GRB_stream.c File Reference

Generic GRB file reading/writing facility. More...

#include <GRBP/io/GRB_stream.h>
#include <GRB/GRB_pktdef.h>
#include <GRB/GRB_msgdef.h>
#include <PBI/PTR.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>

Classes

struct _streamCloseCbp

Structure to bind the close callback routine with a parameter. More...

struct _GRB_streamHeader

Standard header for both input and output streams. More...

struct _GRB_istreamBody

Internal structure used to control the input access to an GRB file or data section. More...

struct _GRB_ostreamBody

Context for an output event/packet stream. More...

union _GRB_streamBody

Union of all the stream types. More...

union _GRB_stream

Standard GRB stream handle. More...

Typedefs

typedef enum _GRB_STREAM_DIR_K GRB_STREAM_DIR_K

Typedef for enum _GRB_STREAM_DIR_K.

typedef enum _GRB_STREAM_DIR_K GRB_streamDir

Typedef for enum _GRB_STREAM_DIR_K.

typedef enum _GRB_STREAM_BOUNDS_K GRB_STREAM_BOUNDS_K

Typedef for enum _GRB_STREAM_BOUNDS_K.

typedef int(* streamClose )(void *file)

Generic stream close function signature.

typedef struct _streamCloseCbp streamCloseCbp

Typedef for struct _streamCloseCbp.

typedef struct _GRB_streamHeader GRB_streamHeader

Typedef for struct _GRB_streamHeader.

typedef struct _GRB_istreamBody GRB_istreamBody

Typedef for struct _GRB_streamFileBody.

typedef struct _GRB_ostreamBody GRB_ostreamBody

Typedef for struct _GRB_ostreamBody.

typedef union _GRB_streamBody GRB_streamBody

Typedef for union _GRB_streamBody.

typedef struct _GRB_stream GRB_stream

Typedef for struct _GRB_stream.

Enumerations

enum _GRB_STREAM_DIR_K {
  GRB_STREAM_DIR_K_READ = 0,
  GRB_STREAM_DIR_K_WRITE = 1
}

Enumeration of the IO direction, i.e. read or write. More...

enum _GRB_STREAM_BOUNDS_K {
  GRB_STREAM_BOUNDS_K_MAXPKT = 1 << 16,
  GRB_STREAM_BOUNDS_K_LOWER = GRB_STREAM_BOUNDS_K_MAXPKT,
  GRB_STREAM_BOUNDS_K_UPPER = 1 << 25,
  GRB_STREAM_BOUNDS_K_OVERFLOW = GRB_STREAM_BOUNDS_K_MAXPKT
}

Upper and lower bounds on the segment size. More...

Functions

static GRB_stream * fileOpen (GRB_streamType type, const char *fileName, int segment_size)

Opens a GRB file for reading.

static int fileRead (GRB_stream *stream, unsigned int *ptr, size_t nbytes)

Reads the specified number of bytes.

static GRB_stream * map (unsigned int *grb_data)

Prepares an in memory data structure to be read.

static void * mem_alloc (unsigned int nbytes)

Wraps the 'real' allocator.

static void mem_free (void *ptr)

Wraps the 'real' deallocator.

GRB_stream * GRB_streamOpen (GRB_streamType type, const void *name, int segment)

Opens a GRB file or data for reading.

int GRB_streamRead (GRB_stream *stream)

Reads the GRB file associated with the event stream handle istream.

const GRB_pktHdr * GRB_streamLocate (GRB_stream *stream, int advance)

Returns a pointer to the current packet and, optionally, advances the current packet to the next packet.

int GRB_streamClose (GRB_stream *stream)

Closes the file after it has been read.

GRB_stream * GRB_streamCreate (GRB_streamType type, const char *name, int rsvd_mbz)

Creates a file to receive the GRB data packets.

int GRB_streamFree (GRB_stream *stream)

Frees the memory associated with the contents of the file.

Detailed Description

Generic GRB file reading/writing facility.

Author:: JJRussell - russell@slac.stanford.edu


   CVS $Id: GRB_stream.c,v 1.2 2011/03/30 22:11:52 russell Exp $

Typedef Documentation

GRB_STREAM_BOUNDS_K

Typedef for enum _GRB_STREAM_BOUNDS_K.

The lower bound must be constrained to be at least the size of the largest event.

The upper bound is really only a default used and can be easily overridden. The upper bound must be at least the size of a maximally sized event, but really is set by purely practical considerations. Since this is unit of a read, on very large files, there can be a considerable delay if this read is very large. This is annoying, particularly in the case where one working interactively and just processing the first few events. So this size is set to be something that is large enough to be accommodate most files, but small enough to keep the read latency to something tolerable on human timescales.

The overflow region is provided as an area where discontigious events may be reassembled into a contigious block of memory. As such, it must be at least the size of a maximally sized event.

Enumeration Type Documentation

enum _GRB_STREAM_BOUNDS_K

Upper and lower bounds on the segment size.

Enumerator:

GRB_STREAM_BOUNDS_K_MAXPKT	Size of maximum event
GRB_STREAM_BOUNDS_K_LOWER	Lower bound must be at least the size of a maximally sized event
GRB_STREAM_BOUNDS_K_UPPER	Upper bound must be at least the lower bound, choose 32Mbytes
GRB_STREAM_BOUNDS_K_OVERFLOW	Overflow area must at least the size of a maximally sized event

enum _GRB_STREAM_DIR_K

Enumeration of the IO direction, i.e. read or write.

Enumerator:

GRB_STREAM_DIR_K_READ	Stream direction is read
GRB_STREAM_DIR_K_WRITE	Stream directory is write

Function Documentation

static GRB_stream * fileOpen	(	GRB_streamType	type,
		const char *	fileName,
		int	segmentSize
	)			`[static]`

Opens a GRB file for reading.

Parameters:

	type	The stream type, must be either GRB_STREAM_TYPE_K_FILE, this potentially offers the most efficient processing GRB_STREAM_TYPE_K_FILESEG, this should be used when 2 or more streams are needed
	fileName	The name of the GRB file to open
	segmentSize	The largest portion of the file that will be buffered on a read. If specified as 0, an upper limit of about 32Mbytes will be used. If smaller than 64Kbytes (the maximum size of an event), the segment size will be reset to 64Kbytes If specified as -1, no uppper limit will be imposed, i.e. the whole file will be processed.

Returns:: If successful, the file handle else, NULL

Referenced by GRB_streamOpen().

static int fileRead	(	GRB_stream *	stream,
		unsigned int *	ptr,
		size_t	nbytes
	)			`[static]`

Reads the specified number of bytes.

Returns:: Status

References _GRB_stream::bdy, _GRB_istreamBody::fd, _GRB_streamBody::in, and _GRB_istreamBody::name.

Referenced by GRB_streamRead().

int GRB_streamClose ( GRB_stream * stream )

Closes the file after it has been read.

This function only closes the file after its contents have been read into memory. To free the memory associated with the contents of the file, the user must call GRB_streamFree().

References _GRB_streamHeader::close, _GRB_stream::hdr, _streamCloseCbp::prm, and _streamCloseCbp::rtn.

GRB_stream * GRB_streamCreate	(	GRB_streamType	type,
		const char *	name,
		int	rsvd_mbz
	)

Creates a file to receive the GRB data packets.

Returns:: The stream handle

Parameters:

	type	The type of stream to open GRB_STREAM_TYPE_K_FILE or GRB_STREAM_TYPE_K_DATA
	name	The name of the section or file to create
	rsvd_mbz	Reserved, must be 0

References _GRB_stream::bdy, _GRB_streamHeader::close, _GRB_streamHeader::dir, _GRB_ostreamBody::file, _GRB_streamHeader::free, GRB_STREAM_DIR_K_WRITE, GRB_STREAM_TYPE_K_DATA, GRB_STREAM_TYPE_K_FILE, _GRB_stream::hdr, mem_alloc(), _GRB_ostreamBody::name, _GRB_streamBody::out, _streamCloseCbp::prm, _streamCloseCbp::rtn, and _GRB_streamHeader::type.

int GRB_streamFree ( GRB_stream * stream )

Frees the memory associated with the contents of the file.

Parameters:

stream

The event builder stream handle

After calling GRB_streamFree, the contents and the GRB stream handle are no longer valid.

References _GRB_streamHeader::free, _GRB_stream::hdr, and mem_free().

const GRB_pktHdr * GRB_streamLocate	(	GRB_stream *	stream,
		int	advance
	)

Returns a pointer to the current packet and, optionally, advances the current packet to the next packet.

Returns:: Pointer to the current packet or NULL if at EOF

Parameters:

	stream	The stream handle
	advance	If non-zero, advance to the internal next packet pointer to the next packet. This is the normal way one would serially read a file of packets

References _GRB_stream::bdy, _GRB_istreamBody::cur_pkt, _GRB_streamBody::in, and _GRB_istreamBody::lst_pkt.

GRB_stream * GRB_streamOpen	(	GRB_streamType	type,
		const void *	name,
		int	segment
	)

Opens a GRB file or data for reading.

Parameters:

	type	The type of stream to open GRB_STREAM_TYPE_K_FILE GRB_STREAM_TYPE_K_FILESEG GRB_STREAM_TYPE_K_DATA If GRB_STREAM_TYPE_K_FILE is opened, the segment size argument determines whether to the file is processed as a whole or in segments. This option can be specified for at most one stream in an application. It is meant to mimic the single stream properties of the LCB event stream. Specifying GRB_STREAM_TYPE_K_FILESEG always forces the file to processed in segments. (Of course if the file size is smaller than segment, then there is only one segment. This option should always be specified when processing more than one stream, the classic example would be a compare application.
	name	The name of the GRB file or the address of data section to open
	segment	Interpretation is dependent on type. If type is GRB_STREAM_TYPE_K_FILE, then this is the largest portion of the file that will be buffered on a read. If this is specified as 0, then a default upper bound of ~32Mbytes will be imposed. This was select purely to avoid long delays (as the whole file is being read in) on very large files. This can be annoying, particularly when working interactively and processing only the first few events. If this is specified as -1, then the there is no segmenting, and the entire file is processed at once. If a value < 64Kbytes is used, the segment size will be set to 64Kbytes. (The segment size must be large enough to hold one maximally sized event). GRB_STREAM_TYPE_K_DATA, then is ignored

Returns:: If successful, the file handle else, NULL

References fileOpen(), GRB_STREAM_TYPE_K_DATA, GRB_STREAM_TYPE_K_FILE, GRB_STREAM_TYPE_K_FILESEG, and map().

int GRB_streamRead ( GRB_stream * stream )

Reads the GRB file associated with the event stream handle istream.

Parameters:

stream

A previously opened event builder stream handle

Return values:

	0,if	successful
	-1,if	not

References _GRB_stream::bdy, _GRB_istreamBody::buf, _GRB_istreamBody::cur_pkt, fileRead(), GRB_STREAM_TYPE_K_FILE, _GRB_stream::hdr, _GRB_streamBody::in, _GRB_istreamBody::lst_pkt, _GRB_istreamBody::seg_size, and _GRB_streamHeader::type.

static GRB_stream * map ( unsigned int * data ) [static]

Prepares an in memory data structure to be read.

Parameters:

data

Pointer to the in memory data structure.

Returns:: If successful, the stream handle, else NULL

References _GRB_streamHeader::close, _GRB_streamHeader::dir, _GRB_streamHeader::free, GRB_STREAM_DIR_K_READ, GRB_STREAM_TYPE_K_DATA, _GRB_stream::hdr, mem_alloc(), _streamCloseCbp::prm, _streamCloseCbp::rtn, and _GRB_streamHeader::type.

Referenced by GRB_streamOpen().

static void * mem_alloc ( unsigned int nbytes ) [static]

Wraps the 'real' allocator.

Returns:: Pointer to the allocated memory or NULL if not available

Parameters:

nbytes

The number of bytes

: On the embedded systems, the memory pool available to malloc is too small for the larger .ebf files. Typically in this environment one should use MBA_alloc/MBA_free, However, because the EDS package is supposed to be independent of other packages (including P B S) one can not link against in the traditional manner.

The solution is two fold

Do the usual thing and define wrapper functions that are different for the host and embedded systems.
Go one step further and define MBA_alloc/MBA_free as weak symbols.

: This second point effectively transfers the burden of determining when the MBA_alloc/MBA_free symbols need to be satistied. Instead of at the static link or dynamic load stage, this is deferred till runtime.

Note:: There is likely an even trickier way to do this. Currently the code checks if MBA_alloc/MBA_free are non-NULL and if so uses them, else it reverts to malloc/free. Ideally the dynamic loader should do this, thus eliminating the need for any run-time code.

Referenced by fileOpen(), GRB_streamCreate(), and map().

static void mem_free ( void * ptr ) [static]

Wraps the 'real' deallocator.

See also:: mem_alloc

Parameters:

ptr

Pointer to the memory to free

Note:: See mem_alloc for a discussion on why mem_alloc and mem_free exist and for the implementation details

Referenced by GRB_streamFree().


Classes
struct	_streamCloseCbp
	Structure to bind the close callback routine with a parameter. More...
struct	_GRB_streamHeader
	Standard header for both input and output streams. More...
struct	_GRB_istreamBody
	Internal structure used to control the input access to an GRB file or data section. More...
struct	_GRB_ostreamBody
	Context for an output event/packet stream. More...
union	_GRB_streamBody
	Union of all the stream types. More...
union	_GRB_stream
	Standard GRB stream handle. More...
Typedefs
typedef enum _GRB_STREAM_DIR_K	GRB_STREAM_DIR_K
	Typedef for enum _GRB_STREAM_DIR_K.
typedef enum _GRB_STREAM_DIR_K	GRB_streamDir
	Typedef for enum _GRB_STREAM_DIR_K.
typedef enum _GRB_STREAM_BOUNDS_K	GRB_STREAM_BOUNDS_K
	Typedef for enum _GRB_STREAM_BOUNDS_K.
typedef int(*	streamClose )(void *file)
	Generic stream close function signature.
typedef struct _streamCloseCbp	streamCloseCbp
	Typedef for struct _streamCloseCbp.
typedef struct _GRB_streamHeader	GRB_streamHeader
	Typedef for struct _GRB_streamHeader.
typedef struct _GRB_istreamBody	GRB_istreamBody
	Typedef for struct _GRB_streamFileBody.
typedef struct _GRB_ostreamBody	GRB_ostreamBody
	Typedef for struct _GRB_ostreamBody.
typedef union _GRB_streamBody	GRB_streamBody
	Typedef for union _GRB_streamBody.
typedef struct _GRB_stream	GRB_stream
	Typedef for struct _GRB_stream.
Enumerations
enum	_GRB_STREAM_DIR_K { GRB_STREAM_DIR_K_READ = 0, GRB_STREAM_DIR_K_WRITE = 1 }
	Enumeration of the IO direction, i.e. read or write. More...
enum	_GRB_STREAM_BOUNDS_K { GRB_STREAM_BOUNDS_K_MAXPKT = 1 << 16, GRB_STREAM_BOUNDS_K_LOWER = GRB_STREAM_BOUNDS_K_MAXPKT, GRB_STREAM_BOUNDS_K_UPPER = 1 << 25, GRB_STREAM_BOUNDS_K_OVERFLOW = GRB_STREAM_BOUNDS_K_MAXPKT }
	Upper and lower bounds on the segment size. More...
Functions
static GRB_stream *	fileOpen (GRB_streamType type, const char *fileName, int segment_size)
	Opens a GRB file for reading.
static int	fileRead (GRB_stream stream, unsigned int ptr, size_t nbytes)
	Reads the specified number of bytes.
static GRB_stream *	map (unsigned int *grb_data)
	Prepares an in memory data structure to be read.
static void *	mem_alloc (unsigned int nbytes)
	Wraps the 'real' allocator.
static void	mem_free (void *ptr)
	Wraps the 'real' deallocator.
GRB_stream *	GRB_streamOpen (GRB_streamType type, const void *name, int segment)
	Opens a GRB file or data for reading.
int	GRB_streamRead (GRB_stream *stream)
	Reads the GRB file associated with the event stream handle istream.
const GRB_pktHdr *	GRB_streamLocate (GRB_stream *stream, int advance)
	Returns a pointer to the current packet and, optionally, advances the current packet to the next packet.
int	GRB_streamClose (GRB_stream *stream)
	Closes the file after it has been read.
GRB_stream *	GRB_streamCreate (GRB_streamType type, const char *name, int rsvd_mbz)
	Creates a file to receive the GRB data packets.
int	GRB_streamFree (GRB_stream *stream)
	Frees the memory associated with the contents of the file.