Archive Manager

Usage - Examples - Creating new archives

The ArchiveManager tool started as a test for the archive I/O library. It can be used for basic archive data file maintainance. It evolved into command-line monster, but archive repairs on GB files take time and this tool can run batch-driven over weekends. An alternate and soon to be preferred version is the python-based cardminer tool.

Before using the ArchiveManager, be sure to read and understand the information about the binary archive format.

Usage

The tool displays an option summary like this: 
USAGE:  ArchiveManager [Options] <archive>

You should not run the this tool on a live archive.
Stop the ArchiveEngine before operating on an archive.
Create backups in case this tool damages your archive.

Options:
        -info                          Show archive information
        -test                          Test archive for errors
        -channel <channel>             Specify channel name
        -match <regular expression>    List matching channels
        -Match <regular expression>    Dump values for matching channels
        -start <time>                  Start time as mm/dd/yyyy hh:mm:ss[.nano-secs]
        -end <time>                    End time (exclusive)
        -xport <new archive>           export data into new archive
        -repeat_limit <seconds>        remove 'repeat' entries beyond limit (export)
        -FileSize <days>               Days per binary data file (export, binary file format detail)  
        -headers <channel>             show headers for channel
        -Output <channel>              output ASCII dump for channel
        -Input <ascii file>            read ASCII dump for channel into archive
        -Compare <target archive>      Compare with target archive
        -Seek                          Seek test (use with -start)
        -Rename <new name>             Rename channel name, requires -c for old channel
        -DELETE <channel>              Delete channel from directory file

Examples:

Suppose "ARCH" is set to some valid archive:
ARCH=/home/fred/Epics/extensions/src/ChannelArchiver/Engine/Test/freq_directory

Show time range of archive

ArchiveManager $ARCH -i

Test archive

ArchiveManager $ARCH -T

Tries to read all values for all channels, reporting errors while doing this. Since a broken archive might be broken in many unexpected ways, this test will always be uncomplete, but if the test succeeds, retrival tools should have no problem looking at the samples.

Show time of first/last sample for specific channel

ArchiveManager $ARCH -c jane -i

List all channel names

ArchiveManager $ARCH

List all values for specific channel

ArchiveManager $ARCH -c janet

... between given times

ArchiveManager $ARCH -c is_hvps_actv -s "02/10/2000 17:21:17" -e "02/10/2000 17:43:22"

Delete, rename entries

ArchiveManager $ARCH -DELETE fred
will delete the entry 'fred' from the directory file. Note that this does not remove any samples or compact the data files, you have to copy the data files to accomplish this. (see '-xport' option). And by the way there is no way back, the channel is gone.

ArchiveManager $ARCH -channel fred -Rename egon
will rename a channel in the directory file.

Dump channel in ASCII

ArchiveManager $ARCH -O is_hvps_actv >is_hvps_actv.txt

Add values from ASCII dump (see note)

ArchiveManager /tmp/new_dir -I is_hvps_actv.txt

Copy archive (see note)

This is the major repair facility for now (besides editing the ASCII dump):

ArchiveManager $ARCH -x /tmp/new_dir

While copying the samples, bad CtrlInfo entries or samples that go "back in time" are skipped. With the "-r" option, outrageous repeat counts can be removed.

The newly generated archive is more tighly packed, consuming less disk space and allowing faster retrival.

Create monthly extract, e.g. for CD-ROM (see note)

ArchiveManager $ARCH -x /tmp/new_dir -s "01/01/2000 00:00:00" -e "02/01/2000 00:00:00"

Restore extract from CD-ROM to new archive (see note)

ArchiveManager /mnt/cdrom/dir -x /tmp/new_dir -s "01/01/2000 00:00:00"

(assuming that /mnt/cdrom/dir holds a "January 2000" extract)

Note on creating new archives

The ArchiveManager can create new archives by extracting samples from an existing archive or reading ASCII files. It can also add samples to an existing archive.

But: It cannot remove entries from an archive.
It cannot add a sample stamped before the last sample for that channel.

Example:
If channel X has samples up to "02/01/2000 18:35:12",
you cannot add a sample stamped "02/01/2000 18:00:00" because that would go back in time.

Idea for maintaining the archive:

ChannelArchiver Manual