hdfwriter

A tableio plugin for writing HDF5 files.

Usage

The I3HDFWriter() segment provides output to HDF5 tables. Use it like so:

from icecube import icetray, dataio, dataclasses
from icecube.icetray import I3Tray
from icecube.hdfwriter import I3HDFWriter

tray = I3Tray()
tray.Add('I3Reader', Filename='foo.i3.gz')

tray.Add(I3HDFWriter,
    Output='foo.hd5',
    Keys=['LineFit','InIceRawData'],
    SubEventStreams=["InIceSplit"]
)

tray.Execute()

I3SimHDFWriter

For pre-trigger simulation files that contain only Q frames, use I3SimHDFWriter(), e.g.:

from icecube.hdfwriter import I3SimHDFWriter

tray.Add(I3SimHDFWriter,
    Output='foo.hd5',
    Keys=['MCMuon','I3MCPESeriesMap']
)

Note that the SubEventStreams parameter is forbidden in this context.

Reading with pytables

I3HDFTableService creates a flat structure of tables under the file root. Since the HDF5 high-level Table API does not explicitly support units and docstrings, these are stored in the attributes FIELD_N_UNIT and FIELD_N_DOC for each field N. Support for reading these attributes is provided by the hdfwriter.pytables_ext module:

In [1]: from icecube.hdfwriter import pytables_ext
In [2]: import tables
In [3]: f=tables.openFile('foo.hdf5')
In [4]: f.root.LineFit.coldoc
Out[4]:
['run number',
 'event number',
 'object was found in the frame',
 'x-position of particle',
 'y-position of particle',
 'z-position of particle',
 'origin time of particle',
 'zenith angle of particle origin',
 'azimuthal angle of particle origin',
 'energy of particle',
 'particle speed',
 'length of track',
 '',
 '',
 '',
 '']
In [5]: f.root.LineFit.colunits
Out[5]:
['',
 '',
 'bool',
 'm',
 'm',
 'm',
 'ns',
 'radian',
 'radian',
 'GeV',
 'Gm/s',
 'm',
 '',
 '',
 '',
 '']

Utilities

The hdfwriter project provides 3 utility scripts:

  • hdfwriter-merge concatenates HDF5 tables and writes them to any of the supported output formats (hdf5, root, or csv)

  • hdfwriter-mix creates HDF5 files containing the union of all tables in its input files, e.g. if file A.hdf5 contains tables FooFit and BarFit and file B.hdf5 contains BazFit, hdfwriter-mix A.hdf5 B.hdf5 -o C.hdf5 will produce a file with tables FooFit, BarFit, and BazFit.

  • hdfwriter-book is a simple wrapper around the I3HDFWriter() snippet above.