Working with tableio .root files¶

File Structure¶

Rootwriter creates a TTree for each key you book with tableio. The fields created by the I3Converter will be branches of this tree. All trees will be stored inside a single output file along with a master tree, which all other trees are friends of. The master tree is usually called MasterTree, but that name can be configured (see the documentation of I3ROOTTableService). All data can be accessed via the master tree.

Thus, if you open a tableio-created .root file:

$ root -l output.root

you may find the following contents:

root [1] .ls
TFile**         output.root
 TFile*         output.root
  KEY: TTree    IceTopHLCVEMPulsesInfo;1        IceTopHLCVEMPulsesInfo
  KEY: TTree    IceTopSLCVEMPulses;1    IceTopSLCVEMPulses
  KEY: TTree    MCPrimary;1     MCPrimary
  KEY: TTree    MCPrimaryInfo;1 MCPrimaryInfo
  KEY: TTree    CleanedTankPulses;1     CleanedTankPulses
  KEY: TTree    CleanedTankPulsesInfo;1 CleanedTankPulsesInfo
  KEY: TTree    IceTopHLCVEMPulses;1    IceTopHLCVEMPulses
  KEY: TTree    ShowerCOG;1     ShowerCOG
  KEY: TTree    ShowerCombined;1        ShowerCombined
  KEY: TTree    ShowerCombinedInfo;1    ShowerCombinedInfo
  KEY: TTree    ShowerCombinedParams;1  ShowerCombinedParams
  KEY: TTree    ShowerPlane;1   ShowerPlane
  KEY: TTree    ShowerPlaneParams;1     ShowerPlaneParams
  KEY: TTree    MasterTree;1    MasterTree

As mentioned above, all data can be accessed via the MasterTree. You can check this by having a look at the tree viewer:

root [2] MasterTree->StartViewer()

The window that opens might look like that:

As you can see, all the other trees show up as if they were branches of MasterTree. The quickest way to create a plot is simply double-clicking a branch in this viewer.

Tree Structure¶

Rootwriter TTrees are completely flat. Thus, each branch contains only one leaf. However, branches can be arrays, either of fixed length (e.g. for filter masks: condition_passed and prescale_passed) or variable length (e.g. when booking I3RecoPulseSeriesMaps).

Each tree contains at least six branches:

UInt_t Run - the run number,
UInt_t Event - the event number,
UInt_t SubEvent - the sub-event number,
Int_t SubEventStream - the ID of the splitter module that made this stream,
Char_t SubEventStream_String[] - the name of the splitter module that made this stream as a string,
Bool_t exists - set to false, if the corresponding object did not exist in the frame.

In order to align the trees every tree contains one line for each event. Therefore it is important to always check the value of the branch called exists.

If the object stored in the tree is an array like structure (like e.g. an I3RecoPulseSeriesMap) the data will be stored in variable-length arrays and an additional branch is added to the tree

ULong64_t Count_<tree_name> - the number of items in the current event.

In case of fixed-length arrays the length is not stored anywhere in the root file, but it is always the same as defined by the I3Converter.

Note

I3FilterResultMapConverter creates one branch for each filter. Each of these branches is an array of two bools. The first one for the condition_passed flag, the second one for prescale_passed.

In some cases, fixed and variable-length arrays are combined. For instance, when booking ATWD waveforms, a branch of type double[Count_<tree_name>][128] will be created. Each entry will be an array of 128 doubles.

Note

To workaround issues with ROOT’s interpretation of branch types, rootwriter will replace all arrays (not single values) of type char or unsigned char with arrays of int16_t or uint16_t, respectively. Thus, the tree structure might differ from what one would expect from the converter implementation.

Besides opening a TreeViewer you can also use TTree::Print() to get information about the structure of a tree and the stored variable types:

root [3] IceTopHLCVEMPulses->Print();

******************************************************************************
*Tree    :IceTopHLCVEMPulses: IceTopHLCVEMPulses                             *
*Entries :   348915 : Total =       417028258 bytes  File  Size =   89738443 *
*        :          : Tree compression factor =   4.64                       *
******************************************************************************
*Br    0 :Count_IceTopHLCVEMPulses :                                         *
*         | ULong64_t Number of objects in each field                        *
*Entries :   348915 : Total  Size=    2828723 bytes  File Size  =     414516 *
*Baskets :       35 : Basket Size=     386560 bytes  Compression=   6.74     *
*............................................................................*
*Br    1 :Run       : UInt_t run number                                      *
*Entries :   348915 : Total  Size=   18123225 bytes  File Size  =     827981 *
*Baskets :      202 : Basket Size=    2502144 bytes  Compression=  21.85     *
*............................................................................*
*Br    2 :Event     : UInt_t event number                                    *
*Entries :   348915 : Total  Size=   18123641 bytes  File Size  =    2239237 *
*Baskets :      202 : Basket Size=    2502144 bytes  Compression=   8.08     *
*............................................................................*
*Br    3 :SubEvent  : UInt_t sub-event number                                *
*Entries :   348915 : Total  Size=   18124266 bytes  File Size  =     875798 *
*Baskets :      202 : Basket Size=    2502144 bytes  Compression=  20.66     *
*............................................................................*

// etc

Note

MasterTree->Print() will only print the structure of MasterTree, which is probably not what you want. You will have to call Print() on each tree.

The tableio description field is stored in the branch titles. You can retrieve the description of an individual branch as follows:

root [4] IceTopHLCVEMPulses->GetBranch("Event")->GetTitle()
(const char* 0x255e5d8)"event number"

These descriptions are provided by the individual converters and are the same as those stored in the hdf header. Unfortunately, ROOT trees do not have a field where the tableio unit field can be stored.

Using C++¶

Of course, the easiest way to create a plot from a tableio root file is using the Draw() method, for instance:

MasterTree->Draw("IceTopHLCVEMPulses.charge");

This will fill all pulses in all events into a histogram.

However, there are more complicated cases, where Draw() and simple cuts are insufficient and you might have to actually loop over the tree by hand.

To do this, assign a variable to the branches you want to inspect and call TTree::GetEntry() inside a loop. For example:

double energy;
MasterTree->SetBranchAddress("MCPrimary.energy", &energy);
for (Long64_t evt = 0; evt < MasterTree->GetEntries(); ++evt) {
    MasterTree->GetEntry(evt);
    // do something with energy
}

For multi-row tables you will need sufficiently large arrays and make use of Count_<tree_name>:

ULong64_t nPulses;
MasterTree->SetBranchAddress("IceTopHLCVEMPulses.Count_IceTopHLCVEMPulses", &nPulses);
double charge[MAX_PULSES];    // set MAX_PULSES to a number large enough for any event you might encounter
MasterTree->SetBranchAddress("IceTopHLCVEMPulses.charge", charge);  // no & before charge!
for (Long64_t evt = 0; evt < MasterTree->GetEntries(); ++evt) {
    MasterTree->GetEntry(evt);
    for (int i = 0; i < nPulses; ++i) {
        // do something with charge[i]
    }
}

Warning

You might be tempted to simplify this task using TTree::MakeClass(). However, this can lead to undesired behaviour for multi-row tables. TTree::MakeClass() will allocate arrays for these tables whose length is inferred from the longest array occurring in the file used when running TTree::MakeClass(). If you then use the resulting code to read other files, longer arrays can lead to crashes through segmentation faults.

When using this approach you need to know the types of branches created by converters. The easiest way to find out (besides checking the code) is TTree::Print() as described above.

Using Python¶

PyROOT offers several ways to read data from root trees. In order to work with pyROOT simply import the ROOT module and open a file:

import ROOT

f = ROOT.TFile('output.root')
tree = f.MasterTree

Just like in C++ you can then use TTree.Draw() to create histograms:

tree.Draw("log10(MCPrimary.energy)")

Looping over trees is a little simpler in python. All branches of a tree can be access as attributes of the tree. TTrees are iterable just as branches that contain arrays. PyROOT will automatically make sure that all loops are stopped in time. You do not have to care about the Count_<tree_name> branch.

An example:

for event in f.IceTopHLCVEMPulses:
    for charge in event.charge:
        # do something with charge

Unfortunately, this is relatively slow.

Note

While branches are accessible as attributes of a tree, friend trees are not. Thus code like MasterTree.IceTopHLCVEMPulses.charge will not work.

A faster way to loop over trees is to assign variables to branches just as in C++. In Python this is only possible using array.array or numpy.array because you have to pass a pointer to a fixed-type variable to TTree.SetBranchAddress().

Here is a simple example using numpy.arrays:

import ROOT
import numpy as n

f=ROOT.TFile('output.root')
t=f.MasterTree

energy = n.array([0], dtype=n.double)
t.SetBranchAddress('MCPrimary.energy', energy)

for evt in range(t.GetEntries()):
    t.GetEntry(evt)
    # do something with energy[0], e.g.
    print energy[0]

In the same way you can also read arrays stored in trees. However, you will have to know in advance how many entries you will expect. Let’s add to the example above:

count = n.array([0], dtype=n.uint64)
MAX_PULSES=324       # for IceTop this is enough
charge = n.zeros(MAX_PULSES, dtype=n.double)

t.SetBranchAddress('IceTopHLCVEMPulses.Count_IceTopHLCVEMPulses', count)
t.SetBranchAddress('IceTopHLCVEMPulses.charge', charge)

for evt in range(t.GetEntries()):
    t.GetEntry(evt)
    for q in range(count[0]):
        # do something with charge, e.g.
        print charge[q]

The obvious disadvantage of this way is that you have to write just as much code as in C++.

A note on FilterResultMaps¶

I3FilterResultMapConverter creates a branch containing an array of two bools for each filter. The first of these two bools stores the condition_passed flag, the second one the prescale_passed flag. So to determine the rate of a filter (e.g. for comparison with monitoring data) you have to look at only one of them, namely the second one, for example:

MasterTree->Draw("FilterMask.IceTopSTA3_11[1]")

Working with tableio .root files¶

File Structure¶

Tree Structure¶

Using C++¶

Using Python¶

A note on FilterResultMaps¶

Table of Contents

Previous topic

Next topic

This Page