.. SPDX-FileCopyrightText: 2024 The IceTray Contributors
..
.. SPDX-License-Identifier: BSD-2-Clause

.. _tray_segments:

Tray segments
=============

In addition to services and modules, IceTray has a concept of a tray
segment, which is an automatic configuration function that adds
one or more modules, services and/or other segments to a
I3Tray. These segments would be defined by module experts, live
within the project source tree and be released with project and
meta-projects.  This is designed to reduce configuration duplication
(and remove the need to cut-paste from other scripts) and to simplify
scripts in several cases (by building massive data processing
scripts from segments of segments).  With this in mind, segments provide a way to:

1. Logically connect sets of services and modules that together perform a single task (e.g. Gulliver-based reconstructions or pulse extraction with NFE)
2. Build large segments of standard processing (e.g. modularization of bulk filtering and processing scripts)
3. Support modules with widely used alternate set of defaults for some specific use (e.g. when different settings are required for real data vs MC data)

.. note::

  When creating a segment for use in production, adhere to the
  "Give one entity, one cohesive responsibility." standard.
  We've seen several cases of segments taking on too much responsibility.


A Basic Tray Segment Example
""""""""""""""""""""""""""""

Tray segments are defined by a python function, such as the following::

  @icetray.traysegment
  def small_segment(tray, name, NDumps=1):
    for i in range(0, NDumps):
      tray.AddModule('Dump', name + ('_dump_%d' % i))

This can then be added to a tray by using the following in your python scripts::

  tray.AddSegment(small_segment, 'dumps', NDumps=4)

A more realistic example from the payload-parsing project::

  @icetray.traysegment
  def I3DOMLaunchExtractor(tray, name, OMKey2MBIDXML=os.path.expandvars("$I3_SRC") + "/phys-services/resources/mainboard_ids.xml", BufferID="I3DAQData", ExceptionId="I3DAQDecodeException"):
      if OMKey2MBIDXML != None:
         tray.AddService("I3XMLOMKey2MBIDFactory", name + "_OmKey2MbId", InFile=OMKey2MBIDXML)
      tray.AddService("I3PayloadParsingEventDecoderFactory", name + "_EventDecoder")
      tray.AddModule("I3FrameBufferDecode", name + "_fbdecoder", BufferID=BufferID, ExceptionId=ExceptionId)

The decorator (@icetray.traysegment) marks the function as a tray segment, allowing it to be found later (for example, by I3Tray and icetray-inspect). Note that this also allows you to have multiple tray segments in one file. The arguments of the segment are specified as keywords (e.g. OMKey2MBIDXML), with default values following the = sign. This segment can be added to your tray like so::

  tray.AddSegment(payload_parsing.I3DOMLaunchExtractor, "extract")


How to find Tray Segments
"""""""""""""""""""""""""

For tasks connected to a single module, relevant standard configurations
should be added to the module's python directory (more details below).

Since tray segments are not stored in a single place but
distributed throughout the source projects, finding them could be
an issue.  To address this, icetray-inspect also reports all
available tray segments as well::

  blaufuss@squeak[build]% icetray-inspect --no-modules --no-services payload-parsing
   -----------------------------------------------------------------------------
   icecube.payload_parsing.I3DOMLaunchExtractor (I3Tray segment)

     Extract DOMLaunches and triggers from raw DAQ data.

   Parameters:
     BufferID
       Description : Name of the raw data buffer in the frame.
       Default     : 'I3DAQData'

     ExceptionId
       Description : Name of the boolean to put in the frame when an error occurs.
       Default     : 'I3DAQDecodeException'

     OMKey2MBIDXML
       Description : Path to XML file mapping mainboard IDs to OMKeys.
       Default     : '/data/i3home/blaufuss/icework/offline-software/trunk/src/phys-services/resources/mainboard_ids.xml'

   -----------------------------------------------------------------------------

You can get a list of all available tray-segments from icetray-inspect::

   blaufuss@squeak[build]% icetray-inspect --no-modules --no-services -a ./lib


Alternative configurations in Segments
""""""""""""""""""""""""""""""""""""""

.. |module_altconfig| replace:: :py:func:`~.icetray.traysegment.module_altconfig`.

For case 3 above (modules with alternate defaults), although still
a tray segment, can be handled very quickly using |module_altconfig|
For example, adding an alternate configuration for I3WaveCalibrator to be
used with DOMSimulator-based simulation can be done as follows::

  DOMSimulatorCalibrator = icetray.module_altconfig('I3WaveCalibrator',
    DOMsimulatorWorkArounds=True, FADCSaturationMargin=1)

This can then be added to a tray using :py:meth:`.I3Tray.AddSegment` in the same
way::

  tray.AddSegment(WaveCalibrator.DOMSimulatorCalibrator, 'calibrator')

When using |module_altconfig|, those parameters specified in the arguments to |module_altconfig| override the module's defaults. These can be overridden again in :py:meth:`.I3Tray.AddSegment` and any other arguments to the original module can also be specified there.

These alternative configurations also are reported by icetray-inspect:

.. code-block:: console

  blaufuss@squeak[build_deb(I3)]% icetray-inspect --no-modules --no-services WaveCalibrator
  *** WaveCalibrator ***
  -------------------------------------------------------------------------------
      icecube.WaveCalibrator.DOMSimulatorCalibrator (I3Tray segment)

      Alternate configuration for I3WaveCalibrator

      Overridden defaults:

      DOMsimulatorWorkArounds=True
      FADCSaturationMargin=1


Writing a Segment for your Modules
""""""""""""""""""""""""""""""""""

Segments can be put in any location accessible to python, but those that are part of Icetray projects typically live in the module's pybdindings. They are then imported from the module's namespace.

Modules without any python component will need to add a PYTHON_DIR directive to their project's CMakeLists.txt::

  i3_project(myproject PYTHON_DIR python)

and an `__init__.py` file to a new python subdirectory like this::

  from icecube import icetray
  import os
  icetray.load('myproject', False)

Short segments can be added directly to this file, but longer ones should be added to another .py file, and then brought into the project's namespace with an import command in `__init__.py`.

There is one additional variant of the icetray.traysegment decorator that can be useful in some circumstances. This is an example from the hdfwriter project::

  @icetray.traysegment_inherit(tableio.I3TableWriter)
  def I3HDFWriter(tray, name, Output=None, **kwargs):
        """Tabulate data to an HDF5 file.

        :param Output: Path to output file
        """
        tabler = I3HDFTableService(Output)
        tray.AddModule(tableio.I3TableWriter, name, TableService=tabler,
            **kwargs)

This is a wrapper around :cpp:class:`I3TableWriter` that adds one additional service. Any options besides `Output` are passed through to :cpp:class:`I3TableWriter` via the kwargs parameter, but what those available options are would not ordinarily show up in the output of icetray-inspect. The use of :py:func:`~.traysegment_inherit` here makes no functional changes, but causes the options taken by :cpp:class:`I3TableWriter` to be appended to the segment's own options (in this case, `Output`) when shown in ``icetray-inspect``.

Segments of Segments
""""""""""""""""""""

Segments can of course include :py:func:`tray.AddSegment() <icecube.icetray.i3tray.I3Tray.AddSegment>` calls within them, and in this way large processing chains can be built up. A working example from the IC86 L2 processing::

  @icetray.traysegment
  def RawDataToPulses(tray, name, superdstname = 'I3SuperDST',
    pulses='UncleanedPulses'):
       # Raw Data
       tray.AddSegment(payload_parsing.I3DOMLaunchExtractor, name + '_launches')
       tray.AddModule('I3WaveCalibrator', name + '_wavecal')
       tray.AddModule('I3Wavedeform', name + '_wavedeform', Output=pulses)
       tray.AddModule('Delete', name + '_delete', Keys=['CalibratedWaveforms'])

       # Super DST
       if superdstname != None:
          tray.AddModule('Rename', name + '_sdstrename',
            Keys=[superdstname, pulses], If=lambda fr: pulses not in fr)

For common processing chains like the L2 processing, such a segment would typically live in a pure python project. For small personal projects, they can live in any python file (including in the script from which they are being used).

Expanding segments in the I3Tray
""""""""""""""""""""""""""""""""

There are several ways to see what is inside of a segment. The simplest is using the --expand-segments argument to icetray-inspect:

.. code-block:: console

  [nwhitehorn@wanderer ~/i3/offline/build]$ icetray-inspect --expand-segments --no-modules --no-services payload-parsing

  *** payload-parsing ***
  -------------------------------------------------------------------------------
    icecube.payload_parsing.I3DOMLaunchExtractor (I3Tray segment)

      Extract DOMLaunches and triggers from raw DAQ data.



      Extra keyword arguments are passed to I3PayloadParsingEventDecoderFactory.

    Equivalent to:
      AddService('I3XMLOMKey2MBIDFactory', 'example_OmKey2MbId', InFile='/home/nwhitehorn/i3/offline/src/phys-services/resources/mainboard_ids.xml.gz')
      AddService('I3PayloadParsingEventDecoderFactory', 'example_EventDecoder')
      AddModule('I3FrameBufferDecode', 'example_fbdecoder', BufferID='I3DAQData', ExceptionId='I3DAQDecodeException')


It is also possible to print out the contents of an I3Tray or TrayInfo object using the Python :py:func:`print` function to get the contents and configuration of the entire tray, with all segments expanded, in a human-readable form. The Python :py:func:`repr` operator can also be used to get a more-tractable (and potentially executable) version of a tray or TrayInfo frame::

  print(repr(tray))

gives::

  tray = I3Tray.I3Tray()
  tray.AddModule('I3Reader', 'reader', DropBuffers=False,
    Filename='/tmp/bork.i3', FilenameList=[], SkipKeys=[])
  tray.AddModule('Dump', 'dump', IcePickServiceKey='', If=None)
  tray.AddModule('I3Writer', 'writer', CompressionLevel=-2,
    DropOrphanStreams=[], Filename='bork-out.i3', IcePickServiceKey='',
    If=None, SkipKeys=[], Streams=[])

You can also, of course, read the source code for the segment.

.. _default_parameters:

Default Parameters for Segments
"""""""""""""""""""""""""""""""

.. autodata:: icecube.icetray.I3Default
   :no-value:

Often times segments will contain default parameters. When segments are layered
with default parameters which are passed to modules it can often be difficult to
track down where the parameter was actually set. There is a global singleton
:py:obj:`~.icetray.I3Default` which if passed to a module will be equivalent to not setting
the parameter. Setting the segment to :py:obj:`~.I3Default` will simplify the segment so
you don't have to guess the correct default to pass the the module.

Example usage::

  @icetray.traysegment
  def MySegment(tray,name,
      Param1 = icetray.I3Default,
      Param1 = icetray.I3Default,
      If = lambda f: True):

      tray.AddModule("MyModule",
          Param1=Param1,
          Param2=Param2,
          )