icecube.gulliver_modules.fortytwo module

Test suite for Gulliver

Implementation of an IceTray module to test reproducibility of processing results.

Frame object values from current processing are compared with values obtained with the same script in the past and/or on some other platform.

The primary (and currently only) use case is to include this module in the resource/test scripts that are run by the build bots. Failure to reproduce results results in un-green checkboxes.

If this ever turns out to become more popular then it should move to some project in offline-software. Then it should also be split up, each class in a separate file.

class icecube.gulliver_modules.fortytwo.DummyChecker

Bases: I3FrameChecker

This checker just serves as an example of a very simple implementation of an I3FrameChecker. It computes only one test value.

compare_test_values(values, reference_values)

This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the get_test_values method.

If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).

If the current test values and the reference ones are are all compatible with each other then an empty string (or None, or False) should be returned.

get_test_values(index, frame)

This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.

class icecube.gulliver_modules.fortytwo.I3FortyTwo(context)

Bases: I3ConditionalModule

Check processing results

A module to check that processing results (e.g. track fits, multiplicities) are the same on different platforms and do not change over time due to changes in the processing software. Reference results for any number of processing/testing scripts can be stored in a single picklefile. This module only cares about the final answers of other module(s) and that they have the expected value(s). Hence the name.

An I3FortyTwo module operates in two different modes:

• ‘reference’: test results of the current run of the script are

  stored in a pickle file. This pickle file can be stored in SVN and then be used by for instance the build bots.

• ‘check’: test results from the latest reference run are read in from the pickle file. They are also computed for the current run, and tested for equality (or equivalence). If any (too large) deviations then the Finish() method will throw an exception in order to trigger the build bots to un-green the appropriate checbox on builds.icecube.wisc.edu.

Check(frame)

Configure((I3ConditionalModule)arg1) → None :

C++ signature :

void Configure(PythonModule<I3ConditionalModule> {lvalue})

Finish((I3ConditionalModule)arg1) → None :

C++ signature :

void Finish(PythonModule<I3ConditionalModule> {lvalue})

Physics((I3ConditionalModule)arg1, (I3Frame)arg2) → None :

C++ signature :

void Physics(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)

ReadPickle()

Reference(frame)

class icecube.gulliver_modules.fortytwo.I3FrameChecker

Bases: object

This class serves to define the interface for checks on specific (types of) objects within a frame, to be used by an I3FortyTwo module.

compare(index, frame)

This method gets run in check mode. The checker object is then instantiated by reading it from a pickle file, so the reference_list should be filled with the test values from a previous run. We compute the test values now again in the current run, and check that they are compatible with the values stored in reference_list.

compare_test_values(values, reference_values)

This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the get_test_values method.

If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).

If the current test values and the reference ones are are all compatible with each other then an empty string (or None, or False) should be returned.

finish()

Optional, write diagnostic stuff at the end.

get_test_values(index, frame)

This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.

num_reference_events()

Return the number of reference events. All of them should be checked.

reference(index, frame)

This method gets called by the I3FortyTwo module in reference mode. The checker object is then installed in the processing script and passed to the module via the configurable parameters. We compute the test values and store them within this checker object in the reference_list.

report_missing(key)

class icecube.gulliver_modules.fortytwo.I3LogLikelihoodFitParamsChecker(params=[])

Bases: I3FrameChecker

Checks the diagnostics object of gulliver fits.

Diagnostics:

• likelihood value

• reduced likelihood value

• number of degree of freedom

• number of minimizer steps

atols = (0.05, 0.05, 0.0, 100.0)

compare_test_values(values, reference_values)

This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the get_test_values method.

If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).

If the current test values and the reference ones are are all compatible with each other then an empty string (or None, or False) should be returned.

get_test_values(index, frame)

This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.

problems(values, reference_values, mask)

rtols = (0.01, 0.01, 0.0, 0.0)

class icecube.gulliver_modules.fortytwo.I3ParticleChecker(particles=[])

Bases: I3FrameChecker

Checks fit status and direction of I3Particle.

Needs a parameter to tweak the tolerance at which floating point numbers should be compared (with numpy.isclose).

atols = (0.0, 0.0, 0.0, 0.02, 0.02, 0.1, 0.1, 0.1, 0.1, 0.1, 0.01)

compare_test_values(values, reference_values)

This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the get_test_values method.

If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).

If the current test values and the reference ones are are all compatible with each other then an empty string (or None, or False) should be returned.

get_test_values(index, frame)

This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.

problems(values, reference_values, mask)

rtols = (0.0, 0.0, 0.0, 0.001, 0.001, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01)

icecube.gulliver_modules.fortytwo.MultiplicityCutModule(frame, pulses, nchmin=0, nchmax=10000, nstrmin=0, nstrmax=10000)

class icecube.gulliver_modules.fortytwo.Pcount(context)

Bases: I3ConditionalModule

Progress meter

Silly for default test, but useful if you use the cmdline options to e.g. run this on a larger input file. There is an I3EventCounter in phys-services, but that is clunky. It requires an XML files service.

Configure((I3ConditionalModule)arg1) → None :

C++ signature :

void Configure(PythonModule<I3ConditionalModule> {lvalue})

DAQ((I3ConditionalModule)arg1, (I3Frame)arg2) → None :

C++ signature :

void DAQ(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)

Physics((I3ConditionalModule)arg1, (I3Frame)arg2) → None :

C++ signature :

void Physics(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)

icecube.gulliver_modules.fortytwo.isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)

Copied from a modern numpy distribution.

We are using this copy here because the build bots builds.icecube.wisc.edu are still using a very old numpy distribution which apparently does not know about the isclose function. Once they are updated, we cane remove this copy and just use numpy.isclose directly.

Modifcation for integer types: just require a and b to be equal, ignore tolerances.