Execution Functions

The core execution functions for running on a node. These are all called from iceprod.core.i3exec.

The fundamental design of the core is to run a task composed of trays and modules. The general heirarchy looks like:

task
|
|- tray1
   |
   |- module1
   |
   |- module2
|
|- tray2
   |
   |- module3
   |
   |- module4

Parameters can be defined at every level, and each level is treated as a scope (such that inner scopes inherit from outer scopes). This is accomplished via an internal evironment for each scope.

Environment Functions

The internal IceProd environment, designed to be nested and clean up after itself.

class iceprod.core.exe.Config(config=None, parser=None, rpc=None, logger=None)

Contain the configuration and related methods

parseValue(value, env={})

Parse a value from the available env and global config.

Uses the Meta Parser on any string value. Pass-through for any other object.

Parameters:

• value – The value to parse

• env – The environment to use, optional

Returns:

The parsed value

parseObject(obj, env)

Recursively parse a dict or list

Run Functions

The main execution flow goes through here.

async iceprod.core.exe.runtask(cfg, globalenv, task, logger=None)

Run the specified task

async iceprod.core.exe.runtray(cfg, globalenv, tray, stats={}, logger=None)

Run the specified tray

async iceprod.core.exe.runmodule(cfg, globalenv, module, stats={}, logger=None)

Run the specified module

Object Setup

Download/upload and setup the main IceProd objects (Resources, Data, Classes).

async iceprod.core.exe.downloadResource(env, resource, remote_base=None, local_base=None, checksum=None, logger=None)

async iceprod.core.exe.downloadData(env, data, logger=None)

Download data and put location in the env

async iceprod.core.exe.uploadData(env, data, logger=None)

Upload data

async iceprod.core.exe.setupClass(env, class_obj, logger=None)

Set up a class for use in modules, and put it in the env