I3Exec

The task runner.

Run it with python -m iceprod.core.i3exec.

optional arguments:
-h, --help

show this help message and exit

-f CFGFILE, --cfgfile CFGFILE

Specify config file

-u URL, --url URL

URL of the iceprod server

-p PASSKEY, --passkey PASSKEY

passkey for communcation with iceprod server

--pilot_id PILOTID

ID of the pilot (if this is a pilot)

-d, --debug

Enable debug actions and logging

--offline

Enable offline mode (don’t talk with server)

–offline_transfer True/False

Enable offline file transfer

--logfile LOGFILE

Specify the logfile to use

--job JOB

Index of the job to run

--task TASK

Name of the task to run

--gzip-logs

gzip the iceprod logs

iceprod.core.i3exec.load_config(cfgfile)[source]

Load a config from file, serialized string, dictionary, etc

iceprod.core.i3exec.main(cfgfile=None, logfile=None, url=None, debug=False, passkey='', pilot_id=None, offline=False, offline_transfer=False, gzip_logs=False)[source]

Main task runner for iceprod

async iceprod.core.i3exec.runner(config, rpc=None, debug=False, offline=False, offline_transfer=False, resources=None)[source]

Run a config.

  1. Set some default options if not set in configuration.

  2. Set up global env based on the configuration.

  3. Run tasks * If a task is specified in the configuration options:

    If the task is specified by name or number, run only that task. If there is a problem finding the task specified, raise a critical error.

    • Otherwise, run all tasks in the configuration in the order they were written.

  4. Destroy the global env, uploading and deleting files as needed.

  5. Upload the log, error, and output files if specified in options.

Parameters:

  • config (iceprod.core.dataclasses.Job) – Dataset configuration

  • rpc (iceprod.core.exe_json.ServerComms) – RPC object

  • debug (bool) – (optional) turn on debug logging

  • offline (bool) – (optional) enable offline mode

  • offline_transfer (bool) – (optional) enable/disable offline data transfers

  • resources (iceprod.core.resources.Resources) – (optional) Resources object