Configuration and logging

The configuration is automatically read from /etc/livereduce.confunless specified as a command line argument. Defaults will be attempted to be determined from the environment. A minimal configuration to specify using nightly builds of mantid installed in a conda environment mantid-dev is

{
  "instrument": "PG3",
  "CONDA_ENV": "mantid-dev"
}

For testing a configuration file can be supplied as a command line argument when running

$ python scripts/livereduce.py ./livereduce.conf

The logfile of what was setup for running, as well as other messages, is /var/log/SNS_applications/livereduce.log if run as the user snsdata, or livereduce.log in the current working directory (if run from the command line).

If run from inside systemctl, use the standard commands for starting and stopping it.

sudo systemctl start livereduce
sudo systemctl stop livereduce
sudo systemctl restart livereduce

The status of the service can be found via

sudo systemctl status livereduce status

the logs are stored in /var/log/SNS_applications/livereduce.log and are readable by anyone. People with extra permissions can run sudo journalctl -u livereduce -f and see all of the logs without them flushing on restart of the service. Sometimes the service refuses to restart, in that case stop then start it in separate commands.

If the instrument is not defined in the configuration file (default is /etc/livereduce.conf) the software will ask mantid for the default instrument using mantid.kerel.ConfigService.getInstrument() (docs). The default instrument is controlled in the mantid properties files and is typically defined in /etc/mantid.local.properties.

The script files that are used/looked for are

• <script_dir>/reduce_<instrument>_proc.py is the processing script (for each chunk). This is required.
• <script_dir>/reduce_<instrument>_post_proc.py is the post-processing script (for the accumulated data). To disable this step rename the python script so it is not found by the daemon.

Example filenames for NOMAD with default script location is /SNS/NOM/shared/livereduce/reduce_NOM_live_proc.py and /SNS/NOM/shared/livereduce/reduce_NOM_live_post_proc.py.

Behavior

The daemon will immediately cancel StartLiveData and MonitorLiveData and restart them when one of processing scripts is changed (verified by md5sum) or removed. This is to be resilient against changes in the scripts.

The process will exit and systemd will restart it if the configuration file is changed. This is done in case the version of mantid wanted is changed.

Building and packaging

Testing is described in the test/ subdirectory.

This package uses a hand-written spec file for releasing on rpm based systems rather than the one generated by python. To run it execute

./rpmbuild.sh

And look for the results in the dist directory.

This package depends on pyinotify and (of course) mantid.

Developer notes

This repository is configured to use pre-commit. This can be done using conda via

conda env create  # looks at environment.yml
pre-commit install

Python 3 compatibility

livereduce.py is python2/3 compatible. Because the livereduce.service file is written to use system python, whether it is actually using python 2 or 3 will depend on your system's default python. The other thing to note, is that livereduce.py imports mantid directly in its namespace so you must run livereduce.py using the same python that mantid was built against.

Acknowledgements and other links

Information and ideas taken from:

• StatisticsService
• autoreduce
• post_processing_agent
• Logging in python
• systemd and systemd.service