Tests

Overview

We run tests via Tox. Tox will set up a separate environment for each test in .tox. This directory can be deleted, as tox will reinstall it on-demand.

Installation

Python 3.10-3.12 tested.

From toml file:

$ pip install ".[tests]"

Manually in your virtual environment:

$ pip install tox

List available tests

$ tox list
default environments:
py310            -> Run pytest, coverage.
py311            -> Run pytest, coverage.
py312            -> Run pytest, coverage.
bandit           -> "Check for security issues. Ignoring asserts for now."
ruff_reformat    -> "Reformat code with ruff format, a Python formatter like black."
ruff_check       -> "Run ruff check, a Python linter. Auto-fix only isort, see pyproject.toml."

additional environments:
prospector       -> "Run static analysis using prospector, don't fail on errors"
ruff_checkformat -> "Run ruff format and complain on changes"
mypy             -> "Check typing with mypy. Only warn, don't fail on errors."
train            -> "Run a test training. Will download data from github. Slow and large! May take about 15 minutes. Result end up in .tox/train/tmp/training-output"

Disk usage

Only some of the tests need a full installation:

$ du -hsc .tox/*
41M	.tox/bandit
63M	.tox/mypy
6,7G	.tox/prospector
12K	.tox/py310
12K	.tox/py311
6,7G	.tox/py312
41M	.tox/ruff_check
41M	.tox/ruffcheck
41M	.tox/ruff_checkformat
41M	.tox/ruffformat
41M	.tox/ruffformat_check
41M	.tox/ruff_reformat
6,8G	.tox/train

Run default tests

$ tox
py310: skipped because could not find python interpreter with spec(s): py310
py310: SKIP ⚠ in 0.02 seconds
py311: skipped because could not find python interpreter with spec(s): py311
py311: SKIP ⚠ in 0.01 seconds
.pkg: _optional_hooks> python /home/larsfp/src/bris-inference/venv/lib/python3.12/site-packages/pyproject_api/_backe
...
All checks passed!
  py310: SKIP (0.02 seconds)
  py311: SKIP (0.01 seconds)
  py312: OK (11.17=setup[2.83]+cmd[8.18,0.16] seconds)
  bandit: OK (0.34=setup[0.01]+cmd[0.33] seconds)
  ruff_reformat: OK (0.05=setup[0.01]+cmd[0.04] seconds)
  ruff_check: OK (0.05=setup[0.01]+cmd[0.05] seconds)
  congratulations :) (11.69 seconds)

Details for each test

Unit tests

py310
py311
py312

pytest will run unit tests in all python versions between 3.10 and 3.12 that are available on the system. Our CI use 3.10.

Security

bandit will check for known security issues in the code (not in libraries), like writing files to a shared temporary directory.

Formatting

Formatting is checked with The Ruff Formatter.

CI will run ruff_checkformat which only reports if formatting is needed. You will have to run the actual formatting using tox -e ruff_reformat, and then commit it.

Lint

Linting is done with The Ruff Linter. isort-changes will be auto-fixed, other errors will be reported. Must run without errors reported before pushing.

tox -e ruff_check

Typing

Type checking with mypy is still a work in process.

Prospector

Other code issues with prospector is still a work in process.

Train

Will train 🚂 a test model using a synthetic dataset from github. Slow!

Requires wget unzip git