Simplest way to run the same suite of tests on multiple devices

[tadeas-vintrlik]

Hi,

I was wondering anyone might have stumbled upon a similar problem.

Infrastructure and setup

Let's say I have 5 devices connected via serial to raspberry pi running the exporter which expoerts these 5 devices as separate places.

Test

For simplicity sake let's suppose I want to run a trivial test to see if all of the serial ports are working by using a simple kind of hello world test like this:

def test_echo(target):
    shell = target.get_driver('CommandProtocol')
    result = shell.run_check('echo OK')
    assert 'OK' in result

My current solution is a setup little like this:

Configuration files for devices

• I have environment configuration file for the tests per device like this

targets:
  main:
    resources:
      RemotePlace:
        name: device-1
    drivers:
      ManualPowerDriver:
        name: "example"
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'
        password: ****

• these are all identcial except the RemotePlace

Shell script to run all the tests

• I then have a shell script which does the following:
  1. loop through all the devices and acquire their places
  2. run the test with the matching environment config passed via --lg-env
  3. unlock all of the places

Questions

My question is: Is there a simpler way to do this?

Is there some way to specify the environment for all places in a single yaml?

When I tried that it either only ran on the target called main or would move the duplication from configuration into the tests themselves having to do it like in #1375

Is there a way acquire the devices automatically and unlock them within the tests?

Ideally I would like to run a simple pytest --lg-env shared.yaml tests/test_shared.yaml without duplication in neither the environment nor the tests code, but if that isn't possible what is presently the simplest way possible? Or is my approach good enough?

Thank you for any suggestions.

[sessl3r]

The method I used in the past was to use templating in the environment config and then call pytest multiple times.

For your config that would mean:

targets:
  main:
    resources:
      RemotePlace:
        name: !template $LG_PLACE
    drivers:
      ManualPowerDriver:
        name: "example"
      SerialDriver: {}
      ShellDriver:
        prompt: 'root@\w+:[^ ]+ '
        login_prompt: ' login: '
        username: 'root'
        password: ****

And then call it like:

for p in $(cat places.txt); do LG_PLACE=$p pytest; done

Of course in this for-loop the acquire/release should be also done to not block all the places all the time.
Also you could run tests in parallel with this approach I think (never tested).