Test Action Reference¶

git¶

A remote git repository which needs to be cloned by the dispatcher.

inline¶

A simple test definition present in the same file as the job submission, instead of from a separate file or VCS repository. This allows tests to be run based on a single file. When combined with file:// URLs to the deploy parameters, this allows tests to run without needing external access. See Inline test definition example.

path¶

The path within that repository to the YAML file containing the test definition.

name¶

(required) - replaces the name from the YAML.

params¶

(optional): Pass parameters to the Lava Test Shell Definition. The format is a YAML dictionary - the key is the name of the variable to be made available to the test shell, the value is the value of that variable.

definitions:
    - repository: https://git.linaro.org/lava-team/hacking-session.git
      from: git
      path: hacking-session-debian.yaml
      name: hacking
      params:
       IRC_USER: ""
       PUB_KEY: ""

definitions:
    - repository: git://git.linaro.org/lava-team/lava-functional-tests.git
      from: git
      path: lava-test-shell/smoke-tests-basic.yaml
      name: smoke-tests
    - repository: https://git.linaro.org/lava-team/lava-functional-tests.git
      from: git
      path: lava-test-shell/single-node/singlenode03.yaml
      name: singlenode-advanced

deps¶

skip running lava-install-packages for the deps: list of the install: section of the definition.

keys¶

skip running lava-add-keys for the keys: list of the install: section of the definition.

sources¶

skip running lava-add-sources for the sources: list of the install: section of the definition.

steps¶

skip running any of the steps:``of the ``install: section of the definition.

all¶

identical to ['deps', 'keys', 'sources', 'steps']

Example syntax:

- test:
    failure_retry: 3
    name: kvm-basic-singlenode
    timeout:
      minutes: 5
    definitions:
      - repository: git://git.linaro.org/lava-team/lava-functional-tests.git
        from: git
        path: lava-test-shell/smoke-tests-basic.yaml
        name: smoke-tests
      - repository: http://git.linaro.org/lava-team/lava-functional-tests.git
        skip_install:
        - all
        from: git
        path: lava-test-shell/single-node/singlenode03.yaml
        name: singlenode-advanced

The following will skip dependency installation and key addition in the same definition:

- test:
    failure_retry: 3
    name: kvm-basic-singlenode
    timeout:
      minutes: 5
    definitions:
      - repository: git://git.linaro.org/lava-team/lava-functional-tests.git
        from: git
        path: lava-test-shell/smoke-tests-basic.yaml
        name: smoke-tests
      - repository: http://git.linaro.org/lava-team/lava-functional-tests.git
        skip_install:
        - deps
        - keys
        from: git
        path: lava-test-shell/single-node/singlenode03.yaml
        name: singlenode-advanced

Result checks¶

LAVA collects results from internal operations as well as from the submitted test definitions, these form the lava test suite results. The full set of results for a job are available at:

results/1234

LAVA records when a submitted test definition starts execution on the test device. If the number of test definitions which started is not the same as the number of test definitions submitted (allowing for the lava test suite results), a warning will be displayed on this page.

TestSets¶

A TestSet is a group of lava test cases which will be collated within the LAVA Results. This allows queries to look at a set of related test cases within a single definition.

name: testset-def
  run:
      steps:
          - lava-test-set start first_set
          - lava-test-case date --shell ntpdate-debian
          - ls /
          - lava-test-case mount --shell mount
          - lava-test-set stop
          - lava-test-case uname --shell uname -a

This results in the date and mount test cases being included into a first_set TestSet, independent of other test cases. The TestSet is concluded with the lava-test-set stop command, meaning that the uname test case has no test set, providing a structure like:

results:
  first_set:
    date: pass
    mount: pass
  uname: pass

{'results': {'first_set': {'date': 'pass', 'mount': 'pass'}, 'uname': 'pass'}}

Each TestSet name must be valid as a URL, which is consistent with the requirements for test definition names and test case names in the V1 dispatcher.

For TestJob 1234, the uname test case would appear as:

results/1234/testset-def/uname

The date and mount test cases are referenced via the TestSet:

results/1234/testset-def/first_set/date
results/1234/testset-def/first_set/mount

A single test definition can start and stop different TestSets in sequence, as long as the name of each TestSet is unique for that test definition.