Writing a monitors test action¶
See also
Writing a Lava-Test Test Definition 1.0 involves adding an overlay of POSIX scripts to
the DUT but some devices cannot support an overlay. The monitors
test action supports IoT devices using pattern matching against
non-interactive output streams. As with pattern matching in the
Lava-Test Test Definition 1.0, this
can be difficult at times.
The IoT application needs to be specially written to create output
which is compatible with automation. Test actions for IoT devices tend
to be short and specific to one dedicated application executing on the
DUT. Applications need to be written to emit a START
and STOP
message which have to be unique across all test job output, not just
the test action. Test writers then need to correlate specific pattern
match expressions in the test job with the behavior of the application
deployed in that test job. This puts the work of creating a parser into
the hands of the writers of the test application and can thus interfere
with development of the test application.
Restrictions¶
Test jobs using Monitors must:
Use carefully designed applications which are designed to automatically execute after boot.
Emit a unique start string:
Only once per boot operation.
Before any test operation starts.
Emit a unique end string:
Only once per boot operation.
After all test operations have completed.
Provide a regular expression which matches all expected test output and maps the output to results without leading to excessively long test case names.
start and end strings will match part of a line but make sure that each string is long enough that it can only match once per boot.
If start does not match, the job will timeout with no results.
If end does not match, the job will timeout but the results (of the current boot) will already have been reported.
Example test monitors job¶
1# Zephyr JOB definition for IoT
2device_type: frdm-kw41z
3job_name: frdm-kw41z zephyr IoT example
4
5timeouts:
6 job:
7 minutes: 10
8 action:
9 minutes: 3
10 actions:
11 wait-usb-device:
12 seconds: 40
13
14priority: medium
15visibility: public
16
17actions:
18- deploy:
19 timeout:
20 minutes: 3
21 to: tmpfs
22 images:
23 zephyr:
24 url: http://images.validation.linaro.org/snapshots.linaro.org/components/kernel/zephyr/master/zephyr/frdm_kw41z/737/tests/kernel/libs/test/zephyr.bin
25
26- boot:
27 method: pyocd
28 timeout:
29 minutes: 10
30
31- test:
32 monitors:
33 - name: tests
34 start: BOOTING ZEPHYR
35 end: PROJECT EXECUTION SUCCESSFUL
36 pattern: '(?P<test_case_id>.*) (?P<measurement>.*) tcs = [0-9]* nsec'
37 fixupdict:
38 PASS: pass
39 FAIL: fail
Download or view example test monitors job: examples/test-jobs/frdm-kw41z-zephyr.yaml