Users can receive notifications containing information about submitted test jobs and the accompanying results by using settings in job definition yaml. Users can also compare the job results within notifications against previous results which satisfy certain conditions.
The basic setup of the
notify block in job definitions will have
criteria, verbosity, recipients and compare blocks.
The basic email template includes job details, device details and results,
based on the level of verbosity (
The method and destination of each notification can be set for each recipient. Currently, two notification methods are supported, email and IRC.
Recipients can be specified using the LAVA username or in full.
LAVA users can configure the
IRC settings or email address in their own
profile data in the instance. This allows the recipient to be specified using
only the LAVA username.
If the user has not configured their own profile data, the recipient details must be specified in full.
Examples for user vs manual addressing:
notify: - to: method: irc user: neil.williams - to: method: irc handle: codehelp server: irc.freenode.net - to: method: email user: neil.williams - to: method: email email: firstname.lastname@example.org
IRC and email notifications use different templates since emails allow for more verbosity, so some options which are present in notify block for email recipients will be obsolete for IRC recipients, like comparing options and verbosity.
Users can use notifications to compare the job status against the last few jobs and also compare job results, like test case/suites difference. The notification will report if there are new test cases compared to previous test jobs, if there are some missing and how the said results differ.
Which test jobs the current one is compared to is determined by the LAVA Queries system, where those previous jobs will be taken from the query specified in the notification block. Custom queries (Query by URL) can also be specified.
Blacklisting test cases is also an option. Ignoring some of the test cases in the comparing section of the notification is possible through an option in the notification block.
Here are some comparing setup examples from test definition excerpts:
notify: - to: method: email user: neil.williams criteria: status: complete verbosity: verbose compare: query: name: query_name username: query_owner blacklist: [lava, singlenode-advanced]
notify: - to: method: email user: neil.williams criteria: status: complete verbosity: verbose compare: query: entity: testjob conditions: submitter: stevan.radakovic requested_device_type: qemu blacklist: [singlenode-advanced]