Deploying test images using LXC¶
Containers are lightweight virtualization technology. LXC is a userspace interface for the Linux kernel containment features. Through a powerful API and simple tools, it lets Linux users easily create and manage system or application containers. The container provides a lightweight method to allow custom software to be used on the dispatcher. The container is used to provide transparent access.
LAVA supports LXC containers both as a standalone device type and as dynamic transparent environments in order to interact with external devices. In either case the LXC protocol is used.
Using LXC as Device Type¶
LXC is a device type of its own and devices could be added to dispatchers under this device type. A device of LXC device type is created within the dispatcher in which the device is configured, as illustrated in the following figure:
protocols: lava-lxc: name: pipeline-lxc-test distribution: debian release: sid arch: amd64
Sample Job Definition¶
device_type: lxc job_name: lxc-debian timeouts: job: minutes: 30 action: minutes: 5 priority: medium visibility: public metadata: source: https://git.linaro.org/lava-team/refactoring.git path: lxc-debian.yaml protocols: lava-lxc: name: pipeline-lxc-test distribution: debian release: sid arch: amd64 actions: - deploy: timeout: minutes: 30 to: lxc - boot: prompts: - '[root@(.*) /]#' timeout: minutes: 5 method: lxc - test: timeout: minutes: 5 definitions: - repository: http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests
Namespaces were introduced to handle use-cases specific to LXC, but the principle can be expanded to other use-cases as and when required. In a job definition where multiple deploy, boot and test actions are specified, there must be a mechanism to describe how the actions are connected. This is the primary purpose of a namespace; it is the way to tie related actions together. The namespace itself is simply a label, test writers are advised to make the label chosen for each namespace meaningful for the purposes of the test job.
In the example below, there are two namespaces - one for the deploy, boot and test actions to perform inside the LXC and one for the deploy, boot and test actions to be performed on the DUT. To support this particular device, the test job needs to:
deploy the container, including:
install software inside the container to control the device. In this case
boot the container
deploy files to the device
In this case, this connects to and turns on power to the device then uses software in the container to push files to the device using the bootloader.
boot the device
run a test shell on the device
run a test shell in the container.
Note how the deploy, boot and test actions are interleaved. The use of
namespaces is essential for the test shell in the container to be able to find
and execute commands in the container. In this example, the software running in
the container and the software running on the device need to be handled quite
differently in each test shell. For example, when installing dependencies
inside the container running Debian, the
apt package manager is available.
When installing dependencies in the test shell on the device, running
OpenEmbedded, there might not be any package manager support. The namespace
data is used to let each test shell identify the default shell and other data
about the environment in each namespace.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81
actions: - deploy: namespace: tlxc timeout: minutes: 15 to: lxc packages: - android-tools-fastboot - boot: namespace: tlxc prompts: - 'root@(.*):/#' - 'hikey:/' timeout: minutes: 5 method: lxc - deploy: timeout: minutes: 30 to: fastboot namespace: hikey-oe connection: lxc images: ptable: url: http://images.validation.linaro.org/snapshots.linaro.org/openembedded/lkft/morty/hikey/rpb/4.9/83/bootloader/ptable-linux-8g.img reboot: hard-reset boot: url: http://images.validation.linaro.org/snapshots.linaro.org/openembedded/lkft/morty/hikey/rpb/4.9/83/boot-0.0+AUTOINC+06e4def583-fb1158a365-r0-hikey-20170713193031-83.uefi.img reboot: hard-reset system: url: http://images.validation.linaro.org/snapshots.linaro.org/openembedded/lkft/morty/hikey/rpb/4.9/83/rpb-console-image-hikey-20170808001820-83.rootfs.img.gz compression: gz apply-overlay: true # ensure that this job raises a network interface with DHCP before relying on internet access protocols: lava-lxc: - action: fastboot-deploy request: pre-power-command timeout: minutes: 2 - boot: namespace: hikey-oe auto_login: login_prompt: 'login:' username: root prompts: - 'root@hikey:~#' timeout: minutes: 5 method: uefi-menu commands: fastboot protocols: lava-lxc: # other action could be boot-fastboot - action: uefi-commands request: pre-os-command timeout: minutes: 2 - test: namespace: hikey-oe timeout: minutes: 5 definitions: - repository: http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests-basic-oe - test: namespace: tlxc timeout: minutes: 5 definitions: - repository: http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests-basic-ubuntu
The two test shells are almost identical but remember that all the
results of this one test job will be reported together. The name of each
test shell definition needs to be different for each test action. So the
name: smoke-tests-basic-oe for the
name: smoke-tests-basic-ubuntu for the
Using the LXC protocol to support Android¶
LAVA Android Naming Conventions¶
production image - a build of Android which, when deployed to a device, means that the device is not visible to
adb. This is typically how a device is configured when first sold to the consumer.
developer image - a build of Android which, when deployed to a device, means that the device is visible to
adb. Devices configured this way will be able to have the image replaced using any machine, just by connecting a suitable cable, so these images are not typically deployed onto hardware which will be sold to the customer without having this image replaced with a production image.
Installing tools like
fastboot on the dispatcher can be
problematic. Some of these issues arise from the need to put many different
types of devices onto a single dispatcher, other issues arise from needing to
use different versions of the build on the devices. Testing an old system may
require downgrading support like
openjdk, new devices or new builds may
require upgrading the same support. Containers isolate this variation so that
each testjob can have a suitable container instead of needing to deal with
changes on the dispatcher:
Shared lock issues - Tools can require use of
flockand similar methods to distinguish a connection to one device from another.
Version disparities - different device versions, different OS versions, may require different support in debug tools like
hardware issues - USB hub variability.
Deploying LXC devices for more information on the administration of LXC for LAVA.
lava-lxc protocol, a Lava Test Shell is provided inside the LXC
to support installing and configuring whatever tools, packages and files which
the testjob will need to use. Installing
adb in this test shell removes the
need to have a POSIX type shell on the device. Files can be pushed and pulled
from the device and executed using the Android support in the image.
Requirements and Limitations¶
The image deployed to the device must enable the Android Debug Bridge, i.e. a developer image. This means enabling developer access over USB or TCP. This rules out the use of production images.
A list of packages to install into the bare container to provide the necessary tools to communicate with the device.
The LXC depends on underlying kernel architecture. For armel, armhf, etc. dispatcher should run on these architectures.
Each distribution has its own template and the templates do not have common options. It can be difficult to have generic support for all distributions.
namespaces to relate different job actions to run in the LXC and for the device.
protocols: lava-lxc: name: lxc-hikey-test template: debian distribution: debian release: jessie mirror: http://mirror.bytemark.co.uk/debian verbose: true actions: # DEPLOY_LXC_BLOCK - deploy: namespace: tlxc timeout: minutes: 5 to: lxc packages: - android-tools-adb - android-tools-fastboot # BOOT_LXC_BLOCK
Feedback from the device¶
Actions within the LXC can cause the device to emit messages on the serial console. Some devices can have problems maintaining the serial connection if this data is not flushed and the data itself can be useful to test writers to debug issues and failures.
LAVA automatically reads from all other namespaces whilst
processing the test shell in another namespace and outputs this as
data. When viewing a test job log file, feedback can be turned on or off using
the buttons at the top of the log file.
To support feedback, the
Individual connection overrides is set to 10 seconds by default.
(There are no suitable prompts to match, so reading feedback continues until
the connection timeout is reached, without failing the test shell itself.)
Differences between LXC releases¶
The release specified in the
lava-lxc protocol will determine some of the
packages which will need to be installed in the container. In particular, any
container based on a Debian later than
jessie will need two packages to be
added to the setup of the container before the container can be used:
systemd-sysv. These two packages must be specified in
the deployment list.
In addition, some packages will have been renamed between releases. For example,
android-tools-adb exists in Debian unstable but it is an old build and will
at some point be replaced by
adb which is also available in unstable but not
Always check the availability of the packages needed for particular releases by using a local chroot or VM. Only packages which are included in the specified release can be installed using the deployment list. Packages from other repositories will have to be installed using the test definition.
Sample Job Definition¶
device_type: hi6220-hikey job_name: lxc-hi6220-hikey timeouts: job: minutes: 60 action: minutes: 15 connection: minutes: 2 priority: medium visibility: public metadata: source: https://git.linaro.org/lava-team/refactoring.git path: hi6220-hikey.yaml protocols: lava-lxc: name: lxc-hikey-test template: debian distribution: debian release: jessie mirror: http://mirror.bytemark.co.uk/debian verbose: true actions: # DEPLOY_LXC_BLOCK - deploy: namespace: tlxc timeout: minutes: 5 to: lxc packages: - android-tools-adb - android-tools-fastboot # BOOT_LXC_BLOCK - boot: namespace: tlxc prompts: - 'root@(.*):' - 'hikey: ' timeout: minutes: 5 method: lxc # DEPLOY_TARGET_BLOCK - deploy: timeout: minutes: 15 namespace: droid to: fastboot images: ptable: url: http://example.com/hikey/ptable-aosp-8g.img reboot: hard-reset boot: url: http://example.com/hikey/boot.img.xz compression: xz reboot: hard-reset cache: url: http://example.com/hikey/cache.img.xz compression: xz userdata: url: http://example.com/hikey/userdata.img.xz compression: xz system: url: http://example.com/hikey/system.img.xz sha256sum: e0e82b5adfae84ff97f4f6488e5b4c64b0dfc7ad8a37b4bcbb887d9f85a6be0a compression: xz protocols: lava-lxc: - action: fastboot-deploy request: pre-power-command timeout: minutes: 2 # BOOT_TARGET_BLOCK - boot: namespace: droid prompts: - 'root@(.*):/#' - 'hikey:/' timeout: minutes: 15 method: fastboot - test: namespace: tlxc timeout: minutes: 10 definitions: - from: inline repository: metadata: format: Lava-Test Test Definition 1.0 name: check-devices description: "LXC dev list" run: steps: - /sbin/ifconfig name: lxc-dev-inline path: inline/lxc-dev.yaml - repository: https://git.linaro.org/lava-team/refactoring.git/ from: git path: android/lava-android-basic-lxc.yaml name: v2-make-adb-connection