210 lines
6.7 KiB
ReStructuredText
210 lines
6.7 KiB
ReStructuredText
.. SPDX-License-Identifier: GPL-2.0+
|
|
|
|
Sandbox tests
|
|
=============
|
|
|
|
Test Design
|
|
-----------
|
|
|
|
Most uclasses and many functions of U-Boot have sandbox tests. This allows much
|
|
of the code to be checked in an developer-friendly environment.
|
|
|
|
Sandbox provides a way to write and run unit tests. The traditional approach to
|
|
unit tests is to build lots of little executables, one for each test or
|
|
category of tests. With sandbox, so far as possible, all the tests share a
|
|
small number of executables (e.g. 'u-boot' for sandbox, 'u-boot-spl' and
|
|
'u-boot' for sandbox_spl) and can be run very quickly. The vast majority of
|
|
tests can run on the 'sandbox' build,
|
|
|
|
Available tests
|
|
---------------
|
|
|
|
Some of the available tests are:
|
|
|
|
- command_ut: Unit tests for command parsing and handling
|
|
- compression: Unit tests for U-Boot's compression algorithms, useful for
|
|
security checking. It supports gzip, bzip2, lzma and lzo.
|
|
- image: Unit tests for images:
|
|
|
|
- test/image/test-imagetools.sh - multi-file images
|
|
- test/py/tests/test-fit.py - FIT images
|
|
- tracing: test/trace/test-trace.sh tests the tracing system (see
|
|
README.trace)
|
|
- verified boot: test/py/tests/test_vboot.py
|
|
|
|
If you change or enhance any U-Boot subsystem, you should write or expand a
|
|
test and include it with your patch series submission. Test coverage in some
|
|
older areas of U-Boot is still somewhat limited and we need to work to improve
|
|
it.
|
|
|
|
Note that many of these tests are implemented as commands which you can
|
|
run natively on your board if desired (and enabled).
|
|
|
|
To run all tests, use 'make check'.
|
|
|
|
|
|
Running sandbox tests directly
|
|
------------------------------
|
|
|
|
Typically tests are run using the pytest suite. Running pytests on sandbox is
|
|
easy and always gets things right. For example some tests require files to be
|
|
set up before they can work.
|
|
|
|
But it is also possible to run some sandbox tests directly. For example, this
|
|
runs the dm_test_gpio() test which you can find in test/dm/gpio.c::
|
|
|
|
$ ./u-boot -T -c "ut dm gpio"
|
|
|
|
|
|
U-Boot 2021.01
|
|
|
|
Model: sandbox
|
|
DRAM: 128 MiB
|
|
WDT: Started with servicing (60s timeout)
|
|
MMC: mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
|
|
In: serial
|
|
Out: vidconsole
|
|
Err: vidconsole
|
|
Model: sandbox
|
|
SCSI:
|
|
Net: eth0: eth@10002000, eth5: eth@10003000, eth3: sbe5, eth6: eth@10004000
|
|
Test: dm_test_gpio: gpio.c
|
|
Test: dm_test_gpio: gpio.c (flat tree)
|
|
Failures: 0
|
|
|
|
The -T option tells the U-Boot sandbox to run with the 'test' devicetree
|
|
(test.dts) instead of -D which selects the normal sandbox.dts - this is
|
|
necessary because many tests rely on nodes or properties in the test devicetree.
|
|
If you try running tests without -T then you may see failures, like::
|
|
|
|
$ ./u-boot -c "ut dm gpio"
|
|
|
|
|
|
U-Boot 2021.01
|
|
|
|
DRAM: 128 MiB
|
|
WDT: Not found!
|
|
MMC:
|
|
In: serial
|
|
Out: serial
|
|
Err: serial
|
|
SCSI:
|
|
Net: No ethernet found.
|
|
Please run with test device tree:
|
|
./u-boot -d arch/sandbox/dts/test.dtb
|
|
Test: dm_test_gpio: gpio.c
|
|
test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
|
|
Test: dm_test_gpio: gpio.c (flat tree)
|
|
test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
|
|
Failures: 2
|
|
|
|
The message above should provide a hint if you forget to use the -T flag. Even
|
|
running with -D will produce different results.
|
|
|
|
You can easily use gdb on these tests, without needing --gdbserver::
|
|
|
|
$ gdb u-boot --args -T -c "ut dm gpio"
|
|
...
|
|
(gdb) break dm_test_gpio
|
|
Breakpoint 1 at 0x1415bd: file test/dm/gpio.c, line 37.
|
|
(gdb) run -T -c "ut dm gpio"
|
|
Starting program: u-boot -T -c "ut dm gpio"
|
|
Test: dm_test_gpio: gpio.c
|
|
|
|
Breakpoint 1, dm_test_gpio (uts=0x5555558029a0 <global_dm_test_state>)
|
|
at files/test/dm/gpio.c:37
|
|
37 ut_assertok(gpio_lookup_name("b4", &dev, &offset, &gpio));
|
|
(gdb)
|
|
|
|
You can then single-step and look at variables as needed.
|
|
|
|
|
|
Running sandbox_spl tests directly
|
|
----------------------------------
|
|
|
|
SPL is the phase before U-Boot proper. It is present in the sandbox_spl build,
|
|
so you can run SPL like this::
|
|
|
|
./spl/u-boot-spl
|
|
|
|
SPL tests are special in that they run (only in the SPL phase, of course) if the
|
|
-u flag is given::
|
|
|
|
./spl/u-boot-spl -u
|
|
|
|
U-Boot SPL 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
|
|
Running 5 driver model tests
|
|
Test: dm_test_of_plat_base: of_platdata.c (flat tree)
|
|
Test: dm_test_of_plat_dev: of_platdata.c (flat tree)
|
|
Test: dm_test_of_plat_parent: of_platdata.c (flat tree)
|
|
Test: dm_test_of_plat_phandle: of_platdata.c (flat tree)
|
|
Test: dm_test_of_plat_props: of_platdata.c (flat tree)
|
|
Failures: 0
|
|
|
|
|
|
U-Boot 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
|
|
|
|
DRAM: 128 MiB
|
|
...
|
|
|
|
It is not possible to run SPL tests in U-Boot proper, firstly because they are
|
|
not built into U-Boot proper and secondly because the environment is very
|
|
different, e.g. some SPL tests rely on of-platdata which is only available in
|
|
SPL.
|
|
|
|
Note that after running, SPL continues to boot into U-Boot proper. You can add
|
|
'-c exit' to make U-Boot quit without doing anything further. It is not
|
|
currently possible to run SPL tests and then stop, since the pytests require
|
|
that U-Boot produces the expected banner.
|
|
|
|
You can use the -k flag to select which tests run::
|
|
|
|
./spl/u-boot-spl -u -k dm_test_of_plat_parent
|
|
|
|
Of course you can use gdb with sandbox_spl, just as with sandbox.
|
|
|
|
|
|
Running all tests directly
|
|
--------------------------
|
|
|
|
A fast way to run all sandbox tests is::
|
|
|
|
./u-boot -T -c "ut all"
|
|
|
|
It typically runs single-thread in 6 seconds on 2021 hardware, with 2s of that
|
|
to the delays in the time test.
|
|
|
|
This should not be considered a substitute for 'make check', but can be helpful
|
|
for git bisect, etc.
|
|
|
|
|
|
What tests are built in?
|
|
------------------------
|
|
|
|
Whatever sandbox build is used, which tests are present is determined by which
|
|
source files are built. For sandbox_spl, the of_platdata tests are built
|
|
because of the build rule in test/dm/Makefile::
|
|
|
|
ifeq ($(CONFIG_SPL_BUILD),y)
|
|
obj-$(CONFIG_SPL_OF_PLATDATA) += of_platdata.o
|
|
else
|
|
...other tests for non-spl
|
|
endif
|
|
|
|
You can get a list of tests in a U-Boot ELF file by looking for the
|
|
linker_list::
|
|
|
|
$ nm /tmp/b/sandbox_spl/spl/u-boot-spl |grep 2_dm_test
|
|
000000000001f200 D _u_boot_list_2_dm_test_2_dm_test_of_plat_base
|
|
000000000001f220 D _u_boot_list_2_dm_test_2_dm_test_of_plat_dev
|
|
000000000001f240 D _u_boot_list_2_dm_test_2_dm_test_of_plat_parent
|
|
000000000001f260 D _u_boot_list_2_dm_test_2_dm_test_of_plat_phandle
|
|
000000000001f280 D _u_boot_list_2_dm_test_2_dm_test_of_plat_props
|
|
|
|
|
|
Writing tests
|
|
-------------
|
|
|
|
See :doc:`tests_writing` for how to write new tests.
|
|
|