Commit Graph

2568 Commits

Author SHA1 Message Date
Simon Glass
f1a83abe60 buildman: Use bytes for the environment
At present we sometimes see problems in gitlab where the environment has
0x80 characters or sequences which are not valid UTF-8.

Avoid this by using bytes for the environment, both internal to buildman
and when writing out the 'env' file. Add a test to make sure this works
as expected.

Reported-by: Marek Vasut <marex@denx.de>
Fixes: e5fc79ea71 ("buildman: Write the environment out to an 'env' file")
Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Simon Glass
8116c78ffd buildman: Handle exceptions in threads gracefully
There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.

Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Simon Glass
ab9b4f35e3 buildman: Use common code to send an result
At present the code to report a build result is duplicated. Put it in a
common function to avoid this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Simon Glass
249933136f buildman: Tidy up a few comments
Add some function comments which are missing, or missing arguments.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Simon Glass
1736575b0c binman: Support adding sections to FMAPs
When used with hierarchical images, use the Chromium OS convention of
adding a section before all the subentries it contains.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Simon Glass
c7722e8417 binman: Tweak implementation of fmap
Use an interator in two of the fmap tests so it is easier to add new
items. Also check the name first since that is the first indication
that something is wrong. Use a variable for the expected size of the
fmap to avoid repeating the code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Evan Benn
ec6db6c297 patman: Parse checkpatch by message instead of by line
Parse each empty-line-delimited message separately. This saves having to
deal with all the different line content styles, we only care about the
header ERROR | WARNING | NOTE...

Also make checkpatch print line information for a uboot specific
warning.

Signed-off-by: Evan Benn <evanbenn@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Tom Rini
2959a8e3a5 patman: Assume we always have pygit2 for tests
Given that we have tests that require pygit2 and it can be installed
like any other python module, fail much more loudly if it is missing.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2021-04-29 03:23:39 -07:00
Simon Glass
170732523b dtoc: Correct dtoc output when testing
At present each invocation of run_steps() updates OUTPUT_FILES_COMMON,
since it does not make a copy of the dict. This is fine for a single
invocation, but for tests, run_steps() is invoked many times.

As a result it may include unwanted items from the previous run, if it
happens that a test runs twice on the same CPU. The problem has not been
noticied previously, as there are few enough tests and enough CPUs that
is is rare for the 'wrong' combination of tests to run together.

Fix this by making a copy of the dict, before updating it. Update the
tests to suit, taking account of the files that are no-longer generated.

With this fix, we no-longer generate files which are not needed for a
particular state of OF_PLATDATA_INST, so the check_instantiate() function
is not needed anymore. It has become dead code and so fails the
code-coverage test (dtoc -T). Remove it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:37 -07:00
Simon Glass
0fe44dc676 binman: Correct testSplNoDtb() and Tpl also
These two tests require an ELF image so that symbol information can be
written into the SPL/TPL binary. At present they rely on other tests
having set it up first, but every test must run independently. This can
cause occasional errors in CI.

Fix this by setting up the required files, as other tests do.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2021-04-29 03:20:54 -07:00
Alexandru Gagniuc
36bfcb62b3 mkimage: Add a 'keyfile' argument for image signing
It's not always desirable to use 'keydir' and some ad-hoc heuristics
to get the filename of the signing key. More often, just passing the
filename is the simpler, easier, and logical thing to do.

Since mkimage doesn't use long options, we're slowly running out of
letters. I've chosen '-G' because it was available.

Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14 15:23:01 -04:00
Alexandru Gagniuc
ed6c9e0b66 lib: Add support for ECDSA image signing
mkimage supports rsa2048, and rsa4096 signatures. With newer silicon
now supporting hardware-accelerated ECDSA, it makes sense to expand
signing support to elliptic curves.

Implement host-side ECDSA signing and verification with libcrypto.
Device-side implementation of signature verification is beyond the
scope of this patch.

Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14 15:06:08 -04:00
Alexandru Gagniuc
4c17e5f691 lib/rsa: Make fdt_add_bignum() available outside of RSA code
fdt_add_bignum() is useful for algorithms other than just RSA. To
allow its use for ECDSA, move it to a common file under lib/.

The new file is suffixed with '-libcrypto' because it has a direct
dependency on openssl. This is due to the use of the "BIGNUM *" type.

Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14 15:06:08 -04:00
Alexandru Gagniuc
0bcb28dfb9 lib: Rename rsa-checksum.c to hash-checksum.c
rsa-checksum.c sontains the hash_calculate() implementations. Despite
the "rsa-" file prefix, this function is useful for other algorithms.

To prevent confusion, move this file to lib/, and rename it to
hash-checksum.c, to give it a more "generic" feel.

Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14 15:06:08 -04:00
Trevor Woerner
dc514d7ea2 moveconfig.py: add to the "do not process" list
Skip the processing of *.aml and *.dat files while iterating through the
source in order to process header files.

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
2021-04-12 17:44:55 -04:00
Sean Anderson
d9c3050710 checkpatch: Add warnings for using strn(cat|cpy)
strn(cat|cpy) has a bad habit of not nul-terminating the destination,
resulting in constructions like

	strncpy(foo, bar, sizeof(foo) - 1);
	foo[sizeof(foo) - 1] = '\0';

However, it is very easy to forget about this behavior and accidentally
leave a string unterminated. This has shown up in some recent coverity
scans [1, 2] (including code recently touched by yours truly).

Fortunately, the guys at OpenBSD came up with strl(cat|cpy), which always
nul-terminate strings. These functions are already in U-Boot, so we should
encourage new code to use them instead of strn(cat|cpy).

[1] https://lists.denx.de/pipermail/u-boot/2021-March/442888.html
[2] https://lists.denx.de/pipermail/u-boot/2021-January/438073.html

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-12 17:44:55 -04:00
Heinrich Schuchardt
2164063678 tools/mkeficapsule: improve online help
Show short arguments along with long arguments in online help:

    $ tools/mkeficapsule -h
    Usage: mkeficapsule [options] <output file>
    Options:
        -f, --fit <fit image>       new FIT image file
        -r, --raw <raw image>       new raw image file
        -i, --index <index>         update image index
        -I, --instance <instance>   update hardware instance
        -K, --public-key <key file> public key esl file
        -D, --dtb <dtb file>        dtb file
        -O, --overlay               the dtb file is an overlay
        -h, --help                  print a help message

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2021-04-10 12:00:24 +02:00
Heinrich Schuchardt
ba7e5dbf38 imx6: icorem6: chmod 644 enigcam.bmp
Bitmap files should not be executable.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Michael Trimarchi <michael@amarulasolutions.com>
2021-04-08 23:59:50 +02:00
Peng Fan
16841a6a50 tools: imx image: fix write warning
Fix the warning by set the variable zero to uint64_t
"warning: ‘write’ reading 5 bytes from a region of size 4"

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2021-04-08 09:18:29 +02:00
Tom Rini
a64322548c tools: Integrate the Dockerfile used for CI
Integrate the Dockerfile from
https://source.denx.de/u-boot/gitlab-ci-runner.git as of
commit bc6130d572f1 ("Dockerfile: Remove high UID/GID") and introduce a
short rST on how to build the container.

Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Tom Rini <trini@konsulko.com>
2021-04-07 14:47:05 -04:00
Simon Glass
7570d9bb47 moveconfig: Handle binary files cleanly
Some files are not actually source code and thus can produce unicode
errors. Report this and continue.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:19 +12:00
Simon Glass
da39341223 dtoc: Improve handling of reg properties
This existing code assumes that a reg property is larger than one cell,
but this is not always the case. Fix this assumption.

Also if a node's parent is missing the #address-cells and #size-cells
properties we use 2 as a default for each. But this should not happen in
practice. More likely the properties were removed for SPL due to there
being no 'u-boot,dm-pre-reloc' property, or similar. Add a warning for
this as the failure can be very confusing.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:19 +12:00
Simon Glass
3e200caff0 dtoc: Adjust detection of 64-bit properties
At present an empty size is considered to be a 64-bit value. This does not
seem useful and wastes space. Limit the 64-bit detection to where one or
both of the addr/size is two cells or more.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:19 +12:00
Simon Glass
0c59acef34 dtoc: Show driver warnings once at the end
At present warnings are shown as soon as they are discovered in the
source scannner. But the function that detects them may be called multiple
times.

Collect all the warnings and show them at the end.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:19 +12:00
Simon Glass
5f86454b3d buildman: Add an encoding to the out-env file
The environment may contain some unicode characters. At least that is what
seemed to happen on one commit:

Building current source for 1 boards (0 threads, 64 jobs per thread)
    0    0    0 /1       -1      (starting)
Traceback (most recent call last):
  File ".../tools/buildman/buildman", line 64, in <module>
    ret_code = control.DoBuildman(options, args)
  File "tools/buildman/control.py", line 372, in DoBuildman
    options.keep_outputs, options.verbose)
  File ".../tools/buildman/builder.py", line 1704, in BuildBoards
    results = self._single_builder.RunJob(job)
  File ".../tools/buildman/builderthread.py", line 526, in RunJob
    self._WriteResult(result, job.keep_outputs, job.work_in_output)
  File ".../tools//buildman/builderthread.py", line 349, in _WriteResult
    print('%s="%s"' % (var, env[var]), file=fd)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
	311-312: ordinal not in range(128)

The problem defies repetition with any change at all to buildman. But
let's set an encoding in any case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:19 +12:00
Simon Glass
42bc156f80 patman: Continue on if warnings are found outside a commit
While we cannot know which commit the warning relates to, this should not
be fatal. Print the warning and carry on.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:10 +12:00
Tom Rini
1057b1be75 Prepare v2021.04-rc5
-----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmBiRbMACgkQFHw5/5Y0
 tyz0KAv/T8glQGL6L2CYeLuwbs9tndQoaH2YNb3teNUJrqehBleJAQ6ubHP30d87
 rHrokJ4Zu3TC15sACrK4GOMvsAVkeCuU/jIqFJA4ieDZr8g7yfUMOZMsDiIFdd36
 Rdmi71Jtek/csyS5BfcUUYoXVbUCGQ8yAx7z0VrK0sqsqZYYJNwPmqgkR8U2ulNb
 FiLxwamFwH7+bsvcvHajO1dOyko0zR7GTisf/fOgmlHuF3BldqG4SDcNd528uEWb
 r3fXs2Ut6IaC/beCnmKUPVG3ZjdPtb+T230rRRhUOfjTwqR7SzXp5bTyzdgyw3gU
 rOgfi/mAGQPPG2E2fzPz0JCI1irbnkSE2fVJ4epaVUCoHIEsQQdy034aD3qLcHFY
 65ihEpRvCM7s9jfKX8XeIG4rKFB4i60TX5orzdMvx7wV3rxmPE9qQmF9lkD+fEA+
 TPuFtkjGmluSe+xWBVK7+6xuF6rcLEbNXuFWRi3qMSMgH8rtGfeJRonusOGPolwC
 jQ1T7szW
 =UJ/O
 -----END PGP SIGNATURE-----

Merge tag 'v2021.04-rc5' into next

Prepare v2021.04-rc5
2021-03-29 18:00:21 -04:00
Jessica Clarke
3b142045e8 Support building on macOS/arm64
On Arm-based Macs, -no_pie is ignored and gives a linker warning.
Moreover, the build falls over with:

  ld: Absolute addressing not allowed in arm64 code but used in '_image_type_ptr_aisimage' referencing '_image_type_aisimage'

for dumpimage and mkimage, since we put data structs in text sections
not data sections and so cannot have dynamic relocations. Instead, move
the sections to __DATA and drop disabling PIE.

Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2021-03-28 17:30:35 -04:00
Simon Glass
5d1bec30ef dtoc: Add new check that offsets are correct
Add a few more internal checks to make sure offsets are correct, before
updating the dtb.

To make this easier, update the functions which add a property to return
that property,.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
f6176651bc dtoc: Support adding subnodes alongside existing ones
So far we have only needed to add subnodes to empty notds, so have not
had to deal with ordering. However this feature is needed for binman's
expanded nodes, since there may be another node in the same section.

While libfdt adds new properties after existing properties, it adds new
subnodes before existing subnodes. This means that we must reorder the
nodes in the cached version, so that the ordering remains consistent.

Update the sync implementation to sync existing subnodes first, then
add new ones, then tidy up the ordering in the cached version. Update the
test to cover this behaviour.

Also improve the comment about property syncing while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
76677dd2b2 dtoc: Add a subnode test for multiple nodes
Add a new test that adds a subnode alongside an existing one, as well as
adding properties to a subnode. This will expand to adding multiple
subnodes in future patches. Put a node after the one we are adding to so
we can check that things sync correctly.

The testAddNode() test should be in the TestNode class since it is a node
test, so move it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
71719e15e8 dtoc: Tweak ordering of fdt-offsets refreshing
Once the tree has been synced, thus potentially moving things around in the
fdt, we set _cached_offsets to False so that a refresh will happen next
time a property is accessed.

This 'lazy' refresh doesn't really save much time, since refresh is a very
fast operation, just a single walk of the tree. Also, having the refresh
happen in the bowels of property access it makes it harder to figure out
what is going on.

Simplify the code by always doing a refresh before and after a sync. Set
_cached_offsets to True immediately after this, in the Refresh() function,
since this makes more sense than doing it in the caller.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
37ba9844c2 dtoc: Tidy up property-offset handling
If a property does not yet have an offset, then that means it exists in
the cache'd fdt but has not yet been synced back to the flat tree. Use
the dirty flag for this so we don't need to check the offset too. Improve
the comments for Prop and Node to make it clear what an offset of None
means.

Also clear the dirty flag after the property is synced.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
acd98611c3 dtoc: Improve internal error for Refresh()
Add the node name too so it is easy to see which node failed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
5ff9fedc9b binman: Support default alignment for sections
Sometimes it is useful to specify the default alignment for all entries
in a section, such as when word-alignment is necessary, for example. It
is tedious and error-prone to specify this individually for each section.

Add a property to control this for a section.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
631f752de5 binman: Support obtaining section contents immediately
Generally the content of sections is not built until the final assembly
of the image. This is partly to avoid wasting time, since the entries
within sections may change multiple times as binman works through its
various stages. This works quite well since sections exist in a strict
hierarchy, so they can be processed in a depth-first manner.

However the 'collection' entry type does not have this luxury. If it
contains a section within its 'content' list, then it must produce the
section contents, if available. That section is typically a sibling
node, i.e. not part oc the collection's hierarchy.

Add a new 'required' argument to section.GetData() to support this. When
required is True, any referenced sections are immediately built. If this
is not possible (because one of the subentries does not have its data yet)
then an error is produced.

The test for this uses a 'collection' entry type, referencing a section as
its first member. This forces a call to _BuildSectionData() with required
set to False, at first, then True later, when the image is assembled.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
189f291914 binman: Add support for a collection of entries
The vblock entry type includes code to collect the data from a number of
other entries (not necessarily subentries) and concatenating it. This is
a useful feature for other entry types.

Make it a base class, so that vblock can use it, along with other entry
types.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
3d43338293 binman: Allow disabling expanding an entry
At present there is a command-line flag to disable substitution of expanded
entries. Add an option to the entry node as well, so it can be controlled
at the node level.

Add a test to cover this. Fix up the comment to the checkSymbols() function
it uses, while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
e9e0db8894 binman: Use a unique number for the symbols test file
Two test devicetree files currently have 192 as their unique number. Fix
this by separating them out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-27 16:26:48 +13:00
Simon Glass
fcae6682a7 binman: Update various pieces of the documentation
A few sections are a little out of date now. Update them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:10 +13:00
Simon Glass
96d340e9f6 binman: Drop repetitive heading for each entry
Many entries start 'Entry containing a'. This looks fine in the source
code but is annoying when viewed in the htmldocs table of contents. Drop
these unnecessary words.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:10 +13:00
Simon Glass
6bc4309be7 binman: Incorporate entry documentation
Update this to avoid sphinx warnings and incorporate it into the new
documentaiton tree.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:10 +13:00
Simon Glass
072026e7bb binman: Rearrange documentation into headings
Collect the material into different top-level headings to make it easier
to read.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:10 +13:00
Simon Glass
61adb2d247 binman: doc: Add documentation to htmldocs
Add a link to binman's documentation and adjust the files so that it is
accessible. Use the name README.rst so it is easy to discover when binman
is installed without U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:10 +13:00
Simon Glass
0668492728 binman: Automatically expand phase binaries into sections
When creating an entry, check for an expanded version of that entry, then
use it instead. This allows, for example use of:

   u-boot {
   };

instead of having to write out in full:

   u-boot {
      type = "section";

      u-boot-nodtb {
      };

      u-boot-dtb {
      };
   };

Add an implementaion of this and associated documentation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Simon Glass
0b6023ee41 binman: Plumb expanded entries through fully
Add support for this feature in the control, image and section modules, so
that expanded entries will be selected by default. So far there are no
expanded entry types, so this is a nop.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Simon Glass
63aeaeb463 binman: Allow a way to select expanded entries
Add a new command-line option to disable expanded entries. This is needed
for most tests, since it is much easier to 'factor out' this function into
a separate test and keep the existing packing tests simple.

Add the option and select it by default from tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Simon Glass
b35fb17936 binman: Allow using an an 'expanded' entry type
As the first step in supporting expanded entries, add a way for binman to
automatically select an 'expanded' version of an entry type, if requested.
This is controlled by a class method.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Simon Glass
5187b80617 binman: Use standard filenames for SPL/TPL devicetree
At present, before any entry expansion is done (such as a 'files' entry
expanding out to individual entries for each file it contains), we check
the binman definition (i.e. '/binman' node) to find out what devicetree
files are used in the images.

This is a pain, since the definition may change during expansion. For
example if there is no u-boot-spl-dtb entry in the definition at the start,
we assume that the SPL devicetree is not used. But if an entry later
expands to include this, then we don't notice.

In fact the flexibility provided by the current approach of checking the
definition is not really useful. We know that we can have SPL and TPL
devicetrees. We know the pathname to each, so we can simply check if the
files are present. If they are present, we can prepare them and update
them regardless of whether they are actually used. If they are not present,
we cannot prepare/update them anyway, i.e. an error will be generated.

Simplify state.Prepare() so it uses a hard-coded list of devicetree files.

Note that state.PrepareFromLoadedData() is left untouched, since in that
case we have a complete definition from the loaded file, but cannot of
course rely on the devicetree files that created it still being present.
So in that case we still check the image defitions.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Simon Glass
adb67bb94e binman: Move the comment for GetFdts() to the base class
Like with other methods this comment should be in the base class. Move it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00