Commit Graph

458 Commits

Author SHA1 Message Date
Simon Glass 2424057b2a binman: Avoid calculated section data repeatedly
Refactor the implementation slightly so that section data is not
rebuilt when it is already available.

We still have GetData() set up to rebuild the section, since we don't
currently track when things change that might affect a section. For
example, if a blob is updated within a section, we must rebuild it.
Tracking that would be possible but is more complex, so it left for
another time.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 8f5ef89f00 binman: Support compression of sections
With the previous changes, it is now possible to compress entire
sections. Add some tests to check that compression works correctly,
including updating the metadata.

Also update the documentation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 0ff83da634 binman: Use the actual contents in CheckSize()
At present this function adds up the total size of entries to work out the
size of a section's contents. With compression this is no-longer enough.

We may as well bite the bullet and build the section contents instead.
Call _BuildSectionData() to get the (possibly compressed) contents and
GetPaddedData() to get the same but with padding added.

Note that this is inefficient since the section contents is calculated
twice. Future work will improve this.

This affects testPackOverlapMap() since the error is reported with a
different section size now (enough to hold the contents). Update that at
the same time.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass b004bf3906 binman: Update CheckEntries() for compressed sections
At present this function assumes that the size of a section is at least as
large as its contents. With compression this is often not the case. Relax
this constraint by using the uncompressed size, if available.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 0b65769c2e binman: Drop CheckEntries()
This method introduces a separation between packing and checking that is
different for sections. In order to handle compression properly, we need
to be able to deal with a section's size being smaller than the
uncompressed size of its contents. It is easier to make this work if
everything happens in the Pack() method.

The only real user of CheckEntries() is entry_Section and it can call it
directly. Drop the call from 'control' and handle it locally.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 93f3c2ea14 binman: Call CheckSize() from the section's Pack() method
At present CheckSize() is called from the function that packs the entries.
Move it up to the main Pack() function so that _PackEntries() can just
do the packing.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 601b69aa8a binman: Drop the Entry.CheckSize() method
This is only used by entry_Section and that class already calls it. Avoid
calling it twice. Also drop it from the documentation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass c1af7a86b3 binman: Move sort and expand to the main Pack() function
At present sorting and expanding entries are side-effects of the
CheckEntries() function. This is a bit confusing, as 'checking' would
not normally involve making changes.

Move these steps into the Pack() function instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 6ddd61131f binman: Drop Entry.CheckOffset()
This function just calls CheckEntries() in the only non-trivial
implementation. Drop it and use CheckEntries() directly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass a9fad07d4b binman: Avoid reporting image-pos with compression
When a section is compressed, all entries within it are grouped together
into a compressed block of data. This obscures the start of each
individual child entry.

Avoid reporting bogus 'image-pos' properties in this case, since it is
not possible to access the entry at the location provided. The entire
section must be decompressed first.

CBFS does not support compressing whole sections, only individual files,
so needs no special handling here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 63e7ba6c18 binman: Set section contents in GetData()
Section contents is not set up when ObtainContents() is called, since
packing often changes the layout of the contents. Ensure that the contents
are correctly recorded by making this function regenerate the section. It
is normally only called by the parent section (when packing) or by the
top-level image code, when writing out the image. So the performance
impact is fairly small.

Now that sections have their contents in their 'data' property, update
testSkipAtStartSectionPad() to check it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 97c3e9a6fa binman: Store the original data before compression
When compressing an entry, the original uncompressed data is overwritten.
Store it so it is available if needed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 7d398bb1c7 binman: Make section padding consistent with other entries
At present padding of sections is inconsistent with other entry types, in
that different pad bytes are used.

When a normal entry is padded by its parent, the parent's pad byte is
used. But for sections, the section's pad byte is used.

Adjust logic to always do this the same way.

Note there is still a special case in entry_Section.GetPaddedData() where
an image is padded with the pad byte of the top-level section. This is
necessary since otherwise there would be no way to set the pad byte of
the image, without adding a top-level section to every image.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass d1d3ad7d1f binman: Move section padding to the parent
Each section is padded up to its size, if the contents are not large
enough. Move this logic from _BuildSectionData() to
GetPaddedDataForEntry() so that all the padding is in one place.

With this, the testDual test is working again, so enable it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 4a655c9bd7 binman: Refactor _BuildSectionData()
At present this function does the padding needed around an entry. It is
easier to understand what is going on if we have a function that returns
the contents of an entry, with padding included.

Refactor the code accordingly, adding a new GetPaddedData() method.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 17ea9f35e7 binman: Move section-building code into a function
Create a new _BuildSectionData() to hold the code that is now in
GetData(), so that it is clearly separated from entry.GetData() base
function.

Separate out the 'pad-before' processing to make this easier to
understand.

Unfortunately this breaks the testDual test. Rather than squash several
patches into an un-reviewable glob, disable the test for now.

This also affects testSkipAtStartSectionPad(), although it still not
quite what it should be. Update that temporarily for now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 4eec34c91f binman: Expand docs and test for alignment
Alignment does form part of the entry once the image is written out, but
within binman the entry contents does not include the padding. Add
documentation to make this clear, as well as a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass f90d906a27 binman: Expand docs and test for padding
Padding becomes part of the entry once the image is written out, but
within binman the entry contents does not include the padding. Add
documentation to make this clear, as well as a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass ef439ed191 binman: Update testPackExtra with more checks
Check the contents of each section to make sure it is actually in the
right place.

Also fix a whitespace error in the .dts file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 9248c8d9c9 binman: Use 'files-compress' to set compression for files
At present we use 'compress' as the property to set the compression of
a 'files' entry. But this conflicts with the same property for entries,
of which Entry_section is a subclass.

Strictly speaking, since Entry_files is in fact a subclass of
Entry_section, the files can be compressed individually but also the
section (that contains all the files) can itself be compressed. With this
change, it is possible to express that.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 87c962943a binman: Move CompressData() into Entry base class
At present this is only used by blobs. To allow it to be used by other
entry types (such as sections), move it into the base class.

Also read the compression type in the base class.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass e6bed4f181 binman: Expand the error message for breaching a section
Add in a few more details to this error message to make it easier to see
what is going on.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 72628cdf58 binman: Remove references to 'image' in entry_Section
While a section is the base class of Image, it is more correct to refer
to sections in most places in this file. Fix these comments.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass a81294671c binman: Fix return from u-boot-ucode if there is no DT
This should return empty contents, not leave it unset. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass 680e3c6edb binman: Give a sensible error if no command is given
At present if 'binman' is typed on the command line, a strange error about
a missing argument is displayed. Fix this.

These does not seem to be standard way to add the 'required' argument in
all recent Python versions, so set it manually.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass f2c0dd85ad binman: Add tests for skip-at-start sections
At present this feature is tested view the end-at-4gb feature. Add some
tests of its own, including the operation of padding.

The third test here shows binman's current, inconsistent approach to
padding in the top-level section. Future patches in this series will
address this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass a4dfe3e473 binman: Drop unused return variable for _DoTestFile()
This function returns the exit code from binman, not any data. Fix up a
few callers in the tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Simon Glass f3243303a0 binman: Update the entry docs
This has got out of sync with the entries. Regenerate it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-10-29 14:42:59 -06:00
Samuel Holland e72a6be4fc sunxi: binman: Add support for including SCP firmware
Allwinner sun50i SoCs contain an OpenRISC 1000 CPU that functions as a
System Control Processor, or SCP. ARM Trusted Firmware (ATF)
communicates with the SCP over SCPI to implement the PSCI system
suspend, shutdown and reset functionality. Currently, SCP firmware is
optional; the system will boot and run without it, but system suspend
will be unavailable.

Since all communication with the SCP is mediated by ATF, the only thing
U-Boot needs to do is load the firmware into SRAM. The SCP firmware
occupies the last 16KiB of SRAM A2, immediately following ATF.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Jagan Teki <jagan@amarulasolutions.com>
2020-10-22 11:43:43 +05:30
Samuel Holland 18bd45592c binman: Add support for SCP firmware
Add an entry type for a firmware blob for a System Control Processor,
given by an entry arg. This firmware is a raw binary blob.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Jagan Teki <jagan@amarulasolutions.com>
2020-10-22 11:43:43 +05:30
Samuel Holland 68158d59d2 sunxi: binman: Update FIT component descriptions
Since commit d879616e9e ("spl: fit: simplify logic for FDT loading for
non-OS boots"), the SPL looks at the "os" properties of FIT images to
determine where to append the FDT.

The "os" property of the "firmware" image also determines how to execute
the next stage of the boot process, as in 1d3790905d ("spl: atf:
introduce spl_invoke_atf and make bl31_entry private"). For this reason,
the next stage must be specified in "firmware", not in "loadables".

To support this additional functionality, and to properly model the boot
process, where ATF runs before U-Boot, add the "os" properties and swap
the firmware/loadable images in the FIT image.

Since this description was copied as an example in commit 70248d6a2916
("binman: Support generating FITs with multiple dtbs"), update those
examples as well for correctness and consistency.

Acked-by: Patrick Wildt <patrick@blueri.se>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Jagan Teki <jagan@amarulasolutions.com>
2020-10-22 11:43:43 +05:30
Samuel Holland b26536ad3d binman: Only write FDT once per node
Due to an extra level of indentation, the "data" property containing the
FDT was being written repeatedly after every other property in the node.
This caused the generated FIT image to be invalid.

Move the block up one level, so the property is added exactly once.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Jagan Teki <jagan@amarulasolutions.com>
2020-10-22 11:43:43 +05:30
Simon Glass fb91d5675e binman: Support adding a U-Boot environment
In some cases it is useful to include a U-Boot environment region in an
image. This allows the board to start up with an environment ready to go.

Add a new entry type for this. The input is a text file containing the
environment entries, one per line, in the format:

   var=value

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
2020-09-25 11:27:28 +08:00
Simon Glass 204aa78e04 binman: Show an error when a file is missing
The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.

Fix this and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
2020-09-25 11:27:28 +08:00
Simon Glass 68de0679c9 binman: sunxi: Add help message for missing sunxi ATF BL31
Add a special help message pointing to the relevant README.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Simon Glass b238143db9 binman: Support help messages for missing blobs
When an external blob is missing it can be quite confusing for the user.
Add a way to provide a help message that is shown.

Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2020-09-22 12:54:13 -06:00
Simon Glass c0f1ebe9c1 binman: Allow selecting default FIT configuration
Add a new entry argument to the fit entry which allows selection of the
default configuration to use. This is the 'default' property in the
'configurations' node.

Update the Makefile to pass in the value of DEVICE_TREE or
CONFIG_DEFAULT_DEVICE_TREE to provide this information.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Michal Simek <michal.simek@xilinx.com>
2020-09-22 12:54:13 -06:00
Alper Nebi Yasak 4ec40a7208 binman: Document how CROSS_COMPILE, CC, HOSTCC etc. are used in README
Explain that binman interprets these environment variables in the
"External tools" section to run target/host specific versions of the
tools, and add a new section on how to use CROSS_COMPILE to run the
tests on non-x86 machines.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Alper Nebi Yasak 1e4687aa47 binman: Use target-specific tools when cross-compiling
Currently, binman always runs the compile tools like cc, objcopy, strip,
etc. using their literal name. Instead, this patch makes it use the
target-specific versions by default, derived from the tool-specific
environment variables (CC, OBJCOPY, STRIP, etc.) or from the
CROSS_COMPILE environment variable.

For example, the u-boot-elf etype directly uses 'strip'. Trying to run
the tests with 'CROSS_COMPILE=i686-linux-gnu- binman test' on an arm64
host results in the '097_elf_strip.dts' test to fail as the arm64
version of 'strip' can't understand the format of the x86 ELF file.

This also adjusts some command.Output() calls that caused test errors or
failures to use the target versions of the tools they call. After this,
patch, an arm64 host can run all tests with no errors or failures using
a correct CROSS_COMPILE value.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Alper Nebi Yasak 5ac7687827 binman: Support cross-compiling test files to x86
These test files are currently "intended for use on x86 hosts", but most
of the tests using them can still pass when cross-compiled to x86 on an
arm64 host.

This patch enables non-x86 hosts to run the tests by specifying a
cross-compiler via CROSS_COMPILE. The list of variables it sets is taken
from the top-level Makefile. It would be possible to automatically set
an x86 cross-compiler with a few blocks like:

    ifneq ($(shell i386-linux-gnu-gcc --version 2> /dev/null),)
    CROSS_COMPILE = i386-linux-gnu-
    endif

But it wouldn't propagate to the binman process calling this Makefile,
so it's better just raise an error and expect 'binman test' to be run
with a correct CROSS_COMPILE.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Simon Glass 6cf9953bfb binman: Support generating FITs with multiple dtbs
In some cases it is useful to generate a FIT which has a number of DTB
images, selectable by configuration. Add support for this in binman,
using a simple iterator and string substitution.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass dc2f81a2c8 binman: Add support for ATF BL31
Add an entry for ARM Trusted Firmware's 'BL31' payload, which is the
device's main firmware. Typically this is U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass 8795898a53 binman: Move 'external' support into base class
At present we have an Entry_blob_ext which implement a blob which holds an
external binary. We need to support other entry types that hold external
binaries, e.g. Entry_blob_named_by_arg. Move the support into the base
Entry class to allow this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass e9d336d866 binman: Fix up a few missing comments
Tidy up a few test functions which lack argument comments. Rename one that
has the same name as a different test.

Also fix up the comment for PrepareImagesAndDtbs().

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass 3decfa3a87 binman: Allow entry args to be required
If an entry argument is needed by an entry but the entry argument is not
present, then a strange error can occur when trying to read the file.

Fix this by allowing arguments to be required. Select this option for the
cros-ec-rw entry. If a filename is provided in the node, allow that to be
used.

Also tidy up a few related tests to make the error string easier to find,
and fully ignore unused return values.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass 2b522f1e79 tools: Drop unnecessary use of __file__
There are few places where the path of the current modules is calculated
but not used. Drop them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass 9fbfaba0a7 binman: Use pkg_resources to find resources
At present we look for resources based on the path of the Python module
that wants them. Instead we should use Python's pkg_resources feature
which is designed for this purpose.

Update binman to use this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Alper Nebi Yasak fe05701b05 binman: Build FIT image subentries with the section etype
When reading subentries of each image, the FIT entry type directly
concatenates their contents without padding them according to their
offset, size, align, align-size, align-end, pad-before, pad-after
properties.

This patch makes sure these properties are respected by offloading this
image-data building to the section etype, where each subnode of the
"images" node is processed as a section. Alignments and offsets are
respective to the beginning of each image. For example, the following
fragment can end up having "u-boot-spl" start at 0x88 within the final
FIT binary, while "u-boot" would then end up starting at e.g. 0x20088.

	fit {
		description = "example";

		images {
			kernel-1 {
				description = "U-Boot with SPL";
				type = "kernel";
				arch = "arm64";
				os = "linux";
				compression = "none";

				u-boot-spl {
				};
				u-boot {
					align = <0x10000>;
				};
			};
		};
	}

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reinstate check in testPadInSections(), squash in
   "binman: Allow FIT binaries to have missing external blobs"
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2020-09-22 12:50:43 -06:00
Alper Nebi Yasak 3fdeb14d95 binman: Respect pad-before property of section subentries
Other relevant properties (pad-after, offset, size, align, align-size,
align-end) already work since Pack() sets correct ranges for subentries'
data (.offset, .size variables), but some padding here is necessary to
align the data within this range to match the pad-before property.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Alper Nebi Yasak 8001d0b162 binman: Ignore hash*, signature* nodes in sections
Switch to str.startswith for matching like the FIT etype does since the
current version doesn't ignore 'hash-1', 'hash-2', etc.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass 02c102074d binman: Add a setup script for Python
Allow binman to be installed by adding a suitable setup.py script.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-08-22 08:53:38 -06:00
Simon Glass 07237988dc binman: Correct some import statements
Some of these were not converted when binman moved to use absolute paths.
Fix them.

Also drop the import of 'test' which is a directory, not a module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-08-22 08:53:38 -06:00
Simon Glass 87d43329ef binman: Move GetEntryModules() to control
When binman is installed its main program is in a different directory
to its modules. This means that __file__ is different and we cannot use
it to obtain the path to etype/ from main.py

To fix this, move the function to the 'control' module, since it is
installed with all the other modules, including the etype/ directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-08-22 08:53:38 -06:00
Simon Glass e6385c7e9c Makefile: Rename ALL-y to INPUTS-y
When binman is in use, most of the targets built by the Makefile are
inputs to binman. We then need a final rule to run binman to produce the
final outputs.

Rename the variable to indicate this, and add a new 'inputs' target.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-28 19:30:39 -06:00
Simon Glass 52b10dd7de binman: Don't change the descriptor in tests
At present testPackX86RomMeNoDesc removes the contents of the
descriptor.bin file and testPackX86RomMeMissingDesc removes the file
completely.

If a test that relies on this file happens to run after it is removed, it
will not work. Since we have no control over the selecting of tests that
run in parallel and series, we must avoid changing the files.

Update this tests to use separate files instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-26 19:59:57 -06:00
Simon Glass fdc34368dd binman: Add support for generating a FIT
FIT (Flat Image Tree) is the main image format used by U-Boot. In some
cases scripts are used to create FITs within the U-Boot build system. This
is not ideal for various reasons:

- Each architecture has its own slightly different script
- There are no tests
- Some are written in shell, some in Python

To help address this, add support for FIT generation to binman. This works
by putting the FIT source directly in the binman definition, with the
ability to adjust parameters, etc. The contents of each FIT image come
from sub-entries of the image, as is normal with binman.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Simon Glass b3295fd4e3 binman: Allow zero-length entries to overlap
Some binary blobs unfortunately obtain their position in the image from
other binary blobs, such as Intel's 'descriptor'. In this case we cannot
rely on packing to work. It is not possible to produce a valid image in
any case, due to the missing blobs.

Allow zero-length overlaps so that this does not cause any problems.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass 0ba4b3dfee binman: Allow missing Intel blobs
Update the Intel blob entries to support missing binaries.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass b1cca9552c binman: Detect when valid images are not produced
When external blobs are missing, show a message indicating that the images
are not functional.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Simon Glass 38fdb4cb35 patman: Update errors and warnings to use stderr
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass 4f9f1056ec binman: Allow external binaries to be missing
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.

U-Boot does with with ATF (ARM Trusted Firmware) today.

Add a new flag to binman to request this behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass 04e6a6b9ec binman: Convert existing binary blobs to blob_ext
Many of the existing blobs rely on external binaries which may not be
available. Move them over to use blob_ext to indicate this.

Unfortunately cros-ec-rw cannot use this class because it inherits
another. So set the 'external' value for that class.

While we are here, drop the import of Entry since it is not used (and
pylint3 complains).

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Simon Glass ce867ad7c8 binman: Add an etype for external binary blobs
It is useful to be able to distinguish between ordinary blobs such as
u-boot.bin and external blobs that cannot be build by the U-Boot build
system. If the external blobs are not available for some reason, then we
know that a value image cannot be built.

Introduce a new 'blob-ext' entry type for that.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass 34861d506c binman: Use super() instead of specifying parent type
It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass c6162cf87c binman: Fix a few typos in the entry docs
Some typos have been fixed in the generated entry docs but the code was
not updated. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass 0dc706fe54 binman: Add support for calling mkimage
As a first step to integrating mkimage into binman, add a new entry type
that feeds data into mkimage for processing and incorporates that output
into the image.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass b5287c4126 binman: Set a default toolpath
When binman is run from 'make check' it is given a toolpath so that the
latest tools (e.g. mkimage) are used. When run manually with no toolpath,
it relies on the system mkimage. But this may be missing or old.

Make some effort to find the built-from-soruce version by looking in the
current directory and in the builds created by 'make check'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass 32eb66d2d4 binman: Specify the toolpath when running test coverage
At present binman's test coverage runs without a toolpath set. This means
that the system tools will be used. That may not be correct if they are
out of date or missing and this can result in a reduction in test coverage
below 100%.

Provide the toolpath to binman in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass fdb3040e96 binman: Correct the search patch for pylibfdt
Now that binman uses tools/ as its base directory for importing modules,
the path to the pylibfdt build by U-Boot is incorrect. Fix it with a new
path.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-24 19:25:15 -06:00
Simon Glass 5de9b9c03c binman: cbfs: Fix IFWI typo
This comment references the wrong thing. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-24 19:25:15 -06:00
Simon Glass 6c8e0bfe6e binman: Output errors to stderr
At present binman outputs errors to stdout which means that fails are
effectively silent when printed by buildman, for example. Fix this by
outputing errors to stderr.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-24 19:25:15 -06:00
Tom Rini 7208396bbf Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"
This reverts commit 5d3a21df66, reversing
changes made to 56d37f1c56.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>
2020-07-24 08:42:06 -04:00
Simon Glass 3b9a87321c binman: Add support for generating a FIT
FIT (Flat Image Tree) is the main image format used by U-Boot. In some
cases scripts are used to create FITs within the U-Boot build system. This
is not ideal for various reasons:

- Each architecture has its own slightly different script
- There are no tests
- Some are written in shell, some in Python

To help address this, add support for FIT generation to binman. This works
by putting the FIT source directly in the binman definition, with the
ability to adjust parameters, etc. The contents of each FIT image come
from sub-entries of the image, as is normal with binman.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass 8200d8871a binman: Allow zero-length entries to overlap
Some binary blobs unfortunately obtain their position in the image from
other binary blobs, such as Intel's 'descriptor'. In this case we cannot
rely on packing to work. It is not possible to produce a valid image in
any case, due to the missing blobs.

Allow zero-length overlaps so that this does not cause any problems.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass 2f5c3a4d1d binman: Allow missing Intel blobs
Update the Intel blob entries to support missing binaries.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass 13262c9362 binman: Detect when valid images are not produced
When external blobs are missing, show a message indicating that the images
are not functional.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass f9793a12c5 patman: Update errors and warnings to use stderr
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass 5f850fb9a6 binman: Allow external binaries to be missing
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.

U-Boot does with with ATF (ARM Trusted Firmware) today.

Add a new flag to binman to request this behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass 894f635755 binman: Convert existing binary blobs to blob_ext
Many of the existing blobs rely on external binaries which may not be
available. Move them over to use blob_ext to indicate this.

Unfortunately cros-ec-rw cannot use this class because it inherits
another. So set the 'external' value for that class.

While we are here, drop the import of Entry since it is not used (and
pylint3 complains).

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass d498630ea9 binman: Add an etype for external binary blobs
It is useful to be able to distinguish between ordinary blobs such as
u-boot.bin and external blobs that cannot be build by the U-Boot build
system. If the external blobs are not available for some reason, then we
know that a value image cannot be built.

Introduce a new 'blob-ext' entry type for that.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass 1216448573 binman: Use super() instead of specifying parent type
It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass fe04f647a2 binman: Fix a few typos in the entry docs
Some typos have been fixed in the generated entry docs but the code was
not updated. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass 3e8fba4cd4 binman: Add support for calling mkimage
As a first step to integrating mkimage into binman, add a new entry type
that feeds data into mkimage for processing and incorporates that output
into the image.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass ce774e94de binman: Set a default toolpath
When binman is run from 'make check' it is given a toolpath so that the
latest tools (e.g. mkimage) are used. When run manually with no toolpath,
it relies on the system mkimage. But this may be missing or old.

Make some effort to find the built-from-soruce version by looking in the
current directory and in the builds created by 'make check'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass 92dee5fcc5 binman: Specify the toolpath when running test coverage
At present binman's test coverage runs without a toolpath set. This means
that the system tools will be used. That may not be correct if they are
out of date or missing and this can result in a reduction in test coverage
below 100%.

Provide the toolpath to binman in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass 5d8b338462 binman: Correct the search patch for pylibfdt
Now that binman uses tools/ as its base directory for importing modules,
the path to the pylibfdt build by U-Boot is incorrect. Fix it with a new
path.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass ccbe7dbc3f binman: cbfs: Fix IFWI typo
This comment references the wrong thing. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass de43855091 binman: Output errors to stderr
At present binman outputs errors to stdout which means that fails are
effectively silent when printed by buildman, for example. Fix this by
outputing errors to stderr.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass b4fa94959d patman: Tidy up sys.path changes
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.

We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.

The special paths for finding pylibfdt remain.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass bf776679a7 patman: Move to absolute imports
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass 16287933a8 binman: Move to absolute imports
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass ce0dc2edfc patman: Move test running/reporting to test_util
This code is useful in other tools. Move it into a common file so it can
be shared.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass c07ab6effb binman: Rename the main module
Python does not like the module name being the same as the module
directory. To allow buildman modules to be used from other tools, rename
it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass 83a4518771 patman: Drop references to __future__
We don't need these now that the tools using Python 3. Drop them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Masahiro Yamada f6a8c0f468 binman: fix default filename of u-boot-with-ucode-ptr in documentation
The suffix should be ".bin" instead of ".dtb" .

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-01-07 16:02:39 -07:00
Thomas Hebb 32f2ca2a7e cosmetic: Fix spelling and whitespace errors
Signed-off-by: Thomas Hebb <tommyhebb@gmail.com>
2019-12-03 08:43:23 -05:00
Simon Glass 7c15013639 binman: tegra: Adjust symbol calculation depending on end-at-4gb
A recent change adjusted the symbol calculation to work on x86 but broke
it for Tegra. In fact this is because they have different needs.

On x86 devices the code is linked to a ROM address and the end-at-4gb
property is used for the image. In this case there is no need to add the
base address of the image, since the base address is already built into
the offset and image-pos properties.

On other devices we must add the base address since the offsets start at
zero.

In addition the base address is currently added to the 'offset' and 'size'
values. It should in fact only be added to 'image-pos', since 'offset' is
relative to its parent and 'size' is not actually an address. This code
should have been adjusted when support for 'image-pos' and 'size' was
added, but it was not.

To correct these problems:
- move the code that handles adding the base address to section.py, which
  can check the end-at-4gb property and which property
  (offset/size/image-pos) is being read
- add the base address only when needed (only for image-pos and not if the
  image uses end-at-4gb)
- add a note to the documentation
- add a separate test to cover x86 behaviour

Fixes: 15c981cc (binman: Correct symbol calculation with non-zero image base)

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
2019-11-11 14:20:35 -05:00
Simon Glass 388560134b binman: Move to use Python 3
Update this tool to use Python 3 to meet the 2020 deadline.

Unfortunately this introduces a test failure due to a problem in pylibfdt
on Python 3. I will investigate.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-11-04 18:15:32 -07:00
Simon Glass b6ee0cf89f binman: Convert a few tests to Python 3
Some tests have crept in with Python 2 strings and constructs. Convert
then.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-11-04 18:15:32 -07:00
Simon Glass 9a5d3dcff7 binman: Remember the pre-reset entry size
When preparing to possible expand or contract an entry we reset the size
to the original value from the binman device-tree definition, which is
often None.

This causes binman to forget the original size of the entry. Remember this
so that it can be used when needed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-11-04 18:15:32 -07:00