As a first step to integrating mkimage into binman, add a new entry type
that feeds data into mkimage for processing and incorporates that output
into the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
When binman is run from 'make check' it is given a toolpath so that the
latest tools (e.g. mkimage) are used. When run manually with no toolpath,
it relies on the system mkimage. But this may be missing or old.
Make some effort to find the built-from-soruce version by looking in the
current directory and in the builds created by 'make check'.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman's test coverage runs without a toolpath set. This means
that the system tools will be used. That may not be correct if they are
out of date or missing and this can result in a reduction in test coverage
below 100%.
Provide the toolpath to binman in this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
Now that binman uses tools/ as its base directory for importing modules,
the path to the pylibfdt build by U-Boot is incorrect. Fix it with a new
path.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present binman outputs errors to stdout which means that fails are
effectively silent when printed by buildman, for example. Fix this by
outputing errors to stderr.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.
We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.
The special paths for finding pylibfdt remain.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Python does not like the module name being the same as the module
directory. To allow buildman modules to be used from other tools, rename
it.
Signed-off-by: Simon Glass <sjg@chromium.org>
The suffix should be ".bin" instead of ".dtb" .
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
A recent change adjusted the symbol calculation to work on x86 but broke
it for Tegra. In fact this is because they have different needs.
On x86 devices the code is linked to a ROM address and the end-at-4gb
property is used for the image. In this case there is no need to add the
base address of the image, since the base address is already built into
the offset and image-pos properties.
On other devices we must add the base address since the offsets start at
zero.
In addition the base address is currently added to the 'offset' and 'size'
values. It should in fact only be added to 'image-pos', since 'offset' is
relative to its parent and 'size' is not actually an address. This code
should have been adjusted when support for 'image-pos' and 'size' was
added, but it was not.
To correct these problems:
- move the code that handles adding the base address to section.py, which
can check the end-at-4gb property and which property
(offset/size/image-pos) is being read
- add the base address only when needed (only for image-pos and not if the
image uses end-at-4gb)
- add a note to the documentation
- add a separate test to cover x86 behaviour
Fixes: 15c981cc (binman: Correct symbol calculation with non-zero image base)
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Update this tool to use Python 3 to meet the 2020 deadline.
Unfortunately this introduces a test failure due to a problem in pylibfdt
on Python 3. I will investigate.
Signed-off-by: Simon Glass <sjg@chromium.org>
When preparing to possible expand or contract an entry we reset the size
to the original value from the binman device-tree definition, which is
often None.
This causes binman to forget the original size of the entry. Remember this
so that it can be used when needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present all the 'command' methods return bytes. Most of the time we
actually want strings, so change this. We still need to keep the internal
representation as bytes since otherwise unicode strings might break over
a read() boundary (e.g. 4KB), causing errors. But we can convert the end
result to strings.
Add a 'binary' parameter to cover the few cases where bytes are needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
This entry is used to hold an Intel FSP-T (Firmware Support Package
Temp-RAM init) binary. Add support for this in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
This entry is used to hold an Intel FSP-S (Firmware Support Package
Silicon init) binary. Add support for this in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present binman adds the image base address to the symbol value before
it writes it to the binary. This is not correct since the symbol value
itself (e.g. image position) has no relationship to the image base.
Fix this and update the tests to cover this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present the symbol information is written to binaries just before
binman exits. This is fine for entries within sections since the section
contents is calculated when it is needed, so the updated symbol values are
included in the image that is written.
However some binaries are inside entries which have already generated
their contents and do not notice that the entries have changed (e.g. Intel
IFWI).
Move the symbol writing earlier to cope with this.
Signed-off-by: Simon Glass <sjg@chromium.org>
The Intel IFWI (Integrated Firmware Image) is effectively a section with
other entries inside it. Support writing symbol information into entries
within it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for the ProcessContents() method in this entry so that it is
possible to support entries which change after initial creation.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this class reads its entries in the constructor. This is not
how things should be done now. Update it.
Signed-off-by: Simon Glass <sjg@chromium.org>
The Intel FSP supports initialising memory early during boot using a binary
blob called 'fspm'. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to access the size of an image in SPL, with
something like:
binman_sym_declare(unsigned long, u_boot_any, size);
...
ulong u_boot_size = binman_sym(ulong, u_boot_any, size);
Add support for this and update the tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present these are large enough to hold 20 bytes of symbol data. Add
four more bytes so we can add another test.
Unfortunately at present this involves changing a few test files to make
room. We could adjust the test files to not specify sizes for entries.
Then we could make the tests check the actual sizes. But for now, leave it
as it is, since the effort is minor.
Signed-off-by: Simon Glass <sjg@chromium.org>
Entries which include a section and need to obtain its contents call
GetData(), as with any other entry. But the current implementation of this
method in entry_Section requires the size of the section to be known. If
it is unknown, an error is produced, since size is None:
TypeError: can't multiply sequence by non-int of type 'NoneType'
There is no need to know the size in advance since the code can be
adjusted to build up the section piece by piece, instead of patching each
entry into an existing bytearray.
Update the code to handle this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Two of the test files somehow were not converted to three digits. Fix
them, using the next available numbers.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we only support symbols inside binaries which are at the top
level of an image. This restrictions seems unreasonable since more complex
images may want to group binaries within different sections.
Relax the restriction, adding a new _SetupTplElf() helper function.
Also fix a typo in the comment for testTpl().
Signed-off-by: Simon Glass <sjg@chromium.org>
We use the Makefile for all ELF test files now, so drop all the code that
checks whether to get the test file from the Makefile or from the git
repo.
Also add a comment to the Makefile indicating that it is run from binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Remove this file from git and instead build it using the Makefile.
With this change a few things need to be adjusted:
1. The 'notes' section no-longer appears at the start of the ELF file
(before the code), so update testSymbols to adjust the offsets.
2. The dynamic linker is disabled to avoid errors like:
"Not enough room for program headers, try linking with -N"
3. The interpreter note is moved to the end of the image, so that the
binman symbols appear first.
Signed-off-by: Simon Glass <sjg@chromium.org>
Remove this file from git and instead build it using the Makefile.
Update tools.GetInputFilename() to support reading files from an absolute
path, so that we can read the Elf test files easily. Also make sure that
the temp directory is report in ELF tests as this was commented out.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the ELF test files are checked into the U-Boot tree. This is
covenient since the files never change and can be used on non-x86
platforms. However it is not good practice to check in binaries and in
this case it does not seem essential.
Update the binman test-file Makefile to support having source in a
different directory. Adjust binman to run it to build bss_data, as a
start. We can add other files as needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this command silently fails if something goes wrong. Use the
tools.Run() function instead, since it reports errors.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this entry does not work correctly when a FIT image is used as
the input. It updates the FIT instead of the output image. The test passed
because the FIT image happened to have the right data already.
Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
A Firmware Image Table (FIT) is a data structure defined by Intel which
contains information about various things needed by the SoC, such as
microcode.
Add support for this entry as well as the pointer to it. The contents of
FIT are fixed at present. Future work is needed to support adding
microcode, etc.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present these two sections of code are linked together into a single
2KB chunk in a single file. Some Intel SoCs like to have a FIT (Firmware
Interface Table) in the ROM and the pointer for this needs to go at
0xffffffc0 which is in the middle of these two sections.
Make use of the new 'reset' entry and change the existing 16-bit entry to
include just the 16-bit data.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman has a single entry type for the 16-bit code code needed
to start up an x86 processor. This entry is intended to include both the
reset vector itself as well as the code to move to 32-bit mode.
However this is not very flexible since in some cases other data needs to
be included at the top of the SPI flash, in between these two pieces. For
example Intel requires that a FIT (Firmware Image Table) pointer be placed
0x40 bytes before the end of the ROM.
To deal with this, add a new reset entry for just the reset vector. A
subsequent change will adjust the existing 'start16' entry.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the Intel IFWI entry uses 'replace' without the 'ifwi-' prefix.
This is a fairly generic name which might conflict with the main Entry
base class at some point, if more features are added. Add a prefix.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some versions of binutils generate hidden symbols which are currently not
parsed by binman. Correct this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Recent versions of binutils add a '.note.gnu.property' into the ELF file.
This is not required and interferes with the expected output. Drop it.
Also fix testMakeElf() to use a different file for input and output.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present if libfdt is not available binman can't do anything much.
Improve the situation a little.
Ideally there should be a test to cover this, but I'm not quite sure how
to fake this.
Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed up missing ReadChildData() enty test)
It is more common to use the name 'cls' for the class object of a class
method, to distinguish it from normal methods, which use 'self' Update the
binman tests accordingly.
Signed-off-by: Simon Glass <sjg@chromium.org>
The doc currently uses sandbox_defconfig as examples of enabling
debug/verbose output of binman. However during a sandbox build it
does not call binman at all. Change it to qemu-x86_defconfig.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The image-header currently sets it offset assuming that skip-at-start is
zero. This does not work on x86 where offsets end at 4GB. Add in this
value so that the offset is correct.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Bin Meng <bmeng.cn@gmail.com>
Some x86 sections have special offsets which currently result in empty
data being returned from the 'extract' command. Fix this by taking account
of the skip-at-start property.
Add a little more debugging while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Bin Meng <bmeng.cn@gmail.com>
At present this function is not present in the Entry base class so it is
hard to find the documentation for it. Move the docs from the section
class and expand it a little.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present the verbose flag only works for the 'build' command. This is
not intended, nor is it useful. Update the code to support the verbose
flag and make use of a command exception handler.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
[bmeng: rebase the patch against u-boot-x86/next to get it applied cleanly]
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Support a new BINMAN_VERBOSE option to the build, to allow passing the
-v flag to binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Add a 'replace' command to binman to permit entries to be replaced, either
individually or all at once (using a filter).
Signed-off-by: Simon Glass <sjg@chromium.org>
This code has three distinct phases:
1. The image is loaded and the state module is set up
2. The entry is written to the image
3. The image is repacked and written back to the file
Split the code out with three separate functions, one for each phase.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present some tests leave behind output directories. This happens
because some tests call binman, which sets up an output directory, then
call it again, which sets up another output directory and leaves the
original one behind.
Fix this by using a separate temporary directory when binman is called
twice, or by manually removing the output directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since the state module holds references to all the device trees used by
binman, it must be updated when the device trees are updated. Add support
for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman cannot replace data within a CBFS since it does not
allow rewriting of the files in that CBFS. Implement this by using the
new WriteData() method to handle the case.
Add a header to compressed data so that the amount of compressed data can
be determined without reference to the size of the containing entry. This
allows the entry to be larger that the contents, without causing errors in
decompression. This is necessary to cope with a compressed device tree
being updated in such a way that it shrinks after the entry size is
already set (an obscure case). It is not used with CBFS since it has its
own metadata for this. Increase the number of passes allowed to resolve
the position of entries, to handle this case.
Add a test for this new logic.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this method assumes that the parent section does not need
to recalculate its position or adjust any metadata it may contain. But
when the entry changes size this may not be true. Also if the parent
section is more than just a container (e.g. it is a CBFS) then the
section may need to regenerate its output.
Add a new WriteChildData() method to sections and call this from the
WriteData() method, to handle this situation.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we simply extract the data directly from entries using the
image_pos information. This happens to work on current entry types, but
cannot work if the entry type encodes the data in some way. Update the
ReadData() method to provide the data by calling a new ReadChildData()
method in the parent. This allows the entry_Section class, or possibly
any other container class, to return the correct data in all cases.
Signed-off-by: Simon Glass <sjg@chromium.org>
The Intel descriptor must always appear at the start of an (x86) image,
so it is supposed to position itself there always. However there is no
explicit test for this. Add one and fix a bug introduced by the recent
change to adjust Entry to read the node in a separate call.
Signed-off-by: Simon Glass <sjg@chromium.org>
The FMAP is not intended to show the files inside a CBFS. The FMAP can be
used to locate the CBFS itself, but then the CBFS must be read to find out
what is in it.
Update the FMAP to work this way and add some debugging while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes an entry may shrink after it has already been packed. In that
case we must repack the items. Of course it is always possible to just
leave the entry at its original size and waste space at the end. This is
what binman does by default, since there is the possibility of the entry
changing size every time binman calculates its contents, thus causing a
loop.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes entries shrink after packing. As a start towards supporting
this, update the _testing entry to handle the test case.
Signed-off-by: Simon Glass <sjg@chromium.org>
So far we don't allow entries to change size when repacking. But this is
not very useful since it is common for entries to change size after an
updated binary is built, etc.
Add support for this, respecting the original offset/size/alignment
constraints of the image layout. For this to work the original image
must have been created with the 'allow-repack' property.
This does not support entry types with sub-entries such as files and
CBFS, but it does support sections.
Signed-off-by: Simon Glass <sjg@chromium.org>
The positioning does not currently work correctly if at the end of an
image with no fixed size. Also if the header is in the middle of an image
it can cause a gap in the image since the header position is normally at
the image end, so entries after it are placed after the end of the image.
Fix these problems and add more tests to cover these cases.
Signed-off-by: Simon Glass <sjg@chromium.org>
Now that an Image is an Entry_section, there is no need for the separate
BuildSection() function. Drop it and add a bit of logging.
Signed-off-by: Simon Glass <sjg@chromium.org>
When reading an image in, write its fdtmap to a file in the output
directory. This is useful for debugging. Update the 'ls' command to set up
the output directory; otherwise it will fail.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present it is not possible to discover the contraints to repacking an
image (e.g. maximum section size) since this information is not preserved
from the original image description.
Add new 'orig-offset' and 'orig-size' properties to hold this. Add them to
the main device tree in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
While it is useful and efficient to build images in a single pass from a
unified description, it is sometimes desirable to update the image later.
Add support for replace an existing file with one of the same size. This
avoids needing to repack the file. Support for more advanced updates will
come in future patches.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present EnsureCompiled() uses an file from the 'output' directory (in
the tools module) when compiling the device tree. This is fine in most
cases, allowing useful inspection of the output files from binman.
However in functional tests, _SetupDtb() creates an output directory and
immediately removes it afterwards. This serves no benefit and just
confuses things, since the 'official' output directory is supposed to be
created and destroyed in control.Binman().
Add a new parameter for the optional temporary directory to use, and use a
separate temporary directory in _SetupDtb().
Signed-off-by: Simon Glass <sjg@chromium.org>
When updating an existing image where the size of all entries remains the
same, we should not need to regenerate the fdtmap. Update the entry to
return the same fdtmap as was read from the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
We use this same combination of properties several times in tests. Add a
constant for it to avoid typos, etc.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we have an 'image' property in the entry for this purpose, but
this is not necessary and seems error-prone in the presence of
inheritance. Add a function instead. The Entry_section class overrides
this with a special version, since top-level sections are in fact images,
since Image inherits Entry_section.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the Entry constructor sets up the object and then immediately
reads its device-tree node to obtain its properties.
This breaks a convention that constructors should not do any processing.
A consequence is that we must pass all arguments to the constructor and
cannot have the node-reading proceed in a different way unless we pass
flags to that constructor. We already have a 'test' flag in a few cases,
and now need to control whether the 'orig_offset' and 'orig_size'
properties are set or not.
Adjust the code to require a separate call to ReadNode() after
construction. The Image class remains as it was.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since binman supports multiple images it is useful to know which one
created the image that has been read. Then it is possible to look up that
name in the 'master' device tree (containing the description of all
images).
Add a property for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases we want to access the Entry object for a particular device
tree. This allows us to read its contents or update it. Add this
information to output_fdt_files and provide a function to read it.
Also rename output_fdt_files since its name is no-longer descriptive.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present these state functions raise an exception if they cannot find
what is requested. But in some cases the information is optional (e.g. an
fdtmap in a coming patch) so it is better to return gracefully.
Update these two functions to return None when the data cannot be found.
Signed-off-by: Simon Glass <sjg@chromium.org>
When modifying an image it is convenient to load the data from the file
into each entry so that it can be reprocessed. Add a new LoadData() method
to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
This data provides all the information about the position and size of each
entry. Store it for later use when loading an image. It can be reused as
is if the image is modified without changing offsets/sizes.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new logging feature to log information about progress with
packing. This is useful to see how binman is figuring things out.
Also update elf.py to use the same feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this excludes the device tree passed in to binman since it
is always returned first by GetAllFdts(). However, this is easy to ensure
by adding a check in that function. Change this dict to includes all
device trees, and rename it to fdt_set.
Signed-off-by: Simon Glass <sjg@chromium.org>
It makes more sense to use entry type as the key for this dictionary,
since the filename can in principle be anything. Make this change and also
rename fdt_files and add a comment to explain it better.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the FDTs are keyed by their default filename (not their actual
filename). It seems easier to key by the entry type, since this is always
the same for each FDT type.
To do this, add a new Entry method called GetFdtEtype(). This is necessary
since some entry types contain a device tree which are not the simple
three entry types 'u-boot-dtb', 'u-boot-spl' or 'u-boot-tpl'.
The code already returns a dict for GetFdt(). Update the value of that
dict to include the filename so that existing code can work.
Signed-off-by: Simon Glass <sjg@chromium.org>
