Sometimes an entry may shrink after it has already been packed. In that
case we must repack the items. Of course it is always possible to just
leave the entry at its original size and waste space at the end. This is
what binman does by default, since there is the possibility of the entry
changing size every time binman calculates its contents, thus causing a
loop.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes entries shrink after packing. As a start towards supporting
this, update the _testing entry to handle the test case.
Signed-off-by: Simon Glass <sjg@chromium.org>
So far we don't allow entries to change size when repacking. But this is
not very useful since it is common for entries to change size after an
updated binary is built, etc.
Add support for this, respecting the original offset/size/alignment
constraints of the image layout. For this to work the original image
must have been created with the 'allow-repack' property.
This does not support entry types with sub-entries such as files and
CBFS, but it does support sections.
Signed-off-by: Simon Glass <sjg@chromium.org>
The positioning does not currently work correctly if at the end of an
image with no fixed size. Also if the header is in the middle of an image
it can cause a gap in the image since the header position is normally at
the image end, so entries after it are placed after the end of the image.
Fix these problems and add more tests to cover these cases.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present it is not possible to discover the contraints to repacking an
image (e.g. maximum section size) since this information is not preserved
from the original image description.
Add new 'orig-offset' and 'orig-size' properties to hold this. Add them to
the main device tree in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
While it is useful and efficient to build images in a single pass from a
unified description, it is sometimes desirable to update the image later.
Add support for replace an existing file with one of the same size. This
avoids needing to repack the file. Support for more advanced updates will
come in future patches.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present EnsureCompiled() uses an file from the 'output' directory (in
the tools module) when compiling the device tree. This is fine in most
cases, allowing useful inspection of the output files from binman.
However in functional tests, _SetupDtb() creates an output directory and
immediately removes it afterwards. This serves no benefit and just
confuses things, since the 'official' output directory is supposed to be
created and destroyed in control.Binman().
Add a new parameter for the optional temporary directory to use, and use a
separate temporary directory in _SetupDtb().
Signed-off-by: Simon Glass <sjg@chromium.org>
We use this same combination of properties several times in tests. Add a
constant for it to avoid typos, etc.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since binman supports multiple images it is useful to know which one
created the image that has been read. Then it is possible to look up that
name in the 'master' device tree (containing the description of all
images).
Add a property for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Current test coverage is likely sufficient for the logic used to place
sections in the image. However it seems useful to add a test specifically
for nested sections, since these could have some unusual interactions.
Add a new test for this and aligned sections. This test failed before the
refactor to drop the bsection.py file (Section class), but passes now.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to extract all binaries from the image, or a
subset of them. Add a new 'extract' command to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.
Signed-off-by: Simon Glass <sjg@chromium.org>
When support for sections (and thus hierarchical images) was added to
binman, the decision was made to create a new Section class which could
be used by both Image and an Entry_section class. The decision between
using inheritance and composition was tricky to make, but in the end it
was decided that Image was different enough from Entry that it made sense
to put the implementation of sections in an entirely separate class. It
also has the advantage that core Image code does have to rely on an entry
class in the etype directory.
This work was mostly completed in commit:
8f1da50ccc "binman: Refactor much of the image code into 'section'
As a result of this, the Section class has its own version of things like
offset and size and these must be kept in sync with the parent
Entry_section class in some cases.
In the last year it has become apparent that the cost of keeping things in
sync is larger than expected, since more and more code wants to access
these properties.
An alternative approach, previously considered and rejected, now seems
better.
Adjust Image to be a subclass of Entry_section. Move the code from Section
(in bsection.py) to Entry_section and delete Section. Update all tests
accordingly.
This requires substantial changes to Image. Overall the changes reduce
code size by about 240 lines. While much of that is just boilerplate from
Section, there are quite a few functions in Entry_section which now do not
need to be overiden from Entry. This suggests the change is beneficial
even without further functionality being added.
A side benefit is that the properties of sections are now consistent with
other entries. This fixes a problem in testListCmd() where some properties
are missing for sections.
Unfortunately this is a very large commit since it is not feasible to do
the migration piecemeal. Given the substantial tests available and the
100% code coverage of binman, we should be able to do this safely.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is possible to read an Image, locate its FDT map and then read it into
the binman data structures. This allows full access to the entries that
were written to the image. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for locating an image's Fdt map which is used to determine
the contents and structure of the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to summarise all the entries in an image, e.g. to
display this to this user. Add a new ListEntries() method to Entry, and
set up a way to call it through the Image class.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to add the CBFS file information (offset, size, etc.) into
the FDT so that the layout is complete. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the logic skips the blob class' handling of compression, so
this is not supported with device tree entries. Fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for detecting entries that change size after they have already
been packed, and re-running packing when it happens.
This removes the limitation that entry size cannot change after
PackEntries() is called.
Signed-off-by: Simon Glass <sjg@chromium.org>
This function raises an exception with its arguments around the wrong way
so the message is incorrect. Fix this as well as a few minor comment
problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
This class is the new way to handle arguments in Python. Convert binman
over to use it. At the same time, introduce commands so that we can
separate out the different parts of binman functionality.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to quickly locate the FDT map in the image. An
easy way to do this is with a pointer at the start or end of the image.
Add an 'image header' entry, which places a magic number followed by a
pointer to the FDT map. This can be located at the start or end of the
image, or at a chosen location.
As part of this, update GetSiblingImagePos() to detect missing siblings.
Signed-off-by: Simon Glass <sjg@chromium.org>
An FDT map is an entry which holds a full description of the image
entries, in FDT format. It can be discovered using the magic string at
its start. Tools can locate and read this entry to find out what entries
are in the image and where each entry is located.
Signed-off-by: Simon Glass <sjg@chromium.org>
Quite a few tests will use a real device tree and need it updated with the
binman metadata. Add a helper function for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
A feature of CBFS is that it allows files to be positioned at particular
offset (as with binman in general). This is useful to support
execute-in-place (XIP) code, since this may not be relocatable.
Add a new cbfs-offset property to control this.
Signed-off-by: Simon Glass <sjg@chromium.org>
An Integrated Firmware Image is used to hold various binaries used for
booting with Apollolake and some later devices. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for putting CBFSs (Coreboot Filesystems) in an image. This
allows binman to produce firmware images used by coreboot to boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tools like ifwitool may not be available in the PATH, but are available in
the build. These tools may be needed by tests, so allow tests to use the
--toolpath flag.
Also use this flag with travis.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes when debugging tests it is useful to keep the input and output
directories so they can be examined later. Add an option for this and
update the binman tests to support it. This affects both the test class
and the tearDown() function called after each test.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the -v flag is ignored with tests, so that (for example) -v2
does not have any effect. Update binman to pass this flag through to tests
so that they work just like running binman normally, except in a few
special cases where we are actually testing behaviour with different
levels of verbosity.
Signed-off-by: Simon Glass <sjg@chromium.org>
We currenty support using the ELF file in U-Boot proper and SPL, but not
TPL. Add this as it is useful both with sandbox and for CBFS to allow
adding TPL as a 'stage'.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present text entries use an indirect method to specify the text to use,
with a label pointing to the text itself.
Allow the text to be directly written into the node. This is more
convenient in cases where the text is constant.
Signed-off-by: Simon Glass <sjg@chromium.org>
FD is a bit confusing so write this out in full. Also avoid splitting the
string so that people can grep for the error message more easily.
Signed-off-by: Simon Glass <sjg@chromium.org>
These files are text files so should be read as such, so that
string-equality assertions work as expected.
With this binman tests work correctly on Python 2 and Python 3:
PYTHONPATH=/tmp/b/sandbox_spl/scripts/dtc/pylibfdt \
python ./tools/binman/binman -t
Change first line of binman.py to end "python3":
PYTHONPATH=~/cosarm/dtc/pylibfdt:tools/patman \
python3 ./tools/binman/binman -t
Signed-off-by: Simon Glass <sjg@chromium.org>
With Python 3 we want to use the 'bytes' type instead of 'str'. Adjust the
code accordingly so that it works on both Python 2 and Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
The method of multiplying a character by a number works well for creating
a repeated string in Python 2. But in Python 3 we need to use bytes()
instead, to avoid unicode problems, since 'bytes' is no-longer just an
alias of 'str'.
Create a function to handle this detail and call it from the relevant
places in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
The lz4 utility inserts binary data in its output which cannot always be
converted to unicode (nor should it be). Fix this by using the new binary
mode for program output.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present some files are opened in text mode despite containing binary
data. This works on Python 2 but not always on Python 3, due to unicode
problems. BC&D are not my favourite people. Adjust the affected open()
statements to use binary mode.
Signed-off-by: Simon Glass <sjg@chromium.org>
Python 3 requires this, and Python 2 allows it. Convert the code over to
ensure compatibility with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present sections are always placed automatically. Even if an 'offset'
property is provided it is ignored. Update the logic to support an offset
for sections.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
This message is not very important since it is simply indicating that the
user's instructions are being followed. Only show it when the verbosity
level is above the default.
Also drop the unnecessary extra newline on this message, which causes two
line breaks.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Unfortunately the test was not included in the original implementation.
Add one.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
