binman: Automatically expand phase binaries into sections

When creating an entry, check for an expanded version of that entry, then
use it instead. This allows, for example use of:

   u-boot {
   };

instead of having to write out in full:

   u-boot {
      type = "section";

      u-boot-nodtb {
      };

      u-boot-dtb {
      };
   };

Add an implementaion of this and associated documentation.

Signed-off-by: Simon Glass <sjg@chromium.org>
This commit is contained in:
Simon Glass 2021-03-18 20:25:07 +13:00
parent 0b6023ee41
commit 0668492728
16 changed files with 571 additions and 18 deletions

View File

@ -840,6 +840,83 @@ of the image) can be used to point to the FDT map. See fdtmap and image-header
entries for more information.
Expanded entries
----------------
Binman automatically replaces 'u-boot' with an expanded version of that, i.e.
'u-boot-expanded'. This means that when you write:
u-boot {
};
you actually get:
u-boot {
type = "u-boot-expanded';
};
which in turn expands to:
u-boot {
type = "section";
u-boot-nodtb {
};
u-boot-dtb {
};
};
U-Boot's various phase binaries actually comprise two or three pieces.
For example, u-boot.bin has the executable followed by a devicetree.
With binman we want to be able to update that devicetree with full image
information so that it is accessible to the executable. This is tricky
if it is not clear where the devicetree starts.
The above feature ensures that the devicetree is clearly separated from the
U-Boot executable and can be updated separately by binman as needed. It can be
disabled with the --no-expanded flag if required.
The same applies for u-boot-spl and u-boot-spl. In those cases, the expansion
includes the BSS padding, so for example:
spl {
type = "u-boot-spl"
};
you actually get:
spl {
type = "u-boot-expanded';
};
which in turn expands to:
spl {
type = "section";
u-boot-spl-nodtb {
};
u-boot-spl-bss-pad {
};
u-boot-spl-dtb {
};
};
Of course we should not expand SPL if it has no devicetree. Also if the BSS
padding is not needed (because BSS is in RAM as with CONFIG_SPL_SEPARATE_BSS),
the 'u-boot-spl-bss-pad' subnode should not be created. The use of the expaned
entry type is controlled by the UseExpanded() method. In the SPL case it checks
the 'spl-dtb' entry arg, which is 'y' or '1' if SPL has a devicetree.
For the BSS case, a 'spl-bss-pad' entry arg controls whether it is present. All
entry args are provided by the U-Boot Makefile.
Compression
-----------

View File

@ -87,6 +87,15 @@ See cros_ec_rw for an example of this.
Entry: blob-phase: Section that holds a phase binary
----------------------------------------------------
This is a base class that should not normally be used directly. It is used
when converting a 'u-boot' entry automatically into a 'u-boot-expanded'
entry; similarly for SPL.
Entry: cbfs: Entry containing a Coreboot Filesystem (CBFS)
----------------------------------------------------------
@ -840,8 +849,7 @@ Properties / Entry arguments:
This is the U-Boot binary, containing relocation information to allow it
to relocate itself at runtime. The binary typically includes a device tree
blob at the end of it. Use u-boot-nodtb if you want to package the device
tree separately.
blob at the end of it.
U-Boot can access binman symbols at runtime. See:
@ -849,6 +857,9 @@ U-Boot can access binman symbols at runtime. See:
in the binman README for more information.
Note that this entry is automatically replaced with u-boot-expanded unless
--no-expanded is used.
Entry: u-boot-dtb: U-Boot device tree
@ -902,6 +913,21 @@ Properties / Entry arguments:
Entry: u-boot-expanded: U-Boot flat binary broken out into its component parts
------------------------------------------------------------------------------
This is a section containing the U-Boot binary and a devicetree. Using this
entry type automatically creates this section, with the following entries
in it:
u-boot-nodtb
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
Entry: u-boot-img: U-Boot legacy image
--------------------------------------
@ -925,8 +951,8 @@ Properties / Entry arguments:
This is the U-Boot binary, containing relocation information to allow it
to relocate itself at runtime. It does not include a device tree blob at
the end of it so normally cannot work without it. You can add a u-boot-dtb
entry after this one, or use a u-boot entry instead (which contains both
U-Boot and the device tree).
entry after this one, or use a u-boot entry instead, normally expands to a
section containing u-boot and u-boot-dtb
@ -952,6 +978,9 @@ in the binman README for more information.
The ELF file 'spl/u-boot-spl' must also be available for this to work, since
binman uses that to look up symbols to write into the SPL binary.
Note that this entry is automatically replaced with u-boot-spl-expanded
unless --no-expanded is used.
Entry: u-boot-spl-bss-pad: U-Boot SPL binary padded with a BSS region
@ -999,6 +1028,29 @@ be relocated to any address for execution.
Entry: u-boot-spl-expanded: U-Boot SPL flat binary broken out into its component parts
--------------------------------------------------------------------------------------
Properties / Entry arguments:
- spl-dtb: Controls whether this entry is selected (set to 'y' or '1' to
select)
This is a section containing the U-Boot binary, BSS padding if needed and a
devicetree. Using this entry type automatically creates this section, with
the following entries in it:
u-boot-spl-nodtb
u-boot-spl-bss-pad
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
This entry is selected based on the value of the 'spl-dtb' entryarg. If
this is non-empty (and not 'n' or '0') then this expanded entry is selected.
Entry: u-boot-spl-nodtb: SPL binary without device tree appended
----------------------------------------------------------------
@ -1008,8 +1060,9 @@ Properties / Entry arguments:
This is the U-Boot SPL binary, It does not include a device tree blob at
the end of it so may not be able to work without it, assuming SPL needs
a device tree to operate on your platform. You can add a u-boot-spl-dtb
entry after this one, or use a u-boot-spl entry instead (which contains
both SPL and the device tree).
entry after this one, or use a u-boot-spl entry instead' which normally
expands to a section containing u-boot-spl-dtb, u-boot-spl-bss-pad and
u-boot-spl-dtb
SPL can access binman symbols at runtime. See:
@ -1054,6 +1107,9 @@ in the binman README for more information.
The ELF file 'tpl/u-boot-tpl' must also be available for this to work, since
binman uses that to look up symbols to write into the TPL binary.
Note that this entry is automatically replaced with u-boot-tpl-expanded
unless --no-expanded is used.
Entry: u-boot-tpl-bss-pad: U-Boot TPL binary padded with a BSS region
@ -1111,6 +1167,29 @@ be relocated to any address for execution.
Entry: u-boot-tpl-expanded: U-Boot TPL flat binary broken out into its component parts
--------------------------------------------------------------------------------------
Properties / Entry arguments:
- tpl-dtb: Controls whether this entry is selected (set to 'y' or '1' to
select)
This is a section containing the U-Boot binary, BSS padding if needed and a
devicetree. Using this entry type automatically creates this section, with
the following entries in it:
u-boot-tpl-nodtb
u-boot-tpl-bss-pad
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
This entry is selected based on the value of the 'tpl-dtb' entryarg. If
this is non-empty (and not 'n' or '0') then this expanded entry is selected.
Entry: u-boot-tpl-nodtb: TPL binary without device tree appended
----------------------------------------------------------------
@ -1120,8 +1199,9 @@ Properties / Entry arguments:
This is the U-Boot TPL binary, It does not include a device tree blob at
the end of it so may not be able to work without it, assuming TPL needs
a device tree to operate on your platform. You can add a u-boot-tpl-dtb
entry after this one, or use a u-boot-tpl entry instead (which contains
both TPL and the device tree).
entry after this one, or use a u-boot-tpl entry instead, which normally
expands to a section containing u-boot-tpl-dtb, u-boot-tpl-bss-pad and
u-boot-tpl-dtb
TPL can access binman symbols at runtime. See:

View File

@ -0,0 +1,51 @@
# SPDX-License-Identifier: GPL-2.0+
# Copyright 2021 Google LLC
# Written by Simon Glass <sjg@chromium.org>
#
# Entry-type base class for U-Boot or SPL binary with devicetree
#
from binman.etype.section import Entry_section
class Entry_blob_phase(Entry_section):
"""Section that holds a phase binary
This is a base class that should not normally be used directly. It is used
when converting a 'u-boot' entry automatically into a 'u-boot-expanded'
entry; similarly for SPL.
"""
def __init__(self, section, etype, node, root_fname, dtb_file, bss_pad):
"""Set up a new blob for a phase
This holds an executable for a U-Boot phase, optional BSS padding and
a devicetree
Args:
section: entry_Section object for this entry's parent
etype: Type of object
node: Node defining this entry
root_fname: Root filename for the binary ('u-boot',
'spl/u-boot-spl', etc.)
dtb_file: Name of devicetree file ('u-boot.dtb', u-boot-spl.dtb',
etc.)
bss_pad: True to add BSS padding before the devicetree
"""
# Put this here to allow entry-docs and help to work without libfdt
global state
from binman import state
super().__init__(section, etype, node)
self.root_fname = root_fname
self.dtb_file = dtb_file
self.bss_pad = bss_pad
def ExpandEntries(self):
"""Create the subnodes"""
names = [self.root_fname + '-nodtb', self.root_fname + '-dtb']
if self.bss_pad:
names.insert(1, self.root_fname + '-bss-pad')
for name in names:
subnode = state.AddSubnode(self._node, name)
# Read entries again, now that we have some
self._ReadEntries()

View File

@ -2,7 +2,7 @@
# Copyright (c) 2016 Google, Inc
# Written by Simon Glass <sjg@chromium.org>
#
# Entry-type module for U-Boot binary
# Entry-type module for the expanded U-Boot binary
#
from binman.entry import Entry
@ -16,14 +16,16 @@ class Entry_u_boot(Entry_blob):
This is the U-Boot binary, containing relocation information to allow it
to relocate itself at runtime. The binary typically includes a device tree
blob at the end of it. Use u-boot-nodtb if you want to package the device
tree separately.
blob at the end of it.
U-Boot can access binman symbols at runtime. See:
'Access to binman entry offsets at run time (fdt)'
in the binman README for more information.
Note that this entry is automatically replaced with u-boot-expanded unless
--no-expanded is used.
"""
def __init__(self, section, etype, node):
super().__init__(section, etype, node)

View File

@ -0,0 +1,24 @@
# SPDX-License-Identifier: GPL-2.0+
# Copyright 2021 Google LLC
# Written by Simon Glass <sjg@chromium.org>
#
# Entry-type module for U-Boot binary
#
from binman.etype.blob_phase import Entry_blob_phase
class Entry_u_boot_expanded(Entry_blob_phase):
"""U-Boot flat binary broken out into its component parts
This is a section containing the U-Boot binary and a devicetree. Using this
entry type automatically creates this section, with the following entries
in it:
u-boot-nodtb
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
"""
def __init__(self, section, etype, node):
super().__init__(section, etype, node, 'u-boot', 'u-boot-dtb', False)

View File

@ -17,8 +17,8 @@ class Entry_u_boot_nodtb(Entry_blob):
This is the U-Boot binary, containing relocation information to allow it
to relocate itself at runtime. It does not include a device tree blob at
the end of it so normally cannot work without it. You can add a u-boot-dtb
entry after this one, or use a u-boot entry instead (which contains both
U-Boot and the device tree).
entry after this one, or use a u-boot entry instead, normally expands to a
section containing u-boot and u-boot-dtb
"""
def __init__(self, section, etype, node):
super().__init__(section, etype, node)

View File

@ -30,6 +30,9 @@ class Entry_u_boot_spl(Entry_blob):
The ELF file 'spl/u-boot-spl' must also be available for this to work, since
binman uses that to look up symbols to write into the SPL binary.
Note that this entry is automatically replaced with u-boot-spl-expanded
unless --no-expanded is used.
"""
def __init__(self, section, etype, node):
super().__init__(section, etype, node)

View File

@ -0,0 +1,45 @@
# SPDX-License-Identifier: GPL-2.0+
# Copyright 2021 Google LLC
# Written by Simon Glass <sjg@chromium.org>
#
# Entry-type module for expanded U-Boot SPL binary
#
from patman import tout
from binman import state
from binman.etype.blob_phase import Entry_blob_phase
class Entry_u_boot_spl_expanded(Entry_blob_phase):
"""U-Boot SPL flat binary broken out into its component parts
Properties / Entry arguments:
- spl-dtb: Controls whether this entry is selected (set to 'y' or '1' to
select)
This is a section containing the U-Boot binary, BSS padding if needed and a
devicetree. Using this entry type automatically creates this section, with
the following entries in it:
u-boot-spl-nodtb
u-boot-spl-bss-pad
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
This entry is selected based on the value of the 'spl-dtb' entryarg. If
this is non-empty (and not 'n' or '0') then this expanded entry is selected.
"""
def __init__(self, section, etype, node):
bss_pad = state.GetEntryArgBool('spl-bss-pad')
super().__init__(section, etype, node, 'u-boot-spl', 'u-boot-spl-dtb',
bss_pad)
@classmethod
def UseExpanded(cls, node, etype, new_etype):
val = state.GetEntryArgBool('spl-dtb')
tout.DoOutput(tout.INFO if val else tout.DETAIL,
"Node '%s': etype '%s': %s %sselected" %
(node.path, etype, new_etype, '' if val else 'not '))
return val

View File

@ -18,8 +18,9 @@ class Entry_u_boot_spl_nodtb(Entry_blob):
This is the U-Boot SPL binary, It does not include a device tree blob at
the end of it so may not be able to work without it, assuming SPL needs
a device tree to operate on your platform. You can add a u-boot-spl-dtb
entry after this one, or use a u-boot-spl entry instead (which contains
both SPL and the device tree).
entry after this one, or use a u-boot-spl entry instead' which normally
expands to a section containing u-boot-spl-dtb, u-boot-spl-bss-pad and
u-boot-spl-dtb
SPL can access binman symbols at runtime. See:

View File

@ -30,6 +30,9 @@ class Entry_u_boot_tpl(Entry_blob):
The ELF file 'tpl/u-boot-tpl' must also be available for this to work, since
binman uses that to look up symbols to write into the TPL binary.
Note that this entry is automatically replaced with u-boot-tpl-expanded
unless --no-expanded is used.
"""
def __init__(self, section, etype, node):
super().__init__(section, etype, node)

View File

@ -0,0 +1,45 @@
# SPDX-License-Identifier: GPL-2.0+
# Copyright 2021 Google LLC
# Written by Simon Glass <sjg@chromium.org>
#
# Entry-type module for expanded U-Boot TPL binary
#
from patman import tout
from binman import state
from binman.etype.blob_phase import Entry_blob_phase
class Entry_u_boot_tpl_expanded(Entry_blob_phase):
"""U-Boot TPL flat binary broken out into its component parts
Properties / Entry arguments:
- tpl-dtb: Controls whether this entry is selected (set to 'y' or '1' to
select)
This is a section containing the U-Boot binary, BSS padding if needed and a
devicetree. Using this entry type automatically creates this section, with
the following entries in it:
u-boot-tpl-nodtb
u-boot-tpl-bss-pad
u-boot-dtb
Having the devicetree separate allows binman to update it in the final
image, so that the entries positions are provided to the running U-Boot.
This entry is selected based on the value of the 'tpl-dtb' entryarg. If
this is non-empty (and not 'n' or '0') then this expanded entry is selected.
"""
def __init__(self, section, etype, node):
bss_pad = state.GetEntryArgBool('tpl-bss-pad')
super().__init__(section, etype, node, 'u-boot-tpl', 'u-boot-tpl-dtb',
bss_pad)
@classmethod
def UseExpanded(cls, node, etype, new_etype):
val = state.GetEntryArgBool('tpl-dtb')
tout.DoOutput(tout.INFO if val else tout.DETAIL,
"Node '%s': etype '%s': %s %sselected" %
(node.path, etype, new_etype, '' if val else 'not '))
return val

View File

@ -18,8 +18,9 @@ class Entry_u_boot_tpl_nodtb(Entry_blob):
This is the U-Boot TPL binary, It does not include a device tree blob at
the end of it so may not be able to work without it, assuming TPL needs
a device tree to operate on your platform. You can add a u-boot-tpl-dtb
entry after this one, or use a u-boot-tpl entry instead (which contains
both TPL and the device tree).
entry after this one, or use a u-boot-tpl entry instead, which normally
expands to a section containing u-boot-tpl-dtb, u-boot-tpl-bss-pad and
u-boot-tpl-dtb
TPL can access binman symbols at runtime. See:

View File

@ -4285,6 +4285,180 @@ class TestFunctional(unittest.TestCase):
self.assertIn('Expected __bss_size symbol in tpl/u-boot-tpl',
str(e.exception))
def checkDtbSizes(self, data, pad_len, start):
"""Check the size arguments in a dtb embedded in an image
Args:
data: The image data
pad_len: Length of the pad section in the image, in bytes
start: Start offset of the devicetree to examine, within the image
Returns:
Size of the devicetree in bytes
"""
dtb_data = data[start:]
dtb = fdt.Fdt.FromData(dtb_data)
fdt_size = dtb.GetFdtObj().totalsize()
dtb.Scan()
props = self._GetPropTree(dtb, 'size')
self.assertEqual({
'size': len(data),
'u-boot-spl/u-boot-spl-bss-pad:size': pad_len,
'u-boot-spl/u-boot-spl-dtb:size': 801,
'u-boot-spl/u-boot-spl-nodtb:size': len(U_BOOT_SPL_NODTB_DATA),
'u-boot-spl:size': 860,
'u-boot-tpl:size': len(U_BOOT_TPL_DATA),
'u-boot/u-boot-dtb:size': 781,
'u-boot/u-boot-nodtb:size': len(U_BOOT_NODTB_DATA),
'u-boot:size': 827,
}, props)
return fdt_size
def testExpanded(self):
"""Test that an expanded entry type is selected when needed"""
self._SetupSplElf()
self._SetupTplElf()
# SPL has a devicetree, TPL does not
entry_args = {
'spl-dtb': '1',
'spl-bss-pad': 'y',
'tpl-dtb': '',
}
self._DoReadFileDtb('194_fdt_incl.dts', use_expanded=True,
entry_args=entry_args)
image = control.images['image']
entries = image.GetEntries()
self.assertEqual(3, len(entries))
# First, u-boot, which should be expanded into u-boot-nodtb and dtb
self.assertIn('u-boot', entries)
entry = entries['u-boot']
self.assertEqual('u-boot-expanded', entry.etype)
subent = entry.GetEntries()
self.assertEqual(2, len(subent))
self.assertIn('u-boot-nodtb', subent)
self.assertIn('u-boot-dtb', subent)
# Second, u-boot-spl, which should be expanded into three parts
self.assertIn('u-boot-spl', entries)
entry = entries['u-boot-spl']
self.assertEqual('u-boot-spl-expanded', entry.etype)
subent = entry.GetEntries()
self.assertEqual(3, len(subent))
self.assertIn('u-boot-spl-nodtb', subent)
self.assertIn('u-boot-spl-bss-pad', subent)
self.assertIn('u-boot-spl-dtb', subent)
# Third, u-boot-tpl, which should be not be expanded, since TPL has no
# devicetree
self.assertIn('u-boot-tpl', entries)
entry = entries['u-boot-tpl']
self.assertEqual('u-boot-tpl', entry.etype)
self.assertEqual(None, entry.GetEntries())
def testExpandedTpl(self):
"""Test that an expanded entry type is selected for TPL when needed"""
self._SetupTplElf()
entry_args = {
'tpl-bss-pad': 'y',
'tpl-dtb': 'y',
}
self._DoReadFileDtb('195_fdt_incl_tpl.dts', use_expanded=True,
entry_args=entry_args)
image = control.images['image']
entries = image.GetEntries()
self.assertEqual(1, len(entries))
# We only have u-boot-tpl, which be expanded
self.assertIn('u-boot-tpl', entries)
entry = entries['u-boot-tpl']
self.assertEqual('u-boot-tpl-expanded', entry.etype)
subent = entry.GetEntries()
self.assertEqual(3, len(subent))
self.assertIn('u-boot-tpl-nodtb', subent)
self.assertIn('u-boot-tpl-bss-pad', subent)
self.assertIn('u-boot-tpl-dtb', subent)
def testExpandedNoPad(self):
"""Test an expanded entry without BSS pad enabled"""
self._SetupSplElf()
self._SetupTplElf()
# SPL has a devicetree, TPL does not
entry_args = {
'spl-dtb': 'something',
'spl-bss-pad': 'n',
'tpl-dtb': '',
}
self._DoReadFileDtb('194_fdt_incl.dts', use_expanded=True,
entry_args=entry_args)
image = control.images['image']
entries = image.GetEntries()
# Just check u-boot-spl, which should be expanded into two parts
self.assertIn('u-boot-spl', entries)
entry = entries['u-boot-spl']
self.assertEqual('u-boot-spl-expanded', entry.etype)
subent = entry.GetEntries()
self.assertEqual(2, len(subent))
self.assertIn('u-boot-spl-nodtb', subent)
self.assertIn('u-boot-spl-dtb', subent)
def testExpandedTplNoPad(self):
"""Test that an expanded entry type with padding disabled in TPL"""
self._SetupTplElf()
entry_args = {
'tpl-bss-pad': '',
'tpl-dtb': 'y',
}
self._DoReadFileDtb('195_fdt_incl_tpl.dts', use_expanded=True,
entry_args=entry_args)
image = control.images['image']
entries = image.GetEntries()
self.assertEqual(1, len(entries))
# We only have u-boot-tpl, which be expanded
self.assertIn('u-boot-tpl', entries)
entry = entries['u-boot-tpl']
self.assertEqual('u-boot-tpl-expanded', entry.etype)
subent = entry.GetEntries()
self.assertEqual(2, len(subent))
self.assertIn('u-boot-tpl-nodtb', subent)
self.assertIn('u-boot-tpl-dtb', subent)
def testFdtInclude(self):
"""Test that an Fdt is update within all binaries"""
self._SetupSplElf()
self._SetupTplElf()
# SPL has a devicetree, TPL does not
self.maxDiff = None
entry_args = {
'spl-dtb': '1',
'spl-bss-pad': 'y',
'tpl-dtb': '',
}
# Build the image. It includes two separate devicetree binaries, each
# with their own contents, but all contain the binman definition.
data = self._DoReadFileDtb(
'194_fdt_incl.dts', use_real_dtb=True, use_expanded=True,
update_dtb=True, entry_args=entry_args)[0]
pad_len = 10
# Check the U-Boot dtb
start = len(U_BOOT_NODTB_DATA)
fdt_size = self.checkDtbSizes(data, pad_len, start)
# Now check SPL
start += fdt_size + len(U_BOOT_SPL_NODTB_DATA) + pad_len
fdt_size = self.checkDtbSizes(data, pad_len, start)
# TPL has no devicetree
start += fdt_size + len(U_BOOT_TPL_DATA)
self.assertEqual(len(data), start)
if __name__ == "__main__":
unittest.main()

View File

@ -141,12 +141,16 @@ def SetEntryArgs(args):
global entry_args
entry_args = {}
tout.Debug('Processing entry args:')
if args:
for arg in args:
m = re.match('([^=]*)=(.*)', arg)
if not m:
raise ValueError("Invalid entry arguemnt '%s'" % arg)
entry_args[m.group(1)] = m.group(2)
name, value = m.groups()
tout.Debug(' %20s = %s' % (name, value))
entry_args[name] = value
tout.Debug('Processing entry args done')
def GetEntryArg(name):
"""Get the value of an entry argument
@ -159,6 +163,19 @@ def GetEntryArg(name):
"""
return entry_args.get(name)
def GetEntryArgBool(name):
"""Get the value of an entry argument as a boolean
Args:
name: Name of argument to retrieve
Returns:
False if the entry argument is consider False (empty, '0' or 'n'), else
True
"""
val = GetEntryArg(name)
return val and val not in ['n', '0']
def Prepare(images, dtb):
"""Get device tree files ready for use

View File

@ -0,0 +1,17 @@
// SPDX-License-Identifier: GPL-2.0+
/dts-v1/;
/ {
#address-cells = <1>;
#size-cells = <1>;
binman {
u-boot {
};
u-boot-spl {
};
u-boot-tpl {
};
};
};

View File

@ -0,0 +1,13 @@
// SPDX-License-Identifier: GPL-2.0+
/dts-v1/;
/ {
#address-cells = <1>;
#size-cells = <1>;
binman {
u-boot-tpl {
};
};
};