At present we sometimes see problems in gitlab where the environment has
0x80 characters or sequences which are not valid UTF-8.
Avoid this by using bytes for the environment, both internal to buildman
and when writing out the 'env' file. Add a test to make sure this works
as expected.
Reported-by: Marek Vasut <marex@denx.de>
Fixes: e5fc79ea71 ("buildman: Write the environment out to an 'env' file")
Signed-off-by: Simon Glass <sjg@chromium.org>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move buildman to use absolute imports. Also adjust moveconfig.py too since
it uses some buildman modules and cannot work without this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At kernel.org aarch64 toolchains are published in folder
arm64. Fix the URL for that case, so that we can fetch
toolchains on aarch64 machines.
Signed-off-by: Matthias Brugger <mbrugger@suse.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE
Signed-off-by: Simon Glass <sjg@chromium.org>
The parser responsible for the '[make-flags]' section in
the '.buildman' settings file is currently not able to
handle quoted strings, as given in the sample bellow:
[make-flags]
qemu_arm=HOSTCC="cc -isystem /add/include" HOSTLDFLAGS="-L/add/lib"
This patch replaces the simple string splitter based on the <space>
delimiter with a regex tokenizer that preserves spaces inside double
quoted strings.
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@gmail.com>
Unfortunately, for some releases the kernel.org toolchain tarball names adhere
to the following pattern:
<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
while others use the following pattern:
<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz
Notice that the first pattern has dashes throughout, while the second has
dashes throughout except just before the target architecture which has an
underscore.
The "dash throughout" versions from kernel.org are:
8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1
while the "dash and underscore" versions from kernel.org are:
7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4
This tweak allows the code to handle both versions. Note that this tweak also
causes the architecture parsing to get confused and find the following two
bogus architectures, "2.0" and "64", which are explicitly checked for, and
removed.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Change single quotes to double quotes:
Signed-off-by: Simon Glass <sjg@chromium.org>
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in
a way that is different from most toolchains. The first entry when unpacking
most toolchain tarballs is:
gcc-<version>-nolib/<targetarch>-<system>
e.g.:
gcc-8.1.0-nolibc/aarch64-linux/
The first entry of the hexagon toolchain, however, is:
gcc-4.6.1-nolibc/
This causes the buildman logic in toolchain.py::ScanPath() to not be able to
find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin}
instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman
tries to download a set of toolchains that includes hexagon, the script fails.
This update takes the second line of the tarball unpacking (which works for
all the toolchains I've tested from kernel.org) and parses it to take the
first two elements, separated by '/'. It makes this logic a bit more robust.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add support for gcc versions 7.3.0, 6.4.0 and 4.9.4.
Also use a regex for matching the tarball names. Some gcc versions
use '-ARCH-' instead of '_ARCH-'.
As part of this, we switch TravisCI to also using these toolchains for
all platforms.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Build tools like Make, gcc or binutils support localized output
or unicode encoded output dependent on the default system locale.
This is not useful for buildman, where we want reproducible
warning or error messages or where the output of binutils is
further processed.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Many toolchains for ARM use the 'gnueabihf' suffix rather than just
'gnueabi', so allow these to be used, but with a lower priority than
'gnueabi' ones.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Now we can use compiler wrapper such as ccache or distcc for buildman.
Signed-off-by: York Sun <york.sun@nxp.com>
CC: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
The current code for setting up the toolchain config always writes the new
paths to an item called 'toolchain'. This means that it will overwrite any
existing toolchain item with the same name. In practice, this means that:
buildman --fetch-arch all
will fetch all toolchains, but only the path of the final one will be added
to the config. This normally works out OK, since most toolchains are the
same version (e.g. gcc 4.9) and will be found on the same path. But it is
not correct and toolchains for archs which don't use the same version will
not function as expected.
Adjust the code to use a complete glob of the toolchain path.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Use colour to make it easier to see what is going on. Also print a message
before downloading a new toolchain. Mention --fetch-arch in the message that
is shown when there are no available toolchains, since this is the quickest
way to resolve the problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
When there are no toolchains a warning is printed. But in some cases this is
confusing, such as when the user is fetching new toolchains.
Adjust the function to supress the warning in this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman allows you to specify the directory containing the
toolchain, but not the actual toolchain prefix. If there are multiple
toolchains in a single directory, this can be inconvenient.
Add a new 'toolchain-prefix' setting to the settings file, which allows
the full prefix (or path to the C compiler) to be specified.
Update the documentation to match.
Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
At present the architecture is deduced from the toolchain filename. Allow it
to be specified by the caller.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com
At present the priority of a toolchain is calculated from its filename based
on hard-coded rules. Allow it to be specified by the caller. We will use
this in a later patch. Also display the priority and provide a message when
it is overriden by another toolchain of higher priority.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Normally we use a single quote for strings unless there is a reason not to
(such as an embedded single quote). Fix a few counter-examples in this file.
Also add a missing function-argument comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Also read gcc 4.9.0 at kernel.org which also have Microblaze toolchain.
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Fixed unit test failure by updating the test:
Signed-off-by: Simon Glass <sjg@chromium.org>
Commit d908898 updated the ScanPath() function but not its documentation
and not all its callers.
This breaks the toolchain check after it is downloaded. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Heiko Schocher <hs@denx.de>
When buildman scans a toolchain path, it stops at the
first toolchain found. However, a single path can contains
several toolchains, each with its own prefix.
This patch lets buildman scan all toolchains in the path.
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
The site at https://www.kernel.org/pub/tools/crosstool/ is a convenient
repository of toolchains which can be used for U-Boot. Add a feature to
download and install a toolchain for a selected architecture automatically.
It isn't clear how long this site will stay in the current place and
format, but we should be able to rely on bug reports if it changes.
Suggested-by: Marek Vašut <marex@denx.de>
Suggested-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases there may be multiple toolchains with the same name in the
path. Provide an option to use the full path in the CROSS_COMPILE
environment variable.
Note: Wolfgang mentioned that this is dangerous since in some cases there
may be other tools on the path that are needed. So this is set up as an
option, not the default. I will need test confirmation (i.e. that this
commit fixes a real problem) before merging it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Steve Rae <srae@broadcom.com>
If:
1. Toolchains A and B have the same filename
2. Toolchain A is in the PATH
3. Toolchain B is given in ~/.buildman and buildman uses it to build
then buildman will add toolchain B to the end of its path but will not
necessarily use it since U-Boot will find toolchain A first in the PATH.
Try to fix this by putting the toolchain first in the path instead of
last.
Signed-off-by: Simon Glass <sjg@chromium.org>
The assumption that the compiler name will always end in gcc is incorrect
for clang and apparently on BSD.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman currently lacks testing in many areas, including its use of git,
make and many command-line flags.
Add a functional test which covers some of these areas. So far it does
a fake 'build' of all boards for the current source tree.
This version reads the real ~/.buildman and boards.cfg files. Future work
will improve this.
Signed-off-by: Simon Glass <sjg@chromium.org>
These characters are commonly used in variables, so permit them. Also
document the permitted characters.
Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
'-elf' appears twice in the toolchain priority_list.
The second one is rudundant.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Toolchains.__init__ is expected to display a warning message
when the [toolchain] section is missing from ~/.buildman file.
But it never works.
In that case, instead, buildmain fails with an error message
which is difficult to understand:
Traceback (most recent call last):
File "tools/buildman/buildman", line 126, in <module>
control.DoBuildman(options, args)
File "/home/foo/u-boot/tools/buildman/control.py", line 78, in DoBuildman
toolchains = toolchain.Toolchains()
File "/home/foo/u-boot/tools/buildman/toolchain.py", line 106, in __init__
config_fname)
NameError: global name 'config_fname' is not defined
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
When a toolchain invocation fails, an exception is thrown but not caught
which then aborts the entire toolchain detection process. To solve this,
request that exceptions not be thrown, since the toolchain init code
already error-checks the command result. This solves e.g.:
- found '/usr/bin/winegcc'
Traceback (most recent call last):
...
Exception: Error running '/usr/bin/winegcc --version'
Change-Id: I579c72ab3b021e38b14132893c3375ea257c74f0
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
(formatted to 80cols)
There are a few make options such as BUILD_TAG which can be provided when
building U-Boot. Provide a way for buildman to pass these flags to make
also.
The flags should be in a [make-flags] section and arranged by target name
(the 'target' column in boards.cfg. See the README for more details.
Signed-off-by: Simon Glass <sjg@chromium.org>
This tool handles building U-Boot to check that you have not broken it
with your patch series. It can build each individual commit and report
which boards fail on which commits, and which errors come up. It also
shows differences in image sizes due to particular commits.
Buildman aims to make full use of multi-processor machines.
Documentation and caveats are in tools/buildman/README.
Signed-off-by: Simon Glass <sjg@chromium.org>
