There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.
Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present even if only a single thread is in use, buildman still uses
threading.
For some debugging it is helpful to do everything in the main process.
Allow -T0 to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Older versions of this script don't support the -q flag. Since buildman
runs this script from when it starts, we may get the old version.
Fix this in two ways:
1. Use the version from the same tree as buildman is run from, if
available
2. Failing that, allow the -q flag to be missing
Signed-off-by: Simon Glass <sjg@chromium.org>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move buildman to use absolute imports. Also adjust moveconfig.py too since
it uses some buildman modules and cannot work without this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is a bad idea to use the default output directory ('..') with -w since
it does a build in that directory and writes various files these.
Require that -o is given to avoid this problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current exit codes of 128 and 129 are useful in that they do not
conflict with those returned by tools, but they are not actually valid.
It seems better to pick some codes which work with 'bit bisect run'.
Update them to 100 (for errors) and 101 (for warnings).
Signed-off-by: Simon Glass <sjg@chromium.org>
These are becoming more common now. They cause boards to show warnings
which can be mistaking for compiler warnings.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately the plague of device-tree warnings has not lifted. These
warnings infiltrate almost every build, adding noise and confusion.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman defaults to running 'mrproper' on every thread before
it starts building commits for each board. This can add a delay of about 5
seconds to the start of the process, since the tools and other invariants
must be rebuilt.
In particular, a build without '-b', to build current source, runs much
slower without -I, since any existing build is removed, thus losing the
possibility of an incremental build.
Partly this behaviour was to avoid strange build-system problems caused by
running 'make defconfig' for one board and then one with a different
architecture. But these problems were fixed quite a while ago.
The -I option (which disabled mrproper) was introduced four years ago and
does not seem to cause any problems with builds.
So make -I the default and deprecate the option. To allow use of
'mrproper', add a new -m flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
There is no point in setting the ARCH environment variable since the
U-Boot build system no-longer uses it.
It seems safe to drop this feature since it was only recently added.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
This is useful in some situations, in particular with -w and when building
in-tree. Now that we are more careful about what we remove in
_PrepareOutputSpace(), it should be safe to relax this restriction.
Update the progress information also so it is clear what buildman is
doing. Remove files can take a long time.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes we don't want buildman to return failure if it seems warnings.
Add a -W option to support this. If buildman detects warnings (and no
errors) it will return an exit code of 0 (success).
Note that the definition of 'warnings' includes the migration warnings
produced by U-Boot, such as:
===================== WARNING ======================
This board does not use CONFIG_DM_MMC. Please update
...
====================================================
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
It is useful to run a simple build and put all the output in a single
directory. Add a -w option to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman looks at toolchains, then commits and then boards.
Move the board processing up above the commit processing, since it relates
to the toolchain code. This will make it easier to check the toolchains
needed for a board without processing commits first.
Signed-off-by: Simon Glass <sjg@chromium.org>
Now that we have buildman telling genboards.cfg to use an output
directory we need to ensure that it exists.
Cc: Bin Meng <bmeng.cn@gmail.com>
Cc: Simon Glass <sjg@chromium.org>
Fixes: bc750bca12 ("tools: buildman: Honor output directory when generating boards.cfg")
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
buildman always generates boards.cfg in the U-Boot source tree.
When '-o' is given, we should generate boards.cfg to the given
output directory.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Fix a typo in the error message from CheckOutputDir().
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present --list-tool-chains prints a lot of information about the
toolchain-probing process. This is generally not very interesting.
Update buildman to print this only if --list-tool-chains is given
with -v.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present 'buildman sandbox' will build all 5 boards for the sandbox
architecture rather than the single board 'sandbox'. The only current way
to exclude sandbox_spl, sandbox_noblk, etc. is to use -x which is a bit
clumbsy.
Add a --boards option to allow individual build targets to be specified.
Signed-off-by: Simon Glass <sjg@chromium.org>
When summarising the builds, add the -U option to emit delta lines for
the default environment built into U-Boot at each commit.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
When the U-Boot base directory happens to have the same name as the branch
that buildman is directed to use via the '-b' option and no output
directory is specified with '-o', buildman happily starts removing the
whole U-Boot sources eventually only stopped with the error message:
OSError: [Errno 20] Not a directory: '../<branch-name>/boards.cfg
Add a check to avoid this and also deal with the case where '-o' points
to the source directory, or any subdirectory of it.
Finally, tidy up the confusing logic for removing the old tree when using
-b. This is only done when building a branch.
Signed-off-by: Lothar Waßmann <LW@KARO-electronics.de>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Lothar Waßmann <LW@KARO-electronics.de>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Add a new option '-E' for treating all compiler warnings as errors.
Eventually this will pass 'KCFLAGS=-Werror' to Kbuild.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
As well as showing the number of boards, allow showing the actual list of
boards that would be built, if -v is provided.
Signed-off-by: Simon Glass <sjg@chromium.org>
When using #define CONFIG_SOME_OPTION, the value it set to '1'. When using
defconfig (i.e. CONFIG_SOME_OPTION=y) the value is set to 'y'. This results
in differences showing up with -K. These differences are seldom useful.
Adjust buildman to suppress these differences by default.
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally buildman does a full build of a board. This includes creating the
u-boot.cfg file which contains all the configuration options. Buildman uses
this file with the -K option, to show differences in effective configuration
for each commit.
Doing a full build of U-Boot just to create the u-boot.cfg file is wasteful.
Add a -D option which causes buildman to only create the configuration. This
is enough to support use of -K and can be done much more quickly (typically
5-10 times faster).
Signed-off-by: Simon Glass <sjg@chromium.org>
It is more useful to have this method raise an error when something goes
wrong. Make this the default and adjust the few callers that don't want to
use it this way.
Signed-off-by: Simon Glass <sjg@chromium.org>
It doesn't make sense to complain about missing toolchains when the
--fetch-arch option is being used. The user is presumably aware that there
is a toolchain problem and is actively correcting it by running with this
option.
Refactor the code to avoid printing this confusing message.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use colour to make it easier to see what is going on. Also print a message
before downloading a new toolchain. Mention --fetch-arch in the message that
is shown when there are no available toolchains, since this is the quickest
way to resolve the problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
One use-case for buildman is to continually run it interactively after
each small step in a large refactoring operation. This gives more
immediate feedback than making a number of commits and then going back and
testing them. For this to work well, buildman needs to be extremely fast.
At present, a couple issues prevent it being as fast as it could be:
1) Each time buildman runs "make %_defconfig", it runs "make mrproper"
first. This throws away all previous build results, requiring a
from-scratch build. Optionally avoiding this would speed up the build, at
the cost of potentially causing or missing some build issues.
2) A build tree is created per thread rather than per board. When a thread
switches between building different boards, this often causes many files
to be rebuilt due to changing config options. Using a separate build tree
for each board would avoid this. This does put more strain on the system's
disk cache, but it is worth it on my system at least.
This commit adds two command-line options to implement the changes
described above; -I ("--incremental") turns of "make mrproper" and -P
("--per-board-out-dir") creats a build directory per board rather than per
thread.
Tested:
./tools/buildman/buildman.py tegra
./tools/buildman/buildman.py -I -P tegra
./tools/buildman/buildman.py -b tegra_dev tegra
./tools/buildman/buildman.py -b tegra_dev -I -P tegra
... each once after deleting the buildman result/work directory, and once
"incrementally" after a previous identical invocation.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org> # v1
Tested-by: Simon Glass <sjg@chromium.org> # v1
Acked-by: Simon Glass <sjg@chromium.org>
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
It is useful to be able to see CONFIG changes made by commits. Add this
feature to buildman using the -K flag so that all CONFIG changes are
reported.
The CONFIG options exist in a number of files. Each is reported
individually as well as a summary that covers all files. The output
shows three parts: green for additions, red for removals and yellow for
changes.
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally buildman runs with 'make -s' meaning that only errors and warnings
appear in the log file. Add a -V option to run make in verbose mode, and
with V=1, causing a full build log to be created.
Signed-off-by: Simon Glass <sjg@chromium.org>
The site at https://www.kernel.org/pub/tools/crosstool/ is a convenient
repository of toolchains which can be used for U-Boot. Add a feature to
download and install a toolchain for a selected architecture automatically.
It isn't clear how long this site will stay in the current place and
format, but we should be able to rely on bug reports if it changes.
Suggested-by: Marek Vašut <marex@denx.de>
Suggested-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases there may be multiple toolchains with the same name in the
path. Provide an option to use the full path in the CROSS_COMPILE
environment variable.
Note: Wolfgang mentioned that this is dangerous since in some cases there
may be other tools on the path that are needed. So this is set up as an
option, not the default. I will need test confirmation (i.e. that this
commit fixes a real problem) before merging it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Steve Rae <srae@broadcom.com>
Adjust the -b flag to permit a range expression as well as a branch.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Tested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
When running tests the output directory is often wiped. This is only safe if
a branch is being built. The output directory may contain other things
besides the buildman test output.
Signed-off-by: Simon Glass <sjg@chromium.org>
When building current source for a single board, buildman puts the output
in <output_dir>/current/current/<board>. Add an option to make it use
<output_dir>/<board> instead. This removes the unnecessary directories
in that case, controlled by the --no-subdirs/-N option.
Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman normally obtains the upstream commit by asking git. Provided that
the branch was created with 'git checkout -b <branch> <some_upstream>' then
this normally works.
When there is no upstream, we can try to guess one, by looking up through
the commits until we find a branch. Add a function to try this and print
a warning if buildman ends up relying on it.
Also update the documentation to match.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
Buildman currently puts current-source builds in a current/current
subdirectory, but there is no need for the extra depth.
Suggested-by: Albert Aribaud <albert.u.boot@aribaud.net>
Signed-off-by: Simon Glass <sjg@chromium.org>
When using summary mode (-s) we don't always want to display errors.
Allow this option to be omitted.
Series-to: u-boot
Series-cc: albert
Change-Id: I6b37754d55eb920ecae114fceba55834b43ea3b9
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Albert Aribaud <albert.u.boot@aribaud.net>
