Commit Graph

326 Commits

Author SHA1 Message Date
Evan Benn ec6db6c297 patman: Parse checkpatch by message instead of by line
Parse each empty-line-delimited message separately. This saves having to
deal with all the different line content styles, we only care about the
header ERROR | WARNING | NOTE...

Also make checkpatch print line information for a uboot specific
warning.

Signed-off-by: Evan Benn <evanbenn@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-29 03:23:39 -07:00
Tom Rini 2959a8e3a5 patman: Assume we always have pygit2 for tests
Given that we have tests that require pygit2 and it can be installed
like any other python module, fail much more loudly if it is missing.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2021-04-29 03:23:39 -07:00
Sean Anderson d9c3050710 checkpatch: Add warnings for using strn(cat|cpy)
strn(cat|cpy) has a bad habit of not nul-terminating the destination,
resulting in constructions like

	strncpy(foo, bar, sizeof(foo) - 1);
	foo[sizeof(foo) - 1] = '\0';

However, it is very easy to forget about this behavior and accidentally
leave a string unterminated. This has shown up in some recent coverity
scans [1, 2] (including code recently touched by yours truly).

Fortunately, the guys at OpenBSD came up with strl(cat|cpy), which always
nul-terminate strings. These functions are already in U-Boot, so we should
encourage new code to use them instead of strn(cat|cpy).

[1] https://lists.denx.de/pipermail/u-boot/2021-March/442888.html
[2] https://lists.denx.de/pipermail/u-boot/2021-January/438073.html

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-12 17:44:55 -04:00
Simon Glass 42bc156f80 patman: Continue on if warnings are found outside a commit
While we cannot know which commit the warning relates to, this should not
be fatal. Print the warning and carry on.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-04-06 16:33:10 +12:00
Simon Glass 5187b80617 binman: Use standard filenames for SPL/TPL devicetree
At present, before any entry expansion is done (such as a 'files' entry
expanding out to individual entries for each file it contains), we check
the binman definition (i.e. '/binman' node) to find out what devicetree
files are used in the images.

This is a pain, since the definition may change during expansion. For
example if there is no u-boot-spl-dtb entry in the definition at the start,
we assume that the SPL devicetree is not used. But if an entry later
expands to include this, then we don't notice.

In fact the flexibility provided by the current approach of checking the
definition is not really useful. We know that we can have SPL and TPL
devicetrees. We know the pathname to each, so we can simply check if the
files are present. If they are present, we can prepare them and update
them regardless of whether they are actually used. If they are not present,
we cannot prepare/update them anyway, i.e. an error will be generated.

Simplify state.Prepare() so it uses a hard-coded list of devicetree files.

Note that state.PrepareFromLoadedData() is left untouched, since in that
case we have a complete definition from the loaded file, but cannot of
course rely on the devicetree files that created it still being present.
So in that case we still check the image defitions.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-26 17:03:09 +13:00
Nicolas Boichat b1b6efc343 patman: Use less for help file, if available
It's convenient to be able to scroll up in `patman -H`.

Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-03-22 19:23:26 +13:00
Simon Glass 0fb560d9a7 patman: Quieten down the alias checking
When a tag is used in a patch subject (e.g. "tag: rest of message") and
it cannot be found as an alias, patman currently reports a fatal error,
unless -t is provided, in which case it reports a warning.

Experience suggest that the fatal error is not very useful. Instead,
default to reporting a warning, with -t tell patman to ignore it
altogether.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-22 19:23:26 +13:00
Simon Glass c7e42cabed patman: Update documentation to match new usage
With the subcommands some of the documentation examples are no-longer
correct. Fix all of them, so it is consistent.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-03-22 19:23:26 +13:00
Bernhard Kirchen 67637d4b5a fix patman --limit-cc option
patman's --limit-cc option parses its argument to an integer and uses
that to trim the list of CC recipients to a particular maximum. but that
only works if the cc variable is a list, which it is not.

Signed-off-by: Bernhard Kirchen <bernhard.kirchen@mbconnectline.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2021-03-03 19:17:25 -07:00
Simon Glass 6deff872cf patman: Correct lz4 compression parameters
At present on large files, lz4 uses a larger block size (e.g. 256KB) than
the 64KB supported by the U-Boot decompression implementation. Also it is
optimised for maximum compression speed, producing larger output than we
would like.

Update the parameters to correct these problems.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-01-30 14:25:41 -07:00
Tom Rini b11f634b1c Driver model: make some udevice fields private
Driver model: Rename U_BOOT_DEVICE et al.
 dtoc: Tidy up and add more tests
 ns16550 code clean-up
 x86 and sandbox minor fixes for of-platdata
 dtoc prepration for adding build-time instantiation
 -----BEGIN PGP SIGNATURE-----
 
 iQFFBAABCgAvFiEEslwAIq+Gp8wWVbYnfxc6PpAIreYFAl/09LURHHNqZ0BjaHJv
 bWl1bS5vcmcACgkQfxc6PpAIrebjwwf/fHRjYsAY/Yj/+y1xgo3L3sphIvQUqTDF
 KkLl+kHdV5r8W/HJULxLQcF2r7pcPEI6TAQxuj3qQ5SUvm2HviS8GHGPawDEwyht
 HgBp9VD56+HUadMfnbG//DVS73ycbL4XSKlYqpkINEejtnlttsCIawUXX5cTyGM/
 59VkgnKrKvJQRUXvYLa8MTugTs4fkPJGDqhActBk/7SP1SImj+rfalNSqA2/dx6y
 2RnPCSzB1x2231KSj+B1NgGlR3Xb8P8zgh20ijcEU/hrlXBTZyi7K7f4SJR30Efu
 LYkkuj4VbxcV/25RozR0fmknqCs0QyAI+/dql6TNtbTSPC/jAfj0jQ==
 =9kN3
 -----END PGP SIGNATURE-----

Merge tag 'dm-pull-5jan21' of git://git.denx.de/u-boot-dm into next

Driver model: make some udevice fields private
Driver model: Rename U_BOOT_DEVICE et al.
dtoc: Tidy up and add more tests
ns16550 code clean-up
x86 and sandbox minor fixes for of-platdata
dtoc prepration for adding build-time instantiation
2021-01-05 22:34:43 -05:00
Tom Rini 720620e691 Prepare v2021.01-rc5
-----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAl/0YVIACgkQFHw5/5Y0
 tywtEwv/cJWlKgcSnYjuJrxwuJdauUTfXdbUgtCxOtBw/BP4dsKkbGTJPw5q5M+4
 LJJSKyksmJVTX26h1dpkzQjOpWtTDnWqm5CTIxD52oQD7pxK+zCQ9T6S+QbQD0Se
 ogHmZluzFoluxbNgo8tiO52xvMhDO3TVAzxsNDdGfkd5/tAXOHClPc34RmAkdRHU
 VsR89AKdT2q543fiUfrRZYDzdctaNWhRGXMDcJ4+QU/8hQhrpcr8EtHbF+3mWX4K
 pA01pDz150Rn4UI6S2xKEWrjSTHe55fxVj/Qj0rq9z2E/+NqGXemf5s13AR0G/z3
 PqHdVLHzDe64pbOvmyU1pVQ0aMb8vMJUnqx68SQZY3On2c+MjRWQ+7aVVaKOcPGp
 uatk6QMrggHp3Li+3yZrLBE0qPr/sNMVb7mUesdZb6lFd2VIs8siwhfeGXMS+nDI
 xePzsR43Fnn5Q5KIqqvcWUb+TTTqUDUff0wyAU8NBgCaIBIZK8h2ppS1jjnbms0I
 mr8Er2vb
 =Dfum
 -----END PGP SIGNATURE-----

Merge tag 'v2021.01-rc5' into next

Prepare v2021.01-rc5

Signed-off-by: Tom Rini <trini@konsulko.com>
2021-01-05 16:20:26 -05:00
Simon Glass 10cbd3b7da dtoc: Add an 'all' command
With upcoming changes, dtoc will output several files for different
of-platdata components.

Add a way to output all ava!ilable files at once ('all'), to the
appropriate directories, without needing to specify each one invidually.

This puts the commands in alphabetical order, so update the tests
accordingly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2021-01-05 12:26:35 -07:00
Philipp Tomsich ddc44c2c92 patman: Add project-default for 'gcc'
Add defaults for FSF/GNU projects, such as gcc, that provide sensible
settings for those projects.

Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-12-22 20:39:25 -07:00
Philipp Tomsich b3aff15ee4 patman: Add --no-signoff to suppress adding signoffs
To enable use of patman with FSF/GNU projects, such as GCC or
Binutils, no Signed-off-by may be added.  This adds a command
line flag '--no-signoff' to suppress adding signoffs in patman
when processing commits.

Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Reviewed-by: Simon Glass <sjg@chromium.org>
Fix patman testBranch() test:
Signed-off-by: Simon Glass <sjg@chromium.org>
2020-12-22 20:38:58 -07:00
Simon Glass b7bbd553de checkpatch: Add warnings for unexpected struct names
As a way of keeping the driver declarations more consistent, add a warning
if the struct used does not end with _priv or _plat.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-12-13 16:51:09 -07:00
Simon Glass 479dd30f4a patman: Drop tools.ToChar() and ToChars()
This is useful anymore, since we always want to call chr() in Python 3.
Drop it and adjust callers to use chr().

Also drop ToChars() which is no-longer used.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-12-13 07:58:17 -07:00
Simon Glass 6a4ccad8e0 patman: Drop tools.ToByte()
This is not needed in Python 3. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-12-13 07:58:17 -07:00
Simon Glass fc0056e8d5 patman: Drop unicode helper functions
We don't need these now that everything uses Python 3. Remove them and
the extra code in GetBytes() and ToBytes() too.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-12-13 07:58:17 -07:00
Simon Glass f31e83d6cf binman: Handle tool paths containing '~' correctly
At present if CROSS_COMPILE contains a tilde, such as
~/.buildman-toolchains/gcc-7.3.0-nolibc/i386-linux/bin/i386-linux-gcc
then binman gives a confusing error:

   binman: Error 255 running '~/..buildman-toolchains/gcc-7.3.0- ...

Fix this by expanding it out before running the tool.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-30 13:14:43 -07:00
Simon Glass fcbec650e6 patman: Add a Series-patchwork-url option
Add a commit tag to allow the Patchwork URL to be specified in a commit.
This can be handy for when you submit code to multiple projects but don't
want to use the -p option.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass a55be354c0 patman: Add a setting for the Patchwork URL
Add an argument to allow specifying the the patchwork URL. This also adds
this feature to the settings file, either globally, or on a per-project
basis.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass 7cbf02e94d patman: Allow specifying the patchwork URL
Add a new argument to allow the URL of the patchwork server to be
speciified. For now this is hard-coded in the main file, but future
patches will move it to the settings file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass 3145b63513 patman: Update defaults in subparsers
At present values from the settings file are only applied to the main
parser. With the new parser structure this means that some settings are
ignored.

Update the implementation to set defaults across the main parser and all
subparsers. Also fix up the comments, since ArgumentParser is being used
now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass 53336e6ca8 patman: Correct Change-Ids error message args
The arguments of this error are incorrectly formatted. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass 4600767d29 patman: Refactor how the default subcommand works
At present patman tries to assume a default subcommand of 'send', to
maintain backwards compatibility. However it does not cope with
arguments added to the default command, so for example 'patman -t'
does not work.

Update the logic to handle this. Also update the CC command to use 'send'
explicitly, since otherwise patman gets confused with the patch-filename
argument.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass dc4b2a9770 patman: Support listing comments from patchwork
While reviewing feedback it is helpful to see the review comments on the
command line to check that each has been addressed. Add an option to
support that.

Update the workflow documentation to describe the new features.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 6b3252e230 patman: Support parsing of review snippets
Add support for parsing the contents of a patchwork 'patch' web page
containing comments received from reviewers. This allows patman to show
these comments in a simple 'snippets' format.

A snippet is some quoted code plus some unquoted comments below it. Each
review is from a unique person/email and can produce multiple snippets,
one for each part of the code that attracts a comment.

Show the file and line-number info at the top of each snippet if
available.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 8f9ba3ab56 patman: Support updating a branch with review tags
It is tedious to add review tags into the local branch and errors can
sometimes be made. Add an option to create a new branch with the review
tags obtained from patchwork.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass dc6df972c9 patman: Support checking for review tags in patchwork
Before sending out a new version of a series for review, it is important
to add any review tags (e.g. Reviewed-by, Acked-by) collected by
patchwork. Otherwise people waste time reviewing the same patch
repeatedly, become frustrated and stop reviewing your patches.

To help with this, add a new 'status' subcommand that checks patchwork
for review tags, showing those which are not present in the local branch.

This allows users to see what new review tags have been received and then
add them.

Sample output:
   $ patman status
     1 Subject 1
       Reviewed-by: Joe Bloggs <joe@napierwallies.co.nz>
     2 Subject 2
       Tested-by: Lord Edmund Blackaddër <weasel@blackadder.org>
       Reviewed-by: Fred Bloggs <f.bloggs@napier.net>
     + Reviewed-by: Mary Bloggs <mary@napierwallies.co.nz>
   1 new response available in patchwork

The '+' indicates a new tag. Colours are used to make it easier to read.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass be051c0c77 patman: Detect missing upstream in CountCommitsToBranch
At present if we fail to find the upstream then the error output is piped
to wc, resulting in bogus results. Avoid the pipe and check the output
directly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass b3348522b7 patman: Improve handling of files
Sometimes warnings are associated with a file and sometimes with the
patch as a whole. Update the regular expression to handle both cases,
even in emacs mode. Also add support for detecting new files.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass a12ad7c940 patman: Allow showing a Commit as a string
Use the subject of the Commit object when printing it out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass b5e188131f patman: Don't ignore lines starting with hash
These lines can indicate a continuation of an error and should not be
ignored. Fix this.

Fixes: 666eb15e92 ("patman: Handle checkpatch output with notes and code")

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 9994baadc0 patman: Fix spelling of plural for warning
Tidy up the extra 's' when there is only a single warning. Fix the empty
print statement also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 8c17f8c5d2 patman: Convert testBasic() to use an interator
On balance it is easier to use an iterator here, particularly if we need
to insert lines due to new functionality. The only niggle is the need to
keep the previous iterator value around in one case.

Convert this test to use iter().

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 4af9987456 patman: Add some tests for warnings
Add tests that check that warnings are generated when expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 7457051e41 patman: Add a test for PatchStream tags
The current functional tests run most of patman. Add a smaller test that
just checks tag handling with the PatchStream class.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 47f62952cc patman: Drop unused signoff member
This is not used. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass dffa42c3ef patman: Convert 'Series-xxx' tag errors into warnings
If the Series-xxx tag is not recognised patman currently reports a fatal
error. This is inconvenient if a new feature is later added to patman that
an earlier version does not support.

Report a warning instead, to allow the user to take action if needed, but
still allow operation to proceed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 313ef5f897 patman: Attach warnings to individual patches
At present warnings are produced across the whole set of patches when
parsing them. It is more useful to associate each warning with the patch
(or commit) that generated it.

Attach warnings to the Commit object and move them out of PatchStream.
Also avoid generating duplicate warnings for the same commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass b5cc399053 patman: Move warning collection to a function
Add a new function in PatchStream to collect the warnings generated while
parsing the stream. This will allow us to adjust the logic, such as
dealing with per-commit warnings.

Two of the warnings are in fact internal errors, so change them to raise
and exception.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 1cb1c0fc8d patman: Fix up argument/return docs in patchstream
Add missing documentation and type information. Fix up some missing docs
on exceptions also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass e3a816b9f4 patman: Drop unused args in patchstream
Drop a few arguments that are not used in functions.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass dd147eda10 patman: Rename variables in patchstream
Some variables are too short or shadow other variables or types. Fix these
to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass d93720e138 patman: Rename functions in patchstream
Rename these functions to lower case as per PEP8.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass 5769904082 patman: Fix constant style in patchstream
These constants should use upper case. Update them to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass d06e55a7c6 patman: Fix indenting in patchstream
Update the indenting to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass f9e428489b patman: Allow linking a series with patchwork
Add a new Series-links tag to tell patman how to find the series in
patchwork. Each item is the series ID optionally preceded by the series
version that the link refers to. An empty version indicates this is the
latest series.

For example:

   Series-links: 209816 1:203302

Documentation is added in a later patch.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass fca99117a5 patman: Fix remaining pylint3 warnings in func_test
This fixes all but the ones about too many variables/statements.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00