Commit Graph

249 Commits

Author SHA1 Message Date
Simon Glass
fd70986a62 patman: Add a test that uses gitpython
It is convenient to use gitpython to create a real git repo for testing
patman's operation. Add a test for this. So far it just checks that patman
produces the right number of patches for a branch.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass
7d5b04e8a5 patman: Move main code out to a control module
To make testing easier, move the code out from main into a separate
'control' module and split it into four parts: setup, preparing patches,
checking patches and emailing patches.

Add comments and fix a few code-style issues while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Simon Glass
0b3d24a779 patman: Use test_util to show test results
This handles skipped tests correctly, so use it instead of the existing
code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00
Tom Rini
7208396bbf Revert "Merge tag 'dm-pull-20jul20' of git://git.denx.de/u-boot-dm"
This reverts commit 5d3a21df66, reversing
changes made to 56d37f1c56.

Unfortunately this is causing CI failures:
https://travis-ci.org/github/trini/u-boot/jobs/711313649

Signed-off-by: Tom Rini <trini@konsulko.com>
2020-07-24 08:42:06 -04:00
Nicolas Boichat
28e0367fc8 patman: When no tracking branch is provided, tell the user
The user can either count the number of patches, or provide a
tracking branch.

Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Nicolas Boichat
754c05caf3 patman: Make sure sendemail.suppresscc is (un)set correctly
Setting sendemail.suppresscc to all or cccmd leads to --cc-cmd
parameter being ignored, and emails going either nowhere, or
just to the To: line maintainer.

Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Patrick Delaunay
3ce7f75f78 Add information for skipped commit options
The unsupported Commit-xxx option are silently skipped
and removed as 're_remove=Commit-\w*', this patch adds
warning message in this case to detect misspelled issue
for the 2 supported options:
  Commit-notes:
  Commit-changes:

For example: the final 's' is missing (Commit-note:)

NB: no issue for Series-xxx option as only the supported
    options are accepted (see valid_series in series.py)

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Patrick Delaunay
df1fa4b223 patman: Detect unexpected END
Detect unexpected 'END' line when a section is not detected.

This patch detect issue when tag name for section start is misspelled,
for example 'Commit-note:' for 'Commit-notes:'

  Commit-note:
  ....
  END

Then 'Commit-note:' is removed silently by re_remove = "Commit-\w*:"
but 'END' is kept in commit message.

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
13262c9362 binman: Detect when valid images are not produced
When external blobs are missing, show a message indicating that the images
are not functional.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
f9793a12c5 patman: Update errors and warnings to use stderr
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass
5f850fb9a6 binman: Allow external binaries to be missing
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.

U-Boot does with with ATF (ARM Trusted Firmware) today.

Add a new flag to binman to request this behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-20 11:37:47 -06:00
Simon Glass
0b9116e31a binman: Re-enable concurrent tests
With the change to absolute imports the concurrent tests feature
unfortunately broke. Fix it.

We cannot easy add a warning, since the output messes up tests which check
the output.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
92dee5fcc5 binman: Specify the toolpath when running test coverage
At present binman's test coverage runs without a toolpath set. This means
that the system tools will be used. That may not be correct if they are
out of date or missing and this can result in a reduction in test coverage
below 100%.

Provide the toolpath to binman in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
0fc2e632f5 patman: Add a -D option to enable debugging
Most users don't want to see traceback errors. Add an option to enable
them for debugging. Disable them by default.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
df3fc0757b patman: Support collecting response tags in Patchstream
Collect response tags such as 'Reviewed-by' while parsing the stream.
This allows us to see what tags are present.

Add a new 'Fixes' tag also, since this is now quite common.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
ef6629128c patman: Allow disabling 'bright' mode with Print output
At present all text is marked bright, which makes it stand out on the
terminal. Add a way to disable that, as is done with the Color class.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
57374b09ec patman: Add a 'test' subcommand
At present we use --test to indicate that tests should be run. It is
better to use a subcommand for list, like binman. Change it and adjust
the existing code to fit under a 'send' subcommand, the default.

Give this subcommand the same default arguments as the others.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
4806fa3006 patman: Allow different commands
At present patman only does one thing so does not have any comments. We
want to add a few more command, so create a sub-parser for the default
command ('send').

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
e676fab729 patman: Convert to ArgumentParser
Convert from OptionParser to ArgumentParser to match binman. With this we
can easily add sub-commands.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:47 -06:00
Simon Glass
d9dc99e327 patman: Allow skipping patches at the end
The -s option allows skipping patches at the top of the branch. Sometimes
there are commits at the bottom that need to be skipped. At present it is
necessary to count the number of commits and then use -c to tell patman
how many to process.

Add a -e option to easily skip a number of commits at the bottom of the
branch.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:46 -06:00
Simon Glass
e9799e0890 patman: Allow creating patches for another branch
Add a -b option to allow patches to be created from a branch other than
the current one.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:46 -06:00
Simon Glass
5d597584db patman: Add a test that uses gitpython
It is convenient to use gitpython to create a real git repo for testing
patman's operation. Add a test for this. So far it just checks that patman
produces the right number of patches for a branch.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:46 -06:00
Simon Glass
f365375975 patman: Move main code out to a control module
To make testing easier, move the code out from main into a separate
'control' module and split it into four parts: setup, preparing patches,
checking patches and emailing patches.

Add comments and fix a few code-style issues while we are here.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:46 -06:00
Simon Glass
2e9a0cdfa8 patman: Use test_util to show test results
This handles skipped tests correctly, so use it instead of the existing
code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-20 11:37:46 -06:00
Simon Glass
8af45b1f20 checkpatch: Don't warn about PREFER_IF in headers/DT files
This warning should only be displayed for C files. Fix it and update the
test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
5c430761e6 patman: Add tests for the rest of the checkpatch checks
Finish off the tests for our small collection of checkpatch checks.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
89fb8b75bc patman: Decode output from the '--show-types' option
Collect the 'checkpatch type' from each error, warning and check. Provide
this to patman and update the uclass test to use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
4148c20f53 patman: Add a test for the 'possible new uclass' check
It is quite likely that the number of U-Boot-specific tests in
checkpatch.pl will increase over time. We should have tests for these to
avoid undefined behaviour and bugs being introduced, which might cause
people to ignore the warnings.

Add a simple new class that can generate a patch with a single-line
addition in it. Use that to add a test for one of the checkpatch checks.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
40d9734cb1 patman: Rename test.py to test_checkpatch.py
These tests check checkpatch.pl operation and can server as our tests for
the U-Boot-specific updates to that script. Rename the file and update
comments to indicate this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
1b48782ad3 patman: Drop import of test_util in test_util
This module doesn't need to import itself. It causes problems on
very old Python 3 (e.g. 3.4.0). Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:22 -06:00
Simon Glass
dd3dac2f2c patman: Avoid importing gitutil in settings
Pass this module in so that settings does not need to import it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stefan Bosch <stefan_b@posteo.net>
2020-07-09 18:57:22 -06:00
Simon Glass
156e655372 patman: Pass in maintainer dirs to avoid and import
Adjust the get_maintainer module to accept a list of directories to search
for the script. This avoids needing to import gitutil.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stefan Bosch <stefan_b@posteo.net>
2020-07-09 18:57:22 -06:00
Simon Glass
38a9d3b31a patman: Use a dict in gitutil to avoid importing series
Only a few members of this class are used and only in a test. To avoid
importing the module, convert the test to use a dict.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stefan Bosch <stefan_b@posteo.net>
2020-07-09 18:57:22 -06:00
Simon Glass
ddd65b0156 patman: Avoid circular dependency between command and tools
This seems to cause problems in some cases. Split the dependency by
copying the code to command.

Reported-by: Stefan Bosch <stefan_b@posteo.net>
Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-09 18:57:21 -06:00
Simon Glass
676f98a870 patman: Drop unnecessary import in gitutil
The checkpatch module is not used, so drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stefan Bosch <stefan_b@posteo.net>
2020-07-09 18:57:21 -06:00
Simon Glass
96daa41aab patman: Complain if a checkpatch line is not understood
Rather than suffering in silence, output a warning if something about the
checkpatch output cannot be understood.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:11 -06:00
Simon Glass
37f3bb53b7 patman: Support warnings in the patch subject
Sometimes checkpatch outputs problems in the patch subject. Add support
for parsing this output and reporting it correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:11 -06:00
Simon Glass
666eb15e92 patman: Handle checkpatch output with notes and code
If checkpatch is configured to output code we should ignore it. Similarly,
notes should be ignored.

Update the logic to handle these situations.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:11 -06:00
Simon Glass
7175b085e1 patman: Don't try to process checkpatch lines twice
Once we have determined what the line refers to there is no point in
processing it further. Update the logic to continue to the next line in
these cases.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:10 -06:00
Simon Glass
7d5b5e8965 patman: Support emacs mode with checkpatch
If checkpatch is run in 'emacs' mode it shows the filename at the
start of each line. Add support for this so that the warnings and errors
are correctly detected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:10 -06:00
Simon Glass
870bd56799 patman: Fix 'warning' typo
If no warnings are detected due to checkpatch having unexpected options,
patman currently shows an error:

   TypeError: unsupported operand type(s) for +=: 'int' and 'property'

Fix this by initing the variable correctly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-06-11 20:52:10 -06:00
Sean Anderson
dc03ba48be patman: Modify functional tests for new behavior
This patch adds or modifies functional tests for the Cover-changes,
Commit-changes, and Series-process-log tags in order to account for new
behavior added in the previous few patches. The '(no changes since v1)'
case is not tested for, since that would need an additional commit to test
in addition to testing the existing code paths.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Sean Anderson
0411fff36f patman: Support multi-line changes in changelogs
This patch adds support to multi-line changes. That is, if one has a line
in a changelog like
- Do a thing but
  it spans multiple lines
Using Series-process-log sort would sort as if those lines were unrelated.
With this patch, any change line starting with whitespace will be
considered part of the change before it.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Sean Anderson
6949f70c6d patman: Add new tags for finer-grained changelog control
By default patman generates a combined changelog for the cover letter. This
may not always be desirable.

Many patches may have the same changes. These can be coalesced with
"Series-process-log: uniq", but this is imperfect. Similar changes like
"Move foo to patch 7" will not be merged with the similar "Move foo to this
patch from patch 6".

Changes may not make sense outside of the patch they are written for. For
example, a change line of "Add check for bar" does not make sense outside
of the context in which bar might be checked for. Some changes like "New"
or "Lint" may be repeated many times throughout different change logs, but
carry no useful information in a summary.

Lastly, I like to summarize the broad strokes of the changes I have made in
the cover letter, while documenting all the details in the appropriate
patches. I think this makes it easier to get a good feel for what has
changed, without making it difficult to wade through every change in the
whole series.

This patch adds two new tags to add changelog entries which only appear in
the cover letter, or only appear in the commit. Changes documented with
"Commit-changes" will only appear in the commit, and will not appear in the
cover letter. Changes documented with "Cover-changes" will not appear in
any commit, and will only appear in the cover letter.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Sean Anderson
b0436b9404 patman: Suppress empty changelog entries
Patman outputs a line for every edition of the series in every patch,
regardless of whether any changes were made. This can result in many
redundant lines in patch changelogs, especially when a patch did not exist
before a certain revision. For example, the existing behaviour could result
in a changelog of

Changes in v7: None
Changes in v6: None
Changes in v5:
- Make a change

Changes in v4: None

Changes in v3:
- New

Changes in v2: None

With this patch applied and with --no-empty-changes, the same patch would
look like

(no changes since v5)

Changes in v5:
- Make a change

Changes in v3:
- New

This is entirely aesthetic, but I think it reduces clutter, especially for
patches added later on in a series.

Signed-off-by: Sean Anderson <seanga2@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Bin Meng
14aa35ad55 patman: Add an option to create patches without binary contents
Some mailing lists have size limits and when we add binary contents
to our patches it's easy to exceed the size limits.

Git supports a command line option "--no-binary" to generate patches
without any binary contents. Add an option in patman to handle this.
Note with this option patches cannot be applied properly, but they
are still useful for code review.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Bin Meng
0fc01bf826 patman: Sort the command line options
Sort the existing command line options by:

- help comes first
- option starts with '-'
- option starts with '--'

Lower case followed by upper case letters, in alphabetical order.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-05-29 20:55:45 -06:00
Simon Glass
b4fa94959d patman: Tidy up sys.path changes
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.

We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.

The special paths for finding pylibfdt remain.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass
bf776679a7 patman: Move to absolute imports
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Simon Glass
16287933a8 binman: Move to absolute imports
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.

Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00