Commit Graph

202 Commits

Author SHA1 Message Date
Simon Glass
b1793a531e patman: Update cros_subprocess to use bytes
At present this function uses lists and strings. This does not work so
well with Python 3, and testing against '' does not work for a bytearray.
Update the code to fix these issues.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:43 -06:00
Simon Glass
11ae93eef4 binman: Run tests concurrently
At present the tests run one after the other using a single CPU. This is
not very efficient. Bring in the concurrencytest module and run the tests
concurrently, using one process for each CPU by default. A -P option
allows this to be overridden, which is necessary for code-coverage to
function correctly.

This requires fixing a few tests which are currently not fully
independent.

At some point we might consider doing this across all pytests in U-Boot.
There is a pytest version that supports specifying the number of processes
to use, but it did not work for me.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
008b030009 patman: Don't clear progress in tout unless it was used
At present calling Uninit() always called ClearProgress() which outputs
a \r character as well as spaces to remove any progress information on the
line. This can mess up the normal output of binman and other tools. Fix
this by outputing this only when progress information has actually been
previous written.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
1fda18205f tools: Set an initial value for indir
This variable is not documented or set up in the module. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
ec9e0f4712 patman: Handle unicode in _ProjectConfigParser tests
With Python 2.7.15rc1, ConfigParser.SafeConfigParser has unfortunately
started returning unicode, for unknown reasons. Adjust the code to handle
this by converting everything to unicode. We cannot convert things to
ASCII since email addresses may be encoded with UTF-8.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
0a98b28b06 binman: Support adding files
In some cases it is useful to add a group of files to the image and be
able to access them at run-time. Of course it is possible to generate
the binman config file with a set of blobs each with a filename. But for
convenience, add an entry type which can do this.

Add required support (for adding nodes and string properties) into the
state module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
04187a845c patman: Detect missing tools and report them
When tools are needed but not present, at present we just get an error
which can be confusing for the user. Try to be helpful by reporting the
tool as missing and suggesting a possible remedy.

Also update the Run() method to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
41d7535cba patman: Correct unit test failure
A recent rename of the function did not rename the test file. Fix this.

Fixes: 12308b128f (lib: fdtdec: Rename routine fdtdec_setup_memory_size())

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-08 12:49:36 +01:00
Simon Glass
4f5dea4543 patman: Show the current directory in GetInputFilename()
When this fails it is useful to see the current directory, since U-Boot's
build system will typically change into the output directory during the
build. Add this information to the error.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01 16:30:48 -06:00
Simon Glass
aeffc5e956 patman: Add a few more helpers to the tools library
Add functions to read and write a file, looking through a list of search
paths to find it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01 16:30:48 -06:00
Simon Glass
2b19321ef9 patman: Allow test commands to fall back to real ones
Tests use the 'test_result' feature to return a predetermined command
result for particular commands. The avoids needing to have the real
command available just to run a test. It works by calling the function
provided by the test, to get the value.

However sometimes the test does need to run the real command. Allow it to
fall back to do this when the function does not return a result.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01 16:30:48 -06:00
Adam Sampson
b8a48fbca3 patman: Split alias entries on arbitrary whitespace
Previously the first three words in a git-mailrc alias entry could only
be separated by spaces. git-send-email and Mutt both allow arbitrary
whitespace here.

Signed-off-by: Adam Sampson <ats@offog.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-07-26 14:18:38 -06:00
Siva Durga Prasad Paladugu
12308b128f lib: fdtdec: Rename routine fdtdec_setup_memory_size()
This patch renames the routine fdtdec_setup_memory_size()
to fdtdec_setup_mem_size_base() as it now fills the
mem base as well along with size.

Signed-off-by: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2018-07-19 10:49:56 +02:00
Simon Glass
c3f9454103 binman: Move capture_sys_output() to test_util
This function is useful in various tests. Move it into the common test
utility module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-07-09 09:11:00 -06:00
Simon Glass
ff1fd6ccde binman: Move coverage logic into a new test_util file
At present only binman has the logic for determining Python test coverage
but this is useful for other tools also. Move it out into a separate file
so it can be used by other tools.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-07-09 09:11:00 -06:00
Simon Glass
a60aedfd31 patman: Support using a particular SMTP server
Some environments require providing the '--smtp-server' argument to
'git send-email'. Add support for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-06-23 08:03:43 -06:00
Chris Packham
fe6ef1e9ba patman: add test for SPDX license
Add a test to exercise the check for a valid SPDX license.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-23 08:03:43 -06:00
Chris Packham
4fb3502972 patman: add option for limiting the Cc list
Many mailing-lists consider a long Cc list a sign of spam and will
either drop the message or mark it for moderation. Because patman
automatically invokes get_maintainer.pl the Cc list can expand
unexpectedly. Allow the user to specify a limit for the Cc list.

This limit is applied after removing any known bouncing addresses. By
default no limit is applied.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-23 08:03:43 -06:00
Simon Glass
4225a4e4a9 patman: Fix unit tests for SPDX
The format of this line has changed. Update the patman test to suit.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Alex Kiernan <alex.kiernan@gmail.com>
2018-06-07 11:25:03 -08:00
Tom Rini
83d290c56f SPDX: Convert all of our single license tags to Linux Kernel style
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from.  So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry.  Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents.  There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-07 09:34:12 -04:00
Masahiro Yamada
94b13bbae9 host-tools: use python2 explicitly for shebang
All of these host tools are apparently written for Python2,
not Python3.

Use 'python2' in the shebang line according to PEP 394
(https://www.python.org/dev/peps/pep-0394/).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-28 12:27:30 -05:00
Stefan Brüns
d1ccaa4760 patman: Unquote output from get_maintainer.pl
get_maintainer.pl quotes names which it considers unsafe, i.e. anything
containing [^a-zA-Z0-9_ \-]. This confuses patman, it will duplicate
addresses which are also in Series-to/cc. Strip the quotes.

Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-21 10:01:02 -07:00
Simon Glass
6c328f2975 patman: Fix up tests to pass with newest checkpatch
The checkpatch tool was updated but the patman tests were not. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-11-22 18:05:38 -07:00
Simon Glass
ad8931400b patman: Fix error when the email blacklist is missing
This section of the settings file may be missing. Handle that gracefully
rather than emitting an error.

Also update patman to write this section when a new settings file is
created.

Fixes: e11aa602 (patman: add support for omitting bouncing addresses)

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Chris Packham <judge.pckham@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
2017-09-13 19:32:13 -06:00
Chris Packham
e11aa602ab patman: add support for omitting bouncing addresses
Add support for reading a list of bouncing addresses from a in-tree file
(doc/bounces) and from the ~/.patman config file. These addresses are
stripped from the Cc list.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com <mailto:philipp.tomsich@theobroma-systems.com>>
2017-09-11 21:43:58 -06:00
Simon Glass
6e87ae1c07 patman: Add a functional test
The existing test (patman --test) only covers basic checkpatch output.
We have had some problems with unicode processing and could use test
coverage for the various tags patman supports.

Add a new functional test which runs most of the patman flow on a few
test commits and checks that the results are correct.

See the documentation in the test for a description of what it does.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
a44f4fb72b patman: Rename 'list' variable in MakeCcFile()
This is not a good variable name in Python because 'list' is a type. It
shows up highlighted in some editors. Rename it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
1f487f85d2 patman: Add a maintainer test feature to MakeCcFile()
Allow the add_maintainers parameter to be a list of maintainers, thus
allowing us to simulate calling the script in tests without actually
needing it to work.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
2eb5fc13b3 patman: Add unicode to test patches
Add some unicode to the test patches to make sure that patman does the
right thing.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
db116cc8d0 patman: Don't return the series in FixPatches()
There is no need for this function to return the same object that was
passed in. Drop the return value.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
04f7870635 patman: Don't report unicode character
Unicode characters may appear in input patches so we should not warn about
them. Drop this warning.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
2df3a01974 patman: Rename 'str' variable in EmailPatches()
This is not a good variable name in Python because 'str' is a type. It
shows up highlighted in some editors. Rename it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
5c724dc440 patman: Don't convert input data to unicode
The communication filter reads data in blocks and converts each block to
unicode (if necessary) one at a time. In the unlikely event that a unicode
character in the input spans a block this will not work. We get an error
like:

UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1022-1023:
   unexpected end of data

There is no need to change the input to unicode, so the easiest fix is to
drop this feature.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Simon Glass
6f8abf765b patman: Adjust handling of unicode email address
Don't mess with the email address when outputting them. Just make sure
they are encoded with utf-8.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
Philipp Tomsich
21caa558ca patman: encode CC list to UTF-8
This change encodes the CC list to UTF-8 to avoid failures on
maintainer-addresses that include non-ASCII characters (observed on
Debian 7.11 with Python 2.7.3).

Without this, I get the following failure:
  Traceback (most recent call last):
    File "tools/patman/patman", line 159, in <module>
      options.add_maintainers)
    File "[snip]/u-boot/tools/patman/series.py", line 234, in MakeCcFile
      print(commit.patch, ', '.join(set(list)), file=fd)
  UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 81: ordinal not in range(128)
from Heiko's email address:
  [..., u'"Heiko St\xfcbner" <heiko@sntech.de>', ...]

While with this change added this encodes to:
  "=?UTF-8?q?Heiko=20St=C3=BCbner?= <heiko@sntech.de>"

Signed-off-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-06-08 20:21:59 -06:00
George McCollister
6db06f94e1 patman: Convert byte arrays to strings
os.read() returns a byte array in Python 3.5.2 and needs to be converted
into a string. Check if the returned value is an instance of bytes and
if it is decode it as a utf-8 string. If it is not a utf-8 encoded string
the decoding may fail with an exception.

Prior to this fix the comparisions check data == "" would fail when data
was b'' and would cause an infinite memory leaking loop. joins would
also fail with an exception below but due to the infinite loop it never
made it that far.

Signed-off-by: George McCollister <george.mccollister@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2017-04-13 11:43:49 -06:00
Chris Packham
f11a0af713 patman: Handle non-ascii characters in names
When gathering addresses for the Cc list patman would encounter a
UnicodeDecodeError due to non-ascii characters in the author name.
Address this by explicitly using utf-8 when building the Cc list.

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2017-02-08 06:12:16 -07:00
Simon Glass
8aa41363eb patman: Format checkpatch messages for IDE throwback
It is convenient to be able to deal with checkpatch warnings in the same
way as build warnings. Tools such as emacs and kate can quickly locate
the source file and line automatically.

To achieve this, adjust the format to match the C compiler, and output to
stderr.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-02-08 06:07:35 -07:00
Paul Burton
f5d44b9bae patman: Fix doctest StringIO import for python 3.x
In python 3.x StringIO is no longer a module, and the class can instead
be found in the io module. Adjust the code in the doctest input to
account for both.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Paul Burton
c9eac38a25 patman: Use items() to iterate over dictionaries
In python 3.x the iteritems() method has been removed from dictionaries,
and the items() method does effectively the same thing. On python 2.x
using items() is a little less efficient since it involves copying data,
but as speed isn't a concern in this code switch to using items() anyway
for simplicity.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Paul Burton
2ce7b21e6c patman: Import 'configparser' lower case to be python 3.x safe
In python 3.x module names used in import statements are case sensitive,
and the configparser module is named in all lower-case. Import it as such
in order to avoid errors when running with python 3.x.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Paul Burton
ac3fde9394 patman: Make exception handling python 3.x safe
Syntax for exception handling is a little more strict in python 3.x.
Convert all uses to a form accepted by both python 2.x & python 3.x.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Paul Burton
a920a17b2f patman: Make print statements python 3.x safe
In python 3.x, print must be used as a function call. Convert all print
statements to the function call style, importing from __future__ where
we print with no trailing newline or print to a file object.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Paul Burton
12e5476df3 patman: Replace tabs with spaces
In preparation for running on python 3.x, which will refuse to run
scripts which mix tabs & spaces for indentation, replace 2 tab
characters present in series.py with spaces.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
8b4919ed29 patman: Flush output when there is no newline
Output which does not include a newline will not be displayed unless
flushed. Add a flush to ensure that it becomes visible.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
0faf6144fd patman: Add a library to handle logging and progress
When tools want to display information of varying levels of importance, it
helps to provide the user with control over the verbosity of these messages.
Progress messages work best if they are displayed and then removed from the
display when no-longer relevant.

Add a new tout library (terminal out) to handle these tasks.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-18 21:04:39 -06:00
Simon Glass
1f1864b408 patman: Add a tools library for using temporary files
For tools which want to use input files and temporary output, it is useful
to have the handling of these dealt with in one place. Add a new library
which allows input files to be read, and output files to be written, all
based on a common directory structure.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-18 21:04:38 -06:00
Simon Glass
785f1548a9 patman: Adjust command.Output() to raise an error by default
It is more useful to have this method raise an error when something goes
wrong. Make this the default and adjust the few callers that don't want to
use it this way.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-18 21:04:38 -06:00
Bin Meng
94fbd3e37d tools: patman: Handle missing 'END' in non-last commit of a series
The following python error:

Traceback (most recent call last):
  File "./tools/patman/patman", line 144, in <module>
    series = patchstream.FixPatches(series, args)
  File "./tools/patman/patchstream.py", line 477, in FixPatches
    commit = series.commits[count]
IndexError: list index out of range

is seen when:

- 'END' is missing in those tags
- those tags are put in the last part in a commit message
- the commit is not the last commit of the series

Add testing logic to see if a new commit starts.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-11 14:06:44 -06:00
Bin Meng
57b6b190a8 tools: patman: Handle missing blank line for 'Series-changes'
'Series-changes' uses blank line to indicate its end. If that is
missing, series internal state variable 'in_change' may be wrong.
Correct its state.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-11 14:06:44 -06:00
Bin Meng
0d57718775 tools: patman: Generate cover letter correctly when 'END' is missing
If 'END' is missing in a 'Cover-letter' section, and that section
happens to show up at the very end of the commit message, and the
commit is the last commit of the series, patman fails to generate
cover letter for us. Handle this in CloseCommit of patchstream.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-11 14:06:44 -06:00
Bin Meng
13b98d95ba tools: patman: Handle tag sections without an 'END'
'Cover-letter', 'Series-notes' and 'Commit-notes' tags require an
'END' to be put at the end of its section. If we forget to put an
'END' in those sections, and these sections are followed by another
patman tag, patman generates incorrect patches. This adds codes to
handle such scenario.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-11 14:06:44 -06:00
Bin Meng
e7df218c3b tools: patman: Use cover_match for 'Cover-letter'
Like other patman tags, use a new variable cover_match to indicate
a match for 'Cover-letter'.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-11 14:06:44 -06:00
Simon Glass
d4c8572b71 buildman: Allow branch names which conflict with directories
At present if you try to use buildman with the branch 'test' it will
complain that it is unsure whether you mean the branch or the directory.
This is a feature of the 'git log' command that buildman uses. Fix it
by resolving the ambiguity.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-17 21:27:39 -06:00
Simon Glass
2bdeade0ea buildman: patman: Fix -H when installed as a symlink
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Simon Glass
9ad96982e8 patman: Add a missing space in GetMetaDataForList()
Fix this nit to keep the code consistent.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Albert ARIBAUD
070b781b2b patman: fix series-notes handling for buildman
A patman series with a 'Series-notes' section causes
buildman to crash with:

    self.series.notes += self.section
    TypeError: cannot concatenate 'str' and 'list' objects

Fix by initializing series.notes as a one-element array
rather than a scalar.

Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
2016-02-24 20:06:19 -08:00
Bin Meng
a187559e3d Use correct spelling of "U-Boot"
Correct spelling of "U-Boot" shall be used in all written text
(documentation, comments in source files etc.).

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Minkyu Kang <mk7.kang@samsung.com>
2016-02-06 12:00:59 +01:00
Mateusz Kulikowski
27067a46c5 patman: Add --thread option
Add option to create threaded series of patches.
With it, it will be possible to create patch threads like this:
[PATCH 0/10] Add support for time travel
  [PATCH 1/10] Add Flux Capacitor driver
  [PATCH 2/10] Add Mr. Fusion driver
  (...)

Internally it will call git send-email with --thread option

Signed-off-by: Mateusz Kulikowski <mateusz.kulikowski@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-20 19:06:23 -07:00
Masahiro Yamada
8d3595a42b Revert "patman: use -D option for git format-patch"
This reverts commit 19b4a33698.

Since that commit, patman generates useless patches for file removal;
"git format -D" prints only the header but not the diff when deleting
files, and "git am" always refuses such patches.

The following is the quotation from "man git-format-patch":

  -D, --irreversible-delete
    Omit the preimage for deletes, i.e. print only the header but
    not the diff between the preimage and /dev/null. The resulting
    patch is not meant to be applied with patch nor git apply; this
    is solely for people who want to just concentrate on reviewing
    the text after the change. In addition, the output obviously
    lack enough information to apply such a patch in reverse, even
    manually, hence the name of the option.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-09-09 07:48:03 -06:00
Simon Glass
d9917b0b6c buildman: Correct 'Series-cover-cc' detection logic
This requires 'Series-cover_cc' at present which is incorrect. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2015-08-31 07:57:26 -06:00
Masahiro Yamada
19b4a33698 patman: use -D option for git format-patch
This allows Patman to generate smaller patches for file removal.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Simon Glass <sjg@chromium.org>
Acked-by: Heiko Schocher <hs@denx.de>
2015-08-26 07:54:13 -07:00
Simon Glass
9649e152cb patman: Don't run patman when it is imported as a module
Commit 488d19c (patman: add distutils based installer) has the side effect
of making patman run twice with each invocation. Fix this by checking for
'main program' invocation in patman.py. This is good practice in any case.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Chris Packham <judge.packham@gmail.com>
2015-07-31 07:31:12 -06:00
Chris Packham
488d19cbca patman: add distutils based installer
To make it easier to use patman on other projects add a distutils style
installer. Now patman can be installed with

  cd u-boot/tools/patman && python setup.py install

There are also the usual distutils options for creating source/binary
distributions of patman.

Tested-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-07-28 10:36:25 -06:00
Wu, Josh
3871cd858f patman: check git format.subjectprefix setting when generate patches prefix
For the local project, we may specified format.subjectprefix setting.
Then the patch will be formated as [Project_prefix][PATCH].
But patman will not check this setting. It will remove the
format.subjectprefix.

So This patch will let patman check this setting and add it as a
project prefix.

Signed-off-by: Josh Wu <josh.wu@atmel.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
2015-04-23 12:35:50 -06:00
Wu, Josh
35ce2dc4d1 patman: cover letter shows like 00/xx if more than 10 patches
Make cover letter shows like 0/x, 00/xx and 000/xxx etc.

Signed-off-by: Josh Wu <josh.wu@atmel.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00
Simon Glass
8895b3e16c patman: Read in the git-mailrc alias file
We should read this file to obtain a set of aliases. This reduces the need
to create them in the ~/.patman file.

This feature did exist in some version of patman, and is mentioned in the
help but it did not find its way upstream.

Reported-by: Graeme Russ <gruss@tss-engineering.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-02-15 14:34:06 -07:00
Simon Glass
71edbe5cda buildman: Fix incorrect arguemnt in GetUpstream()
This causes an error when trying to build a local branch which has a local
branch as its upstream.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2015-02-15 14:34:06 -07:00
Simon Glass
3d4de98696 patman: Explain how to make doc/git-mailrc work
Add an explanation for how to set up git so that patman can find the alias
file. Fix up the get_maintainers message too.

Reported-by: Scott Wood <scottwood@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-01-30 15:55:30 -07:00
Scott Wood
4b89b8135f patman: Check commit_match before stripping leading whitespace
True commit lines start at column zero.  Anything that is indented
is part of the commit message instead.  I noticed this by trying to
run buildman with commit e3a4facdfc
as master, which contained a reference to a Linux commit inside
the commit message.  ProcessLine saw that as a genuite commit
line, and thus buildman tried to build it, and died with an
exception because that SHA is not present in the U-Boot tree.

Signed-off-by: Scott Wood <scottwood@freescale.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-01-30 15:55:26 -07:00
Peter Tyser
2181830f11 patman: Make dry-run output match real functionality
When run with the --dry-run argument patman prints out information
showing what it would do.  This information currently doesn't line up
with what patman/git send-email really do.  Some basic examples:
- If an email address is addressed via "Series-cc" and "Patch-cc" patman
  shows that email address would be CC-ed two times.
- If an email address is addressed via "Series-to" and "Patch-cc" patman
  shows that email address would be sent TO and CC-ed.
- If an email address is addressed from a combination of tag aliases,
  get_maintainer.pl output, "Series-cc", "Patch-cc", etc patman shows
  that the email address would be CC-ed multiple times.

Patman currently does try to send duplicate emails like the --dry-run
output shows, but "git send-email" intelligently removes duplicate
addresses so this patch shouldn't change the non-dry-run functionality.

Change patman's output and email addressing to line up with the
"git send-email" logic.  This trims down patman's dry-run output and
prevents confusion about what patman will do when emails are actually
sent.

Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
2015-01-30 15:52:29 -07:00
Simon Glass
5abab20dfb buildman: Allow specifying a range of commits to build
Adjust the -b flag to permit a range expression as well as a branch.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Tested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2015-01-14 21:16:53 -08:00
Simon Glass
2a9e2c6a09 buildman: Try to guess the upstream commit
Buildman normally obtains the upstream commit by asking git. Provided that
the branch was created with 'git checkout -b <branch> <some_upstream>' then
this normally works.

When there is no upstream, we can try to guess one, by looking up through
the commits until we find a branch. Add a function to try this and print
a warning if buildman ends up relying on it.

Also update the documentation to match.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
2015-01-14 21:16:52 -08:00
Simon Glass
0b5b409acc patman: Use the full commit hash for 'git checkout'
Even with the initial 8 characeters of the hash we will sometimes get a
collision. Use the full hash.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-11-26 20:25:39 -07:00
Simon Glass
983a2749e2 patman: Add a -m option to avoid copying the maintainers
The get_maintainers script is a useful default, but sometimes is copies
too many people, or takes a long time to run.

Add an option to disable it and update the README.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-21 12:03:07 -06:00
Vadim Bendebury
1f7278851e patman: make run results better visible
For an occasional user of patman some failures are not obvious: for
instance when checkpatch reports warnings, the dry run still reports
that the email would be sent. If it is not dry run, the warnings are
shown on the screen, but it is not clear that the email was not sent.

Add some code to report failure to send email explicitly.

Tested by running the script on a patch with style violations,
observed error messages in the script output.

Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:31 -06:00
Simon Glass
950a23133d buildman: Ignore conflicting tags
Tags like Series-version are normally expected to appear once, and with a
unique value. But buildman doesn't actually look at these tags. So ignore
conflicts.

This allows bulidman to build a branch containing multiple patman series.

Reported-by: Steve Rae <srae@broadcom.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:31 -06:00
Simon Glass
891b7a0761 patman: Start with a clean series when needed
For reasons that are not well-understood, GetMetaDataForList() can end up
adding to an existing series even when it appears that it should be
starting a new one.

Change from using a default constructor parameter to an explicit one, to
work around this problem.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:30 -06:00
Simon Glass
82012dd284 patman: Provide a way to intercept commands for testing
Add a test point for the command module. This allows tests to emulate
the execution of commands. This provides more control (since we can make
the fake 'commands' do whatever we like), makes it faster to write tests
since we don't need to set up as much environment, and speeds up test
execution.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
ddaf5c8f30 patman: RunPipe() should not pipe stdout/stderr unless asked
RunPipe() currently pipes the output of stdout and stderr to a pty, but
this is not the intended behaviour. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:24 -06:00
Simon Glass
3c6c0f81bf patman: Add a way of recording terminal output for testing
When running unit tests we don't want output to go to the terminal.
Provide a way of collecting it so that it can be examined by test code
later.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:22 -06:00
Simon Glass
0b703dbcee patman: Fix detection of git version
A missing 'global' declaration means that this feature does not currently
work. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
d0c5719d92 patman: Avoid changing the order of tags
patman collects tags that it sees in the commit and places them nicely
sorted at the end of the patch. However, this is not really necessary and
in fact is apparently not desirable.

Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-09-05 13:40:42 -06:00
Simon Glass
9447a6b2f6 patman: Use --no-pager' to stop git from forking a pager
In a headless environment the pager can apparently hang. We don't want a
pager anyway so let's request that none be used.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
7428dc14b0 patman: Remove the -a option
It seems that this is no longer needed, since checkpatch.pl will catch
whitespace problems in patches. Also the option is not widely used, so
it seems safe to just remove it.

Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
e752edcb6b patman: Correct unit tests to run correctly
It seems that doctest behaves differently now, and some of the unit tests
do not run. Adjust the tests to work correctly.

 ./tools/patman/patman --test
<unittest.result.TestResult run=10 errors=0 failures=0>

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:41 -06:00
Simon Glass
6ba5737ff0 patman: Fix indentation in terminal.py
This code came from a different project with 2-character indentation. Fix
it for U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:41 -06:00
Simon Glass
042a732cf5 patman: Support the 'reverse' option for 'git log'
This option is currently not supported, but needs to be, for buildman to
operate as expected.

Reported-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-28 07:56:19 -07:00
Masahiro Yamada
e0a4d06af2 patman: refactor help message
"patman [options]" is displayed by default.

Append the rest of help messages to parser.usage
instead of replacing it.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-22 10:41:50 -04:00
Masahiro Yamada
31e2141d5a tools, scripts: refactor error-out statements of Python scripts
In Python, sys.exit() function can also take an object other
than an integer.

If an integer is given to the argument, Python exits with the return
code of it.  If a non-integer argument is given, Python outputs it
to stderr and exits with the return code of 1.

That means,

    print >> sys.stderr, "Blah Blah"
    sys.exit(1)

is equivalent to

    sys.exit("Blah Blah")

The latter is a useful shorthand.

Note:
Some error messages in Buildman and Patman were output to stdout.
But they should go to stderr.  They are also fixed by this commit.
This is a nice side effect.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-21 12:01:11 -04:00
Simon Glass
e49f14af13 patman: Only use git's --no-decorate when available
Older versions of git (e.g. Ubuntu 10.04) do not support this flag. By
default they do not decorate. So only enable this flag when supported.

Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Simon Glass
cda2a61152 patman: Move the 'git log' command into a function
Move the code that builds a 'git log' command into a function so we can more
easily adjust it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Masahiro Yamada
ee860c60d2 patman: make "No recipient" checking more tolerant
If Series-to tag is missing, Patman exits with a message
"No recipient".

This is just annoying for those who had already added
sendemail.to configuration.

I guess many developers have

  [sendemail]
          to = u-boot@lists.denx.de

in their .git/config because the 'To: u-boot@lists.denx.de' field
should always be added when sending patches.

That seems more reasonable rather than adding
'Series-to: u-boot@lists.denx.de' to every patch series.

Patman should exit only when both Series-to tag and sendemail.to
configuration are mising.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-07-28 04:57:24 +01:00
Simon Glass
4251978afc patman: Only apply patches when we know the original HEAD
When patman applies the patches it checks out a new branch, uses 'git am'
to apply the patches one by one, and then tries to go back to the old
branch. If you try this when the branch is 'undefined', this doesn't work
as patman cannot restore the correct branch after applying the patches.
It seems that 'undefined' is created by git and is persistent after it is
created, so that you can end up on quite an old branch.

Add a check for the 'undefined' branch to avoid this.

Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-07-07 17:22:03 -06:00
Simon Glass
6be6b6bcba patman: Suppress duplicate signoffs only for real patches
There is an unfortunate bug in the signoff suppression logic. The first
pass is performed with 'git log', and all signoffs are added to the
supression set, such that the second time (when processing the real
patches) we always suppress the signoffs.

Correct this by only suppressing signoffs in the second pass.

Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Michal Simek <monstr@monstr.eu>
Tested-by: Andreas Bießmann <andreas.devel@googlemail.com>
2014-05-16 11:42:29 -07:00
Simon Glass
102061bd8b patman: Avoid duplicate sign-offs
Keep track of all Signed-off-by tags in a commit and silently suppress any
duplicates.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-05-09 14:50:45 -06:00
Simon Glass
757f64a89b patman: Deal with 'git apply' failures correctly
This sort of failure is rare, but the code to deal with it is wrong.
Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-05-09 14:50:39 -06:00
Simon Glass
659c89da8e patman: Use Patch-cc: instead of Cc:
Add a new Patch-cc: tag which performs the service now provided by
the Cc: tag. The Cc: tag is interpreted by git send-email but
ignored by patman.

So now:

  Cc: patman does nothing. (git send-email can cc patches)
  Patch-cc: patman Cc's patch and removes this tag from the patch

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-03-22 14:47:30 -06:00
Albert ARIBAUD
5c8fdd91dc patman: add Commit-notes tag and section
Sometimes a commit should have notes enclosed with it rather
than withing the cover letter -- possibly even because there
is no cover letter. Add a 'Commit-notes' tag, similar to the
'Series-notes' one; lines between this tag and the next END
line are inserted in the patch right after the '---' commit
delimiter.

Change-Id: I01e99ae125607dc6dec08f3be8a5a0b37f0a483d
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Signed-off-by: Simon Glass <sjg@chromium.org>
(Updated README)
2013-11-21 13:35:51 -07:00
Wolfgang Denk
1a4596601f Add GPL-2.0+ SPDX-License-Identifier to source files
Signed-off-by: Wolfgang Denk <wd@denx.de>
[trini: Fixup common/cmd_io.c]
Signed-off-by: Tom Rini <trini@ti.com>
2013-07-24 09:44:38 -04:00