From: "Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>
To: Jeremy Cline <jcline@redhat.com>
Cc: Thierry Reding <thierry.reding@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Thomas Gleixner <tglx@linutronix.de>,
Jonathan Corbet <corbet@lwn.net>, Joe Perches <joe@perches.com>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] scripts/spdxcheck.py: Always open files in binary mode
Date: Thu, 13 Dec 2018 08:37:08 +0100 [thread overview]
Message-ID: <20181213073708.nwj4nmnccuugvrhc@pengutronix.de> (raw)
In-Reply-To: <20181212181410.GC2352@laptop.jcline.org>
On Wed, Dec 12, 2018 at 01:14:10PM -0500, Jeremy Cline wrote:
> Hi,
>
> On Wed, Dec 12, 2018 at 02:12:09PM +0100, Thierry Reding wrote:
> > From: Thierry Reding <treding@nvidia.com>
> >
> > The spdxcheck script currently falls over when confronted with a binary
> > file (such as Documentation/logo.gif). To avoid that, always open files
> > in binary mode and decode line-by-line, ignoring encoding errors.
I suggest pointing out that the breakage only happens with python3 and
results in a UnicodeDecodeError.
> > One tricky case is when piping data into the script and reading it from
> > standard input. By default, standard input will be opened in text mode,
> > so we need to reopen it in binary mode.
> >
> > Signed-off-by: Thierry Reding <treding@nvidia.com>
> > ---
> > scripts/spdxcheck.py | 6 ++++--
> > 1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/scripts/spdxcheck.py b/scripts/spdxcheck.py
> > index 5056fb3b897d..e559c6294c39 100755
> > --- a/scripts/spdxcheck.py
> > +++ b/scripts/spdxcheck.py
> > @@ -168,6 +168,7 @@ class id_parser(object):
> > self.curline = 0
> > try:
> > for line in fd:
> > + line = line.decode(locale.getpreferredencoding(False), errors='ignore')
> > self.curline += 1
> > if self.curline > maxlines:
> > break
> > @@ -249,12 +250,13 @@ if __name__ == '__main__':
> >
> > try:
> > if len(args.path) and args.path[0] == '-':
> > - parser.parse_lines(sys.stdin, args.maxlines, '-')
> > + stdin = os.fdopen(sys.stdin.fileno(), 'rb')
> > + parser.parse_lines(stdin, args.maxlines, '-')
> > else:
> > if args.path:
> > for p in args.path:
> > if os.path.isfile(p):
> > - parser.parse_lines(open(p), args.maxlines, p)
> > + parser.parse_lines(open(p, 'rb'), args.maxlines, p)
> > elif os.path.isdir(p):
> > scan_git_subtree(repo.head.reference.commit.tree, p)
> > else:
> > --
> > 2.19.1
> >
>
> It might be worth noting this fixes commit 6f4d29df66ac
> ("scripts/spdxcheck.py: make python3 compliant") and also Cc this for
> stable since 6f4d29df66ac got backported to v4.19. While that commit
> did indeed make the script work with Python 3 for piping data, it broke
> Python 2 and made its way to stable.
It didn't break for me. Can you provide details about how and when it
broke for you?
Best regards
Uwe
--
Pengutronix e.K. | Uwe Kleine-König |
Industrial Linux Solutions | http://www.pengutronix.de/ |
next prev parent reply other threads:[~2018-12-13 7:37 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-12 13:12 Thierry Reding
2018-12-12 13:12 ` [PATCH 2/2] scripts: Add spdxcheck.py self test Thierry Reding
2018-12-12 18:14 ` [PATCH 1/2] scripts/spdxcheck.py: Always open files in binary mode Jeremy Cline
2018-12-13 2:51 ` Joe Perches
2018-12-13 16:14 ` Thierry Reding
2018-12-13 7:37 ` Uwe Kleine-König [this message]
2018-12-13 15:10 ` Jeremy Cline
2018-12-13 19:40 ` Uwe Kleine-König
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181213073708.nwj4nmnccuugvrhc@pengutronix.de \
--to=u.kleine-koenig@pengutronix.de \
--cc=akpm@linux-foundation.org \
--cc=corbet@lwn.net \
--cc=jcline@redhat.com \
--cc=joe@perches.com \
--cc=linux-kernel@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=thierry.reding@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®