git grep/sed to standardize "/* SPDX-License-Identifier: <license>"

git grep/sed to standardize "/* SPDX-License-Identifier: <license>"

[Joe Perches]

Almost all source files in the kernel use a standardized SPDX header
at line 1 with a comment /* initiator and terminator */:

/* SPDX-License-Identifier: <license> */

$ git grep -PHn '^/\* SPDX-License-Identifier:.*\*/\s*$' | \
  wc -l
17847

$ git grep -PHn '^/\* SPDX-License-Identifier:.*\*/\s*$' | \
  grep ":1:" | cut -f1 -d":" | grep -oP '\.\w+$' | \
  sort | uniq -c | sort -rn
  16769 .h
    972 .S
     87 .c
      6 .lds
      3 .l
      2 .y
      2 .py
      2 .dtsi
      1 .sh
      1 .dts
      1 .cpp
      1 .bc

But about 2% of the files do not use a use comment termination at
line 1 and use either:

/* SPDX-License-Identifier: <license>
 * additional comment or blank

or

/* SPDX-License-Identifier: <license>
<blank line>

$ git grep -PHn '^/\* SPDX-License-Identifier:(?!.*\*/\s*$)' | \
  wc -l
407

$ git grep -PHn '^/\* SPDX-License-Identifier:(?!.*\*/\s*$)' | \
  grep '\:1:' | cut -f1 -d':' | grep -oP '\.\w+$' | \
  sort | uniq -c | sort -rn
    357 .h
     34 .S
     16 .c

Here's a trivial script to convert and standardize the
first and second lines of these 407 files to make it easier
to categorize and sort.

$ git grep -PHn '^/\* SPDX-License-Identifier:(?!.*\*/\s*$)' | \
  grep ':1:' | cut -f1 -d":" | \
  xargs sed -i -e '1s@[[:space:]]*$@ */@' -r -e '2s@^( \*|)@/*@'

Re: git grep/sed to standardize "/* SPDX-License-Identifier: <license>"

[Linus Torvalds]

On Tue, Oct 6, 2020 at 4:13 PM Joe Perches <joe@perches.com> wrote:
>
> Almost all source files in the kernel use a standardized SPDX header
> at line 1 with a comment /* initiator and terminator */:
>
> /* SPDX-License-Identifier: <license> */
>
> $ git grep -PHn '^/\* SPDX-License-Identifier:.*\*/\s*$' | \
>   wc -l
> 17847

That grep pattern makes zero sense.

Why would */ be special at all? It isn't.

  $ git grep SPDX-License-Identifier: | wc -l
  52418

and a *LOT* of those are shell scripts and use "#", or are C sources
and use "//" etc.

So your "standardization" is completely pointless. Anybody who expects
that pattern just doing something fundamentally wrong, because the
pattern you want to standardize around is simply not valid.

             Linus

Re: git grep/sed to standardize "/* SPDX-License-Identifier: <license>"

[Joe Perches]

On Sun, 2020-10-11 at 11:42 -0700, Linus Torvalds wrote:
> On Tue, Oct 6, 2020 at 4:13 PM Joe Perches <joe@perches.com> wrote:
> > Almost all source files in the kernel use a standardized SPDX header
> > at line 1 with a comment /* initiator and terminator */:
> > 
> > /* SPDX-License-Identifier: <license> */
> > 
> > $ git grep -PHn '^/\* SPDX-License-Identifier:.*\*/\s*$' | \
> >   wc -l
> > 17847
> 
> That grep pattern makes zero sense.
> 
> Why would */ be special at all? It isn't.
> 
>   $ git grep SPDX-License-Identifier: | wc -l
>   52418
> 
> and a *LOT* of those are shell scripts and use "#", or are C sources
> and use "//" etc.
> 
> So your "standardization" is completely pointless. Anybody who expects
> that pattern just doing something fundamentally wrong, because the
> pattern you want to standardize around is simply not valid.

It's just a trivial grep pattern to determine if the c90 style
SPDX-License-Identifier is in an individual single line comment.

Almost all are.

Re: git grep/sed to standardize "/* SPDX-License-Identifier: <license>"

[Joe Perches]

On Sun, 2020-10-11 at 11:47 -0700, Joe Perches wrote:
> On Sun, 2020-10-11 at 11:42 -0700, Linus Torvalds wrote:
> > On Tue, Oct 6, 2020 at 4:13 PM Joe Perches <joe@perches.com> wrote:
> > > Almost all source files in the kernel use a standardized SPDX header
> > > at line 1 with a comment /* initiator and terminator */:
> > > 
> > > /* SPDX-License-Identifier: <license> */
> > > 
> > > $ git grep -PHn '^/\* SPDX-License-Identifier:.*\*/\s*$' | \
> > >   wc -l
> > > 17847
> > 
> > That grep pattern makes zero sense.
> > 
> > Why would */ be special at all? It isn't.
> > 
> >   $ git grep SPDX-License-Identifier: | wc -l
> >   52418
> > 
> > and a *LOT* of those are shell scripts and use "#", or are C sources
> > and use "//" etc.
> > 
> > So your "standardization" is completely pointless. Anybody who expects
> > that pattern just doing something fundamentally wrong, because the
> > pattern you want to standardize around is simply not valid.

btw:

The script would merely change these c90 comments to use the style
mandated by/proposed in Documentation/process/license-rules.rst