sparse on scripts/kconfig/*.c

sparse on scripts/kconfig/*.c

[Randy Dunlap]

Hi,

I've been attempting to run sparse on the kconfig/ C files -- without success.

The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
and just running sparse on the C files has issues with not being able to
find header files.

Has anyone done this?  Any clues about how to do it?

thanks,
-- 
~Randy

Re: sparse on scripts/kconfig/*.c

[Christoph Hellwig]

On Tue, May 09, 2017 at 09:47:41AM -0700, Randy Dunlap wrote:
> Hi,
> 
> I've been attempting to run sparse on the kconfig/ C files -- without success.
> 
> The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
> and just running sparse on the C files has issues with not being able to
> find header files.
> 
> Has anyone done this?  Any clues about how to do it?

As a wild guess from using sparse on various userspace projects:

have you tried simply setting HOSTCC to cgcc?

Re: sparse on scripts/kconfig/*.c

[Randy Dunlap]

On 05/09/17 13:17, Christoph Hellwig wrote:
> On Tue, May 09, 2017 at 09:47:41AM -0700, Randy Dunlap wrote:
>> Hi,
>>
>> I've been attempting to run sparse on the kconfig/ C files -- without success.
>>
>> The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
>> and just running sparse on the C files has issues with not being able to
>> find header files.
>>
>> Has anyone done this?  Any clues about how to do it?
> 
> As a wild guess from using sparse on various userspace projects:
> 
> have you tried simply setting HOSTCC to cgcc?

I don't quite see what that has to do with running sparse ($CHECK, not $HOSTCC).

Anyway, I tried it and got the same message:

nconf.h:19:11: error: unable to open 'menu.h'


thanks.
-- 
~Randy

Re: sparse on scripts/kconfig/*.c

[Christoph Hellwig]

On Tue, May 09, 2017 at 05:27:01PM -0700, Randy Dunlap wrote:
> On 05/09/17 13:17, Christoph Hellwig wrote:
> > On Tue, May 09, 2017 at 09:47:41AM -0700, Randy Dunlap wrote:
> >> Hi,
> >>
> >> I've been attempting to run sparse on the kconfig/ C files -- without success.
> >>
> >> The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
> >> and just running sparse on the C files has issues with not being able to
> >> find header files.
> >>
> >> Has anyone done this?  Any clues about how to do it?
> > 
> > As a wild guess from using sparse on various userspace projects:
> > 
> > have you tried simply setting HOSTCC to cgcc?
> 
> I don't quite see what that has to do with running sparse ($CHECK, not $HOSTCC).

cgcc is a gcc wrappr that calls sparse.  I just trie quickly to patch
Makefile to run cgcc instead of gcc as HOSTCC an it seems to work:

  HOSTCC  scripts/basic/fixdep
scripts/basic/fixdep.c:117:5: warning: symbol 'insert_extra_deps' was not declared. Should it be static?
scripts/basic/fixdep.c:118:6: warning: symbol 'target' was not declared. Should it be static?
scripts/basic/fixdep.c:119:6: warning: symbol 'depfile' was not declared. Should it be static?
scripts/basic/fixdep.c:120:6: warning: symbol 'cmdline' was not declared. Should it be static?

But then I run into the known cgcc bug that it also calls sparse when
called for linking.  Which reminds me that I need to go back and fix
that.

Re: sparse on scripts/kconfig/*.c

[Randy Dunlap]

On 05/10/17 00:28, Christoph Hellwig wrote:
> On Tue, May 09, 2017 at 05:27:01PM -0700, Randy Dunlap wrote:
>> On 05/09/17 13:17, Christoph Hellwig wrote:
>>> On Tue, May 09, 2017 at 09:47:41AM -0700, Randy Dunlap wrote:
>>>> Hi,
>>>>
>>>> I've been attempting to run sparse on the kconfig/ C files -- without success.
>>>>
>>>> The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
>>>> and just running sparse on the C files has issues with not being able to
>>>> find header files.
>>>>
>>>> Has anyone done this?  Any clues about how to do it?
>>>
>>> As a wild guess from using sparse on various userspace projects:
>>>
>>> have you tried simply setting HOSTCC to cgcc?
>>
>> I don't quite see what that has to do with running sparse ($CHECK, not $HOSTCC).
> 
> cgcc is a gcc wrappr that calls sparse.  I just trie quickly to patch
> Makefile to run cgcc instead of gcc as HOSTCC an it seems to work:
> 
>   HOSTCC  scripts/basic/fixdep

Agreed, that does it.

> scripts/basic/fixdep.c:117:5: warning: symbol 'insert_extra_deps' was not declared. Should it be static?
> scripts/basic/fixdep.c:118:6: warning: symbol 'target' was not declared. Should it be static?
> scripts/basic/fixdep.c:119:6: warning: symbol 'depfile' was not declared. Should it be static?
> scripts/basic/fixdep.c:120:6: warning: symbol 'cmdline' was not declared. Should it be static?
> 
> But then I run into the known cgcc bug that it also calls sparse when
> called for linking.  Which reminds me that I need to go back and fix
> that.
> 

Thanks to both you and Dan for your help.

-- 
~Randy

Re: sparse on scripts/kconfig/*.c

[Dan Carpenter]

On Tue, May 09, 2017 at 09:47:41AM -0700, Randy Dunlap wrote:
> Hi,
> 
> I've been attempting to run sparse on the kconfig/ C files -- without success.
> 
> The kbuild files don't try to support CHECK in scripts/kconfig/ AFAICT,
> and just running sparse on the C files has issues with not being able to
> find header files.
> 
> Has anyone done this?  Any clues about how to do it?
> 

Smatch has pretty much only been used on kernel code, and it really
shows.  For userspace code, you'd want to make a list of the no return
functions and some other customizations.  Also the kernel uses a quite
limitted sample of C.

Anyway, what Christophe said is right.  Change the Makefile:
-HOSTCC       = gcc
+HOSTCC       = ~/progs/smatch/devel/cgcc

make CHECK="~/progs/smatch/devel/smatch --file-output" scripts/kconfig/
find -name \*.c.smatch -exec cat \{\} \; > smatch_warns.txt

I just did that now, and there is nothing interesting there.  If there
were something there, then there is a good chance I would have seen it
already because I've been running smatch_scripts/kchecker on every .c
file.  Say you don't have your .config set up for sparc then it doesn't
really work great, but what it does it uses the x86 headers but if there
are any header files missing it just includes a promising header with
the same name or it continues without the .h file.  The output is mostly
garbage but I had five sparc patches merged in 2016 so I'm a top ten
sparc dev for the year.  :P

regards,
dan carpenter

Re: sparse on scripts/kconfig/*.c

[Dan Carpenter]

I have created some new tools to make this process easier.

1) First you still have to edit the Makefile:

-HOSTCC       = gcc
+HOSTCC       = ~/progs/smatch/devel/cgcc

2) Build the data with this command:

	~/progs/smatch/devel/smatch_scripts/build_generic_data.sh --target scripts/

The --target scripts/ is because we want to run smatch on the scripts/
directory.

3) Run smatch:

	~/progs/smatch/devel/smatch_scripts/test_generic.sh --target scripts/

I just wrote this code today so it might be a bit rough perhaps.  Let me
know if you have any issues.  It doesn't find anything like I said
before, but hopefully this can help other people who want to run Smatch
on user space code.

regards,
dan carpenter

Re: sparse on scripts/kconfig/*.c

[Randy Dunlap]

On 05/10/17 13:53, Dan Carpenter wrote:
> I have created some new tools to make this process easier.
> 
> 1) First you still have to edit the Makefile:
> 
> -HOSTCC       = gcc
> +HOSTCC       = ~/progs/smatch/devel/cgcc
> 
> 2) Build the data with this command:
> 
> 	~/progs/smatch/devel/smatch_scripts/build_generic_data.sh --target scripts/
> 
> The --target scripts/ is because we want to run smatch on the scripts/
> directory.
> 
> 3) Run smatch:
> 
> 	~/progs/smatch/devel/smatch_scripts/test_generic.sh --target scripts/
> 
> I just wrote this code today so it might be a bit rough perhaps.  Let me
> know if you have any issues.  It doesn't find anything like I said
> before, but hopefully this can help other people who want to run Smatch
> on user space code.

Thanks, Dan.

I get this when running build_generic_data.sh:

readline() on closed filehandle FILE at /home/rdunlap/projects/smatch/smatch_scripts/../smatch_data/db/fill_db_caller_info.pl line 33.

I see scripts/mod/* being built/checked ... but not scripts/kconfig/*.

Any ideas?


-- 
~Randy

Re: sparse on scripts/kconfig/*.c

[Dan Carpenter]

On Mon, May 15, 2017 at 11:00:22AM -0700, Randy Dunlap wrote:
> On 05/10/17 13:53, Dan Carpenter wrote:
> > I have created some new tools to make this process easier.
> > 
> > 1) First you still have to edit the Makefile:
> > 
> > -HOSTCC       = gcc
> > +HOSTCC       = ~/progs/smatch/devel/cgcc
> > 
> > 2) Build the data with this command:
> > 
> > 	~/progs/smatch/devel/smatch_scripts/build_generic_data.sh --target scripts/
> > 
> > The --target scripts/ is because we want to run smatch on the scripts/
> > directory.
> > 
> > 3) Run smatch:
> > 
> > 	~/progs/smatch/devel/smatch_scripts/test_generic.sh --target scripts/
> > 
> > I just wrote this code today so it might be a bit rough perhaps.  Let me
> > know if you have any issues.  It doesn't find anything like I said
> > before, but hopefully this can help other people who want to run Smatch
> > on user space code.
> 
> Thanks, Dan.
> 
> I get this when running build_generic_data.sh:
> 
> readline() on closed filehandle FILE at /home/rdunlap/projects/smatch/smatch_scripts/../smatch_data/db/fill_db_caller_info.pl line 33.
> 

Thanks, I will push a fix for this later today.

> I see scripts/mod/* being built/checked ... but not scripts/kconfig/*.
> 
> Any ideas?
> 

Hm...  It turns out we need to do a `make mrproper` between runs.

make mrproper ; make allmodconfig ; ~/progs/smatch/release/smatch_scripts/build_generic_data.sh --target scripts/
make mrproper ; make allmodconfig ; ~/progs/smatch/release/smatch_scripts/test_generic.sh --target scripts/

When I remove the grep out the autogenerated code this is what is left
grep -v .lex.c smatch_warns.txt | grep -v .tab.c

scripts/kconfig/expr.c:1027 expr_compare_type() info: ignoring unreachable code.
scripts/kconfig/symbol.c:1152 sym_check_print_recursive() error: we previously assumed 'prop' could be null (see line 1144)
scripts/kconfig/symbol.c:1153 sym_check_print_recursive() warn: curly braces intended?
scripts/kconfig/symbol.c:1155 sym_check_print_recursive() warn: inconsistent indenting
scripts/kconfig/menu.c:662 get_symbol_str() error: we previously assumed 'sym' could be null (see line 649)
scripts/mod/modpost.c:460 parse_elf() info: ignoring unreachable code.
scripts/mod/modpost.c:504 parse_elf() info: ignoring unreachable code.

regards,
dan carpenter

Re: sparse on scripts/kconfig/*.c

[Randy Dunlap]

On 05/16/17 02:43, Dan Carpenter wrote:
> On Mon, May 15, 2017 at 11:00:22AM -0700, Randy Dunlap wrote:
>> On 05/10/17 13:53, Dan Carpenter wrote:
>>> I have created some new tools to make this process easier.
>>>
>>> 1) First you still have to edit the Makefile:
>>>
>>> -HOSTCC       = gcc
>>> +HOSTCC       = ~/progs/smatch/devel/cgcc
>>>
>>> 2) Build the data with this command:
>>>
>>> 	~/progs/smatch/devel/smatch_scripts/build_generic_data.sh --target scripts/
>>>
>>> The --target scripts/ is because we want to run smatch on the scripts/
>>> directory.
>>>
>>> 3) Run smatch:
>>>
>>> 	~/progs/smatch/devel/smatch_scripts/test_generic.sh --target scripts/
>>>
>>> I just wrote this code today so it might be a bit rough perhaps.  Let me
>>> know if you have any issues.  It doesn't find anything like I said
>>> before, but hopefully this can help other people who want to run Smatch
>>> on user space code.
>>
>> Thanks, Dan.
>>
>> I get this when running build_generic_data.sh:
>>
>> readline() on closed filehandle FILE at /home/rdunlap/projects/smatch/smatch_scripts/../smatch_data/db/fill_db_caller_info.pl line 33.
>>
> 
> Thanks, I will push a fix for this later today.
> 
>> I see scripts/mod/* being built/checked ... but not scripts/kconfig/*.
>>
>> Any ideas?
>>
> 
> Hm...  It turns out we need to do a `make mrproper` between runs.
> 
> make mrproper ; make allmodconfig ; ~/progs/smatch/release/smatch_scripts/build_generic_data.sh --target scripts/
> make mrproper ; make allmodconfig ; ~/progs/smatch/release/smatch_scripts/test_generic.sh --target scripts/
> 
> When I remove the grep out the autogenerated code this is what is left
> grep -v .lex.c smatch_warns.txt | grep -v .tab.c
> 
> scripts/kconfig/expr.c:1027 expr_compare_type() info: ignoring unreachable code.
> scripts/kconfig/symbol.c:1152 sym_check_print_recursive() error: we previously assumed 'prop' could be null (see line 1144)
> scripts/kconfig/symbol.c:1153 sym_check_print_recursive() warn: curly braces intended?
> scripts/kconfig/symbol.c:1155 sym_check_print_recursive() warn: inconsistent indenting
> scripts/kconfig/menu.c:662 get_symbol_str() error: we previously assumed 'sym' could be null (see line 649)
> scripts/mod/modpost.c:460 parse_elf() info: ignoring unreachable code.
> scripts/mod/modpost.c:504 parse_elf() info: ignoring unreachable code.


OK, I don't need to do it then. However, there are several other config
programs in scripts/kconfig/ that are not being checked by this.


I did a 'git pull' for updates. On the rebuild, I get this error:

macro_table.c:30:31: fatal error: cwchash/hashtable.h: No such file or directory
 #include "cwchash/hashtable.h"

My problem?

thanks for your help.
-- 
~Randy

Re: sparse on scripts/kconfig/*.c

[Dan Carpenter]

On Wed, May 17, 2017 at 07:53:35PM -0700, Randy Dunlap wrote:
> 
> I did a 'git pull' for updates. On the rebuild, I get this error:
> 
> macro_table.c:30:31: fatal error: cwchash/hashtable.h: No such file or directory
>  #include "cwchash/hashtable.h"
> 
> My problem?
> 
> thanks for your help.

That's weird.  I haven't touched that code in years so I can't imagine
what's going on.  :/  My fresh clone works.

regards,
dan carpenter