Re: [Celinux-dev] Re: [Announce] Linux-tiny project revival

Re: [Celinux-dev] Re: [Announce] Linux-tiny project revival

[Scott Preece]

----- Original Message ----
From: Tim Bird <tim.bird@am.sony.com>

Rob Landley wrote:


Given that there are about 60,000 printks in the kernel (and that's
not counting wrappers like dprintk() and other locally-defined
functions and macros) it would be a huge task to examine the code
and differentiate strings that really start a new log message
(and thus should have an attached log level) and strings
that don't.
 -- Tim

---

So, if this is a good idea, maybe the new version with the separate argument should have a new name? Then you can mechanically convert the onces that fit the pattern to the new name and form and leave the others to clean up later.

scott

[Announce] Linux-tiny project revival

[Tim Bird]

Recently, the CE Linux forum has been working to revive the
Linux-tiny project.  At OLS, I asked for interested parties
to volunteer to become the new maintainer for the Linux-tiny patchset.

A few candidates came forward, but eventually Michael Opdenacker
was selected as the new primary maintainer.  A few other
people, including John Cooper of Wind River and myself
are working to support this effort.

Recently, many of the Linux-tiny patches have been brought up-to-date
and are now available for use with a 2.6.22 kernel.  The intent
is to test these, and begin mainlining the most effective sub-patches,
in the next few months.

Some automated testing has already been set up, with some
preliminary results published at a CELF conference in Japan.
(See the linux-tiny page below for a link to the presentation.)
Hopefully, results publishing will also be automated soon.

We encourage anyone with interest in this project to get involved.
If you have ideas how to reduce the static or dynamic memory footprint
of Linux, or, even better, patches for this, please let us know about
them.

Please see http://elinux.org/Linux_Tiny

A related document: http://elinux.org/Kernel_Size_Tuning_Guide
is undergoing an update this week.

Thanks,
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [Announce] Linux-tiny project revival

[Rob Landley]

On Wednesday 19 September 2007 1:03:09 pm Tim Bird wrote:
> Recently, the CE Linux forum has been working to revive the
> Linux-tiny project.  At OLS, I asked for interested parties
> to volunteer to become the new maintainer for the Linux-tiny patchset.
>
> A few candidates came forward, but eventually Michael Opdenacker
> was selected as the new primary maintainer.  A few other
> people, including John Cooper of Wind River and myself
> are working to support this effort.
>
> Recently, many of the Linux-tiny patches have been brought up-to-date
> and are now available for use with a 2.6.22 kernel.  The intent
> is to test these, and begin mainlining the most effective sub-patches,
> in the next few months.

Cool!

Could you update http://www.selenic.com/linux-tiny/ to mention the new 
maintainer and new URLs?

> Some automated testing has already been set up, with some
> preliminary results published at a CELF conference in Japan.
> (See the linux-tiny page below for a link to the presentation.)
> Hopefully, results publishing will also be automated soon.
>
> We encourage anyone with interest in this project to get involved.
> If you have ideas how to reduce the static or dynamic memory footprint
> of Linux, or, even better, patches for this, please let us know about
> them.

I've been playing with an idea for a while to improve the printk() situation, 
but it's a more intrusive change than I've had time to bang on.

Right now, the first argument to printk() is a loglevel, but it's handled via 
string concatenation.  I'd like to change that to be an integer, and make it 
an actual comma-separated first argument.  (Mandatory, not optional.)

So instead of:
  printk(KERN_NOTICE "Fruit=%d\n", banana);
It would now be:
  printk(KERN_NOTICE, "Fruit=%d\n", banana);

Change the header from:
  #define KERN_NOTICE "<5>"
to:
  #define KERN_NOTICE 5

Then you can change the printk guts to do something vaguely like (untested):
#define printk(arg1, arg2, ...) actual_printk("<" #arg1 ">" arg2, __VA_ARGS__)

And so far no behavior has changed.  But now the _fun_ part is, you can add a 
config symbol for "what is the minimum loglevel I care about?"  Set that as a 
number from 0-9.  And then you can define the printk to do:

#define printk(level, str, ...) \
  do { \
    if (level < CONFIG_PRINTK_DOICARE) \
      actual_printk("<" #level ">" str, __VA_ARGS__); \
  } while(0);

And viola (however you spell that, I think I'm using the stringed instrument 
but it's french and I'm not trying to type a diacritical mark anyway), the 
compiler's dead code eliminator zaps the printks you don't care about so they 
don't bloat the kernel image.  But this doesn't _completely_ eliminate 
printks, so you can still get the panic() calls and such.  You tweak precisly 
how much bloat you want, using the granularity information that's already 
there in the source code...

Opinions?

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

Re: [Announce] Linux-tiny project revival

[Alexey Dobriyan]

On Thu, Sep 20, 2007 at 03:38:42PM -0500, Rob Landley wrote:
> I've been playing with an idea for a while to improve the printk() situation, 
> but it's a more intrusive change than I've had time to bang on.
> 
> Right now, the first argument to printk() is a loglevel, but it's handled via 
> string concatenation.  I'd like to change that to be an integer, and make it 
> an actual comma-separated first argument.  (Mandatory, not optional.)
> 
> So instead of:
>   printk(KERN_NOTICE "Fruit=%d\n", banana);
> It would now be:
>   printk(KERN_NOTICE, "Fruit=%d\n", banana);
> 
> Change the header from:
>   #define KERN_NOTICE "<5>"
> to:
>   #define KERN_NOTICE 5
> 
> Then you can change the printk guts to do something vaguely like (untested):
> #define printk(arg1, arg2, ...) actual_printk("<" #arg1 ">" arg2, __VA_ARGS__)
> 
> And so far no behavior has changed.  But now the _fun_ part is, you can add a 
> config symbol for "what is the minimum loglevel I care about?"

Given that
a) there're plenty of printks without any KERN_* bloat,
b) there're printks that SHOULD NOT have KERN_* bloat,
c) debugging-by-printk method is widely used and this will force
   additional typing, head-scratching  and swear words
d) time wasted on pointless discussions whether some particular
   printk ALERT or CRIT
e) flag day for printk,

I think that this idea is not worth it.

> #define printk(level, str, ...) \
>   do { \
>     if (level < CONFIG_PRINTK_DOICARE) \
>       actual_printk("<" #level ">" str, __VA_ARGS__); \
>   } while(0);

> Opinions?

Ick.

	Alexey "ignore_loglevel" Dobriyan



--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -52,6 +52,7 @@ extern const char linux_proc_banner[];
  */
 #define upper_32_bits(n) ((u32)(((n) >> 16) >> 16))
 
+#ifdef CONFIG_FOO
 #define	KERN_EMERG	"<0>"	/* system is unusable			*/
 #define	KERN_ALERT	"<1>"	/* action must be taken immediately	*/
 #define	KERN_CRIT	"<2>"	/* critical conditions			*/
@@ -59,6 +60,15 @@ extern const char linux_proc_banner[];
 #define	KERN_WARNING	"<4>"	/* warning conditions			*/
 #define	KERN_NOTICE	"<5>"	/* normal but significant condition	*/
 #define	KERN_INFO	"<6>"	/* informational			*/
+#else
+#define	KERN_EMERG	""
+#define	KERN_ALERT	""
+#define	KERN_CRIT	""
+#define	KERN_ERR	""
+#define	KERN_WARNING	""
+#define	KERN_NOTICE	""
+#define	KERN_INFO	""
+#endif
 #define	KERN_DEBUG	"<7>"	/* debug-level messages			*/
 
 extern int console_printk[];

Re: [Announce] Linux-tiny project revival

[Rob Landley]

On Thursday 20 September 2007 2:58:44 pm Alexey Dobriyan wrote:
> On Thu, Sep 20, 2007 at 03:38:42PM -0500, Rob Landley wrote:
> > I've been playing with an idea for a while to improve the printk()
> > situation, but it's a more intrusive change than I've had time to bang
> > on.
> >
> > Right now, the first argument to printk() is a loglevel, but it's handled
> > via string concatenation.  I'd like to change that to be an integer, and
> > make it an actual comma-separated first argument.  (Mandatory, not
> > optional.)
> >
> > So instead of:
> >   printk(KERN_NOTICE "Fruit=%d\n", banana);
> > It would now be:
> >   printk(KERN_NOTICE, "Fruit=%d\n", banana);
> >
> > Change the header from:
> >   #define KERN_NOTICE "<5>"
> > to:
> >   #define KERN_NOTICE 5
> >
> > Then you can change the printk guts to do something vaguely like
> > (untested): #define printk(arg1, arg2, ...) actual_printk("<" #arg1 ">"
> > arg2, __VA_ARGS__)
> >
> > And so far no behavior has changed.  But now the _fun_ part is, you can
> > add a config symbol for "what is the minimum loglevel I care about?"
>
> Given that
> a) there're plenty of printks without any KERN_* bloat,

> b) there're printks that SHOULD NOT have KERN_* bloat,

So define a level 0 that doesn't prepend any level to the string, and have the 
macro filter that out at the same default level it counts as now.  
(KERN_INFO, I think?)  The tests are all on contants which should resolve at 
compile time and the dead code eliminator should zap it, even if the macro 
gets more complicated it shouldn't result in a bigger binary.

> c) debugging-by-printk method is widely used and this will force
>    additional typing, head-scratching  and swear words

Because we never change kernel internal APIs.  Oh yeah.  Never happens.

> d) time wasted on pointless discussions whether some particular
>    printk ALERT or CRIT

Let me get this straight: you're objecting to actually making the printk 
levels useful enough that developers start to care what they're set to, 
because then they might be motivated to want some of them changed?

Make it useful, people might care, thus they might talk about it...

Sorry, I'm still missing the downside here.

> e) flag day for printk,

That's the main reason I haven't played with it so far, although it would be 
easy to define a new symbol (dprintk or some such, although I note several 
drivers are already using that) and transition gradually.

> I think that this idea is not worth it.

*Shrug*.

My problem is that switching off printk is the single biggest bloat cutter in 
the kernel, yet it makes the resulting system very hard to support.  It 
combines a big upside with a big downside, and I'd like something in between.

> > #define printk(level, str, ...) \
> >   do { \
> >     if (level < CONFIG_PRINTK_DOICARE) \
> >       actual_printk("<" #level ">" str, __VA_ARGS__); \
> >   } while(0);
> >
> > Opinions?
>
> Ick.
>
> 	Alexey "ignore_loglevel" Dobriyan

But ignore_loglevel doesn't decrease the size of the _binary_.  That's what 
we're talking about here with the -tiny tree.  Embedded developers want to 
squeeze more code onto smaller flash/rom chips.  Setting ignore_loglevel does 
prevent these messages from ever being emitted, but they're still in the 
kernel image as dead weight.  It saves noise but doesn't save _space_.

I'm proposing allowing an ignore_loglevel to remove the unused messages at 
compile time so they don't take up space.  Doing that requires the levels to 
be integers so they can be compared with < or >, and the remaining changes 
follow logically.  (To me, anyway...)

Rob
-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.

RE: [Celinux-dev] Re: [Announce] Linux-tiny project revival

[Gross, Mark]

>-----Original Message-----
>From: celinux-dev-bounces@tree.celinuxforum.org [mailto:celinux-dev-
>bounces@tree.celinuxforum.org] On Behalf Of Rob Landley
>Sent: Thursday, September 20, 2007 3:02 PM
>To: Alexey Dobriyan
>Cc: Michael Opdenacker; linux-tiny@selenic.com; CE Linux Developers
List;
>linux kernel
>Subject: [Celinux-dev] Re: [Announce] Linux-tiny project revival
>
>On Thursday 20 September 2007 2:58:44 pm Alexey Dobriyan wrote:
>> On Thu, Sep 20, 2007 at 03:38:42PM -0500, Rob Landley wrote:
>> > I've been playing with an idea for a while to improve the printk()
>> > situation, but it's a more intrusive change than I've had time to
bang
>> > on.
>> >
>> > Right now, the first argument to printk() is a loglevel, but it's
>handled
>> > via string concatenation.  I'd like to change that to be an
integer,
>and
>> > make it an actual comma-separated first argument.  (Mandatory, not
>> > optional.)
>> >
>> > So instead of:
>> >   printk(KERN_NOTICE "Fruit=%d\n", banana);
>> > It would now be:
>> >   printk(KERN_NOTICE, "Fruit=%d\n", banana);
>> >
>> > Change the header from:
>> >   #define KERN_NOTICE "<5>"
>> > to:
>> >   #define KERN_NOTICE 5
>> >
>> > Then you can change the printk guts to do something vaguely like
>> > (untested): #define printk(arg1, arg2, ...) actual_printk("<" #arg1
">"
>> > arg2, __VA_ARGS__)
>> >
>> > And so far no behavior has changed.  But now the _fun_ part is, you
can
>> > add a config symbol for "what is the minimum loglevel I care
about?"
>>
>> Given that
>> a) there're plenty of printks without any KERN_* bloat,
>
>> b) there're printks that SHOULD NOT have KERN_* bloat,
>
>So define a level 0 that doesn't prepend any level to the string, and
have
>the
>macro filter that out at the same default level it counts as now.
>(KERN_INFO, I think?)  The tests are all on contants which should
resolve
>at
>compile time and the dead code eliminator should zap it, even if the
macro
>gets more complicated it shouldn't result in a bigger binary.
>
>> c) debugging-by-printk method is widely used and this will force
>>    additional typing, head-scratching  and swear words
>
>Because we never change kernel internal APIs.  Oh yeah.  Never happens.
>
>> d) time wasted on pointless discussions whether some particular
>>    printk ALERT or CRIT
>
>Let me get this straight: you're objecting to actually making the
printk
>levels useful enough that developers start to care what they're set to,
>because then they might be motivated to want some of them changed?
>
>Make it useful, people might care, thus they might talk about it...
>
>Sorry, I'm still missing the downside here.
>
>> e) flag day for printk,
>
>That's the main reason I haven't played with it so far, although it
would
>be
>easy to define a new symbol (dprintk or some such, although I note
several
>drivers are already using that) and transition gradually.
>
>> I think that this idea is not worth it.
>
>*Shrug*.
>
>My problem is that switching off printk is the single biggest bloat
cutter
>in
>the kernel, yet it makes the resulting system very hard to support.  It
>combines a big upside with a big downside, and I'd like something in
>between.

What about getting even more hard core? 

Use compiler tricks to remove ALL the static printk string from the
kernel and replace the printk with something that outputs an decimal
index followed by tuples, of zero to N, hex-strings on a single line.
Then have the syslogd or some other utility take this cryptic output and
convolve it with a table (created at compile time) to re-create what
would have been dumped to the sys-log ring buffer.  This way you strip
out most of the static text from the kernel and yet can still re-create
the kernlog output.

At least as a post processing operation....

Is this an old idea?  I'm guessing this has been at least proposed
before....

--mgross

the 
>
>> > #define printk(level, str, ...) \
>> >   do { \
>> >     if (level < CONFIG_PRINTK_DOICARE) \
>> >       actual_printk("<" #level ">" str, __VA_ARGS__); \
>> >   } while(0);
>> >
>> > Opinions?
>>
>> Ick.
>>
>> 	Alexey "ignore_loglevel" Dobriyan
>
>But ignore_loglevel doesn't decrease the size of the _binary_.  That's
what
>we're talking about here with the -tiny tree.  Embedded developers want
to
>squeeze more code onto smaller flash/rom chips.  Setting
ignore_loglevel
>does
>prevent these messages from ever being emitted, but they're still in
the
>kernel image as dead weight.  It saves noise but doesn't save _space_.
>
>I'm proposing allowing an ignore_loglevel to remove the unused messages
at
>compile time so they don't take up space.  Doing that requires the
levels
>to
>be integers so they can be compared with < or >, and the remaining
changes
>follow logically.  (To me, anyway...)
>
>Rob
>--
>"One of my most productive days was throwing away 1000 lines of code."
>  - Ken Thompson.
>_______________________________________________
>Celinux-dev mailing list
>Celinux-dev@tree.celinuxforum.org
>http://tree.celinuxforum.org/mailman/listinfo/celinux-dev

Re: [Announce] Linux-tiny project revival

[Joe Perches]

On Thu, 2007-09-20 at 15:38 -0500, Rob Landley wrote:
> And so far no behavior has changed.  But now the _fun_ part is, you can add a 
> config symbol for "what is the minimum loglevel I care about?"  Set that as a 
> number from 0-9.  And then you can define the printk to do:
> 
> #define printk(level, str, ...) \
>   do { \
>     if (level < CONFIG_PRINTK_DOICARE) \
>       actual_printk("<" #level ">" str, __VA_ARGS__); \
>   } while(0);
> 
> And viola (however you spell that, I think I'm using the stringed instrument 

> But this doesn't _completely_ eliminate 
> printks, so you can still get the panic() calls and such.  You tweak precisly 
> how much bloat you want, using the granularity information that's already 
> there in the source code...
> Opinions?

I'd rather take the opportunity to convert all the printks to
use pr_<level>.  That way, you can pick'n'choose if you want
arbitrary combinations of KERN_<level> compiled in or not.

Re: [Celinux-dev] Re: [Announce] Linux-tiny project revival

[Geert Uytterhoeven]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1842 bytes --]

On Thu, 20 Sep 2007, Joe Perches wrote:
> On Thu, 2007-09-20 at 15:38 -0500, Rob Landley wrote:
> > And so far no behavior has changed.  But now the _fun_ part is, you can add a 
> > config symbol for "what is the minimum loglevel I care about?"  Set that as a 
> > number from 0-9.  And then you can define the printk to do:
> > 
> > #define printk(level, str, ...) \
> >   do { \
> >     if (level < CONFIG_PRINTK_DOICARE) \
> >       actual_printk("<" #level ">" str, __VA_ARGS__); \
> >   } while(0);
> > 
> > And viola (however you spell that, I think I'm using the stringed instrument 
> 
> > But this doesn't _completely_ eliminate 
> > printks, so you can still get the panic() calls and such.  You tweak precisly 
> > how much bloat you want, using the granularity information that's already 
> > there in the source code...
> > Opinions?
> 
> I'd rather take the opportunity to convert all the printks to
> use pr_<level>.  That way, you can pick'n'choose if you want
> arbitrary combinations of KERN_<level> compiled in or not.

Or:

    if ((1 << level) & CONFIG_PRINTK_MASK)
	actual_printk("<" #level ">" str, __VA_ARGS__);

But it would indeed be nice to have all of pr_<level> (and not only pr_info()
and pr_debug()) anyway.

With kind regards,
 
Geert Uytterhoeven
Software Architect

Sony Network and Software Technology Center Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
 
Phone:    +32 (0)2 700 8453	
Fax:      +32 (0)2 700 8622	
E-mail:   Geert.Uytterhoeven@sonycom.com	
Internet: http://www.sony-europe.com/
 	
Sony Network and Software Technology Center Europe	
A division of Sony Service Centre (Europe) N.V.	
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium	
VAT BE 0413.825.160 · RPR Brussels	
Fortis Bank Zaventem · Swift GEBABEBB08A · IBAN BE39001382358619