Re: Hardwired drivers are going away?

Re: Hardwired drivers are going away?

[Pete Zaitcev]

> 3. simplicity in building kernels for other machines. with a monolithic
> kernel you have one file to move (and a bootloader to run) with modules
> you have to move quite a few more files.

It is easy to automate. Simply build RPMs and install them.

Die-hard RPM haters may use some ad-hoc tools as well, see:
http://people.redhat.com/zaitcev/linux/linux-collect

-- Pete

Re: Hardwired drivers are going away?

[Alan Cox]

> 1) The main kernel is contiguous in physical memory and is mapped with
>    large (4 MB) pages.  This reduces pressure on the TLB.  Modules are
> 2) Space for module code is allocated in page units.  Thus, each module
>    wastes an average of 2K.  If I'm going to have dozens of modules
>    loaded, small machines are going to notice.

If at boot time we keep a big chunk of ram free at the kernel end and just
load modules one after each other into that space until we get into real
paging that problem goes away

Re: Hardwired drivers are going away?

[peter]

I don't mind how people do it, but there are currently two significant
penalties associated with modules on x86.  I think other architectures
have analogues.

1) The main kernel is contiguous in physical memory and is mapped with
   large (4 MB) pages.  This reduces pressure on the TLB.  Modules are
   loaded in vmalloc memory, which uses small pages, and therefore
   competes for TLB space.  This is a performance penalty, especially
   as most current machines have undersized TLBs already.  (A 64-entry
   TLB with 4K pages maps 256K at a time.  On-chip L2 caches are this
   large or larger.  Thus, as a crude approximation, every L2 miss also
   causes a TLB miss.)

   Also, vmalloc space is limited, and contributes to the "why isn't
   the kernel seeing all of my 1 GB?" question.

2) Space for module code is allocated in page units.  Thus, each module
   wastes an average of 2K.  If I'm going to have dozens of modules
   loaded, small machines are going to notice.


On a related subject, I'd like a kernel image to consist of a series
of concatenated "chunks".  The first one is the kernel proper, and then
come various modules and initrd image(s).  (There may be an "EOF" chunk,
or just the absence of a magic number.)

All the boot loader has to do is copy the image into contiguous memory
and jump to it.  The start of the image (ideally, it'd be PIC) then
relocates itself to wherever it wants and starts assembling the pieces.

Whether this is done by a sort of linker before the kernel boots, or
if the chunks are all ramdisk components and form a userland that the
kernel boots to which assemble the rest of the kernel is not particularly
important.  I just want the unnecessary bits to get thrown away once
we've booted.  (Kernel bloat should NOT be designed in.)

That would make assembling a custom boot image simple enough for a fancy
boot loader (or fancy tftp server) to do.

Re: Hardwired drivers are going away?

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 15 Jan 2002 12:44:38 +0000 (GMT)
   
   If at boot time we keep a big chunk of ram free at the kernel end and just
   load modules one after each other into that space until we get into real
   paging that problem goes away

And we do have module_map/module_unmap interfaces already so it's
easy to toy with this.

I've always been meaning to do a "alloc 4MB page at boot, lock into
TLB, carve module pages out of that, vmalloc when that runs out" on
sparc64.

Re: Hardwired drivers are going away?

[Rob Landley]

On Tuesday 15 January 2002 07:39 am, David S. Miller wrote:
>    From: Alan Cox <alan@lxorguk.ukuu.org.uk>
>    Date: Tue, 15 Jan 2002 12:44:38 +0000 (GMT)
>
>    If at boot time we keep a big chunk of ram free at the kernel end and
> just load modules one after each other into that space until we get into
> real paging that problem goes away
>
> And we do have module_map/module_unmap interfaces already so it's
> easy to toy with this.
>
> I've always been meaning to do a "alloc 4MB page at boot, lock into
> TLB, carve module pages out of that, vmalloc when that runs out" on
> sparc64.

How would this handle modules being unloaded?  Are you going to try to pack 
the space, try to re-use the holes (allowing fragmentation), handle unloading 
the last module only (high water mark), or just ignore it and basically not 
allow modules to ever be unloaded?

Just curious...

Rob

Re: Hardwired drivers are going away?

[Andi Kleen]

peter@horizon.com writes:

> 1) The main kernel is contiguous in physical memory and is mapped with
>    large (4 MB) pages.  This reduces pressure on the TLB.  Modules are
>    loaded in vmalloc memory, which uses small pages, and therefore
>    competes for TLB space.  This is a performance penalty, especially
>    as most current machines have undersized TLBs already.  (A 64-entry
>    TLB with 4K pages maps 256K at a time.  On-chip L2 caches are this
>    large or larger.  Thus, as a crude approximation, every L2 miss also
>    causes a TLB miss.)

-aa tries to load modules into the linear mapping when possible.
That usually works when you load the modules early when the memory 
isn't that fragmented yet.

I agree on that trying to put everything into modules isn't a good idea,
especially because of your second point.

-Andi

Re: Hardwired drivers are going away?

[Chris Adams]

In article <fa.ksahkuv.tg60gl@ifi.uio.no>, David Lang wrote:
>I can see a couple reasons for building a kernel without useing modules.
>
>1. security, if you don't need any modules you can disable modules entirly
>and then it's impossible to add a module without patching the kernel first
>(the module load system calls aren't there)

There is no security gain from disabling module loading.  If someone has
the level of access needed to load modules, they also have access to
/dev/mem.  Run-time patching of the kernel is something that can be done
(and probably is done by some rootkits).  For bonus points, patch in the
system call(s) required by insmod and voila: module loading now works
(instead of having to patch all your rootkit code into the running
kernel, patch in insmod and let the kernel load the code for you).
-- 
Chris Adams <cmadams@hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

Re: ISA hardware discovery -- the elegant solution

[Mr. James W. Laferriere]

	Hello All ,  And what mechanism is going to be used for an -all-
	compiled in kernel ?  Everyone and there brother is so enamoured
	of Modules .  Not everyone uses nor will use modules .
		Tia ,  JimL

On Mon, 14 Jan 2002, Alan Cox wrote:

> > That would be fine with me.  But wouldn't it involve adding a new
> > initialization-time call to every driver.
>
> man lsmod
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: ISA hardware discovery -- the elegant solution

[Alan Cox]

> 	Hello All ,  And what mechanism is going to be used for an -all-
> 	compiled in kernel ?  Everyone and there brother is so enamoured
> 	of Modules .  Not everyone uses nor will use modules .

For 2.5 if things go to plan there will be no such thing as a "compiled in"
driver. They simply are not needed with initramfs holding what were once the
"compiled in" modules.

Alan

Hardwired drivers are going away?

[Eric S. Raymond]

Alan Cox <alan@lxorguk.ukuu.org.uk>:
> For 2.5 if things go to plan there will be no such thing as a "compiled in"
> driver. They simply are not needed with initramfs holding what were once the
> "compiled in" modules.

This is something of a bombshell.  Not necessarily a bad one, but...

Alan, do you have *any* *freakin'* *idea* how much more complicated
the CML2 deduction engine had to be because the basic logical entity
was a tristate rather than a bool?  If this plan goes through, I'm
going to be able to drop out at least 20% of the code, with most of
that 20% being in the nasty complicated bits where the maintainability
improvement will be greatest.  And I can get rid of the nasty "vitality"
flag, which probably the worst wart on the language.

Yowza...so how soon is this supposed to happen?
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Government should be weak, amateurish and ridiculous. At present, it
fulfills only a third of the role.	-- Edward Abbey

Re: Hardwired drivers are going away?

[David Lang]

I can see a couple reasons for building a kernel without useing modules.

1. security, if you don't need any modules you can disable modules entirly
and then it's impossible to add a module without patching the kernel first
(the module load system calls aren't there)

2. speed, there was a discussion a few weeks ago pointing out that there
is some overhead for useing modules (far calls need to be used just in
case becouse the system can't know where the module will be located IIRC)

3. simplicity in building kernels for other machines. with a monolithic
kernel you have one file to move (and a bootloader to run) with modules
you have to move quite a few more files.

modules are good for distros and for machines that have to worry about
hotpluged equipment (but even that is less nessasary then it used to be)
but for servers that don't change their hardware they aren't nessasary,
why should they have to deal with them?

David Lang




On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Date: Mon, 14 Jan 2002 13:10:50 -0500
> From: Eric S. Raymond <esr@thyrsus.com>
> To: Alan Cox <alan@lxorguk.ukuu.org.uk>
> Cc: Mr. James W. Laferriere <babydr@baby-dragons.com>,
>      Giacomo Catenazzi <cate@debian.org>,
>      Linux Kernel List <linux-kernel@vger.kernel.org>
> Subject: Hardwired drivers are going away?
>
> Alan Cox <alan@lxorguk.ukuu.org.uk>:
> > For 2.5 if things go to plan there will be no such thing as a "compiled in"
> > driver. They simply are not needed with initramfs holding what were once the
> > "compiled in" modules.
>
> This is something of a bombshell.  Not necessarily a bad one, but...
>
> Alan, do you have *any* *freakin'* *idea* how much more complicated
> the CML2 deduction engine had to be because the basic logical entity
> was a tristate rather than a bool?  If this plan goes through, I'm
> going to be able to drop out at least 20% of the code, with most of
> that 20% being in the nasty complicated bits where the maintainability
> improvement will be greatest.  And I can get rid of the nasty "vitality"
> flag, which probably the worst wart on the language.
>
> Yowza...so how soon is this supposed to happen?
> --
> 		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>
>
> Government should be weak, amateurish and ridiculous. At present, it
> fulfills only a third of the role.	-- Edward Abbey
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Hardwired drivers are going away?

[Alan Cox]

> 1. security, if you don't need any modules you can disable modules entirly
> and then it's impossible to add a module without patching the kernel first
> (the module load system calls aren't there)

Urban legend.

> 2. speed, there was a discussion a few weeks ago pointing out that there
> is some overhead for useing modules (far calls need to be used just in
> case becouse the system can't know where the module will be located IIRC)

I defy you to measure it on x86

> 3. simplicity in building kernels for other machines. with a monolithic
> kernel you have one file to move (and a bootloader to run) with modules
> you have to move quite a few more files.

tar or nfs mount; make modules_install.

Re: Hardwired drivers are going away?

[David Lang]

On Mon, 14 Jan 2002, Alan Cox wrote:

> > 1. security, if you don't need any modules you can disable modules entirly
> > and then it's impossible to add a module without patching the kernel first
> > (the module load system calls aren't there)
>
> Urban legend.
>

If this is the case then why do I get systemcall undefined error messages
when I make a mistake and attempt to load a module on a kernel without
modules enabled?

> > 2. speed, there was a discussion a few weeks ago pointing out that there
> > is some overhead for useing modules (far calls need to be used just in
> > case becouse the system can't know where the module will be located IIRC)
>
> I defy you to measure it on x86

during the discussion a few weeks ago there were people pointing out cases
where this overhead would be a problem.

> > 3. simplicity in building kernels for other machines. with a monolithic
> > kernel you have one file to move (and a bootloader to run) with modules
> > you have to move quite a few more files.
>
> tar or nfs mount; make modules_install.
>
not on my firewalls thank you.

David Lang

Re: Hardwired drivers are going away?

[Andrew Pimlott]

On Mon, Jan 14, 2002 at 10:57:19AM -0800, David Lang wrote:
> On Mon, 14 Jan 2002, Alan Cox wrote:
> 
> > > 1. security, if you don't need any modules you can disable modules entirly
> > > and then it's impossible to add a module without patching the kernel first
> > > (the module load system calls aren't there)
> >
> > Urban legend.
> 
> If this is the case then why do I get systemcall undefined error messages
> when I make a mistake and attempt to load a module on a kernel without
> modules enabled?

It's an urban legend that this is a security benefit.  grep the
hacker zines for injecting code into a non-modular kernel.

> > > 2. speed, there was a discussion a few weeks ago pointing out that there
> > > is some overhead for useing modules (far calls need to be used just in
> > > case becouse the system can't know where the module will be located IIRC)
> >
> > I defy you to measure it on x86
                  ^^^^^^^^^^
> during the discussion a few weeks ago there were people pointing out cases
> where this overhead would be a problem.

Above.

> > > 3. simplicity in building kernels for other machines. with a monolithic
> > > kernel you have one file to move (and a bootloader to run) with modules
> > > you have to move quite a few more files.
> >
> > tar or nfs mount; make modules_install.
> >
> not on my firewalls thank you.

You won't install tar on your firewalls?  Weird :-)

Andrew

Re: Hardwired drivers are going away?

[Alan Cox]

> > Urban legend.
> 
> If this is the case then why do I get systemcall undefined error messages
> when I make a mistake and attempt to load a module on a kernel without
> modules enabled?

Because you are using insmod. The guys who break into your computer probably 
are not in their own language "that lame".

> > I defy you to measure it on x86
> 
> during the discussion a few weeks ago there were people pointing out cases
> where this overhead would be a problem.

TLB miss overhead for kernel modules is very very hard to measure, at least
on the x86 platform. It can be an issue in userspace as you address much
larger objects and in certain cases like scanning an element in an array
of objects hit a TLB reload each scan or two (think about vertical line draw
unaccelerated in X11)

> > tar or nfs mount; make modules_install.
> >
> not on my firewalls thank you.

If you are that worried just burn a new root file system on CD when you 
upgrade your firewall. At least I hope if you are going to pretend to be 
so secure that tar is a bad idea you have no writable file store on the
machine ?

Alan

Re: Hardwired drivers are going away?

[Horst von Brand]

> On Mon, 14 Jan 2002, Alan Cox wrote:

> > > 1. security, if you don't need any modules you can disable modules
> > > entirly and then it's impossible to add a module without patching
> > > the kernel first (the module load system calls aren't there)

> > Urban legend.

> If this is the case then why do I get systemcall undefined error messages
> when I make a mistake and attempt to load a module on a kernel without
> modules enabled?

AFAIU the security improvement of no-modules are way overrated.

[...]

> > > 3. simplicity in building kernels for other machines. with a monolithic
> > > kernel you have one file to move (and a bootloader to run) with modules
> > > you have to move quite a few more files.

> > tar or nfs mount; make modules_install.

> not on my firewalls thank you.

You keep compilers and stuff on your firewalls then?!

I get them over via ssh or by floppy/CD. Same idea.
-- 
Horst von Brand			     http://counter.li.org # 22616

Re: Hardwired drivers are going away?

[crispin]

On Tue, Jan 15, 2002 at 11:45:39AM +0100, Horst von Brand wrote:
> > On Mon, 14 Jan 2002, Alan Cox wrote:
> 
> > > > 1. security, if you don't need any modules you can disable modules
> > > > entirly and then it's impossible to add a module without patching
> > > > the kernel first (the module load system calls aren't there)
> 
> > > Urban legend.
> 
> > If this is the case then why do I get systemcall undefined error messages
> > when I make a mistake and attempt to load a module on a kernel without
> > modules enabled?
> 
> AFAIU the security improvement of no-modules are way overrated.

insmod knark.o

Crispin

Re: Hardwired drivers are going away?

[David Lang]

one other thing re: simplicity.

bootloaders

I know that many of you who are advocating this new approach despise lilo,
but for many people it does the job just fine. with the new approach when
switching between kernels you now not only need to switch the kernel
images, but also the initrd (or equivalent) images.

also don't forget that the current way of storing modules can have
problems when some kernel comile options change (SMP for example, but IIRC
there are others) if you make modules mandatory you will have to fix this
first.

David Lang


>
> > > > 3. simplicity in building kernels for other machines. with a monolithic
> > > > kernel you have one file to move (and a bootloader to run) with modules
> > > > you have to move quite a few more files.
>

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello Alan ,

On Mon, 14 Jan 2002, Alan Cox wrote:
> > 1. security, if you don't need any modules you can disable modules entirly
> > and then it's impossible to add a module without patching the kernel first
> > (the module load system calls aren't there)
>
> Urban legend.
	I do not agree .  Got proof ?  Yes that is a valid question .

> > 2. speed, there was a discussion a few weeks ago pointing out that there
> > is some overhead for useing modules (far calls need to be used just in
> > case becouse the system can't know where the module will be located IIRC)
> I defy you to measure it on x86
	OK ,How about sparc-64/alpha/ia64/... ?

> > 3. simplicity in building kernels for other machines. with a monolithic
> > kernel you have one file to move (and a bootloader to run) with modules
> > you have to move quite a few more files.
> tar or nfs mount; make modules_install.
	Please my laugh'o meter is stuck already .  Sorry .  JimL

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[Alan Cox]

> > Urban legend.
> 	I do not agree .  Got proof ?  Yes that is a valid question .

Most of the rootkit type stuff I see nowdays includes code for loading
patches into module free kernels. Its a real no win. The better ones support
regexp scanning so they can patch kernels where the sysadmin thinks he/she
is cool and has hidden or crapped in System.map

> > > case becouse the system can't know where the module will be located IIRC)
> > I defy you to measure it on x86
> 	OK ,How about sparc-64/alpha/ia64/... ?

Not generally found in your grandmothers PC

> > > 3. simplicity in building kernels for other machines. with a monolithic
> > > kernel you have one file to move (and a bootloader to run) with modules
> > > you have to move quite a few more files.
> > tar or nfs mount; make modules_install.
> 	Please my laugh'o meter is stuck already .  Sorry .  JimL

Then fix it, because the above works well. Also remember that autoconfig
tools won't be able to guess remote machines very well 8)

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello Alan ,

On Mon, 14 Jan 2002, Alan Cox wrote:
> > > Urban legend.
> > 	I do not agree .  Got proof ?  Yes that is a valid question .
> Most of the rootkit type stuff I see nowdays includes code for loading
> patches into module free kernels. Its a real no win. The better ones support
> regexp scanning so they can patch kernels where the sysadmin thinks he/she
> is cool and has hidden or crapped in System.map
	Same as I am seeing .  This does not eleviate the fact that
	modules can & are a still a security issue .  Yes , sloppy
	administration is still the biggest culprit in 'most' compromises .

> > > > case becouse the system can't know where the module will be located IIRC)
> > > I defy you to measure it on x86
> > 	OK ,How about sparc-64/alpha/ia64/... ?
> Not generally found in your grandmothers PC
	No,  But it is -mine- !  If I can not have complete control over
	my system due to the way the Fathers of this operating system
	are taking it I will have no choice but to use another or be stuck
	at a non-maintained version .  Now mind you I LIKE LINUX !

> > > > 3. simplicity in building kernels for other machines. with a monolithic
> > > > kernel you have one file to move (and a bootloader to run) with modules
> > > > you have to move quite a few more files.
> > > tar or nfs mount; make modules_install.
> > 	Please my laugh'o meter is stuck already .  Sorry .  JimL
> Then fix it, because the above works well. Also remember that autoconfig
> tools won't be able to guess remote machines very well 8)
	Show me , Please .  I am not convinced .  I shall try out the new
	level of kernel without a doubt .  But ,  I find the concept that
	-everything- is dynamic to be wrong .  Mind you My opinion .
		Twyl ,  JimL

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[Alan Cox]

> 	No,  But it is -mine- !  If I can not have complete control over
> 	my system due to the way the Fathers of this operating system
> 	are taking it I will have no choice but to use another or be stuck
> 	at a non-maintained version .  Now mind you I LIKE LINUX !

Right but you aren't going to go and use autoconfig except maybe to get
an initial Configuration to frob with ?

Alan

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello Alan ,

On Mon, 14 Jan 2002, Alan Cox wrote:
> > 	No,  But it is -mine- !  If I can not have complete control over
> > 	my system due to the way the Fathers of this operating system
> > 	are taking it I will have no choice but to use another or be stuck
> > 	at a non-maintained version .  Now mind you I LIKE LINUX !
> Right but you aren't going to go and use autoconfig except maybe to get
> an initial Configuration to frob with ?
	Ok , Now I am confused .  I guess it's not hard to do .
	Is the Kernel headed toward Soft drivers or not ?

	I get my initial Config (say from autoconf.) how will that keep
	the Drivers in the Kernel ?  Tia ,  JimL

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[David Lang]

On Mon, 14 Jan 2002, Alan Cox wrote:

> Date: Mon, 14 Jan 2002 19:17:46 +0000 (GMT)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: babydr@baby-dragons.com
> Cc: alan@lxorguk.ukuu.org.uk, linux-kernel@vger.kernel.org
> Subject: Re: Hardwired drivers are going away?
>
> > > Urban legend.
> > 	I do not agree .  Got proof ?  Yes that is a valid question .
>
> Most of the rootkit type stuff I see nowdays includes code for loading
> patches into module free kernels. Its a real no win. The better ones support
> regexp scanning so they can patch kernels where the sysadmin thinks he/she
> is cool and has hidden or crapped in System.map
>
> > > > case becouse the system can't know where the module will be located IIRC)
> > > I defy you to measure it on x86
> > 	OK ,How about sparc-64/alpha/ia64/... ?
>
> Not generally found in your grandmothers PC

doesn't matter, they are likly to be found on dedicated servers where
the flexibility of modules is not needed and the slight performance
advantage is desired.

making everything modular is fine for desktops/laptops but why should
dedicated servers pay the price?

> > > > 3. simplicity in building kernels for other machines. with a monolithic
> > > > kernel you have one file to move (and a bootloader to run) with modules
> > > > you have to move quite a few more files.
> > > tar or nfs mount; make modules_install.
> > 	Please my laugh'o meter is stuck already .  Sorry .  JimL
>
> Then fix it, because the above works well. Also remember that autoconfig
> tools won't be able to guess remote machines very well 8)

I don't want autoconfig to detect my remote machines I want to specify
them directly. (there are likly to be things on the motherboards of
machines that I don't want to compile into the kernel, sound cards for
servers are one example, video drivers are another, and USB is a third)

autoconfig is supposed to be an option, not the only way to compile a
kernel (or are you saying that you don't want to be able to use your 1.2G
athlon to build the kernel for your 486?)

David Lang

Re: Hardwired drivers are going away?

[Alexander Viro]

On Mon, 14 Jan 2002, David Lang wrote:

> doesn't matter, they are likly to be found on dedicated servers where
> the flexibility of modules is not needed and the slight performance
> advantage is desired.
> 
> making everything modular is fine for desktops/laptops but why should
> dedicated servers pay the price?

There should be no price.  And AFAICS that's doable.
 
> autoconfig is supposed to be an option, not the only way to compile a
> kernel (or are you saying that you don't want to be able to use your 1.2G
> athlon to build the kernel for your 486?)

autoconfig is completely separate story.  There are very good reasons to
avoid built-in/modular crap, regardless of the autoconf.  Every #ifdef MODULE
is a bug waiting to happen - simply because the stuff gets out of sync.
Every place that consists of
#ifdef CONFIG_FOO
	init_foo();
#endif
....
is a permanent source of PITA for out-of-tree drivers/patch merging/etc.
And that's aside of the fun with ordering, etc.

If we can get rid of performance hit on modular code - we should go
for that variant simply because it reduces the amount of bug sources.

Autoconfig and other details of ESR's Aunt Tillie fetish belong to
alt.sex.* - from time to time they may be amusing, but let's not
mix them with the technical arguments, OK?

Re: Hardwired drivers are going away?

[Chris Friesen]

Alexander Viro wrote:
> 
> On Mon, 14 Jan 2002, David Lang wrote:
> 
> > doesn't matter, they are likly to be found on dedicated servers where
> > the flexibility of modules is not needed and the slight performance
> > advantage is desired.
> >
> > making everything modular is fine for desktops/laptops but why should
> > dedicated servers pay the price?
> 
> There should be no price.  And AFAICS that's doable.

I haven't been following the initramfs stuff, but now I have a question. 
Currently we're using initrd to store a kernel and compressed ramdisk bundled
together as an ~7MB single file that gets netbooted by firmware in a card.  Will
it be possible to bundle initramfs together with the kernel into a single file
in this same manner?

Thanks,

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: Hardwired drivers are going away?

[Alexander Viro]

On Mon, 14 Jan 2002, Chris Friesen wrote:

> I haven't been following the initramfs stuff, but now I have a question. 
> Currently we're using initrd to store a kernel and compressed ramdisk bundled
> together as an ~7MB single file that gets netbooted by firmware in a card.  Will
> it be possible to bundle initramfs together with the kernel into a single file
> in this same manner?

<shrug> kernel unpacks to rootfs after init_vfs_caches().  It does
	populate_root(archive_start, archive_size);
followed by releasing that area.  That's it.  If you have these variables
set - it will work, no matter how did the archive get there.

Re: Hardwired drivers are going away?

[David Lang]

note: this discussion has moved away from autoconfig to requiring
everything to be modules.

autoconfig is a useful tool in some cases, but is not going to be used
everywhere.

I view modules as being in the same catagory.

David Lang


On Mon, 14 Jan 2002, Alan Cox wrote:

> Date: Mon, 14 Jan 2002 19:17:46 +0000 (GMT)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: babydr@baby-dragons.com
> Cc: alan@lxorguk.ukuu.org.uk, linux-kernel@vger.kernel.org
> Subject: Re: Hardwired drivers are going away?
>
> > > Urban legend.
> > 	I do not agree .  Got proof ?  Yes that is a valid question .
>
> Most of the rootkit type stuff I see nowdays includes code for loading
> patches into module free kernels. Its a real no win. The better ones support
> regexp scanning so they can patch kernels where the sysadmin thinks he/she
> is cool and has hidden or crapped in System.map
>
> > > > case becouse the system can't know where the module will be located IIRC)
> > > I defy you to measure it on x86
> > 	OK ,How about sparc-64/alpha/ia64/... ?
>
> Not generally found in your grandmothers PC
>
> > > > 3. simplicity in building kernels for other machines. with a monolithic
> > > > kernel you have one file to move (and a bootloader to run) with modules
> > > > you have to move quite a few more files.
> > > tar or nfs mount; make modules_install.
> > 	Please my laugh'o meter is stuck already .  Sorry .  JimL
>
> Then fix it, because the above works well. Also remember that autoconfig
> tools won't be able to guess remote machines very well 8)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Hardwired drivers are going away?

[Ian Molton]

On a sunny Mon, 14 Jan 2002 11:44:59 -0800 (PST) David Lang gathered a
sheaf of electrons and etched in their motions the following immortal
words:

> doesn't matter, they are likly to be found on dedicated servers where
> the flexibility of modules is not needed and the slight performance
> advantage is desired.

Exactly WHAT performance advantage? once the module is loaded, its loaded.
most modules use the same code to handle modular and non-modular builds
anyhow (look at the ide drivers, for example)

Re: Hardwired drivers are going away?

[David Lang]

the impact is in all calls to the module, if they are far calls instead of
near calls each and every call is (a hair) slower.

so the code can be the same and still be slower to get to.

you can argue that it's not enough slower to matter, but even Alan admits
there is some impact.

David Lang

On Mon, 14 Jan 2002, Ian Molton wrote:

> Date: Mon, 14 Jan 2002 20:51:24 +0000
> From: Ian Molton <spyro@armlinux.org>
> To: linux-kernel@vger.kernel.org
> Subject: Re: Hardwired drivers are going away?
>
> On a sunny Mon, 14 Jan 2002 11:44:59 -0800 (PST) David Lang gathered a
> sheaf of electrons and etched in their motions the following immortal
> words:
>
> > doesn't matter, they are likly to be found on dedicated servers where
> > the flexibility of modules is not needed and the slight performance
> > advantage is desired.
>
> Exactly WHAT performance advantage? once the module is loaded, its loaded.
> most modules use the same code to handle modular and non-modular builds
> anyhow (look at the ide drivers, for example)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Hardwired drivers are going away?

[Brian Gerst]

David Lang wrote:
> 
> the impact is in all calls to the module, if they are far calls instead of
> near calls each and every call is (a hair) slower.
> 
> so the code can be the same and still be slower to get to.
> 
> you can argue that it's not enough slower to matter, but even Alan admits
> there is some impact.
> 
> David Lang

Let's get the terminology right here (for x86 at least):
Far jump: Changes to a new code segment, absolute address
Near jump: Same code segment, 4-byte relative offset
Short jump: Same code segment, 1-byte signed offset

The kernel never uses far jumps except for some BIOS calls and during
booting.  The difference betwen near and short jumps is very minute. 
Short jumps are limited to +/- 128 bytes, so are really only applicable
for small loops.  All function calls between object files must be near
jumps, as the assembler does not not know the distance of an unresolved
symbol and must assume the largest possible offset.

--

				Brian Gerst

Re: Hardwired drivers are going away?

[David Lang]

my apologies for the terminology errors, I was trying to go from memory
from the thread here late last year.

David Lang

On Mon, 14 Jan 2002, Brian Gerst wrote:

> Date: Mon, 14 Jan 2002 18:12:43 -0500
> From: Brian Gerst <bgerst@didntduck.org>
> To: David Lang <dlang@diginsite.com>
> Cc: Ian Molton <spyro@armlinux.org>, linux-kernel@vger.kernel.org
> Subject: Re: Hardwired drivers are going away?
>
> David Lang wrote:
> >
> > the impact is in all calls to the module, if they are far calls instead of
> > near calls each and every call is (a hair) slower.
> >
> > so the code can be the same and still be slower to get to.
> >
> > you can argue that it's not enough slower to matter, but even Alan admits
> > there is some impact.
> >
> > David Lang
>
> Let's get the terminology right here (for x86 at least):
> Far jump: Changes to a new code segment, absolute address
> Near jump: Same code segment, 4-byte relative offset
> Short jump: Same code segment, 1-byte signed offset
>
> The kernel never uses far jumps except for some BIOS calls and during
> booting.  The difference betwen near and short jumps is very minute.
> Short jumps are limited to +/- 128 bytes, so are really only applicable
> for small loops.  All function calls between object files must be near
> jumps, as the assembler does not not know the distance of an unresolved
> symbol and must assume the largest possible offset.
>
> --
>
> 				Brian Gerst
>

Re: Hardwired drivers are going away?

[Ian Molton]

On a sunny Mon, 14 Jan 2002 14:11:48 -0800 (PST) David Lang gathered a
sheaf of electrons and etched in their motions the following immortal
words:

> the impact is in all calls to the module, if they are far calls instead
> of near calls each and every call is (a hair) slower.
> 
> so the code can be the same and still be slower to get to.
> 
> you can argue that it's not enough slower to matter, but even Alan admits
> there is some impact.

Ok, #1 please dont send me courtesy copies without indicating so.

#2 Not all architectures have a problem with 'far' or 'near' calls, and
frankly, I'm glad the kernels design isnt being crippled just to serve the
fundamentally CRAP x86 architecture, for once.

Re: Hardwired drivers are going away?

[H. Peter Anvin]

Followup to:  <20020114232247.195bf441.spyro@armlinux.org>
By author:    Ian Molton <spyro@armlinux.org>
In newsgroup: linux.dev.kernel
> 
> #2 Not all architectures have a problem with 'far' or 'near' calls, and
> frankly, I'm glad the kernels design isnt being crippled just to serve the
> fundamentally CRAP x86 architecture, for once.
> 

Especially since the x86 *ISN'T* one of the architectures with this
particular problem.  'far' and 'near' as used in this thread don't
refer to the x86 segmentation crud.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Hardwired drivers are going away?

[Aaron Lehmann]

On Mon, Jan 14, 2002 at 07:17:46PM +0000, Alan Cox wrote:
> Not generally found in your grandmothers PC

What kind of an argument is this?!? Linux developers used to care
about portability to and performance on non-x86 platforms.

Re: Hardwired drivers are going away?

[Alexander Viro]

On Tue, 15 Jan 2002, Aaron Lehmann wrote:

> On Mon, Jan 14, 2002 at 07:17:46PM +0000, Alan Cox wrote:
> > Not generally found in your grandmothers PC
> 
> What kind of an argument is this?!? Linux developers used to care
> about portability to and performance on non-x86 platforms.

<rant>

Guys, could you lay off whatever the fuck you are smoking?  And
I mean everybody involved.

*	autoconfig is part of ESR's wet dream.  Ignore him and he'll go away.
*	Aunt Tillie and her magical mistery whorehouse <<--->>
*	performance problems with modules are real, need to be solved and
	can be solved.
*	disabling module loading doesn't buy any security.
*	it's easy to make irrevertible disabling modules _after_ boot.
*	initramfs is independent from modules
*	it is _not_ initrd or an analog
*	it doesn't involve a shitload of extra files
*	it removes tons of crap from kernel
*	it allows to simplify build process and get rid of quite a few sources
	of bugs, _if_ performance problems with modules are solved

Now could we fscking drop these threads?  Or at least take care to separate
myths from reality before posting more crap.

</rant>

Re: Hardwired drivers are going away?

[Alan Cox]

> On Mon, Jan 14, 2002 at 07:17:46PM +0000, Alan Cox wrote:
> > Not generally found in your grandmothers PC
> 
> What kind of an argument is this?!? Linux developers used to care
> about portability to and performance on non-x86 platforms.

Read the entire thread (I know thats like asking you to read war and peace)
but if you've not read it in context you will misunderstand arguments (as
you do above).

Re: Hardwired drivers are going away?

[Florian Weimer]

"Mr. James W. Laferriere" <babydr@baby-dragons.com> writes:

> 	Hello Alan ,
>
> On Mon, 14 Jan 2002, Alan Cox wrote:
>> > 1. security, if you don't need any modules you can disable modules entirly
>> > and then it's impossible to add a module without patching the kernel first
>> > (the module load system calls aren't there)
>>
>> Urban legend.
> 	I do not agree .  Got proof ?  Yes that is a valid question .

http://www.phrack.org/phrack/58/p58-0x07

Globally preloading a shared library in user space is almost as
effective, BTW, unless your critical binaries are linked statically
(which is unusual on most systems nowadays).

-- 
Florian Weimer 	                  Weimer@CERT.Uni-Stuttgart.DE
University of Stuttgart           http://CERT.Uni-Stuttgart.DE/people/fw/
RUS-CERT                          +49-711-685-5973/fax +49-711-685-5898

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello Florian ,

On Fri, 18 Jan 2002, Florian Weimer wrote:
> "Mr. James W. Laferriere" <babydr@baby-dragons.com> writes:
> > 	Hello Alan ,
> > On Mon, 14 Jan 2002, Alan Cox wrote:
> >> > 1. security, if you don't need any modules you can disable modules entirly
> >> > and then it's impossible to add a module without patching the kernel first
> >> > (the module load system calls aren't there)
> >> Urban legend.
> > 	I do not agree .  Got proof ?  Yes that is a valid question .
> http://www.phrack.org/phrack/58/p58-0x07
	Thank you for the pointer .  Fine you do not need to allow modules
	in order for a hacker to insert their code .  It is still another
	thing to allow modules & not put & use signatures on them .  AFAIK
	Linux doesn't have a method to load encrypted & signed modules at
	this time .  Please ,  someone prove me wrong .  I -personally-
	like statically compiled kernels .  The method being pushed forth
	at present doesn't allow that ,  Unless I am completely mistaken
	about what Alan & the rest have been discussing .  Again PLEASE
	someone prove me wrong about this also .

> Globally preloading a shared library in user space is almost as
> effective, BTW, unless your critical binaries are linked statically
> (which is unusual on most systems nowadays).
	I can beleive that 8-} .  Statically or written to readonly media
 	or the drive hard set to readonly ;-) .  I have followed most of
	Alan's suggestions security concerns over the years & a few that
	I thought of along the way .  Which later I found had been being
	done alot longer than I would have thought .  Tia ,  JimL

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[Keith Owens]

On Fri, 18 Jan 2002 17:20:02 -0500 (EST), 
"Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
>	Linux doesn't have a method to load encrypted & signed modules at
>	this time .

And never will.  Who loads the module - root.  Who maintains the list
of signatures - root.  Who controls the code that verifies the
signature - root.

Your task Jim, should you choose to accept it, is to make the kernel
distinguish between a good use of root and a malicious use by some who
has broken in and got root privileges.  When you can do that, then we
can add signed modules.

Re: Hardwired drivers are going away?

[Frank van de Pol]

On Sat, Jan 19, 2002 at 10:22:43AM +1100, Keith Owens wrote:
> On Fri, 18 Jan 2002 17:20:02 -0500 (EST), 
> "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
> >	Linux doesn't have a method to load encrypted & signed modules at
> >	this time .
> 
> And never will.  Who loads the module - root.  Who maintains the list
> of signatures - root.  Who controls the code that verifies the
> signature - root.
> 
> Your task Jim, should you choose to accept it, is to make the kernel
> distinguish between a good use of root and a malicious use by some who
> has broken in and got root privileges.  When you can do that, then we
> can add signed modules.
> 

If you want to secure your box, why don't you simply put a lock on it and
throw away the key? Really, what might help the paranoid admins in this case
is a setting in the kernel which basically disables the ability to load or
unload modules. Of course once set this setting can not been turned with
rebooting the box.

Frank.


-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?

Re: Hardwired drivers are going away?

[Bruce Harada]

On Mon, 21 Jan 2002 00:20:41 +0100
Frank van de Pol <fvdpol@home.nl> wrote:

> If you want to secure your box, why don't you simply put a lock on it and
> throw away the key? Really, what might help the paranoid admins in this case
> is a setting in the kernel which basically disables the ability to load or
> unload modules. Of course once set this setting can not been turned with
> rebooting the box.

...and how would you guarantee that this setting remains set, in the face of
some nasty little cracker screwing around with /dev/kmem?

Re: Hardwired drivers are going away?

[Andreas Ferber]

On Mon, Jan 21, 2002 at 09:54:58AM +0900, Bruce Harada wrote:
> 
> ...and how would you guarantee that this setting remains set, in the face of
> some nasty little cracker screwing around with /dev/kmem?

If the attacker gained the ability to play with /dev/kmem, he can
already load modules into the kernel, regardless if the kernel is
actually compiled with module support or not. You can find various
papers describing how to do it via google, and AFAIK some rootkits are
already using those techniques, so it's even "scriptkiddy-ready".

Face it, there is absolutely /no/ security gain in disabling module
support.

Andreas
-- 
       Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
     ---------------------------------------------------------
         +49 521 1365800 - af@devcon.net - www.devcon.net

Re: Hardwired drivers are going away?

[Bruce Harada]

On Mon, 21 Jan 2002 04:49:52 +0100
Andreas Ferber <aferber@techfak.uni-bielefeld.de> wrote:

> On Mon, Jan 21, 2002 at 09:54:58AM +0900, Bruce Harada wrote:
> > 
> > ...and how would you guarantee that this setting remains set, in the face
> > of some nasty little cracker screwing around with /dev/kmem?
> 
> If the attacker gained the ability to play with /dev/kmem, he can
> already load modules into the kernel, regardless if the kernel is
> actually compiled with module support or not. You can find various
> papers describing how to do it via google, and AFAIK some rootkits are
> already using those techniques, so it's even "scriptkiddy-ready".

Er... that was exactly what I meant - perhaps you meant to reply to the parent
post?

Re: Hardwired drivers are going away?

[Andreas Ferber]

On Mon, Jan 21, 2002 at 01:45:16PM +0900, Bruce Harada wrote:
> 
> Er... that was exactly what I meant - perhaps you meant to reply to the parent
> post?

Hmm, I understood your post as if you consider a simple flag disabling
module loading a step-back regarding security. I stand corrected.

Andreas
-- 
       Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
     ---------------------------------------------------------
         +49 521 1365800 - af@devcon.net - www.devcon.net

Re: Hardwired drivers are going away?

[Anton Altaparmakov]

At 23:20 20/01/02, Frank van de Pol wrote:

>On Sat, Jan 19, 2002 at 10:22:43AM +1100, Keith Owens wrote:
> > On Fri, 18 Jan 2002 17:20:02 -0500 (EST),
> > "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
> > >     Linux doesn't have a method to load encrypted & signed modules at
> > >     this time .
> >
> > And never will.  Who loads the module - root.  Who maintains the list
> > of signatures - root.  Who controls the code that verifies the
> > signature - root.
> >
> > Your task Jim, should you choose to accept it, is to make the kernel
> > distinguish between a good use of root and a malicious use by some who
> > has broken in and got root privileges.  When you can do that, then we
> > can add signed modules.
>
>If you want to secure your box, why don't you simply put a lock on it and
>throw away the key? Really, what might help the paranoid admins in this case
>is a setting in the kernel which basically disables the ability to load or
>unload modules. Of course once set this setting can not been turned with
>rebooting the box.

Er that sounds like just disabling modules in the kernel altogether (kernel 
compile option exists for this since the beginning of time)... I do that on 
all servers I control. Not only for security reasons but also because I 
suspect it produces smaller and probably faster kernels (I haven't tested 
this in any way, just a guess).

Best regards,

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello Anton ,

On Mon, 21 Jan 2002, Anton Altaparmakov wrote:
> At 23:20 20/01/02, Frank van de Pol wrote:
> >On Sat, Jan 19, 2002 at 10:22:43AM +1100, Keith Owens wrote:
> > > On Fri, 18 Jan 2002 17:20:02 -0500 (EST),
> > > "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
> > > >     Linux doesn't have a method to load encrypted & signed modules at
> > > >     this time .
> > > And never will.  Who loads the module - root.  Who maintains the list
> > > of signatures - root.  Who controls the code that verifies the
> > > signature - root.
> > > Your task Jim, should you choose to accept it, is to make the kernel
> > > distinguish between a good use of root and a malicious use by some who
> > > has broken in and got root privileges.  When you can do that, then we
> > > can add signed modules.
> >If you want to secure your box, why don't you simply put a lock on it and
> >throw away the key? Really, what might help the paranoid admins in this case
> >is a setting in the kernel which basically disables the ability to load or
> >unload modules. Of course once set this setting can not been turned with
> >rebooting the box.

> Er that sounds like just disabling modules in the kernel altogether (kernel
> compile option exists for this since the beginning of time)... I do that on
> all servers I control. Not only for security reasons but also because I
> suspect it produces smaller and probably faster kernels (I haven't tested
> this in any way, just a guess).
	This is just what the Heads are trying to do away with .  There
	will only be module enabled kernels .  JimL

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[Andrew Morton]

"Mr. James W. Laferriere" wrote:
> 
>         This is just what the Heads are trying to do away with .  There
>         will only be module enabled kernels .  JimL

I suspect none of these "Heads" spend much time in protracted
email debug sessions.  Because the *first* thing you do is
ask the tester to compile the relevant driver into the
kernel.

The problems which the removal of this option will cause include:

1: Inability to look up symbols in the kernel elf image.
2: Breaks the kernel profiler
3: breaks kgdb
4: breaks ksymoops.

How often have we seen nonsensical backtraces here because
modules were involved?   Possibly we can include a table
of module base addresses in the Oops output and teach ksymoops
about it.

This proposal is, frankly, brain-damaged.  It will significantly
impeded kernel developers in remote problem diagnosis and it will
weaken the kernel development toolchain.

There's a lot of work to be done to overcome this damage, and
given the difficulty of getting debug tools into the mainstream
kernel, the damage may well be permanent.

-

Re: Hardwired drivers are going away?

[Keith Owens]

On Sun, 20 Jan 2002 17:30:12 -0800, 
Andrew Morton <akpm@zip.com.au> wrote:
>I suspect none of these "Heads" spend much time in protracted
>email debug sessions.  Because the *first* thing you do is
>ask the tester to compile the relevant driver into the
>kernel.
>
>The problems which the removal of this option will cause include:
>
>1: Inability to look up symbols in the kernel elf image.
>2: Breaks the kernel profiler
>3: breaks kgdb
>4: breaks ksymoops.
>
>How often have we seen nonsensical backtraces here because
>modules were involved?   Possibly we can include a table
>of module base addresses in the Oops output and teach ksymoops
>about it.

You see nonsensical backtraces because people persist in using the oops
decode option of klogd which is broken when faced with modules.  Turn
off klogd oops (klogd -x) and you get a raw backtrace which ksymoops
can handle.  Guess why these entries are in /proc/ksyms?

c48a2300 __insmod_3c589_cs_S.bss_L4	[3c589_cs]
c48a0000 __insmod_3c589_cs_O/lib/modules/2.4.17-xfs/kernel/drivers/net/pcmcia/3c589_cs.o_M3C332CFF_V132113	[3c589_cs]
c48a22a0 __insmod_3c589_cs_S.data_L96	[3c589_cs]
c48a1820 __insmod_3c589_cs_S.rodata_L1152	[3c589_cs]
c48a0060 __insmod_3c589_cs_S.text_L6064	[3c589_cs]

ksymoops uses the __insmod entries to work out exactly where each
module is, it gives an accurate backtrace with modules.  man insmod for
details.

Kernel debuggers like kgdb and kdb use kallsyms which has full support
for modules.  kgdb can also use the __insmod entries in /proc/ksyms to
tell gdb where each module was loaded.

ksymoops has a save map option (-s) which writes out the combined
system map, including the kernel and all symbols from all modules.

Sure, a dynamic system requires a little more work, but it has all been
done.  Just kill the broken klogd code so it stops corrupting log data.

Re: Hardwired drivers are going away?

[Andrew Morton]

Keith Owens wrote:
> 
> On Sun, 20 Jan 2002 17:30:12 -0800,
> Andrew Morton <akpm@zip.com.au> wrote:
> >I suspect none of these "Heads" spend much time in protracted
> >email debug sessions.  Because the *first* thing you do is
> >ask the tester to compile the relevant driver into the
> >kernel.
> >
> >The problems which the removal of this option will cause include:
> >
> >1: Inability to look up symbols in the kernel elf image.
> >2: Breaks the kernel profiler
> >3: breaks kgdb
> >4: breaks ksymoops.
> >
> >How often have we seen nonsensical backtraces here because
> >modules were involved?   Possibly we can include a table
> >of module base addresses in the Oops output and teach ksymoops
> >about it.
> 
> You see nonsensical backtraces because people persist in using the oops
> decode option of klogd which is broken when faced with modules.  Turn
> off klogd oops (klogd -x) and you get a raw backtrace which ksymoops
> can handle.

ksymoops doesn't know what modules were loaded at the time
of the crash, and it doesn't know where they were loaded.

The `klogd -x' problem has been with us for *years* and
distributors still persist in turning it on.


>  Guess why these entries are in /proc/ksyms?
> 
> c48a2300 __insmod_3c589_cs_S.bss_L4     [3c589_cs]
> c48a0000 __insmod_3c589_cs_O/lib/modules/2.4.17-xfs/kernel/drivers/net/pcmcia/3c589_cs.o_M3C332CFF_V132113      [3c589_cs]
> c48a22a0 __insmod_3c589_cs_S.data_L96   [3c589_cs]
> c48a1820 __insmod_3c589_cs_S.rodata_L1152       [3c589_cs]
> c48a0060 __insmod_3c589_cs_S.text_L6064 [3c589_cs]
> 
> ksymoops uses the __insmod entries to work out exactly where each
> module is, it gives an accurate backtrace with modules.  man insmod for
> details.

It assumes too much.  Arjan has a kksymoops thingy which does the symbol
resolution at crash-time.  This seems much more reliable to me.  It also
handles the common case where the running vmlinux/System.map/etc no longer
exist.

> Kernel debuggers like kgdb and kdb use kallsyms which has full support
> for modules.  kgdb can also use the __insmod entries in /proc/ksyms to
> tell gdb where each module was loaded.
> 
> ksymoops has a save map option (-s) which writes out the combined
> system map, including the kernel and all symbols from all modules.
> 
> Sure, a dynamic system requires a little more work, but it has all been
> done.  Just kill the broken klogd code so it stops corrupting log data.

Keith, I have spent a *lot* of time working weird kernel bugs
with people via email.  Usually, things peter out simply because
the other party lacks the expertise or time to keep on doing
things.  So the bug doesn't get fixed.

I would prefer that all this become easier, simpler and more reliable.
We need a damn good reason for deprecating statically linked kernels
and certainly none has been presented yet.

-

Re: Hardwired drivers are going away?

[Keith Owens]

On Sun, 20 Jan 2002 18:04:00 -0800, 
Andrew Morton <akpm@zip.com.au> wrote:
>ksymoops doesn't know what modules were loaded at the time
>of the crash, and it doesn't know where they were loaded.

/var/log/ksymoops.  man insmod and look for ksymoops assistance.

>The `klogd -x' problem has been with us for *years* and
>distributors still persist in turning it on.

Tell me about it :(.

>It assumes too much.  Arjan has a kksymoops thingy which does the symbol
>resolution at crash-time.

It can only get symbols that are in /proc/ksyms, i.e. the exported
symbols.  Proper crash analysis needs the full symbol table.

>It also
>handles the common case where the running vmlinux/System.map/etc no longer
>exist.

So does ksymoops, with reduced detail because it only has exported
symbols.

>I would prefer that all this become easier, simpler and more reliable.
>We need a damn good reason for deprecating statically linked kernels
>and certainly none has been presented yet.

That is a different problem.  Saying that modular kernels cause
problems for debugging is not a good enough reason to deprecate modular
kernels, all the problems have been solved.

Re: Hardwired drivers are going away?

[Andrew Morton]

Keith Owens wrote:
> 
> On Sun, 20 Jan 2002 18:04:00 -0800,
> Andrew Morton <akpm@zip.com.au> wrote:
> >ksymoops doesn't know what modules were loaded at the time
> >of the crash, and it doesn't know where they were loaded.
> 
> /var/log/ksymoops.  man insmod and look for ksymoops assistance.

Oh I have, extensively, when trying to get kgdb to play with
modules.  I did what any sensible person would do: gave up and went
for a static link.

I'm not aware of anyone getting kgdb working fully with modules.

> >The `klogd -x' problem has been with us for *years* and
> >distributors still persist in turning it on.
> 
> Tell me about it :(.
> 
> >It assumes too much.  Arjan has a kksymoops thingy which does the symbol
> >resolution at crash-time.
> 
> It can only get symbols that are in /proc/ksyms, i.e. the exported
> symbols.  Proper crash analysis needs the full symbol table.

Proper crash analysis needs to know the load address of each module
at the time of the crash.  We should print them out at Oops time.

> >It also
> >handles the common case where the running vmlinux/System.map/etc no longer
> >exist.
> 
> So does ksymoops, with reduced detail because it only has exported
> symbols.
> 
> >I would prefer that all this become easier, simpler and more reliable.
> >We need a damn good reason for deprecating statically linked kernels
> >and certainly none has been presented yet.
> 
> That is a different problem.  Saying that modular kernels cause
> problems for debugging is not a good enough reason to deprecate modular
> kernels, all the problems have been solved.

They are patently *not* solved, because we continue to get a
stream of partially and competely useless oops reports.  It
is my empirical observation that this problem is worse when
modules are involved.  It is my personal experience that
modular code is harder to debug.

It is also my personal experience that the refusal of sbp2.o to
work correctly when statically linked was a complete pain in the
ass when trying to diagnose its deadlock and module refcount
problems.  It was only when I worked out that I could link it
statically and then get it to play via rescan-scsi-bus.sh that
I was able to make reasonably-paced progress in finding the bugs.

We don't want this to happen to every damn driver in the kernel.

-

Re: Hardwired drivers are going away?

[Keith Owens]

On Sun, 20 Jan 2002 18:56:34 -0800, 
Andrew Morton <akpm@zip.com.au> wrote:
>I'm not aware of anyone getting kgdb working fully with modules.

kgdb has a script that tells gdb where each module is loaded.  AFAIK it
uses add-symbol-file FILENAME -sSECTION ADDRESS, the __insmod entries
contain enough data to tell gdb what is going on.

>Proper crash analysis needs to know the load address of each module
>at the time of the crash.  We should print them out at Oops time.

You need 13 bits of data per module.  Where each of text, rodata, data
and bss sections were loaded, you cannot assume they are contiguous.
The length of those four sections.  Where the module is on disk, you
cannot assume the object name is the same as the module name, insmod
-o.  The timestamp and kernel version of the module when it was loaded,
to detect updates after the event.  All of that is encoded in the
__insmod entries in /proc/ksyms, 5 lines per module.

>> That is a different problem.  Saying that modular kernels cause
>> problems for debugging is not a good enough reason to deprecate modular
>> kernels, all the problems have been solved.
>
>They are patently *not* solved, because we continue to get a
>stream of partially and competely useless oops reports.

Rule 1.  Users do not read documentation.
Rule 2.  You can't do anything about rule 1.

If you want complete bug reports, write a script that forces the user
to submit all the required data.  Why do I feel the "Linux should have
a bug reporting system" thread starting again?

Re: Hardwired drivers are going away?

[Andrew Morton]

Keith Owens wrote:
> 
> On Sun, 20 Jan 2002 18:56:34 -0800,
> Andrew Morton <akpm@zip.com.au> wrote:
> >I'm not aware of anyone getting kgdb working fully with modules.
> 
> kgdb has a script that tells gdb where each module is loaded.  AFAIK it
> uses add-symbol-file FILENAME -sSECTION ADDRESS, the __insmod entries
> contain enough data to tell gdb what is going on.
> 
> >Proper crash analysis needs to know the load address of each module
> >at the time of the crash.  We should print them out at Oops time.
> 
> You need 13 bits of data per module.  Where each of text, rodata, data
> and bss sections were loaded, you cannot assume they are contiguous.
> The length of those four sections.  Where the module is on disk, you
> cannot assume the object name is the same as the module name, insmod
> -o.  The timestamp and kernel version of the module when it was loaded,
> to detect updates after the event.  All of that is encoded in the
> __insmod entries in /proc/ksyms, 5 lines per module.

That sounds OK.  Just print it first.  If it hits the logs then
great.  If it just scrolls off the screen then no loss.

> >> That is a different problem.  Saying that modular kernels cause
> >> problems for debugging is not a good enough reason to deprecate modular
> >> kernels, all the problems have been solved.
> >
> >They are patently *not* solved, because we continue to get a
> >stream of partially and competely useless oops reports.
> 
> Rule 1.  Users do not read documentation.
> Rule 2.  You can't do anything about rule 1.

Often the users know nothing about elf internals, kernel tools
and build processes and all of that.  They just want to get their
box back up and reliable, and they consider that any work they do 
in problem disgnosis is a favour for the developers (and I have no
problem with that attitude).

Often, we only get one shot at it.
 
> If you want complete bug reports, write a script that forces the user
> to submit all the required data.

It's in our interest to make it as easy as possible for the reporter.
Cut-and-paste is easy, so we should err on the side of overkill when
choosing which data to dump out.

> Why do I feel the "Linux should have
> a bug reporting system" thread starting again?

Yeah, let's not do that.  (But I'd be on the "no" side :-))

-

Re: Hardwired drivers are going away?

[yodaiken]

On Mon, Jan 21, 2002 at 01:25:18PM +1100, Keith Owens wrote:
> >I would prefer that all this become easier, simpler and more reliable.
> >We need a damn good reason for deprecating statically linked kernels
> >and certainly none has been presented yet.
> 
> That is a different problem.  Saying that modular kernels cause
> problems for debugging is not a good enough reason to deprecate modular
> kernels, all the problems have been solved.


Interesting definition of "solved".

Re: Hardwired drivers are going away?

[Kai Henningsen]

yodaiken@fsmlabs.com  wrote on 20.01.02 in <20020120204413.B6096@hq.fsmlabs.com>:

> On Mon, Jan 21, 2002 at 01:25:18PM +1100, Keith Owens wrote:
> > >I would prefer that all this become easier, simpler and more reliable.
> > >We need a damn good reason for deprecating statically linked kernels
> > >and certainly none has been presented yet.
> >
> > That is a different problem.  Saying that modular kernels cause
> > problems for debugging is not a good enough reason to deprecate modular
> > kernels, all the problems have been solved.
>
>
> Interesting definition of "solved".

Make that "all problems except the one of how to get everybody to use the  
correct solution", and it is the standard definition of "solved".

I.e., everything's solved except the marketing.

MfG Kai

Re: Hardwired drivers are going away?

[John Levon]

On Mon, Jan 21, 2002 at 12:53:28PM +1100, Keith Owens wrote:

> >How often have we seen nonsensical backtraces here because
> >modules were involved?   Possibly we can include a table
> >of module base addresses in the Oops output and teach ksymoops
> >about it.
> 
> You see nonsensical backtraces because people persist in using the oops
> decode option of klogd which is broken when faced with modules.  Turn
> off klogd oops (klogd -x) and you get a raw backtrace which ksymoops
> can handle.  Guess why these entries are in /proc/ksyms?
> 
> c48a2300 __insmod_3c589_cs_S.bss_L4	[3c589_cs]

and quite often the user has unloaded / loaded modules in the meantime
and the oops is useless.

It would be nice if klogd's oops detection just passed everything to ksymoops
untouched, and stored everything somewhere using -m

I don't see any reason why the internal profiler can't have an extended EIP
range to catch module samples either. Perhaps I'm missing something. Perhaps
no one cares enough ...

regards
john

-- 
"I hope you will find the courage to keep on living 
 despite the existence of this feature."
	- Richard Stallman

Re: Hardwired drivers are going away?

[Keith Owens]

On Mon, 21 Jan 2002 02:13:55 +0000, 
John Levon <movement@marcelothewonderpenguin.com> wrote:
>On Mon, Jan 21, 2002 at 12:53:28PM +1100, Keith Owens wrote:
>> Guess why these entries are in /proc/ksyms?
>> 
>> c48a2300 __insmod_3c589_cs_S.bss_L4	[3c589_cs]
>
>and quite often the user has unloaded / loaded modules in the meantime
>and the oops is useless.

/var/log/ksymoops.  I added the code and documented it nicely, man
insmod or ksymoops.  It's not my fault if nobody reads the docs!

>It would be nice if klogd's oops detection just passed everything to ksymoops
>untouched, and stored everything somewhere using -m

It would be better if klogd got out of the way completely.  Everything
is stored, just created /var/log/ksymoops.

>I don't see any reason why the internal profiler can't have an extended EIP
>range to catch module samples either. Perhaps I'm missing something. Perhaps
>no one cares enough ...

IMNSHO the profiler should use kallsyms.  Then it gets every symbol
from the kernel and from all modules, instead of only being able to
report on exported symbols.

int kallsyms_address_to_symbol(
        unsigned long     address,      /* Address to lookup */
        const char      **mod_name,     /* Set to module name */
        unsigned long    *mod_start,    /* Set to start address of module */
        unsigned long    *mod_end,      /* Set to end address of module */
        const char      **sec_name,     /* Set to section name */
        unsigned long    *sec_start,    /* Set to start address of section */
        unsigned long    *sec_end,      /* Set to end address of section */
        const char      **sym_name,     /* Set to full symbol name */
        unsigned long    *sym_start,    /* Set to start address of symbol */
        unsigned long    *sym_end       /* Set to end address of symbol */
        )

We have the technology.

Re: Hardwired drivers are going away?

[Thomas Zimmerman]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On 21-Jan 01:31, Keith Owens wrote:
> On Mon, 21 Jan 2002 02:13:55 +0000, 
> John Levon <movement@marcelothewonderpenguin.com> wrote:
> >On Mon, Jan 21, 2002 at 12:53:28PM +1100, Keith Owens wrote:
> >> Guess why these entries are in /proc/ksyms?
> >> 
> >> c48a2300 __insmod_3c589_cs_S.bss_L4	[3c589_cs]
> >
> >and quite often the user has unloaded / loaded modules in the meantime
> >and the oops is useless.
> 
> /var/log/ksymoops.  I added the code and documented it nicely, man
> insmod or ksymoops.  It's not my fault if nobody reads the docs!
> 
> >It would be nice if klogd's oops detection just passed everything to ksymoops
> >untouched, and stored everything somewhere using -m
> 
> It would be better if klogd got out of the way completely.  Everything
> is stored, just created /var/log/ksymoops.

Isn't part of the problem with klogd mangling is that is sort of works?
Change the format so that klogd doesn't touch it and that gets _wrong_ Oops
reports out of the way. [you do great work, now make someone else do some
too!] 

Just my $0.02, from a thankful user.

Thomas

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Hardwired drivers are going away?

[Horst von Brand]

Andrew Morton <akpm@zip.com.au> said:
> "Mr. James W. Laferriere" wrote:
> >         This is just what the Heads are trying to do away with .  There
> >         will only be module enabled kernels .  JimL

> I suspect none of these "Heads" spend much time in protracted
> email debug sessions.  Because the *first* thing you do is
> ask the tester to compile the relevant driver into the
> kernel.
> 
> The problems which the removal of this option will cause include:
> 
> 1: Inability to look up symbols in the kernel elf image.

ksymoops(8) is able to find them...

> 2: Breaks the kernel profiler

Which one?

> 3: breaks kgdb

Officially not wanted by the heads, so irrelevant

> 4: breaks ksymoops.

How so?
-- 
Horst von Brand			     http://counter.li.org # 22616

Re: Hardwired drivers are going away?

[John Levon]

On Mon, Jan 21, 2002 at 10:34:07AM +0100, Horst von Brand wrote:

> > 2: Breaks the kernel profiler
> 
> Which one?

readprofile(3) - the histogram only extends across the .text of the monolithic
kernel image

regards
john

-- 
"I hope you will find the courage to keep on living 
 despite the existence of this feature."
	- Richard Stallman

Re: Hardwired drivers are going away?

[Anton Altaparmakov]

At 01:07 21/01/02, Mr. James W. Laferriere wrote:
>On Mon, 21 Jan 2002, Anton Altaparmakov wrote:
> > At 23:20 20/01/02, Frank van de Pol wrote:
> > >On Sat, Jan 19, 2002 at 10:22:43AM +1100, Keith Owens wrote:
> > > > On Fri, 18 Jan 2002 17:20:02 -0500 (EST),
> > > > "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
> > > > >     Linux doesn't have a method to load encrypted & signed modules at
> > > > >     this time .
> > > > And never will.  Who loads the module - root.  Who maintains the list
> > > > of signatures - root.  Who controls the code that verifies the
> > > > signature - root.
> > > > Your task Jim, should you choose to accept it, is to make the kernel
> > > > distinguish between a good use of root and a malicious use by some who
> > > > has broken in and got root privileges.  When you can do that, then we
> > > > can add signed modules.
> > >If you want to secure your box, why don't you simply put a lock on it and
> > >throw away the key? Really, what might help the paranoid admins in 
> this case
> > >is a setting in the kernel which basically disables the ability to load or
> > >unload modules. Of course once set this setting can not been turned with
> > >rebooting the box.
>
> > Er that sounds like just disabling modules in the kernel altogether (kernel
> > compile option exists for this since the beginning of time)... I do that on
> > all servers I control. Not only for security reasons but also because I
> > suspect it produces smaller and probably faster kernels (I haven't tested
> > this in any way, just a guess).
>         This is just what the Heads are trying to do away with .  There
>         will only be module enabled kernels .  JimL

Ah! Serves me right for jumping into a thread without reading the previous 
posts... I better shut up as I have no idea why this is being done. From my 
unknowledgeable point of view it seems daft but there probably are good 
reasons... I suppose even non-modular kernels can be modified by the root 
user to be exploited, it's just a little bit harder...

Best regards,

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: Hardwired drivers are going away?

[Alexander Viro]

On Mon, 14 Jan 2002, David Lang wrote:

> I can see a couple reasons for building a kernel without useing modules.
> 
> 1. security, if you don't need any modules you can disable modules entirly
> and then it's impossible to add a module without patching the kernel first
> (the module load system calls aren't there)

Oh, please...  Give me permissions sufficient to make these syscalls
and in a couple of minutes your kernel will be replaced with ELIZA.
As a bonus - ELIZA running under TOPS-10 on PDP-10 emulator.  And talking
to PARRY.

Anyway, it's trivial to disable said system calls just before doing
execve("/sbin/init",...).  It won't buy you any security, but if you
insist...

> 2. speed, there was a discussion a few weeks ago pointing out that there
> is some overhead for useing modules (far calls need to be used just in
> case becouse the system can't know where the module will be located IIRC)

_That_ has to be addressed - regardless of anything else, if that suckitude
can be fixed it should be.

> 3. simplicity in building kernels for other machines. with a monolithic
> kernel you have one file to move (and a bootloader to run) with modules
> you have to move quite a few more files.

FVO"quite a few" equal to 2.  Kernel and initramfs.cpio.gz

Re: Hardwired drivers are going away?

[Aaron Lehmann]

On Mon, Jan 14, 2002 at 10:50:20AM -0800, David Lang wrote:
> I can see a couple reasons for building a kernel without useing modules.

I agree with all of yours. IMHO, the proposed scheme for 2.5 is plain
bad. It will require an initrd (initramfs, or whatever), and force
kernel installation to be more difficult. The performance overhead
sounds like a major downside, epsecially when currently people know
what hardware they have and build things into the kernel accordingly.

Between this and CML2, my mental image of 2.5+ is starting to look
very grim.

Re: Hardwired drivers are going away?

[David Lang]

CML2 is a tradeoff in the kernel config process. it (currently) trades
time for correctness. It has no effect after you have built the kernel.

I like this becouse there have been several times where I have had to go
on a scavanger hunt to figure out what option I need to turn on before I
can turn on another option (something that CML2 will fix)

as for the kbuild 2.5 performance 'penalty', is it really slower then
always doign make dep; make clean; make bzImage? Yes I know I don't always
have to do that, but I don't understand the rules as to when I do and when
I don't (they sound simple at first but there keep being exceptions
raised). having the new kernel build process always get this right is
valuble.

also the time penalties associated with these two options is something
that can (and will) be dealt with over time as they get optimized and
rewritten to be faster.

these are VERY different from the modules thing which is permanently
introducing additional penalties into the system that can affect it for
as long as it's running.

David Lang


 On Tue, 15 Jan 2002, Aaron Lehmann wrote:

> On Mon, Jan 14, 2002 at 10:50:20AM -0800, David Lang wrote:
> > I can see a couple reasons for building a kernel without useing modules.
>
> I agree with all of yours. IMHO, the proposed scheme for 2.5 is plain
> bad. It will require an initrd (initramfs, or whatever), and force
> kernel installation to be more difficult. The performance overhead
> sounds like a major downside, epsecially when currently people know
> what hardware they have and build things into the kernel accordingly.
>
> Between this and CML2, my mental image of 2.5+ is starting to look
> very grim.
>

Re: Hardwired drivers are going away?

[Aaron Lehmann]

On Tue, Jan 15, 2002 at 10:17:29AM -0800, David Lang wrote:
> CML2 is a tradeoff in the kernel config process. it (currently) trades
> time for correctness. It has no effect after you have built the kernel.

My gripe is more that I find it ugly. I'm very used to the menuconfig
appearance, and I do not appreciate the interface change.

And (we've been through this before) I am *NOT* going to install yet
another language runtime just to configure a kernel.

Re: Hardwired drivers are going away?

[Mr. James W. Laferriere]

	Hello All ,  And on that day linux will loose one small person .
		Taf ,  JimL

On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Alan Cox <alan@lxorguk.ukuu.org.uk>:
> > For 2.5 if things go to plan there will be no such thing as a "compiled in"
> > driver. They simply are not needed with initramfs holding what were once the
> > "compiled in" modules.
>
> This is something of a bombshell.  Not necessarily a bad one, but...
>
> Alan, do you have *any* *freakin'* *idea* how much more complicated
> the CML2 deduction engine had to be because the basic logical entity
> was a tristate rather than a bool?  If this plan goes through, I'm
> going to be able to drop out at least 20% of the code, with most of
> that 20% being in the nasty complicated bits where the maintainability
> improvement will be greatest.  And I can get rid of the nasty "vitality"
> flag, which probably the worst wart on the language.
>
> Yowza...so how soon is this supposed to happen?
> --
> 		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>
>
> Government should be weak, amateurish and ridiculous. At present, it
> fulfills only a third of the role.	-- Edward Abbey
>

       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Hardwired drivers are going away?

[Alan Cox]

> that 20% being in the nasty complicated bits where the maintainability
> improvement will be greatest.  And I can get rid of the nasty "vitality"
> flag, which probably the worst wart on the language.
> 
> Yowza...so how soon is this supposed to happen?

Subject to availability. No contract implied or etc.. Its something to
tackle after the rest of initramfs works. Even if not then the lsmod case
can be made to work since its just a matter of putting the names in a segment
for the linker to collate.

We already have module_init/module_exit which is a big chunk of the cleanup.

Re: Hardwired drivers are going away?

[Eric S. Raymond]

Alan Cox <alan@lxorguk.ukuu.org.uk>:
> > Yowza...so how soon is [the end of hard-compiled drivers] supposed to 
> > happen?
> 
> Subject to availability. No contract implied or etc.. Its something to
> tackle after the rest of initramfs works.

Dang.  This will make the CML2 inference engine work better in some
funky corner cases, too.  And its behavior will be easier to
understand all around.

Sign me up.  This will be a good change; I like it when I can make things
better by taking features *out* of my code.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

Every Communist must grasp the truth, 'Political power grows out of
the barrel of a gun.'
        -- Mao Tse-tung, 1938, inadvertently endorsing the Second Amendment.

Re: Hardwired drivers are going away?

[Alexander Viro]

On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Alan Cox <alan@lxorguk.ukuu.org.uk>:
> > For 2.5 if things go to plan there will be no such thing as a "compiled in"
> > driver. They simply are not needed with initramfs holding what were once the
> > "compiled in" modules.
> 
> This is something of a bombshell.  Not necessarily a bad one, but...
> 
> Alan, do you have *any* *freakin'* *idea* how much more complicated
> the CML2 deduction engine had to be because the basic logical entity
> was a tristate rather than a bool?  If this plan goes through, I'm
> going to be able to drop out at least 20% of the code, with most of
> that 20% being in the nasty complicated bits where the maintainability
> improvement will be greatest.  And I can get rid of the nasty "vitality"
> flag, which probably the worst wart on the language.
> 
> Yowza...so how soon is this supposed to happen?

Two technical obstacles:
	a) on some architecures modular code is slower (IIRC, the problem is
with medium-range calls being faster than far ones and usable only in the
kernel proper).  We probaly want to leave a gap after the .text and remap
.text of module in there - if I understand the problem that should be
enough, but that's really a question to folks dealing with these ports (PPC64
and Itanic?)
	b) current differences between options parsing in case of built-in
and modular drivers.

The rest is trivial and should be done in 2.5.4-pre* or so.

But it still leaves you with tristate - instead of yes/module/no it's
yes/yes, but don't put it on initramfs/no.  However, dependencies become
simpler - all you need is "I want this, that and that on initramfs" and
the rest can be found by depmod (i.e. configurator doesn't have to deal
with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
(== can't be M)").

Re: Hardwired drivers are going away?

[Rob Landley]

On Monday 14 January 2002 02:09 pm, Alexander Viro wrote:

> But it still leaves you with tristate - instead of yes/module/no it's
> yes/yes, but don't put it on initramfs/no.  However, dependencies become
> simpler - all you need is "I want this, that and that on initramfs" and
> the rest can be found by depmod (i.e. configurator doesn't have to deal
> with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
> (== can't be M)").

This is something I've wondered about and would like to ask for clarification 
on: the relationship between the initramfs image and the kernel, build 
process-wise.

How much of the build process for the initramfs will be integrated with the 
kernel build?  Since the kernel won't boot without a matching initramfs, I 
take it that some kind of initramfs will be a kernel build target now?

There's been a lot of talk about having the source for a mini-libc (uclibc, 
dietlibc, some combo) in the kernel tree, and other people saying we should 
just grab the binary for build purposes.  The most obvious model I can think 
of for klibc staying seperate from the kernel is the user-space 
pcmcia/cardbus hotplug stuff, but that DID get integrated into the kernel.

The klibc source/binary debate still seems to be ongoing, but apart from 
that, will the build process for initramfs be part of the kernel build or not?

Rob.

Re: Hardwired drivers are going away?

[Greg KH]

On Mon, Jan 14, 2002 at 09:19:01AM -0500, Rob Landley wrote:
> 
> The klibc source/binary debate still seems to be ongoing, but apart from 
> that, will the build process for initramfs be part of the kernel build or not?

I would vote for yes.  I would like this, if for no other reason than
the number of kernel builds most kernel developers do :)

I was going to wait until kbuild 2.5 went into the tree before looking
at what would be needed to add this to the build process.

thanks,

greg k-h

Re: Hardwired drivers are going away?

[Keith Owens]

On Mon, 14 Jan 2002 09:19:01 -0500, 
Rob Landley <landley@trommello.org> wrote:
>How much of the build process for the initramfs will be integrated with the 
>kernel build?  Since the kernel won't boot without a matching initramfs, I 
>take it that some kind of initramfs will be a kernel build target now?
>
>There's been a lot of talk about having the source for a mini-libc (uclibc, 
>dietlibc, some combo) in the kernel tree, and other people saying we should 
>just grab the binary for build purposes.  The most obvious model I can think 
>of for klibc staying seperate from the kernel is the user-space 
>pcmcia/cardbus hotplug stuff, but that DID get integrated into the kernel.
>
>The klibc source/binary debate still seems to be ongoing, but apart from 
>that, will the build process for initramfs be part of the kernel build or not?

I am in two minds about this (but I am at one with my duality).  Part
of me says that users will want to tweak the initramfs process and they
may want to use different versions of klibc, so klibc and building
initramfs should be outside the kernel build.  Another part says that
users have to build initramfs before doing the install so initramfs and
klibc should be part of kbuild.

It will probably end up with klibc as part of the kernel tarball
supported by kbuild.  Generating initramfs will be done by a script,
kbuild will supply a sample but users can copy and change the sample to
suit their own requirements.

Re: Hardwired drivers are going away?

[Oliver Xymoron]

On Tue, 15 Jan 2002, Keith Owens wrote:

> On Mon, 14 Jan 2002 09:19:01 -0500,
> Rob Landley <landley@trommello.org> wrote:
> >How much of the build process for the initramfs will be integrated with the
> >kernel build?  Since the kernel won't boot without a matching initramfs, I
> >take it that some kind of initramfs will be a kernel build target now?
> >
> >There's been a lot of talk about having the source for a mini-libc (uclibc,
> >dietlibc, some combo) in the kernel tree, and other people saying we should
> >just grab the binary for build purposes.  The most obvious model I can think
> >of for klibc staying seperate from the kernel is the user-space
> >pcmcia/cardbus hotplug stuff, but that DID get integrated into the kernel.
> >
> >The klibc source/binary debate still seems to be ongoing, but apart from
> >that, will the build process for initramfs be part of the kernel build or not?
>
> I am in two minds about this (but I am at one with my duality).  Part
> of me says that users will want to tweak the initramfs process and they
> may want to use different versions of klibc, so klibc and building
> initramfs should be outside the kernel build.  Another part says that
> users have to build initramfs before doing the install so initramfs and
> klibc should be part of kbuild.
>
> It will probably end up with klibc as part of the kernel tarball
> supported by kbuild.  Generating initramfs will be done by a script,
> kbuild will supply a sample but users can copy and change the sample to
> suit their own requirements.

Better yet, we should do away with all install targets beyond assembling
the bootable images and provide sample install scripts in /scripts. Most
of it is already at odds with the distro's install and the remainder
becomes a maintenance issue with initramfs. Distro folks are going to be
working on initramfs-libc for their boot disks anyway and will undoubted
want to tweak such stuff greatly. Let's include Viro's
minimal-userspace-boot.c in scripts along with a
build-minimal-initramfs.sh and leave it at that.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: Hardwired drivers are going away?

[Alan Cox]

> 	a) on some architecures modular code is slower (IIRC, the problem is
> with medium-range calls being faster than far ones and usable only in the
> kernel proper).  We probaly want to leave a gap after the .text and remap

Also with TLB mapping sizes (4Mb versus 4K)

> .text of module in there - if I understand the problem that should be
> enough, but that's really a question to folks dealing with these ports (PPC64
> and Itanic?)

For x86 you just need to leave a nice chunk of physical memory that is there
to copy the module text/data into as you insmod them during boot then give
the rest of the pool back to the paging system. This also solves the problem
with "can't be a module because I need 1Mb of linear space" drivers.

Alan

Re: Hardwired drivers are going away?

[Eric S. Raymond]

Alexander Viro <viro@math.psu.edu>:
> But it still leaves you with tristate - instead of yes/module/no it's
> yes/yes, but don't put it on initramfs/no.  However, dependencies become
> simpler - all you need is "I want this, that and that on initramfs" and
> the rest can be found by depmod (i.e. configurator doesn't have to deal
> with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
> (== can't be M)").

Actually I think we may no longer be in tristate-land.  Instead, some
devices have the property "This belongs in initramfs if it's configured
at all" -- specifically, drivers for potential boot devices.  Everything
else can dynamic-load after boot time.  

In CML2 you can assign a symbol properties, which are written into
trailing comments in the config file on the same line as the symbol
value assignment.  One such property, "PRIVATE", is already
interpreted by the postprocessor that generates autoconf.h from the
configuration output; it prevents the symbol from being written to
autoconf.h.

The critical-for-boot property could be interpreted by the same
postprocessor script and turned into a manifest for initramfs.  There
would be no need for the inference engine or configurator to know
about this property at all, just as it doesn't need to know anything
about PRIVATE.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

"To disarm the people... was the best and most effectual way to enslave them."
        -- George Mason, speech of June 14, 1788

Re: Hardwired drivers are going away?

[Rob Landley]

On Monday 14 January 2002 03:17 pm, Eric S. Raymond wrote:
> Alexander Viro <viro@math.psu.edu>:
> > But it still leaves you with tristate - instead of yes/module/no it's
> > yes/yes, but don't put it on initramfs/no.  However, dependencies become
> > simpler - all you need is "I want this, that and that on initramfs" and
> > the rest can be found by depmod (i.e. configurator doesn't have to deal
> > with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
> > (== can't be M)").
>
> Actually I think we may no longer be in tristate-land.  Instead, some
> devices have the property "This belongs in initramfs if it's configured
> at all" -- specifically, drivers for potential boot devices.  Everything
> else can dynamic-load after boot time.

No.  Just about every block device there is is a POTENTIAL root device.  Boot 
and root devices are seperate things (just as boot and root partitions are 
seperate things.)  Your kernel can decompress from an enternet boot prom if 
you're clever enough about.  You could theoretically have your root device on 
a USB storage device.  Booting from CD-ROM is fairly common, yet not 
something you normally want your installed kernel to have compiled in.

I've even seen kernels that load via tftp, including an initial ram disk, and 
either never have local storage of any kind or use it purely as swap space.  
(Why?  Really really really easy system administration of multiple terminals 
in libraries and kiosks and such.  It breaks, you drop in a new one (don't 
even have to image it).  No local hard drive liminates the #1 cause of system 
failure.  And there's no way an end-user can screw up its configuration in a 
way a reboot won't fix.)

What's interesting is what devices the root partition IS going to live on and 
this has NOTHING to do with the device itself.  It can be any device any 
normal partition lives on.  You can have IDE and SCSI in the same system.  
You can have hard drives, CD-ROM drives, and floppies.  The fact you can boot 
off of an IDE cd-rom doesn't mean you will, or that's how you want to 
configure your kernel.

When manually configuring, you select this.  And if the autoprober isn't 
smart enough to figure out what your root device is, it needs to be fixed.

Rob

Re: Hardwired drivers are going away?

[David Lang]

some systems don't want modules to be loaded after boot (yes it's security
that can be bypassed, but as long as it's not your only security it's
still worth doing)

also with hardwired drivers going away does this mean that other things
that are currently compiled in will become modules, like say the IP stack?

David Lang

 On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Date: Mon, 14 Jan 2002 15:17:48 -0500
> From: Eric S. Raymond <esr@thyrsus.com>
> To: Alexander Viro <viro@math.psu.edu>
> Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>,
>      Mr. James W. Laferriere <babydr@baby-dragons.com>,
>      Giacomo Catenazzi <cate@debian.org>,
>      Linux Kernel List <linux-kernel@vger.kernel.org>
> Subject: Re: Hardwired drivers are going away?
>
> Alexander Viro <viro@math.psu.edu>:
> > But it still leaves you with tristate - instead of yes/module/no it's
> > yes/yes, but don't put it on initramfs/no.  However, dependencies become
> > simpler - all you need is "I want this, that and that on initramfs" and
> > the rest can be found by depmod (i.e. configurator doesn't have to deal
> > with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
> > (== can't be M)").
>
> Actually I think we may no longer be in tristate-land.  Instead, some
> devices have the property "This belongs in initramfs if it's configured
> at all" -- specifically, drivers for potential boot devices.  Everything
> else can dynamic-load after boot time.
>
> In CML2 you can assign a symbol properties, which are written into
> trailing comments in the config file on the same line as the symbol
> value assignment.  One such property, "PRIVATE", is already
> interpreted by the postprocessor that generates autoconf.h from the
> configuration output; it prevents the symbol from being written to
> autoconf.h.
>
> The critical-for-boot property could be interpreted by the same
> postprocessor script and turned into a manifest for initramfs.  There
> would be no need for the inference engine or configurator to know
> about this property at all, just as it doesn't need to know anything
> about PRIVATE.
> --
> 		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>
>
> "To disarm the people... was the best and most effectual way to enslave them."
>         -- George Mason, speech of June 14, 1788
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Hardwired drivers are going away?

[Oliver Xymoron]

On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Alexander Viro <viro@math.psu.edu>:
> > But it still leaves you with tristate - instead of yes/module/no it's
> > yes/yes, but don't put it on initramfs/no.  However, dependencies become
> > simpler - all you need is "I want this, that and that on initramfs" and
> > the rest can be found by depmod (i.e. configurator doesn't have to deal
> > with "FOO goes on initramfs (== old Y), so BAR and BAZ must go there
> > (== can't be M)").
>
> Actually I think we may no longer be in tristate-land.  Instead, some
> devices have the property "This belongs in initramfs if it's configured
> at all" -- specifically, drivers for potential boot devices.  Everything
> else can dynamic-load after boot time.

The knowledge of what's required to boot can and should be moved out of
the kernel to distro land. 'make install' should call the distro's install
script that sez /boot and /lib are on ext3 on /dev/sd?, ergo I need to
include the ext3 and relevant SCSI modules.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: Hardwired drivers are going away?

[Horst von Brand]

"Eric S. Raymond" <esr@thyrsus.com> said:

[...]

> Actually I think we may no longer be in tristate-land.  Instead, some
> devices have the property "This belongs in initramfs if it's configured
> at all" -- specifically, drivers for potential boot devices.  Everything
> else can dynamic-load after boot time.  

Then all SCSI drivers end up in the initramfs for the install kernel for
a distro? There might be _many_ devices configured that don't need to
reside on the initramfs.

OTOH, the initramfs will get built and populated by separate tools (just
like initrd is done today), so the alternatives would boil down to M/N.
_If_ the grand scheme of getting rid of builtins flies. I for one wouldn't
hold my breath.
-- 
Horst von Brand			     http://counter.li.org # 22616

Re: Hardwired drivers are going away?

[Eric S. Raymond]

Horst von Brand <brand@jupiter.cs.uni-dortmund.de>:
> > Actually I think we may no longer be in tristate-land.  Instead, some
> > devices have the property "This belongs in initramfs if it's configured
> > at all" -- specifically, drivers for potential boot devices.  Everything
> > else can dynamic-load after boot time.  
> 
> Then all SCSI drivers end up in the initramfs for the install kernel for
> a distro? There might be _many_ devices configured that don't need to
> reside on the initramfs.

No worries.  I have working code for a better approach now.  It turns
out not to be that hard to discover the root device, controller, and bus.
-- 
		<a href="http://www.tuxedo.org/~esr/">Eric S. Raymond</a>

The end move in politics is always to pick up a gun.
	-- R. Buckminster Fuller

Re: Hardwired drivers are going away?

[Rusty Russell]

On Mon, 14 Jan 2002 14:09:16 -0500 (EST)
Alexander Viro <viro@math.psu.edu> wrote:
> Two technical obstacles:
> 	a) on some architecures modular code is slower (IIRC, the problem is
> with medium-range calls being faster than far ones and usable only in the
> kernel proper).  We probaly want to leave a gap after the .text and remap
> .text of module in there - if I understand the problem that should be
> enough, but that's really a question to folks dealing with these ports (PPC64
> and Itanic?)

Sparc64 has a special allocator.  PPC32 is not so lucky, and if the module
doesn't end up under 32MB we use trampolines (PLT in the ppc parlance).
PPC64 has similar issues.

> 	b) current differences between options parsing in case of built-in
> and modular drivers.

I have a patch for this, of course, but it's tied to my replacement module
loader:

	http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Module/param.patch.bz2

Cheers!
Rusty.
-- 
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: Hardwired drivers are going away?

[Kai Henningsen]

aaronl@vitelus.com (Aaron Lehmann)  wrote on 15.01.02 in <20020115172413.GD7030@vitelus.com>:

> I agree with all of yours. IMHO, the proposed scheme for 2.5 is plain
> bad. It will require an initrd (initramfs, or whatever), and force
> kernel installation to be more difficult. The performance overhead
> sounds like a major downside, epsecially when currently people know
> what hardware they have and build things into the kernel accordingly.

Installation more difficult? Significant performance overhead?

What *are* you smoking?

MfG Kai

Re: Hardwired drivers are going away?

[Dr. Kelsey Hudson]

This bothers me. I prefer building a monolithic kernel with only a few 
drivers compiled as modules (e.g. those drivers which are under active 
development and are upgraded often). I believe firmly that certain drivers 
should be part of the monolithic kernel.... either way, i'm not about to 
debate it. i'd just like to know exactly why this is being done, and to 
lodge an objection against it.

On Mon, 14 Jan 2002, Eric S. Raymond wrote:

> Alan Cox <alan@lxorguk.ukuu.org.uk>:
> > For 2.5 if things go to plan there will be no such thing as a "compiled in"
> > driver. They simply are not needed with initramfs holding what were once the
> > "compiled in" modules.
> 
> This is something of a bombshell.  Not necessarily a bad one, but...
> 
> Alan, do you have *any* *freakin'* *idea* how much more complicated
> the CML2 deduction engine had to be because the basic logical entity
> was a tristate rather than a bool?  If this plan goes through, I'm
> going to be able to drop out at least 20% of the code, with most of
> that 20% being in the nasty complicated bits where the maintainability
> improvement will be greatest.  And I can get rid of the nasty "vitality"
> flag, which probably the worst wart on the language.
> 
> Yowza...so how soon is this supposed to happen?
> 

-- 
 Kelsey Hudson                                           khudson@ctica.com 
 Software Engineer
 Compendium Technologies, Inc                               (619) 725-0771
---------------------------------------------------------------------------