Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Nick Craig-Wood]

Zan Lynx <zlynx@acm.org> wrote:
>  There *are* crashes from LVM and ext3.  I had to change kernels to avoid
>  them.
> 
>  I had crashes with ext3 on LVM snapshot on DM mirror on SATA.

We've noticed these too... ext3/LVM/raid0/sata seems fine.  If you add
snapshot in that mix then it becomes rather unreliable.

-- 
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Thu, 19 Jul 2007 03:33:58 +0200
> Andrea Arcangeli <andrea@suse.de> wrote:

>> > 8K stacks without IRQ stacks are not "safer" so I don't understand your
>> > comment ?
>> 
>> Ouch, see the reports about 4k stack crashes. I agree they're not
>> safe w/o irq stacks (like on x86-64), but they're generally safer.
> 
> Still don't follow. How is "exceeds stack space but less likely to be
> noticed" safer.

If there is a tree in the forest, is it as likely to fall as the three
that's being chopped in front of our eyes? It is, because each tree will
fall eventually, but you'd still not allow your kids to play on the
tree being chopped, but you'd probably allow them to climb that other tree
like all the other kids do.

The same applies to the stack: We don't know if or when we'll see all
possible interrupts fire and kill the 8K stack, but we know for sure the
8K stack has been climbed for years and there is an axe on that 4K stack.
So where do you send the users to play?
-- 
What's worse than a Male Chauvinist Pig?
A woman that won't do what she's told.

Friß, Spammer: Fq@Bbu29.7eggert.dyndns.org od1Gkubx@hrBky.7eggert.dyndns.org

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Nick Craig-Wood]

Nick Craig-Wood <nick@craig-wood.com> wrote:
>  Zan Lynx <zlynx@acm.org> wrote:
> >  There *are* crashes from LVM and ext3.  I had to change kernels to avoid
> >  them.
> > 
> >  I had crashes with ext3 on LVM snapshot on DM mirror on SATA.
> 
>  We've noticed these too... ext3/LVM/raid0/sata seems fine.  If you add
>  snapshot in that mix then it becomes rather unreliable.

I meant raid1 up there not raid0!

-- 
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

Matt Mackall <mpm@selenic.com> wrote:
> On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:

>> As far as I'm aware, the actual reason for 4K stacks is that after the
>> system has been up and running for some time getting "1 physically
>> contiguous pages" becomes significantly easier than 2 which wouldn't be
>> arbitrary.
> 
> If there are exactly two free pages in the system, the odds of them
> being buddies (ie adjacent AND properly aligned) is quite small. The
> available page pool has to grow quite a bit before the availability of
> order-1 page pairs approaches 100%.

If there are exactly two free pages in a system, the odds of starting any
program are not very good. You'll have to swap, and if you do, you can swap
two more pages in order to free enough RAM for the stack.
-- 
The secret of the universe is #@*%! NO CARRIER 

Friß, Spammer: TqB@mxlqnP.7eggert.dyndns.org -kplBVgN@7eggert.dyndns.org
 BpRVhx@1Q.7eggert.dyndns.org ozy0XBmkB@m1Sh.7eggert.dyndns.org

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Arjan van de Ven]

On Sun, 2007-07-15 at 19:17 +0200, Bodo Eggert wrote:
> Matt Mackall <mpm@selenic.com> wrote:
> > On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:
> 
> >> As far as I'm aware, the actual reason for 4K stacks is that after the
> >> system has been up and running for some time getting "1 physically
> >> contiguous pages" becomes significantly easier than 2 which wouldn't be
> >> arbitrary.
> > 
> > If there are exactly two free pages in the system, the odds of them
> > being buddies (ie adjacent AND properly aligned) is quite small. The
> > available page pool has to grow quite a bit before the availability of
> > order-1 page pairs approaches 100%.
> 
> If there are exactly two free pages in a system, the odds of starting any
> program are not very good. You'll have to swap, and if you do, you can swap
> two more pages in order to free enough RAM for the stack.


even if you have several thousand pages the odds aren't good; or rather,
they start out reasonably ok until something starts eating very
deliberately at the 8k pages pool, for example the new app creating
about 10 threads....

The 4K issue is "tricky", only a few selected workload/compiler combos
seem to hit the dirt, yet distros like Fedora and RHEL use 4K stacks
since forever, and if it gave massive problems they wouldn't do that.
On the upside, especially on very-threaded workloads, it helps
reliability and the VM a lot...

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Måns Rullgård]

Arjan van de Ven <arjan@infradead.org> writes:

> On Sun, 2007-07-15 at 19:17 +0200, Bodo Eggert wrote:
>> Matt Mackall <mpm@selenic.com> wrote:
>> > On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:
>> 
>> >> As far as I'm aware, the actual reason for 4K stacks is that after the
>> >> system has been up and running for some time getting "1 physically
>> >> contiguous pages" becomes significantly easier than 2 which wouldn't be
>> >> arbitrary.
>> > 
>> > If there are exactly two free pages in the system, the odds of them
>> > being buddies (ie adjacent AND properly aligned) is quite small. The
>> > available page pool has to grow quite a bit before the availability of
>> > order-1 page pairs approaches 100%.
>> 
>> If there are exactly two free pages in a system, the odds of starting any
>> program are not very good. You'll have to swap, and if you do, you can swap
>> two more pages in order to free enough RAM for the stack.
>
> even if you have several thousand pages the odds aren't good; or rather,
> they start out reasonably ok until something starts eating very
> deliberately at the 8k pages pool, for example the new app creating
> about 10 threads....
>
> The 4K issue is "tricky", only a few selected workload/compiler combos
> seem to hit the dirt, yet distros like Fedora and RHEL use 4K stacks
> since forever, and if it gave massive problems they wouldn't do that.
> On the upside, especially on very-threaded workloads, it helps
> reliability and the VM a lot...

I guess no Fedora users run md+lvm+xfs then.  That combination has
quite reliably crashed any 4k-stack kernel I've ever cared to try.

-- 
Måns Rullgård
mans@mansr.com

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/15/2007 07:17 PM, Bodo Eggert wrote:

> Matt Mackall <mpm@selenic.com> wrote:
>> On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:
> 
>>> As far as I'm aware, the actual reason for 4K stacks is that after the
>>> system has been up and running for some time getting "1 physically
>>> contiguous pages" becomes significantly easier than 2 which wouldn't be
>>> arbitrary.
>> 
>> If there are exactly two free pages in the system, the odds of them
>> being buddies (ie adjacent AND properly aligned) is quite small. The
>> available page pool has to grow quite a bit before the availability of
>> order-1 page pairs approaches 100%.
> 
> If there are exactly two free pages in a system, the odds of starting any
> program are not very good. You'll have to swap, and if you do, you can swap
> two more pages in order to free enough RAM for the stack.

A thread's kernel stack is a kernel allocation. If you'd fail to allocate it 
you'd supposedly _already_ have swapt out everything that could be swapped out.

Moreoveover -- literally two pages free was hardly his point. The point is 
just that (with a page being the allocation unit) single page allocations 
are guaranteed to succeed if _any_ memory is free, while two adjacent (yes, 
and stacksize aligned) pages will be pretty hard to get by once the system 
has been up and running for some time.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Sun, 15 Jul 2007, Rene Herman wrote:
> On 07/15/2007 07:17 PM, Bodo Eggert wrote:
> > Matt Mackall <mpm@selenic.com> wrote:
> >> On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:

> >>> As far as I'm aware, the actual reason for 4K stacks is that after the
> >>> system has been up and running for some time getting "1 physically
> >>> contiguous pages" becomes significantly easier than 2 which wouldn't be
> >>> arbitrary.
> >> 
> >> If there are exactly two free pages in the system, the odds of them
> >> being buddies (ie adjacent AND properly aligned) is quite small. The
> >> available page pool has to grow quite a bit before the availability of
> >> order-1 page pairs approaches 100%.
> > 
> > If there are exactly two free pages in a system, the odds of starting any
> > program are not very good. You'll have to swap, and if you do, you can swap
> > two more pages in order to free enough RAM for the stack.
> 
> A thread's kernel stack is a kernel allocation. If you'd fail to allocate it 
> you'd supposedly _already_ have swapt out everything that could be swapped out.

So we are in a desperate situation, we can almost make no progress, adding 
another task is going to push the system into an unrecoverable situation, 
and we make sure this task can be started.-)

> Moreoveover -- literally two pages free was hardly his point. The point is 
> just that (with a page being the allocation unit) single page allocations 
> are guaranteed to succeed if _any_ memory is free, while two adjacent (yes, 
> and stacksize aligned) pages will be pretty hard to get by once the system 
> has been up and running for some time.

That never happened on my servers, therefore I'd opt for the little extra 
security of having spare 4k on the stack. (I made a patch which would 
printk a message if allocating a stack would ever fail).

I'm not at all opposed to letting the guys with zillions of threads 
benefit from having less unused kernel stack, but unless it's secure for
all users, it should not be default=y.

-- 
A beggar walked up to a well-dressed woman shopping on Rodeo
Drive and said, "I haven't eaten anything for days."
She looked at him and said, "God, I wish I had your willpower."

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/16/2007 03:43 PM, Bodo Eggert wrote:

> So we are in a desperate situation, we can almost make no progress,
> adding another task is going to push the system into an unrecoverable
> situation, and we make sure this task can be started.-)

Hnng. He was just saying that odds of two pages being buddies is quite 
small, not talking about literally two pages.

>> Moreoveover -- literally two pages free was hardly his point. The point is 
>> just that (with a page being the allocation unit) single page allocations 
>> are guaranteed to succeed if _any_ memory is free, while two adjacent (yes, 
>> and stacksize aligned) pages will be pretty hard to get by once the system 
>> has been up and running for some time.
> 
> That never happened on my servers, therefore I'd opt for the little extra 
> security of having spare 4k on the stack. (I made a patch which would 
> printk a message if allocating a stack would ever fail).
> 
> I'm not at all opposed to letting the guys with zillions of threads 
> benefit from having less unused kernel stack, but unless it's secure for
> all users, it should not be default=y.

Given that as Arjan stated Fedora and even RHEL have been using 4K stacks 
for some time now, and certainly the latter being a distribution which I 
would expect to both host a relatively large number of lvm/md/xfs and what 
stackeaters have you users and to be fairly conservative with respect to the 
chances of scribbling over kernel memory (I'm a trusting person...) it seems 
  there might at this stage only be very few offenders left.

Seeing as how single-page stacks are much easier on the VM so that creating 
those zillion threads should also be faster, at _some_ percentage we get to 
say "and now to hell with the rest".

Do also note that with interrupts of the process stack, available stack is 
definitely not halved. I don't have data (if anyone reading does, please 
say) but I expect that on the kinds of busy networked systems that want 
many-thread creation to be fastest, their many concurrent interrupt sources 
might mean they are not actually experiencing less stack at all. That is, 
that "little extra security" you speak of might very well be none at all in 
practice and perhaps even negative.

Getting interrupts onto their own stack(s) certainly made for better (more 
deterministic that is) behaviour as well -- you're then independent on how 
deep into the stack you already are when the interrupt comes in which is 
otherwise anyone's guess. Now I must say I'm not particularly sure why you 
couldn't still also have those even if you don't pick 4K stacks, but as far 
as I'm aware they're a package deal at least today.

Single page stacks are much nicer on anyone. For all I care, that single 
page migth be a larger soft-page. No idea how the hell you'd investigate the 
optimum page-size for any given system, but I quite fully expect it's larger 
than 4K these days _anyway_ even on x86 and for modern loads.

Since Linux doesn't yet have those that's also not very important currently 
though. 4K is 1024 32-bit datums which isn't all that little if recursion is 
limited and programmers somewhat competent. With the interrupt stacks 
meaning available stack might not be worse or even _better_ for some 
systems, well, I's argue to just go with 4K if at all possible.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> Seeing as how single-page stacks are much easier on the VM so that creating
> those zillion threads should also be faster, at _some_ percentage we get to
> say "and now to hell with the rest".

This is the core dispute here. Stated differently, I hope you never
design a bridge that I have to drive over.

Correctness first, optimization second. Introducing random and
difficult to trace crashes upon an unsuspecting audience of sysadmins
and users is not a viable option.

If at some point one of the pro-4k stacks crowd can prove that all
code paths are safe, or introduce another viable alternative (such as
Matt's idea for extending the stack dynamically), then removing the 8k
stacks option makes sense.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> If at some point one of the pro-4k stacks crowd can prove that all
> code paths are safe, or introduce another viable alternative (such as
> Matt's idea for extending the stack dynamically), then removing the 8k
> stacks option makes sense.

Any x86-32 path unsafe with 4K stacks is almost certainly unsafe with 8K
stacks because the 8K stacks do not have seperate IRQ stack paths, so you
have the same space but split. It might be less predictable on 8K stacks
but it isn't absent.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/16/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > If at some point one of the pro-4k stacks crowd can prove that all
> > code paths are safe, or introduce another viable alternative (such as
> > Matt's idea for extending the stack dynamically), then removing the 8k
> > stacks option makes sense.
>
> Any x86-32 path unsafe with 4K stacks is almost certainly unsafe with 8K
> stacks because the 8K stacks do not have seperate IRQ stack paths, so you
> have the same space but split. It might be less predictable on 8K stacks
> but it isn't absent.

Understood, but isn't that an argument pro interrupt stacks, rather
than one against 8k? wli has a patch to break out the interrupt stack
feature from the 4k/8k choice.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 12:53 AM, Ray Lee wrote:

> On 7/16/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

>> Any x86-32 path unsafe with 4K stacks is almost certainly unsafe with 8K
>> stacks because the 8K stacks do not have seperate IRQ stack paths, so you
>> have the same space but split. It might be less predictable on 8K stacks
>> but it isn't absent.
> 
> Understood, but isn't that an argument pro interrupt stacks, rather
> than one against 8k? wli has a patch to break out the interrupt stack
> feature from the 4k/8k choice.

Yes but it's also an argument that the 4K stacks don't make the _current_ 
situation without CONFIG_4KSTACKS selected worse and given that you trust 
that current situation, that leaves you without your argument :-)

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> Yes but it's also an argument that the 4K stacks don't make the _current_
> situation without CONFIG_4KSTACKS selected worse and given that you trust
> that current situation, that leaves you without your argument :-)

Heh :-). No, it's not a question of trust. First and foremost, it's
that there are still users who say that they can crash a current
4k+interrupt stacks kernel, while the 8k without interrupt stacks is
fine.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

On Mon, 16 Jul 2007 16:15:28 -0700
"Ray Lee" <ray-lk@madrabbit.org> wrote:

> On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> > Yes but it's also an argument that the 4K stacks don't make the _current_
> > situation without CONFIG_4KSTACKS selected worse and given that you trust
> > that current situation, that leaves you without your argument :-)
> 
> Heh :-). No, it's not a question of trust. First and foremost, it's
> that there are still users who say that they can crash a current
> 4k+interrupt stacks kernel, while the 8k without interrupt stacks is
> fine.

You forgot "most of the time". Its statistically less likely, which
merely means its evilly hard to debug

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/17/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Mon, 16 Jul 2007 16:15:28 -0700
> "Ray Lee" <ray-lk@madrabbit.org> wrote:
> > Heh :-). No, it's not a question of trust. First and foremost, it's
> > that there are still users who say that they can crash a current
> > 4k+interrupt stacks kernel, while the 8k without interrupt stacks is
> > fine.
>
> You forgot "most of the time".

Yeah, fair enough.

> Its statistically less likely, which
> merely means its evilly hard to debug

Not being able to debug the cases that occur (and the fact that
they're rare, as you're pointing out) is as much of a problem as the
crashes themselves. 8k + IRQ stacks with a warning when 4k of process
stack is exceeded would seem like a reasonable first step to making 4k
a palatable default.

Ray

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[William Lee Irwin III]

At some point in the past, I wrote:
>> If at some point one of the pro-4k stacks crowd can prove that all
>> code paths are safe, or introduce another viable alternative (such as
>> Matt's idea for extending the stack dynamically), then removing the 8k
>> stacks option makes sense.

On Mon, Jul 16, 2007 at 11:54:38PM +0100, Alan Cox wrote:
> Any x86-32 path unsafe with 4K stacks is almost certainly unsafe with 8K
> stacks because the 8K stacks do not have seperate IRQ stack paths, so you
> have the same space but split. It might be less predictable on 8K stacks
> but it isn't absent.

At hch's suggestion I rewrote the separate IRQ stack configurability
patch into one making IRQ stacks mandatory and unconfigurable, and
hence enabled with 8K stacks.


-- wli

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 17/07/07, William Lee Irwin III <wli@holomorphy.com> wrote:
> At some point in the past, I wrote:
> >> If at some point one of the pro-4k stacks crowd can prove that all
> >> code paths are safe, or introduce another viable alternative (such as
> >> Matt's idea for extending the stack dynamically), then removing the 8k
> >> stacks option makes sense.
>
> On Mon, Jul 16, 2007 at 11:54:38PM +0100, Alan Cox wrote:
> > Any x86-32 path unsafe with 4K stacks is almost certainly unsafe with 8K
> > stacks because the 8K stacks do not have seperate IRQ stack paths, so you
> > have the same space but split. It might be less predictable on 8K stacks
> > but it isn't absent.
>
> At hch's suggestion I rewrote the separate IRQ stack configurability
> patch into one making IRQ stacks mandatory and unconfigurable, and
> hence enabled with 8K stacks.
>
For what it's worth, that sounds good to me - like something that we
would want merged.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/18/2007 01:39 AM, Jesper Juhl wrote:

> On 17/07/07, William Lee Irwin III <wli@holomorphy.com> wrote:

>> At hch's suggestion I rewrote the separate IRQ stack configurability
>> patch into one making IRQ stacks mandatory and unconfigurable, and
>> hence enabled with 8K stacks.
>>
> For what it's worth, that sounds good to me - like something that we
> would want merged.

Yes, seperate IRQ stacks make eminent sense in their own right.

Andrea Arcangeli's current thread on soft pages:

http://lkml.org/lkml/2007/7/6/346

is also interesting though in the context of 1-page stacks.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 17/07/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> > Seeing as how single-page stacks are much easier on the VM so that creating
> > those zillion threads should also be faster, at _some_ percentage we get to
> > say "and now to hell with the rest".
>
> This is the core dispute here. Stated differently, I hope you never
> design a bridge that I have to drive over.
>
> Correctness first, optimization second. Introducing random and
> difficult to trace crashes upon an unsuspecting audience of sysadmins
> and users is not a viable option.
>
> If at some point one of the pro-4k stacks crowd can prove that all
> code paths are safe, or introduce another viable alternative (such as
> Matt's idea for extending the stack dynamically), then removing the 8k
> stacks option makes sense.
>

Please note that I was not trying to remove the 8K stack option right
now - heck, I didn't even add anything to feature-removal-schedule.txt
- all I wanted to accomplish with the patch that started this threas
was;  a) indicate that the 4K option is no longer a debug thing  and
b) make 4K stacks the default option in vanilla kernel.org kernels as
a gentle nudge towards getting people to start fixing the code paths
that are not 4K stack safe.
Distros that currently use 8K stacks can continue to do so just fine,
individuals compiling their own kernel.org kernels can as well and
people using oldconfig wouldn't get any change, only people
configuring a new kernel.org kernel from scratch would see a change.
It was mostly meant as a hint that we want to move in the 4K stack
direction over time...
In the future (perhaps far future) when all 4K unsafe codepaths are
believed to have been fixed an entry could be made in
feature-removal-schedule.txt stating that the 8K option would go away
in 6, 12 or whatever, months.   That was my intention with the patch I
posted, I never intended to rip out 8K stacks anytime *soon*.


-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 12:54 AM, Jesper Juhl wrote:

> Please note that I was not trying to remove the 8K stack option right
> now - heck, I didn't even add anything to feature-removal-schedule.txt
> - all I wanted to accomplish with the patch that started this threas
> was;  a) indicate that the 4K option is no longer a debug thing  and
> b) make 4K stacks the default option in vanilla kernel.org kernels as
> a gentle nudge towards getting people to start fixing the code paths
> that are not 4K stack safe.

Definite Acked-by: Rene Herman <rene.herman@gmail.com> where others can 
decide what that ack's worth.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Tue, 17 Jul 2007, Jesper Juhl wrote:

> Please note that I was not trying to remove the 8K stack option right
> now - heck, I didn't even add anything to feature-removal-schedule.txt
> - all I wanted to accomplish with the patch that started this threas
> was;  a) indicate that the 4K option is no longer a debug thing  and

Very ACK.

> b) make 4K stacks the default option in vanilla kernel.org kernels as
> a gentle nudge towards getting people to start fixing the code paths
> that are not 4K stack safe.

That's the big NACK. It's OK for MM, where things are supposed to be in a 
not well-tested state, but for running possibly mission-critical systems,
you should take no risk.

If you'd run a 4K stack on the NFS+XFS+LVM+dmcrypt+MD+somethingmore 
setup driving your loved one's life support, you may go ahead.
-- 
I'm a member of DNA (National Assocciation of Dyslexics).
	-- Storm in <5Z4Z7.52353$4x4.6445347@news2-win.server.ntlworld.com>

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 17/07/07, Bodo Eggert <7eggert@gmx.de> wrote:
> On Tue, 17 Jul 2007, Jesper Juhl wrote:
>
> > Please note that I was not trying to remove the 8K stack option right
> > now - heck, I didn't even add anything to feature-removal-schedule.txt
> > - all I wanted to accomplish with the patch that started this threas
> > was;  a) indicate that the 4K option is no longer a debug thing  and
>
> Very ACK.
>
Ok, so perhaps a patch to do just that bit could go towards mainline
(possibly after a short stay in -mm).

> > b) make 4K stacks the default option in vanilla kernel.org kernels as
> > a gentle nudge towards getting people to start fixing the code paths
> > that are not 4K stack safe.
>
> That's the big NACK. It's OK for MM, where things are supposed to be in a
> not well-tested state, but for running possibly mission-critical systems,
> you should take no risk.
>
Hmm, I guess it would make sense to stick it in -mm for an extended
period of time, but hold off on mainline until we start hearing
positive noises that people can no longer break 4K stacks...

> If you'd run a 4K stack on the NFS+XFS+LVM+dmcrypt+MD+somethingmore
> setup driving your loved one's life support, you may go ahead.

;-)


How about if I split it into two tiny patches
 - one that just makes it no longer a debug option and moves the
option out of the "kernel hacking" submenu and into "general setup"
instead, but leave it to default to N.
 - and then one that makes it default Y.
and then submit those two patches to Andrew for inclusion in -mm with
a clear note that only the first one should go upstream anytime soon
and the second one should be kept "strictly -mm" until we arrive at
the point in time where people can no longer break 4K.
How does that sound?
>From my point of view that would accomplish my objective of getting 4K
stacks some more testing and the indication that long-term that's the
way we probably want to move.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Denis Vlasenko]

On Tuesday 17 July 2007 00:42, Bodo Eggert wrote:
> > Please note that I was not trying to remove the 8K stack option right
> > now - heck, I didn't even add anything to feature-removal-schedule.txt
> > - all I wanted to accomplish with the patch that started this threas
> > was;  a) indicate that the 4K option is no longer a debug thing  and
> 
> Very ACK.
> 
> > b) make 4K stacks the default option in vanilla kernel.org kernels as
> > a gentle nudge towards getting people to start fixing the code paths
> > that are not 4K stack safe.
> 
> That's the big NACK. It's OK for MM, where things are supposed to be in a 
> not well-tested state, but for running possibly mission-critical systems,
> you should take no risk.

Mission-critical machines are not supposed to have kernel configured
with incompetent/careless sysadmin who didn't think about
config choices he made at kernel build time.
--
vda

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Thu, 19 Jul 2007, Denis Vlasenko wrote:
> On Tuesday 17 July 2007 00:42, Bodo Eggert wrote:

> > > b) make 4K stacks the default option in vanilla kernel.org kernels as
> > > a gentle nudge towards getting people to start fixing the code paths
> > > that are not 4K stack safe.
> > 
> > That's the big NACK. It's OK for MM, where things are supposed to be in a 
> > not well-tested state, but for running possibly mission-critical systems,
> > you should take no risk.
> 
> Mission-critical machines are not supposed to have kernel configured
> with incompetent/careless sysadmin who didn't think about
> config choices he made at kernel build time.

Is it careless to asume good code quality for default options?
Does the 4K stack come with a big red warning about crashing the kernel?
(I just checked, it does not, only benefits are listed.)
Are 4K stacks so obviously flawed nobody would use them for reliable systems?
Or is each sysadmin supposed to read LKML in order to find out about the
pitfalls you designed for them?
-- 
Top 100 things you don't want the sysadmin to say:
55. NO!  Not _that_ button!

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 12:37 AM, Ray Lee wrote:

> On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:

>> Seeing as how single-page stacks are much easier on the VM so that 
>> creating those zillion threads should also be faster, at _some_
>> percentage we get to say "and now to hell with the rest".
> 
> This is the core dispute here. Stated differently, I hope you never
> design a bridge that I have to drive over.
> 
> Correctness first, optimization second. Introducing random and
> difficult to trace crashes upon an unsuspecting audience of sysadmins
> and users is not a viable option.

Quite. But unfortunately you didn't actually go into the bit on how given 
seperate interrupt stacks, available stackspace might not actually _be_ less 
after selecting CONFIG_4KSTCKS nor into Fedora and RHEL shipping it already.

> If at some point one of the pro-4k stacks crowd can prove that all
> code paths are safe

I'll do that the minute you prove the current shared 8K stacks are safe. Do 
we have a deal?

> or introduce another viable alternative (such as Matt's idea for
> extending the stack dynamically), then removing the 8k stacks option
> makes sense.

I'm still waiting for larger soft-pages... does anyone in this thread have a 
clue on their status?

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Tue, Jul 17, 2007 at 12:55:36AM +0200, Rene Herman wrote:
> On 07/17/2007 12:37 AM, Ray Lee wrote:
> 
> >On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> 
> >>Seeing as how single-page stacks are much easier on the VM so that 
> >>creating those zillion threads should also be faster, at _some_
> >>percentage we get to say "and now to hell with the rest".
> >
> >This is the core dispute here. Stated differently, I hope you never
> >design a bridge that I have to drive over.
> >
> >Correctness first, optimization second. Introducing random and
> >difficult to trace crashes upon an unsuspecting audience of sysadmins
> >and users is not a viable option.
> 
> Quite. But unfortunately you didn't actually go into the bit on how given 
> seperate interrupt stacks, available stackspace might not actually _be_ 
> less after selecting CONFIG_4KSTCKS nor into Fedora and RHEL shipping it 
> already.
> 
> >If at some point one of the pro-4k stacks crowd can prove that all
> >code paths are safe
> 
> I'll do that the minute you prove the current shared 8K stacks are safe. Do 
> we have a deal?
> 
> >or introduce another viable alternative (such as Matt's idea for
> >extending the stack dynamically), then removing the 8k stacks option
> >makes sense.
> 
> I'm still waiting for larger soft-pages... does anyone in this thread have 
> a clue on their status?

Given that most x86 users won't want anything to do with them, it's
not going to help us at all here.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:07 AM, Matt Mackall wrote:

> On Tue, Jul 17, 2007 at 12:55:36AM +0200, Rene Herman wrote:

>> I'm still waiting for larger soft-pages... does anyone in this thread
>> have a clue on their status?
> 
> Given that most x86 users won't want anything to do with them, it's not
> going to help us at all here.

No idea why not? Is this something you expect, or know, or... ? (and who are 
users in this context?)

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Tue, Jul 17, 2007 at 01:12:51AM +0200, Rene Herman wrote:
> On 07/17/2007 01:07 AM, Matt Mackall wrote:
> 
> >On Tue, Jul 17, 2007 at 12:55:36AM +0200, Rene Herman wrote:
> 
> >>I'm still waiting for larger soft-pages... does anyone in this thread
> >>have a clue on their status?
> >
> >Given that most x86 users won't want anything to do with them, it's not
> >going to help us at all here.
> 
> No idea why not? Is this something you expect, or know, or... ? (and who 
> are users in this context?)

Larger soft pages waste tremendous amounts of memory (mostly in page
cache) for minimal benefit on, say, the typical desktop. While there
are workloads where it's a win, it's probably on a small percentage of
machines.

So it's absolutely no help in fixing our order-1 allocation problem
because we don't want to force large pages on people.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:27 AM, Matt Mackall wrote:

> Larger soft pages waste tremendous amounts of memory (mostly in page
> cache) for minimal benefit on, say, the typical desktop. While there
> are workloads where it's a win, it's probably on a small percentage of
> machines.
> 
> So it's absolutely no help in fixing our order-1 allocation problem 
> because we don't want to force large pages on people.

Okay. I would've expected that 4K was fairly tiny for today's loads but as 
usual I'm relatively data challenged so I guess I'll take your word for it. 
Bummer.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]

On 07/17/2007 01:27 AM, Matt Mackall wrote:

> Larger soft pages waste tremendous amounts of memory (mostly in page
> cache) for minimal benefit on, say, the typical desktop. While there
> are workloads where it's a win, it's probably on a small percentage of
> machines.
> 
> So it's absolutely no help in fixing our order-1 allocation problem
> because we don't want to force large pages on people.

I was just now looking at how much space is in fact wasted in pagecache for 
various pagesizes by running the attached dumb little program from a few 
selected directories (heavy stack recursion, never mind).

Well, hmmm. This is on a (compiled) git tree:

rene@7ixe4:~/src/linux/local$ pageslack
total	: 447350347
  4k	: 67738037 (15%)
  8k	: 147814837 (33%)
16k	: 324614581 (72%)
32k	: 724629941 (161%)
64k	: 1592785333 (356%)

Nicely constant factor 2.2 instead of the 2 one would expect but oh well. On 
a collection of larger files the percentages obviously drop. This is on a 
directory of ogg vorbis files:

root@7ixe4:/mnt/ogg/.../... # pageslack
total	: 70817974
  4k	: 26442 (0%)
  8k	: 67402 (0%)
16k	: 124746 (0%)
32k	: 288586 (0%)
64k	: 419658 (0%)

The "typical desktop" is presented by neither I guess but does involve audio 
and (much larger still) video and bloody huge browser apps.

Not too sure then that 8K wouldn't be something I'd want, given fewer 
pagefaults and all that...

Rene.


[-- Attachment #2: pageslack.c --]
[-- Type: text/plain, Size: 1552 bytes --]

/* gcc -W -Wall -o pageslack pageslack.c */

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <dirent.h>

#define PAGE_SIZE (1UL << PAGE_SHIFT)
#define PAGE_MASK (~(PAGE_SIZE - 1))

unsigned long long total;
unsigned long long slack[5];

void do_dir(const char *name)
{
	DIR *dir; 
	struct dirent *ent;

	dir = opendir(name);
	if (!dir) {
		perror("opendir");
		exit(EXIT_FAILURE);
	}
	while ((ent = readdir(dir))) {
		struct stat buf;
		char path[PATH_MAX];

		if (!strcmp(ent->d_name, "."))
			continue;
		if (!strcmp(ent->d_name, ".."))
			continue;

		sprintf(path, "%s/%s", name, ent->d_name);
		if (stat(path, &buf)) {
			perror("stat");
			exit(EXIT_FAILURE);
		}
		if (S_ISDIR(buf.st_mode)) {
			do_dir(path);
			continue;
		}
		if (S_ISREG(buf.st_mode)) {
			int i;

			for (i = 0; i < 5; i++) {
				unsigned long PAGE_SHIFT = 12 + i;
				slack[i] += (PAGE_SIZE - (buf.st_size % PAGE_SIZE)) % PAGE_SIZE;
			}
			total += buf.st_size;
		}
	}
	if (closedir(dir)) {
		perror("closedir");
		exit(EXIT_FAILURE);
	}
}

int main(void)
{
	do_dir(".");
	printf("total\t: %llu\n", total);
	printf(" 4k\t: %llu (%llu%%)\n", slack[0], (100 * slack[0]) / total);
	printf(" 8k\t: %llu (%llu%%)\n", slack[1], (100 * slack[1]) / total);
	printf("16k\t: %llu (%llu%%)\n", slack[2], (100 * slack[2]) / total);
	printf("32k\t: %llu (%llu%%)\n", slack[3], (100 * slack[3]) / total);
	printf("64k\t: %llu (%llu%%)\n", slack[4], (100 * slack[4]) / total);
	return EXIT_SUCCESS;
}

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Wed, Jul 18, 2007 at 04:38:19AM +0200, Rene Herman wrote:
> On 07/17/2007 01:27 AM, Matt Mackall wrote:
> 
> >Larger soft pages waste tremendous amounts of memory (mostly in page
> >cache) for minimal benefit on, say, the typical desktop. While there
> >are workloads where it's a win, it's probably on a small percentage of
> >machines.
> >
> >So it's absolutely no help in fixing our order-1 allocation problem
> >because we don't want to force large pages on people.
> 
> I was just now looking at how much space is in fact wasted in pagecache for 
> various pagesizes by running the attached dumb little program from a few 
> selected directories (heavy stack recursion, never mind).
> 
> Well, hmmm. This is on a (compiled) git tree:
> 
> rene@7ixe4:~/src/linux/local$ pageslack
> total	: 447350347
>  4k	: 67738037 (15%)
>  8k	: 147814837 (33%)
> 16k	: 324614581 (72%)
> 32k	: 724629941 (161%)
> 64k	: 1592785333 (356%)
> 
> Nicely constant factor 2.2 instead of the 2 one would expect but oh well. 
> On a collection of larger files the percentages obviously drop. This is on 
> a directory of ogg vorbis files:
> 
> root@7ixe4:/mnt/ogg/.../... # pageslack
> total	: 70817974
>  4k	: 26442 (0%)
>  8k	: 67402 (0%)
> 16k	: 124746 (0%)
> 32k	: 288586 (0%)
> 64k	: 419658 (0%)
> 
> The "typical desktop" is presented by neither I guess but does involve 
> audio and (much larger still) video and bloody huge browser apps.

I'd be surprised if a user had substantially more than one OGG, video,
or browser in memory at one time. In fact, you're likely to find only
a fraction of each of those in memory at any given time.

Meanwhile, they're likely to have thousands of small browser cache,
thumbnail, config, icon, maildir, etc. files in cache. And hundreds of
medium-sized libraries, utilities, applications, and so on.

You can expect the distribution of file sizes to follow a gamma
distribution, with a large hump towards the small end of the spectrum
around 1-10K, dropping off very rapidly as file sizes grow.

> Not too sure then that 8K wouldn't be something I'd want, given fewer 
> pagefaults and all that...

Fewer minor pagefaults, perhaps. Readahead already deals with most of
the major pagefaults that larger pages would.

Anyway, raising the systemwide memory overhead by up to 15% seems an
awfully silly way to address the problem of not being able to allocate
a stack when you're down to your last 1 or 2% of memory! In all
likelihood, we'll fail sooner because we're completely OOM.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/18/2007 06:54 PM, Matt Mackall wrote:

> You can expect the distribution of file sizes to follow a gamma
> distribution, with a large hump towards the small end of the spectrum
> around 1-10K, dropping off very rapidly as file sizes grow.

Okay.

>> Not too sure then that 8K wouldn't be something I'd want, given fewer 
>> pagefaults and all that...
> 
> Fewer minor pagefaults, perhaps. Readahead already deals with most of
> the major pagefaults that larger pages would.

Mmm, yes.

> Anyway, raising the systemwide memory overhead by up to 15% seems an 
> awfully silly way to address the problem of not being able to allocate a
> stack when you're down to your last 1 or 2% of memory!

Well, I've seen larger pagesizes submerge in more situations, specifically 
in allocation overhead -- ie, making the struct page's fit in lowmem for 
hugemem x86 boxes was the first I heard of it. But yes, otherwise (also) 
mostly database loads which obviously have moved to 64-bit since.

Pagecache tail-packing seems like a promising idea to deal with the downside 
of larger pages but I'll admit I'm not particularly sure how many _up_ sides 
to them are left on x86 (not -64) now that's becoming a legacy architecture 
(and since you just shot down the pagefaults thing).

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Andrea Arcangeli]

On Mon, Jul 16, 2007 at 06:27:55PM -0500, Matt Mackall wrote:
> So it's absolutely no help in fixing our order-1 allocation problem
> because we don't want to force large pages on people.

Using kmalloc(8k) instead of alloc_page() doesn't sound a too big deal
and that will solve the problem. The whole idea is to avoid the memcpy
+ pte mangling of defrag while hopefully lowering cpu utilization in
allocations at the same time.

About 4k stacks I was generally against them, much better to fail in
fork than to risk corruption. The per-irq stack part is great feature
instead (too bad it wasn't enabled for the safer 8k stacks).

Failing in a do_no_page with variable order page size allocation is a
fatal event (the task will be killed), failing in fork is graceful,
userland can retry etc... Fork can fail for different reasons, ulimit
itself is the most likely source of fork failures. I don't think the
8k stacks have ever been a problem, yes you will run out of stack
sooner (sooner also because the 4k stacks takes less memory) but
nothing is terribly wrong if the 8k allocation fails.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> About 4k stacks I was generally against them, much better to fail in
> fork than to risk corruption. The per-irq stack part is great feature
> instead (too bad it wasn't enabled for the safer 8k stacks).

8K stacks without IRQ stacks are not "safer" so I don't understand your
comment ?

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Andrea Arcangeli]

On Thu, Jul 19, 2007 at 01:39:55AM +0100, Alan Cox wrote:
> > About 4k stacks I was generally against them, much better to fail in
> > fork than to risk corruption. The per-irq stack part is great feature
> > instead (too bad it wasn't enabled for the safer 8k stacks).
> 
> 8K stacks without IRQ stacks are not "safer" so I don't understand your
> comment ?

Ouch, see the reports about 4k stack crashes. I agree they're not
safe w/o irq stacks (like on x86-64), but they're generally safer.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Thu, Jul 19, 2007 at 03:33:58AM +0200, Andrea Arcangeli wrote:
> On Thu, Jul 19, 2007 at 01:39:55AM +0100, Alan Cox wrote:
> > > About 4k stacks I was generally against them, much better to fail in
> > > fork than to risk corruption. The per-irq stack part is great feature
> > > instead (too bad it wasn't enabled for the safer 8k stacks).
> > 
> > 8K stacks without IRQ stacks are not "safer" so I don't understand your
> > comment ?
> 
> Ouch, see the reports about 4k stack crashes. I agree they're not
> safe w/o irq stacks (like on x86-64), but they're generally safer.

Here's a way to make forward progress on this whole thing:

Turn on irqstacks when using 8k stacks
Detect when usage with 8k stacks would overrun a 4k stack when doing
 our stack switch and do a WARN_ONCE
Fix up the damn bugs

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/19/2007 03:37 AM, Matt Mackall wrote:

> Here's a way to make forward progress on this whole thing:
> 
> Turn on irqstacks when using 8k stacks

WLI: are you submitting? Makes great sense regardless of anything and 
they've been tested silly with 4KSTACKS already...

> Detect when usage with 8k stacks would overrun a 4k stack when doing
> our stack switch and do a WARN_ONCE

Our stack switch?

> Fix up the damn bugs

DM ofcourse is fairly "layered-by-design" so I _hope_ they can be classified 
simple bugs...

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Andrea Arcangeli]

On Wed, Jul 18, 2007 at 08:37:25PM -0500, Matt Mackall wrote:
> Turn on irqstacks when using 8k stacks

Indeed.

> Detect when usage with 8k stacks would overrun a 4k stack when doing
>  our stack switch and do a WARN_ONCE
> Fix up the damn bugs

I don't think they're necessarily bugs. IMHO the WARN_ON is better off
at 7k level like it is today with the current STACK_WARN. 4k for a
stack for common code really is small. I doubt you're going to find
obvious culprits that way, more likely you'll have to mangle the code
to call kmalloc for fairly small structures which isn't necessarily a
good thing in the long term. It comes to mind the folio ptes array
that Hugh allocated on the stack in his large PAGE_SIZE patch of jul
2001, that thing like any other local array, would need to be
kmalloced with a 4k stack. With 4k I'm afraid you better not use the
stack for anything but pointers, especially if you run in common code
that may invoke I/O like that.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> I don't think they're necessarily bugs. IMHO the WARN_ON is better off
> at 7k level like it is today with the current STACK_WARN. 4k for a
> stack for common code really is small. I doubt you're going to find

You want the limit settable. On a production system you want to set the
limit to somewhere appropriate for the stack size used. When debugging
(eg to remove any last few bogus users of 8K stack space) you want to be
able to set it to just under 4K

Alan

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Eric Sandeen]

Alan Cox wrote:
>> I don't think they're necessarily bugs. IMHO the WARN_ON is better off
>> at 7k level like it is today with the current STACK_WARN. 4k for a
>> stack for common code really is small. I doubt you're going to find
> 
> You want the limit settable. On a production system you want to set the
> limit to somewhere appropriate for the stack size used. When debugging
> (eg to remove any last few bogus users of 8K stack space) you want to be
> able to set it to just under 4K

Hm, when cramming cxfs into 4k at sgi, I had a patch that did just that
for debugging (warn about encroaching on 4k without actually tipping
over, with a settable threshold...)

Maybe I should resurrect it & send it out...

(FWIW I think I recall that the warning itself sometimes tipped the
scales enough on 4k stacks to bring the box down)

-eric

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> Maybe I should resurrect it & send it out...
> 
> (FWIW I think I recall that the warning itself sometimes tipped the
> scales enough on 4k stacks to bring the box down)

You can always switch stack for the printk and it probably should panic
at that point and give a trace then die as that is what we are trying to
prove does not occur

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Satyam Sharma]

On 7/27/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > Maybe I should resurrect it & send it out...

Hmm, something that hooks in not only at do_IRQ time (as the present
in-mainline stackoverflow check thing)?

> > (FWIW I think I recall that the warning itself sometimes tipped the
> > scales enough on 4k stacks to bring the box down)
>
> You can always switch stack for the printk and it probably should panic
> at that point and give a trace then die as that is what we are trying to
> prove does not occur

Yes, only yesterday I saw exactly this happening DEBUG_STACKOVERFLOW
when doing a udf -> pktcdvd -> cdrom -> ide_cd thing. It's one of those
reproducible will-crash-4k-stacks tests, especially if you have debug stuff
enabled in your build that would make on-stack structures (where such
exist on the codepath) a bit heavier.

Admittedly, what seems to have happened is a bit pathological:

[  481.836378] cdrom: entering cdrom_count_tracks
[  481.844266] BUG: sleeping function called from invalid context at
include/asm/semaphore.h:98
[  481.844434] do_IRQ: stack overflow: 164
[  481.844540]  [<c0405cfe>] show_trace_log_lvl+0x19/0x2e
[  481.844707]  [<c0405dfe>] show_trace+0x12/0x14
[  481.844867]  [<c0405e14>] dump_stack+0x14/0x16
[  481.845027]  [<c0406ff6>] do_IRQ+0x7b/0xe1
[  481.845186]  [<c040583e>] common_interrupt+0x2e/0x34
[  481.845348]  [<c042b8e7>] printk+0x1b/0x1d
[  481.845507]  [<c0422c05>] __might_sleep+0x81/0xdc
[  481.845668]  [<c066d869>] __reacquire_kernel_lock+0x2d/0x4f
[  481.845833]  [<c066b09b>] schedule+0x78a/0x7a4
[  481.845996]  [<c066b538>] wait_for_completion+0x72/0x97
[  481.846160]  [<c05937a6>] ide_do_drive_cmd+0xeb/0x109
[  481.846324]  [<f89172a2>] cdrom_queue_packet_command+0x40/0xc5 [ide_cd]
[  481.846503]  [<f89175b7>] ide_cdrom_packet+0x86/0xa4 [ide_cd]
[  481.846669]  [<f8854dc1>] cdrom_get_disc_info+0x48/0x87 [cdrom]
[  481.846839]  [<f8854ec6>] cdrom_get_last_written+0x2a/0xfe [cdrom]
[  481.847009]  [<f891831b>] cdrom_read_toc+0x39d/0x3f3 [ide_cd]
[  481.847231]  [<f8918e7e>] ide_cdrom_audio_ioctl+0x130/0x1ce [ide_cd]
[  481.847414]  [<f8854123>] cdrom_count_tracks+0x5c/0x126 [cdrom]
[  481.847583]  [<f8855688>] cdrom_open+0x147/0x79c [cdrom]
[  481.847748]  [<f891799a>] idecd_open+0x75/0x8a [ide_cd]
[  481.847912]  [<c04aac0e>] do_open+0x1d1/0x284
[  481.848079]  [<c04aad89>] __blkdev_get+0x73/0x7e
[  481.848242]  [<c04aada9>] blkdev_get+0x15/0x17
[  481.848411]  [<f8b34b6b>] pkt_open+0x99/0xc6e [pktcdvd]
[  481.848583]  [<c04aaad3>] do_open+0x96/0x284
[  481.848745]  [<c04aad89>] __blkdev_get+0x73/0x7e
[  481.848910]  [<c04aada9>] blkdev_get+0x15/0x17

(... the trace cut off there, and then the box froze hard, no sysrq ...)

The mount(2) hit the wait_for_completion() in ide_do_drive_cmd(),
little stack was left at this point. But then I have no idea why the
__reacquire_kernel_lock() from schedule() gave a might_sleep() there,
the code in sched.c and kernel_lock.c looks obviously correct -- the
down(&kernel_sem) only happens with both irqs and preemption on.

Anyway, the second line of printk() in __might_sleep (the one that
tells us in_atomic() and irqs_disabled()) was about to be printed when
an interrupt decided to join the fun. do_IRQ() comes in, with debug
stackoverflows on, it notices that only 164 bytes worth of stack is left
and decides to dump_stack ... and while we were doing just that,
we died. (this was 2.6.23-rc1-mm1)

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Eric Sandeen]

Satyam Sharma wrote:
> On 7/27/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>>> Maybe I should resurrect it & send it out...
> 
> Hmm, something that hooks in not only at do_IRQ time (as the present
> in-mainline stackoverflow check thing)?

No, what I had did only that, so it was still a matter of probabilities...

-Eric

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Dan Merillat]

On 7/31/07, Eric Sandeen <sandeen@sandeen.net> wrote:

> No, what I had did only that, so it was still a matter of probabilities...

How expensive would it be to allocate two , then use the MMU mark the
second page unwritable? Hardware wise it should be possible,  (for
constant 4k pagesizes, I have not worked with variable pagesize MMUs)
and since it's a per-context-switch constant operation, it would be a
special case in the fault handler rather then adding another entry to
the VM for every process.

Using large hardware pages to cover the kernel mapping could be worked
around by leaving the area where the current process stack resides
mapped via 4k pages.  Of course, I haven't touched a modern PC MMU in
ages, so I could be missing something fundamentally difficult.

The other issue is with the layered IO design - no matter what we
configure the stack size to, it is still possible to create a set of
translation layers that will cause it to crash regularly:  XFS on
dm_crypt on loop on XFS on dm_crypt on loop on ad infinitum.

That said, I'm missing something here - why is the stack growing?
Filesystems should be issuing bios with callbacks, so they should be
back off the stack, same with dm, loop, etc.   Am I missing step where
they use a wrapper function that pretends to be syncronous?

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Neil Brown]

On Wednesday August 1, dan.merillat@gmail.com wrote:
> 
> The other issue is with the layered IO design - no matter what we
> configure the stack size to, it is still possible to create a set of
> translation layers that will cause it to crash regularly:  XFS on
> dm_crypt on loop on XFS on dm_crypt on loop on ad infinitum.

No, this does not use indefinite stack.

loop will schedule each request to be handled by a kernel thread, so
requests to 'loop' are serialised, never stacked.

In 2.6.22, generic_make_request detects and serialises recursive calls,
so unlimited recursion is not possible there either.

It is still possible to do
  dm on dm on dm on dm on md on md on md on md

and calls to ->issue_flush_fn or ->unplug_fn could use an arbitrarily
large amount of stack.  But the stack usage of each stage is very
small so it is unlikely to be a problem (though it should still be
fixed).

NeilBrown

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Dan Merillat]

On 8/1/07, Neil Brown <neilb@suse.de> wrote:

> No, this does not use indefinite stack.
>
> loop will schedule each request to be handled by a kernel thread, so
> requests to 'loop' are serialised, never stacked.
>
> In 2.6.22, generic_make_request detects and serialises recursive calls,
> so unlimited recursion is not possible there either.

Is that saying "before 2.6.22, a read/write on a deeply layered device
would use a lot of stack?"

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Neil Brown]

On Friday August 10, dan.merillat@gmail.com wrote:
> On 8/1/07, Neil Brown <neilb@suse.de> wrote:
> 
> > No, this does not use indefinite stack.
> >
> > loop will schedule each request to be handled by a kernel thread, so
> > requests to 'loop' are serialised, never stacked.
> >
> > In 2.6.22, generic_make_request detects and serialises recursive calls,
> > so unlimited recursion is not possible there either.
> 
> Is that saying "before 2.6.22, a read/write on a deeply layered device
> would use a lot of stack?"

before 2.6.22, a stack of dm and/or md devices (not loop, and not
md/raid0 or md/linear) would use more stack the more devices were
involved.  If you made a very deep stack, you could push the stack
over any limit you chose.

I won't say "a lot of stack" as I haven't measured the exact amount,
just "more stack as you add more devices".

NeilBrown

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Andrea Arcangeli]

On Wed, Aug 01, 2007 at 04:11:23AM -0400, Dan Merillat wrote:
> How expensive would it be to allocate two , then use the MMU mark the
> second page unwritable? Hardware wise it should be possible,  (for

Tweaking kernel ptes is prohibitive during clone() because that's
kernel memory and it would require a flush tlb all with IPIs that
won't scale (IPIs are really the blocker). Basically vmalloc already
does what you suggest with the gap page and yet we can't use it for
performance reasons. Kernel stack should be readable by any context to
allow sysrq+t kind of things, so I doubt it's feasible to do tricks to
avoid ipis.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

On Wed, 1 Aug 2007 15:33:58 +0200
Andrea Arcangeli <andrea@suse.de> wrote:

> On Wed, Aug 01, 2007 at 04:11:23AM -0400, Dan Merillat wrote:
> > How expensive would it be to allocate two , then use the MMU mark the
> > second page unwritable? Hardware wise it should be possible,  (for
> 
> Tweaking kernel ptes is prohibitive during clone() because that's
> kernel memory and it would require a flush tlb all with IPIs that
> won't scale (IPIs are really the blocker)

Agreed - except when doing debug work then its an acceptable cost. You
still have to sort the debug side out because you are going to fault the
kernel stack which will probably then cause a triple fault and reboot on
the spot.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Dan Merillat]

On 8/1/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Wed, 1 Aug 2007 15:33:58 +0200
> Andrea Arcangeli <andrea@suse.de> wrote:
> > Tweaking kernel ptes is prohibitive during clone() because that's
> > kernel memory and it would require a flush tlb all with IPIs that
> > won't scale (IPIs are really the blocker)
>
> Agreed - except when doing debug work then its an acceptable cost. You
> still have to sort the debug side out because you are going to fault the
> kernel stack which will probably then cause a triple fault and reboot on
> the spot.

I was assuming debugging work, yes.  I was also thinking it wouldn't
be done at clone() time, but mapped (on a single CPU) at the time of a
context switch.  It would eliminate IPI, but would probably make the
rest of the TLB handling much too ugly to contemplate.    As an
alternative, could the TLB flush and associated IPI be deferred until
the process migrates?   First migration would trigger flush/IPI,
further migration would be as now, no?   I'd happily run it with
various dm/md layers underneath

On 8/1/07, Denis Vlasenko <vda.linux@googlemail.com> wrote:
> Hmm, neat. Why do you need to _allocate second page_ at all?
> Just mark it "not present"...

Because the kernel mapping covers all physical memory contiguously, so
if the page isn't allocated, it could be used by a kernel data
structure you need to access.  Same reason the kernel stack has to be
contiguous pages.   Well, for non-highmem at least.  Either way, you
don't want to mark an in-use page as inaccessable, you never know
what's under there.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

On Thu, 19 Jul 2007 03:33:58 +0200
Andrea Arcangeli <andrea@suse.de> wrote:

> > 8K stacks without IRQ stacks are not "safer" so I don't understand your
> > comment ?
> 
> Ouch, see the reports about 4k stack crashes. I agree they're not
> safe w/o irq stacks (like on x86-64), but they're generally safer.

Still don't follow. How is "exceeds stack space but less likely to be
noticed" safer.

Alan

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Andrea Arcangeli]

On Thu, Jul 19, 2007 at 10:23:59AM +0100, Alan Cox wrote:
> Still don't follow. How is "exceeds stack space but less likely to be
> noticed" safer.

Statistically speaking it clearly is. The reason is probably that the
irq theoretical issue happens only on large boxes with lots of
reentrant irqs. Not all irqs are reentrant, not all systems runs lots
of irqs at the same time etc..

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Eric Sandeen]

Alan Cox wrote:
>> About 4k stacks I was generally against them, much better to fail in
>> fork than to risk corruption. The per-irq stack part is great feature
>> instead (too bad it wasn't enabled for the safer 8k stacks).
> 
> 8K stacks without IRQ stacks are not "safer" so I don't understand your
> comment ?

Hmm was it SuSE or RH kernels (or mainline?) I saw which had a test to
defer soft IRQs if they occurred too deep in the stack for the current
thread.

-Eric

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Krzysztof Halasa]

Eric Sandeen <sandeen@sandeen.net> writes:

>> 8K stacks without IRQ stacks are not "safer" so I don't understand your
>> comment ?
>
> Hmm was it SuSE or RH kernels (or mainline?) I saw which had a test to
> defer soft IRQs if they occurred too deep in the stack for the current
> thread.

Perhaps the "8 KB softpage" should be an option instead
of 8 KB stack size?

Not sure about ABI compatibility.
-- 
Krzysztof Halasa

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Thu, Jul 19, 2007 at 02:15:39AM +0200, Andrea Arcangeli wrote:
> On Mon, Jul 16, 2007 at 06:27:55PM -0500, Matt Mackall wrote:
> > So it's absolutely no help in fixing our order-1 allocation problem
> > because we don't want to force large pages on people.
> 
> Using kmalloc(8k) instead of alloc_page() doesn't sound a too big deal
> and that will solve the problem.

How do you figure?

If you're saying that soft pages helps our 8k stack allocations, it
doesn't. The memory overhead of soft pages will be higher (5-15%,
mostly due to file tails in pagecache) than the level at which 8k
stacks currently run into trouble (1-2% free?).

Not helpful.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/19/2007 02:41 AM, Matt Mackall wrote:

> On Thu, Jul 19, 2007 at 02:15:39AM +0200, Andrea Arcangeli wrote:

>> Using kmalloc(8k) instead of alloc_page() doesn't sound a too big deal
>> and that will solve the problem.
> 
> How do you figure?
> 
> If you're saying that soft pages helps our 8k stack allocations, it
> doesn't. The memory overhead of soft pages will be higher (5-15%,
> mostly due to file tails in pagecache) than the level at which 8k
> stacks currently run into trouble (1-2% free?).
> 
> Not helpful.

With tail-packing it is.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Thu, Jul 19, 2007 at 02:48:37AM +0200, Rene Herman wrote:
> On 07/19/2007 02:41 AM, Matt Mackall wrote:
> 
> >On Thu, Jul 19, 2007 at 02:15:39AM +0200, Andrea Arcangeli wrote:
> 
> >>Using kmalloc(8k) instead of alloc_page() doesn't sound a too big deal
> >>and that will solve the problem.
> >
> >How do you figure?
> >
> >If you're saying that soft pages helps our 8k stack allocations, it
> >doesn't. The memory overhead of soft pages will be higher (5-15%,
> >mostly due to file tails in pagecache) than the level at which 8k
> >stacks currently run into trouble (1-2% free?).
> >
> >Not helpful.
> 
> With tail-packing it is.

Tail packing is a whole new can of worms. Especially as it's very
likely to make performance suffer on small files (the common case).

On the other hand, if someone can demonstrate that tail-packed page
cache doesn't suck, we should put it in mainline pronto. The poor
architectures that are stuck with real 64k pages are sure to
appreciate it.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/17/2007 12:37 AM, Ray Lee wrote:
> > If at some point one of the pro-4k stacks crowd can prove that all
> > code paths are safe
>
> I'll do that the minute you prove the current shared 8K stacks are safe. Do
> we have a deal?

Language barrier, I think, or perhaps I was unclear. Please read that
as "4k stacks introduce no new stack bugs." And if we put wli's
unconditional interrupt stacks into the kernel, it's pretty obvious
that 8k stacks are at least as safe in that case as 4k stacks.

Given that there's actual, y'know, reports of people who can easily
crash a 4k+interrupt stacks kernel, and not an 8k one, I think the
current evidence speaks for itself.

The point remains that the burden of proof of the safety of the 4k
only option is upon those people who want to remove the 8k option.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:13 AM, Ray Lee wrote:

> Given that there's actual, y'know, reports of people who can easily
> crash a 4k+interrupt stacks kernel, and not an 8k one, I think the
> current evidence speaks for itself.

Where?

> The point remains that the burden of proof of the safety of the 4k only
> option is upon those people who want to remove the 8k option.

Removing any such option was not the objective of this thread, just lifting 
4K stacks from debug and making it the default. People fortunate enough to 
use workloads where some piece of crap code by accident works more often 
with the current shared 8K stacks then it does with the unshared 4K stacks 
can then still nicely not select it (or fix the code if possible).

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 17/07/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/17/2007 01:13 AM, Ray Lee wrote:
>
> > Given that there's actual, y'know, reports of people who can easily
> > crash a 4k+interrupt stacks kernel, and not an 8k one, I think the
> > current evidence speaks for itself.
>
> Where?
>
> > The point remains that the burden of proof of the safety of the 4k only
> > option is upon those people who want to remove the 8k option.
>
> Removing any such option was not the objective of this thread, just lifting
> 4K stacks from debug and making it the default. People fortunate enough to
> use workloads where some piece of crap code by accident works more often
> with the current shared 8K stacks then it does with the unshared 4K stacks
> can then still nicely not select it (or fix the code if possible).
>
I may have worded the initial email a bit too heavily towards 8K
removal, but that was mainly to provoke some discussion.  Nothing
should get removed without a fair (and long) warning in
feature-removal-schedule.txt and ofcourse not before we know that the
new option is at least as safe as what we intend to remove, so the
patch really was just intended as a fairly harmless nudge towards
getting 4K stacks into a shape where we can eventually start
considering 8K removal.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/17/2007 01:13 AM, Ray Lee wrote:
> > Given that there's actual, y'know, reports of people who can easily
> > crash a 4k+interrupt stacks kernel, and not an 8k one, I think the
> > current evidence speaks for itself.
>
> Where?

The second message in this thread, according to my reader, from Zan
Lynx. Another from Utz Lehmann a few minutes ago.

> > The point remains that the burden of proof of the safety of the 4k only
> > option is upon those people who want to remove the 8k option.
>
> Removing any such option was not the objective of this thread, just lifting
> 4K stacks from debug and making it the default.

True, but your messages are reading as advocacy for removing the 8k
option. I'm saying that's a bad idea. If I misunderstood your
position, then my bad.

> People fortunate enough to
> use workloads where some piece of crap code by accident works more often
> with the current shared 8K stacks then it does with the unshared 4K stacks
> can then still nicely not select it (or fix the code if possible).

If they even realize that it's the cause of the problem. In the
meantime, we're generating more bug reports to lkml. As the general
opinion is that the ones getting received now aren't getting enough
attention (see regression tracking threads), setting a default that is
known to break setups in hard to debug ways seems counterproductive.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:40 AM, Ray Lee wrote:

> On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:
>> On 07/17/2007 01:13 AM, Ray Lee wrote:
>> 
>> > Given that there's actual, y'know, reports of people who can easily
>> > crash a 4k+interrupt stacks kernel, and not an 8k one, I think the
>> > current evidence speaks for itself.
>>
>> Where?
> 
> The second message in this thread, according to my reader, from Zan
> Lynx. Another from Utz Lehmann a few minutes ago.

True enough. I'm rather wondering though why RHEL is shipping with it if 
it's a _real_ problem. Scribbling junk all over kernel memory would be the 
kind of thing I'd imagine you'd mightely piss-off enterprise customers with. 
But well, sure, that rather quickly becomes a self-referential argument I guess.

>> Removing any such option was not the objective of this thread, just 
>> lifting 4K stacks from debug and making it the default.
> 
> True, but your messages are reading as advocacy for removing the 8k
> option. I'm saying that's a bad idea. If I misunderstood your
> position, then my bad.

I personally believe that CONFIG_4KSTACKS is not better only for very few 
users but well, no, if even some users exist, then I wouldn't want to 
suggest they'd be disallowed (shared or unshared) 8K stacks.

I _would_ in fact suggest there are few enough left that rather than 4K, 8K 
should really be the option that only those few would select, but ofcourse, 
given config defaults that's mostly a matter of semantics, so who cares in 
the end.

> If they even realize that it's the cause of the problem. In the
> meantime, we're generating more bug reports to lkml. As the general
> opinion is that the ones getting received now aren't getting enough
> attention (see regression tracking threads), setting a default that is
> known to break setups in hard to debug ways seems counterproductive.

Well, no. "oldconfig" works fine, and other than that, all failure modes 
I've heard about also in this thread are MD/LVM/XFS. This is extremely 
widely tested stuff in at least Fedora and RHEL. "hard to debug" is simply 
not the case -- every one will immediately start yelling 4KSTACKS when that 
software-stack appears anywhere.

Don't try and hang this off generic development unease... ;-)

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Shawn Bohrer]

On Tue, Jul 17, 2007 at 02:57:45AM +0200, Rene Herman wrote:
>  True enough. I'm rather wondering though why RHEL is shipping with it if 
>  it's a _real_ problem. Scribbling junk all over kernel memory would be the 
>  kind of thing I'd imagine you'd mightely piss-off enterprise customers with. 
>  But well, sure, that rather quickly becomes a self-referential argument I 
>  guess.

I can't speak for Fedora, but RHEL disables XFS in their kernel likely
because it is known to cause problems with 4K stacks.

>  Well, no. "oldconfig" works fine, and other than that, all failure modes 
>  I've heard about also in this thread are MD/LVM/XFS. This is extremely 
>  widely tested stuff in at least Fedora and RHEL.

Again don't assume that because Fedora and RHEL have 4K stacks means
that MD/LVM/XFS is widely tested.

Additionally I think I should point out that the problems pointed out so
far are not the only problem areas with 4K stacks.  There are out of
tree drivers to consider as well, and use cases like ndiswrapper.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 06:14 PM, Shawn Bohrer wrote:

> I can't speak for Fedora, but RHEL disables XFS in their kernel likely
> because it is known to cause problems with 4K stacks.

Okay. So is it fair to say it's largely XFS that's the problem? No problems 
with LVM/MD and say plain ext? If that's the case, I believe it could be 
concluded that it's not something in any sense fundamentally unfixable and 
the question becomes why XFS isn't fixed...

>>  Well, no. "oldconfig" works fine, and other than that, all failure modes 
>>  I've heard about also in this thread are MD/LVM/XFS. This is extremely 
>>  widely tested stuff in at least Fedora and RHEL.
> 
> Again don't assume that because Fedora and RHEL have 4K stacks means
> that MD/LVM/XFS is widely tested.

No, quite, that specific combination was reported in this thread alone 3 
times again, so that one's clear, but _other_ than that, I've heard of no 
other failure modes.

> Additionally I think I should point out that the problems pointed out so
> far are not the only problem areas with 4K stacks.  There are out of
> tree drivers to consider as well, and use cases like ndiswrapper.

Except these. Good to have pointed out, thanks, but as far as I'm concerned 
both these cases do not get a say in what's default configuration for the 
kernel.org kernel. They might get a say in what's removed or not removed 
from that kernel but that's not under discussion at the moment (nor would I 
expect it to be anytime soon if ever).

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Zan Lynx]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

On Tue, 2007-07-17 at 18:52 +0200, Rene Herman wrote:
> On 07/17/2007 06:14 PM, Shawn Bohrer wrote:
> 
> > I can't speak for Fedora, but RHEL disables XFS in their kernel likely
> > because it is known to cause problems with 4K stacks.
> 
> Okay. So is it fair to say it's largely XFS that's the problem? No problems 
> with LVM/MD and say plain ext?

There *are* crashes from LVM and ext3.  I had to change kernels to avoid
them.

I had crashes with ext3 on LVM snapshot on DM mirror on SATA.
-- 
Zan Lynx <zlynx@acm.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> I can't speak for Fedora, but RHEL disables XFS in their kernel likely
> because it is known to cause problems with 4K stacks.

-was- - the SGI folks submitted patches to deal with some gcc problems
with stack usage.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Tue, 17 Jul 2007, Rene Herman wrote:
> On 07/17/2007 12:37 AM, Ray Lee wrote:
> > On 7/16/07, Rene Herman <rene.herman@gmail.com> wrote:

> > If at some point one of the pro-4k stacks crowd can prove that all
> > code paths are safe
> 
> I'll do that the minute you prove the current shared 8K stacks are safe. Do 
> we have a deal?

You claim 4k+4k is safe, therefore 8k must be safe, too. But if 8k is 
safe, this does not yet prove that you can store 5k+3k in 4k+4k.
-- 
Funny quotes:
38. Last night I played a blank tape at full blast. The mime next door went
    nuts.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:45 AM, Bodo Eggert wrote:

> On Tue, 17 Jul 2007, Rene Herman wrote:
>> On 07/17/2007 12:37 AM, Ray Lee wrote:

>>> If at some point one of the pro-4k stacks crowd can prove that all 
>>> code paths are safe
>> I'll do that the minute you prove the current shared 8K stacks are
>> safe. Do we have a deal?
> 
> You claim 4k+4k is safe, therefore 8k must be safe, too.

No, I most certainly do not. I claim proving that 4K and seperate (per cpu) 
interrupt stacks are safe are exactly the same as proving unshared 8K stacks 
are safe. That is, you don't, no such proof exists other than in the eating 
of the pudding. Ray (and you) in considering !CONFIG_4KSTACKS to be "safer" 
than CONFIG_4KSTACKS suggest that _inevitably_ CONFIG_4KSTACKS would leave 
you with less available stack and I pointed out this isn't be the case.

And in fact, I shouldn't have said "exactly" the same. Unshared interrupt 
stacks make for more determistisc behaviour, so you'd have a harder time 
proven anything to some set limit of uncertainty with the shared 8K stacks 
than with the unshared 4K stacks.

> But if 8k is safe, this does not yet prove that you can store 5k+3k in
> 4k+4k.

I really have not made any claim of the kind. The argument is that with 
CONFIG_4KSTACKS, availeble stack space isn't inevitably less at any point in 
time.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Tue, 17 Jul 2007, Rene Herman wrote:
> On 07/17/2007 01:45 AM, Bodo Eggert wrote:
> > On Tue, 17 Jul 2007, Rene Herman wrote:
> >> On 07/17/2007 12:37 AM, Ray Lee wrote:

> >>> If at some point one of the pro-4k stacks crowd can prove that all 
> >>> code paths are safe
> >> I'll do that the minute you prove the current shared 8K stacks are
> >> safe. Do we have a deal?
> > 
> > You claim 4k+4k is safe, therefore 8k must be safe, too.
> 
> No, I most certainly do not. I claim proving that 4K and seperate (per cpu) 
> interrupt stacks are safe are exactly the same as proving unshared 8K stacks 
> are safe. That is, you don't, no such proof exists other than in the eating 
> of the pudding.

And yet you have a more strict claim than I do. If you are right, I'll be
right, too, because two times less-than-4K is less tham 8K. If I'm wrong
and 8K is not enough, you must be wrong, too, because you can impossibly
fit more than 8K into 4K+4K. That's the law of mathematics.

> Ray (and you) in considering !CONFIG_4KSTACKS to be "safer" 
> than CONFIG_4KSTACKS suggest that _inevitably_ CONFIG_4KSTACKS would leave 
> you with less available stack and I pointed out this isn't be the case.

Why do you insist on 4Kstacks being good as long as there is _one_ usevase 
not crashing the kernel? _All_ usecases have to be safe!

> And in fact, I shouldn't have said "exactly" the same. Unshared interrupt 
> stacks

, which are a completely different thing which was bundled to 4K-stacks
  because you need more than 4K,

> make for more determistisc behaviour, so you'd have a harder time 
> proven anything to some set limit of uncertainty with the shared 8K stacks 
> than with the unshared 4K stacks.

I don't want my stack to overflow in order to be theoretically able to
prove it does not overflow. I'd rather go for 8K+4K-stacks, and if _you_
have done the proof _you_ wanted to make, we can talk again about
4K-stacks. Then I'll just add up the maximum stack usages and have the
proof that 8K stacks are safe.

> > But if 8k is safe, this does not yet prove that you can store 5k+3k in
> > 4k+4k.
> 
> I really have not made any claim of the kind. The argument is that with 
> CONFIG_4KSTACKS, availeble stack space isn't inevitably less at any point in 
> time.

I claim, you can store 5k + 3k on the 8k stack, where 5k is something like
the current worst case for non-interrupt stack and 3k is plenty for
interrupts. Thousands of stable systems with 8K stacks support my claim.

You claimed with 4k + 4k, there is not less available stack space.
(At least for usecases you are interested in, but I'll asume you don't 
 want other usecases to crash.)

If you were right, I'd have enough space on 4k + 4k to store that 5k.
Obviously, thousands of systems disagree by crashing with 4K-stacks.
That's most simple logic.

Off cause I may be wrong and the kernels don't crash because of 4K stacks, 
but because of bad karma ... But even then, you'd first have to get rid of
that bad karma before defaulting to 4K stacks.

-- 
Top 100 things you don't want the sysadmin to say:
41. OH, SH*T! (as they scrabble at the keyboard for ^c).

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 12:06 PM, Bodo Eggert wrote:

> On Tue, 17 Jul 2007, Rene Herman wrote:
>> On 07/17/2007 01:45 AM, Bodo Eggert wrote:

>>> You claim 4k+4k is safe, therefore 8k must be safe, too.
>> 
>> No, I most certainly do not. I claim proving that 4K and seperate (per cpu) 
>> interrupt stacks are safe are exactly the same as proving unshared 8K stacks 
>> are safe. That is, you don't, no such proof exists other than in the eating 
>> of the pudding.
> 
> And yet you have a more strict claim than I do. If you are right, I'll be
> right, too, because two times less-than-4K is less tham 8K.

Firstly, it's not two times 4K but 4K + (4K + 4K) * NR_CPUS. Secondly, _you_ 
are the one making claims -- specifically that !CONFIG_4KSTACKS is "safer", 
happily ignoring the fact that generally speaking available process stack 
can be _better_ with CONFIG_4KSTACKS and there seems to exist but _one_ 
(één, ein, une) known situation where it's problematic.

Must there be none rather than one? In some senses maybe, if the problem is 
more than bad, fixable code but I doubt you know this. CONFIG_4KSTACKS is 
much better on the VM (and hence faster) and as such, any user not using the 
one nicely isolated and identified problem case benefits from it. This means 
it's either very close or already _at_ the point of being the best default 
for the kernel. Changing options is for users with special needs, as you 
believe you are.

I truly apologise for taking it into this direction but you're wearing me 
down rapidly. Every single time you insert some uninformed crap comment that 
shows that you both don't understand the issue and didn't understand what 
the other person was saying and then after being made aware of such, ignore 
that and follow up with the next uninformed crap comment. That is, you seem 
to care less about the issue then about the discussion and since for me it's 
quite the other way around I'm leaving it at that.

RedHat is the one with the actual data available, and they've been enabling 
4KSTACKS for quite some time now (with some of their users apparently 
unhappy about it but not many it would seem).

Jesper also already posted how he's going to proceed: lift 4K from debug 
status and submit it as default for -mm. As to the latter bit, unless I 
remember wrong, it already _was_ default in -mm for some time a while ago so 
Andrew no doubt has an informed opinion on how to proceed with that.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Tue, 17 Jul 2007, Rene Herman wrote:
> On 07/17/2007 12:06 PM, Bodo Eggert wrote:
> > On Tue, 17 Jul 2007, Rene Herman wrote:
> >> On 07/17/2007 01:45 AM, Bodo Eggert wrote:

> >>> You claim 4k+4k is safe, therefore 8k must be safe, too.
> >> 
> >> No, I most certainly do not. I claim proving that 4K and seperate (per cpu) 
> >> interrupt stacks are safe are exactly the same as proving unshared 8K stacks 
> >> are safe. That is, you don't, no such proof exists other than in the eating 
> >> of the pudding.
> > 
> > And yet you have a more strict claim than I do. If you are right, I'll be
> > right, too, because two times less-than-4K is less tham 8K.
> 
> Firstly, it's not two times 4K but 4K + (4K + 4K) * NR_CPUS. Secondly, _you_ 
> are the one making claims -- specifically that !CONFIG_4KSTACKS is "safer", 
> happily ignoring the fact that generally speaking available process stack 
> can be _better_ with CONFIG_4KSTACKS

It can be better, but the worst case stays 4K + 4K - unless one CPU will 
walk over to the next and nicely ask for a cup of stack.

Therefore you can discuss 4K + 4K or 4K + 4K + 4K, or 4K + 4K * \inf. It 
won't change a thing:

1) It all can be reduced to 4K + 4K by asuming all IRQ happen on one CPU.
2) Even if the interrupts decide not to happen on one CPU, you still can't 
   fit that possible 5K into 4K.

Having a local stack per CPU helps locality, and it's gootd, but that's 
about it.

> and there seems to exist but _one_ 
> (één, ein, une) known situation where it's problematic.

One case is reason enough not to enable 4K-stacks per default, and this 
is a common server setup. "server" as in "I need a reliable system".

> Must there be none rather than one? In some senses maybe, if the problem is 
> more than bad, fixable code but I doubt you know this. CONFIG_4KSTACKS is 
> much better on the VM (and hence faster) and as such,

"Look how fast I crashed!" doesn't buy you anything. In order to finish 
first, you first got to finish.

> any user not using the 
> one nicely isolated and identified problem case benefits from it.

And they can turn it on.

> This means 
> it's either very close or already _at_ the point of being the best default 
> for the kernel. Changing options is for users with special needs, as you 
> believe you are.

If you designed a car, you would also go for breaks with a well-known 
problem just because they weight less and all that people not crossing 
mountains would be happy about the weight benefit - that is if they'd 
notice, wouldn't you?

Please post a list of things you have designed, so I can avoid them.

> I truly apologise for taking it into this direction but you're wearing me 
> down rapidly.

I put the facts onto the ground. If you're getting down, you may stumble 
on them. Beware

> Every single time you insert some uninformed crap comment that 
> shows that you both don't understand the issue and didn't understand what 
> the other person was saying and then after being made aware of such, ignore 
> that and follow up with the next uninformed crap comment.

So what did you say about the worst case stack size being bigger than 4K?
That's correct, you choose to put it aside as a minor use case. Yea, it's
just the combination you'd choose for a reliable server setup, the users
won't have a problem when their systems crash ...

Was your claim about each CPU having a separate stack helping your cause?
No, everybody can see it's not. That is, except for you, your CPU will
just borrow some, since their neighbours have some free stack.


But let's not stop here: You claimed: "Unshared interrupt 
stacks make for more determistisc behaviour, so you'd have a harder time 
proven anything to some set limit of uncertainty with the shared 8K stacks 
than with the unshared 4K stacks."

So you want to tell me I can't prove 8K stacks are safe - you are right.
But can you prove 4K stacks are safe? You can't either. But you want to be
able to prove it. I told you to stick to your words - go and prove 4K+4K
to be safe. What did you do? You chose to ignore that.

I bet you don't even consider proving 4K stacks to be correct, nor do you
know anyone who would try that in the near future. And besides that, you
know at least one case where your proof would fail. So why would you talk
about proofs? Do you think you can trick me into believing a crashing 
system would work more correctly than a non-crashing one, just by 
mentioning the possibility of having it easier to make a proof?


Besides that, I told you you can separate unshared interrupt stacks from 
4K stacks. And yet, you still argue as if 4K stacks are required for 
having a separate interrupt stack.


So who's ignoring facts, who is talking crap?

> That is, you seem 
> to care less about the issue then about the discussion and since for me it's 
> quite the other way around I'm leaving it at that.

I know very well you're going to turn linux into a bleeding edge system 
where you're supposed to get some bad surprises on an update, unless you 
happen not to hit the error case. That's not the way I'd like it to go.
You may use -mm, you'll get the 4K stacks per default, be happy wih them.

> RedHat is the one with the actual data available, and they've been enabling 
> 4KSTACKS for quite some time now (with some of their users apparently 
> unhappy about it but not many it would seem).

Would you use a distribution crashing on your servers? Or would you change
the distribution and spread the word not to use redhat?

-- 
"'Multiple exclamation marks,' he went on, shaking his head, 'are a 
sure sign of a diseased mind.'"
	-- Terry Pratchett in "Eric"

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Arjan van de Ven]

> 1) It all can be reduced to 4K + 4K by asuming all IRQ happen on one CPU.

no it's separate stacks for soft and hard irqs, so it's really 4+4+4


another angle is that while correctness rules, userspace correctness
rules as well. If you can't fork enough threads for what you need the
machine for, why have the machine in the first place?

-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Tue, 17 Jul 2007, Arjan van de Ven wrote:

> > 1) It all can be reduced to 4K + 4K by asuming all IRQ happen on one CPU.
> 
> no it's separate stacks for soft and hard irqs, so it's really 4+4+4

Thanks, I missed that information. Unfortunately this change still does 
not help if one of these stacks needs to grow beyond 4K.

> another angle is that while correctness rules, userspace correctness
> rules as well. If you can't fork enough threads for what you need the
> machine for, why have the machine in the first place?

Userspace can't work correctly after the kernel crashed, but it can fail 
gracefully if it can't create enough threads.

I'd really like to be able to select 4K stacks, but as long as that stack
would overflow, I can't, and it can't be default, too.
-- 
Top 100 things you don't want the sysadmin to say:
8. ...and after I patched the microcode...

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/18/2007 01:19 AM, Bodo Eggert wrote:

> Please post a list of things you have designed, so I can avoid them.

- The ability to read
- The ability to understand

You're doing a hell of a job already.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Bodo Eggert]

On Wed, 18 Jul 2007, Rene Herman wrote:
> On 07/18/2007 01:19 AM, Bodo Eggert wrote:

> > Please post a list of things you have designed, so I can avoid them.
> 
> - The ability to read
> - The ability to understand
> 
> You're doing a hell of a job already.

If you designed them like you design secure systems, that explains a lot.

-- 
Top 100 things you don't want the sysadmin to say:
83. Damn, and I just bought that pop...

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[utz lehmann]

On Tue, 2007-07-17 at 00:28 +0200, Rene Herman wrote:
> Given that as Arjan stated Fedora and even RHEL have been using 4K stacks 
> for some time now, and certainly the latter being a distribution which I 
> would expect to both host a relatively large number of lvm/md/xfs and what 
> stackeaters have you users and to be fairly conservative with respect to the 
> chances of scribbling over kernel memory (I'm a trusting person...) it seems 
>   there might at this stage only be very few offenders left.

I have to recompile the fedora kernel rpms (fc6, f7) with 8k stacks on
my i686 server. It's using NFS -> XFS -> DM -> MD (raid1) -> IDE disks.
With 4k stacks it crash (hang) within minutes after using NFS.
With 8k stacks it's rock solid. No crashes within months.

utz

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Måns Rullgård]

utz lehmann <lkml123@s2y4n2c.de> writes:

> On Tue, 2007-07-17 at 00:28 +0200, Rene Herman wrote:
>> Given that as Arjan stated Fedora and even RHEL have been using 4K
>> stacks for some time now, and certainly the latter being a
>> distribution which I would expect to both host a relatively large
>> number of lvm/md/xfs and what stackeaters have you users and to be
>> fairly conservative with respect to the chances of scribbling over
>> kernel memory (I'm a trusting person...) it seems there might at
>> this stage only be very few offenders left.
>
> I have to recompile the fedora kernel rpms (fc6, f7) with 8k stacks on
> my i686 server. It's using NFS -> XFS -> DM -> MD (raid1) -> IDE disks.
> With 4k stacks it crash (hang) within minutes after using NFS.
> With 8k stacks it's rock solid. No crashes within months.

Running either vanilla or Gentoo kernels, I've had no luck at all with
4k stacks.  On SATA+MD(raid1)+LVM+XFS I've had numerous crashes when
doing even light I/O.  One particularly crash-prone scenario is
writing to random locations in sparse files, thus forcing new blocks
to be allocated.

I don't have dedicated testing machines, so I can't afford the time
and potential data loss of testing this regularly.  I have no shortage
on RAM with 8k stacks, so for me the choice is quite simple.

-- 
Måns Rullgård
mans@mansr.com

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:26 AM, utz lehmann wrote:

> I have to recompile the fedora kernel rpms (fc6, f7) with 8k stacks on my
> i686 server. It's using NFS -> XFS -> DM -> MD (raid1) -> IDE disks. With
> 4k stacks it crash (hang) within minutes after using NFS. With 8k stacks
> it's rock solid. No crashes within months.

Okay, thanks. That's the usual offender. And only one I've heard of...

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[John Stoffel]

utz> On Tue, 2007-07-17 at 00:28 +0200, Rene Herman wrote:
>> Given that as Arjan stated Fedora and even RHEL have been using 4K stacks 
>> for some time now, and certainly the latter being a distribution which I 
>> would expect to both host a relatively large number of lvm/md/xfs and what 
>> stackeaters have you users and to be fairly conservative with respect to the 
>> chances of scribbling over kernel memory (I'm a trusting person...) it seems 
>> there might at this stage only be very few offenders left.

utz> I have to recompile the fedora kernel rpms (fc6, f7) with 8k
utz> stacks on my i686 server. It's using NFS -> XFS -> DM -> MD
utz> (raid1) -> IDE disks.  With 4k stacks it crash (hang) within
utz> minutes after using NFS.  With 8k stacks it's rock solid. No
utz> crashes within months.

Does it give any useful information when it does crash?  Can you make
a simple test case using ram disks instead of IDE disks and then
building upon that?  

I think I should try to do this myself at some point...

John

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[utz lehmann]

On Tue, 2007-07-17 at 10:45 -0400, John Stoffel wrote:
> utz> I have to recompile the fedora kernel rpms (fc6, f7) with 8k
> utz> stacks on my i686 server. It's using NFS -> XFS -> DM -> MD
> utz> (raid1) -> IDE disks.  With 4k stacks it crash (hang) within
> utz> minutes after using NFS.  With 8k stacks it's rock solid. No
> utz> crashes within months.
> 
> Does it give any useful information when it does crash?
>  
No, sorry. Nearly always it lock up so hard that even sysrq didn't work
anymore. Most times the console was blanked. If not, there was a line
with "do_irq" or something like that (if i remember correctly).
A few times it continuous oopsing (scrolling like mad).

I think it's just a stack overflow. Knowing that XFS + long IO stack
have problems with 4k stacks. And i have zero crashes with the
recompiled 8k stack kernels. (All kernel are the fedora ones).

Btw: In the past the server runs on slightly different hardware and
without raid1 (NFS -> XFS -> DM -> IDE disk). It runs with 4k stacks. I
had a few crashes, but i blame the hardware for it.

I don't want to make tests with the server. It's my main data storage
and i don't want to risk it.

>  Can you make
> a simple test case using ram disks instead of IDE disks and then
> building upon that?

Sorry, i don't think i can do this. My other computer, which i can use
for tests, is x86_64 based.
And IFAIK the problem on the XFS side has something to do with looking
for freespace on many AGs. So maybe a bigger and filled filesystem is
needed. And 50GB ram disks are out of question.

utz

[PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

Hi,

I'm wondering if it's time to make 4K stacks the default and to start 
considering removing the 8K stack option alltogether soon?

One of the big problem spots was XFS, but that got some stack usage 
fixes recently, and the 4K stack option has been around for quite a 
while now, so people really should have gotten around to fixing any 
code that can't handle it.   Are there still any big problem areas 
remaining?

As a starting point, something like the patch below that makes 4K 
stacks default and make them no longer depend on DEBUG_KERNEL 
could be the way to go. Then after a while of this being in effect we 
could remove the option of a 8K stack alltogether.

What say you?


Signed-off-by: Jesper Juhl <jesper.juhl@gmail.com>
---
 arch/i386/Kconfig.debug |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/arch/i386/Kconfig.debug b/arch/i386/Kconfig.debug
index b31c080..9b23bff 100644
--- a/arch/i386/Kconfig.debug
+++ b/arch/i386/Kconfig.debug
@@ -58,7 +58,7 @@ config DEBUG_RODATA
 
 config 4KSTACKS
 	bool "Use 4Kb for kernel stacks instead of 8Kb"
-	depends on DEBUG_KERNEL
+	default y
 	help
 	  If you say Y here the kernel will use a 4Kb stacksize for the
 	  kernel stack attached to each process/thread. This facilitates




-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Zan Lynx]

Jesper Juhl wrote:
> Hi,
>
> I'm wondering if it's time to make 4K stacks the default and to start 
> considering removing the 8K stack option alltogether soon?
>
> One of the big problem spots was XFS, but that got some stack usage 
> fixes recently, and the 4K stack option has been around for quite a 
> while now, so people really should have gotten around to fixing any 
> code that can't handle it.   Are there still any big problem areas 
> remaining?
>   
Has anyone fixed the infrequent crashes with 4K stacks and ext3 -> LVM 
snapshot -> LVM -> DM mirror -> libata?

The latest (as of 150 days ago) Fedora 6 32-bit kernels crashed every 
week or two, the 64-bit kernel, which I believe uses 8K stacks, has been 
up 150 days.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Daniel Phillips]

On Wednesday 11 July 2007 10:54, Zan Lynx wrote:
> Jesper Juhl wrote:
> > Hi,
> >
> > I'm wondering if it's time to make 4K stacks the default and to start
> > considering removing the 8K stack option alltogether soon?
> >
> > One of the big problem spots was XFS, but that got some stack usage
> > fixes recently, and the 4K stack option has been around for quite a
> > while now, so people really should have gotten around to fixing any
> > code that can't handle it.   Are there still any big problem areas
> > remaining?
>
> Has anyone fixed the infrequent crashes with 4K stacks and ext3 -> LVM
> snapshot -> LVM -> DM mirror -> libata?

Ahem: ext3 -> LVM snapshot -> LVM -> DM mirror -> DM crypt -> md -> libata,
or worse.

No, it's not fixed.  The model is wrong.  Virtual block drivers should not
be callling submit_bio.  The recursive IO submissions should be handled
on a dedicated stack, most probably allocated as part of the request queue.
This could be done easily in device mapper and md, or better, in
submit_bio.

Regards,

Daniel

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Neil Brown]

On Wednesday July 11, phillips@phunq.net wrote:
> On Wednesday 11 July 2007 10:54, Zan Lynx wrote:
> > Jesper Juhl wrote:
> > > Hi,
> > >
> > > I'm wondering if it's time to make 4K stacks the default and to start
> > > considering removing the 8K stack option alltogether soon?
> > >
> > > One of the big problem spots was XFS, but that got some stack usage
> > > fixes recently, and the 4K stack option has been around for quite a
> > > while now, so people really should have gotten around to fixing any
> > > code that can't handle it.   Are there still any big problem areas
> > > remaining?
> >
> > Has anyone fixed the infrequent crashes with 4K stacks and ext3 -> LVM
> > snapshot -> LVM -> DM mirror -> libata?
> 
> Ahem: ext3 -> LVM snapshot -> LVM -> DM mirror -> DM crypt -> md -> libata,
> or worse.
> 
> No, it's not fixed.  The model is wrong.  Virtual block drivers should not
> be callling submit_bio.  The recursive IO submissions should be handled
> on a dedicated stack, most probably allocated as part of the request queue.
> This could be done easily in device mapper and md, or better, in
> submit_bio.

Maybe you should read that latest kernel source code.  Particularly
generic_make_request in block/ll_rw_blk.c.
The comment preceding it read:

/*
 * We only want one ->make_request_fn to be active at a time,
 * else stack usage with stacked devices could be a problem.
 * So use current->bio_{list,tail} to keep a list of requests
 * submited by a make_request_fn function.
 * current->bio_tail is also used as a flag to say if
 * generic_make_request is currently active in this task or not.
 * If it is NULL, then no make_request is active.  If it is non-NULL,
 * then a make_request is active, and new requests should be added
 * at the tail
 */

NeilBrown

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Daniel Phillips]

On Wednesday 11 July 2007 15:09, Neil Brown wrote:
> > > Has anyone fixed the infrequent crashes with 4K stacks and ext3
> > > -> LVM snapshot -> LVM -> DM mirror -> libata?
> >
> > Ahem: ext3 -> LVM snapshot -> LVM -> DM mirror -> DM crypt -> md ->
> > libata, or worse.
> >
> > No, it's not fixed.  The model is wrong.  Virtual block drivers
> > should not be callling submit_bio.  The recursive IO submissions
> > should be handled on a dedicated stack, most probably allocated as
> > part of the request queue. This could be done easily in device
> > mapper and md, or better, in submit_bio.
>
> Maybe you should read that latest kernel source code.  Particularly
> generic_make_request in block/ll_rw_blk.c.

And plus you've had that one sitting around since 2005, hats off for
nailing the issue from way out.  Sorry for missing the action, I was 
elsewhere.  Niggles begin here.

I'm not sure I like the additional task_struct encumbrance when the
functions themselves could sort it out, albeit with an API change
affecting a gaggle of md and dm drivers.

Hopefully there are other users of the bio list fields, otherwise I
would point out that a per-queue stack is less memory than two
per-bio fields.  I didn't go delving that far.

The pointer to the description of the barrier deadlock is not right, it
points to the problem report when it really out to point to the
definitive analysis and include a subject line, because list archives
come and go:

   [PATCH] block: always requeue !fs requests at the front
   http://thread.gmane.org/gmane.linux.kernel/537473

Is there a good reason why we should not just put the whole
analysis from Tejun Heo in as a comment?  It is terse enough.

In other words, looks good to me :)

Regards,

Daniel

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[David Chinner]

On Wed, Jul 11, 2007 at 07:16:34PM +0200, Jesper Juhl wrote:
> Hi,
> 
> I'm wondering if it's time to make 4K stacks the default and to start 
> considering removing the 8K stack option alltogether soon?
> 
> One of the big problem spots was XFS, but that got some stack usage 
> fixes recently, and the 4K stack option has been around for quite a 
> while now, so people really should have gotten around to fixing any 
> code that can't handle it.   Are there still any big problem areas 
> remaining?

Yes. NFS on top of XFS on top of DM/MD.

XFS got some fixes recently, of which the main one was to fix the
regressions that gcc 4.x introduced by inlining single use functions
that we'd split out of the common code to reduce stack usage. So in
reality, there hasn't been a great deal of change in the status quo
here.

In fact, things have probably got worse because the generic
writeback code is slowly increasing it's stack usage....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Ray Lee]

On 7/11/07, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> I'm wondering if it's time to make 4K stacks the default and to start
> considering removing the 8K stack option alltogether soon?

Why? Leaving the option for 8k stacks isn't killing any kittens, AFAICS.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 12/07/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> On 7/11/07, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> > I'm wondering if it's time to make 4K stacks the default and to start
> > considering removing the 8K stack option alltogether soon?
>
> Why? Leaving the option for 8k stacks isn't killing any kittens, AFAICS.
>
Well, as long as 8K stacks are an option we'll never get all the code
that doesn't work properly with 4K stacks fixed. We *might* get all
the in-tree code fixed eventually, but there's still going to be lots
of out-of-tree code that will keep depending on 8K stacks. And as long
as 4K stacks are not reliable, distributions are stick with 8K for
their kernels, people who build their own kernel are probably also
going to stick to 8K most of the time, since that's the default.  This
makes the 4K stack option fairly pointless for most users.

If we consider 4K stacks broken, not used much, then we should just
get rid of that option. That would remove a config choice (of which we
have IMHO way too many already) and it would remove code that we then
no longer need to maintain.

If, on the other hand, we consider 4K stacks to be the superior
solution, then we should work to get all code fixed to be able to
handle it so that it's actually something distros will start to enable
so that we can eventually get rid of the 8K stack option alltogether.
Making 4K stacks default and no longer a debug option is just the
first step in that direction.


-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jeremy Fitzhardinge]

Jesper Juhl wrote:
> If, on the other hand, we consider 4K stacks to be the superior
> solution, then we should work to get all code fixed to be able to
> handle it so that it's actually something distros will start to enable
> so that we can eventually get rid of the 8K stack option alltogether.
> Making 4K stacks default and no longer a debug option is just the
> first step in that direction.

First step is to apply wli's patch which makes separate interrupt stacks 
orthogonal to the 4K stack option.

    J

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jesper Juhl]

On 13/07/07, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> Jesper Juhl wrote:
> > If, on the other hand, we consider 4K stacks to be the superior
> > solution, then we should work to get all code fixed to be able to
> > handle it so that it's actually something distros will start to enable
> > so that we can eventually get rid of the 8K stack option alltogether.
> > Making 4K stacks default and no longer a debug option is just the
> > first step in that direction.
>
> First step is to apply wli's patch which makes separate interrupt stacks
> orthogonal to the 4K stack option.
>
Yes and no.  If that will get things moving in the direction of
getting rid of the stack size as a config option, then I'm all for it.
But on the other hand it is my personal opinion that this is an area
where we should just make up our minds as to whether we want 4K or 8K
stacks and whether we want interrupt stacks or not, and then not have
it configurable at all.  I believe the goal is 4K stacks + interrupt
stacks, so let's just aim for that and get rid of the configurability
of the damn thing - make a choice, make it work, make it be that
that's what we use and rid ourselves of the alternatives...

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jeremy Fitzhardinge]

Jesper Juhl wrote:
> Yes and no.  If that will get things moving in the direction of
> getting rid of the stack size as a config option, then I'm all for it.
> But on the other hand it is my personal opinion that this is an area
> where we should just make up our minds as to whether we want 4K or 8K
> stacks and whether we want interrupt stacks or not, and then not have
> it configurable at all.  

Well, smaller stacks are better where possible, but there's nothing 
magic about 4k.  Sure, its mostly enough, but there's no particular 
reason to believe it will be enough for everything.  You could state a 
priori that all kernel code paths must fit into 4k of stack, but that's 
pretty arbitrary.

I'm tempted to post a patch making XFS depend on 8K stacks.  I know is 
simplistic, given that it seems to be xfs+lvm which is particularly 
problematic, and other filesystems+lvm are also a problem.  But its 
better than crashing.

    J

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jan Engelhardt]

On Jul 12 2007 21:14, Jeremy Fitzhardinge wrote:
>
> Well, smaller stacks are better where possible, but there's nothing magic about
> 4k.

There is. It is exactly one page on x86. I cannot remember who said it when,
but anything greater than a page implies some penalty.

> Sure, its mostly enough, but there's no particular reason to believe it
> will be enough for everything.  You could state a priori that all kernel
> code paths must fit into 4k of stack, but that's pretty arbitrary.

	Jan
-- 

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/13/2007 06:14 AM, Jeremy Fitzhardinge wrote:

> Jesper Juhl wrote:

>> Yes and no.  If that will get things moving in the direction of
>> getting rid of the stack size as a config option, then I'm all for it.
>> But on the other hand it is my personal opinion that this is an area
>> where we should just make up our minds as to whether we want 4K or 8K
>> stacks and whether we want interrupt stacks or not, and then not have
>> it configurable at all.  
> 
> Well, smaller stacks are better where possible, but there's nothing 
> magic about 4k.

Except offcourse it happens to also be the value of PAGE_SIZE at least on x86...

> Sure, its mostly enough, but there's no particular reason to believe it
> will be enough for everything.  You could state a priori that all kernel
> code paths must fit into 4k of stack, but that's pretty arbitrary.

As far as I'm aware, the actual reason for 4K stacks is that after the 
system has been up and running for some time getting "1 physically 
contiguous pages" becomes significantly easier than 2 which wouldn't be 
arbitrary.

Rene

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:
> On 07/13/2007 06:14 AM, Jeremy Fitzhardinge wrote:
> 
> >Jesper Juhl wrote:
> 
> >>Yes and no.  If that will get things moving in the direction of
> >>getting rid of the stack size as a config option, then I'm all for it.
> >>But on the other hand it is my personal opinion that this is an area
> >>where we should just make up our minds as to whether we want 4K or 8K
> >>stacks and whether we want interrupt stacks or not, and then not have
> >>it configurable at all.  
> >
> >Well, smaller stacks are better where possible, but there's nothing 
> >magic about 4k.
> 
> Except offcourse it happens to also be the value of PAGE_SIZE at least on 
> x86...
> 
> >Sure, its mostly enough, but there's no particular reason to believe it
> >will be enough for everything.  You could state a priori that all kernel
> >code paths must fit into 4k of stack, but that's pretty arbitrary.
> 
> As far as I'm aware, the actual reason for 4K stacks is that after the 
> system has been up and running for some time getting "1 physically 
> contiguous pages" becomes significantly easier than 2 which wouldn't be 
> arbitrary.

If there are exactly two free pages in the system, the odds of them
being buddies (ie adjacent AND properly aligned) is quite small. The
available page pool has to grow quite a bit before the availability of
order-1 page pairs approaches 100%. 

So if we fail to allocate an 8k stack when we could have allocated a
4k stack, we're almost certainly failing significantly prematurely.

As I've pointed out before, it's fairly easy to make our stack
growable with a trampoline in the troublesome paths. Something like:

int growstack(int headroom, int func, void *data)
{
	void *new_stack;
	int ret;

	if (likely(available_stack() > headroom))
		return func(data);

#ifdef CONFIG_GROWSTACK_STATS
	/* gather statistics about stack-heavy paths */
#endif
	/* warn/abort if we're recursing too deeply */

	new_stack = get_free_page();
	switch_to_new_stack(new_stack);
	ret = func(data);
	cleanup_stack(new_stack);
	return ret;
}

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/14/2007 09:17 PM, Matt Mackall wrote:

> On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:

>> As far as I'm aware, the actual reason for 4K stacks is that after the 
>> system has been up and running for some time getting "1 physically 
>> contiguous pages" becomes significantly easier than 2 which wouldn't be 
>> arbitrary.
> 
> If there are exactly two free pages in the system, the odds of them
> being buddies (ie adjacent AND properly aligned) is quite small. The
> available page pool has to grow quite a bit before the availability of
> order-1 page pairs approaches 100%. 
> 
> So if we fail to allocate an 8k stack when we could have allocated a
> 4k stack, we're almost certainly failing significantly prematurely.

Quite. Ofcourse, saying "our stacks are 1 page" would be the by far easiest 
solution to that. Personally, I've been running with 4K stacks exclusively 
on a variety of machines for quite some time now, but I can't say I'm all 
too adventurous with respect to filesystems (especially) so I'm not sure how 
many problems remain with 4K stacks. I did recently see Andrew Morton say 
that problems _do_ still exist. If it's just XFS -- well, heck...

Moreover though, rather than 4K, the issue is "single page" stacks meaning a 
larger (soft-) pagesize would seem to fix things nicely. I've been reading 
about that on this list off and on for some time -- no idea where that 
stands though.

> As I've pointed out before, it's fairly easy to make our stack
> growable with a trampoline in the troublesome paths. Something like:
> 
> int growstack(int headroom, int func, void *data)
> {
> 	void *new_stack;
> 	int ret;
> 
> 	if (likely(available_stack() > headroom))
> 		return func(data);
> 
> #ifdef CONFIG_GROWSTACK_STATS
> 	/* gather statistics about stack-heavy paths */
> #endif
> 	/* warn/abort if we're recursing too deeply */
> 
> 	new_stack = get_free_page();
> 	switch_to_new_stack(new_stack);
> 	ret = func(data);
> 	cleanup_stack(new_stack);
> 	return ret;
> }

This would also need something to tell func() where its current_thread_info 
  is now at. Which might not be much of a problem. Can't think of much else 
either but it's the kind of thing you'd _like_ to be a problem just to have 
an excuse to shoot down an icky notion like that...

Would you intend this just as a "make this path work until we fix it 
properly" kind of thing?

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Sun, Jul 15, 2007 at 12:19:15AM +0200, Rene Herman wrote:
> On 07/14/2007 09:17 PM, Matt Mackall wrote:
> 
> >On Fri, Jul 13, 2007 at 03:20:54PM +0200, Rene Herman wrote:
> 
> >>As far as I'm aware, the actual reason for 4K stacks is that after the 
> >>system has been up and running for some time getting "1 physically 
> >>contiguous pages" becomes significantly easier than 2 which wouldn't be 
> >>arbitrary.
> >
> >If there are exactly two free pages in the system, the odds of them
> >being buddies (ie adjacent AND properly aligned) is quite small. The
> >available page pool has to grow quite a bit before the availability of
> >order-1 page pairs approaches 100%. 
> >
> >So if we fail to allocate an 8k stack when we could have allocated a
> >4k stack, we're almost certainly failing significantly prematurely.
> 
> Quite. Ofcourse, saying "our stacks are 1 page" would be the by far easiest 
> solution to that. Personally, I've been running with 4K stacks exclusively 
> on a variety of machines for quite some time now, but I can't say I'm all 
> too adventurous with respect to filesystems (especially) so I'm not sure 
> how many problems remain with 4K stacks. I did recently see Andrew Morton 
> say that problems _do_ still exist. If it's just XFS -- well, heck...

One long-standing problem is DM/LVM. That -may- be fixed now, but I
suspect issues remain.
 
> >As I've pointed out before, it's fairly easy to make our stack
> >growable with a trampoline in the troublesome paths. Something like:
> >
> >int growstack(int headroom, int func, void *data)
> >{
> >	void *new_stack;
> >	int ret;
> >
> >	if (likely(available_stack() > headroom))
> >		return func(data);
> >
> >#ifdef CONFIG_GROWSTACK_STATS
> >	/* gather statistics about stack-heavy paths */
> >#endif
> >	/* warn/abort if we're recursing too deeply */
> >
> >	new_stack = get_free_page();
> >	switch_to_new_stack(new_stack);
> >	ret = func(data);
> >	cleanup_stack(new_stack);
> >	return ret;
> >}
> 
> This would also need something to tell func() where its current_thread_info 
>  is now at.

That'd be handled in the usual way by switch_to_new_stack. That is,
we'd store the location of the old stack at the top of the new stack
and then literally change everything to point to the new stack.

> Which might not be much of a problem. Can't think of much else 
> either but it's the kind of thing you'd _like_ to be a problem just to have 
> an excuse to shoot down an icky notion like that...

It's not any ickier than explicitly calling schedule().

> Would you intend this just as a "make this path work until we fix it 
> properly" kind of thing?

Maybe.

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/17/2007 01:38 AM, Matt Mackall wrote:

> On Sun, Jul 15, 2007 at 12:19:15AM +0200, Rene Herman wrote:

>> Quite. Ofcourse, saying "our stacks are 1 page" would be the by far
>> easiest solution to that. Personally, I've been running with 4K stacks
>> exclusively on a variety of machines for quite some time now, but I
>> can't say I'm all too adventurous with respect to filesystems
>> (especially) so I'm not sure how many problems remain with 4K stacks. I
>> did recently see Andrew Morton say that problems _do_ still exist. If
>> it's just XFS -- well, heck...
> 
> One long-standing problem is DM/LVM. That -may- be fixed now, but I
> suspect issues remain.

Three cases were reported again in this thread alone yes. Problems do seem 
to be nicely isolated to that specific issue...

>>> int growstack(int headroom, int func, void *data)
>>> {

[ ... ]

>>> }

>> This would also need something to tell func() where its current_thread_info 
>>  is now at.
> 
> That'd be handled in the usual way by switch_to_new_stack. That is,
> we'd store the location of the old stack at the top of the new stack
> and then literally change everything to point to the new stack.

I might not understand what you're saying but I don't believe that would do.
The current thread_info _itself_ (ie, the struct itself, not a pointer) is 
located at esp & ~(THREAD_SIZE - 1) meaning you'd either have to copy over 
the struct to the new stack, or forego that historic optimization (don't get 
me wrong, either may be okay).

>> Which might not be much of a problem. Can't think of much else 
>> either but it's the kind of thing you'd _like_ to be a problem just to have 
>> an excuse to shoot down an icky notion like that...
> 
> It's not any ickier than explicitly calling schedule().

Somewhat comparable in notion perhaps, but I disagree on the relative level 
of ickyness. Calling schedule() you do when you know you no longer have to 
hog te CPU and when you know it's safe to do so. Calling via growstack() 
looks to be a "ah, heck, let's err on the safe side since we don't have a 
bleedin' clue otherwise" sort of thing.

>> Would you intend this just as a "make this path work until we fix it 
>> properly" kind of thing?
> 
> Maybe.

If you know, _can_ MD/LVM (and/or XFS) in fact be sanely/timely fixed, or is 
this looking at something fundamental?

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Phillip Susi]

Matt Mackall wrote:
>> As far as I'm aware, the actual reason for 4K stacks is that after the 
>> system has been up and running for some time getting "1 physically 
>> contiguous pages" becomes significantly easier than 2 which wouldn't be 
>> arbitrary.
> 
> If there are exactly two free pages in the system, the odds of them
> being buddies (ie adjacent AND properly aligned) is quite small. The
> available page pool has to grow quite a bit before the availability of
> order-1 page pairs approaches 100%. 
> 
> So if we fail to allocate an 8k stack when we could have allocated a
> 4k stack, we're almost certainly failing significantly prematurely.

Why do the two pages have to be physically contiguous?  The stack just 
needs to be two contiguous pages in virtual memory, but they can map to 
any two pages anywhere in physical memory.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman]

On 07/18/2007 07:19 PM, Phillip Susi wrote:

> Why do the two pages have to be physically contiguous?  The stack just 
> needs to be two contiguous pages in virtual memory, but they can map to 
> any two pages anywhere in physical memory.

As far as I'm aware that's just a consequence of the way linux does memory 
management. If we ignore highmem, virtual memory is simply +/- PAGE_OFFSET 
away from physical so allocating virtually contiguous pages that are _not_ 
physically contiguous requires mapping them somewhere (the vmalloc area) 
which is limited. Given that large number of threads _are_ the problem you 
wouldn't solve things -- you'd again be out of space, although now for a 
different reason.

Rene.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Alan Cox]

> Why do the two pages have to be physically contiguous?  The stack just 
> needs to be two contiguous pages in virtual memory, but they can map to 
> any two pages anywhere in physical memory.

Historically we allowed DMA off the stack on old x86 systems. Removing
that while a good idea would take a lot of auditing. We also have a very
limited vmalloc window for mapped pages and filling that with stacks
would be bad.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Phillip Susi]

Alan Cox wrote:
>> Why do the two pages have to be physically contiguous?  The stack just 
>> needs to be two contiguous pages in virtual memory, but they can map to 
>> any two pages anywhere in physical memory.
> 
> Historically we allowed DMA off the stack on old x86 systems. Removing
> that while a good idea would take a lot of auditing. We also have a very
> limited vmalloc window for mapped pages and filling that with stacks
> would be bad.

Wow, DMA off the stack?  That's just crazy.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Matt Mackall]

On Fri, Jul 13, 2007 at 04:25:56AM +0200, Jesper Juhl wrote:
> On 13/07/07, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> >Jesper Juhl wrote:
> >> If, on the other hand, we consider 4K stacks to be the superior
> >> solution, then we should work to get all code fixed to be able to
> >> handle it so that it's actually something distros will start to enable
> >> so that we can eventually get rid of the 8K stack option alltogether.
> >> Making 4K stacks default and no longer a debug option is just the
> >> first step in that direction.
> >
> >First step is to apply wli's patch which makes separate interrupt stacks
> >orthogonal to the 4K stack option.
> >
> Yes and no.  If that will get things moving in the direction of
> getting rid of the stack size as a config option, then I'm all for it.
> But on the other hand it is my personal opinion that this is an area
> where we should just make up our minds as to whether we want 4K or 8K
> stacks and whether we want interrupt stacks or not, and then not have
> it configurable at all.  I believe the goal is 4K stacks + interrupt
> stacks, so let's just aim for that and get rid of the configurability
> of the damn thing - make a choice, make it work, make it be that
> that's what we use and rid ourselves of the alternatives...

There are two downsides to interrupt stacks:

- an extra 4k/8k used per CPU
- stack switch time

Both of those are pretty minimal. And I'd be surprised if the latter
was measurable. 

-- 
Mathematics is the supreme nostalgia of our time.

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Jeremy Fitzhardinge]

Jesper Juhl wrote:
> One of the big problem spots was XFS, but that got some stack usage 
> fixes recently, and the 4K stack option has been around for quite a 
> while now, so people really should have gotten around to fixing any 
> code that can't handle it.   Are there still any big problem areas 
> remaining?
>   

I get frequent crashes when I use 4k stacks with XFS+lvm, using recent 
kernels.  When did XFS stack use reductions go in?

    J

Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[David Chinner]

On Wed, Jul 11, 2007 at 11:08:36PM -0700, Jeremy Fitzhardinge wrote:
> Jesper Juhl wrote:
> >One of the big problem spots was XFS, but that got some stack usage 
> >fixes recently, and the 4K stack option has been around for quite a 
> >while now, so people really should have gotten around to fixing any 
> >code that can't handle it.   Are there still any big problem areas 
> >remaining?
> >  
> 
> I get frequent crashes when I use 4k stacks with XFS+lvm, using recent 
> kernels.  When did XFS stack use reductions go in?

February. They were fixes for gcc4.x stack usage regressions (i.e. a ~25%
increase) over gcc 3.3.5. They were not new stack reduction patches....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group