Preempt note in the logs

Preempt note in the logs

[Dag Nygren]

Hi,

just installed the preempt stuff on a 2.4.19 here
running on a laptop and keep getting the
"dreaded"  exited with preempt_count 1
messages.
there is modprobe,kdeinit,aplay,less,find,grep,automount
and others misbehaving.

As this is a laptop and I would like to save as much battery as
I can by keeping the disk not spinning, this behaviour will
flush the log to the disk all the time.

Would it be too much to ask if I requested that the creator of
this very useful patch would make it possible to switch this note
off through a sysctl ?

BRGDS


-- 
Dag Nygren                               email: dag@newtech.fi
Oy Espoon NewTech Ab                     phone: +358 9 8024910
Träsktorpet 3                              fax: +358 9 8024916
02360 ESBO                              Mobile: +358 400 426312
FINLAND

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

On Sat, 24 Aug 2002, Dag Nygren wrote:
> just installed the preempt stuff on a 2.4.19 here
> running on a laptop and keep getting the
> "dreaded"  exited with preempt_count 1
> messages.
> there is modprobe,kdeinit,aplay,less,find,grep,automount
> and others misbehaving.

Not much. That's just unclean preempt.

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs (and patch)

[Dag Nygren]

[-- Attachment #1: Type: text/plain, Size: 960 bytes --]

> Hi,
> 
> On Sat, 24 Aug 2002, Dag Nygren wrote:
> > just installed the preempt stuff on a 2.4.19 here
> > running on a laptop and keep getting the
> > "dreaded"  exited with preempt_count 1
> > messages.
> > there is modprobe,kdeinit,aplay,less,find,grep,automount
> > and others misbehaving.
> 
> Not much. That's just unclean preempt.

Anyway I cannot have it on the laptop so I made the sysctl patch
I was talking about myself (The one that switches that message on
or off. The default is on as wo. the patch)

It is diffed against a kernel with the ck2 patch applied,
ie. lowlatency+preempt.

Here it is, if you are interested out there:

Best from Finland

-- 
Dag Nygren                               email: dag@newtech.fi
Oy Espoon NewTech Ab                     phone: +358 9 8024910
Träsktorpet 3                              fax: +358 9 8024916
02360 ESBO                              Mobile: +358 400 426312
FINLAND


[-- Attachment #2: preemponoff.patch --]
[-- Type: application/x-patch , Size: 2005 bytes --]

Re: Preempt note in the logs

[Robert Love]

On Sat, 2002-08-24 at 05:45, Thunder from the hill wrote:

> Not much. That's just unclean preempt.

No, that is not just preempt being unclean.

Dan, you have a problem.  Something in your kernel is doing stupid
things with locks and most of your tasks are not even preemptible.

Do you use XFS?  If not, what fs?

	Robert Love

Re: Preempt note in the logs

[Arador]

On 24 Aug 2002 12:15:32 -0400
Robert Love <rml@tech9.net> escribió:
> No, that is not just preempt being unclean.
> 
> Dan, you have a problem.  Something in your kernel is doing stupid
> things with locks and most of your tasks are not even preemptible.

I can see all those messages, too. A lot of tasks (if not all)
seems to exit with a "note: task[PID] exited with preempt_count 1"

The kernel I'm using is 2.4.19-ck2-rmap from Con Colivas
wich can be found at http://kernel.kolivas.net

It's a plain 2.4.29 + 
sched-2.4.19-rc2-A4
preempt-kernel-rml-2.4.19-rc5-3.patch
2.4.19-low-latency.patch
2.4.19-rmap14a or 2.4.19-rc5aa1 05_vm_* (rmap in my case)
 
> Do you use XFS?  If not, what fs?

Reiserfs

Diego Calleja

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

On Sat, 24 Aug 2002, Arador wrote:
> I can see all those messages, too. A lot of tasks (if not all)
> seems to exit with a "note: task[PID] exited with preempt_count 1"

Symptoms confirmed, that is.

> > Do you use XFS?  If not, what fs?
> 
> Reiserfs

IMHO we can exclude reiserfs, at least the code must be clean. I'm on
2.4.19-rc5-aa1 w/reiserfs 3.6, which acts just the hell of a perfect
system. I'm serious. Of course it can be triggering.

We have to check
1. process kicking code
2. process killing code
3. memory allocation code
4. read/write code

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs

[Robert Love]

On Sat, 2002-08-24 at 13:12, Thunder from the hill wrote:

> We have to check
> 1. process kicking code
> 2. process killing code
> 3. memory allocation code
> 4. read/write code

The problem can be anywhere, actually.  #4 (and other file I/O) is the
best bet though, I suspect.

It is caused by any mismatched locking - it is a good debugging check
regardless of preemption.

The only known place I or others see it now is on shutdown, since we
have kernel threads that do not drop the BKL and other locks when they
shutdown since the system is about to halt anyway.

There was some reports of XFS earlier... I thought they were fixed (I
talked to those guys).  Maybe some other rogue module...

	Robert Love

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

On 24 Aug 2002, Robert Love wrote:
> It is caused by any mismatched locking - it is a good debugging check
> regardless of preemption.

Do you think it's useful to temporarily put a lock counter into struct 
task (TEMPORARILY, Linus, temporarily!) and check that as well? Maybe that 
will point us something.

Or we should extend that whole crap a bit so we could see exactly what 
caused the preemption count to rise. I don't know if we can do that, but 
we can try doing that.

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs

[Robert Love]

On Sat, 2002-08-24 at 14:01, Thunder from the hill wrote:

> Do you think it's useful to temporarily put a lock counter into struct 
> task (TEMPORARILY, Linus, temporarily!) and check that as well? Maybe that 
> will point us something.

There already is, it is called preempt_count.

> Or we should extend that whole crap a bit so we could see exactly what 
> caused the preemption count to rise. I don't know if we can do that, but 
> we can try doing that.

A lock trace would be helpful here.  But since I know normal kernel code
is not suspect, all these users need to do is find which oddball module
or patch they are using... e.g. watch it be nvidia.

	Robert Love

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

On 24 Aug 2002, Robert Love wrote:
> On Sat, 2002-08-24 at 14:01, Thunder from the hill wrote:
> 
> > Do you think it's useful to temporarily put a lock counter into struct 
> > task (TEMPORARILY, Linus, temporarily!) and check that as well? Maybe that 
> > will point us something.
> 
> There already is, it is called preempt_count.

Yes, I know about this one. But you never know how many different-type 
locks you'll find. In the worst case there's a kernelful of crap around, 
you know.

> A lock trace would be helpful here.  But since I know normal kernel code
> is not suspect, all these users need to do is find which oddball module
> or patch they are using... e.g. watch it be nvidia.

Hey, it's not all the world that is bad. I'm not just fixed on this bug 
here. I'm suggesting this as a serious finder for broken things which 
could be used all over the place.

(Of course my (and possibly even your) great dream is that we can find 
these things before we boot the broken kernel. Good old genparsetree.pl 
and a few AI might help here.)

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

Hmmm... If we run a program that does neither read nor write, can the 
library opens still trigger the preempt_count++ if it's really the r/w 
code to blaim?

Oh no, it's raining in again!

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs

[Dag Nygren]

Whhops,

Lots of discussion,
but this is what I like with Linux:
lots of people that know what they are talking about
commenting/discussing things.

Anyway:
- I am not running xfs, but I do run resiserfs.
- I also have the ck2 patch that includes the low latency patch,
  could that be it?

My patch got rid of the messages and my laptop behaves again,
but really I think that there is something fishy going on,
the messages shouldn't happen....

I am leaving it in your hands guys, as I don't have the experience
nor the time to go get this one.

BRGDS


-- 
Dag Nygren                               email: dag@newtech.fi
Oy Espoon NewTech Ab                     phone: +358 9 8024910
Träsktorpet 3                              fax: +358 9 8024916
02360 ESBO                              Mobile: +358 400 426312
FINLAND

Re: Preempt note in the logs

[Thunder from the hill]

Hi,

On Sat, 24 Aug 2002, Dag Nygren wrote:
> Anyway:
> - I am not running xfs, but I do run resiserfs.
> - I also have the ck2 patch that includes the low latency patch,
>   could that be it?

So let's be clear on that:

2.4.19+preempt + reiserfs = charm
2.4.19+preempt + ck2 = possibly no problem, dunno
2.4.19+preempt + ck2 + reiserfs = bad dude

ck2 seems to introduce a problem that reiserfs triggers. It's not a 
solution to say anything against reiserfs here, never fix the symptoms, 
but the root cause...

Have a look at ck2 then...

			Thunder
-- 
--./../...-/. -.--/---/..-/.-./..././.-../..-. .---/..-/.../- .-
--/../-./..-/-/./--..-- ../.----./.-../.-.. --./../...-/. -.--/---/..-
.- -/---/--/---/.-./.-./---/.--/.-.-.-
--./.-/-.../.-./.././.-../.-.-.-

Re: Preempt note in the logs

[conman]

Robert

Sorry you're getting involved in this. It all centres around me merging a few of
the more popular performance patches [O(1),preempt,low latency,rmap]. 

The preempt count in the log they're referring to only occurs frequently with my
patches so it's not due to your code; it happens with every single process using
my butchered patch. I based the merging on your latest patch for 2.4.19 and a
previous patch you had created for preempt on O(1) for 2.4.18

http://kernel.kolivas.net

I don't expect you to debug this, and I expect it's a fault in my version of
sched.c (most obvious choice).

Cheers,
Con Kolivas