self piping and context switching

self piping and context switching

[Iker]

Assume a thread is monitoring a set of fd's which include both ends of
a pipe (using poll, for example). If the thread writes to the pipe (in
order to notify itself for whatever reason) is it reasonable to expect
that it will be able to return to its poll loop and get the event
without a context switch? (provided it quickly returns to the poll
loop).

Regards,
Iker

RE: self piping and context switching

[David Schwartz]

> Assume a thread is monitoring a set of fd's which include both ends of
> a pipe (using poll, for example). If the thread writes to the pipe (in
> order to notify itself for whatever reason) is it reasonable to expect
> that it will be able to return to its poll loop and get the event
> without a context switch? (provided it quickly returns to the poll
> loop).

	It's reasonable to expect that this will be the most common case and the
one to optimize. It is unreasonable to fail if this doesn't happen, since
it's not guaranteed to happen. Note that if by "without a context switch"
you really mean without another thread getting a chance to run, then it is
totally unreasonable. On SMP systems and with hyper-threading, threads can
run concurrently.

	DS

Re: [lkml] RE: self piping and context switching

[Iker]

More specifically, I was wondering if the write to the pipe or the call back
into poll involved anything that might prompt the scheduler to replace the
thread in this scenario.

> It's reasonable to expect that this will be the most common case and the
> one to optimize. It is unreasonable to fail if this doesn't happen, since
> it's not guaranteed to happen. Note that if by "without a context switch"
> you really mean without another thread getting a chance to run, then it is
> totally unreasonable.

Are you referring to transitions to/from kernel space? If so, wouldn't the
write
on the pipe and the call to poll both result in transitions?

Regards,
Iker


----- Original Message -----
From: "David Schwartz" <davids@webmaster.com>
To: "Iker" <iker@computer.org>; <linux-kernel@vger.kernel.org>
Sent: Friday, September 12, 2003 10:04 PM
Subject: [lkml] RE: self piping and context switching


>
> > Assume a thread is monitoring a set of fd's which include both ends of
> > a pipe (using poll, for example). If the thread writes to the pipe (in
> > order to notify itself for whatever reason) is it reasonable to expect
> > that it will be able to return to its poll loop and get the event
> > without a context switch? (provided it quickly returns to the poll
> > loop).
>
> It's reasonable to expect that this will be the most common case and the
> one to optimize. It is unreasonable to fail if this doesn't happen, since
> it's not guaranteed to happen. Note that if by "without a context switch"
> you really mean without another thread getting a chance to run, then it is
> totally unreasonable. On SMP systems and with hyper-threading, threads can
> run concurrently.
>
> DS
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

RE: [lkml] RE: self piping and context switching

[David Schwartz]

> More specifically, I was wondering if the write to the pipe or
> the call back
> into poll involved anything that might prompt the scheduler to replace the
> thread in this scenario.

	Sure, it could, but there's no particular reason to expect it to. The one
exception would be if another thread was already blocked waiting for that
particular pipe. In this case, your sending data on the pipe would unblock
that thread, which could trigger a pre-emption.

> > It's reasonable to expect that this will be the most common case and the
> > one to optimize. It is unreasonable to fail if this doesn't
> > happen, since
> > it's not guaranteed to happen. Note that if by "without a
> > context switch"
> > you really mean without another thread getting a chance to run,
> > then it is
> > totally unreasonable.

> Are you referring to transitions to/from kernel space? If so, wouldn't the
> write
> on the pipe and the call to poll both result in transitions?

	Yes, but there's no reason that transition should cause the kernel to want
to change processes. The kernel generally tries to let processes complete
their timeslice. An exception could certainly be if the write to the pipe
causes a new thread to unblock and that that thread has a higher dynamic
priority (which it well might, since it was waiting).

	Personally, I don't think the kernel should ever pre-empt threads/processes
with equivalent static priorities, but that's another whole argument. ;)

	DS

Re: [lkml] RE: self piping and context switching

[Alan Cox]

On Sad, 2003-09-13 at 03:56, Iker wrote:
> More specifically, I was wondering if the write to the pipe or the call back
> into poll involved anything that might prompt the scheduler to replace the
> thread in this scenario.

Unless it happens to cause page faults or fill up the pipe nothing in
paticular.  Sending a message to yourself down a pipe is pretty standard
in event based programs as a way of turning a signal from asynchronous
event and thus nuisance to handle into a message.

Re: [lkml] RE: self piping and context switching

[Jamie Lokier]

Alan Cox wrote:
> On Sad, 2003-09-13 at 03:56, Iker wrote:
> > More specifically, I was wondering if the write to the pipe or the call back
> > into poll involved anything that might prompt the scheduler to replace the
> > thread in this scenario.
> 
> Unless it happens to cause page faults or fill up the pipe nothing in
> paticular.  Sending a message to yourself down a pipe is pretty standard
> in event based programs as a way of turning a signal from asynchronous
> event and thus nuisance to handle into a message.

This "pretty standard" method is the reason Netscape 4.07 hangs on
computers which are too fast: It writes too many events to a pipe
before reading from the pipe, and the pipe fills.

Of course that is a programming error.  Just a reminder to be careful :)

-- Jamie

RE: self piping and context switching

[Robert White]

Not to knarf at you, and not to pollute this list, but depending on what
*else* you might be expecting on that pipe, this kind of code could deadlock
with itself rather easily on the write() to the pipe.  If nothing else is
can write to the pipe, then the pipe has no use that could not be better
served by a more local data item.  A thread should "never" (trademark, all
rights reserved 8-) control *itself* through this kind of mechanism.

It is far better, (and cheaper, and less likely to lose the CPU, etc.) to
have a non-local thread-specific (managed, etc) data item that you check
"just before" the poll.

(from a pure performance perspective) Because *any* kernel call is an
opportunity to lose you your timslice, the sequence you mention is a write()
a poll() and presumably a read() to clear the state.  That is three chances
to get "anti-scheduled" (8-) between your decision to "wake yourself" and
the actual desired processing.

Consider the truth of each of the following:

1) If you can "find" the file descriptor for "this thread" then you could
set up and "find" a simple boolean/integer flag that you could test before
even calling the poll().  [This is a self-wake right? So why poll() if you
can avoid it.]

2) If the pipe-special device/file is private to the thread, the same thing
can be accomplished by a simple queue/stack data structure.  You don't even
need locking because it is private so there is no possibility of contention.

2a) In this private case, the thread can never wake itself inside the poll
anyway, because it is waiting inside the poll() [if you catch my meaning.]

3) If the pipe-special device/file is *NOT* private, but this thread is its
only reader, then it is possible that the pipe could be full, in which case
the write() would block waiting for a read that, by definition, will never
happen.

3a) see item 2a.

4) If the pipe-special device/file is *NOT* private AND there are multiple
readers, then you don't know which thread you would be waking (so the
loss-of-timeslice issue is insoluble) and it is *STILL* possible (though
less likely) for all the threads to form a completely deadlocked dependency
graph with blocking write() semantics.

5) If you do the write() in non-blocking mode, and it would block, you have
no means to communicate what you wanted to communicate, so you would have to
discard the event.  This leads to an unstable model, or it leads to writing
the queue/stack mentioned above to solve this case.

The short version is: If a thread is writing to itself using a pipe, you
almost certainly have a semantic problem that could be solved better and
faster using local data or something.

I can't get more specific without seeing your code.

Rob White.

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of Iker
Sent: Friday, September 12, 2003 6:19 PM
To: linux-kernel@vger.kernel.org
Subject: self piping and context switching

Assume a thread is monitoring a set of fd's which include both ends of
a pipe (using poll, for example). If the thread writes to the pipe (in
order to notify itself for whatever reason) is it reasonable to expect
that it will be able to return to its poll loop and get the event
without a context switch? (provided it quickly returns to the poll
loop).

Regards,
Iker


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: self piping and context switching

[Jamie Lokier]

Robert White wrote:
> The guy is writing on a pipe in an attempt to have a thread signal itself,
> and wanted to know if he could do this in a way that wouldn't cause that
> thread to lose its timeslice.

Indeed he was.  I responded to Alan, though, who was talking of
signals and a classic method of handling them:

Alan Cox wrote:
> Sending a message to yourself down a pipe is pretty standard in
> event based programs as a way of turning a signal from asynchronous
> event and thus nuisance to handle into a message.

Robert:
> Thing the first: NEVER write to ANYTHING that can block (like a pipe) in a
> signal handler.  Period.
>
> Thing the second:  Mixing signals and threads is bad, especially if you
> handle it as naively as you outline below.

I, and I believe Alan, were talking about systems _without_ threads...

I.e. a time-honoured technique for turning signals into events in a
portable program.  Of course modern platforms with good threading
don't need to use the technique.

> There are several good techniques for dealing with SIGALRM and SIGCHLD etc
> that will not deadlock the poll, that doesn't need to use any of the
> longjump() variants, and which doesn't do exterior IO.

All the ones you gave require threads, hence I would not consider any
of them portable.  I'll be impressed if you come up with something
that doesn't use threads, pipes or more signals.

> 2) Pick a signal (I like SIGUSR2) and install a handler for it without the
> SA_RESTART bit set.

That's all very well if you're on a system which _has_ SA_RESTART.
Here are some portability notes:

	- Traditional BSD-like systems don't have SA_RESTART.
	  They always restart some system calls on signals.

	- POSIX specifies that signals always interrupt system
	  calls (according to the GNU libc documentation).
	  You cannot make signals restartable.

	- On some threading systems, establishing a signal handler
	  with SA_RESTART not set will cause it to interrupt systems
	  calls in all thread, not just the one it is destined for.

	  If you used this technique, you'd need to take it into
	  account in the code of all threads.

	- Of course, truly portable code has to assume EINTR may be
	  returned by all interruptible system calls anyway.

	- The select() system call is interrupted by signals
	  even when they have SA_RESTART set anyway.  If that's all
	  you want to interrupt, you don't need to mess with the flag.

> When you get a SIGALRM etc, in the handler you check the state of an
> atomic boolean-equivalent (normally an int, but on some embedded
> platforms you should use a char)

It's called sigatomic_t.

> and, if that boolean is set you promote the signal to the
> non-restarting signal by pthread_kill(ing) the specific thread you
> want to wake.  The poll/select being run in that thread will then
> return -1 instead of restarting.

What was that about not mixing signals and threads?

Even when you have a thread library, and it claims to be pthreads, I'd
consider pthread_kill() the most unlikely construct to work portably,
without crashing poll/select, or failing to terminate it, that we've
come up with so far.

Having seen siglongjmp() crash when it's done during select(), I have
not a lot of faith in things working the unix way on all the platforms
I target, let alone pthreads and signals working as specified.

> The above systems will outperform a write-on-pipe wakeup model "on the
> average" anyway, because you almost never drop your timeslice prematurely
> and you spend way less time in kernel-entry/exit space.

Certainly, optimising for the common case is good and so is saving
system calls.  (Believe me, I am very familiar with saving system calls).

I hope this exchange has been entertaining.  As you can see, we have
very different target platforms in mind when we talk of portability.

-- Jamie