Re: Bash not reacting to Ctrl-C

Re: Bash not reacting to Ctrl-C

[julie Sullivan]

(apologies for not replying to original thread, I wasn't subscribed)

>Might be some Bash assumption or race that works under other OSs but somehow Linux
>does differently. IIRC Bash is being developed on MacOS-X.

Yep, and I can confirm that this behaviour is the same on MacOS-X too
(I tested Ingo's
bash scripts in my break on a machine at work today).
Ctrl-C is lost about once in every 10 times.

>See that '^C^C' line? That is where i had to do Ctrl-C twice.

I beat Ingo - I got three '^C's in a row on the Mac a couple of times :-)

Cheers
Julie

Re: Bash not reacting to Ctrl-C

[Ingo Molnar]

* julie Sullivan <kernelmail.jms@gmail.com> wrote:

> (apologies for not replying to original thread, I wasn't subscribed)
> 
> >Might be some Bash assumption or race that works under other OSs but somehow Linux
> >does differently. IIRC Bash is being developed on MacOS-X.
> 
> Yep, and I can confirm that this behaviour is the same on MacOS-X too
> (I tested Ingo's
> bash scripts in my break on a machine at work today).
> Ctrl-C is lost about once in every 10 times.

Ok, so it's likely a Bash bug.

[ It's annoying nevertheless :) ]

> >See that '^C^C' line? That is where i had to do Ctrl-C twice.
> 
> I beat Ingo - I got three '^C's in a row on the Mac a couple of times :-)

:-)

Thanks,

	Ingo

[PATCHSET] ptrace,signal: group stop / ptrace updates

[Tejun Heo]

Hello,

This is another posting of ptrace and group stop interaction update.
The last posting was split over two patchsets[1][2].  Changes are,

* Rebased on top of v2.6.38-rc2

* 0010-ptrace-clean-transitions-between-TASK_STOPPED-and-TR.patch
  updated as per Oleg's comments - the TRACED/TRAPPING race condition
  closed and trapping clearing separated out from group_stop clearing.

0001-signal-fix-SIGCONT-notification-code.patch
0002-ptrace-remove-the-extra-wake_up_process-from-ptrace_.patch
0003-signal-remove-superflous-try_to_freeze-loop-in-do_si.patch
0004-ptrace-kill-tracehook_notify_jctl.patch
0005-ptrace-add-why-to-ptrace_stop.patch
0006-signal-fix-premature-completion-of-group-stop-when-i.patch
0007-signal-use-GROUP_STOP_PENDING-to-stop-once-for-a-sin.patch
0008-ptrace-participate-in-group-stop-from-ptrace_stop-if.patch
0009-ptrace-make-do_signal_stop-use-ptrace_stop-if-the-ta.patch
0010-ptrace-clean-transitions-between-TASK_STOPPED-and-TR.patch

0001-0004 are cleanup/bugfix patches.  0005-0010 improve group stop
handling.

Discussions are still on-going on the following points.

1. Removal of spurious wake_up_process() by 0002 may not be safe[3].

2. STOPPED -> RUNNING -> TRACED transition window may be visible to
   tasks which are not the tracer[4].  Tracee always entering TRACED
   also causes one ptrace test case to fail[5].

3. After immediately re-attaching to a detached task in stopped state,
   WNOHANG wait(2) may fail.

This patchset does change ptrace behavior but the changed aspects are
somewhere between awkward and outright buggy before the changes and
the changes are visible only through very convoluted use cases.
Regardless of future directions from here, I don't think the patches
posted in this patchset would be a problem.

The patchset is available in the following git tree.

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc.git ptrace-review

Thank you.

 fs/exec.c                 |    1 
 include/linux/sched.h     |   11 ++
 include/linux/tracehook.h |   27 -----
 kernel/ptrace.c           |   51 ++++++++--
 kernel/signal.c           |  226 ++++++++++++++++++++++++++++++++++------------
 5 files changed, 225 insertions(+), 91 deletions(-)

--
tejun

[1] http://thread.gmane.org/gmane.linux.kernel/1079975
[2] http://thread.gmane.org/gmane.linux.kernel/1080700
[3] http://thread.gmane.org/gmane.linux.kernel/1079975/focus=1088490
[4] http://thread.gmane.org/gmane.linux.kernel/1080700/focus=1088538
[5] http://thread.gmane.org/gmane.linux.kernel/1080700/focus=1093056

Re: [PATCHSET] ptrace,signal: group stop / ptrace updates

[Ingo Molnar]

Hi,

I'm hijacking this thread, to report a signal handling bug that Linux and Bash has, 
and which has been there at least for 10 years since i started using SMP Linux 
systems ...

It's not easy to reproduce but today i found a reproducer - maybe you guys have an 
idea what's going on.

There's two very simple scripts, one calls the other in an infinite loop:

 $ cat test-signal
 #!/bin/bash

 while true; do ./test-signal2; done

 $ cat test-signal2
 #!/bin/bash

 true

The bug is that occasionally Ctrl-C does not get processed, and that the Ctrl-C is 
'lost'. It can be reproduced here by running ./test-signal several times, and 
Ctrl-C-ing it:

 $ ./test-signal
 ^C
 $ ./test-signal
 ^C^C
 $ ./test-signal
 ^C

See that '^C^C' line? That is where i had to do Ctrl-C twice.

It only fails here about once every 10 times, so it's very rare. I have a stock F14 
system running on that box, with the very latest .38 based kernel.

Any ideas what's going on?

Thanks,

	Ingo

Re: [PATCHSET] ptrace,signal: group stop / ptrace updates

[Oleg Nesterov]

On 01/28, Ingo Molnar wrote:
>
> The bug is that occasionally Ctrl-C does not get processed, and that the Ctrl-C is
> 'lost'. It can be reproduced here by running ./test-signal several times, and
> Ctrl-C-ing it:
>
>  $ ./test-signal
>  ^C
>  $ ./test-signal
>  ^C^C
>  $ ./test-signal
>  ^C
>
> See that '^C^C' line? That is where i had to do Ctrl-C twice.

Reproduced.

At first glance, /bin/sh should be blamed... Hmm, probably yes,
I even reproduced this under strace, and this is what I see

	wait4(-1, 0x7fff388431c4, 0, NULL) = ? ERESTARTSYS (To be restarted)
	--- SIGINT (Interrupt) @ 0 (0) ---
	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)
	wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9706

So, ^C is not lost, but ./test-signal doesn't want to exit.




This is what ./test-signal does when ^C does work:

	wait4(-1, 0x7fff1c283b74, 0, NULL)      = ? ERESTARTSYS (To be restarted)
	--- SIGINT (Interrupt) @ 0 (0) ---
	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)

OK, it doesn't exit immediately, but then it kills itself:

	wait4(-1, [{WIFSIGNALED(s) && WTERMSIG(s) == SIGINT}], 0, NULL) = 19585
	rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f3c3035b150}, {0x433d30, [], SA_RESTORER, 0x7f3c3035b150}, 8) = 0
	rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f3c3035b150}, {SIG_DFL, [], SA_RESTORER, 0x7f3c3035b150}, 8) = 0
	kill(19584, SIGINT)




Looking into the previous log (when it doesn't exit) again,

	wait4(-1, 0x7fff388431c4, 0, NULL) = ? ERESTARTSYS (To be restarted)
	--- SIGINT (Interrupt) @ 0 (0) ---
	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)
	wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9706
	rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
	--- SIGCHLD (Child exited) @ 0 (0) ---
	wait4(-1, 0x7fff38842d24, WNOHANG, NULL) = -1 ECHILD (No child processes)
	rt_sigreturn(0x8)                       = 0
	rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f3cbdbd0150}, {0x433d30, [], SA_RESTORER, 0x7f3cbdbd0150}, 8) = 0
	rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
	rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
	rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
	clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f3cbe9ab780) = 9707

Perhaps the handler for SIGCHLD clears some internal i_am_going_to_exit flag,
I dunno.

Oleg.

Re: Bash not reacting to Ctrl-C

[Ingo Molnar]

* Oleg Nesterov <oleg@redhat.com> wrote:

> On 01/28, Ingo Molnar wrote:
> >
> > The bug is that occasionally Ctrl-C does not get processed, and that the Ctrl-C is
> > 'lost'. It can be reproduced here by running ./test-signal several times, and
> > Ctrl-C-ing it:
> >
> >  $ ./test-signal
> >  ^C
> >  $ ./test-signal
> >  ^C^C
> >  $ ./test-signal
> >  ^C
> >
> > See that '^C^C' line? That is where i had to do Ctrl-C twice.
> 
> Reproduced.
> 
> At first glance, /bin/sh should be blamed... Hmm, probably yes,
> I even reproduced this under strace, and this is what I see
> 
> 	wait4(-1, 0x7fff388431c4, 0, NULL) = ? ERESTARTSYS (To be restarted)
> 	--- SIGINT (Interrupt) @ 0 (0) ---
> 	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)
> 	wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9706
> 
> So, ^C is not lost, but ./test-signal doesn't want to exit.

Might be some Bash assumption or race that works under other OSs but somehow Linux 
does differently. IIRC Bash is being developed on MacOS-X.

But it's happening all the time (with yum for example - but also with makejobs, as 
Thomas has reported it) - this is simply the first time i managed to reproduce it 
with something really simple.

Thanks,

	Ingo

Re: Bash not reacting to Ctrl-C

[Oleg Nesterov]

On 01/28, Ingo Molnar wrote:
>
> * Oleg Nesterov <oleg@redhat.com> wrote:
>
> > On 01/28, Ingo Molnar wrote:
> > >
> > > The bug is that occasionally Ctrl-C does not get processed, and that the Ctrl-C is
> > > 'lost'. It can be reproduced here by running ./test-signal several times, and
> > > Ctrl-C-ing it:
> > >
> > >  $ ./test-signal
> > >  ^C
> > >  $ ./test-signal
> > >  ^C^C
> > >  $ ./test-signal
> > >  ^C
> > >
> > > See that '^C^C' line? That is where i had to do Ctrl-C twice.
> >
> > Reproduced.
> >
> > At first glance, /bin/sh should be blamed... Hmm, probably yes,
> > I even reproduced this under strace, and this is what I see
> >
> > 	wait4(-1, 0x7fff388431c4, 0, NULL) = ? ERESTARTSYS (To be restarted)
> > 	--- SIGINT (Interrupt) @ 0 (0) ---
> > 	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)
> > 	wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9706
> >
> > So, ^C is not lost, but ./test-signal doesn't want to exit.
>
> Might be some Bash assumption or race that works under other OSs but somehow Linux
> does differently. IIRC Bash is being developed on MacOS-X.
>
> But it's happening all the time (with yum for example - but also with makejobs, as
> Thomas has reported it) - this is simply the first time i managed to reproduce it
> with something really simple.

OK, I seem to understand what happens. Of course I am not sure, I never
looked into these sources before...

Suppose that jctl ^C races with the normal child exit. In this case
waitchld() sets child->status = status (zero in this case) and calls
set_job_status_and_cleanup().

set_job_status_and_cleanup() notice wait_sigint_received and send
SIGINT to itself (termsig_handler (SIGINT)), but somehow it assumes
that the last foreground job should be terminated by SIGINT too:

	 else if (wait_sigint_received && (WTERMSIG (child->status) == SIGINT) &&

Then the next wait_for() clears wait_sigint_received and bash
looses ^C

Oleg.

Re: Bash not reacting to Ctrl-C

[Oleg Nesterov]

On 02/05, Oleg Nesterov wrote:
>
> On 01/28, Ingo Molnar wrote:
> >
> > * Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > > On 01/28, Ingo Molnar wrote:
> > > >
> > > > The bug is that occasionally Ctrl-C does not get processed, and that the Ctrl-C is
> > > > 'lost'. It can be reproduced here by running ./test-signal several times, and
> > > > Ctrl-C-ing it:
> > > >
> > > >  $ ./test-signal
> > > >  ^C
> > > >  $ ./test-signal
> > > >  ^C^C
> > > >  $ ./test-signal
> > > >  ^C
> > > >
> > > > See that '^C^C' line? That is where i had to do Ctrl-C twice.
> > >
> > > Reproduced.
> > >
> > > At first glance, /bin/sh should be blamed... Hmm, probably yes,
> > > I even reproduced this under strace, and this is what I see
> > >
> > > 	wait4(-1, 0x7fff388431c4, 0, NULL) = ? ERESTARTSYS (To be restarted)
> > > 	--- SIGINT (Interrupt) @ 0 (0) ---
> > > 	rt_sigreturn(0)                         = -1 EINTR (Interrupted system call)
> > > 	wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9706
> > >
> > > So, ^C is not lost, but ./test-signal doesn't want to exit.
> >
> > Might be some Bash assumption or race that works under other OSs but somehow Linux
> > does differently. IIRC Bash is being developed on MacOS-X.
> >
> > But it's happening all the time (with yum for example - but also with makejobs, as
> > Thomas has reported it) - this is simply the first time i managed to reproduce it
> > with something really simple.
>
> OK, I seem to understand what happens. Of course I am not sure, I never
> looked into these sources before...
>
> Suppose that jctl ^C races with the normal child exit. In this case
> waitchld() sets child->status = status (zero in this case) and calls
> set_job_status_and_cleanup().
>
> set_job_status_and_cleanup() notice wait_sigint_received and send
> SIGINT to itself (termsig_handler (SIGINT)), but somehow it assumes
> that the last foreground job should be terminated by SIGINT too:
>
> 	 else if (wait_sigint_received && (WTERMSIG (child->status) == SIGINT) &&
>
> Then the next wait_for() clears wait_sigint_received and bash
> looses ^C

IOW.

Now that it is clear what happens, the test-case becomes even more
trivial:

	bash-4.1$ ./bash -c 'while true; do /bin/true; done'
	^C^C

needs 4-5 attempts on my machine.

The patch below fixes the problem, but most probably it is not
correct. Although I don't understand the point of "status == SIGINT"
check, we already checked this job is dead. But I won't pretend I
really understand this code.

Oleg.

--- bash-4.1/jobs.c~ctrlc_exit_race	2011-02-07 13:52:48.000000000 +0100
+++ bash-4.1/jobs.c	2011-02-07 13:55:30.000000000 +0100
@@ -3299,7 +3299,7 @@ set_job_status_and_cleanup (job)
 	 signals are sent to process groups) or via kill(2) to the foreground
 	 process by another process (or itself).  If the shell did receive the
 	 SIGINT, it needs to perform normal SIGINT processing. */
-      else if (wait_sigint_received && (WTERMSIG (child->status) == SIGINT) &&
+      else if (wait_sigint_received /*&& (WTERMSIG (child->status) == SIGINT)*/ &&
 	      IS_FOREGROUND (job) && IS_JOBCONTROL (job) == 0)
 	{
 	  int old_frozen;

Re: Bash not reacting to Ctrl-C

[Michael Witten]

On Mon, Feb 7, 2011 at 07:08, Oleg Nesterov <oleg@redhat.com> wrote:
> Now that it is clear what happens, the test-case becomes even more
> trivial:
>
>        bash-4.1$ ./bash -c 'while true; do /bin/true; done'
>        ^C^C
>
> needs 4-5 attempts on my machine.

I feel like the odd penguin out.

I can't reproduce the behavior in question when using that example (I
haven't tried the other).

I'm running:

    * bash version 4.1.9(2)-release (i686-pc-linux-gnu)

    * linux 2.6.38-rc4 (100b33c8bd8a3235fd0b7948338d6cbb3db3c63d)

Re: Bash not reacting to Ctrl-C

[Ingo Molnar]

* Michael Witten <mfwitten@gmail.com> wrote:

> On Mon, Feb 7, 2011 at 07:08, Oleg Nesterov <oleg@redhat.com> wrote:
> > Now that it is clear what happens, the test-case becomes even more
> > trivial:
> >
> >        bash-4.1$ ./bash -c 'while true; do /bin/true; done'
> >        ^C^C
> >
> > needs 4-5 attempts on my machine.
> 
> I feel like the odd penguin out.
> 
> I can't reproduce the behavior in question when using that example (I
> haven't tried the other).
> 
> I'm running:
> 
>     * bash version 4.1.9(2)-release (i686-pc-linux-gnu)
> 
>     * linux 2.6.38-rc4 (100b33c8bd8a3235fd0b7948338d6cbb3db3c63d)

Oleg provided another testcase, can you reproduce the Ctrl-C problem with this 
it?

#!/bin/bash

perl -we '$SIG{INT} = sub {exit}; sleep'

echo "Hehe, I am going to sleep after ^C"
sleep 100


Thanks,

	Ingo

Re: Bash not reacting to Ctrl-C

[Michael Witten]

On Wed, Feb 9, 2011 at 08:53, Ingo Molnar <mingo@elte.hu> wrote:
>
> * Michael Witten <mfwitten@gmail.com> wrote:
>
>> On Mon, Feb 7, 2011 at 07:08, Oleg Nesterov <oleg@redhat.com> wrote:
>> > Now that it is clear what happens, the test-case becomes even more
>> > trivial:
>> >
>> >        bash-4.1$ ./bash -c 'while true; do /bin/true; done'
>> >        ^C^C
>> >
>> > needs 4-5 attempts on my machine.
>>
>> I feel like the odd penguin out.
>>
>> I can't reproduce the behavior in question when using that example (I
>> haven't tried the other).
>>
>> I'm running:
>>
>>     * bash version 4.1.9(2)-release (i686-pc-linux-gnu)
>>
>>     * linux 2.6.38-rc4 (100b33c8bd8a3235fd0b7948338d6cbb3db3c63d)
>
> Oleg provided another testcase, can you reproduce the Ctrl-C problem with this
> it?
>
> #!/bin/bash
>
> perl -we '$SIG{INT} = sub {exit}; sleep'
>
> echo "Hehe, I am going to sleep after ^C"
> sleep 100
>
>
> Thanks,
>
>        Ingo
>

Yes, that requires me to press Ctrl-C twice in order to escape the
entire script. However, what do you expect the following to do:

#!/bin/bash

perl -we '$SIG{INT} = "IGNORE"; sleep 10'

echo "Hehe, I am going to sleep after ^C"
sleep 100

Re: Bash not reacting to Ctrl-C

[Pavel Machek]

Hi!

> > set_job_status_and_cleanup() notice wait_sigint_received and send
> > SIGINT to itself (termsig_handler (SIGINT)), but somehow it assumes
> > that the last foreground job should be terminated by SIGINT too:
> >
> > 	 else if (wait_sigint_received && (WTERMSIG (child->status) == SIGINT) &&
> >
> > Then the next wait_for() clears wait_sigint_received and bash
> > looses ^C
> 
> IOW.
> 
> Now that it is clear what happens, the test-case becomes even more
> trivial:
> 
> 	bash-4.1$ ./bash -c 'while true; do /bin/true; done'
> 	^C^C
> 
> needs 4-5 attempts on my machine.

Huh, this happened so often to me that I assumed it is a feature
:-(. Reproducible on both up arm and 4way x86...

Ok, it would be very good to get it fixed.


									Pavel

> --- bash-4.1/jobs.c~ctrlc_exit_race	2011-02-07 13:52:48.000000000 +0100
> +++ bash-4.1/jobs.c	2011-02-07 13:55:30.000000000 +0100
> @@ -3299,7 +3299,7 @@ set_job_status_and_cleanup (job)
>  	 signals are sent to process groups) or via kill(2) to the foreground
>  	 process by another process (or itself).  If the shell did receive the
>  	 SIGINT, it needs to perform normal SIGINT processing. */
> -      else if (wait_sigint_received && (WTERMSIG (child->status) == SIGINT) &&
> +      else if (wait_sigint_received /*&& (WTERMSIG (child->status) == SIGINT)*/ &&
>  	      IS_FOREGROUND (job) && IS_JOBCONTROL (job) == 0)
>  	{
>  	  int old_frozen;


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html