Re: 5.8-rc*: kernel BUG at kernel/signal.c:1917

[peterz@infradead.org]

On Mon, Jul 20, 2020 at 10:41:06AM +0200, Peter Zijlstra wrote:
> On Mon, Jul 20, 2020 at 10:26:58AM +0200, Oleg Nesterov wrote:
> > Peter,
> > 
> > Let me add another note. TASK_TRACED/TASK_STOPPED was always protected by
> > ->siglock. In particular, ttwu(__TASK_TRACED) must be always called with
> > ->siglock held. That is why ptrace_freeze_traced() assumes it can safely
> > do s/TASK_TRACED/__TASK_TRACED/ under spin_lock(siglock).
> > 
> > Can this change race with
> > 
> > 		if (signal_pending_state(prev->state, prev)) {
> > 			prev->state = TASK_RUNNING;
> > 		}
> > 
> > in __schedule() ? Hopefully not, signal-state is protected by siglock too.
> > 
> > So I think this logic was correct even if it doesn't look nice. But "doesn't
> > look nice" is true for the whole ptrace code ;)
> 
> *groan*... another bit of obscure magic :-(
> 
> let me go try and wake up and figure out how best to deal with this.

So clearly I'm still missing something, the below results in:

[   63.760863] ------------[ cut here ]------------
[   63.766019] !(tmp_state & __TASK_TRACED)
[   63.766030] WARNING: CPU: 33 PID: 33801 at kernel/sched/core.c:4158 __schedule+0x6bd/0x8e0

Also, is there any way to not have ptrace do this? How performance
critical is this ptrace path? Because I really hate having to add code
to __schedule() to deal with this horrible thing.


---
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e15543cb84812..f65a801d268b6 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4100,9 +4100,9 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
  */
 static void __sched notrace __schedule(bool preempt)
 {
+	unsigned long prev_state, tmp_state;
 	struct task_struct *prev, *next;
 	unsigned long *switch_count;
-	unsigned long prev_state;
 	struct rq_flags rf;
 	struct rq *rq;
 	int cpu;
@@ -4140,16 +4140,38 @@ static void __sched notrace __schedule(bool preempt)
 	rq_lock(rq, &rf);
 	smp_mb__after_spinlock();
 
+	/*
+	 * We must re-load prev->state in case ttwu_remote() changed it
+	 * before we acquired rq->lock.
+	 */
+	tmp_state = prev->state;
+	if (unlikely(prev_state != tmp_state)) {
+		if (prev_state & __TASK_TRACED) {
+			/*
+			 * ptrace_{,un}freeze_traced() think it is cool
+			 * to change ->state around behind our backs
+			 * between TASK_TRACED and __TASK_TRACED.
+			 *
+			 * Luckily this transformation doesn't affect
+			 * sched_contributes_to_load.
+			 */
+			SCHED_WARN_ON(!(tmp_state & __TASK_TRACED));
+		} else {
+			/*
+			 * For any other case, a changed prev_state
+			 * must be to TASK_RUNNING, such that...
+			 */
+			SCHED_WARN_ON(tmp_state != TASK_RUNNING);
+		}
+		prev_state = tmp_state;
+	}
+
 	/* Promote REQ to ACT */
 	rq->clock_update_flags <<= 1;
 	update_rq_clock(rq);
 
 	switch_count = &prev->nivcsw;
-	/*
-	 * We must re-load prev->state in case ttwu_remote() changed it
-	 * before we acquired rq->lock.
-	 */
-	if (!preempt && prev_state && prev_state == prev->state) {
+	if (!preempt && prev_state) {
 		if (signal_pending_state(prev_state, prev)) {
 			prev->state = TASK_RUNNING;
 		} else {