[PATCH RFC 12/14] rcu: Skip per-CPU list addition when GP already started

[Joel Fernandes <joelagnelf@nvidia.com>]

When a grace period is already started or waiting on this CPU, skip
adding the blocked task to the per-CPU rdp->blkd_list. The task goes
directly to rnp->blkd_tasks via rcu_preempt_ctxt_queue(), which is the
same behavior as before per-CPU lists were added.

When no GP is waiting, add the task to BOTH lists as before this patch. This
maintains the existing behavior while preparing for the next patch which will
skip rnp blocked list addition when no GP is waiting.

Because the rnp->blkd_tasks handling remains unchanged (tasks still go
through rcu_preempt_ctxt_queue() in all cases), this work same as
before this patch.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
 kernel/rcu/tree_plugin.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 8622e79660ed..d43dd153c152 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -339,10 +339,18 @@ void rcu_note_context_switch(bool preempt)
 		t->rcu_read_unlock_special.b.blocked = true;
 		t->rcu_blocked_node = rnp;
 #ifdef CONFIG_RCU_PER_CPU_BLOCKED_LISTS
-		t->rcu_blocked_cpu = rdp->cpu;
-		raw_spin_lock(&rdp->blkd_lock);
-		list_add(&t->rcu_rdp_entry, &rdp->blkd_list);
-		raw_spin_unlock(&rdp->blkd_lock);
+		/*
+		 * If no GP is waiting on this CPU, add to per-CPU list as well
+		 * so promotion can find it if a GP starts later. If GP waiting,
+		 * skip per-CPU list - task goes only to rnp->blkd_tasks (same
+		 * behavior as before per-CPU lists were added).
+		 */
+		if (!rcu_gp_in_progress() && !rdp->cpu_no_qs.b.norm && !rdp->cpu_no_qs.b.exp) {
+			t->rcu_blocked_cpu = rdp->cpu;
+			raw_spin_lock(&rdp->blkd_lock);
+			list_add(&t->rcu_rdp_entry, &rdp->blkd_list);
+			raw_spin_unlock(&rdp->blkd_lock);
+		}
 #endif
 
 		/*
-- 
2.34.1