[PATCH RFC 10/14] rcu: Promote blocked tasks before QS report in rcu_gp_init()

[Joel Fernandes <joelagnelf@nvidia.com>]

During grace period initialization, when rcu_gp_init() reports QS for
offline CPUs, any tasks blocked on those CPUs' per-CPU blocked lists
must first be promoted to the rcu_node's blkd_tasks list.

Without this promotion, blocked tasks on offline CPUs' per-CPU lists
won't have gp_tasks point to them, so the GP machinery won't wait for
them. This can cause "Wrong-GP reads" errors where a GP completes while
readers are still in their critical sections.

Therefore, call call rcu_promote_blocked_tasks_rdp() for each offline CPU
before reporting QS for them.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
 kernel/rcu/tree.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 5e73ebb260e3..468388970c98 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2001,8 +2001,21 @@ static noinline_for_stack bool rcu_gp_init(void)
 		 */
 		mask = rnp->qsmask & ~rnp->qsmaskinitnext;
 		rnp->rcu_gp_init_mask = mask;
-		if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp))
+		if ((mask || rnp->wait_blkd_tasks) && rcu_is_leaf_node(rnp)) {
+			int cpu;
+
+			/*
+			 * Promote blocked tasks from offline CPUs before
+			 * reporting QS, so they properly block the GP.
+			 */
+			for_each_leaf_node_cpu_mask(rnp, cpu, mask) {
+				struct rcu_data *rdp_cpu;
+
+				rdp_cpu = per_cpu_ptr(&rcu_data, cpu);
+				rcu_promote_blocked_tasks_rdp(rdp_cpu, rnp);
+			}
 			rcu_report_qs_rnp(mask, rnp, rnp->gp_seq, flags);
+		}
 		else
 			raw_spin_unlock_irq_rcu_node(rnp);
 		cond_resched_tasks_rcu_qs();
-- 
2.34.1