[PATCH] sched/loadavg: Use READ_ONCE/WRITE_ONCE for calc_load_idx

[PATCH] sched/loadavg: Use READ_ONCE/WRITE_ONCE for calc_load_idx

[Hui Su]

The NO_HZ fold path reads calc_load_idx locklessly from
calc_load_write_idx(), while calc_global_nohz() updates it from the
jiffies_lock-serialized global load path.

Commit caeb5882979b ("sched/loadavg: Use {READ,WRITE}_ONCE() for sample
window") marked the lockless accesses to calc_load_update. calc_load_idx
participates in the same cross-CPU protocol, ordered by the existing
smp_wmb()/smp_rmb() pair, but its accesses remain plain C accesses.

Mark the lockless read and publisher access with READ_ONCE() and
WRITE_ONCE(). KCSAN otherwise observes the concurrent access:

  BUG: KCSAN: data-race in calc_global_load / calc_load_nohz_fold

  write by interrupt on cpu 3:
   calc_global_load+0x392/0x420
   tick_do_update_jiffies64+0x171/0x1e0

  read by task 0 on cpu 5:
   calc_load_nohz_fold+0x7a/0x110
   calc_load_nohz_start+0x26/0x30

  value changed: 0x00000033 -> 0x00000034

Keep the existing memory ordering and writer serialization unchanged.
calc_load_read_idx() is only used from the jiffies_lock-serialized
global load path and does not need an accessor.

Fixes: 5167e8d5417b ("sched/nohz: Rewrite and fix load-avg computation -- again")
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/loadavg.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/loadavg.c b/kernel/sched/loadavg.c
index b601e0243d0e..f87059e0e251 100644
--- a/kernel/sched/loadavg.c
+++ b/kernel/sched/loadavg.c
@@ -209,7 +209,7 @@ static int calc_load_idx;
 
 static inline int calc_load_write_idx(void)
 {
-	int idx = calc_load_idx;
+	int idx = READ_ONCE(calc_load_idx);
 
 	/*
 	 * See calc_global_nohz(), if we observe the new index, we also
@@ -333,7 +333,7 @@ static void calc_global_nohz(void)
 	 * index, this avoids a double flip messing things up.
 	 */
 	smp_wmb();
-	calc_load_idx++;
+	WRITE_ONCE(calc_load_idx, READ_ONCE(calc_load_idx) + 1);
 }
 #else /* !CONFIG_NO_HZ_COMMON: */
 
-- 
2.55.0