[PATCH 1/1] rcu/kvfree: Use READ_ONCE() when access to krcp->head

[PATCH 1/1] rcu/kvfree: Use READ_ONCE() when access to krcp->head

[Uladzislau Rezki (Sony)]

A need_offload_krc() function is now lock-free. A compiler
can optimize readers in way that they see an old value even
though writers already updated the krcp->head from another
path.

Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
---
 kernel/rcu/tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index efc79f942b30..d155f2594317 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)
 		if (!list_empty(&krcp->bulk_head[i]))
 			return true;
 
-	return !!krcp->head;
+	return !!READ_ONCE(krcp->head);
 }
 
 static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)
 		// in that case the monitor work is rearmed.
 		if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) ||
 			(!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) ||
-				(krcp->head && !krwp->head_free)) {
+				(READ_ONCE(krcp->head) && !krwp->head_free)) {
 
 			// Channel 1 corresponds to the SLAB-pointer bulk path.
 			// Channel 2 corresponds to vmalloc-pointer bulk path.
-- 
2.30.2

Re: [PATCH 1/1] rcu/kvfree: Use READ_ONCE() when access to krcp->head

[Paul E. McKenney]

On Fri, Dec 02, 2022 at 02:18:37PM +0100, Uladzislau Rezki (Sony) wrote:
> A need_offload_krc() function is now lock-free. A compiler
> can optimize readers in way that they see an old value even
> though writers already updated the krcp->head from another
> path.
> 
> Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>

Queued, thank you!!!

With the usual wordsmithing, so please let me know if I messed something
up.

							Thanx, Paul

------------------------------------------------------------------------

commit 9615ca28bc2c94cf2e092f06baace941fbf3b7e8
Author: Uladzislau Rezki (Sony) <urezki@gmail.com>
Date:   Fri Dec 2 14:18:37 2022 +0100

    rcu/kvfree: Use READ_ONCE() when access to krcp->head
    
    The need_offload_krc() function is now lock-free, which gives the
    compiler freedom to load old values from plain C-language loads from
    the kfree_rcu_cpu struture's ->head pointer.  This commit therefore
    applied READ_ONCE() to these loads.
    
    Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
    Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 2d16a0eee792d..ee8a6a711719a 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)
 		if (!list_empty(&krcp->bulk_head[i]))
 			return true;
 
-	return !!krcp->head;
+	return !!READ_ONCE(krcp->head);
 }
 
 static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)
 		// in that case the monitor work is rearmed.
 		if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) ||
 			(!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) ||
-				(krcp->head && !krwp->head_free)) {
+				(READ_ONCE(krcp->head) && !krwp->head_free)) {
 
 			// Channel 1 corresponds to the SLAB-pointer bulk path.
 			// Channel 2 corresponds to vmalloc-pointer bulk path.