* [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity @ 2022-12-15 9:00 Zqiang 2022-12-17 1:09 ` Paul E. McKenney 0 siblings, 1 reply; 8+ messages in thread From: Zqiang @ 2022-12-15 9:00 UTC (permalink / raw) To: paulmck, frederic, quic_neeraju, joel; +Cc: rcu, linux-kernel Currently, when CPUs going offline, the rcu boost kthreads CPU affinity has been reset in rcutree_offline_cpu(), therefore this commit remove duplicate settings in rcutree_dead_cpu(). Signed-off-by: Zqiang <qiang1.zhang@intel.com> --- kernel/rcu/tree.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 65f3dd2fd3ae..20de83ed0c7e 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4122,15 +4122,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) */ int rcutree_dead_cpu(unsigned int cpu) { - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) return 0; WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); - /* Adjust any no-longer-needed kthreads. */ - rcu_boost_kthread_setaffinity(rnp, -1); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); return 0; -- 2.25.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-15 9:00 [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity Zqiang @ 2022-12-17 1:09 ` Paul E. McKenney 2022-12-17 2:00 ` Zhang, Qiang1 0 siblings, 1 reply; 8+ messages in thread From: Paul E. McKenney @ 2022-12-17 1:09 UTC (permalink / raw) To: Zqiang; +Cc: frederic, quic_neeraju, joel, rcu, linux-kernel On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote: > Currently, when CPUs going offline, the rcu boost kthreads CPU > affinity has been reset in rcutree_offline_cpu(), therefore this > commit remove duplicate settings in rcutree_dead_cpu(). > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> One question... Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity() to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu() instead passes in -1, are these two invocations really redundant? Thanx, paul > --- > kernel/rcu/tree.c | 5 ----- > 1 file changed, 5 deletions(-) > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index 65f3dd2fd3ae..20de83ed0c7e 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -4122,15 +4122,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) > */ > int rcutree_dead_cpu(unsigned int cpu) > { > - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); > - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ > - > if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) > return 0; > > WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); > - /* Adjust any no-longer-needed kthreads. */ > - rcu_boost_kthread_setaffinity(rnp, -1); > // Stop-machine done, so allow nohz_full to disable tick. > tick_dep_clear(TICK_DEP_BIT_RCU); > return 0; > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-17 1:09 ` Paul E. McKenney @ 2022-12-17 2:00 ` Zhang, Qiang1 2022-12-17 13:02 ` Zhang, Qiang1 0 siblings, 1 reply; 8+ messages in thread From: Zhang, Qiang1 @ 2022-12-17 2:00 UTC (permalink / raw) To: paulmck; +Cc: frederic, quic_neeraju, joel, rcu, linux-kernel On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote: > Currently, when CPUs going offline, the rcu boost kthreads CPU > affinity has been reset in rcutree_offline_cpu(), therefore this > commit remove duplicate settings in rcutree_dead_cpu(). > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > >One question... > >Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity() >to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu() >instead passes in -1, are these two invocations really redundant? Although -1 is passed in, but at this time, the outgoing CPU has been removed from rcu_rnp_online_cpus(rnp), in other words, we will eliminate the same CPU, which is also we passed in cpu in the rcutree_offline_cpu() before. Thanks Zqiang > > Thanx, paul > --- > kernel/rcu/tree.c | 5 ----- > 1 file changed, 5 deletions(-) > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index 65f3dd2fd3ae..20de83ed0c7e 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -4122,15 +4122,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) > */ > int rcutree_dead_cpu(unsigned int cpu) > { > - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); > - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ > - > if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) > return 0; > > WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); > - /* Adjust any no-longer-needed kthreads. */ > - rcu_boost_kthread_setaffinity(rnp, -1); > // Stop-machine done, so allow nohz_full to disable tick. > tick_dep_clear(TICK_DEP_BIT_RCU); > return 0; > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-17 2:00 ` Zhang, Qiang1 @ 2022-12-17 13:02 ` Zhang, Qiang1 2022-12-21 19:26 ` Paul E. McKenney 0 siblings, 1 reply; 8+ messages in thread From: Zhang, Qiang1 @ 2022-12-17 13:02 UTC (permalink / raw) To: Zhang, Qiang1, paulmck; +Cc: frederic, quic_neeraju, joel, rcu, linux-kernel On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote: > Currently, when CPUs going offline, the rcu boost kthreads CPU > affinity has been reset in rcutree_offline_cpu(), therefore this > commit remove duplicate settings in rcutree_dead_cpu(). > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > >One question... > >Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity() >to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu() >instead passes in -1, are these two invocations really redundant? > >Although -1 is passed in, but at this time, the outgoing CPU has been removed from >rcu_rnp_online_cpus(rnp), in other words, we will eliminate the same CPU, which is >also we passed in cpu in the rcutree_offline_cpu() before. > I also used " qemu -smp 4 "for a simple test. 1) rcutree_offline_cpu() (gdb) bt #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888100ce7cd0) at kernel/sched/core.c:2939 #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, outgoingcpu=outgoingcpu@entry=3) at kernel/rcu/tree_plugin.h:1250 #2 0xffffffff811cfa9f in rcutree_affinity_setting (outgoing=3, cpu=3) at kernel/rcu/tree.c:4297 #3 rcutree_offline_cpu (cpu=3) at kernel/rcu/tree.c:4341 #4 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=state@entry=CPUHP_AP_RCUTREE_ONLINE, bringup=bringup@entry=false, node=<optimized out>, lastp=lastp@entry=0xffff88815b027568) at kernel/cpu.c:192 #5 0xffffffff810c6e24 in cpuhp_thread_fun (cpu=<optimized out>) at kernel/cpu.c:785 #6 0xffffffff8111784e in smpboot_thread_fn (data=0xffff88810014c3c0) at kernel/smpboot.c:164 #7 0xffffffff8110c88e in kthread (_create=0xffff8881009b8b00) at kernel/kthread.c:376 #8 0xffffffff81002902 in ret_from_fork () at arch/x86/entry/entry_64.S:306 #9 0x0000000000000000 in ?? () (gdb) p/x *0xffff888100ce7cd0 $1 = 0x7 2) rcutree_dead_cpu() #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888109fc7948) at kernel/sched/core.c:2939 #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, outgoingcpu=outgoingcpu@entry=-1) at kernel/rcu/tree_plugin.h:1250 #2 0xffffffff811cf4a1 in rcutree_dead_cpu (cpu=3) at kernel/rcu/tree.c:4176 #3 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=CPUHP_RCUTREE_PREP, bringup=bringup@entry=false, node=node@entry=0x0 <fixed_percpu_data>, lastp=lastp@entry=0x0 <fixed_percpu_data>) at kernel/cpu.c:192 #4 0xffffffff810c58f5 in cpuhp_invoke_callback_range (bringup=bringup@entry=false, cpu=cpu@entry=3, st=st@entry=0xffff88815b027540, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:675 #5 0xffffffff827a5f6a in cpuhp_down_callbacks (target=CPUHP_OFFLINE, st=0xffff88815b027540, cpu=3) at kernel/cpu.c:1170 #6 _cpu_down (cpu=cpu@entry=3, tasks_frozen=tasks_frozen@entry=0, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1170 #7 0xffffffff810c6ff8 in cpu_down_maps_locked (target=CPUHP_OFFLINE, cpu=3) at kernel/cpu.c:1202 #8 cpu_down (cpu=3, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1210 #9 0xffffffff810c7906 in cpu_device_down (dev=dev@entry=0xffff88815b026988) at kernel/cpu.c:1227 (gdb) p/x *0xffff888109fc7948 $2 = 0x7 As can be seen from the above, we have set the same cpumask twice for the same boost kthreads. Thanks Zqiang >Thanks >Zqiang > > Thanx, paul > --- > kernel/rcu/tree.c | 5 ----- > 1 file changed, 5 deletions(-) > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index 65f3dd2fd3ae..20de83ed0c7e 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -4122,15 +4122,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) > */ > int rcutree_dead_cpu(unsigned int cpu) > { > - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); > - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ > - > if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) > return 0; > > WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); > - /* Adjust any no-longer-needed kthreads. */ > - rcu_boost_kthread_setaffinity(rnp, -1); > // Stop-machine done, so allow nohz_full to disable tick. > tick_dep_clear(TICK_DEP_BIT_RCU); > return 0; > -- > 2.25.1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-17 13:02 ` Zhang, Qiang1 @ 2022-12-21 19:26 ` Paul E. McKenney 2022-12-22 9:42 ` Zhang, Qiang1 2023-01-12 15:07 ` Frederic Weisbecker 0 siblings, 2 replies; 8+ messages in thread From: Paul E. McKenney @ 2022-12-21 19:26 UTC (permalink / raw) To: Zhang, Qiang1; +Cc: frederic, quic_neeraju, joel, rcu, linux-kernel On Sat, Dec 17, 2022 at 01:02:03PM +0000, Zhang, Qiang1 wrote: > On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote: > > Currently, when CPUs going offline, the rcu boost kthreads CPU > > affinity has been reset in rcutree_offline_cpu(), therefore this > > commit remove duplicate settings in rcutree_dead_cpu(). > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > >One question... > > > >Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity() > >to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu() > >instead passes in -1, are these two invocations really redundant? > > > >Although -1 is passed in, but at this time, the outgoing CPU has been removed from > >rcu_rnp_online_cpus(rnp), in other words, we will eliminate the same CPU, which is > >also we passed in cpu in the rcutree_offline_cpu() before. > > > > I also used " qemu -smp 4 "for a simple test. > > 1) rcutree_offline_cpu() > > (gdb) bt > #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888100ce7cd0) > at kernel/sched/core.c:2939 > #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, > outgoingcpu=outgoingcpu@entry=3) at kernel/rcu/tree_plugin.h:1250 > #2 0xffffffff811cfa9f in rcutree_affinity_setting (outgoing=3, cpu=3) at kernel/rcu/tree.c:4297 > #3 rcutree_offline_cpu (cpu=3) at kernel/rcu/tree.c:4341 > #4 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=state@entry=CPUHP_AP_RCUTREE_ONLINE, > bringup=bringup@entry=false, node=<optimized out>, lastp=lastp@entry=0xffff88815b027568) at kernel/cpu.c:192 > #5 0xffffffff810c6e24 in cpuhp_thread_fun (cpu=<optimized out>) at kernel/cpu.c:785 > #6 0xffffffff8111784e in smpboot_thread_fn (data=0xffff88810014c3c0) at kernel/smpboot.c:164 > #7 0xffffffff8110c88e in kthread (_create=0xffff8881009b8b00) at kernel/kthread.c:376 > #8 0xffffffff81002902 in ret_from_fork () at arch/x86/entry/entry_64.S:306 > #9 0x0000000000000000 in ?? () > > (gdb) p/x *0xffff888100ce7cd0 > $1 = 0x7 > > 2) rcutree_dead_cpu() > > #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888109fc7948) > at kernel/sched/core.c:2939 > #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, > outgoingcpu=outgoingcpu@entry=-1) at kernel/rcu/tree_plugin.h:1250 > #2 0xffffffff811cf4a1 in rcutree_dead_cpu (cpu=3) at kernel/rcu/tree.c:4176 > #3 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=CPUHP_RCUTREE_PREP, bringup=bringup@entry=false, > node=node@entry=0x0 <fixed_percpu_data>, lastp=lastp@entry=0x0 <fixed_percpu_data>) at kernel/cpu.c:192 > #4 0xffffffff810c58f5 in cpuhp_invoke_callback_range (bringup=bringup@entry=false, cpu=cpu@entry=3, > st=st@entry=0xffff88815b027540, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:675 > #5 0xffffffff827a5f6a in cpuhp_down_callbacks (target=CPUHP_OFFLINE, st=0xffff88815b027540, cpu=3) at kernel/cpu.c:1170 > #6 _cpu_down (cpu=cpu@entry=3, tasks_frozen=tasks_frozen@entry=0, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1170 > #7 0xffffffff810c6ff8 in cpu_down_maps_locked (target=CPUHP_OFFLINE, cpu=3) at kernel/cpu.c:1202 > #8 cpu_down (cpu=3, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1210 > #9 0xffffffff810c7906 in cpu_device_down (dev=dev@entry=0xffff88815b026988) at kernel/cpu.c:1227 > > (gdb) p/x *0xffff888109fc7948 > $2 = 0x7 > > As can be seen from the above, we have set the same cpumask twice for the same boost kthreads. Very good, thank you for checking! I have queued this as shown below, so please let me know of any issues. Thanx, Paul ------------------------------------------------------------------------ commit 5815245b56229198d5f5d0c48c26d3c0c53ee222 Author: Zqiang <qiang1.zhang@intel.com> Date: Wed Dec 21 11:15:43 2022 -0800 rcu: Remove redundant call to rcu_boost_kthread_setaffinity() The rcu_boost_kthread_setaffinity() function is invoked at rcutree_online_cpu() and rcutree_offline_cpu() time, early in the online timeline and late in the offline timeline, respectively. It is also invoked from rcutree_dead_cpu(), however, in the absence of userspace manipulations (for which userspace must take responsibility), this call is redundant with that from rcutree_offline_cpu(). This redundancy can be demonstrated by printing out the relevant cpumasks This commit therefore removes the call to rcu_boost_kthread_setaffinity() from rcutree_dead_cpu(). Signed-off-by: Zqiang <qiang1.zhang@intel.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index c3255d27286c4..a6181b76fd5ab 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4165,15 +4165,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) */ int rcutree_dead_cpu(unsigned int cpu) { - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) return 0; WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); - /* Adjust any no-longer-needed kthreads. */ - rcu_boost_kthread_setaffinity(rnp, -1); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); return 0; ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-21 19:26 ` Paul E. McKenney @ 2022-12-22 9:42 ` Zhang, Qiang1 2023-01-12 15:07 ` Frederic Weisbecker 1 sibling, 0 replies; 8+ messages in thread From: Zhang, Qiang1 @ 2022-12-22 9:42 UTC (permalink / raw) To: paulmck; +Cc: frederic, quic_neeraju, joel, rcu, linux-kernel On Sat, Dec 17, 2022 at 01:02:03PM +0000, Zhang, Qiang1 wrote: > On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote: > > Currently, when CPUs going offline, the rcu boost kthreads CPU > > affinity has been reset in rcutree_offline_cpu(), therefore this > > commit remove duplicate settings in rcutree_dead_cpu(). > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > > >One question... > > > >Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity() > >to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu() > >instead passes in -1, are these two invocations really redundant? > > > >Although -1 is passed in, but at this time, the outgoing CPU has been removed from > >rcu_rnp_online_cpus(rnp), in other words, we will eliminate the same CPU, which is > >also we passed in cpu in the rcutree_offline_cpu() before. > > > > I also used " qemu -smp 4 "for a simple test. > > 1) rcutree_offline_cpu() > > (gdb) bt > #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888100ce7cd0) > at kernel/sched/core.c:2939 > #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, > outgoingcpu=outgoingcpu@entry=3) at kernel/rcu/tree_plugin.h:1250 > #2 0xffffffff811cfa9f in rcutree_affinity_setting (outgoing=3, cpu=3) at kernel/rcu/tree.c:4297 > #3 rcutree_offline_cpu (cpu=3) at kernel/rcu/tree.c:4341 > #4 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=state@entry=CPUHP_AP_RCUTREE_ONLINE, > bringup=bringup@entry=false, node=<optimized out>, lastp=lastp@entry=0xffff88815b027568) at kernel/cpu.c:192 > #5 0xffffffff810c6e24 in cpuhp_thread_fun (cpu=<optimized out>) at kernel/cpu.c:785 > #6 0xffffffff8111784e in smpboot_thread_fn (data=0xffff88810014c3c0) at kernel/smpboot.c:164 > #7 0xffffffff8110c88e in kthread (_create=0xffff8881009b8b00) at kernel/kthread.c:376 > #8 0xffffffff81002902 in ret_from_fork () at arch/x86/entry/entry_64.S:306 > #9 0x0000000000000000 in ?? () > > (gdb) p/x *0xffff888100ce7cd0 > $1 = 0x7 > > 2) rcutree_dead_cpu() > > #0 set_cpus_allowed_ptr (p=p@entry=0xffff888100bd9cc0, new_mask=new_mask@entry=0xffff888109fc7948) > at kernel/sched/core.c:2939 > #1 0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>, > outgoingcpu=outgoingcpu@entry=-1) at kernel/rcu/tree_plugin.h:1250 > #2 0xffffffff811cf4a1 in rcutree_dead_cpu (cpu=3) at kernel/rcu/tree.c:4176 > #3 0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@entry=3, state=CPUHP_RCUTREE_PREP, bringup=bringup@entry=false, > node=node@entry=0x0 <fixed_percpu_data>, lastp=lastp@entry=0x0 <fixed_percpu_data>) at kernel/cpu.c:192 > #4 0xffffffff810c58f5 in cpuhp_invoke_callback_range (bringup=bringup@entry=false, cpu=cpu@entry=3, > st=st@entry=0xffff88815b027540, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:675 > #5 0xffffffff827a5f6a in cpuhp_down_callbacks (target=CPUHP_OFFLINE, st=0xffff88815b027540, cpu=3) at kernel/cpu.c:1170 > #6 _cpu_down (cpu=cpu@entry=3, tasks_frozen=tasks_frozen@entry=0, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1170 > #7 0xffffffff810c6ff8 in cpu_down_maps_locked (target=CPUHP_OFFLINE, cpu=3) at kernel/cpu.c:1202 > #8 cpu_down (cpu=3, target=target@entry=CPUHP_OFFLINE) at kernel/cpu.c:1210 > #9 0xffffffff810c7906 in cpu_device_down (dev=dev@entry=0xffff88815b026988) at kernel/cpu.c:1227 > > (gdb) p/x *0xffff888109fc7948 > $2 = 0x7 > > As can be seen from the above, we have set the same cpumask twice for the same boost kthreads. > >Very good, thank you for checking! > >I have queued this as shown below, so please let me know of any issues. I have checked and thank you for the clear description. Thanks Zqiang > > Thanx, Paul > >------------------------------------------------------------------------ commit 5815245b56229198d5f5d0c48c26d3c0c53ee222 Author: Zqiang <qiang1.zhang@intel.com> Date: Wed Dec 21 11:15:43 2022 -0800 rcu: Remove redundant call to rcu_boost_kthread_setaffinity() The rcu_boost_kthread_setaffinity() function is invoked at rcutree_online_cpu() and rcutree_offline_cpu() time, early in the online timeline and late in the offline timeline, respectively. It is also invoked from rcutree_dead_cpu(), however, in the absence of userspace manipulations (for which userspace must take responsibility), this call is redundant with that from rcutree_offline_cpu(). This redundancy can be demonstrated by printing out the relevant cpumasks This commit therefore removes the call to rcu_boost_kthread_setaffinity() from rcutree_dead_cpu(). Signed-off-by: Zqiang <qiang1.zhang@intel.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index c3255d27286c4..a6181b76fd5ab 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4165,15 +4165,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) */ int rcutree_dead_cpu(unsigned int cpu) { - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) return 0; WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); - /* Adjust any no-longer-needed kthreads. */ - rcu_boost_kthread_setaffinity(rnp, -1); // Stop-machine done, so allow nohz_full to disable tick. tick_dep_clear(TICK_DEP_BIT_RCU); return 0; ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2022-12-21 19:26 ` Paul E. McKenney 2022-12-22 9:42 ` Zhang, Qiang1 @ 2023-01-12 15:07 ` Frederic Weisbecker 2023-01-12 15:29 ` Paul E. McKenney 1 sibling, 1 reply; 8+ messages in thread From: Frederic Weisbecker @ 2023-01-12 15:07 UTC (permalink / raw) To: Paul E. McKenney; +Cc: Zhang, Qiang1, quic_neeraju, joel, rcu, linux-kernel On Wed, Dec 21, 2022 at 11:26:22AM -0800, Paul E. McKenney wrote: > On Sat, Dec 17, 2022 at 01:02:03PM +0000, Zhang, Qiang1 wrote: > commit 5815245b56229198d5f5d0c48c26d3c0c53ee222 > Author: Zqiang <qiang1.zhang@intel.com> > Date: Wed Dec 21 11:15:43 2022 -0800 > > rcu: Remove redundant call to rcu_boost_kthread_setaffinity() > > The rcu_boost_kthread_setaffinity() function is invoked at > rcutree_online_cpu() and rcutree_offline_cpu() time, early in the online > timeline and late in the offline timeline, respectively. It is also > invoked from rcutree_dead_cpu(), however, in the absence of userspace > manipulations (for which userspace must take responsibility), this call > is redundant with that from rcutree_offline_cpu(). This redundancy can > be demonstrated by printing out the relevant cpumasks > > This commit therefore removes the call to rcu_boost_kthread_setaffinity() > from rcutree_dead_cpu(). > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > Signed-off-by: Paul E. McKenney <paulmck@kernel.org> Reviewed-by: Frederic Weisbecker <frederic@kernel.org> Thanks! > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index c3255d27286c4..a6181b76fd5ab 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -4165,15 +4165,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) > */ > int rcutree_dead_cpu(unsigned int cpu) > { > - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); > - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ > - > if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) > return 0; > > WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); > - /* Adjust any no-longer-needed kthreads. */ > - rcu_boost_kthread_setaffinity(rnp, -1); > // Stop-machine done, so allow nohz_full to disable tick. > tick_dep_clear(TICK_DEP_BIT_RCU); > return 0; ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity 2023-01-12 15:07 ` Frederic Weisbecker @ 2023-01-12 15:29 ` Paul E. McKenney 0 siblings, 0 replies; 8+ messages in thread From: Paul E. McKenney @ 2023-01-12 15:29 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: Zhang, Qiang1, quic_neeraju, joel, rcu, linux-kernel On Thu, Jan 12, 2023 at 04:07:07PM +0100, Frederic Weisbecker wrote: > On Wed, Dec 21, 2022 at 11:26:22AM -0800, Paul E. McKenney wrote: > > On Sat, Dec 17, 2022 at 01:02:03PM +0000, Zhang, Qiang1 wrote: > > commit 5815245b56229198d5f5d0c48c26d3c0c53ee222 > > Author: Zqiang <qiang1.zhang@intel.com> > > Date: Wed Dec 21 11:15:43 2022 -0800 > > > > rcu: Remove redundant call to rcu_boost_kthread_setaffinity() > > > > The rcu_boost_kthread_setaffinity() function is invoked at > > rcutree_online_cpu() and rcutree_offline_cpu() time, early in the online > > timeline and late in the offline timeline, respectively. It is also > > invoked from rcutree_dead_cpu(), however, in the absence of userspace > > manipulations (for which userspace must take responsibility), this call > > is redundant with that from rcutree_offline_cpu(). This redundancy can > > be demonstrated by printing out the relevant cpumasks > > > > This commit therefore removes the call to rcu_boost_kthread_setaffinity() > > from rcutree_dead_cpu(). > > > > Signed-off-by: Zqiang <qiang1.zhang@intel.com> > > Signed-off-by: Paul E. McKenney <paulmck@kernel.org> > > Reviewed-by: Frederic Weisbecker <frederic@kernel.org> Thank you! I will apply this on my next rebase. Thanx, Paul > Thanks! > > > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > > index c3255d27286c4..a6181b76fd5ab 100644 > > --- a/kernel/rcu/tree.c > > +++ b/kernel/rcu/tree.c > > @@ -4165,15 +4165,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) > > */ > > int rcutree_dead_cpu(unsigned int cpu) > > { > > - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); > > - struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ > > - > > if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) > > return 0; > > > > WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); > > - /* Adjust any no-longer-needed kthreads. */ > > - rcu_boost_kthread_setaffinity(rnp, -1); > > // Stop-machine done, so allow nohz_full to disable tick. > > tick_dep_clear(TICK_DEP_BIT_RCU); > > return 0; ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-01-12 15:39 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-12-15 9:00 [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity Zqiang 2022-12-17 1:09 ` Paul E. McKenney 2022-12-17 2:00 ` Zhang, Qiang1 2022-12-17 13:02 ` Zhang, Qiang1 2022-12-21 19:26 ` Paul E. McKenney 2022-12-22 9:42 ` Zhang, Qiang1 2023-01-12 15:07 ` Frederic Weisbecker 2023-01-12 15:29 ` Paul E. McKenney
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®