From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Frederic Weisbecker <frederic@kernel.org>,
Yong He <alexyonghe@tencent.com>,
Joel Fernandes <joel@joelfernandes.org>,
Neeraj upadhyay <Neeraj.Upadhyay@amd.com>,
Like Xu <likexu@tencent.com>, Sasha Levin <sashal@kernel.org>,
jiangshanlai@gmail.com, paulmck@kernel.org,
josh@joshtriplett.org, rcu@vger.kernel.org
Subject: [PATCH AUTOSEL 6.6 09/13] srcu: Only accelerate on enqueue time
Date: Mon, 6 Nov 2023 18:14:22 -0500 [thread overview]
Message-ID: <20231106231435.3734790-9-sashal@kernel.org> (raw)
In-Reply-To: <20231106231435.3734790-1-sashal@kernel.org>
From: Frederic Weisbecker <frederic@kernel.org>
[ Upstream commit 8a77f38bcd28d3c22ab7dd8eff3f299d43c00411 ]
Acceleration in SRCU happens on enqueue time for each new callback. This
operation is expected not to fail and therefore any similar attempt
from other places shouldn't find any remaining callbacks to accelerate.
Moreover accelerations performed beyond enqueue time are error prone
because rcu_seq_snap() then may return the snapshot for a new grace
period that is not going to be started.
Remove these dangerous and needless accelerations and introduce instead
assertions reporting leaking unaccelerated callbacks beyond enqueue
time.
Co-developed-by: Yong He <alexyonghe@tencent.com>
Signed-off-by: Yong He <alexyonghe@tencent.com>
Co-developed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Co-developed-by: Neeraj upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj upadhyay <Neeraj.Upadhyay@amd.com>
Reviewed-by: Like Xu <likexu@tencent.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
kernel/rcu/srcutree.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index a1fcb8566b2e3..dbb5116bb0200 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -782,8 +782,7 @@ static void srcu_gp_start(struct srcu_struct *ssp)
spin_lock_rcu_node(sdp); /* Interrupts already disabled. */
rcu_segcblist_advance(&sdp->srcu_cblist,
rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq));
- (void)rcu_segcblist_accelerate(&sdp->srcu_cblist,
- rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq));
+ WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL));
spin_unlock_rcu_node(sdp); /* Interrupts remain disabled. */
WRITE_ONCE(ssp->srcu_sup->srcu_gp_start, jiffies);
WRITE_ONCE(ssp->srcu_sup->srcu_n_exp_nodelay, 0);
@@ -1692,6 +1691,7 @@ static void srcu_invoke_callbacks(struct work_struct *work)
ssp = sdp->ssp;
rcu_cblist_init(&ready_cbs);
spin_lock_irq_rcu_node(sdp);
+ WARN_ON_ONCE(!rcu_segcblist_segempty(&sdp->srcu_cblist, RCU_NEXT_TAIL));
rcu_segcblist_advance(&sdp->srcu_cblist,
rcu_seq_current(&ssp->srcu_sup->srcu_gp_seq));
if (sdp->srcu_cblist_invoking ||
@@ -1721,8 +1721,6 @@ static void srcu_invoke_callbacks(struct work_struct *work)
*/
spin_lock_irq_rcu_node(sdp);
rcu_segcblist_add_len(&sdp->srcu_cblist, -len);
- (void)rcu_segcblist_accelerate(&sdp->srcu_cblist,
- rcu_seq_snap(&ssp->srcu_sup->srcu_gp_seq));
sdp->srcu_cblist_invoking = false;
more = rcu_segcblist_ready_cbs(&sdp->srcu_cblist);
spin_unlock_irq_rcu_node(sdp);
--
2.42.0
next prev parent reply other threads:[~2023-11-06 23:15 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-06 23:14 [PATCH AUTOSEL 6.6 01/13] perf/core: Bail out early if the request AUX area is out of bound Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 02/13] rcu: Dump memory object info if callback function is invalid Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 03/13] srcu: Fix srcu_struct node grpmask overflow on 64-bit systems Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 04/13] selftests/lkdtm: Disable CONFIG_UBSAN_TRAP in test config Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 05/13] binfmt_elf: Support segments with 0 filesz and misaligned starts Sasha Levin
2023-11-07 0:04 ` Kees Cook
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 06/13] clocksource/drivers/timer-imx-gpt: Fix potential memory leak Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 07/13] binfmt_misc: cleanup on filesystem umount Sasha Levin
2023-11-07 0:04 ` Kees Cook
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 08/13] clocksource/drivers/timer-atmel-tcb: Fix initialization on SAM9 hardware Sasha Levin
2023-11-06 23:14 ` Sasha Levin [this message]
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 10/13] smp,csd: Throw an error if a CSD lock is stuck for too long Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 11/13] cpu/hotplug: Don't offline the last non-isolated CPU Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 12/13] workqueue: Provide one lock class key per work_on_cpu() callsite Sasha Levin
2023-11-06 23:14 ` [PATCH AUTOSEL 6.6 13/13] x86/mm: Drop the 4 MB restriction on minimal NUMA node memory size Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231106231435.3734790-9-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=Neeraj.Upadhyay@amd.com \
--cc=alexyonghe@tencent.com \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=likexu@tencent.com \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@kernel.org \
--cc=rcu@vger.kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome