From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46369500967; Sun, 18 Jan 2026 11:25:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768735502; cv=none; b=NB0Hxyxn+rKYkUIhe0+92yeuiDHzc+0WzoMeuSWCuLu/QbOyLxgF/M0acxlpaDA4/WspV16sPG4WeVsJs7qRBN8bgu4ChH04B4tsqZT1mA4T5kT0mDjwB4eu2BB6pbfiH7CsBVK3D6tM/AUusiu1zfXIB4/6jvyYCiGoVYVF+Nk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768735502; c=relaxed/simple; bh=rRmNHUGPqRMm/3JopDjPqtCwlL5E/rEJsFwo32qsDt0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=gTdqI98VMURxyrhTng2zuVhMj1TgIt3BDzMCBRnUoWVM7YP4XNlrBYgiAHuvDTxCfuLGvqjn7gB35s/lysb23nVA/DRHOnGq5utwJY1g7ryaKMatGjQhb+QI7fr4gbZ/Sm54MU/TaHWebBk0eXB576jhTfZBfbiHS7lNdDQxoGA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=bZ6uXaCy; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="bZ6uXaCy" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60IANojX011442; Sun, 18 Jan 2026 11:24:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=224k99 1wunepNTMjBvoXK0Z8BMo+9M6kNaO6+6ya/a8=; b=bZ6uXaCyKT7wdGJ4upZrkw 9Jn8Nk9BNfzRZI+XzDQZI5pM1Igv3hIxEpuEjukV0up/RI2DqDirr4KWrJ8z2c0C n42UNqK5cVB42EOxYTxKE1CqevewXVLO7jaUU9EttNIruxfJRidNlk08WbCCWUiF aUGXkfKhPO6eRy51ieBjZeP/BizbotG+HcPhuXGld7CdJQEXu/HEPLf8ownRYdiP QMsbIgjfeIvKSQvIenUw/laBgYaOpZ4H4wryzWOWWWVVguTALdVeeUyGXJJc/rzN nTQ+vqIuhsWsimEy6VHIL/wdohNRYRAJG/XRuvsimfz8NQgWKTzNg1ws8Y9EFPtA == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bqyujuw3e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 18 Jan 2026 11:24:55 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60IBOtlY018151; Sun, 18 Jan 2026 11:24:55 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bqyujuw3c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 18 Jan 2026 11:24:55 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60IB4599024610; Sun, 18 Jan 2026 11:24:54 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([172.16.1.9]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brxar81xm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 18 Jan 2026 11:24:54 +0000 Received: from smtpav02.wdc07v.mail.ibm.com (smtpav02.wdc07v.mail.ibm.com [10.39.53.229]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60IBOqwa23200344 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 18 Jan 2026 11:24:53 GMT Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BBCDF58058; Sun, 18 Jan 2026 11:24:52 +0000 (GMT) Received: from smtpav02.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 665945805B; Sun, 18 Jan 2026 11:24:48 +0000 (GMT) Received: from [9.39.20.96] (unknown [9.39.20.96]) by smtpav02.wdc07v.mail.ibm.com (Postfix) with ESMTP; Sun, 18 Jan 2026 11:24:47 +0000 (GMT) Message-ID: <9c544560-1448-42a4-b16d-000f06733d0b@linux.ibm.com> Date: Sun, 18 Jan 2026 16:54:46 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] rcu: Latch normal synchronize_rcu() path on flood To: Shrikanth Hegde , Joel Fernandes Cc: Uladzislau Rezki , Paul E McKenney , Vishal Chourasia , Neeraj upadhyay , RCU , LKML , Frederic Weisbecker References: <20260114183415.286489-1-urezki@gmail.com> <1c6b741e-acfd-432b-bd04-4534c2e2511a@linux.ibm.com> <00e91ebc-0783-4519-9727-53dd3a625298@linux.ibm.com> Content-Language: en-US From: Samir M In-Reply-To: <00e91ebc-0783-4519-9727-53dd3a625298@linux.ibm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTE4MDA5OCBTYWx0ZWRfX10hqgQSP9/tX eClE+kZ3F1vf86xQLtGXRhkbmL35piWCZx+CNviU4HPQ+QAICsEaQ7QAvHwrqTnSpVWycv8P7wk RPwkFIlL5w2DlXeBcLrqa2BUnimhLnlM2WGG4skxjmL/EO1Y7Wdkp48vshZp/rVJtb/ntrQ03a9 5/4aS8YNrDNR3dvtA7g/kOWvTPvcuCnQbONFXvYOPPsTXwo2Q07Y+TGC1q0Fa2l6K05aYaWz5cK z3AjfCGnww+CfiBmq3ySMefKffevwOCINqe/YjRzSRyZJ7uNeZo0roev/2UvrjZSqcrud+01ceL iM02446PoUmiwHrsJaUm8nP6ndk8F1bWOINx5E7AgGPdH39Tmhn4bH3A3zTdIw0//zWp5YTs9v9 b72RBukN7YNkQ+o/vUo1aHDz/DT/Sox8j/7ogVlR5rd4ZeI7sghzhv6m4Urn9JgcqKFqgyBw2MW HZbvAMbWDxBW7Wl2K7A== X-Authority-Analysis: v=2.4 cv=bsBBxUai c=1 sm=1 tr=0 ts=696cc307 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=Ikd4Dj_1AAAA:8 a=pGLkceISAAAA:8 a=DFK25qOm3bQXS9wycRcA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: m0z6rcqgcoQlqd9Ii-90Szgcn3v17K5C X-Proofpoint-GUID: 5xu4arF_7DyWOaOhVYZMZ5N9gE9Rn8Ul X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-17_03,2026-01-15_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 phishscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 malwarescore=0 clxscore=1015 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601180098 On 17/01/26 5:48 pm, Shrikanth Hegde wrote: > > > On 1/17/26 2:18 PM, Joel Fernandes wrote: >> >> >>> On Jan 17, 2026, at 1:17 AM, Samir M wrote: >>> >>>  >>>> On 15/01/26 12:04 am, Uladzislau Rezki (Sony) wrote: >>>> Currently, rcu_normal_wake_from_gp is only enabled by default >>>> on small systems(<= 16 CPUs) or when a user explicitly set it >>>> enabled. >>>> >>>> This patch introduces an adaptive latching mechanism: >>>>   * Tracks the number of in-flight synchronize_rcu() requests >>>>     using a new atomic_t counter(rcu_sr_normal_count); >>>> >>>>   * If the count exceeds RCU_SR_NORMAL_LATCH_THR(64), it sets >>>>     the rcu_sr_normal_latched, reverting new requests onto the >>>>     scaled wait_rcu_gp() path; >>>> >>>>   * The latch is cleared only when the pending requests are fully >>>>     drained(nr == 0); >>>> >>>>   * Enables rcu_normal_wake_from_gp by default for all systems, >>>>     relying on this dynamic throttling instead of static CPU >>>>     limits. >>>> >>>> Suggested-by: Joel Fernandes >>>> Signed-off-by: Uladzislau Rezki (Sony) >>>> --- >>>>   kernel/rcu/tree.c | 37 ++++++++++++++++++++++++++----------- >>>>   1 file changed, 26 insertions(+), 11 deletions(-) >>>> >>>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c >>>> index 293bbd9ac3f4..c42d480d6e0b 100644 >>>> --- a/kernel/rcu/tree.c >>>> +++ b/kernel/rcu/tree.c >>>> @@ -1631,17 +1631,21 @@ static void rcu_sr_put_wait_head(struct >>>> llist_node *node) >>>>       atomic_set_release(&sr_wn->inuse, 0); >>>>   } >>>>   -/* Enable rcu_normal_wake_from_gp automatically on small >>>> systems. */ >>>> -#define WAKE_FROM_GP_CPU_THRESHOLD 16 >>>> - >>>> -static int rcu_normal_wake_from_gp = -1; >>>> +static int rcu_normal_wake_from_gp = 1; >>>>   module_param(rcu_normal_wake_from_gp, int, 0644); >>>>   static struct workqueue_struct *sync_wq; >>>>   +#define RCU_SR_NORMAL_LATCH_THR 64 >>>> + >>>> +/* Number of in-flight synchronize_rcu() calls queued on srs_next. */ >>>> +static atomic_long_t rcu_sr_normal_count; >>>> +static atomic_t rcu_sr_normal_latched; >>>> + >>>>   static void rcu_sr_normal_complete(struct llist_node *node) >>>>   { >>>>       struct rcu_synchronize *rs = container_of( >>>>           (struct rcu_head *) node, struct rcu_synchronize, head); >>>> +    long nr; >>>>         WARN_ONCE(IS_ENABLED(CONFIG_PROVE_RCU) && >>>> !poll_state_synchronize_rcu_full(&rs->oldstate), >>>> @@ -1649,6 +1653,15 @@ static void rcu_sr_normal_complete(struct >>>> llist_node *node) >>>>         /* Finally. */ >>>>       complete(&rs->completion); >>>> +    nr = atomic_long_dec_return(&rcu_sr_normal_count); >>>> +    WARN_ON_ONCE(nr < 0); >>>> + >>>> +    /* >>>> +     * Unlatch: switch back to normal path when fully >>>> +     * drained and if it has been latched. >>>> +     */ >>>> +    if (nr == 0) >>>> +        (void)atomic_cmpxchg(&rcu_sr_normal_latched, 1, 0); >>>>   } >>>>     static void rcu_sr_normal_gp_cleanup_work(struct work_struct >>>> *work) >>>> @@ -1794,7 +1807,14 @@ static bool rcu_sr_normal_gp_init(void) >>>>     static void rcu_sr_normal_add_req(struct rcu_synchronize *rs) >>>>   { >>>> +    long nr; >>>> + >>>>       llist_add((struct llist_node *) &rs->head, &rcu_state.srs_next); >>>> +    nr = atomic_long_inc_return(&rcu_sr_normal_count); >>>> + >>>> +    /* Latch: only when flooded and if unlatched. */ >>>> +    if (nr >= RCU_SR_NORMAL_LATCH_THR) >>>> +        (void)atomic_cmpxchg(&rcu_sr_normal_latched, 0, 1); >>>>   } >>>>     /* >>>> @@ -3268,7 +3288,8 @@ static void synchronize_rcu_normal(void) >>>>         trace_rcu_sr_normal(rcu_state.name, &rs.head, TPS("request")); >>>>   -    if (READ_ONCE(rcu_normal_wake_from_gp) < 1) { >>>> +    if (READ_ONCE(rcu_normal_wake_from_gp) < 1 || >>>> +            atomic_read(&rcu_sr_normal_latched)) { >>>>           wait_rcu_gp(call_rcu_hurry); >>>>           goto trace_complete_out; >>>>       } >>>> @@ -4892,12 +4913,6 @@ void __init rcu_init(void) >>>>       sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM | >>>> WQ_UNBOUND, 0); >>>>       WARN_ON(!sync_wq); >>>>   -    /* Respect if explicitly disabled via a boot parameter. */ >>>> -    if (rcu_normal_wake_from_gp < 0) { >>>> -        if (num_possible_cpus() <= WAKE_FROM_GP_CPU_THRESHOLD) >>>> -            rcu_normal_wake_from_gp = 1; >>>> -    } >>>> - >>>>       /* Fill in default value for rcutree.qovld boot parameter. */ >>>>       /* -After- the rcu_node ->lock fields are initialized! */ >>>>       if (qovld < 0) >>> >>> >>> Hi Uladzislau, >>> >>> I verified this patch using the configuration described below. >>> Configuration: >>>      •    Kernel version: 6.19.0-rc5 >>>      •    Number of CPUs: 2048 >>> >>> Using this setup, I evaluated the patch with both SMT enabled and >>> SMT disabled. The results indicate that when SMT is enabled, the >>> system time is noticeably higher. In contrast, with SMT disabled, no >>> significant increase in system time is observed. >>> >>> SMT=ON  -> sys 31m22.922s >>> SMT=OFF -> sys 0m0.046s >>> >>> >>> SMT Mode    | Without Patch    | With Patch   | % Improvement    | >>> ------------------------------------------------------------------ >>> SMT=off     | 30m 53.194s      | 26m 24.009s  | +14.53%          | >>> SMT=on      | 49m 5.920s       | 47m 5.513s   | +4.09% >> >> So it takes you 47 minutes to offline CPUs and you are Ok with that? >> >> - Joel >> > > > This is certainly quite long. IMO not worth the added complexity > of atomic inc/dec reads happening(even though till 64 CPUs) > > Samir, > can you post number's with vishal's patch of using rcu expedited? > > >> >> >>>         | >>> >>> >>> Please add below tag: Tested-by: Samir M >>> >>> Regards, >>> Samir >>> >>> > Shrikanth, I verified Vishal’s patch as described below, using the following configuration. Configuration:     •    Kernel version: 6.19.0-rc5     •    Number of CPUs: 2048 Vishal's patch link: https://lore.kernel.org/all/20260112094332.66006-2-vishalc@linux.ibm.com/ Verification of Vishal’s patch shows a significant improvement in the *SMT=off* case and a measurable improvement in the *SMT=on* case. SMT Mode    | Without Patch    | With Patch | % Improvement    | ------------------------------------------------------------------ SMT=off     | 30m 53.194s      |  6m 4.250s  | +80.40%          | SMT=on      | 49m 5.920s       | 36m 50.386s | +25.01%          | Regards, Samir