From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C7F22DC355 for ; Mon, 12 Jan 2026 17:09:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768237755; cv=none; b=cMrr4Rzr2Fh/1bEBBl2LoObCaRllWZcDUuBP/Z/YggRkCkeSk9vlfByG87FBiwTaCKUoexdC+HX32kqWv0663M6IGiYldS/0WRpSps52bn9MzQxmlKIpc8z2/X7TSYKZv5APwiYf+enbWTKxnSTknpWDJwUismVAoKL5Ww1wU/4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768237755; c=relaxed/simple; bh=OJB3qQLixmLbSxVGQskeXIa9IR2SDMZNpVzBkBE9CJs=; h=From:Date:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=B09OwSpzEa2/uCUwH0xRTxlvLdqHzCOhEKU23jEL5ixi7mOBpjy80BkOeol6qo43Xhdl8Y0YnjeRmzJNcnusvfLGyGDDr7ar5FdY/Xy6ayS5b/NAMCnACfqGbPOCKnOrMVkINJl10Tx4R0rvyyAcJ49KCPSH1jCVqNCRLU/vPnw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N4VWuPJ9; arc=none smtp.client-ip=209.85.167.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N4VWuPJ9" Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-59b7882e605so5865588e87.0 for ; Mon, 12 Jan 2026 09:09:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768237752; x=1768842552; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from:from:to :cc:subject:date:message-id:reply-to; bh=WLV1NsxjV3ejRzmN5EWp0k3Qy7VB+brJ9PeF6ga5Kbc=; b=N4VWuPJ9e0HmtB9Eh4qH6yX7Mg7Dt4vRFggTPRkieTRqcp6dBmB837LNGV1rnGw1A4 nfHvsKfkQhe0DX8lH5y9wXQsPqwMbXM4Y13Ig5TOIxU75QrKWkmtyUXR0DaoCTdNQj5V Snsal/asC10uocLuLnThymdl/f4H+OZuSOw6Xtr2P0H2ONpBXvhevnKWiSk69+J8KFz5 VdFv++kVci7CFE+x+K/f2u0h+lywQ0kyETYbhxI4d4bz+A7U0AVZxPhEi82uNnHgsYRI FjXOZS7Y/E2OEhiyjmcawxmlqH6Sh5VjDY/UjNYLsXKqF8AwyN3KuZzwz9bIEwJaD/En U3WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768237752; x=1768842552; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WLV1NsxjV3ejRzmN5EWp0k3Qy7VB+brJ9PeF6ga5Kbc=; b=qqvIxZls7M5eRnZ+/ksNfD2pIdNofDwnaQm0y5L3cdocF0o8GE0Ndhv8BP4Ro9ssbZ rQMH2R3maYOhA8JhF61QGlsQqBfr8SC71G4VZVhDYmuTTPEsq4RxiW6qLP6g0uUp4eAf J9q4UESAxNjHf4++ozugyRyL8wJzvBZIB4t5MuKGIbq4SYlha1zKAUwbBzCKefdfGWI1 3jpoZgCaX2p+23u0tu0aIByfbscvZTHarHViu87UWaNkzEKrQLVsyRMq/H9d+QxGxmBP uw6Gcts/wcArKBMNAVAIr4PtCbSuZ+cMQIhOgHQ0krL+LVWqGiHjgwUzg4CMp9Ae8k/o zWCQ== X-Forwarded-Encrypted: i=1; AJvYcCXcgyMfod2aypo+9ivv9Hrqu4JjYHQ/ffDCDJ1FRXv1xU0Rzc+7zyApCxKqjM2lOwJtR9aGrWUlfFmPruA=@vger.kernel.org X-Gm-Message-State: AOJu0Ywcf1nH4nLUuGGCLbY8sa6bQgbM3i69BjgEHNMOOdD0aHvqxzzG oIAijIkkeBgPDv9SPnq996sY1fmxtq7/BK67oCZwOK6qP+tIm92jK/q/ X-Gm-Gg: AY/fxX5YPkcxx+K0h42hJyMsEjajDUWNBfsLa5ifnA+fGvVmd65Gtkg6/YLtOjx9p7r I+4TE258wRJ4SC5naNd7YZNMgO46CwDqXtwEhxZY4TwvZUgWmjHaEVYsQkIip1RNv+NIVj8ak9p 207Xa9tfF4fGBytzdNfdE3Gi08ukBuhzxng8ESP0M1KzaXcp4cPIEExTatAWprD4IW1PZ7HCRAe QJB5bdbQg1cXlfrbTyp+jioRcEhPBp4uwfgSP0Z3zBYAy7MD8MQYNBdAg3O0i6utchDBidio1US iortwxpZpvyYEfI7VteVg/av/SAaRZN8rOPPqq1jwRJYiT9z2GIKU1OIxjnetU88Y0g7gxCS01R ZwUdIpipV0GGBtT7LkjxdHofxdPGSSFLkngCvGFImv0BGpXFCSuPi X-Google-Smtp-Source: AGHT+IEvqK8oHcfGwCDrhoKWko25WVhi/Q+5LwtVlHdVJPwRlLaIiZmCfnbRPTjwihLiBNvr1keRhw== X-Received: by 2002:a05:6512:4004:b0:594:2f25:d488 with SMTP id 2adb3069b0e04-59b6ef1e286mr5998653e87.21.1768237751382; Mon, 12 Jan 2026 09:09:11 -0800 (PST) Received: from milan ([2001:9b1:d5a0:a500::24b]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-59b65d0d108sm4925107e87.27.2026.01.12.09.09.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jan 2026 09:09:10 -0800 (PST) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Mon, 12 Jan 2026 18:09:09 +0100 To: Joel Fernandes Cc: Uladzislau Rezki , Shrikanth Hegde , Vishal Chourasia , "rcu@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "paulmck@kernel.org" , "frederic@kernel.org" , "neeraj.upadhyay@kernel.org" , "josh@joshtriplett.org" , "boqun.feng@gmail.com" , "rostedt@goodmis.org" , "tglx@linutronix.de" , "peterz@infradead.org" , "srikar@linux.ibm.com" Subject: Re: [PATCH] cpuhp: Expedite synchronize_rcu during CPU hotplug operations Message-ID: References: <20260112094332.66006-2-vishalc@linux.ibm.com> <5a2b00f2-5e73-4c89-89b5-1a69cb8a7fa2@linux.ibm.com> <91138C31-EF47-4CA6-BD9F-A41981F543EE@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <91138C31-EF47-4CA6-BD9F-A41981F543EE@nvidia.com> On Mon, Jan 12, 2026 at 04:09:49PM +0000, Joel Fernandes wrote: > > > > On Jan 12, 2026, at 7:57 AM, Uladzislau Rezki wrote: > > > > Hello, Shrikanth! > > > >> > >>> On 1/12/26 3:38 PM, Uladzislau Rezki wrote: > >>> On Mon, Jan 12, 2026 at 03:13:33PM +0530, Vishal Chourasia wrote: > >>>> Bulk CPU hotplug operations—such as switching SMT modes across all > >>>> cores—require hotplugging multiple CPUs in rapid succession. On large > >>>> systems, this process takes significant time, increasing as the number > >>>> of CPUs grows, leading to substantial delays on high-core-count > >>>> machines. Analysis [1] reveals that the majority of this time is spent > >>>> waiting for synchronize_rcu(). > >>>> > >>>> Expedite synchronize_rcu() during the hotplug path to accelerate the > >>>> operation. Since CPU hotplug is a user-initiated administrative task, > >>>> it should complete as quickly as possible. > >>>> > >>>> Performance data on a PPC64 system with 400 CPUs: > >>>> > >>>> + ppc64_cpu --smt=1 (SMT8 to SMT1) > >>>> Before: real 1m14.792s > >>>> After: real 0m03.205s # ~23x improvement > >>>> > >>>> + ppc64_cpu --smt=8 (SMT1 to SMT8) > >>>> Before: real 2m27.695s > >>>> After: real 0m02.510s # ~58x improvement > >>>> > >>>> Above numbers were collected on Linux 6.19.0-rc4-00310-g755bc1335e3b > >>>> > >>>> [1] https://lore.kernel.org/all/5f2ab8a44d685701fe36cdaa8042a1aef215d10d.camel@linux.vnet.ibm.com > >>>> > >>> Also you can try: echo 1 > /sys/module/rcutree/parameters/rcu_normal_wake_from_gp > >>> to speedup regular synchronize_rcu() call. But i am not saying that it would beat > >>> your "expedited switch" improvement. > >>> > >> > >> Hi Uladzislau. > >> > >> Had a discussion on this at LPC, having in kernel solution is likely > >> better than having it in userspace. > >> > >> - Having it in kernel would make it work across all archs. Why should > >> any user wait when one initiates the hotplug. > >> > >> - userspace tools are spread across such as chcpu, ppc64_cpu etc. > >> though internally most do "0/1 > /sys/devices/system/cpu/cpuN/online". > >> We will have to repeat the same in each tool. > >> > >> - There is already /sys/kernel/rcu_expedited which is better if at all > >> we need to fallback to userspace. > >> > > Sounds good to me. I agree it is better to bypass parameters. > > Another way to make it in-kernel would be to make the RCU normal wake from GP optimization enabled for > 16 CPUs by default. > > I was considering this, but I did not bring it up because I did not know that there are large systems that might benefit from it until now. > IMO, we can increase that threshold. 512/1024 is not a problem at all. But as Paul mentioned, we should consider scalability enhancement. From the other hand it is also probably worth to get into the state when we really see them :) -- Uladzislau Rezki