From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98E803AFD0A for ; Thu, 30 Jul 2026 10:16:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785406586; cv=none; b=EAbgeuXdLr+EEmMLNUK7G+aRErlZbH/GNmo+Tamo0xfMQm53uvkeWZXsx3iWTyDOI9Zkudat0c5WussI1EVpcSl+WyZUGKw+s/8lW83o22p7ljiSJWVAXXwBJJLfCEU+6WEIt9/4sT1i7XmKneVNbU55/DaUSZHDuyeN/vDdNWI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785406586; c=relaxed/simple; bh=UJPQ8ZdTEBI1Xvyh2Evh5pLo6a+5xv0PKQhrswoofWk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=loHHfCAFXv/b47SD3qg2uz6QzTtdQSHr1OMCgPprf/YV3t9BOBblFN88IR8Jg9g++DJ9P1JQ7GQ5LFCl4osZ6uxx4deYSUD5PEtsBOVbobphwPxqLFVQcrQlqNjkK4PPPf27IrLOFfGv1DkO+iGsy8qKyMsF5m4cEkevhZnBY84= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g/xYDbwi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g/xYDbwi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1E641F000E9; Thu, 30 Jul 2026 10:16:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785406585; bh=hwQXtI5NkzEH7Kjv8pvtqVVW6iAaKpBh3/KqqndjeGE=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=g/xYDbwi9nH+4w3ZdzklsgnymASpCq/zmRuiTDKBJuD3OlmsZBzlS8kYCMb73kRi4 mdllQn7gnYyUt9CfcNPkRmhza7mbyb1BoEQKu1NlVsW3xAlcA42VKTJnrJ8zY5fiBu ezCgLcHi7cR3v43PK1Pwu4WoDzYDf7z4wORRw+52E3HzwkIL9B1byalpZpVQuMf6sc UQVPyev2jgAg1NbOdu7dqMfTf8jrLouKz+nXKleo9mvscvtZc7iRQ9SZU8UU+4mLja Y1jh+2Sn1dadWfH52WlPQ4I+ZF5fh0npB7Q+dh/TbEjd9VUrOppMvNnPo8t72R/DL5 K8Lo8gU8IDUCg== Date: Thu, 30 Jul 2026 11:16:21 +0100 From: Conor Dooley To: Paul Sherman Cc: palmer@dabbelt.com, pjw@kernel.org, aou@eecs.berkeley.edu, alex@ghiti.fr, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] riscv: smp: fix non-SPINWAIT secondary hart rendezvous for fw_dynamic platforms Message-ID: <20260730-crusader-disfigure-0ab121347aa1@spud> References: <20260729083414.39339-1-shermanpauldylan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="IQgEodWEVcoUETSD" Content-Disposition: inline In-Reply-To: <20260729083414.39339-1-shermanpauldylan@gmail.com> --IQgEodWEVcoUETSD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 29, 2026 at 01:34:14AM -0700, Paul Sherman wrote: > On platforms where firmware (e.g. fw_dynamic) releases all harts to the > same Linux entry point simultaneously, the hart selected by OpenSBI as > Domain0 Boot HART is already in SBI_HSM_STATE_STARTED when Linux later > attempts to bring it online as a secondary CPU via SBI HART_START. > OpenSBI correctly returns SBI_ERR_ALREADY_STARTED, but Linux has no > recovery path: with CONFIG_RISCV_BOOT_SPINWAIT=3Dn, there is no secondary > wait path in _start_kernel for harts that entered Linux directly from > firmware, and sbi_cpu_start() has no handler for SBI_ERR_ALREADY_STARTED. >=20 > This causes one CPU to be permanently dropped per boot. The missing CPU > is always the OpenSBI Domain0 Boot HART, which varies between boots on > Sophgo SG2042 (hart 1, 2, or 3), explaining the apparent 'moving victim'. >=20 > Fix with three cooperating changes: >=20 > 1. Initialize boot_cpu_hartid to INVALID_HARTID instead of relying on > BSS zero-initialization. Without this, boot_cpu_hartid aliases with > hart 0, causing hart 0 to always appear to win the boot CPU race > regardless of which hart actually stored its hartid first. >=20 > 2. Add a non-SPINWAIT secondary wait path in _start_kernel. When > firmware releases multiple harts to the same entry point, non-primary > harts divert into the existing spinwait rendezvous arrays (previously > used only by CONFIG_RISCV_BOOT_SPINWAIT) and wait for cpu_start() to > provide boot data before proceeding to secondary startup. >=20 > 3. Handle SBI_ERR_ALREADY_STARTED (-EALREADY) in sbi_cpu_start(). When > HART_START returns -EALREADY, the hart is already executing in Linux > and spinning in .Lwait_for_cpu_up_sbi. Write the spinwait rendezvous > arrays to release it into secondary startup, matching the approach > used by cpu_ops_spinwait.c. >=20 > The arrays __cpu_spinwait_stack_pointer and __cpu_spinwait_task_pointer > are defined unconditionally in cpu_ops_spinwait.c but their extern > declarations in head.h were guarded by CONFIG_RISCV_BOOT_SPINWAIT. > Move the declarations outside the guard since the arrays are always > present and now used by both boot paths. >=20 > Note: The NR_CPUS bound check mirrors the identical pattern in > cpu_ops_spinwait.c:32 which guards the same arrays against out-of-range > hartids on platforms with discontiguous hart numbering. >=20 > Link: https://lore.kernel.org/linux-riscv/20260727221508.5179-1-shermanpa= uldylan@gmail.com/ > Tested-on: Milk-V Pioneer (Sophgo SG2042, 64-hart RISC-V, 4-NUMA nodes, > 128GB DDR4, OpenSBI v1.5, Linux v7.2-rc5) > Result: boot_cpu_hartid correctly reflects Domain0 Boot HART, all 64 > CPUs online in 2.7 seconds (was 63 CPUs, boot always on hart 0) FWIW, neither of these are valid tags, just put this information in the commit message proper. > Signed-off-by: Paul Sherman --IQgEodWEVcoUETSD Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRh246EGq/8RLhDjO14tDGHoIJi0gUCamskcgAKCRB4tDGHoIJi 0rPWAQCRC236qWEk7INUdaQxu3SiRxaoOw7opvQ+G34KDAUMTgEAlaIIWgqX9dsj KyVz/JEbRQUpYalqqFUOLzjkzUnZBQo= =95V8 -----END PGP SIGNATURE----- --IQgEodWEVcoUETSD--