From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from va-2-111.ptr.blmpb.com (va-2-111.ptr.blmpb.com [209.127.231.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE73442314F for ; Thu, 9 Jul 2026 12:31:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.127.231.111 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783600275; cv=none; b=T40iIRgyh/Qfv6XhHSHeGNkzq/gLWX8CYtOhVLgI6bruAvYkKjPRk6jM5Fn4pCP+OQ5d4cUBbBNxLfgOUGPELZ2/UGilSoXyL7UjTfqk5uXqgjCXfsZvf2EYePlfyQ2ZY9tBEjgRg+SdpG9jJuoXzn2PU12QIvF9VOZESHQr3FY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783600275; c=relaxed/simple; bh=R7faOJ929c2PCJcFq+Dk6kkaNQw2kQCLns9yUo6xhDY=; h=In-Reply-To:References:From:Date:Message-Id:To:Subject: Content-Type:Cc:Mime-Version; b=MbYUyf4mc960BD1bA7AWzcyE7B5VC1sGYYgwT4uuaDKUrI8BFa4HNiNZ82dE+Bne7Bwuy8mNwT0HRXo+1i+z0nvQpFC0A7HRKGNTaeUzzZHf6Guy8xP5sMCesDn5S2c98AyJJ3iJcztID4NNr6PWcK45HWdGAKJb/OqBQEx8L8M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=L/DEtblp; arc=none smtp.client-ip=209.127.231.111 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="L/DEtblp" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1783600262; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=tBoFytz6cA8ILrpJXluLdSOQPOneediXsnQek8X+OpU=; b=L/DEtblp/5M20jds4+883Hq3dYSlD5XbmmLCgkBrubZbVihF5WJ37osyuxOr7O4F9RSJ29 5BWJMGLXtxs+FYxnDzRxXqbnGZCIDDlJ19BbyOr2EilgOTN+h0iRLCmI6nuTC4m2Hl42SV ZM1GbrjMp6bOIvTtMJ29Hpw/ZLLgYP22/HaJSfFZFFowPTpmvnDMhsCM124lbcuCjtjtyX a3HJCdgIxnyIQAeE1dPuRcnJGRU1iqB2Qc7JTibelnwZPsWDN83eknRZ7IubGAvmQhvS2+ Fn5KHSoChdnQmQN3tMtkYPsHddblsfjbVIahGwGDPhxD38Q5N8WEYGnfKfeLEg== X-Mailer: git-send-email 2.20.1 In-Reply-To: <20260709122933.4021501-1-zhouchuyi@bytedance.com> X-Lms-Return-Path: References: <20260709122933.4021501-1-zhouchuyi@bytedance.com> From: "Chuyi Zhou" Date: Thu, 9 Jul 2026 20:29:22 +0800 Message-Id: <20260709122933.4021501-4-zhouchuyi@bytedance.com> To: , , , , , , , , , , , , , Subject: [PATCH v10 03/14] smp: Refactor remote CPU selection in smp_call_function_any() Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=UTF-8 Cc: , "Chuyi Zhou" Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Original-From: Chuyi Zhou smp_call_function_any() disables preemption across the entire operation: selecting a target CPU, enqueueing the IPI, and synchronously waiting for the remote CPU. smp_call_function_single() already re-enables preemption before the synchronous csd_lock_wait(), so callers of smp_call_function_any() should benefit from the same shorter preemption-disabled section. Simply removing get_cpu() and put_cpu() from smp_call_function_any() would leave the preemption disablement entirely to smp_call_function_single(). That opens a preemption window between selecting the remote CPU, for example via sched_numa_find_nth_cpu(), and dispatching the IPI in smp_call_function_single(). If the selected CPU is fully offlined in that window, smp_call_function_single() fails its cpu_online() check and returns -ENXIO to the caller, violating the guarantee that smp_call_function_any() executes on any online CPU in the mask. Move the remote CPU selection into a common __smp_call_function_single() helper. Keep the target CPU selection and IPI dispatch within the same preemption-disabled region, while still allowing the wait path to use the shorter preemption-disabled section provided by smp_call_function_single(). Signed-off-by: Chuyi Zhou Tested-by: Paul E. McKenney Reviewed-by: Sebastian Andrzej Siewior --- include/linux/smp.h | 3 +-- kernel/smp.c | 56 ++++++++++++++++++++++++--------------------- kernel/up.c | 3 +-- 3 files changed, 32 insertions(+), 30 deletions(-) diff --git a/include/linux/smp.h b/include/linux/smp.h index 6925d15ccaa7..11e36c7bc4d6 100644 --- a/include/linux/smp.h +++ b/include/linux/smp.h @@ -47,8 +47,7 @@ extern void __smp_call_single_queue(int cpu, struct llist_node *node); /* total number of cpus in this system (may exceed NR_CPUS) */ extern unsigned int total_cpus; -int smp_call_function_single(int cpuid, smp_call_func_t func, void *info, - int wait); +int smp_call_function_single(int cpuid, smp_call_func_t func, void *info, bool wait); void on_each_cpu_cond_mask(smp_cond_func_t cond_func, smp_call_func_t func, void *info, bool wait, const struct cpumask *mask); diff --git a/kernel/smp.c b/kernel/smp.c index 292eefadddbc..92e1dffe4589 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -641,17 +641,9 @@ void flush_smp_call_function_queue(void) local_irq_restore(flags); } -/** - * smp_call_function_single - Run a function on a specific CPU - * @cpu: Specific target CPU for this function. - * @func: The function to run. This must be fast and non-blocking. - * @info: An arbitrary pointer to pass to the function. - * @wait: If true, wait until function has completed on other CPUs. - * - * Returns: %0 on success, else a negative status code. - */ -int smp_call_function_single(int cpu, smp_call_func_t func, void *info, - int wait) +static int __smp_call_function_single(int cpu, smp_call_func_t func, + void *info, const struct cpumask *mask, + bool wait) { call_single_data_t *csd; call_single_data_t csd_stack = { @@ -668,6 +660,14 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info, */ this_cpu = get_cpu(); + if (mask) { + /* Try for same CPU (cheapest) */ + if (!cpumask_test_cpu(this_cpu, mask)) + cpu = sched_numa_find_nth_cpu(mask, 0, cpu_to_node(this_cpu)); + else + cpu = this_cpu; + } + /* * Can deadlock when called with interrupts disabled. * We allow cpu's that are not yet online though, as no one else can @@ -712,6 +712,20 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info, return err; } + +/** + * smp_call_function_single - Run a function on a specific CPU + * @cpu: Specific target CPU for this function. + * @func: The function to run. This must be fast and non-blocking. + * @info: An arbitrary pointer to pass to the function. + * @wait: If true, wait until function has completed on other CPUs. + * + * Returns: %0 on success, else a negative status code. + */ +int smp_call_function_single(int cpu, smp_call_func_t func, void *info, bool wait) +{ + return __smp_call_function_single(cpu, func, info, NULL, wait); +} EXPORT_SYMBOL(smp_call_function_single); /** @@ -762,10 +776,10 @@ EXPORT_SYMBOL_GPL(smp_call_function_single_async); /** * smp_call_function_any - Run a function on any of the given cpus - * @mask: The mask of cpus it can run on. - * @func: The function to run. This must be fast and non-blocking. - * @info: An arbitrary pointer to pass to the function. - * @wait: If true, wait until function has completed. + * @mask: The mask of cpus it can run on. + * @func: The function to run. This must be fast and non-blocking. + * @info: An arbitrary pointer to pass to the function. + * @wait: If true, wait until function has completed. * * Selection preference: * 1) current cpu if in @mask @@ -776,17 +790,7 @@ EXPORT_SYMBOL_GPL(smp_call_function_single_async); int smp_call_function_any(const struct cpumask *mask, smp_call_func_t func, void *info, int wait) { - unsigned int cpu; - int ret; - - /* Try for same CPU (cheapest) */ - cpu = get_cpu(); - if (!cpumask_test_cpu(cpu, mask)) - cpu = sched_numa_find_nth_cpu(mask, 0, cpu_to_node(cpu)); - - ret = smp_call_function_single(cpu, func, info, wait); - put_cpu(); - return ret; + return __smp_call_function_single(-1, func, info, mask, wait); } EXPORT_SYMBOL_GPL(smp_call_function_any); diff --git a/kernel/up.c b/kernel/up.c index df50828cc2f0..6d4ac9502e8b 100644 --- a/kernel/up.c +++ b/kernel/up.c @@ -9,8 +9,7 @@ #include #include -int smp_call_function_single(int cpu, void (*func) (void *info), void *info, - int wait) +int smp_call_function_single(int cpu, void (*func)(void *info), void *info, bool wait) { unsigned long flags; -- 2.20.1