From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from foss.arm.com (foss.arm.com [217.140.110.172])
	by smtp.subspace.kernel.org (Postfix) with ESMTP id 303BE326D62
	for <linux-kernel@vger.kernel.org>; Mon, 17 Nov 2025 10:34:29 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172
ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
	t=1763375671; cv=none; b=tMlBgI/iFEznba+bk+bxxySNGLPPG5bj2H4edUN50rkATouEHa8vmqV7mTXHHfB4nc4wTZxgRX6d6gsu1mlmaBuPSWNYZcBHxGr9JR1ubieA+6A/qz2vHicyc43XWikTBxayUCRlXCJOBv8mk5p/EvvM2xQMkgDsJiopqunFxiQ=
ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org;
	s=arc-20240116; t=1763375671; c=relaxed/simple;
	bh=Z+52sbIpiNoNA9Y69lySwnsy5JOKArWNNai4IdwZi94=;
	h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:
	 Content-Type:Content-Disposition:In-Reply-To; b=H7VC2XCyW1WXaVrOdYHyNWIfpeYG151CNbQn+Gb+/YkpBxnEIj1Lh2mgAiGYfnOvAdmQwps4ku8CtRb5HL8G1AzEAWqSr6WEatp+6hWvLoHuPyCvagOEq5xf2rc8sRJwg8WdRdZZRwK4SygDD+Gg3N/mIEDIf/+1AoMVCb9Jivg=
ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172
Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com
Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com
Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14])
	by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BF2F9FEC;
	Mon, 17 Nov 2025 02:34:21 -0800 (PST)
Received: from J2N7QTR9R3 (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14])
	by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C9E773F740;
	Mon, 17 Nov 2025 02:34:27 -0800 (PST)
Date: Mon, 17 Nov 2025 10:34:22 +0000
From: Mark Rutland <mark.rutland@arm.com>
To: Ben Niu <BenNiu@meta.com>
Cc: catalin.marinas@arm.com, will@kernel.org, tytso@mit.edu,
	Jason@zx2c4.com, linux-arm-kernel@lists.infradead.org,
	niuben003@gmail.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] tracing: Enable kprobe tracing for Arm64 asm functions
Message-ID: <aRr6Ll7h29dpt7zz@J2N7QTR9R3>
References: <20251027181749.240466-1-benniu@meta.com>
 <aQIltqoIVDwh4A6p@J2N7QTR9R3>
 <aQKpDIYwO9VTP7qM@meta.com>
 <aQNbjap69C6inZwb@J2N7QTR9R3>
 <aQOpd14RXnyQLOWR@meta.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <aQOpd14RXnyQLOWR@meta.com>

On Thu, Oct 30, 2025 at 11:07:51AM -0700, Ben Niu wrote:
> On Thu, Oct 30, 2025 at 12:35:25PM +0000, Mark Rutland wrote:
> > Is there something specific you want to trace, but cannot currently
> > trace (on arm64)?
> 
> For some reason, we only saw Arm64 Linux asm functions __arch_copy_to_user and
> __arch_copy_from_user being hot in our workloads, not those counterpart asm
> functions on x86, so we are trying to understand and improve performance of
> those Arm64 asm functions.

Are you sure that's not an artifact of those being out-of-line on arm64,
but inline on x86? On x86, the out-of-line forms are only used when the
CPU doesn't have FSRM, and when the CPU *does* have FSRM, the logic gets
inlined. See raw_copy_from_user(), raw_copy_to_user(), and
copy_user_generic() in arch/x86/include/asm/uaccess_64.h.

Have you checked that inlining is not skewing your results, and
artificially making those look hotter on am64 by virtue of centralizing
samples to the same IP/PC range?

Can you share any information on those workloads? e.g. which callchains
were hot?

Mark.