From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30DD270818 for ; Wed, 7 Jan 2026 04:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.189 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767758880; cv=none; b=ERSZA6IxKLQUjtDZw24Hu2AUbRXBA2o+0SrUmfpXRn924OhefM25W12JSffIFRFsiMntZ4T8nWjGxTKrzsLOSHmKq13jHiqQrqavZrT2ghMZrzw7IIs0o5H6gH9KHO9Y/SKskCBrGCw1hVKEFV+N+ikPraC2LOa/GiA2EK+PPWI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767758880; c=relaxed/simple; bh=a/6F3Fua/rS8HFUuR945Fqc7GB6xy8g+Srmt92qAqtg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=DTe3pzesayonmox+XzEeR6whlC0ytzLAB7n5A6n2z5TNMSEMDWE7SMhTGoRf1WoUiXsHGCoXzy2FWiAQQsOommVBpVBBnEYfaLF/Ay47793y57Gwtzx1Az08Hx3od+yfGm0foO7j0TMCFXOygEu9ZN14chPhqRgRB32N9oi8BdI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=JChzz8JP; arc=none smtp.client-ip=91.218.175.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="JChzz8JP" Date: Tue, 6 Jan 2026 23:07:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1767758876; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gpQ5rJZWPuYDYZOiEs7ucS7UwBx/7FvHyEGjqrQa6bU=; b=JChzz8JPhTtft7tIloMgX1wvE/JiSsAUMmMhb6Cermsqcm1dL7/mzbjjbGuAowXHX70fW4 z62ipmFe8hARr9BTobOl8CHtBtAnjQWuVVyorJTe2QPvhsBrCgIGSLbLz0r/pfoP0p0UaU k0fmQCLxIqSkeofX1Rma95c+gsHKT98= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: David Wang <00107082@163.com> Cc: Suren Baghdasaryan , akpm@linux-foundation.org, hannes@cmpxchg.org, pasha.tatashin@soleen.com, souravpanda@google.com, vbabka@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH RFC] alloc_tag: add option to pick the first codetag along callchain Message-ID: References: <20251216064349.74501-1-00107082@163.com> <75285cc4.3c52.19b916d9490.Coremail.00107082@163.com> <37169c79.a0e5.19b93a2768f.Coremail.00107082@163.com> <4736b304.38a2.19b96888104.Coremail.00107082@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4736b304.38a2.19b96888104.Coremail.00107082@163.com> X-Migadu-Flow: FLOW_OUT On Wed, Jan 07, 2026 at 11:38:06AM +0800, David Wang wrote: > > At 2026-01-07 07:26:18, "Kent Overstreet" wrote: > >On Tue, Jan 06, 2026 at 10:07:36PM +0800, David Wang wrote: > >> I agree, the accounting would be incorrect for alloc sites down the callchain, and would confuse things. > >> When the call chain has more than one codetag, correct accounting for one codetag would always mean incorrect > >> accounting for other codetags, right? But I don't think picking the first tag would make the accounting totally incorrect. > > > >The trouble is you end up in situations where you have an alloc tag on > >the stack, but then you're doing an internal allocation that definitely > >should not be accounted to the outer alloc tag. > > > >E.g. there's a lot of internal mm allocations like this; object > >extension vectors was I think the first place where it came up, > >vmalloc() also has its own internal data structures that require > >allocations. > > > >Just using the outermost tag means these inner allocations will get > >accounted to other unrelated alloc tags _effectively at random_; meaning > >if we're burning more memory than we should be in a place like that it > >will never show up in a way that we'll notice and be able to track it > >down. > > Kind of feel that the same thing could be said for drivers: the driver could use more memory > than the data says....this is actually true.... > Different developer may have different focus concerning the allocation site. > > > > >> Totally agree. > >> I used to sum by filepath prefix to aggregate memory usage for drivers. > >> Take usb subsystem for example, on my system, the data say my usb drivers use up 200k memory, > >> and if pick first codetag, the data say ~350K. Which one is lying, or are those two both lying. I am confused. > >> > >> I think this also raises the question of what is the *correct* way to make use of /proc/allocinfo... > > > >So yes, summing by filepath prefix is the way we want things to work. > > > >But getting there - with a fully reliable end result - is a process. > > > >What you want to do is - preferably on a reasonably idle machine, aside > >from the code you're looking at - just look at everything in > >/proc/allocinfo and sort by size. Look at the biggest ones that might be > >relevant to your subsystem, and look for any that are suspicious and > >perhaps should be accounted to your code. Yes, that may entail reading > >code :) > > > >This is why accounting to the innermost tag is important - by doing it > >this way, if an allocation is being accounted at the wrong callsite > >they'll all be lumped together at the specific callsite that needs to be > >fixed, which then shows up higher than normal in /proc/allocations, so > >that it gets looked at. > > > >> >The fact that you have to be explicit about where the accounting happens > >> >via _noprof is a feature, not a bug :) > >> > >> But it is tedious... :( > > > >That's another way of saying it's easy :) > > > >Spot an allocation with insufficiently fine grained accounting and it's > >generally a 3-5 line patch to fix it, I've been doing those here and > >there - e.g. mempools, workqueues, rhashtables. > > > >One trick I did with rhashtables that may be relevant to other > >subsystems: rhashtable does background processing for your hash table, > >which will do new allocations for your hash table out of a workqueue. > > > >So rhashtable_init() gets wrapped in alloc_hooks(), and then it stashes > >the pointer to that alloc tag in the rhashtable, and uses it later for > >all those asynchronous allocations. > > > >This means that instead of seeing a ton of memory accounted to the > >rhashtable code, with no idea of which rhashtable is burning memory - > >all the rhashtable allocations are accounted to the callsit of the > >initialization, meaning it's trivial to see which one is burning memory. > > Not that easy, ....code keeps being refactored, _noprof need to be changed along. > I was trying to split the accounting for __filemap_get_folio to its callers in 6.18, > it was easy, only ~10 lines of code changes. But 6.19 starts with code refactors to > __filemap_get_folio, adding another level of indirection, allocation callchain becomes > longer, and more _noprof should be added...quite unpleasant... > > Sometimes I would feel too many _noprof could be obstacle for future code refactors.... > > PS: There are several allocation sites have *huge* memory accounting, __filemap_get_folio is > one of those. splitting those accounting to its callers would be more informative I'm curious why you need to change __filemap_get_folio()? In filesystem land we just lump that under "pagecache", but I guess you're doing more interesting things with it in driver land?