From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B9D13019BF for ; Mon, 1 Dec 2025 10:19:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764584353; cv=none; b=Z4tAk5fG0VwYpq8rlII/4G9rZIfh0ACsqtTy9rclDljoTDGMEegVFRotWO0Q7UVqzPCfEuF5EvkxbF26oKfGdGSJtwN8GMFv/pZ/TuTcThsLNOX4ARX/ClWqe17K3fUV0L6pSR/sHzidh3m819wf4kmgZWEF7GkNFDV3R8mETLg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764584353; c=relaxed/simple; bh=xNjLM4V4MqHGMgpf2NW2wWz3QXuoLL9CSJchibOclpk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=pqrEYh3sNBcpe9DjslHHBafSKwt+fRAwIofCvcB6mXOk8nmXb46cnuBZw75F3Ma8A2nY6NdX6zJKc8Ovp72sfS9mGynrwj2yWgDpPOeXSRqJdBVZ1ky6LN5LG/wWgjoJozCIgOf9/AygzGIJTWyYFwl2enxBxGKmQtWviJ++rew= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JHM8+8tL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JHM8+8tL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D851DC4CEF1; Mon, 1 Dec 2025 10:19:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1764584352; bh=xNjLM4V4MqHGMgpf2NW2wWz3QXuoLL9CSJchibOclpk=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=JHM8+8tLHacbP4wV+snjkXpnDRU77Jmqv1LFHFUyjgRG9uB+wXh6Gnll2/rBDcKxL d4x3bFQnQ3cWyPwKm/pNUxGPJIDgcfw2M+ZxQSrq8P4heDa/bcYZD5A+fMXn17dFVu zH81sh/2V3oBKgJqdpGuvbAY4ySCem8W1/95GoSC3+k14OdQxu+u0ceHs6RhM1WLCY 3D51fV9eZXfhPFfuDETwaZaGFCDXX1M7qNfI62GV0q1EEvylDvzscXErDzibr+YPd9 rcU3XotlAls8cEA9ogmi9ERzZ96+voTu1NXyezO86S9nE74cg1VJ1aZiJeynF9KyzB MqYiibqWJV80Q== Message-ID: Date: Mon, 1 Dec 2025 11:19:04 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 4/4] mm: Split a slow path for updating mm counters To: Gabriel Krisman Bertazi , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, jack@suse.cz, Mateusz Guzik , Shakeel Butt , Michal Hocko , Mathieu Desnoyers , Dennis Zhou , Tejun Heo , Christoph Lameter , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan References: <20251127233635.4170047-1-krisman@suse.de> <20251127233635.4170047-5-krisman@suse.de> From: "David Hildenbrand (Red Hat)" Content-Language: en-US In-Reply-To: <20251127233635.4170047-5-krisman@suse.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 11/28/25 00:36, Gabriel Krisman Bertazi wrote: > For cases where we know we are not coming from local context, there is > no point in touching current when incrementing/decrementing the > counters. Split this path into another helper to avoid this cost. > > Signed-off-by: Gabriel Krisman Bertazi > --- > arch/s390/mm/gmap_helpers.c | 4 ++-- > arch/s390/mm/pgtable.c | 4 ++-- > fs/exec.c | 2 +- > include/linux/mm.h | 14 +++++++++++--- > kernel/events/uprobes.c | 2 +- > mm/filemap.c | 2 +- > mm/huge_memory.c | 22 +++++++++++----------- > mm/khugepaged.c | 6 +++--- > mm/ksm.c | 2 +- > mm/madvise.c | 2 +- > mm/memory.c | 20 ++++++++++---------- > mm/migrate.c | 2 +- > mm/migrate_device.c | 2 +- > mm/rmap.c | 16 ++++++++-------- > mm/swapfile.c | 6 +++--- > mm/userfaultfd.c | 2 +- > 16 files changed, 58 insertions(+), 50 deletions(-) > > diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c > index d4c3c36855e2..6d8498c56d08 100644 > --- a/arch/s390/mm/gmap_helpers.c > +++ b/arch/s390/mm/gmap_helpers.c > @@ -29,9 +29,9 @@ > static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) > { > if (!non_swap_entry(entry)) > - dec_mm_counter(mm, MM_SWAPENTS); > + dec_mm_counter_other(mm, MM_SWAPENTS); > else if (is_migration_entry(entry)) > - dec_mm_counter(mm, mm_counter(pfn_swap_entry_folio(entry))); > + dec_mm_counter_other(mm, mm_counter(pfn_swap_entry_folio(entry))); > free_swap_and_cache(entry); > } > > diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c > index 0fde20bbc50b..021a04f958e5 100644 > --- a/arch/s390/mm/pgtable.c > +++ b/arch/s390/mm/pgtable.c > @@ -686,11 +686,11 @@ void ptep_unshadow_pte(struct mm_struct *mm, unsigned long saddr, pte_t *ptep) > static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) > { > if (!non_swap_entry(entry)) > - dec_mm_counter(mm, MM_SWAPENTS); > + dec_mm_counter_other(mm, MM_SWAPENTS); > else if (is_migration_entry(entry)) { > struct folio *folio = pfn_swap_entry_folio(entry); > > - dec_mm_counter(mm, mm_counter(folio)); > + dec_mm_counter_other(mm, mm_counter(folio)); > } > free_swap_and_cache(entry); > } > diff --git a/fs/exec.c b/fs/exec.c > index 4298e7e08d5d..33d0eb00d315 100644 > --- a/fs/exec.c > +++ b/fs/exec.c > @@ -137,7 +137,7 @@ static void acct_arg_size(struct linux_binprm *bprm, unsigned long pages) > return; > > bprm->vma_pages = pages; > - add_mm_counter(mm, MM_ANONPAGES, diff); > + add_mm_counter_local(mm, MM_ANONPAGES, diff); > } > > static struct page *get_arg_page(struct linux_binprm *bprm, unsigned long pos, > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 29de4c60ac6c..2db12280e938 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -2689,7 +2689,7 @@ static inline unsigned long get_mm_counter_sum(struct mm_struct *mm, int member) > > void mm_trace_rss_stat(struct mm_struct *mm, int member); > > -static inline void add_mm_counter(struct mm_struct *mm, int member, long value) > +static inline void add_mm_counter_local(struct mm_struct *mm, int member, long value) > { > if (READ_ONCE(current->mm) == mm) > lazy_percpu_counter_add_fast(&mm->rss_stat[member], value); > @@ -2698,9 +2698,17 @@ static inline void add_mm_counter(struct mm_struct *mm, int member, long value) > > mm_trace_rss_stat(mm, member); > } > +static inline void add_mm_counter_other(struct mm_struct *mm, int member, long value) > +{ > + lazy_percpu_counter_add_atomic(&mm->rss_stat[member], value); > + > + mm_trace_rss_stat(mm, member); > +} > > -#define inc_mm_counter(mm, member) add_mm_counter(mm, member, 1) > -#define dec_mm_counter(mm, member) add_mm_counter(mm, member, -1) > +#define inc_mm_counter_local(mm, member) add_mm_counter_local(mm, member, 1) > +#define dec_mm_counter_local(mm, member) add_mm_counter_local(mm, member, -1) > +#define inc_mm_counter_other(mm, member) add_mm_counter_other(mm, member, 1) > +#define dec_mm_counter_other(mm, member) add_mm_counter_other(mm, member, -1) I'd have thought that there is a local and !local version, whereby the latter one would simply maintain the old name. The "_other()" sticks out a bit. E.g., cmpxch() vs. cmpxchg_local(). Or would "_remote()" better describe what "_other()" intends to do? -- Cheers David