From: Balbir Singh <balbirs@nvidia.com>
To: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Cc: "David Hildenbrand" <david@kernel.org>,
"Liam R. Howlett" <liam@infradead.org>,
"Vlastimil Babka" <vbabka@kernel.org>,
"Mike Rapoport" <rppt@kernel.org>,
"Suren Baghdasaryan" <surenb@google.com>,
"Michal Hocko" <mhocko@suse.com>,
"David Rientjes" <rientjes@google.com>,
"Shakeel Butt" <shakeel.butt@linux.dev>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Jann Horn" <jannh@google.com>,
"Pedro Falcato" <pfalcato@suse.de>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
"Jan Kara" <jack@suse.cz>, "Nhat Pham" <nphamcs@gmail.com>,
"Johannes Weiner" <hannes@cmpxchg.org>, "Zi Yan" <ziy@nvidia.com>,
"Matthew Brost" <matthew.brost@intel.com>,
"Joshua Hahn" <joshua.hahnjy@gmail.com>,
"Byungchul Park" <byungchul@sk.com>,
"Gregory Price" <gourry@gourry.net>,
"Ying Huang" <ying.huang@linux.alibaba.com>,
"Alistair Popple" <apopple@nvidia.com>,
"Usama Arif" <usama.arif@linux.dev>,
"Kairui Song" <kasong@tencent.com>,
"Jérôme Glisse" <jglisse@redhat.com>,
"Kiryl Shutsemau" <kas@kernel.org>,
"Yu-cheng Yu" <yu-cheng.yu@intel.com>,
"Dave Hansen" <dave.hansen@linux.intel.com>,
"Dennis Zhou" <dennis@kernel.org>, "Tejun Heo" <tj@kernel.org>,
"Christoph Lameter" <cl@gentwo.org>,
"Peng Fan" <peng.fan@nxp.com>, "Josef Bacik" <jbacik@fb.com>,
"Chris Mason" <mason@kernel.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
linux-fsdevel@vger.kernel.org,
"Borislav Petkov (AMD)" <bp@alien8.de>,
"Mark Brown" <broonie@kernel.org>,
"Dennis Zhou (Facebook)" <dennisszhou@gmail.com>,
"Rick Edgecombe" <rick.p.edgecombe@intel.com>,
stable@vger.kernel.org
Subject: Re: [PATCH v2 01/13] mm/mmap: free gathered pages before MMF_OOM_SKIP
Date: Mon, 16 Nov 2026 12:53:13 +1100 [thread overview]
Message-ID: <avpbxk6pzvh4hoLG@parvat> (raw)
In-Reply-To: <20261113-mason-migrate-et-al-fixes-v2-1-50a2ed382b61@kernel.org>
On Fri, Nov 13, 2026 at 09:45:19PM +0000, Lorenzo Stoakes (ARM) wrote:
> The OOM killer can quickly trigger again after having selected a
> victim. That victim might not have freed its memory yet, but its mm will
> be marked MMF_OOM_SKIP by this point, so the OOM killer will ignore it.
>
> This can cause the OOM killer to kill an innocent process when the first
> might have freed enough memory to relieve the memory pressure.
>
> exit_mmap() marks the mm MMF_OOM_SKIP once unmap_vmas() returns.
> However unmap_vmas() does not free the last batch of pages it
> gathered. That is left to tlb_finish_mmu(), which runs after
> MMF_OOM_SKIP is set, and the batch can be large.
>
> Fix the issue by freeing the gathered pages of an OOM victim before
> marking the mm MMF_OOM_SKIP.
>
> Note that the task which calls mmput() might not be the victim - if it
> took a reference on the mm (e.g. ptrace), it may hold the last reference
> and release it when it calls mmput().
>
> Therefore we cannot rely on current to tell whether the mm belongs to an
> OOM victim. Instead, partially revert commit b3541d912a84 ("mm: delete
> unused MMF_OOM_VICTIM flag") to reinstate MMF_OOM_VICTIM, which
> mark_oom_victim() sets on the victim's mm, and check that.
>
> Reported-by: Chris Mason <mason@kernel.org>
> Fixes: bf3980c85212 ("mm: drop oom code from exit_mmap")
> Cc: stable@vger.kernel.org
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> Tested-by: Gregory Price (Meta) <gourry@gourry.net>
> Acked-by: Michal Hocko <mhocko@suse.com>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
> Acked-by: Usama Arif <usama.arif@linux.dev>
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> ---
> include/linux/mm_types.h | 2 ++
> include/linux/oom.h | 9 +++++++++
> mm/mmap.c | 9 ++++++++-
> mm/oom_kill.c | 4 +++-
> 4 files changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 1ed35d258d2e..09e63ff12b04 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -2030,6 +2030,8 @@ enum {
> #define MMF_TOPDOWN 31 /* mm searches top down by default */
> #define MMF_TOPDOWN_MASK BIT(MMF_TOPDOWN)
>
> +#define MMF_OOM_VICTIM 32 /* mm is the oom victim */
> +
> #define MMF_INIT_LEGACY_MASK (MMF_DUMP_FILTER_MASK |\
> MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK |\
> MMF_VM_MERGE_ANY_MASK | MMF_TOPDOWN_MASK)
> diff --git a/include/linux/oom.h b/include/linux/oom.h
> index 00da05d227e6..a9390cb70541 100644
> --- a/include/linux/oom.h
> +++ b/include/linux/oom.h
> @@ -76,6 +76,15 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk)
> return tsk->signal->oom_mm;
> }
>
> +/*
> + * Use this helper if tsk->mm != mm and the victim mm needs a special
> + * handling. This is guaranteed to stay true after once set.
> + */
> +static inline bool mm_is_oom_victim(struct mm_struct *mm)
> +{
> + return mm_flags_test(MMF_OOM_VICTIM, mm);
> +}
> +
> /*
> * Checks whether a page fault on the given mm is still reliable.
> * This is no longer true if the oom reaper started to reap the
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 7618e72766a5..4cb18b217048 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -1313,11 +1313,18 @@ void exit_mmap(struct mm_struct *mm)
> /* update_hiwater_rss(mm) here? but nobody should be looking */
> /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */
> unmap_vmas(&tlb, &unmap);
> +
> + /*
> + * Free the memory before marking MMF_OOM_SKIP so a re-triggered OOM
> + * killer will only skip this mm if it's already freed all it can.
> + */
> + if (mm_is_oom_victim(mm))
> + tlb_flush_mmu(&tlb);
> mmap_read_unlock(mm);
>
> /*
> * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
> - * because the memory has been already freed.
> + * because the memory has been already unmapped.
> */
> mm_flags_set(MMF_OOM_SKIP, mm);
> mmap_write_lock(mm);
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 5d48bd862c27..9de57a23893c 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -752,8 +752,10 @@ static void mark_oom_victim(struct task_struct *tsk)
> return;
>
> /* oom_mm is bound to the signal struct life time. */
> - if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm))
> + if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) {
> mmgrab(tsk->signal->oom_mm);
> + mm_flags_set(MMF_OOM_VICTIM, mm);
> + }
>
> /*
> * Make sure that the process is woken up from uninterruptible sleep
>
Acked-by: Balbir Singh <balbirs@nvidia.com>
next prev parent reply other threads:[~2026-11-16 1:53 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-13 21:45 [PATCH v2 00/13] mm: batch of (urgentish?) fixes in oomk, cachestat, dev-migrate, percpu Lorenzo Stoakes (ARM)
2026-11-13 21:45 ` [PATCH v2 01/13] mm/mmap: free gathered pages before MMF_OOM_SKIP Lorenzo Stoakes (ARM)
2026-11-14 19:56 ` Chris Down
2026-11-16 1:53 ` Balbir Singh [this message]
2026-11-13 21:45 ` [PATCH v2 02/13] mm: cachestat: count each page only once Lorenzo Stoakes (ARM)
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 03/13] mm: cachestat: restart the walk after each entry Lorenzo Stoakes (ARM)
2026-11-14 18:08 ` Nhat Pham
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 04/13] mm/migrate_device: allocate THP deferred split heads Lorenzo Stoakes (ARM)
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 05/13] mm/migrate_device: don't collect a range twice Lorenzo Stoakes (ARM)
2026-11-14 5:09 ` Balbir Singh
2026-11-14 6:45 ` Mika Penttilä
2026-11-14 18:34 ` Lorenzo Stoakes (ARM)
2026-11-15 9:24 ` Mika Penttilä
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 06/13] mm/migrate_device: don't let go of the fault page Lorenzo Stoakes (ARM)
2026-11-13 22:31 ` Balbir Singh
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 07/13] mm/migrate_device: look up fault folio after a split Lorenzo Stoakes (ARM)
2026-11-13 22:20 ` Balbir Singh
2026-11-14 19:56 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 08/13] mm/migrate_device: fix ksmd oops on a migrated KSM page Lorenzo Stoakes (ARM)
2026-11-14 19:56 ` Chris Down
2026-11-15 12:43 ` Lorenzo Stoakes (ARM)
2026-11-13 21:45 ` [PATCH v2 09/13] mm/migrate_device: fix use-after-free of the anon rmap Lorenzo Stoakes (ARM)
2026-11-14 19:57 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 10/13] mm/migrate_device: refuse to migrate shadow stacks Lorenzo Stoakes (ARM)
2026-11-14 19:57 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 11/13] percpu: fix contig hints that are too low Lorenzo Stoakes (ARM)
2026-11-14 19:57 ` Chris Down
2026-11-15 13:20 ` Lorenzo Stoakes (ARM)
2026-11-13 21:45 ` [PATCH v2 12/13] percpu: search the whole reserved chunk Lorenzo Stoakes (ARM)
2026-11-14 19:57 ` Chris Down
2026-11-13 21:45 ` [PATCH v2 13/13] percpu: queue the balance work on an unbound workqueue Lorenzo Stoakes (ARM)
2026-11-14 19:57 ` Chris Down
2026-11-14 23:07 ` Balbir Singh
2026-11-15 12:41 ` Lorenzo Stoakes (ARM)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avpbxk6pzvh4hoLG@parvat \
--to=balbirs@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=bp@alien8.de \
--cc=broonie@kernel.org \
--cc=byungchul@sk.com \
--cc=cl@gentwo.org \
--cc=dave.hansen@linux.intel.com \
--cc=david@kernel.org \
--cc=dennis@kernel.org \
--cc=dennisszhou@gmail.com \
--cc=gourry@gourry.net \
--cc=hannes@cmpxchg.org \
--cc=jack@suse.cz \
--cc=jannh@google.com \
--cc=jbacik@fb.com \
--cc=jglisse@redhat.com \
--cc=joshua.hahnjy@gmail.com \
--cc=kas@kernel.org \
--cc=kasong@tencent.com \
--cc=liam@infradead.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mason@kernel.org \
--cc=matthew.brost@intel.com \
--cc=mhocko@suse.com \
--cc=nphamcs@gmail.com \
--cc=peng.fan@nxp.com \
--cc=pfalcato@suse.de \
--cc=rick.p.edgecombe@intel.com \
--cc=rientjes@google.com \
--cc=rppt@kernel.org \
--cc=shakeel.butt@linux.dev \
--cc=stable@vger.kernel.org \
--cc=surenb@google.com \
--cc=tj@kernel.org \
--cc=usama.arif@linux.dev \
--cc=vbabka@kernel.org \
--cc=willy@infradead.org \
--cc=ying.huang@linux.alibaba.com \
--cc=yu-cheng.yu@intel.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®