PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Zachary M. Raines]

Greetings,

When mounting and unmounting many filesystems, /proc/<pid>/mountinfo sometimes
contains entries which are duplicated many times.

Summary
=======

Sometimes on a system that is mounting and unmounting filesystems frequently,
for example running lots of docker containers, the size of /proc/1/mountinfo,
can become very large -- 100s, to 1000s of entries or more -- with the vast
majority being a single entry duplicated many times.

This causes other problems on the system, due to systemd parsing the mount table
whenever it changes, and eating up a lot of memory, for example [1]. Waiting
long enough there are rare events where the length of mountinfo can go into the
millions of lines and lead to OOM and kernel panics.

Running the reproducers below, I pretty reliably see an Ubuntu virtual machine
kernel panic due to lack of memory within about 24hrs.

Versions
========

Bisecting the kernel git history, I was able to track the issue back to
'2eea9ce4310d8 mounts: keep list of mounts in an rbtree' [2].

I've tested on 6.19-rc7 in a virtual machine and the issue is still present
there. /proc/version:

Linux version 6.19.0-rc7+ (ubuntu@kernel-builder) (gcc (Ubuntu 15.2.0-4ubuntu4)
15.2.0, GNU ld (GNU Binutils for Ubuntu) 2.45) #8 SMP PREEMPT_DYNAMIC Tue Jan 27
22:33:35 UTC 2026

running on Ubuntu 25.10

Reproducer
==========

The problem can be reproduced by mounting and then unmounting tmpfs in a loop
and in a seperate process reading /proc/1/mountinfo and checking for duplicates.

I used the following scripts:

1. Mounts and unmounts tmpfs

#!/bin/bash
counter=0
while true; do
    unique_name="tmpfs_$$_$counter"
   	mkdir -p "/tmp/$unique_name"
   	sudo mount -t tmpfs "$unique_name" "/tmp/$unique_name"
   	sudo umount "/tmp/$unique_name"
   	rmdir "/tmp/$unique_name"
   	((counter++))
   	sleep 0.1
done

2. Reads `/prod/1/mountinfo` and checks for duplicates

#!/bin/bash
THRESHOLD=75
echo "Starting monitoring at $(date)"
while true; do
    # Get mountinfo entries and count total
    mountinfo="$(cat /proc/1/mountinfo)"
    mountinfo_count=$(echo "$mountinfo" | wc -l)

    if ((mountinfo_count > THRESHOLD)); then
        echo "$(date): Mount count ($mountinfo_count) exceeds threshold ($THRESHOLD)"

        # Find and log duplicate mount points with their counts
        duplicates=$(echo "$mountinfo" | sort | uniq -cd)

        if [[ -n "$duplicates" ]]; then
            echo "Duplicate mounts :"
            echo "$duplicates"
        fi
        echo "====="
        echo "$mountinfo"
        echo "---"
    fi

    sleep 0.1
done

Typically, within 5-10 minutes duplicates can be observed, often including
hundreds or thousands of copies of the same mount point -- although the number
can rarely spike to much higher values. Given a long enough uptime, I've
observed up to 1.4 million duplicates at a time.

The duplication in mountinfo is very intermittent. `cat /proc/1/mountinfo` 100ms
later shows no duplication.

Additional diagnostics
======================

While running the script (2.) above, I also ran the following bpftrace script

3. Trace vfs_mounts as by `cat /proc/1/mountinfo`

#!/usr/bin/env bpftrace

fentry:show_mountinfo / comm == "cat"/ {
    @mnts[args->mnt] = count();
}

tracepoint:sched:sched_process_exit / comm == "cat"/ {
    for ($mnt : @mnts) {
        if ($mnt.1 > 1) {
            printf("Duplicate mount %p\n", $mnt.0);
            @dups[$mnt.0] = $mnt.1;
        }
    }
    clear(@mnts);
}

and observed that a single mount struct was reached multiple times -- perhaps
unsurprisingly exactly the same number as there were duplicates detected by
the above script.

Typical outputs of script (2.) and the bpftrace script above are

Starting monitoring at Tue Jan 27 20:48:13 UTC 2026
Tue Jan 27 20:50:43 UTC 2026: Mount count (696) exceeds threshold (75)
Duplicate mounts :
  /proc/sys/fs/binfmt_misc: 2 occurrences
  /tmp/tmpfs_856614_41491: 666 occurrences

and

@dups[0xffff88e5fb9f10a0]: 666

Best,
Zachary Raines

[1]: https://github.com/systemd/systemd/issues/37939
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2eea9ce4310d8c0f8ef1dbe7b0e7d9219ff02b97

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Christian Brauner]

[-- Attachment #1: Type: text/plain, Size: 4812 bytes --]

On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
> Greetings,
> 
> When mounting and unmounting many filesystems, /proc/<pid>/mountinfo sometimes
> contains entries which are duplicated many times.
> 
> Summary
> =======
> 
> Sometimes on a system that is mounting and unmounting filesystems frequently,
> for example running lots of docker containers, the size of /proc/1/mountinfo,
> can become very large -- 100s, to 1000s of entries or more -- with the vast
> majority being a single entry duplicated many times.
> 
> This causes other problems on the system, due to systemd parsing the mount table
> whenever it changes, and eating up a lot of memory, for example [1]. Waiting
> long enough there are rare events where the length of mountinfo can go into the
> millions of lines and lead to OOM and kernel panics.
> 
> Running the reproducers below, I pretty reliably see an Ubuntu virtual machine
> kernel panic due to lack of memory within about 24hrs.
> 
> Versions
> ========
> 
> Bisecting the kernel git history, I was able to track the issue back to
> '2eea9ce4310d8 mounts: keep list of mounts in an rbtree' [2].
> 
> I've tested on 6.19-rc7 in a virtual machine and the issue is still present
> there. /proc/version:
> 
> Linux version 6.19.0-rc7+ (ubuntu@kernel-builder) (gcc (Ubuntu 15.2.0-4ubuntu4)
> 15.2.0, GNU ld (GNU Binutils for Ubuntu) 2.45) #8 SMP PREEMPT_DYNAMIC Tue Jan 27
> 22:33:35 UTC 2026
> 
> running on Ubuntu 25.10
> 
> Reproducer
> ==========
> 
> The problem can be reproduced by mounting and then unmounting tmpfs in a loop
> and in a seperate process reading /proc/1/mountinfo and checking for duplicates.
> 
> I used the following scripts:
> 
> 1. Mounts and unmounts tmpfs
> 
> #!/bin/bash
> counter=0
> while true; do
>     unique_name="tmpfs_$$_$counter"
>    	mkdir -p "/tmp/$unique_name"
>    	sudo mount -t tmpfs "$unique_name" "/tmp/$unique_name"
>    	sudo umount "/tmp/$unique_name"
>    	rmdir "/tmp/$unique_name"
>    	((counter++))
>    	sleep 0.1
> done
> 
> 2. Reads `/prod/1/mountinfo` and checks for duplicates
> 
> #!/bin/bash
> THRESHOLD=75
> echo "Starting monitoring at $(date)"
> while true; do
>     # Get mountinfo entries and count total
>     mountinfo="$(cat /proc/1/mountinfo)"
>     mountinfo_count=$(echo "$mountinfo" | wc -l)
> 
>     if ((mountinfo_count > THRESHOLD)); then
>         echo "$(date): Mount count ($mountinfo_count) exceeds threshold ($THRESHOLD)"
> 
>         # Find and log duplicate mount points with their counts
>         duplicates=$(echo "$mountinfo" | sort | uniq -cd)
> 
>         if [[ -n "$duplicates" ]]; then
>             echo "Duplicate mounts :"
>             echo "$duplicates"
>         fi
>         echo "====="
>         echo "$mountinfo"
>         echo "---"
>     fi
> 
>     sleep 0.1
> done
> 
> Typically, within 5-10 minutes duplicates can be observed, often including
> hundreds or thousands of copies of the same mount point -- although the number
> can rarely spike to much higher values. Given a long enough uptime, I've
> observed up to 1.4 million duplicates at a time.
> 
> The duplication in mountinfo is very intermittent. `cat /proc/1/mountinfo` 100ms
> later shows no duplication.
> 
> Additional diagnostics
> ======================
> 
> While running the script (2.) above, I also ran the following bpftrace script
> 
> 3. Trace vfs_mounts as by `cat /proc/1/mountinfo`
> 
> #!/usr/bin/env bpftrace
> 
> fentry:show_mountinfo / comm == "cat"/ {
>     @mnts[args->mnt] = count();
> }
> 
> tracepoint:sched:sched_process_exit / comm == "cat"/ {
>     for ($mnt : @mnts) {
>         if ($mnt.1 > 1) {
>             printf("Duplicate mount %p\n", $mnt.0);
>             @dups[$mnt.0] = $mnt.1;
>         }
>     }
>     clear(@mnts);
> }
> 
> and observed that a single mount struct was reached multiple times -- perhaps
> unsurprisingly exactly the same number as there were duplicates detected by
> the above script.
> 
> Typical outputs of script (2.) and the bpftrace script above are
> 
> Starting monitoring at Tue Jan 27 20:48:13 UTC 2026
> Tue Jan 27 20:50:43 UTC 2026: Mount count (696) exceeds threshold (75)
> Duplicate mounts :
>   /proc/sys/fs/binfmt_misc: 2 occurrences
>   /tmp/tmpfs_856614_41491: 666 occurrences
> 
> and
> 
> @dups[0xffff88e5fb9f10a0]: 666

Thanks for the report. So it's a bit unfortunate that you're showing
duplication by source path. That's not as useful as that can
legitimately happen. So the better test would be to see whether you get
any duplicated unique mount ids, i.e., whether the same
mnt->mnt_id_unique appears multiple times. Because that's a bug for
sure.

I suspect the issue is real though. I'm appending a patch as a proposed
fix. Can you test that and report back, please? I'm traveling tomorrow
so might take a little.

[-- Attachment #2: 0001-namespace-fix-proc-mount-iteration.patch --]
[-- Type: text/x-diff, Size: 2099 bytes --]

From 26907be24e631329ba963d66196fe4f5d0863388 Mon Sep 17 00:00:00 2001
From: Christian Brauner <brauner@kernel.org>
Date: Thu, 29 Jan 2026 14:52:22 +0100
Subject: [PATCH] namespace: fix proc mount iteration

The m->index isn't updated when m->show() overflows and retains its
value before the current mount causing a restart to start at the same
value. If that happens in short order to due a quickly expanding mount
table this would cause the same mount to be shown again and again.

Ensure that *pos always equals the mount id of the mount that was
returned by start/next. On restart after overflow mnt_find_id_at(*pos)
finds the exact mount. This should avoid duplicates, avoid skips and
should handle concurrent modification just fine.

Cc: <stable@vger.kernel.org>
Fixed: 2eea9ce4310d8 ("mounts: keep list of mounts in an rbtree")
Signed-off-by: Christian Brauner <brauner@kernel.org>
---
 fs/namespace.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index c58674a20cad..ad35f8c961ef 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1537,23 +1537,33 @@ static struct mount *mnt_find_id_at_reverse(struct mnt_namespace *ns, u64 mnt_id
 static void *m_start(struct seq_file *m, loff_t *pos)
 {
 	struct proc_mounts *p = m->private;
+	struct mount *mnt;
 
 	down_read(&namespace_sem);
 
-	return mnt_find_id_at(p->ns, *pos);
+	mnt = mnt_find_id_at(p->ns, *pos);
+	if (mnt)
+		*pos = mnt->mnt_id_unique;
+	return mnt;
 }
 
 static void *m_next(struct seq_file *m, void *v, loff_t *pos)
 {
-	struct mount *next = NULL, *mnt = v;
+	struct mount *mnt = v;
 	struct rb_node *node = rb_next(&mnt->mnt_node);
 
-	++*pos;
 	if (node) {
-		next = node_to_mount(node);
+		struct mount *next = node_to_mount(node);
 		*pos = next->mnt_id_unique;
+		return next;
 	}
-	return next;
+
+	/*
+	 * No more mounts. Set pos past current mount's ID so that if
+	 * iteration restarts, mnt_find_id_at() returns NULL.
+	 */
+	*pos = mnt->mnt_id_unique + 1;
+	return NULL;
 }
 
 static void m_stop(struct seq_file *m, void *v)
-- 
2.47.3

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Zachary M. Raines]

On Thu Jan 29, 2026 at 8:28 AM CST, Christian Brauner wrote:
> On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
>> ...
>> 2. Reads `/prod/1/mountinfo` and checks for duplicates
>>
>> #!/bin/bash
>> THRESHOLD=75
>> echo "Starting monitoring at $(date)"
>> while true; do
>>     # Get mountinfo entries and count total
>>     mountinfo="$(cat /proc/1/mountinfo)"
>>     mountinfo_count=$(echo "$mountinfo" | wc -l)
>>
>>     if ((mountinfo_count > THRESHOLD)); then
>>         echo "$(date): Mount count ($mountinfo_count) exceeds threshold ($THRESHOLD)"
>>
>>         # Find and log duplicate mount points with their counts
>>         duplicates=$(echo "$mountinfo" | sort | uniq -cd)
>>
>>         if [[ -n "$duplicates" ]]; then
>>             echo "Duplicate mounts :"
>>             echo "$duplicates"
>>         fi
>>         echo "====="
>>         echo "$mountinfo"
>>         echo "---"
>>     fi
>>
>>     sleep 0.1
>> done
>> ...

> Thanks for the report. So it's a bit unfortunate that you're showing
> duplication by source path. That's not as useful as that can
> legitimately happen. So the better test would be to see whether you get
> any duplicated unique mount ids, i.e., whether the same
> mnt->mnt_id_unique appears multiple times. Because that's a bug for
> sure.

It turns out I pasted the output of an old version of my test script,
above, but if you look at the script itself, it checks for duplicates,
of the entire mountinfo line, i.e., duplicates have the same unique id.

> I suspect the issue is real though. I'm appending a patch as a proposed
> fix. Can you test that and report back, please? I'm traveling tomorrow
> so might take a little.

Thank you for the quick turnaround on that patch. I applied it on top
of 6.19-rc7 and after about 3 hrs I haven't seen any duplicates, in
contrast to without the patch where they appear in under 10 minutes.

Let me know if there's any other testing that would help.

Best,
Zach

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Zachary M. Raines]

On Thu Jan 29, 2026 at 1:04 PM CST, Zachary M. Raines wrote:
> On Thu Jan 29, 2026 at 8:28 AM CST, Christian Brauner wrote:
>> On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
>> I suspect the issue is real though. I'm appending a patch as a proposed
>> fix. Can you test that and report back, please? I'm traveling tomorrow
>> so might take a little.

Just following up on the patch you sent and thanks again.

> Thank you for the quick turnaround on that patch. I applied it on top
> of 6.19-rc7 and after about 3 hrs I haven't seen any duplicates, in
> contrast to without the patch where they appear in under 10 minutes.
>
> Let me know if there's any other testing that would help.

According to my testing, your patch resolves the issue. Do you plan to
submit it upstream? Please let me know if there's anything
I can do to help on that front.

Best,
Zach

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Christian Brauner]

On Wed, Feb 11, 2026 at 12:59:25PM -0600, Zachary M. Raines wrote:
> On Thu Jan 29, 2026 at 1:04 PM CST, Zachary M. Raines wrote:
> > On Thu Jan 29, 2026 at 8:28 AM CST, Christian Brauner wrote:
> >> On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
> >> I suspect the issue is real though. I'm appending a patch as a proposed
> >> fix. Can you test that and report back, please? I'm traveling tomorrow
> >> so might take a little.
> 
> Just following up on the patch you sent and thanks again.
> 
> > Thank you for the quick turnaround on that patch. I applied it on top
> > of 6.19-rc7 and after about 3 hrs I haven't seen any duplicates, in
> > contrast to without the patch where they appear in under 10 minutes.
> >
> > Let me know if there's any other testing that would help.
> 
> According to my testing, your patch resolves the issue. Do you plan to
> submit it upstream? Please let me know if there's anything
> I can do to help on that front.

It'll go upstream early next week. :) I deferred some pull requests.

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Jeff Layton]

On Sat, 2026-02-14 at 13:28 +0100, Christian Brauner wrote:
> On Wed, Feb 11, 2026 at 12:59:25PM -0600, Zachary M. Raines wrote:
> > On Thu Jan 29, 2026 at 1:04 PM CST, Zachary M. Raines wrote:
> > > On Thu Jan 29, 2026 at 8:28 AM CST, Christian Brauner wrote:
> > > > On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
> > > > I suspect the issue is real though. I'm appending a patch as a proposed
> > > > fix. Can you test that and report back, please? I'm traveling tomorrow
> > > > so might take a little.
> > 
> > Just following up on the patch you sent and thanks again.
> > 
> > > Thank you for the quick turnaround on that patch. I applied it on top
> > > of 6.19-rc7 and after about 3 hrs I haven't seen any duplicates, in
> > > contrast to without the patch where they appear in under 10 minutes.
> > > 
> > > Let me know if there's any other testing that would help.
> > 
> > According to my testing, your patch resolves the issue. Do you plan to
> > submit it upstream? Please let me know if there's anything
> > I can do to help on that front.
> 
> It'll go upstream early next week. :) I deferred some pull requests.

I noticed that this still hasn't gone to Linus and it's marked for
stable in your tree. Any reason you're waiting on it?
-- 
Jeff Layton <jlayton@kernel.org>

Re: PROBLEM: Duplicated entries in /proc/<pid>/mountinfo

[Christian Brauner]

On Wed, Feb 25, 2026 at 11:55:07AM -0500, Jeff Layton wrote:
> On Sat, 2026-02-14 at 13:28 +0100, Christian Brauner wrote:
> > On Wed, Feb 11, 2026 at 12:59:25PM -0600, Zachary M. Raines wrote:
> > > On Thu Jan 29, 2026 at 1:04 PM CST, Zachary M. Raines wrote:
> > > > On Thu Jan 29, 2026 at 8:28 AM CST, Christian Brauner wrote:
> > > > > On Wed, Jan 28, 2026 at 08:49:12AM -0600, Zachary M. Raines wrote:
> > > > > I suspect the issue is real though. I'm appending a patch as a proposed
> > > > > fix. Can you test that and report back, please? I'm traveling tomorrow
> > > > > so might take a little.
> > > 
> > > Just following up on the patch you sent and thanks again.
> > > 
> > > > Thank you for the quick turnaround on that patch. I applied it on top
> > > > of 6.19-rc7 and after about 3 hrs I haven't seen any duplicates, in
> > > > contrast to without the patch where they appear in under 10 minutes.
> > > > 
> > > > Let me know if there's any other testing that would help.
> > > 
> > > According to my testing, your patch resolves the issue. Do you plan to
> > > submit it upstream? Please let me know if there's anything
> > > I can do to help on that front.
> > 
> > It'll go upstream early next week. :) I deferred some pull requests.
> 
> I noticed that this still hasn't gone to Linus and it's marked for
> stable in your tree. Any reason you're waiting on it?

As mentioned off-line: we got our wires crossed and this went out as a
pull-request right before you sent the question here.
TL;DR it's upstream now.