From: Yuanhe Shu <xiangzao@linux.alibaba.com>
To: Kiryl Shutsemau <kas@kernel.org>, Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Joerg Roedel <joro@8bytes.org>,
Nicolin Chen <nicolinc@nvidia.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>,
Pranjal Shrivastava <praan@google.com>,
Mostafa Saleh <smostafa@google.com>,
Thierry Reding <thierry.reding@kernel.org>,
Krishna Reddy <vdumpa@nvidia.com>,
Jonathan Hunter <jonathanh@nvidia.com>,
Breno Leitao <leitao@debian.org>, Kyle McMartin <jkkm@meta.com>,
Usama Arif <usama.arif@linux.dev>,
kernel-team@meta.com, linux-arm-kernel@lists.infradead.org,
iommu@lists.linux.dev, linux-tegra@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v6 2/2] iommu/arm-smmu-v3: Default queue depths to one page in a kdump kernel
Date: Tue, 15 Sep 2026 19:20:36 +0800 [thread overview]
Message-ID: <20260915112037.1495272-1-xiangzao@linux.alibaba.com> (raw)
In-Reply-To: <20260909095228.2174031-3-kas@kernel.org>
Tested-by: Yuanhe Shu <xiangzao@linux.alibaba.com>
We hit the same problem on our arm64 platform: the kdump capture kernel
runs out of memory because the SMMUv3 queues are still allocated at full
hardware size. We had started looking into it ourselves, then came across
this series, so we tested it on the affected hardware rather than prepare
a duplicate fix.
It resolves the problem for us - details below.
Test platform: an arm64 server, 128 cores, 2 NUMA nodes, 512 GiB RAM,
exposing 6 SMMUv3 instances. 64K-page kernel (PAGE_SHIFT=16) based on
Linux 7.0.14 with the series applied on top; crashkernel reservation is
512 MiB.
On this platform kdump FAILS without the patch, and the series fixes it.
Before - capture kernel WITHOUT the series:
All 6 SMMUv3 instances still allocate the hardware-maximum queues:
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for cmdq
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for evtq
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for priq
(... same for all 6 instances)
That is 8 + 16 + 8 = 32 MiB per instance, 192 MiB total - 37.5% of the
512 MiB reservation. The capture kernel OOMs before makedumpfile runs:
swapper/0 invoked oom-killer: gfp_mask=0x2040cc0(GFP_KERNEL|...), order=0
Out of memory and no killable processes...
Rebooting in 10 seconds..
=> no vmcore is saved.
After - capture kernel WITH the series:
All three queues are floored to one page per instance:
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 4096 entries for cmdq
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 2048 entries for evtq
arm-smmu-v3 arm-smmu-v3.0.auto: allocated 4096 entries for priq
(... same for all 6 instances)
=> ~1.1 MiB total (~191 MiB freed). All 6 instances re-probe cleanly
(no -ENXIO, no allocation failure) and a 608 MB vmcore is saved.
Same machine, same 512 MiB reservation; the only difference is the v6
series, so this is a controlled before/after.
Thanks,
Yuanhe
prev parent reply other threads:[~2026-09-15 11:20 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-09 9:52 [PATCH v6 0/2] iommu/arm-smmu-v3: Make the queue depths tunable, and shrink them " Kiryl Shutsemau (Meta)
2026-09-09 9:52 ` [PATCH v6 1/2] iommu/arm-smmu-v3: Add a cmdq_max_n_shift module parameter Kiryl Shutsemau (Meta)
2026-09-15 12:16 ` Breno Leitao
2026-09-09 9:52 ` [PATCH v6 2/2] iommu/arm-smmu-v3: Default queue depths to one page in a kdump kernel Kiryl Shutsemau (Meta)
2026-09-15 11:20 ` Yuanhe Shu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260915112037.1495272-1-xiangzao@linux.alibaba.com \
--to=xiangzao@linux.alibaba.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=jkkm@meta.com \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=kas@kernel.org \
--cc=kernel-team@meta.com \
--cc=leitao@debian.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=nicolinc@nvidia.com \
--cc=praan@google.com \
--cc=robin.murphy@arm.com \
--cc=smostafa@google.com \
--cc=thierry.reding@kernel.org \
--cc=usama.arif@linux.dev \
--cc=vdumpa@nvidia.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®