* [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers?
@ 2026-11-19 9:00 ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
0 siblings, 2 replies; 6+ messages in thread
From: ping.li @ 2026-11-19 9:00 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li
From: Ping Li <leeonion.muyu@gmail.com>
While investigating a throughput regression on one of our platforms
(long uptime, fragmented memory, large read()/write() on /dev/mtdX)
I noticed an inconsistency inside mtdchar:
- the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce
buffers with kvmalloc() (83208e106a8e8),
- but the read()/write() paths still use mtd_kmalloc_up_to(),
which halves the buffer size until kmalloc() succeeds.
The halving behavior is not a correctness problem: the loop in
mtdchar_read()/mtdchar_write() makes progress with whatever chunk
size it gets. But under fragmentation a multi-megabyte transfer
silently degrades into many small mtd_read()/mtd_write() calls,
which is visible when the low-level driver cost scales with the
number of transactions (e.g. drivers that forward each call to a
remote controller over a bus).
This RFC proposes to align the read()/write() paths with the ioctl
paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap.
The open question I would like to sort out before submitting a
proper patch:
How concerned should we be about low-level drivers receiving a
vmalloc'd buffer?
Looking at the tree, most drivers only memcpy() the buffer, and
the controllers that DMA directly into it (brcmnand, gpmi,
cadence, atmel, ...) already check is_vmalloc_addr()/
virt_addr_valid() and fall back to a bounce path. So vmalloc'd
buffers are supported everywhere today -- just slower on those
controllers. With mtd_kmalloc_up_to() the same controllers get a
kmalloc'd (smaller) buffer instead, so either way there is a
slowdown; it is mostly a question of which one userspace prefers:
few large transactions through a bounce path, or many small
transactions without one.
If the direction is agreed, I will follow up with a proper patch
and numbers from our platform.
Ping Li (1):
mtd: mtdchar: use kvmalloc() for large read/write buffers
drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
@ 2026-11-19 9:08 ` Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
1 sibling, 0 replies; 6+ messages in thread
From: Ping Li @ 2026-11-19 9:08 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li
mtdchar_read() and mtdchar_write() use mtd_kmalloc_up_to() for their
bounce buffer. Under memory fragmentation this helper halves the
requested size until kmalloc() succeeds, so a userspace read()/write()
of a few megabytes may silently degrade into many small mtd_read()/
mtd_write() calls: more per-chunk overhead, and device drivers whose
backend cost scales with the number of transactions pay a visible
throughput penalty.
The ioctl paths (MEMREAD/MEMWRITE) already use kvmalloc() for the same
purpose (83208e106a8e8 "mtdchar: use kvmalloc() for potentially large
allocations"). Use kvmalloc() in the read/write paths too: it tries
kmalloc() first and falls back to vmalloc() when the buddy allocator
cannot serve the request contiguously, so the buffer keeps its original
size instead of shrinking.
The size is still capped at KMALLOC_MAX_SIZE, as mtd_kmalloc_up_to()
did.
Most low-level MTD drivers only memcpy() the buffer, so vmalloc'd
buffers are fine there; controllers that DMA directly into the buffer
(brcmnand, gpmi, cadence, atmel, ...) check is_vmalloc_addr()/
virt_addr_valid() and fall back to a bounce path already.
Signed-off-by: Ping Li <leeonion.muyu@gmail.com>
---
drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index bf01e6ac7293..442fac076940 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -148,7 +148,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
if (!count)
return 0;
- kbuf = mtd_kmalloc_up_to(mtd, &size);
+ /*
+ * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc()
+ * then falls back to vmalloc() when the fragmented buddy
+ * allocator cannot serve the request contiguously.
+ */
+ size = min_t(size_t, count, KMALLOC_MAX_SIZE);
+ kbuf = kvmalloc(size, GFP_KERNEL);
if (!kbuf)
return -ENOMEM;
@@ -192,7 +198,7 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
if (!ret || mtd_is_bitflip_or_eccerr(ret)) {
*ppos += retlen;
if (copy_to_user(buf, kbuf, retlen)) {
- kfree(kbuf);
+ kvfree(kbuf);
return -EFAULT;
}
else
@@ -204,13 +210,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count,
count = 0;
}
else {
- kfree(kbuf);
+ kvfree(kbuf);
return ret;
}
}
- kfree(kbuf);
+ kvfree(kbuf);
return total_retlen;
} /* mtdchar_read */
@@ -237,7 +243,13 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
if (!count)
return 0;
- kbuf = mtd_kmalloc_up_to(mtd, &size);
+ /*
+ * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc()
+ * then falls back to vmalloc() when the fragmented buddy
+ * allocator cannot serve the request contiguously.
+ */
+ size = min_t(size_t, count, KMALLOC_MAX_SIZE);
+ kbuf = kvmalloc(size, GFP_KERNEL);
if (!kbuf)
return -ENOMEM;
@@ -245,7 +257,7 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
len = min_t(size_t, count, size);
if (copy_from_user(kbuf, buf, len)) {
- kfree(kbuf);
+ kvfree(kbuf);
return -EFAULT;
}
@@ -292,12 +304,12 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c
buf += retlen;
}
else {
- kfree(kbuf);
+ kvfree(kbuf);
return ret;
}
}
- kfree(kbuf);
+ kvfree(kbuf);
return total_retlen;
} /* mtdchar_write */
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers?
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
@ 2026-11-19 9:50 ` Miquel Raynal
1 sibling, 0 replies; 6+ messages in thread
From: Miquel Raynal @ 2026-11-19 9:50 UTC (permalink / raw)
To: ping.li; +Cc: richard, vigneshr, linux-mtd, linux-kernel
Hi Ping Li,
On 19/11/2026 at 17:00:54 +08, "ping.li" <leeonion.muyu@gmail.com> wrote:
> From: Ping Li <leeonion.muyu@gmail.com>
>
> While investigating a throughput regression on one of our platforms
> (long uptime, fragmented memory, large read()/write() on /dev/mtdX)
> I noticed an inconsistency inside mtdchar:
>
> - the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce
> buffers with kvmalloc() (83208e106a8e8),
> - but the read()/write() paths still use mtd_kmalloc_up_to(),
> which halves the buffer size until kmalloc() succeeds.
>
> The halving behavior is not a correctness problem: the loop in
> mtdchar_read()/mtdchar_write() makes progress with whatever chunk
> size it gets. But under fragmentation a multi-megabyte transfer
> silently degrades into many small mtd_read()/mtd_write() calls,
> which is visible when the low-level driver cost scales with the
> number of transactions (e.g. drivers that forward each call to a
> remote controller over a bus).
>
> This RFC proposes to align the read()/write() paths with the ioctl
> paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap.
>
> The open question I would like to sort out before submitting a
> proper patch:
>
> How concerned should we be about low-level drivers receiving a
> vmalloc'd buffer?
>
> Looking at the tree, most drivers only memcpy() the buffer, and
> the controllers that DMA directly into it (brcmnand, gpmi,
> cadence, atmel, ...) already check is_vmalloc_addr()/
> virt_addr_valid() and fall back to a bounce path. So vmalloc'd
> buffers are supported everywhere today -- just slower on those
> controllers. With mtd_kmalloc_up_to() the same controllers get a
> kmalloc'd (smaller) buffer instead, so either way there is a
> slowdown; it is mostly a question of which one userspace prefers:
> few large transactions through a bounce path, or many small
> transactions without one.
>
> If the direction is agreed, I will follow up with a proper patch
> and numbers from our platform.
I would be curious to know how this gets faster and why? Because the
preferred path is obviously via DMA to avoid keeping the CPU busy for
such transfers, in which case you have no other choice than kmalloc'ing
your buffer for the low-level driver.
I am not generally opposed to the idea. I actually think it's best to
have a unified allocation method if that is possible (without strong
preference myself, but perhaps Richard will share a more educated
opinion?). The main problem is how to handle the legacy cases. Typically
Sashiko raised that some devices use __pa() over the buffer virtual
address, which is absolutely horrible in 2027. Those will have to be
fixed in your serie (as prerequisites). So be aware that the series is
gonna be larger than what you expect.
Ideally, I believe the low-level drivers should tell the MTD layer
whether they expect continuous buffers > PAGE_SIZE or not. We could
therefore have our own devm_mtd_kvmalloc(mtd, ...) which attaches the
buffer to the MTD device, tries kzalloc() if requested by the mtd device
(some kind of mtd->expects_dma_bufs boolean set during probe, typically
based on the availability of DMA callbacks), fallbacks to vmalloc()
otherwise.
Thanks,
Miquèl
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers
2026-11-19 9:38 ` Ping Li
@ 2026-11-19 9:39 ` Ping Li
0 siblings, 0 replies; 6+ messages in thread
From: Ping Li @ 2026-11-19 9:39 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu
From: leeonion.muyu@gmail.com
Apologies for the duplicate -- that was a resend glitch on my side;
the two messages are identical, please disregard one of them.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers
[not found] <sashiko-outbox-222838@kernel.org>
2026-11-19 9:37 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
@ 2026-11-19 9:38 ` Ping Li
2026-11-19 9:39 ` Ping Li
1 sibling, 1 reply; 6+ messages in thread
From: Ping Li @ 2026-11-19 9:38 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu
From: leeonion.muyu@gmail.com
Hi,
Thanks for the review. I verified the three findings in the tree and
all of them check out.
1. powernv_flash registers _read/_write directly and passes __pa(buf)
down to OPAL (powernv_flash.c:69, :220), so a vmalloc'd buffer
would hand firmware a bogus physical address. fsmc_nand.c:505,
intel-nand-controller.c:312 and r852.c:196 dma_map_single() the
buffer without any virt_addr_valid() check. This also explains
why the 2022 ioctl conversion (83208e106a8e8) was safe: MEMREAD
and MEMWRITE require _read_oob/_write_oob, which these drivers do
not implement.
2. Agreed on the low-memory regression: kvmalloc() makes a single
kmalloc() attempt and then goes straight to vmalloc(), so the
size backoff that used to make progress under fragmentation is
gone. The stale comment above mtdchar_read() is my oversight.
3. Agreed that "shrink first, vmalloc last" is the right shape;
jumping to vmalloc while intermediate contiguous sizes are still
available trades a cheap shrink for an expensive bounce path.
So the answer to the cover letter's question is: quite concerned.
Plain kvmalloc() in the _read/_write path is a no-go as long as
drivers assume a linear-map buffer.
The options I see from here:
a) a small pre-cleanup series making the affected drivers
vmalloc-safe (bounce or reject), then converting
mtdchar_read()/mtdchar_write() with the size backoff kept and
vmalloc only as a last resort;
b) keeping the current halving-only behavior and dropping this
RFC.
I am happy to work on (a) if that direction is acceptable;
otherwise I will withdraw the RFC.
Thanks,
Ping
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers
[not found] <sashiko-outbox-222838@kernel.org>
@ 2026-11-19 9:37 ` Ping Li
2026-11-19 9:38 ` Ping Li
1 sibling, 0 replies; 6+ messages in thread
From: Ping Li @ 2026-11-19 9:37 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu
From: leeonion.muyu@gmail.com
Hi,
Thanks for the review. I verified the three findings in the tree and
all of them check out.
1. powernv_flash registers _read/_write directly and passes __pa(buf)
down to OPAL (powernv_flash.c:69, :220), so a vmalloc'd buffer
would hand firmware a bogus physical address. fsmc_nand.c:505,
intel-nand-controller.c:312 and r852.c:196 dma_map_single() the
buffer without any virt_addr_valid() check. This also explains
why the 2022 ioctl conversion (83208e106a8e8) was safe: MEMREAD
and MEMWRITE require _read_oob/_write_oob, which these drivers do
not implement.
2. Agreed on the low-memory regression: kvmalloc() makes a single
kmalloc() attempt and then goes straight to vmalloc(), so the
size backoff that used to make progress under fragmentation is
gone. The stale comment above mtdchar_read() is my oversight.
3. Agreed that "shrink first, vmalloc last" is the right shape;
jumping to vmalloc while intermediate contiguous sizes are still
available trades a cheap shrink for an expensive bounce path.
So the answer to the cover letter's question is: quite concerned.
Plain kvmalloc() in the _read/_write path is a no-go as long as
drivers assume a linear-map buffer.
The options I see from here:
a) a small pre-cleanup series making the affected drivers
vmalloc-safe (bounce or reject), then converting
mtdchar_read()/mtdchar_write() with the size backoff kept and
vmalloc only as a last resort;
b) keeping the current halving-only behavior and dropping this
RFC.
I am happy to work on (a) if that direction is acceptable;
otherwise I will withdraw the RFC.
Thanks,
Ping
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-19 9:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
[not found] <sashiko-outbox-222838@kernel.org>
2026-11-19 9:37 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:38 ` Ping Li
2026-11-19 9:39 ` Ping Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®