* [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers?
@ 2026-11-19 9:00 ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
0 siblings, 2 replies; 6+ messages in thread
From: ping.li @ 2026-11-19 9:00 UTC (permalink / raw)
To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li
From: Ping Li <leeonion.muyu@gmail.com>
While investigating a throughput regression on one of our platforms
(long uptime, fragmented memory, large read()/write() on /dev/mtdX)
I noticed an inconsistency inside mtdchar:
- the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce
buffers with kvmalloc() (83208e106a8e8),
- but the read()/write() paths still use mtd_kmalloc_up_to(),
which halves the buffer size until kmalloc() succeeds.
The halving behavior is not a correctness problem: the loop in
mtdchar_read()/mtdchar_write() makes progress with whatever chunk
size it gets. But under fragmentation a multi-megabyte transfer
silently degrades into many small mtd_read()/mtd_write() calls,
which is visible when the low-level driver cost scales with the
number of transactions (e.g. drivers that forward each call to a
remote controller over a bus).
This RFC proposes to align the read()/write() paths with the ioctl
paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap.
The open question I would like to sort out before submitting a
proper patch:
How concerned should we be about low-level drivers receiving a
vmalloc'd buffer?
Looking at the tree, most drivers only memcpy() the buffer, and
the controllers that DMA directly into it (brcmnand, gpmi,
cadence, atmel, ...) already check is_vmalloc_addr()/
virt_addr_valid() and fall back to a bounce path. So vmalloc'd
buffers are supported everywhere today -- just slower on those
controllers. With mtd_kmalloc_up_to() the same controllers get a
kmalloc'd (smaller) buffer instead, so either way there is a
slowdown; it is mostly a question of which one userspace prefers:
few large transactions through a bounce path, or many small
transactions without one.
If the direction is agreed, I will follow up with a proper patch
and numbers from our platform.
Ping Li (1):
mtd: mtdchar: use kvmalloc() for large read/write buffers
drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers 2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li @ 2026-11-19 9:08 ` Ping Li 2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal 1 sibling, 0 replies; 6+ messages in thread From: Ping Li @ 2026-11-19 9:08 UTC (permalink / raw) To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, Ping Li mtdchar_read() and mtdchar_write() use mtd_kmalloc_up_to() for their bounce buffer. Under memory fragmentation this helper halves the requested size until kmalloc() succeeds, so a userspace read()/write() of a few megabytes may silently degrade into many small mtd_read()/ mtd_write() calls: more per-chunk overhead, and device drivers whose backend cost scales with the number of transactions pay a visible throughput penalty. The ioctl paths (MEMREAD/MEMWRITE) already use kvmalloc() for the same purpose (83208e106a8e8 "mtdchar: use kvmalloc() for potentially large allocations"). Use kvmalloc() in the read/write paths too: it tries kmalloc() first and falls back to vmalloc() when the buddy allocator cannot serve the request contiguously, so the buffer keeps its original size instead of shrinking. The size is still capped at KMALLOC_MAX_SIZE, as mtd_kmalloc_up_to() did. Most low-level MTD drivers only memcpy() the buffer, so vmalloc'd buffers are fine there; controllers that DMA directly into the buffer (brcmnand, gpmi, cadence, atmel, ...) check is_vmalloc_addr()/ virt_addr_valid() and fall back to a bounce path already. Signed-off-by: Ping Li <leeonion.muyu@gmail.com> --- drivers/mtd/mtdchar.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c index bf01e6ac7293..442fac076940 100644 --- a/drivers/mtd/mtdchar.c +++ b/drivers/mtd/mtdchar.c @@ -148,7 +148,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count, if (!count) return 0; - kbuf = mtd_kmalloc_up_to(mtd, &size); + /* + * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc() + * then falls back to vmalloc() when the fragmented buddy + * allocator cannot serve the request contiguously. + */ + size = min_t(size_t, count, KMALLOC_MAX_SIZE); + kbuf = kvmalloc(size, GFP_KERNEL); if (!kbuf) return -ENOMEM; @@ -192,7 +198,7 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count, if (!ret || mtd_is_bitflip_or_eccerr(ret)) { *ppos += retlen; if (copy_to_user(buf, kbuf, retlen)) { - kfree(kbuf); + kvfree(kbuf); return -EFAULT; } else @@ -204,13 +210,13 @@ static ssize_t mtdchar_read(struct file *file, char __user *buf, size_t count, count = 0; } else { - kfree(kbuf); + kvfree(kbuf); return ret; } } - kfree(kbuf); + kvfree(kbuf); return total_retlen; } /* mtdchar_read */ @@ -237,7 +243,13 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c if (!count) return 0; - kbuf = mtd_kmalloc_up_to(mtd, &size); + /* + * Cap the buffer like mtd_kmalloc_up_to() used to; kvmalloc() + * then falls back to vmalloc() when the fragmented buddy + * allocator cannot serve the request contiguously. + */ + size = min_t(size_t, count, KMALLOC_MAX_SIZE); + kbuf = kvmalloc(size, GFP_KERNEL); if (!kbuf) return -ENOMEM; @@ -245,7 +257,7 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c len = min_t(size_t, count, size); if (copy_from_user(kbuf, buf, len)) { - kfree(kbuf); + kvfree(kbuf); return -EFAULT; } @@ -292,12 +304,12 @@ static ssize_t mtdchar_write(struct file *file, const char __user *buf, size_t c buf += retlen; } else { - kfree(kbuf); + kvfree(kbuf); return ret; } } - kfree(kbuf); + kvfree(kbuf); return total_retlen; } /* mtdchar_write */ -- 2.34.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? 2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li 2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li @ 2026-11-19 9:50 ` Miquel Raynal 1 sibling, 0 replies; 6+ messages in thread From: Miquel Raynal @ 2026-11-19 9:50 UTC (permalink / raw) To: ping.li; +Cc: richard, vigneshr, linux-mtd, linux-kernel Hi Ping Li, On 19/11/2026 at 17:00:54 +08, "ping.li" <leeonion.muyu@gmail.com> wrote: > From: Ping Li <leeonion.muyu@gmail.com> > > While investigating a throughput regression on one of our platforms > (long uptime, fragmented memory, large read()/write() on /dev/mtdX) > I noticed an inconsistency inside mtdchar: > > - the ioctl paths (MEMREAD/MEMWRITE) allocate their bounce > buffers with kvmalloc() (83208e106a8e8), > - but the read()/write() paths still use mtd_kmalloc_up_to(), > which halves the buffer size until kmalloc() succeeds. > > The halving behavior is not a correctness problem: the loop in > mtdchar_read()/mtdchar_write() makes progress with whatever chunk > size it gets. But under fragmentation a multi-megabyte transfer > silently degrades into many small mtd_read()/mtd_write() calls, > which is visible when the low-level driver cost scales with the > number of transactions (e.g. drivers that forward each call to a > remote controller over a bus). > > This RFC proposes to align the read()/write() paths with the ioctl > paths by using kvmalloc(), keeping the KMALLOC_MAX_SIZE cap. > > The open question I would like to sort out before submitting a > proper patch: > > How concerned should we be about low-level drivers receiving a > vmalloc'd buffer? > > Looking at the tree, most drivers only memcpy() the buffer, and > the controllers that DMA directly into it (brcmnand, gpmi, > cadence, atmel, ...) already check is_vmalloc_addr()/ > virt_addr_valid() and fall back to a bounce path. So vmalloc'd > buffers are supported everywhere today -- just slower on those > controllers. With mtd_kmalloc_up_to() the same controllers get a > kmalloc'd (smaller) buffer instead, so either way there is a > slowdown; it is mostly a question of which one userspace prefers: > few large transactions through a bounce path, or many small > transactions without one. > > If the direction is agreed, I will follow up with a proper patch > and numbers from our platform. I would be curious to know how this gets faster and why? Because the preferred path is obviously via DMA to avoid keeping the CPU busy for such transfers, in which case you have no other choice than kmalloc'ing your buffer for the low-level driver. I am not generally opposed to the idea. I actually think it's best to have a unified allocation method if that is possible (without strong preference myself, but perhaps Richard will share a more educated opinion?). The main problem is how to handle the legacy cases. Typically Sashiko raised that some devices use __pa() over the buffer virtual address, which is absolutely horrible in 2027. Those will have to be fixed in your serie (as prerequisites). So be aware that the series is gonna be larger than what you expect. Ideally, I believe the low-level drivers should tell the MTD layer whether they expect continuous buffers > PAGE_SIZE or not. We could therefore have our own devm_mtd_kvmalloc(mtd, ...) which attaches the buffer to the MTD device, tries kzalloc() if requested by the mtd device (some kind of mtd->expects_dma_bufs boolean set during probe, typically based on the availability of DMA callbacks), fallbacks to vmalloc() otherwise. Thanks, Miquèl ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <sashiko-outbox-222838@kernel.org>]
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers [not found] <sashiko-outbox-222838@kernel.org> @ 2026-11-19 9:37 ` Ping Li 2026-11-19 9:38 ` Ping Li 1 sibling, 0 replies; 6+ messages in thread From: Ping Li @ 2026-11-19 9:37 UTC (permalink / raw) To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu From: leeonion.muyu@gmail.com Hi, Thanks for the review. I verified the three findings in the tree and all of them check out. 1. powernv_flash registers _read/_write directly and passes __pa(buf) down to OPAL (powernv_flash.c:69, :220), so a vmalloc'd buffer would hand firmware a bogus physical address. fsmc_nand.c:505, intel-nand-controller.c:312 and r852.c:196 dma_map_single() the buffer without any virt_addr_valid() check. This also explains why the 2022 ioctl conversion (83208e106a8e8) was safe: MEMREAD and MEMWRITE require _read_oob/_write_oob, which these drivers do not implement. 2. Agreed on the low-memory regression: kvmalloc() makes a single kmalloc() attempt and then goes straight to vmalloc(), so the size backoff that used to make progress under fragmentation is gone. The stale comment above mtdchar_read() is my oversight. 3. Agreed that "shrink first, vmalloc last" is the right shape; jumping to vmalloc while intermediate contiguous sizes are still available trades a cheap shrink for an expensive bounce path. So the answer to the cover letter's question is: quite concerned. Plain kvmalloc() in the _read/_write path is a no-go as long as drivers assume a linear-map buffer. The options I see from here: a) a small pre-cleanup series making the affected drivers vmalloc-safe (bounce or reject), then converting mtdchar_read()/mtdchar_write() with the size backoff kept and vmalloc only as a last resort; b) keeping the current halving-only behavior and dropping this RFC. I am happy to work on (a) if that direction is acceptable; otherwise I will withdraw the RFC. Thanks, Ping ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers [not found] <sashiko-outbox-222838@kernel.org> 2026-11-19 9:37 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li @ 2026-11-19 9:38 ` Ping Li 2026-11-19 9:39 ` Ping Li 1 sibling, 1 reply; 6+ messages in thread From: Ping Li @ 2026-11-19 9:38 UTC (permalink / raw) To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu From: leeonion.muyu@gmail.com Hi, Thanks for the review. I verified the three findings in the tree and all of them check out. 1. powernv_flash registers _read/_write directly and passes __pa(buf) down to OPAL (powernv_flash.c:69, :220), so a vmalloc'd buffer would hand firmware a bogus physical address. fsmc_nand.c:505, intel-nand-controller.c:312 and r852.c:196 dma_map_single() the buffer without any virt_addr_valid() check. This also explains why the 2022 ioctl conversion (83208e106a8e8) was safe: MEMREAD and MEMWRITE require _read_oob/_write_oob, which these drivers do not implement. 2. Agreed on the low-memory regression: kvmalloc() makes a single kmalloc() attempt and then goes straight to vmalloc(), so the size backoff that used to make progress under fragmentation is gone. The stale comment above mtdchar_read() is my oversight. 3. Agreed that "shrink first, vmalloc last" is the right shape; jumping to vmalloc while intermediate contiguous sizes are still available trades a cheap shrink for an expensive bounce path. So the answer to the cover letter's question is: quite concerned. Plain kvmalloc() in the _read/_write path is a no-go as long as drivers assume a linear-map buffer. The options I see from here: a) a small pre-cleanup series making the affected drivers vmalloc-safe (bounce or reject), then converting mtdchar_read()/mtdchar_write() with the size backoff kept and vmalloc only as a last resort; b) keeping the current halving-only behavior and dropping this RFC. I am happy to work on (a) if that direction is acceptable; otherwise I will withdraw the RFC. Thanks, Ping ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers 2026-11-19 9:38 ` Ping Li @ 2026-11-19 9:39 ` Ping Li 0 siblings, 0 replies; 6+ messages in thread From: Ping Li @ 2026-11-19 9:39 UTC (permalink / raw) To: miquel.raynal, richard, vigneshr; +Cc: linux-mtd, linux-kernel, leeonion.muyu From: leeonion.muyu@gmail.com Apologies for the duplicate -- that was a resend glitch on my side; the two messages are identical, please disregard one of them. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-19 9:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-19 9:00 [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? ping.li
2026-11-19 9:08 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:50 ` [RFC PATCH 0/1] mtdchar: kvmalloc() for large read/write buffers? Miquel Raynal
[not found] <sashiko-outbox-222838@kernel.org>
2026-11-19 9:37 ` [RFC PATCH 1/1] mtd: mtdchar: use kvmalloc() for large read/write buffers Ping Li
2026-11-19 9:38 ` Ping Li
2026-11-19 9:39 ` Ping Li
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®