[PATCH 0/2] mtd: ubi: fix direct reclaim deadlock with synchronous block devices

[PATCH 0/2] mtd: ubi: fix direct reclaim deadlock with synchronous block devices

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

Two-part fix for ubiblock swap deadlock: forbid direct reclaim I/O in the synchronous dispatch path and switch the ltree entry allocation to GFP_NOIO.

Ye Linfei (2):
  mtd: ubi: block: forbid direct reclaim I/O during synchronous read
  mtd: ubi: use GFP_NOIO for lock tree entry allocation

 drivers/mtd/ubi/block.c | 15 +++++++++++++++
 drivers/mtd/ubi/eba.c   |  2 +-
 2 files changed, 16 insertions(+), 1 deletion(-)

-- 
2.25.1

[PATCH 1/2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

ubiblock implements ->queue_rq() synchronously: the task submitting the
bio executes the whole UBI/MTD read itself, from inside the block layer
submission loop (current->bio_list is non-empty) and while holding the
tag_set SRCU read lock (blk_mq_run_dispatch_ops(), taken for
BLK_MQ_F_BLOCKING drivers).

An allocation in that context may trigger direct reclaim, which with
GFP_NOFS - used by all UBI allocations - may synchronously write dirty
pages to swap.  On systems where swap is backed by a device with
BLK_FEAT_SYNCHRONOUS (zram), that ends in swap_writepage_bdev_sync()
-> submit_bio_wait().  The swap bio is only queued onto
current->bio_list, never dispatched: per submit_bio_noacct_nocheck(),
bios submitted while a submission loop is active are only collected on
current->bio_list, and their sole dispatcher is the task currently
executing ->queue_rq() - which is now sleeping waiting for the very
same bio's completion.  The task hangs forever, the pending SRCU read
lock freezes grace periods (blk_mq_timeout_work() -> synchronize_srcu()
never returns), and the unfinished request blocks all other I/O to the
device.

This was observed in the field on a 1 GiB RISC-V networking product
running squashfs-on-ubiblock with zram swap: 47 tasks stuck in disk
sleep for 600+ seconds, ending in a hung task panic.  The allocator
never reached the OOM decision point, because the task died inside
try_to_free_pages().

Mark the whole read with memalloc_noio_save()/memalloc_noio_restore()
so direct reclaim never issues block I/O while this driver executes:
reclaim is restricted to dropping clean page cache and shrinking slabs,
page (swap) reclaim is performed asynchronously by kswapd, and the
worst case is -ENOMEM, which propagates to the caller as an ordinary
recoverable I/O error - a recoverable read failure instead of a total
system hang.

This covers every allocation performed below ubi_read_sg(), including
lock-tree entries, VID buffers and I/O helpers, regardless of the gfp
flags they use internally.

Fixes: 9d54c8a33eec ("UBI: R/O block driver on top of UBI volumes")
Cc: stable@vger.kernel.org
Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
---
 drivers/mtd/ubi/block.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
index 29c0d6941a81..c9d42181bf64 100644
--- a/drivers/mtd/ubi/block.c
+++ b/drivers/mtd/ubi/block.c
@@ -38,6 +38,7 @@
 #include <linux/blkdev.h>
 #include <linux/blk-mq.h>
 #include <linux/hdreg.h>
+#include <linux/sched/mm.h>
 #include <linux/scatterlist.h>
 #include <linux/idr.h>
 #include <asm/div64.h>
@@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
 	int leb = pos;
 	struct req_iterator iter;
 	struct bio_vec bvec;
+	unsigned int noio_flags;
 	int ret;
 
 	blk_mq_start_request(req);
@@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
 	ubi_sgl_init(&pdu->usgl);
 	blk_rq_map_sg(req, pdu->usgl.sg);
 
+	/*
+	 * This entire callback runs in the synchronous dispatch path of
+	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
+	 * from here onto current->bio_list, and the sole dispatcher of
+	 * that list is this very task, busy inside ->queue_rq().  Any
+	 * block I/O issued from below would therefore deadlock against
+	 * itself (e.g. synchronous swap writeback reached via direct
+	 * reclaim).  Forbid it for the whole read instead of relying on
+	 * every callee using safe allocation flags.
+	 */
+	noio_flags = memalloc_noio_save();
+
 	while (bytes_left) {
 		/*
 		 * We can only read one LEB at a time. Therefore if the read
@@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
 
 	rq_for_each_segment(bvec, req, iter)
 		flush_dcache_page(bvec.bv_page);
+	memalloc_noio_restore(noio_flags);
 
 	blk_mq_end_request(req, errno_to_blk_status(ret));
 
-- 
2.25.1

Re: [PATCH 1/2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[Zhihao Cheng]

在 2026/10/15 15:33, Yelin Fei 写道:
> From: Ye Linfei <yelinfei@ruijie.com.cn>
> 
> ubiblock implements ->queue_rq() synchronously: the task submitting the
> bio executes the whole UBI/MTD read itself, from inside the block layer
> submission loop (current->bio_list is non-empty) and while holding the
> tag_set SRCU read lock (blk_mq_run_dispatch_ops(), taken for
> BLK_MQ_F_BLOCKING drivers).
> 
> An allocation in that context may trigger direct reclaim, which with
> GFP_NOFS - used by all UBI allocations - may synchronously write dirty
> pages to swap.  On systems where swap is backed by a device with
> BLK_FEAT_SYNCHRONOUS (zram), that ends in swap_writepage_bdev_sync()
> -> submit_bio_wait().  The swap bio is only queued onto
> current->bio_list, never dispatched: per submit_bio_noacct_nocheck(),
> bios submitted while a submission loop is active are only collected on
> current->bio_list, and their sole dispatcher is the task currently
> executing ->queue_rq() - which is now sleeping waiting for the very
> same bio's completion.  The task hangs forever, the pending SRCU read
> lock freezes grace periods (blk_mq_timeout_work() -> synchronize_srcu()
> never returns), and the unfinished request blocks all other I/O to the
> device.
> 
> This was observed in the field on a 1 GiB RISC-V networking product
> running squashfs-on-ubiblock with zram swap: 47 tasks stuck in disk
> sleep for 600+ seconds, ending in a hung task panic.  The allocator
> never reached the OOM decision point, because the task died inside
> try_to_free_pages().
> 
> Mark the whole read with memalloc_noio_save()/memalloc_noio_restore()
> so direct reclaim never issues block I/O while this driver executes:
> reclaim is restricted to dropping clean page cache and shrinking slabs,
> page (swap) reclaim is performed asynchronously by kswapd, and the
> worst case is -ENOMEM, which propagates to the caller as an ordinary
> recoverable I/O error - a recoverable read failure instead of a total
> system hang.
> 
> This covers every allocation performed below ubi_read_sg(), including
> lock-tree entries, VID buffers and I/O helpers, regardless of the gfp
> flags they use internally.
> 
> Fixes: 9d54c8a33eec ("UBI: R/O block driver on top of UBI volumes")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
> ---
>   drivers/mtd/ubi/block.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)

Looks like other block drivers(eg. nvme/nbd/rbd..)  won't trigger direct 
mem reclaim by using flags 
GFP_ATOMIC(nvme_pci_setup_meta_iter)/GFP_NOIO(nbd 
sock_xmit)/GFP_NOWAIT(loop).

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
> 
> diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
> index 29c0d6941a81..c9d42181bf64 100644
> --- a/drivers/mtd/ubi/block.c
> +++ b/drivers/mtd/ubi/block.c
> @@ -38,6 +38,7 @@
>   #include <linux/blkdev.h>
>   #include <linux/blk-mq.h>
>   #include <linux/hdreg.h>
> +#include <linux/sched/mm.h>
>   #include <linux/scatterlist.h>
>   #include <linux/idr.h>
>   #include <asm/div64.h>
> @@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
>   	int leb = pos;
>   	struct req_iterator iter;
>   	struct bio_vec bvec;
> +	unsigned int noio_flags;
>   	int ret;
>   
>   	blk_mq_start_request(req);
> @@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
>   	ubi_sgl_init(&pdu->usgl);
>   	blk_rq_map_sg(req, pdu->usgl.sg);
>   
> +	/*
> +	 * This entire callback runs in the synchronous dispatch path of
> +	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
> +	 * from here onto current->bio_list, and the sole dispatcher of
> +	 * that list is this very task, busy inside ->queue_rq().  Any
> +	 * block I/O issued from below would therefore deadlock against
> +	 * itself (e.g. synchronous swap writeback reached via direct
> +	 * reclaim).  Forbid it for the whole read instead of relying on
> +	 * every callee using safe allocation flags.
> +	 */
> +	noio_flags = memalloc_noio_save();
> +
>   	while (bytes_left) {
>   		/*
>   		 * We can only read one LEB at a time. Therefore if the read
> @@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
>   
>   	rq_for_each_segment(bvec, req, iter)
>   		flush_dcache_page(bvec.bv_page);
> +	memalloc_noio_restore(noio_flags);
>   
>   	blk_mq_end_request(req, errno_to_blk_status(ret));
>   
> 

[PATCH 2/2] mtd: ubi: use GFP_NOIO for lock tree entry allocation

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

The per-LEB lock tree entry can be allocated while UBI is executing on
behalf of a synchronous block-layer consumer (ubiblock), where issuing
further block I/O from direct reclaim would deadlock; see the
preceding commit "mtd: ubi: block: forbid direct reclaim I/O during
synchronous read" for the full analysis.

Make the allocation self-contained: GFP_NOIO restricts direct reclaim
to dropping clean page cache and shrinking slabs, page (swap) reclaim
is left to kswapd, and the worst case is -ENOMEM, which callers
already propagate as an ordinary recoverable I/O error.

memalloc_noio_save()/memalloc_noio_restore() in ubiblock_read()
already enforce this for the whole read scope; keeping the allocation
here self-contained as well means the safety property does not depend
on every future caller remembering to set up that scope.

Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
---
 drivers/mtd/ubi/eba.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/mtd/ubi/eba.c b/drivers/mtd/ubi/eba.c
index f0c549e62c55..8fe68e7fff58 100644
--- a/drivers/mtd/ubi/eba.c
+++ b/drivers/mtd/ubi/eba.c
@@ -247,7 +247,7 @@ static struct ubi_ltree_entry *ltree_add_entry(struct ubi_device *ubi,
 {
 	struct ubi_ltree_entry *le, *le1, *le_free;
 
-	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOFS);
+	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOIO); /* see ubiblock_read() */
 	if (!le)
 		return ERR_PTR(-ENOMEM);
 
-- 
2.25.1

Re: [PATCH 2/2] mtd: ubi: use GFP_NOIO for lock tree entry allocation

[Zhihao Cheng]

在 2026/10/15 15:33, Yelin Fei 写道:
> From: Ye Linfei <yelinfei@ruijie.com.cn>
> 
> The per-LEB lock tree entry can be allocated while UBI is executing on
> behalf of a synchronous block-layer consumer (ubiblock), where issuing
> further block I/O from direct reclaim would deadlock; see the
> preceding commit "mtd: ubi: block: forbid direct reclaim I/O during
> synchronous read" for the full analysis.
> 
> Make the allocation self-contained: GFP_NOIO restricts direct reclaim
> to dropping clean page cache and shrinking slabs, page (swap) reclaim
> is left to kswapd, and the worst case is -ENOMEM, which callers
> already propagate as an ordinary recoverable I/O error.
> 
> memalloc_noio_save()/memalloc_noio_restore() in ubiblock_read()
> already enforce this for the whole read scope; keeping the allocation
> here self-contained as well means the safety property does not depend
> on every future caller remembering to set up that scope.

The ltree_add_entry() has many callers that not need GFP_NOIO, I think 
'mtd: ubi: block: forbid direct reclaim I/O during
 > synchronous read' is enough.
> 
> Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
> ---
>   drivers/mtd/ubi/eba.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/mtd/ubi/eba.c b/drivers/mtd/ubi/eba.c
> index f0c549e62c55..8fe68e7fff58 100644
> --- a/drivers/mtd/ubi/eba.c
> +++ b/drivers/mtd/ubi/eba.c
> @@ -247,7 +247,7 @@ static struct ubi_ltree_entry *ltree_add_entry(struct ubi_device *ubi,
>   {
>   	struct ubi_ltree_entry *le, *le1, *le_free;
>   
> -	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOFS);
> +	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOIO); /* see ubiblock_read() */
>   	if (!le)
>   		return ERR_PTR(-ENOMEM);
>   
> 

Re: [PATCH 2/2] mtd: ubi: use GFP_NOIO for lock tree entry allocation

[Ye Linfei]

On Thu, 2026-10-15 at 19:03 +0800, Zhihao Cheng wrote:
> The ltree_add_entry() has many callers that not need GFP_NOIO, I think
> 'mtd: ubi: block: forbid direct reclaim I/O during synchronous read'
> is enough.

Agreed, and thanks for catching this.

With memalloc_noio_save() in place, the ltree allocation inside
ubiblock_read() already drops __GFP_IO implicitly, so the explicit
GFP_NOIO there adds nothing.  The remaining ltree_add_entry() callers
(ubi_eba_write_leb() and friends from cdev/update/wear-leveling paths)
run in process or kthread context rather than a blk-mq synchronous
dispatch loop, where swap I/O triggered by direct reclaim can complete
normally; forcing GFP_NOIO on them is unnecessary.

I will drop this patch and send a v2 with the block driver change only.

Ye Linfei

[PATCH v2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[Ye Linfei]

ubiblock implements ->queue_rq() synchronously: the task submitting the
bio executes the whole UBI/MTD read itself, from inside the block layer
submission loop (current->bio_list is non-empty) and while holding the
tag_set SRCU read lock (blk_mq_run_dispatch_ops(), taken for
BLK_MQ_F_BLOCKING drivers).

An allocation in that context may trigger direct reclaim, which with
GFP_NOFS - used by all UBI allocations - may synchronously write dirty
pages to swap.  On systems where swap is backed by a device with
BLK_FEAT_SYNCHRONOUS (zram), that ends in swap_writepage_bdev_sync()
-> submit_bio_wait().  The swap bio is only queued onto
current->bio_list, never dispatched: per submit_bio_noacct_nocheck(),
bios submitted while a submission loop is active are only collected on
current->bio_list, and their sole dispatcher is the task currently
executing ->queue_rq() - which is now sleeping waiting for the very
same bio's completion.  The task hangs forever, the pending SRCU read
lock freezes grace periods (blk_mq_timeout_work() -> synchronize_srcu()
never returns), and the unfinished request blocks all other I/O to the
device.

This was observed in the field on a 1 GiB RISC-V networking product
running squashfs-on-ubiblock with zram swap: 47 tasks stuck in disk
sleep for 600+ seconds, ending in a hung task panic.  The allocator
never reached the OOM decision point, because the task died inside
try_to_free_pages().

Mark the whole read with memalloc_noio_save()/memalloc_noio_restore()
so direct reclaim never issues block I/O while this driver executes:
reclaim is restricted to dropping clean page cache and shrinking slabs,
page (swap) reclaim is performed asynchronously by kswapd, and the
worst case is -ENOMEM, which propagates to the caller as an ordinary
recoverable I/O error - a recoverable read failure instead of a total
system hang.

This covers every allocation performed below ubi_read_sg(), including
lock-tree entries, VID buffers and I/O helpers, regardless of the gfp
flags they use internally.

Fixes: 9d54c8a33eec ("UBI: R/O block driver on top of UBI volumes")
Cc: stable@vger.kernel.org
Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
---

Changes in v2:
- Drop the ltree GFP_NOIO patch: its allocation rides under
  ubiblock_read()'s memalloc_noio_save() scope, which already implies
  NOIO semantics, and the remaining ltree_add_entry() callers (cdev,
  update and wear-leveling paths) run in process context where swap
  I/O from direct reclaim can complete, so widening the flag change
  was unnecessary.  Suggested by Zhihao Cheng.
- Add Reviewed-by: Zhihao Cheng.

 drivers/mtd/ubi/block.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
index 29c0d6941a81..c9d42181bf64 100644
--- a/drivers/mtd/ubi/block.c
+++ b/drivers/mtd/ubi/block.c
@@ -38,6 +38,7 @@
 #include <linux/blkdev.h>
 #include <linux/blk-mq.h>
 #include <linux/hdreg.h>
+#include <linux/sched/mm.h>
 #include <linux/scatterlist.h>
 #include <linux/idr.h>
 #include <asm/div64.h>
@@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
 	int leb = pos;
 	struct req_iterator iter;
 	struct bio_vec bvec;
+	unsigned int noio_flags;
 	int ret;
 
 	blk_mq_start_request(req);
@@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
 	ubi_sgl_init(&pdu->usgl);
 	blk_rq_map_sg(req, pdu->usgl.sg);
 
+	/*
+	 * This entire callback runs in the synchronous dispatch path of
+	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
+	 * from here onto current->bio_list, and the sole dispatcher of
+	 * that list is this very task, busy inside ->queue_rq().  Any
+	 * block I/O issued from below would therefore deadlock against
+	 * itself (e.g. synchronous swap writeback reached via direct
+	 * reclaim).  Forbid it for the whole read instead of relying on
+	 * every callee using safe allocation flags.
+	 */
+	noio_flags = memalloc_noio_save();
+
 	while (bytes_left) {
 		/*
 		 * We can only read one LEB at a time. Therefore if the read
@@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
 
 	rq_for_each_segment(bvec, req, iter)
 		flush_dcache_page(bvec.bv_page);
+	memalloc_noio_restore(noio_flags);
 
 	blk_mq_end_request(req, errno_to_blk_status(ret));
 
-- 
2.25.1