mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* Re: task hung in xlog_grant_head_check - still reproducible on 7.2-rc7 (2018 report went dormant) - and on 5.15.216-stable, with a cascading inodegc victim profile
@ 2026-10-26 18:19 Ivan
  2026-10-26 20:34 ` Dave Chinner
  0 siblings, 1 reply; 2+ messages in thread
From: Ivan @ 2026-10-26 18:19 UTC (permalink / raw)
  To: linux-xfs; +Cc: djwong, yangguikun, syzkaller, linux-kernel

Hi,

Picking up Yang Guikun's August report [1] with independent evidence from
our fuzzing fleet -- we agree with the analysis and would like to help get
this unstuck, because the history of this bug now looks like the pre-history
of CVE-2026-109618 (PHP2Shell): a known issue sitting on hard trade-offs
until it becomes an emergency.

INDEPENDENT CONFIRMATION (5.15.216, KASAN, syzkaller):

Our focused XFS campaign reproduces the same hang with a different image
generation and an extra victim profile:

  1. syz_mount_image$xfs -- crafted V4 image (small log)
  2. one setxattr(...) with a large value

  -> xfs_attr_set -> xfs_trans_alloc_inode -> xfs_log_reserve
     -> xlog_grant_head_check -> xlog_grant_head_wait (xfs_log.c:256)

Unlike the August report (V5 image, single victim), our logs show a CASCADE:
the first task wedged is the xfs-inodegc kworker in xfs_inactive_truncate
(D-state >122s), after which user tasks pile up in xfs_trans_alloc behind
the grant wait. All victims are unkillable. So the blast radius is the
whole filesystem, not one unlucky task.

Same root cause as described in [1]: a transaction reservation larger than
the usable log size can never be granted, so xlog_grant_head_wait() is an
unbounded wait -- no amount of log tail pushing can satisfy it.

THE RECORD (why we are pushing now):

- syzbot's main tracker: first report 2018-05-21, open 3079 days,
  1471 crashes, 0 of 29 patches merged, last crash within the last day.
- Sister trackers all open with C repros: linux-4.14 (2450 days),
  linux-4.19 (1412 days), linux-5.15 (1311 days), linux-6.1 (1302 days,
  394 crashes), linux-6.6 (Dec 2025, corrupt-fs images).
- August's report included a concrete, low-risk fix direction (sanity-check
  the reservation against usable log size -> clean -ENOSPC instead of an
  unkillable hang). Neighbor fixes over the years (integer overflow in
  xlog_grant_head_check, the nowait case, ioend log exhaustion) never
closed the unbounded-wait itself.

WHAT WE ARE ASKING FOR:

1. Either accept a reservation-size sanity check (we are happy to test
   patches against our fleet and provide before/after traces), or
2. record an explicit decision (even "won't fix, requires crafted image")
   so that the dormancy becomes a documented choice rather than silence.

We would rather spend our cycles testing your patch than re-reporting this
for the 30th time in 2028.

OFFER:
- Full syzkaller reproducer package (image + prog + C + logs) from both
  campaigns; can also provide a hand-minimized C repro on request.
- Bisect/testing assistance on 5.15-stable and mainline.

[1] https://lists.openwall.net/linux-kernel/2026/08/24/265
    ("task hung in xlog_grant_head_check - still reproducible on 7.2-rc7")

Reported-by: Ivan Vinichuk <ivan.vin.ctf@gmail.com>

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: task hung in xlog_grant_head_check - still reproducible on 7.2-rc7 (2018 report went dormant) - and on 5.15.216-stable, with a cascading inodegc victim profile
  2026-10-26 18:19 task hung in xlog_grant_head_check - still reproducible on 7.2-rc7 (2018 report went dormant) - and on 5.15.216-stable, with a cascading inodegc victim profile Ivan
@ 2026-10-26 20:34 ` Dave Chinner
  0 siblings, 0 replies; 2+ messages in thread
From: Dave Chinner @ 2026-10-26 20:34 UTC (permalink / raw)
  To: Ivan; +Cc: linux-xfs, djwong, yangguikun, syzkaller, linux-kernel

On Mon, Oct 26, 2026 at 11:19:38AM -0700, Ivan wrote:
> Hi,
> 
> Picking up Yang Guikun's August report [1] with independent evidence from
> our fuzzing fleet -- we agree with the analysis and would like to help get
> this unstuck, because the history of this bug now looks like the pre-history
> of CVE-2026-109618 (PHP2Shell): a known issue sitting on hard trade-offs
> until it becomes an emergency.
> 
> INDEPENDENT CONFIRMATION (5.15.216, KASAN, syzkaller):
> 
> Our focused XFS campaign reproduces the same hang with a different image
> generation and an extra victim profile:
> 
>   1. syz_mount_image$xfs -- crafted V4 image (small log)

"Doctor, it hurts when I punch myself in the face!"

> Same root cause as described in [1]: a transaction reservation larger than
> the usable log size can never be granted, so xlog_grant_head_wait() is an
> unbounded wait -- no amount of log tail pushing can satisfy it.

The root cause is that the journal was intentionally corrupted to
make an empty journal appear full. The normal mechanisms to free
journal space for transaction reservations or return space after
recovery don't work because -there is nothing to free-. V5
filesystems detect this specific issues and error out; v4
filesystems only warn.

IOWs, this hang has nothing to do with transaction reservation or
log sizes, and everything to do with passing the filesystem a
carefully crafted image with corrupted index values that will never
actually happen in the real world....

> THE RECORD (why we are pushing now):
> 
> - syzbot's main tracker: first report 2018-05-21, open 3079 days,
>   1471 crashes, 0 of 29 patches merged, last crash within the last day.
> - Sister trackers all open with C repros: linux-4.14 (2450 days),
>   linux-4.19 (1412 days), linux-5.15 (1311 days), linux-6.1 (1302 days,
>   394 crashes), linux-6.6 (Dec 2025, corrupt-fs images).
> - August's report included a concrete, low-risk fix direction (sanity-check
>   the reservation against usable log size -> clean -ENOSPC instead of an
>   unkillable hang). Neighbor fixes over the years (integer overflow in
>   xlog_grant_head_check, the nowait case, ioend log exhaustion) never
> closed the unbounded-wait itself.
> 
> WHAT WE ARE ASKING FOR:
> 
> 1. Either accept a reservation-size sanity check (we are happy to test
>    patches against our fleet and provide before/after traces), or
> 2. record an explicit decision (even "won't fix, requires crafted image")
>    so that the dormancy becomes a documented choice rather than silence.

Deeds speak louder than words. If you want it fixed, fix it and
propose the fix for inclusion.

As I said in 2018 about this specific v4 fuzzer "issue":

| And this comes back to my repeated comments about fuzzers needing
| to fuzz properly made V5 filesystems as we catch and error out on
| things like this. Fuzzing random collections of v4 filesystem
| fragments will continue to trip over problems we've avoided with v5
| filesystems, and this is further evidence to point to that.
| 
| I'd suggest that at this point, syzbot XFS reports should be
| redirected to /dev/null. It's not worth our time to triage
| unreviewed bot generated bug reports until the syzbot developers
| start listening and acting on what we have been telling them
| about fuzzing filesystems and reproducing bugs that are meaningful
| and useful to us.

It should be pretty obvious that in 2018 we weren't fixing this sort
of stuff in v4 filesystems because the default v5 format catches
them and errors out correctly.

Now it's 2026, v4 format is officially deprecated, we don't build v4
support by default, and V4 support is officially going to be removed
in 2030. We're not going to fix this.

> We would rather spend our cycles testing your patch than re-reporting this
> for the 30th time in 2028.

I welcome your 2028 report. You'll get the same answer.

-Dave.
-- 
Dave Chinner
dgc@kernel.org

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-10-26 20:34 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-26 18:19 task hung in xlog_grant_head_check - still reproducible on 7.2-rc7 (2018 report went dormant) - and on 5.15.216-stable, with a cascading inodegc victim profile Ivan
2026-10-26 20:34 ` Dave Chinner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®