mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: K Prateek Nayak <kprateek.nayak@amd.com>
To: Oleg Nesterov <oleg@redhat.com>, Mateusz Guzik <mjguzik@gmail.com>
Cc: syzbot <syzbot+62262fdc0e01d99573fc@syzkaller.appspotmail.com>,
	<brauner@kernel.org>, <dhowells@redhat.com>, <jack@suse.cz>,
	<jlayton@kernel.org>, <linux-fsdevel@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <netfs@lists.linux.dev>,
	<swapnil.sapkal@amd.com>, <syzkaller-bugs@googlegroups.com>,
	<viro@zeniv.linux.org.uk>
Subject: Re: [syzbot] [netfs?] INFO: task hung in netfs_unbuffered_write_iter
Date: Mon, 24 Mar 2025 20:22:40 +0530	[thread overview]
Message-ID: <f7585a27-aaef-4334-a1de-5e081f10c901@amd.com> (raw)
In-Reply-To: <7e377feb-a78b-4055-88cc-2c20f924bf82@amd.com>

Hello folks,

Some updates.

On 3/24/2025 6:49 PM, K Prateek Nayak wrote:
>>
>> Per syzbot this attempt did not work out either.
>>
>> I think the blind stabs taken by everyone here are enough.
>>
>> The report does not provide the crucial bit: what are the other
>> threads doing. Presumably someone else is stuck somewhere, possibly
>> not even in pipe code and that stuck thread was supposed to wake up
>> the one which trips over hung task detector. Figuring out what that
>> thread is imo the next step.
>>
>> I failed to find a relevant command in
>> https://github.com/google/syzkaller/blob/master/docs/syzbot.md
>>
>> So if you guys know someone on syzkaller side, maybe you can ask them
>> to tweak the report *or* maybe syzbot can test a "fix" which makes
>> hung task detector also report all backtraces? I don't know if that
>> can work, the output may be long enough that it will get trimmed by
>> something.
>>
>> I don't have to time work on this for now, just throwing ideas.
> 
> I got the reproducer running locally. Tracing stuff currently to see
> what is tripping. Will report back once I find something interesting.
> Might take a while since the 9p bits are so far spread out.
> 

So far, with tracing, this is where I'm:

o Mainline + Oleg's optimization reverted:

     ...
     kworker/43:1-1723    [043] .....   115.309065: p9_read_work: Data read wait 55
     kworker/43:1-1723    [043] .....   115.309066: p9_read_work: Data read 55
     kworker/43:1-1723    [043] .....   115.309067: p9_read_work: Data read wait 7
     kworker/43:1-1723    [043] .....   115.309068: p9_read_work: Data read 7
            repro-4138    [043] .....   115.309084: netfs_wake_write_collector: Wake collector
            repro-4138    [043] .....   115.309085: netfs_wake_write_collector: Queuing collector work
            repro-4138    [043] .....   115.309088: netfs_unbuffered_write: netfs_unbuffered_write
            repro-4138    [043] .....   115.309088: netfs_end_issue_write: netfs_end_issue_write
            repro-4138    [043] .....   115.309089: netfs_end_issue_write: Write collector need poke 0
            repro-4138    [043] .....   115.309091: netfs_unbuffered_write_iter_locked: Waiting on NETFS_RREQ_IN_PROGRESS!
  kworker/u1030:1-1951    [168] .....   115.309096: netfs_wake_write_collector: Wake collector
  kworker/u1030:1-1951    [168] .....   115.309097: netfs_wake_write_collector: Queuing collector work
  kworker/u1030:1-1951    [168] .....   115.309102: netfs_write_collection_worker: Write collect clearing and waking up!
     ... (syzbot reproducer continues)

o Mainline:

    kworker/185:1-1767    [185] .....   109.485961: p9_read_work: Data read wait 7
    kworker/185:1-1767    [185] .....   109.485962: p9_read_work: Data read 7
    kworker/185:1-1767    [185] .....   109.485962: p9_read_work: Data read wait 55
    kworker/185:1-1767    [185] .....   109.485963: p9_read_work: Data read 55
            repro-4038    [185] .....   114.225717: netfs_wake_write_collector: Wake collector
            repro-4038    [185] .....   114.225723: netfs_wake_write_collector: Queuing collector work
            repro-4038    [185] .....   114.225727: netfs_unbuffered_write: netfs_unbuffered_write
            repro-4038    [185] .....   114.225727: netfs_end_issue_write: netfs_end_issue_write
            repro-4038    [185] .....   114.225728: netfs_end_issue_write: Write collector need poke 0
            repro-4038    [185] .....   114.225728: netfs_unbuffered_write_iter_locked: Waiting on NETFS_RREQ_IN_PROGRESS!
    ... (syzbot reproducer hangs)

There is a third "kworker/u1030" component that never gets woken up for
reasons currently unknown to me with Oleg's optimization. I'll keep
digging.

-- 
Thanks and Regards,
Prateek


  reply	other threads:[~2025-03-24 14:52 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-03-22 15:54 syzbot
2025-03-23 18:48 ` Oleg Nesterov
2025-03-23 19:17   ` syzbot
2025-03-23 19:47     ` Oleg Nesterov
2025-03-23 19:50       ` K Prateek Nayak
2025-03-23 19:52       ` Mateusz Guzik
2025-03-23 21:02         ` Oleg Nesterov
2025-03-24 10:47           ` K Prateek Nayak
2025-03-24 11:15             ` syzbot
2025-03-24 13:17             ` Mateusz Guzik
2025-03-24 13:19               ` K Prateek Nayak
2025-03-24 14:52                 ` K Prateek Nayak [this message]
2025-03-24 16:03                   ` Mateusz Guzik
2025-03-24 16:25                     ` K Prateek Nayak
2025-03-24 16:36                       ` K Prateek Nayak
2025-03-25  2:52                   ` K Prateek Nayak
2025-03-25 12:15                     ` Oleg Nesterov
2025-03-25 12:36                       ` Dominique Martinet
2025-03-25 13:04                       ` Oleg Nesterov
2025-03-25 14:49                         ` K Prateek Nayak
2025-03-25 14:58                           ` Dominique Martinet
2025-03-26 12:19                             ` Oleg Nesterov
2025-03-26 12:44                               ` Oleg Nesterov
2025-03-26 13:05                                 ` Oleg Nesterov
2025-03-27 17:46                             ` K Prateek Nayak
2025-03-27 21:19                               ` syzbot
2025-03-27 22:18                                 ` asmadeus
2025-03-28  4:01                                   ` K Prateek Nayak
2025-03-28  4:43                                     ` syzbot
2025-03-28 13:06                                     ` Oleg Nesterov
2025-03-28 13:07                                       ` syzbot
2025-03-28 13:25                                         ` Oleg Nesterov
2025-03-28 13:49                                           ` syzbot
2025-03-28 14:49                                             ` Oleg Nesterov
2025-03-28 15:22                                               ` syzbot
2025-03-28 17:00                                                 ` Oleg Nesterov
2025-03-28 17:56                                                   ` K Prateek Nayak
2025-03-28 18:20                                                     ` Oleg Nesterov
2025-03-29  0:00                                                   ` asmadeus
2025-03-29 14:21                                                     ` Oleg Nesterov
2025-03-29 23:27                                                       ` asmadeus
2025-03-30 10:21                                                         ` Oleg Nesterov
2025-03-23 20:03     ` Mateusz Guzik
2025-03-23 20:43       ` syzbot
2025-03-24 12:49 ` Edward Adam Davis
2025-03-24 13:37   ` syzbot
2025-03-28 18:14 ` David Howells
2025-03-28 18:44   ` syzbot
2025-03-28 19:01   ` Oleg Nesterov
2025-08-03 12:09 ` syzbot
     [not found] <20250323002028.3563-1-hdanton@sina.com>
2025-03-23  1:15 ` syzbot
     [not found] <20250323021219.3580-1-hdanton@sina.com>
2025-03-23  2:48 ` syzbot
     [not found] <20250324111613.3620-1-hdanton@sina.com>
2025-03-24 11:34 ` syzbot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f7585a27-aaef-4334-a1de-5e081f10c901@amd.com \
    --to=kprateek.nayak@amd.com \
    --cc=brauner@kernel.org \
    --cc=dhowells@redhat.com \
    --cc=jack@suse.cz \
    --cc=jlayton@kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mjguzik@gmail.com \
    --cc=netfs@lists.linux.dev \
    --cc=oleg@redhat.com \
    --cc=swapnil.sapkal@amd.com \
    --cc=syzbot+62262fdc0e01d99573fc@syzkaller.appspotmail.com \
    --cc=syzkaller-bugs@googlegroups.com \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®