task kworker / kloopd blocked for more than 120 seconds

task kworker / kloopd blocked for more than 120 seconds

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Hello everybody,

I am experiencing issues on a live system based on Arch Linux [0][1] with
Linux kernel 4.0.x when booted in RAM mode (parameter 'copytoram') [2].
Probably the problem exists some time longer, I am not sure when this first
happened and what versions are effected.

The boot sequence looks like this:

-> boot kernel
-> start initramfs
-> get the squashfs image file (there is no difference with loop mount,
   nbd or http)
-> root filesystem is set up with loop devices and device mapper
-> boot continues

The root filesystem is accessed as follows:

-> ext4 filesystem
  -> device mapper snapshot target
    -> /dev/loop1: r/o ext4 image
      -> mounted squashfs
        -> /dev/loop0: squashfs image
    -> /dev/loop2: r/w COW image (sparse file)

    
Assembling the root filesystem stack always succeeds, but sometimes boot hangs
with a lot of:

INFO: task kworker/u#:#:### blocked for more than 120 seconds

and a single:

INFO: task kloopd/### blocked for more than 120 seconds

Any idea what goes wrong? Looks like a race condition or deadlock.

[0] https://wiki.archlinux.org/index.php/Archiso
[1] https://projects.archlinux.org/archiso.git/
[2] https://projects.archlinux.org/archiso.git/tree/docs/README.bootparams
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: task kworker / kloopd blocked for more than 120 seconds

[Ming Lei]

Hello Christian,

On Thu, May 28, 2015 at 5:37 PM, Christian Hesse <list@eworm.de> wrote:
> Hello everybody,
>
> I am experiencing issues on a live system based on Arch Linux [0][1] with
> Linux kernel 4.0.x when booted in RAM mode (parameter 'copytoram') [2].
> Probably the problem exists some time longer, I am not sure when this first
> happened and what versions are effected.
>
> The boot sequence looks like this:
>
> -> boot kernel
> -> start initramfs
> -> get the squashfs image file (there is no difference with loop mount,
>    nbd or http)
> -> root filesystem is set up with loop devices and device mapper
> -> boot continues
>
> The root filesystem is accessed as follows:
>
> -> ext4 filesystem
>   -> device mapper snapshot target
>     -> /dev/loop1: r/o ext4 image
>       -> mounted squashfs
>         -> /dev/loop0: squashfs image
>     -> /dev/loop2: r/w COW image (sparse file)
>
>
> Assembling the root filesystem stack always succeeds, but sometimes boot hangs
> with a lot of:

Could you try the following two patches to see if they can fix your issue?

http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=f4aa4c7bbac6c4afdd4adccf90898c1a3685396d

http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=4d4e41aef9429872ea3b105e83426941f7185ab6

Thanks,
Ming

>
> INFO: task kworker/u#:#:### blocked for more than 120 seconds
>
> and a single:
>
> INFO: task kloopd/### blocked for more than 120 seconds
>
> Any idea what goes wrong? Looks like a race condition or deadlock.
>
> [0] https://wiki.archlinux.org/index.php/Archiso
> [1] https://projects.archlinux.org/archiso.git/
> [2] https://projects.archlinux.org/archiso.git/tree/docs/README.bootparams
> --
> main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
> "CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
> putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}



-- 
Ming Lei

Re: task kworker / kloopd blocked for more than 120 seconds

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

Ming Lei <tom.leiming@gmail.com> on Thu, 2015/05/28 20:20:
On Thu, May 28, 2015 at 5:37 PM, Christian Hesse <list@eworm.de> wrote:
> > Assembling the root filesystem stack always succeeds, but sometimes boot
> > hangs with a lot of:  
> 
> Could you try the following two patches to see if they can fix your issue?
> 
> http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=f4aa4c7bbac6c4afdd4adccf90898c1a3685396d
> 
> http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=4d4e41aef9429872ea3b105e83426941f7185ab6

Sounds reasonable. System works perfectly stable and reliable since booted
with a patched kernel. Thanks a lot for the hint!

Looks like the changes did not make it into Linus' tree yet. So it will take
some time for the patches to show up in a stable release, no?
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: task kworker / kloopd blocked for more than 120 seconds

[Ming Lei]

On Fri, May 29, 2015 at 8:15 PM, Christian Hesse <list@eworm.de> wrote:
> Ming Lei <tom.leiming@gmail.com> on Thu, 2015/05/28 20:20:
> On Thu, May 28, 2015 at 5:37 PM, Christian Hesse <list@eworm.de> wrote:
>> > Assembling the root filesystem stack always succeeds, but sometimes boot
>> > hangs with a lot of:
>>
>> Could you try the following two patches to see if they can fix your issue?
>>
>> http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=f4aa4c7bbac6c4afdd4adccf90898c1a3685396d
>>
>> http://git.kernel.org/cgit/linux/kernel/git/axboe/linux-block.git/commit/?h=for-next&id=4d4e41aef9429872ea3b105e83426941f7185ab6
>
> Sounds reasonable. System works perfectly stable and reliable since booted
> with a patched kernel. Thanks a lot for the hint!
>
> Looks like the changes did not make it into Linus' tree yet. So it will take
> some time for the patches to show up in a stable release, no?

The patches will land 4.2, and then reach stable release because both
two are marked as stable.


-- 
Ming Lei