Re: [PATCH RFC 1/1] kernfs: keep kernfs node alive for __kernfs_remove()

[Konstantin Khorenko <khorenko@virtuozzo.com>]

On 04/16/2019 10:17 PM, Tejun Heo wrote:
> On Tue, Apr 16, 2019 at 06:53:35PM +0300, Konstantin Khorenko wrote:
>> __kernfs_remove() which is called under kernfs_mutex,
>> assumes nobody kills kernfs node whie it's working on it
>> and "get"s current kernfs node for that.
>>
>> But we hit a warning in kernfs_get(): kn->counter == 0 already:
>>   ------------[ cut here ]------------
>>   WARNING: CPU: 2 PID: 63923 at fs/kernfs/dir.c:377 kernfs_get+0x2f/0x40
>>   ...
>>   Call Trace:
>>    [<ffffffffa7f92e67>] dump_stack+0x19/0x1b
>>    [<ffffffffa78987b8>] __warn+0xd8/0x100
>>    [<ffffffffa78988fd>] warn_slowpath_null+0x1d/0x20
>>    [<ffffffffa7aecaff>] kernfs_get+0x2f/0x40
>>    [<ffffffffa7aed233>] __kernfs_remove+0x113/0x260
>>    [<ffffffffa7aee201>] kernfs_remove+0x21/0x30
>>    [<ffffffffa7af1010>] sysfs_remove_dir+0x50/0x80
>>    [<ffffffffa7b9fb38>] kobject_del+0x18/0x50
>>    [<ffffffffa7a38a4d>] sysfs_slab_remove+0x3d/0x50
>>    [<ffffffffa79f1e6b>] do_kmem_cache_release+0x3b/0x70
>>    [<ffffffffa79f2aa1>] memcg_destroy_kmem_caches+0xb1/0xf0
>>    [<ffffffffa7a4ed5c>] mem_cgroup_css_free+0x4c/0x280
>>    [<ffffffffa79377fc>] cgroup_free_fn+0x4c/0x120
>>    [<ffffffffa78bc222>] process_one_work+0x182/0x440
>>    [<ffffffffa78bd3d6>] worker_thread+0x126/0x3c0
>>    [<ffffffffa78c4441>] kthread+0xd1/0xe0
>>
>> This could be for example because of kernfs_notify_workfn() which
>> does kernfs_put(kn) out of kernfs_mutex held section,
>> so move kernfs_put(kn) under the mutex.
>
> This patch doesn't really make sense to me.  Can you give a more
> concrete scenario where this would help?

i don't know the full scenario unfortunately, but the idea is the following:

__kernfs_remove() is called under kernfs_mutex and if
   !(!kn || (kn->parent && RB_EMPTY_NODE(&kn->rb)))

it assumes that nothing can change while we hold the mutex and
for each kernfs descendant should have kn->count > 0.

=====
         /* deactivate and unlink the subtree node-by-node */
         do {
                 pos = kernfs_leftmost_descendant(kn);

                 /*
                  * kernfs_drain() drops kernfs_mutex temporarily and @pos's
                  * base ref could have been put by someone else by the time
                  * the function returns.  Make sure it doesn't go away
                  * underneath us.
                  */
                 kernfs_get(pos);
=====

At the same time kernfs_notify_workfn() can do a kernfs_put() out of kernfs_mutex
which probably can be the last put and dec kn->count to 0 any moment.


Thank you.

--
Best regards,

Konstantin Khorenko,
Virtuozzo Linux Kernel Team