From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752237AbdHPN3I convert rfc822-to-8bit (ORCPT ); Wed, 16 Aug 2017 09:29:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:14591 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751661AbdHPN3G (ORCPT ); Wed, 16 Aug 2017 09:29:06 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 5184FC04F4D7 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=longman@redhat.com Subject: Re: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries To: "Wangkai (Kevin,C)" , Alexander Viro , Jonathan Corbet Cc: "linux-kernel@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley References: <1501266880-26288-1-git-send-email-longman@redhat.com> <1cdfb670-42ed-f8f6-aeab-c38ea0567651@redhat.com> <87B246BB5ED53A4C98E4F9A35839EDE1853965F7@nkgeml514-mbx.china.huawei.com> From: Waiman Long Organization: Red Hat Message-ID: <1756a532-ed56-f2f8-0463-bd659923acd1@redhat.com> Date: Wed, 16 Aug 2017 09:29:04 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <87B246BB5ED53A4C98E4F9A35839EDE1853965F7@nkgeml514-mbx.china.huawei.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT Content-Language: en-US X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 16 Aug 2017 13:29:06 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/16/2017 06:33 AM, Wangkai (Kevin,C) wrote: >> -----Original Message----- >> From: linux-fsdevel-owner@vger.kernel.org >> [mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of Waiman Long >> Sent: Wednesday, August 16, 2017 1:15 AM >> To: Alexander Viro; Jonathan Corbet >> Cc: linux-kernel@vger.kernel.org; linux-doc@vger.kernel.org; >> linux-fsdevel@vger.kernel.org; Paul E. McKenney; Andrew Morton; Ingo Molnar; >> Miklos Szeredi; Matthew Wilcox; Larry Woodman; James Bottomley >> Subject: Re: [PATCH v3 0/5] fs/dcache: Limit # of negative dentries >> >> On 07/28/2017 02:34 PM, Waiman Long wrote: >>> v2->v3: >>> - Add a faster pruning rate when the free pool is closed to depletion. >>> - As suggested by James Bottomley, add an artificial delay waiting >>> loop before killing a negative dentry and properly clear the >>> DCACHE_KILL_NEGATIVE flag if killing doesn't happen. >>> - Add a new patch to track number of negative dentries that are >>> forcifully killed. >>> >>> v1->v2: >>> - Move the new nr_negative field to the end of dentry_stat_t structure >>> as suggested by Matthew Wilcox. >>> - With the help of Miklos Szeredi, fix incorrect locking order in >>> dentry_kill() by using lock_parent() instead of locking the parent's >>> d_lock directly. >>> - Correctly account for positive to negative dentry transitions. >>> - Automatic pruning of negative dentries will now ignore the reference >>> bit in negative dentries but not the regular shrinking. >>> >>> A rogue application can potentially create a large number of negative >>> dentries in the system consuming most of the memory available. This >>> can impact performance of other applications running on the system. >>> >>> This patchset introduces changes to the dcache subsystem to limit the >>> number of negative dentries allowed to be created thus limiting the >>> amount of memory that can be consumed by negative dentries. >>> >>> Patch 1 tracks the number of negative dentries used and disallow the >>> creation of more when the limit is reached. >>> >>> Patch 2 enables /proc/sys/fs/dentry-state to report the number of >>> negative dentries in the system. >>> >>> Patch 3 enables automatic pruning of negative dentries when it is >>> close to the limit so that we won't end up killing recently used >>> negative dentries. >>> >>> Patch 4 prevents racing between negative dentry pruning and umount >>> operation. >>> >>> Patch 5 shows the number of forced negative dentry killings in >>> /proc/sys/fs/dentry-state. End users can then tune the neg_dentry_pc= >>> kernel boot parameter if they want to reduce forced negative dentry >>> killings. >>> >>> Waiman Long (5): >>> fs/dcache: Limit numbers of negative dentries >>> fs/dcache: Report negative dentry number in dentry-state >>> fs/dcache: Enable automatic pruning of negative dentries >>> fs/dcache: Protect negative dentry pruning from racing with umount >>> fs/dcache: Track count of negative dentries forcibly killed >>> >>> Documentation/admin-guide/kernel-parameters.txt | 7 + >>> fs/dcache.c | 451 >> ++++++++++++++++++++++-- >>> include/linux/dcache.h | 8 +- >>> include/linux/list_lru.h | 1 + >>> mm/list_lru.c | 4 +- >>> 5 files changed, 435 insertions(+), 36 deletions(-) >>> >> I haven't received any comment on this v3 patch for over 2 weeks. Is there >> anything I can do to make it more ready to be merged? >> >> Thanks, >> Longman > Hi Longman, > I am a fresher of fsdevel, about 2 weeks before, I have joined this mail list, recently I have met the same problem of negative dentries, > in my opinion, the dentries should be remove together with the files or directories, I don't know you have submit this patch, I have > another patch about this: > > http://marc.info/?l=linux-fsdevel&m=150209902215266&w=2 > > maybe this is a foo idea... > > regards > Kevin If you look at the code, the front dentries of the LRU list are removed when there are too many negative dentries. That includes positive dentries as well as it is not practical to just remove the negative dentries. I have looked at your patch. The dentry of a removed file becomes a negative dentry. The kernel can keep track of those negative entries and there is no need to add an additional flag for that. Cheers, Longman