[PATCH] f2fs: reduce lock overhead of extent node releasing

[PATCH] f2fs: reduce lock overhead of extent node releasing

[Chao Yu]

>From e5c6600d01c4462c4e1ee0c70ec1d9319862077d Mon Sep 17 00:00:00 2001
From: Chao Yu <chao2.yu@samsung.com>
Date: Thu, 2 Jul 2015 18:52:46 +0800
Subject: [PATCH] f2fs: reduce lock overhead of extent node releasing

Open and close critical section for each extent node when traversing rb-tree
results in high overhead of cpu, slows thing down.

This patch alternates to batch mode for removing extent nodes under spin lock.

Signed-off-by: Chao Yu <chao2.yu@samsung.com>
---
 fs/f2fs/data.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 6a706dd..7fb56a0 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -441,19 +441,31 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi,
 	struct extent_node *en;
 	unsigned int count = et->count;
 
-	node = rb_first(&et->root);
-	while (node) {
-		next = rb_next(node);
-		en = rb_entry(node, struct extent_node, rb_node);
+	if (!et->count)
+		return 0;
+
+	/* 1. remove all extent nodes in global lru list */
+	if (free_all) {
+		spin_lock(&sbi->extent_lock);
+		node = rb_first(&et->root);
+		while (node) {
+			next = rb_next(node);
+			en = rb_entry(node, struct extent_node, rb_node);
 
-		if (free_all) {
-			spin_lock(&sbi->extent_lock);
 			if (!list_empty(&en->list))
 				list_del_init(&en->list);
-			spin_unlock(&sbi->extent_lock);
+			node = next;
 		}
+		spin_unlock(&sbi->extent_lock);
+	}
+
+	/* 2. release all extent nodes which are not in global lru list */
+	node = rb_first(&et->root);
+	while (node) {
+		next = rb_next(node);
+		en = rb_entry(node, struct extent_node, rb_node);
 
-		if (free_all || list_empty(&en->list)) {
+		if (list_empty(&en->list)) {
 			__detach_extent_node(sbi, et, en);
 			kmem_cache_free(extent_node_slab, en);
 		}
-- 
2.4.2

Re: [PATCH] f2fs: reduce lock overhead of extent node releasing

[Jaegeuk Kim]

On Thu, Jul 02, 2015 at 08:40:12PM +0800, Chao Yu wrote:
> >From e5c6600d01c4462c4e1ee0c70ec1d9319862077d Mon Sep 17 00:00:00 2001
> From: Chao Yu <chao2.yu@samsung.com>
> Date: Thu, 2 Jul 2015 18:52:46 +0800
> Subject: [PATCH] f2fs: reduce lock overhead of extent node releasing
> 
> Open and close critical section for each extent node when traversing rb-tree
> results in high overhead of cpu, slows thing down.
> 
> This patch alternates to batch mode for removing extent nodes under spin lock.
> 
> Signed-off-by: Chao Yu <chao2.yu@samsung.com>
> ---
>  fs/f2fs/data.c | 28 ++++++++++++++++++++--------
>  1 file changed, 20 insertions(+), 8 deletions(-)
> 
> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> index 6a706dd..7fb56a0 100644
> --- a/fs/f2fs/data.c
> +++ b/fs/f2fs/data.c
> @@ -441,19 +441,31 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi,
>  	struct extent_node *en;
>  	unsigned int count = et->count;
>  
> -	node = rb_first(&et->root);
> -	while (node) {
> -		next = rb_next(node);
> -		en = rb_entry(node, struct extent_node, rb_node);
> +	if (!et->count)
> +		return 0;
> +
> +	/* 1. remove all extent nodes in global lru list */
> +	if (free_all) {
> +		spin_lock(&sbi->extent_lock);
> +		node = rb_first(&et->root);
> +		while (node) {
> +			next = rb_next(node);
> +			en = rb_entry(node, struct extent_node, rb_node);
>  
> -		if (free_all) {
> -			spin_lock(&sbi->extent_lock);
>  			if (!list_empty(&en->list))
>  				list_del_init(&en->list);
> -			spin_unlock(&sbi->extent_lock);
> +			node = next;
>  		}
> +		spin_unlock(&sbi->extent_lock);
> +	}
> +
> +	/* 2. release all extent nodes which are not in global lru list */

Hmm,
Is there any overhead to traverse the rb_tree twice and any spin_lock delay
caused by contention?

Thanks,

> +	node = rb_first(&et->root);
> +	while (node) {
> +		next = rb_next(node);
> +		en = rb_entry(node, struct extent_node, rb_node);
>  
> -		if (free_all || list_empty(&en->list)) {
> +		if (list_empty(&en->list)) {
>  			__detach_extent_node(sbi, et, en);
>  			kmem_cache_free(extent_node_slab, en);
>  		}
> -- 
> 2.4.2

RE: [f2fs-dev] [PATCH] f2fs: reduce lock overhead of extent node releasing

[Chao Yu]

> -----Original Message-----
> From: Jaegeuk Kim [mailto:jaegeuk@kernel.org]
> Sent: Saturday, July 04, 2015 3:04 PM
> To: Chao Yu
> Cc: linux-kernel@vger.kernel.org; linux-f2fs-devel@lists.sourceforge.net
> Subject: Re: [f2fs-dev] [PATCH] f2fs: reduce lock overhead of extent node releasing
> 
> On Thu, Jul 02, 2015 at 08:40:12PM +0800, Chao Yu wrote:
> > >From e5c6600d01c4462c4e1ee0c70ec1d9319862077d Mon Sep 17 00:00:00 2001
> > From: Chao Yu <chao2.yu@samsung.com>
> > Date: Thu, 2 Jul 2015 18:52:46 +0800
> > Subject: [PATCH] f2fs: reduce lock overhead of extent node releasing
> >
> > Open and close critical section for each extent node when traversing rb-tree
> > results in high overhead of cpu, slows thing down.
> >
> > This patch alternates to batch mode for removing extent nodes under spin lock.
> >
> > Signed-off-by: Chao Yu <chao2.yu@samsung.com>
> > ---
> >  fs/f2fs/data.c | 28 ++++++++++++++++++++--------
> >  1 file changed, 20 insertions(+), 8 deletions(-)
> >
> > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> > index 6a706dd..7fb56a0 100644
> > --- a/fs/f2fs/data.c
> > +++ b/fs/f2fs/data.c
> > @@ -441,19 +441,31 @@ static unsigned int __free_extent_tree(struct f2fs_sb_info *sbi,
> >  	struct extent_node *en;
> >  	unsigned int count = et->count;
> >
> > -	node = rb_first(&et->root);
> > -	while (node) {
> > -		next = rb_next(node);
> > -		en = rb_entry(node, struct extent_node, rb_node);
> > +	if (!et->count)
> > +		return 0;
> > +
> > +	/* 1. remove all extent nodes in global lru list */
> > +	if (free_all) {
> > +		spin_lock(&sbi->extent_lock);
> > +		node = rb_first(&et->root);
> > +		while (node) {
> > +			next = rb_next(node);
> > +			en = rb_entry(node, struct extent_node, rb_node);
> >
> > -		if (free_all) {
> > -			spin_lock(&sbi->extent_lock);
> >  			if (!list_empty(&en->list))
> >  				list_del_init(&en->list);
> > -			spin_unlock(&sbi->extent_lock);
> > +			node = next;
> >  		}
> > +		spin_unlock(&sbi->extent_lock);
> > +	}
> > +
> > +	/* 2. release all extent nodes which are not in global lru list */
> 
> Hmm,
> Is there any overhead to traverse the rb_tree twice and 

Yes, it will cost us more time to traverse the rb tree twice, adding one more
shrink list may reduce the overhead.

> any spin_lock delay caused by contention?

Maybe, since our critical region is enlarged, but the overhead of operation
for re-entering the critical region will be decreased as we invoke spin_lock
only one time.

Thanks,

> 
> Thanks,
> 
> > +	node = rb_first(&et->root);
> > +	while (node) {
> > +		next = rb_next(node);
> > +		en = rb_entry(node, struct extent_node, rb_node);
> >
> > -		if (free_all || list_empty(&en->list)) {
> > +		if (list_empty(&en->list)) {
> >  			__detach_extent_node(sbi, et, en);
> >  			kmem_cache_free(extent_node_slab, en);
> >  		}
> > --
> > 2.4.2
> 
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel