mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Eric Dumazet <eric.dumazet@gmail.com>
To: Andi Kleen <andi@firstfloor.org>
Cc: herbert@gondor.hengli.com.au, dm-devel@redhat.com,
	linux-kernel@vger.kernel.org, agk@redhat.com, ak@linux.intel.com
Subject: Re: [PATCH] DM-CRYPT: Scale to multiple CPUs
Date: Mon, 31 May 2010 18:34:55 +0200	[thread overview]
Message-ID: <1275323695.3291.122.camel@edumazet-laptop> (raw)
In-Reply-To: <20100531160425.GA20344@basil.fritz.box>

Le lundi 31 mai 2010 à 18:04 +0200, Andi Kleen a écrit :
> DM-CRYPT: Scale to multiple CPUs
> 
> Currently dm-crypt does all encryption work per dmcrypt mapping in a
> single workqueue. This does not scale well when multiple CPUs
> are submitting IO at a high rate. The single CPU running the single
> thread cannot keep up with the encryption and encrypted IO performance
> tanks.
> 
> This patch changes the crypto workqueue to be per CPU. This means
> that as long as the IO submitter (or the interrupt target CPUs
> for reads) runs on different CPUs the encryption work will be also
> parallel.
> 
> To avoid a bottleneck on the IO worker I also changed those to be
> per CPU threads.
> 
> There is still some shared data, so I suspect some bouncing
> cache lines. But I haven't done a detailed study on that yet.
> 
> All the threads are global, not per CPU. That is to avoid a thread
> explosion on systems with a large number of CPUs and a larger
> number of dm-crypt mappings. The code takes care to avoid problems
> with nested mappings.
>     
> Signed-off-by: Andi Kleen <ak@linux.intel.com>


>  	/*
> +	 * Duplicated per cpu state. Access through
> +	 * per_cpu_ptr() only.
> +	 */
> +	struct crypt_cpu *cpu;
> +
> +	/*
>  	 * Layout of each crypto request:
>  	 *

Since commit e0fdb0e050eae331, we have a __percpu annotation so that
sparse can be augmented. This would also make the comment unnecessary...


We also have this_cpu_ accessors

> +       return per_cpu_ptr(cc->cpu, smp_processor_id());

	this_cpu_ptr(cc->cpu)

and __thic_cpu_ptr() for the 'raw' version


> +       __get_cpu_var(io_wq_cpu) = current;

	this_cpu_write(io_wq_cpu, current)



  reply	other threads:[~2010-05-31 16:57 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-05-31 16:04 Andi Kleen
2010-05-31 16:34 ` Eric Dumazet [this message]
2010-05-31 17:46   ` Andi Kleen
2010-05-31 17:52     ` Eric Dumazet
2010-05-31 17:22 ` [dm-devel] " Milan Broz
2010-05-31 17:42   ` Andi Kleen
2010-05-31 18:10     ` Milan Broz
2010-05-31 18:27       ` Andi Kleen
2010-05-31 18:55         ` Milan Broz
2010-05-31 19:04           ` Andi Kleen
2010-05-31 22:07             ` Herbert Xu
2010-06-01  1:46               ` huang ying
2010-06-01  2:39         ` Mikulas Patocka
2010-06-01  2:44 ` Mikulas Patocka
2010-06-01  4:39   ` Herbert Xu
2010-06-02  5:10     ` Mikulas Patocka
2010-06-02  5:14       ` Herbert Xu
2010-06-02  5:25         ` Mikulas Patocka
2010-06-02  6:07           ` Herbert Xu
2010-06-02  6:30             ` Steffen Klassert
2010-06-02  7:07               ` Herbert Xu
2010-06-02  7:51           ` Andi Kleen
2010-06-01  6:40   ` Andi Kleen
2010-06-02  5:15     ` Mikulas Patocka
2010-06-02  5:20       ` Herbert Xu
2010-06-02  7:53       ` Andi Kleen
2010-06-02 10:20         ` Herbert Xu
2010-05-31 18:47 Tomasz Chmielewski
2010-06-01  7:21 ` Andi Kleen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1275323695.3291.122.camel@edumazet-laptop \
    --to=eric.dumazet@gmail.com \
    --cc=agk@redhat.com \
    --cc=ak@linux.intel.com \
    --cc=andi@firstfloor.org \
    --cc=dm-devel@redhat.com \
    --cc=herbert@gondor.hengli.com.au \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®