Re: [PATCH] Split futex global spinlock futex_lock

[Rusty Russell <rusty@rustcorp.com.au>]

In message <37FBBA5F3A361C41AB7CE44558C3448E01C0B8DE@pdsmsx403.ccr.corp.intel.com> you write:
> +/*
> + * Split the global futex_lock into every hash list lock.
> + */
> +struct futex_hash_bucket {
> +       spinlock_t              lock;
> +       struct list_head       chain;
> +};
> +
>  /* The key for the hash is the address + index + offset within page */
> -static struct list_head futex_queues[1<<FUTEX_HASHBITS];
> -static spinlock_t futex_lock = SPIN_LOCK_UNLOCKED;
> +static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS] \
> +	__cacheline_aligned_in_smp;

This doesn't do what you expect, unfortunately.  You need:

struct futex_hash_bucket {
       spinlock_t              lock;
       struct list_head       chain;
} ____cacheline_aligned_in_smp;

static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS]
	__cacheline_aligned_in_smp;

Also, Jamie was hinting at a sectored approach: optimal memory
footprint/speed balance might be one lock in one cache-line-worth of
list_head.  But the above will do unless someone gets extremely
excited.

Uli, can we ask you for benchmarks with this change, too?

Cheers,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.