Re: Possible memory ordering bug in page reclaim?

["David S. Miller" <davem@davemloft.net>]

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Sun, 16 Oct 2005 08:16:30 +1000

> > Even a write barrier is required on the left side, the read barrier on
> > the right side is useless if there is no write barrier on the left side.
> > 
> > Note that the barrier in atomic_add_negative is useless here because it
> > happens way too late, _after_ the count is decremented (not _before_)
> > so the decreased count could be already visible to the other cpu.
> 
> Not on ppc64.

Or sparc64, or ppc, or ... any platform that adheres to the
rules specified in Documentation/atomic_ops.txt :-)

One really needs the barriers both before and after the atomic
operation when you return a value, because this is the only way to get
sane semantics for dropping the final reference to some object.  When
one does:

	if (atomic_dec_and_test(&obj->count))
		__free_stuff(obj);

All other modifications to object state before the atomic
decrement must reach global visibility _first_, and then
the atomic decrement itself must be strongly ordered wrt.
future memory operations.

That's why it is documented that the memory barriers both before and
after the atomic operation are required when the operation returns a
result value of some kind.

There is a lot of code in the kernel that depends upon this.