Re: Opinion on ordering of writel vs. stores to RAM

[David Miller <davem@davemloft.net>]

From: Paul Mackerras <paulus@samba.org>
Date: Sat, 9 Sep 2006 13:02:27 +1000

> I suspect the best thing at this point is to move the sync in writeX()
> before the store, as you suggest, and add an "eieio" before the load
> in readX().  That does mean that we are then relying on driver writers
> putting in the mmiowb() between a writeX() and a spin_unlock, but at
> least that is documented.

I think not matching what PC systems do is, at least from one
perspective, a very bad engineering decision for 2 reasons.

1) You will be chasing down these kinds of problems forever,
   you will fix tg3 today, but tomorrow it will be another driver
   for which you will invest weeks of delicate debugging that
   could have been spent on much more useful coding

2) Driver authors will not get these memory barriers right,
   you can say they will because it will be "documented" but
   that does not change reality which is that driver folks
   will get simple interfaces right but these memory barriers
   are relatively advanced concepts, which they thus will get
   wrong half the time

Sure it's more expensive, but at least on sparc64 I'd much rather
spend my time working on more interesting things than "today's
missing memory barrier" :-)

I also don't want to see all of these memory barriers crapping up our
drivers.  I do a MMIO, then I access a descriptor, or vice versa, then
those should be ordered because they are both technically accesses to
"physical device state".  Having to say this explicitly seems really
the wrong thing to do, at least to me.