Re: readl/writel and memory barriers

[David Mosberger <davidm@hpl.hp.com>]

>>>>> On Tue, 19 Feb 2002 10:35:06 -0800, Jesse Barnes <jbarnes@sgi.com> said:

  Jesse> Making a variable volatile doesn't guarantee that the
  Jesse> compiler won't reorder references to it, AFAIK.  And on some
  Jesse> platforms, even uncached I/O references aren't necessarily
  Jesse> ordered.

It certainly does for on ia64-compliant system.  Check section 9.3
"Multi-threaded Code" in the "Itanium Software Conventions and Runtime
Architecture manual".

  Jesse> To avoid the overhead of having I/O flushed on every memory
  Jesse> barrier and readX/writeX operation, we've introduced mmiob()
  Jesse> on ia64, which explicity orders I/O space accesses.  Some
  Jesse> ports have chosen to take the performance hit in every
  Jesse> readX/writeX, memory barrier, and spinlock however
  Jesse> (e.g. PPC64, MIPS).

I think this is a bit of a different problem.  On non-NUMA platforms,
the performance hit of enforcing order is not huge.  Basically, as
long as the CPU issues the accesses in order, you'll be fine.

Now, with NUMA platforms, where the chipsets/switch may re-order
accesses, the performance hit will be much bigger, so the old scheme
may not be sufficient.

  Jesse> Is this a reasonable approach?  Is it acceptable to have a
  Jesse> seperate barrier operation for I/O space?  If so, perhaps
  Jesse> other archs would be willing to add mmiob() ops?

I'm no NUMA expert, but my guess is that nobody will want to go
through all the existing drivers to change them to use mmiob().  For
new drivers, it might be OK.

	--david