Re: [PATCH] xen: Use wmb instead of rmb in xen_evtchn_do_upcall().

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Tuesday 10 June 2008 17:57, Jeremy Fitzhardinge wrote:
> Nick Piggin wrote:
> > On Tuesday 10 June 2008 17:35, Isaku Yamahata wrote:
> >> This patch is ported one from 534:77db69c38249 of linux-2.6.18-xen.hg.
> >> Use wmb instead of rmb to enforce ordering between
> >> evtchn_upcall_pending and evtchn_pending_sel stores
> >> in xen_evtchn_do_upcall().
> >
> > There are a whole load of places in the kernel that should be using
> > smp_ variants of memory barriers. This seemed to me like one of them,
> > but I could be wrong.
>
> No, it needs to be an unconditional barrier.  This is synchronizing with
> the hypervisor - even if the kernel is compiled UP, the SMP hypervisor
> may be testing/setting the events pending bits from another (physical) cpu.

OK. What you *really* want is smp_*mb_even_if_compiled_for_UP() ;)
That is, a small set of primitives that are compiled with CONFIG_SMP
(and given some xxx_ prefix to distinguish).

IO barriers are probably the best thing you can use for the moment.


> > Also, if you do that can you get rid of the ifdef? If it really *really*
> > mattered, we could introduce smp_mb before/after xchg... but if you
> > use smp_wmb anyway then it definitely does not matter because that is a
> > noop on x86.
>
> Yes, I'd like to lose the #ifdef.  Unfortunately I think putting a
> "lock; addl $0,0(%%esp)" style barrier had a measurable negative
> performance impact, but I may be thinking of something else.  I don't
> know how expensive sfence is.
>
> The alternative is to make ia64's xchg a barrier (or to add a barrier
> variant of it).  It seems like a wart to have a cross-architecture
> function like xchg(), but then have different architectures differ in
> important details like barrier-ness.

Well, no you have to be careful. Because even if we did ask for ia64's
xchg to be a full barrier, you wouldn't get the right behaviour on UP
because it would be free to optimise that away -- those kinds of barriers
referred to in all those primitives are defined for cacheable memory and
CPU-to-CPU only...

I guess under the circumstances, leaving the ifdef there is probably
reasonable.