Re: Comments on UV tlb flushing

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Tuesday 29 July 2008 10:28, Jeremy Fitzhardinge wrote:
> I'm just reworking the x86 tlb code to use smp_call_function_mask, and I
> see how the UV tlb flushing hooks in.  A few things occur to me:
>
>    1. There should be a CONFIG_X86_UV to select this code.  tlb_uv.o is
>       around 6k, which is not trivial overhead to subject every x86_64
>       kernel to.

Definitely.


>    2. CONFIG_X86_UV should either depend on or select CONFIG_PARAVIRT.
>    3. You should hook into paravirt_ops to enable your tlb-flush code.
>       That is, in - say - uv_bau_init() you do
>       "pv_mmu_ops.flush_tlb_others = uv_flush_tlb_others".  This removes
>       a test/branch in the generic code.  Using paravirt_ops may open
>       other opportunities to put UV-optimised functions in place without
>       having to modify generic code.

Really? It's not virtualized at all, although I don't like adding that
branch for such a small class of systems either.

It would possibly be better to have a new function (eg.
override_flush_tlb_others()), which returns 0 if
CONFIG_OVERRIDE_FLUSH_TLB is set, otherwise branches. And have *that*
selected by CONFIG_PARAVIRT and X86_UV.


> My understanding is that the UV hardware has some kind of
> payload-carrying IPI mechanism, which is a capability could be useful to
> express in a higher-level way in the kernel.  Certainly I could imagine
> using it in a virtual environment as a way to do inter-VCPU messaging
> with less context switch overhead.

Yes, as I said in my review of that part of the UV tlb flushing, it would
be nice to have a generic mechanism to IPI with payload, which falls back
to a smp_call_function-like approach on platforms that don't have the
capability.