Re: [PATCHv4] SGI UV: TLB shootdown using broadcast assist unit

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Thursday 12 June 2008 22:23, Cliff Wickman wrote:
> From: Cliff Wickman <cpw@sgi.com>
>
> TLB shootdown for SGI UV.
>
> v1: 6/2 original
> v2: 6/3 corrections/improvements per Ingo's review
> v3: 6/4 split atomic operations off to a separate patch (Jeremy's review)
> v4: 6/12 include <mach_apic.h> rather than <asm/mach-bigsmp/mach_apic.h>
>          (fixes a !SMP build problem that Ingo found)
>          fix the index on uv_table_bases[blade]
> Now depends on patch:
>     x86 atomic operations: atomic_or_long atomic_inc_short
>   which was split off (and improved) at the suggestion of
>   Jeremy Fitzhardinge <jeremy@goop.org>
>
> This patch provides the ability to flush TLB's in cpu's that are not on
> the local node.  The hardware mechanism for distributing the flush
> messages is the UV's "broadcast assist unit".
>
> The hook to intercept TLB shootdown requests is a 2-line change to
> native_flush_tlb_others() (arch/x86/kernel/tlb_64.c).
>
> This code has been tested on a hardware simulator. The real hardware
> is not yet available.
>
> The shootdown statistics are provided through /proc/sgi_uv/ptc_statistics.
> The use of /sys was considered, but would have required the use of
> many /sys files.  The debugfs was also considered, but these statistics
> should be available on an ongoing basis, not just for debugging.
>
> Issues to be fixed later:
> - The IRQ for the messaging interrupt is currently hardcoded as 200
>   (see UV_BAU_MESSAGE).  It should be dynamically assigned in the future.
> - The use of appropriate udelay()'s is untested, as they are a problem
>   in the simulator.

For someone not too familiar with low level x86 (or UV) code, can
you explain why you are hooking at this point? I mean, what it
looks like is either a performance improvement, or for some reason
UV does not support send_IPI_mask out to CPUs "not on the local node".

If the former, what sort of improvement to you expect / see?

If the latter, then why aren't you hooking at send_IPI_mask? If
possible that would obviously be preferable so you aren't
making the tlb flushing harder to follow...

Can you set me straight? :)

Thanks,
Nick