Re: x86_64: 2.6.14-rc4 swiotlb broken

[Alex Williamson <alex.williamson@hp.com>]

On Mon, 2005-10-17 at 11:20 -0700, Christoph Lameter wrote:
> On Mon, 17 Oct 2005, Ravikiran G Thirumalai wrote:
> 
> > Maybe someone with access to ia64 NUMA boxen can check if the NODE(0)
> > solution works (and does not break anything) on ia64?  Chrisoph, can you help?
> 
> Umm... SGI does not use the swiotlb and we do not have these issues. HP 
> does use the swiotlb on IA64. CCing John and Alex.
...
> @@ -123,7 +123,7 @@
>  	/*
>  	 * Get IO TLB memory from the low pages
>  	 */
> -	io_tlb_start = alloc_bootmem_low_pages(io_tlb_nslabs *
> +	io_tlb_start = alloc_bootmem_node(NODE_DATA(0), io_tlb_nslabs *

   HP ia64 boxes typically use a hardware I/O TLB, so this is not the
normal case.  However, the sx1000 boxes are exactly an example that will
break because of this assumption about memory layout.  These boxes can
be configured to have various ratios of node local memory and
interleaved memory.  Node local memory starts well above 4GB.
Interleaved memory is zero-based, and described in it's own proximity
domain.  It therefore looks like a memory-only node.  I believe the
above code change would cause us to allocate memory from the node local
range, way too high in the address space for bounce buffers.

   BTW, I've got a patch in Tony's testing branch that allows a late
initialization of the swiotlb.  This is currently only useful on ia64
since we have 4GB of ZONE_DMA, but may be useful on x86-ish archs when
the 4GB zone is introduced.

	Alex

--