Re: Hugepages demand paging V2 [0/8]: Discussion and overview

[Robin Holt <holt@sgi.com>]

On Mon, Oct 25, 2004 at 07:40:30PM -0700, Jesse Barnes wrote:
> On Monday, October 25, 2004 7:23 pm, William Lee Irwin III wrote:
> > On Mon, Oct 25, 2004 at 06:26:42PM -0700, Christoph Lameter wrote:
> > > - Clearing hugetlb pages is time consuming using clear_highpage in
> > > alloc_huge_page. Make it possible to use hw assist via DMA or so there?
> >
> > It's possible, but it's been found not to be useful. What has been found
> > useful is assistance from much lower-level memory hardware of a kind
> > not to be had in any extant mass-manufactured machines.
> 
> Do you have examples?  SGI hardware has a so-called 'BTE' (for Block Transfer 
> Engine) that can arbitrarily zero or copy pages w/o CPU assistance.  It's 
> builtin to the memory controller.  Using it to zero the pages has the 
> advantages of being asyncrhonous and not hosing the CPU cache.
> 

Jesse,

Sorry for being a stickler here, but the BTE is really part of the
I/O Interface portion of the shub.  That portion has a seperate clock
frequency from the memory controller (unfortunately slower).  The BTE
can zero at a slightly slower speed than the processor.  It does, as
you pointed out, not trash the CPU cache.

One other feature of the BTE is it can operate asynchronously from
the cpu.  This could be used to, during a clock interrupt, schedule
additional huge page zero filling on multiple nodes at the same time.
This could result in a huge speed boost on machines that have multiple
memory only nodes.  That has not been tested thoroughly.  We have done
considerable testing of the page zero functionality as well as the
error handling.

Robin