Re: [Feature] x86_64 page tracking for Stratus servers

["Kimball Murray" <kimball.murray@gmail.com>]

On 9/6/06, Andi Kleen <ak@suse.de> wrote:
>
> > Silly question, why can't you do all this from stop_machine_run context (or
> > your SMI) that doesn't have to worry about other CPUs dirtying memory?
>
> Because that would be too slow for continuous mirroring.
>
> You can't go through 10+GB of virtual memory (or more with shared
> memory because the scan has to be virtual) in an interrupt.
>
> The only sane way is to do it continuously.

That's exactly right on all counts.  We can "harvest" dirty pages in a
continuous fashion, as a background task.  But when we finally decide
to dive in to machine blackout for the final copy, we need to ensure
that the blackout is as short as possible.  Toward that goal, we have
built a sort of DMA engine that can move pages over the backplane to
the spare cpu/memory module without interfering (much) with the active
module's activities.  We continue doing this until the number of pages
for final copy is small enough that we can be sure the final blackout
copy will take a sufficiently short time.

And you're right about the virtual scan as well.  Trouble is, it's a
potentially unbounded scan.  With the right machine load, we can find
ourselves in a state where it takes so long to scan all the page
tables that the pages are all dirty again before we start the next
pass. For this situation we have two choices.  We can either give up
and dive into the blackout passing a dirty list that basically
includes all of memory (long blackout time!), or we can continue
scanning for a long time, waiting and hoping the machine load will
decrease to a point where are target number of dirty pages is reached.
 This sort of thing will always be a problem with a software solution.

-kimball