From: Mel Gorman <mel@csn.ul.ie>
To: Andrea Arcangeli <aarcange@redhat.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Linus Torvalds <torvalds@linux-foundation.org>,
Andrew Morton <akpm@linux-foundation.org>,
Peter Zijlstra <pzijlstr@redhat.com>, Ingo Molnar <mingo@elte.hu>,
Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Hillf Danton <dhillf@gmail.com>,
Andrew Jones <drjones@redhat.com>, Dan Smith <danms@us.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Paul Turner <pjt@google.com>, Christoph Lameter <cl@linux.com>,
Suresh Siddha <suresh.b.siddha@intel.com>,
Mike Galbraith <efault@gmx.de>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: [PATCH 05/33] autonuma: pte_numa() and pmd_numa()
Date: Thu, 11 Oct 2012 20:54:50 +0100 [thread overview]
Message-ID: <20121011195450.GM3317@csn.ul.ie> (raw)
In-Reply-To: <20121011165847.GO1818@redhat.com>
On Thu, Oct 11, 2012 at 06:58:47PM +0200, Andrea Arcangeli wrote:
> On Thu, Oct 11, 2012 at 12:15:45PM +0100, Mel Gorman wrote:
> > huh?
> >
> > #define _PAGE_NUMA _PAGE_PROTNONE
> >
> > so this is effective _PAGE_PRESENT | _PAGE_PROTNONE | _PAGE_PROTNONE
> >
> > I suspect you are doing this because there is no requirement for
> > _PAGE_NUMA == _PAGE_PROTNONE for other architectures and it was best to
> > describe your intent. Is that really the case or did I miss something
> > stupid?
>
> Exactly.
>
> It reminds that we need to return true in pte_present when the NUMA
> hinting page fault is on.
>
> Hardwiring _PAGE_NUMA to _PAGE_PROTNONE conceptually is not necessary
> and it's actually an artificial restrictions. Other archs without a
> bitflag for _PAGE_PROTNONE, may want to use something else and they'll
> have to deal with pte_present too, somehow. So this is a reminder for
> them as well.
>
That's all very reasonable.
> > > static inline int pte_hidden(pte_t pte)
> > > @@ -420,7 +421,63 @@ static inline int pmd_present(pmd_t pmd)
> > > * the _PAGE_PSE flag will remain set at all times while the
> > > * _PAGE_PRESENT bit is clear).
> > > */
> > > - return pmd_flags(pmd) & (_PAGE_PRESENT | _PAGE_PROTNONE | _PAGE_PSE);
> > > + return pmd_flags(pmd) & (_PAGE_PRESENT | _PAGE_PROTNONE | _PAGE_PSE |
> > > + _PAGE_NUMA);
> > > +}
> > > +
> > > +#ifdef CONFIG_AUTONUMA
> > > +/*
> > > + * _PAGE_NUMA works identical to _PAGE_PROTNONE (it's actually the
> > > + * same bit too). It's set only when _PAGE_PRESET is not set and it's
> >
> > same bit on x86, not necessarily anywhere else.
>
> Yep. In fact before using _PAGE_PRESENT the two bits were different
> even on x86. But I unified them. If I vary them then they will become
> _PAGE_PTE_NUMA/_PAGE_PMD_NUMA and the above will fail to build without
> risk of errors.
>
Ok.
> >
> > _PAGE_PRESENT?
>
> good eye ;) corrected.
>
> > > +/*
> > > + * pte/pmd_mknuma sets the _PAGE_ACCESSED bitflag automatically
> > > + * because they're called by the NUMA hinting minor page fault.
> >
> > automatically or atomically?
> >
> > I assume you meant atomically but what stops two threads faulting at the
> > same time and doing to the same update? mmap_sem will be insufficient in
> > that case so what is guaranteeing the atomicity. PTL?
>
> I meant automatically. I explained myself wrong and automatically may
> be the wrong word. It also is atomic of course but it wasn't about the
> atomic part.
>
> So the thing is: the numa hinting page fault hooking point is this:
>
> if (pte_numa(entry))
> return pte_numa_fixup(mm, vma, address, entry, pte, pmd);
>
> It won't get this far:
>
> entry = pte_mkyoung(entry);
> if (ptep_set_access_flags(vma, address, pte, entry, flags & FAULT_FLAG_WRITE)) {
>
> So if I don't set _PAGE_ACCESSED in pte/pmd_mknuma, the TLB miss
> handler will have to set _PAGE_ACCESSED itself with an additional
> write on the pte/pmd later when userland touches the page. And that
> will slow us down for no good.
>
All clear now. Letting it fall through to reach that point would be
convulated and messy. This is a better option.
> Because mknuma is only called in the numa hinting page fault context,
> it's optimal to set _PAGE_ACCESSED too, not only _PAGE_PRESENT (and
> clearing _PAGE_NUMA of course).
>
> The basic idea, is that the numa hinting page fault can only trigger
> if userland touches the page, and after such an event, _PAGE_ACCESSED
> would be set by the hardware no matter if there is a NUMA hinting page
> fault or not (so we can optimize away the hardware action when the NUMA
> hinting page fault triggers).
>
> I tried to reword it:
>
> diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
> index cf1d3f0..3dc6a9b 100644
> --- a/arch/x86/include/asm/pgtable.h
> +++ b/arch/x86/include/asm/pgtable.h
> @@ -449,12 +449,12 @@ static inline int pmd_numa(pmd_t pmd)
> #endif
>
> /*
> - * pte/pmd_mknuma sets the _PAGE_ACCESSED bitflag automatically
> - * because they're called by the NUMA hinting minor page fault. If we
> - * wouldn't set the _PAGE_ACCESSED bitflag here, the TLB miss handler
> - * would be forced to set it later while filling the TLB after we
> - * return to userland. That would trigger a second write to memory
> - * that we optimize away by setting _PAGE_ACCESSED here.
> + * pte/pmd_mknuma sets the _PAGE_ACCESSED bitflag too because they're
> + * only called by the NUMA hinting minor page fault. If we wouldn't
> + * set the _PAGE_ACCESSED bitflag here, the TLB miss handler would be
> + * forced to set it later while filling the TLB after we return to
> + * userland. That would trigger a second write to memory that we
> + * optimize away by setting _PAGE_ACCESSED here.
> */
> static inline pte_t pte_mknonnuma(pte_t pte)
> {
>
Much better.
--
Mel Gorman
SUSE Labs
next prev parent reply other threads:[~2012-10-11 19:54 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1349308275-2174-1-git-send-email-aarcange@redhat.com>
[not found] ` <20121004113943.be7f92a0.akpm@linux-foundation.org>
2012-10-05 23:14 ` [PATCH 00/33] AutoNUMA27 Andi Kleen
2012-10-05 23:57 ` Tim Chen
2012-10-06 0:11 ` Andi Kleen
2012-10-08 13:44 ` Don Morris
2012-10-08 20:34 ` Rik van Riel
[not found] ` <20121011101930.GM3317@csn.ul.ie>
2012-10-11 14:56 ` Andrea Arcangeli
2012-10-11 15:35 ` Mel Gorman
2012-10-12 0:41 ` Andrea Arcangeli
2012-10-12 14:54 ` Mel Gorman
[not found] ` <1349308275-2174-2-git-send-email-aarcange@redhat.com>
[not found] ` <20121011105036.GN3317@csn.ul.ie>
2012-10-11 16:07 ` [PATCH 01/33] autonuma: add Documentation/vm/autonuma.txt Andrea Arcangeli
2012-10-11 19:37 ` Mel Gorman
[not found] ` <1349308275-2174-5-git-send-email-aarcange@redhat.com>
[not found] ` <20121011110137.GQ3317@csn.ul.ie>
2012-10-11 16:43 ` [PATCH 04/33] autonuma: define _PAGE_NUMA Andrea Arcangeli
2012-10-11 19:48 ` Mel Gorman
[not found] ` <1349308275-2174-6-git-send-email-aarcange@redhat.com>
[not found] ` <20121011111545.GR3317@csn.ul.ie>
2012-10-11 16:58 ` [PATCH 05/33] autonuma: pte_numa() and pmd_numa() Andrea Arcangeli
2012-10-11 19:54 ` Mel Gorman [this message]
[not found] ` <1349308275-2174-7-git-send-email-aarcange@redhat.com>
[not found] ` <20121011122255.GS3317@csn.ul.ie>
2012-10-11 17:05 ` [PATCH 06/33] autonuma: teach gup_fast about pmd_numa Andrea Arcangeli
2012-10-11 20:01 ` Mel Gorman
[not found] ` <1349308275-2174-8-git-send-email-aarcange@redhat.com>
[not found] ` <20121011122827.GT3317@csn.ul.ie>
2012-10-11 17:15 ` [PATCH 07/33] autonuma: mm_autonuma and task_autonuma data structures Andrea Arcangeli
2012-10-11 20:06 ` Mel Gorman
[not found] ` <5076E4B2.2040301@redhat.com>
[not found] ` <0000013a525a8739-2b4049fa-1cb3-4b8f-b3a7-1fa77b181590-000000@email.amazonses.com>
2012-10-12 0:52 ` Andrea Arcangeli
[not found] ` <1349308275-2174-9-git-send-email-aarcange@redhat.com>
[not found] ` <20121011134643.GU3317@csn.ul.ie>
2012-10-11 17:34 ` [PATCH 08/33] autonuma: define the autonuma flags Andrea Arcangeli
2012-10-11 20:17 ` Mel Gorman
[not found] ` <1349308275-2174-11-git-send-email-aarcange@redhat.com>
[not found] ` <20121011145805.GW3317@csn.ul.ie>
2012-10-12 0:25 ` [PATCH 10/33] autonuma: CPU follows memory algorithm Andrea Arcangeli
2012-10-12 8:29 ` Mel Gorman
[not found] ` <20121011213432.GQ3317@csn.ul.ie>
2012-10-12 1:45 ` [PATCH 00/33] AutoNUMA27 Andrea Arcangeli
2012-10-12 8:46 ` Mel Gorman
[not found] ` <1349308275-2174-16-git-send-email-aarcange@redhat.com>
[not found] ` <20121011155302.GA3317@csn.ul.ie>
[not found] ` <50770314.7060800@redhat.com>
[not found] ` <20121011175953.GT1818@redhat.com>
2012-10-12 14:03 ` [PATCH 15/33] autonuma: alloc/free/init task_autonuma Rik van Riel
2012-10-13 18:40 ` [PATCH 00/33] AutoNUMA27 Srikar Dronamraju
2012-10-14 4:57 ` Andrea Arcangeli
2012-10-15 8:16 ` Srikar Dronamraju
2012-10-23 16:32 ` Srikar Dronamraju
[not found] ` <1349308275-2174-20-git-send-email-aarcange@redhat.com>
[not found] ` <20121013180618.GC31442@linux.vnet.ibm.com>
2012-10-15 8:24 ` [PATCH 19/33] autonuma: memory follows CPU algorithm and task/mm_autonuma stats collection Srikar Dronamraju
2012-10-15 9:20 ` Mel Gorman
2012-10-15 10:00 ` Srikar Dronamraju
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121011195450.GM3317@csn.ul.ie \
--to=mel@csn.ul.ie \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=danms@us.ibm.com \
--cc=dhillf@gmail.com \
--cc=drjones@redhat.com \
--cc=efault@gmx.de \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@elte.hu \
--cc=paulmck@linux.vnet.ibm.com \
--cc=pjt@google.com \
--cc=pzijlstr@redhat.com \
--cc=riel@redhat.com \
--cc=suresh.b.siddha@intel.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®