From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S269682AbUIRXha (ORCPT ); Sat, 18 Sep 2004 19:37:30 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S269680AbUIRXaM (ORCPT ); Sat, 18 Sep 2004 19:30:12 -0400 Received: from omx1-ext.sgi.com ([192.48.179.11]:53714 "EHLO omx1.americas.sgi.com") by vger.kernel.org with ESMTP id S269671AbUIRX15 (ORCPT ); Sat, 18 Sep 2004 19:27:57 -0400 Date: Sat, 18 Sep 2004 16:27:34 -0700 (PDT) From: Christoph Lameter X-X-Sender: clameter@schroedinger.engr.sgi.com To: akpm@osdl.org cc: "David S. Miller" , benh@kernel.crashing.org, wli@holomorphy.com, davem@redhat.com, raybry@sgi.com, ak@muc.de, manfred@colorfullife.com, linux-ia64@vger.kernel.org, linux-kernel@vger.kernel.org, vrajesh@umich.edu, hugh@veritas.com Subject: page fault scalability patch V8: [3/7] atomic pte operations for ia64 In-Reply-To: Message-ID: References: <20040816143903.GY11200@holomorphy.com> <1094012689.6538.330.camel@gaston> <1094080164.4025.17.camel@gaston> <20040901215741.3538bbf4.davem@davemloft.net> <20040902131057.0341e337.davem@davemloft.net> <20040902140759.5f1003d5.davem@davemloft.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Changelog * Provide atomic pte operations for ia64 * Enhanced parallelism in page fault handler if applied together with the generic patch Signed-off-by: Christoph Lameter Index: linus/include/asm-ia64/pgalloc.h =================================================================== --- linus.orig/include/asm-ia64/pgalloc.h 2004-09-18 14:25:23.000000000 -0700 +++ linus/include/asm-ia64/pgalloc.h 2004-09-18 15:43:25.000000000 -0700 @@ -34,6 +34,10 @@ #define pmd_quicklist (local_cpu_data->pmd_quick) #define pgtable_cache_size (local_cpu_data->pgtable_cache_sz) +/* Empty entries of PMD and PGD */ +#define PMD_NONE 0 +#define PGD_NONE 0 + static inline pgd_t* pgd_alloc_one_fast (struct mm_struct *mm) { @@ -78,12 +82,19 @@ preempt_enable(); } + static inline void pgd_populate (struct mm_struct *mm, pgd_t *pgd_entry, pmd_t *pmd) { pgd_val(*pgd_entry) = __pa(pmd); } +/* Atomic populate */ +static inline int +pgd_test_and_populate (struct mm_struct *mm, pgd_t *pgd_entry, pmd_t *pmd) +{ + return ia64_cmpxchg8_acq(pgd_entry,__pa(pmd), PGD_NONE) == PGD_NONE; +} static inline pmd_t* pmd_alloc_one_fast (struct mm_struct *mm, unsigned long addr) @@ -132,6 +143,13 @@ pmd_val(*pmd_entry) = page_to_phys(pte); } +/* Atomic populate */ +static inline int +pmd_test_and_populate (struct mm_struct *mm, pmd_t *pmd_entry, struct page *pte) +{ + return ia64_cmpxchg8_acq(pmd_entry, page_to_phys(pte), PMD_NONE) == PMD_NONE; +} + static inline void pmd_populate_kernel (struct mm_struct *mm, pmd_t *pmd_entry, pte_t *pte) { Index: linus/include/asm-ia64/pgtable.h =================================================================== --- linus.orig/include/asm-ia64/pgtable.h 2004-09-18 14:25:23.000000000 -0700 +++ linus/include/asm-ia64/pgtable.h 2004-09-18 15:43:25.000000000 -0700 @@ -423,6 +423,19 @@ extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern void paging_init (void); +/* Atomic PTE operations */ +static inline pte_t +ptep_xchg (struct mm_struct *mm, pte_t *ptep, pte_t pteval) +{ + return __pte(xchg((long *) ptep, pteval.pte)); +} + +static inline int +ptep_cmpxchg (struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t oldval, pte_t newval) +{ + return ia64_cmpxchg8_acq(&ptep->pte, newval.pte, oldval.pte) == oldval.pte; +} + /* * Note: The macros below rely on the fact that MAX_SWAPFILES_SHIFT <= number of * bits in the swap-type field of the swap pte. It would be nice to @@ -558,6 +571,7 @@ #define __HAVE_ARCH_PTEP_MKDIRTY #define __HAVE_ARCH_PTE_SAME #define __HAVE_ARCH_PGD_OFFSET_GATE +#define __HAVE_ARCH_ATOMIC_TABLE_OPS #include #endif /* _ASM_IA64_PGTABLE_H */