From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756092AbbHFRp6 (ORCPT ); Thu, 6 Aug 2015 13:45:58 -0400 Received: from mga02.intel.com ([134.134.136.20]:7115 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S964798AbbHFRnn (ORCPT ); Thu, 6 Aug 2015 13:43:43 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,623,1432623600"; d="scan'208";a="620637962" From: Ross Zwisler To: linux-kernel@vger.kernel.org, linux-nvdimm@ml01.01.org, dan.j.williams@intel.com Cc: Ross Zwisler , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Borislav Petkov , Toshi Kani , "Luis R. Rodriguez" Subject: [PATCH 3/6] x86: add clwb_cache_range() Date: Thu, 6 Aug 2015 11:43:17 -0600 Message-Id: <1438883000-9011-4-git-send-email-ross.zwisler@linux.intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1438883000-9011-1-git-send-email-ross.zwisler@linux.intel.com> References: <1438883000-9011-1-git-send-email-ross.zwisler@linux.intel.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add support for writing back a cache range using CLWB instead of just flushing it using CLFLUSH or CLFLUSHOPT. This allows you to ensure that your data has become durable on your DIMM, but potentially leaves a clean version in the processor cache hierarchy for future loads. This will be used in DAX to write back stores to persistent memory. Details on CLWB can be found here: https://software.intel.com/sites/default/files/managed/0d/53/319433-022.pdf Signed-off-by: Ross Zwisler --- arch/x86/include/asm/cacheflush.h | 1 + arch/x86/mm/pageattr.c | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/arch/x86/include/asm/cacheflush.h b/arch/x86/include/asm/cacheflush.h index ae00766..490b3d6 100644 --- a/arch/x86/include/asm/cacheflush.h +++ b/arch/x86/include/asm/cacheflush.h @@ -88,6 +88,7 @@ int set_pages_rw(struct page *page, int numpages); void clflush_cache_range(void *addr, unsigned int size); +void clwb_cache_range(void *addr, size_t size); #ifdef CONFIG_DEBUG_RODATA void mark_rodata_ro(void); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 727158c..ce84d05 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -144,6 +144,29 @@ void clflush_cache_range(void *vaddr, unsigned int size) } EXPORT_SYMBOL_GPL(clflush_cache_range); +/** + * clwb_cache_range - write back a cache range with clwb + * @vaddr: virtual start address + * @size: number of bytes to write back + * + * clwb is an unordered instruction which needs fencing with mfence or sfence + * to avoid ordering issues. + */ +void clwb_cache_range(void *vaddr, size_t size) +{ + u16 x86_clflush_size = boot_cpu_data.x86_clflush_size; + unsigned long clflush_mask = x86_clflush_size - 1; + char *vend = (char *)vaddr + size; + char *p; + + for (p = (char *)((unsigned long)vaddr & ~clflush_mask); + p < vend; p += x86_clflush_size) + clwb(p); + + wmb(); +} +EXPORT_SYMBOL_GPL(clwb_cache_range); + static void __cpa_flush_all(void *arg) { unsigned long cache = (unsigned long)arg; -- 2.1.0