From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753177AbbAEBgM (ORCPT ); Sun, 4 Jan 2015 20:36:12 -0500 Received: from lgeamrelo04.lge.com ([156.147.1.127]:45850 "EHLO lgeamrelo04.lge.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753119AbbAEBgK (ORCPT ); Sun, 4 Jan 2015 20:36:10 -0500 X-Original-SENDERIP: 10.177.220.158 X-Original-MAILFROM: iamjoonsoo.kim@lge.com From: Joonsoo Kim To: Andrew Morton Cc: Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jesper Dangaard Brouer , rostedt@goodmis.org, Thomas Gleixner Subject: [PATCH 2/2] mm: don't use compound_head() in virt_to_head_page() Date: Mon, 5 Jan 2015 10:36:05 +0900 Message-Id: <1420421765-3209-2-git-send-email-iamjoonsoo.kim@lge.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1420421765-3209-1-git-send-email-iamjoonsoo.kim@lge.com> References: <1420421765-3209-1-git-send-email-iamjoonsoo.kim@lge.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org compound_head() is implemented with assumption that there would be race condition when checking tail flag. This assumption is only true when we try to access arbitrary positioned struct page. The situation that virt_to_head_page() is called is different case. We call virt_to_head_page() only in the range of allocated pages, so there is no race condition on tail flag. In this case, we don't need to handle race condition and we can reduce overhead slightly. This patch implements compound_head_fast() which is similar with compound_head() except tail flag race handling. And then, virt_to_head_page() uses this optimized function to improve performance. I saw 1.8% win in a fast-path loop over kmem_cache_alloc/free, (14.063 ns -> 13.810 ns) if target object is on tail page. Signed-off-by: Joonsoo Kim --- include/linux/mm.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index f80d019..0460e2e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -453,6 +453,13 @@ static inline struct page *compound_head(struct page *page) return page; } +static inline struct page *compound_head_fast(struct page *page) +{ + if (unlikely(PageTail(page))) + return page->first_page; + return page; +} + /* * The atomic page->_mapcount, starts from -1: so that transitions * both from it and to it can be tracked, using atomic_inc_and_test @@ -531,7 +538,8 @@ static inline void get_page(struct page *page) static inline struct page *virt_to_head_page(const void *x) { struct page *page = virt_to_page(x); - return compound_head(page); + + return compound_head_fast(page); } /* -- 1.7.9.5