From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S932421AbcHIRAa (ORCPT <rfc822;w@1wt.eu>);
	Tue, 9 Aug 2016 13:00:30 -0400
Received: from mga09.intel.com ([134.134.136.24]:55942 "EHLO mga09.intel.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1752346AbcHIRA3 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Tue, 9 Aug 2016 13:00:29 -0400
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.28,495,1464678000"; 
   d="scan'208";a="862445589"
From: "Huang\, Ying" <ying.huang@intel.com>
To: Dave Hansen <dave.hansen@intel.com>
Cc: "Huang\, Ying" <ying.huang@intel.com>,
        Andrew Morton <akpm@linux-foundation.org>, <tim.c.chen@intel.com>,
        <andi.kleen@intel.com>, <aaron.lu@intel.com>, <linux-mm@kvack.org>,
        <linux-kernel@vger.kernel.org>, Hugh Dickins <hughd@google.com>,
        Shaohua Li <shli@kernel.org>, Minchan Kim <minchan@kernel.org>,
        Rik van Riel <riel@redhat.com>,
        Mel Gorman <mgorman@techsingularity.net>, Tejun Heo <tj@kernel.org>,
        Wu Fengguang <fengguang.wu@intel.com>
Subject: Re: [RFC] mm: Don't use radix tree writeback tags for pages in swap cache
References: <1470759443-9229-1-git-send-email-ying.huang@intel.com>
	<57AA061B.2050002@intel.com>
Date: Tue, 09 Aug 2016 10:00:28 -0700
In-Reply-To: <57AA061B.2050002@intel.com> (Dave Hansen's message of "Tue, 9
	Aug 2016 09:34:35 -0700")
Message-ID: <87oa51513n.fsf@yhuang-mobile.sh.intel.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, Dave,

Dave Hansen <dave.hansen@intel.com> writes:

> On 08/09/2016 09:17 AM, Huang, Ying wrote:
>> File pages uses a set of radix tags (DIRTY, TOWRITE, WRITEBACK) to
>> accelerate finding the pages with the specific tag in the the radix tree
>> during writing back an inode.  But for anonymous pages in swap cache,
>> there are no inode based writeback.  So there is no need to find the
>> pages with some writeback tags in the radix tree.  It is no necessary to
>> touch radix tree writeback tags for pages in swap cache.
>
> Seems simple enough.  Do we do any of this unnecessary work for the
> other radix tree tags?  If so, maybe we should just fix this once and
> for all.  Could we, for instance, WARN_ONCE() in radix_tree_tag_set() if
> it sees a swap mapping get handed in there?

Good idea!  I will do that and try to catch other places if any.

> In any case, I think the new !PageSwapCache(page) check either needs
> commenting, or a common helper for the two sites that you can comment.

Sure.  I will add that.

>> With this patch, the swap out bandwidth improved 22.3% in vm-scalability
>> swap-w-seq test case with 8 processes on a Xeon E5 v3 system, because of
>> reduced contention on swap cache radix tree lock.  To test sequence swap
>> out, the test case uses 8 processes sequentially allocate and write to
>> anonymous pages until RAM and part of the swap device is used up.
>
> What was the swap device here, btw?  What is the actual bandwidth
> increase you are seeing?  Is it 1MB/s -> 1.223MB/s? :)

The swap device here is a DRAM simulated persistent memory block device
(pmem).

   1207402 ±  7%     +22.3%    1476578 ±  6%  vmstat.swap.so

The actual bandwidth increase is from 1.21GB/s -> 1.48 GB/s.  This is
lower than that of NVMe disk, so the bottleneck is in swap subsystem
instead of block subsystem and device.

Best Regards,
Huang, Ying