From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84D64C43387 for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5CBEF20665 for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730793AbfAIMUm (ORCPT ); Wed, 9 Jan 2019 07:20:42 -0500 Received: from relay.sw.ru ([185.231.240.75]:42330 "EHLO relay.sw.ru" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730761AbfAIMUj (ORCPT ); Wed, 9 Jan 2019 07:20:39 -0500 Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1ghCqJ-0004gq-0w; Wed, 09 Jan 2019 15:20:19 +0300 Content-Transfer-Encoding: 7bit Subject: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction From: Kirill Tkhai To: akpm@linux-foundation.org, hannes@cmpxchg.org, josef@toxicpanda.com, jack@suse.cz, hughd@google.com, ktkhai@virtuozzo.com, darrick.wong@oracle.com, mhocko@suse.com, aryabinin@virtuozzo.com, guro@fb.com, mgorman@techsingularity.net, shakeelb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 09 Jan 2019 15:20:18 +0300 Message-ID: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On nodes without memory overcommit, it's common a situation, when memcg exceeds its limit and pages from pagecache are shrinked on reclaim, while node has a lot of free memory. Further access to the pages requires real device IO, while IO causes time delays, worse powerusage, worse throughput for other users of the device, etc. Cleancache is not a good solution for this problem, since it implies copying of page on every cleancache_put_page() and cleancache_get_page(). Also, it requires introduction of internal per-cleancache_ops data structures to manage cached pages and their inodes relationships, which again introduces overhead. This patchset introduces another solution. It introduces a new scheme for evicting memcg pages: 1)__remove_mapping() uncharges unmapped page memcg and leaves page in pagecache on memcg reclaim; 2)putback_lru_page() places page into root_mem_cgroup list, since its memcg is NULL. Page may be evicted on global reclaim (and this will be easily, as page is not mapped, so shrinker will shrink it with 100% probability of success); 3)pagecache_get_page() charges page into memcg of a task, which takes it first. Below is small test, which shows profit of the patchset. Create memcg with limit 20M (exact value does not matter much): $ mkdir /sys/fs/cgroup/memory/ct $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes $ echo $$ > /sys/fs/cgroup/memory/ct/tasks Then twice read 1GB file: $ time cat file_1gb > /dev/null Before (2 iterations): 1)0.01user 0.82system 0:11.16elapsed 7%CPU 2)0.01user 0.91system 0:11.16elapsed 8%CPU After (2 iterations): 1)0.01user 0.57system 0:11.31elapsed 5%CPU 2)0.00user 0.28system 0:00.28elapsed 100%CPU With the patch set applied, we have file pages are cached during the second read, so the result is 39 times faster. This may be useful for slow disks, NFS, nodes without overcommit by memory, in case of two memcg access the same files, etc. --- Kirill Tkhai (3): mm: Uncharge and keep page in pagecache on memcg reclaim mm: Recharge page memcg on first get from pagecache mm: Pass FGP_NOWAIT in generic_file_buffered_read and enable ext4 fs/ext4/inode.c | 1 + include/linux/pagemap.h | 1 + mm/filemap.c | 38 ++++++++++++++++++++++++++++++++++++-- mm/vmscan.c | 22 ++++++++++++++++++---- 4 files changed, 56 insertions(+), 6 deletions(-) -- Signed-off-by: Kirill Tkhai