From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0B63C47071 for ; Wed, 15 Nov 2023 03:00:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229596AbjKODAr (ORCPT ); Tue, 14 Nov 2023 22:00:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234288AbjKODAo (ORCPT ); Tue, 14 Nov 2023 22:00:44 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64C08E1 for ; Tue, 14 Nov 2023 19:00:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700017240; x=1731553240; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=0ABMAvlua1+dMNScNiJgwLMEsJzQQD0nYg82pujlRYY=; b=ISnSwe6l9eSX2n0szcUMw/6qwE29FiW83LltCXxWMfPa+K4uHFMxIV89 /jJfOVAC4dRv+5hhWNq65lifrIdutahtZxLhSQHyxNiwvnCU6ZkgqCgB0 8TbJePnxfsgdZjgfqYPgGca1Nl4nmiM4zcHqtjMKIiNyjXkrCjXNorHHe E2XHrLjcC4ncUP9Cw4BWlXQkcvi1ejpB7jeBWUkZ2NYSxC73jgpqFA1yX m3GQMhvb45prxVvVOhV7/kzZeCowWoz5oaanDCZGahWhPPjyGsA57BN/A RgC3N4Dkh57fqYM/1QB0CsDwoBhpDzFR4+i4qYdWHnKEiNM1/PhDygxTC w==; X-IronPort-AV: E=McAfee;i="6600,9927,10894"; a="394714510" X-IronPort-AV: E=Sophos;i="6.03,303,1694761200"; d="scan'208";a="394714510" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2023 19:00:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10894"; a="1012127430" X-IronPort-AV: E=Sophos;i="6.03,303,1694761200"; d="scan'208";a="1012127430" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2023 19:00:35 -0800 From: "Huang, Ying" To: Baolin Wang Cc: David Hildenbrand , , , , , , John Hubbard , Mel Gorman , Peter Zijlstra Subject: Re: [RFC PATCH] mm: support large folio numa balancing In-Reply-To: (Baolin Wang's message of "Tue, 14 Nov 2023 19:11:56 +0800") References: <606d2d7a-d937-4ffe-a6f2-dfe3ae5a0c91@redhat.com> <871qctf89m.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 15 Nov 2023 10:58:32 +0800 Message-ID: <87sf57en8n.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Baolin Wang writes: > On 11/14/2023 9:12 AM, Huang, Ying wrote: >> David Hildenbrand writes: >> >>> On 13.11.23 11:45, Baolin Wang wrote: >>>> Currently, the file pages already support large folio, and supporting for >>>> anonymous pages is also under discussion[1]. Moreover, the numa balancing >>>> code are converted to use a folio by previous thread[2], and the migrate_pages >>>> function also already supports the large folio migration. >>>> So now I did not see any reason to continue restricting NUMA >>>> balancing for >>>> large folio. >>> >>> I recall John wanted to look into that. CCing him. >>> >>> I'll note that the "head page mapcount" heuristic to detect sharers will >>> now strike on the PTE path and make us believe that a large folios is >>> exclusive, although it isn't. >> Even 4k folio may be shared by multiple processes/threads. So, numa >> balancing uses a multi-stage node selection algorithm (mostly >> implemented in should_numa_migrate_memory()) to identify shared folios. >> I think that the algorithm needs to be adjusted for PTE mapped large >> folio for shared folios. > > Not sure I get you here. In should_numa_migrate_memory(), it will use > last CPU id, last PID and group numa faults to determine if this page > can be migrated to the target node. So for large folio, a precise > folio sharers check can make the numa faults of a group more accurate, > which is enough for should_numa_migrate_memory() to make a decision? A large folio that is mapped by multiple process may be accessed by one remote NUMA node, so we still want to migrate it. A large folio that is mapped by one process but accessed by multiple threads on multiple NUMA node may be not migrated. > Could you provide a more detailed description of the algorithm you > would like to change for large folio? Thanks. I haven't thought about that thoroughly. So, please evaluate the algorithm by yourself. For example, the 2 sub-pages of a shared PTE-mapped large folio may be accessed together by a task. This may cause the folio be migrated wrongly. One possible solution is to restore all other PTE mappings of the large folio in do_numa_page() as the first step. This resembles the PMD-mapped THP behavior. >> And, as a performance improvement patch, some performance data needs to > > Do you have some benchmark recommendation? I know the the autonuma can > not support large folio now. There are autonuma-benchmark, and specjbb is used by someone before. >> be provided. And, the effect of shared folio detection needs to be >> tested too -- Best Regards, Huang, Ying