From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F635C433EF for ; Sat, 2 Apr 2022 08:50:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352989AbiDBIwH (ORCPT ); Sat, 2 Apr 2022 04:52:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230100AbiDBIwD (ORCPT ); Sat, 2 Apr 2022 04:52:03 -0400 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 910306380 for ; Sat, 2 Apr 2022 01:50:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648889412; x=1680425412; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=y7sSkd9UD22NCQ4FcQr0EgofGUC0OSFealjI83DxPQc=; b=SZVcy5yavA755IeqdjXXDGQU5t2XKmMwZt5L0p4XJ96tlQXHvnVfkigA RDAFdjFC6xPu3uEP4N6B7J3aoTdQW93xr5UxfuCKOQz2+X/zvPYmffj2R OjS/gzcA4xf9bslBBVNhda0Aq6wGYBuOeznnKVLS7ahA2VMztFrREnvcf ShxnoJg2iAOjHMLithTVgRydxQPkeErNV6d0L0PWWlvkK7knLkT7usddv +OatqCwnmBGSKGkSb3Hczgttm2XgQWwB+9kOOTpcnCJ/C2wZZfo15AOzB tDtTIlvGpq6rVeqzAQqLBndwrYyFl773MTUmyYHeg+OFIaqitkYxWQxQj w==; X-IronPort-AV: E=McAfee;i="6200,9189,10304"; a="240876820" X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="240876820" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2022 01:50:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,229,1643702400"; d="scan'208";a="568141682" Received: from shbuild999.sh.intel.com (HELO localhost) ([10.239.146.138]) by orsmga008.jf.intel.com with ESMTP; 02 Apr 2022 01:50:06 -0700 Date: Sat, 2 Apr 2022 16:50:05 +0800 From: Feng Tang To: Linus Torvalds Cc: kernel test robot , Yang Shi , Baolin Wang , Johannes Weiner , Oscar Salvador , Michal Hocko , Rik van Riel , Mel Gorman , Peter Zijlstra , Dave Hansen , Zi Yan , Wei Xu , Shakeel Butt , zhongjiang-ali , Randy Dunlap , Andrew Morton , LKML , lkp@lists.01.org, kernel test robot , "Huang, Ying" , Zhengjun Xing , fengwei.yin@intel.com Subject: Re: [NUMA Balancing] e39bb6be9f: will-it-scale.per_thread_ops 64.4% improvement Message-ID: <20220402085005.GC32311@shbuild999.sh.intel.com> References: <20220401094214.GA8368@xsang-OptiPlex-9020> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Linus, On Fri, Apr 01, 2022 at 09:35:24AM -0700, Linus Torvalds wrote: > On Fri, Apr 1, 2022 at 2:42 AM kernel test robot wrote: > > > > FYI, we noticed a 64.4% improvement of will-it-scale.per_thread_ops due to commit: > > e39bb6be9f2b ("NUMA Balancing: add page promotion counter") > > That looks odd and unlikely. > > That commit only modifies some page counting statistics. Sure, it > could be another cache layout thing, and maybe it's due to the subtle > change in how NUMA_PAGE_MIGRATE gets counted, but it still looks a bit > odd. We did a quick check about cache stuff by disabling HW cache prefetch completely (writing 0xf to MSR 0x1a4), and the performance change is almost gone: ee97347fe058d020 e39bb6be9f2b39a6dbaeff48436 ---------------- --------------------------- 134793 -1.4% 132867 will-it-scale.per_thread_ops The test box is a Cascadelake machine with 4 nodes, and the similar trend is found on a 2 nodes machine, that the commit has 55% improvement with HW cache prefetch enabled, and has less than 1% change when disabled. Though we still cannot pin-point the exact place affected. Also per our experience, the patch changing vm statistics can easily trigger strange performance bumps for micro-benchmarks like will-it-scale, stress-ng etc. Thanks, Feng > Linus