From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9196DC7EE23 for ; Wed, 31 May 2023 22:30:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229925AbjEaWay (ORCPT ); Wed, 31 May 2023 18:30:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229672AbjEaWax (ORCPT ); Wed, 31 May 2023 18:30:53 -0400 Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5966D107 for ; Wed, 31 May 2023 15:30:52 -0700 (PDT) Received: by mail-oi1-x233.google.com with SMTP id 5614622812f47-38c35975545so43756b6e.1 for ; Wed, 31 May 2023 15:30:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20221208.gappssmtp.com; s=20221208; t=1685572251; x=1688164251; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/fxfV5VcNo1HuNl1BAlDq7x3auo0HGUulH0i8hWzvcQ=; b=aZHYvexed48WM5IBidrzNTTUHgyWvwj9jzFpiCEH/huHzSVxtvHsAWNH8JIHLFj67n 4G0IYsOzn+FciEUpUgWQoR1hQKb2q1dm7vWFlTY+EyzkYvPTuy8mNZ+f11iJsy8+yZJn EKyubvVaua/MsplXg1t04POt9rYmzvWnkNjuYZoeY9f6+1X9FPfewNrOX6YFLeip1dAs LdObE2wvIsCWYpbdflvb21BNLg/QneriumlKUBicYw22PNqInLjM4SVdeUMh128KMLjU poox4XkN5tobPAgLGJ9uN7SOXZn1VMv4XLq8GYmNr++ilqWGT8xIEuRed4oXZFiPSg6z 9ybw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685572251; x=1688164251; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=/fxfV5VcNo1HuNl1BAlDq7x3auo0HGUulH0i8hWzvcQ=; b=MmsIExheSkdZEdQCfO+MGqGUna2w3u5bU33wi+3p39kQKsebVtSpNFnIdDgwAKe5yb KXwvgheg7A4zIOPSZvJF9Dw2lHYBNSGJ4EEYz9QYXYExynSCq7jQnCUhOULJPH9uTK2B m8Jhc5nyzAdk/e9XMeLe2vQrocsS+YSQi8dzIuXX1/8LT+TSy69VVx1yY3IgignVLu9P QyEHWyAqfjBwoHMZnn7tmgdombFyRJ107NGEZrZm76Md/8lGIIMAOZON9A66mKP78XlI TEa4uJbGOcpJJtm1XtA1efWFy1In/62RU8plorlSamU4GmbWVmjKdipAHWfrnaBJJ+SK FC8w== X-Gm-Message-State: AC+VfDyfkGUWAakcGpIDAdNgEWeThBmmdX3q5L+wzjZaIRpxGO+j/jij 7JY2afYPEnOxKruQ/jB9WBGQyILyaSUfEbCgb8U= X-Google-Smtp-Source: ACHHUZ5eCf2tJFtsvUL8ppbW7rTCZstcQT9dxFkvwabquAdExDT8tCX3d8b1NHSMngPsfAhFMsLxwQ== X-Received: by 2002:a05:6808:3090:b0:395:f4fd:9fe9 with SMTP id bl16-20020a056808309000b00395f4fd9fe9mr6871739oib.50.1685572251471; Wed, 31 May 2023 15:30:51 -0700 (PDT) Received: from dread.disaster.area (pa49-179-0-188.pa.nsw.optusnet.com.au. [49.179.0.188]) by smtp.gmail.com with ESMTPSA id jd5-20020a170903260500b001a245b49731sm1908603plb.128.2023.05.31.15.30.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 May 2023 15:30:50 -0700 (PDT) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1q4ULA-006HBu-12; Thu, 01 Jun 2023 08:30:48 +1000 Date: Thu, 1 Jun 2023 08:30:48 +1000 From: Dave Chinner To: "Chen, Zhiyin" Cc: Eric Biggers , Christian Brauner , "viro@zeniv.linux.org.uk" , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "Zou, Nanhai" , "Feng, Xiaotian" Subject: Re: [PATCH] fs.h: Optimize file struct to prevent false sharing Message-ID: References: <20230530020626.186192-1-zhiyin.chen@intel.com> <20230530-wortbruch-extra-88399a74392e@brauner> <20230531015549.GA1648@quark.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 31, 2023 at 10:31:09AM +0000, Chen, Zhiyin wrote: > As Eric said, CONFIG_RANDSTRUCT_NONE is set in the default config > and some production environments, including Ali Cloud. Therefore, it > is worthful to optimize the file struct layout. > > Here are the syscall test results of unixbench. Results look good, but the devil is in the detail.... > Command: numactl -C 3-18 ./Run -c 16 syscall So the test is restricted to a set of adjacent cores within a single CPU socket, so all the cachelines are typically being shared within a single socket's CPU caches. IOWs, the fact there are 224 CPUs in the machine is largely irrelevant for this microbenchmark. i.e. is this a microbenchmark that is going faster simply because the working set for the specific benchmark now fits in L2 or L3 cache when it didn't before? Does this same result occur for different CPUs types, cache sizes and architectures? What about when the cores used by the benchmark are spread across mulitple sockets so the cost of remote cacheline access is taken into account? If this is actually a real benefit, then we should see similar or even larger gains between CPU cores that are further apart because the cost of false cacheline sharing are higher in those systems.... > Without patch > ------------------------ > 224 CPUs in system; running 16 parallel copies of tests > System Call Overhead 5611223.7 lps (10.0 s, 7 samples) > System Benchmarks Partial Index BASELINE RESULT INDEX > System Call Overhead 15000.0 5611223.7 3740.8 > ======== > System Benchmarks Index Score (Partial Only) 3740.8 > > With patch > ------------------------------------------------------------------------ > 224 CPUs in system; running 16 parallel copies of tests > System Call Overhead 7567076.6 lps (10.0 s, 7 samples) > System Benchmarks Partial Index BASELINE RESULT INDEX > System Call Overhead 15000.0 7567076.6 5044.7 > ======== > System Benchmarks Index Score (Partial Only) 5044.7 Where is all this CPU time being saved? Do you have a profile showing what functions in the kernel are running far more efficiently now? Yes, the results look good, but if all this change is doing is micro-optimising a single code path, it's much less impressive and far more likley that it has no impact on real-world performance... More information, please! -Dave. -- Dave Chinner david@fromorbit.com