From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74D65C433EF for ; Fri, 25 Feb 2022 05:22:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237525AbiBYFWn (ORCPT ); Fri, 25 Feb 2022 00:22:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237485AbiBYFWN (ORCPT ); Fri, 25 Feb 2022 00:22:13 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6BDB366B6 for ; Thu, 24 Feb 2022 21:21:40 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21P4jfZf006248; Fri, 25 Feb 2022 05:21:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=H/MTTrg1Y1OvDYnkNIyADXVI0viOQp8B7sbZh21c2lU=; b=dfKhMkF9Dx6tATkf/yJ8rfw2CDMciwsH9KCxm3DxRJpfm6LyeOaFbrm4SMiIu+5tMxCI LoSaPol6cIGsS0wJNEORbF/zAKJg0sI36qSSYcS52BjD1tn4dtFBFYUO0TYOnrH2pBEf tclPm1zD0XH4AlsNY9QRd8rYNWNdL3SiLqMfVdu5X+D5w7LGt3yzf1YwMPKF431GP+5e odmpq+B+lgsHMHU3Pv7FLtEpR7mrkfN2aUhpcMHxFioMojj5U96Lob3EXbcFaTL4RW+a RXkPytsD/JQVRNDrJgMiedW7D4VErF883oE2AJlcs3sYz1YXoV2uiG2q2hbQZN4OtKPC tQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx9a5k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Feb 2022 05:21:37 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21P5AxUp181372; Fri, 25 Feb 2022 05:21:36 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2109.outbound.protection.outlook.com [104.47.58.109]) by aserp3030.oracle.com with ESMTP id 3eapkm8a45-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Feb 2022 05:21:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aChM7DpW8nclij8yt2blEBKYF6SRmeMxQ9uN555QMX4VqOQ9PKCt+42xLZXyLPbHsLkA5z2dWy/M/YkcbPBpgikPiKrIuU1WQ3AOCgVG3LnPGkvn929vc2xEND7UpGXsByDzBM21S4uwZF0ruUOjgnDWd68jR5SnOt/5a8ujBKoZ1hkrUV577Drc7OY+YaATBFQV/9HSylyJOeCmCmUdsDZezXC/erXD4hGceqdjm6c6KO5+IfBQfe12k+BmHXkVGLqBEKcerQB7V2AhLCjqnjHIwOuEMHYRlAs9byc6qgG07M/J5AbgVJse+eaEvBSnUfs6zsBVddFkb1+5eybz3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H/MTTrg1Y1OvDYnkNIyADXVI0viOQp8B7sbZh21c2lU=; b=UIr+r7bZs1FrQXD4cFwwfaQS2XOufbLDsQA2B1WvkVTFCzqM7zS6qd9AUwTJJFWxrHF1q4AKetoDQVABBEgLeAcu3imzXo5ZxXXHDjZdpcpVVNex5ObMUQOaSLV/6CeOpFnMpi2D+ZiNuEnpGBofuVutALZeotQYiZq6QKsVy+hDcBvvqEFbttIRJFAgv7kj+tPE7TeK3YKcMe2lpqK07ywYunqT2uiFo0aMZ6lnkzls/wU659QDqUhsl7V+Ekdpt8vrQBgh4DIhHPk7obn/kf3afd1B9Xcmz9zEm3LTYnzDqZPdkfToUz4Bf2wt0hzpBEBg+YNxsIThPSiP5IEVOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H/MTTrg1Y1OvDYnkNIyADXVI0viOQp8B7sbZh21c2lU=; b=U+64TEDrLGEtlXj54/gFxdDshrxrIpwitXYdGRFdhzm/0GoJdX5oqSmvEyMzRvRO0/2vjQgGTQRuQTxES82OW7fGy+p0/yfXqmEKa6fNXC1sLULQ7q7GAFh5A5q53V81cLyyRHlPaaefVNR8Wop5aQMJH55gViIsOy/mva0Fnv0= Received: from CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) by MWHPR10MB1677.namprd10.prod.outlook.com (2603:10b6:301:a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.24; Fri, 25 Feb 2022 05:21:34 +0000 Received: from CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::b5ab:1c3e:6540:d2fa]) by CO1PR10MB4468.namprd10.prod.outlook.com ([fe80::b5ab:1c3e:6540:d2fa%9]) with mapi id 15.20.5017.025; Fri, 25 Feb 2022 05:21:34 +0000 From: Imran Khan To: tj@kernel.org, gregkh@linuxfoundation.org, viro@zeniv.linux.org.uk Cc: linux-kernel@vger.kernel.org Subject: [PATCH v7 4/8] kernfs: Replace global kernfs_open_node_lock with hashed spinlocks. Date: Fri, 25 Feb 2022 16:21:12 +1100 Message-Id: <20220225052116.1243150-5-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220225052116.1243150-1-imran.f.khan@oracle.com> References: <20220225052116.1243150-1-imran.f.khan@oracle.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SYAPR01CA0043.ausprd01.prod.outlook.com (2603:10c6:1:1::31) To CO1PR10MB4468.namprd10.prod.outlook.com (2603:10b6:303:6c::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 42ece961-5a12-48e7-fb8e-08d9f81eb072 X-MS-TrafficTypeDiagnostic: MWHPR10MB1677:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: l6oiVYLbd3rSmQTWzfc1hyakHerthXxEU0C+dkJMEka9JhcLrwkiL7PzJQs1fAzif44Yja/m62nYKtEPNDE7a8BuEvRm3f5gEmat8+VUjmIcMc0vpJlhaA0h7LkRUNgu/PodL2Vshsk4HPLREkZOOjcozCdSpadlJwJBMmZgB8rr+/BlG+3WCgeG6blm1sNwAl4Y27LwU5XZ/WZK7+1u+2N0MoCDnt861m6cuLGj/Hv7w6CECu6ecFnNSIRjGUF53PF83EQbBYnxtTWa9sPfA57AG4RzcayQkT580MyB3nP8uqNoCeYf8n/eV4V3t0VdJSgpbpLh6n04wIKZcKYXtdfYoX9CHSmM6HMNdmlV/tOhTR0lF3UX/7HAtEdBkrRRJyafW+rJCTKkWBdMybTVHr/ApZs7sKGmWGt5OkckaetmAJOnxDnAkDseuTu1qStdpd9CtGhH5w3T1A6tpX0ETEvRpHP7XiMi1ywkjXWMzanhuBp5xhlswa944Kngwy1cGU3whlUnzKy+BJhg5q6qxhcOvH654RfFXhqemzuqAZEWsJhqpkliVs3FW+VmWyyjc4VwknBW1/+l6tibaDDer1wZBf9SPpLqL4nGZ78i+I74zA9oOkfJMQFPfCGo2nxPkSc0QLcrO2vHB74kXDb50BtQgSl/Yry/cPvJxh8yS+50XoVLKrW7PofA/oYR8a9q+e85OfL/MrXjyvnqfXmMvg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR10MB4468.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(186003)(26005)(38350700002)(6666004)(8936002)(38100700002)(83380400001)(1076003)(6512007)(508600001)(86362001)(6486002)(6506007)(2906002)(52116002)(316002)(66946007)(36756003)(4326008)(5660300002)(8676002)(66476007)(103116003)(66556008)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dKq4BHBgDc+X4CCDbT/V+sm4F/QOszm2sq6ym9qj8SWMfPUjJa34Y4ccrcYD?= =?us-ascii?Q?dmt3ldpzHVZUEFe8IMplnrmgCqY4iCWFeVN95EHI1kbGtvs/wm0qVqnCpmEn?= =?us-ascii?Q?LjgBeYuRQqQcZZGMHetCtITGgQKgODiDd1VzKBcbgc1U+n7ZWfh3GEr6AAfv?= =?us-ascii?Q?AcruDDXbXL5juWKQ05dzVx7ZwgK48SKgQ3lKvWqC1ce8oB/8dsPz7JOX9WJC?= =?us-ascii?Q?nkUNG9tt0Sp5miPsFhmR96KyoGPyQiJCNZV0gqa0EBAVvdldiwieyJZ40YSW?= =?us-ascii?Q?U/TI17/1FeDEO/elreOgeWxxJcQ+AOvc0v3J9Wl+7fVaZy84G5wQOmuCmQwV?= =?us-ascii?Q?uot7saQ2fYRSL+V6F+405k1kAvg8plponNbtO6jbxYnsomHEihlcqbX6+D6n?= =?us-ascii?Q?uWYCkfFdEoGJ+ntA9Mm8P7BxfriFfUoQEQrT2ByO/WkpwuCIrkO0t9i2X0pL?= =?us-ascii?Q?SZ/HCwyqg631iJuYJYAkdjnE73rtqjPUNJmULoJxozY7QoZ2kH5rbxk+uiV/?= =?us-ascii?Q?KzjdZfeWFqmQCKCJSDTe6enz4SUnOPBvh3tE20FS29oNAeqCLpJw4l6zdkoB?= =?us-ascii?Q?M7I/br+MlKmYHqIaTaN8xBDYBFOCbVvwJtQOcb7b2lN19nLNT5u03+Wd+tJ8?= =?us-ascii?Q?RXNwr/J3X3m9HNZyVH9q7u6NZJnFVpOaN1TMATCGNrRwi1XHSqUylgdS+5jn?= =?us-ascii?Q?Oe8e8vs83ZXsDh35QEYaOG5BwMRLbX4hzSCD8NvYKQcBpXVNQlGJAz0mYgon?= =?us-ascii?Q?9Ll/QIhkzjzN2ikzkUDFP9fGY9V9LUkMQkI+lRrk392BpSkgdRDQLmg3UDRD?= =?us-ascii?Q?8YLdZvf9LKdWJi8bbGF4BC2rIClxwas+kpJlfeTQ0NtzOL69PtA5N4eWIrID?= =?us-ascii?Q?W4Q7CYvN0nGljLWe3JGJUFnxadIEK/aZKeWoBrQNcXWyKeH934Qa6rvF0tAA?= =?us-ascii?Q?9Jx1tLUOCI/5YmgFTMiUSr0WSV0/B9t7/0t22kBZ11pnqk9cNCGXaAiVlGJr?= =?us-ascii?Q?ZxA9DiUKQ5J/ATp9TLqUEbGy9soJVP1qCTCWQY+LTCOsS8ItuRdN06HfL/1L?= =?us-ascii?Q?4Wwj1inRMWuunE2a6uemKWI4UT4krj51qS2tCl9KMnoXYW2sMhwrGYww/mdc?= =?us-ascii?Q?qC22l+b9+bBHJ4KWrQ81r3byi5fDdNaZXQ6Oxpu1wskf4TxNQIJhlNKhyehW?= =?us-ascii?Q?3EYLw6gDg2v+R1O8Ad84ZLobm49GCRmQW4g3+NwSGcfaquKxICENdESNrIrq?= =?us-ascii?Q?6APHxivbHTYU3SzrFwUy/lidj4fQcL3kjY7/qnpLJIQI2TTWz7/rxlgOjBLY?= =?us-ascii?Q?xW1LDpsum9w6HAKU8Ak6mQTeCVXZ1FR/V0QJkAYAg/Jf4QH/5/Yn9wvs647s?= =?us-ascii?Q?LZl+CuA49/U7Wgb+f83axdrxleSTyDQaKfixjX4py1hRD9a5he4W24E7zbix?= =?us-ascii?Q?R8qxNt1lw7nKSNgkSj5lvW+K154m2Qw6h+yYTOXzZahEjjLzA3qm/Z8Rqxa8?= =?us-ascii?Q?ZP7BX7qWZ7V7KkwkHT9DunYgB+E5v09Av6flCUKHOJ96pHGTc7RG1j4JNabC?= =?us-ascii?Q?vfi7ruUy3CjVSsVSqZgHq0p49hT3smkJi2e5eZ1Ke1ossaS22k/+P8Ixk1B+?= =?us-ascii?Q?t4XIsc+soLrjurfm98rHis0=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42ece961-5a12-48e7-fb8e-08d9f81eb072 X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2022 05:21:34.3762 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qrF+4uS6OA1tW30/xKuzSSOg9ME0B0rbOcSAgXVLuAKy7v8OiawKDIR3fEzjlV9Lu89veYDqYsx8WJ3tr/aZ8Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR10MB1677 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10268 signatures=684655 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202250027 X-Proofpoint-ORIG-GUID: 6u2tXuXnV46ZREmcUsQda_UMWM6O9DoW X-Proofpoint-GUID: 6u2tXuXnV46ZREmcUsQda_UMWM6O9DoW Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In current kernfs design a single spinlock, kernfs_open_node_lock, protects the kernfs_node->attr.open i.e kernfs_open_node instances corresponding to a sysfs attribute. So separate tasks, which are opening or closing separate sysfs files, can contend on this spinlock. The contention is more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed spinlocks in place of a single global spinlock, can reduce contention around global spinlock and hence provide better scalability. Moreover as these hashed spinlocks are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. Modify interface introduced in previous patch to make use of hashed spinlocks. Use kernfs_node address as hashing key. Signed-off-by: Imran Khan --- fs/kernfs/file.c | 9 --------- fs/kernfs/kernfs-internal.h | 6 +++--- fs/kernfs/mount.c | 4 +++- include/linux/kernfs.h | 10 +++++++++- 4 files changed, 15 insertions(+), 14 deletions(-) diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 1658bfa048df3..95426df9f0304 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -18,15 +18,6 @@ #include "kernfs-internal.h" -/* - * kernfs_node->attr.open points to kernfs_open_node. attr.open is - * protected by kernfs_open_node_lock. - * - * filp->private_data points to seq_file whose ->private points to - * kernfs_open_file. - */ -DEFINE_SPINLOCK(kernfs_open_node_lock); - struct kernfs_open_node { atomic_t refcnt; atomic_t event; diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index ef5b04d43ef1b..64e9cca66d436 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -152,8 +152,6 @@ extern const struct inode_operations kernfs_symlink_iops; */ extern struct kernfs_global_locks *kernfs_locks; -extern spinlock_t kernfs_open_node_lock; - static inline struct mutex *kernfs_open_file_mutex_ptr(struct kernfs_node *kn) { int idx = hash_ptr(kn, NR_KERNFS_LOCK_BITS); @@ -174,7 +172,9 @@ static inline struct mutex *kernfs_open_file_mutex_lock(struct kernfs_node *kn) static inline spinlock_t *kernfs_open_node_spinlock_ptr(struct kernfs_node *kn) { - return &kernfs_open_node_lock; + int idx = hash_ptr(kn, NR_KERNFS_LOCK_BITS); + + return &kernfs_locks->open_node_locks[idx].lock; } static inline spinlock_t *kernfs_open_node_spinlock(struct kernfs_node *kn) diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index fa3fa22c95b21..809b738739b18 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -395,8 +395,10 @@ void __init kernfs_lock_init(void) kernfs_locks = kmalloc(sizeof(struct kernfs_global_locks), GFP_KERNEL); WARN_ON(!kernfs_locks); - for (count = 0; count < NR_KERNFS_LOCKS; count++) + for (count = 0; count < NR_KERNFS_LOCKS; count++) { mutex_init(&kernfs_locks->open_file_mutex[count].lock); + spin_lock_init(&kernfs_locks->open_node_locks[count].lock); + } } void __init kernfs_init(void) diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 3f72d38d48e31..e50528c45bcd4 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -19,6 +19,7 @@ #include #include #include +#include struct file; struct dentry; @@ -75,20 +76,27 @@ struct kernfs_iattrs; * kernfs_open_file. * kernfs_open_files are chained at kernfs_open_node->files, which is * protected by kernfs_open_file_mutex.lock. + * + * kernfs_node->attr.open points to kernfs_open_node. attr.open is + * protected by kernfs_open_node_lock.lock. */ struct kernfs_open_file_mutex { struct mutex lock; } ____cacheline_aligned_in_smp; +struct kernfs_open_node_lock { + spinlock_t lock; +} ____cacheline_aligned_in_smp; + /* * To reduce possible contention in sysfs access, arising due to single * locks, use an array of locks and use kernfs_node object address as * hash keys to get the index of these locks. */ - struct kernfs_global_locks { struct kernfs_open_file_mutex open_file_mutex[NR_KERNFS_LOCKS]; + struct kernfs_open_node_lock open_node_locks[NR_KERNFS_LOCKS]; }; enum kernfs_node_type { -- 2.30.2