From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2076.outbound.protection.outlook.com [40.107.92.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C50F1A0730 for ; Wed, 8 Jan 2025 16:33:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.76 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736353989; cv=fail; b=OhjF0GQH8I0335xr+qIggSNNyy+ajwFJxBev4Kez8nlKSt6OUDU9EMZt8hNiqNXZ2atRu913EvhywuWh+r8BZH0r84q2AZWRsrzTIAbTw23PrwJNY02V+AzvSUXlDxFBsWNsXpOHF9br+QzHJRAekRaoudYqbtaHJ5InUk9hquU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736353989; c=relaxed/simple; bh=ee+V2OjR48P8l5MrF+2d4eEBoRWdE4CE9plY9eqhrAE=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=qk3Zp7CFDXtAgJR1gJDJbU3+39QgukoYSIRpZ+5WEXjwlRdIEWjh7zct8/Y3+9p0Ymoxx9ReqlqCbGJdLiEQq7GUaz2c226hTf/C/V0I1Zikk0pmUbZPJTIuWUwTm+jlVGKuxxlUkQP2MhE2CTFxDz0GcGI5VoE8OklIV5M0oiA= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=MCEOSCaQ; arc=fail smtp.client-ip=40.107.92.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="MCEOSCaQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TwH6Aa9hXk69/mzJe/vGiGBWdpQunj3sE6mtxxMxlpLS3LdiUy3KJq4mg4IMUUEhYJhSxlidrGPAYXuz7FpJVQP3M2kYy0BxlPtHjUnHn1HL9BnfvYWBfQVeMqXQZXiQ4pQcgfQw60OEHql6vzM19vIUOWLY83a43S7CtRHkEV9bSHEF0tF2/7iAwBoSSgUal/xNZOApWMZzqjuUPOJy6gKNUgqa9T/+GfVc87UBTMqWINU/gIAxh9/iiiKoJgY70OMkMZ9yDh/TUmBc+ldQgRX21VCDczyT9pE7liP3DD5U4Or30GV9aF+T14htPTONXow8WD0EX58Za4lfG9SrQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3n5rpFcEC6VqoVTeExgsYqrTHaXyid1dAz0fTGYAT7A=; b=MKDXmasJAbUjhsWc7AOsBwypb3i58TSrmPOy8zi2KJbJp2g1ApLB/YW3tSEKh+UmjJH63MBh5q0bspj4VUFn00s0VTwnRotcHIW3dSOYLG3rOMH+d6+WbyZDcQvOK6fjxyqTcvNRY2Zpx/35OMdsrVsraEJUwK/qiCeCYreehQ3yrfITi72ER16OD4Dy8Bio30BChsJS+0gyVa0lTLvZAYtcTCd16J76S1F1OxzYkM024Tlg3Zl2qr0SCTLZPpA7XepX88Kt/X1Ig17oOmxIC1C31LOWOnFQ/tSDpAxc936+yFI0jKUIMVb9o5P8WqsccF0OEHL5xL6HdojTPDNtvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3n5rpFcEC6VqoVTeExgsYqrTHaXyid1dAz0fTGYAT7A=; b=MCEOSCaQR7I2svGMi9sEog2ivDtw3B66tQzp/cylWUXRkhcMpxgTo8OLtfNcOYqgTGwJEra0S9USD4wAc8M9hMAaVWf9ScJD/WSUwBcCJXkEuuR7Nq4NbRfKgXKXsB+GYRkGwy/syxVgi7iCCSjbVc0hJXOatgQrFTXLupOAvO95ad5Vc1pyr1IRX3QXcmZhcZ3frNqKHBZJdRSlWRzOjNvA5P34r3+zvPWUJLBciu5/My9sZVd/8xuK2d9WVQkJQ76rlD4VJ6LpRB4/TKKZHmbvwUyazI5QlPK39/ZcX1gIo7PTP7In8eK359mgIqcipXEftCdN2SxNkVsEaKbnWA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) by DM4PR12MB6421.namprd12.prod.outlook.com (2603:10b6:8:b7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8335.10; Wed, 8 Jan 2025 16:32:58 +0000 Received: from CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5]) by CY5PR12MB6405.namprd12.prod.outlook.com ([fe80::2119:c96c:b455:53b5%3]) with mapi id 15.20.8314.015; Wed, 8 Jan 2025 16:32:58 +0000 Date: Wed, 8 Jan 2025 17:32:53 +0100 From: Andrea Righi To: Changwoo Min Cc: tj@kernel.org, void@manifault.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3] sched_ext: Replace rq_lock() to raw_spin_rq_lock() in scx_ops_bypass() Message-ID: References: <20250108150806.101555-1-changwoo@igalia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250108150806.101555-1-changwoo@igalia.com> X-ClientProxiedBy: FR0P281CA0253.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:af::8) To CY5PR12MB6405.namprd12.prod.outlook.com (2603:10b6:930:3e::17) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY5PR12MB6405:EE_|DM4PR12MB6421:EE_ X-MS-Office365-Filtering-Correlation-Id: e7209a2a-8534-4df9-1a20-08dd30021c3a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ZERq/o9ZgOxkg5vU2k2mEzZtsaPxLWLBdp1NSbYStgNesx8PlE1YFUHZctJA?= =?us-ascii?Q?hyVozoo26ElqNEURPqexx0bCe01MEqa5yc9J3nKDHdFufv5vCs6hcsRSV60R?= =?us-ascii?Q?YbRr0mygiatgKLrEws6C5440MrhAYXEKHbR46lAPZJtrAjqY+5wMAzql+kho?= =?us-ascii?Q?SpX+Vdd4mH5bc0vnawzoEuPVUIVZskkNkbzD/fPYdfSwGQClgP6xzErXi6Er?= =?us-ascii?Q?Qym4N0/HTnO5wcN/gxxxchl49FifXog0t9Me8m7eRO0qQiDP7lzMzn+rKabH?= =?us-ascii?Q?5M31WO/crHGQZrQVBQ/VFWHCLh+otgBPQiz6313a93P3EY3WM8q3kXC86d/N?= =?us-ascii?Q?r8TITKrP9ovH09AgUGGzZXV2Bwy3NFrTylN1iWTcMgLKtnLfZ2N+Q6HRO5ZY?= =?us-ascii?Q?gdTDRS14hbLteCHQjyVdNBq5D28H5pIi+xS8XjESq9z5K2C9McWtdRNkfeF5?= =?us-ascii?Q?1AdXNAEvl79NC2ZVanm7wPE985sJPUvJ6LBeY1bYe5vCzvG1YXhesJaH6KPu?= =?us-ascii?Q?mq8CKu4xw8+R1LDY+YW5eYMyPuQzRC8p8fr3o4bzi5BGHqLyqn1g7DUUGjAv?= =?us-ascii?Q?ETKH8y/hrPrU6n4MWpfy1UPrPqYtMwHfzXT88BZVS8I6dj2w9R8twpFkIO2V?= =?us-ascii?Q?9rrCLyMHVMyeBsFa50l1jHoTP9fIDRU/5BnYKz+0gstQd+5iWt4cxpyRGWRQ?= =?us-ascii?Q?9BV/YUCzdbmkDPV+CzuP86SAx/A+tXI9TaACMtteAHH/v9DyQYzSSU8qHde8?= =?us-ascii?Q?XfLt6I/w4mX4v5F63cJ92mkuuoYpSuW4iou3Vwhk0hUlVgFU1YVnfFcySYkA?= =?us-ascii?Q?5fM/dcvasfziTM1aqtLieW2DHfLBA3SFGdWGK4Il1UbEYtnMlaROP5PLz6mA?= =?us-ascii?Q?SxNDoEYwX+R3LMNwGCH6bey7nF4Ovwa9G8fOkh9eVk3i5MiaLOKSFKGU8M88?= =?us-ascii?Q?tRvMoteYkk+RkfC32+Cc6L4dS3b/8tPcKLOFFXz8HGGX0qY8ZJzj2HJ2X3/K?= =?us-ascii?Q?tt8HdIrDnkgcENMJzH0xz+MHXh8swdDMEXucox4SexxoYC9q+fkEufFkcE7l?= =?us-ascii?Q?2qsUKhCfe/tUuKsu+5DFTXjhzYaBKIlgl/TMliKjUmaNfwdjt7yFd5w6PTJJ?= =?us-ascii?Q?eBArgQSf94Gete/qpk7PxMmQhVJrJEwh1PdVnCMot0rjDcUCuYF7Jr+i/CCV?= =?us-ascii?Q?lfoj3k8VHLszFuf85TlRh1EjI2nYQCgnaHUNFwxxenHygaAtjiqRTOYuqIkl?= =?us-ascii?Q?3zb5dXFyt/GhPl862XH2vN0LnglyKCtOuXR7b8XjvN++LilFwr4QFxpICcxi?= =?us-ascii?Q?NGTXZH+2AVL58CEfQmCy/Sa0AtR/SyoJT/NYa7KsWBc2RQfJHLBkZ4yof2kR?= =?us-ascii?Q?DcdizJFA70qMWa6vrc7lBwwOOS29?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY5PR12MB6405.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ER6b384P78mqzTcpOOOv3pxWY/LeT17nwAjbWn8zRpdfzEPWCkxuyaKPtxID?= =?us-ascii?Q?RNPaWzBuP20+Cb1fj7cdvA7eGYZui1qRQHO5StMQV4qKzK5/x2HtxYsBDOIr?= =?us-ascii?Q?Xs5EsbIr42xU4T1snheJfRfwlEilLQpIDbkZ7iZBBJRq0keXj/sNMvRET4z0?= =?us-ascii?Q?vZQO9gWQygc0yNRYe/hlG9lZKcGQ0X3NXIF+XTHErt8Epa01J2OHj5gzO/pg?= =?us-ascii?Q?UKeX7P51yo5dy1t+GKqeJPGI/s3G2TTnQkXRG4Piojz0OoENRm/YLFSbrf/d?= =?us-ascii?Q?6UhL85dUJLHVDY8LEOcbiHkrbdKsB16cDIQ+HC6QP41cOm0s62cPJ7Qpyei5?= =?us-ascii?Q?jCVN/rSxcH6EOFdbAuX25Di9oJ8mL25m5SjbafYAASzC7uJrSUXtBCS42EG1?= =?us-ascii?Q?5KxLvuptx6JvUD52Ga4XYhEMNp6XjwVhaOcAS+vLt04OUyUQE/S2wiXSBSpu?= =?us-ascii?Q?TS+2NmP40ufT6JMKHggI5W8NP8SqS6xJuIPNg/SfsruUYTTntC+y6WYVdsJm?= =?us-ascii?Q?cZ02K7qLI1A9o3Q8TX48/h6tiChSBwTl+anT4hp0iWhdxN8QtDnVRSfG/Rz0?= =?us-ascii?Q?BF3J+1glvBMlAcUGkrtKMNjHPh3+4o2BRITm0vxgkd1aaqCaGkXOGnoF9PMO?= =?us-ascii?Q?HtTqBJk7kdMaTeQy52X1z7LvapdNQHL5By0l82E89oTn50+gbwBc9zMkZ2qj?= =?us-ascii?Q?E96LjbjVSDidTcY7DffI6asKHkX2+4sdJd1zMX5Tql6pYrmJb8FMrAR1ChgU?= =?us-ascii?Q?rqj2bUASHR5u1TqeuFC232tSOZj3VD9g8aflRNumN2lssVtIRnZhzSNV0aaw?= =?us-ascii?Q?8w7d+qhAbS8GHhamYjynxJW/cXBL1l8mIs9oGz7Ya8IcHxVVkRtRPVlp8AHU?= =?us-ascii?Q?oOdSetA76SDWMDys9JrqhU1aOyPsF0zpG2BWXI0VDpgVQHL/4m/cSdkxgnX2?= =?us-ascii?Q?cQZtVMZEyLBMcpQLTjM/ekoPmNXu6zZrE3EIrE+fJ1dpt2ZbdoKbvE0blytW?= =?us-ascii?Q?cbTWZZcPb96TxfJkr9MT4bK+6UBKhO+4OU7TtWTWQKKWFet9u3TjR7Z/jDM6?= =?us-ascii?Q?l1Wayvdg0uJNY0i95oq0SosOfxxDfrTVe2J6WOd+G8qaNQ8C/pz5+ouE2Eo7?= =?us-ascii?Q?GXj8k2GUtUDa3o0ntzx7/kDPgooQ6fOkbyf/rh1YTiz8CiubLIitpYMjC2J+?= =?us-ascii?Q?YR/CgBxEoX4pL+KHfFym5ryAgTqS4DQMk4xYMG/qStVlFx3HmR3Ora8GNYxk?= =?us-ascii?Q?xda4RNIDgqeEYwGlVuEkQU7YkQs0Dm0XsS85a1hRXB1Fto+7TiIbf2AWVEjx?= =?us-ascii?Q?UreCNtUEk3nwo4isgIXzgvUwrzJjc6Wy5WrqL3ZHSVccIuKG3DFJadmJQ08b?= =?us-ascii?Q?8fVNPce0bPFVWOpMUD8/7+XxaOcst8xBXCuWdw89qE191R/obWFrIhfwL0Zs?= =?us-ascii?Q?2/JnsOjIL7l+wVsXM+thzpR1AdFKxa85Em+VLiyuy2s3Hoos45djs3abCZpD?= =?us-ascii?Q?7DUtK8pSaDSlba+mTIxWKq7GHVW+wpO2x0dlGwDV/PhoPTsEDJ8+coOHVuWF?= =?us-ascii?Q?e32b0Jz7zC548ceES7cUuONUeMzJ5TiM/LYUIwY4?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7209a2a-8534-4df9-1a20-08dd30021c3a X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6405.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2025 16:32:57.9461 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: DhoKuYwLTd08+TyLSY0rnCY9TlYxXkyneP5V1icQyJneTGKIyRDYCzTKz25DHnpikUdnE43cBqpj6JYDyAmwmA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6421 On Thu, Jan 09, 2025 at 12:08:06AM +0900, Changwoo Min wrote: > scx_ops_bypass() iterates all CPUs to re-enqueue all the scx tasks. > For each CPU, it acquires a lock using rq_lock() regardless of whether > a CPU is offline or the CPU is currently running a task in a higher > scheduler class (e.g., deadline). The rq_lock() is supposed to be used > for online CPUs, and the use of rq_lock() may trigger an unnecessary > warning in rq_pin_lock(). Therefore, replace rq_lock() to > raw_spin_rq_lock() in scx_ops_bypass(). > > Without this change, we observe the following warning: > > ===== START ===== > [ 6.615205] rq->balance_callback && rq->balance_callback != &balance_push_callback > [ 6.615208] WARNING: CPU: 2 PID: 0 at kernel/sched/sched.h:1730 __schedule+0x1130/0x1c90 > ===== END ===== > > Fixes: 0e7ffff1b811 ("scx: Fix raciness in scx_ops_bypass()") > Signed-off-by: Changwoo Min Looks good to me. Acked-by: Andrea Righi Thanks, -Andrea > --- > > ChangeLog v2 -> v3: > - Trim long warning messages for readability. > - Properly add the Fixes tag. > > ChangeLog v1 -> v2: > - Add warning messages without this change. > - Use resched_curr() instead of resched_cpu() to avoid redundant locking (Andrea) > > kernel/sched/ext.c | 12 ++++++------ > 1 file changed, 6 insertions(+), 6 deletions(-) > > diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c > index 8fe64c27004e..cb6eb49d16be 100644 > --- a/kernel/sched/ext.c > +++ b/kernel/sched/ext.c > @@ -4803,10 +4803,9 @@ static void scx_ops_bypass(bool bypass) > */ > for_each_possible_cpu(cpu) { > struct rq *rq = cpu_rq(cpu); > - struct rq_flags rf; > struct task_struct *p, *n; > > - rq_lock(rq, &rf); > + raw_spin_rq_lock(rq); > > if (bypass) { > WARN_ON_ONCE(rq->scx.flags & SCX_RQ_BYPASSING); > @@ -4822,7 +4821,7 @@ static void scx_ops_bypass(bool bypass) > * sees scx_rq_bypassing() before moving tasks to SCX. > */ > if (!scx_enabled()) { > - rq_unlock(rq, &rf); > + raw_spin_rq_unlock(rq); > continue; > } > > @@ -4842,10 +4841,11 @@ static void scx_ops_bypass(bool bypass) > sched_enq_and_set_task(&ctx); > } > > - rq_unlock(rq, &rf); > - > /* resched to restore ticks and idle state */ > - resched_cpu(cpu); > + if (cpu_online(cpu) || cpu == smp_processor_id()) > + resched_curr(rq); > + > + raw_spin_rq_unlock(rq); > } > > atomic_dec(&scx_ops_breather_depth); > -- > 2.47.1 >