From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2041.outbound.protection.outlook.com [40.107.96.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A09A1339B1 for ; Fri, 25 Oct 2024 15:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.41 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729870360; cv=fail; b=pYhrCdALNK4xNfq6jWXqnKSFWB1k3G1qLb3PWeGEYi4k47xnAhganGUZQCYHcmBXTiyRVRaScCpmc9mXVn9Diuujg777cKdbUeXCh4237PA8SHXKpaogJHuNzt4mM7godlSZAC7fPb2n1CWv8gLsukCc/+CfMRBAgAgnzCJPVnM= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729870360; c=relaxed/simple; bh=62nLEWg6tyzR0KLeQ0rzjEEDozw/XO1rxMFz9gAzKTg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Io0zbkYSb0tXXv5XtOzkpYidYolWST07+Yk93rwmF84l33pzIadBFY3jJ3lbiBYiXkyBKmXkiNzKA5GGwT9fyy79qWzo/5tZ79DroPlz1DAeNnMRsSGkHVoxUf2gS3R9RQJ0hJu68fcVHdkisQazI/13XJRoT6do5+DsHl+v8js= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eHOts+m7; arc=fail smtp.client-ip=40.107.96.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eHOts+m7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WVNHsCd2Lp+i6JFCpHAIm63lc5ZIC+EWBZXKRy1hGyqqIGtVm6PPn2cI5c7LsISjJTUiDw6sMfchXXuSIMNHDqTQ85zFm6QfInOZBXk2TiAjcMs3IRzPUVbayfhqp6hO16l7hf80XXN6i0yZKulGG5C36+RFiklj9dA4nsPujJMAFyveYTTrBKC5ikTErBGRA8hCKAGNMCPuXsoIUVD+BCvqU/JCSLQx4/THbHtJRG8mL5h4IzCWcg+wJlojGvA6QuhW6+4Vu9XvDfa/K1DyvhICxa2qTDNI66Bpx/glE+cKYPQ6besVHNzjYYjT9gXHYO3iFZG+vOC320wiUR9Yrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UldyBszstme9CCKyzJQfAloJxdBvFaEhW2UOnFwdVKU=; b=Fqa0RPzXcsNYQqDaQxMLyMTJqrF9F8HSn57ME2khaP710OfRm2bCeRnrUhnFbz3I8jYOHTlJpCSOGe1+GY97YVLYxqPL8mn5RCpM0Dvt9jkroktIN5xWfKVicEpXzpF3w0HTpvs4m964lbco/mlHCS0RgVIEtMv7Xvi4Q74q3Ltpf6kfhrlPjwFeDRPjMn/ILoCMISOwENef1+p4d+O/pGkiXvrpuFN02ousav7ys1RyZN8x7TeQCPa2pWzuO4vdrLOEDIGSh3I6SG03elYMNdR5F2kMKKop4+G2anRnoPJpi5b108t9l3r0rNdAOQzOy0Sw479b7ojRSDIuTWGTZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UldyBszstme9CCKyzJQfAloJxdBvFaEhW2UOnFwdVKU=; b=eHOts+m7M3HEVrI/6TH+SWa4l8yr3CYJkIXThl2JSEIvkSCAgW1KZR1dHVlplc7yWFlwJZcwitKF67tGrD5oYY2+eGwqpUZ4/ecSwTfqSG9cpLbmZqWQSDDcxpsOLer+3VrW4mflCp+cXGKk/CwGwTCtit/F53n6QEXQ4m/YfSNNxo/4Z66xzXKTcC5lcEc489FvJpI8VL9bIEbXqgilKQAi3QKwKp0k4zeGybrGOfgST0U/knNHoK/+Vn6Bi37hqTp8SjqDGfya0bFzYJALXVeqwKdxipFM5Jb6kL5UEfPSVNs4IO9mqWrL0/wqrxX4QPgk0TbLw75RS4GvIcpI4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by PH8PR12MB7423.namprd12.prod.outlook.com (2603:10b6:510:229::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Fri, 25 Oct 2024 15:32:34 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%7]) with mapi id 15.20.8093.021; Fri, 25 Oct 2024 15:32:33 +0000 From: Zi Yan To: Hugh Dickins Cc: Andrew Morton , Usama Arif , Yang Shi , Wei Yang , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Johannes Weiner , Baolin Wang , Barry Song , Kefeng Wang , Ryan Roberts , Nhat Pham , Chris Li , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH hotfix 1/2] mm/thp: fix deferred split queue not partially_mapped Date: Fri, 25 Oct 2024 11:32:30 -0400 X-Mailer: MailMate (1.14r6065) Message-ID: In-Reply-To: <966a4aff-f587-c4bb-1e10-2673734c2aa0@google.com> References: <760237a3-69d6-9197-432d-0306d52c048a@google.com> <3A1E5353-D8C5-4D38-A3FF-BFC671FC25CE@nvidia.com> <966a4aff-f587-c4bb-1e10-2673734c2aa0@google.com> Content-Type: text/plain X-ClientProxiedBy: BN9PR03CA0105.namprd03.prod.outlook.com (2603:10b6:408:fd::20) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|PH8PR12MB7423:EE_ X-MS-Office365-Filtering-Correlation-Id: 0eb645df-8ccd-408d-0b94-08dcf50a3efa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?EV+0YQctt6kvJ7zmoJDlT9BtVSMloSXtH4LxaZ9ot3J31iOnXPloLyrYhjtS?= =?us-ascii?Q?Rj2/SAfakHNz/m0J2bYTNZI4EkiFAF0qYdqMWiqDI/fGiOY0Ce1cKQBHkGAM?= =?us-ascii?Q?tzK6m8+e1cA4BTa72U9fHs/YDE20A0P6sPXgkKFFiuaTVsa4hGRmd9JKVA+J?= =?us-ascii?Q?evylRAnHqhDsGXK32dv71mwPtf+Ojpz5asb65Lf/tpOoHjKGZuUpMpGDk1hY?= =?us-ascii?Q?kbioOjPnQ8fm8LKnhzmsBQ7Seukn9Xc4egAFNv6cyQAEHCpybfNObEUe2vIp?= =?us-ascii?Q?7quTvSUiL3BX+b89GucJvTlBhe3mftUhSbYfDosmdHR8s6H/egXAzMdN0Gvm?= =?us-ascii?Q?WyQralflg0WhndCDPC7aTFuefJUjjZlrCvP0e9L1g7HnJbFSV6hJx+qBikiw?= =?us-ascii?Q?mD5bVsogWoqBg3lV1bX2j5yJND7VbNBLhXQw8STZBUt0EexlW2bC8lCm+Q6Q?= =?us-ascii?Q?e+k+K3Pw9P5LcEycTCPhog1wdpoKvp+u9biZVsbJ7+xR4eOH68a4o4y0mEqi?= =?us-ascii?Q?eLfmvj7FjyVw9oyCeJP2CDTFwfjHQvW9vrAsS6/GY5OZCVwe6GxXZCUoLofl?= =?us-ascii?Q?LPjZf145rGosX0VuTOIjh0iIvsp2m+hxWT3nTRVsKJMC2rv2Hw9gD6F8oeYl?= =?us-ascii?Q?AeBed90ON79b0xiA6m3ozyCYaWqwzJ4IE6KQx1R3OmV341Fu+SOFJxSbtFVb?= =?us-ascii?Q?vvJ0bWcyhQTj2xgmZpxXUZFc/U2WqvIuqzNAFSsx96oTWjrif34TcAPHi4GW?= =?us-ascii?Q?xbIIiOXfHZbooXJrHQiI1zMyMR6bcxmotnoBZUwQ1HKgArd7PS6pUArHPwln?= =?us-ascii?Q?75p0XeHjayHeOT4zw8+O913ORM5T8zeaxB98MGsHN0KudUjwg2n2PMgzChS/?= =?us-ascii?Q?qjQM9o6ke88H5lBXinxmhvDhIYgy6h+XV82kREYJHpQ08d8ouUi0mH3o3pTO?= =?us-ascii?Q?kL0jzpj1kGIUlJjsXtPflcE2ypOPs33xEr4dg7dfEn2G+lrdza1l/2Z6WF/e?= =?us-ascii?Q?BKDI3pmj67po6Yxc6YtVrunawO67oZRqJm7SnOLeGaVhtjX4EMr/oIhfiHJq?= =?us-ascii?Q?/GckT/ek+OB2aUUrdXUc626ZjyMnpjw6aDLL1VUPuHladeSJUv603aIfAeNg?= =?us-ascii?Q?PRpyj6ZSpZ3p90HUjPwtpHeqN7xFLhkxSvHhngPehWBgV56THhbtO2NDg/A3?= =?us-ascii?Q?UNfSZAnN4qjRkhOilO7Yp1A0po2zOuw00pHwfKSXqPxGoAgbV6YrJ+Z7Gwoz?= =?us-ascii?Q?dTEysdv2leruvcycJ9dsBZJB47mkaAL8Z2TdIDCYfO7rILck73sn141Z3Ii7?= =?us-ascii?Q?AGRg3AUWSKkXNGQyDifwAqnE?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?vor2c5XWsmYX2LLMvPqmpMTZFZNnMhqWfG0TojaasGcFUF0i177YDrNpBH1T?= =?us-ascii?Q?JhksuxTtZnAw43JbyY1AhSpMqRnOEOGjOvhyLeSVMmCiWJzCM8TrDbWw4oWK?= =?us-ascii?Q?mjloRSH3P2dNIK34bYm3Vxc7ffxjtkRN8wTmBP0CW0z2QzjHBCeNa1u1+Xn1?= =?us-ascii?Q?JxgZ93A70fbN1BcXon5a256gkruF6CuHNPFxbN0UCDPI1xyQr9Ky+2kqzYah?= =?us-ascii?Q?23tvYNrIYvafQEmNicDqfM9NXCZrkVtQebkMYQ8g9Hk39mxz/bpgVKfGAvG8?= =?us-ascii?Q?q9R0v9YOSDYaQv1W648STt7hR27+0tnIKmfo5DZAAFJziGvun+HmU+gbrnqA?= =?us-ascii?Q?jtH0tWpPiqYmJ7NMs2bWXOoMuTgrEcAU/W6FacNmddlbL+7ep2AaTS4H7n1h?= =?us-ascii?Q?uQ9A7qMvuh/Sv/6avXExZpT8oUhAla3GeN9RKw7dwOzbv5I559l4XXkvV8Ta?= =?us-ascii?Q?HTv0Jz/Don2vJb1KZ0sz04jnvi6dRkn6uzLVxDwnjDL4dfNXxjiqh24+vHe5?= =?us-ascii?Q?Tj5BdsoiHXpo/GshZGoiuLJ2AiDL8XuicHnRI1vcVpR8+/iXAULvxCjH2MCb?= =?us-ascii?Q?76nzTW8d0/1dEwtv2qFebl2aVNOosBIPUzsSQoQIAZtFizzBDRkR0Xy8EWIg?= =?us-ascii?Q?WmNVY4zbW/TtPGxzYERCUwgGmgcxaiiK/HGLs9W9Us0Q0wktje76hu6Rpw/4?= =?us-ascii?Q?HpIEmuy6+WFQBKASBRZBf2FpuheFu7bWfW1y+rIMbew6eF0jDQQUuo0e2sEi?= =?us-ascii?Q?ib1Yopt3HtKl0V7lwZ+UWrxg6zavXi/cyJE4aPFAmk3b6aGSyZEueUId/XDT?= =?us-ascii?Q?K5Um561WuJpDAeGk3tPHtnrtgLvdKC/lMBGYKtQx5iVAoG+J++mssEbuG5iT?= =?us-ascii?Q?kODrz1J8CkpaRcHs+xaP0CLilCClLQ+FO/GnxDRfQ+iOKu52CLgOhnr5vBY/?= =?us-ascii?Q?GLd+sEiIsmf09swGDkCMoQQNHgXH/LlGCfcDrV+DtPub68TajAc31d+LVFX+?= =?us-ascii?Q?x4QnwQ3WYGwjueylDVYqGczJJerIDyCDaukOiV711z1Oj1rCfpVMi9GAKVpp?= =?us-ascii?Q?ui6lBR+s7bGM8rwOCUG90+srsidhwN/8rA5jpb+WpoJjYREtSIonC6YzZFrQ?= =?us-ascii?Q?Aj326kWl0NHHxBm5VIc0VyDywT0miuy5FPeXuV0U+fsoo+p5+CYqDYoBNs+G?= =?us-ascii?Q?I/hSfmzoYW/v19hAdkKFxQGvmnhQHKE2KomMdy2dacxZYzippd2B/T6UEnVO?= =?us-ascii?Q?eBcRHZEobnHkiYYKGTGlYepn63l278oM5ogMFnkUoWHHrBHWFlTiS4NXoSTU?= =?us-ascii?Q?5vjCZeaLd0NVVD5sADMtLTi0QcAUMs33Kp188cTIoU50iO9rCxyIhK4U0VNF?= =?us-ascii?Q?kzAnOd7XgNmvlEAtTZUmZhA50+/ShmE3bfsyGMDqbyVd2/w7FbM1MBAbI2N0?= =?us-ascii?Q?5wqypekeAmwvzppnC1DQr+N/EIjc3R5KABzRGcwJGUaOlrKsFwvbqUas64qh?= =?us-ascii?Q?vswsIwq+FLpzhCs4cNjqnrSCa+mQLjAQsyBNLdFoWmwa5HwyKZOvjEGUJbX2?= =?us-ascii?Q?XSkfldFLi7O8svimybRDvVxA+zWAm9l7rgchb2bO?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0eb645df-8ccd-408d-0b94-08dcf50a3efa X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 15:32:33.6532 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z92mUXdEyROsm7/b9sY+QzSR9rlqq36yoKB27bL5iU2Ul5fPflO4NwR5dQVxNG1L X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7423 On 25 Oct 2024, at 1:41, Hugh Dickins wrote: > On Thu, 24 Oct 2024, Zi Yan wrote: >> On 24 Oct 2024, at 0:10, Hugh Dickins wrote: >> >>> The new unlocked list_del_init() in deferred_split_scan() is buggy. >>> I gave bad advice, it looks plausible since that's a local on-stack >>> list, but the fact is that it can race with a third party freeing or >>> migrating the preceding folio (properly unqueueing it with refcount 0 >>> while holding split_queue_lock), thereby corrupting the list linkage. >>> >>> The obvious answer would be to take split_queue_lock there: but it has >>> a long history of contention, so I'm reluctant to add to that. Instead, >>> make sure that there is always one safe (raised refcount) folio before, >>> by delaying its folio_put(). (And of course I was wrong to suggest >>> updating split_queue_len without the lock: leave that until the splice.) >> >> I feel like this is not the right approach, since it breaks the existing >> condition of changing folio->_deferred_list, namely taking >> ds_queue->split_queue_lock for serialization. The contention might not be >> as high as you think, since if a folio were split, the split_queue_lock >> needed to be taken during split anyway. So the worse case is the same >> as all folios are split. Do you see significant perf degradation due to >> taking the lock when doing list_del_init()? >> >> I am afraid if we take this route, we might hit hard-to-debug bugs >> in the future when someone touches the code. > > You have a good point: I am adding another element of trickiness > to that already-tricky local-but-not-quite list - which has tripped > us up a few times in the past. > > I do still feel that this solution is right in the spirit of that list; > but I've certainly not done any performance measurement to justify it, > nor would I ever trust my skill to do so. I just tried to solve the > corruptions in what I thought was the best way. > > (To be honest, I found this solution to the corruptions first, and thought > the bug went back to the original implemention: that its put_page() at the > end of the loop was premature all along. It was only when writing the > commit message two days ago, that I came to realize that even put_page() > or folio_put() would be safely using the lock to unqueue: that it is only > this new list_del_init() which is the exception which introduces the bug.) > > Looking at vmstats, I'm coming to believe that the performance advantage > of this way is likely to be in the noise: that mTHPs alone, and the > !partially_mapped case on top, are greatly increasing the split_deferred > stats: and may give rise to renewed complaints of lock contention, with > or without this optimization. > > While I still prefer to stick with what's posted and most tested, I am > giving the locked version a run overnight. Thanks a lot for the reviews > and acks everyone: at present Zi Yan is in the minority preferring a > locked version, but please feel free to change your vote if you wish. Thank you a lot for taking the time to check the locked version. Looking forward to the result. BTW, I am not going to block this patch since it fixes the bug. The tricky part in deferred_list_scan() is always the use of folio->_deferred_list without taking split_queue_lock. I am thinking about use folio_batch to store the out-of-split_queue folios, so that _deferred_list will not be touched when these folios are tried to be split. Basically, 1. loop through split_queue and move folios to a folio_batch until the folio_batch is full; 2. loop through the folio_batch to try to split each folio; 3. move the remaining folios back to split_queue. With this approach, split_queue_lock might be taken more if there are more than 31 (folio_batch max size) folios on split_queue and split_queue_lock will be held longer in step 3, since the remaining folios need to be added back to split_queue one by one instead of a single list splice. Let me know your thoughts. I can look into this if this approach sounds promising. Thanks. Best Regards, Yan, Zi