From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012006.outbound.protection.outlook.com [40.107.209.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E99F679CD for ; Tue, 14 Jul 2026 00:11:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.6 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783987872; cv=fail; b=VA4au/zUVkziGvTOr+jBV9U3u8DVqqAAIWfDIfvLUQ8Y7Xoyji0GzLYKMkUnjRD2AVzclHUnYdb6zzBbodePwjxoOwtv/YTMhJ9PrOqyE89z+2QFHn79M+trUzfbYq3ABcrYDy0gVCMjI7EFvuk8cHbXSgkdXLlcPIGnQc90718= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783987872; c=relaxed/simple; bh=tTPCb9I3m6oAUBWyUpsjSzciIOEjOzq//4n5HCMiDrM=; h=Message-ID:Date:Subject:To:Cc:References:From:In-Reply-To: Content-Type:MIME-Version; b=bZgSRvana55paMlvWPuozZsaog2B5+YQwSm/bQfCKSVTs8AzLkFaEL2ECbuvrMiJva3pU2ZY5TSkvl9JZPKNbDEl65FSn1toK1XB4dkq8g15MUJQ5my3xqTcyWse8wRTPG2nsFpNHQh61smI/1wCFRuzYjHJj46IIhznebNbZsM= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=i2H7qS16; arc=fail smtp.client-ip=40.107.209.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="i2H7qS16" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ld/Q18ccc67ZjQgLeyprkM+iPWKHoyqbB8v+wGdaK52C+q4aCUUhM5z25cTygkWtawPYXpdKnZTkQLqt+UlezTm3qXegDOMBsoLf7cH9GMY5axEiHf0CJh9EPc4fbNqpYL22vWgEMuB6wXZEtcNLMUiR415HgNFCJ1mzsMU1lOMaR+qeGlbjRP3gzg9huLZBxbm74VsBmrhMzjTzUODY5kHIasL+7v6Wlo1zF+/37aBPVqvYJmOwN8SzA1HPfPv3dhsCtlJR2VOWV3aZzFtlzIvqfNLEcoAxToxdMdlWRXjd9MNzaPXNjgV1GBVyKwyXjHBjytOOWJNrwt9oVZdjmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=47ROepTMiYiWFVpKLb/v8LN/D3YyyeqMTQDgBZzWuRs=; b=KRfMpbImpM4GVmWsiDEr7EOhZ1KiIGC/hC/JP6JhXiMRKe3D4bFmYZrj/PL479xb/pbNd3zkeK+wJZf+kv4JqHQigYaKipP52jaUtSL9ScQuv6ERwUVRYDLAM06vxzosFFMxlnct30Jga54NdUYe0dpZ5UX6h/ycj0HJJIXj4afzE6L6y1Q+9GGlC2+JjIV5lhpZUkuc8cnCgY+xVFzJ2+jJ845xNrBjUS8GjM4OLIT3iIu50yalIyWDl1omcM2otCMyCDuZQbx9z3JtSLTjByTRhw7Zhs3mF5Y3PbjzlLP30jAJuD4aKuJgIdI/p/QjokPmNIGKjQK+xDiSMZ0DjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=47ROepTMiYiWFVpKLb/v8LN/D3YyyeqMTQDgBZzWuRs=; b=i2H7qS16H05/P0ctgDJ5/YfRIKpgvUEFTZBqmD2Jgo5j9qG4bw5AlH0wh9lKjamZVOB1hYSAKcnlt0rdWs7ewh78RM77NdMh9tNbLzpIgHtUt7KDeBnx/fGnfl3sK7OvcJ/uXKbBLg7lyTgAcXVTW6sg2+8o7VAeS+R9WZfM/obNMyS955Z+ZWJl5hJFZyouHK97qaua2j5iIEamGp3Hi5T70bMxSiR47JqT0CfAaMSSQzob+rrmpK9/twrkUhxwInv8vC/U99kxHPTyCakX2wheKutymgwqTe7RtIVBmM+y7y70I2dQ108E9KkdTZmApweEy/dErQGwh8E5ZNk2fQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH2PR12MB5001.namprd12.prod.outlook.com (2603:10b6:610:61::18) by LV3PR12MB9215.namprd12.prod.outlook.com (2603:10b6:408:1a0::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.202.19; Tue, 14 Jul 2026 00:11:07 +0000 Received: from CH2PR12MB5001.namprd12.prod.outlook.com ([fe80::89e3:6df0:de90:8dfe]) by CH2PR12MB5001.namprd12.prod.outlook.com ([fe80::89e3:6df0:de90:8dfe%3]) with mapi id 15.21.0181.014; Tue, 14 Jul 2026 00:11:06 +0000 Message-ID: Date: Tue, 14 Jul 2026 10:10:59 +1000 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] mm/migrate_device: avoid overflowing migrate_vma collection arrays To: Alistair Popple , Zi Yan Cc: "David Hildenbrand (Arm)" , Matthew Brost , Andrew Morton , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Kefeng Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sashiko References: <89d20411-4915-43e8-9808-3a7fbd98d2ab@kernel.org> <57cc487b-6ec3-4ca6-8735-f37644e2b0ab@kernel.org> Content-Language: en-US From: Balbir Singh In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: ME3PR01CA0045.ausprd01.prod.outlook.com (2603:10c6:220:f7::13) To CH2PR12MB5001.namprd12.prod.outlook.com (2603:10b6:610:61::18) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB5001:EE_|LV3PR12MB9215:EE_ X-MS-Office365-Filtering-Correlation-Id: 0e6d4be3-91aa-4be9-ad2e-08dee13c6681 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|366016|1800799024|22082099003|18002099003|4143699003|56012099006|11063799006; X-Microsoft-Antispam-Message-Info: Sqkobhd+I6NV1opYZFYPEevHhVO059SvxQOMcC8lx2Cnu+/MfGa2tihOhgp+1dxwSPMRbcVv9J+Vw1X77Hf9ZHV1sG4FydtAm5qzWjhINKWZVTDkd0/MpFlhD22KBtyzYGloy6SYo0AobDr/0zrFhsjMq0TjWXPwQdXTwSP4XID3IGJvwu13Tr5Y6G0crFzv0sIya0cUrsWECQDde9yoRVFHEZlB+3UWUqEF4ggcrY11ynCdpMeiwL4C/LO/KVhVneQdU35dhiXyOCDKtYvprI/w+CzK8BMr0tXpYbQcXnadalUDVVfxbKNhmOey+g1onvGvYHlmuwJHGipcnBsillxWl9v5HJoB5jHHhRfFLeIv72Zofi64icwq01qEi65AwKlSWcx9FZf/P0OJfZhj+klCdadg442o8jtaTClJyy4+lrbldHwmx5oAl0a/aaKdFZ09Osz0OlGiLYOUYVhkccLqwVTnB/fkKmrpj/c6wVLeS7f2MYRLnh0cgGBzrta7llA5TvKhT8aRtvSJDnBB4dGMwt3DPLJPDEXpVNkHuQo9Iybjs+6qPaugES6vc/Et8hQ3NiPpEADKWSWD4iQ6ddE092WBgvv/Ub+UBeA5TJ0aTzqsJmeQQ3nyEya75uDp7h8YUJPGUVAMKqeZxAZrPLoceSbL+QKcHNewlQL3QDw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH2PR12MB5001.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(366016)(1800799024)(22082099003)(18002099003)(4143699003)(56012099006)(11063799006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Y05VSHo5alNvSTdkRi9KQmdTbFNGM09Nejl4ejAyN0pMYzkzVWpwbjE2Tk5X?= =?utf-8?B?eGpMN2JXLzM2dTh5czNIMnQ0UzJvZC9ZTHZUSWRpQnpHNWxTVU9UMnRPYVR6?= =?utf-8?B?NFdPQUV3Q1RHL3FrcGNkSEVta28yMFNxVm8yaXk3NU5xSXNtWmZTa2hRTnVP?= =?utf-8?B?SGJvVEdpem1lSzhHZzFzSGE1SzMzRGEycWphQjBzV0xWdVNoTVd6SVVuSFBS?= =?utf-8?B?TEp6Z1hOMTdZY2tPK2FDWkdIblFuSmFzL09lc2VaejBVV0FYNlBaREJlRXNW?= =?utf-8?B?dUdRUmxiS1hFWUM2R1lVd21idlV2ckFFNjg2MmlJQThvN1dFb2Q4Y0ZTaGdR?= =?utf-8?B?K1lmSStnSERRNTlVRkVxVFZhNjA2clBMTVRXTnJYeVZ2MVFRMXJyN1pQeURt?= =?utf-8?B?YmpKRURwd3cvdlZ0WXczcytGSndXUk9RekV0dEM2WGFZMG5xZ0tTb2YzUitw?= =?utf-8?B?WXB2eWpLU2txVGx5T1lodzR1YmlVcURVV1V1OWF6Y29IS25leFFPeGRpYVEx?= =?utf-8?B?Umw1WDBWR2dDNnIveENjVkNnS05oc1k3Z1IyLzViRW5pVm9hcUNKdjVLRGFS?= =?utf-8?B?Y0tZdjIvQWtuaVFNNEhjWVRNWFQ4c1NYQlFsNDFPTlBrMkJ0WG56ZGF0MC83?= =?utf-8?B?S0tjWnRKbkhvV3NXR3diKzZieW9IblV6ZlRaR0J0Q3NHOTQ4VU82Vi9nc1hS?= =?utf-8?B?RjA3K1pZWHE2eEZabm1ydzM4QWMvWFlDVWJSd1lDMTN0K28zTStUaVJ0Qzcv?= =?utf-8?B?R21hMno1N2x6ZGVPeThRc0Y0NFhlOWVVNW5jak8xbHp2a1Q0WXpXNnJSY1lS?= =?utf-8?B?TzkxYnhTT1NnNndaSWZKMHVDbFBjb0t2K1pLTXJHOTFKRzJlUG91QklHcUpJ?= =?utf-8?B?ZjhHTTBRUWdHR2xWbFdZZVpDSVFRZXhtTk95bVFSWlV6bDNoeWRicmE3WDk1?= =?utf-8?B?eG0wblFKVEdqZURmZTBiUmJMM01PVzQ0Unk4U2JTRlkyUVlLTFl1cjVTaHgw?= =?utf-8?B?S0NlY3NTVWhhM3dzdGVlc3V2SWtYcmhMYTJtNGREU2RXdDJEUm84bG5WWmpL?= =?utf-8?B?aG9RK3BOcFFqWEdzQ0FtZzFDbG9PZ3pNeWY5ZGIzN1NVVjZkYnhTYXpuMVhF?= =?utf-8?B?R0lWbTJ0aGNHcGEvczh4SlBGZ2tOZDRPYmxob092WFM1ZE02SVNUZXpwQXEw?= =?utf-8?B?R3V6bVBzNEFzbEUybFhiVVhpYWZjL0FDU3A0L0N4TURLcDBBWEFRbnBjUkRX?= =?utf-8?B?TmJ3VldNdnJRcHo0Z2EvRGNoUmdxeDY0N0VEV2NKVzkwUndlMU1rajRvbHJY?= =?utf-8?B?Z0pLWnpZNFpWYmRaNndwZStZSUxYMk1jd3A2ajRkdDRpVXBPa0tqYys0UXVh?= =?utf-8?B?a0N1ek9pRUlpalNwWUhZVUZmM3ZCejlVV0JOOUs4L2pYUysrMUJiTC95Snpk?= =?utf-8?B?L0s3NjRnc3pkTForVXlYU3ZTNDUyamVZbm5DQVcwa1JxdzF3TnpQSXVDYm9n?= =?utf-8?B?WEJvYkRKYjBIZkU0K3hoVG5hV0laS0xFUk85ZTI5U24zVjB2Vk5lS2g1MGFy?= =?utf-8?B?OURaQ0RsTmNqVEtXR0RxYjE4anEwb2VoSjVZdzBSUDhNNjQxTjZPRzRYOTJF?= =?utf-8?B?NkJyTkJwQnNKVXNaSzNBTG1HY2U3Zi9OdVgrWnA1b2NoMTZ0eVhNYXhTM2RM?= =?utf-8?B?Ny9MVVdxSkJ0Nit6dHl3dWpwZEhlWGM3QTg1L3c2YnZtNjBrczhieXBkS0pw?= =?utf-8?B?OW9PM2tQN1BzMnAyUUVQRFR1Ulg4ME1NL3JOc0hQK3NpWFpDeFpCa1B4US82?= =?utf-8?B?L014dXpSZWQ4bEo1L2N1ZFZQbi9heTN2RkpCUjRsWTdmYXVuZVprK2FKNE1W?= =?utf-8?B?bVQvOUR5djRjQXlEN1FvakU3a09YM0NDZUV5UXVwMzgwYUgrSkdBK0dDeVNB?= =?utf-8?B?NHhwcE1jbERqZEY3SGhySkFmSkZIbExZdG9WaWUyUlNyL0NFQnMzQ1E2Q0sw?= =?utf-8?B?bWdnbUZ4Q21OZlhqcjNNTmVmVVlsVTZsNThKRHRJWnVxQk4rOFUzWVAyS3FE?= =?utf-8?B?QmlwNkY4TE1jWUg2bkhQZW9MaUJCenNMWEJEdkI0blF2S0svL0kzZDZPL2RS?= =?utf-8?B?SmlSejNNZi9YblZDaW1oUUdEOE16TWt6b1dGbVVnZ3V1WGxzOHJ6Q3Z3ZlpF?= =?utf-8?B?RjZRRi9BWk0rd1U1QkY1QmtNZCtnMWFndk5FODNxZVpIeTRsOGlsRm5YalpF?= =?utf-8?B?b0t6NmhobHlGQ0dWZmxSN3FCRWFqUzRqYmR5Zmd0Ryt3WjZNTGdtTm5VbWJS?= =?utf-8?B?YUNubVZNRG9hZEk5RVp6MFgvS1NyWWVVeE1pWVd0cW5yQnNJM1RiZz09?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e6d4be3-91aa-4be9-ad2e-08dee13c6681 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB5001.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2026 00:11:06.9056 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HIHzsy51bhvEh14YbnWceafJwiwB8U2aTtALWIunKSGBkmHM2Kk//FvKQShsWXvlgJpPrfsWbU6VVmLkQLMSQA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9215 On 7/14/26 8:24 AM, Alistair Popple wrote: > On 2026-07-14 at 01:39 +1000, Zi Yan wrote... >> On 13 Jul 2026, at 11:25, David Hildenbrand (Arm) wrote: >> >>> On 7/13/26 17:20, Zi Yan wrote: >>>> On 13 Jul 2026, at 8:57, David Hildenbrand (Arm) wrote: >>>> >>>>> On 7/10/26 17:35, Zi Yan wrote: >>>>>> >>>>>> Not all the time. After a folio split and PTE lock is dropped, there are >>>>>> different cases: >>>>>> >>>>>> 1. no page table change, the collection grabs the PTE lock and continues; >>>>> >>>>> Yes. >>>>> >>>>>> >>>>>> 2. some PTE(s) is changed, the same as 1 and the collection continues; >>>>> >>>>> Yes. >>>>> >>>>>> >>>>>> 3. the PMD pointing to the PTE page table is changed to either a leaf >>>>>> PMD or an invalid PMD, the collection cannot grab the PTE lock to >>>>>> work on the remaming PTEs, since they are gone. For the collected >>>>>> PFNs (they are mapped more than once) and folios with elevated >>>>>> refcount (they are mapped once and unmapped here as an optimization), >>>>>> they will be processed later in migrate_vma_unmap(). Since >>>>>> migrate_vma_collect() is best effort, there is no need to revert and >>>>>> try to recollect from the beginning (to get that possible large >>>>>> folio or skip). >>>>> >>>>> Yes. >>>>> >>>>>> >>>>>> BTW, recollection will be more feasible if migrate_vma_collect() does >>>>>> not do unmap singly-mapped optimization, since in the case, no PTE is >>>>>> changed, we just need to reset migrate->->cpages and migrate->npages and >>>>>> restart from the beginning. >>>>>> >>>>>> >>>>>> Right. The PTE page table should be gone and the original PMD, pointing >>>>>> to the PTE page table, becomes a leaf PMD or an invalid PMD, since >>>>>> someone changed it when the PTE lock is dropped. >>>>> >>>>> Right, so I guess we'd have to remember how far to rollback, and rollback would >>>>> mean resetting migrate->->cpages and migrate->npages. >>>> >>>> For case 1 and 2, we do not roll back. For case 3, since the PMD is changed, >>>> we just roll back to the start address and set migrate->cpages and migrate->npages >>>> both to 0. >>> >>> Can't we walk multiple PMDs and have data already in there? >> >> migrate_vma_collect() is the pmd_entry of struct mm_walk_ops, so it only >> walks a single PMD at a time. >> >>> >>>>> >>>>> And for the entries we rollback, we have to decide whether to folio_put() and >>>>> whether to folio_unlock() [fault_folio != folio]. That's the confusing bit given >>>>> that the code is rather "interesting". >>>> >>>> IMHO, it might be much cleaner to remove the “unmap the mapping if the folio is >>>> only mapped once” optimization, so there is no need to roll back at all. >>> >>> I guess most folios are only mapped once. Do we have any numbers on that? >> >> I will defer the question to Balbir, Alistair, and Matthew. > > Unfortunately previous tests that I've done showed the optimisation is critical > to performance in the common case (ie. singly mapped folios). > > I say unfortunately because I have similar feelings to others here about this > code :) I tested this a while ago because I was trying to clean up the collect > step by mostly removing it and replacing it with a hmm_range_fault() type thing, > but the optimisation proved too important due to unmap having to do another > rmap/page table walk. > > I don't have the numbers on hand atm, but will recreate them just in case my > recollection is faulty. > I had a patch to add tracepoints to the code and some of that instrumentation included this data. A combination of the trace points + hmm_tests will help us get some data to start with. Balbir >> BTW, for this patch, since there is no bug report on it, we would like to >> fix it properly without worrying about backport, right? >>