From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2055.outbound.protection.outlook.com [40.107.243.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4FCC318143 for ; Wed, 10 Sep 2025 12:12:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.55 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757506328; cv=fail; b=gY7q/iog7fKoSV/8Gtkg3qJ2hGNfoLuXSMXy9gxU0f3H6QA7NUjnh96yyXPGAD3YMluoET5uwh9lrwmZJ/JB7ZtJ2gcKALqDHw8lhydWcr5clOUKb/7GzK/JEsaVg4rmFdarcRXW11KM+NT/Fe/G2oRr/LlN4X/SB/NeGce+A60= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757506328; c=relaxed/simple; bh=HIQPMTiJ5fWdt/gJy52yIkcbR2qsjMuKiHt6wWfqSgs=; h=Message-ID:Date:Subject:To:Cc:References:From:In-Reply-To: Content-Type:MIME-Version; b=FkVLdaeqQiHUQFjJW4ltIaR4M4p21RYTXdIo/eHrIPZt14hK4/rFjSqOXfEGKv/odwZ1v5WtAu4i0s5lAtf527ohswH29wJehhBkdiJW/D/j2X+TdoQ2UegAPUHjVUPZP2l7LDDLxc7q5QLPYPIfpnuSfCCC7hOQc5vE7lJV5Ic= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=wC4Sww/f; arc=fail smtp.client-ip=40.107.243.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="wC4Sww/f" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wRNeasMgrCrVfpE2qRTooWTBZEWzdNyQXADdKhW/t3OhYX5qkFq0H2FJFlOtyoGMoKb/iapFEiaqf49Ch4BxlSt24/swZwbaRw2YzPJ74XkWWZXTUVHSzukmvw3xh4PKsLQ6MZr23gnk8XyEctYWJ8nCTEaqyhG0BJsmfAg4Mw+QxIGldnvzrYrXgoUHKB52prnGOj7Urhwopo4qCLHrPVXlT7NigSJdGJcCykzNRNMYt1tK7Q4hhZNdtZefJMqPUOj1tgwjCwr9bj852MTdAHn+LMmb/wo34YhUAwiJYbs9y1fTM08BjRUerSJsmlp/iEqRIu8q1iQ0rTQLXFkoog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z2Kee2l5oTJbaxhPJiwU+A8nN4uP1vO6e4esfXzswkM=; b=C98FBdQX2E/nbZtIVxoCm5OgEokabZNehM5JYfG54slwpLWC4D54YQhYQEfv/VDFRSeFYCvuHJxyEfHYAf+iRrweKPcJ1yym9FvChEnlyWEIexWClpwcHR6Y2iI16lkSok6hRdMltdYnGDBaFv2HMG0D/K5fTAqPyjmD37KS6jqM64MsY6YZ6kro5uYH+YiyZm1XM07//8cqQVnRddLQCLxgjBpDX7of3PfoFNiYKGC5qWTp6wggFdColJsdogK/sJRb244zpNDuewp670kH+wLcJln4MSYShqBNhNuDKGSzl78hobG6nRPr4Iu89f/Mg4hlragE6ORw6srcliImPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z2Kee2l5oTJbaxhPJiwU+A8nN4uP1vO6e4esfXzswkM=; b=wC4Sww/fpDUOnnNRyo4p8ix7AQ7ToM02wk3PoScCGusOdvj6CzFlqDq7/29IpFwkofsjsiKZxTyzbKqyWn1b2YAQId2dhBAJdtVhgZs7mG5EzucPZ955riy7C312/q3hCKHeXAxhs7cZK29AFU/XjQD59JOqDPdjIFMkH5I70C8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from PH7PR12MB5685.namprd12.prod.outlook.com (2603:10b6:510:13c::22) by CY5PR12MB6276.namprd12.prod.outlook.com (2603:10b6:930:f::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9094.22; Wed, 10 Sep 2025 12:12:03 +0000 Received: from PH7PR12MB5685.namprd12.prod.outlook.com ([fe80::46fb:96f2:7667:7ca5]) by PH7PR12MB5685.namprd12.prod.outlook.com ([fe80::46fb:96f2:7667:7ca5%5]) with mapi id 15.20.9094.021; Wed, 10 Sep 2025 12:12:03 +0000 Message-ID: Date: Wed, 10 Sep 2025 14:11:58 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] drm: ttm: do not direct reclaim when allocating high order pages To: Thadeu Lima de Souza Cascardo , Huang Rui , Matthew Auld , Matthew Brost , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter Cc: dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, kernel-dev@igalia.com, Sergey Senozhatsky References: <20250910-ttm_pool_no_direct_reclaim-v1-1-53b0fa7f80fa@igalia.com> Content-Language: en-US From: =?UTF-8?Q?Christian_K=C3=B6nig?= In-Reply-To: <20250910-ttm_pool_no_direct_reclaim-v1-1-53b0fa7f80fa@igalia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR2P281CA0144.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:98::18) To PH7PR12MB5685.namprd12.prod.outlook.com (2603:10b6:510:13c::22) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH7PR12MB5685:EE_|CY5PR12MB6276:EE_ X-MS-Office365-Filtering-Correlation-Id: 02157fd1-f44d-469a-a811-08ddf0634093 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?B?eTZ2a1VqYURRVTh3RjhIT2tJR0llalZFK2FBWUt5NUFUZ0IzNHlXbWpZVHVm?= =?utf-8?B?ZE81bDNsN0FmZytmUmJ3dTJDZVVCMlJKd0w1QWxPVmhKb0Uwa2hTOG5TM1kr?= =?utf-8?B?VEoyZkFyTDF6bXNpdG9IbTRDbVZXelRlRWRQakIrcjBZN1FZZldwUVdJR081?= =?utf-8?B?OVA5enh1bWxLSHNzUVY2UGJrZzdmYWpYRlh0d1lxNFlKQnNGTG5uL0tNclBk?= =?utf-8?B?VjBYU3lSdFN4NDlLazd3bUNDZjhoT05ORjRHYVJSb1lVb3FTNG5ldmY1QnVx?= =?utf-8?B?MlFyN1pDRHk5dXFaYVVMdS8yNUVZaDlGVEh1ME1SN0pVTjFwWUdPeXFkSDRu?= =?utf-8?B?Wjl3eHFiRytBeHlaVE83U2RKVEUySmtRUmZqTXlWTkNsY3lFbk9VTHhYWVBp?= =?utf-8?B?OXI1TUVWL3pXbmFpZHNnWkY4NWplY1VzSStBVE5nUWl1QS9tNDJxQUZCdWha?= =?utf-8?B?YURKa205dTluU0NPdzMxcE5CZXdsSHVqU2pmNFFQd0E5K0szQU9OODBZT203?= =?utf-8?B?a0JZajNSWGx3OHZoc1lNL2dRR0Nya2FPcksrOE0wWlhvQmdIaWtPYVNLTHl4?= =?utf-8?B?bGNMTmR5RzNiZTBKWEVDYWZqU2Y3VXhGZmw4ajRzSTZvSmptS2piUmJjcE5Z?= =?utf-8?B?aDZsUjNtekR4Z2Q2QUNpZkI1ZmRSekM4aDJpcWM4V3hsOFJZUzJ0Uk9TeXds?= =?utf-8?B?cExsTnd0aW5HVlRRQ0RHQkY4RHNTR1dtVHBFUktaZm5NOE9EVGpxWUNFaWI3?= =?utf-8?B?czQzOHAzZzAxdDUrdlJaclVqeGJUZlN5dVRmZHd3Yzk5VTBuZjU1dDlUd2dU?= =?utf-8?B?ZFU2ZHRpcVl5bldwYWt4ek52WFlYbkN5Q2hFL1laU0dET2xHVDFSak4vbG5L?= =?utf-8?B?cnJYLzN3M1U5VVlwWTRZQjFGcEoxdU9FMWxZR0JoUEJCSXRsbll0QkI1OVpx?= =?utf-8?B?WlloTldkNU1vZ293YUp1RVFEelMyTXJJZzJtU0RwSG9UendKcEhwdE05dFJ0?= =?utf-8?B?bXlBNjArS2ovTjFoV3dneHNZMG81MmVDbHBobkVHaGJwLzA3NklNWE1ZU1F4?= =?utf-8?B?TERhTGZlK3JLK0NGVkhVa3RadXZrTmorUFlEbUNiTStKK3FCbGRYVStVdjhm?= =?utf-8?B?UGRVNHdIYzJoSytZelR0UUNzMGRHYXZxSUdXSk1WUmN2NkRNY0tuUHNuMUE2?= =?utf-8?B?UHJwUWs4NTZPZS9VNG5xaFRrbGtaWkFmUlhlV2dDT1dHcVZjTERRLzI2U1dY?= =?utf-8?B?bUxoZ1BaWDdIYXB0MER3QlJJa2VzZGNQU1g0L1RKb3p1bzg0cGRoYU5zcHBo?= =?utf-8?B?S0hEb21YZjhqeHpQZWtMY1ZxM1dWSkpyNDl2aENkaXB4bmltM3ozNmplMDdn?= =?utf-8?B?S25LdUJjMHJZOHAxZFcrcS8wOElidzNrWjgzMi96ZGUyamljYU9CeUpxeElG?= =?utf-8?B?dVBiMzRCeXIvUDIxUmtycTdIZ2tpRHl2czZxM1dqZUVFaVZXbnRlY2gzQ29Y?= =?utf-8?B?eXhtRFlwMnNiUXJyRExkZ0R5cXRWZTMrekpMU0NJTitTdVFLNlA5UXhJMVNU?= =?utf-8?B?NWFuSkNvNWtrZTlScmxFTVA2S1BiS3RXVG51RHBQb0loMEhFNWVMbXBlRzlD?= =?utf-8?B?aFUwSkZVY2t0VUV1bUJtN3hOQjhuMVhhWm94akprOGxocDFvMHVKb3pxckdC?= =?utf-8?B?bDFJd0tka3NaeGViNmZUTjREU0JKSkFPN1RjUTRoSHIzWXZWZC9UbTFMOUsv?= =?utf-8?B?ZmZuMEZySHZrMFlvRXE3cFRlbGMxeTBLTXJNY0RUYTdTanFGQlNwMU5HaHVk?= =?utf-8?B?aUZXQStuTjVrQ0N4MVZ0ZFdXbHB6emVRVy9lOW9ZbGZRWU1XWWwyYklVSHNh?= =?utf-8?B?UnJpRlRVUnNPMjlwZjdJOGRZOVlCdUhRUEc5T2lueHk3R1h2bE82S00zdXVN?= =?utf-8?Q?b3UwPRxiOL4=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH7PR12MB5685.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZVc2VDQvRUVVcENSTUhrNVFuR2lHMTJTTExidG5RNHhDZFdyNzN4UnhQL2Y4?= =?utf-8?B?S2dHL3F6UmtyajRxaTFwck0zTEV6blVRYzRVWEtNUXl5NkNBUXU3OWVLWSth?= =?utf-8?B?bnRMMUxVbmQycWpxbVR1L3RjYllZNmQwdEY5OG40Mk5ZUjh6azNkSkUyWlEx?= =?utf-8?B?TmJIS2s5KzlsN3FiU1p6b2xLd0liTU9sOUtOZkhEUlZLZnArVUs0MVJ0S2Iy?= =?utf-8?B?bFFpV1ZtZ3RyN3lGU0V4U1RWSTdZUDVBVDV6NU9lOG8rdHE4b0Q2K3A1Rzcy?= =?utf-8?B?NmMwRlhhWXh3QlRYMGUvbTlEdFdBZGFTcmk3Z2p1YUo0TkZqdDkxVVh0RUxz?= =?utf-8?B?QVo4NTcyczMrQUM5eXVCN2tYS0laRENoTTY1RDRrK1NIYWJVWTAyM1dGc3dx?= =?utf-8?B?NzZPR1grcHhTZUNSRlQ1eE83bEwwWi9ad3FzOXNmK1dnNUw2YUJuTUV2Visv?= =?utf-8?B?akxLRzRGNTZvb0lHNThrcEFHY2d5U3oyK0hvbEdubXZPN3drM1BKU2FYRlVQ?= =?utf-8?B?TzNHT01RdFFiSUlDZTRaaGlodUdiMHcweVMxeXByRWdQRVZRWHVhZjlYci82?= =?utf-8?B?RHhMbnNibmsrZTlTdW1wYlhicDU0VU5mdERPWVRCZ0hGcXphTTI3Tlc5T0dJ?= =?utf-8?B?VFY5M1lMc2xzcXdZdWlDbi91Mk1ucmNHbTNPOVpJaWFJaVZIRUFFS2NpeVRV?= =?utf-8?B?SVhjYzZIc1MrUFFQWWVqQUpZT1dncVM1OUtUU3owM3RhYzdIMFdZVXY5bm9J?= =?utf-8?B?c2NDSlI5bjg2bmR4bjNHTiszWTJVZ0d1N0o4OGZpRmxMam10cytpVXZiOHRD?= =?utf-8?B?ZTVhdWVvU3IxUXNNYjcwV0lDWlhuNXF5SVMyQm5yNHJJZ2dwRktManErZ0JW?= =?utf-8?B?K0JlRkp1YzVmUWdJcUZLVDN0TXNPVmx1bFdXakdkcENlQWhha0wzRm1RQXJT?= =?utf-8?B?SWJzUjBjWmM1bGV0YUZaUEVMSlN2Tm1oaTBYNml0ZHNXaGhoNENJeHdZb3Ri?= =?utf-8?B?MEx0T2JnTzMvc0hYWFV4K0R6QXVMcmUxU2NvVXNjdkdNY3FYV3NFVmZaR1Yv?= =?utf-8?B?VWtPamVhclowTHJTdkJlb3o0bUNQWHo4YitZdysxcndMbExxMkt3UUV0SEt4?= =?utf-8?B?Qkk4bm84YXZjNTBLZUVDOFcyMHd2MUZiRU53Uks1WUZObmdhUzg4OGxDcDlP?= =?utf-8?B?S1hxR2lBQnUxaVczWWdQR2ZjRnZ3cGt3NTJhekNMaS9KbzNoUnV5Q1pEL29G?= =?utf-8?B?WGxYS1JWTWFyL29GMU5jQ0JvVXZ1aXd1dlZ0WnlQc0RRTUdpRTV5RHd3Rml4?= =?utf-8?B?M0x1NkxMVlRVeGpqa1ZLNytuckt3bGVCaFhNSGU4QnRad3pmTUloNFBnL21l?= =?utf-8?B?R3JmMEJtdVBsUmxPS3p3aW1IVCs3K1FLY0N1TUlyR0ZoYnBXaTdEbHZuZWVu?= =?utf-8?B?TkZJbDdLU1Y0dnhMSDdBQ1VjRjRpYVhtcWRkWkJQUkF0Q01UVVUxaklaNmR3?= =?utf-8?B?Z2FrNmJOelAwemoycnN6bjFKKzBMcGNMekFzbDJ3cVd2UlQvdjFTNGdGTWk4?= =?utf-8?B?R29kMVpld011dDZXd0MxdElTNHpPMEQ5dWNjYTJMM0U3ZGRGdnhuaHV3aUpX?= =?utf-8?B?VmRzcWovQXlkVFdscWpNQlB4UjRRN0pHcVAvL25Od1gvRUhRQ1BCVGk0TldB?= =?utf-8?B?Q1VaajhoY2U3SEVtN1hIRTgvQmd4VmhzdWRPTU1jUTA1aFJFZkMrRHE4T2tp?= =?utf-8?B?dkhFWUFOWitLclM4STFKNmVqa0V5dG0zanl5SEwwSkhrZFZ6MUp6bHIyNy9Q?= =?utf-8?B?ZWpRajFBRXhVYlBGMS9Fa1E4NnkySStOcmJ3a1hFSTRHWmdobTVKYWxtQUxY?= =?utf-8?B?a1Z0ZnRrbkVlaGVydEppZlp4cHdoU2tiUWRYbEZaUk9FZWdxdEZxZEFpY3Fs?= =?utf-8?B?WnJXRDBESDhNTXRSRHFOZ0R0ODFNM2pDRlo2ZWh3N1pTSzhMVEthSjRjYmFN?= =?utf-8?B?Z1NvcHR6ejFTd1I5MGM3bVh5akxzMzBjMy9tMmxqbTQ2TVJVc3BMTkRSalU5?= =?utf-8?B?QXQvYUhCUG5TeWFqdnNaY3YwMDM3RTZRUjNaM0NndTVLRlVPUjVpMGdxaity?= =?utf-8?Q?tDK6uhXT4AUWNCMYLpZorQLWf?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 02157fd1-f44d-469a-a811-08ddf0634093 X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB5685.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2025 12:12:03.5232 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7+bwpwtCI2z+j7itHfC5wjKW/dv3uzVycklr2LOaYkPrad8L1zb5Qfxn10KDeWN7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6276 On 10.09.25 13:59, Thadeu Lima de Souza Cascardo wrote: > When the TTM pool tries to allocate new pages, it stats with max order. If > there are no pages ready in the system, the page allocator will start > reclaim. If direct reclaim fails, the allocator will reduce the order until > it gets all the pages it wants with whatever order the allocator succeeds > to reclaim. > > However, while the allocator is reclaiming, lower order pages might be > available, which would work just fine for the pool allocator. Doing direct > reclaim just introduces latency in allocating memory. > > The system should still start reclaiming in the background with kswapd, but > the pool allocator should try to allocate a lower order page instead of > directly reclaiming. > > If not even a order-1 page is available, the TTM pool allocator will > eventually get to start allocating order-0 pages, at which point it should > and will directly reclaim. Yeah that was discussed before quite a bit but at least for AMD GPUs that is absolutely not something we should do. The performance difference between using high and low order pages can be up to 30%. So the added extra latency is just vital for good performance. We could of course make that depend on the HW you use if it isn't necessary for some other GPU, but at least both NVidia and Intel seem to have pretty much the same HW restrictions. NVidia has been working on extending this to even use 1GiB pages to reduce the TLB overhead even further. Regards, Christian. > > Signed-off-by: Thadeu Lima de Souza Cascardo > --- > drivers/gpu/drm/ttm/ttm_pool.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c > index baf27c70a4193a121fbc8b4e67cd6feb4c612b85..6124a53cd15634c833bce379093b557d2a2660fd 100644 > --- a/drivers/gpu/drm/ttm/ttm_pool.c > +++ b/drivers/gpu/drm/ttm/ttm_pool.c > @@ -144,9 +144,11 @@ static struct page *ttm_pool_alloc_page(struct ttm_pool *pool, gfp_t gfp_flags, > * Mapping pages directly into an userspace process and calling > * put_page() on a TTM allocated page is illegal. > */ > - if (order) > + if (order) { > gfp_flags |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN | > __GFP_THISNODE; > + gfp_flags &= ~__GFP_DIRECT_RECLAIM; > + } > > if (!pool->use_dma_alloc) { > p = alloc_pages_node(pool->nid, gfp_flags, order); > > --- > base-commit: b320789d6883cc00ac78ce83bccbfe7ed58afcf0 > change-id: 20250909-ttm_pool_no_direct_reclaim-ee0807a2d3fe > > Best regards,