From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2058.outbound.protection.outlook.com [40.107.237.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFC87EED6 for ; Tue, 28 Jan 2025 11:00:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.58 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738062023; cv=fail; b=nSL/QJrmwDRdVvSGDY9hul4bZQRI+wJ2nZIdKBeiniL9IST5fXDRfcZZtXK1Hd2edOzg8a/MZN6eRhOs6Uhh1vXiePeiH0E1eoO2bPoRUoF1GS5NXBba7NkYoPecILXZ675df+reV/9g+FR2L4j9nRuPbLP6t+9OF/RpOR87iQI= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738062023; c=relaxed/simple; bh=BYNnF4rJLvOWtw8aJcJlPW7GQO3mhH2PBpw5/VBD6N0=; h=Message-ID:Date:Subject:To:Cc:References:From:In-Reply-To: Content-Type:MIME-Version; b=e4DcyoYSXGiA8ERDH3vujK+E85efkc70m5Y+8AXzSMTjXjsKPppcV2ESTmTHWFGv7hj1ZsyOoIepcp6fG71Xd9SI35D2Ow68rQzznI+aP3K7p6MU7/xiW8eeAX/j79PCFxp4I3+bC59WWI4Wpv9zRDp49b/g7tQq5Xd4sREUlZY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ZHomkrW/; arc=fail smtp.client-ip=40.107.237.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ZHomkrW/" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=o89wrnryeqJaDZBGW+ux8kBETlZJYq0o6wlacR4C5y9dbHbfTUY8uBtH+Gwq3Spopps8NCyA/617ixgyafv1seQbt4kCYUo85L+rCTo8UqzviMVy9fqoO4Zl1cBSup5CIvTno/a67YaG1wbnHM9PZ7wl3zrE134vJzSVi58XpdwdRIYhT+jQnZufo4T0LUCLF8VYxqpGM6ZENRpUsUuXMtUoUxLLE2+eiIU1I+UEJv1eBnIGZCyV6/luact6ygyruOw5G4xiNsKKwT9jsVoLZFzbOO1bZHBtdMV67YW1WSBZ1ji6b4S3mFZhYxCcV0+DpnuTDWDlKCk3Gk1pezhAgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VGKS9o+f64iRNdANArTObuklmjeAYJk+p6YMgCVPlMQ=; b=px6S5koozsNQ/tSzXNhHwjdTRP9TvLgO/DoO/Mvl3Apb9RhR+f+X2fINKfU+8j5KfPKSHy24G8ku7slmJI+FDwbqCrdikcp7/QP8GqCHumOVSYqMJPNX9YEvtWCPacYmbGMyXbBuWhXE0xMfjjjAAx6SpT9gP/ln9yRsTg+T34xMC5VLuMbysgeIzqkwczEJ3xMMI7ocoEMWzQ6uDCSHVJLcDVoSHxI1Dl/eVUPW4dnV8hXAiXbwty7Xg4/siNPeMTMWNZVYVbxEWUWyvHrgT7/l3rWBdqdV7FmoTQA1HVLkVfmFxs7o7vLNaYWf9pcuMFbtGGwpDO0FqnnGeHgJYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VGKS9o+f64iRNdANArTObuklmjeAYJk+p6YMgCVPlMQ=; b=ZHomkrW/gJUXuoiI/D8RvEWhRPNXByI8X0ecqTaQgCl6q0xHWjfCHwfwvR/WHl6ouLoc5yhIK3JZVN2969ECI6hujZ6JyxE6jj8vWgsGUYMSC0eES9aL+jv11yzyGw2U9c+pr/UkeGZmzv5Z5trJSp+0n8LF09aqEEkJD5ayoPs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from DS7PR12MB6214.namprd12.prod.outlook.com (2603:10b6:8:96::13) by BL1PR12MB5995.namprd12.prod.outlook.com (2603:10b6:208:39b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8377.23; Tue, 28 Jan 2025 11:00:15 +0000 Received: from DS7PR12MB6214.namprd12.prod.outlook.com ([fe80::17e6:16c7:6bc1:26fb]) by DS7PR12MB6214.namprd12.prod.outlook.com ([fe80::17e6:16c7:6bc1:26fb%3]) with mapi id 15.20.8377.021; Tue, 28 Jan 2025 11:00:15 +0000 Message-ID: <93792758-bc88-4d84-bdea-f366988c2d53@amd.com> Date: Tue, 28 Jan 2025 16:30:05 +0530 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v7 00/12] AMD broadcast TLB invalidation To: Rik van Riel , x86@kernel.org Cc: linux-kernel@vger.kernel.org, bp@alien8.de, peterz@infradead.org, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Manali Shukla References: <20250123042447.2259648-1-riel@surriel.com> Content-Language: en-US From: Manali Shukla In-Reply-To: <20250123042447.2259648-1-riel@surriel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: PN2PR01CA0240.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c01:eb::17) To DS7PR12MB6214.namprd12.prod.outlook.com (2603:10b6:8:96::13) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB6214:EE_|BL1PR12MB5995:EE_ X-MS-Office365-Filtering-Correlation-Id: d408ab8f-007f-4d48-741e-08dd3f8af12c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?eEd3VGZmVWdZTjlsLzVXQjNLS0YvQWZQNzNpVlRobm1BVUt5RVIrRFVieE95?= =?utf-8?B?VERNbTRGd2JPT25vV1BPSC9tMnJDeFlLbm9nYUpOeitQVHlrTWtSWGs2QUdx?= =?utf-8?B?a2hteU1CUERiOHNnaGRHTi9RcEhlY2xOME5RclFNU293eG1HQXZiSlIxaW51?= =?utf-8?B?Vkl5RUNidThIN1F2TFlqUXJ6TzMzbHNKMkpOSGdWM3NEY0tLenRpdVhWREg5?= =?utf-8?B?emNRSkVtYVkyTjRzTXFDZER5ZytGSCtnUVQyU0x3T1NlaHFMeVR1U0tscjBm?= =?utf-8?B?em1pQWwrSXdpOHNJMWkrSjZPWkJwU2kwRmxnY1J4djN4QXhwL3JGZ2pPRFNV?= =?utf-8?B?MEI2OVRDbFhFR0hiVGE3SFV2RzE4bHdPWTVPOHVXQ2Vza0JxaVRFM1M5VjU3?= =?utf-8?B?cllQN0YzWkdCN2oxQ1V2djQwVjRPajJxdTc3UnFoUlNXWHBRUStkTkdWNDNv?= =?utf-8?B?cXVGRHluUTlQWTBCbGpqZFFvOWdpOFpaNjZIUnU0b2p1dXFRQ1VJaVJneUUy?= =?utf-8?B?RE1CUldQcEN3SVFJMjF6aUlSSXltUEFlclkrcUVPQ0Y4RmNuNEViYnQ2OGRi?= =?utf-8?B?enpmZ2JvckhSL2MvOWhvMVRWZmllZHlrWElLZDhneFlsRG9wd1VsT1lXVCt0?= =?utf-8?B?Nm84NnNpM1BwVFBIN2hiVFV4V2hTR1dVUFRLK3huS3g3V1hPZTY5QUZ5R1hS?= =?utf-8?B?ZVBSOUVMNkF3RjFnSStNUW9XSWRhVEQrYWlyVk0yR3Jod0cxR2t1WVlQbFVI?= =?utf-8?B?dUlON0VMYlQyOXowM3pYZU16QzZGTWJGbFd0MmFvbDJ0L3dBdmRUWkNaSHZw?= =?utf-8?B?UjNuTXVOLzR0MWtUY3BwejAwck9TdEZwd0xkOHFicHpXNVlxUngrc2pYUWZ5?= =?utf-8?B?VXo5OTlUZEJlYjRqOVA3QnpjRlF4L3BFU016NHh4eTF4a0ZjLzkraHB5VEg0?= =?utf-8?B?V2xMSG50RXpId0p4eUpLbG5oUmNlQ2Q3V1hmcitwcWNaNDliL28yZC94ZHl2?= =?utf-8?B?UXdKaUxpcTE1VW1jM1RXZUFhcDlvUVRtaGp4cmRvZkVjbnpUaDhTSUdnbkdZ?= =?utf-8?B?Ykx1cHhtQmRDL3UwOFpsWUFHbEFpYnZEdnVheDRwM1UrLzVicDZDN3FwTnVP?= =?utf-8?B?YUxuUzNyL2d2U1F3Z09oVmhyL01LV0FIT05QUlBuUHZ1amVxWmpMaGZ0cVlh?= =?utf-8?B?cjFueVdUMUcrZ1VnOGFjMDM4M3k5Q2w3ckprb256M1ZTWW8yRE1ad0YwYmVO?= =?utf-8?B?b2NFRkJ3VzBDMzdjWkRVNlNzaWZQS2YzWnNvNXVFZitoRWo0QnhzU0lMWWls?= =?utf-8?B?dEhSSGFYclR3SDl5SWthOFhEYXk5Nkx3VFhJVy8rODdnSDZCekdBNXNQM2xY?= =?utf-8?B?OWV0NWVhbGJEN1NmWDhITit4KzhPNXFGSERtQXVhaTY4d21nbG5hSUpSbll0?= =?utf-8?B?NlJ0YkFKWkU3dER3YktDR3NHazg3UzRPYVpMcW9uVUpac0pKR1RLZFczb0NF?= =?utf-8?B?YnNCOEQxNFVCZXRKTTBNSVZhK1VOU1hUYXZEOC92eVpGTFh1UGpPcjh6bjho?= =?utf-8?B?RzVwVHhtYzZmeFo3N1owZ0lLVjNIYllKdW1JWTFIaktSc3dhdzlHSWd4ZG5m?= =?utf-8?B?eEYzeFBjRERGM0JqZ1FjcUxqZnpubmprRklVOWZaejZteXVoSGlKaExkdWVT?= =?utf-8?B?cHZxaG1WL1l0bjB4d0Zvc0FwUEVlMkJ4UHYvQjYrZEFxYm9Vekl2d2ZQN1Fa?= =?utf-8?B?bUpEaW1GRVJheEhTZSs5Z2NiTndPSlVKMU13ZHdqaXRpZDBjc1Vrci9yMDRu?= =?utf-8?B?R0NBRWtVbjh6UVFzVFJYQT09?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB6214.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UXBsNmpydGJZNWtsSU5xYWtBd2pDNXZmdU9KUzl0enRIb25ZcEpqZVdJOHhC?= =?utf-8?B?cTZFOCtoRUZWSmFiSU5sR3hJalR0c1dWdlhGZk5uTU1wbzF4UHFITFRybndR?= =?utf-8?B?WFQ0N3gwU2o5ZTU4R2lKRkVwSk9yR1IwM2kvNnRQY3V6Q0l6Uzl4elhkcXR2?= =?utf-8?B?cHpMSlBROXR2MWNUSEdVb0ZPdnlsRTlsQ0YvWVo0UEUzb1lBdURLNTdRZU5J?= =?utf-8?B?MXR4YUs0RCtJSkxIakVCbWJNaFlYQkRwMTcyM2k3Wi9WM25uTU5vQkVKdS83?= =?utf-8?B?bmQ5R0RKQ0ZOdTh1SW45WU5mSndqVWRhQTJFZVlrT3huOXp6dklpUjdJeDN6?= =?utf-8?B?YzZIOFZvQWFnTXhOYngweWFEN1RKY25zNk02d1RrSDlEVmlMVWI2M21SOFAy?= =?utf-8?B?dUlWL21BZFJsN2lrMUNDczZWeGRzcHZmOFM0dDRKRkNydGwyTHFSTEdRemZV?= =?utf-8?B?aEx3WCtEMzh1QkFWeVlVUkhVeDA0Y0tQbkt5UHUxL2J2YjZqVzF3VmtZTWdm?= =?utf-8?B?bldpakcvOFkycVhBa1hQakxsZmRtMjhTL1luSm1jNFlXTnlZM2NWSGowQkxq?= =?utf-8?B?bnBMYnlSQno1SXBUbVB4QkNSNml1RTUwbUdueG9qRmJVV0grdUtRZlhBZkZR?= =?utf-8?B?VkVZdXlJTjBtNVhQS1hzM0ZkNFRFeWFITTdSVnk5eW5iTEVVakV2akovM1ZL?= =?utf-8?B?enhFbm0xeXRpNm5nM21OZVl0TTZxTXhYb0wrc0F4RnR3LzZlK01WYkZ0ZVl5?= =?utf-8?B?LzRETzJoNUJvUGFjSGZrdVFybytnOEhhY0dDQW5MNVJpZnp6ckk5WjIzVUtW?= =?utf-8?B?Wk43bkw4MFFCNmg3YlBBWWJVOHROMWY4N1diWFIvWGdFYWdIdTkxOCtINUhO?= =?utf-8?B?YWFGb2RvTjJYNGlmei9LKytOcmVpbUpEQmtMMlVuK0RWazVIc2dMQlkxL0FU?= =?utf-8?B?Unp2Y0NkKy9JeUNkM293YlZ5UURXZ2o3dFRTTktXbWFjUkoyUWxKaXl3b0Nn?= =?utf-8?B?Rlh5blB4YzBkejNzRndDSnhvQUt5cERmRHRWWEtpcm9xV2FFTXRFdXhpZ1VS?= =?utf-8?B?R3BWZllUSFRHNTNnZWlOaXdsOEJRdlNVbWhhTXFXY3Znb203OW5wNzV2bGd3?= =?utf-8?B?YzhNRmNoVkR1RGV2a2k2ZElEUTgvbmNpcGRleWYyL0QxNkc0VFFBeUlTcnlO?= =?utf-8?B?UXNZc0QwYm8vV1ZTZTV5UHVhd1FzRHQrV0l4Nm5CU1lOOFVVcHJNcGgzbGMr?= =?utf-8?B?NlRDTGtDZ2pYd0VCMjVScFhZYjA4MTFhR1A4SG9JL0pyell4TnE4cjRSL3Nn?= =?utf-8?B?WVRCUUdTcnJxTTBPRDVMOEFCaG96Yk9TN1Q3OWE0cFFsY1E5UkEyZTRpSE1p?= =?utf-8?B?RlRROURhMzJQclhNOE5lRzJJVXIyV3p5d0Fib0s5aVFJRm85TUdPSmJyQWFp?= =?utf-8?B?UUVKaU5NQkxIRGo0d1RwOGpTY0N2aFV6TmF4MFhFeXVYbzlsUVJIV2s4cmZo?= =?utf-8?B?MlJkaVprUkFFZW9tdXdCRHVWbHIyZzRxc0NLL3pQY0VlNjF2aFJnN2prWnlG?= =?utf-8?B?czJRWUpIVm5KMko2S2RSWG1qZEJ4Z1QvbXA3VENNV2h3aUk3U0YycHBpemZD?= =?utf-8?B?K2o4UkJrNzE4Z3cxQzZBdGluaE9Mb2FEaTZ0cUk1L2VVRkE1Q2hJV1czTnRG?= =?utf-8?B?OWhXUkR0dDlUYkJZamRjVUtNSlBxS2M1d3NIVjJkNVE2N1VEQTNJOVhmOUUr?= =?utf-8?B?a2tINXl4SmV3bzQ0cHVpQXM3N2V2TjNWNUllbmFpTCs1QUYvSnhTem1MY0Iz?= =?utf-8?B?QWc1YVNsWDQ2T2VCanczc0NoVlZ3YmxZb1FtL2JjZ2xLdzRlUlluRktRVWdw?= =?utf-8?B?eW9iNmd5VFN6N21PVjBjR3FsalFqZ0RYWm9BeTRNY3BUR2RIalpiUDV0NVpn?= =?utf-8?B?ZEJoMU80QnloU0xnVHNCV3hhb2pHbHkxc09JODR0Um1XRjMyS21EYkJmQjRl?= =?utf-8?B?dkZnTnBRUVBKbllPRW9xZVRHdzh2Nm5IR0huaWkwN1MvWkpDWmwvbnpSMlVr?= =?utf-8?B?VU1IaUY1MW1sVUdPcnN6R095b2ZBMkRQdnp1OUc0VmNrWUhSMU9UVWQvZTJV?= =?utf-8?Q?I9tPyBBawOr6Zyxu3nrWZ4G1k?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d408ab8f-007f-4d48-741e-08dd3f8af12c X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB6214.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2025 11:00:15.0907 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tacqqqD0dhfwiSVU02wxfZgPvhANzv2cgKH4+Lr8SLcdLHZ3ygNGXsvkLz/T1ZBH8PEVxip9hdQXTzhNRLOuJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5995 On 1/23/2025 9:53 AM, Rik van Riel wrote: > Add support for broadcast TLB invalidation using AMD's INVLPGB instruction. > > This allows the kernel to invalidate TLB entries on remote CPUs without > needing to send IPIs, without having to wait for remote CPUs to handle > those interrupts, and with less interruption to what was running on > those CPUs. > > Because x86 PCID space is limited, and there are some very large > systems out there, broadcast TLB invalidation is only used for > processes that are active on 3 or more CPUs, with the threshold > being gradually increased the more the PCID space gets exhausted. > > Combined with the removal of unnecessary lru_add_drain calls > (see https://lkml.org/lkml/2024/12/19/1388) this results in a > nice performance boost for the will-it-scale tlb_flush2_threads > test on an AMD Milan system with 36 cores: > > - vanilla kernel: 527k loops/second > - lru_add_drain removal: 731k loops/second > - only INVLPGB: 527k loops/second > - lru_add_drain + INVLPGB: 1157k loops/second > > Profiling with only the INVLPGB changes showed while > TLB invalidation went down from 40% of the total CPU > time to only around 4% of CPU time, the contention > simply moved to the LRU lock. > > Fixing both at the same time about doubles the > number of iterations per second from this case. > > Some numbers closer to real world performance > can be found at Phoronix, thanks to Michael: > > https://www.phoronix.com/news/AMD-INVLPGB-Linux-Benefits > > My current plan is to implement support for Intel's RAR > (Remote Action Request) TLB flushing in a follow-up series, > after this thing has been merged into -tip. Making things > any larger would just be unwieldy for reviewers. > > v7: > - a few small code cleanups (Nadav) > - fix spurious VM_WARN_ON_ONCE in mm_global_asid > - code simplifications & better barriers (Peter & Dave) > v6: > - fix info->end check in flush_tlb_kernel_range (Michael) > - disable broadcast TLB flushing on 32 bit x86 > v5: > - use byte assembly for compatibility with older toolchains (Borislav, Michael) > - ensure a panic on an invalid number of extra pages (Dave, Tom) > - add cant_migrate() assertion to tlbsync (Jann) > - a bunch more cleanups (Nadav) > - key TCE enabling off X86_FEATURE_TCE (Andrew) > - fix a race between reclaim and ASID transition (Jann) > v4: > - Use only bitmaps to track free global ASIDs (Nadav) > - Improved AMD initialization (Borislav & Tom) > - Various naming and documentation improvements (Peter, Nadav, Tom, Dave) > - Fixes for subtle race conditions (Jann) > v3: > - Remove paravirt tlb_remove_table call (thank you Qi Zheng) > - More suggested cleanups and changelog fixes by Peter and Nadav > v2: > - Apply suggestions by Peter and Borislav (thank you!) > - Fix bug in arch_tlbbatch_flush, where we need to do both > the TLBSYNC, and flush the CPUs that are in the cpumask. > - Some updates to comments and changelogs based on questions. > > I have collected performance data using the will-it-scale tlb_flush2_threads benchmark on my AMD Milan, Genoa, and Turin systems. As expected, I don't see any discrepancies in the data. (Performance Testing is done based on 6.13.0-rc7). ------------------------------------------------------------------------------------------------------------------------------------------------ | ./tlb_flush2_threads -s 5 -t 128 | Milan 1P (NPS1) | Milan 1P (NPS2) | Genoa 1P (NPS1) | Genoa 1P (NPS2) | Turin 2P (NPS1) | Turin 2P (NPS2) | ------------------------------------------------------------------------------------------------------------------------------------------------ | Vanila | 357647 | 419631 | 319885 | 311069 | 380559 | 379286 | ------------------------------------------------------------------------------------------------------------------------------------------------ | LRU drain removal | 784734 | 796056 | 540862 | 530472 | 549168 | 482683 | ------------------------------------------------------------------------------------------------------------------------------------------------ | INVLPGB | 581069 | 950848 | 501033 | 553987 | 528660 | 536535 | ------------------------------------------------------------------------------------------------------------------------------------------------ | LRU drain removal + INVLPGB | 1094941 | 1086826 | 980293 | 979005 | 1228823 | 1238440 | ------------------------------------------------------------------------------------------------------------------------------------------------ | LRU drain vs. Vanila | 54.42% | 47.29% | 40.86% | 41.36% | 30.70% | 21.42% | ------------------------------------------------------------------------------------------------------------------------------------------------ | INVLPGB vs. Vanila | 38.45% | 55.87% | 55.87% | 43.85% | 28.01% | 29.31% | ------------------------------------------------------------------------------------------------------------------------------------------------ | (LRU drain + INVLPGB) vs. Vanila | 67.34% | 61.39% | 67.37% | 68.23% | 69.03% | 69.37% | ------------------------------------------------------------------------------------------------------------------------------------------------ Feel free to add: Tested-by: Manali Shukla I would be happy to test future versions, if needed. -Manali