From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from YQZPR01CU011.outbound.protection.outlook.com (mail-canadaeastazon11020087.outbound.protection.outlook.com [52.101.191.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6908E2066C9 for ; Fri, 10 Jan 2025 19:16:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.191.87 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736536621; cv=fail; b=aPCrU9vo5lRKiYKgCKxshl+B+iH7+bBdevUov2K/6XlzlTfH2TKtSXU2Q21QmR1rP/RSzSQqspHoRcxUpd9RDswneUj1h2MvlhahS8LpZPVNGmb7LCOxeN/3ug4s2OaBBzuChZD/2JV8LXxcidT0Sk3aQP2R5ee6AyJP0dIt/q8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736536621; c=relaxed/simple; bh=L+fOpkb0RuNxZF0LgvlrIwjkBoD7MLzcOUBj0e8sQcY=; h=Message-ID:Date:Subject:To:Cc:References:From:In-Reply-To: Content-Type:MIME-Version; b=uNmDRwlJD0V7LqCUVQdnOlq2aRpIOv3OlbTh1iJ+VN0QektGSawW0SFXarIaObSDRc05AHW8ZFHYNz+17VOfCLv0+wt+VmS+mhHPSwNrBrBGcGW3Bn8r67s7cI2++WRmJtTle7ikJh0XNXlziuPH5UkqZjbn9Je+SU8wabegSwY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=efficios.com; spf=pass smtp.mailfrom=efficios.com; dkim=pass (2048-bit key) header.d=efficios.com header.i=@efficios.com header.b=BJYr8sRe; arc=fail smtp.client-ip=52.101.191.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=efficios.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=efficios.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=efficios.com header.i=@efficios.com header.b="BJYr8sRe" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NBe1YNosddWSwjXZOFYsT6NHPMmPlwW7JvwPyPTRtvVgZuNbbhswX8LfYBzkh8+eERwUl2/N/pwMm4+OzDAjKe8zVmJFT0vVTjUDlvKhfMXYkEalXG0RFwG8PUsrsJ4plGTL3+ZfqzED3kgqdANABDChXcBMiXhjOG/n+BudIGEneK6N9zj2Ejyzf9tI3jnB7dq0PzTncc18izNapJjBu0Gn0WQh0QZhpiiDZZOVuFQVenjQrgqDVja5ozRTBeL1EHWs9wDTRwRUgTe+bFidRKKnWSWqbbLtrvrGzXKUUrdylwfzvmEVRrH56GvBRfoQiY9jsbwLkKfMPZ4SVaYt8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Urk2N1I57Th6ZBnM8a/946RI0ziMDdI68hXzP1o/wO8=; b=JG7dM/2IcWg0nKjyjC8MeuzAfQEv00dL2+vmZIJCPkrsgxQezS2KRWh7Vg4W4bFduRF5Pz3ajK2RYZt8cPsV2EfN27mZIIRc7nR6X6q48z+Hx+EDL2aCgJFECX/7HjoHLxLq5JiKZCw2DXHl6udBWvOgWjDKUDpL1pAAHivpKIuNYhvgIY823NB/iPrIzyWLL02OzMJFEBtn1RiA4HzOy4pf9PG7+e9DWNjn25yXDVFTsmdQ4HWDlwHN8aBqq4T2IiX6KCVdFS/Wbh+cGbN/FaWP3V9BzVVCvfQGaf6UKOVPfPYRZMYqBjZ0CL7x3WHAzgCrNSGaSn7YHuou/x42YA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=efficios.com; dmarc=pass action=none header.from=efficios.com; dkim=pass header.d=efficios.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficios.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Urk2N1I57Th6ZBnM8a/946RI0ziMDdI68hXzP1o/wO8=; b=BJYr8sReAoWxeBLU/S7gLxmGFmg1A4ktBpQ191X1uq7y/c9nbYwnW9j0F4CZ1LqNhCMgkw0IYB+7l8j2S7GShJp9ql0JoVuxZaQzgWb3bejdeSbFcNpsiakQEa8ClTBjyXKAw5FE24M8H1wJfbRtbtmI7k66zRzVAwoUAkoUfKzho+PkjSY5Jd7VVijJYK3tkPc6WvEJs0KBbwrN6saA5hHBwY7ErvUHnaIVh6h4UG6wvE/Cl2+JZVkJj/HUTbm6dQg822untcNt373/vq3Uu8VUCKgoFZEx/qn2pMdLyhIix5WH932o733jQ/nqW/OUbUQ3/qXOsRt1JTpvt+Zd9g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=efficios.com; Received: from YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:be::5) by YQBPR0101MB5765.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:39::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8335.14; Fri, 10 Jan 2025 19:16:56 +0000 Received: from YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM ([fe80::50f1:2e3f:a5dd:5b4]) by YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM ([fe80::50f1:2e3f:a5dd:5b4%4]) with mapi id 15.20.8335.012; Fri, 10 Jan 2025 19:16:55 +0000 Message-ID: <2c625f72-ed5d-4426-abd9-5d80bfe40694@efficios.com> Date: Fri, 10 Jan 2025 14:16:54 -0500 User-Agent: Mozilla Thunderbird Subject: Re: Prevent inconsistent CPU state after sequence of dlclose/dlopen To: Florian Weimer Cc: Peter Zijlstra , "libc-alpha@sourceware.org" , "carlos@redhat.com" , Mark Rutland , linux-kernel , x86@kernel.org, paulmck , Michael Jeanson References: <20250110165412.GC4213@noisy.programming.kicks-ass.net> <8c1ad304-61bb-4bdf-aa75-8633f3d0196c@efficios.com> <87ldvitx0t.fsf@oldenburg.str.redhat.com> <87cygutvds.fsf@oldenburg.str.redhat.com> From: Mathieu Desnoyers Content-Language: en-US In-Reply-To: <87cygutvds.fsf@oldenburg.str.redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: YQBPR0101CA0353.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:6b::27) To YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:be::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: YT2PR01MB9175:EE_|YQBPR0101MB5765:EE_ X-MS-Office365-Filtering-Correlation-Id: 08503a44-0e20-4fa5-553c-08dd31ab58a2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?SDNJZWphYjVrbW9rU240eXhUZ0pKc25seGIxVGJOK0lFLzZtOXp4Ky82TDkv?= =?utf-8?B?d2dvM3FuNG5iYkZvRytnZHV3L2RiY3lSY3l3akFsQ0M0ZHp5QjB0UG8reXNN?= =?utf-8?B?em9iT09LNU92dHRtWGZwSm9oU2NJVjZYU3JoVDZGbGd1R3dXMGw1ZGR0UjNr?= =?utf-8?B?WU9HOW1BN1hvTy8wQTBzeFBPZjQ3VW9mU1FyRGVNeXEvOVJzdEQzNG91cTZl?= =?utf-8?B?U09MV3JzR2taQkNIVWZ6WTkycFYzZndWb0RTaHFkZWxrWjd4ZUV5a1pQVElk?= =?utf-8?B?UXdGcDJucW9ab0hkNnpZa0ZOY2pSUVFDNVBVSHdGci8xa0JxOGlUR2JNbUtU?= =?utf-8?B?TlRsRVlqbXljU1pnVzFPNWJ3dG5YWUU4SVQ5dHd4QmMxc1hLV0RJSURBTzRr?= =?utf-8?B?MUJmQ1dMNWlTYWtmeTNNK2Nqcllid0dtOUo2NG1Mdkc2clZ5Y0kxQ0UyVHpS?= =?utf-8?B?cC9OOCtjY3d4Q3prdHNJYXpXd1RqdVMzcG5LQ0pNK2NLQVp4OVBlMjI1N3Vi?= =?utf-8?B?TTFacE1zU0s1L3hCcml0RG0raFNMaDdNNTN3N2FhNktnTzR3T29FTFJ2ZTgx?= =?utf-8?B?MUVmVjh0NjhWeXZha0RMN0hmemRZTnZtZ01BVEVUbWdlQVVFRXEzTFArN2x3?= =?utf-8?B?Z3ZhZVppUHVQMzgzVUd0alArMXpvT1JaVXNaeGZiY2RNazJTTmRrWUJna1lt?= =?utf-8?B?K3JsZFlLZHEzdEppTktUWDc2YUd6dHNLV3JvbXJ2N3JmdWFJWVdqd3YyTlNJ?= =?utf-8?B?YmlKZVNIS1hpeUExVmZrdHdFaXpUcnRkbWhEOG5EREpwZ2tDLzZEQVk0LzRm?= =?utf-8?B?MmJGWmJEWCtiLzNSSkpZTWFoUmtzelFwWWY3VytZd04wRmpDdWdaaVVCYjB0?= =?utf-8?B?U3B2VHhHMFZNKzZsNC9aa0tXalhrR1FMekZQS1JpMGVkNHB6K2F2TUJHaitK?= =?utf-8?B?MVpCd09nZ2xWM3F4SzhFV0tvMUNoVFY2S1JiZXdqclE4WlZBbWZxbVhMS3hL?= =?utf-8?B?VWt0cnNIM1dRSjQvcGlYWlUyaWE1bmlJN0Jxem9sNGRwbXpOQ1RWYUlVbnZy?= =?utf-8?B?Nks5TFp2U1hwNGZmOG9tZ3lRVlFoRjAzRjlpK0psWnhoOEd0Z1JXanZMaFNW?= =?utf-8?B?cmY1UDVKd1NEd1VUYllVU1VndFVCWmhxSlFBTERoQUlWZ2V5U2hvNGxsdzdi?= =?utf-8?B?MVppRTIyLzNmdE5vamlzRUVKd2huakt5eFpTbjRXVmJpRXROa20zZHFqOVJt?= =?utf-8?B?aS82WkxFTFUyalQwSXlTL0tHV1NnQklIUVhtOER3V3Qzbk84T1dQTjFlMENI?= =?utf-8?B?eEhjVzMzbGdwQzUrSjBTdzZxenhzZlNjZmdWMXhQKzVGcFFHQlQreXlVczBB?= =?utf-8?B?WUIwNWw2ZnpDdlN4QVNDOTlMMW9lV2lxcFZMaDNuMUJOR2trU2J4MjJ1MGVx?= =?utf-8?B?RHg2TmY4M0Q3bHZha2hLUlEzUHIzd3l3ejEyV2Jnay9MVVlZZzdpZUwzZEY5?= =?utf-8?B?UDUzeHRCSXIrSlV2MVBldHpsL3ZWalVCMzNEWXF5MnNGSmdZWC9HeWU3cTli?= =?utf-8?B?ZExGREFETW1LSkhMTDdJUWU4YWt3cmZSRDJmNFI5bUJ6bkQ2OEdwdktuQjM5?= =?utf-8?B?cStrTjd6bUQ5aUxZdzg4NkZud2Vmb2pJZHoxcmRFMDBhdzRNSFFxQmg2QzZk?= =?utf-8?B?REJmL1hJdzRrajNVSjlMaEtIaDQwYlRvOFdUMnlvWEpycWwrSTYrc3FtRUtW?= =?utf-8?B?d2doVGl2UFZQVWlCWDFTc04rU1pnd1Y2YzdaaHVzRDRsUjlxLzVtM1BmOVBm?= =?utf-8?B?NW9WeXp4QzVFWWtmTVY0QT09?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d01VNmJNTHNoNXY5ZXgzcXY2cGF6dXFISk9Eb1p0RDJiZU9NUkdLK0VnZkxq?= =?utf-8?B?dmFDODZ6RUNGb0x5TG5raVp5Y1dYb0w3dHdzTHlUbFdvVlNXZExLTEw4UUFZ?= =?utf-8?B?b1NsV1V5YWhNdS8reFV5MW9CNUg4aktHT3p1VjI3VTE2bHB5d1k1dTFDVCtJ?= =?utf-8?B?TTZLWHZJalZIMU1SNENHaTNla3pISUNLUHlINm5SQ01wcnhhVDl6bGNyM1pi?= =?utf-8?B?NzZWSzgwSU1wVUtob1BZNWRSdTZtSVJWWWRvbG9tWFJkUjhSeDJRaHJtdXFo?= =?utf-8?B?cWFTMUxQM3dNVFRRUWZrL29pTUFrRmhSdHgwUVBGTW1wV0o0b1RaMnd6TzRC?= =?utf-8?B?STIwKzhoaUY2WlFITDVQd1ozeUFvK2I1aUVGQlFocEliUmQyRTI4Z0w0Njhu?= =?utf-8?B?VGxMbTJSZThBNFN1aTM1NHkxOWdsOStnR21teFYrODRBVUVNWjFkRFFHR092?= =?utf-8?B?TkRYZ0lvSGNZQWpBUWk3Wk1pZ1lneGZlM0c2R09sdnZ3aUN2Q1NsaERocGVS?= =?utf-8?B?Z3lCUjdwaC9xU0x6YVlTUXZuY0dwR0FvcC9yTjRqSDlYUUl6MTRXejEweENh?= =?utf-8?B?YW1aR2hvN294OGVoaFJSNlZ3SE5NQmkyckJKc1MybTRzRG1aNkpxRVJoR0U4?= =?utf-8?B?WVIxZTZ0TDJQN29DWG54U3ZtMndzTklCVSsrK0plcVVVRGlNR05xTjhFcCtY?= =?utf-8?B?MXNjN01tekl5eGkzNE10UFlkTHQ2aHM0MGpEZXM4aElwL3pSVWV2ZEtpbTFF?= =?utf-8?B?clp6cmNyTDVEQU80SmdNSzhhOVRYOGNWYVZGOXA2eTFyZTQ5Z0JseVVCTU1i?= =?utf-8?B?QktrNTlUSEV6Q3lrQ2Z3MzE4dk1kbFNMdU9WVlVSZTlVVGp2aUFUVzV0UTV0?= =?utf-8?B?WW1zN0FzUWpzbWZ4VGZDMGZ6Z1NXK1Q2cTZ6cHlaVXdVcDM4cXR2UzZ3S1dG?= =?utf-8?B?NzQrU3N1YktYWDVrUHNyenoxSC9CTFZGWFN6L1hCMWNxOFVMM0RVSktCRDdu?= =?utf-8?B?UmFlanR2VzR1TVczUmcyQmpoM0gzSUUxdWhNT0xqc1JNamhpUDFmbFN2QTM1?= =?utf-8?B?WG5XSkZtRFNVUEthejEwcy84WUFtOXdqQVo5VnhmcEFQT0NsaGlTZGRvV3lL?= =?utf-8?B?UG1nWVpBOXRRZzlIUGl6MmlpYndvSVNSYUNTeWpnVk9NdzJuRmZXNy9xbkpx?= =?utf-8?B?RHdoUkVJK3ZWWldBb0JRZmVVRldCRXlVYVRJT3JZZC9ISWthaWRmRkJKeW1D?= =?utf-8?B?MnpSRXhYdHNzOGdyMEY5RDVlYXpvOEtWbjRJSmx1R3dyVFE0K0hMTXRzN0l2?= =?utf-8?B?eU54Z1JvTE54VjhjZEtyZ0FWYVAwTCtYT2tQS0tQbmVZbmtrdk4xNWNmam93?= =?utf-8?B?dGVTM3l0Z0N4dldmdHl2NEpvTS9ISm1zaldBbEhBTkZ2eU9PSExMZWdsZGVw?= =?utf-8?B?TmZ6TGZ6QzJ3aE5SSkFFbHpVWllZVEM0OWlWRUdKUERtUG9kK1BZQXJwc0ww?= =?utf-8?B?M21RTGZiS2dvZU11MG9XRlFXSmJkSUVuMld1cS9JcHZsUG9MM3JLZlh6SW43?= =?utf-8?B?a2VheVZiT3NqSGxBQXVQdE5pK1hGZGpMK0hmdHFjczVkMnlCWTdxTk05dG1X?= =?utf-8?B?clVtck5CVkJBenE0RjVKb3VrclFlbjV3RHZ4RmVDODk4ZTk0MDVkOFcrVU9L?= =?utf-8?B?aVNRMTZWMm9NUFFxbTVLUHRiN0VzSmx4UlRnMEEwWGcyUFFHdkxzRnZ0cFZ0?= =?utf-8?B?QmtXdnNsbmV6WExTSVRuWWMxM3hJUSt2MTNhamlBR2x3ak5XMksyT21uUzlH?= =?utf-8?B?bDVjVXUyZUR0QjVFSkgrbkR5VHAxT1lHSUpXeW1VZWprYVl5OXVtZjdpVFhv?= =?utf-8?B?L201REJMNHVEbmk2V2RxcjNnSnd2TVgzZ3AvOGRhbGJYZGJKcEt3NU9oNjRY?= =?utf-8?B?clpDMGhNanhvLzB0TXlERFp3WFJYWHprR3F0NUkyZTF6YUROWEErZ0JoUkRV?= =?utf-8?B?QlpVTEhWM1dOYWw5azVOd2lFTFFWZGo1QjdIcGNleURUeVlhNXdVeTdrTTE3?= =?utf-8?B?cEFuK0NMMUFQVXdFb1BaNDN6Tk5KU0JDRkQ1QTdZQXVTWlJwdks3VVRMdjhZ?= =?utf-8?B?TTdEQ3ZwNi9uV2VhLzBnVXlNU080bEd6T0c1Z25SV0FEaU5tQmpwK2MrWFhz?= =?utf-8?Q?ZusAbiu6uEHv/00RuzJjjQM=3D?= X-OriginatorOrg: efficios.com X-MS-Exchange-CrossTenant-Network-Message-Id: 08503a44-0e20-4fa5-553c-08dd31ab58a2 X-MS-Exchange-CrossTenant-AuthSource: YT2PR01MB9175.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2025 19:16:55.3649 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4f278736-4ab6-415c-957e-1f55336bd31e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: l0ZglMxiCxXFkQsCdT9d2j8cOFqfCucAV0eQ/GTXekIxvlb6BHXxThU8GuPh1w5lY2E4nhZy8fe3BS/W3DlxeYklp0pnwlMLpO4Mm7H99Y8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB5765 On 2025-01-10 12:46, Florian Weimer wrote: > * Mathieu Desnoyers: > >> On 2025-01-10 12:10, Florian Weimer wrote: >>> * Mathieu Desnoyers: >>> >>>> On 2025-01-10 11:54, Peter Zijlstra wrote: >>>>> On Fri, Jan 10, 2025 at 10:55:36AM -0500, Mathieu Desnoyers wrote: >>>>>> Hi, >>>>>> >>>>>> I was discussing with Mark Rutland recently, and he pointed out that a >>>>>> sequence of dlclose/dlopen mapping new code at the same addresses in >>>>>> multithreaded environments is an issue on ARM, and possibly on Intel/AMD >>>>>> with the newer TLB broadcast maintenance. >>>>> What is the exact race? Should not munmap() invalidate the TLBs >>>>> before >>>>> it allows overlapping mmap() to complete? >>>> >>>> The race Mark mentioned (on ARM) is AFAIU the following scenario: >>>> >>>> CPU 0 CPU 1 >>>> >>>> - dlopen() >>>> - mmap PROT_EXEC @addr >>>> - fetch insn @addr, CPU state expects unchanged insn. >>>> - execute unrelated code >>>> - dlclose(addr) >>>> - munmap @addr >>>> - dlopen() >>>> - mmap PROT_EXEC @addr >>>> - fetch new insn @addr. Incoherent CPU state. >>> Unmapping an object while code is executing in it is undefined. >> >> That's not the scenario though. In this scenario, CPU 1 executes >> _unrelated code_ while we unmap @addr. > > Oh, so CPU 1 initially executes some code, returns to some safe, > persistent code (“the execute unrelated code” part), this code > synchronizes with the dlclose and the dlopen that execute on CPU 0, > obtains a pointer to some supposedly safely published function in the > newly mapped object, and calls it. And that fails because previously > cached information about the code is invalid? Correct. > > Additional awkwardness may result if the initial execution is > speculative, and the code on CPU 1 only synchronizes with the dlopen, > and not the previous dlclose because it does not know about it at all? I'm not sure I follow this last example. Can you explain further what you have in mind ? Thanks, Mathieu > > Thanks, > Florian > -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com