From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67581206F05 for ; Fri, 10 Jan 2025 06:08:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736489298; cv=none; b=YuE8k37Jmv2wkQYsz7Jtd2S6mmb51SJ3fys9RF8WLUM+msYgdXEG021jMmCPI0xC1N7IeWI6UZ5G3PUdFE08PEalMD8Gm8+EdmpZUmOzDW8/K/mtmPwMHD7+NK5sZTblIangSIRCeeltKUx8X7S9xQW0DOhfe11QcBgjzsivSrs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736489298; c=relaxed/simple; bh=88oRuWvYHW2P+pwcx8ZmEj1KOqoYj8ynBFVphUSB1c0=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc: Message-Id:References:To; b=L8XKM/yQCfdErgLspB295WGCJ5jH4k6oQxmhxF9s8m+MNiOnH3PjlTBr4DvdDOzgG/GYSnnNR+UNCyaxnzF5W2Kqj+vxQXLLpo2B/yOQUajklLLO/DQuwlkW2o8t/i7JSwJ3ylo2rYtb6xI5u0oy97G64kNhCD1jYhSWCh6okaA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TqqFuZz0; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TqqFuZz0" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-4361c705434so12523465e9.3 for ; Thu, 09 Jan 2025 22:08:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736489295; x=1737094095; darn=vger.kernel.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=AIF1eAxxJoLk/2NWoF0H87GjzPYtUPj+T1xrd8DQ6Jg=; b=TqqFuZz03BfFmUkhFOqJ1dFpn2mP9vL3Au0nrk0Bq6vItjK7hgA1XgSqChVR2KvcJQ uHrL8UtkOeZ50V0bFlIdeZ9r2ZylFa0T2nXb30N3G50LNX5nASHySGOoDO5t1VYFXYgJ aU26tFM5de/Hu7djPyhpg+rFWZmUxn1z7ohKeaOXEkDfBiDpJzCIRE6NOF/wUCV9X+rR nHf2r2kkFURQvhvExDJaTcD7ErvEjNUxaUhrnG54QgYGxTp3ckEMzZXm5uSHbmJqdtVW 89P/HaKV5jZCOlYYeAHCZ9kvFnt0Fm9t9BFIHZQkkE7rE5RYCjT23ZU4dFxOEL2/Y1J+ dL4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736489295; x=1737094095; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AIF1eAxxJoLk/2NWoF0H87GjzPYtUPj+T1xrd8DQ6Jg=; b=kLkWVY9Y5LkcfIPxGuAEv+Y/rugTJZFM6wxaKz4o45FPZclRSQmeq5V706QqY15LhB WO9e7Ndw70jkMMjTIQD3W300gWlJXISNZv69pg1V/dXwRFGpmlIE/j2H0CXB0y6AS9ou SbPisECNB9TTnSwYhUicgNfLgtB+bh8yL/Y2d5IZ3Ir8rU4Gz53XD2jklr2wcHdixxay YLsF+HvSpdLuDQc/0vmOa/XRplReqvxYH+ManAv5fevMVCnIP2NxU0DdgFUtLvlXaXyw S3uWE7vNsn1J9bo8+PISZUUGIkLKDZSL/D27QnIqAp56VqHbJTKIeANITDhczh3QdnX3 ZldQ== X-Forwarded-Encrypted: i=1; AJvYcCVJSko5zWgGjdsG2PqyuBGU8hMSm6RESfDGepIXAIt07ej7sU5qWso0CqwaEdKP/MqWZMleMGXdO75uqoA=@vger.kernel.org X-Gm-Message-State: AOJu0Yzhshy1i9xDkSWYiqVIHKX2Hnv0o7X2ZGlXCA9/dB05SSMC+N19 su4Fs2pmg4b6FtCdQV5BSbgMj1W25UtEi5chS3d5JnMX46zEfGYe X-Gm-Gg: ASbGncs4Vk4Pn/8/IOF/SGVACp4yPTBZrO7madYEhAvqFz4ESnhVwm3UrsW+a1jOun5 xya21K+pyhuiy0JRCFLpiN3aEmSgs7XwUax7aTO6jwreHxkWrLH6/vcoRWwHHsIB+dr6ElX/1bJ Lthn2hZ+B7vq6s0C5wlmNss20yNYhyyEIS58rtnHrM/vjbRPAOTSv0hSK2EVfD0R4CS0H91NvBF uKBTinbPSDdxJZwPgSVU6RDWMsVvtIm6kKZ6i2zaKM5zSuxd4iK7eOv7Ft69igFUH5YmA== X-Google-Smtp-Source: AGHT+IFpXyaxjC8xi/oV4JH+nccKAN6bFaS024FqLgGassM9bqlo+KptiQ6c6GwwWuoItH5iZH5Xxw== X-Received: by 2002:a05:600c:3b0c:b0:434:fbda:1f36 with SMTP id 5b1f17b1804b1-436e26e51a0mr82585625e9.20.1736489294392; Thu, 09 Jan 2025 22:08:14 -0800 (PST) Received: from smtpclient.apple ([5.29.8.141]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-436e9e37c2esm41296195e9.28.2025.01.09.22.08.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Jan 2025 22:08:13 -0800 (PST) Content-Type: text/plain; charset=utf-8 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\)) Subject: Re: [PATCH 06/12] x86/mm: use INVLPGB for kernel TLB flushes From: Nadav Amit In-Reply-To: <426011a9-1fbc-415c-bac7-df5d67417df3@intel.com> Date: Fri, 10 Jan 2025 08:07:56 +0200 Cc: Rik van Riel , the arch/x86 maintainers , Linux Kernel Mailing List , kernel-team@meta.com, Dave Hansen , luto@kernel.org, peterz@infradead.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Andrew Morton , zhengqi.arch@bytedance.com, "open list:MEMORY MANAGEMENT" Content-Transfer-Encoding: quoted-printable Message-Id: References: <20241230175550.4046587-1-riel@surriel.com> <20241230175550.4046587-7-riel@surriel.com> <855298e6e981378c3afeab93b8c3cb821a7a5b88.camel@surriel.com> <426011a9-1fbc-415c-bac7-df5d67417df3@intel.com> To: Dave Hansen X-Mailer: Apple Mail (2.3826.300.87.4.3) > On 9 Jan 2025, at 23:18, Dave Hansen wrote: >=20 > But actually I think INVLPGB is *WAY* better than INVLPG here. INVLPG > doesn't have ranged invalidation. It will only architecturally > invalidate multiple 4K entries when the hardware fractured them in the > first place. I think we should probably take advantage of what INVLPGB > can do instead of following the INVLPG approach. >=20 > INVLPGB will invalidate a range no matter where the underlying entries > came from. Its "increment the virtual address at the 2M boundary" mode > will invalidate entries of any size. That's my reading of the docs at > least. Is that everyone else's reading too? This is not my reading. I think that this reading assumes that besides the broadcast, some new =E2=80=9Crange flush=E2=80=9D was added to the = TLB. My guess is that this not the case, since presumably it would require a different TLB structure (and who does 2 changes at once ;-) ). My understanding is therefore that it=E2=80=99s all in microcode. There = is a =E2=80=9Cstride=E2=80=9D and =E2=80=9Cnumber=E2=80=9D which are used by = the microcode for iterating and on every iteration the microcode issues a TLB invalidation. This invalidation is similar to INVLPG, just as it was always done (putting aside the variants that do not invalidate the PWC). IOW, the page-size is not given as part of the INVLPG and not as part of INVLPGB = (regardless of the stride) for whatever entries used to invalidate a given address. I think my understanding is backed by the wording of "regardless of the page size=E2=80=9D appearing for INVLPG as well in AMD=E2=80=99s manual. My guess is that invalidating more entries will take longer time, maybe not on the sender, but at least on the receiver. I also guess that in certain setups - big NUMA machines - INVLPGB might perform worse. I remember vaguely ARM guys writing something about such behavior.