From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2C5A38B7D4 for ; Sat, 13 Jun 2026 02:02:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781316133; cv=none; b=Cy4qOIuFSoQ2BIBXWgzl/cRIGVBn3KQdR47HLhYPlisVOs/kJwwfnMZEm/qBvf7X642wzU3GCpyuTyhs1gtqcDzrN3Tsv8Lbt2x+4i2Wrff96N8HjWqjY2BG5pv5faEI1ZzNU1kyLUuUy8k/V0oqjc9bgJMohDaolvP4kMLFKDM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781316133; c=relaxed/simple; bh=72jhbdFx3hPZipRFlbcX+3bfbBOY7aoBt8nHrP7HRQ8=; h=Message-ID:Date:MIME-Version:From:Subject:To:Cc:Content-Type; b=ojnQX737xO89yznvWV/Ox8qUxqxAfbFo2c6kIjdpv2WglguR/btQZplssWqnnFqft2hp8rFZVL8sCLOzh7rLsW00xpjK0bn+mfFPi6vjWiGZlBkqRRDJvMk85ntRsyzEhQL24l+wAGiSuPVyykSBRciAKpUPD9Vrx9QS6PUnn/E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=DO4gFjCD; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="DO4gFjCD" Received: from [172.27.3.125] (c-76-133-66-138.hsd1.ca.comcast.net [76.133.66.138]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 65D1jC0b2055834 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 12 Jun 2026 18:45:19 -0700 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 65D1jC0b2055834 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2026052701; t=1781315120; bh=D7y+es0JjDfj5epziE0NvS5pnqO2YXx1lJFCsVQ/fSI=; h=Date:From:Subject:To:Cc:From; b=DO4gFjCDfsc1p9KvBEmoRRxh14QQ2wCQ1q4I64/uW5o5T5RdH7rW0EFy4X53pbnIJ Eut9OJgcxHnWeGX82unNQ2oHjcZitWZwt8V3mot9f51alz63lCeIWtnnqo6MZhb9H6 w0nmXppij8JqmedDt3lCQ86IaInrLE8vOT5kjbMzq7nlZ8zBoh70P/QWeu+2BbMz4F m/f65b9mBkMAWhR5CHwNlCLWqNcQEGlgdb29Krvk7amsZ4hcMnLgf1ZLnbtoYKrNKp 32auAh7UH35KKnDQHg6dQeYk5W+5Mtlr50EQWBrMI/oSmLGuXqDiuZ6adB+M38gJqL nKT8KfZMz1UYQ== Message-ID: Date: Fri, 12 Jun 2026 18:45:06 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US, sv-SE From: "\"H. Peter Anvin\" (Intel)" Subject: 8aeb879baf12 - significant system call latency regression, bisected To: "Peter Zijlstra (Intel)" Cc: tglx@kernel.org, mingo@redhat.com, bp@alien8.de, Nathan Chancellor , Calvin Owens , Dave Hansen , torvalds@linux-foundation.org, x86-ML , LKML Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit So I was trying to figure out a significant -- about 13% -- increase in system call latency between v7.0 and the current master, and it bisects down to: 8aeb879baf12 x86/kvm/vmx: Fix x86_64 CFI build This is on Panther Lake (Core Ultra X7 358H) with FRED enabled. This is a bare metal boot, no KVM. I'm personally extremely puzzled how this could possibly be related, and I will be investigating the possibility that this is a false bisect, but it is not a Heisenbug in any way; it has been extremely reproducible, and the difference is statistically valid by close to 10 sigma. Futhermore, the bisection at least gave the appearance of stability. Given how late in the cycle this is I wanted to send an alert sooner rather than later; I will update as I get more data. -hpa