From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DEF83FB079 for ; Fri, 10 Jul 2026 09:54:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783677257; cv=none; b=Si8buKnkHUxAGFmu9Kr9PCCH4lh/sVoSe6RYUA19iPprUAliu9JgqSabTPdw8T6Mh7S3G/lJe/wd8QmxcTA6FlkupCd5EwVbkcAlWHO6EE3XTsAi8zXKItQdafvlxfGnr4LkbvRNGfgU9VXNbD9LqeaF0s1hyqz5+tr4B9R3py4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783677257; c=relaxed/simple; bh=v8P2Uf45mI/biqxnOxHWfgXQmu+JaXEwT05r7bGVu80=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=fgUDfuJhDRIddjYTQCQ46yhWkk6E4M+B5QRLsyEEuup4cOk7SppwpXoVraoaFIZmtR+sJiXXmFlF1Bj8nx/QYJZT6uXOwoqXzAaonTcNQnTqiz49PUNulP0jV9Y6phcwlq1JJpn7iqTOO2C/vxfx1lJd1mVp8f1GlTH3DYb626Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=Ip/hqoeG; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="Ip/hqoeG" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-47df6a5202bso419110f8f.0 for ; Fri, 10 Jul 2026 02:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1783677252; x=1784282052; darn=vger.kernel.org; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:from:to :cc:subject:date:message-id:reply-to:content-type; bh=v8P2Uf45mI/biqxnOxHWfgXQmu+JaXEwT05r7bGVu80=; b=Ip/hqoeGAr/myjEmeIAVuqq/vHDGOW82Jij0v9V7U9rEvPVd9xIOWYyooT6BVXBmQX oQtmGVl0uMY1ruyFgsmEHUkL4uUGHUyVEIfFDGOZQIlD95I00sEkYCvvLbsUVaCRcdE/ 21l9ux9MUb0iFF5oP9hKQH6roO+9UKLSfUNsGIWQFMT0Tc948Tjponv4UABojt2vCMnC gLTlIF36tLwLlPnNOmJ3i9z8rMMq3NEiQtkOpsuomare+m7yXnB+/mZ0F6cQEI72XISf Mm8Wpf/L6vFfce6tR71mQY2uUfqusyRMIGEgWKU19OdDbVQgTfvp33yjhc0PQ3Zo1D9L hSmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783677252; x=1784282052; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=v8P2Uf45mI/biqxnOxHWfgXQmu+JaXEwT05r7bGVu80=; b=FzyrBPmk8D4MQGu3W/Zd7K5CWLy+Md83hnI45fHRnaCeKNtvA1BAeYIRzm22FzIg43 vEB0di26VAvJpOp9A0XAtmI4gCb68MUi08y2Gxdu6yP5MfjgtK76i2K9p1eYZTXZ7UbV yAtBKwzmwsKz2y1sAGifMgL/lByf9uayKKd/km8Is7huuMFBVShGj2rtwTGCd1ISkRGQ dphZSygm9mLTaPbR95f9RKB5egLAz/0EG07RwMHXul3wxKQfwGhWBBQKc0XM3SRb50V7 nXRQj09jWcvYmUYcMIgdoW7LE8O/KmM2oxgdKudC5B16S54ujCSMwpuzVpJW+hYDxJyK /bIA== X-Forwarded-Encrypted: i=1; AHgh+Rpn+7+haadqXcpBEA54iko5fgfzrYVAWFbj0sUjMNeOEWa4I/ol+usa7RHBON/6D7Qc61nX2dDCMXjCJus=@vger.kernel.org X-Gm-Message-State: AOJu0Yxi/w/UmdZwtoDojE7Kn1zeK09etAquyTQHd4AnHD2cxa+CRThA 1By/0HxUMXqvTJVkIY28eND2MtvWDXbP/G4Ju/uALcWUhroDGR4C90sSAGkQ1KZ4aU4= X-Gm-Gg: AfdE7clYHQ8rJYVemBpXBBU9omS4HdJIZFRgW6lJJuG2fYPFeEFG2m5hiRM1g6lNI9p uUan6b1ygJV3JS3ka8PsvfScAWWMHHaKc1HKjupFhSnXKGmKUD7S1vXcuGhP0FxfGDfUnAGttVC U5J3liiEoCcQGHDiNrfoT7S+0D945X96hixYCG0zg8oqHss64IB6CloQLH9ODSsILx/eZ1fNsQt s+IJ2EQKuklJnGbFUJE+QnkextuMXN+Y+Gh55GesIzkIMSGTRYq49zL3HsgbbFUzXSvggamUbsW yjLElXoQ0FpTWRPMpIeqiabmLvkti+px5vyMdEx65tYPAFUonm3dNPFt04hbVzojGPinoJqMNVB smPxDm0vfiI/DpWyzcrqNu89uvhq8Lejfzk4Cx1+URj39KLTypFDAKIvAQwZFi/7rFWNc86jgmk 86ruBtVRkgT7/CwvyX5idBhjws X-Received: by 2002:a05:6000:27d4:b0:47d:f441:5f93 with SMTP id ffacd0b85a97d-47df4416019mr6854730f8f.25.1783677252374; Fri, 10 Jul 2026 02:54:12 -0700 (PDT) Received: from [192.168.219.26] ([212.129.74.16]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47aa039ae44sm55993210f8f.23.2026.07.10.02.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jul 2026 02:54:12 -0700 (PDT) Message-ID: Subject: Re: [RFC] Significant Random I/O Performance Regression in Linux Kernel 6.18 (Up to 27.7%) Likely Caused by Commit 3c7ac40d7322 From: =?ISO-8859-1?Q?Andr=E9?= Draszik To: =?UTF-8?Q?=E5=AD=99=E9=AD=81?= "(Kui Sun)" , Neil Armstrong , Bart Van Assche , Alim Akhtar , Avri Altman , "James E.J. Bottomley" , "\"Martin K. Petersen\"" Cc: Peter Griffin , Tudor Ambarus , Will McVicker , Manivannan Sadhasivam , "kernel-team@android.com" , "linux-samsung-soc@vger.kernel.org" , "linux-scsi@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "stable@vger.kernel.org" , "linux-arm-msm@vger.kernel.org" , =?UTF-8?Q?=E5=BC=A0=E5=A6=82=E6=B3=89?= "(Rain Zhang)" , "cixi.geng@linux.dev" , =?UTF-8?Q?=E5=94=90=E6=9C=88=E6=9E=97?= "(Yuelin Tang)" , =?UTF-8?Q?=E9=99=88=E6=96=87=E8=B6=85?= "(Wenchao Chen)" Date: Fri, 10 Jul 2026 10:54:19 +0100 In-Reply-To: <12a8417dc8644a71b9cb25c53c93805a@zeshmbx08.spreadtrum.com> References: <12a8417dc8644a71b9cb25c53c93805a@zeshmbx08.spreadtrum.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.2-8+build1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Hi, On Fri, 2026-07-10 at 07:17 +0000, =E5=AD=99=E9=AD=81 (Kui Sun) wrote: > Dear Kernel Maintainers, >=20 > During our upgrade from Linux kernel 5.15 to Linux kernel 6.18, we observ= ed a significant performance regression in random I/O > workloads=E2=80=94with a maximum degradation of 27.7%. [...] > This issue is particularly pronounced in single-threaded, small-block I/O= scenarios=E3=80=82 >=20 > To illustrate the impact, we conducted benchmark tests using AnTuTu on Un= isoc T615 devices. > The results are summarized below: [...] > Root Cause Identification >=20 > Through investigation, we identified that upstream commit 3c7ac40d732232f= ec0ba31d0a5e3cc9c112fc2e7, merged in April 2025, is likely > responsible for this performance drop. > After locally reverting this commit on kernel 6.18, performance fully rec= overed: >=20 > Table 4=EF=BC=9AMixed Random Read/Write Speed Scores=EF=BC=88After Revert= =EF=BC=89 > Device=C2=A0 Kernel Version=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Test1=C2=A0=C2=A0= Test2=C2=A0=C2=A0 Test3=C2=A0=C2=A0 Average > T615=C2=A0=C2=A0=C2=A0 5.15=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 18604=C2=A0=C2=A0 18314=C2=A0=C2=A0 17732=C2=A0=C2=A0 18216= .67 > T615=C2=A0=C2=A0=C2=A0 6.18=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 13372=C2=A0=C2=A0 13081=C2=A0=C2= =A0 13081=C2=A0=C2=A0 13178.00=EF=BC=88=E2=86=9327.66%=EF=BC=89 > T615=C2=A0=C2=A0=C2=A0 6.18=EF=BC=88reverted 3c7ac40)=C2=A0 18314=C2=A0= =C2=A0 18604=C2=A0=C2=A0 18604=C2=A0=C2=A0 18507.33 >=20 Thank you for your above analysis. Your numbers match up well with my own observations at the time in https://lore.kernel.org/all/88d31a258feb36425ad73d0323077972f85f8341.camel@= linaro.org/ [...] > Request and Recommendations >=20 > Given the tangible impact on mobile user experience, we kindly request th= e community to: > 1.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Consider reverting commit 3c7ac40d732232= fec0ba31d0a5e3cc9c112fc2e7, or > 2.=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Re-evaluate the proposed change in light= of its effect on low-concurrency I/O paths, as discussed here: > https://lore.kernel.org/lkml/88d31a258feb36425ad73d0323077972f85f8341.cam= el@linaro.org/ While originally I was trying to find a solution that doesn't regress exist= ing platforms and still works for the newly added platform, I can only second t= he request for revert, given how important UFS is for mobile. Mani was also in favour of reverting: https://lore.kernel.org/all/4enen7mopxtx4ijl5qyrd2gnxvv3kygtlnhxpr64egckpvk= ja4@hjli25ndhxwc/ Cheers, Andre'