From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D404F1DC985 for ; Tue, 21 Apr 2026 13:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=212.227.17.21 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776777301; cv=none; b=XfhhZeDJKvQnWJ3EZSL2ozlmIezVIFT56uGWe9/sEK7e4fLPBdgw5E9oK3Aq5EyfcRBz4Rsd4IIDFtCySZBYMuheucQnyKFxD9vPApyQwHJ8f/teDkyGuNNIGO9CBqvgq3sJVyL0Ix6mc3M48i5Qt/n6+PhUeHJMXgYbMkq7mmE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776777301; c=relaxed/simple; bh=OHv7T7wHJn6PGgAV++PLIFWJkPJk3KeNxMLFIDn/rAY=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=KaBKg73OnymQEOYX+RsVRx7NLALn7jpMCbInz33DxRD+20roDlfkbIXKybebDjFa8szY84VpBNPZupnL0jDSkmetirTzzRN+Kns1EMF9y4V3v+ICRfu6VMay60/fXRJgB7cyiI2Tl8S//JtpEiFbzMhZFZD+djiAy6Y0Vrnxn9Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gmx.de; spf=pass smtp.mailfrom=gmx.de; dkim=pass (2048-bit key) header.d=gmx.de header.i=efault@gmx.de header.b=BeL5WdUs; arc=none smtp.client-ip=212.227.17.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gmx.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmx.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmx.de header.i=efault@gmx.de header.b="BeL5WdUs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.de; s=s31663417; t=1776777296; x=1777382096; i=efault@gmx.de; bh=qGTQNwwXwmlav/rD8J1QIRbmqNRqTXG9pwdogYUXuuQ=; h=X-UI-Sender-Class:Message-ID:Subject:From:To:Cc:Date:In-Reply-To: References:Content-Type:Content-Transfer-Encoding:MIME-Version:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=BeL5WdUsCCnuUwNln3iHr/MOEQVuV5v5TpugzT1/1fmuS4FoFSbUhnFbeXCI+Z6P M2x+ICDCHLLFP/9SE/roC4vDFL4tkjHhcwc8dvz9UeR8T45wRcSkfFScDFY5W5EVp TtFHwFA8g6nSUbZRFzMoeH3awtFJKI6naWOUly5nmbpBqzoek7g7MSiCn8g6FiHl5 CC/PPZkipgCu/Bvulm/LKktoA8qXU4vugXsyBnb9bMm/zE+D+ULERkZZBR/lFpdRy YUOTpM1QXUe55QopLqgV5NE+5HWdtzjwCT7Suykeec3p26J9yq1t1DRq466oq6EjG l5tHGMiCxhtfQG7H2A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from client.hidden.invalid by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MMofc-1vwPbQ1HfQ-00Z4nM; Tue, 21 Apr 2026 15:14:56 +0200 Message-ID: <3243f2b7d7e5c3121c17fe84c328137f91b322a7.camel@gmx.de> Subject: Re: sched/fair: DELAY_DEQUEUE causes ~25% pipe IPC regression on Raspberry Pi 5 From: Mike Galbraith To: Tom Gebhardt Cc: peterz@infradead.org, mingo@redhat.com, linux-kernel@vger.kernel.org Date: Tue, 21 Apr 2026 15:14:55 +0200 In-Reply-To: References: <2877d886d8470f4ea3393e8c2e570a059c2b8b1b.camel@gmx.de> <0a93288ace2a95616066f7a26cc45d6407dc5147.camel@gmx.de> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Provags-ID: V03:K1:xnfX3+DB40AUoqsThcgE/dsWuUSReN+2hqud42ZJctdxts4gynS JrfQAbdmn20UdRa7uNgiGDewBD16n7lLII8D/x+73zpdAW8e+B3NjYw0aqM3YuYNMsQAaoQ cuACqMzJcI3bPUOgJTWoPm2gnzPg0xv7zHJ5yzexSUgI2Gc3T0mxqh8jVS2IJfTY/GFb3rO BFmYo1+f8tiIZaklTId3Q== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:7Oki2BkqsLg=;ZeT/1cmP0wxyhKzXYLAAUObuX7o FJxTdWsUbZ6Hs7ug0+6SwZNZXJWY1YIYRCSRl2ET0N8iiMDvE5Su8WAU549EIvwGKBo2igH4H 5o8tczfX9pJ98ZahhNBR0sT3ga0YlGPmqiKfCIqpc7EuPqkSrQtaLTCvFDQhVkeHjFdsCV+CJ 7V805HDY/Fb47QYpJWiDLZzz1PvfBLThJA/8JwaRtj58R1a2UMYmBnCWa5B256RzcwIfhlUt/ LuL3+7OsvpSeOOyBJ9l7hDgXPGFOjJdRBUhtMl2fw0MR6gV3qDPHrw2ZHIlO/qPXlP90ILqFJ v31p66VQhhYAMACOy2P2yCJwonbMMSBTfaLt7KbwF3uih9MP8m3OBHfhA4ch5TGu09RdQ9elJ Fgnw6suokZKMtKIObgbADgqD8GOqR4x9P+XShLRULO7J2lSCAzlk11OIGvMvB2Ro48k6img2V cFCY7LcRMqYG5daVgv2pJN56EZdnuyRLB5pykuodDtxMt94jAGdAWqmbFepP+hQGMe9i8ZHt+ cfRkW7YgT3x5SgjzPmv+OAgPyRdYspFAIZ0wM8BDPXsex+n7qPYRJ9qmSklUag9YlSyKRi22B cKAs6OKnjJHtICTSoQkc41FGiXMe9prUX0/JH1cikiEclPbnGtoc3+H5USuoJQCbw10VtsDOC Woro/idrBGECq8savnQSLjlF91Kt6ly7az9yrWfyHq/gnrAaF0pY2C6X+3Be2iFowdaIbdY41 VGYEfWCDBsIyWALGFeeNXVc+8I6R6FTNYDnugB9RtamClxemdGrvlR3gmE23VgmbzpbiuIELw 9ILJ7sC5pPhyjPGqqVjB8ThIThyD+tqE8KZpwxfASbSCPayVB3LybZ9ZDSyapQu3npI07oXW7 y0UA7avjlp1aHHFwJn2yG0GEM7PaPaR3uri8cE2C/yvEkdZyu6SI/+mnFXALTadp2X2U8uFLs DaB0zcyukfeYVo0zjMVIZAtL7fPSIcyMHELsfZLmy1OYu+h4GWS+BS5heT8Rd7dmtb51hjKjS zeGGQzn3ExsQ5G3Vp1r3dRfqII5rH+qUsuPU3kfLMOA+UOgsfb70enYYZ7vnDeLYvlJYLGCHi 8TfopqXMhyo1amasuXLuEtgmBiMxh1PEW5rBQbID73QDTYSRdFOEK2X48z/+tSrmvRX+WFSCS 2ZKXkL6h1+4ozwcllTyzWYZMptNjOCZNvJWj0Mdy8INVZZ9L5hbcR5qqRKipWLR9x4lm+oRD9 zByWy7NzRFX7gNuZmKO3dPfHwGYXNn/hmNE81GMfREOm0Ff3S4I6CYoDTCAsCDek8xASSo2he xoIqcA2r39ReG5cC2Wnf5LdkJptk66k+1TeWY8ki/LeaYHytRSFEemLI1O7iiLuzEVGeVO9t0 dZU9laq/fHnQKK7NwSzmKXxeo8hov92ftozD8ZN1i7kIXrpaShXwYyfHovOow9x2QrvgL0KH/ JN570SBL8+z6pl/GnKs4Qhb+scVVpPBM+lNBOlq4INkl6rbpKxhttYG+4sJGYaZ6DM3LZSxXG tkUGN61P4NJMsiowcienmqkF9Y97oxkHMiOmrTd/fCOi5W+Jc2GLw16S03hSioOwai/tASGIf co9JX57GMmc8F8N73if88jMyHd8KE9vr8NSakU8HLRs/5+2SBT/a1pRrQf6giZOWDDxwyNsTW 3/BOSD+9ifeuEBzgWo0JVrPfGdl2CSvW+9wJak61flLCjVCHFtKOzRrYUoyIHV1exgLt3UocD oKN8MuZIs2Nxd8XfhRJMzxDr2xLHWyGJHokMR4Cg6ZmLlyn+x4uQ5nccw8L45NayW9Z2HMh0H +aYH+PD9zuf8aoZz07SU9D1/iemPcuVBkNGRi41GNEpLLHYz4O2cwkrpG+74Rq7yzEeUcWMXe KdDwwdfyHdviMcosTbaA/w6bLaPnvyzWAtrVYA8pD84Bni6ttDKR/W00Td2X3cTkINvOdeijQ eJP986ThVaJ4ikHOOjvZbkV1J0C+gdkXZcX1q1mjGpUaUzwz7kmuN4H39ArlMS9kOSjt/jV8v y2ZJYdqtZy1IHQi6iDn9Odjs1DnAPDBX2JpJkc69nY1cIQfDwrbZX5Ant7AcDltkkgkbFpXbZ A+O05mxKdmUI/gwSnbFeT09oFg+SXPZZXUx3/SAkev8/pCAhnwAXgPRKZRpVg0admsXV/LYL/ TBIer8Ii7ZTkXlbyKEsRgopKHVSn0yHLKhflu1MrKTd+u60DN8YO+PZTTkVJ6qf/ykUxMTbxJ jBEEjHH7x1Ntnr5EnkeDMxqeNXM0m7sbXUtEDxP5PwZT2Fg3anAEFk9yBXp+3kDT0V3Q31MBS CBaOBBZDuR2iFHQHsSTwk5/L8VrmGhfsfOLCtp6R/R+6eQmKOynveBGtQ2PPl0guvaT6dxLij OAMbODeeOAtxekrm5bOVxElT1nhOF7ipZ/rJeAdUtKrjy7qT0Es7C71p4z4X+FNbiM9SBuw1B NKszCDrdErssgbUxWkiqMJhwYoBxppB4zxdt8L6tt07gNx5mVrED50xS+4wU0mkgIMOEu7u6W JVKXh3vrny5qvxYNKRGx4fRFgzc/xIRXOY/O7Yc+3g4wZ1WrRevsiI3OnZ0WbQkPfCHw8YFIy tlb8Jt/uZaSnoXpNoD06OD9+TypSYH9XtSyAc4rraWQfauJ1hqdBv9AJHQIhctxQnlURKpJ5p F2DghY9UJhfHbsFlm6IPisHQvHwaxZn40hPRchi8Y6zgwSEI285AEl/ElW3NTEkuGKiNZSGNl Zm/XsGLmogAi3Vkr+B9IV98WS0OIPtyHOO52CJ5BT1B9H3a7EF5+QwbNfS1P1dGOM4Hlm51sX uco4SU6154tnN8jERnUhQLgflbg9h2gSZ0s6Ad7aJRu11HfVvQ7DlfCFuubBftI8vGvShNTqZ 1AsBQSD1Y5t/D2mm9/Q+dJcFfsbb7DdPc+cIssE10EWJiXVBDQQVx9tJOUWTx5Q78UQ04iabv SHGr3lDEpVTzvPO4i8FwJaO5ZoBKmMxXQY63nGe1rWb5NXKfgI2A0QPOfyyL2V2ZXWOzrMWNt Zm5bmeYv/V0QEvj8E1+bGy7urEf/etTK5X23BMrW7BpWVObJ66H3Z0s+zG4q9qe2BHDs+vIam uvNPsKc0MARcUU3pT9nK1FcLwl2lu9x369fBA0CY9UwkgVNpkMIoQmGIR9beE2HYsZdkasVuo 4UhP+77skO3Et6PApuy/VXRZ5F2rNSI1t3VCTdOlg2fU3vCx+NNNj66u9TTMyi+cXjaoG6lPJ jngdV9iGhbOG7DFChkiplylWhoy1RT2MN+MXdsfoak1hWTASFO+mIPLX7UeSxdbReCqwRwdxB kfH9HXgim4yQ1ylo+jG2Zjxk82AIEzAeqCAKHBJ3KkG0fufl/6d3SPylpE2iPHiaeb5HUUgL7 1rdx2LevQQBm31bXohL9i38xAUQRdbQPfpjYZGE6K8d2wil2f2BVE+S4RYJXKIpe2er1HJ/qL tTuqGJzvOZvS3cnM6VCCmZq7oECqlZb6WhJRvT7xGx+WPqNocV8I6mr+Ey5eJWbNGTMjjyqfB 3JSKnxltHLlQnpwoKYm327oVe4JsmNfD/POI6qkYz2csWurfJ5SUM7nZU+8zheF8cIKDjRfxD 9UBoSMfR1JiFOrGZ2zFIsGMgy3JgaQsY9tqpBEgKBw1RC8aoF9Juwm46lnfVCjUPf7OqY/g1f TZiQpUKZe0ATYE4Wun6ynVMdUeUVHqloV5PUyIJRHhwVSVRVSbvNBqwSEjV9UnSgGQutmOY2z j0JJJAS66eFcnu8vrri8WDPkYak9Ho31f0OesCEiCwkPHT1wDzcjRBDllAzE7gB0CFDdJxIKm YGxBoyOpfgK3PC8Gb/YifPSJYtl+LcDOoaWmpdZd5BBYwSp/1XlvGwhlZYItnmwoFBMlhQZed igquXZtVPGcLzhA2VVhkvNsg2KoPNUFMLz7Yn9EZco6MOHX4T3WO6ACmWtd8BOuqw5DiNWoWm dJo0TrScEB7+JrZMvQ2KqHnoLOXc3PRp1rEV5wj3BLa+WaDpd++jNCxu3uKsW5bTz7x3gtWfS 6qlFGXm0y81iVENQ9fU5y4lyFgM/yamcB0r8XpclVH4Ad25cdr+MqYqHXw4zmg66pvUp8+dJd Oxr0z/iDZmyQD2tjRMjfXJ3Cx+s2b5bBl1bUF9VKe2uavHlGT6hmvvDSRWv3B1n0Aqd7/TJGD ZG49SidbHe8yK6t+iRVNdd4pUY25ikT2vpZZsZn+glDKgFq5SbFxFd13+s0ORfeYWJ8H8YlLg p4oHVdUxwKX8f5yyiA8xgTHoyCMbJeSQ9cc0ZvFIh/D6qF04QExW15o6G6W0CDgezu9mHxZUI 0Y91lSVFVpcC+y/sP++cJZn2658keRo3Mg3PtluXs+knPIA77Hyye99AWAVqsAa9gNr3Hwre5 zgdl1YfrpfjqAyLhbHNBaMwOqQ3U89QICxct5efUOaHl8mRtRBuPmvjYvF1x/GGobzN/L8L5E 31dwC/GwHKaUZPG70xGTznFJ/94f0qK1wMPvF8lGuMMX0aGzErduqa3lRfLG/48YJt97l4AyV Sr7fuG4KlEB6zPGjh+lYxkFryUiNFiQok1/pWYHKbZ7i3rAnd1fX7JNZ3JgHLYH38bWrjvu+w qoAW3l6qmE/YsGmDv1mlPAVojZ7FOcn65ik8BNzfXsW939UBpf1ZSats3W13TUwh89UbXmS91 8w/4ZYpdJbupEHtdRjNcMYJo/u1CRs1HacTOobmDyy5xOCASfzPdq7CSkfwgShyaXhIZt7IqB 6SU+JY28TjviBSdjqTUNYNxh2DoeaEddU1Lfx1h8E+n/CGKwujcyqzxie2U+3LiyYU4aap6x3 /0kGt7ssjZbz9C5F85KxrDmApQ0p3kCHMPpkfgX+nzT/gQo5kNqjHR5lZHSiNaOMB2V/bZtZM /9VnoXvOgHwKkL/RoPfX1f2w0zvkqCf93cM7oU2tMYzF2RmBnXFv8qP53bWgsLhkroY/xZF3Z oQs0PtqCASueT3NmaxylhKTVskX4mnqKeWJo2BECcUgrAMuRp92BX4I5tgzwtu4Qh7xje1rjQ xC0JKHcw3lC2tMgS8AE5kRGQ355Cvp7UWNhLKQp45JRwuoUFo48+/qaAjVs3DYp+6DTaQjnlQ iCZrwlYhnOKsVWL7dJcE5jbCo1ItMpWTlMIiZRwvDA+MY3Mec6zUH7FTvdNsdBo1QHn9M7d8q pmFmag9Gp7IQf6nhbDVtl/6OpP4R3qBhctP4vdzqIdrbBDrBGEKrOtDfmiQBMGDjXuK/krz2R ++qAPQValnG6DwPLQu9MwWEhleJ/rduzrXJYJN2b0CRnfR9tbarcNtfU/cSCAujitleVhVqJF 52/5CZa8LuK0c7sEV6QHKsvFc0T+9wGhwrc5fxcUYpgB6mXYaYJqGV3Wk2OlWrqr/h7TdhJqi QWNnNsBFyoeCdXS7ElvDSkHJ4Xpo+JOw9HKYXyWkgmXCLA9gSLugPaOwKR7pOo+b+INkG/ZGW bgc5CQuVuO3DmX+AumrorpgX5WZ/RmIw0pP0= On Tue, 2026-04-21 at 14:46 +0200, Tom Gebhardt wrote: >=20 > The fact that your local 6.12.y branch with patches 0047 and 0089 > (Christian's "Prevent negative lag increase during delayed dequeue") > does not reproduce the issue is very encouraging. It suggests that the > downstream RPi kernel just hasn't picked up those fixes yet, which > explains why we still see the regression on stock rpi-6.12.y. Not sure. Taking stress-ng 0.15.06 to trusty ole x86_64 i4790 desktop box, since I see no excuse for it to be arch specific, the virgin 6.12.82 DELAY_DEQUEUE delta for a pure voluntary config (lowest overhead) was -6%. Adding all eevdf backports took it to -3%. To me that looks very much like a nothing-burger, ie routine bench jitter when diddling *anything* in fastpaths. The only clear signal has come from that pi5 6.12.75+rpt-rpi-2712 plus stress-ng 0.15.06 combo. -Mike