From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF5BE13A244 for ; Fri, 3 Apr 2026 22:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775256380; cv=none; b=Q+Kz1JbjxU1V3oFv0SO6vsPufsB0qQc8Uy1Y7KUR2VjMsoTSZsagGGF2PIaZhVakZl5a8x7BFbD5ryU6R1K0Nqj3UTpsJL/O4rHCxCU4BkQIM0bhwdc98Fu3ngqUOMZ+ZQjsIB2WVL3IJ8JD0DEwJDG8KCvpDGB4cnqkKgbWMK8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775256380; c=relaxed/simple; bh=cWoaBgYobBiSIWjRIJOprI5Fa4YoOSNggKxCCQte1II=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TpCx0mAvbBBHrSVpRp4ZT8MLif/nS/5RK+PUH5gzAibsvaqbaWwr6GLsYqYP9kbj8QLBzLsYTd4rHGIFgrmysGhBrljt/eVrMCrn+8Wxt7q5UdGN9e2FxbcbvsKWSlfYoJT+OSJIB9FFcDnrrq08wlwF0iy+F4A4ZcTZ8HR8Hzk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=UMV+Iowt; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UMV+Iowt" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=aYEcf+hHheoM5HjgxZS/PmrmrOO9UBWBx2SKmQIfX50=; b=UMV+IowtGl0hZpujHbWZ3SYkQy 5m46Ats/+ek9g+nSlppeIyhXGZ+HNNevI1qrIgIpKfymlPqDBj7AU7CeF/YISsxBNUNe22anx6LPl hZGJciPFRcrbrBfEff0KAZfL8yMUxtZh6MoZBEB3PrRGSUx4kbHTiqpDQL9s4vrmTf3s0OcZai/Zw fHUU894KcEh8WpLETNx9YJ0DXoMFcomJTf8c58RPjh6PiJF5T/p9yWJ4t18ghpeI3LbtWYMkXXkau lcwZKPwQsgCCmjy+SmzIEkfQsWBHwP4DMWeME2C1WWts3eoDkdARUT8SYRTe9vLA5VVTAHpj9JlUi n8+Pxb2g==; Received: from 2001-1c00-8d85-4b00-266e-96ff-fe07-7dcc.cable.dynamic.v6.ziggo.nl ([2001:1c00:8d85:4b00:266e:96ff:fe07:7dcc] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1w8nHH-00000004ogN-28w3; Fri, 03 Apr 2026 22:46:11 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id B22A3302FFB; Sat, 04 Apr 2026 00:46:10 +0200 (CEST) Date: Sat, 4 Apr 2026 00:46:10 +0200 From: Peter Zijlstra To: John Stultz Cc: soolaugust@gmail.com, juri.lelli@redhat.com, mingo@redhat.com, linux-kernel@vger.kernel.org, zhidao su , Andrea Righi Subject: Re: [PATCH] sched/deadline: Fix stale dl_defer_running in update_dl_entity() if-branch Message-ID: <20260403224610.GJ2872@noisy.programming.kicks-ass.net> References: <20260403081215.3942454-1-soolaugust@gmail.com> <20260403134256.GH3558198@noisy.programming.kicks-ass.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Fri, Apr 03, 2026 at 12:31:19PM -0700, John Stultz wrote: > Using a 8 cpu VM with CONFIG_SCHED_PROXY_EXEC disabled: > > With commit 115135422562 ("sched/deadline: Fix 'stuck' dl_server") > reverted, I see the (expected, maybe) behavior where the starvation > lasts ~1second, then dl_server allows all the threads to spawn right > away, and then the test runs for 10 seconds. > > See perfetto chart: > https://ui.perfetto.dev/#!/?s=a729fd2dd4b224d6335c5b2e727dc1a1c302c11a > (click the Kernel-threads track and scroll down to see the test > threads named referee/defense/offense/crazy-fan) > > With commit 115135422562 ("sched/deadline: Fix 'stuck' dl_server") > applied, it seems the dl_server boosting the kthreadd spawning is much > more staggered. Again we spin up NR_CPU low priority threads, and > there's ~1second of starvation, then we spawn one of the mid threads, > and another second delay, then there's a two second delay befofe we > get the third running, then we get a small burst of 5 threads at once, > then it falls back to 1 second or more per thread as it spawns off the > rest. All in all it takes ~44 seconds just to spawn the threads before > running the test. > > Perfetto chart: > https://ui.perfetto.dev/#!/?s=ab8e487375d0c82ceea478ee4534a7189269c0d4 > > With higher cpu counts (64), the test effectively prevents the system > from booting (trips the hung task watchdog). > > I haven't really diagnosed the issue, but it feels a little like the > dl_server is boosting until the fair rq is empty but then giving up > the rest of its time, so if a fair task runs repeatedly but for a very > short period of time, it won't get to run again until the next > dl_server period? Causing this rate-limiting one-task-per-second > effect for thread spawning? I still need to stare at the dl_server > logic some more. I'm getting a sense of deja-vu here. Didn't we cure this once before? I'll go stare at this somewhere next week I suppose -- we have a long weekend here.