From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D54230C606 for ; Sat, 4 Apr 2026 10:22:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775298173; cv=none; b=i8xxu9n8r0+4JU1WJ0THGC0MCug0xtixzkGPNUN6Nguq+LB+FJl93efE8YE3o707VcD5MXrwIO95yVxV9qGJzfqS4BKj2UjwaG9xalYXqwJnr52ptvIQtXuCWjYxyuMSmF7uR1GvhwbX4lwC3jVIEmkXTAcMAhEVhE/47Rgi8DM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775298173; c=relaxed/simple; bh=h6njPlE8o2vFiE63IkUERqu2eMjgsHufPPhwwYKd/pc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RQvktSUeVVPXMbC4Nqho39/kPRYRjlNuPf0gDPEjp/h4T9L3zGLU6Mhcz8oEPpm22QOu+9I2zPR/t/DNDh9URFPLnMRfyyMeNxHL63Q+PitsSg1ixSHGt2UAswGG8kybM6yIMKqmcuEbao1FWHBj5VHZfvn8vDEOUjCmwYVpj6M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=MTk8uglU; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="MTk8uglU" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=t1nzMWmMpCrFifLMvmbPxX9PEgBQzl0pmY4/EVgyG98=; b=MTk8uglU+TRlb14L4q866pWFzs BTWL+i6WzOcWjFc2NEAYqc8jPSPPSWL8L8vJKE8CnbhnDdhrUIDYgHeOHrJnuol7tGkZYlxiWNH85 cPkKKllZnVrWuIOO/hHi6WyPKneyxS68RGV5EX9V9zBsOnj1oRYjNfVFMgesElKBgmFpjXyP3JT7o QWjFH0njUnnASHPlSQDtdCrJAGgTUhgYHGMgZcMOSDO4SykyksewzTstoHBJxP/JpaWvyt8ljoXoO u69N0+af3RJ6HSKyRet8opAbx2qGNADNglw2gCD7L3mCr++fw0qzR6x7XhIXxsqhsSUcJyYFHxK+H LZBX7yYg==; Received: from 2001-1c00-8d85-4b00-266e-96ff-fe07-7dcc.cable.dynamic.v6.ziggo.nl ([2001:1c00:8d85:4b00:266e:96ff:fe07:7dcc] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1w8y9O-00000005BxI-1vcT; Sat, 04 Apr 2026 10:22:46 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 3F7833021A4; Sat, 04 Apr 2026 12:22:44 +0200 (CEST) Date: Sat, 4 Apr 2026 12:22:44 +0200 From: Peter Zijlstra To: John Stultz Cc: soolaugust@gmail.com, juri.lelli@redhat.com, mingo@redhat.com, linux-kernel@vger.kernel.org, zhidao su , Andrea Righi Subject: Re: [PATCH] sched/deadline: Fix stale dl_defer_running in update_dl_entity() if-branch Message-ID: <20260404102244.GB22575@noisy.programming.kicks-ass.net> References: <20260403081215.3942454-1-soolaugust@gmail.com> <20260403134256.GH3558198@noisy.programming.kicks-ass.net> <20260403224610.GJ2872@noisy.programming.kicks-ass.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260403224610.GJ2872@noisy.programming.kicks-ass.net> On Sat, Apr 04, 2026 at 12:46:10AM +0200, Peter Zijlstra wrote: > On Fri, Apr 03, 2026 at 12:31:19PM -0700, John Stultz wrote: > > > Using a 8 cpu VM with CONFIG_SCHED_PROXY_EXEC disabled: > > > > With commit 115135422562 ("sched/deadline: Fix 'stuck' dl_server") > > reverted, I see the (expected, maybe) behavior where the starvation > > lasts ~1second, then dl_server allows all the threads to spawn right > > away, and then the test runs for 10 seconds. > > > > See perfetto chart: > > https://ui.perfetto.dev/#!/?s=a729fd2dd4b224d6335c5b2e727dc1a1c302c11a > > (click the Kernel-threads track and scroll down to see the test > > threads named referee/defense/offense/crazy-fan) > > > > With commit 115135422562 ("sched/deadline: Fix 'stuck' dl_server") > > applied, it seems the dl_server boosting the kthreadd spawning is much > > more staggered. Again we spin up NR_CPU low priority threads, and > > there's ~1second of starvation, then we spawn one of the mid threads, > > and another second delay, then there's a two second delay befofe we > > get the third running, then we get a small burst of 5 threads at once, > > then it falls back to 1 second or more per thread as it spawns off the > > rest. All in all it takes ~44 seconds just to spawn the threads before > > running the test. > > > > Perfetto chart: > > https://ui.perfetto.dev/#!/?s=ab8e487375d0c82ceea478ee4534a7189269c0d4 > > > > With higher cpu counts (64), the test effectively prevents the system > > from booting (trips the hung task watchdog). > > > > I haven't really diagnosed the issue, but it feels a little like the > > dl_server is boosting until the fair rq is empty but then giving up > > the rest of its time, so if a fair task runs repeatedly but for a very > > short period of time, it won't get to run again until the next > > dl_server period? Causing this rate-limiting one-task-per-second > > effect for thread spawning? I still need to stare at the dl_server > > logic some more. > > I'm getting a sense of deja-vu here. Didn't we cure this once before? > > I'll go stare at this somewhere next week I suppose -- we have a long > weekend here. Random brain wave... Since the dl_server is LLF (deferred), it will pretty much always trip the dl_entity_overflow() when interrupted, right? Does it make sense to use the revised wake-up rule for it, when appropriate? --- diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index d08b00429323..674de6a48551 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1027,7 +1027,7 @@ static void update_dl_entity(struct sched_dl_entity *dl_se) if (dl_time_before(dl_se->deadline, rq_clock(rq)) || dl_entity_overflow(dl_se, rq_clock(rq))) { - if (unlikely(!dl_is_implicit(dl_se) && + if (unlikely((!dl_is_implicit(dl_se) || dl_se->dl_defer) && !dl_time_before(dl_se->deadline, rq_clock(rq)) && !is_dl_boosted(dl_se))) { update_dl_revised_wakeup(dl_se, rq);