From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2CA033D4EC for ; Mon, 26 Jan 2026 14:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769437222; cv=none; b=gsovX5NJW/VkVWbxxuWo/c7g4S5cxeXEJO8uxju9k6X9A6Cb2w3AfX5p1k5r3Y/+dH0u9YskpW9W2Zi7SakplWx/GJIqoffzsmdjK3ukBLBogVPhJXM9BPEFvlObH8oYZV80/FfF5D4CyEp5xk+yIeN+6wfDC04LnUPc9z2Ilkc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769437222; c=relaxed/simple; bh=fBV/Sjxi45WbP/O31hj0SwBeWpmB5ZhGCiEXMnH47y8=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=c0XanmxNDYSBCQZSfykFOZC/QwO7NKhqd3FCBtW4vn8NrzmuqceSGcXMggFVM9/ToyCpdxpCdXkEDKRNrBusTrPewrcfhBjx2BsKobBuNBfMmG7Z+w0sYfWV0obgsNgyAx/9iD2UT1d6Rf6L4OS7mOAnu0NW9EZOZrs1E9Z1d68= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z5dxVgDG; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=LQyNNIWR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z5dxVgDG"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="LQyNNIWR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1769437220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=fG1OxMornZ72vrWdkK/L/931jxhh8ttLn2tvmGHTing=; b=Z5dxVgDGs6jlkzFX5l5QbFS499MjkRoQaNt0z1/sXPqA7DRP/HYGdkcgPHznx2jdl5nGgy gDIJnwv+XxX9QlhtwX9dEARTox8APzIbk2iU0HtIvSiUpPW5kib0t4bzqKWPuh6ymF5iAC ghfW628x2k3WHKwIIKINiXkG9vc5ryU= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-427-aOANvvvqOxaoKE6ij8EyCg-1; Mon, 26 Jan 2026 09:20:16 -0500 X-MC-Unique: aOANvvvqOxaoKE6ij8EyCg-1 X-Mimecast-MFC-AGG-ID: aOANvvvqOxaoKE6ij8EyCg_1769437215 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-4358f90fe8dso2748306f8f.0 for ; Mon, 26 Jan 2026 06:20:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1769437215; x=1770042015; darn=vger.kernel.org; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id:from:to :cc:subject:date:message-id:reply-to; bh=fG1OxMornZ72vrWdkK/L/931jxhh8ttLn2tvmGHTing=; b=LQyNNIWR0JG6rjL9rAN7qW9pxsa5V10mBj48XvkeR4y0fYO1r0BIiTJxkCUYsXEP1e Y2fduYeGgo9i0m53n1XY2hDzNrjCfqvpd6eeoFCyx9EiFw8EIszZ08BWUPm0Kwsgztz7 KtSjMYHzeMP64ajOfum5BhBjwLnC2z/kQuRubr05UxfByAs5jHDh0XRWQhc8yKcpXB0F IM9xwvsGSjWWvJ+FRVNdAiYjQLVeewJAg+Z1skqb4HCOr4yF4FgcBXd+BZzvcEIogbvF dE2HrZ3ko6PfKIrmcR8zko3hlYjIgcQUA5xLfzhJirN86YEFpQrDgYQRp8sBSzQm5cks xWxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769437215; x=1770042015; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fG1OxMornZ72vrWdkK/L/931jxhh8ttLn2tvmGHTing=; b=FgWdYnLPi5L8I5U+kx7uYSoabNoprbGbp1RFD4VeQgBh3rDHTeflC/zAYZP634F9q3 ahwWV9wG1tNryTCLbZk17YxPHtv2bvjklXxtxrH55+SNZvhoXlvtwivby4Wks9vNWm3L ylC+FTnvdNPzqgM4ozwo40tqwCB7TWEDpyGSHsj1gtsM9UvSsJ7vYfjeJDeGwBUivZry 1H7lvkpG2SM+KA9DaS4zuMn5EAkIJXRuds9vKqeD5SNdFeQZzZB3FemnZGyNriK5ObRb xIdENPaE+uQr+Ucg72hKyhLVjH1r/v30Wzxtamy9/fiHPvdkmMOXU650HznhvtwUCgDV 1ong== X-Forwarded-Encrypted: i=1; AJvYcCV+MRSs+9f4yhm1/1IS3IOxBiBgYWgQIyp/aWsaYif0T1yfTOjMUlTH/rFH4EZ4HLHZlU39yWBzIJC7iV4=@vger.kernel.org X-Gm-Message-State: AOJu0YyjRolFyrB0DGzWseTGfynTjTTeTZVcOIKzzea4BzoOlOcKE4pu tpF1W3Kj9iftF58SGzgJ4P9MvayiktaV8lCAUVq0xhZQ8+Yc9xK+GEwfGlEKRAj5R0iC2SWxXVT R3wlBSLkzYEaXGOhpbiLQfYqRJwLU0vJJ/1fpSPsqjIRCL7a/9Bse87MF28v5y3Tq1w== X-Gm-Gg: AZuq6aI1/kZUAYcoDCXVsWLvtU4ePs50KAfmTS3C9I44moArrozdVjF+GdJn9sybFgs FtVRSyI2BQJgxXWIkBcvDB+opnWD+LkDuexIMBiiX+T3od5O3BZoRJP9AUihhpM7t9VOxOuZrsq UekCbBZjJLrmsbJ+SWk1GZv7Fnr9xSqMv9vgq4Q2Z5aq2u3rPB/3l4/KEhWqLP5p3IiM7nXitrM Fiw7BbmZd2hT4h3Q5NIW8pbqLC8269uF0gMGZtpvY2/Ed7dB05Ego3ELrXZgZwCDMvRPzvTBx09 arQ3VJCXhPP7L4qS8WzbNJFoE7qeUA583yG1xGmx7xM+ZWlBAfqPieWk9juICIjWNllK0ovDejm bW21xRwRXZiOerjLco3CpR0TaH0Hzpjdr6DzAC0mOpFr9oz2Ed7Gsa30myy4YGBzmoS951mzZnf dwktDN3+My X-Received: by 2002:a05:6000:ed1:b0:435:a43b:2dde with SMTP id ffacd0b85a97d-435c9b1af15mr6839232f8f.15.1769437215313; Mon, 26 Jan 2026 06:20:15 -0800 (PST) X-Received: by 2002:a05:6000:ed1:b0:435:a43b:2dde with SMTP id ffacd0b85a97d-435c9b1af15mr6839182f8f.15.1769437214786; Mon, 26 Jan 2026 06:20:14 -0800 (PST) Received: from gmonaco-thinkpadt14gen3.rmtit.csb (185-132-178-103.hosted-by-worldstream.net. [185.132.178.103]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-435b1f7c8efsm30933323f8f.42.2026.01.26.06.20.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jan 2026 06:20:14 -0800 (PST) Message-ID: Subject: Re: [PATCH v2] sched/deadline: Reset dl_server execution state on stop From: Gabriele Monaco To: Andrea Righi , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tejun Heo , Joel Fernandes , David Vernet , Changwoo Min , Daniel Hodges , sched-ext@lists.linux.dev, linux-kernel@vger.kernel.org Date: Mon, 26 Jan 2026 15:20:12 +0100 In-Reply-To: <20260123161645.2181752-1-arighi@nvidia.com> References: <20260123161645.2181752-1-arighi@nvidia.com> Autocrypt: addr=gmonaco@redhat.com; prefer-encrypt=mutual; keydata=mDMEZuK5YxYJKwYBBAHaRw8BAQdAmJ3dM9Sz6/Hodu33Qrf8QH2bNeNbOikqYtxWFLVm0 1a0JEdhYnJpZWxlIE1vbmFjbyA8Z21vbmFjb0BrZXJuZWwub3JnPoiZBBMWCgBBFiEEysoR+AuB3R Zwp6j270psSVh4TfIFAmjKX2MCGwMFCQWjmoAFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgk Q70psSVh4TfIQuAD+JulczTN6l7oJjyroySU55Fbjdvo52xiYYlMjPG7dCTsBAMFI7dSL5zg98I+8 cXY1J7kyNsY6/dcipqBM4RMaxXsOtCRHYWJyaWVsZSBNb25hY28gPGdtb25hY29AcmVkaGF0LmNvb T6InAQTFgoARAIbAwUJBaOagAULCQgHAgIiAgYVCgkICwIEFgIDAQIeBwIXgBYhBMrKEfgLgd0WcK eo9u9KbElYeE3yBQJoymCyAhkBAAoJEO9KbElYeE3yjX4BAJ/ETNnlHn8OjZPT77xGmal9kbT1bC1 7DfrYVISWV2Y1AP9HdAMhWNAvtCtN2S1beYjNybuK6IzWYcFfeOV+OBWRDQ== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.2 (3.58.2-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Fri, 2026-01-23 at 17:16 +0100, Andrea Righi wrote: > dl_server_stop() can leave a deadline server in an inconsistent internal > state across stop/start transitions, causing it to bypass its required > deferral phase when restarted. This breaks the scheduler invariant that > a restarted server must re-establish eligibility before being allowed to > execute. >=20 > When the server is stopped (e.g., because the associated task blocks), > it's expected to transition back to an inactive, initial state. However, > dl_server_stop() does not fully reset the execution state. As a result, > the server can be logically inactive while still appearing as if it was > still running. >=20 > When the server is restarted via dl_server_start(), the following > sequence occurs: > =C2=A0 1. dl_server_start() calls enqueue_dl_entity(ENQUEUE_WAKEUP), > =C2=A0 2. enqueue_dl_entity() calls update_dl_entity(), > =C2=A0 3. update_dl_entity() checks (!dl_se->dl_defer_running) to decide > =C2=A0=C2=A0=C2=A0=C2=A0 whether to arm the deferral mechanism, > =C2=A0 4. because dl_defer_running is stale, the check fails, > =C2=A0 5. dl_defer_armed and dl_throttled are not set, > =C2=A0 6. enqueue_dl_entity() skips start_dl_timer(), because > =C2=A0=C2=A0=C2=A0=C2=A0 dl_throttled =3D=3D 0, > =C2=A0 7. the server is enqueued via __enqueue_dl_entity(), > =C2=A0 8. the scheduler picks the server to run, > =C2=A0 9. update_curr_dl_se() detects that the server has exhausted its > =C2=A0=C2=A0=C2=A0=C2=A0 runtime (or has negative runtime), as it wasn't = properly > =C2=A0=C2=A0=C2=A0=C2=A0 replenished/deferred, > =C2=A010. the server is throttled (dl_throttled set to 1) and dequeued, > =C2=A011. the server repeatedly cycles through wakeup and throttling, > =C2=A0=C2=A0=C2=A0=C2=A0 effectively receiving no usable CPU bandwidth. Hello, I remember wondering why defer_running was kept after stop and Peter sugges= ted it's to avoid penalising tasks with short sleeps. [1] Clearing defer_running on stop is in fact removing the edge from A:init to D:running , isn't it? The server should be able to start as running and not= only deferred (dl_defer_armed and dl_throttled set). In the sequence you described above, I wonder why the enqueue is never replenishing. As far as I understand the runtime should remain <=3D 0 only = as long as the enqueue occurs before the deadline, after that it should simply repl= enish a new period (pushing deadline and restoring runtime). What am I missing here? Thanks, Gabriele [1] - https://lore.kernel.org/lkml/20251111111716.GL278048@noisy.programming.kick= s-ass.net >=20 > This results in starvation of the tasks serviced by the deadline server > in the presence of competing RT workloads. >=20 > This issue can be confirmed adding debugging traces, which show that the > server skips the deferral timer and is immediately throttled upon > execution with negative runtime: >=20 > =C2=A0DEBUG: dl_server_start: dl_defer_running=3D1 active=3D0 > =C2=A0DEBUG: enqueue_dl_entity: flags=3D1 dl_throttled=3D0 dl_defer=3D1 > =C2=A0DEBUG: update_dl_entity: dl_defer_running=3D1 > =C2=A0DEBUG: enqueue_dl_entity: SKIPPING start_dl_timer! dl_throttled=3D0 > =C2=A0... > =C2=A0DEBUG: update_curr_dl_se: THROTTLED runtime=3D-954758 >=20 > Fix this by properly resetting dl_defer_running in dl_server_stop(), > ensuring the server correctly enters the defer phase upon restart. >=20 > This issue is quite difficult to observe when only the fair server > is present, as the required stop/start patterns are relatively rare. > However, it becomes easier to trigger with an additional deadline server > with more frequent server lifecycle transitions (such as a sched_ext > deadline server). >=20 > This change is a prerequisite for introducing a sched_ext deadline > server, as it ensures correct and predictable behavior across server > stop/start cycles. >=20 > Link: https://lore.kernel.org/all/aXEMat4IoNnGYgxw@gpd4/ > Signed-off-by: Andrea Righi > --- > Changes in v2: > =C2=A0- Update state machine documentation > =C2=A0- Link to v1: > https://lore.kernel.org/all/20260122140833.1655020-1-arighi@nvidia.com/ >=20 > =C2=A0kernel/sched/deadline.c | 4 +++- > =C2=A01 file changed, 3 insertions(+), 1 deletion(-) >=20 > diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c > index c509f2e7d69de..e42867061ea77 100644 > --- a/kernel/sched/deadline.c > +++ b/kernel/sched/deadline.c > @@ -1615,7 +1615,7 @@ void dl_server_update(struct sched_dl_entity *dl_se= , s64 > delta_exec) > =C2=A0 *=C2=A0=C2=A0 dl_server_active =3D 0 > =C2=A0 *=C2=A0=C2=A0 dl_throttled =3D 0 > =C2=A0 *=C2=A0=C2=A0 dl_defer_armed =3D 0 > - *=C2=A0=C2=A0 dl_defer_running =3D 0/1 > + *=C2=A0=C2=A0 dl_defer_running =3D 0 > =C2=A0 *=C2=A0=C2=A0 dl_defer_idle =3D 0 > =C2=A0 * > =C2=A0 * [B] - zero_laxity-wait > @@ -1704,6 +1704,7 @@ void dl_server_update(struct sched_dl_entity *dl_se= , s64 > delta_exec) > =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 hrtimer_try_to_cancel(); > =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dl_defer_armed =3D 0; > =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dl_throttled =3D 0; > + *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dl_defer_running =3D 0; > =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dl_server_active =3D 0; > =C2=A0 *=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 // [A] > =C2=A0 *=C2=A0=C2=A0 return p; > @@ -1813,6 +1814,7 @@ void dl_server_stop(struct sched_dl_entity *dl_se) > =C2=A0 hrtimer_try_to_cancel(&dl_se->dl_timer); > =C2=A0 dl_se->dl_defer_armed =3D 0; > =C2=A0 dl_se->dl_throttled =3D 0; > + dl_se->dl_defer_running =3D 0; > =C2=A0 dl_se->dl_defer_idle =3D 0; > =C2=A0 dl_se->dl_server_active =3D 0; > =C2=A0}