From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 243E72DBF4B for ; Wed, 7 Jan 2026 10:27:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767781634; cv=none; b=o3fKZVS5EX6lPbh/r+cLJy3IY/DxaOxRDplL0NiC77rtvKThViDrVo6l6IHQQlIZMRAwwlyDZL7btXqya3KgIPqFlaLhMMhB2AeHuIGCna9RPlsa6ZGutAxMV/bhH4jaeswPlrBQ3XqY3Ljd2rw2I8RCCcBdOf0s2gfo2wyWZ4w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767781634; c=relaxed/simple; bh=/50yRi6YRSTU/2kYkx9ziZXLdBGr3xEksZmwJeJXwI8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nr6MArSvqbhORDMMcg/+/nOC4g31dc0v2d6td8lxK5oLBIkvjCDh1I6rAHUr0OUqcn/O1RwI4urfoCEc9PpZRKbT3vjOaPS9S0GuSYXjGW7petXJwSF5ilvIPvu02sh4AmoJSnHKpaKo15aJFkewh20lpSVqhypMpahGx4vbm9Q= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=c1u3oSwm; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="c1u3oSwm" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0BEekeHtAJsf/Jn6ZZIFs7FDODHkSvw1u4IPW12GSKY=; b=c1u3oSwmBJocn73zl3yYMb6+lg paTWR0TJuVCJza4ORbNXnBnfnuwtmTkX2lAj/uu19JZga9F0THSA/8AmNapw1qqNq/odCyK3CQi/k a7rdTDdSaN1RReeeI1bRrV592eQOOwpF1xEeZmJAsMFCbPBJatMHGyyjs68SWRxArB6JksPyskLAZ tzNgRxSnl3FFQCw4FlagdG/tYbGXydMklaZQg8ZKyLHfpum14/9Eo9C15rHW8R0gKCDBud8BVhDo3 YS06t6RLz82pVOnzREoqWejZqrHo9pNXTiv5TOvB/fkFPSoL/Cm7lWDUtcn4GpEf+qN9E3rpb0NpH Hh41r0tw==; Received: from 2001-1c00-8d85-5700-266e-96ff-fe07-7dcc.cable.dynamic.v6.ziggo.nl ([2001:1c00:8d85:5700:266e:96ff:fe07:7dcc] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1vdQkn-0000000DF0V-23X8; Wed, 07 Jan 2026 10:27:01 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 0823430056B; Wed, 07 Jan 2026 11:27:00 +0100 (CET) Date: Wed, 7 Jan 2026 11:26:59 +0100 From: Peter Zijlstra To: Juri Lelli Cc: Aaron Tomlin , Shrikanth Hegde , neelx@suse.com, sean@ashe.io, mproche@gmail.com, linux-kernel@vger.kernel.org, mingo@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Subject: Re: [RFC PATCH 0/1] sched/fair: Feature to suppress Fair Server for NOHZ_FULL isolation Message-ID: <20260107102659.GE2393663@noisy.programming.kicks-ass.net> References: <20260106034209.2703289-1-atomlin@atomlin.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Wed, Jan 07, 2026 at 10:48:12AM +0100, Juri Lelli wrote: > Hello! > > On 06/01/26 09:49, Aaron Tomlin wrote: > > On Tue, Jan 06, 2026 at 02:37:49PM +0530, Shrikanth Hegde wrote: > > > If all your SCHED_FIFO is pinned and their scheduling decisions > > > are managed in userspace, using isolcpus would offer you better > > > isolations compared to nohz_full. > > > > Hi Shrikanth, > > > > You are entirely correct; isolcpus=domain (or isolcpus= without flags as > > per housekeeping_isolcpus_setup()) indeed offers superior isolation by > > removing the CPU from the scheduler load-balancing domains. > > > > I must apologise for the omission in my previous correspondence. I > > neglected to mention that our specific configuration utilises isolcpus= in > > conjunction with nohz_full=. > > > > > > However, the extant "Fair Server" (Deadline Server) architecture > > > > compromises this isolation guarantee. At present, should a background > > > > SCHED_OTHER task be enqueued, the scheduler initiates the Fair Server > > > > (dl_server_start). As the Fair Server functions as a SCHED_DEADLINE entity, > > > > its activation increments rq->dl.dl_nr_running. > > > > > > > > > > There is runtime allocated to fair server. If you make them 0 on CPUs of > > > interest, wouldn't that work? > > > > > > /sys/kernel/debug/sched/fair_server//runtime > > > > Yes, you are quite right; setting the fair server runtime to 0 (via > > /sys/kernel/debug/sched/fair_server/[cpu]/runtime) does indeed achieve the > > desired effect. In my testing, the SCHED_FIFO task on the fully > > adaptive-tick CPU remains uninterrupted by the restored clock-tick when > > this configuration is applied. Thank you. > > > > However, I believe it would be beneficial if this scheduling feature were > > available as an automatic kernel detection mechanism. While the manual > > runtime adjustment works, having the kernel automatically detect the > > condition - where an RT task is running and bandwidth enforcement is > > disabled - would provide a more seamless and robust solution for > > partitioned systems without requiring external intervention. > > I may consider an improved version of the patch that includes a "Fair > > server disabled" warning much like in sched_fair_server_write(). > > I am not sure either we need/want the automatic mechanism, as we already > have the fair_server interface. I kind of think that if any (kthread > included) CFS task is enqueued on an "isolated" CPU the problem might > reside in sub-optimal isolation (usually a config issue or a kernel > issue that might need solving - e.g. a for_each_cpu loop that needs > changing). Starving such tasks might anyway end in a system crash of > sort. We must not starve fair tasks -- this can severely affect the system health. Specifically per-cpu kthreads getting starved can cause complete system lockup when other CPUs go wait for completion and such. We must not disable the fair server, ever. Doing do means you get to keep the pieces. The only sane way is to ensure these tasks do not get queued in the first place.