From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from canpmsgout01.his.huawei.com (canpmsgout01.his.huawei.com [113.46.200.216]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 981463ECBEB for ; Tue, 7 Jul 2026 12:51:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.216 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783428685; cv=none; b=oAxHNvlRh5kE7PV66uSAmrm6jlD80Ohff03ixx7yDVVJOhzqRITIlmKivhMytyrRj4vUJKNqDqRSG36ySCFieA/wBA4VyaGlCvQYCyJWUDj7hU5qRAaCdTN0xsAD7EDggSbGK80EU7aEddIYF7yJ768f0ucb5kTAAqmj8AC1vBE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783428685; c=relaxed/simple; bh=G3uLbwUtibkCOO/oYywJfNy+oeo1x8bOt9njRuApRBQ=; h=Message-ID:Date:MIME-Version:From:Subject:To:CC:References: In-Reply-To:Content-Type; b=d8519LrGYTRGSe4dlZYkuA5/c2AMsiM4w60kcwC7xI9wBBfvq69vnwsQo+Pymphgpj3BNWlMAwJ2cfzY6ERp/shMjMrq/ckjaXflOPkEGYiMLnXfAYUA088BquVMDeU4GrJ+Rp0mRX5pDnnnrFfnqgLkz0zVSdpjk/Uu1gkQZvc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=Lo8Hv7rt; arc=none smtp.client-ip=113.46.200.216 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="Lo8Hv7rt" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=prxrcM73xjvyk1MiBYlNXSL52q5hspDUDOmphscjUZE=; b=Lo8Hv7rtKeh9Zcs+aO0TLtdREwdO/cf0d3F/GHtmH9vnxJH6pxszGGbs8TsvxTh7Wt26u5xWx 0odK3nSJL4sNoT+qq3bI+DHRSxsYSEMkmDrCHd0op7hrFTcDoXaDtxfE68PP7aOIroA0fHseJ1w UOoRwV71j1Eb4OOH9pCVn84= Received: from mail.maildlp.com (unknown [172.19.162.144]) by canpmsgout01.his.huawei.com (SkyGuard) with ESMTPS id 4gvgpr4rFcz1T4G0; Tue, 7 Jul 2026 20:42:12 +0800 (CST) Received: from kwepemr500016.china.huawei.com (unknown [7.202.195.68]) by mail.maildlp.com (Postfix) with ESMTPS id 2981E4056D; Tue, 7 Jul 2026 20:51:12 +0800 (CST) Received: from [10.67.111.161] (10.67.111.161) by kwepemr500016.china.huawei.com (7.202.195.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 7 Jul 2026 20:51:11 +0800 Message-ID: <9f4f6b31-1cfb-4d2d-9f52-d3dcd00f3a3a@huawei.com> Date: Tue, 7 Jul 2026 20:51:10 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: chenjinghuang Subject: Re: [RFC PATCH] sched/fair: Address avruntime drift caused by entity_lag() To: Vincent Guittot CC: , , , , , , , , , References: <20260703092635.583332-1-chenjinghuang2@huawei.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-ClientProxiedBy: kwepems200002.china.huawei.com (7.221.188.68) To kwepemr500016.china.huawei.com (7.202.195.68) On 7/7/2026 12:03 AM, Vincent Guittot wrote: > On Fri, 3 Jul 2026 at 11:50, Chen Jinghuang wrote: >> >> While running EEVDF-related test cases, we observed an anomalous >> phenomenon where the vruntime of a continuously running scheduling >> entity (se) exhibits an overall downward trend. >> >> The test topology and observed behavior are as follows: >> >> CPU 0 CPU 1 / Other CPUs >> +----------------------------+ +----------------------------+ >> | cgroup A cgroup B | | cgroup A cgroup B | >> | |- A1 (Run) |- B1 | | |- A2, A3... |- B2, B3...| >> | (50us S)| | (Frequent sleep/wake) | >> | (10us W)| | | >> +----------------------------+ +----------------------------+ >> | >> v >> A2/A3 frequent sleep/wake -> Frequent reweight of gse(A) on CPU 0 >> -> Triggers vlag clamping -> Unidirectional avruntime drift >> -> gse->vruntime drops abnormally (while gse weight remains stable) >> >> According to function_graph traces, frequent sleep/wake cycles of >> sibling tasks within the same cgroup cause the running task to repeatedly >> trigger update_cfs_cgroup() -> reweight_entity(). >> >> The root cause appears to point to the entity_lag() boundary limit >> introduced in `reweight_entity()` to prevent overflows. During repeated >> weight scaling, this limit introduces an asymmetric cumulative error. >> Connsider an example here where se->vruntime > avruntime: >> >> 1. Downscale: >> se->vlag = entity_lag(cfs_rq, se, avruntime); > > The clamp applied in entity_lag() should "in theory never" happen. > However, I observed that vlag does indeed get clamped in entity_lag() on CPU 0 in the test case > Do you have more details to share about your setup? > Here is the exact layout of the cgroups: - cgroup A: Besides task A1 (which is bound to CPU 0), there are 20 other tasks running on the remaining CPUs (excluding CPU 0) that frequently sleep and wake up. This setup is designed to continuously trigger weight adjustments for A1. - cgroup B: There is only a single task B1 bound to CPU 0, which sleeps for long periods and wakes up briefly to compete with A1.(Please note that my previous diagram mistakenly showed other tasks running under cgroup B on other CPUs; that was an error on my part). > Which kernel version do you use ? A number of changes have been queued > recently that should ensure that the vlag never goes above the > theoretical limit. > I am running the mainline kernel, specifically on the v7.2-rc2 tag. > How many CPUs are in the system ? > Is there other tasks running in the cgroups ? > The system has 32 CPUs in total. > Could you try to run without RUN_TO_PARITY ? > echo NO_RUN_TO_PARITY > /sys/kernel/debug/sched/features > I tried disabling RUN_TO_PARITY, but the issue persists—vlag still exceeds the limit and triggers the clamp. > Do you set the share of task groups to particular values? > Do you change the slice of the tasks ? > No, I haven't set specific shares for the task groups (they are all at the default value of 1024), nor have I modified the default task slices. > Note that even without clamping, part of the vlag is lost in div64_long > >> se->vlag = div64_long(se->vlag * old_weight, weight); >> se->vruntime = avruntime - se->vlag; >> >> If the calculated vlag exceeds the limit boundary, it is clamped. >> This causes the scaled vlag to be bigger than its theoretical value, >> making the resulting se->vruntime smaller than it should be (forced >> closer to avruntime). >> >> 2. Upscale: >> Because se->vruntime already carries the clamping error from the >> previous step, it implicitly shifts the subsequent global avruntime >> computation trajectory. When the weight is restored, se->vruntime >> deviates from its original true value, preventing the true vlag from >> being reconstructed and leading to a unidirectional avruntime drift. >> >> Under frequent cgroup reweighting, this asymmetric truncation error >> accumulates, causing a unidirectional downward drift of vruntime. >> >> Regarding this vlag limit mechanism in reweight_entity(), we would like >> to raise a few questions for discussion: >> >> Question 1: Is this deviation of se->vruntime expected? >> For a currently running entity (curr se), due to RUN_TO_PARITY, it >> typically operates in a state where se->vruntime > avruntime. During >> reweighting, the limit forces its vruntime downward (closer to avruntime), >> while attributes like deadline remain unaffected by the limit. >> >> Question 2: Is the unidirectional drift of avruntime reasonable? >> Since avruntime = avg_vruntime(), the deviation of se->vruntime >> before and after reweighting (while its weight remains unchanged) shifts >> the global avruntime downward. According to EEVDF core principles, >> avruntime should remain invariant across weight changes. >> (https://lore.kernel.org/lkml/20260415211149.2658910-1-jstultz@google.com/), >> >> Question 3: Is the reduction of avruntime fair to other entities? >> A non-functional reduction in global avruntime passively decreases the >> se->vlag of other queued entities on the runqueue, which practically >> worsens the starvation of those tasks. >> >> Question 4: Does this lead to complete preemption suppression? >> In entity_tick -> update_deadline, because curr->vruntime is abnormally >> pulled down by the clamping error while its deadline is unchanged, the >> condition vruntime < deadline can be continuously satisfied at every >> tick. This skips resched_curr(), suppressing preemption until the >> frequent reweighting stops or curr's vlag falls back within the limit. >> >> While entity_lag() was originally introduced to prevent overflow-induced >> NULL pointer dereferences, given its significant impact on EEVDF fairness, >> should the necessity and implementation of this limit be reconsidered? >> >> Analyzed-by: Hui Tang >> Analyzed-by: Zhang Qiao >> Analyzed-by: Zicheng Qu >> Analyzed-by: Chen Jinghuang >