From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 83AF5346AC4 for ; Wed, 1 Apr 2026 13:00:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775048411; cv=none; b=mDyhnkjndpw3cvlHgPhNGqQEmlcsJoc6mrr8DcaMqB8Nm0e9O9yT2zKvoP3c2uCkrxyMlBqcAX7bFxqgwhf35DTQCQ2aRkPUT7EH+tibR7EpPJha34dY5XdKknSSjiKTgcv7syY8FK3VvUgSfxADN6bUNJ1D8RdE6rahkLciW4Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775048411; c=relaxed/simple; bh=6uHl6B/W7ILJ4NgzIzCT9MotTPr/A68X63LVI0rT2XM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=KHG/ttM7YlN4xLmAI4A/G0mFHqNc4q6/JokHKe1soqlyg1sOS4S3WHwm60XGd7MRzRRZt1oGZNs4CBBZX++QF07vR2Cpz/GxElbm7rRELHKC1RX9kFn5qWkC8v3Y4XyHSjhZZfgyFM4ikCdN77HEr77IvYRtIQVJRGelDniXM/A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=QhjkiZot; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="QhjkiZot" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 065241F91; Wed, 1 Apr 2026 06:00:03 -0700 (PDT) Received: from [192.168.178.100] (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7019D3F915; Wed, 1 Apr 2026 06:00:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1775048408; bh=6uHl6B/W7ILJ4NgzIzCT9MotTPr/A68X63LVI0rT2XM=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=QhjkiZotpk9FPcTvO5nJ8JyCKJDQWgVoM+2FEpmZst6UQ+ZYzoKTzaqYOJvece4mi uBEmIw7qTF28de67PoH/CKML7TNF/zSRrxXoJucDxkgex0r9KpXygAh3O1foSO7jyS ddzcM2ccs21VeEkqSe6dULNPYrSyofZC/W2Nn7Ts= Message-ID: <1c0e5a70-22b7-4cbd-8ca9-0a7da27ed7a9@arm.com> Date: Wed, 1 Apr 2026 15:00:04 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [Question] Sched: Severe scheduling latency (>10s) observed on kernel 6.12 with specific workload To: Xuewen Yan , Vincent Guittot Cc: John Stultz , Peter Zijlstra , Ingo Molnar , Juri Lelli , Steven Rostedt , Benjamin Segall , Mel Gorman , Valentin Schneider , linux-kernel , =?UTF-8?B?546L56eRIChLZSBXYW5nKQ==?= , Xuewen Yan , "hongyu.jin@unisoc.com" , guohua.yan@unisoc.com References: From: Dietmar Eggemann Content-Language: en-GB In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 01.04.26 12:48, Xuewen Yan wrote: > On Wed, Apr 1, 2026 at 6:05 PM Vincent Guittot > wrote: >> >> On Wed, 1 Apr 2026 at 08:04, Xuewen Yan wrote: >>> >>> On Wed, Apr 1, 2026 at 12:25 PM John Stultz wrote: >>>> >>>> On Tue, Mar 31, 2026 at 7:32 PM Xuewen Yan wrote: >>>>> >>>>> Dear Linux maintainers and reviewers, >>>>> >>>>> I am writing to report a severe scheduling latency issue we recently >>>>> discovered on Linux Kernel 6.12. >>>>> >>>>> Issue Description >>>>> >>>>> We observed that when running a specific background workload pattern, >>>>> certain tasks experience excessive scheduling latency. The delay from >>>>> the runnable state to running on the CPU exceeds 10 seconds, and in >>>>> extreme cases, it reaches up to 100 seconds. >>>>> >>>>> Environment Details >>>>> >>>>> Kernel Version: 6.12.58-android16-6-g3835fd28159d-ab000018-4k >>>>> Architecture: [ ARM64] >>>>> Hardware: T7300 Is this 4 big & 4 little CPUs? >>>>> Config: gki_defconfig >>>>> >>>>> RT-app‘s workload Pattern: >>>>> >>>>> { >>>>> "tasks" : { >>>>> "t0" : { >>>>> "instance" : 40, >>>>> "priority" : 0, >>>>> "cpus" : [ 0, 1, 2, 3 ], >>>>> "taskgroup" : "/background", >>>>> "loop" : -1, >>>>> "run" : 200, >>>>> "sleep" : 50 >>>>> } >>>>> } >>>>> } >>>>> >>>>> And we have applied the following patchs: >>>>> >>>>> https://lore.kernel.org/all/20251216111321.966709786@linuxfoundation.org/ >>>>> https://lore.kernel.org/all/20260106170509.413636243@linuxfoundation.org/ >>>>> https://lore.kernel.org/all/20260323134533.805879358@linuxfoundation.org/ Does the issue happen on v6.12.58 plain (android) or only when those 3 additional patches are applied on top? d5843e1530d8 - sched/fair: Forfeit vruntime on yield (2025-12-18 Fernand Sieber) v6.12.63 bddd95054e33 - sched/eevdf: Fix min_vruntime vs avg_vruntime (2026-01-08 Peter Zijlstra) v6.12.64 d2fc2dcfce47 - sched/fair: Fix zero_vruntime tracking (2026-03-25 Peter Zijlstra) v6.12.78 [...]