From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966554AbcIZOMa (ORCPT ); Mon, 26 Sep 2016 10:12:30 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:60533 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S965676AbcIZOMY (ORCPT ); Mon, 26 Sep 2016 10:12:24 -0400 Subject: Re: group scheduler regression since 4.3 (bisect 9d89c257d sched/fair: Rewrite runnable load and utilization average tracking) To: Peter Zijlstra References: <45222b6f-4849-f1f4-fdf5-2a26ac9a3ed4@de.ibm.com> <20160926105621.GZ5016@twins.programming.kicks-ass.net> <20160926115300.GA5016@twins.programming.kicks-ass.net> <4c4e8838-9a6a-62b9-a8b7-48e4d375604e@de.ibm.com> <20160926121025.GC5016@twins.programming.kicks-ass.net> Cc: Yuyang Du , Ingo Molnar , Linux Kernel Mailing List , vincent.guittot@linaro.org, Morten.Rasmussen@arm.com, dietmar.eggemann@arm.com, pjt@google.com, bsegall@google.com From: Christian Borntraeger Date: Mon, 26 Sep 2016 16:12:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160926121025.GC5016@twins.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16092614-0040-0000-0000-000002349334 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16092614-0041-0000-0000-000021EFD2A9 Message-Id: <13ad6712-a3e2-f084-e12d-6cc947895e32@de.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-26_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609020000 definitions=main-1609260268 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/26/2016 02:10 PM, Peter Zijlstra wrote: > On Mon, Sep 26, 2016 at 02:01:43PM +0200, Christian Borntraeger wrote: >> They applied ok on next from 9/13. Things go even worse. >> With this host configuration: >> >> CPU NODE BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED ADDRESS >> 0 0 0 0 0 0:0:0:0 yes yes 0 >> 1 0 0 0 0 1:1:1:1 yes yes 1 >> 2 0 0 0 1 2:2:2:2 yes yes 2 >> 3 0 0 0 1 3:3:3:3 yes yes 3 >> 4 0 0 1 2 4:4:4:4 yes yes 4 >> 5 0 0 1 2 5:5:5:5 yes yes 5 >> 6 0 0 1 3 6:6:6:6 yes yes 6 >> 7 0 0 1 3 7:7:7:7 yes yes 7 >> 8 0 0 1 4 8:8:8:8 yes yes 8 >> 9 0 0 1 4 9:9:9:9 yes yes 9 >> 10 0 0 1 5 10:10:10:10 yes yes 10 >> 11 0 0 1 5 11:11:11:11 yes yes 11 >> 12 0 0 1 6 12:12:12:12 yes yes 12 >> 13 0 0 1 6 13:13:13:13 yes yes 13 >> 14 0 0 1 7 14:14:14:14 yes yes 14 >> 15 0 0 1 7 15:15:15:15 yes yes 15 >> >> the guest was running either on 0-3 or on 4-15, but never >> used the full system. With group scheduling disabled everything was good >> again. So looks like that this bug has also some dependency on on the >> host topology. > > OK, so CPU affinities that unevenly straddle topology boundaries like > that are hard (and is generally not recommended), but its not Ok so I created with cpu hotplug a symmetrical CPU topology: CPU NODE BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED ADDRESS 0 0 0 0 0 0:0:0:0 yes yes 0 1 0 0 0 0 1:1:1:1 yes yes 1 2 0 0 0 1 2:2:2:2 yes yes 2 3 0 0 0 1 3:3:3:3 yes yes 3 4 0 0 0 2 4:4:4:4 yes yes 4 5 0 0 0 2 5:5:5:5 yes yes 5 6 0 0 0 3 6:6:6:6 yes yes 6 7 0 0 0 3 7:7:7:7 yes yes 7 8 0 0 0 - ::: no yes 8 9 0 0 0 - ::: no yes 9 10 0 0 0 - ::: no yes 10 11 0 0 0 - ::: no yes 11 12 0 0 1 4 8:8:8:8 yes yes 12 13 0 0 1 4 9:9:9:9 yes yes 13 14 0 0 1 5 10:10:10:10 yes yes 14 15 0 0 1 5 11:11:11:11 yes yes 15 16 0 0 1 6 12:12:12:12 yes yes 16 17 0 0 1 6 13:13:13:13 yes yes 17 18 0 0 1 7 14:14:14:14 yes yes 18 19 0 0 1 7 15:15:15:15 yes yes 19 Same effect: Only half of the CPUs are used, but the number of guest CPUs == number of host cpus. Turns out that this is totally unrelated to this patch set, so it must be something else.