From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S941479AbcIZMuC (ORCPT ); Mon, 26 Sep 2016 08:50:02 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:57838 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933115AbcIZMuA (ORCPT ); Mon, 26 Sep 2016 08:50:00 -0400 Subject: Re: group scheduler regression since 4.3 (bisect 9d89c257d sched/fair: Rewrite runnable load and utilization average tracking) To: Peter Zijlstra References: <45222b6f-4849-f1f4-fdf5-2a26ac9a3ed4@de.ibm.com> <20160926105621.GZ5016@twins.programming.kicks-ass.net> <20160926115300.GA5016@twins.programming.kicks-ass.net> <4c4e8838-9a6a-62b9-a8b7-48e4d375604e@de.ibm.com> <20160926121025.GC5016@twins.programming.kicks-ass.net> Cc: Yuyang Du , Ingo Molnar , Linux Kernel Mailing List , vincent.guittot@linaro.org, Morten.Rasmussen@arm.com, dietmar.eggemann@arm.com, pjt@google.com, bsegall@google.com From: Christian Borntraeger Date: Mon, 26 Sep 2016 14:49:53 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20160926121025.GC5016@twins.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16092612-0008-0000-0000-000002D280E3 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16092612-0009-0000-0000-00001A1AF908 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-26_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609020000 definitions=main-1609260244 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/26/2016 02:10 PM, Peter Zijlstra wrote: > On Mon, Sep 26, 2016 at 02:01:43PM +0200, Christian Borntraeger wrote: >> They applied ok on next from 9/13. Things go even worse. >> With this host configuration: >> >> CPU NODE BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED ADDRESS >> 0 0 0 0 0 0:0:0:0 yes yes 0 >> 1 0 0 0 0 1:1:1:1 yes yes 1 >> 2 0 0 0 1 2:2:2:2 yes yes 2 >> 3 0 0 0 1 3:3:3:3 yes yes 3 >> 4 0 0 1 2 4:4:4:4 yes yes 4 >> 5 0 0 1 2 5:5:5:5 yes yes 5 >> 6 0 0 1 3 6:6:6:6 yes yes 6 >> 7 0 0 1 3 7:7:7:7 yes yes 7 >> 8 0 0 1 4 8:8:8:8 yes yes 8 >> 9 0 0 1 4 9:9:9:9 yes yes 9 >> 10 0 0 1 5 10:10:10:10 yes yes 10 >> 11 0 0 1 5 11:11:11:11 yes yes 11 >> 12 0 0 1 6 12:12:12:12 yes yes 12 >> 13 0 0 1 6 13:13:13:13 yes yes 13 >> 14 0 0 1 7 14:14:14:14 yes yes 14 >> 15 0 0 1 7 15:15:15:15 yes yes 15 >> >> the guest was running either on 0-3 or on 4-15, but never >> used the full system. With group scheduling disabled everything was good >> again. So looks like that this bug has also some dependency on on the >> host topology. > > OK, so CPU affinities that unevenly straddle topology boundaries like > that are hard (and is generally not recommended), but its not > immediately obvious why it would be so much worse with cgroups enabled. Well thats what I get from LPAR... With CPUs 0-3 disabled things are better, but there is still 10% difference between group/nogroup. Will test Vincents v4 soon. In any case, would a 5 second sequence of /proc/sched_debug for the good/bad case with all 16 host CPUs (or the reduced 12 cpu set) be useful? Christian