From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161736AbdEXXki (ORCPT ); Wed, 24 May 2017 19:40:38 -0400 Received: from mga11.intel.com ([192.55.52.93]:54753 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1033927AbdEXXkg (ORCPT ); Wed, 24 May 2017 19:40:36 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.38,388,1491289200"; d="scan'208";a="1173980876" Subject: Re: [PATCH v2 for-4.12-fixes 1/2] sched/fair: Use task_groups instead of leaf_cfs_rq_list to walk all cfs_rqs To: Tejun Heo Cc: Ingo Molnar , Peter Zijlstra , linux-kernel@vger.kernel.org, Linus Torvalds , Vincent Guittot , Mike Galbraith , Paul Turner , Chris Mason , kernel-team@fb.com, mohini.narkhede@intel.com References: <20170509161740.GD8609@htj.duckdns.org> From: Tim Chen Message-ID: <9dad4417-4b9e-e466-9561-7569cec28ab6@linux.intel.com> Date: Wed, 24 May 2017 16:40:34 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <20170509161740.GD8609@htj.duckdns.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/09/2017 09:17 AM, Tejun Heo wrote: > Currently, rq->leaf_cfs_rq_list is a traversal ordered list of all > live cfs_rqs which have ever been active on the CPU; unfortunately, > this makes update_blocked_averages() O(total number of CPU cgroups) > which isn't scalable at all. > > The next patch will make rq->leaf_cfs_rq_list only contain the cfs_rqs > which are currently active. In preparation, this patch converts users > which need to traverse all cfs_rqs to use task_groups list instead. > > task_groups list is protected by its own lock and allows RCU protected > traversal and the order of operations guarantees that all online > cfs_rqs will be visited, but holding rq->lock won't protect against > iterating an already unregistered cfs_rq. However, the operations of > the two users that get converted - update_runtime_enabled() and > unthrottle_offline_cfs_rqs() - should be safe to perform on already > dead cfs_rqs, so adding rcu read protection around them should be > enough. > > Note that print_cfs_stats() is not converted. The next patch will > change its behavior to print out only active cfs_rqs, which is > intended as there's not much point in printing out idle cfs_rqs. > > v2: Dropped strong synchronization around removal and left > print_cfs_stats() unchanged as suggested by Peterz. > > Tejun, We did some preliminary testing of this patchset for a well known database benchmark on a 4 socket Skylake server system. It provides a 3.7% throughput boost which is significant for this benchmark. Thanks. Tim