From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94CFFC001DC for ; Thu, 27 Jul 2023 13:43:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232532AbjG0NnH (ORCPT ); Thu, 27 Jul 2023 09:43:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233609AbjG0NnE (ORCPT ); Thu, 27 Jul 2023 09:43:04 -0400 Received: from mblankhorst.nl (lankhorst.se [IPv6:2a02:2308:0:7ec:e79c:4e97:b6c4:f0ae]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC6EC2D64; Thu, 27 Jul 2023 06:43:03 -0700 (PDT) Message-ID: <05178cf3-df1c-80a7-12ad-816fafbc2df7@linux.intel.com> Date: Thu, 27 Jul 2023 15:42:58 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: [PATCH 16/17] cgroup/drm: Expose memory stats Content-Language: en-US To: Tejun Heo Cc: Tvrtko Ursulin , Intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?UTF-8?Q?St=c3=a9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?UTF-8?Q?Christian_K=c3=b6nig?= , Brian Welty , Tvrtko Ursulin , Eero Tamminen References: <20230712114605.519432-1-tvrtko.ursulin@linux.intel.com> <20230712114605.519432-17-tvrtko.ursulin@linux.intel.com> From: Maarten Lankhorst In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hey, On 2023-07-26 21:44, Tejun Heo wrote: > Hello, > > On Wed, Jul 26, 2023 at 12:14:24PM +0200, Maarten Lankhorst wrote: >>> So, yeah, if you want to add memory controls, we better think through how >>> the fd ownership migration should work. >> >> I've taken a look at the series, since I have been working on cgroup memory >> eviction. >> >> The scheduling stuff will work for i915, since it has a purely software >> execlist scheduler, but I don't think it will work for GuC (firmware) >> scheduling or other drivers that use the generic drm scheduler. >> >> For something like this, you would probably want it to work inside the drm >> scheduler first. Presumably, this can be done by setting a weight on each >> runqueue, and perhaps adding a callback to update one for a running queue. >> Calculating the weights hierarchically might be fun.. > > I don't have any idea on this front. The basic idea of making high level > distribution decisions in core code and letting individual drivers enforce > that in a way which fits them the best makes sense to me but I don't know > enough to have an opinion here. > >> I have taken a look at how the rest of cgroup controllers change ownership >> when moved to a different cgroup, and the answer was: not at all. If we > > For persistent resources, that's the general rule. Whoever instantiates a > resource gets to own it until the resource gets freed. There is an exception > with the pid controller and there are discussions around whether we want > some sort of migration behavior with memcg but yes by and large instantiator > being the owner is the general model cgroup follows. > >> attempt to create the scheduler controls only on the first time the fd is >> used, you could probably get rid of all the tracking. >> This can be done very easily with the drm scheduler. >> >> WRT memory, I think the consensus is to track system memory like normal >> memory. Stolen memory doesn't need to be tracked. It's kernel only memory, >> used for internal bookkeeping only. >> >> The only time userspace can directly manipulate stolen memory, is by mapping >> the pinned initial framebuffer to its own address space. The only allocation >> it can do is when a framebuffer is displayed, and framebuffer compression >> creates some stolen memory. Userspace is not >> aware of this though, and has no way to manipulate those contents. > > So, my dumb understanding: > > * Ownership of an fd can be established on the first ioctl call and doesn't > need to be migrated afterwards. There are no persistent resources to > migration on the first call. > > * Memory then can be tracked in a similar way to memcg. Memory gets charged > to the initial instantiator and doesn't need to be moved around > afterwards. There may be some discrepancies around stolen memory but the > magnitude of inaccuracy introduced that way is limited and bound and can > be safely ignored. > > Is that correct? Hey, Yeah mostly, I think we can stop tracking stolen memory. I stopped doing that for Xe, there is literally nothing to control for userspace in there. Cheers, Maarten