From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E43FEC433EF for ; Fri, 24 Jun 2022 20:17:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231220AbiFXURR (ORCPT ); Fri, 24 Jun 2022 16:17:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230495AbiFXURP (ORCPT ); Fri, 24 Jun 2022 16:17:15 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2391C82892 for ; Fri, 24 Jun 2022 13:17:11 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id lw20so6905251ejb.4 for ; Fri, 24 Jun 2022 13:17:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to; bh=A7KsArznjDszXEtjfLGnA6REJfLUYfQ3Oszk2m5SudA=; b=DysS05ypy7Dj7deDljGrAOAbGl2Xyt4/z99Z0sAnWou95elElB1LSY5/Ts4Ksh/mXP DLv6fOZpbapyMhr7MkdZvU+yZ2/QnFZ1mpgFnWqAY41HyRd5w3WAVUbpmAQFwqme6Ilf NCYlIIWT8tJDBPd7kF3JbEbHU58J7PTZbpGK0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=A7KsArznjDszXEtjfLGnA6REJfLUYfQ3Oszk2m5SudA=; b=QBVrrdbtR0sfaunTvbfhYI0VbXqN1l+cDO2TvFivZoR207OPU+zMtKG6ndI1B/81vn N6HShFkZY9hIG35fq5ZYWAmXvpo2O4cOEyacalSYGOWXdt7Ax5q1Cw3A9EeEyD8L/oSO B7lnyxE26ETlCaCJ495i8nDHeFfpBebo3syat+WVSyBXRppNGFFf6ENGEfiBY9RVDJc5 qwE6TGoPqEfhiNtu9Hsa80W6cJUwFM3ibemhPgKqzDzH2AWKoEp7oolw0D5ij9tMcrWK UmSrnk6YYGEI+W7IAuV9thwQ1M+grjVufWkjMJAkeyRkn1r3m9EnznBOyW0jM/ppnUGJ iOig== X-Gm-Message-State: AJIora9sAUSI1jT2e6s90AaDY9xJpd2eF/BJVioTaBC7/Ti+rEWwdnfs 93WUxxjEEjCqdDqTQs+busbEYw== X-Google-Smtp-Source: AGRyM1tbrhSpLiAB9dqU6edRxuLbyecMJr7L30I5wUrP26zUyqvNNkEwbuedAAWD5HRC48ej/oEHbg== X-Received: by 2002:a17:907:7d88:b0:726:2adb:2073 with SMTP id oz8-20020a1709077d8800b007262adb2073mr778546ejc.466.1656101829623; Fri, 24 Jun 2022 13:17:09 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id w20-20020aa7dcd4000000b0042dc882c823sm2642667edu.70.2022.06.24.13.17.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Jun 2022 13:17:08 -0700 (PDT) Date: Fri, 24 Jun 2022 22:17:06 +0200 From: Daniel Vetter To: "T.J. Mercier" Cc: Tejun Heo , Nicolas Dufresne , Zefan Li , Johannes Weiner , Jonathan Corbet , Greg Kroah-Hartman , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Christian Brauner , Hridya Valsaraju , Suren Baghdasaryan , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Benjamin Gaignard , Liam Mark , Laura Abbott , Brian Starkey , John Stultz , Shuah Khan , Daniel Vetter , John Stultz , Carlos Llamas , Kalesh Singh , Kenny.Ho@amd.com, Michal =?iso-8859-1?Q?Koutn=FD?= , Shuah Khan , kernel-team@android.com, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org Subject: Re: [PATCH v7 0/6] Proposal for a GPU cgroup controller Message-ID: Mail-Followup-To: "T.J. Mercier" , Tejun Heo , Nicolas Dufresne , Zefan Li , Johannes Weiner , Jonathan Corbet , Greg Kroah-Hartman , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Christian Brauner , Hridya Valsaraju , Suren Baghdasaryan , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Benjamin Gaignard , Liam Mark , Laura Abbott , Brian Starkey , John Stultz , Shuah Khan , John Stultz , Carlos Llamas , Kalesh Singh , Kenny.Ho@amd.com, Michal =?iso-8859-1?Q?Koutn=FD?= , Shuah Khan , kernel-team@android.com, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kselftest@vger.kernel.org References: <20220510235653.933868-1-tjmercier@google.com> <3365cd1d750e84fedc8e75d646a77ffd85619d35.camel@ndufresne.ca> <81026ef07c1ce20f8673b75b17bab79a2b39c548.camel@ndufresne.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: Linux phenom 5.10.0-8-amd64 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 15, 2022 at 10:31:21AM -0700, T.J. Mercier wrote: > On Fri, May 20, 2022 at 9:25 AM T.J. Mercier wrote: > > > > On Fri, May 20, 2022 at 12:47 AM Tejun Heo wrote: > > > > > > Hello, > > > > > > On Tue, May 17, 2022 at 04:30:29PM -0700, T.J. Mercier wrote: > > > > Thanks for your suggestion. This almost works. "dmabuf" as a key could > > > > work, but I'd actually like to account for each heap. Since heaps can > > > > be dynamically added, I can't accommodate every potential heap name by > > > > hardcoding registrations in the misc controller. > > > > > > On its own, that's a pretty weak reason to be adding a separate gpu > > > controller especially given that it doesn't really seem to be one with > > > proper abstractions for gpu resources. We don't want to keep adding random > > > keys to misc controller but can definitely add limited flexibility. What > > > kind of keys do you need? > > > > > Well the dmabuf-from-heaps component of this is the initial use case. > > I was envisioning we'd have additional keys as discussed here: > > https://lore.kernel.org/lkml/20220328035951.1817417-1-tjmercier@google.com/T/#m82e5fe9d8674bb60160701e52dae4356fea2ddfa > > So we'd end up with a well-defined core set of keys like "system", and > > then drivers would be free to use their own keys for their own unique > > purposes which could be complementary or orthogonal to the core set. > > Yesterday I was talking with someone who is interested in limiting gpu > > cores and bus IDs in addition to gpu memory. How to define core keys > > is the part where it looks like there's trouble. > > > > For my use case it would be sufficient to have current and maximum > > values for an arbitrary number of keys - one per heap. So the only > > part missing from the misc controller (for my use case) is the ability > > to register a new key at runtime as heaps are added. Instead of > > keeping track of resources with enum misc_res_type, requesting a > > resource handle/ID from the misc controller at runtime is what I think > > would be required instead. > > > Quick update: I'm going to make an attempt to modify the misc > controller to support a limited amount of dynamic resource > registration/tracking in place of the new controller in this series. > > Thanks everyone for the feedback. Somehow I missed this entire chain here. I'm not a fan, because I'm kinda hoping we could finally unify gpu memory account. Atm everyone just adds their one-off solution in a random corner: - total tracking in misc cgroup controller - dma-buf sysfs files (except apparently too slow so it'll get deleted again) - random other stuff on open device files os OOM killer can see it This doesn't look good. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch