From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4C3F21CA0D for ; Mon, 20 Jul 2026 22:16:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784585786; cv=none; b=HVxhzOUl33kWSM1tjtXK7Vf2Ntc5G4G+Op5pl+wnjcUQEDJF9OjSyMUxQKMypsKRnIylY/P3euziVbpdH7QcA6Kxok3zS28bzsmQqM77FWqVHmfepSnJrfqTLHx8EyO84b/t5YVFtUYn+PkadfSSnLaMeF2GnCmyZYOiE2g0AOs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784585786; c=relaxed/simple; bh=EC8yNHjTDXyuwDYD6Gak8bNS/hg5PwjNliAg5IrJPg8=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=jvdENEH8jsshK5h1Rz71onNjmN/EKK2YaWsmSlCDjd2w4Bxa/kDgZraB2xbhFlBmjnTqiZYRjZPpuWjJ98jlLfw9VUakZy1dI53BBuwcswIf28SS9ZRziJqp/fCXEcgXDvwu3KwKyRk/cUER6ALrd/UNYM2VXS6dyUShWhYlrwU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bCDho1KK; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=hRqUAgZD; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bCDho1KK"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="hRqUAgZD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1784585783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EC8yNHjTDXyuwDYD6Gak8bNS/hg5PwjNliAg5IrJPg8=; b=bCDho1KKfUEBEpxTjBj7iYNsX/TtUhbPZwhRXN6h5hHVujx4I0d25wlKDg5SQAReYxF1El 3HZLf93Q1SpD0bLwJ+mq8X4ks/DN6yw8ASbEQr8f4fz+EX97C05NZ5tF3NpRHEEUSVJJFg VHaqG4zrzXLAzMDbcc2OveOvJlJQrfc= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-696-lfWuLVyQPlSwCbwt0MX3Kg-1; Mon, 20 Jul 2026 18:16:21 -0400 X-MC-Unique: lfWuLVyQPlSwCbwt0MX3Kg-1 X-Mimecast-MFC-AGG-ID: lfWuLVyQPlSwCbwt0MX3Kg_1784585781 Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-92e538afe65so1720758885a.3 for ; Mon, 20 Jul 2026 15:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1784585781; x=1785190581; darn=vger.kernel.org; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:from:to :cc:subject:date:message-id:reply-to:content-type; bh=EC8yNHjTDXyuwDYD6Gak8bNS/hg5PwjNliAg5IrJPg8=; b=hRqUAgZDB4xIU0JgCumyws++3KgAtcfR6GD9rJy8DjNNKScrBCbSz+oIgyVeqxE8CE uKVmZmNz82PbtOYu8ui8Xk37Mzs0Hws+HlmKFTGEL8wZK7CtQjzVxO9UYjWq/qh3g1dU 4/zcEg/HjC4e1QzIFQfwF5kjkfGoM1Try1HvP8LqpA08Uj9PyshJ4kiFBR2kjKeaL7GO oUHrUkkgc2AGdTB40y79tLr5PtaEkG6ydJN9OXJvTQzdY8sjGnL/ZhNfEHXisuCN0sKL 6eHkYU2Q6AWMZcnf/cYurBq5S+cfpgLQHr1n0SAdDJz4fRi6W0Ya/upTVPucQ4192Hhj gMsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784585781; x=1785190581; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=EC8yNHjTDXyuwDYD6Gak8bNS/hg5PwjNliAg5IrJPg8=; b=WOnI2B+xrkbFhVkzitCRJ9RJlBaNMkSqkIu830Q5GJ9jWFegSFTamJ36tfHt9TxCx6 YynzA1uvyhO8MHI7jVhDNx4YX1e/GQiMCgzE4mH2IgCDbcgo4vhxUt+fhB/du0XzL4Nr JpRrlmffm/Hz94VD0Z2CpwaiCp+j0beQSi+75wlNlqQsyqw32TUV+TGGJDOR+1P3j+YS mI2SwwEw1oWr84IozgSRW49DDVoS2HvjidcepIhIb2sswrCLj8PqXRj5ZAnxA3PtgW5w J0nmVfUgNu7FqO/SM87Ot/BSSSDNdNAIxRAs/BiY2+X3qNwHaPvTqqhiQ8Gmp/EL10RR nVRg== X-Forwarded-Encrypted: i=1; AHgh+RrCyxOSsZXquJJbk2aQMlbYwATK9GyDEyqwQEOzS7HwMIX3tzOhnlt2uKRZQBJcUXl8Qt99W6OweD+Eq3g=@vger.kernel.org X-Gm-Message-State: AOJu0YxXbMgkYges5m9oXCYthLBM09Pm5/WhFzTHKGOataRxOCOAJtrl rEIEakOjaHdtMBx6ryJu6QTGyspMJl/2iOTk/ayEO++frwzvDKAmsx+aYrSXs68FM2LKxV6PvEU tfkA96ecI7A4SvvlWHAIAedBTXkGlZdkHqnpy/W8NOUIGL7eqfFxGjDAzF6+V2hn3Q+Xnq71BLQ == X-Gm-Gg: AfdE7clQufGnZbY0sLo94QFAQwI1CQaO2J2ztvCv0CCk6QgdDwxyN13eODmZ8LzUtZi zPTXP/kz6ZwXHxxRdWzxXY70ekSPtFFmuxB2tkFmTVkpK+U1BJ+wtsPXNSXidzYNgxU+D/Lq/Xo thzbBLMxhipFUAADkogwJcc9mdIo4t6ql2A2huUu/PjuR6UsxY3jc96CPSeyw1R0V+db5KJOn4v Pz0niCF+zU4cqzx1MBDn4vcK8ubaIwtgE7gHL54Y7lp3jHIyoZfVNaoILwHMkOicS3ge9Q+Y9Er tXDJ2ZTMaL2E+UQNtd0nipGjzn1ILsaVU51afLq4/ULxSVNu3+5gBSLEEDFzdDcsO6phjEe0 X-Received: by 2002:a05:620a:708a:b0:92e:fd2c:6a21 with SMTP id af79cd13be357-930b3ed4e6bmr1576935485a.27.1784585780739; Mon, 20 Jul 2026 15:16:20 -0700 (PDT) X-Received: by 2002:a05:620a:708a:b0:92e:fd2c:6a21 with SMTP id af79cd13be357-930b3ed4e6bmr1576932885a.27.1784585780256; Mon, 20 Jul 2026 15:16:20 -0700 (PDT) Received: from [192.168.8.4] ([100.0.180.93]) by smtp.gmail.com with ESMTPSA id af79cd13be357-930b52fa8dfsm970105485a.11.2026.07.20.15.16.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jul 2026 15:16:19 -0700 (PDT) Message-ID: <147d6760338f314016bf4f5c0b33b5c5d08d9d3a.camel@redhat.com> Subject: Re: [PATCH 0/6] drm/nouveau: GSP telemetry via RUSD, and fdinfo telemetry exposure From: lyude@redhat.com To: Mohamed Ahmed , linux-kernel@vger.kernel.org Cc: dri-devel@lists.freedesktop.org, Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Mary Guillemard , Milos Tijanic , nouveau@lists.freedesktop.org Date: Mon, 20 Jul 2026 18:16:18 -0400 In-Reply-To: <20260714211429.238355-1-mohamedahmedegypt2001@gmail.com> References: <20260714211429.238355-1-mohamedahmedegypt2001@gmail.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.3 (3.58.3-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (just for other nouveau maintainers, e.g. danilo, I'm currently planning on looking at this during this week. Already started reviewing some of this, slightly underestimated how much I had to review :) On Wed, 2026-07-15 at 01:14 +0400, Mohamed Ahmed wrote: > On the GSP firmware, nouveau reports nothing about the GPU. The GSP > owns > the sensors, so the legacy nvkm therm/clk/volt hwmon paths never run, > and > monitoring tools have no data to read. >=20 > GSP on later firmware does publish all of it through a stable > interface. > It maintains a shared-memory region called "RM User Shared Data" > (RUSD), > which the firmware fills with all the relevant sensor readings like > power, > temperatures, clocks, and many others. Consuming it needs only two > internal subdevice controls: INIT_USER_SHARED_DATA to register the > buffer, > and SET_DATA_POLL to tell the GSP which data groups to poll and how > often. >=20 > This series makes nouveau a RUSD client and exposes the data through > two > main surfaces: >=20 > =C2=A0 - hwmon, for the values that map onto standard channels: GPU and > memory > =C2=A0=C2=A0=C2=A0 temperature, GPU/board/VRAM power, power cap.=20 >=20 > =C2=A0 - sysfs, under /sys/class/drm/cardN/device/rusd/, for everything > hwmon > =C2=A0=C2=A0=C2=A0 has no channel type for: clocks, utilisation, pstate, = throttle > reason, > =C2=A0=C2=A0=C2=A0 ECC counters, PCIe error counters, and everything expo= sed to > hwmon as > =C2=A0=C2=A0=C2=A0 well. One value per file according to sysfs convention= . >=20 > Polling is demand-driven. GSP polls a group only while a client is > reading > it, and a group idles off after 10s after the last read. Polling > costs > power, so nothing is polled unless someone is looking. A section that > has > not been polled yet, or that the board does not support, reads - > ENODATA. >=20 > The sysfs interface is documented in > Documentation/ABI/testing/sysfs-driver-nouveau. >=20 > While the original scope was RUSD support only, monitoring tools > needed > two extra additions to complete the picture: >=20 > =C2=A0 - VRAM size and usage on sysfs (patch 5): since this series adds a > sysfs > =C2=A0=C2=A0=C2=A0 interface for nouveau, it's better to expose VRAM metr= ics there > rather > =C2=A0=C2=A0=C2=A0 than force monitoring apps to pull in libdrm for the G= ETPARAM > memory > =C2=A0=C2=A0=C2=A0 ioctls. >=20 > =C2=A0 - fdinfo (patch 6): nouveau implemented none at all, and monitorin= g > apps > =C2=A0=C2=A0=C2=A0 could not show per-process GPU and memory usage. It no= w reports > =C2=A0=C2=A0=C2=A0 per-client memory and per-engine busy time, accounted = from the > DRM > =C2=A0=C2=A0=C2=A0 scheduler that the EXEC/VM_BIND path already uses. >=20 > Limitations, caveats, and additional considerations or questions: >=20 > =C2=A0 - RUSD needs r570 or newer GSP firmware; r535 does not have the > needed > =C2=A0=C2=A0=C2=A0 controls. Pre-GSP and r535 behavior is identical to wh= at it was: > the > =C2=A0=C2=A0=C2=A0 rusd/ group is not created, and hwmon keeps its curren= t behavior. >=20 > =C2=A0 - This is all read-only telemetry. Clock, voltage, or power contro= l > is > =C2=A0=C2=A0=C2=A0 through other interfaces out of scope of this series. >=20 > =C2=A0 - The sysfs interface exposed here should be stable and valid for > nova > =C2=A0=C2=A0=C2=A0 as well. RUSD is stable and only grows more fields wit= h newer GSP > =C2=A0=C2=A0=C2=A0 firmware (e.g., fan telemetry gets added in r580), so = it is > possible > =C2=A0=C2=A0=C2=A0 to keep the same interface as we use newer firmware or= as we move > to > =C2=A0=C2=A0=C2=A0 nova and any potential considerations should be ironed= out before > this > =C2=A0=C2=A0=C2=A0 gets merged and the interface gets frozen. >=20 > =C2=A0 - amdgpu offers a single binary blob with all the telemetry > embedded in > =C2=A0=C2=A0=C2=A0 addition to the sysfs sensor files. This series curren= tly only > exposes > =C2=A0=C2=A0=C2=A0 sensor files, but it is possible to add a similar bina= ry for a > v2. > =C2=A0=C2=A0=C2=A0 This mainly offers a more efficient way to read the te= lemetry as > =C2=A0=C2=A0=C2=A0 clients would read only a few files instead of reading= all of > them. >=20 > =C2=A0 - The last two patches don't depend on RUSD and work on every GPU. > But > =C2=A0=C2=A0=C2=A0 they are here because they complete the sensor telemet= ry story. I > am > =C2=A0=C2=A0=C2=A0 happy to split them into their own series if that is p= referred. >=20 > Userspace WIP work built on this sysfs surface exists in form of > nouveau > backends for nvtop, LACT, and a NVK HUD: >=20 > =C2=A0 - nvtop: https://github.com/mohamexiety/nvtop/tree/nvtop-nouvtop >=20 > =C2=A0 - LACT: https://github.com/mohamexiety/LACT/tree/lnouvct >=20 > =C2=A0 - NVK HUD: > =C2=A0=C2=A0=C2=A0 https://gitlab.freedesktop.org/mesa/mesa/-/merge_reque= sts/39898 >=20 > None of these are merged yet and exist for testing only. Upstreaming > will > begin when this series gets merged and the interface is stable. >=20 > Mohamed Ahmed (6): > =C2=A0 drm/nouveau/gsp: vendor the RUSD header > =C2=A0 drm/nouveau/gsp: add RUSD telemetry support > =C2=A0 drm/nouveau: add GSP hwmon support > =C2=A0 drm/nouveau: expose RUSD telemetry via sysfs > =C2=A0 drm/nouveau: expose global VRAM size and usage via sysfs > =C2=A0 drm/nouveau: expose per-client GPU usage via fdinfo >=20 > =C2=A0.../ABI/testing/sysfs-driver-nouveau=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 | 167 ++++++ > =C2=A0MAINTAINERS=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= |=C2=A0=C2=A0 1 + > =C2=A0drivers/gpu/drm/nouveau/Kbuild=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 1 + > =C2=A0.../gpu/drm/nouveau/include/nvkm/subdev/gsp.h |=C2=A0 33 ++ > =C2=A0.../drm/nouveau/include/nvkm/subdev/rusd.h=C2=A0=C2=A0=C2=A0 | 114 = ++++ > =C2=A0.../gpu/drm/nouveau/include/nvrm/nvtypes.h=C2=A0=C2=A0=C2=A0 |=C2= =A0=C2=A0 3 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_abi16.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 16 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_chan.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_drm.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 22 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_drv.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 29 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_exec.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 |=C2=A0=C2=A0 3 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_gem.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 16 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_hwmon.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 375 ++++++++++--- > =C2=A0drivers/gpu/drm/nouveau/nouveau_sched.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 87 +++ > =C2=A0drivers/gpu/drm/nouveau/nouveau_sched.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 11 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_sysfs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 311 ++++++++++ > =C2=A0drivers/gpu/drm/nouveau/nouveau_sysfs.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 10 + > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c |=C2=A0 26 + > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r535/rm.c=C2=A0 |=C2=A0=C2=A0 1 = + > =C2=A0.../nouveau/nvkm/subdev/gsp/rm/r570/Kbuild=C2=A0=C2=A0=C2=A0 |=C2= =A0=C2=A0 1 + > =C2=A0.../nvkm/subdev/gsp/rm/r570/nvrm/rusd.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 303 ++++++++++ > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r570/rm.c=C2=A0 |=C2=A0=C2=A0 1 = + > =C2=A0.../nouveau/nvkm/subdev/gsp/rm/r570/rusd.c=C2=A0=C2=A0=C2=A0 | 531 > ++++++++++++++++++ > =C2=A0.../gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h=C2=A0=C2=A0 |=C2=A0=C2= =A0 7 + > =C2=A024 files changed, 1990 insertions(+), 81 deletions(-) > =C2=A0create mode 100644 Documentation/ABI/testing/sysfs-driver-nouveau > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/include/nvkm/subdev/rusd.h > =C2=A0create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.c > =C2=A0create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.h > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/rusd.h > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/rusd.c