From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC16333A9FC for ; Tue, 21 Jul 2026 20:40:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784666450; cv=none; b=WuSnbxoLITH6iJHYVTe/pw/C+pxZp3/yboHUjH47VUbUNEH6WwVH4u9A4ueLnrCHXGrbnh0BNAD0UNMDDfnZUX16WVwoOHh2bl2qyJNDYVyi3NaSz01MPhME8/pmYsmQi7abosVpJci+GBQr4jrT9hWF1qc9kgZ3yAdqynTNZWI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784666450; c=relaxed/simple; bh=gXgX0uNO+WJdsVZ7unE8kmr92+eVG9rapgaFoCHjavU=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=AkmXSzoWnH9unkEZJbo+CAXrztU5Vp3XiUjNo6LaEp7A91l3YScO941EaQ+d0chI1gyZbr1uHeyy24ifSnng85pSwTC6AL9SdRi52ez6YbZ8Cp9ybEFA8FBXiJPVeTYzh7jRQESXEYOOuLsAP7PkJSrnUBUrt0iyfUXogiqH9Mk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=S7O14MBc; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=tHIBbw7H; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="S7O14MBc"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="tHIBbw7H" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1784666447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gXgX0uNO+WJdsVZ7unE8kmr92+eVG9rapgaFoCHjavU=; b=S7O14MBcG8LGyCswUYbEJPMBAmGSc4Tvc5U/74eyNRO3vcEEUDfPWuM7LzbqIneDy/bJSV jm4aYnfxU7WsPWu6feFcy+Fxmuhg5kAh4HjSXjrAgWZrog5ngEuO8bcFGuKIqa43Wio7gX Y73nGq2kOQ2rDMptpteQoD2jqnYpVqA= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-591-0Pa8OZ_eN6KlR3Ugkdexcg-1; Tue, 21 Jul 2026 16:40:46 -0400 X-MC-Unique: 0Pa8OZ_eN6KlR3Ugkdexcg-1 X-Mimecast-MFC-AGG-ID: 0Pa8OZ_eN6KlR3Ugkdexcg_1784666446 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-51c1d137a68so319392981cf.3 for ; Tue, 21 Jul 2026 13:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1784666446; x=1785271246; darn=vger.kernel.org; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:from:to :cc:subject:date:message-id:reply-to:content-type; bh=gXgX0uNO+WJdsVZ7unE8kmr92+eVG9rapgaFoCHjavU=; b=tHIBbw7HvzlMQwtOHK9KliasG2dSeUgGcPpNdHX6jJ/I+uRlN/2M1GCjc89SeKAlra eEYPEvJd3xMXZ3sCF6KVSf6aQ/7T/mNVKMmlJZuQX7PhQ0ZRAsTJ1KbX8VBH0jexGVWY kQae+63gY90IMZX+/ArVanGNfY8nrSBX8hFOcypNSsoDNVs8WPD03aw5z+dPOt53q4Qa hugi/KtbNY2r6q5SmNVbL9L26N8Yi5NXSP+YtLax1naizW6XCi/l8UWtDYsKpjvE8ceT H4B1tC/eqJ3E8WQ5n7fP9AnzkDezZ8/IOOGu/5DEnqRJKv5lXyD+WMmUhcYSNXWPo2X+ PrSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784666446; x=1785271246; h=mime-version:user-agent:content-transfer-encoding:content-type :references:in-reply-to:date:cc:to:from:subject:message-id:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=gXgX0uNO+WJdsVZ7unE8kmr92+eVG9rapgaFoCHjavU=; b=OJjTL/8i/siE6qSDfSmHn6XVWxC3DmNfXbnLG9lnLIVF5kZZYoxa+6Zk6N5q+dq9UH GSxH+Plz4Oinh3VwgHp5txbrWRSisHX8duJxYm45V3/WVGYcNyQQrM8YJ/Pnww0aXDlT Tda3dVD8yOm/nqfBjwhCLSVFPTkqAJz7KYmB+cbH37WrrXXuvCwfJM8R5Js7mQ7+gVxg wXG5zrdw+bIid8twKbevRJUcXwlaEkLA9NVKQrH9exP4SLPIBwvwzEA6cYATkAMbmF8r INc1HfR93Y3pfJaUP2gYcfx16Dc/OnWLLmTwTECEq2M2rrwqygH/hE919mmAkBxJm5uS H0xA== X-Forwarded-Encrypted: i=1; AHgh+RpbwSLQb1FcVYnd7xpQPIDN54cGWEynBmK/gs7wiZn1TOtFiCkpujrmyn5CZOc4pPv/T6MkWurp5r4q10Q=@vger.kernel.org X-Gm-Message-State: AOJu0Yz0Ys6runSbmkDEbUKkBOuqICAK86TchWRb48dvui8GjsAMICCY lOPye1ofNbK2cOt5IyMr0+manXqvRvTvZ3L86kTMKqNhe2ErZgh3PlMTzllD02rEvWCwtrEehPf XZe0EJKN9T/uv9IXDLp7kZSpc/MrAbT9pj9gAVtbtUINEfeZsI1AXMewa3kk0BM17fA== X-Gm-Gg: AfdE7cnUrsN7pm9LrQVSOv29qW2m5dzApkPpr6+gwkx6pIhBUA9LApPO1cud0AyBQ0A na1zZuawzolULslmJ5DjmWDfY1I+aN+7lB5AS7e5r7jVRQRZKtSPgCdaVA/7svcQhNJT9RBeWgj McVEublUKvcaZw7HD3cmuXiH1xOzqFvWoG8DirnLS/PX5pCOxdtKJiL+WhLQXU6jbg2Qw2gEUYD wjAwRSLAhG1Hsf2jChsAJzEyX7nrcTOB5ma6Fz3vw6SLmtlcE03NX+Frcwl6nEas2SsmRUtYOOP jn0NtEBC0HtKZhIqe4jomv0oTVQrdnw2QtTNZcyFYCVS0YhXdrT+Fl4ZUdY1cHnUpJsbiprF X-Received: by 2002:a05:622a:1e11:b0:51c:88b:d7e3 with SMTP id d75a77b69052e-5213c8e3864mr187389071cf.3.1784666445654; Tue, 21 Jul 2026 13:40:45 -0700 (PDT) X-Received: by 2002:a05:622a:1e11:b0:51c:88b:d7e3 with SMTP id d75a77b69052e-5213c8e3864mr187388891cf.3.1784666445181; Tue, 21 Jul 2026 13:40:45 -0700 (PDT) Received: from [192.168.8.4] ([100.0.180.93]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-907ba9e19f6sm4502016d6.27.2026.07.21.13.40.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 13:40:44 -0700 (PDT) Message-ID: Subject: Re: [PATCH 0/6] drm/nouveau: GSP telemetry via RUSD, and fdinfo telemetry exposure From: lyude@redhat.com To: Mohamed Ahmed , linux-kernel@vger.kernel.org Cc: dri-devel@lists.freedesktop.org, Danilo Krummrich , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Mary Guillemard , Milos Tijanic , nouveau@lists.freedesktop.org Date: Tue, 21 Jul 2026 16:40:43 -0400 In-Reply-To: <20260714211429.238355-1-mohamedahmedegypt2001@gmail.com> References: <20260714211429.238355-1-mohamedahmedegypt2001@gmail.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.3 (3.58.3-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With the various nits addressed or responded to, for the whole series: Reviewed-by: Lyude Paul (I assume with you moving stuff into nvkm I might need to re-review later, but that's fine with me) On Wed, 2026-07-15 at 01:14 +0400, Mohamed Ahmed wrote: > On the GSP firmware, nouveau reports nothing about the GPU. The GSP > owns > the sensors, so the legacy nvkm therm/clk/volt hwmon paths never run, > and > monitoring tools have no data to read. >=20 > GSP on later firmware does publish all of it through a stable > interface. > It maintains a shared-memory region called "RM User Shared Data" > (RUSD), > which the firmware fills with all the relevant sensor readings like > power, > temperatures, clocks, and many others. Consuming it needs only two > internal subdevice controls: INIT_USER_SHARED_DATA to register the > buffer, > and SET_DATA_POLL to tell the GSP which data groups to poll and how > often. >=20 > This series makes nouveau a RUSD client and exposes the data through > two > main surfaces: >=20 > =C2=A0 - hwmon, for the values that map onto standard channels: GPU and > memory > =C2=A0=C2=A0=C2=A0 temperature, GPU/board/VRAM power, power cap.=20 >=20 > =C2=A0 - sysfs, under /sys/class/drm/cardN/device/rusd/, for everything > hwmon > =C2=A0=C2=A0=C2=A0 has no channel type for: clocks, utilisation, pstate, = throttle > reason, > =C2=A0=C2=A0=C2=A0 ECC counters, PCIe error counters, and everything expo= sed to > hwmon as > =C2=A0=C2=A0=C2=A0 well. One value per file according to sysfs convention= . >=20 > Polling is demand-driven. GSP polls a group only while a client is > reading > it, and a group idles off after 10s after the last read. Polling > costs > power, so nothing is polled unless someone is looking. A section that > has > not been polled yet, or that the board does not support, reads - > ENODATA. >=20 > The sysfs interface is documented in > Documentation/ABI/testing/sysfs-driver-nouveau. >=20 > While the original scope was RUSD support only, monitoring tools > needed > two extra additions to complete the picture: >=20 > =C2=A0 - VRAM size and usage on sysfs (patch 5): since this series adds a > sysfs > =C2=A0=C2=A0=C2=A0 interface for nouveau, it's better to expose VRAM metr= ics there > rather > =C2=A0=C2=A0=C2=A0 than force monitoring apps to pull in libdrm for the G= ETPARAM > memory > =C2=A0=C2=A0=C2=A0 ioctls. >=20 > =C2=A0 - fdinfo (patch 6): nouveau implemented none at all, and monitorin= g > apps > =C2=A0=C2=A0=C2=A0 could not show per-process GPU and memory usage. It no= w reports > =C2=A0=C2=A0=C2=A0 per-client memory and per-engine busy time, accounted = from the > DRM > =C2=A0=C2=A0=C2=A0 scheduler that the EXEC/VM_BIND path already uses. >=20 > Limitations, caveats, and additional considerations or questions: >=20 > =C2=A0 - RUSD needs r570 or newer GSP firmware; r535 does not have the > needed > =C2=A0=C2=A0=C2=A0 controls. Pre-GSP and r535 behavior is identical to wh= at it was: > the > =C2=A0=C2=A0=C2=A0 rusd/ group is not created, and hwmon keeps its curren= t behavior. >=20 > =C2=A0 - This is all read-only telemetry. Clock, voltage, or power contro= l > is > =C2=A0=C2=A0=C2=A0 through other interfaces out of scope of this series. >=20 > =C2=A0 - The sysfs interface exposed here should be stable and valid for > nova > =C2=A0=C2=A0=C2=A0 as well. RUSD is stable and only grows more fields wit= h newer GSP > =C2=A0=C2=A0=C2=A0 firmware (e.g., fan telemetry gets added in r580), so = it is > possible > =C2=A0=C2=A0=C2=A0 to keep the same interface as we use newer firmware or= as we move > to > =C2=A0=C2=A0=C2=A0 nova and any potential considerations should be ironed= out before > this > =C2=A0=C2=A0=C2=A0 gets merged and the interface gets frozen. >=20 > =C2=A0 - amdgpu offers a single binary blob with all the telemetry > embedded in > =C2=A0=C2=A0=C2=A0 addition to the sysfs sensor files. This series curren= tly only > exposes > =C2=A0=C2=A0=C2=A0 sensor files, but it is possible to add a similar bina= ry for a > v2. > =C2=A0=C2=A0=C2=A0 This mainly offers a more efficient way to read the te= lemetry as > =C2=A0=C2=A0=C2=A0 clients would read only a few files instead of reading= all of > them. >=20 > =C2=A0 - The last two patches don't depend on RUSD and work on every GPU. > But > =C2=A0=C2=A0=C2=A0 they are here because they complete the sensor telemet= ry story. I > am > =C2=A0=C2=A0=C2=A0 happy to split them into their own series if that is p= referred. >=20 > Userspace WIP work built on this sysfs surface exists in form of > nouveau > backends for nvtop, LACT, and a NVK HUD: >=20 > =C2=A0 - nvtop: https://github.com/mohamexiety/nvtop/tree/nvtop-nouvtop >=20 > =C2=A0 - LACT: https://github.com/mohamexiety/LACT/tree/lnouvct >=20 > =C2=A0 - NVK HUD: > =C2=A0=C2=A0=C2=A0 https://gitlab.freedesktop.org/mesa/mesa/-/merge_reque= sts/39898 >=20 > None of these are merged yet and exist for testing only. Upstreaming > will > begin when this series gets merged and the interface is stable. >=20 > Mohamed Ahmed (6): > =C2=A0 drm/nouveau/gsp: vendor the RUSD header > =C2=A0 drm/nouveau/gsp: add RUSD telemetry support > =C2=A0 drm/nouveau: add GSP hwmon support > =C2=A0 drm/nouveau: expose RUSD telemetry via sysfs > =C2=A0 drm/nouveau: expose global VRAM size and usage via sysfs > =C2=A0 drm/nouveau: expose per-client GPU usage via fdinfo >=20 > =C2=A0.../ABI/testing/sysfs-driver-nouveau=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 | 167 ++++++ > =C2=A0MAINTAINERS=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= |=C2=A0=C2=A0 1 + > =C2=A0drivers/gpu/drm/nouveau/Kbuild=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 1 + > =C2=A0.../gpu/drm/nouveau/include/nvkm/subdev/gsp.h |=C2=A0 33 ++ > =C2=A0.../drm/nouveau/include/nvkm/subdev/rusd.h=C2=A0=C2=A0=C2=A0 | 114 = ++++ > =C2=A0.../gpu/drm/nouveau/include/nvrm/nvtypes.h=C2=A0=C2=A0=C2=A0 |=C2= =A0=C2=A0 3 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_abi16.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 16 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_chan.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 |=C2=A0=C2=A0 2 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_drm.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 22 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_drv.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 29 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_exec.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 |=C2=A0=C2=A0 3 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_gem.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 |=C2=A0 16 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_hwmon.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 375 ++++++++++--- > =C2=A0drivers/gpu/drm/nouveau/nouveau_sched.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 87 +++ > =C2=A0drivers/gpu/drm/nouveau/nouveau_sched.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 11 + > =C2=A0drivers/gpu/drm/nouveau/nouveau_sysfs.c=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 311 ++++++++++ > =C2=A0drivers/gpu/drm/nouveau/nouveau_sysfs.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 |=C2=A0 10 + > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r535/gsp.c |=C2=A0 26 + > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r535/rm.c=C2=A0 |=C2=A0=C2=A0 1 = + > =C2=A0.../nouveau/nvkm/subdev/gsp/rm/r570/Kbuild=C2=A0=C2=A0=C2=A0 |=C2= =A0=C2=A0 1 + > =C2=A0.../nvkm/subdev/gsp/rm/r570/nvrm/rusd.h=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 | 303 ++++++++++ > =C2=A0.../drm/nouveau/nvkm/subdev/gsp/rm/r570/rm.c=C2=A0 |=C2=A0=C2=A0 1 = + > =C2=A0.../nouveau/nvkm/subdev/gsp/rm/r570/rusd.c=C2=A0=C2=A0=C2=A0 | 531 > ++++++++++++++++++ > =C2=A0.../gpu/drm/nouveau/nvkm/subdev/gsp/rm/rm.h=C2=A0=C2=A0 |=C2=A0=C2= =A0 7 + > =C2=A024 files changed, 1990 insertions(+), 81 deletions(-) > =C2=A0create mode 100644 Documentation/ABI/testing/sysfs-driver-nouveau > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/include/nvkm/subdev/rusd.h > =C2=A0create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.c > =C2=A0create mode 100644 drivers/gpu/drm/nouveau/nouveau_sysfs.h > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/nvrm/rusd.h > =C2=A0create mode 100644 > drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r570/rusd.c