From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44C4140DB5C for ; Thu, 20 Aug 2026 10:57:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787223444; cv=none; b=bbuTxAPxUuJkxXr0zwEWX3whIXgCweLB80gqPVpwlhmcBn+v5lgWZld5z0Xt6RJUpXXeoSuCE8223TT9JK5RPtx1A4UZuXrZe9rUwxLfTP4b7Z03vOgttqdXBLJLJjObtSOorYmtAcgXNoRgu304wUPCh23Fr+o/gm5EKkQzejA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787223444; c=relaxed/simple; bh=FicelAfK0ZLqsFbS2etTPyd7dQW3jGUKt+iuxGkuS/s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KK8pr0+dOnIeNNSPt2E2vEdMFpxmgF5b9G5FioTigNaRV8T1F7AyX+26VN277hpdBRrgqkTNuZc0tvD0Kdoa7p41rFr9BNFmj598rlmsDC1M2bpvYzr3A5tXqhU98ZC45H9VH8YPWsvHosp3mf6W4xnzokDeOEQAlBUykYG6pL4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O+8Abr0A; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O+8Abr0A" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-47db714766aso619979f8f.0 for ; Thu, 20 Aug 2026 03:57:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787223441; x=1787828241; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=h/HhSZtTG2M/MhbMK/mYTVAf7lwkJQIPzztoCbrat6E=; b=O+8Abr0AbGVgbJU7XuGuEKnzXH5XO7yiqZcZYOD0evnzT7O/oHfin3tJdcQwEVz1bu YuLGUrbnl/nqgz+BNRGkw8dErWz0cICXnmFEXBsUmEiAj5uyOL35bRJda4mefP/QQ6eJ Vy+cjPgt4iQqHyiSBK/qf4ZbZogq0cLyp1eNmUB6gyOQuYZHgzDWsoxUceg46OU8SoFF Iyeaxxy/fdOcgXck2mxY4Z7j7XsUlnU0zV3Zq2qICKO9fmNQZeI1jaGRQq2rOvvaWGKa 8F2BWxSAbXqnVpM9XG3RDtNZjtZbZ0+ns0rGCyGy5aPanH27mImmjys+7O5knNCSAtDZ zAsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787223441; x=1787828241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=h/HhSZtTG2M/MhbMK/mYTVAf7lwkJQIPzztoCbrat6E=; b=LcfyqDhgPWTKRFzE35u4thz0FlSQxokHr6WmTcVnkRju3yr73+F0fwpn35spRrzgbS gWju1PoQNRbhWZ+yH+GszXfYbMM79BDDHH49TnMadbnI6L/5Wo2x032nz9jcGKPnuIS1 dQaIPfARaWxzZbw7gh51w9XBAMnzBawdRfhq4o7oYJGu+SAGDo5cFJxryrIfSszSQOr1 PxzeKF///dWI4ZZdAGT8VV/A3BpI/03AGL9RDXQXf0NO7iF/irbCerOXkDQwy8BL8R4k z53jUFCXl9lZ+DdoWAd5a4323iskjpxlibhY90tJz0RxYGI0GU08Fo8hjIXO0rshnBqb mjhw== X-Forwarded-Encrypted: i=1; AHgh+RqBEyXxR0RrvpcviUcj6BRFtn9rZmfIq90qkxHWYWmoChVvpmyhjpEkx8Y0b2WqmWYDN+oMTP9E3d0cGOE=@vger.kernel.org X-Gm-Message-State: AOJu0Yw+WTW5TxoXFrxV+NIeHw2nFQCgPbQo7Jq1eNy7GMHHhDEBZN0e V210NzURnoaKoz6i2hWrg4mrMRD5Ddj8UNq7r85/V8C00bCN+xafbJY9 X-Gm-Gg: AR+sD13X4c4l3eDnYPOPqaFNDZVebCG+WmQJ25wVqmNowP4O8EI/Q2693tXylh2dKL7 yspnbNL1soQgahmv5GkP2aRAbGcazQj1oCoupQW9E36MEQp8QQ1elnCFoFY1m/jH7kwM5VYPtAz 7XiTnnUXJ2Mw8HyfmY0UCCopGefqidKKfwsBFQyHtLV110ryZ5FRn8PfyMOaRYz/GGAgVrircT2 6VXix8NDRrNv7v21xFBl2YieYLkWYcrSnRg/b/Gs74e1sReQPIePMkM+K5K9xTjbzkfgZJyzNOc cW/QcIvKF44vl++gw2gfI+Eq94sqDZZHGZlM7kg++TCZCcFNuho3TUSxoEr/K8NFNzjllnC4vEa RsqdDNgdhJOolfZXeOqn5jyqdY1H3EpF2duHFEZ4+ii994b71IjR5GxOF78dvBKvUH+T99Yz6nW 0vu6eSOIYtNsejSJBKZUMOgXjMjtdyn0aIOAFBTnpSvh93921YqStyPQq3yv/UWyYpkGnLTi7K1 fTOi6J3893PBAzoTijWdhJ9/2ubIHmPlmaH8GhC2Di0t1BBLBf9/L8yHfzxeaO1AVez8/LBKi8U O4E2WmssSgF0rgyl X-Received: by 2002:a05:600c:3487:b0:495:5d6d:9cc1 with SMTP id 5b1f17b1804b1-499b0633f1bmr82252225e9.0.1787223441255; Thu, 20 Aug 2026 03:57:21 -0700 (PDT) Received: from horsehead.lan (p548c9e2a.dip0.t-ipconnect.de. [84.140.158.42]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-499b4568a65sm13872775e9.2.2026.08.20.03.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Aug 2026 03:57:20 -0700 (PDT) From: Denis Pisarev To: amd-gfx@lists.freedesktop.org Cc: alexander.deucher@amd.com, christian.koenig@amd.com, mario.limonciello@amd.com, ionut_n2001@yahoo.com, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Denis Pisarev Subject: [RFC PATCH v2 0/1] drm/amdgpu: MMIO TLB invalidation fallback when KIQ is wedged after S4 resume Date: Thu, 20 Aug 2026 12:57:15 +0200 Message-ID: <20260820105716.148732-1-pisarevden@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260819185349.29407-1-pisarevden@gmail.com> References: <20260819185349.29407-1-pisarevden@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Hi, v2 of this RFC, addressing the automated sashiko-bot review of v1. The failure data and trigger isolation are unchanged from v1 (bugzilla 219492): - S4 resume on Cezanne (gmc_v9, GFXOFF) -> KIQ TLB flush reg failures at 80-140/hour for 9+ hours, sched.ready true throughout; each failure burns the ~5 s retry window and the invalidation is dropped - GFXOFF held off across S4: 0 errors (vs ~70-140/30 min stock); re-enabled 40 min after a clean resume: 0 errors in 15 min - so the wedge forms in the S4 resume window while GFXOFF is allowed Changes since v1 (all from the bot review, all reasonable): 1. [Critical] the runtime MMIO fallback now holds the GC block awake with amdgpu_gfx_off_ctrl(adev, false) across the direct register access - the unguarded v1 could touch power-gated registers. Because that call may sleep, the fallback is restricted to process context. 2. [High] failure counter moved from a global in amdgpu_gmc to the per-instance struct amdgpu_kiq - no cross-talk between XCCs. 3. [High] SR-IOV VFs never take the runtime MMIO fallback (they lack privileges for direct GMC invalidation register writes). 4. [Medium] gmc_v9_0_flush_gpu_tlb_mmio() (extracted helper) takes invalidate_lock with irqsave since the path is now runtime-reachable. 5. [High, pre-existing] the MES branch now propagates amdgpu_mes_reg_write_reg_wait()'s error instead of hardcoding success. 6. [Medium] threshold warning no longer claims a fallback that gmc_v10/v11/v12 do not implement. Open question kept from v1: a KIQ command that timed out remains queued in the ring; if the ring recovers late, a duplicate (idempotent) invalidation could race a CPU MMIO flush. Input on whether the req/ack handshake needs protection for that case is welcome. Alternative direction also still open: fix the S4 resume ordering itself (RLC/ME vs GFXOFF) instead of a runtime fallback - happy to run tracing on the affected hardware. Patch 1/1 follows. Denis Pisarev Denis Pisarev (1): drm/amdgpu: fall back to MMIO TLB invalidation when KIQ is unresponsive drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 18 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 2 +- drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 111 ++++++++++++++++++------ 5 files changed, 102 insertions(+), 33 deletions(-) -- 2.55.0