From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6B2C379EC8 for ; Fri, 19 Jun 2026 12:18:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.176 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781871512; cv=none; b=IaMMPEh6fhu4rl1nEjnN2773VW/Mgfp6WXC9tJBh52goOEIzj8c/jbAnkg2kZAUFqmKZKqHhsG92Bz5xIyGepWhqLcvGXQC3W0zvNRWlaSfcS+XZiSoLfz1vut8HZ4TOBlOegYUAYCSQK6FvbVavfvdmytYUan4CVKj4ITImET8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781871512; c=relaxed/simple; bh=9f+nn5nDDk9GSmx5JBKJvDEPRUwJmfjFEPUzdnNlAQE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MObaewj6pjhQj5kZn+YxYNfb8N2AjdNHa2YUSQ4jVY8fQOtri0NnrnGKRMevEskihElGMsMjP821FzirX3lj3Z563roQzAadQPDru1cqnkNC8t4OMV/l/gfbNKRGhOXvi/sWrcPZZmXZVE3ZZ7fLlfsLTz/Ah1PNlQGYOR4T0PQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YGRto7uT; arc=none smtp.client-ip=209.85.208.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YGRto7uT" Received: by mail-lj1-f176.google.com with SMTP id 38308e7fff4ca-3967738c801so21263861fa.2 for ; Fri, 19 Jun 2026 05:18:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781871507; x=1782476307; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dLZ+pCnD0P0yEjXHP7s+d0OIOgshDvs2k5lhyVXVghQ=; b=YGRto7uTPf8+MhQxS3lnyYdcUDL+oqnsrQq2o3M/FieiHzYM1QHF4vAtWMCs4cci8r URrMOMbyyKBAokxhj3whhbiZRtQpKvoTIDIFde3jeWhqT6xeyt05aaZ8dpiy18qfdxqP vD/+Q6w8oWxKL7H2T9aRLkho37c/KoK9a700mKdDKEBGu5J+E/bX6dTzJzpRmQIGamm0 8gxxhaiy/jR9+V77NKqRpmojPH1mRoeD14ANZ4QC14NjwsvZ3BGYbPXAn76Mqm4PuX8X OVXiwen7lAmJGwGlWWsPDq291TVVaHLCUTAEPswFdT3nIrP7vt/JibPMzH7S4nQbgATG lE1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781871507; x=1782476307; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=dLZ+pCnD0P0yEjXHP7s+d0OIOgshDvs2k5lhyVXVghQ=; b=Oyn78MnR67bFHmFzjEjVVq0NDZN9zQqrvGtM2StkBtjOlPcgkDqu/lsooFWAKe4bnL A+00rQWqcZEjiU3n/Hq3ZrfuMJtKNvHicRMp+8vJGOrJTcF6LrgXgSwE1psoZhgiSUVR xwhoXdFheTRkq8TtwtQAWXfNZogzXDPJ2sOtKRLcIWfKWnQVYWjWub8S2DUACGeG8VeL AzggRub2qfmo6TnmFnMVeQQnbBIAhR86T9apx2INJpPIYtuqc6xPZEpwfzHpEdqjB/NO MygDtv2LVdT05jPVuxdJ/Tn/vrPx4yOcay2ZH6spoNtIu0K2YxHdF9KHvbZkVX2nZMoq ZzBg== X-Forwarded-Encrypted: i=1; AFNElJ+C4vKDyF+ff7rOmqeDNe1dNfAAP7ZYZwI7MMRsbA0KQZWMP2rOM7CCzXA+BHzUeqAYN91+hOCp35YypIc=@vger.kernel.org X-Gm-Message-State: AOJu0YxRu/RuN/ALbOd7jOHkEp+LBwBL6c11IMCEHFYZQ0geQWafuZvW bBSCuTkpTsCxhPWWIhLKRTba9Qs/ULDDV7lXtRDppkBaOKZeng+vPbz4 X-Gm-Gg: AfdE7cnajZ0naBPuK5wKqc86SpuwNJ9kVx1NozxcdvG89oYyMywuKmSzkcnuxKEZ5Rt QcQ29r4QkQRWmie96Zv9GbR6OfLKxV5uVjYF4VQIqB/retj/bMUhACFPepqWQJ1c9WeV8IRnViV xNspLRPVejxG6iLBDEDBaYvXqoRgjrrpHEQKyhMTQ/kV8XpB1YOBdpG3B5loqwhIuiM2CMgzxkZ e2N6Cl8feGAKp5zQulwpnAT7WlpOlV5Yvy6HmISlVs3h+M3NYnIoeEh5uPYQ749SVo93TB9RBMm 3gh1tihpPXNNfn/+6cPN/7Vt2J1QXheHVkv6jWqZVFVT0t/ylIua4JGP1qAt5zRWaXNBKTMKSsb XeNX06XWZFcIrrWjUAgG+ynKNKow2QTkAFoc6SYqnzsIMzJPBvkrzWBmRvkiIAtl7wjU824Y/Pv MdT2o50t1hSudqURyTpvJsPA== X-Received: by 2002:a05:651c:211f:b0:399:1fbd:8f2f with SMTP id 38308e7fff4ca-3998bd06f64mr8439831fa.9.1781871506548; Fri, 19 Jun 2026 05:18:26 -0700 (PDT) Received: from localhost ([188.234.148.119]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-3998beca743sm4598331fa.5.2026.06.19.05.18.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Jun 2026 05:18:24 -0700 (PDT) From: Mikhail Gavrilov To: =?UTF-8?q?Christian=20K=C3=B6nig?= , Alex Deucher , Vitaly Prosyak Cc: Mikhail Gavrilov , David Airlie , Simona Vetter , amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] drm/amdgpu: do not enter fs_reclaim under notifier_lock in lockdep training Date: Fri, 19 Jun 2026 17:18:18 +0500 Message-ID: <20260619121819.189293-1-mikhail.v.gavrilov@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Makes sense, thanks. I won't respin this one then. Vitaly, for the reorder, here is a deterministic reproducer so you can confirm the splat on your side without a round-trip. It arms an mmu_interval_notifier via GEM_USERPTR over anonymous memory, then forces reclaim of that exact range with madvise(MADV_PAGEOUT), so amdgpu_hmm_invalidate_gfx() takes notifier_lock under fs_reclaim in the calling thread. Needs CONFIG_PROVE_LOCKING and a fresh boot; build/run notes are in the header. Happy to give Tested-by once you post. // SPDX-License-Identifier: MIT /* * amdgpu-notifier-reclaim-repro.c * * Deterministic reproducer for the false circular-locking-dependency splat * produced by drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c. * * amdgpu_lockdep_init() (run at module load) calls fs_reclaim_acquire() * while holding the dummy notifier_lock, teaching lockdep that it is legal * to enter reclaim with the MMU-notifier lock held. The real notifier lock * is taken in amdgpu_hmm_invalidate_gfx(), which mm/ calls from inside * reclaim, so the reverse edge fs_reclaim -> mmu_notifier -> notifier_lock * is mandatory. The cycle is closed the first time reclaim unmaps a page * covered by an amdgpu userptr interval notifier. * * This program installs such a notifier (GEM_USERPTR) over anonymous memory * and then forces synchronous reclaim of that exact range with * MADV_PAGEOUT, which runs try_to_unmap()->invalidate_range_start() with * fs_reclaim held in the calling thread, closing the loop on demand. * * Requirements: * - kernel built with CONFIG_PROVE_LOCKING (lockdep) * - amdgpu loaded; run from a FRESH boot (the first lockdep splat of any * kind calls debug_locks_off() and silences all later reports) * * Build: cc -O2 -o repro amdgpu-notifier-reclaim-repro.c * Run: ./repro # picks the first amdgpu render node * ./repro /dev/dri/renderD129 * Watch: sudo dmesg -w */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include /* --- minimal amdgpu uapi (self-contained, no libdrm needed) ----------- */ #ifndef DRM_IOCTL_BASE #define DRM_IOCTL_BASE 'd' #endif #define DRM_COMMAND_BASE 0x40 #define DRM_AMDGPU_GEM_USERPTR 0x11 struct drm_amdgpu_gem_userptr { uint64_t addr; uint64_t size; uint32_t flags; uint32_t handle; }; #define DRM_IOCTL_AMDGPU_GEM_USERPTR \ _IOWR(DRM_IOCTL_BASE, DRM_COMMAND_BASE + DRM_AMDGPU_GEM_USERPTR, \ struct drm_amdgpu_gem_userptr) #define AMDGPU_GEM_USERPTR_READONLY (1 << 0) #define AMDGPU_GEM_USERPTR_ANONONLY (1 << 1) #define AMDGPU_GEM_USERPTR_VALIDATE (1 << 2) #define AMDGPU_GEM_USERPTR_REGISTER (1 << 3) #ifndef MADV_PAGEOUT #define MADV_PAGEOUT 21 #endif /* ------------------------------------------------------------ */ #define BUF_SIZE (64ull * 1024 * 1024) /* 64 MiB, page aligned by mmap */ static int open_amdgpu_render(const char *forced) { if (forced) { int fd = open(forced, O_RDWR | O_CLOEXEC); if (fd < 0) perror(forced); return fd; } /* try renderD128..renderD143 and keep the first that accepts GEM_USERPTR */ for (int i = 128; i < 144; i++) { char path[64]; snprintf(path, sizeof(path), "/dev/dri/renderD%d", i); int fd = open(path, O_RDWR | O_CLOEXEC); if (fd < 0) continue; /* probe: a zero-size userptr returns -EINVAL on amdgpu but * -ENOTTY/-ENODEV on a non-amdgpu driver, which lets us tell * the nodes apart without pulling in libdrm version ioctls. */ struct drm_amdgpu_gem_userptr probe = { 0 }; errno = 0; ioctl(fd, DRM_IOCTL_AMDGPU_GEM_USERPTR, &probe); if (errno == ENOTTY || errno == ENODEV || errno == EOPNOTSUPP) { close(fd); continue; } fprintf(stderr, "using %s\n", path); return fd; } fprintf(stderr, "no amdgpu render node found under /dev/dri\n"); return -1; } int main(int argc, char **argv) { int fd = open_amdgpu_render(argc > 1 ? argv[1] : NULL); if (fd < 0) return 1; /* anonymous, private, page-aligned region for the userptr */ void *buf = mmap(NULL, BUF_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (buf == MAP_FAILED) { perror("mmap"); return 1; } memset(buf, 0xa5, BUF_SIZE); /* fault every page in */ /* REGISTER installs the mmu_interval_notifier (amdgpu_hmm_register()); * VALIDATE additionally faults the pages via hmm_range_fault() and * binds them into GTT. ANONONLY matches our MAP_ANONYMOUS region. */ struct drm_amdgpu_gem_userptr up = { .addr = (uint64_t)(uintptr_t)buf, .size = BUF_SIZE, .flags = AMDGPU_GEM_USERPTR_ANONONLY | AMDGPU_GEM_USERPTR_REGISTER | AMDGPU_GEM_USERPTR_VALIDATE, }; if (ioctl(fd, DRM_IOCTL_AMDGPU_GEM_USERPTR, &up)) { perror("GEM_USERPTR (VALIDATE)"); /* retry without VALIDATE: the notifier is still registered, the * pages are present from the memset, MADV_PAGEOUT still works */ up.flags = AMDGPU_GEM_USERPTR_ANONONLY | AMDGPU_GEM_USERPTR_REGISTER; if (ioctl(fd, DRM_IOCTL_AMDGPU_GEM_USERPTR, &up)) { perror("GEM_USERPTR (REGISTER)"); return 1; } } fprintf(stderr, "userptr handle=%u, interval notifier armed over %p..%p\n", up.handle, buf, (char *)buf + BUF_SIZE); /* Force synchronous reclaim of the notifier-covered range. MADV_PAGEOUT * runs shrink_folio_list()->try_to_unmap()->invalidate_range_start() * with fs_reclaim held in THIS thread, so amdgpu_hmm_invalidate_gfx() * takes notifier_lock under fs_reclaim and lockdep closes the cycle. * * A few iterations (re-touching in between) cover the race where pages * are already paged out on the first pass. */ for (int it = 0; it < 8; it++) { if (madvise(buf, BUF_SIZE, MADV_PAGEOUT)) perror("madvise(MADV_PAGEOUT)"); usleep(50 * 1000); memset(buf, 0xa5, BUF_SIZE); /* fault back in for the next pass */ } fprintf(stderr, "done: check dmesg for " "\"possible circular locking dependency\" / amdgpu_hmm_invalidate_gfx\n"); munmap(buf, BUF_SIZE); close(fd); return 0; }