From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C1C43B2FF6 for ; Wed, 27 May 2026 20:48:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.177 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779914893; cv=none; b=uqa0W4doyM9ebzI5XNLDsMHgIQ5ddBnVL3skHxFkCpKv6YcKKkI9zL+QKNNF2GqQ5GivfHekVdqZ4BiGG6oQG9TrvnZ/19WAd31JtqtMmjSp1R6t2Cc5Ky4hK5ZXTj5GsxOy1OdTDHmR5fV9lKXR7siMiVvFPoYqbVNCD6VP+NM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779914893; c=relaxed/simple; bh=gquPeyIrgrXq7dLjLlFiTq8DRz00IYvKdULLyzd3nQI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NijtHHBegez9g08SfTh58qjzBwiZGj5JS1HEnLwYOc9zvsM41YxVIL+nSxn6fFpNdS9eGU8wAYK50A0MNX0/Qnms8ebB+rHmLwVNOjT46Q4rJvW8tPGgLkr7NCi8iND3alS3Jy9s1YdJt31FOsgeorPkhzF5zv1wtYVOVthew5M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b=QB0CiFIL; arc=none smtp.client-ip=209.85.160.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b="QB0CiFIL" Received: by mail-qt1-f177.google.com with SMTP id d75a77b69052e-5102582e23eso95868211cf.1 for ; Wed, 27 May 2026 13:48:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg.org; s=google; t=1779914888; x=1780519688; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GBXzyPhWkzGmNH/N9A+/kfLtlL9nu+YFCZMUKmgaFzk=; b=QB0CiFILJDYJbNEV//W8WxUN7gH4RvSyq3pCkLHgldimXgbNAR7sqZYaSmrg9uIQEl oc4zjfpob2K2S3id7/3gaensZcIMiWDlGvZnlvkDy5L7RzkFd5ysUg85xop5oBdNJZyi IlzxyKOkqCVWneZs3Dt+5pR+qu8L7hfis6IFx9vCKiMlepAvHmOc1kvpCXJjv0YWa8O1 TH2jS0ll2jZUNUfCy9rAMktfA+EJDoqtZlSU+ldwZAjrIs/VaPexAuLTFICSbx9EPcBF wtmSWAiMP1OxMS+OnmPjkFIIIoRfjnnhhWkhIfBaQSY6IonAXkTIRVwFUBinik9pSvTw 9fgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779914888; x=1780519688; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=GBXzyPhWkzGmNH/N9A+/kfLtlL9nu+YFCZMUKmgaFzk=; b=LHim+uGbSJPqXerPwR12cKJzi1eMyncU7E0JPKWSsauj5+cTILXFcuykt+EKCNhuQI SgyEKcswJ8EBP9PZ2qs2rvJ5uUQDA7DGHFmUAFoPQ8r9o2QW1Rkj+IMbNeAY942sZ1bL L78PoxV6ovBynoZ/C1OB92dwQPCRpFLzxYfXDrufvImS0Q45d056RTfqWS/pnLpLLC1k 5+GEMNja3sTufBpaZuhpWwIDTq0lE4Ue0jNOnGuhttElx/uBRGS7wAgTtsjY+0oE8zP9 Dynh5vpGdkynWSzZsAp/+rIPnlHc2RxrtsgQlfgONj78D508Qke2VwJEF5bXL9wO2hQ7 FQLQ== X-Forwarded-Encrypted: i=1; AFNElJ/3jwd7p9OjZNBgaw4v1vCQfhL3MIo/owoh7gnzMIKdFfmpvphXy9MJPu6krI3nsCSRuDsBF2kQtQfNZEQ=@vger.kernel.org X-Gm-Message-State: AOJu0YxNLSm8Q6bfbyvOsEkryuCMK7GGDjNUWk75X2hGGMZfScoN0W4G k1YeVwlAEHyLa0QX1vuqPklLiBrptSs3FTLXgaJffUZ/9s+V9IEYfo26PAJRE8X4EAo= X-Gm-Gg: Acq92OF2kZdcXOZmwdlJ/iKNHSp7AYivSmcLV37kPuo19BQdKu0A3s0sLbizlp439tr Ew66naDjcTIQitzzO/CtjJ20u/ndkX8AnmoXJKUR4oPVeQoIOVcaEmj/LSZ39ISzqV3GhmIgNdO bJxSTILRg31kKOvFHxpGkZVYGeLaYQDsu8n+kwPcSTy6ji2ldbhkp8Lnu/RUgqYmuTr2QcqyUGk XJiqxFdTehcm6mnWP4/+PcZ6VyCn4vNnOqVi2v1eopsKobZLFOFI7JjRHADyF+BkBX2iz++cr/G n25IpyMR7l+Gszde8IWLORd2nn1zd8r1kVZgSytI4D768OJU5qlFHflH2uPkDXLryAOoeI16TPJ 7d/MeMgb2ABN6V7JMTRu6b+TblYmqeYihXgzVzJ6ZdARtyDMJQvPY6E8ui7bLQmWgHt0p3koKIb +u2CvytJeZteMg/SzNQrpAInnZcoj5hLez X-Received: by 2002:a05:622a:244b:b0:50d:e471:2d1e with SMTP id d75a77b69052e-516d43cb2dbmr336768581cf.35.1779914887963; Wed, 27 May 2026 13:48:07 -0700 (PDT) Received: from localhost ([2603:7001:f100:500:365a:60ff:fe62:ff29]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-517069f2d2fsm59376831cf.3.2026.05.27.13.48.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 May 2026 13:48:07 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Michal Hocko , Dave Chinner , Roman Gushchin , Muchun Song , Qi Zheng , Yosry Ahmed , Zi Yan , "Liam R . Howlett" , Usama Arif , Kiryl Shutsemau , Vlastimil Babka , Kairui Song , Mikhail Zaslonko , Vasily Gorbik , Baolin Wang , Barry Song , Dev Jain , Lance Yang , Nico Pache , Ryan Roberts , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 1/9] mm: list_lru: fix set_shrinker_bit() call during race with cgroup deletion Date: Wed, 27 May 2026 16:45:08 -0400 Message-ID: <20260527204757.2544958-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260527204757.2544958-1-hannes@cmpxchg.org> References: <20260527204757.2544958-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When list_lru_add() races with cgroup deletion, the shrinker bit is set on the wrong group and lost. This can cause a shrinker run to miss the cgroup that actually has the object. When the passed in memcg is dead, the function finds the first non-dead parent from the passed in memcg and adds the object there; but the shrinker bit is set on the memcg that was passed in. This bug is as old as the shrinker bitmap itself. Fix it by returning the "effective" memcg from the locking function, and have the caller use that. Fixes: fae91d6d8be5 ("mm/list_lru.c: set bit in memcg shrinker bitmap on first list_lru item appearance") Reported-by: Usama Arif Reported-by: Sashiko Signed-off-by: Johannes Weiner --- mm/list_lru.c | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/mm/list_lru.c b/mm/list_lru.c index dd29bcf8eb5f..45d1b97737ea 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -77,14 +77,14 @@ static inline bool lock_list_lru(struct list_lru_one *l, bool irq) } static inline struct list_lru_one * -lock_list_lru_of_memcg(struct list_lru *lru, int nid, struct mem_cgroup *memcg, - bool irq, bool skip_empty) +lock_list_lru_of_memcg(struct list_lru *lru, int nid, + struct mem_cgroup **memcg, bool irq, bool skip_empty) { struct list_lru_one *l; rcu_read_lock(); again: - l = list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(memcg)); + l = list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(*memcg)); if (likely(l) && lock_list_lru(l, irq)) { rcu_read_unlock(); return l; @@ -97,8 +97,8 @@ lock_list_lru_of_memcg(struct list_lru *lru, int nid, struct mem_cgroup *memcg, rcu_read_unlock(); return NULL; } - VM_WARN_ON(!css_is_dying(&memcg->css)); - memcg = parent_mem_cgroup(memcg); + VM_WARN_ON(!css_is_dying(&(*memcg)->css)); + *memcg = parent_mem_cgroup(*memcg); goto again; } @@ -135,8 +135,8 @@ list_lru_from_memcg_idx(struct list_lru *lru, int nid, int idx) } static inline struct list_lru_one * -lock_list_lru_of_memcg(struct list_lru *lru, int nid, struct mem_cgroup *memcg, - bool irq, bool skip_empty) +lock_list_lru_of_memcg(struct list_lru *lru, int nid, + struct mem_cgroup **memcg, bool irq, bool skip_empty) { struct list_lru_one *l = &lru->node[nid].lru; @@ -164,12 +164,16 @@ bool list_lru_add(struct list_lru *lru, struct list_head *item, int nid, struct list_lru_node *nlru = &lru->node[nid]; struct list_lru_one *l; - l = lock_list_lru_of_memcg(lru, nid, memcg, false, false); + l = lock_list_lru_of_memcg(lru, nid, &memcg, false, false); if (!l) return false; if (list_empty(item)) { list_add_tail(item, &l->list); - /* Set shrinker bit if the first element was added */ + /* + * Set shrinker bit on the memcg that owns the locked + * sublist - lock_list_lru_of_memcg() may have walked up + * past a dying memcg, and the bit must be set there. + */ if (!l->nr_items++) set_shrinker_bit(memcg, nid, lru_shrinker_id(lru)); unlock_list_lru(l, false); @@ -204,7 +208,7 @@ bool list_lru_del(struct list_lru *lru, struct list_head *item, int nid, { struct list_lru_node *nlru = &lru->node[nid]; struct list_lru_one *l; - l = lock_list_lru_of_memcg(lru, nid, memcg, false, false); + l = lock_list_lru_of_memcg(lru, nid, &memcg, false, false); if (!l) return false; if (!list_empty(item)) { @@ -288,7 +292,7 @@ __list_lru_walk_one(struct list_lru *lru, int nid, struct mem_cgroup *memcg, unsigned long isolated = 0; restart: - l = lock_list_lru_of_memcg(lru, nid, memcg, irq_off, true); + l = lock_list_lru_of_memcg(lru, nid, &memcg, irq_off, true); if (!l) return isolated; list_for_each_safe(item, n, &l->list) { -- 2.54.0