From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AED963749F6 for ; Wed, 22 Jul 2026 15:00:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.177 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784732417; cv=none; b=D+R+UnZL9ULkDi9YrgKf1IlJiajNCkoGxofO83qbWjp7L7U18bf1DnTvjNno3ii99SXL7H820YODEiKZdUz+USnvmV+NKMV+6qJ3RnL71YWTj4AaN5OGpfrUE4OFBydpjAC6kS0MG7Lw4ruBtSlrkrqRurBWfa4DftbSeAwTzFY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784732417; c=relaxed/simple; bh=INHUvS4Xk54MpBqgU7R471x+YkLibsTfRlnI/xBZiMk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gnlLE0b+QEV2fB8WgDiS9SPU6eYg7YkwU2y/gj627Xku6tCBTumxFC7Bk7ysK7HgBaM/0yIMOmb7VFJp3/aHJlwnLbbNNUJs8uU+PC7iyaAxp7sMHHavTPUP0lOL7vvwlXuPENPNL2RjvbDSZfSsV/ILY0crcajLLEm8G/9M9oM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b=Tyu+LVCE; arc=none smtp.client-ip=209.85.222.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg.org header.i=@cmpxchg.org header.b="Tyu+LVCE" Received: by mail-qk1-f177.google.com with SMTP id af79cd13be357-92e67555e24so672520885a.3 for ; Wed, 22 Jul 2026 08:00:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg.org; s=google; t=1784732414; x=1785337214; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=WzWEJqU7j0ZiAwHq29jiPVthqAQlNA5+hbQ4+opl0xY=; b=Tyu+LVCEldgJ4p+k6aXx2lli1Sps4iSLwArxt8IsHvn300w9XRxEA8YwdmKR7jh1Mi PGJsYvlxa9pHhOlDtn7JIsNiHzXMBiboZw5kOH6xMMebhglpf9LWZY/IqbuQKEZ0WtVI SYXzNknxggsP7oOh9s6afaRTraj2UpnWhUiTMuM+X3BSXxAm3j2BbBsmdlb+SYcFc+hW F6qN6kdmOu2J675/m+lrKXPaeL+b1fX8xa0rsEhK2FzwjiDESNGz74dMPHqBNb89ISpG c0Ehy83UQ5nxSz3rO0SBysN6xGNGudCEVNvz9uq1YgY2z1lg0ucT5YLeeWTvgzXABJHR VLgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784732414; x=1785337214; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=WzWEJqU7j0ZiAwHq29jiPVthqAQlNA5+hbQ4+opl0xY=; b=MG31E4nhUz2IXQJVjfYNo6r3pHbowcV7g3TFHhaHmEWqmW4mi9x7c7CS7L+HCIxpJ/ G+CfWbPSOwR2MvZbfeZ1TG7xm4SgtKe/lVTcH/xwPo1rsDunnMoYdm6WAILlRUa58BdD mn6NdpFBPD2i8zeI4mbN73/WkvvB2cvxb+Trtxf6h1fQSNcg4+AeIovEEyYHxfavUzBg ZZFveAZJru4owcIXXhacMW6PGNiA1elZNNfwmh0A3Ds+QRpNn2YBoRsE+cSAdfcaqNrw Ed+LHm8VyAsBl47Zf7tXV+32aTv3sRW/BnguKScAaOW1Dy8ETb3k6dJYiyXoowhNXnxG NokA== X-Forwarded-Encrypted: i=1; AHgh+RqzxgqirGeBA8H6VnDuLWsP8Ijezy6Ifa/FlxiiUVE++W1QYGkdBhTogZ5Y72c+ApAX1yHsaZY4S43Vbss=@vger.kernel.org X-Gm-Message-State: AOJu0YzMeIQ4zR7fTJ3qdJqDps4NZ/YV2bV6l9yxXGXGvrnO7qZonlxW Tg/eIhJV2XBxu5R4cK2DTiCmGN7m5wyBrx+s5I0psUmD9ZfwYoc0U9d4a6W7Fp3E84U= X-Gm-Gg: AR+sD12ciodkqc+dB9hcxLyQVySMzhyxhiF27pj8alPCAD50SIS3lhAZeaQcFXpo6nr GrqLQZHqhDnPpXqQizFJAdQ1zurbO8mh05VYptmHzGYa7DpwqCcGGIG7CWE4pYFFi+T+Bb41L61 gKrcKZxcqcA8WnwoCMLepWyAQLER6JHA5xWxdQTSjF2vop2/2CdKVEPXSUv/dozLupq/ELZzrnl xFJ7CQSyr8Tw1QhkjgwS74Z0Rwh8LNBo0QOxJMeRGRv2Vavr7b4gmkC3dI9noU7AbEkfg6XSVrP kox/yPnhnZLUS5M4ly1JTAnFjhM+epPoUVUi2JJxyVXT+rucaJg3Vy7pxp3QQpeSRV2VKi3xkwH W9keAmyfPFPXx8ovjeyHTjCkGnSjiPpQIuOdtl56v1gsnCt5G47gMcLBVr44iZm0tTVrK/mFByn Zauc5X81aDA3U= X-Received: by 2002:a05:620a:4008:b0:92d:5504:72dd with SMTP id af79cd13be357-930b3ed9d45mr2222220285a.10.1784732414019; Wed, 22 Jul 2026 08:00:14 -0700 (PDT) Received: from localhost ([2603:7001:f100:500:365a:60ff:fe62:ff29]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-907ba672806sm23149956d6.0.2026.07.22.08.00.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jul 2026 08:00:13 -0700 (PDT) From: Johannes Weiner To: Andrew Morton , Vlastimil Babka Cc: Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Zi Yan , David Hildenbrand , Lorenzo Stoakes , "Liam R . Howlett" , Mike Rapoport , Shakeel Butt , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/4] mm: compaction: support non-movable compaction for pageblock requests Date: Wed, 22 Jul 2026 10:56:45 -0400 Message-ID: <20260722150006.3848560-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260722150006.3848560-1-hannes@cmpxchg.org> References: <20260722150006.3848560-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit While trying to fix a reclaim storm in defrag_mode, I noticed that non-movable direct compaction is extremely inefficient. When searching for space to evacuate, compaction only allows blocks of the same type as the incoming request. This is to prevent migratetype pollution, where a small non-movable request frees space in a movable block and provokes the allocator to fall back and pollute it. This protection is reasonable on one hand, but the downside is that it makes non-movable direct compaction nearly useless: if we get the type annotations right, by definition there aren't any movable pages inside the non-movable blocks it is allowed to scan. With defrag_mode, the goal is the production of whole blocks, which are essentially type neutral: __rmqueue_claim() will convert them wholesale on alloc. This makes type mixing and pollution a non-issue. Fix the pollution gates to take the requested order into account, and allow whole-block requests to scan blocks of other types. The only exception is CMA blocks. That type is sticky and these blocks cannot be claimed to other types. Continue to be strict with them, and allow only explicit ALLOC_CMA requests and kcompactd to evacuate them. Reviewed-by: Vlastimil Babka (SUSE) Signed-off-by: Johannes Weiner --- mm/compaction.c | 46 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 7 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index f08765ade014..225862a00380 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1381,12 +1381,44 @@ static bool suitable_migration_source(struct compact_control *cc, if (pageblock_skip_persistent(page)) return false; - if ((cc->mode != MIGRATE_ASYNC) || !cc->direct_compaction) + /* + * Background compaction produces blocks for the zone at + * large, with no particular allocation context. Allow all + * block types, including CMA. + */ + if (!cc->direct_compaction) return true; block_mt = get_pageblock_migratetype(page); - if (cc->migratetype == MIGRATE_MOVABLE) + /* + * CMA pages can only be taken by ALLOC_CMA requests. For anybody + * else, vacating a CMA block consumes free pages the caller + * could have used, and produces free pages it cannot. + */ + if (is_migrate_cma(block_mt) && !(cc->alloc_flags & ALLOC_CMA)) + return false; + + /* + * Per default, scans are restricted to blocks compatible with + * the request, to prevent cross-contamination. Once + * compaction priority escalates to synchronous scans, though, + * scan all blocks to try to make forward progress. For + * movable request, this likely helps little: there shouldn't + * be many migratable pages inside non-movable blocks besides + * allocator fallbacks. For non-movable requests, this helps a + * lot, as they can finally scan movable blocks. + */ + if (cc->mode != MIGRATE_ASYNC) + return true; + + /* + * Prevent migratetype == MIGRATE_MOVABLE || cc->order >= pageblock_order) return is_migrate_movable(block_mt); else return block_mt == cc->migratetype; @@ -1974,12 +2006,12 @@ static unsigned long fast_find_migrateblock(struct compact_control *cc) return pfn; /* - * Only allow kcompactd and direct requests for movable pages to - * quickly clear out a MOVABLE pageblock for allocation. This - * reduces the risk that a large movable pageblock is freed for - * an unmovable/reclaimable small allocation. + * Prevent direct_compaction && cc->migratetype != MIGRATE_MOVABLE) + if (cc->direct_compaction && cc->migratetype != MIGRATE_MOVABLE && + cc->order < pageblock_order) return pfn; /* -- 2.55.0