From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=91.218.175.224 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794321800; b=5ANE3qEYRZnxgQw0c+j36GLrE4JxaupBLntNri/jcK8n6ESuweRUo0BMaO+detHrJlnt 7/B1X9DLJWbSU/04dH/W+qENPcqx6km0DIjGcfHk2aqHlGh1W1PH8aeNOG3vScGIoQ65B T8tKto0FG2/XN5HPlAqQqpkOMO9ekodJT2KYUMgQjmq7UIazcsBDJ3Vs8b2J4JODcsqTK IO2eGlbF2HfUTFAs6C2alPKIrskvG/sKynNhjCt5SX2BgoXmrhnwhuVpbMS6JDwErK7ns 7vKsx1nH+lHOFuEGLCV6Jdv0IgeQp16v8tBoao8CwB/rXPXfv3ZOACy9sGBeDx+giWg== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794321800; h=DMARC-Filter:X-Envelope-To:DKIM-Signature:X-Envelope-To:Received: X-Mizu-Trace-ID:X-Migadu-Flow:From:To:Cc:Subject:Date:Message-ID: X-Mailer:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding; bh=ONtiqfBY7XF9tcsGZa6BGd36+aq3hcBRWhYmk4rQLvM=; b=IJn/iLF8Yu1IDzc/SXFljpH6gBexmWZIYX88csOfMAF02xd7LD1/Ej5EJZV6J7ZHILc7 g2/u9nL2yzHqpfnfHx+dt2mMKsnv+TQpAceOG7aNJum3xQl+SLG1KdSXGukD1h/dRVBu5 U9Yq4dBHXfJ2tvfS8H8BXSvc6snMsGDr3RE+nrbQQYuMq10YDMij/Tm065wK2/Qo2/PwA 5ITBwP27Fu+CsOCTxdk4ZWomg06hjDUCIyJsZ44sqo8A0LU8B3rHi8b6011y++ZHMDQ/M 4+P3HvU21RioZlRVmuaVci+KCf2kLsLgCC3zCvmdZ5jLGi/byjF1SqEoYKxHSiXQhIw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=nhgEZOPf; arc=none smtp.remote-ip=91.218.175.224 Received: from mta0.migadu.com (out-224.mta0.migadu.com [91.218.175.224]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C5EF60A2609 for ; Tue, 10 Nov 2026 14:43:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 6C5EF60A2609 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key, unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=nhgEZOPf X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=x8taPRgwxDpWF2n1mLGTs5sZSctanW78ZPHgi3RsjbQ=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1794321796; v=1; x=1794926596; b=nhgEZOPfXyqrxK2Kd1sLKf8FyfBDXX+1LFf24hZCfiaUOAcahMUM/FDuRhq2kChjnO9QxCA3 lyrgx86DoLqc/s3VNrURV0k2XAzR56ufMbKCCTkYAglq/+K4rUsrFx9lcize8UCNPNOl/57UMWH K6rC3IhRAMJ4Cd+6mwySI7xQ= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id c0dc64717c01038c; Tue, 10 Nov 2026 14:43:16 +0000 X-Mizu-Trace-ID: c0dc64717c01038c X-Migadu-Flow: FLOW_OUT From: Qiliang Yuan To: Vlastimil Babka Cc: Qiliang Yuan , Andrew Morton , Kairui Song , Qi Zheng , Shakeel Butt , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Baoquan He , Baolin Wang , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: Re: [PATCH v2 2/2] mm/compaction: defer failed async direct compaction Date: Tue, 10 Nov 2026 09:42:56 -0500 Message-ID: <20261110144256.3841504-1-qiliang.yuan@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20261102-bug-mm-thp-async-compact-defer-v2-2-b407e974db54@gmail.com> References: <20261102-bug-mm-thp-async-compact-defer-v2-0-b407e974db54@gmail.com> <20261102-bug-mm-thp-async-compact-defer-v2-2-b407e974db54@gmail.com> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit On Mon, 02 Nov 2026 01:01:08 -0500, Qiliang Yuan wrote: > @@ -2600,9 +2600,12 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) > > /* > * Clear pageblock skip if there were failures recently and compaction > - * is about to be retried after being deferred. > + * is about to be retried after being deferred. Leave this to sync > + * compaction: async compaction relies on the skip hints, and clearing > + * them on its retries would rescan the whole zone each time. > */ > - if (compaction_restarting(cc->zone, cc->order)) > + if (cc->mode != MIGRATE_ASYNC && > + compaction_restarting(cc->zone, cc->order)) > __reset_isolation_suitable(cc->zone); As was pointed out to me off-list, this test removes a recovery path. I'm not asking to reconsider the series, only leaving the analysis and a reproducer here in case they help when you look at the issue. The async run a THP fault does on the local node is never deferred: try_to_compact_pages() calls defer_compaction() only for sync priorities, because the deferral state is shared and deferring it after an async failure would defer sync compaction too. The async path checks that same sync state in compaction_deferred(), so as long as sync isn't deferred, every fault scans the zone again. Patch 1 splits the state so that async can be deferred on its own. Skip hints are cleared in two places: in compact_zone(), when compaction_restarting() sees the sync deferral at its maximum, and by kswapd before it sleeps. Madvised THP faults don't wake kswapd, and with proactive compaction off nothing else resets the hints, so once the pinned pages become movable again, the reset in compact_zone() is what lets local THP attempts use those pageblocks. After sync reaches the maximum deferral, that reset comes from an async retry, which is what the test in this patch blocks. I reproduced it on the same QEMU guest as the series: 32 vCPUs, node 0 with 16 CPUs and 24 GiB, node 1 with 16 CPUs and 8 GiB, compaction_proactiveness set to 0 and defrag left at madvise: 1. Fill node 0 with anonymous memory, free every other 4K page and long-term pin the rest through io_uring fixed buffers. 2. With defrag set to always, fault in a 1 GiB MADV_HUGEPAGE buffer bound to node 0, so THP faults there run sync direct compaction and fail until sync compaction reaches the maximum deferral. 3. Close the io_uring rings. The pins are gone, the memory stays in use and is movable again. 4. Populate a 4 GiB MADV_HUGEPAGE buffer from a CPU of node 0. Compactions that succeeded in step 4, one boot per number: mainline 1184, 963 v2 8, 0, 516, 400, 534, 521, 495 v2 without the test 1025, 994, 906, 957 v2 recovered in five of seven boots without resetting any skip hint (no call to __reset_isolation_suitable() in the two boots I traced): with async compaction deferred, it scans few pageblocks while they are pinned and leaves most of them unmarked. In the other two boots the marked pageblocks kept it from recovering at all. Without the test, the reset after the maximum sync deferral recovers every boot, as on mainline, and the steady pinned case stays as cheap as with v2 (7.8 to 20 ms in compaction against 310 to 332 ms on mainline). The reproducer is below, run as CPUS=0-15 sh thp-recover.sh. Thanks, Qiliang ---8<--- thp-recover.sh #!/bin/sh # Run as root on a two-node machine; CPUS lists the CPUs of node 0. CPUS=${CPUS:-0} D=/sys/kernel/mm/transparent_hugepage/defrag echo 0 > /proc/sys/vm/compaction_proactiveness echo madvise > $D echo 1 > /proc/sys/vm/drop_caches M=/sys/devices/system/node/node0/meminfo free=$(awk '/MemFree/ { print int($4 / 1024) }' $M) ./thp-recover hog $((free - 300)) > hog.log & hog=$! until grep -q ready hog.log; do sleep 1; done echo "steady, pinned:" for i in 1 2 3; do taskset -c $CPUS ./thp-recover fault 4096; done # Fail sync compaction on node 0 until it reaches the maximum deferral echo always > $D echo "bound to node 0:" taskset -c $CPUS ./thp-recover bfault 1024 echo madvise > $D kill -USR1 $hog until grep -q unpinned hog.log; do sleep 1; done echo "after dropping the pins:" for i in 1 2 3 4 5; do taskset -c $CPUS ./thp-recover fault 4096; done kill $hog ---8<--- thp-recover.c // SPDX-License-Identifier: GPL-2.0 /* * thp-recover hog fill node 0, free every other page and long-term * pin the rest; SIGUSR1 drops the pins and keeps * the memory in use * thp-recover fault populate a MADV_HUGEPAGE buffer and print how * much of it got THPs and on which node * thp-recover bfault the same, bound to node 0 */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #define MB (1UL << 20) #define MPOL_BIND 2 #define RING_BUFS 16384 static volatile sig_atomic_t unpin; static void on_usr1(int sig) { unpin = sig; } static void bind_node0(void) { unsigned long mask = 1; if (syscall(__NR_set_mempolicy, MPOL_BIND, &mask, 64)) perror("set_mempolicy"), exit(1); } static long vmstat(const char *key) { char name[64]; long v, ret = 0; FILE *f = fopen("/proc/vmstat", "r"); while (f && fscanf(f, "%63s %ld", name, &v) == 2) if (!strcmp(name, key)) ret = v; if (f) fclose(f); return ret; } static long thp_mb(void) { const char *key = "AnonHugePages:"; char line[256]; long kb = 0; FILE *f = fopen("/proc/self/smaps_rollup", "r"); while (f && fgets(line, sizeof(line), f)) if (!strncmp(line, key, strlen(key))) kb = atol(line + strlen(key)); if (f) fclose(f); return kb / 1024; } static int hog(size_t size) { size_t npages = size / 8192, done = 0, i, off; struct iovec *iov = calloc(RING_BUFS, sizeof(*iov)); int fds[4096], nr = 0, r; char *buf; bind_node0(); buf = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); madvise(buf, size, MADV_NOHUGEPAGE); memset(buf, 1, size); /* Plenty of free memory, but no free 2M block */ for (off = 4096; off < size; off += 8192) madvise(buf + off, 4096, MADV_DONTNEED); /* Long-term pin the pages left, as RDMA registration does */ while (done < npages) { struct io_uring_params p = { 0 }; size_t n = npages - done > RING_BUFS ? RING_BUFS : npages - done; int fd = syscall(__NR_io_uring_setup, 4, &p); for (i = 0; i < n; i++) { iov[i].iov_base = buf + (done + i) * 8192; iov[i].iov_len = 4096; } if (fd < 0 || syscall(__NR_io_uring_register, fd, IORING_REGISTER_BUFFERS, iov, n)) perror("pin"), exit(1); fds[nr++] = fd; done += n; } printf("ready\n"); fflush(stdout); signal(SIGUSR1, on_usr1); while (!unpin) pause(); for (r = 0; r < nr; r++) close(fds[r]); printf("unpinned\n"); fflush(stdout); for (;;) pause(); } static int fault(size_t size) { size_t n = size / (2 * MB), i, node[2] = { 0 }; long stall = vmstat("compact_stall"), ok = vmstat("compact_success"); void **pages = malloc(n * sizeof(*pages)); int *status = malloc(n * sizeof(*status)); struct timespec t0, t1; double ms; char *buf; buf = mmap(NULL, size + 2 * MB, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); buf = (char *)(((unsigned long)buf + 2 * MB - 1) & ~(2 * MB - 1)); madvise(buf, size, MADV_HUGEPAGE); clock_gettime(CLOCK_MONOTONIC, &t0); for (i = 0; i < size; i += 4096) buf[i] = 1; clock_gettime(CLOCK_MONOTONIC, &t1); ms = (t1.tv_sec - t0.tv_sec) * 1e3 + (t1.tv_nsec - t0.tv_nsec) / 1e6; for (i = 0; i < n; i++) pages[i] = buf + i * 2 * MB; syscall(__NR_move_pages, 0, n, pages, NULL, status, 0); for (i = 0; i < n; i++) if (status[i] == 0 || status[i] == 1) node[status[i]]++; printf("%zu MB in %.0f ms, %ld MB in THPs, 2M ranges on node 0/1: %zu/%zu, ", size / MB, ms, thp_mb(), node[0], node[1]); printf("compactions: %ld, succeeded: %ld\n", vmstat("compact_stall") - stall, vmstat("compact_success") - ok); return 0; } int main(int argc, char **argv) { size_t size; if (argc != 3) { fprintf(stderr, "usage: %s hog|fault|bfault \n", argv[0]); return 1; } size = strtoul(argv[2], NULL, 0) * MB; if (!strcmp(argv[1], "hog")) return hog(size); if (!strcmp(argv[1], "bfault")) bind_node0(); return fault(size); }