From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f52.google.com (mail-dl1-f52.google.com [74.125.82.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB3553E5577 for ; Thu, 18 Jun 2026 09:48:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.52 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781776132; cv=none; b=S05VwYVphor0xoa4UdJR8drkFNHpRzUWVTVOkm0tVip0a5vIQ08qzL9o2PegNRM8AoS5QNKU6ZghduIYKXDllJ3f2WCMU2o8QiIebXEF2j5MgyAbAh73T6sZNLVSgaaETd5StjCmKurL8mZMYDcQrH0hTGfiea5MjfFFjlP1qF4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781776132; c=relaxed/simple; bh=H6pG1KH1HWzroQ2xN0ysBJpEFlDxQDOFKkiRbInW2+U=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=dZTBXOcu4GlT0b+hibp5p+1Rs8zAHutaKwWQYMdwPeGeVYk9xpABJpoSJB1Y3jEt8zIzTQ47LfS+rX/xUQV/3fmZOJsZ2MRU5ryYnETM9pqUYthB1fKFRbbUOIMPhMuaGC0qbszfnZcUkiljk/M2xYH8uBwea09akbKSPCYtI08= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kysVfXcN; arc=none smtp.client-ip=74.125.82.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kysVfXcN" Received: by mail-dl1-f52.google.com with SMTP id a92af1059eb24-1384ebe7a10so1531742c88.1 for ; Thu, 18 Jun 2026 02:48:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781776128; x=1782380928; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=YMLQTADz6AnzpUepxHGqYwO7011QTCioGMDUqQIMy2A=; b=kysVfXcNfWdWTaHjMVq6gaIEZgLNaBcZv5LmzCwI6SGxaYWkg808ZpCv1dZwbto/FI uasR1qcXEAjTO/h9ht/9nzGJ84h4k+8n7pMX3Ho3d/Mm0V+l+YYT3Fvp2rgDsA/LznLJ ePmLy+a+ydySdPZTJhByLXNVgqWIG5prqPSEGYUckxNcg/Rbtz3YtPrSWl7Ez7iQD1Ly Sy+Nzw9HPzZ7tQU82uvPCDTB2pAhSbg/kWc4gdzEYpaIqx1vsptywujZhQXU2IIXBeg1 bjHC9XyldRUvBMoJjQkKfxpqW6EyX3HNbQdh5S/V0aKhADd1U5mJVPUdM5puCRNVBeQL eMfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781776128; x=1782380928; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=YMLQTADz6AnzpUepxHGqYwO7011QTCioGMDUqQIMy2A=; b=mmwrJEeXIBWrlFaFogQcVkFqlvxUjFaTO8DYDyTzHcy/KdRePY9Vky2l4zTmlnHSXD f+KmzYlUMB8pzAMzZygojV1Os61voJZamOZq37s6lPRtdFXYjyI4JWaWo+Aqhm9qPCV3 3njBoaGsKwrC4iaOqk9vAdrcp2v7BM1u/4kW7t2swfS7hU6wjbUENZxoGit55GPj9guv 637kwOAWidaD5SEJPd9SzIGF484tKp0A0pUsn3PMJ31M4jA0FGOg10D7/LCzB/06IHVp nJ65IHDNFGtOiqOrGxvKsQv231mDDUi0ceavVQtmbOo1kH2lrrBZTYpYTXRneW6YUnYQ ywBQ== X-Forwarded-Encrypted: i=1; AFNElJ8OPP23Q7/C2aTScC1OHBmNztbcsgXAY9OqIDB8Z3s/OE+KUVMINjxu2aZQGZHdo9aBIqf6wr7oMH6toXI=@vger.kernel.org X-Gm-Message-State: AOJu0YwWtb4XimtZnzPc+KDaFDnqtbK5x2/3Iv7mWdZqJxA2NQafI+GM mawPw1jUxDDswkjUUCdXlMkfyeQ1dnYsiRY+RFi9Agic/k9NYdzUUvSl X-Gm-Gg: Acq92OEvhmMCEsykpq8K1BsQtRGfiR+1aei7w8LMFOot+w4tTACouc9vqNFw6x0vISZ zLX1kHtOK8VJMImiC4kFNasXD8sjAW51f4d1oxMAfeasFwgFJ9BGsIwlkAxu17Yer4dgLLuBxIs QJI9GBlbYMi+nep1PimOG5OVEUdgJWm9pgv2cJRIcC2SMV+UnPgpsSmMcAa41NsM4L6ADL3u+kJ ozh+nwQ7XH5fY2NgfSkUESKwBiY2pc5an+TUH/YN9OlpRPqm2iDppFbspD1ICWO+YA94YMgELy0 6cVS52n0kkrojMJWxrUnfwf8BukTMPY7pMStj9ReivRsRgkC5DzFGOUTtpcmZMHZLzW2H0RD0EM CkwfZvZyTKGzHqMy+PEyCBi0LtAgLOXEJ///S3qiGNGm11aiGnoZPw6+YXRoWUD9cNFnxogog3B S74prhDfoDgVXTviG/DhOllUFq X-Received: by 2002:a05:7022:2206:b0:128:d4be:7428 with SMTP id a92af1059eb24-1398f6cbc71mr4432771c88.19.1781776127632; Thu, 18 Jun 2026 02:48:47 -0700 (PDT) Received: from localhost.localdomain ([104.28.158.224]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-1384b91a4c7sm18069971c88.8.2026.06.18.02.48.43 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 18 Jun 2026 02:48:46 -0700 (PDT) From: Wang Lian To: sj@kernel.org, akpm@linux-foundation.org Cc: npache@redhat.com, gutierrez.asier@huawei-partners.com, daichaobing@sangfor.com.cn, linux-mm@kvack.org, linux-kernel@vger.kernel.org, lianux.mm@gmail.com, kunwu.chan@gmail.com Subject: [RFC PATCH 0/6] mm/damon: Add mTHP-aware collapse/split with ARM SPE feedback Date: Thu, 18 Jun 2026 17:48:32 +0800 Message-ID: <20260618094838.32805-1-lianux.mm@gmail.com> X-Mailer: git-send-email 2.50.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Received an off-list report that DAMON significantly overestimates hot memory in KVM/QEMU deployments with THP-backed tmpfs guest memory running Oracle workloads. The root cause is structural: a PMD entry covers 512 4KB subpages with a single Access Flag (AF) bit. When any one subpage is accessed, the entire 2MB region appears "hot" to DAMON. On ARM64, this is compounded by the hardware AF mechanism -- the AF is only set on a TLB miss. Consequently, when the working set fits entirely within the L2 TLB (e.g., a 16MB working set with 2MB THP running on a Kunpeng 920's 2048-entry L2 TLB), DAMON becomes completely blind to subsequent accesses. x86 is not subject to this specific blindness under similar conditions. We reproduced this memory inflation on a Kunpeng 920 platform using a synthetic workload (8GB mmap with a 0.2% sparse hotspot, i.e. 16MB actually hot): THP=always causes DAMON to report the entire 8GB as hot, while THP=never reports only a few hundred MB -- a 512x overestimate relative to the actual 16MB hotspot under THP, and a ~33x gap between the two THP modes. ARM SPE hardware profiling independently confirms this asymmetry: out of 2,005 THPs sampled system-wide over 10 seconds, 97% had fewer than 10% of their 4KB subpages actually accessed. To mitigate this, this series extends the existing DAMOS_COLLAPSE action to be mTHP-aware via a new target_order field, and introduces a new DAMOS_MTHP_SPLIT action. This enables DAMON to proactively split PMD THPs into smaller mTHPs when most subpages are probed as cold, and collapse them back when beneficial. To resolve the sub-PMD monitoring blindness, the split path can incorporate fine-grained hardware feedback from ARM SPE. The hardware feedback loop (damon_spe_folio_heatmap) implements a two-pass signal filter: it first identifies the peak chunk access count, and then marks sub-chunks with >= 1/10 of the peak count as hot, effectively filtering out SPE sampling noise. A configurable hot_threshold (default 30%) controls the split decision: only folios with a hot fraction below this threshold are eligible for splitting. When no SPE data is available, the infrastructure gracefully falls back to explicit PTE-level scanning via folio_walk. Currently, SPE data is fed from userspace via debugfs (e.g., perf script piped through a histogram builder into /sys/kernel/debug/damon/spe_feed). Collapse path (patches 1-3): DAMON scheme action=COLLAPSE, target_order=N -> damos_va_collapse() -> damon_collapse_folio_range() -> collapse_huge_page() Split path (patches 4-5): DAMON scheme action=MTHP_SPLIT, target_order=N, hot_threshold=M -> damos_va_mthp_split() -> damon_spe_hot_fraction() -> split_folio_to_order() SPE feedback infrastructure (patch 6): perf script -> spe_hist -> debugfs spe_feed -> per-folio rbtree {THP-aligned PFN -> access_count[512]} -> damon_spe_folio_heatmap() -> hot_bitmap -> split decision The userspace helper tools (including the spe_hist histogram builder and validation scripts) are archived at: https://github.com/lianux-mm/damon_spe Testing was performed on a Kunpeng 920 system (256 cores, 249GB RAM, base kernel 7.1.0-rc5+): T1 ARM64 blind spot: A 16MB THP workload (where 8 PMDs fit entirely within the L2 TLB) resulted in DAMON detecting 0 regions. Conversely, using 512MB with 4KB base pages, or a 16GB THP layout (exceeding L2 TLB reach), allowed DAMON to function normally. T2 THP inflation: With an 8GB mmap and 16MB actually hot (0.2%), THP=always: DAMON reported 8GB hot (512x vs ground truth); THP=never: ~245MB (15x vs ground truth). The THP-induced gap between the two modes was ~33x. T3 RocksDB: Fragmented malloc allocation prevented THP formation, and DAMON behaved normally. We could not reproduce THP inflation with RocksDB. The workloads fundamentally vulnerable to this structural issue remain KVM guests, JVM large heaps, and PostgreSQL shared_buffers. T4 min=0 deadlock break: A 256MB THP induced the DAMON blind spot. Triggering an unconditional mthp_split (via nr_accesses/min=0) successfully shattered the space into 16384x16KB folios, allowing DAMON to fully recover. T5 ARM SPE histogram: Out of 2005 sampled THPs, 97% exhibited <10% hot subpages. A typical trace showed PFN 0x820db800 accumulated 39,794 hardware accesses concentrated across only 3 out of 512 subpages. End-to-end: Verified hot/cold discrimination. The SPE feed preserved a 90% hot THP intact, while successfully splitting a 25% cold THP into 128x16KB folios. Known limitations: - The full KVM + Oracle production chain has not yet been benchmarked end-to-end. While individual component verification is complete, full integration testing is planned in collaboration with Sangfor. - khugepaged may aggressively re-collapse the mTHPs that DAMON splits. A coordination/back-off mechanism is required to avoid ping-pong effects. - SPE data is currently funneled via a userspace daemon and debugfs. Direct kernel-side perf_event sampling integration is planned as a follow-up. - The rbtree entry TTL (30s) and signal threshold (1/10 of peak) are empirical defaults subject to further tuning. - The ARM64 DAMON blind spot (WSS < L2 TLB reach) is a pre-existing hardware-MMU characteristic, not introduced by this series. Setting nr_accesses/min=0 serves as an effective workaround for the split path. Reported-by: Chaobing Dai Cc: SeongJae Park Cc: Andrew Morton Cc: Nico Pache Cc: Asier Gutierrez Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Wang Lian Wang Lian (6): mm/damon: add target_order field for DAMOS_COLLAPSE mm/khugepaged: add damon_collapse_folio_range() for external callers mm/damon/vaddr: implement mTHP-aware DAMOS_COLLAPSE handler mm/damon: introduce DAMOS_MTHP_SPLIT action and hot_threshold mm/damon/vaddr: implement DAMOS_MTHP_SPLIT handler mm/damon: add SPE feedback for sub-THP split decisions include/linux/damon.h | 18 ++ include/linux/khugepaged.h | 3 + mm/damon/Kconfig | 12 + mm/damon/Makefile | 1 + mm/damon/core.c | 3 + mm/damon/spe.c | 505 +++++++++++++++++++++++++++++++++++++ mm/damon/spe.h | 62 +++++ mm/damon/sysfs-schemes.c | 96 +++++++ mm/damon/vaddr.c | 118 +++++++++ mm/khugepaged.c | 39 +++ 10 files changed, 857 insertions(+) create mode 100644 mm/damon/spe.c create mode 100644 mm/damon/spe.h -- 2.50.1 (Apple Git-155)