From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dy1-f171.google.com (mail-dy1-f171.google.com [74.125.82.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4E1C3168E1 for ; Fri, 19 Jun 2026 03:40:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781840450; cv=none; b=rUHxoKI5A9XkT+8AGbHQXnvbXpNBdGMTg/Fi4OX6gLmERrlASzXyT6Cqaux28jUwAz6M6Qf+qy8FLOcUUOH27YoLUNFeOwHWUlHdcksBlN1olnTAy+EEpaFQTJmwVbX3ERNL+3KMXXc3tP5Qqmwf0XqKyWJWUGMnAuYXCfTU+7s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781840450; c=relaxed/simple; bh=g84wbLuiJh972m18NHw6vVlpe+AOMpGp+RD6oHOrJu4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kUXyjiZv05l9wbYMIhoU6CaHh0riXCA3xuLqNRfrIuW9BnlFJ2PfbtYaidleh/lZg8rhkqNr/MvlLQeu8zT2sp8AP3WH5iihXGpXkQJtxGV+ADvOVMsAY7IC/3yPK0590e+trK+rxyfkjljBsJOnGE/35Hu7UIApl9/mGsE9PGk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gV5fUDP5; arc=none smtp.client-ip=74.125.82.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gV5fUDP5" Received: by mail-dy1-f171.google.com with SMTP id 5a478bee46e88-30c09f29b64so182783eec.0 for ; Thu, 18 Jun 2026 20:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781840448; x=1782445248; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZsT2A6vcJvvOIiYPu7Uwiedbr6l9fXs1eEHFyyaKQfw=; b=gV5fUDP5cxeQlZZlGXPFmGix61uF0SMBuS93cLh1/NixKtp80rnmxUCidcViBnG47l 7wEmvoB/Y+TS83yfv80K0+6PSzJcIkZj00xYXaSwLTtuS7mPoS69DzGliivBLUA5IOdd /xNF0bBfT9Z1PoO8UP77HmTTQyZLeT6syeyLGIbBe0TXny+e+k8Xfs4w6kM0qmQ1kvzK t9pFhfzE67yjy8hoB1m5buiWjl1w9yWGDU5JApwnvORsZUtmU5UyyRu9gY8ZiJRGpi+o GMitaEHw6JZ2qPomBAzr0UJtUjVYsXaid6tiusHkO0nQs1ErM0kKTPgzUYuLWgbWyLwo LAlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781840448; x=1782445248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZsT2A6vcJvvOIiYPu7Uwiedbr6l9fXs1eEHFyyaKQfw=; b=qB9Ana1IHCLEwKfMagyOf2c79uCvSIO+cZvsgPJTR+NVMgQoLq8Pch0/t67gtwsW3l vS2YNhGFiZhS5Am6h3k9RaO45OHmDa9PWQ+j+wvtjg9SVrKgmsMJ470ELCGIhej/+vNP S5RHPGRCkkngzexvLOmXIW4cBYZk9UQ2rEu+fgkqkjeHFsSZ0uxSiawe0BTsd+1h4cxB l2ZpSYzkXGyxJC/IahUChyeLPzYRKDhEbq7oeiJibGw12q24qtwZpIEmgEyU8fnd3u/t yYbYV//UW6QEs4U1PHmAnp0j5h1uL/1MmW+ewogrF+LHo51H0noK5B8m3sjUjcNJ+Vno 2WHw== X-Forwarded-Encrypted: i=1; AFNElJ9o9g6daMiM9FcUufYdo7ZAc13abEs91SVI26JrR5DLAYwLSf6EjEmm0JaBEKkfnkc6H/OjuV45uP73CaQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwYmrfPqVJe60OUf7ggA6HaGM8IcdoKvbn0FC0KPqNOrHXWCxFf qYYWpvm71EprWgIEg9jHAagDaon8fxHRyakj5/k2M7tGTFMoai9xEOlm X-Gm-Gg: AfdE7cm1ltljXNzBdPTrkg87d5Li2fua6VGQMXbiQT4AEOwG0pJAaXvf7fhEaKCHV0D u3CBMvEgetHipott+azozGFfleGcvX8+icxUjOnSjWDHQiWrG07o3zzWRYseztiiCA/VJkTJwmG s3AYfQHYqXz7eVT/hpN6PxIxJXsBuWu4JqlglFSBRXKXGpsQN1TZujIJ72z6Vn2+ukagRJmVqVr oOoBU1JIOrSeaYNxAaXFICM9lENxc0VDcQz3Cu+xYDeu7zM54sCy+onrrEz+0j4e0Lp0tPNmQqk Mw3hL3/+esjj+zTgKn+55tSowYDYKkz0PQXMo4rjkZSJn0FUfsAa88wkmVQ2HKo5MylQThFU8ep BQvliqm6FsIzZ4Nfm+P5z/4DLUJnA31M2CBB1moyW5aJxa9rV9ouX/WxGZu79NRqMsKJDMb3Xq4 IF+Duljp+aeqnYMR/mml3Twhg= X-Received: by 2002:a05:693c:3110:b0:304:56fc:775 with SMTP id 5a478bee46e88-30c07a0f2d0mr1157147eec.21.1781840447613; Thu, 18 Jun 2026 20:40:47 -0700 (PDT) Received: from localhost.localdomain ([104.28.158.91]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-30c06d6766fsm1973267eec.29.2026.06.18.20.40.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 18 Jun 2026 20:40:47 -0700 (PDT) From: Wang Lian To: sj@kernel.org Cc: akpm@linux-foundation.org, daichaobing@sangfor.com.cn, gutierrez.asier@huawei-partners.com, kunwu.chan@gmail.com, lianux.mm@gmail.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, npache@redhat.com Subject: Re: [RFC PATCH 0/6] mm/damon: Add mTHP-aware collapse/split with ARM SPE feedback Date: Fri, 19 Jun 2026 11:40:39 +0800 Message-ID: <20260619034039.35643-1-lianux.mm@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260619014707.9297-1-sj@kernel.org> References: <20260619014707.9297-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi SeongJae, Thank you for the thorough and thoughtful review. Your feedback on the x86 AF behavior was an important correction -- I'll address that and your other questions below. On Thu, 18 Jun 2026 SeongJae Park wrote: > This makes sense to me. I also agree this could caused the reported > problem. And this is a known limitation of DAMON. My suggestion for > straightforward workaround of this problem is, using 'age' information > of DAMON for better identification of the hot memory. Thank you for pointing out idle time percentiles [1]. We agree that 'age' helps differentiate frequently-accessed from occasionally-accessed regions, and it is a good workaround for many cases. However, age operates at region granularity, which is still at or above PMD level for THP-mapped memory. When only a few 4KB subpages within a 2MB THP are hot, age tells us the region has been accessed recently, but not which subpages are hot. The split decision needs sub-PMD information, which is what the SPE heatmap provides. That said, combining age with split could be valuable: split only regions that have been consistently hot (high age) AND have sparse sub-page access patterns. We will explore this. > > On ARM64, this is compounded by the hardware AF mechanism -- the AF > > is only set on a TLB miss. > > This makes sense to me. However, I don't get how this is contributing > to the problem. Could you please elaborate? The AF-on-TLB-miss behavior creates a second-order problem that directly exacerbates the overestimation. When DAMON's mkold path clears the PMD AF, it deliberately skips the TLB flush to minimize overhead. If the dense working set fits entirely within the L2 TLB (e.g., 16MB workload using 8 PMD entries on Kunpeng 920's 2048-entry L2 TLB), subsequent hardware accesses hit the valid, stale TLB entries directly. The hardware MMU never generates a page table walk, so the in-memory PMD AF stays 0. Consequently, DAMON sees `nr_accesses = 0` and assumes the region is completely cold, making it impossible to naturally track the sub-page usage shifts. When sporadic/noise accesses later hit other parts of this "seemingly cold" PMD and trigger an isolated TLB refilling, DAMON abruptly sees the whole 2MB as hot. This binary oscillation (completely blind vs. fully hot) is what drives the massive overestimation under THP. We confirmed this TLB-reach aspect empirically via our T1 test: 16MB THP (8 PMDs, 0.4% of L2 TLB reach) -> DAMON tracks 0 accesses (blind) 16GB THP (8192 PMDs, 400% of L2 TLB reach) -> DAMON tracks normally due to natural eviction > > x86 is not subject to this specific blindness under similar > > conditions. > > To my understanding on x86, same issue exists. If TLB hits, Aceessed > bit is not set, and DAMON shows it as unaccessed. Am I missing > something? You are entirely right, and I was wrong on this point. I re-checked the kernel source and verified that x86's ptep_test_and_clear_young() does NOT flush the TLB. Even ptep_clear_flush_young() on x86 deliberately skips the flush as a performance optimization (arch/x86/mm/pgtable.c:486-502). The same optimization architectural behavior exists on PowerPC and RISC-V. Therefore, both architectures are theoretically vulnerable to this stale-TLB blind spot under identical tightly-fit workloads. Our initial assumption was biased because T1 was only conducted on ARM64. We will reproduce the T1 setup on x86 to verify the exact behavior, and I will correct this claim in the v2 cover letter. Thank you for catching this mistake. > Nice! Asier was planning to do similar work in future. I think you > could collaborate to reduce unnecessary duplicates! Great to hear! We would be happy to collaborate with Asier. I'll reach out to him to coordinate our efforts. > I'd suggest making the name simpler and consistent to DAMOS_COLLAPSE, > though. Say, DAMOS_SPLIT ? Agreed. DAMOS_SPLIT is cleaner and fits the existing naming convention perfectly. Will rename in v2. > So you implemented a debugfs interface? That must be a nice approach > for PoC. But it may be difficult to be upstreamed as is. > > You could build a control plane that decides the exact address ranges > to split, and directly feed it to DAMOS using DAMOS address filter. The native perf event approach [3] aligns perfectly with our long-term Phase 2c plan, and we are highly interested in collaborating on it to eliminate the userspace daemon and debugfs bridge entirely. However, since native kernel-side SPE handling is a long-term item, we will follow your pragmatic alternative suggestion for v2: use DAMOS address filters or user_input quota goals [2] to feed the split decisions from userspace cleanly. This allows us to upstream the core infrastructure (mTHP target_order for collapse and the new DAMOS_SPLIT action) first. > Do you really need to khugepaged together, when you already have > DAMOS_COLLAPSE, and anyway you are running DAMON for hugepage splits? Excellent point. Running both concurrently on the same VMA introduces redundancy and heavy ping-pong effects. Option (b) is definitely cleaner: we will let DAMON handle both split and re-collapse decisions using its own access data. To make this robust in production environments where khugepaged is globally enabled, we will explore having the DAMOS_SPLIT path temporarily mark the target ranges (e.g., via a pseudo-VM_NOHUGEPAGE backing off mechanism) to prevent khugepaged from immediately undoing DAMON's work. > > - The rbtree entry TTL (30s) and signal threshold (1/10 of peak) are > > empirical defaults subject to further tuning. > > I don't fully understand this part. Could you please elaborate? Since ARM SPE samples hardware accesses instruction-by-instruction, the raw data is highly statistical and noisy. The TTL (30s) defines the lifecycle of our per-folio rbtree tracking entries. Entries not updated within 30 seconds are pruned to prevent stale tracking data from corrupting split decisions after a workload phase change. 30s is selected to comfortably outlive DAMON's aggregation intervals while keeping the rbtree memory footprint tightly bounded. The signal threshold (1/10 of peak) filters out the statistical sampling noise. Instead of treating any subpage with access > 0 as hot, the algorithm finds the peak access count inside the 2MB region and only marks sub-chunks with >= 1/10 of that peak as genuinely hot. On Kunpeng 920, this specific threshold successfully reduced false-hot subpage classifications from ~50% to <5%. We plan to make these parameters sysfs-configurable. > > - The ARM64 DAMON blind spot (WSS < L2 TLB reach) is a pre-existing > > hardware-MMU characteristic, not introduced by this series. Setting > > nr_accesses/min=0 serves as an effective workaround for the split path. > > I don't fully understand this, too. Could you please elaborate and > enlighten me? The blind spot creates an operational deadlock for the split infrastructure: 1. WSS < TLB reach -> All THP entries stay cached in TLB. 2. DAMON's page-table scan yields `nr_accesses = 0` globally. 3. A scheme requiring `nr_accesses.min = 1` never fires -> DAMOS_SPLIT is never invoked. 4. THPs remain unsplit -> WSS remains within TLB reach -> Loop returns to step 1. Setting `nr_accesses.min = 0` and `max = 0` breaks this deadlock. It forces DAMON to evaluate these seemingly "dead/cold" regions. Once the split handler invokes, it checks the ARM SPE telemetry (which captures data directly from the instruction pipeline, completely bypassing the MMU page-table AF limitation). If SPE reveals a sparse access heatmap, the split is executed. Once shattered into mTHP/base pages, the TLB reach drops, natural TLB misses resume, and DAMON's standard page-table tracking fully recovers. Thanks again for your guidance. The action items for v2 are locked in: 1. Rename DAMOS_MTHP_SPLIT -> DAMOS_SPLIT. 2. Drop debugfs in favor of DAMOS address filters / control plane. 3. Correct x86 AF behavior statements in the cover letter. 4. Coordinate with Asier on split/collapse unification. 5. Implement back-off to prevent khugepaged ping-pong under Option (b). Best regards, Wang Lian