From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA033AAF62 for ; Wed, 29 Jul 2026 16:25:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.184 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785342346; cv=none; b=VToiJJeUxKuApi7W0Lz3OVW+c8LgE3asAi/SlUAfUpY1TrLGqcCjAV/5ZJHKvcWiPeuA6o3ODp40RR+PePnujT5BM6yL1ioNTxxlMX9lW8kJOufHLb0BPYFmakqjq+JrfRuUOqIFKItCMD9an9km95WB3R7buX5HMyh+Tq8Fyu4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785342346; c=relaxed/simple; bh=/t+McpZzFsFMJOwXwMU3NjteDCv1vn3CUjKW1GuMAY0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tjBiIt6eEAk44X5OYpmd7rMLYMfTErTrYRPlOtW5vAIm5afrnnjnNm3tN8J3BZpkrDp+yF0y16StT5p+EPFUsVI0KobBZxeWMU7wjsVg+K4qw9UOkSfJ0W1WfsyvRfKdS6LWgMxygtYhIvi7zzuw+E1ug1LLV7ZKGYAOzlr31Pk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Jg/ruSG9; arc=none smtp.client-ip=91.218.175.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Jg/ruSG9" Date: Thu, 30 Jul 2026 00:25:27 +0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1785342339; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D3757SJdmqZVERewPVHqeMVYgjkUIDr+1uHBBRmZIMQ=; b=Jg/ruSG9Q19yLyn2YpylYetRoq5tibhBI8BE85ok2sKXB5DAqvloeml1mJlNo4dXbiqkAU xXGdTKTsRBUXJZ2Fss066KONse6v6hVUzLL0qqZcCKtmFwCTQDUox/va1uOxRT6R8znmNu vuQfCzvaBulJE/aDRiP3wdLt0N1uvX4= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Baoquan He To: Nhat Pham Cc: linux-mm@kvack.org, akpm@linux-foundation.org, chrisl@kernel.org, kasong@tencent.com, baohua@kernel.org, youngjun.park@lge.com, hannes@cmpxchg.org, yosry@kernel.org, david@kernel.org, shikemeng@huaweicloud.com, chengming.zhou@linux.dev, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH 00/11] mm, swap: dynamic cluster management for xswap devices Message-ID: References: <20260727135029.1059441-1-baoquan.he@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT On 07/27/26 at 08:48am, Nhat Pham wrote: > On Mon, Jul 27, 2026 at 6:50 AM Baoquan He wrote: > > > > This series implements step 3 of the incremental path proposed in [1]: > > a virtual swap device with dynamic cluster growth and shrink, backed > > only by zswap, no writeback yet. > > > > During the discussion in [2], Chris and Nhat debated xarray-based vs > > array-based cluster lookup for virtual swap. Chris argued for keeping > > the swap table and cluster_info in the same place with O(1) array > > indexing, and suggested that the kvmalloc grow approach here should > > provide the cluster management VS needs. He further proposed using > > the same grow technique for the future per-slot backend pointer array > > (vs_table), keeping a single coherent array layout throughout. > > > > This series therefore serves as the foundation: once the dynamic > > cluster management lands, Nhat's VS series can build the per-slot > > backend pointer, writeback, and rmap on top of it — without an extra > > xarray lookup in the cluster access path. > > Thanks Baoquan. I've given it some more thoughts over the past week or > so, to figure out a way to so that everyone would be happy with the > final results. > > To be perfectly clear, I'm not married with the xarray data structure. > I only care about use case aspect (as you have seen, I switch over to > swap table the moment I figure out a way to realize Kairui's design > proposal - it's way cleaner) - and I've seen that you have addressed a > lot of the design concerns I had that would hindered the use cases I > presented. If the vmalloc data structure outperforms the xarray, it > should *definitely* be the data structure we use when everything > settles. > > That just leaves the sequencing of our efforts :) > > My only fear with landing one piece at a time is that that we land > something that ended up not working for the entire big series in some > unforeseen cases, and we have to rip it out or rewrite a huge chunks > of it. Besides, without an actual use case, we can't really test its > correctness, and purported performance improvement benefits. Thanks for the thoughtful and honest reply, Nhat. I really appreciate the clarity — and the fact that this is a sequencing discussion, not a technical objection. Personally, I still lean toward the incremental approach. The cluster management layer feels like a well-defined, self-contained foundation that can be reviewed and tested independently, and landing it first would let the later change built on it with less effort. I think the overall direction is converging nicely. We've reached rough consensus on the big-picture architecture — decoupling backends, writeback, dynamic cluster grow/shrink, and per-device xswap/vswap — that's a solid foundation. Once we settle on the cluster_info implementation, everything else should fall into place quickly. From there we can move on to actual swap-tier usage of xswap/vswap and other extensions on top. However, if the consensus leans toward merging your xarray-based vswap series first and then switching the data structure as a follow-up optimization, I have no problem with that either. One concern: if we land the xarray-based approach first and then switch to a pointer array later, the cluster_info type change (e.g to struct swap_cluster_info * flat array or struct swap_cluster_info ** pointer array) will cause us touching those codes twice. Just something to weigh against the "land early" argument. I'm quite optimistic about how things are shaping up. It's been a long discussion but I think we're very close to something that everyone can get behind. Thanks Baoquan > > That's why so far I have bitten the bullet and sent out the whole > series (even if it's really large) with the core use cases > implemented. I understand that it is big, and is burdensome for > reviewers - but at least you know that the core use cases works - and > if you don't take my work for it, you can test with it and benchmark > it, etc. And with Kairui's proposal, we still maintain the old > machinery, which allows us to even merge with something a bit > suboptimal, then optimize later. FWIW, so far in my testing, the > numbers have been quite close, at least for zswap backend, and the > core concern (memory overhead) has been resolved. So it might actually > already be usable out of the box? :) > > This patch series has reached 700 LoCs, and I still have some > correctness concerns - I have commented one in another patch. It has > gotten more complicated now, which is no fault of you - we just need > more logic to handle the things xarray provided us out of the box, > with the hope that it would be more optimal. And that's fine. I just > think it would be better to land the xarray version first to unblock > users, then just slot your patch series on top as a follow-up > optimization. It would show the actual win of your data structure over > xarray data structure with concrete numbers :) Win-win? > > But thanks again for taking a stab at it. Seriously, you're awesome. I > want to re-iterate again - this is NOT a Nacked-by Nhat. Not with this > patch series, nor with the direction. I'm just trying to figure out > the sequencing of our efforts that would make everything work. I'll > spend sometimes on my own trying to synthesize this data structure > with vswap design too, to see if I can fish out more gaps, and to see > if I can clean things up in a way that would make the switch from > xarray to this new data structure as painless as possible. As well as > the other design points we have touched on in our last conversation - > it makes sense to me at the time, but I need to code some prototype > out and stare at it a bit more.