From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC1384457B5 for ; Wed, 22 Jul 2026 18:35:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.182 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784745344; cv=none; b=AExXa3wDwhhmRMlvfCXVmVD3BNbvvxw1mJynMacuQQMeY9dQXD22sZvCfo+hFWCAbdI1nS3PQqMpeZ1N9do27frcZ93rrlwJUDSPki64gbXpJO1ZlOozCXwe3kQUhZRVOhIAeRymnqNmcRUOYbW0f4l4/M/CQ9AJbIHXaUoUEm0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784745344; c=relaxed/simple; bh=nJ35d+eSRwcW0nsb3UMSMmMEzxzLr5EKcff45F9EGHk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KfZf+SS72SAOJ3EfEY060nm/0WADXA/SPxRdmE+JiIM72o65+W5ORP8PQHd3Z7C8Mk+gMuGQZPLzwh2sg3IUYzmZSZhzcCHOtE5UcsY/iKDejQ5qvEuAb1MXo9oFry7QVM659T5YIkuiZMKdk6w3245Cpgh2ZXKu/VPNNioiiXc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Sl9+Z9jT; arc=none smtp.client-ip=209.85.222.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Sl9+Z9jT" Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-9305a2121a1so961530085a.2 for ; Wed, 22 Jul 2026 11:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1784745336; x=1785350136; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=rb3wioIcpW4xOhPfXJEKXcvsIiP8tbJgl4SCMKfLhpI=; b=Sl9+Z9jTE3gnS9DBZMHJ8oL9nCIJiTS6xfiaRNCN1Brjme023rH/b3x54CvoPkhXj4 7ivZjJuASH8xM5ZzPaPsbQsDzWfuNZONrqDMJxZUuAeKY+UgTdLRpi6ZfznzezN1T6n2 5BsaiK9eo4QXKYwufyIT6vWmwUomne2hOTDcFljKgyZx6pW+X8RiMH9mLtI0evht5Y5x ymcJKZr0WmbeSOl+Rh90kY2D/F1j63+Q1PsS6a2nvH/ITkJZuokqY/lpkM7FzugOj7f1 25Uc0MuukQf1FglToPOk0pxL3slkaalpCIS+05ixVTHiENFZpCCkb0NwGHW4nXihuBje FAPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784745336; x=1785350136; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=rb3wioIcpW4xOhPfXJEKXcvsIiP8tbJgl4SCMKfLhpI=; b=R+ll4AS9uEZGn9bymhXNsVLyjEas8Ewzfhpc9B8u82lNAjJ9zWVGOEjYGbNoki5Uof mXIBkocD0uCalqX3nazA78b/R54xRxAO8fOWn75lcJikOPLk0cHS1An9vJYRh857DGho 2B80W44pBYj8tFeWljqGuKQ9MwW/EgIeql9jc/R5UnLMfttFVAU5WWCCRfjL42682UPS vwb6i5zQG7leGVXwpsiklhghtpOJqMYFEXTw2rHYv1xd+Yctng3EohNk3IlvE/XyzSE4 ARmQ9zoxGtsu09Xj/X+IOZWSw8pZ5KrDd8bRAMVbpFqon8pgrWz3jOv/cYdffSLq4PhI ionQ== X-Forwarded-Encrypted: i=1; AHgh+Rorl5cP9QeLHuMXj6Jfn4OEn0nR8jskFpRSxtbpqfvWsPfEA8eVuyqPyDbi4ZQzHsrVoKS9LfX/LCwz2CE=@vger.kernel.org X-Gm-Message-State: AOJu0YyOup/dRdPd3j+0QPIt59S6R/dXNTzKX3+nqGgXiUlEqmGPK/xz 1lcoQ0n+T80nee9cuQVZzDuU+sn21yj9cPG00+jOiCtvsuwtJkaMPWizoFPBpvcZAw8= X-Gm-Gg: AR+sD10m06dguKlB+nAefXRFzo6k1Nf0vWGk2WUhY2yoPM/vzsH5gjYUEh8P+gLKUQn 9ymdBgsTo9utXta1ouTeYtKzojnQx3jZHQxTABsceJxVHL88ygAFnIw0HgzCBfPIMzFTfXM9snR EbHWaSxAzTfZy1FDvz+oroWMLD9BtGFCaJeE++/5l0Sy9jx0G0E/bMDA9Tn7ulNVS+lDe0ClPt9 AEN3OKxCu593Tb7/sui67SQdYKXX2mlTjPzwXh65Gs9XENAYma5KIulEx9FZ4BDQNapVB5UEvYq IICzaKAaQzJUYlwXUhfgJHed5BQElZGKxnCh0tv/NqxAZNvLstnaz4MB8vL0a30BSQ5T484O859 K7AfCFEW7qf/RrpWXQPFSFEC+xaS4dEmIvrGaw6klJru8P8FJ8NEGyf4YsaAv8m8/V2homUQmFv S2ZMsXylhT3HFBmkPR8f1OWz9EuNBWoCSanqBGwbFVpH4b8QDE6PS8Y9fE4uL9bTs= X-Received: by 2002:a05:620a:4885:b0:92e:d5f6:2e71 with SMTP id af79cd13be357-930b3eea5damr2411579085a.14.1784745335608; Wed, 22 Jul 2026 11:35:35 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-907ba8b3e96sm27630216d6.1.2026.07.22.11.35.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jul 2026 11:35:35 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org Cc: nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, kernel-team@meta.com, djbw@kernel.org, vishal.l.verma@intel.com, dave.jiang@intel.com, alison.schofield@intel.com Subject: [RFC PATCH 8/8] dax/kmem: add dax/adistance to make private node adistance configurable Date: Wed, 22 Jul 2026 14:35:19 -0400 Message-ID: <20260722183519.935213-9-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260722183519.935213-1-gourry@gourry.net> References: <20260722183519.935213-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit For testing, it is useful to allow adjustments to node location in the tiering hierarchy - but this is presently a function of static config data (SRAT/HMAT). Private nodes have a contract of one owning service per node, meaning the registering service can set the adistance for that node without interfering with other hotplugged memory (no other service can hotplug). Extend kmem to have dax0.0/adistance when `private=1`, so that the targeted node's tier membership can be moved around. Signed-off-by: Gregory Price --- drivers/dax/kmem.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index c33dac5939edd..9fd50ef44d56b 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -51,6 +51,7 @@ struct dax_kmem_data { int numa_node; int state; struct mutex lock; /* protects hotplug state transitions and config */ + int adistance; /* node abstract distance */ bool dax_file; /* when set, cdev allows mmap */ struct mempolicy *policy; /* device-lifetime bind for dax-file mmap */ bool private; /* when set, memory is onlined as N_MEMORY_PRIVATE */ @@ -68,6 +69,27 @@ static struct memory_dev_type *kmem_find_alloc_memory_type(int adist) return mt_find_alloc_memory_type(adist, &kmem_memory_types); } +/* + * Re-register the node's memory type at given abstract distance, refined + * by the platform algorithm. This lets us move nodes to different memory + * tiers for testing and if the node does not have static perf data. + * + * Caller holds data->lock, and device must be in unplugged state. + */ +static int kmem_set_adistance(struct dax_kmem_data *data, int adist) +{ + struct memory_dev_type *memtype; + + mt_calc_adistance(data->numa_node, &adist); + memtype = kmem_find_alloc_memory_type(adist); + if (IS_ERR(memtype)) + return PTR_ERR(memtype); + clear_node_memory_type(data->numa_node, NULL); + init_node_memory_type(data->numa_node, memtype); + data->adistance = adist; + return 0; +} + static void kmem_put_memory_types(void) { guard(mutex)(&kmem_memory_type_lock); @@ -495,6 +517,45 @@ static ssize_t state_store(struct device *dev, struct device_attribute *attr, return len; } +static ssize_t adistance_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct dax_kmem_data *data = dev_get_drvdata(dev); + + if (!data) + return -ENXIO; + return sysfs_emit(buf, "%d\n", data->adistance); +} + +static ssize_t adistance_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len) +{ + struct dax_kmem_data *data = dev_get_drvdata(dev); + int adist; + ssize_t rc; + + if (!data) + return -ENXIO; + + rc = kstrtoint(buf, 0, &adist); + if (rc) + return rc; + if (adist <= 0) + return -EINVAL; + + guard(mutex)(&data->lock); + + /* adistance only reconfigures the device while it holds no memory. */ + if (data->state != DAX_KMEM_UNPLUGGED) + return -EBUSY; + + rc = kmem_set_adistance(data, adist); + if (rc) + return rc; + return len; +} +static DEVICE_ATTR_RW(adistance); + static const struct attribute_group dax_kmem_private_group; static ssize_t private_show(struct device *dev, struct device_attribute *attr, @@ -529,6 +590,11 @@ static ssize_t private_store(struct device *dev, struct device_attribute *attr, if (enable == data->private) return len; + /* When toggling private, reset custom adistance to default */ + rc = kmem_set_adistance(data, MEMTIER_DEFAULT_DAX_ADISTANCE); + if (rc) + return rc; + if (enable) { /* Add the per-service opt-in attributes. */ rc = sysfs_create_group(&dev->kobj, &dax_kmem_private_group); @@ -637,6 +703,7 @@ KMEM_PRIVATE_CAP_ATTR(ltpin, NODE_PRIVATE_CAP_LTPIN); /* Per-service opt-ins. Visibility toggled by 'private' control */ static struct attribute *dax_kmem_private_attrs[] = { + &dev_attr_adistance.attr, &dev_attr_reclaim.attr, &dev_attr_user_numa.attr, &dev_attr_hotunplug.attr, @@ -719,6 +786,7 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax) data->mgid = rc; data->numa_node = numa_node; data->state = DAX_KMEM_UNPLUGGED; + data->adistance = adist; data->np.owner = data; mutex_init(&data->lock); -- 2.53.0-Meta