From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60AB01426C for ; Sun, 22 Dec 2024 17:03:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734887008; cv=none; b=mUUuv4Mz7iQHVakvY5o6Fecrkp1X3l7rmYD380/qpspRbeKUA1hUNZoFbrp7O7t2FmvJMoyBWHEa737XjjJFgso3MrA1GX84DwCh5ljM/tShsxhbZnFvA/wMhk/jiKJ4PN+UDFNw0on1ZiV+28jFeqNSIDs8sB26UnGAft4uHpo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734887008; c=relaxed/simple; bh=IflVJ6J9XzbXYmOLhpidkZjoAwBqPzc+2r/K98T3J/k=; h=From:Date:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NgPZaizat0THpzP/ikKx2fOH/lhRfn3DofRzmVrr+l8plVOBo/AA1ofGzcHPen33zu1F9VC1eabTQ62LyfdCwV8nGGVnjnzEI/B8ncCB59Bk/LOAHnQ/cIQiDIETF0O63227xsOmDH9b5tH7hkubdf7zFy7VshDvotgked4lsT8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=PCSMLRnr; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="PCSMLRnr" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-728e81257bfso2843421b3a.2 for ; Sun, 22 Dec 2024 09:03:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1734887007; x=1735491807; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=tcOdHdWrcugtyWcHoCfyk0R8M1W+7zds2dEJm6C+mDI=; b=PCSMLRnrhXM9s9SJPWjAm2Rz44mOJvh4Pdu/US/RYn9MtiBHaWgDHtFRAJl9205NlE CGiTmNOn6MUnk0CqOklW5By9NjUmR/HvAcIexrz5M/seFxwRY8Ag6p4Cg0+DDFa0iazX jHd+Oka3MsXUGKboIVGiaI67FFNZ/CCxNVxgMCiB/FcTCalMThYHTNV3Ey7Dn9CtoaaR ymOP/0k5SwDp3D2cxuH0B0lM151PbfvRSv/6edDp+8PQHDu+AguQxhxnCEEBlrKcscut 2miQ6o0qS0CiDSjaVQE6FxayIcHPd9MJRbiqKFFVHj5m7OADEtp+IwIs6p2IhS0W/uyC N6Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734887007; x=1735491807; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=tcOdHdWrcugtyWcHoCfyk0R8M1W+7zds2dEJm6C+mDI=; b=nw8kE7kZwggnRrO5+ojahN3jRPJ+6u+0KgdUzHnpzK+nb9oINzlPeN1vZC9NKdsTav /ITJBIvNiMbFJVjYoeW30queO6l2611aAaWmtN3WlQIEaRIg6q7iYWHVYYSz/ywsmpGN f8/soy3xF75QYX49Y6vzg3P0tJo1a/OC0Z/rqoSsyUNo1w9b21t5lKcXffqheo9WMLd8 lzNWom+Gcj3mTs8n287Uz8ZuDeTN8AFw1pPQHUISJpcgKXfpsR69Ljf23JKLiRXWEtue 21cDSvZOUphsAfwlWwzaJQ1erhi/CV4vHbZAbQf4wxOb9ToPKlzNs8MQlx8dFVtcxI3i r1rQ== X-Forwarded-Encrypted: i=1; AJvYcCUsRqchBivV8/iTh7tnCDgkLJwsMYVLKh3z3r2IiFI4/FQ/wTpMZrPBE3WWLRdIdbGYhaQh31B7B4RsOoQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyjHpfd1llp4EQTAfkuPWkPhao8pJ92SlQLwjuBPRXY75jbqvMY RAxonFC251sKp1mdd5sZLdStBrYhlYI63oKgXR9LpREAC0N43jTEdC8V0mHw3Jo= X-Gm-Gg: ASbGncvemCSmLhZVzp0SiZjFqucwaXE89q7FZwjV93PxToyTv/YyKdcRRyjzmvX7XLp DsNQc6M7BLa/h7efBAii3FpwJ0qyIEPMQx8GcFNk2cHNm8UCfFfyzU/CeuqXyZ71mgF73J4HXz4 CzEZPo8ZznryfpQHXziR6goOy1a/mZs2uu3LH67Qoob2ssXgS1fDHo/WfAt4d/JYbSvuOM6fhhx 4fcbpcVELqow08UFD4jRD4k8ADepDDhtwHyIjZmGMcI9+WOdQvHqEzA1YngsnnrVgSazAb0k83U g7k1YQA= X-Google-Smtp-Source: AGHT+IHyDRNtmt1EM6O/D9w+dg7/VuoeBXOzeAbhhFbYiVHsrJa0pCNJI4n5WSuy+0/pLWrop6UFLQ== X-Received: by 2002:a05:6a00:2449:b0:728:e906:e45a with SMTP id d2e1a72fcca58-72abdecdbfdmr16342619b3a.24.1734887006789; Sun, 22 Dec 2024 09:03:26 -0800 (PST) Received: from gourry-fedora-PF4VCD3F ([75.167.163.61]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad831160sm6343053b3a.59.2024.12.22.09.03.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Dec 2024 09:03:26 -0800 (PST) From: Gregory Price X-Google-Original-From: Gregory Price Date: Sun, 22 Dec 2024 12:03:22 -0500 To: "Huang, Ying" Cc: Hyeonggon Yoo , Joshua Hahn , "gourry@gourry.net" , kernel_team@skhynix.com, 42.hyeyoo@gmail.com, "rafael@kernel.org" , "lenb@kernel.org" , "gregkh@linuxfoundation.org" , "akpm@linux-foundation.org" , Honggyu Kim , Rakie Kim , "dan.j.williams@intel.com" , "Jonathan.Cameron@huawei.com" , "dave.jiang@intel.com" , "horen.chuang@linux.dev" , "hannes@cmpxchg.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "kernel-team@meta.com" Subject: Re: [External Mail] [RFC PATCH v2] Weighted interleave auto-tuning Message-ID: References: <20241219191845.3506370-1-joshua.hahnjy@gmail.com> <3682b9cf-213c-497d-ab81-f70e1a785716@sk.com> <87r060i3nn.fsf@DESKTOP-5N7EMDA> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87r060i3nn.fsf@DESKTOP-5N7EMDA> On Sun, Dec 22, 2024 at 03:21:32PM +0800, Huang, Ying wrote: > Hyeonggon Yoo writes: > > > On this server, ideally weighted interleaving should be configured > > within a socket (e.g. local NUMA node + local CXL node) because > > weighted interleaving does not consider the bandwidth when accessed > > from a remote socket. > > If multiple sockets are considered, what is the best behavior? > > The process may be cross-socket too. So, we will need to use > set_mempolicy() to bind tasks to sockets firstly. Then, it may be > better to use per-task weights. > If we want to revisit this, we might be able to make task-local weights work without a new syscall, but the use case was not clear enough which is why it was soft-nak'd originally. vma-local weights are arguably more usable, but require the task to be numa-aware and probably require a new mempolicy syscall because mbind has no remaining arguments. recall my original testing results from stream: https://lore.kernel.org/linux-mm/20240202170238.90004-1-gregory.price@memverge.com/ Stream Benchmark (vs DRAM, 1 Socket + 1 CXL Device) Default interleave : -78% (slower than DRAM) Global weighting : -6% to +4% (workload dependant) Targeted weights : +2.5% to +4% (consistently better than DRAM) Just some context ~Gregory