From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 869B13B19A8 for ; Tue, 9 Jun 2026 16:17:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781021867; cv=none; b=TnhkUEBpgd9IugS/AUZWHfSHpun6wWLcEcomT23h87sef2FjB6rJSm5qd6NQQx1yeJmkgYmkdhwhd0FCLA5LTfLl+/mz4JDzpFWVD23wZXPTPYzXaE3GdtWgsKWLnE/LsQOfOYpxyTQ8IiS2kZUSE6XlbQ9USmHiOHI8v2WXbM4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781021867; c=relaxed/simple; bh=lUR9hrdKbI6QRrf+wGC1f7/B+yQ7JmVqvxZ0haw9tgw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WkSiVHm8QdBWzRY5uBOH56Py8icu/wKcynqs/4eSm+hAGK8dpShWJSBf/9v4sKKsH2GtyQe38VEq4OH3EfxwbDG3wR68u+ik+2vp0yBnG25tj87iudJiFIoLOEnSG9eOhjxbrpagEqgoDmozB5oVM7QtdKRs9l4DOAR3zkvNhpk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IMf55VfW; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IMf55VfW" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-490bb83a3f6so47713635e9.0 for ; Tue, 09 Jun 2026 09:17:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781021865; x=1781626665; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=VcNntx/NGAvfH+oGg7l4M6S28fmu1J5RocdEzpCM4Lw=; b=IMf55VfWnv24bF3NAWMSpVzhAOXmYPT6yZ2L4skIUIkjkgHXYZpdyqTqyQSF4ns53y CUJua15z6nD0Ps+p538RUv4YxiCzxV/QvGoeRerzllSRWjSb780w4677iQYgv0SvBTRz SLlNojDaK/61TVL+JgNGrGnjQ3XhHhVYTxxeV2vy37PbTqH8edcRGpP+kZ8Et3AJQJWV 5g2PejWh2xcTqo5cmFtA4ikBDhm2CLmlRikzuRcqJucc+cSwPrevbmJ6RlVa+tohBEoG Q2IBqNhwCTrIGQr1Znw13YCnXquS1u4IDR5X8h70/NRGBxVexkeiw4d+f49dYOFd3Tfw 9FGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781021865; x=1781626665; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VcNntx/NGAvfH+oGg7l4M6S28fmu1J5RocdEzpCM4Lw=; b=qR0voGefSlkuGQ8A1BRLbTjgGWhLfQA57pePwUWIbZ3+LuKA2zlDqePUVUZw/cQXtj Bp4m0L5MSOY6NPYZvmrNIGoOglD5Ljsw1z1TDkR5eNooUCl1b5X5Q5QSci8X0MSX5Odi meKNKi6aBHEhGRgK9Y/cbvDJ6vf8TcJOvAzBSzOQUh96lakqMz63AWEU53IgJAoZdnp/ HLwEQr4+SzstGX8zD+qrEjLT2K4q0sfw+EYG0Db5A5hQ3LdbJ8XwlrmcDptBOIYXB815 w8SVsPeM9zAQBc3Jmqp0j9u55C61pQr7opxed79RkX67/PJuPsCct/+hQrWv4wnEypYh 895Q== X-Forwarded-Encrypted: i=1; AFNElJ/y7oD6e6nw3vbRrpFsbunQdbxgBjTKYE5+PD9SAtbwPq4QEqR+bvziJKiLosuC+Wr9q29+zMDpfMJHN0U=@vger.kernel.org X-Gm-Message-State: AOJu0Yww5QXHVj9/7jI+lpM/enEKi9vqsqjyi/DPyNsTqaYe9n6z+s9J bEkaZAkcQTjzbIUVNMU7vixHO8Z23QanhlGzk7eBFK8UKZ1HEbzigr6X X-Gm-Gg: Acq92OH6brQuAztcbxeNua9PhiEI+Rb9Us2AiXttUuKL9HAHnD2yZ41DAytENTs7ODo Rwu3tsfTVYI2TCBCQ/Hh89l5tab8UDpyAoki+2lDQphCJxx+tTET0p4IqS5UewubHBMWa1sRAZA qsve62jua5wMku2sFkaternl8AcervO9WcCnB/aH4n29cjn9n5jG1ApP1Zv5BJmj6kqH8bVskLq ySp5kIwNKMCUvMcCbYc902LKoSyWoBFCxhuLWE4VAvXWVFGuXaZm5/CXGe2gMK6yFOnnjWObGB6 TugFfjxylbuCS6jLODoJGJEDcuiHLb8Wr4Dh4BRd6GxBHi2FaMYGou3xXQrAxDIxK+vFNyWhVPO wA572GMYkWf9b4Bd8L2hO+Htglf9mVjgw/I8HoqQHpTVRsnvpBCAzMf4cO2RMDMuucCkpv2fyR5 jW4PFzpFpy47mWjHYqonUfNPeu1PmxFWV5e6oz5lE0AL5MQ/9DzP8OuZ7yrNOHleJ/YF7az5UrO +2BAVQkAL9uNcjoR5p67ssVtjgr6uy07M0= X-Received: by 2002:a05:600c:5292:b0:490:688b:f9f8 with SMTP id 5b1f17b1804b1-490d722e5ffmr47179415e9.27.1781021864447; Tue, 09 Jun 2026 09:17:44 -0700 (PDT) Received: from localhost (pat-125-253.wlan.net.ed.ac.uk. [192.41.125.253]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3fd502sm492555605e9.11.2026.06.09.09.17.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2026 09:17:43 -0700 (PDT) Date: Tue, 9 Jun 2026 17:17:42 +0100 From: Karim Manaouil To: "Garg, Shivank" Cc: akpm@linux-foundation.org, david@kernel.org, kinseyho@google.com, weixugc@google.com, ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, willy@infradead.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, ziy@nvidia.com, matthew.brost@intel.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, byungchul@sk.com, gourry@gourry.net, ying.huang@linux.alibaba.com, apopple@nvidia.com, dave@stgolabs.net, Jonathan.Cameron@huawei.com, rkodsara@amd.com, vkoul@kernel.org, bharata@amd.com, sj@kernel.org, rientjes@google.com, xuezhengchu@huawei.com, yiannis@zptcorp.com, dave.hansen@intel.com, hannes@cmpxchg.org, jhubbard@nvidia.com, peterx@redhat.com, riel@surriel.com, shakeel.butt@linux.dev, stalexan@redhat.com, tj@kernel.org, nifan.cxl@gmail.com, jic23@kernel.org, aneesh.kumar@kernel.org, nathan.lynch@amd.com, Frank.li@nxp.com, djbw@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 6/7] drivers/migrate_offload: add DMA batch copy driver (dcbm) Message-ID: <20260609161742.phipg7eid3yy4j7f@wrangler> References: <20260428155043.39251-2-shivankg@amd.com> <20260428155043.39251-14-shivankg@amd.com> <20260609000009.dvh7gfbmiz76eliu@wrangler> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Tue, Jun 09, 2026 at 01:01:21PM +0530, Garg, Shivank wrote: > > > On 6/9/2026 5:30 AM, Karim Manaouil wrote: > > [You don't often get email from kmanaouil.dev@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] > > > > In my opinion the driver should be at drivers/dma/dcbm, other DMA drivers > > in the same directory like idxd and sdxi can just export a struct > > dma_migrator. It makes it easy to have everything in the driver's own > > directory. For example, idxd has some internals headers as well as an > > already existing dmaengine implementation in drivers/dma/idxd. > > > > PS. I am writing an implementation of this based on idxd. I'll try to > > post it soon with numbers. > > > > You can add > > Reviewed-by: Karim Manaouil > > > > Hi Karim, > > Thanks for the review. > > I'm excited to see your idxd based implementation. > > For plain memory to memory copy, which is all migration needs right now, > the public dmaengine API is good enough, and idxd/sdxi/ptdma ... can > all work through it. I'm curious which features you're planning to use > from idxd internal headers that dmaengine can't provide? Hi Shivank, I'm still testing, but the initial implementation I wrote with DMAEngine had too much overhead because of the sgtable allocations and the conversion between kernel scatterlists to device descriptors. So I entirely bypassed the DMAEngine API by directly passing the folios lists to the driver. I know it depends on the use case. If you just want to offload with no latency requirements, then DMAEngine is fine, but if the goal is to achieve high bandwidth with minimal latency, then it's a problem. Another example, if you have to do several independent copies of 256 or 512 4KiB pages in a short period of time, there will to much stress on sgtable allocations. Another problem for low latency is DMA mapping. Anyway, I need to collect more numbers. I will try to share my insights with idxd asap. > Also, my understanding is that driver/dma houses DMA providers, > whereas DCBM would be consumer of DMAengine APIs. So is that right > place for it? I think you're right. It makes more sense when viewed from that perspective. In fact, maybe at some point we will have some more generic memory offload infrastructure to support sharing, load balancing across several acceelrator instances, numa-awareness, handling batching for drivers and asynchronous offload (to workqueues with completions). -- ~karim