From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S967487AbdAFQq0 convert rfc822-to-8bit (ORCPT ); Fri, 6 Jan 2017 11:46:26 -0500 Received: from hqemgate15.nvidia.com ([216.228.121.64]:2387 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754237AbdAFQqQ (ORCPT ); Fri, 6 Jan 2017 11:46:16 -0500 X-PGP-Universal: processed; by hqnvupgp07.nvidia.com on Thu, 05 Jan 2017 20:42:27 -0800 Subject: Re: [HMM v15 13/16] mm/hmm/migrate: new memory migration helper for use with device memory v2 To: =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , , , References: <1483721203-1678-1-git-send-email-jglisse@redhat.com> <1483721203-1678-14-git-send-email-jglisse@redhat.com> CC: John Hubbard , Evgeny Baskakov , Mark Hairgrove , Sherry Cheung , Subhash Gutti , Cameron Buschardt , Zi Yan , Anshuman Khandual From: David Nellans Message-ID: Date: Fri, 6 Jan 2017 10:46:09 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <1483721203-1678-14-git-send-email-jglisse@redhat.com> X-Originating-IP: [10.20.174.107] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL103.nvidia.com (172.20.187.11) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/06/2017 10:46 AM, Jérôme Glisse wrote: > This patch add a new memory migration helpers, which migrate memory > backing a range of virtual address of a process to different memory > (which can be allocated through special allocator). It differs from > numa migration by working on a range of virtual address and thus by > doing migration in chunk that can be large enough to use DMA engine > or special copy offloading engine. > > Expected users are any one with heterogeneous memory where different > memory have different characteristics (latency, bandwidth, ...). As > an example IBM platform with CAPI bus can make use of this feature > to migrate between regular memory and CAPI device memory. New CPU > architecture with a pool of high performance memory not manage as > cache but presented as regular memory (while being faster and with > lower latency than DDR) will also be prime user of this patch. Why should the normal page migration path (where neither src nor dest are device private), use the hmm_migrate functionality? 11-14 are replicating a lot of the normal migration functionality but with special casing for HMM requirements. When migrating THP's or a list of pages (your use case above), normal NUMA migration is going to want to do this as fast as possible too (see Zi Yan's patches for multi-threading normal migrations & prototype of using intel IOAT for transfers, he sees 3-5x speedup). If the intention is to provide a common interface hook for migration to use DMA acceleration (which is a good idea), it probably shouldn't be special cased inside HMM functionality. For example, using the intel IOAT for migration DMA has nothing to do with HMM whatsoever. We need a normal migration path interface to allow DMA that isn't tied to HMM.