From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3EF219AD48 for ; Tue, 28 Jan 2025 13:20:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738070438; cv=none; b=FDatr7kOewWtmXG0JR13u6e1nsIs22AMWslzA3IDtaCJrOxHntKKAK0rZNFK+dTJY23cw3A2G+fpCZ8gQ8p6hzTaLzrtOOym5YxQ4JvYL+injVLsnArXXvTp8cTUWUB6iXC2Pdxhg2c2ToX4Ft0QrobYSBrwtGCh/h4sYoEGXs0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738070438; c=relaxed/simple; bh=U4suS5LTaOVNoqADNqMm3cSIM2A9q2pA52JYTTgCklY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jQsi01ySg7rPUl4kKLRbRUdYGJ7wu4wEr9Fnicfv5eFYa/1BSrZWbpACLsKnf7RqbEuBc44JnDFGdVadxhb5mO23mROGcmGySV922t9ItmV5h1qLonjng0QhA1ynWwPiTy0amL/Fw+xD0grKDtWxkDqYomiiP6KXZXfhM606XXo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=Y/iX5bAQ; arc=none smtp.client-ip=209.85.222.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="Y/iX5bAQ" Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-7b6ed0de64aso517980485a.0 for ; Tue, 28 Jan 2025 05:20:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1738070435; x=1738675235; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=GOWXoYYRnWMsoeKK+VVuf/lMuv2V7GO+mfTTLFXfUSE=; b=Y/iX5bAQEEByiwQNOJDftwrQx7wHO5Y9UA51pkGWXkj2Mk83CmggJLSEd8babLOV8V nVlKo7Wns1lCfcty0PWvoFrTXhTdsasEBN3dPYCngRp2hxwOwTKvnQ5TiSV3u109RY3W bOWQsvooFLysWzmORdltM2sACdf2ED1vJs4gJueCSgiW8OxSLiVU2XqEbYZi6BbbMJXj pUz0qZYrwOBJ8fCSWH0vj++MlDlDfljZ9LCMPHIoCZsmsfTzCBvUSYaELvTA2gitzqpX GrXl9drRu0M6N5D+z7h4AK2ssMcwZ1baH3PleulmHmLYnpTHJzV5RftTvqdp0g2oJj6+ ZtMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738070435; x=1738675235; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GOWXoYYRnWMsoeKK+VVuf/lMuv2V7GO+mfTTLFXfUSE=; b=m9SK+VeumhuVo55y31Alis0LOvD5R7+xQ5tbBLCYYLrfShbToGxfmstXw9tASiW3uG j1UHXmo1/DXT7zUlwe1/U5zv+b/VPr+xFH86SB98xxw27ktLL3xF2XE/CCj8YZqvIiWI l5e6380mMsd88aAuoGPOcWWmYAykedK91EaBvcn3WCs4tDkih34OiwOhvvrkNul5YmiU oj5q/MLwQArRxeydzKH+Yij0x1+AuQqfUu/jPeYH9h29y1uaUa12HcOzVrh30z4TcAag Kd6PqSKNW0DnFTqCiyPHPSlf+cHrNegWP2lKX+icF2faQvWvP0E/aRXV7Ak5gDWcLqI7 ByRA== X-Forwarded-Encrypted: i=1; AJvYcCV1jTWwsxCIpWi4w49x1xfTYdxjj2D2/VAIfNaZAwIwlushmANHM2+NRUm27tzrQiHbXAf3IMeyoT2ak1g=@vger.kernel.org X-Gm-Message-State: AOJu0YxWHZ0C4IWZHFZI0QsxQ3JNcAZLlr3I4aEe51sbiFeKMKkMqXz3 X98vzzN1LNjA/lilZmM9Sm7jgpP0N/kraYN71rBPNVtmUHtTmnvsKeeISMp/2Y0= X-Gm-Gg: ASbGncv0NrLIC+GBOQCkqT9N/ZQ354Ns6ThespNRWeaeN3hsSdYas/dnaHN7Wd5oazn eBjRnDBRmiCb6qtkZ1jqkkIV9V7XJ+P+PHmcx6LfSc7DjkV2fMdkB38EC8G+lktrMM6MTGD+YpA xQbV8NZCHa+NGboKlAd5Ux8KZ0JDg48lOZ5wZaSBH4X+ttK0dquDBb5lG4sHgLKp5bwiEz95H41 drFLwoZKO1KyozeXCFFh3JyygDEBZS0vgRTSI/VRhuKCBVtSE11vHAhVFbHsl/NCeoUucgjLH8h LV2G/5A/EnMqYRcJGpSbyZU6vbVBn5lUCzebpUAWsI07wVIpHOzgrony1Tu4imKD/0/wRljHO+M = X-Google-Smtp-Source: AGHT+IFfqsRD0FOVkdIg887UXRm977J4ky26yoqHXaGeXqELEj2EneWq+CuXUYKkdzvufPHaZSVAIg== X-Received: by 2002:a05:620a:3910:b0:7b7:142d:53a8 with SMTP id af79cd13be357-7bff3fb4946mr402618885a.19.1738070435489; Tue, 28 Jan 2025 05:20:35 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-68-128-5.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.128.5]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7be9aeedc68sm512887085a.74.2025.01.28.05.20.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 05:20:34 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1tclW6-00000007TbE-14aw; Tue, 28 Jan 2025 09:20:34 -0400 Date: Tue, 28 Jan 2025 09:20:34 -0400 From: Jason Gunthorpe To: Thomas =?utf-8?Q?Hellstr=C3=B6m?= Cc: Yonatan Maman , kherbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, GalShalom@nvidia.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-tegra@vger.kernel.org Subject: Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages Message-ID: <20250128132034.GA1524382@ziepe.ca> References: <20241201103659.420677-1-ymaman@nvidia.com> <20241201103659.420677-2-ymaman@nvidia.com> <7282ac68c47886caa2bc2a2813d41a04adf938e1.camel@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7282ac68c47886caa2bc2a2813d41a04adf938e1.camel@linux.intel.com> On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > How would the pgmap device know whether P2P is actually possible > without knowing the client device, (like calling pci_p2pdma_distance) > and also if looking into access control, whether it is allowed? The DMA API will do this, this happens after this patch is put on top of Leon's DMA API patches. The mapping operation will fail and it will likely be fatal to whatever is going on. get_dma_pfn_for_device() returns a new PFN, but that is not a DMA mapped address, it is just a PFN that has another struct page under it. There is an implicit assumption here that P2P will work and we don't need a 3rd case to handle non-working P2P.. > but leaves any dma- mapping or pfn mangling to be done after the > call to hmm_range_fault(), since hmm_range_fault() really only needs > to know whether it has to migrate to system or not. See above, this is already the case.. > One benefit of using this alternative > approach is that struct hmm_range can be subclassed by the caller and > for example cache device pairs for which p2p is allowed. If you want to directly address P2P non-uniformity I'd rather do it directly in the core code than using a per-driver callback. Every driver needs exactly the same logic for such a case. Jason