From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751658AbdJVRTJ (ORCPT ); Sun, 22 Oct 2017 13:19:09 -0400 Received: from ale.deltatee.com ([207.54.116.67]:45556 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751466AbdJVRTH (ORCPT ); Sun, 22 Oct 2017 13:19:07 -0400 To: "Petrosyan, Ludwig" Cc: "Deucher, Alexander" , "linux-kernel@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-nvdimm@lists.01.org" , "Linux-media@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , "linux-pci@vger.kernel.org" , "Bridgman, John" , "Kuehling, Felix" , "Sagalovitch, Serguei" , "Blinzer, Paul" , "Koenig, Christian" , "Suthikulpanit, Suravee" , "Sander, Ben" References: <7f5e0303-f4ea-781a-8dec-74b30990d54f@desy.de> <1381807327.12461494.1508652825239.JavaMail.zimbra@desy.de> From: Logan Gunthorpe Message-ID: <85c98f79-e4b4-cbb5-feca-6944497c0c59@deltatee.com> Date: Sun, 22 Oct 2017 11:19:00 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <1381807327.12461494.1508652825239.JavaMail.zimbra@desy.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 68.147.191.165 X-SA-Exim-Rcpt-To: ben.sander@amd.com, Suravee.Suthikulpanit@amd.com, Christian.Koenig@amd.com, Paul.Blinzer@amd.com, Serguei.Sagalovitch@amd.com, Felix.Kuehling@amd.com, John.Bridgman@amd.com, linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Linux-media@vger.kernel.org, linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, Alexander.Deucher@amd.com, ludwig.petrosyan@desy.de X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: Enabling peer to peer device transactions for PCIe devices X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 22/10/17 12:13 AM, Petrosyan, Ludwig wrote: > But at first sight it has to be simple: > The PCIe Write transactions are address routed, so if in the packet header the other endpoint address is written the TLP has to be routed (by PCIe Switch to the endpoint), the DMA reading from the end point is really write transactions from the endpoint, usually (Xilinx core) to start DMA one has to write to the DMA control register of the endpoint the destination address. So I have change the device driver to set in this register the physical address of the other endpoint (get_resource start called to other endpoint, and it is the same address which I could see in lspci -vvvv -s bus-address of the switch port, memories behind bridge), so now the endpoint has to start send writes TLP with the other endpoint address in the TLP header. > But this is not working (I want to understand why ...), but I could see the first address of the destination endpoint is changed (with the wrong value 0xFF), > now I want to try prepare in the driver of one endpoint the DMA buffer , but using physical address of the other endpoint, > Could be it will never work, but I want to understand why, there is my error ... Hmm, well if I understand you correctly it sounds like, in theory, it should work. But there could be any number of reasons why it does not. You may need to get a hold of a PCIe analyzer to figure out what's actually going on. Logan