From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752113AbdI0KlE (ORCPT ); Wed, 27 Sep 2017 06:41:04 -0400 Received: from mail-eopbgr20076.outbound.protection.outlook.com ([40.107.2.76]:63488 "EHLO EUR02-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750703AbdI0KlB (ORCPT ); Wed, 27 Sep 2017 06:41:01 -0400 From: Bharat Bhushan To: Bharat Bhushan , Robin Murphy , Marc Zyngier , "robh+dt@kernel.org" , Mark Rutland , "will.deacon@arm.com" , "oss@buserror.net" , Gang Liu , "devicetree@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "catalin.marinas@arm.com" Subject: RE: [PATCH] RM64: dts: ls208xa: Add iommu-map property for pci Thread-Topic: [PATCH] RM64: dts: ls208xa: Add iommu-map property for pci Thread-Index: AQHTIjs8I4mAanCpD0a4/V9TcR3UWKKeM5KAgAAItcCAAAzwgIAAB65QgAF79KCAABE2gIAGKyMggCKr7eA= Date: Wed, 27 Sep 2017 10:40:57 +0000 Message-ID: References: <1504171424-15536-1-git-send-email-Bharat.Bhushan@nxp.com> <06c57ca4-edaa-df26-b1d9-1906141c5065@arm.com> <10be1849-8071-64fc-5ee2-b6e429c64b9e@arm.com> <901dee07-d1ee-63cc-59a0-bdf37cd7d937@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=bharat.bhushan@nxp.com; x-originating-ip: [192.88.169.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;AM5PR0401MB2548;6:iigxNRJpMAYjcYrFRDimg/lRSaniervhrQAbbwPPJROl1V0c40SpDT9r8z/MlJF2U8FcI5b6yazgzqYDhsKcaRQnVfhmNgGZOCMjdPs0vV6MlJU2k9d3m1iqDJip76Yyly9FzeBHj/MO8k9H0beO56WkQh7WsEFRGxxhozidytjHXd/jxv3fJRuDX3RVzKi5j+A2y/zJghJYBCsjlewf+GP7uKdOtESqQw/4/e2w/HofXkYTGxVEPyeWgSKvicNz0fOxu6UWh2h19NK5zK04SOrgKQ2L98Z7V9VlnEzw5YwpBH59LrYalAXQahGhkN7kMn8M9KUeIvlke/itCSwf5A==;5:JHZWWFADJkLAl7glD6D5YoeFlHwKgROw6OLK3HR6byR82CIfzYx4qh3N6ZEFTvrQ0mazX4XkGZF7nILTUErLVh6BzvIu5pIpN1P2zbd+jX+2bZd25Y3jHhW99v8ZVwnz/br7X5usDpkznBAeqdA1og==;24:lT0z0uRJZ5bHwVhoiX+bHdH5mjUNwMJceyaDwcQrX3KQz4Q2Al2j6pxZrx1k1o5inzCPHJ9cWiT7cHxcYLLyXfV8lO8rGof90xOZnheeY3w=;7:VAUWtpMVOQANdN9zJQEBzu9FxL5DOE3iZLyWh1YHLKTHFgueZGJPsYWoub92GEgbIXstsirJ8aDQvcTB1S22d1grxMuitIN4tO+hOEYO31Zfy0IsfbT+3dMxy2SJVBwvYPAT5Jofi8b0ZUzksN/5QpDHrd8mN6NvDuCjWyRfVSV9pHLcPz3/W9zh6QTb0EeP9ddLvu3pUk6RjjHmy9ZRdE4jHM3ORqEHbt/igHPYSJ8= x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI;SCL:-1;SFV:NSPM;SFS:(10009020)(6009001)(346002)(376002)(199003)(377454003)(189002)(24454002)(13464003)(76176999)(86362001)(478600001)(6506006)(25786009)(110136005)(106356001)(105586002)(6436002)(101416001)(102836003)(229853002)(54356999)(50986999)(6116002)(66066001)(68736007)(3846002)(2906002)(316002)(33656002)(3660700001)(3280700002)(2201001)(7736002)(53936002)(6246003)(7416002)(305945005)(53546010)(5250100002)(2501003)(97736004)(2950100002)(14454004)(74316002)(5660300001)(7696004)(93886005)(189998001)(2900100001)(8936002)(8676002)(81156014)(9686003)(81166006)(99286003)(55016002)(921003)(1121003);DIR:OUT;SFP:1101;SCL:1;SRVR:AM5PR0401MB2548;H:AM5PR0401MB2545.eurprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; x-ms-office365-filtering-correlation-id: 565148b9-6138-4424-7d00-08d505943cf8 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:AM5PR0401MB2548; x-ms-traffictypediagnostic: AM5PR0401MB2548: x-exchange-antispam-report-test: UriScan:(180628864354917)(9452136761055)(185117386973197)(258649278758335); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123558100)(20161123564025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:AM5PR0401MB2548;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:AM5PR0401MB2548; x-forefront-prvs: 04433051BF spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Sep 2017 10:40:57.8541 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0401MB2548 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v8RAfA1i007953 Hi Robin, > -----Original Message----- > From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel- > owner@vger.kernel.org] On Behalf Of Bharat Bhushan > Sent: Wednesday, September 06, 2017 12:47 PM > To: Robin Murphy ; Marc Zyngier > ; robh+dt@kernel.org; Mark Rutland > ; will.deacon@arm.com; oss@buserror.net; Gang > Liu ; devicetree@vger.kernel.org; linux-arm- > kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > catalin.marinas@arm.com > Subject: RE: [PATCH] RM64: dts: ls208xa: Add iommu-map property for pci > > Hi Robin, > > > -----Original Message----- > > From: Robin Murphy [mailto:robin.murphy@arm.com] > > Sent: Friday, September 01, 2017 4:29 PM > > To: Bharat Bhushan ; Marc Zyngier > > ; robh+dt@kernel.org; Mark Rutland > > ; will.deacon@arm.com; oss@buserror.net; > Gang > > Liu ; devicetree@vger.kernel.org; linux-arm- > > kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > > catalin.marinas@arm.com > > Subject: Re: [PATCH] RM64: dts: ls208xa: Add iommu-map property for > > pci > > > > On 01/09/17 11:13, Bharat Bhushan wrote: > > > > > > > > >> -----Original Message----- From: linux-kernel-owner@vger.kernel.org > > >> [mailto:linux-kernel- owner@vger.kernel.org] On Behalf Of Bharat > > >> Bhushan Sent: Thursday, August 31, 2017 4:53 PM To: Marc Zyngier > > >> ; robh+dt@kernel.org; Mark Rutland > > >> ; will.deacon@arm.com; oss@buserror.net; > > Gang > > >> Liu ; devicetree@vger.kernel.org; > > >> linux-arm-kernel@lists.infradead.org; linux- > > >> kernel@vger.kernel.org; catalin.marinas@arm.com Subject: RE: > > >> [PATCH] RM64: dts: ls208xa: Add iommu-map property for pci > > >> > > >> > > >> > > >>> -----Original Message----- From: Marc Zyngier > > >>> [mailto:marc.zyngier@arm.com] Sent: Thursday, August 31, 2017 > > >>> 4:20 PM To: Bharat Bhushan ; > > >>> robh+dt@kernel.org; > > >> Mark > > >>> Rutland ; will.deacon@arm.com; > > >> oss@buserror.net; > > >>> Gang Liu ; devicetree@vger.kernel.org; > > >>> linux-arm-kernel@lists.infradead.org; linux- > > >>> kernel@vger.kernel.org; catalin.marinas@arm.com Subject: Re: > > >>> [PATCH] RM64: dts: ls208xa: Add iommu-map property for pci > > >>> > > >>> [Fixing Mark's address...] > > >>> > > >>> On 31/08/17 11:41, Bharat Bhushan wrote: > > >>>> > > >>>>> -----Original Message----- From: Marc Zyngier > > >>>>> [mailto:marc.zyngier@arm.com] Sent: Thursday, August 31, 2017 > > >>>>> 3:02 PM To: Bharat Bhushan ; > > >>>>> robh+dt@kernel.org; ark.rutland@arm.com; will.deacon@arm.com; > > >>>>> oss@buserror.net; Gang > > >>> Liu > > >>>>> ; devicetree@vger.kernel.org; linux-arm- > > >>>>> kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > > >>>>> catalin.marinas@arm.com Subject: Re: [PATCH] RM64: dts: > > >>>>> ls208xa: Add iommu-map property for pci > > >>>>> > > >>>>> On 31/08/17 10:23, Bharat Bhushan wrote: > > >>>>>> This patch adds iommu-map property for PCIe, which enables > > SMMU > > >>>>>> for these devices on LS208xA devices. > > >>>>>> > > >>>>>> Signed-off-by: Bharat Bhushan --- > > >>>>>> arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi | 4 ++++ 1 file > > >>>>>> changed, 4 insertions(+) > > >>>>>> > > >>>>>> diff --git > > >>>>>> a/arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi > > >>>>>> b/arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi index > > >>>>>> 94cdd30..67cf605 100644 --- > > >>>>>> a/arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi +++ > > >>>>>> b/arch/arm64/boot/dts/freescale/fsl-ls208xa.dtsi @@ -606,6 > > >>>>>> +606,7 @@ num-lanes = <4>; bus-range = <0x0 0xff>; > > >>>>>> msi-parent = <&its>; + iommu-map = <0 > &smmu 0 > > 1>; /* This > > >>>>>> is fixed-up by > > >>>>> u-boot */ > > >>>>> > > >>>>> What does this do when your version of u-boot doesn't fill this > > >>>>> in for > > >> you? > > >>>> > > >>>> Good question, frankly I have not thought of this case before. > > >>>> But if we pass length = 0 in above property then no fixup happen > > >>>> with happen with older u-boot. In this case > > >>>> of_iommu_configure() will return NULL iommu-ops and it switch to > > >>>> swio-tlb. Will that work? > > >>> I really don't like this. You rely on having invalid data in the > > >>> DT, and that seems just wrong. > > >>> > > >>> Why can't u-boot just generate that property, and we leave the DT > > >>> alone? > > >> > > >> We do not have smmu phandle allowing uboot to generate this. > > >> > > >>> Or even better, you provide the right information for the few > > >>> boards that are based on this SoC, not relying on u-boot for > > >>> anything that is in the kernel tree? > > >> > > >> On our platforms we have a h/w table which converts RID->Device-Id. > > >> I will check what will happen if that table is not initialized, can > > >> RID be equal to device-id is that case. If that will be allowed > > >> than we can give right information that will work with u-boot not > > >> updating this property. > > > > > > U-boot uses a stream-id allocator and programs the h/w mapping table > > > (rid to sid mapping table). Also it updates iommu-map property > > > accordingly. But If u-boot does not update iommu-map than we cannot > > > have a valid full proof solution as stream-id allocation can change > > > in u-boot. > > > > > > So the other option of u-boot generating this entry seems correct > > > solution. This requires u-boot to know iommu-phandle, something > > > similar to "msi-parent" used for "msi-map" Device-tree binding need > > > change to add iommu-phandle/iommu-parent for this. > > > > From what I know of this hardware, it's going to be rather difficult > > to concoct a DT which reflects the initial hardware state accurately > > *and* works correctly without updating u-boot in lockstep. IIRC, I > > believe the accurate description for an unprogrammed LUT would be > > "everything comes out on the default ID, which defaults to 0", i.e.: > > > > iommu-map-mask = <0x0>; > > iommu-map = <0x0 &smmu 0x0 0x1>; > > These changes are not enough to make PCI devise working with LUT > disabled, also needed msi-map maps all requester-id to "0", using "msi-map- > mask = 0x0". > > Why we assume that no "msi-map" property means untranslated requested- > id, why not consider that translated to "0". > > > > > (assuming the SMMU has stream-id-mask set appropriately too) > > > > That's OK except if u-boot updates the map without removing the mask, > > whereupon things will go wrong, and I guess that would be the case > > with current u-boot :( > > > > However, the existing MSI description is already bogus - if u-boot > > didn't program the LUT, the ITS driver would treat the invalid > > "msi-parent" property as this: > > > > msi-map = <0x0 &its 0x0 0xffff>; > > > > which means that nobody other than 0:0.0 has working MSIs anyway. > > We can have following version of u-boot: > 1) No LUT setup, does not generate msi-map and does not update/generate > iommu-map (older u-boot) > > For this case the working device tree changes can be: > iommu-map-mask = <0>; > iommu-map = <0x0 &smmu 0 0x1>; > msi-map-mask = <0x0>; > msi-map = <0x0 &its 0 0x1>; > > But these changes will not work with current version of u-boot (below (2)) > > 2) LUT setup and msi-map generated but no iommu-map generated (current > case) > > I do not see we can have a working device tree for this. > Probably generating iommu-map in u-boot is better, with that msi-map and > iommu-map will be consistent (below (4)) > > 3) LUT setup, "msi-map" generated and iommu-map updated > > We can have below change is device tree. > iommu-map = <0x0 &smmu 0 0x1>; > > But this change will not work with (1) and (2) above. > > 4) LUT setup, "msi-map" generated and iommu-map also generated by u- > boot. > There is no iommu-map entry needed in device tree but u-boot need to > know iommu phandle. > While iommus is supposed to be used in iommu-node and not in pci node. > > Looking for suggestion Below properties as suggested by you are correct from h/w behavior perspective iommu-map-mask = <0>; iommu-map = <0x0 &smmu 0 0x1>; Are you ok with these changes? Thanks -Bharat > > Thanks > -Bharat > > > > > If you want an obviously-invalid placeholder equivalent to the use of > > "msi- parent" then I'd suggest just: > > > > iommus = <&smmu>; > > > > which would be ignored by Linux for PCI devices, so provided you > > didn't disable bypass at the SMMU things might in theory still work in > > the "u-boot does nothing" case. Otherwise, the implied identity map is > > probably slightly preferable to the unit-length map, i.e.: > > > > iommu-map = <0x0 &smmu 0x0 0xffff>; > > > > which is at least equally broken as MSIs in the same situation. > > > > Robin.