From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752005AbdK1KRv (ORCPT ); Tue, 28 Nov 2017 05:17:51 -0500 Received: from mail-cys01nam02on0044.outbound.protection.outlook.com ([104.47.37.44]:44756 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751778AbdK1KRr (ORCPT ); Tue, 28 Nov 2017 05:17:47 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Christian.Koenig@amd.com; Subject: Re: [Xen-devel] [PATCH v9 4/5] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors v5 To: Jan Beulich Cc: helgaas@kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, xen-devel , Boris Ostrovsky , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org References: <20171018135821.3248-1-deathsimple@vodafone.de> <20171018135821.3248-5-deathsimple@vodafone.de> <26df0a78-8028-e42c-ce50-4cefe612a7e1@oracle.com> <3443aad0-8c3b-b97e-685a-96b0866827be@amd.com> <7936fdd3-1615-ac1f-7b75-330ccb6a1a3f@amd.com> <0bb468d9-bc1d-e3c5-e313-1cf9408380f0@oracle.com> <275e859a-0ddd-ea7f-a681-67e42a5233fe@amd.com> <38bc76d9-2ed5-61f3-24b3-085031825181@oracle.com> <65885386-3dde-de23-7e95-7d96dfdf6033@oracle.com> <0ab28334-3ccc-2a93-9991-396ee01a3662@amd.com> <5A1D3E9D0200007800192A44@prv-mh.provo.novell.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: Date: Tue, 28 Nov 2017 11:17:33 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <5A1D3E9D0200007800192A44@prv-mh.provo.novell.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Originating-IP: [2a02:908:1251:7981:b169:9ad5:fb55:6cfe] X-ClientProxiedBy: DB6PR0402CA0016.eurprd04.prod.outlook.com (10.172.243.154) To BN6PR12MB1298.namprd12.prod.outlook.com (10.168.227.148) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 38b5f134-f3f4-416b-0749-08d536494496 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603199);SRVR:BN6PR12MB1298; X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1298;3:u1AZ0PWEIcWgTEVadsQuxrhVKJlT+tUTChdtHHoDg8BPkjbpsjp8cAvglH/OFl+7CVedCZM5uVuu+sv0PrylvEROw59fhN7+YemnzrP9sPz6tMyKY13iGEqidFucOmd0xiBlv9lAkt7bd3gsiJz4pgAruU28iC/YMh0exKo3AOyccnGW26U/cpzsuW7hR4jjOYwJ1lapSFtH+2IXIJhp4oUQT4xT4WKsalj7bOvWO34hZtmkI3Jeia7W5ac3ebaX;25:bzrZybmzRffDRnEHwnlrAA9hw1AHeiVDJQGZIW4Rw/YVvXyJG9VyYpev9wuEg8F1M6iRNAn4G+/HJWq5An8wgjfkue1Kqg/xdNUUwNGI85GkZkLP3hbaStz2Jhn70PY9nEaaQf+jEflxd7O9W4gaVfY+ac7vuN4mhYaI7nI8PUYw9PUI7X2p652TWOyUJ+Og/5EwJdTtZxkn//Pl+UeuR/xx3ASgO50b1uxo3rioO4JzpIPG7SAXLNodAU1URsK0rRDj84Fl917Joo9QRxYSu5Y9iN0U4bnF8oFjevy4T2ls0APQi/vIsvsLWQ7TUKC9ipZhrqB2KqAqyNnfGsD/TA==;31:CYbciilU4z0n8BIe9eBLBntUCcU7MtqCDWlcDJ5SeGzbDqGgevMSrIOPAVyHAxaN2CGJk1dtn4G3JXH+QBBDJND9jcjkpejizmSecfq0sT8WWtdeW1tQOqOMeI52KPvFLCLVfpNo4mM1bjKuhAFqhXzuPR+dhUQHHoi9FdnlKimj8sYfDfE1cURWIBD8xRYqzaKDLVFmZQ8THpnrujHnGSB05UnDxMnk2LMWr+RumKY= X-MS-TrafficTypeDiagnostic: BN6PR12MB1298: X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1298;20:XZNV8r6Gp2CtvUDYmw8d0Lhya/xMkE7/KsURihECVXF1XHiu4pYW3D7d968uzz37c45H/zE5Pg1FO3Pa0xrvlwWyjpQEOSzKnMf1dXtQtLl6esledMCoSZSnjq1GrGXjZo552yNVnI2ZfNJa0Zb4gTrVP2WNktYrHo2RGjvQFcdhrkM+Wk8iMrRjUKhpk7spMtcBRFtkIqqTBvgBDyj2GhYjixhURcVAB/3mPnfJv3fs/8Sl0DfILsKbc5dK46C1RYG0Ufxnut3hr3DUIfvNOTBpbq/gfU8I7AnqlRUgNt9zb4gjScPKalzEvsaFTew5HyX1GD4oFsA/UqB840RQV92KIPTk9cGp8cxIWpp11crD42VT25A0JfoBTcgZdNoUWwavdlBKIOcm75/BBjXojtpNiyJtFXMOoHm3v+Gdi/MidEywibHOjOslzjAu4DolN4Anqz+3k/tMEGhZaRA8hTKpPDBoUM3JeYgzYbjD8nM6CiZO9sy8AnjaOr3Fj96h;4:Du98mTBDAc7GSSPs41KnL55baIicWjnvyDD6w0Gvm24kFfq0VAH0gUzd9bi7BRFCZtvq9orO748SYpkWdcXk7f525Ex1f96UKPt5q0iR9qWlwfWBlgMWxoNbyWXYXer5uXWmgwu1QPpRu1tXaf1h2Xr13EVTGC7WFlRM3CiPTViLGbNzt2sjh5s0KuRDFgLEBGHUeEaGjQwvmnnrwgFNjfOZkH+jD/nW3KqGvZrfsvRKoDJDJUX7YjeSCVIC0DxFDR5jZCuwJAdHgC6X0DptMK4RayuTlkhWGQXjp4GBpWmv46XFP4uVzHSIwyNU++W/ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231022)(93006095)(93001095)(6055026)(6041248)(20161123560025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(6072148)(201708071742011);SRVR:BN6PR12MB1298;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:BN6PR12MB1298; X-Forefront-PRVS: 0505147DDB X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(366004)(346002)(376002)(39860400002)(189002)(51444003)(24454002)(199003)(31686004)(58126008)(76176999)(33646002)(101416001)(54356999)(53546010)(65826007)(105586002)(54906003)(106356001)(316002)(229853002)(478600001)(72206003)(36756003)(305945005)(5660300001)(68736007)(25786009)(50986999)(8936002)(2950100002)(6666003)(6916009)(93886005)(81166006)(8676002)(81156014)(1706002)(7736002)(2906002)(52116002)(6116002)(4326008)(67846002)(2486003)(52146003)(23676004)(52396003)(83506002)(230700001)(50466002)(189998001)(64126003)(97736004)(86362001)(65956001)(6486002)(31696002)(65806001)(47776003)(53936002)(6246003);DIR:OUT;SFP:1101;SCL:1;SRVR:BN6PR12MB1298;H:[IPv6:2a02:908:1251:7981:b169:9ad5:fb55:6cfe];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjEyTUIxMjk4OzIzOnVVNFhiSEd5L2Y1TFF6Y295bThWSWZLNGJm?= =?utf-8?B?NnlML3dTc3E2L0ZhTzAwa0x5azExaVg5b2JKdlNENHk1OGRqZ3c0TnNpbWRF?= =?utf-8?B?b3ZRNnBreW1VUEtmSXdZUjl4eDNmS1FhRExwRXVKT2puZm83Tit6Zk5mQ1Uv?= =?utf-8?B?cXZmMlhva0h3Y1ovaEgyRnFVWFFDYU1EMWpoK0sxellYU3lOMWowd1V3c29E?= =?utf-8?B?WVVIY0NyNk11V1lHaFhHUjRCR2ovZmY0ME80Z1M1WnVlOU51V3RqNjFQUk9F?= =?utf-8?B?TFBFWUJRODBobklkc0VYM3VZNzZ2bXZOYlo0STRCRlBKTElwckZXMmk0eWpP?= =?utf-8?B?c1phdkF5OXI3Qm5JOU81U1YzUGUxSk5XY1BIdmJvTU1wV0l4Mlh2OUZwSU5h?= =?utf-8?B?b3h2dVhvR1VnWlNuaVVKeG5vYmJhdkRjMTV4Q05lRkQwc2haNGNtWmNvOElr?= =?utf-8?B?NzZrdE1LazNKR1NyaVRFMFg1WEI3SWZmc2VYb1FGOEVLTkZLYkVQa2g1UCtu?= =?utf-8?B?U3VBdDk3OHp2ZnV0YlhJdmptK1dUM2lRK2crbnFLZHR3bUhjQTBMYXlpNGFS?= =?utf-8?B?N2hwa1JBS3lmMTc0U2xsYktld1U5RWFIZElnZE1Ca0kyQVZCcEhDZ0VmT2to?= =?utf-8?B?bGVlRVRYT0kzaC9lL3BEbmc5TVpOTG5uUmpQd2Z1L3FleDlFZlRhb3RZZC9V?= =?utf-8?B?ODVWbi8yN2NabitFSGk0dTl3d3pNNytYeWtxMlkvQVQyYW14MzZrNnNra0ow?= =?utf-8?B?S3lpekFDaUVUR3VEdVpBNWRuVDlVN2taQmNQQkxHcmJpOVBTeGc2akw1WjNk?= =?utf-8?B?TldWY1NPVVpJWk1FbzNheVNWc1VSQzhOYldscWQ0WTFVU0ZLTzFWekY4ci9Q?= =?utf-8?B?b3c0ejR1d0dncHJIazhDaHl4THFlNnJQUGdVVEZsSjI1eE4vdXVtelAyMk5V?= =?utf-8?B?Rjhsc0QwenpsL2FJOHBIdDhxUXJEQzJRcE9BMldjTXZqeUhZWEFzSnVRWlQ4?= =?utf-8?B?ZUJSUEdXakVZK0FhakEzYkFaUE9IY3J5OUc5S25NaVVkR0ZtVmd3YXdncTNG?= =?utf-8?B?RWtCbDNQUUE0ZVByMFdJalVpdEQ0MWNaeGtSNkZHT0lLV0U0WUJZbFBDMXNX?= =?utf-8?B?TnVHb0F1QUZYdHdkUm1XcVdQUXVneGkvcTNNY096RElERFZFb2J4RWYwbHVN?= =?utf-8?B?c2gra0ZEV2dIZ2VtYWNSN2hLZ2RDUHhHcnJiOGZNQzhQd21ydkNPaXZpTVVE?= =?utf-8?B?UTQwd0xMb3pMemhVcFMrUDdSUENOdzNIS012UnVUaHdHZmpjMW5hdHVLVjBm?= =?utf-8?B?UzVRWFUyOG9PRjN0S1BkK1ZvYlZ6VXczdWJtc3BLSWprMVNxaEp1T2pQbnhV?= =?utf-8?B?c0xoL3pFbHRvSEV5aUNMWFVJSGhSNUwwZmNMOWdxZzF2QUpsbmtCYmFtMnpz?= =?utf-8?B?eC9JcFp6Mzd1RktmZzNtRWtKa0JoUys3bk9EdXh2aGVDQlY5anBwWDR2QjhY?= =?utf-8?B?WlBVWTBsRTFjTGpKRHdrZklKbDZ4Q2daQ094T0JmYnQxc1JBUkNHZ2paa3Vt?= =?utf-8?B?MXZTcEpxcjVGNzluK1JzSXVoVlZwZ0xNSVpkN0hsekFYbjExaDJKRmRWMFFT?= =?utf-8?B?M09VemtyVzNVeFg5YmZ5dnFJMzRiSnFZSG1lSG1KdWFPTUVuSncrck0zRzJC?= =?utf-8?B?L0pCVks5YktmOU9uYWxKUEZqbGk1VXo2bnJudUgrcy9mc1pmUnZMMDh3MFBT?= =?utf-8?B?Q3Exc1dCMHZmVndjNG5ja1JHNXVwQkhtQXdWbUk2Y0FXcS9CWC9VU0ZIcFM3?= =?utf-8?B?NEN5MW9MbTZGZFlOL21WVmd6OTBOZEVBQ3VFZ05vSVRrVlV2eUNRdDVUa0Iw?= =?utf-8?B?STFCR1NURnZucXZ6RTN0c1NpanFTVFZFQUpSbVJmZmJNdHdOS0hzbHBiTTVa?= =?utf-8?Q?5+0SluyJEyCiSbT4r+ivnetttbfi2E=3D?= X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1298;6:YKS7D+aHyV4mUTTVzqSD9rhsD3UVMuED1oAxkMgIaJOhBHzHBBmknthTh0u3vtORczVVKnhqAA8AK4UndtKMni0oE+DLFbyiCpzIjX+VZmqhI+XhwgDc6Q/9qE5scHg/BpBhBxkTA/XvUxa+G/CPu/i+dAwH/C5vC+u+xtcy3qec6BBnfuTf+UBUbX5PPu0raLr/HESUsNyu8tu4DNoUprkK+4HmB0rln7naopn5w5sOk123GfuyEZeiak1fWCxH2brSNMcuUIQEAj/yA4P8p+Y8ipFbUeUnmxg/CMheh+TwjvVIJNh4BkAybLFlwtN/JZd5aLXdw94ZR1TrzHiTva+6p6sS1a4IcI7QSsA8ZV4=;5:00r0nlToBFsEZ+JCM8DzqdB606WM2uSevxB94FuzO39rbOv3tnaVPNShZ8d75uK8vkciCYR2so+xd/IrEJCZ0uD5JVQ626PHosXDt0jcRD5V8zsqT+RGCIoBvkcaAwRBF0bItb8uU0nWZnoD476k3uWDq+PXIC6GdgZ8ADUQLbY=;24:5enIcCwg4msBFPxOrXiSzX8attOsStbOFrk6RB3Yt5dwROTVhiZePnRitqDMPHn/Ic2b/Uiz9mikFmRUiBnJU2Gb+Qpnc6u2hPQCiJcK/x8=;7:p399Q129RKcNNhunBzJww8YAgMqzxquc6V7dT0s/ZdDktPz5tF5WlOwwqptTFzkQ9GaPmYTrstU5Cbkja+ns5DFVWnMMQtwyLyjGHVGJyIMQKO3DpSzc6S2Kxuy9TCHf/gEO16Rw/CGR/2gTMDlGaRTZ1ebpCvjNlioJ/t4eLZSwQzFtpL4Wjm84YUDZA2r3qLVn+bhYYzoAHj6yZ+Wbsl4nP3Klt13CHoV4PlcnNhWxU5qNU2kPHxEQFKSxz4zs SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BN6PR12MB1298;20:sRvaJ8f8VE4lr6U9V84f9HI9/fF+37p9MGvhm7vB0bp7SEjusS6BqoXnIpcws5q6LhQ2OnB+xHoLh19u0hu8dVSq/vMftUINlOtEKY0v2uwplds+IfQKP7xkKRHAATpl49LwRJNjoOwj7ixZOFGE/kVSml67cr/Z7rR4ll6Mx6ZSG8HyhLXMKUvn+jEN+/J6oylPcG+bA99BihOEeOhT3gJvofIrt5Dc/OlHKF+hSB1g8h2vhT1/NmdXvUcqU+9d X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Nov 2017 10:17:43.7269 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 38b5f134-f3f4-416b-0749-08d536494496 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1298 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Am 28.11.2017 um 10:46 schrieb Jan Beulich: >>>> On 28.11.17 at 10:12, wrote: >> In theory the BIOS would search for address space and won't find >> anything, so the hotplug operation should fail even before it reaches >> the kernel in the first place. > How would the BIOS know what the OS does or plans to do? As far as I know the ACPI BIOS should work directly with the register content. So when we update the register content to enable the MMIO decoding the BIOS should know that as well. > I think > it's the other way around - the OS needs to avoid using any regions > for MMIO which are marked as hotpluggable in SRAT. I was under the impression that this is exactly what acpi_numa_memory_affinity_init() does. > Since there is > no vNUMA yet for Xen Dom0, that would need special handling. I think that the problem is rather that SRAT is NUMA specific and if I'm not totally mistaken the content is ignored when NUMA support isn't compiled into the kernel. When Xen steals some memory from Dom0 by hocking up itself into the e820 call then I would say the cleanest way is to report this memory in e820 as reserved as well. But take that with a grain of salt, I'm seriously not a Xen expert. Regards, Christian. > > Jan >