From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935027AbdKQW0A (ORCPT ); Fri, 17 Nov 2017 17:26:00 -0500 Received: from mail-bn3nam01on0086.outbound.protection.outlook.com ([104.47.33.86]:9525 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934087AbdKQWZw (ORCPT ); Fri, 17 Nov 2017 17:25:52 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Subject: Re: [PATCH 2/2] iommu/amd: Add support for fast IOTLB flushing To: Suravee Suthikulpanit , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org Cc: joro@8bytes.org, jroedel@suse.de, alex.williamson@redhat.com References: <1510953080-5619-1-git-send-email-Suravee.Suthikulpanit@amd.com> <1510953080-5619-3-git-send-email-Suravee.Suthikulpanit@amd.com> From: Tom Lendacky Message-ID: <273179a4-43fe-1e5f-802a-31eea5a91bd3@amd.com> Date: Fri, 17 Nov 2017 16:25:45 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <1510953080-5619-3-git-send-email-Suravee.Suthikulpanit@amd.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR18CA0024.namprd18.prod.outlook.com (10.175.188.34) To DM5PR12MB1147.namprd12.prod.outlook.com (10.168.236.142) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cbbae594-c7a3-41a4-eafe-08d52e0a27ab X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603258);SRVR:DM5PR12MB1147; X-Microsoft-Exchange-Diagnostics: 1;DM5PR12MB1147;3:+KmyE0NDX054p73aCOBC/aJDDo9Lp9wIXMjU6/bsCq2OOB2G6AY/XAW7NE3MtnQOZCWEl0dO1It+jPQN22himx728J693Aq5698BV6x3uhBcQTSKjKSYyzrIYm+Aj0SnuDr5F5dUqCqSIPmLAXYcrgMRpXYC05YfkOevRPLq2JRKlVMDnxCULpxO/J9EEsc/OdKV3O8f93eioQE4XdlNj25j02rR4U+F8W0KDUzJihOHGk2foRVUAcFKYaNG5DO5;25:bvXPgQj7ByIyXd/rLXsu6+E6tT96VAhsBJuSVHi5xF/QteafcUmI1cLAivMpZ7XUNJOAKFGh3v9sTEZHT9Yq+ervl2uhRnWDcukK8y+mOvpqsGZdDpbaxpulycA5nkWIucs05IOmnU4+cE4ztjNF9ru0z3kwouQ2zk2yhBrTWLC8mdUgJszNvO/RKnK6L7Nha+g0Qa3DOpUw2RW4oVCCbwt5TPtw+ykBlhU1oznPDFJ/J14Vt0NpwbzHBMNGxhZngCjMfoh0uhZhYjTQ2vYMoEGx0BobUnip6cdEEsRhVPDACtI54Ul1bIjRdcjVSgOfqUHJRJe7eq4AH4WlmZej1Q==;31:c3fcMcvLZAfJ0O+RtNh3xynsewYqre31jHOe72mbvIAEiZV/gbb0qxS4/WQrG3o/4RBhmp8G33tNwQYJ3W83H3uFeM3ZkKAOWoTEmso0OwrwqWp3JHLgGo8uuk/xVz2x3leQAciipP7Cg4FfPLd2B3gjxEJtYQuYaADzQOKT5b+SrmAi3N2b7I+46ut45qg51y/jtOcb11bFe1pghy0dJIP4sLgBqZXQ2rIRgfMMNwU= X-MS-TrafficTypeDiagnostic: DM5PR12MB1147: X-Microsoft-Exchange-Diagnostics: 1;DM5PR12MB1147;20:B/cv8UvgH80S07DsP216jm/CJw9T4JXT0UD9QhAz/XcawzOMTebvAQP62iKI+mwZAKsa/8BWL8RBYPsv+cyxr5fBMEWijkekI5TKLmUXEZM/SG+ZbUg0nZxzjoS58LNCEClL/q7OaaypUEsi5Sj9No8T/LcW+GGZHwJBd7RhU1PnTBr+vNCeOXKDem4VLsQWqwAmDfCCU056BkSDD+MQJkEJw6b7prb4Sk8UbYqhYmxLdvlcXHKAghXRR4hfFLQpf3uim9HCBdSeZz+te+f+HJP9EZscT7OdjUImDs6Wu/E+UJgZ8mRog2KwWRPwwupcamuMEexXZ4h1Tu7WYryr4kENjdsS1TG/N0JV4G6HEJyIeEQAf9SYxZFgL4BbvNUNbtEsabB7v5YKZJ0kFKJzMBxBglOjq6BYZ/Mvv/V2oTaoXsnUtWZ2REWkTJSfcfdXSj4NVezOJfX5p5pwqjQYiGAfdR9eSN5i6gbG+wf9kJ6HxlrBiDmeLLhMRNLd/4dG;4:w/WQV64E9IIy/PNMG0QGX4Nhk5OfD046ceQ8LMG9S7Av89ue0Jvlu8Ut+TkA5XvWMUJboOZj1Vui77Z1f3iSdm6acy/zfwqUPmIxlWcse/o0xljJfRrnAD0fONE0+fkt3U6YpYVgsY9M1h/Qc3ByQVrVxvbbX9dCy9ypCj7TbmgotxA/izBmxdH8GzmEq7fnGl7A1BsBoswLNxmr+1r32H5MgSLPykDcKutCvAdoXL+lsahWhpKwxHU156BWYxlRKQOa3gOeSgDhf1vJTTbLnsxSqhKnDVHT/kBlt2G2Kyoj9JhJfe6B6ws7vTi8wAwM X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(100000703101)(100105400095)(3231022)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123562025)(20161123555025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM5PR12MB1147;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM5PR12MB1147; X-Forefront-PRVS: 049486C505 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6049001)(6009001)(39860400002)(376002)(346002)(199003)(24454002)(189002)(5890100001)(58126008)(106356001)(36756003)(16576012)(316002)(3260700006)(97736004)(25786009)(53936002)(229853002)(6246003)(77096006)(33646002)(6486002)(16526018)(230700001)(90366009)(53546010)(305945005)(31686004)(54356999)(72206003)(50986999)(4326008)(6116002)(47776003)(7736002)(3846002)(50466002)(76176999)(8936002)(64126003)(68736007)(8676002)(81156014)(81166006)(478600001)(2950100002)(101416001)(67846002)(5660300001)(31696002)(86362001)(105586002)(65806001)(2906002)(23676003)(6666003)(66066001)(65956001)(65826007)(83506002)(189998001);DIR:OUT;SFP:1101;SCL:1;SRVR:DM5PR12MB1147;H:[10.236.64.70];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTVQUjEyTUIxMTQ3OzIzOjNvcyt2enBlS0N0U0w5STdNSzlEMHBLUmNQ?= =?utf-8?B?TUdGeG1jU2hUbC9DM3hqZkhzUEg0aGxtRS90TGxSaUNpRlpic3NjdWNKd1Q1?= =?utf-8?B?V0lQVUlyLzB0QkNuYWtUelV0SVFaUG43MHpMVjNGWnUxSEZPa0JsMkZxUnJT?= =?utf-8?B?MnhSYmsrTWJJNXh2RGF5Z0xiUTJqb25qZDMrNDdXSjZaWWZtQUVDQWUzdDAw?= =?utf-8?B?U3ZnOXhLbnJxT2d4M1N4elp0VklVV0lBUkw1TTBYaFJuNU5ZVnFBTTZqKzJ5?= =?utf-8?B?bGxNTTFPaE9uK0Y5bGdLRUtRZzB4TktvQXEwTmFmV3AxcU9BdmZSaFNBZXVQ?= =?utf-8?B?ODQwMHdDODZCTVFKTXNVTEhzb3RBN1djdUZmY3VsVlh5LzhVOUVmU2V3SDBn?= =?utf-8?B?Vk1rWlo4OEN6WlN3OWJlazN5MlBkMm5hVFF2U3dZRWFhQ2svbGExbDRDVW8x?= =?utf-8?B?S1o3dVdYS0V4UnRpOE90OE1LZTkzcFJBRWtORlErZFRLbHk3eWNCNFRiNkpH?= =?utf-8?B?VHlYK0dnYUYyTDhadXBYczFPWUtSd0dYM0o4VFZoNTVydG5ET0szS1RoSzRJ?= =?utf-8?B?eUdYazVjZ0Q4UnV3MG1xM2wxZStXUUw3WDRIa1l0S2NWdGZmcGoxS2E2RXdN?= =?utf-8?B?T2hGQ09Cczl5SjJNZFduSWVnWjc4Wm9tLzVjUTU5eUJQdnA1Y0hTUDZrRzdN?= =?utf-8?B?QUhNS1A2YkRVdEJ4NzM1RVl0L2dNZE44NGhlTlQ3b3M5QytZelZOc2pVanlC?= =?utf-8?B?WmhBdzlBSmk3dmJpWEYvVGxya3RnRy9UK004b2NraSsweUVQRVRYZno0Wjkx?= =?utf-8?B?YjROdEM1NGFtMU41RTV4ZnY1Q3lUL0VVd3NiZCs5TXNZa3BpMW9QNmNRVFVK?= =?utf-8?B?aDQzeFVZSVM0Tm41NW5uSFRuYlRvS001bFRxZXQ1dTNGTW11Yk8rV3phd25B?= =?utf-8?B?ajBuQTgxWHFPK1RHSE5pRGZrZWtKT2Y4MWtsdG1EeTFPenVUZ2NnQkVhTTJr?= =?utf-8?B?UnFLakNDTlZsZFliNW82UENTSUdBZTdBM2JqZXowUHpmaStNbmlRT0tsRHZ2?= =?utf-8?B?bGxhSzFQdUVCMlAzNVB5WnI5SDhFdDIvR1N0S3dtNWtzQ2h2M2hHam14SFdq?= =?utf-8?B?NHVnVDhud29mV21Cd21IdE1adTdsUmszWWFBeHJiemwzWjFDWHdlSmlMbEEv?= =?utf-8?B?Rk9VRFhzY2hHaDFhQ21FWmRkZFZjaFJkRmpTd0FsalYvbCt1dkVaekYweFZV?= =?utf-8?B?bGg5L2Y2VndXUVJPRTM5S3BPMHFEajhOYmRIc1pBcUNUWFNtanlLajdtMlZJ?= =?utf-8?B?VWxIaWEvZ1k4NW5Nc2NuK2RhL1cwWTI0RzUwWmZ3bmJSRE5zd01GMkxyVXNG?= =?utf-8?B?cElBeU1CQUFpdnBFaHdPc1AwUlVYSkY1bkZHMExUZnFoakI3K2RhbHFLcXZT?= =?utf-8?B?S3BER3ZDOTBEdWNGNTN3eGVpd202bEdmVEFIL3AzU3lQZ3dGVndZekZIeVdm?= =?utf-8?B?RW1SbXdxZkUvVzhyMnNjV2VqaWY0Q2hac2d2eFlZM2J6TFNub0tRUmNUTVdu?= =?utf-8?B?YjVXZTJtWHRhOU50SmN3eTFnNzUrOXoyV0VnVXF5V2ZOakpBR1BhbXhrbnp2?= =?utf-8?B?UkZaQlg0VHhFY0xGeUlHR3RGbkEyUmFjbXNvRkhMSWlta1hHY3R0WU1KYkNR?= =?utf-8?B?aUJSQlVRUXMrak9GWnkzRzY1djFvN3F1QUVhbVZaZk5PYklmUm1JdHdnbW9Q?= =?utf-8?B?NUl1ck1mbVVlNFFLNVlSeFlKM2VZZ1lyOHNlU2RBSzFiMTJxZUVLZlBuVU9k?= =?utf-8?B?NFBDa3hWd2h1VHI0QmhMQXlTbTRpNlU5SVFBUFhnUXVsNldZLzMzZXZRdDFE?= =?utf-8?B?RzBYMmczWVZwdld4N2o1R09laVVuUWNLcGtDcS9LRWZxYTROdUowczJkK1Vo?= =?utf-8?Q?wPQgPALaDm7MC9fvU1NxXFdEh3nqog=3D?= X-Microsoft-Exchange-Diagnostics: 1;DM5PR12MB1147;6:QM78WAdDqTBc0ZQgHKOU/KE6EuX8tati5EcY6Nolql6qvGYYtSAMGbMaPXCnP50yQ5ByTBMgSdddT2gke13omwCi7xaer3CHZWhqumAOWweafpP9TJqPY0E0qUgcCEy8A/oOausKW3XaW4zx1g5TiUWbXkf45PWhovZcvqZO1Y6tvbOoDFLstsuVJSjT3zVJfZ3Q4V6xt+pks1e/2++sIJPgg3qIlzuzHiK5GFjMiO0kkqHDR96lDGN9bqtDUpAK0yPvTT2osTxGoShHFw5pWbyNbESGGxcsBdRv/VgA+bip3oOrZ4tKbz34ZRLSBccFh17CxX/sDKPDsd59x5nRZQthrnvLmxGwzjl0M6BmULI=;5:UTQUc2zZWRuxnm30ghSJdACg+374RSdWJ8aJZyyMSijtp3SnMoxul86LrFwBbVKfO8tBVaH859H8DxBjNzUSSZzX67LaJVJA5HZzvEMxFurMfeFC2CdmjbD1m1v7ECFimkm1kmSyqewjwu1sr/3D/CEoqNFB+A0JpMXlv7oW8Kk=;24:1I+C3GmrFZWf9sLmgwyKbO8z6qjsHNDIBywJrTsm2hhq8nSIpzfkBAgXEvqNljTCEdZeMZM73zM/AqTVugvTx9qzY+YMVhoWt88tk2H6g0I=;7:i12AVu9cDrwoEWnBxs7WLBadEyGKMaPbAqdRI//eS+h7twYd3CuRuGJcr/TPh2YYMI1GT/zL0jY3+owHUNPLv5YzvTAXDnYR2gAG5GVAMDZn2yXRusk92Gobirh2LSzAKMSPRoNyqN8E3JxRaE+2lK+4zcIhgR7bk+kwUCAGuNtzZ+i5tPp3ujCLprF6pqIsUPSjnXYqDTT/6OLkNIp+/tFT3/h7l7wmDixgcYE3oTCEy6MN1YT2rb8rH2juL+kn SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM5PR12MB1147;20:yvvNkHWJF2QewbfnRF5TQ8ZyACEMHP3KJtBFTjjUnza7ZnOgOfkVyHyqe2iwAQZU3Utc7FQX1aJ/zlQdIeGMy8vrXp9BxAbCovmLs0/jv608K6aWZ40J/aFyDM1Pox16u2hzIZPy8gz66ZiqGAV+AOXoa1BaoHcrjws6G/Z+mrLUBawQddrsC/PCKK0rW4079LojNjsZpTqo+rlaiCWa1B/xbGJepZD4nqUBXoReJE8kxKrDrlp5lV2ihbYfYzkk X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2017 22:25:48.4797 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cbbae594-c7a3-41a4-eafe-08d52e0a27ab X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1147 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/17/2017 3:11 PM, Suravee Suthikulpanit wrote: > From: Suravee Suthikulpanit > > Implement the newly added IOTLB flushing interface by introducing > per-protection-domain IOTLB flush list, which maintains a list of > IOVAs to be invalidated (by INVALIDATE_IOTLB_PAGES command) during > IOTLB sync. > > Cc: Joerg Roedel > Signed-off-by: Suravee Suthikulpanit > --- > drivers/iommu/amd_iommu.c | 77 ++++++++++++++++++++++++++++++++++++++++- > drivers/iommu/amd_iommu_init.c | 2 -- > drivers/iommu/amd_iommu_types.h | 2 ++ > 3 files changed, 78 insertions(+), 3 deletions(-) > > diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c > index 8e8874d..bf92809 100644 > --- a/drivers/iommu/amd_iommu.c > +++ b/drivers/iommu/amd_iommu.c > @@ -130,6 +130,12 @@ struct dma_ops_domain { > static struct iova_domain reserved_iova_ranges; > static struct lock_class_key reserved_rbtree_key; > > +struct iotlb_flush_entry { > + struct list_head list; > + unsigned long iova; > + size_t size; > +}; > + > /**************************************************************************** > * > * Helper functions > @@ -2838,11 +2844,13 @@ static void protection_domain_free(struct protection_domain *domain) > static int protection_domain_init(struct protection_domain *domain) > { > spin_lock_init(&domain->lock); > + spin_lock_init(&domain->iotlb_flush_list_lock); > mutex_init(&domain->api_lock); > domain->id = domain_id_alloc(); > if (!domain->id) > return -ENOMEM; > INIT_LIST_HEAD(&domain->dev_list); > + INIT_LIST_HEAD(&domain->iotlb_flush_list); > > return 0; > } > @@ -3047,7 +3055,6 @@ static size_t amd_iommu_unmap(struct iommu_domain *dom, unsigned long iova, > unmap_size = iommu_unmap_page(domain, iova, page_size); > mutex_unlock(&domain->api_lock); > > - domain_flush_tlb_pde(domain); > domain_flush_complete(domain); > > return unmap_size; > @@ -3167,6 +3174,71 @@ static bool amd_iommu_is_attach_deferred(struct iommu_domain *domain, > return dev_data->defer_attach; > } > > +static void amd_iommu_flush_iotlb_all(struct iommu_domain *domain) > +{ > + struct protection_domain *dom = to_pdomain(domain); > + > + domain_flush_tlb_pde(dom); > +} > + > +static void amd_iommu_iotlb_range_add(struct iommu_domain *domain, > + unsigned long iova, size_t size) > +{ > + struct protection_domain *pdom = to_pdomain(domain); > + struct iotlb_flush_entry *entry, *p; > + unsigned long flags; > + bool found = false; > + > + spin_lock_irqsave(&pdom->iotlb_flush_list_lock, flags); > + list_for_each_entry(p, &pdom->iotlb_flush_list, list) { > + if (iova != p->iova) > + continue; > + > + if (size > p->size) { > + p->size = size; > + pr_debug("%s: update range: iova=%#lx, size = %#lx\n", > + __func__, p->iova, p->size); > + } > + found = true; > + break; > + } > + > + if (!found) { > + entry = kzalloc(sizeof(struct iotlb_flush_entry), > + GFP_ATOMIC); > + if (!entry) > + return; You need to release the spinlock before returning here. Thanks, Tom > + > + pr_debug("%s: new range: iova=%lx, size=%#lx\n", > + __func__, iova, size); > + > + entry->iova = iova; > + entry->size = size; > + list_add(&entry->list, &pdom->iotlb_flush_list); > + } > + spin_unlock_irqrestore(&pdom->iotlb_flush_list_lock, flags); > +} > + > +static void amd_iommu_iotlb_sync(struct iommu_domain *domain) > +{ > + struct protection_domain *pdom = to_pdomain(domain); > + struct iotlb_flush_entry *entry, *next; > + unsigned long flags; > + > + /* Note: > + * Currently, IOMMU driver just flushes the whole IO/TLB for > + * a given domain. So, just remove entries from the list here. > + */ > + spin_lock_irqsave(&pdom->iotlb_flush_list_lock, flags); > + list_for_each_entry_safe(entry, next, &pdom->iotlb_flush_list, list) { > + list_del(&entry->list); > + kfree(entry); > + } > + spin_unlock_irqrestore(&pdom->iotlb_flush_list_lock, flags); > + > + domain_flush_tlb_pde(pdom); > +} > + > const struct iommu_ops amd_iommu_ops = { > .capable = amd_iommu_capable, > .domain_alloc = amd_iommu_domain_alloc, > @@ -3185,6 +3257,9 @@ static bool amd_iommu_is_attach_deferred(struct iommu_domain *domain, > .apply_resv_region = amd_iommu_apply_resv_region, > .is_attach_deferred = amd_iommu_is_attach_deferred, > .pgsize_bitmap = AMD_IOMMU_PGSIZES, > + .flush_iotlb_all = amd_iommu_flush_iotlb_all, > + .iotlb_range_add = amd_iommu_iotlb_range_add, > + .iotlb_sync = amd_iommu_iotlb_sync, > }; > > /***************************************************************************** > diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c > index 6fe2d03..1659377 100644 > --- a/drivers/iommu/amd_iommu_init.c > +++ b/drivers/iommu/amd_iommu_init.c > @@ -2489,8 +2489,6 @@ static int __init early_amd_iommu_init(void) > */ > __set_bit(0, amd_iommu_pd_alloc_bitmap); > > - spin_lock_init(&amd_iommu_pd_lock); > - > /* > * now the data structures are allocated and basically initialized > * start the real acpi table scan > diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h > index f6b24c7..30e1c68 100644 > --- a/drivers/iommu/amd_iommu_types.h > +++ b/drivers/iommu/amd_iommu_types.h > @@ -462,9 +462,11 @@ struct amd_iommu_fault { > struct protection_domain { > struct list_head list; /* for list of all protection domains */ > struct list_head dev_list; /* List of all devices in this domain */ > + struct list_head iotlb_flush_list; /* store iovas for iotlb sync */ > struct iommu_domain domain; /* generic domain handle used by > iommu core code */ > spinlock_t lock; /* mostly used to lock the page table*/ > + spinlock_t iotlb_flush_list_lock; /* protect iova flush list */ > struct mutex api_lock; /* protect page tables in the iommu-api path */ > u16 id; /* the domain id written to the device table */ > int mode; /* paging mode (0-6 levels) */ >