From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934593AbdBQQwA (ORCPT ); Fri, 17 Feb 2017 11:52:00 -0500 Received: from mail-dm3nam03on0056.outbound.protection.outlook.com ([104.47.41.56]:50336 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934431AbdBQQvp (ORCPT ); Fri, 17 Feb 2017 11:51:45 -0500 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Subject: Re: [RFC PATCH v4 19/28] swiotlb: Add warnings for use of bounce buffers with SME To: Konrad Rzeszutek Wilk References: <20170216154158.19244.66630.stgit@tlendack-t1.amdoffice.net> <20170216154619.19244.76653.stgit@tlendack-t1.amdoffice.net> <20170217155955.GK30272@char.us.ORACLE.com> CC: , , , , , , , , , Rik van Riel , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Toshimitsu Kani , Arnd Bergmann , Jonathan Corbet , Matt Fleming , "Michael S. Tsirkin" , Joerg Roedel , Paolo Bonzini , Brijesh Singh , Ingo Molnar , Alexander Potapenko , Andy Lutomirski , "H. Peter Anvin" , Borislav Petkov , Andrey Ryabinin , Thomas Gleixner , Larry Woodman , Dmitry Vyukov From: Tom Lendacky Message-ID: <17c8099a-5495-5f1d-4c8a-bd9f5d2c5e58@amd.com> Date: Fri, 17 Feb 2017 10:51:31 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <20170217155955.GK30272@char.us.ORACLE.com> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: YQXPR01CA0055.CANPRD01.PROD.OUTLOOK.COM (10.165.103.23) To CY4PR12MB1141.namprd12.prod.outlook.com (10.168.163.149) X-MS-Office365-Filtering-Correlation-Id: cdb19f86-5913-4b6f-c1e3-08d457553f2e X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081);SRVR:CY4PR12MB1141; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1141;3:/ISI+x9rwVkPxGvKh3AyFI7Jeuk2Iqib7QT4RrFnNlUBUb+giLeEY0Ge3G1J+bacZS601EJd57m+nYZl9lKLvd8WTHI3oUOi4/vqYRb4fqjSfQhJ5U9cWQPvQx8qrgjlPjoCiPtN5GjZ7dbu4cFG1J77pdmiRGefY57CUcmjWnxrfND7lL15CJIljHsA622xxLB2BdUpRHm0m+ej22woz3R1f4fLIks75U9eXva8ZGZAqaUTa49grdvkR5ePhIftKfjzKq6CPS1lgDypgHGlvdrAPWZpgbppDn1JZecx9eA=;25:Br4BaIP/jqXVlanowimp9/LvmgPEWcMKQYewPhpXUqS2NoHXsSUYsoOP/VMM9sqkgkRFQF3d8eiCLENUPAQFovbTL2eii/eOjKmfW0LgH3MNKlxJ5aw+L+bU5LebFcSxouVqdRZ58O7NZQworJJySFINLVICPCIqTqYDOKd8RwYdM/fQhN30qrv3cEcio4HUcSZRoubi/bUiijeO2XphtBFJm1sqNBkSWg45OztPnbQO0vyu7sYX9qfZ/BaAHyMywGwF53uqJVwsmEf78/pkwOujzUdpm6ScXrjITQe0O7CnVhVl4uVuRqLIhAR+y7CDn/8ljT8kAa/xyggzweQD4wVgXX4/bcMcKLfGaa8Jo7gsuoG4FAh5qzREj4BUCxkBOOfWkgUkoyDB5bogOPbTRBi1eV9LHmUYVrw5EPLyWzTEKdCsctQ9gqRgUn8Fk1E06XCezn2loAYArRXRkuHjEQ== X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1141;31:HMkneZGQITZmvabzkfzcAhY9aAEGyDjrFfPL+tRqJPUbWLS2affGgjSbCtSxND1Vxy8KHFfKsQLXkvLziU1VR6bduxEOXJsMnGif8DBl34tTr6LQsj1YICnLH+9s1doOsDW3DeQ8/3pURWxwPKIg6v/GrWFeaTFoS7XQI9N3nazdVdyNfX8kjijuT/7CXFtEqN0mANmsByOku5yksgqY5GWNK7UMUvAHubcXZuNWjfYwnhalqqoA11d+rs4BgCvC;20:wus4s77EPvFIG+SNjgYkwf5/nTlVODKi7VAaNl2zBqHdVds4oMwVBbSJLfwIUOsps7udKhpdz3cAi2j01AYBsqpZEAOSu9uqdUpljA5MhpI4U6zwBSsriE15KGSEdhSNtfuGa4/HlNyvyXLgn2FxPDwnd50KNxJCAVP6GUBC5LQuz7WMQBoXW0tjB2q9RXll2T1jBiYvQdX6DnsookpBeoOMw1bWIoo42VF+iqi+T02PSP/7uhBEpkHLtMsnsBS3xu8fLawUxuFL561Dijuv8y6oEPri0k9p7jCv5HKBG0b4caVoNo3vfWlnp76BiI2Q1+3pt9vkAcbsFjDE5bCzeJPL6btGAaXhPunQE28khAqOV2A3hkMLMHhr/uXKEyIV30IqZE3FEUSfqAxubQRfz3Ktd8W60ErSxjGwr8OMqp9csF5bTnbLVWwArAqvCuxLXaUWfgExotAmOCmIZTPDgOeBSxkjlwJWOdw5DiKx7k4QbcTmx8K67gUe6/ejmTyW X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026)(6041248)(20161123555025)(20161123560025)(20161123558025)(20161123562025)(20161123564025)(6072148);SRVR:CY4PR12MB1141;BCL:0;PCL:0;RULEID:;SRVR:CY4PR12MB1141; X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1141;4:jE8bx9SoFqVbF8lpO+Gz+FtzAHSHUSt0Sy16ycOJUqdet1UJRKbPz+sUKKOTdUcl6sfz4iNrMLph92pSPf2YPFMro0DAERCTdSt1qWhO4KrOSd1v0k0yD02E+T4+oZMKv/y0I9AyhV1Nd0u7eejEdvvtw7ia89RAwjMLQwPK2mjN9nE0uWjL8b1wOdItO/AFx0gybPgkBKEulU9MPoW0zpXKDI8K8WlTs3LfAKvQvJAvcBKmXa7PVvpjvNQmQ4Nvx0ua57u0aj7WJjY6CnWJaquU9D3Mivs75vyrHrZAIWR5eMbbzPVZLcccZP9CGJR5fBFX1Wp8Yq+XuIWn6CwZ/xuNJ1AJafdp9jiU1Zy3l3Eo57WWjN067zENAmWgDE3NYFco5j7tNNCBBlq2BZJGjDb0epMYAQDdLAabNB8Ot72lUQDzh44cJX6Ed6hgoBtFteLVkPkCocVW1QrXzBtomAH6UpKEWZZcolzbwygWYV+KqI8XJ2asAVXxhlMvFsOYW0xgvNLhvE8SBznRSEEoGiK/eR4SfaXCakjaRBmA0FicXSEuiq4JQcd9u0W3FSTv7CWtm6k3hzWGviTnGPT9TQMypVhydxz2sYMNjNcx4b9iVFJWiudpCB3dI5+TyDALnyylnPgL28/EBS4PgTdMotL1VGOJ86PclxU9paI9PaY= X-Forefront-PRVS: 02213C82F8 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(979002)(6049001)(6009001)(7916002)(39450400003)(39410400002)(39840400002)(39850400002)(39860400002)(24454002)(189002)(199003)(377454003)(65826007)(110136004)(76176999)(189998001)(6916009)(105586002)(5660300001)(106356001)(101416001)(38730400002)(50466002)(305945005)(7736002)(6666003)(33646002)(6246003)(4326007)(31696002)(86362001)(42186005)(8676002)(64126003)(31686004)(81156014)(54356999)(2950100002)(50986999)(36756003)(81166006)(92566002)(90366009)(53936002)(23746002)(25786008)(2906002)(6116002)(97736004)(47776003)(66066001)(229853002)(83506001)(65806001)(7416002)(4001350100001)(3846002)(230700001)(6486002)(77096006)(53546006)(65956001)(54906002)(68736007)(217873001)(969003)(989001)(999001)(1009001)(1019001);DIR:OUT;SFP:1101;SCL:1;SRVR:CY4PR12MB1141;H:[10.236.64.179];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;CY4PR12MB1141;23:tm283ZLrRPVvpgEUl1JdooC0bRcFp2p1iRFMZ?= =?Windows-1252?Q?V3blN9QCnqBWirSJMfi5LlN53RhJ4zEU5nIC6hbLt1WA3+/WtkgQgRvC?= =?Windows-1252?Q?LtPAgHh3z7oOsgKsJQOFojrs6CRkayS3GZgJ2MY1pzIhLF0wFYSlra7w?= =?Windows-1252?Q?wXz74RkIBkdsMeMYYqMbkZQpRniwRaXP3SfEAso+8eky9jYy0sfI1oq6?= =?Windows-1252?Q?zsmFUgdqqSfDtMLigbZPYtXLAh44xSTA1kHX2tzjbZqhYGrb9+5IrkHt?= =?Windows-1252?Q?S9ej5ZY12KTtVIEYDk/Pk+gOgGIetxEBpOQKxXseJpzfxgM4NGi7rC1c?= =?Windows-1252?Q?tgISAD592YKMN2LGMs/SBY5Ws7/IUfQzIcWcyou+71k9gMNZWcY904cs?= =?Windows-1252?Q?DV4gUdOttHhswPuFVthnvMCJZ36ky0Va4XegnAff+GwJrzeDUYLAEuSX?= =?Windows-1252?Q?ByI8luWVxq/N9zSWuPzUk6L2RpnA58K2a3xtZVBJnVmreu+nn06OE0Iu?= =?Windows-1252?Q?Hr99YJcrCVs0KRuhn8kTYJd7AIvMW7RB/dOnrzwTxyjBbbctNEXn2GhV?= =?Windows-1252?Q?D1bm7YUWLgMGXwt0aN5SKlItn6h42Ys6Zbp9KJrnvrop2mXdGFLAd+YX?= =?Windows-1252?Q?C+GI5w8js8mOIJcdgG0oxkHrSQcSWFtVV4fV6O0wOkGNmy7uPGi4DfVf?= =?Windows-1252?Q?rCwC4sdzBIOjv+DjQ2xUJAXud+HyilaNOc0Fgr32rByhjuGJm3gsN17+?= =?Windows-1252?Q?81LzNraMjmCOZXxGZQdSOMr2P6b69V93aRANXFzRmr9QvMc2C9EFx1h/?= =?Windows-1252?Q?M+jFq/kR2yKZfbL7vCeaA8NYRYPCoAQbqKTnRXWtgx8sGpQDBpvnKFda?= =?Windows-1252?Q?UnAnIkklW9prsqsAJ6kp/jK2UftJU/q6LPXxawvEFKLLHMqs52DWR9UY?= =?Windows-1252?Q?LVvPqrthGJ6idNXK5sKHFi8K61iyqqd5uiEJEClJOia1Q7k5FIG3dFp2?= =?Windows-1252?Q?wWUKKUQmO2aBLMr1yP0MJDZw5XdQZdbjEGYRg2MFsrRTbPTiK53B6muo?= =?Windows-1252?Q?LfWvCacND/ooahmal469evGd+0w9VTLaTch+V/wBmkFpuxy7Sl6SuQP/?= =?Windows-1252?Q?b3ZZANvsXa9dzsfS85Z4WtXu8d6TY4XxxG8JAkOluNXm61UF+qmlL5wl?= =?Windows-1252?Q?j7k3a6GJBnhiyL24txEbRFF7teZhjqj1kIXOBEhau6suAXnbsJeUDhr2?= =?Windows-1252?Q?m4G8Du52pueiunY1zYjnnJazLrb0uu+7vJbNJbe2XWJXaRKSKnUCU9VI?= =?Windows-1252?Q?/bZy2Ayyxy16tD+b2aNBO4mHj2oQMGlX0nKwn9sToQi4K+VQWXVyyXRS?= =?Windows-1252?Q?9fkJeJo5lAfD8cqtZyJrkgHu/K/d53qJpBW+rmiTJYN/I8B9SjR2Jdvp?= =?Windows-1252?Q?aKRRyZZVlTQVHS/pqBrGDnXGCPQpvgMDvP1+8eHQaml2nUf+3sOolJ5C?= =?Windows-1252?Q?zmtjLHKqoAuAyA0KYQNtxTkhLldF5uyillIWQ4eTyHkollgye1BRfzUN?= =?Windows-1252?Q?4aCjlbsmhaI/eFZ4J3oFTo5n1WcKMQ9kjPkemDVRWOlLjvrYyW2sDFiR?= =?Windows-1252?Q?rF9nVYei1wMpjSxtAex/qipcCeZ2zl0s460YQaqlUGNWnD1EvEjiLSeK?= =?Windows-1252?Q?4JzTXuR/KbXT9WWiCk+Nxap5f+WAg18CfX8ZHfrKlXoq3AaVcqBtRZeE?= =?Windows-1252?Q?0TvrtiJRJsqS4uGX0fPALr/aX/+lJzA39SN8EwWlCdFDv5HmmnWBJOFo?= =?Windows-1252?Q?p6Ai+PaZh/JWe1HPoDrCOK8jg=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1141;6:sWRIAKAPNGx1+/+ChjaG69r5CBHgrH229Qyntuhd5zzzEM+ZY4EO+32E6skzGuFhokhiE7JcEp1cm5QcDes14suzs2gSd/oSAJXHGFpUeCBARIMWSHf81xyYuUfDoJwgTby/OPAwo20t4M2KOWy53Iq8f+mvE4fHu1+p2gQJyp3nxR/gDNogpCmKUGEmnJvVRlIt45nLgGAx880GC+2r5GB/rI6WTEiUyFDA4m1ojn/HIAHMqHGOsisRVs9SJhDTPhjvPLFhosfI3sP0JlDdBTDr1/fCB3y2w7v1tz/qzlu8BKEDb0SSbKPiAHQ8+ilTbkpS/uBkekzYyO2stRRwehf5s4h6whYDkIUBMYH87Du60mHvt3h50Hz9Bj5uDnL2xGQoe6Utjx6BFm7ZAZF2dLHZrRJ5TvGBhr60TjOl2x8=;5:ZVHLWJmId6tMj7GI1jI7Ah92cEZXJYbFd2n0jAFic+7SS36ugGE2+MQ4b4ABi/HO075TzyxissjjNL7KHkOy+YwtWLblCTtBDH0xVX0VgqaEZ0Yz3Krdn2WbXKE4NCL1xLXAWcE1nv0+MCYxunXOKMGfS2MeucrfIZHKJpmRvzg=;24:8DvxtUhQCNR6KZxrq9o2yNZQwhHEhE4fj7/OjLhPDhJyGPxKEjCbP1MK9CHx8AO+paepGW0PSNGd5jGevlmWNfCyJPjYSKb3/pYkkO/5NRc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR12MB1141;7:3Q9/Rk1TQkfT0RHQar+o2q0D9Pn/pkcmXc9fT1gHYt9IF+YFAx2bxVy1IG9AT0uzaJvTd3vyF+pA3K/CCpYNknUXTBLAJEqiWSD5dp24MPTpN9pcqYwfceaqutqBlxIEYKiB/VxYfKxpca/IaEFz3zNmof6p2fpPA5HIX6pDbjBs4QPG9si8o/Jw6dcrIQLcy3/zILo7yvNZJRZE+p+SOIs3oYelz0NAFkvniqNOqXOCeGEOpiO3ZfwZ7bzioqXSUx9jI5UYPJbWlZhbQkLfDVoHRpD7osXBibXURiA3Mz0dJyoaQo8E0lAM02D8PSxhX2W/F/bqhqgP+vLuhVGcNw==;20:1qxzeT139qph2QT6IgULUpWgAYmF/vvDMfo/dqUKG9MdHmPzOqygdQkvPH4kC675qBmHaM3DvS/SFIiOOYuOuAhg14fCZNjDs/wmmT8/pZIvDowZ8GZEmXMXvNKE76D8YBPhVmUfktRDLSkT+KjNs62iVX6Ea5AF1TjmwvMHQe7vUPJYTdCz+EA66MbUN5YI3PtsH3OLyfy7c5qLc7ueBFybhzTGyHtc3U7TkNLrPOw3AyX1B5rerH/ezslt+JDI X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2017 16:51:37.3857 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1141 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2/17/2017 9:59 AM, Konrad Rzeszutek Wilk wrote: > On Thu, Feb 16, 2017 at 09:46:19AM -0600, Tom Lendacky wrote: >> Add warnings to let the user know when bounce buffers are being used for >> DMA when SME is active. Since the bounce buffers are not in encrypted >> memory, these notifications are to allow the user to determine some >> appropriate action - if necessary. >> >> Signed-off-by: Tom Lendacky >> --- >> arch/x86/include/asm/mem_encrypt.h | 11 +++++++++++ >> include/linux/dma-mapping.h | 11 +++++++++++ >> include/linux/mem_encrypt.h | 6 ++++++ >> lib/swiotlb.c | 3 +++ >> 4 files changed, 31 insertions(+) >> >> diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h >> index 87e816f..5a17f1b 100644 >> --- a/arch/x86/include/asm/mem_encrypt.h >> +++ b/arch/x86/include/asm/mem_encrypt.h >> @@ -26,6 +26,11 @@ static inline bool sme_active(void) >> return (sme_me_mask) ? true : false; >> } >> >> +static inline u64 sme_dma_mask(void) >> +{ >> + return ((u64)sme_me_mask << 1) - 1; >> +} >> + >> void __init sme_early_encrypt(resource_size_t paddr, >> unsigned long size); >> void __init sme_early_decrypt(resource_size_t paddr, >> @@ -53,6 +58,12 @@ static inline bool sme_active(void) >> { >> return false; >> } >> + >> +static inline u64 sme_dma_mask(void) >> +{ >> + return 0ULL; >> +} >> + >> #endif >> >> static inline void __init sme_early_encrypt(resource_size_t paddr, >> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h >> index 10c5a17..130bef7 100644 >> --- a/include/linux/dma-mapping.h >> +++ b/include/linux/dma-mapping.h >> @@ -10,6 +10,7 @@ >> #include >> #include >> #include >> +#include >> >> /** >> * List of possible attributes associated with a DMA mapping. The semantics >> @@ -557,6 +558,11 @@ static inline int dma_set_mask(struct device *dev, u64 mask) >> >> if (!dev->dma_mask || !dma_supported(dev, mask)) >> return -EIO; >> + >> + if (sme_active() && (mask < sme_dma_mask())) >> + dev_warn(dev, >> + "SME is active, device will require DMA bounce buffers\n"); > > You can make it one line. But I am wondering if you should use > printk_ratelimit as this may fill the console up. I thought the use of dma_set_mask() was mostly a one time probe/setup thing so I didn't think we would get that many of these messages. If dma_set_mask() is called much more often that that I can change this to a printk_ratelimit(). I'll look into it further. > >> + >> *dev->dma_mask = mask; >> return 0; >> } >> @@ -576,6 +582,11 @@ static inline int dma_set_coherent_mask(struct device *dev, u64 mask) >> { >> if (!dma_supported(dev, mask)) >> return -EIO; >> + >> + if (sme_active() && (mask < sme_dma_mask())) >> + dev_warn(dev, >> + "SME is active, device will require DMA bounce buffers\n"); > > Ditto. >> + >> dev->coherent_dma_mask = mask; >> return 0; >> } >> diff --git a/include/linux/mem_encrypt.h b/include/linux/mem_encrypt.h >> index 14a7b9f..6829ff1 100644 >> --- a/include/linux/mem_encrypt.h >> +++ b/include/linux/mem_encrypt.h >> @@ -28,6 +28,12 @@ static inline bool sme_active(void) >> { >> return false; >> } >> + >> +static inline u64 sme_dma_mask(void) >> +{ >> + return 0ULL; >> +} >> + >> #endif >> >> #endif /* CONFIG_AMD_MEM_ENCRYPT */ >> diff --git a/lib/swiotlb.c b/lib/swiotlb.c >> index c463067..aff9353 100644 >> --- a/lib/swiotlb.c >> +++ b/lib/swiotlb.c >> @@ -509,6 +509,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, >> if (no_iotlb_memory) >> panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer"); >> >> + WARN_ONCE(sme_active(), >> + "SME is active and system is using DMA bounce buffers\n"); > > How does that help? > > As in what can the user do with this? It's meant just to notify the user about the condition. The user could then decide to use an alternative device that supports a greater DMA range (I can probably change it to a dev_warn_once() so that a device is identified). I would be nice if I could issue this message once per device that experienced this. I didn't see anything that would do that, though. Thanks, Tom >> + >> mask = dma_get_seg_boundary(hwdev); >> >> tbl_dma_addr &= mask; >>