From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756377AbcE0Ten (ORCPT ); Fri, 27 May 2016 15:34:43 -0400 Received: from mail-db3on0089.outbound.protection.outlook.com ([157.55.234.89]:29616 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753221AbcE0Tei (ORCPT ); Fri, 27 May 2016 15:34:38 -0400 X-Greylist: delayed 75032 seconds by postgrey-1.27 at vger.kernel.org; Fri, 27 May 2016 15:34:38 EDT Authentication-Results: zankel.net; dkim=none (message not signed) header.d=none;zankel.net; dmarc=none action=none header.from=mellanox.com; Subject: Re: [PATCH -v2 4/6] locking, arch: Update spin_unlock_wait() To: Peter Zijlstra References: <20160526141922.163198062@infradead.org> <20160526142354.293350777@infradead.org> <20160527090548.GZ3193@twins.programming.kicks-ass.net> CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , From: Chris Metcalf Message-ID: <903f2076-9e24-afb7-0b17-e41457854907@mellanox.com> Date: Fri, 27 May 2016 15:34:13 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <20160527090548.GZ3193@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [12.216.194.146] X-ClientProxiedBy: DM2PR0501CA0039.namprd05.prod.outlook.com (10.162.29.177) To HE1PR05MB1690.eurprd05.prod.outlook.com (10.169.119.156) X-MS-Office365-Filtering-Correlation-Id: f907f2c4-de11-43cd-93c3-08d38665ee9a X-Microsoft-Exchange-Diagnostics: 1;HE1PR05MB1690;2:yffZQu+b98xe0LZd3Gkosj+qOn2b3gsadfZmLjtaeRMZ89zx7vTWKJoji8ZWfpfuiUGEvjEfOq4KzTC2t3mikzTuIhK4tjTXgWrUZAzfsFtriiVmhn8BCtrLMsUfj/VdPiy3DfNvbza4WZ/p+a9tVvY+RNOf9CpC/Is7EZx4pjJjQHYN59or4V5b9PnGXFa2;3:t/Pc25z4nN4R+q+uwQptUp8TceoeZ0myUn3Dt1SRdmju8jYHSRw4p1cP/DZwJk7OzvHgnotQt8SOhSWRQrf6DwaAYxomQJU9N/jLVdBZaC5VhnCOXTJo021DbpEIipdY;25:/6ZoreQ41aUJTTx07QgtW+dMITNmfW3fprrLuzrIUSYe63WJbfabhhAt0m+l7RRPPhNNQhSRGM+/8lAfSkjsVVX2qIULb4cpK1wN0KOkjJst5ndH3MFej0IPLTgQwyzTZWewh8uwtwaa3HVPzi2eamVmvEqFx6iHiKvDuHw5aTjOh4o2Kw9HZn2jQKxEa6aTwz0tjUxS3amE34xvmma/ai2DzsARP8nFup+l32WJzQFLLrxsyxaqxd5T6FE+4/UdkxXzxBJzxq6zE2skVXvmInfsJvksaeS1K+o+YTU97vMkgxvSXdbfOKByihI7ChbtSn2gML7tPvbPNpol6osyc+cL5D1qYBY+ctd/3sFb9kjKj0UB8O4smxfcPEfrKgZrmzujSy7ulfJu3p/XrlKGrKZ5SzdfU1OcXIK9Vpvue7E= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HE1PR05MB1690; X-MLNXRule-EZCH-Linux: Rule triggered X-Microsoft-Exchange-Diagnostics: 1;HE1PR05MB1690;20:aD1EWY8wFkY9ar/v27/utR2EpixI/II79/4WIK3hz4FntwsOpm58TEc8H+5FGqsNLYC4qKPNhc9y6CPBA/8FuRuSD5TEpQhEnoGMaf+uMr0JGDkYRRMvF+oatP6rw0chhjdt9hy5kIL/gFselSM3vD2Iii8rfg8rAI9yQLnqTh8tXndaMpV4gYOScpbxlXVIUPW+pNwF1EixQFpyeXNxzacNmfz84BkR4HLe71uYIS8Sl81Fr/OK2kitFUY/ERusi39gUpaprNnHPQAlvNq9j9+UkqZN9+i2124TKnbEJnBMHAJSlMooxD2ZckXhgNQUYtF+EPLd8UUitnPWdB/BZ5Tti9izmLGj0unyiJS61hxGqF/eYJkFO+tztGlVlJ71Qx4mLLXDyBdM6Ozg4VeD/FaSNJhQAjKp5t2zM6k2pRtVdi//l46U1g7AIkZv4Rf53B8gCi30TOsxIkxY3GgAISpSrmD1Uy/81h0p5hrbZ+jRD8mBoNatRbz3zgCgAzis;4:BPT4yjUpDHl64EYVttJ+5Zi+6Mo/POcFPNXv2YWe76SnjX76nHcUfg15i+LAB5nVtf0/TICYezijUWHGtM9UvZltFEBXQEiT5FfLby1a5MrPh6IwyeISFpwM9nMOdbTNVYROlnaymU3XcWVz8UfGZJ2ROBsieqngbhM5q7E4BWuwybl9zXOJ/qR6F1hF8QkysEabsYDTsJaYuOfA6cI1UB2S36dBw3ryyJN7gHhSZthlz0SGdClYN15NWpfWk0vllY7E+0MOxs2Lx2e+iSTLgLAbetHwUsOCbhjkyYBdvS/yysDEWhIepq3dr7PgDUJaZRE/pHGlItWW5Ja6F6RWO+2opz5VHxSLX2qdSPTbCUxFgSXyxx2p39lDtkE9kKnMAHWIiTMcf4/hViiUH5TF2hHuo6j+Xx4DtE5F+FYdpPFEO8ElwEuBI2LCr8+xRKQL X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(171992500451332); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026);SRVR:HE1PR05MB1690;BCL:0;PCL:0;RULEID:;SRVR:HE1PR05MB1690; X-Forefront-PRVS: 09555FB1AD X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(6049001)(24454002)(377454003)(81166006)(110136002)(86362001)(47776003)(8676002)(66066001)(189998001)(65806001)(65956001)(33646002)(23746002)(5008740100001)(6116002)(50466002)(3846002)(93886004)(5004730100002)(2950100001)(586003)(4326007)(31686004)(31696002)(2906002)(42186005)(15975445007)(230700001)(77096005)(50986999)(76176999)(19580395003)(54356999)(64126003)(92566002)(83506001)(36756003)(65826006)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:HE1PR05MB1690;H:[10.15.7.169];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;HE1PR05MB1690;23:aL2G7PkbhzTjTSvvFEiD0nMpzHEHes/wHnlwI?= =?Windows-1252?Q?vqQaWrt3tLH6Liu0/ueR1Xi5nBNO+ERZ7uk04TOftXJuMXQeu//YL8gV?= =?Windows-1252?Q?ankCwjW/lNZFBHCoZss6i+kd9dlKla3u0LhrUAw3oDS6LWmFA7uqjbdI?= =?Windows-1252?Q?Juj+Vyp7oz9JAbi+4/sCeZ0od/XvpNeHPRFgkzPGFuepOAgcHIZBNktk?= =?Windows-1252?Q?tg21fvczyGFQf+pQLlNsyIWh5fvO0SSVcd0J0xfcjNzibZIFWnIr9G2S?= =?Windows-1252?Q?3ngmz4Ztuw269kmjJrmbkT0j0HgPLed/v+z4WY1vAnhW2LkD3cLsFBkP?= =?Windows-1252?Q?WRROfGmiOGnSZInn26JX9f2tqnDl18ZRMMfI+LsJk/oAD6AyPhFrH8he?= =?Windows-1252?Q?rN1Zr1pELUKd5/nod6SSaX3jLNG/AiiMoKFDl6GfW8sCzH0GZ/B/vFio?= =?Windows-1252?Q?ddiY+5dk+xnbr0bNQJcoDXGpYMcVgBjily6zS57XoRS602FTwh543+lt?= =?Windows-1252?Q?T5zFevMWM7ysh6iyEIr4wpcFH0l4T90fcEKbptXyYmnkOWFPcIjQnk43?= =?Windows-1252?Q?KBOSmL0Y+DeMJrNEItrU6DqUGbBoRJWpKg800aQoGsq0AmFsGNKkPaFK?= =?Windows-1252?Q?JXGBgZFNBfjlByBonQ/48drnH3002W+hF0vRx0PtHbOR8NIZCESncoKu?= =?Windows-1252?Q?3EshY/HsyHIMf8a8Wody1JYA8qqccrlHle1Y1xxIbjS5HZ7TZk32RXnz?= =?Windows-1252?Q?Nmlkn28tr7qBp8E5y3bVu9Z6eaDgTmrMWOe08NfPy4jZh3NVHKufRWu/?= =?Windows-1252?Q?3pPr1iD6FWVwDT9C9FINS+lPXgI3GfOtbJEcb55ID9CAWsZ+gbsLuLC8?= =?Windows-1252?Q?bVgYjkU1zcTeVObHcP57miSCrgxsAkSnCliamQ2+Jag25lZuEaRR/jq8?= =?Windows-1252?Q?sDXp3Hl9uotPSbhIYuOLh19TR+gObXQH6NefRCcwQUvrOaVIlQiaDKsH?= =?Windows-1252?Q?0QKRLjXfsD2iVT+9WFpe9+gz16BlGXiYxn9d6l3bEqb++mRGvmjI0O/C?= =?Windows-1252?Q?mswRy0ag86V9lJVusUzIc4veh7o2wyljUO5ShmtgeJqqZ7RiofHCjanl?= =?Windows-1252?Q?4bvo8xInF71IhSjKVJRjRFSU/zWzQWCidXYq5unFx/57V8Ov7gCuwNu1?= =?Windows-1252?Q?y0R16xjdA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;HE1PR05MB1690;5:/Fu7Y0DtMRUlCL9C0wKB6SfsPiEMWnWTo1fe7w5dvltzpojmCIzatlNfmGS1C+CfR5P2v1S3bS9ucRT9v0qf1tAOniH3sMDQfEmSRBlaLc8KkWrWVwPjusrGR46Eg11VACQ6xBDhzgtVU7BmKpjsig==;24:qClb3rYsTqAmc68iwZt/vnENTdyFUhm9WGBYJMswd51ZtuRsWFVI4fHm8v88hTcjG3T4gkeP4R0y8L2v9RqxOeqp1FbPUOMZrdktcY1IyO4=;7:q/6MZIjHe9CdLHXdQAyAbsg3Hv/z2n8JNaPs674GhESeLXt42I179N+7kEQXadruoLoLZLt3+Kmk1h5ZieBafXqBbyv1VU79gZGiCH+uhsf6IdciAjgHwcsyBBNsEIi+sx4d4aidy/avhSYusUJeO13cRPt7uy1nD5lGutTZeTTV6G4L7htky7Hnk+t2ulHQ SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2016 19:34:23.2451 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR05MB1690 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 5/27/2016 5:05 AM, Peter Zijlstra wrote: > On Thu, May 26, 2016 at 05:10:36PM -0400, Chris Metcalf wrote: >> On 5/26/2016 10:19 AM, Peter Zijlstra wrote: >>> --- a/arch/tile/lib/spinlock_32.c >>> +++ b/arch/tile/lib/spinlock_32.c >>> @@ -72,10 +72,14 @@ void arch_spin_unlock_wait(arch_spinlock >>> if (next == curr) >>> return; >>> + smp_rmb(); >>> + >>> /* Wait until the current locker has released the lock. */ >>> do { >>> delay_backoff(iterations++); >>> } while (READ_ONCE(lock->current_ticket) == curr); >>> + >>> + smp_acquire__after_ctrl_dep(); >>> } >>> EXPORT_SYMBOL(arch_spin_unlock_wait); >>> --- a/arch/tile/lib/spinlock_64.c >>> +++ b/arch/tile/lib/spinlock_64.c >>> @@ -72,10 +72,14 @@ void arch_spin_unlock_wait(arch_spinlock >>> if (arch_spin_next(val) == curr) >>> return; >>> + smp_rmb(); >>> + >>> /* Wait until the current locker has released the lock. */ >>> do { >>> delay_backoff(iterations++); >>> } while (arch_spin_current(READ_ONCE(lock->lock)) == curr); >>> + >>> + smp_acquire__after_ctrl_dep(); >>> } >>> EXPORT_SYMBOL(arch_spin_unlock_wait); >> The smp_rmb() are unnecessary for tile. We READ_ONCE next/curr from the >> lock and compare them, so we know the load(s) are complete. There's no >> microarchitectural speculation going on so that's that. Then we READ_ONCE >> the next load on the lock from within the wait loop, so our load/load >> ordering is guaranteed. > Does TILE never speculate reads? Because in that case the control > dependency already provides a full load->load,store barrier and you'd > want smp_acquire__after_ctrl_dep() to be a barrier() instead of > smp_rmb(). Yes, that's a good point. I didn't look at the definition of smp_acquire__after_ctrl_dep(), but it certainly sounds like that's exactly a compiler barrier for tile. There is no load speculation performed. The only out-of-order stuff that happens is in the memory subsystem: stores will become visible in arbitrary order, and loads will arrive in arbitrary order, but as soon as the result of a load is used in any other kind of instruction, the instruction issue will halt until the pending load(s) for the instruction operands are available. -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com