From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753328AbcAGTqG (ORCPT ); Thu, 7 Jan 2016 14:46:06 -0500 Received: from mail.fireflyinternet.com ([87.106.93.118]:49398 "EHLO fireflyinternet.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752150AbcAGTqE (ORCPT ); Thu, 7 Jan 2016 14:46:04 -0500 X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Date: Thu, 7 Jan 2016 19:44:13 +0000 From: Chris Wilson To: Andy Lutomirski Cc: "linux-kernel@vger.kernel.org" , Ross Zwisler , "H . Peter Anvin" , Borislav Petkov , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Thomas Gleixner , Imre Deak , Daniel Vetter , DRI Subject: Re: [PATCH] x86: Add an explicit barrier() to clflushopt() Message-ID: <20160107194413.GA25144@nuc-i3427.alporthouse.com> Mail-Followup-To: Chris Wilson , Andy Lutomirski , "linux-kernel@vger.kernel.org" , Ross Zwisler , "H . Peter Anvin" , Borislav Petkov , Brian Gerst , Denys Vlasenko , "H. Peter Anvin" , Linus Torvalds , Thomas Gleixner , Imre Deak , Daniel Vetter , DRI References: <1445248735-11915-1-git-send-email-chris@chris-wilson.co.uk> <20160107101652.GF652@nuc-i3427.alporthouse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jan 07, 2016 at 09:55:51AM -0800, Andy Lutomirski wrote: > On Thu, Jan 7, 2016 at 2:16 AM, Chris Wilson wrote: > >> + /* GCC (4.9.1 and 5.2.1 at least) appears to be very confused when > >> + * meeting this alternative() and demonstrably miscompiles loops > >> + * iterating over clflushopts. > >> + */ > >> + barrier(); > >> } > > > > Or an alternative: > > > > +#define alternative_output(oldinstr, newinstr, feature, output) \ > > + asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) \ > > + : output : "i" (0) : "memory") > > > > I would really appreciate some knowledgeable folks taking a look at the > > asm for clflushopt() as it still affects today's kernel and gcc. > > > > Fwiw, I have confirmed that arch/x86/mm/pageattr.c clflush_cache_range() > > is similarly affected. > > Unless I'm mis-reading the asm, clflush_cache_range() is compiled > correctly for me. (I don't know what the %P is for in the asm, but > that shouldn't matter.) The ALTERNATIVE shouldn't even be visible to > the optimizer. > > Can you attach a bad .s file and let us know what gcc version this is? > (You can usually do 'make foo/bar/baz.s' to get a .s file.) I'd also > be curious whether changing clflushopt to clwb works around the issue. Now I feel silly. Looking at the .s, there is no difference with the addition of the barrier to clflush_cache_range(). And sure enough letting the test run for longer, we see a failure. I fell for a placebo. The failing assertion is always on the last cacheline and is always one value behind. Oh well, back to wondering where we miss the flush. -Chris -- Chris Wilson, Intel Open Source Technology Centre