From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752481AbdI2PT1 (ORCPT ); Fri, 29 Sep 2017 11:19:27 -0400 Received: from g2t1383g.austin.hpe.com ([15.233.16.89]:37227 "EHLO g2t1383g.austin.hpe.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751920AbdI2PTZ (ORCPT ); Fri, 29 Sep 2017 11:19:25 -0400 Subject: Re: [PATCH 0/4] x86/platform/UV: Update TSC support To: Peter Zijlstra Cc: Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Bin Gao , Prarit Bhargava , Dimitri Sivanich , Andrew Banman , Russ Anderson , linux-kernel@vger.kernel.org, x86@kernel.org References: <20170928180339.010141096@stormcage.americas.sgi.com> <20170929084632.gfwugbeaonraadp3@hirez.programming.kicks-ass.net> From: Mike Travis Message-ID: Date: Fri, 29 Sep 2017 08:19:22 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170929084632.gfwugbeaonraadp3@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 9/29/2017 1:46 AM, Peter Zijlstra wrote: > On Thu, Sep 28, 2017 at 01:03:39PM -0500, mike.travis@hpe.com wrote: >> >> The UV BIOS goes to considerable effort to get the TSC synchronization >> accurate across the entire system. Included in that are multiple chassis >> that can have 32+ sockets. The architecture does support an external >> high resolution clock to aid in maintaining this synchronization. >> >> The resulting TSC accuracy set by the UV system BIOS is much better >> than the generic kernel TSC ADJUST functions. This is important for >> applications that read the TSC values directly for accessing data bases. >> >> * These patches disable an assumption made by the kernel tsc sync >> functions that Socket 0 in the system should have a TSC ADJUST >> value of zero. This is not correct when the chassis are reset >> asynchronously to each other so which TSC's should be zero is >> not predictable. >> >> * When the system BIOS determines that the TSC is not stable, it then >> sets a flag so the UV kernel setup can set the "tsc is unstable" >> flag. A patch now prevents the kernel from attempting to fix the >> TSC causing a slew of warning messages. >> >> * It also eliminates another avalanche of warning messages from older >> BIOS that did not have the TSC ADJUST MSR (ex. >3000 msgs in a 32 >> socket Skylake system). It now notes this with a single warning >> message and then moves on with fixing them. > > So I would still like to get clarification on how ART works (or likely > doesn't) on your systems. I think for now its fairly prudent to kill > detect_art() on UV. I tested with both detect_art enabled and disabled and didn't notice a difference though I wasn't sure what test to run to verify whether it was being used or not. (I'd be glad to run some specific test if one can be suggested?) The num/denom setting for a 2100Mhz CPU was 168/2 if that information helps? > Also, while indeed not strictly required, that TSC_ADJUST==0 test on > bootcpu is nice for consumer systems, BIOS did something 'weird' if that > is not true. Is something like is_uv_system() available early enough? My previous version of the patches had me setting a flag that could be checked by the tsc_sanitize_first_cpu() function and disable the requirement of "TSC == 0 on socket 0" for any arch that specified it. (And UV did set that flag.) But Thomas said it was "hackery" and that TSC being 0 on socket 0 was no longer a requirement. So I took it out for this version and made the "TSC == 0 on socket 0" no longer the default for any arch. > > Other than that, the patches look good to me. >