From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753134AbcG1ILq (ORCPT ); Thu, 28 Jul 2016 04:11:46 -0400 Received: from mail-ve1eur01on0063.outbound.protection.outlook.com ([104.47.1.63]:18718 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752686AbcG1ILl (ORCPT ); Thu, 28 Jul 2016 04:11:41 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=cmetcalf@mellanox.com; Subject: Re: clocksource_watchdog causing scheduling of timers every second (was [v13] support "task_isolation" mode) To: Christoph Lameter References: <1468529299-27929-1-git-send-email-cmetcalf@mellanox.com> <7a3f66f7-5011-7d59-2e0e-f57e4e42e6b6@mellanox.com> CC: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Peter Zijlstra , Andrew Morton , "Rik van Riel" , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Viresh Kumar , Catalin Marinas , "Will Deacon" , Andy Lutomirski , "Daniel Lezcano" , , , From: Chris Metcalf Message-ID: Date: Wed, 27 Jul 2016 10:12:02 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [108.20.173.108] X-ClientProxiedBy: BL2PR19CA0019.namprd19.prod.outlook.com (10.167.113.29) To DB5PR05MB1685.eurprd05.prod.outlook.com (10.165.6.151) X-MS-Office365-Filtering-Correlation-Id: 65122a00-c4a4-452d-fcc7-08d3b62801c0 X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1685;2:uRaORM3xFOUsqaUzcdveRUFIuVZ/RGjD3BtcZz2RNcCJ5HzAm/lQwx6OgFazXG/Pq307Mg1KXN4R6ESX4d+Bc47JDiSK6jMHwlXoHgiP++AQ/LstOGQZlgc/HnDYiZv0Omxh3kRHYYXVYuHrMXa72yrUkf8NRBpDPZnIv/MoFLllD6u1jvaq36YQWMKl8Vyx;3:PGhAGBKTkNJJFERR7z2x3uchHD00+6B99qIL2qlfOofsYL4b9pLxJMj2Y9M3lcHkne8GDaj9ZpFRxODIyh5s9JXvea2ZeIG6+3wEAxvPJDi6TjOGXWcCao/Y66ydr4KK X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1685; X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1685;25:GckrT8thyGbYWjkdIkIwlPYCckSq75aoWdYWE+GKIY23HIdkR119xfvsi9HygOKwxpgaUjdhGzXNSbzaBIqupjMol9FCHcyQByGmJ2FwYJqGsi6fkMIicUnz1BomUCdmPsIErrMEUXk/lEcFs5NqJytR7ELHrZM7X+0DXK1cDBZL7WKhHl4C4qYmtkxarcux8PxXc9gIs7PA2dw3sNRqk6sTmjQDnDNyT286IoQmwUnU75hSMcV9QD45zxJkWCVV6isRUsYnB0dVIwmNIfylF1l5BFLP4y6QJu5Z2FRnH4erwd2T8Tup1IFIusDyCDFE1cLxF9GIhdbJITuPJrE2cxWaDpuGn3PjFzd8YFR9ZMKbpRH3WqNbk/Mjamwmz+mcKhxC6vsGa8x03MhrCb3mFfHKXzdleF8Xs8d6PiaDJF7YteYDN8MjjPuejuVt0a+aqG2yn8DZS+V0455rWomieT5HB0Yhs7VpWDloIqZksaaddq/SdZf4T71FQZFiMDskfGNMiShyL2Pq7KV78kSDG0Nsf5G9iuJad/ws0hv0frVhcobYy4pqlwo4T3OoyuTcb+XOxZLa3evGhM87RCxRIkB1j2fvBktePiEZru8ovsyOCElRRk52E11BVzlTub9+EGGUvrLzJhr3bJMrCxpM6wN/pgwyJnwtoB6Z66M0JrwPmTzjilk2F1hRQl9uArVH9HK6SWzVoRfBbY7wkw8YxyF+UsVwcTCI5vfhbWCWFY/O0Bd+VQqA/J3F+nZymtr7c/yatPpYF4acYZJ+2dI9ZcRsFzjEjX32Wop2P1WWyr48MjZC9N3QKinZ4ry46CcC X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1685;31:J+z3WvooSJQiRYEEo9PpKHvilSgNvjqz9328HzdORTbtvK8uQYjEMmA3an1AxRFc0WZ2GdleMw9fL2O92TyEjvGgJddgVSKmZFOpa0sZBDDXCCxd2ypUlCX2L5hXL7Knroc3okybdOQYhENO/VhQQd10FqM4TTkv1W5rd+jtMpgHAYxhEH9hsyroNEwf+Z8zFFBnCIf4diFeuupaJU/BCg==;20:eBusAVBjnkShhdf0Bg9L2g7LrKvJtwfdYWVC3kTq8WZPpL2gS8W02wtQl6PquvTFa05JPN7EOuvy60PuOnewDcWyo5Mc0Ger8U22ZQYqZKkNWYYQRPRnnpOXWcEQKluu8vqj09TDtlFsQ4BDEVmMVPsG5fKawYze++92eAHnstDH36eSPRJTB1+jT1XLFc+SenTd1t0MzlWLqrsmbvH2GZbn0zfF2o3kYjDKjCP4ud08WmfvhTzdrsDXVoBG4HVpOV1khyLvCxSwgVJPXmip13LiMFphw3ukIaRLCQeUN2RAFSDSeCq2aiEtVSEVic6K6o8WafFzRBIGPfuiDJGJwNT+cmeHohOGwZywG8kb1r0gnQDLglg+MX235xPsa+bt88pytoBO1vmhHMpn/gDzuV4TMyCo7bX8Ipv0WXHGFKs62P51xFg2GBY6AWe2U8g7bDlt7nLc10AUfe99eZIXXYsk8IeHrF56lMBsq27q7XZN220OGgOXmgEYlrFzdX// X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851)(171992500451332); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026);SRVR:DB5PR05MB1685;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1685; X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1685;4:+6Iq09MZu5KbGmCOptYTmLnjipfHxMSUoFfntqM5f6qgg1V/T/w7GuIUQXeprcGAkBHoGs5bYCEYOBoo/Ux2JEdgNcFfD+SFAWCx7EXaswCt+Ml4E/DWBmiOS8W3zF2POoc6drkQHJ3P1X9l9lfwx1LwrwOdUDeRV5J2o3y7mUnrCiu7YHZlVfINZ0tSHbNym9TIDgVXJxr+6F4s0BiQa4wfP1pJuirtBOG5f4ObMmZyMZQ2+IwazyB+RZMLThTZc2k2T/XLaRJn1W3GmjwcAgVWJc05q3t39qKtdThS7kmk71M4cFMekX0LeUiPxV62nlm0TNqvO4lLzGV93SDeGDLZEIWj7oOtEn8i0WZpVa363rg8QmENPu2JPWTDgFTyXPlSfrffWVl/T72mLHleCfY2X8hL/wMcdulylU15DmxM538HQrpJzKNLwFAoh7dnGGwT2y5DMMh2zUpO+UTCFA== X-Forefront-PRVS: 0016DEFF96 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(24454002)(377454003)(199003)(189002)(8676002)(64126003)(4326007)(101416001)(68736007)(586003)(3846002)(6116002)(2906002)(50466002)(86362001)(7846002)(50986999)(81156014)(81166006)(36756003)(66066001)(65956001)(54356999)(65806001)(23746002)(47776003)(305945005)(93886004)(76176999)(97736004)(33646002)(117156001)(31696002)(7736002)(106356001)(92566002)(105586002)(110136002)(4001350100001)(31686004)(42186005)(189998001)(230700001)(2950100001)(19580395003)(83506001)(15975445007)(77096005)(65826006)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:DB5PR05MB1685;H:[192.168.1.158];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DB5PR05MB1685;23:/IuBTGfoQyT/aADO/Pf7u2erFBt1WY4psoENw?= =?Windows-1252?Q?0SvNEbxhT2Z8qh8kyTeqxzbuEaXcFXmV7z+H1cIU0KJmnXIA7Ev669Xh?= =?Windows-1252?Q?bI9enPRYbUIgdeNab6X849FN1vrwapRcAwirvFYGY/Y+i02F5pQ47fJV?= =?Windows-1252?Q?AdOQX3OPNZxpj//4KLjcWdxj4T+HeIBUNOlS4e95dn8SxiIEkkTntdzB?= =?Windows-1252?Q?qGzK7gmPh42Bq0CbfBPV21v/nrUxzNYGg0wLjMcLeu86BiBFNGPyxbqT?= =?Windows-1252?Q?vAZHO4aZ5bwGcTgxA3gRQVrqVNH/k70WSnkgYKsLgdkPKQU2B/BingVQ?= =?Windows-1252?Q?Zn9sJvec0KETbZOpLpbJCl2ap+5zyhZRMhPDRyEcS7wBLe7igGDJ2pN3?= =?Windows-1252?Q?27/jHC0nM1haXqIl3j0AN3yU7c0zFiGPUTwCH02ySJtVDoto4gTxKCd1?= =?Windows-1252?Q?gdbfNaUARo2pzab3LHdVq827iOZ+5HFKT8hrDXMa5tR6nEEllxYR+UCJ?= =?Windows-1252?Q?515GEIg0tAZC2K77JxtA6ogq6ai0M6rvZNCP0DgtvquKQE+gEOkW5exo?= =?Windows-1252?Q?LJPl9LCj8OjBimEBCZxzsNe7E+ppY3bXpnxFHnTeXCvzbnRAoyC8qFX7?= =?Windows-1252?Q?mny/R3IDGM9MS4oESrRdp9R8ykxuB68VOlPkjTVUNT8JBzAaUk1Jr6m1?= =?Windows-1252?Q?O58WkgGmE+ipfjFqZtEyKhynO1yf0PA16yZvfxZk5SA38HZDq7xdzb0l?= =?Windows-1252?Q?luXigNz01HDqUUocwcfoHqafbtn5j2lX5ZiV+6+G25FYkZgAWIQaHRyK?= =?Windows-1252?Q?FM2qLzp4c7InRxW77yAFvRKSXB/+t6cQ2BmEQiPqKxg3ULKiTB5SfvUb?= =?Windows-1252?Q?zhnDF4SCe7Iq17IfkZikqnWItCaX3TAQ+x1cNZCH7LcCp7/cbtEYpQsu?= =?Windows-1252?Q?DJKCgGKsADnVt0A9TUI1ff+9oS2D/Gp3ucxsLiWEWZHP7E5GhUVdFV8j?= =?Windows-1252?Q?fZ1H8uiugPkARlvlPA7AHxMqENkcKgy/1duwYQXkgt502eKzKgXkN9Fj?= =?Windows-1252?Q?7OEyvW5HrjyzEteRSfqF5pvTGW1rUxnce0nRAxGRUBmqKdiQEdbSTRgI?= =?Windows-1252?Q?RL4mvWvnOVzO3NIJpVs1Roj1kLTqrpXEE+XaSnu+2VFFMiTFFwm7b7lT?= =?Windows-1252?Q?uzP2etMMBhMpvpnNKzWL5pm60yjHbei8M7uuEmSzrGOhWITagb0BXkH/?= =?Windows-1252?Q?YMn/9/07KRZh88l3vhVgj0oiXiuDU9uZN6GFiDoaoISxRFtSAgTJru63?= =?Windows-1252?Q?vepadpA3BOA4jkFHcoEkDEnZ6VZoC3v3cN5W1g7tQu26i0Bqx8gdfyFf?= =?Windows-1252?Q?nxptroUmDPmR1DUdMK2LXP+zBbzC2HSEKBrusDtj24AG3bF46jtl2TKx?= =?Windows-1252?Q?bZCmHTpGyeSoAERZC+n?= X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1685;6:H+xRUUT7QEcuP3Q/mKqFkaDkDS8BIVDdvsGua1YZphdcrAmr7Q8I3yvJSHG5dohVRnF54BHqey4UT908vbNBrsDEX2rNgtfNZUxrn6Ku4MNgm0cwtg3O/4QdhEjijI3mlCgps0gkJ1uTUCiOLYXssfHx5VCB+19IMTf+zGboXrfebEc7QpiQrhHNpFC8qw9HbJ+QbF4A6ZaHZdZr6EMNAzLW1FDgcSWX+V+CUYhra9NUoVcgKuiVIAg8TkjTymnPwro5Ardac5Fm4YdWrbs06aZq/4Kn3dmMWxHzGU1KOjwL1FS76esmWUz8oh6eI/bts7UxET4yLprxVORkVz/wug==;5:JWVwmcl44foo5ptYdJ1ph5RIn8Z3W+kwywwmvQZRCSKLYJa8ZWTtmFq/MTfl1uQtRr4V1cBWuxlEwFIIpBPbYUH0CGLvKY9tUvQri5SnO4+gnEVvis1tFcICn6MjnrSVGKdf86i7fqSimnP5so84YA==;24:bvbxEShCLbnkLfV/WXQ9Wc6o7DJHJ8GqJUWSmLVnXyu27p4vHXzeu39FSqXMFlLBBEJnUZC8+vr8IKGsouHDpzt4oRFjJ+lBqWDznyIqwpA=;7:42rwdv/3Zg/rzHhySg6JWY5xRk0ZWUQdwRQNu+IWHEcDSv7u0heHOa0dEq0o5Qezw9RulCODoszsS9wMMwpaQ5NN455Qo/joIn9irurK+rHdDXJbOCm164wIXSAlk1rm7I7+maOZSVUNT1q1iqh/NxTVcvtrLHZnv9g9fJ2kJJsKYBN0KGdelPQ1vO2I7MHqus7G6KYydYVWksBTi7BxqlImZsr+F2mZRnmQkOQ/1JCuzp9HhUo5NYjPAUCFJGB+ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jul 2016 14:12:11.0545 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1685 X-Microsoft-Exchange-Diagnostics: 1;DB5PR05MB1686;2:lmLBJcUA2kCrujk771Ul6+MQIJuRYDMllNe/W6Dlv8sTickQoMLsShUaAJhtwRMPXqE1nX+ui89yrDESWBbKc4hZOmfoY1pPcG/EAiYPMf0Yx10PkNzivUE5ngfM9IOcYFyGOaW164E6Htc9mLuNul2rArsPQoNdhBh+qvqE/gkWWqSPoU+K6g0C7A9xxuD/;23:GgE6He4MHkyt6NklLva4gCVPnb4JkRTgS6TLXg5hmijxm5iZSg/vEjGpYFoKn6Wn1dYaQtyu9qnWLJQbOz5zoQ7YZ1Vt4aJVG4aBYXt2Ehm2tYxgXtU3MhEOECQ86nVOdEdXWstkzh69PlUlfzf4T7232aY4hYArq6CqnVKn7jPpO580nVyQNcrBE6K7YTO6 X-OriginatorOrg: Mellanox.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 7/27/2016 9:55 AM, Christoph Lameter wrote: > The critical piece of code is this: > > /* > * Cycle through CPUs to check if the CPUs stay synchronized > * to each other. > */ > next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask); > if (next_cpu >= nr_cpu_ids) > next_cpu = cpumask_first(cpu_online_mask); > watchdog_timer.expires += WATCHDOG_INTERVAL; > add_timer_on(&watchdog_timer, next_cpu); > > > Should we just cycle through the cpus that are not isolated? Otherwise we > need to have some means to check the clocksources for accuracy remotely > (probably impossible for TSC etc). That sounds like the right idea - use the housekeeping cpu mask instead of the cpu online mask. Should be a straightforward patch; do you want to do that and test it in your configuration, and I'll include it in the next spin of the patch series? Thanks for your testing! -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com