From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932717AbcH3RPO (ORCPT ); Tue, 30 Aug 2016 13:15:14 -0400 Received: from mail-db5eur01on0075.outbound.protection.outlook.com ([104.47.2.75]:8848 "EHLO EUR01-DB5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932252AbcH3RPJ (ORCPT ); Tue, 30 Aug 2016 13:15:09 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=cmetcalf@mellanox.com; Subject: Re: [PATCH v14 04/14] task_isolation: add initial support To: Frederic Weisbecker References: <1470774596-17341-1-git-send-email-cmetcalf@mellanox.com> <1470774596-17341-5-git-send-email-cmetcalf@mellanox.com> <20160811181132.GD4214@lerouge> <20160830005550.GB32720@lerouge> CC: Christoph Lameter , Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , "Peter Zijlstra" , Andrew Morton , Rik van Riel , Tejun Heo , Thomas Gleixner , "Paul E. McKenney" , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , Michal Hocko , , , , From: Chris Metcalf Message-ID: <69cbe2bd-3d39-b3ae-2ebc-6399125fc782@mellanox.com> Date: Tue, 30 Aug 2016 11:41:36 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160830005550.GB32720@lerouge> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [12.216.194.146] X-ClientProxiedBy: BY2PR07CA0079.namprd07.prod.outlook.com (10.166.107.32) To DB6PR0501MB2760.eurprd05.prod.outlook.com (10.172.226.12) X-MS-Office365-Filtering-Correlation-Id: 6361a484-353a-46c5-a566-08d3d0ec2b19 X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2760;2:gMBuf3vqEmJaZo4YEPwk3yrN5CplRvFMBRRnZE/3dmPRvaD9i4WwahkEyxkYsIKWiOzKFG6CdKrVfRkPauQ5icIEWdzu5LsbLQBkmJtzf4KkYlO32yxu0elZj950+/SFiL695DeueUPn/ogNSQQHVzhK9dLZWz1cXv+LOq1gBvBYuElwFEHRuEwUAQGwD4Sq;3:CFv3ycY3XuYtIoi6u7capnywuLoM2ftzJiezSHk7BsawP2KYEdL6P8dOui7uoxrocdPks8JaWtYum43QYmBPbT9ZjNiapz7Ef77zk0aRuoXggXtUbU5yBnUB28N8MGPd X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB6PR0501MB2760; X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2760;25:FM/CIE+ubqfmAwqkyeodDkRB+Nv989Mq2qho3PWsqlCljpSucTQpMbciN948ybKclDNLmgF2qWBK2vaRjSP17DTtMClEmerEDBfwAndfLTa4ft9r87lI/j10+Y620+lRc4Pggg2o8p0pDcKIsxkKdHTZ8yfNGOpEeT0NvzEMGsyBeFIVgzqGC9i4QHrh19+yGYZW8DM7DelkX0fJyb45PYQKf+efPcqfMwgddJaxCRjFZRqm1cPJ5eIVPMOjapF+PwVAgSrITqcvEtDo0dWH0KMTbyiS48oFE1I7hIdMOgMPo7GHLnC55k7rLYMC+LNyU7T3VrB8AkXBs37KedLemVoIADcvBID5lcN3fjAe9o6CXFbb9C327dFQWAUwi0HmM45YWJe/Di4+KlArvmCqmSD5c2G9MoER0hzO11zfo5JOltLjqEsvnmCRB5yKipIOnpY0UFyoKyTtULUytKVlQcuBtRDOufPYF6sD3+mqTXlDcKMUrid8rwdJLAgQuj+mU/XVCK66srzUR2OyUE6LFWZ5wGBMZ+jtOsyBAkjoXAALllblLiynpWXYh/rI4SXyWMfEi6Jfv9WPrIFeF8RJSmdbih8uJUPQLIPmNxr97S01B0CTNOtiiVu/V+QrqUwrv4u7QbAsGgOMNtpDDlgxn2aMSFZhyVhDdxBq8f9DjIEoWJA0oaHdNshVmBuVpkHy/AxSx+dYwZ0m6zwmwyHXnBZuu1Bix5ypqzRUvgDSWIx1ugU637Kopx9+2MC3tcKSD9bSKQWw2ru0H7AVgcN44Ii7VDcwjuogB7iqzFrIJqbzDCbLW/8ANwTkQRPNQ5O4hPh2WmRH0Xl2rHPa9EelHg== X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2760;31:TDBktfvGpMjTOIzMQheGOK5vjsbaTTCJIfCUvt91UkpLwGGER3ridCOdF77B71i47G3Yhac675OA11CamT2mvJpGXBKNRQzqhslaKuZtwSiCfsJAgUcCUKM621gYJFV/oDa4+6mXsCYd4HSZWg6aK4hnUesuXVW8QDZwncwtXaRl7cn8Ocxo1HqmVzHyLnBAA6tyBRBUHp1FIG34ep+rXv6je3FobjO57/682orJlEw=;20:bcjCaeUpMRohrdE4q7GzSudcFxMB6i0ahWC0iw3f2eA7UnI4vT0V/xYlYC0R4Q1Wpf0PkZqJzmDGPPfejuFFSaSXsrw0T8fAccUIBegCZyEun6aDEbSjdWnooUYrD/5zEappqEo04bVlNTPVY04hKlKbd+ifQUHalbZO1HAW8tRVzIOTBVsSt4Xk8TOWBD6mc8lPuzF15zMq6nwjs7fz9zfPLVABOGRAPGLQ/kZJFF314Xl6ysccfn2KOQ//b5xQxsLZ5Sl9bWRK+bb2QNZuznZuKlK5dITnF54H9D1goI5DORqkLf75yF+RO7nmRt26kZp/XQtHLcaYpnrz5nRNqrQOe4P2rkUEZF5SWewalk1Xybzd7UYFsa7Abr/5M25z2IcZVvLG4wJibQjxD8PvkeyNE8uI8zBw/zpVawCIKY3m01aPjBdOuUgEYwc4hgn9QMfsYey9NoLSpaxp6eNt5Y9JNB5VWOOjWhnxufuTWGkWRdCSbdR688FLhBgSNNK2 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(171992500451332); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026);SRVR:DB6PR0501MB2760;BCL:0;PCL:0;RULEID:;SRVR:DB6PR0501MB2760; X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2760;4:MbcGXsW32vrTfBxKn8dUk1HpqS3YZtSpEuCDKNf7ldIufVIKMLqXsZ93eqK/7RTOUcsIs8fQdecz4XmBJo8EXKDSnlF7I3xshQiKmrEZUcQdtv5ZOo2YO1WohDdmZL/ti5kuVtH45aoGfDJI07qZippNZvQP+VO+q6zREdWETUjPodErS/L/JFbKJvkp6hFYHc+di4soquyuVcnWo9A1TF1jHanRUi0X1Ysm0ePPpmpQvz8oRFQZHMvvsWUscg++3FU3vbz//bVrwy6ZAdwRcvQhqO3hFlfbuA/msQ/olGh8zYnq+wDGxqae/7p9FCNdGZhUnFLTvYitwHQSC+RdhdGcArwIJa5UHzbp19MAKP6+nA6nVy/heRM0/7slEl0NcuRXvrbf4FbDorN6b30FfW51KVaKGSGZZPMrgd4u02xTx/gvDLaYPdcbOfYpLkJw2ZJctZ8vgY6SFsmyUAcZjw== X-Forefront-PRVS: 0050CEFE70 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(377454003)(57704003)(189002)(199003)(24454002)(101416001)(7416002)(2950100001)(83506001)(5660300001)(68736007)(189998001)(15975445007)(31696002)(92566002)(106356001)(77096005)(65826007)(4001350100001)(86362001)(97736004)(230700001)(110136002)(33646002)(105586002)(65806001)(4326007)(305945005)(2906002)(81156014)(7736002)(47776003)(65956001)(66066001)(50986999)(50466002)(7846002)(81166006)(54356999)(76176999)(31686004)(8676002)(23746002)(93886004)(19580395003)(64126003)(586003)(42186005)(3846002)(6116002)(1411001)(36756003)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:DB6PR0501MB2760;H:[10.15.7.181];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DB6PR0501MB2760;23:DdtZHNIf9zFzIP+lY1bl7Fu4ACLrqZ53xQW?= =?Windows-1252?Q?LPveSMe0lcays6ESeumZkdsCNj1CsSO/eCMEhfMt3HoMaPM/tKkEekWw?= =?Windows-1252?Q?/b9/7vNkkMpf6h7ZXFtUYkYZKlATnlkaI01q784KzMIAPdvyD5hLop7j?= =?Windows-1252?Q?4V7jKx/Q7XlgHczdx2KSerB7TM8Y0UQravQFmgZtgGCrdwWsNjWr4kNp?= =?Windows-1252?Q?SN1eFgH3uH1lJvVrejdOMXIxDqOMdjm2+yUOzVdiHSQJODHP2dI9tbOH?= =?Windows-1252?Q?BS4EEaidB5XuH6o5/zqAlgvSPH/zrOwDV2v430cUwEQit2i+5EqF1gbV?= =?Windows-1252?Q?mqyLvNg172S8bV928WfjkhORHD/npkSSTXVaWYaf8aw/mOKS/Ir92Z2r?= =?Windows-1252?Q?1eFDrykjGv5DwsYYoVdrFgxpi0DmiJNLC866A5CAud+7k3mFZLVo81AH?= =?Windows-1252?Q?QaDUkriIjB7qQRztTzVRXYybEwXlb15v/q8NFWlcnDADCKQj5nagS3PF?= =?Windows-1252?Q?4H4Vi8EtN2RMD/wZbbg5qS7iJbI0hDUljguweIjolE/DB0rXEfoM0HRu?= =?Windows-1252?Q?HAgl58NlQGi5Lxvun7JVNQNFazSG6pP87HrZUU3CFAYo0bBKBBQxox7I?= =?Windows-1252?Q?+AgP8SBgyZtWUILU8emv2lcjxnIIQj/2GnewV/FqR04M0Pe7JLbGSEv1?= =?Windows-1252?Q?Vwd9J53DXCsE328YYmpY2QXQN+Qg8uwdaxF0Jbk21VlJAREcgIaHKRwm?= =?Windows-1252?Q?fxePjoOmrkFEikBuBUAUucqp576xw6iIyzeIipXL52c9ZkhRdNNDTmTS?= =?Windows-1252?Q?hjKLVj/xInGiF1Ow8ZmZJFt1SkF+5qZDuoj+fE0ZxcGFV4M7ckHkmLMX?= =?Windows-1252?Q?Rj7IxpmcuQmBMex5nMVqPSRMLOsln6hMOn0xNxWo31j/4BU+l/X+wRfu?= =?Windows-1252?Q?C7A3FXCEYcwnm88hQvg/lQjttbkHbtpo3xcEpOGJnH6ia+ZBUjn6xUaD?= =?Windows-1252?Q?OLir0pIWrvkAA6LRy+/pa2ejDo5HF5i/T9+v9/Btmqrd/kAHVjx/mY4/?= =?Windows-1252?Q?0Mpqb1KnVvMO8ak2bxJOUEhjDjhq8pspl2eW/oASQap7U+jUwbOrH7/c?= =?Windows-1252?Q?7gkGYwLUwakkcLxJtzg4hn5vJbMFMw0WRg+Zpcs7d6Bg/irGJW0n9OvK?= =?Windows-1252?Q?0WPj7Dh4oIcuYkpsYoMjerzjhkDM/0WO2LMufERve+/Mj4zWgNFRPQCv?= =?Windows-1252?Q?6NuWCk21Ja9HoGBc+GAOXlW7Ac+I8wopVM7cqLka9aev4IeDFKk3oSIW?= =?Windows-1252?Q?2r0HtrJTrNkZkRL2b9HgG7QxouEeChfGTetLAiKcuhRqsiLKai6iBNq2?= =?Windows-1252?Q?AHeWEZDUAcqajKwqAn0YGFP4E7Nb6St+rQScuy9qUqCnZZXXHs5DLQGi?= =?Windows-1252?Q?W9XGuZI3opPslfks6ynJPTZFX4MNO+DlNBvESKlsI85wSQMf+4TzfLHo?= =?Windows-1252?Q?j+nQs6NqC0zCPwavUvIuXl6Pul/nv?= X-Microsoft-Exchange-Diagnostics: 1;DB6PR0501MB2760;6:k3liYRyiWJe6Gu51f51A4PyvYOHaX6AcuXTzEzSVfmr9RIbyc2T7LVYKbv5BoshBhm0oVj3r/5AS7n/VrValK9vK0XnCthXzx8EscJocDAlF9J0q5D4SUCJ4E8OjpHtv8CEA2rke/aS40oN0cPte1+3a9kEBKjJfVhMtTgcqn6EDFiFSsUHi4zFmYbauVgN0L7Hh8UR1GwOhGuFrMSSTEA+NHu0IjkGpTKmSy7vM0+SEcggxfkQtp8cSUsCg0pl79UcFJaYRks7kwLeBp2n1u0ObUR31bJn53TSuvRvc+hn5ppm5hKD1WgRZqnHnnhyJfDmtY6wuppQijheH1d99yg==;5:Bs8LRioFLazdXCeRcw9KUDlKWpEQ8ScKrEydlFtNfJ2HvCdaqMHpclOBg65z1hF5u1SGL1P9LKqo0wcW+470H0Ge9c3Rht8gHCzKKH5Z8QLsudAPvyCQX2o0Uhp4u08XWdTZhI+LQGggWm3IKoA8hg==;24:sOMKsAeE69capHRGD5kS3UTngg2WH2hl4GZk+WHR/b8g6Uqid3OVTw2SUiF8DQtYLNW8Fikm9z/qzYjx86aZPRVKJAH2piTkYa7YEDHzXms=;7:pdTZspudrHwBzkwzmrPsb2D29BHnRfj7/fzd6IlfgPAoer8qkDFfVNQ0CAQukB0GSe+dPHY+QSsz9qIb/68TdAyKeWaDtzrA0fcs3rYcHwx5kxgLsxTT4wGhIEPwiP3/Ep7XjiF5dZEZqCQ3QiD4GJ9ObrsDMOilNTqXNpFSINsDEwIIGApZyOxwr4CMDnMLlj4P8YGYFIvnX6qVSxyCAvMCMul2Mi5iEmkO7n9ESWIblGzjqgJZGsigkj4bJDdv SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2016 15:41:49.1689 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2760 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/29/2016 8:55 PM, Frederic Weisbecker wrote: > On Mon, Aug 15, 2016 at 10:59:55AM -0400, Chris Metcalf wrote: >> On 8/11/2016 2:50 PM, Christoph Lameter wrote: >>> On Thu, 11 Aug 2016, Frederic Weisbecker wrote: >>> >>>> Do we need to quiesce vmstat everytime before entering userspace? >>>> I thought that vmstat only need to be offlined once and for all? >>> Once is sufficient after disabling the tick. >> It's true that task_isolation_enter() is called every time before >> returning to user space while task isolation is enabled. >> >> But once we enter the kernel again after returning from the initial >> prctl() -- assuming we are in NOSIG mode so doing so is legal in the >> first place -- almost anything can happen, certainly including >> restarting the tick. Thus, we have to make sure that normal quiescing >> happens again before we return to userspace. > Yes but we need to sort out what needs to be called only once on prctl(). > > Once vmstat is quiesced, it's not going to need quiescing again even if we > restart the tick. That's true, but I really do like the idea of having a clean structure where we verify all our prerequisites in task_isolation_ready(), and have code to try to get things fixed up in task_isolation_enter(). If we start moving some things here and some things there, it gets harder to manage. I think by testing "!vmstat_idle()" in task_isolation_enter() we are avoiding any substantial overhead. I think it would be clearer to rename task_isolation_enter() to task_isolation_prepare(); it might be less confusing. Remember too that in general, we really don't need to think about return-to-userspace as a hot path for task isolation, unlike how we think about it all the rest of the time. So it makes sense to prioritize keeping things clean from a software development perspective first, and high-performance only secondarily. >> The thing to remember is that this is only relevant if the user has >> explicitly requested the NOSIG behavior from task isolation, which we >> don't really expect to be the default - we are implicitly encouraging >> use of the default semantics of "you can't enter the kernel again >> until you turn off isolation". > That's right. Although NOSIG is the only thing we can afford as long as > we drag around the 1Hz. True enough. Hopefully we'll finish sorting that out soon enough. >>>> + if (!tick_nohz_tick_stopped()) >>>> + set_tsk_need_resched(current); >>>> Again, that won't help >> It won't be better than spinning in a loop if there aren't any other >> schedulable processes, but it won't be worse either. If there is >> another schedulable process, we at least will schedule it sooner than >> if we just sat in a busy loop and waited for the scheduler to kick >> us. But there's nothing else we can do anyway if we want to maintain >> the guarantee that the dyn tick is stopped before return to userspace. > I don't think it helps either way. If reschedule is pending, the current > task already has TIF_RESCHED set. See the other thread with Peter Z for the longer discussion of this. At this point I'm leaning towards replacing the set_tsk_need_resched() with set_current_state(TASK_INTERRUPTIBLE); schedule(); __set_current_state(TASK_RUNNING); -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com