From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755224AbcESPMt (ORCPT ); Thu, 19 May 2016 11:12:49 -0400 Received: from mail-db3on0078.outbound.protection.outlook.com ([157.55.234.78]:48304 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755206AbcESPMp (ORCPT ); Thu, 19 May 2016 11:12:45 -0400 Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=mellanox.com; From: Chris Metcalf Subject: Re: [PATCH v12 07/13] task_isolation: add debug boot flag To: Peter Zijlstra References: <1459877922-15512-1-git-send-email-cmetcalf@mellanox.com> <1459877922-15512-8-git-send-email-cmetcalf@mellanox.com> <20160518135645.GJ3193@twins.programming.kicks-ass.net> <684587d7-3653-7570-215f-37d3e9e786bc@mellanox.com> <20160518170647.GL3193@twins.programming.kicks-ass.net> CC: Gilad Ben Yossef , Steven Rostedt , Ingo Molnar , Andrew Morton , Rik van Riel , Tejun Heo , Frederic Weisbecker , Thomas Gleixner , "Paul E. McKenney" , Christoph Lameter , Viresh Kumar , Catalin Marinas , Will Deacon , Andy Lutomirski , , Message-ID: Date: Thu, 19 May 2016 11:12:24 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <20160518170647.GL3193@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [12.216.194.146] X-ClientProxiedBy: SN1PR11CA0002.namprd11.prod.outlook.com (10.164.10.12) To VI1PR05MB1694.eurprd05.prod.outlook.com (10.165.235.156) X-MS-Office365-Filtering-Correlation-Id: 010cefbe-73d2-4545-62e4-08d37ff80551 X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1694;2:3eJZtmexABpEdYP1qBwVPKyKdRq0TDk53BvdX+d4FJ9SOUtiDdswE1lw11S7X4dZe3WylUBLn0hd69Ls1bHqMn1S75X1u1YUUGbfuk1/FhoobifbZaYtcyDXzZr7kjum7BfQPpvJ6c3kBcNoBiRCz7TPgNjXlj00UJf5CnOKE1xaqKKHyACTJufh/xTYgnH9;3:HZNrPFaZiyIbwVvIkMOp/VRPiWVUv6xX1v5zm1FwDsi0L7rO3aHvnZXbcYsglClsOEJoJTpOv9fSzftcIqP0g74VGrJoS2x0yTzl1G4xDGeqOW0hPfMnbbpBdcW57vj0 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:VI1PR05MB1694; X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1694;25:XTAq3C/LgKyKw8oGUFHnoclecgebccJom/yBsuPRMIxiF2MKv2kkSAjEEVnnFoBTXtwgAT4WpTgTv2AHgWjuc0tEkfXeMEac60H/wklXh6FQZ4/rDe/76ZuzDmfKdmsCfj2jkG1gbWoyrOQ0He7++eNLSH/Xf9/Jkl5ZL6w90TPhIze1H/MeUEjQXM95LwEax6imHFSrwjJU5IYmXDZ8IXkvtMEALGFrQMP3gsj8sDuw4fSMgFoTdr7M7WNoV5JECux5BjvWltTgpKq72mBg/Mao6XaV9DHq+u4ImGbuLD6ArIBVm0NuLWoStVGch0p6+WXQDqHKCyVy7LDdP9vvwUQOT3rXn0Kdcb/HbKP0w3zppwk+Y746snepNViISmuXR0h98rKeToYAi0jzhriNQ8mRn4HhJLNvw1aIi7HlwtuN0GdIDr9DDTOAI7S54p8mgLnovECYvJfnopZJzCQhKIrRHwxcd+l1rLNhKeIfuovB2zkb7VsZ8t+8c8c3Go9q8o1yPO2cG9VMh5QfJLvcFCINP9XHgilpcB0e6TD7gutoZ1gYaafJewV5BXRBButatvOELuqmyk5i3/zhQFW/g4xIWKUmC/1bst5M8rw/RL2LY0qU0jn84QM3eJDd6B35MUdDA5QfDEa315JGOxDLHDpWAqmJyuHJ9vtyEj8edfWBuOfZALMYlPilEv8rc7R1JhY9eNn9BzZ/eCq545z6FuQnAazW9jGr9+aa8Y+NkZcs8rY7L6rIaaExMrtbM4DLk9qiI1ciPPBOQCLDH56ixMcA4oYpjZy6g3L5h9K9k8e89Yy/mkP9Wg/nbwEfDKyqqvR2kpGgllcWgvgFf6GPzUblMm5r7rLBhdH8J+wWCkc= X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1694;20:tyyGAkJD3LoDoYrOjAEbsH6V+F3aNzNmXIeVt5KW69/yNlxES/MI3IdT+cq54EbY9PlCBFrN5B/8rI2gr995ZUeTbqWnPkCsCTFOrY2O+V+rqssgGePYR43Re5oxvpRaNFIx1zU0E7so1VG+Ff5VmmOwfQd9tsUvVo74WXhvOfgaWqNCHDjI0+CvBfgl5Bt0mgsafJv0HgC9n0po7/KyZFhphPQXvF1we5IOQraNYJbrxYbON4W6ZJwQP209nhQoRHmtp3cuVpPOEdxyLwitec4ZUA0JVqQiynbIwq/zNC+xPG/pRPKqzBzZvI/P9XE0j5kiH90HajW67jEemuw7Lw1H2CgYmburIJbq3iiRVZma6kRx0rsFrAtU2O2v49UxV8vsPerLJHHlRb1afW3x+O0euMPQ4ZWbCpmKiw0lLZDQYnpFS1PvPIKIec4+zAZK2zw5Xp+LBHCMLQGv2vTOLNpUT5s9oegN9fEjS38SSg6RagnwIt7T0YLdk/d9sSzo;4:EnYsdeqORuGSjd42nJsJ0FdOIP7EkXo2bj6TpbikQvP/NgVUgsJRwMHCPkfE6jYVTv1T9PWAyyfg1Z4VURU+KXNO5xhppLIx+AYTzMDzeL9drf6Dw0JgkRTcB46ICcpSR95lDOQ1kBxBwVi/2gqxmp6C63gbboLXAgt3Ajb2Yj25FCQBkZZOKgTBaFGaODnwhRMUaKNBExfwneWkBc6ME/q5vBIHibhl4CHTzFDrIpC0fk51JJED6UVXuyA0ql1dlfloDTjkWl+uM5Fx4sgd7CoCo9KIxrjO9o2L7FG9Q6558t4jYzj3MSOn9CQOo+DA7GJr2lnNM2DGifrO17xfPmObIOdEmcF1Emv8LpcXezBykPBWZJE3rujWwyVKXcOyXHD9qvMthzlJGEsiv1H9DQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:VI1PR05MB1694;BCL:0;PCL:0;RULEID:;SRVR:VI1PR05MB1694; X-Forefront-PRVS: 094700CA91 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6049001)(6009001)(24454002)(377454003)(52314003)(5008740100001)(66066001)(65806001)(47776003)(92566002)(81166006)(8676002)(83506001)(2950100001)(86362001)(93886004)(33646002)(230700001)(54356999)(31696002)(31686004)(2906002)(42186005)(110136002)(4326007)(4001350100001)(23746002)(50466002)(76176999)(77096005)(36756003)(5004730100002)(50986999)(19580395003)(189998001)(586003)(3846002)(6116002)(64126003)(19580405001)(65826006)(18886065003);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR05MB1694;H:[10.15.7.169];FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;VI1PR05MB1694;23:N7sLRlsPMjFCUZut5G5zz1ctEZCsy5s7SBTGL?= =?Windows-1252?Q?ddDXrhJdzPCg3vvGb7LKQypby86bJLZ9I27oSfpXGSVE49Sgwv7dY7qw?= =?Windows-1252?Q?4MgG4EAwhODMC9lCEsCRMIDe6fvdi55rzUUAzUuIyiGtlu8+6E+xMNdo?= =?Windows-1252?Q?6kFdcIuzlD/7aLwKUXwJWVbyI+3Ejm4QRCCNjdmgd21DFDarSo0Dsp1V?= =?Windows-1252?Q?bI748yGGRvNBmYu/8kuasq9I7L2XNYWZ2N7CZ3MeMzDLlHLoY9qaSubQ?= =?Windows-1252?Q?KO3aMNsGUW5Cpa620gYz+KmHDKEiPspvEu8NV6vRmz/AjEpa3i65urEP?= =?Windows-1252?Q?5Xm5YDz1hruwetCR+tqzzWRgOuZ/aAYgu5NM98aewJ9T7/AioLQi48PC?= =?Windows-1252?Q?mjysRekmU0afwTR3xAvSE2d3ADRg4mlcu8jv1NR8CzXgo0JuIK8mDBUN?= =?Windows-1252?Q?Kjj2XW6XV5a7/cibNbXwgeZWN2heLkdPNa6B+CyFZMgy4cnblLo5FVxo?= =?Windows-1252?Q?aj+b2uFyEW2t2kyfC0MKU8I6BIoSSpZWz0LEHDPMbaC9xQgsT7KhwnQd?= =?Windows-1252?Q?4I40Qno60kZAJ2YnL6TMxcMQ4lKJOYo09Sr06FBGVzpdBfB6DtpvmAl8?= =?Windows-1252?Q?y7lXvSgKdNsbNZBOfKeP8AnGILOPZr+Uk8uwjEXJzh+bXJUzAJRLCPZD?= =?Windows-1252?Q?VQKtPXSVMDhxuXLI4OogqkMKlfp32zvtFkuhjG/55cz/Ew4ZgQ40dQfo?= =?Windows-1252?Q?4pxoCv23C1GoQJoos9WYGKoeZ+l5d0fxhxo+1XoFsYghn8zxabQAF1Nj?= =?Windows-1252?Q?bk+MO6j8pjMTl+OjMgxzoppF9nkoXBGLLh5pROw4ccYGVzDkOMoNAjNc?= =?Windows-1252?Q?HR38AiG0LmmnCkAWcQbhT775s4vmebMDMuKm+lKWaT4nHC1DGQWH3fyW?= =?Windows-1252?Q?sYY3BJlNP4VqPYj6r22wz8JqnFhP3Uy+9LovA9uEfaMtkZfY6cRpX+yr?= =?Windows-1252?Q?uKNkYP7VTURsyaKY4zsFRBaTVruWS32OujIDFqiE91t7effXMj+O+k7B?= =?Windows-1252?Q?6rK1E5HfhD7y0f3g6RFFElmQh8QEtiGYtmfEcvuTJpWUUSJz7bJhgMi0?= =?Windows-1252?Q?DX2ze7M6KI/QZKskeXh6K92xWY6f5SyJPENHMhbhtfsdoZUQaur0C8wy?= =?Windows-1252?Q?Yz3Azl80YlMhFYwgX2+A8wbsY9sqplJrRnCDNw93H8NlAhMrAep?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR05MB1694;5:nj/k2SnhYdADb5NymGImwtPDKsk7srsJ7tx10xnr5LfLmg/kkFe6CKG6tgyKGQ+iOtIPctKvvunS9MTTgxs2SgSDvn/Da8LQClwN8vIp6p9A0NRcBJ9d/tPt3JVBewmjsHPsnNodPz/W/wY6DQcpIg==;24:EQgoJdK8rMB00fSFwi8kePvK6a0gSDHYKp1zT/AhtjSbpTVVNnlHtvEy3A9TAV4q5KzB0vtxnrm85V1li2Q/f46WdkB1KZIJvWx86EVj8XA=;7:d+hykgxALVNcTT37dJQqKlns7VpOPM1oEtbnruKiGE4HNXwv5GJ/MhzKYhvzFFRMkenYnWCOQSsft3CQkUVZdjpelw/Vm+4+ihl/ZTapoqPSUl5FoxtJcAqpDPSvLAIkjO9v0LV6sy/Sp8n4ZSYW3qAfnyES8SVG0DBD4jfD+hk= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2016 15:12:36.9082 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB1694 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org (Resending in text/plain. I just screwed around with my Thunderbird config some more in hopes of getting it to pay attention to all the settings that say "use plain text for LKML", but, we'll see.) On 5/18/2016 1:06 PM, Peter Zijlstra wrote: > On Wed, May 18, 2016 at 12:35:19PM -0400, Chris Metcalf wrote: >> On 5/18/2016 9:56 AM, Peter Zijlstra wrote: >>> On Tue, Apr 05, 2016 at 01:38:36PM -0400, Chris Metcalf wrote: >>>> +#ifdef CONFIG_TASK_ISOLATION >>>> +void task_isolation_debug(int cpu) >>>> +{ >>>> + struct task_struct *p; >>>> + >>>> + if (!task_isolation_possible(cpu)) >>>> + return; >>>> + >>>> + rcu_read_lock(); >>>> + p = cpu_curr(cpu); >>>> + get_task_struct(p); >>>> + rcu_read_unlock(); >>>> + task_isolation_debug_task(cpu, p); >>>> + put_task_struct(p); >>> This is still broken... >> I don't know how or why, though. :-) Can you give me a better idiom? >> This looks to my eye just like how it's done for something like >> sched_setaffinity() by one task on another task, and I would have >> assumed the risks there of the other task evaporating part way >> through would be the same as the risks here. > Because rcu_read_lock() does not stop the task pointed to by > cpu_curr(cpu) from disappearing on you entirely. So clearly once we have a task struct with an incremented usage count, we are golden: the task isolation code only touches immediate fields of task_struct, which are guaranteed to stick around until we put_task_struct(), and the other path is into send_sig_info(), which is already robust to the task struct being exited (the ->sighand becomes NULL and we bail out in __lock_task_sighand, otherwise we're holding sighand->siglock until we deliver the signal). So, I think what you're saying is that there is a race between when we read per_cpu(runqueues, cpu).curr, and when we increment the p->usage value in the task, and that the RCU read lock doesn't help with that? My impression was that by being the ".curr" task, we are guaranteed that it hasn't gone through do_exit() yet, and thus we benefit from an RCU guarantee around being able to validly dereference the pointer, i.e. it hasn't yet been freed and so dereferencing is safe. I don't see how grabbing the ->curr from the runqueue is any more fragile from an RCU perspective than grabbing the task from the pid in kill_pid_info(). And in fact, that code doesn't even bump task->usage, as far as I can see, just relying on getting the sighand->siglock. Anyway, whatever more clarity you can offer me, or suggestions for APIs to use are welcome. > See also the discussion around: > > lkml.kernel.org/r/20160518170218.GY3192@twins.programming.kicks-ass.net This makes me wonder if I should use rcu_dereference(&cpu_curr(p)) just for clarity, though I think it's just as correct either way. -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com