From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752974AbcHOXkM (ORCPT ); Mon, 15 Aug 2016 19:40:12 -0400 Received: from mail-by2nam01on0068.outbound.protection.outlook.com ([104.47.34.68]:43056 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750796AbcHOXkK (ORCPT ); Mon, 15 Aug 2016 19:40:10 -0400 Authentication-Results: spf=pass (sender IP is 63.163.107.21) smtp.mailfrom=sandisk.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=bestguesspass action=none header.from=sandisk.com; X-AuditID: 0ac94369-b97ff7000001694f-62-57b250e038b5 Subject: Re: [PATCH] sched: Avoid that __wait_on_bit_lock() hangs To: Oleg Nesterov References: <4091e252-18d9-1795-de63-9fbc678aa6b1@acm.org> <20160808162038.GA25927@redhat.com> <78fafdc1-d4ae-a9a2-169c-1d456b6e6e41@sandisk.com> <20160809171459.GA13840@redhat.com> <3cec7657-caa9-92ca-9f0e-34f073a6ed8c@sandisk.com> <20160810104555.GA3333@redhat.com> <4d2e02f8-c7da-ee1a-1068-25492cbffebe@sandisk.com> <20160811173651.GA31803@redhat.com> <20160812161611.GC30930@redhat.com> <20160813163208.GA938@redhat.com> CC: Peter Zijlstra , "mingo@kernel.org" , Andrew Morton , "Johannes Weiner" , Neil Brown , Michael Shaver , "linux-kernel@vger.kernel.org" From: Bart Van Assche Message-ID: Date: Mon, 15 Aug 2016 16:39:57 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2 MIME-Version: 1.0 In-Reply-To: <20160813163208.GA938@redhat.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCIsWRmVeSWpSXmKPExsXCddJ5ke7jgE3hBu9+yVnMWb+GzWL1Jl+L NZMesllc3jUHyPt3itFi7/wGFout+1sZLY73HmBy4PA4/OY9s8fOWXfZPTav0PLYtKqTzePE jN8sHu/3XWXz2Hy62uPzJrkAjigum5TUnMyy1CJ9uwSujCPTPrAXPOCtuHFnElsD4x7uLkZO DgkBE4npLz+ydDFycQgJLGWSaNn5lR3C2cEosePFL6YuRg6wql1nQ0AagIoYJZqfqoLYwgKO Elv3f2YGsUUElCX+3FvADNH7llniw9f1YA6zwEomiWf3J7KCVLEJGEl8ez+TBcTmFbCTWLxi E1icRUBV4vv2p4wgtqhAmMS0u2/ZIWoEJU7OfAJWzymgJ/FoLUScWcBCYub884wQtrzE9rdz wJZJCDxklbi/fxYbxKnqEieXzGeawCg8C8msWUj6ZyHpX8DIvIpRLDczpzg3PbXA0FSvODEv JbM4Wy85P3cTIySuMncw3n3ifYhRgINRiYe34N/GcCHWxLLiytxDjBIczEoivH/8N4UL8aYk VlalFuXHF5XmpBYfYpTmYFES53W79yVMSCA9sSQ1OzW1ILUIJsvEwSnVwFitWbpboPp2YGD+ o88hswuZpxy8mc7zmdHrup3lOYO4uKDbCXJ2T+qeXmJfo+xzQagu31Hg6ccOQemPmyazGlU8 Z/hhuVcn/aiIFXt5y/T9H7gnJs/6k5y8bjubNldvjouYjFDNZyPvdS7vzqxbfW5yzZQfM1Q/ hz2Yf1XGXiTpW0CS89STWkosxRmJhlrMRcWJAD6imyanAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupiluLIzCtJLcpLzFFi42Lh2siRonshaFO4wfP1JhZz1q9hs1i9yddi zaSHbBaXd80B8v6dYrTYO7+BxWLr/lZGi+O9B5gcODwOv3nP7LFz1l12j80rtDw2repk8zgx 4zeLx/t9V9k8Np+u9vi8SS6AI4rLJiU1J7MstUjfLoEr48i0D+wFD3grbtyZxNbAuIe7i5GD Q0LARGLX2ZAuRi4OIYHFjBLLbjYydzFycggLOEps3f8ZzBYRUJb4c28BM0TRW2aJD1/XgznM AiuZJG7e72UCqWITMJL49n4mC4jNK2AnsXjFJlYQm0VAVeL79qeMILaoQJjEtLtv2SFqBCVO znwCVs8poCfxaC1EnFnATGLe5ofMELa8xPa3c5gnMPLNQtIyC0nZLCRlCxiZVzGK5WbmFOem ZxYYGuoVJ+alZBZn6yXn525iBIc2Z+QOxqcTzQ8xMnFwSjUw9nQfmWDm8M7UJ/fy+2dnS7OZ Lbv0irjyNzxP75fz2/Jq583bEs8Pzm1kzVdZMXvnTbaCCscXjvlRV/WDX+/arOy5WDH9yokN 5h6Po/dras8SzPP6fztQaAJPk4QKnw3TllvW2nIfWCvWVe9kzvTXnhoZ1TxZ2sKsivMx07aS 3Rs3Cpxb7jRbiaU4I9FQi7moOBEA+CUwgB0CAAA= X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:63.163.107.21;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(189002)(24454002)(377454003)(199003)(2270400002)(2950100001)(31686004)(93886004)(77096005)(4001350100001)(36756003)(586003)(86362001)(97736004)(106466001)(305945005)(69596002)(626004)(110136002)(87936001)(7846002)(83506001)(189998001)(7696003)(8936002)(8676002)(81166006)(81156014)(53416004)(23676002)(68736007)(4326007)(92566002)(33646002)(64126003)(2906002)(65956001)(65806001)(76176999)(11100500001)(50986999)(230700001)(31696002)(47776003)(54356999)(50466002)(356003)(65826006);DIR:OUT;SFP:1101;SCL:1;SRVR:BN1PR0201MB0802;H:milsmgep15.sandisk.com;FPR:;SPF:Pass;PTR:InfoDomainNonexistent;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BN1AFFO11FD017;1:ux3uk4Ze6u5DhszXb04TuOJhknI1BHWGdgo9h39TwJWdlK1/XtmLfxgEhJOtxFOxpysJAxRy5tplf1WP7DG2SXlNFCb2/LMu3XrrKCmSoZ+KdFnt36iQmKQ5PynI0LtMFLgDwuPICJ9SWQeqZeFXHzrajVIrgJffd61u32SW9/dGroxdySqhhVk5C8fY7FbVdGDUpLgEnKPVP5PPBHn5PnKMY7ISn2wdat6qba6t+LH1UMM8duoKW4laGqOi4OFAcNQgFgWoUBKo5xI0gXXV1DD6xiKezqFaN2R3+hGrjPQX3yTzx8ZGHil+TN1z3YGtTSz9eSoRUvgU8t48aDDv7ZghyE68NHzZzRVykR6l93z1e07UlPvdngocW3zZ1g0o0waa9dOE47rFkF864Q550ymXsRZ6uwGORXG/FBPhgdjRJphaKJk/eP7jGRi6uHXqb7aZFGiyHE/d+ACctmuctkdw86utAW2/xpK6hD1m9aPWXu+Oam8HF428v8gsMJfp+kbuRoJciwUY1/Nv6vGQrg== X-MS-Office365-Filtering-Correlation-Id: 5b5b5dc8-3a0f-488b-cdb2-08d3c5657cc3 X-Microsoft-Exchange-Diagnostics: 1;BN1PR0201MB0802;2:n/j0+4qt5/N3aCmkxUnv9p+z65le/YfQmvmupOKjdEZkDn8UPMY/MiTw29VbykqayHTu5tYzG4uQU/Tx/yqkJHSOgNRfbcQPizi5+W+y2JzCZ1dWAELinNyFB1gHlPmGxTqbpLjJF5wxmnHr9QMwNEnB6cLSqX0XEVhERYQu0Wp6vXv9o6ZAT4XjOQQAWGa5;3:nYxlT+Ic7fI3cqtJ+sujBHfJ2RrJLQr5ZBRuzsQ5nE5qYVz83/IAyux2YzS2bPdzKSikW8dScdMHsQ1Xs+q4ItDOzUHtOxSVP2WRpd3QPY4t6mB2yu0qX7MwPSsiPs6tACLLcAp+lcfpOz4fFRwRNgDiY+zraZO/vAJ+L8aFfPUMyibKQf795AG+kIN+iRnt1aLGiuFKlCy4BYRXlf18GjNbXNRXHvt7SZCi9kXwewUwb1FONmBeRAxNHtT2FN/JT6YW6MV4ZnkWc/Rg83gCOw==;25:QCdLhz6Aq6a/tNvd11xf4x0m/Q4/svBX2Okq+Vt6YH8lGB+70bOb1WHmrDgjUTSnhVyhBcTNascXvzAvBZll3U6UYJHPleyAUivLA7OJFIsiK/Sv+6yifcD66QMq7rGsKVtjqmOsDfPs+rZTy0bOuPmEhqhV7dDXsHT5Oix7lS5tnHA+nAXAO1bEZcgolA36PvEVowPSWL+7k/ThK83G/+9pz62XzbUKv4TOdkYxSrKaZ/Raj+RXoyI/A+JLJRhg1AzMJE+Xpp+XWgg1HJPF+9TR2JEitmQKO1QYhRHd5vkAdFB/AN4yDF6j/VaYVw5tpvgwitliO8QpcHDfH/Y9LYbZprrYRZoVq9Msd8LelC9Y/BJ7S987Kcg4NtiBV3Muvxg1XNYsjtektixuEYSjnk5Y0glSEEPDZU3UXSZ1JEc= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(8251501002);SRVR:BN1PR0201MB0802; X-Microsoft-Exchange-Diagnostics: 1;BN1PR0201MB0802;31:71irz0Xu8CBKbPwUMlEKxMybCTlMMvVYtPxmJ/s14QtdRv4Z0zPlv/P840TARhzzXHQuHI7gESCFlGG3sBzO+bbdpTvE+jBox3Cfc6DcqDf5VecaHSZH6360gI7w1OiWber+q2uD7Sk6uzz0xHlNmstOuaFdChdxLCHNgA1WVDKE0IjUDZj/4+P2dT+h/T+NXZcuvI72PAGuHIeOfBJp9BEf9zDbHTveININmw506Eg=;20:hoZZ7hQ+Pdq00IzJ48fSLoQszrZ5TeDstn9VdKA4hbuwAV+BAScEIitvWswCV5ZMkeZeuEgIG+NMsvb3PErvge+88OcJc6lXUpNvBFsjaHbHSCzc13KRre/7olkat9t5HjHSl1bU7pzmIzVZs1l8JhBZOr5ExqOMH7hbfrjRaX1IZydV6IQgxUiG4Q87sousBCj9I4liSmmnYkAi2xVo0omfi2d0Ctvc7ao1LEGFOKQJuPoPdXQ+YvlWE/dcBiMmk1qVgie93wVbN+HKyR/g7BtQwOKaaygOn4nzqx41khePuGLlaDtjuHtOqJaCES0E1MneQOkHQWtiPFaLv2ipHZXpWMfFntFB/UBVlH/fv6crRouvqaOStzRSQ14vUMM/fvV7bf6Fn5MERIS4OVB/h5L68mqy15ycgUHTYVCet3i0slkMnPtNOpqHogkcNrPUGiC55FJ/w802vG4jKZSP9OTc/FJ0H2RLXCYEVf8ErTsKPtrM4xPG7SCs2ugVtost X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(5005006)(13016025)(13018025)(8121501046)(10201501046)(3002001)(6055026);SRVR:BN1PR0201MB0802;BCL:0;PCL:0;RULEID:;SRVR:BN1PR0201MB0802; X-Microsoft-Exchange-Diagnostics: 1;BN1PR0201MB0802;4:5rax3r7ED9cqYSpgP8/iV5Qu3UDx5GozE6YAHSFTOM5JMEM276h+nGJmwTKVnTx7h/XHe4RpuZoFKKxapaVR7krt7ptpdQolKKw1/aPxKvG+Kfbi4L4AHvprC0GXR2mvzWT5gfPeafcxLPdGcsBn7jmGvc8lF1+ZsAcNX9fgMVKmhxZReambhgBtuP3EIVmkv7acWW/zA57G1CkjyhcblZauo1dq6zo9UMNMEvL1MFmMdJal8br/pby0z2HrZHhTMMKZ52YjjVdw4XjXij+d2d1biFqBoMJUdUTBPgdMCqoYdj2mIl2Zj2EaRGmmuNo1bPwmZ26JC9BTYlPoil7MDnMcmmInb4U8oz5shub574Gaagd0W5dzO0fOhD2HkQCfVjEm+5g4o5nTIxPtqXYOSIKT0cSNHbfeMl0dT/6EjMCxD9zpELQTXyrDO1/Nix7U X-Forefront-PRVS: 0035B15214 X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjFQUjAyMDFNQjA4MDI7MjM6WUdjRmVGOWQrVE1MVU1GVXpvYjltZ1Zw?= =?utf-8?B?clRaOHFsS2FVWVhXRTF2azZRWUczUDlvR21YMzBxWmpkVmo4STIveXhhdzRU?= =?utf-8?B?S0FteG5zRUx0TVJFSHdTMHZWZXVsdmhyeVpxYkdhMTl1SmgxVmhPcGVLVDE4?= =?utf-8?B?R2h5Y3Z6LzQ0L1N4ZGpuMUNXR1o4Z0tvcTZxL1c0NzZibUljUzdUQXBOSFVX?= =?utf-8?B?NzNuWEx5Vk5zRWNwdHE4c2hTdDdhVVdncGRNUWNFdHhKNzN0a0xINmc1d2lJ?= =?utf-8?B?SE5lOXVWYlQ0L0dYV1d0a0lrajRpTWczN0gyd1JYYlJFU3BhOHJTSjFDc0xh?= =?utf-8?B?Vzd1Z2VkL1VnK0NGajNhVG9VWW83VVNGQThkL241ZXhEUitqNkVzdnVrTHZN?= =?utf-8?B?Unhac0kvY0RhdzBhMDlIK1B5dHVVMStJMTJGQ3ZqNzUxZ0VVWUl3NnhLdGk4?= =?utf-8?B?MDNMWU05Q1JFZFlCQW94SVpnaWprREZmRXMxV0NBK2dwREhTSWNFdG5oeHo5?= =?utf-8?B?VTlnbWI4UTh5WnVNLytHYkI4akhEL2dYM0ViNjZqWU91WE1YdVRuM0RTOXpH?= =?utf-8?B?alZYM3IyWDJibTNiTkYzUytVclVCUTl3YTVGTGF4bU8vZG1hWVV0MTNNUnNE?= =?utf-8?B?MG5pemxPWThzRDZUVVc0b0MrMzhkVTUyeGU0bDVZcmZ1WmFGc2oyOTVFRk5S?= =?utf-8?B?QTZJUUlNNEo1YWlIMmNkUG5QMEZVa0ltMHhvT0MvNU5HY2JqUDkxSE5RSjVL?= =?utf-8?B?SFl2bmxKeExlM0k5a2FvdnhVSHhOQURmdzE5RTU3VHFWZVdQMExyWExRekNh?= =?utf-8?B?Rmo3YkRTbjcybExrTHQycVRjanMrUmM4ZHZUZi9vejFkaVhVblhvVWphVENj?= =?utf-8?B?VjZYUlIxKzZRVWlLVVZxb3pIQjZnbmlvTDMydTN3aVh2RkRIRmtjbVkvT0c1?= =?utf-8?B?WTlpMm1RSiswWWlUMndLSmJUZlZiVDJOdkZ0L2xJYXQzN1NlNFg0MWV4bDJz?= =?utf-8?B?NmxJQ3lMNGFQNUdlbGwwa2N4WkJOTVpTSUF3S01lbW9VZkdlMG94NU5xMFFr?= =?utf-8?B?T0cybGUyNUQ2SVFKcDNacTlRUXF2ektIc3NCaWlSeHUzUHMvcHZSbUFQN3RZ?= =?utf-8?B?YS9KQVpvVDR4NjZUelFnRUNpSEpidDV0TVdOTEpzb3c3TnFBVlRvRFBPYW9W?= =?utf-8?B?QyttcHRpWmVSdXlpTU1RelkrLzA0VnRpWVFiVURKMmtqaGozWVpUMXhpS0tL?= =?utf-8?B?WW5JTzZtRUNtOHpXenZCd05mVk5URmhRMUI3d1pVUHdvMldpZExuSjJCc2dl?= =?utf-8?B?YjFmcnZqMXhvd1hMblVVSysvQ050ZTRPZy9VTHFTeXVUVUd6QnlLZzQybFRD?= =?utf-8?B?N3lGZnhNemF3bG9nNU5pc0VMNVE1ZjVFeUZmUXNSQzFUTndNWks4ck1zK2ZO?= =?utf-8?B?TFlxZ0JjZS8zT25WdHNUcnFrdDIvN0ozSXVHMll2ZUhBNDQ4ditRUnpBZzJK?= =?utf-8?B?NGhIa0NCMFIvZFUrNFowMkZGcnhhT29aK2wvUi9zQmhtc1oydm45Q2plTkJw?= =?utf-8?B?Q1pkRUtQcTJVZGpabHhKbUFRdEl3RXk3RW5uOHBsRFhmV2NsRGdadmw4QXJB?= =?utf-8?B?U2RnNFhablRLeHNrc0d5aVR1K082V2lDVFBxcndoekRFOHZHbUJEZ1BCZGc9?= =?utf-8?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1;BN1PR0201MB0802;6:zg3+PK9gubvyjuO+bkTaN1BcJq4WfBycTp/po1PoteL6LPFxXmtqC/Xj4jpamUbiWD6LxEmmjGXKgU0a+ZYMUzb9P8+ygAFrAEDyS5jWffMo6c2iOf1rbm8zOOXo6IZ59tRX4mc5dCIWdD2xHSPLUUjGLA34dRe0as4LIm1uLtog8AP/CoYBQjjeSToV7nM3DWxNdOze6Y14LQT9zv8QV9NhqqU/6kglIpo99PIq+2rE4BN7Xi22h1k/eFl7xwPp9Q1McaJ9lTje5koCQlIYK3H/OM5pjJ9ZQJ7xGjGPthVs14Fy5GzYEQrfpBX7GwomK/iGknGv72EeKmHhxflwWw==;5:lEPE7+pWR8EyDNBqkSmG7ZXeh9OfNWZFyQg3JdZ51zW4qDxjm5cn/vfskckfyeeC7eJIKMwnQcwCymC9eNSNx21HvyCJ0GFjnI907cgdXRzoNimiYARJ0zKIqpRQWOmHZ2rJHpWYyB8ymrGwM8MA1Q==;24:rWxwjzdveu/bDmsLh7MKAcRjzIVCMUnBKTfAjUSk3S0/bzEq73WN4AmSwa0TibeTVabxkaI3RMmhTKO2Cwrf/tx/bktFt6Gwib4XG52BKXM=;7:dao+FfnrrMnMh8czUti7UtloH9wASDdMsDl2TI6FVI363A+ox0k0oKl81fyNer01SBu86D9O7DGU1RSvaKGNemTelGNdpvhxAJuDjmBkGP6eREvkU9w8++THZbISM8mP3nIMWA/AkJGRlFIuisdsTRCx/QU8/PFFzw+X9xgAuEV9ZUQ5aSn+I5AuGPLQ1gvkJ4chA3Pnqx0BCrJw0OfJVxqAxJJlZBUK8MChQdGzcMjVnxJkbzkRWHhUJpw4JUll SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BN1PR0201MB0802;20:fXwHOfWEMjRC4nDhe8Hv24oxuIiKlcGuqujkWPoLZrsuRzPxhSMamFcEutH8j1vmk3vJ5BJej4tqrtxPYonfN1b09wlotapmcDIiKp5xoabBDlnkLAY6YzYQ9zr4Bxw5NODEuWuzilDZm5V0m2evOh/uEogrT8hjuiqp6/TXFAmU6o5/jLTbUUi7nVVkJFK925TWMdrtp9ZZKXCOVglK6hDvAYLGryLDjaaQuvBjQsNB+0Z0a0CsACjG4h0i3ES2 X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2016 23:40:05.7615 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d;Ip=[63.163.107.21];Helo=[milsmgep15.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR0201MB0802 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/13/2016 09:32 AM, Oleg Nesterov wrote: > On 08/12, Bart Van Assche wrote: >> before I started testing. It took some time >> before I could reproduce the hang in truncate_inode_pages_range(). > > all I can say this contradicts with the previous testing results with > my previous patch or with your change in abort_exclusive_wait(). Hello Oleg, My opinion is that all this means is that we do not yet have a full understanding of what is going on. BTW, I have improved my page lock owner instrumentation patch such that it prints a call stack of the lock owner if lock_page() takes too long. The following call stack was reported: __lock_page / pid 8549 / m 0x2: timeout - continuing to wait for 8549 [] save_stack_trace+0x26/0x50 [] add_to_page_cache_lru+0x7e/0x170 [] mpage_readpages+0xc5/0x170 [] blkdev_readpages+0x18/0x20 [] __do_page_cache_readahead+0x268/0x310 [] force_page_cache_readahead+0xa8/0x100 [] page_cache_sync_readahead+0x39/0x40 [] generic_file_read_iter+0x707/0x920 [] blkdev_read_iter+0x30/0x40 [] __vfs_read+0xbb/0x130 [] vfs_read+0x91/0x130 [] SyS_read+0x44/0xa0 [] entry_SYSCALL_64_fastpath+0x18/0xa8 My understanding of mpage_readpages() is that the page unlock happens after readahead I/O completed (see also page_endio()). So this probably means that an I/O request submitted because of readahead code did not get completed. I will see whether I can find anything that's wrong in the block layer. Bart.