From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756860AbbIXH0b (ORCPT ); Thu, 24 Sep 2015 03:26:31 -0400 Received: from mail-by2on0147.outbound.protection.outlook.com ([207.46.100.147]:13040 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754615AbbIXH02 convert rfc822-to-8bit (ORCPT ); Thu, 24 Sep 2015 03:26:28 -0400 X-Greylist: delayed 25843 seconds by postgrey-1.27 at vger.kernel.org; Thu, 24 Sep 2015 03:26:28 EDT Authentication-Results: spf=pass (sender IP is 206.191.229.116) smtp.mailfrom=microsoft.com; kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=pass action=none header.from=microsoft.com; From: Dexuan Cui To: Tejun Heo , Chris Mason , "Theodore Ts'o" , Andreas Dilger , "linux-ext4@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: RE: ext4: performance regression introduced by the cgroup writeback support Thread-Topic: ext4: performance regression introduced by the cgroup writeback support Thread-Index: AdD142fYZSxUpRKBTkye5CjsMXMB2AAN3K2AAAWTO4AACyRGwAAPChjQ Date: Thu, 24 Sep 2015 07:26:20 +0000 Message-ID: <5b1f9f0d50f94eea922dd0c7405428be@SIXPR30MB031.064d.mgd.msft.net> References: <20150923161359.GB25218@ret.masoncoding.com> <20150923185337.GK26647@mtj.duckdns.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [141.251.58.69] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;BL2FFO11FD033;1:NUs6RlB5LYH/iVQDkIdBdYi9kBEWP2GPKd67886m6q6ZDwleyG/QkIPulShBFQJxbPY1iBvUSh3/Ey9tBeqefaGc7H+CWCyhGSk9bzSsfwBRk6RrbeWuU8zVDqfgnzu5BQ2NqxT51TIQci9pozFUpBij8O91dYTFwsfILCORwkgTie5TutYwufpNsq5YGgTGH9BxcoNnHWHbEWaF5NKpyk6O2nMHI5HHFr/GXPhtPNz9tQmhHNb0DzGghkYGIu0aPRdBgSlYHKCq/bi0Iy47SxO74vSP/1mTcL5SKxiMH5IwwOW7LUGIrQD/zuw8J2O+aizSwyTvGtNohr7wH44t555jilFioVQouS6y2cuqW+I= X-Forefront-Antispam-Report: CIP:206.191.229.116;CTRY:US;IPV:CAL;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10019020)(6009001)(2980300002)(438002)(189002)(13464003)(199003)(24454002)(4001540100001)(24736003)(5001860100001)(19580395003)(92566002)(64706001)(50986999)(5001770100001)(2201001)(54356999)(76176999)(46406003)(19580405001)(2900100001)(77156002)(2501003)(69596002)(66066001)(5001830100001)(26826002)(97736004)(6806005)(5004730100002)(5007970100001)(81156007)(47776003)(62966003)(23726002)(106466001)(189998001)(97756001)(102836002)(46102003)(10090500001)(33646002)(68736005)(5005710100001)(11100500001)(86146001)(5001960100002)(5003600100002)(86362001)(107886002)(108616004)(10400500002)(86612001)(50466002)(2171001)(16796002)(87936001)(10290500002);DIR:OUT;SFP:1102;SCL:1;SRVR:BLUPR03MB359;H:064-smtp-out.microsoft.com;FPR:;SPF:Pass;PTR:ErrorRetry;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BLUPR03MB359;2:4ZA5YgiUHyydMkfa0aEpxJxbFrTYuLonP+tq9C5qdzS6aVTKrv4Mn90zVd09Enl+Dz9+OF+erWlACHOvNwVzI3a7dLIPz8Fn0W2/yai+GFCBnCAujafz9QXLww1PFzpepMbOTocHxAAsCZ+RGK755VBxt9PqLzuSFk5NZ23FhvM=;3:ibRu+dd6N4u+q0qbcdVhA9x8WtzPBJuJse/fnEr4ZhTaHZVuGo7zxuHO9yRhMzetAiG7aoDZh325JFHSNfGTEQlu/VdYAMo4F8zB4//HsglwH2qWsrisyEb0tfdzqUnLrPHxP1yCFl65mKFFnqO7gX/I+yOwXpf7GufoBBeyQkDAtEHTSatQul4a8Uf1rWeFDHTw33CQMIt2OP9WkIjgA2gvW90VG6FyyPIDP5q2vkQ8fP+Rz4FlpI3UzPjZ9H7l2UxXmbY9QQPKSbhm45XEjg==;25:lwP+iQ3+etmyxlNDFEw+vqMz3Eo16+yMtQ/OqgNnCRyuQz7LZzrzWA+FHBYpIibAi2aJzm2enbPvVkcSzficrkZKGA5pQXgV2PvmWJriE8IT2yfI/HIDWp9ERRBZRFiSK5oSG3X526MewonJZAJ0qNBKraupX/8h7imjIq7RLabEFeuwV+B1IiVNOkBQ0P9nDJcnPsUn9HYQ4Dj7mmFPEA65vLmSuI+IHQQwVLaMsIHX9KD5jBB1efILQz9EMpW8 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(8251501001);SRVR:BLUPR03MB359; X-O365EOP-Header: O365_EOP: AllowList from IP - set SCL to -1 X-Microsoft-Exchange-Diagnostics: 1;BLUPR03MB359;20:3nArts7MkjT8F4h6vQWM5TdT/qf3QQNzaVUPcew9gD/DpXFrxSmwpwk2G/d7/kkMRmy1bOCzXtXQDUm7JYQ4DnaIdbTOSzsimGgFZV2+EIXJtvsgQ53aW7kwsxaV2740CY4EW7zyKSQmqaViZ6mGr+uatQou30GDMhX8IdrFndcscVrHsWQ4R4wnST6rARhyuHAlq53iPZ6KFz/zURXF2GxeAvK8arKTevIU0ZtUU4PbeHScqFaATh4vFgy1fWn3NDqNfKptKy6Kt9fzxRc45I2mT+f3kIuUT+XgptFEpfYPBbvpWV7kioiBSLImGW/rdtZ90HXnIM4nLVjkHevpgzMC0auPva7VgLCjG3HZwjUumsnd/J5h6YvqHDzCgvtBWDUUhcrczRcjp8QlmC66HGLRQdAKZzAw6SLPUHUw/HfVZutW5DF4pP5pMAGDT1TyyLZk5IBvuthuNLDgOaCB5TXEnCEZC5xR6sBPR5DMdNpPVnBFerQSlQITQMoS8otk1LtbRyYPeDLZb0vRwxT7AZyae2nPI9s/eJBSRq8GwJNF90T7KjBgFt6Q6ofPAxWrskIsj0DRUjZ88fKMJMHX3LX+fifPHdollLX65F7VHvY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(108003899814671); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(61425024)(601004)(2401047)(8121501046)(520078)(5005006)(3002001)(61426024)(61427024);SRVR:BLUPR03MB359;BCL:0;PCL:0;RULEID:;SRVR:BLUPR03MB359; X-Microsoft-Exchange-Diagnostics: 1;BLUPR03MB359;4:fs8RkAXlrD5V0CMYQwBzO16AhjiwT18BGfKaOogCOuLWKbhLAr5WGPxGYU7lRmLw+rWx+gHbnuP4L+XYplVSMl4cgYhdM/iL53mjq20NpUZnfEZZGN2q2Nm9ILe0qbymmUHJQEN8wcDwaSOtkUcLEg8km1R0DoY/tlEmLzssVYDZk6bCXqdmL5nNhN76ZegB2YjpGYfBU4SOrTQ5QxPmxFO2fTT+iQ6FUKhkTffZK15MNV3jXlNvwnLfmXC8byjHaFv/50ndqwDxuZxgo8+JImXQ9vl752T1Ij3SEPyUFreQdrpPckx7Zk5eDdHVxBRfrkY2fq5Qu3UBVjnPSpaq5zf4WCalKb9xnx2TIlcbNF+q7enjxWs631wM/zRnD7nSMIIvIlLLI7GAKgqaMj42VEnIjCO7z6fZvaJ345IoiJ8= X-Forefront-PRVS: 070912876F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BLUPR03MB359;23:A70PIN3ITMMfcqVCHpMzkv5UPSy3qHxyhe1RikSkN3?= =?us-ascii?Q?9xiktyU4pNXmkL1kjWRroKOwmdiRNfMWdz8WMtraqS+2G65dMm+R0XWrUSuX?= =?us-ascii?Q?x3pzka1rzP4yPJlX0l3Ar4Tjb2AtmcD8F9u+KPgS3nsVFIm32qAqyB8MBCfs?= =?us-ascii?Q?mu0hXm+GSczRhn58jMtRingMrMk/hxKSgeMB00nxS4e2TIZTJ9YTEBa+bhyo?= =?us-ascii?Q?AUUjLFwEhhaiE0GMEBDEi1vdIzhJxJDJ2UAYKsd4zZr3xAkQuCCcYYDYxVss?= =?us-ascii?Q?8pLTbuAE6yj1iXSGM/OmWQvYMU1visxxHOx4I51omWL+ojDM5kpaK63jSvjk?= =?us-ascii?Q?FB59Md2GyUjyoKwik/Uq3OKDbYazOXQqL//d9h3z1nw7orRr5+xklLWWFDUW?= =?us-ascii?Q?6FyXT42s8FmvzSPEyY3qPODsv+IL5zmBk+Re1RMkcUCkOhAxdWdyc1zyu0i6?= =?us-ascii?Q?oThG+20PsRt/KXD5iy95lsyRII+SMCExNHNaZ6D+StePB7B5AYw5bvpeTD+q?= =?us-ascii?Q?yU26iAOkDfBvqzJsXyK5pJ7PzfT40/WrQSbE+zp/W931EMdw6DlSyrmh8zHK?= =?us-ascii?Q?MyRb0rAP3DxVH1nOANc2L2A9Rz0QYmaVq6wasq8TkMoKu0rMBcStxCGCQ8FE?= =?us-ascii?Q?ZZCM2o6nCpSQ4fukHOulS46GaHPUI+hf5aXtbhnEeSGIsSgWMsX+mrmpz6NN?= =?us-ascii?Q?89nQAjRIrMjSBSmHcFtMor5433nIO+kMYBcq0NWH5whgnNi2s1zXCLQphX1y?= =?us-ascii?Q?UVK8ePT5HcdQxWMenUB43KmVOEQPwHC9YlS2lwUqWOJy+/ag2X2ZFyXZvrJY?= =?us-ascii?Q?GMcNVpummM1yyYJTNzFxGULpfnERQiCPNcQQd6i6e4HXG5zGeIS/hcxTdFRG?= =?us-ascii?Q?9KCZ1eBmYFBiKeLFJhOKN4Jq3VAdGMTCxTzCpSfWdVycFShlpdWh+/cQb0Mo?= =?us-ascii?Q?q8IVfJi8EpsjVCv03dzMs6y3lgvIbAqNd8EesvIOr9u1hmElmlyUNBqRviSA?= =?us-ascii?Q?maOuJ5HdYyqOu8pN3HH76iZD5cMdYwe+po0VEw2r4JQKvFYaYc7e1JQg8GIZ?= =?us-ascii?Q?lVcuLsGr1xntAODxIG0uYvuDXmToz5dcf2YSEOrXL8wL4CWnBMEZgvl7JITT?= =?us-ascii?Q?7nvvxGlTfo3ZM1jo447iZEw87QVGilMeJRoaiaayWFBL7I2Zmg0ojqX8VkuP?= =?us-ascii?Q?vRPbN6TsuhhE2iHdsw9mpf4CyDVnEvP9l2/V2OSpl6AyFnFIP7keOknj/j8y?= =?us-ascii?Q?uwKa8/ornBRUzlxEnRvjJxWKBNELogGh8hZ7RzCrv1iieI7nN3hQcdqyew9u?= =?us-ascii?Q?wvOTqHVUD6jUZhkvYAGH6TcNqTEobLtFz213vWnXtrk9DqiYGDVy/VpPsSrr?= =?us-ascii?Q?yKOf0yIoOLuVMAY3Bd6zqQSJifXoACfRmkxg80UNfC9uZPp8adnSvZOBsCEa?= =?us-ascii?Q?XCVixUHnW0o2ycE+DiXGZTHyJeXO0uZJO2cddS6FnKpuWbZhLv9jDiB2oK4O?= =?us-ascii?Q?IKTshOX+AzQtyIykQ5fiDI76/0ADINwoOr7xOsQDL523UyukuqpL3W?= X-Microsoft-Exchange-Diagnostics: 1;BLUPR03MB359;5:1wJmeXv9m4O2zbwT391k5gwIypYSJa13gLdoDFbZ/rfQVzD2Oj08wxoenPBME8nFcsxbqXkeNQCM8q6zye63iO6gKo8Os8IAFpWEU7gm/kwkhgYQdNmCW0xcWbthWZgw6tdkSA6UmpjggjKF5UQ/QQ==;24:2XYiFovo2mNHqRqidpaYRrrxoCI/xTbURvx3EQgbxpdK3P/0JEc5fpd0kfc99ereUoiI6i0QKYNIi1EqXBNJr1A7LCjyog07RbV6+lchW14=;20:2iyqDzIgZqnVKrf3IpxUWV38CO1H1123OHmQSqLt0MOgA4lSVmWMlnUF2DYrj5zajKZkPM/7rIx5nscYZqL7/g== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Sep 2015 07:26:23.1153 (UTC) X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47;Ip=[206.191.229.116];Helo=[064-smtp-out.microsoft.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR03MB359 X-Microsoft-Exchange-Diagnostics: 1;BLUPR03MB1442;2:dNy2lkqU+evOtEpq353vMoAM4EVw+cksAm6HfFUBH4TFDJrWAXGFeBsHaF4uFdDlGW2adLT+gwuPxIfQEYSQDUzhw+caNffkWzvE0HrD6HekEbTzNMegso8Q4d6bAVnK4YMOCOXuvGR1/MoXN713DHwYaWtcRNfrmfpNLod8hUs=;23:/siLyeCrIEErASchEvLLSzudRRKRVLc147VhaF4Nl5UwBAq2kz22J52E0KOX3xWSjIi5q8egvvghH39CD2RQpnRwuxnjOLV6dbeB+AipAbxtp5ej7MjgVADsv264pYC6MPT0rtJIiNEWvfTFHn00qIAJk3J2tWH9LOpWqUxpXGdH5zjuWN8aNqbZovZ7zdg7 X-OriginatorOrg: microsoft.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > From: Dexuan Cui > Sent: Thursday, September 24, 2015 8:16 > To: 'Tejun Heo' ; Chris Mason ; Theodore Ts'o > ; Andreas Dilger ; linux- > ext4@vger.kernel.org; linux-fsdevel@vger.kernel.org; linux- > kernel@vger.kernel.org > Subject: RE: ext4: performance regression introduced by the cgroup writeback > support > > > -----Original Message----- > > From: Tejun Heo [mailto:htejun@gmail.com] On Behalf Of Tejun Heo > > Sent: Thursday, September 24, 2015 2:54 > > To: Chris Mason ; Dexuan Cui ; > > Theodore Ts'o ; Andreas Dilger ; > > linux-ext4@vger.kernel.org; linux-fsdevel@vger.kernel.org; linux- > > kernel@vger.kernel.org > > Subject: Re: ext4: performance regression introduced by the cgroup writeback > > support > > > > On Wed, Sep 23, 2015 at 12:13:59PM -0400, Chris Mason wrote: > > > > The issue is: under high CPU and disk I/O pressure, *some* processes can > > suffer from a very slow write speed (e.g., <1MB/s or even only 20KB/s), while > > the normal write speed should be at least dozens of MB/s. > > > > So, I think I know what caused this regression. Separate wb domains > > shouldn't have been enabled on traditional hierarchies. It doesn't > > work there and leads to multiple wb domains competing on the same > > blkcg and the bw estimation would go completely haywire. Will update > > soon. > > > > Thanks. > > > > -- > > tejun > > Thanks a lot for the quick fix, Tejun! > > I'll test the fix. > I'll report back in case it can't fix the issue --I think this is unlikely. :-) > > -- Dexuan Hi Tejun, Thank you! I believe your patch fixes my issue, according to my test. -- Dexuan