From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753269AbZBDGSG (ORCPT ); Wed, 4 Feb 2009 01:18:06 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751330AbZBDGRy (ORCPT ); Wed, 4 Feb 2009 01:17:54 -0500 Received: from smtp102.mail.mud.yahoo.com ([209.191.85.212]:29064 "HELO smtp102.mail.mud.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1750892AbZBDGRy (ORCPT ); Wed, 4 Feb 2009 01:17:54 -0500 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.au; h=Received:X-YMail-OSG:X-Yahoo-Newman-Property:From:To:Subject:Date:User-Agent:Cc:References:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-Disposition:Message-Id; b=e0wPgDeMGlCAicDciJzkO8f08XGPEgwEyEdnTaklHeZsqk4vRlCvUVS3UVCn10EuVSxwaq9YUsAFnjeWOxdTgmjPPXn0VmMF+oVUsccN2eA4Rfkf/nzUxvtFfoXc844Iokfrcg0TqXV14dBaxe3dN+19og98jsC0alqApn67gQI= ; X-YMail-OSG: 6S.DKFsVM1kKr.aDSQoAZWS2UMa50KoyeqH.5EWli.Om3Bh2.kh5fv1O.l8gSBD3DULF9HCwvWj52u5uTAgBBxsatlqkUtAYe71JkvCTK5tM6.ey3qrQyvMFSKCK_jt8EV6pL7O4zzROx3vy5S1CbY1_Ijrl4AK4_Rnwxn5qbnwokqbGLbaIbK5Gli2JXfTUKubEWibqw1iYQCeYngXoAjGwcAm9e6RORyyCrFKvASum X-Yahoo-Newman-Property: ymail-3 From: Nick Piggin To: Federico Cuello , Ralf Hildebrandt , Artem Bityutskiy Subject: Re: sync-Regression in 2.6.28.2? Date: Wed, 4 Feb 2009 17:17:26 +1100 User-Agent: KMail/1.9.51 (KDE/4.0.4; ; ) Cc: linux-kernel@vger.kernel.org References: <20090127093533.GB7037@charite.de> <200902031209.12615.nickpiggin@yahoo.com.au> <4988A034.20406@lugmen.org.ar> In-Reply-To: <4988A034.20406@lugmen.org.ar> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Message-Id: <200902041717.27320.nickpiggin@yahoo.com.au> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by alpha id n146IAwM020768 On Wednesday 04 February 2009 06:51:16 Federico Cuello wrote:> Nick Piggin wrote:> > On Wednesday 28 January 2009 08:09:10 Federico Cuello wrote:> >> Ralf Hildebrandt escribió:> >>> I recently installed 2.6.28.2 on our postfix/dovecot-based> >>> mailboxserver. Previously, 2.6.28 and 2.6.28.1 have been running there> >>> without a hitch.> >>>> >>> Now with 2.6.28.2S I had two major lockups: All writes to the users'> >>> Maildirs (on ext4) would stall, the load would rise, "sync" would never> >>> return.> >>>> >>> I had to "reboot -f -n" to get the machine back. All hanging processes> >>> were unkillable, even with kill -9.> >>> [...]> >>> >> The same is happening to me, but I have some logs taken with sysrq.[...]> >> 0 2 99028 46536 26368 1569400 0 0 4 0 1407 387 0> >> 0 0 100> >>> >> Notice the 100% iowait.> >>> >> I also managed to reproduce it doing a rsync from one partition to a USB> >> drive. After the lockup I can't read any file from the source partition,> >> but the other partitions can be accessed normally.> >> > Hm, thanks for reporting, can you guys get a sysrq+W trace when the> > system reaches this state?>> Here it is, if you need something else please ask: Thanks, could you reply-to-all when replying to retain ccs please? Common theme is ext4, which uses no_nrwrite_index_update, and I introduceda bug in there which could possibly cause ext4 to go into a loop... Would it be possible if you can test the following patch? Thanks,Nick --- Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 introduced some@wbc->nr_to_write breakage. Here is the change from the commit: >--- a/mm/page-writeback.c>+++ b/mm/page-writeback.c>@@ -963,8 +963,10 @@ retry:> }> }>> if (--nr_to_write <= 0)> done = 1;> if (wbc->sync_mode == WB_SYNC_NONE) {> if (--wbc->nr_to_write <= 0)> done = 1;> }> if (wbc->nonblocking && bdi_write_congested(bdi)) {> wbc->encountered_congestion = 1;> done = 1> } It makes the following changes:1. Decrement wbc->nr_to_write instead of nr_to_write2. Decrement wbc->nr_to_write _only_ if wbc->sync_mode == WB_SYNC_NONE3. If synced nr_to_write pages, stop only if if wbc->sync_mode ==WB_SYNC_NONE, otherwise keep going. However, according to the commit message, the intention was toonly make change 3. Change 1 is a bug. Change 2 does not seem to benecessary, and it breaks UBIFS expectations, so if needed, itshould be done separately later. And change 2 does not seem tobe documented in the commit message. This patch does the following:1. Undo changes 1 and 22. Add a comment explaining change 3 (it very useful to have comments in_code_, not only in the commit). Signed-off-by: Artem Bityutskiy Acked-by: Nick Piggin Cc: Andrew Morton --- mm/page-writeback.c | 21 +++++++++++++++------ 1 files changed, 15 insertions(+), 6 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.cindex b493db7..dc32dae 100644--- a/mm/page-writeback.c+++ b/mm/page-writeback.c@@ -1051,13 +1051,22 @@ continue_unlock: } } - if (wbc->sync_mode == WB_SYNC_NONE) {- wbc->nr_to_write--;- if (wbc->nr_to_write <= 0) {- done = 1;- break;- }+ if (nr_to_write > 0)+ nr_to_write--;+ else if (wbc->sync_mode == WB_SYNC_NONE) {+ /*+ * We stop writing back only if we are not+ * doing integrity sync. In case of integrity+ * sync we have to keep going because someone+ * may be concurrently dirtying pages, and we+ * might have synced a lot of newly appeared+ * dirty pages, but have not synced all of the+ * old dirty pages.+ */+ done = 1;+ break; }+ if (wbc->nonblocking && bdi_write_congested(bdi)) { wbc->encountered_congestion = 1; done = 1;{.n++%ݶw{.n+{G{ayʇڙ,jfhz_(階ݢj"mG?&~iOzv^m ?I