From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932182AbcFTWtE (ORCPT ); Mon, 20 Jun 2016 18:49:04 -0400 Received: from mail-bn1bon0137.outbound.protection.outlook.com ([157.56.111.137]:47857 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752927AbcFTWsp (ORCPT ); Mon, 20 Jun 2016 18:48:45 -0400 From: "Kani, Toshimitsu" To: "snitzer@redhat.com" CC: "linux-kernel@vger.kernel.org" , "sandeen@redhat.com" , "linux-nvdimm@ml01.01.org" , "agk@redhat.com" , "linux-raid@vger.kernel.org" , "viro@zeniv.linux.org.uk" , "axboe@kernel.dk" , "dan.j.williams@intel.com" , "ross.zwisler@linux.intel.com" , "dm-devel@redhat.com" Subject: Re: [PATCH 0/6] Support DAX for device-mapper dm-linear devices Thread-Topic: [PATCH 0/6] Support DAX for device-mapper dm-linear devices Thread-Index: AQHRxcNvFc0gDtWw/UKSuO/qIYbBN5/oAoaAgAqtRoCAAAWsgIAAFjAAgAADUICAAAKLgA== Date: Mon, 20 Jun 2016 20:11:35 +0000 Message-ID: <1466452883.3504.244.camel@hpe.com> References: <1465856497-19698-1-git-send-email-toshi.kani@hpe.com> <20160613225756.GA18417@redhat.com> <20160620180043.GA21261@redhat.com> <1466446861.3504.243.camel@hpe.com> <20160620194026.GA21657@redhat.com> <20160620195217.GB21657@redhat.com> In-Reply-To: <20160620195217.GB21657@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=toshi.kani@hpe.com; x-originating-ip: [15.219.163.9] x-ms-office365-filtering-correlation-id: b2823cde-9182-4aa4-2f89-08d3994714e0 x-microsoft-exchange-diagnostics: 1;CS1PR84MB0008;6:jqlpqWei3/mMitoNP8QjHzbiRQEn2lhN8ROEx42nxVcjZsNrhqeygIQ5FmZkS0X8j+mSMY0/FZiryKYliIsthHK5Pf1hpZ2neWu+j2hkcsriRRIPihDeP+l9RpfVUlsVVbB1pv2r4+bzx1aleTkaN0UiIdSW+3KaJEeREsN9wuDsWHSHRRn0Az1SlfjwyUrZSWi6Ao/281lMlI+gp4yyEVcvj8uOZCYv1uFRiwWHXb27RzDH2pkZKdZ3LO0VEcPp0PIFT2LhAHyKJVqeS8fztymQqm44C3QTgXssU2UyVzI=;5:19exWwsGYWi4gMLteWZM3ac1K7QV96mGnWjyWE97W0Dv0TkFSrOjwGc39RRw4IJIzkt2k45dXRohPtQS/OsyR3rcUyLDm/vrJOuzsB1VbnYmhZnNCVskO48sIs9GDFDXV6mS6VFvkKQ3uTGN62TjjQ==;24:StEpNiTDiFYEpW0ndySvEALNU8QDIxNBEOsceg0K/l/eHdIWo+yCTASNBA2UVPJeBa9tNatoJo/OoDNXHwTOh5rBPaGroS12jgUAXzJZ5AA=;7:s8JoZWlBIDieFcWld30tS326g/Ihdjcri4fReFE6sJ1IKZ4uuy1PcECcuX6DNb40+FuIjczYQnYLCaNX+IAZQGBC26+Ci53vWfPxqBWEJWngYQE+5aMlzG1s7OiV7IH+vWNJ8+p3tf7Cf9h6Z/MjbjbNl7xxHGOr9mQ+rfJ/IEwiZlUS5zyP+nSVN8PPCKXg/jrYyUdLYiTMXOwsotX2K93oVHu5OSvZ3nnHHZu/LzUAXCeOYjFuK69Uwnr6kYAD x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0008; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:CS1PR84MB0008;BCL:0;PCL:0;RULEID:;SRVR:CS1PR84MB0008; x-forefront-prvs: 09796A1B83 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(7916002)(189002)(199003)(377424004)(24454002)(51914003)(106356001)(10400500002)(101416001)(586003)(106116001)(11100500001)(110136002)(33646002)(122556002)(105586002)(3846002)(102836003)(99286002)(5640700001)(97736004)(7736002)(93886004)(36756003)(6116002)(8936002)(19580405001)(19580395003)(103116003)(7846002)(86362001)(189998001)(2501003)(575784001)(2900100001)(2950100001)(68736007)(92566002)(1730700003)(8676002)(81156014)(81166006)(2351001)(3280700002)(77096005)(76176999)(3660700001)(50986999)(66066001)(87936001)(2906002)(54356999)(4326007)(5002640100001)(505234006);DIR:OUT;SFP:1102;SCL:1;SRVR:CS1PR84MB0008;H:CS1PR84MB0005.NAMPRD84.PROD.OUTLOOK.COM;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <72F21725EFE6764B8B0BF9A47C01BB6F@NAMPRD84.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: hpe.com X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2016 20:11:36.0004 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR84MB0008 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u5KMnKa6028239 On Mon, 2016-06-20 at 15:52 -0400, Mike Snitzer wrote: > On Mon, Jun 20 2016 at  3:40pm -0400, > Mike Snitzer wrote: >   > > > > # dd if=/dev/zero of=/mnt/dax/meh bs=1024K oflag=direct > > [11729.754671] XFS (dm-4): Metadata corruption detected at > > xfs_agf_read_verify+0x70/0x120 [xfs], xfs_agf block 0x45a808 > > [11729.766423] XFS (dm-4): Unmount and run xfs_repair > > [11729.771774] XFS (dm-4): First 64 bytes of corrupted metadata buffer: > > [11729.778869] ffff8800b8038000: 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00  ................ > > [11729.788582] ffff8800b8038010: 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00  ................ > > [11729.798293] ffff8800b8038020: 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00  ................ > > [11729.808002] ffff8800b8038030: 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 00 00 00  ................ > > [11729.817715] XFS (dm-4): metadata I/O error: block 0x45a808 > > ("xfs_trans_read_buf_map") error 117 numblks 8 > > > > When this XFS corruption occurs corruption then also manifests in lvm2's > > metadata: > > > > # vgremove pmem > > Do you really want to remove volume group "pmem" containing 1 logical > > volumes? [y/n]: y > > Do you really want to remove active logical volume lv? [y/n]: y > >   Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 > >   WARNING: Failed to write an MDA of VG pmem. > >   Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 > >   WARNING: Failed to write an MDA of VG pmem. > >   Failed to write VG pmem. > >   Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 > >   Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 > > > > If I don't use XFS, and only issue IO directly to the /dev/pmem/lv, I > > don't see this corruption. > I did the same test with ext4 instead of xfs and it resulted in the same > type of systemic corruption (lvm2 metadata corrupted too): > > [12816.407147] EXT4-fs (dm-4): DAX enabled. Warning: EXPERIMENTAL, use at > your own risk > [12816.416123] EXT4-fs (dm-4): mounted filesystem with ordered data mode. > Opts: dax > [12816.766855] EXT4-fs error (device dm-4): ext4_mb_generate_buddy:758: > group 9, block bitmap and bg descriptor inconsistent: 32768 vs 32395 free > clusters > [12816.782016] EXT4-fs error (device dm-4): ext4_mb_generate_buddy:758: > group 10, block bitmap and bg descriptor inconsistent: 32768 vs 16384 free > clusters > [12816.797491] JBD2: Spotted dirty metadata buffer (dev = dm-4, blocknr = > 0). There's a risk of filesystem corruption in case of system crash. > > # vgremove pmem > Do you really want to remove volume group "pmem" containing 1 logical > volumes? [y/n]: y > Do you really want to remove active logical volume lv? [y/n]: y >   Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 >   WARNING: Failed to write an MDA of VG pmem. >   Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 >   WARNING: Failed to write an MDA of VG pmem. >   Failed to write VG pmem. >   Incorrect metadata area header checksum on /dev/pmem0p2 at offset 4096 >   Incorrect metadata area header checksum on /dev/pmem0p1 at offset 4096 I will look into the issue. Thanks for the testing! -Toshi