From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1765587AbdDSPKV (ORCPT ); Wed, 19 Apr 2017 11:10:21 -0400 Received: from mail-eopbgr30091.outbound.protection.outlook.com ([40.107.3.91]:45025 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1765569AbdDSPKN (ORCPT ); Wed, 19 Apr 2017 11:10:13 -0400 Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=virtuozzo.com; Subject: Re: [PATCH 1/4] fs: fix data invalidation in the cleancache during direct IO To: Ross Zwisler , Alexander Viro , , Konrad Rzeszutek Wilk , Eric Van Hensbergen , Ron Minnich , Latchesar Ionkov , Steve French , Matthew Wilcox , Trond Myklebust , Anna Schumaker , Andrew Morton , Jan Kara , Jens Axboe , Johannes Weiner , Alexey Kuznetsov , Christoph Hellwig , , , , , , References: <20170414140753.16108-1-aryabinin@virtuozzo.com> <20170414140753.16108-2-aryabinin@virtuozzo.com> <20170418193808.GA16667@linux.intel.com> From: Andrey Ryabinin Message-ID: Date: Wed, 19 Apr 2017 18:11:31 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170418193808.GA16667@linux.intel.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: HE1PR0402CA0017.eurprd04.prod.outlook.com (10.175.27.27) To DB5PR0801MB2726.eurprd08.prod.outlook.com (10.166.176.22) X-MS-Office365-Filtering-Correlation-Id: a1181c3b-ca3c-4b44-4acb-08d487362772 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(201703131423075)(201703031133081);SRVR:DB5PR0801MB2726; X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;3:UpeP2PfCG+EiEFooihGN2oMl70kQmis7hA7//6LAOu58M5ZUNTy9e7KdZ3wbp6ygH26J7AmyHELIr1RyTdqI06VJw4PV3hk50QxmoayeBPmQHu2ITU+wHOHsalAuVy1Ccs8FFBJSC/i6szCRs/tPDbk8J9L4E7p2L1lnnPAa000AP5fJ7M8unauVkMXi8sO88zuVULFqYgW/dKSxqgPNTz8PRpyruncnB/yQcz1a8uHGH3e4fHoKPx9Pqz6qd+tWPc2GyxVac4sYQJ5LI4mVSQzJ3Xz8kI1e7bPO9XD7zWiCJXD9Y97Ep65QQ5FFEClCfK+EsmK7MWiOhWN9dBXx2g==;25:NzPVvyXJ9xBmJf12qCQE9LX3VhpAnBYUymnJ5zylBiEFSABWj6+muooaNXQO2Orv6fi3m4Y8NQzPP+p928Bq3IIZnbNVWGM6Eni4KwQ6fvFHZXcZYFJ24wTvYCXDR9DNjgRZoDGC+aaaPkn7xfKLIfffRvavlZTz7GqpBvQdl4ZHQlgrakkT9HIcfWCwX2wosmtCsV3D/EZqJQbkImJgIIxDr0zkhnkB/FDtlCrXHpF10+szkZ1/VrWhUrCInnD0URxQxRMlTfo4Jr3qm6pmb/dYtDz8Z7XmoFOFkZVi+FH7HJ53NDdvSPPkq4So0AyK3hWoshWvYbKA9nSPB73Ys0RCNaaPvcbd2RqNNECtVvfuU1RB/BDhXOKjhUL86076e1A4FB0nXuYJ78ZKw6YsLTPzPUxyZkMzZVzzjDztddtvBPsz+RdFYbG/iBCznFLtwytqB7jj1rUuZv+2N0n22Q== X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;31:eQJq47EhXMDWiT3XoDuOSzJLg9E9+JiP+vzex/ajTDFi5F72uxOhmzG7uole2M0J7jJdxDnPduDIJlBL9EXVQHTex9g8NqgFaDJIUIt4itqLGSKZAq2s9ivIr97JWyjHVYEt0wqoH/bMLt/7Be1D0fUhsp22bk3LaLUGDCr6opuX33Y4GV/ke3T5oApXoJ6vhgDuaP6R45ummPU2xj+jnPyolE5yV8UpnrvnU4/E2rbyzVQSF9tZhhjlKYy6q/3c;20:lDf3NHNgCRmLk6Fp/Hgav2K6t7qL7c31uz9/gK7nRDpF7O7o81Jbq1KneeqbkWjWy3gnxJ6lDVzE8xyKBxyzbMPoRXK1Yk6SrFxXEkHdhK20TGxyXpxJqveCD6Y3evoeZ6kFggMbq5RcXqmbDBUapNRQRgd49g1l9Cl9uQoCPc4jEDHohIXbCQ17H7m5aGnKc9GWlbHiJhyESs2IgpY4ME7t/lWvKVSrEoFvXnnrCLE/EffVADdNucyWSTPZcdx0v3/Fq6e0E6qbhEoeIgN4yR0gkB/i4HA8sSeGpmeqBnhRfoiZ+JBbh3mmjnyz7TNkO22nJJ4zxUGD5BFABQH0/UL5Ih/gPjD9eADAiChFoez0vn6DpexF27gdeYX8YszWy1mzaC7cS84bMpA8rd6ZWpTvsf6rC3GH0ZjoIQYS5Uk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(93001095)(6041248)(201703131423075)(201702281528075)(201703061421075)(20161123562025)(20161123560025)(20161123555025)(20161123564025)(6072148);SRVR:DB5PR0801MB2726;BCL:0;PCL:0;RULEID:;SRVR:DB5PR0801MB2726; X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;4:x+hLmdEnnD9crWJiFClZb80V8bLY9ePd1EmqDT9oKPjm4GPx9jv5OcxAYMjeaSiBrNCif8Yt9ZdAMMWJ+sYCbrd5t8r0vWAKtPXHYTHj9i4jRTPauFjKW+qnRj6yctn6yOwvccQO6gGlN5gT0OP+75I+uG9CkCPnrBCgwhSMSkBbGtxRrvbMmJ2/aC+3Lh3SXTEWVk8z4RVwVcAg1WIyEeYVSK2mJv6Ly6d2wfd5OKX1wBHFQMPprL7inJQWj6aI7zBBr20eTC8TCYbQxfz2Fc+fWpDP58N8E1knez5bqSzzXUJRmvHdjk0U+0lpbtkjvKC/zXgYJxJQBUfoicdUjnN2/L+gSIB92NAyID/b2b5o2fPaAPqkTCNAs2yp3uCmg3CMyRh9o/HBhB80wM0mNTn4mUoDZ6S7QGSjWeO6TmDVkpAFlJmikHZnzHGBNMoK308BRqAFz/BhQaOvazRtGkRR1vJGAl/mtfF2FcerzeUoOaTsTKbR6qKhK0VVl9eVPhP9Cjf+IwgTx/e/49eCbXhI4IFgZAiqMHqQztbzYw7ZixF54URgkWzXn5jI0raN1nVLH+SI30TVXpV6GIKU5mNz4vuF2iupuPkXC/lNjlwP/Yin4cwrnfzVFbNidaXXPyTVzCgOGiilokXngvTalBRA5zKRua5emD45QwX25Jj2SijQ0W3hjw7YtO07K8F/96tN4fGAY9UCf0HKpf+IIg== X-Forefront-PRVS: 028256169F X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(979002)(6009001)(6049001)(39830400002)(39410400002)(39400400002)(39450400003)(24454002)(52314003)(377454003)(38730400002)(86362001)(189998001)(50466002)(2201001)(2950100002)(65826007)(6246003)(229853002)(6666003)(31686004)(42186005)(83506001)(64126003)(230700001)(3846002)(31696002)(8676002)(81166006)(6116002)(4001350100001)(2906002)(53546009)(25786009)(65806001)(305945005)(47776003)(65956001)(2421001)(5660300001)(7736002)(36756003)(50986999)(76176999)(54356999)(66066001)(1511001)(33646002)(6486002)(8666007)(90366009)(53936002)(77096006)(23746002)(7416002)(2561002)(921003)(1121003)(2101003)(83996005)(969003)(989001)(999001)(1009001)(1019001);DIR:OUT;SFP:1102;SCL:1;SRVR:DB5PR0801MB2726;H:[172.16.25.12];FPR:;SPF:None;MLV:ovrnspm;PTR:InfoNoRecords;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DB5PR0801MB2726;23:0TCkeF60/kRG8opRwK9zK/R505IGsgmdcew?= =?Windows-1252?Q?UeGqENNOCFF9Hyqpaqcu58Fhjd54Jre3IFnRrOuQ8RYWJY658qgdTZHc?= =?Windows-1252?Q?MqfTKDd6G0wxxZBvG0MnA/C4XfAsiuvqFpjzSz+J+ssHufxTz1PY3FY+?= =?Windows-1252?Q?f4HOGVxqG261usdVd7WgwSy6jNueLuvV/GVEVMfCotI1AUpzFbwsQVKf?= =?Windows-1252?Q?o+//XV8vT93+DT5gvfg5xYBOmKJe6iv8/iDSqaxg2iWeYpqSPg+jWFET?= =?Windows-1252?Q?oH8BKnqG1GW83ZceU3cDUmcbFB1Prrd74qrHFz0cAU9Y2+7hhOtaw0zY?= =?Windows-1252?Q?VEUAirvPe8TthhBBD2uLhyiAyfBN0aKAPMQfTUm3NHKdRicg5u8Xu04r?= =?Windows-1252?Q?vkuSh1vsX1lVGck2I62sG3FecXmL/+uU6kTYio7yvt2xjMKzad5KB3yc?= =?Windows-1252?Q?Iix9vDIYatu18KcEh6cowdQa5Ip7W1y+RAH7LGxuG9MiodWe3lMTbk/3?= =?Windows-1252?Q?3Llz1n+rEwZshm3lf5TwZ+uyWpV7aP0ovmE6OsqIHOe5bN3xIfV4AT9d?= =?Windows-1252?Q?4vinWOqBBRHMx/ZSu92EpmO/KWLgVn9yxNH7G1rFeIu5zYjJkpSbJaIW?= =?Windows-1252?Q?UIdESZh1HnfNGv6X9Rhejfo3+YsBcdiIvMPqIYyh+JZlrmYPud+Am7P8?= =?Windows-1252?Q?pI9M8biqkGcL4Qki857WYsM8BbAzLQaq/lDt0tfmFLx65RXI1noT6L4u?= =?Windows-1252?Q?HL2m4jh8uzY1MsSwHgPI6R+GMspe0I+w+p176CGdcWusnjBLKciv3JWz?= =?Windows-1252?Q?y29hDy41ffU2n014Mqk7Esh9alzNIfIXQtK89/uGYAGVZmR8qv8sbTlx?= =?Windows-1252?Q?S/kF4t/oAg4gt382xGvDNrt0st275O4w1euqYOifhH2IC4iLkp+nbrsJ?= =?Windows-1252?Q?eBUfHGiG6l37uBE/gT0qFpM454HoR6qp22NX84RJznWM90AWpTCXul2t?= =?Windows-1252?Q?bjFVM0klz4XXS87onJEbgDwMIGyb/hfEvaE8JLd9rn6IrRSBDWKdfyhm?= =?Windows-1252?Q?67RxipjQbVkePQ6HM/OGkU1ewm71foGNCMx2mL+QyXRTNKzIus8ccnSj?= =?Windows-1252?Q?xB1XNuDwYKNJ8M6Q2rbDcNRgYsDrM+GUMbk205Lb7Ij/pV/PJkeJG8Pv?= =?Windows-1252?Q?03ZnBDfGmEDdGUia0qmqhDjNA6jnm3yscYWn+es6C2TUXY8Nvc+x9VF9?= =?Windows-1252?Q?6R5rTCDWvg3OOD+lHby9ONZ/SBZo2VHjb+HGOTMrfFV5WZvOXjcbDahj?= =?Windows-1252?Q?oVmUP1RUTHhMRzsHjWzIotPLsCZVv+1gPZbppSsUox2BGWyH+niFFGy+?= =?Windows-1252?Q?vtsZuFm+SfhHZlgA6sdY/3i2Cs3SOpflqmihesi3qbh+SplGzDYzmjH8?= =?Windows-1252?Q?PaORcNfLDoiVEQhyelrr4npiNYmhslCZxlaN8ZTHSL6XJIMJjwIdi8tg?= =?Windows-1252?Q?ndZGVjXPgjKFhZdogaQZ+ntAW3nfbWVhVzNB6UK8ipYAlLiqO7B+wF6e?= =?Windows-1252?Q?K6zqyqHzbc3Qf0rJbTAzNnq6DBWnVmBdGo8lf++dHiHGTGsfqVBXyqLL?= =?Windows-1252?Q?aaUbdw+JlI/Kc8dLL43ngLO7JopiXP71GfRx8YLt7Vfqp?= X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;6:lFyI5ignFIiqLdM1WoDDLiG4a4kfFJfSSHbZeiu0o2eEXjUabhaDCd8rceDyD6vvsS6B09lo3h0jEgf6WfjnsS8I1bV1eB0XRAgZaI4X923Hg7/WIzti7Sp74Xm2xCco/Q4USO8OtZECW2RgbeGHY0671OlYdmoXkly+CEYHs3j2tyLv5zoVaNytN59QLzA8aVJwTlj+ebShjUU99c1JUtX+t56q2DzK9lm2izzN0eq/TxaV+1ykdtEMTIIe4le2Zw7lZu1NmfRAGA8s9/TYYcTjKCMehhKzxvIyuiUfDkQ9DwY3a/l+X60KR2JTGhRhB+HVvZ3ApcXuR2S5rD0VsQ1RiB/mueNzTvTOx3V9z5r7sPnP99jOpqjCwOuyjX7c6kZs70Z0BgHoTVMY6tVHezzNtEXMYPIaBZJ+BoIXpI+hatRtGu6DJYN3DJ7BaDUPb1k8ABFaGQ8hFW/a49Ox+3G26+AnJYRnjARC71lP12JYOJ+a8rtnh8GRrYE6jyY1iaAXu9KXxpZh+Q+u6q+C6w==;5:BAv8Yyuv4m0bHk0lOUypvHdCp63A/D2ofTD3JLv2CeYnc6ffOkDA5tmP2p5TRpGU/+tqFXGqCL/Q0FihFwUzzi+41dI/fmRxESu/GAWGCNT2sBKUD04HsahIPQHO9ZZTPtFUH9Q9X+SUk77VZBiESQ==;24:rjWnvQUqItmnZ2bUTwf/1Ow+V5tuBuOkHIbp6ZhBpZDxJRX15ZpHsTOp1ugvV9apI1djt5dtjxShzzfJuyHHnnfjFXxQw5iEnIfehc7qCAE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DB5PR0801MB2726;7:ze4QGo4qTnAFdATUfcRQW4fvRJCoN5k4Ic3ZEMJ9P2+rrnsFmNoLrY7qeO8Glg29CdSazFqb6IwDsJoPelXtdm5GpA6T5AH2G13830SfZc5DMRJdFUAqKwkuCmcKSII0HmWXdb1KD42Zcdd6LexHGgApH+Ah+DMFrnEGDnhDeCzyieJZecU31up+IcSquRpfjNBQzLhV9vsb/A3jFL/CDyq1x7epvZTHesI6ccbTf1HXJXJ60VYRnoVsrBCWcfDo0pMVDd3LsGm0VMHztwi6ufgSSyeek0CXwMnX+QPCSuTghp1pa1mOuIhNnARTDlfW/NaNQiH8KuJ/xRfHP9FBAw==;20:AMV4mWS6cUNHHneVZGvI/dZN5ftcc7vY6MeqKtY9Tys+in/fGbtgnrU6sdhRKc5NW8nRDbJX0/Bpxun/ShB6yM9xURUemm8nDW+jjSKsNbJMpPWOLSwIvKPFwWiHNRbv+jSGjHka4dnSiA5GtKGR6i6O3g7qaxPjuYTMOi/fhsk= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Apr 2017 15:10:00.1468 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR0801MB2726 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/18/2017 10:38 PM, Ross Zwisler wrote: > On Fri, Apr 14, 2017 at 05:07:50PM +0300, Andrey Ryabinin wrote: >> Some direct write fs hooks call invalidate_inode_pages2[_range]() >> conditionally iff mapping->nrpages is not zero. If page cache is empty, >> buffered read following after direct IO write would get stale data from >> the cleancache. >> >> Also it doesn't feel right to check only for ->nrpages because >> invalidate_inode_pages2[_range] invalidates exceptional entries as well. >> >> Fix this by calling invalidate_inode_pages2[_range]() regardless of nrpages >> state. >> >> Fixes: c515e1fd361c ("mm/fs: add hooks to support cleancache") >> Signed-off-by: Andrey Ryabinin >> --- > <> >> diff --git a/fs/dax.c b/fs/dax.c >> index 2e382fe..1e8cca0 100644 >> --- a/fs/dax.c >> +++ b/fs/dax.c >> @@ -1047,7 +1047,7 @@ dax_iomap_actor(struct inode *inode, loff_t pos, loff_t length, void *data, >> * into page tables. We have to tear down these mappings so that data >> * written by write(2) is visible in mmap. >> */ >> - if ((iomap->flags & IOMAP_F_NEW) && inode->i_mapping->nrpages) { >> + if ((iomap->flags & IOMAP_F_NEW)) { >> invalidate_inode_pages2_range(inode->i_mapping, >> pos >> PAGE_SHIFT, >> (end - 1) >> PAGE_SHIFT); > > tl;dr: I think the old code is correct, and that you don't need this change. > > This should be harmless, but could slow us down a little if we keep > calling invalidate_inode_pages2_range() without really needing to. Really for > DAX I think we need to call invalidate_inode_page2_range() only if we have > zero pages mapped over the place where we are doing I/O, which is why we check > nrpages. > Check for ->nrpages only looks strange, because invalidate_inode_pages2_range() also invalidates exceptional radix tree entries. Is that correct that we invalidate exceptional entries only if ->nrpages > 0 and skip invalidation otherwise? > Is DAX even allowed to be used at the same time as cleancache? From a brief > look at Documentation/vm/cleancache.txt, it seems like these two features are > incompatible. With DAX we already are avoiding the page cache completely. tl;dr: I think you're right. cleancache may store any PageUptodate && PageMappedToDisk page evicted from page cache (see __delete_from_page_cache) DAX deletes hole page via __delete_from_page_cache(), but I don't see we mark such page as Uptodate or MappedToDisk so it will never go into the cleancache. Latter cleancache_get_page() (e.g. it's called from mpage_readpages() which is called from blkdev_read_pages()) I assume that DAX doesn't use a_ops->readpages() method so cleancache_get_page() is never called from DAX. > Anyway, I don't see how this change in DAX can save us from a data corruption > (which is what you're seeing, right?), and I think it could slow us down, so > I'd prefer to leave things as they are. > I'll remove this hunk from v2. Thanks.