From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B7D5386425 for ; Mon, 27 Jul 2026 06:18:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785133106; cv=none; b=W5fgt+QdhyaqqaTzX89wpROXasav2I0mfI0cZbd1rp4KnbqC3c60eDFWOjeUPOXkjlmCqmrWp6eH+FN+Y9boifHknWBjPNrM9vfC8rbVgQlvZkr5pyzk8aVuCeqKcK+JYZyuCB3jycfA1rF2JZaLA48rwiWba8E0zR9VW5uPCWg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785133106; c=relaxed/simple; bh=ipIc5mIjuPdnhjxEQWxY3I4cWevJDCobazMTJt2bLrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lAEztkHURH3HNtEZ1GeCLpKoe7yVTHoSJDdCX66nVFFAqOdLlmBfP4a4+PAKt7VaBVp3JRv7ZW3AMNsp8MwHp+v8Tl/G4CxbcAiKlZTNm0XnsUh2612h6A/y2J8Fi8L24q2ehlpkh/Gos+epyr3YjEMB8Vb3ZE4kSQ1hPTBXlnA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b=LQGihkJT; arc=none smtp.client-ip=209.85.208.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=suse.com header.i=@suse.com header.b="LQGihkJT" Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-698a99b1c37so404682a12.1 for ; Sun, 26 Jul 2026 23:18:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1785133102; x=1785737902; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=w1YaimAULhRxxIPWQ8++mwbsLRf1Yy9B7C9zHgWk02I=; b=LQGihkJT3VcGdjvkVJdoUAJ4gm7voTuPyMrFEvLOx7a+fRzDb3iwb07jNFIJ+Tb5++ oEDZ5SgtkAk9eGbVzT/9u+3RaYS1HpX8ISTTjYdfvy6mbUDjFmslXVTmZKU4paW4esdi ubJseHs7KC48zyRP02S8Og3L2gNJxmUt5b1eYaPaT+1q/PjLYqAz2WLJAFWinQc7nDh6 eKrTWTfMW+u/PWHI1S2x6bTUgYyejwz5pj+o+HEQTJeLPx4+1AGZQko5wvwC3X138Hc2 2BnQ+Bz6cWH6RFqgv8nSVVpzYHmSDl1jLjvbCZuyFhCA0U7bvR677b6IQgNCJgnMzYqZ d2AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785133102; x=1785737902; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=w1YaimAULhRxxIPWQ8++mwbsLRf1Yy9B7C9zHgWk02I=; b=HSrpV4B4mCxddmNaH/eCSb6c9o2RiTALvOZDFnQs1VH2/4TXcsQ197ll3hEEY5ll3o as2A8F1iOGmuqeMm2Nv5mj+KySKtdjrQZpQ8mBPiHif5o0zeGArzfGY8blaO6j69X/U1 zA6M5rdGH5uBTUGNaXTFpKAkXNo5qoZfBWan2HOdhXBL6ypfyw950TvMjeS1h488HIUc /9nrHMvPsxVNToDuX51EZA+S4tfjdx/sglrRuymANGOa6tX5hc3XdcfpTE2ADlIAf4Is lckHE58mfaFEUkktq5neB509rUcc49IpzK3AHn/O+27QT8ivAKKqAk3EbVOLtvttzLFr sptw== X-Forwarded-Encrypted: i=1; AHgh+Rq+ZrPot8dUPdUxbpniT6mT+/YI9AZ4QoWWSZjwWShJcwRKbrrY4b1dPFFnMJF2G84NF6bYddD5VcA0jx8=@vger.kernel.org X-Gm-Message-State: AOJu0YwwMptdpE2YI3EMTBTPW8Q5DAc2oqq28oWj04g/oG6lK+lX4FtZ xep4iWCMd60Mmsp/1S0dfZWhK0lfTdrJ/eY1tgmJYPYv2Db4BULNNYHJNbMoUI6foD8= X-Gm-Gg: AR+sD10SRPK55MoUuiKRjBx2rAyd/jaih+LKA18DJLUCw/0TW+HP8USgz+CPuGV0gGF sdvvH8vpXf+vD2IhQM371n3TQjHHstYVnKak7AAKxeVLdipLoIJD7YxYWEDViUhSyxQ53pwksJm 3OdE0DxWoaCAElN7CH4tzkUQAhByGf7apab6m2HY3m4uOYfUf7wCwfbW2t/82ov5w4Rodc5EU4P 5G7W4yOG6XeVoQNEWv81j7y2BPWqLbhs6dFqi6V6xJAV+hhquwazBHQ3dKAdpXQVH37dtXu6EAv vlIcuhavxeFsY8E9Dr3/cqlcwGfq/rNP1+ncrzudSFP0nOtP7rR0geGnDI3a+jr0SHIdrcoujML K8erUztcWl+OFrL4ZzGd4Ai/4mkLwxX69ZM6vymNAfNRvK/IFvW/7aBXtt6KfRUg8vfltPYHvnK 8n6UU= X-Received: by 2002:a05:6402:e87:b0:69c:7a0a:12f with SMTP id 4fb4d7f45d1cf-69fc0ffd8ebmr1372709a12.1.1785133102295; Sun, 26 Jul 2026 23:18:22 -0700 (PDT) Received: from p15.suse.cz ([202.127.77.110]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-38f2951f0afsm2583300a91.16.2026.07.26.23.18.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Jul 2026 23:18:21 -0700 (PDT) From: Heming Zhao To: joseph.qi@linux.alibaba.com, mark@fasheh.com, jlbec@evilplan.org, hch@lst.de Cc: Heming Zhao , ocfs2-devel@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC PATCH v2 3/4] ocfs2: switch dio write path from buffer_head to iomap Date: Mon, 27 Jul 2026 14:17:59 +0800 Message-ID: <20260727061802.18485-4-heming.zhao@suse.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260727061802.18485-1-heming.zhao@suse.com> References: <20260727061802.18485-1-heming.zhao@suse.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This patch converts OCFS2's DIO write path from the legacy buffer_head infrastructure to the modern iomap framework. Key modifications and designs are as follows: 1. Dynamic Context Allocation: Refactor 'struct ocfs2_write_ctxt' to use a flexible array 'w_desc[]' instead of a fixed-size array. Dynamically allocate the context based on the write length ('w_clen') in 'ocfs2_alloc_write_ctxt()'. This prevents static limits overflow and optimizes kernel heap memory utilization. 2. Robust Mapping and Limits: Introduce 'ocfs2_dio_wr_map_blocks()' to allocate and map direct write blocks. Implement a 1 MiB cap ('OCFS2_DIO_WR_MAX_MAX_BYTES') per mapping call to restrict allocation granularity, preventing JBD2 transaction credit exhaustion during huge asynchronous sequential writes. 3. Reliable Completion Work and Fallback: - Implement 'ocfs2_iomap_dio_end_io_write()' to handle metadata completion. It converts UNWRITTEN extents, updates inode size (EOF), and deletes the inode from the orphan directory if it was appended. - Implement 'ocfs2_dio_write_end_io()' to finalize the dio lifecycle and release cluster locks safely. - Intercept '-ENOTBLK' errors from 'iomap_dio_rw()' caused by page cache invalidation failures (due to mmap/buffered collisions). Gracefully clear the error, strip the IOCB_DIRECT flag, and fall back to buffered write 4. Moved ocfs2_add_inode_to_orphan(): - moved ocfs2_add_inode_to_orphan() from ocfs2_dio_wr_map_blocks() to ocfs2_file_write_iter(). 5. Uncertain logic in code For the following code block in ocfs2_dio_wr_map_blocks(): ``` if (extend) { if (ocfs2_sparse_alloc(osb)) ret = ocfs2_zero_tail(inode, di_bh, pos); else ret = ocfs2_expand_nonsparse_inode(inode, di_bh, pos, map_len, NULL); if (ret < 0) { mlog_errno(ret); goto unlock; } } ``` I am not completely certain whether it is called only once per ocfs2_file_write_iter(), but I believe calling it multiple times will not introduce any side effects. Furthermore, testing across various scenarios showed no instances of multiple calls. Assisted-by: Gemini:gemini-3.5-flash Assisted-by: Claude:claude-sonnet-4-5 Co-developed-by: Joseph Qi Signed-off-by: Joseph Qi Signed-off-by: Heming Zhao --- fs/ocfs2/aops.c | 394 ++++++++++++++++++++++++++++++++++++-- fs/ocfs2/buffer_head_io.c | 7 +- fs/ocfs2/file.c | 91 +++++++-- fs/ocfs2/ocfs2.h | 2 + 4 files changed, 459 insertions(+), 35 deletions(-) diff --git a/fs/ocfs2/aops.c b/fs/ocfs2/aops.c index 12f5f2e3530a..9a079436c9c0 100644 --- a/fs/ocfs2/aops.c +++ b/fs/ocfs2/aops.c @@ -764,6 +764,8 @@ int ocfs2_map_folio_blocks(struct folio *folio, u64 *p_blkno, #define OCFS2_MAX_CTXT_PAGES (OCFS2_MAX_CLUSTERSIZE / PAGE_SIZE) #endif +#define OCFS2_DIO_WR_MAX_MAP_BYTES (1U << 20) /* 1 MiB */ + #define OCFS2_MAX_CLUSTERS_PER_PAGE (PAGE_SIZE / OCFS2_MIN_CLUSTERSIZE) struct ocfs2_unwritten_extent { @@ -799,8 +801,6 @@ struct ocfs2_write_ctxt { /* Type of caller. Must be one of buffer, mmap, direct. */ ocfs2_write_type_t w_type; - struct ocfs2_write_cluster_desc w_desc[OCFS2_MAX_CLUSTERS_PER_PAGE]; - /* * This is true if page_size > cluster_size. * @@ -850,6 +850,8 @@ struct ocfs2_write_ctxt { struct list_head w_unwritten_list; unsigned int w_unwritten_count; + + struct ocfs2_write_cluster_desc w_desc[]; }; void ocfs2_unlock_and_free_folios(struct folio **folios, int num_folios) @@ -917,17 +919,24 @@ static int ocfs2_alloc_write_ctxt(struct ocfs2_write_ctxt **wcp, unsigned len, ocfs2_write_type_t type, struct buffer_head *di_bh) { - u32 cend; + u32 w_cpos, cend, w_clen; struct ocfs2_write_ctxt *wc; + size_t alloc_size; + + w_cpos = pos >> osb->s_clustersize_bits; + cend = (pos + len - 1) >> osb->s_clustersize_bits; + w_clen = cend - w_cpos + 1; + + alloc_size = sizeof(struct ocfs2_write_ctxt) + + w_clen * sizeof(struct ocfs2_write_cluster_desc); - wc = kzalloc_obj(struct ocfs2_write_ctxt, GFP_NOFS); + wc = kzalloc(alloc_size, GFP_NOFS); if (!wc) return -ENOMEM; - wc->w_cpos = pos >> osb->s_clustersize_bits; + wc->w_cpos = w_cpos; wc->w_first_new_cpos = UINT_MAX; - cend = (pos + len - 1) >> osb->s_clustersize_bits; - wc->w_clen = cend - wc->w_cpos + 1; + wc->w_clen = w_clen; get_bh(di_bh); wc->w_di_bh = di_bh; wc->w_type = type; @@ -2218,7 +2227,7 @@ static int ocfs2_dio_wr_get_block(struct inode *inode, sector_t iblock, struct ocfs2_write_cluster_desc *desc = NULL; struct ocfs2_dio_write_ctxt *dwc = NULL; struct buffer_head *di_bh = NULL; - u64 p_blkno; + u64 p_blkno = 0; unsigned int i_blkbits = inode->i_sb->s_blocksize_bits; loff_t pos = iblock << i_blkbits; sector_t endblk = (i_size_read(inode) - 1) >> i_blkbits; @@ -2358,6 +2367,148 @@ static int ocfs2_dio_wr_get_block(struct inode *inode, sector_t iblock, return ret; } +/* copy from ocfs2_dio_wr_get_block */ +static int ocfs2_dio_wr_map_blocks(struct inode *inode, + struct ocfs2_map_block *map, int create) +{ + struct ocfs2_super *osb = OCFS2_SB(inode->i_sb); + struct ocfs2_inode_info *oi = OCFS2_I(inode); + struct ocfs2_write_ctxt *wc; + struct ocfs2_write_cluster_desc *desc = NULL; + struct buffer_head *di_bh = NULL; + u64 p_blkno; + sector_t iblock = map->lblk; + unsigned int i_blkbits = inode->i_sb->s_blocksize_bits; + loff_t pos = iblock << i_blkbits; + sector_t endblk = (i_size_read(inode) - 1) >> i_blkbits; + unsigned int total_len = map->len << i_blkbits; + unsigned int map_len; + unsigned int cont_clusters = 1; + int ret = 0, extend = 0, idx; + + /* + * map->len is count in ocfs2_iomap_begin according to write + * "pos" and "end", we need map twice to return different buffer state: + * 1. area in file size, not set NEW; + * 2. area out file size, set NEW. + * + * iblock endblk + * |--------|---------|---------|--------- + * |<-------area in file------->| + */ + if ((iblock <= endblk) && + ((iblock + ((total_len - 1) >> i_blkbits)) > endblk)) + total_len = (endblk - iblock + 1) << i_blkbits; + + /* + * Bound how much we allocate/prepare (and journal) per call. map->len + * below still reports only the physically-contiguous prefix to iomap; + * iomap calls us again for the remainder. + */ + map_len = min(total_len, OCFS2_DIO_WR_MAX_MAP_BYTES); + + mlog(0, "get block of %llu at %llu:%u req %u\n", + inode->i_ino, pos, map_len, total_len); + + /* + * Because we need to change file size in ocfs2_iomap_dio_end_io_write(), + * or we may need to add it to orphan dir. So can not fall to fast path + * while file size will be changed. + */ + if (pos + total_len <= i_size_read(inode)) { + /* This is the fast path for re-write. */ + ret = ocfs2_map_blocks(inode, map, OCFS2_GET_BLOCKS_CREATE); + if ((map->flags & OCFS2_MAP_MAPPED) && + !(map->flags & OCFS2_MAP_NEW) && ret == 0) + goto out; + + /* Clear state set by ocfs2_map_blocks. */ + map->flags = 0; + } else { + extend = 1; + } + + ret = ocfs2_inode_lock(inode, &di_bh, 1); + if (ret) { + mlog_errno(ret); + goto out; + } + + down_write(&oi->ip_alloc_sem); + + if (extend) { + if (ocfs2_sparse_alloc(osb)) + ret = ocfs2_zero_tail(inode, di_bh, pos); + else + ret = ocfs2_expand_nonsparse_inode(inode, di_bh, pos, + map_len, NULL); + if (ret < 0) { + mlog_errno(ret); + goto unlock; + } + } + + ret = ocfs2_write_begin_nolock(inode->i_mapping, pos, map_len, + OCFS2_WRITE_DIRECT, NULL, + (void **)&wc, di_bh, NULL); + if (ret) { + mlog_errno(ret); + goto unlock; + } + + desc = &wc->w_desc[0]; + + p_blkno = ocfs2_clusters_to_blocks(inode->i_sb, desc->c_phys); + BUG_ON(p_blkno == 0); + p_blkno += iblock & (u64)(ocfs2_clusters_to_blocks(inode->i_sb, 1) - 1); + + /* + * Calculate physically contiguous clusters by checking descriptor arrays. + * This strictly prevents map->len from expanding past non-contiguous physical bounds. + */ + for (idx = 1; idx < wc->w_clen; idx++) { + if (wc->w_desc[idx].c_phys == wc->w_desc[idx - 1].c_phys + 1 && + wc->w_desc[idx].c_needs_zero == wc->w_desc[idx - 1].c_needs_zero) { + cont_clusters++; + } else { + break; + } + } + map->pblk = p_blkno; + /* + * Only report the mapping as UNWRITTEN when the cluster is newly + * allocated or was already unwritten (desc->c_needs_zero). Reporting + * an already-written cluster as UNWRITTEN makes the iomap direct-io + * core zero the sub-block head/tail padding, which destroys existing + * data when a sub-block write lands inside a written cluster. + */ + map->flags |= OCFS2_MAP_MAPPED; + if (desc->c_needs_zero) + map->flags |= OCFS2_MAP_UNWRITTEN; + map->len = cont_clusters * ocfs2_clusters_to_blocks(inode->i_sb, 1); + + if (desc->c_needs_zero) + map->flags |= OCFS2_MAP_NEW; + + if (iblock > endblk) + map->flags |= OCFS2_MAP_NEW; + + map->flags |= OCFS2_MAP_DEFER_COMPLETION; + + ocfs2_free_unwritten_list(inode, &wc->w_unwritten_list); + ret = ocfs2_write_end_nolock(inode->i_mapping, pos, map_len, map_len, wc); + BUG_ON(ret != map_len); + ret = 0; + +unlock: + up_write(&oi->ip_alloc_sem); + ocfs2_inode_unlock(inode, 1); + brelse(di_bh); + +out: + return ret; +} + static int ocfs2_dio_end_io_write(struct inode *inode, struct ocfs2_dio_write_ctxt *dwc, loff_t offset, @@ -2557,6 +2708,12 @@ static ssize_t ocfs2_direct_IO(struct kiocb *iocb, struct iov_iter *iter) ocfs2_dio_end_io, 0); } +static int ocfs2_iomap_alloc(struct inode *inode, struct ocfs2_map_block *map, + unsigned int flags) +{ + return ocfs2_dio_wr_map_blocks(inode, map, 1); +} + static int ocfs2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, unsigned int flags, struct iomap *iomap, struct iomap *srcmap) { @@ -2564,6 +2721,7 @@ static int ocfs2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, struct ocfs2_map_block map; struct ocfs2_inode_info *oi = OCFS2_I(inode); u8 blkbits = inode->i_blkbits; + unsigned int orig_mlen; if ((offset >> blkbits) > OCFS2_MAX_LOGICAL_BLOCK) return -EINVAL; @@ -2575,9 +2733,32 @@ static int ocfs2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, map.len = min_t(loff_t, (offset + length - 1) >> blkbits, OCFS2_MAX_LOGICAL_BLOCK) - map.lblk + 1; map.flags = 0; + orig_mlen = map.len; if (flags & IOMAP_WRITE) { - /* todo */ + /* + * We check here if the blocks are already allocated, then we + * don't need to start a journal txn and we can directly return + * the mapping information. This could boost performance + * especially in multi-threaded overwrite requests. + */ + if (offset + length <= i_size_read(inode)) { + down_read(&oi->ip_alloc_sem); + ret = ocfs2_map_blocks(inode, &map, 0); + up_read(&oi->ip_alloc_sem); + /* + * For atomic writes the entire requested length should + * be mapped. + */ + if (map.flags & OCFS2_MAP_MAPPED) { + if ((!(flags & IOMAP_ATOMIC) && ret > 0) || + (flags & IOMAP_ATOMIC && ret >= orig_mlen)) { + goto out; + } + } + map.len = orig_mlen; + } + ret = ocfs2_iomap_alloc(inode, &map, flags); } else { down_read(&oi->ip_alloc_sem); ret = ocfs2_map_blocks(inode, &map, 0); @@ -2587,6 +2768,7 @@ static int ocfs2_iomap_begin(struct inode *inode, loff_t offset, loff_t length, if (ret < 0) return ret; +out: /* * Before returning to iomap, let's ensure the allocated mapping * covers the entire requested length for atomic writes. @@ -2653,8 +2835,8 @@ static int ocfs2_iomap_dio_end_io(struct kiocb *iocb, ssize_t size, int error, return error; } -static int ocfs2_dio_end_io_r_pr(struct kiocb *iocb, ssize_t size, - int error, unsigned int flags) +static int ocfs2_iomap_dio_end_io_r_pr(struct kiocb *iocb, ssize_t size, + int error, unsigned int flags) { if (error) mlog_ratelimited(ML_ERROR, "Direct IO failed, bytes = %lld errno:%d", @@ -2662,13 +2844,199 @@ static int ocfs2_dio_end_io_r_pr(struct kiocb *iocb, ssize_t size, error = ocfs2_iomap_dio_end_io(iocb, size, error, 0); -bail: return error; } const struct iomap_dio_ops ocfs2_iomap_dio_ops_r_pr = { - .end_io = ocfs2_dio_end_io_r_pr, + .end_io = ocfs2_iomap_dio_end_io_r_pr, +}; + +/* copy from ocfs2_dio_end_io_write */ +static int ocfs2_iomap_dio_end_io_write(struct inode *inode, + loff_t offset, + ssize_t bytes) +{ + struct ocfs2_cached_dealloc_ctxt dealloc; + struct ocfs2_extent_tree et; + struct ocfs2_super *osb = OCFS2_SB(inode->i_sb); + struct ocfs2_inode_info *oi = OCFS2_I(inode); + struct buffer_head *di_bh = NULL; + struct ocfs2_dinode *di; + struct ocfs2_alloc_context *data_ac = NULL; + struct ocfs2_alloc_context *meta_ac = NULL; + handle_t *handle = NULL; + loff_t end = offset + bytes; + int ret = 0, credits = 0; + struct ocfs2_map_block map; + unsigned int blkbits = inode->i_blkbits; + unsigned int max_blocks; + unsigned int ue_cpos = 0, ue_phys = 0, ue_len = 0; + unsigned int curr_lblk, end_lblk; + + map.lblk = offset >> blkbits; + max_blocks = (bytes + offset) >> osb->s_clustersize_bits; + + ocfs2_init_dealloc_ctxt(&dealloc); + + ret = ocfs2_inode_lock(inode, &di_bh, 1); + if (ret < 0) { + mlog_errno(ret); + goto out; + } + + down_write(&oi->ip_alloc_sem); + + di = (struct ocfs2_dinode *)di_bh->b_data; + + ocfs2_init_dinode_extent_tree(&et, INODE_CACHE(inode), di_bh); + + /* Attach dealloc with extent tree in case that we may reuse extents + * which are already unlinked from current extent tree due to extent + * rotation and merging. + */ + et.et_dealloc = &dealloc; + + ret = ocfs2_lock_allocators(inode, &et, 0, (bytes >> osb->s_clustersize_bits)*2, + &data_ac, &meta_ac); + if (ret) { + mlog_errno(ret); + goto unlock; + } + + credits = ocfs2_calc_extend_credits(inode->i_sb, &di->id2.i_list); + + handle = ocfs2_start_trans(osb, credits); + if (IS_ERR(handle)) { + ret = PTR_ERR(handle); + mlog_errno(ret); + goto unlock; + } + ret = ocfs2_journal_access_di(handle, INODE_CACHE(inode), di_bh, + OCFS2_JOURNAL_ACCESS_WRITE); + if (ret) { + mlog_errno(ret); + goto commit; + } + + curr_lblk = offset >> blkbits; + /* + * Round the end up so the final partial block (sub-block direct I/O) + * is included; otherwise the last, partially-written cluster is left + * unwritten and reads back as zero. + */ + end_lblk = (offset + bytes + (1 << blkbits) - 1) >> blkbits; + while (ret >= 0 && curr_lblk < end_lblk) { + memset(&map, 0, sizeof(map)); + map.lblk += curr_lblk; + map.len = end_lblk - curr_lblk; + + ret = ocfs2_assure_trans_credits(handle, credits); + if (ret < 0) { + mlog_errno(ret); + break; + } + + ret = ocfs2_map_blocks(inode, &map, 0); + if (ret < 0) { + mlog_errno(ret); + break; + } + if (ret == 0) + break; + + if (map.flags & OCFS2_MAP_UNWRITTEN) { + ue_cpos = ocfs2_blocks_to_clusters(inode->i_sb, map.lblk); + ue_phys = ocfs2_blocks_to_clusters(inode->i_sb, map.pblk); + ue_len = ocfs2_clusters_for_blocks(inode->i_sb, + map.lblk + map.len) - ue_cpos; + + ret = ocfs2_mark_extent_written(inode, &et, handle, + ue_cpos, ue_len, + ue_phys, + meta_ac, &dealloc); + if (ret < 0) { + mlog_errno(ret); + break; + } + } + curr_lblk += map.len; + } + + if (end > i_size_read(inode)) { + ret = ocfs2_set_inode_size(handle, inode, di_bh, end); + if (ret < 0) + mlog_errno(ret); + } + +commit: + ocfs2_commit_trans(osb, handle); + +unlock: + up_write(&oi->ip_alloc_sem); + + if (data_ac) + ocfs2_free_alloc_context(data_ac); + if (meta_ac) + ocfs2_free_alloc_context(meta_ac); + + down_write(&oi->ip_alloc_sem); + if (!ret && (di->i_flags & cpu_to_le32(OCFS2_DIO_ORPHANED_FL))) { + ret = ocfs2_del_inode_from_orphan(osb, inode, di_bh, 0, 0); + if (ret < 0) + mlog_errno(ret); + } + up_write(&oi->ip_alloc_sem); + ocfs2_inode_unlock(inode, 1); + brelse(di_bh); + +out: + ocfs2_run_deallocs(osb, &dealloc); + return ret; +} + +static int ocfs2_dio_write_end_io(struct kiocb *iocb, ssize_t size, + int error, unsigned int flags, int level) +{ + struct inode *inode = file_inode(iocb->ki_filp); + loff_t offset = iocb->ki_pos; + int ret = 0; + + if (error) + mlog_ratelimited(ML_ERROR, "Direct IO failed, bytes = %lld errno:%d", + (long long)size, error); + + if (size && ((flags & IOMAP_DIO_UNWRITTEN) || + (offset + size > i_size_read(inode)))) { + ret = ocfs2_iomap_dio_end_io_write(inode, offset, size); + if (ret < 0) + mlog_errno(ret); + } + + error = ocfs2_iomap_dio_end_io(iocb, size, error, level); + + return error; +} + +static int ocfs2_iomap_dio_end_io_w_pr(struct kiocb *iocb, ssize_t size, + int error, unsigned int flags) +{ + return ocfs2_dio_write_end_io(iocb, size, error, flags, 0); +} + +static int ocfs2_iomap_dio_end_io_w_ex(struct kiocb *iocb, ssize_t size, + int error, unsigned int flags) +{ + return ocfs2_dio_write_end_io(iocb, size, error, flags, 1); +} + +const struct iomap_dio_ops ocfs2_iomap_dio_ops_w_pr = { + .end_io = ocfs2_iomap_dio_end_io_w_pr, +}; + +const struct iomap_dio_ops ocfs2_iomap_dio_ops_w_ex = { + .end_io = ocfs2_iomap_dio_end_io_w_ex, }; + const struct address_space_operations ocfs2_aops = { .dirty_folio = block_dirty_folio, .read_folio = ocfs2_read_folio, diff --git a/fs/ocfs2/buffer_head_io.c b/fs/ocfs2/buffer_head_io.c index 493f2209cca5..03906ab737ec 100644 --- a/fs/ocfs2/buffer_head_io.c +++ b/fs/ocfs2/buffer_head_io.c @@ -332,6 +332,8 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr, wait_on_buffer(bh); put_bh(bh); bhs[i] = NULL; + } else if (bh && buffer_uptodate(bh)) { + clear_buffer_uptodate(bh); } continue; } @@ -360,11 +362,8 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr, BUG_ON(buffer_jbd(bh)); clear_buffer_needs_validate(bh); status = validate(sb, bh); - if (status) { - if (buffer_uptodate(bh)) - clear_buffer_uptodate(bh); + if (status) goto read_failure; - } } } diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c index a0b3883216bc..2d78f2863acf 100644 --- a/fs/ocfs2/file.c +++ b/fs/ocfs2/file.c @@ -48,6 +48,7 @@ #include "quota.h" #include "refcounttree.h" #include "ocfs2_trace.h" +#include "namei.h" #include "buffer_head_io.h" @@ -2378,6 +2379,8 @@ static int ocfs2_prepare_inode_for_write(struct file *file, static bool ocfs2_should_use_dio(struct kiocb *iocb, struct iov_iter *iter, struct inode *inode) { + struct ocfs2_super *osb = OCFS2_SB(inode->i_sb); + /* * Fallback to buffered I/O if we see an inode without * extents. @@ -2385,6 +2388,11 @@ static bool ocfs2_should_use_dio(struct kiocb *iocb, struct iov_iter *iter, if (OCFS2_I(inode)->ip_dyn_features & OCFS2_INLINE_DATA_FL) return false; + /* Fallback to buffered I/O if we do not support append dio. */ + if (iocb->ki_pos + iter->count > i_size_read(inode) && + !ocfs2_supports_append_dio(osb)) + return false; + return true; } @@ -2405,6 +2413,8 @@ static ssize_t ocfs2_file_write_iter(struct kiocb *iocb, i_size_read(inode) ? 1 : 0); int direct_io = iocb->ki_flags & IOCB_DIRECT ? 1 : 0; int nowait = iocb->ki_flags & IOCB_NOWAIT ? 1 : 0; + int dio_flags = 0; + ssize_t buffered; trace_ocfs2_file_write_iter(inode, file, file->f_path.dentry, (unsigned long long)OCFS2_I(inode)->ip_blkno, @@ -2424,8 +2434,6 @@ static ssize_t ocfs2_file_write_iter(struct kiocb *iocb, } else inode_lock(inode); - ocfs2_iocb_init_rw_locked(iocb); - /* * Concurrent O_DIRECT writes are allowed with * mount_option "coherency=buffered". @@ -2489,26 +2497,73 @@ static ssize_t ocfs2_file_write_iter(struct kiocb *iocb, saved_ki_complete = xchg(&iocb->ki_complete, NULL); } - /* communicate with ocfs2_dio_end_io */ - ocfs2_iocb_set_rw_locked(iocb, rw_level); + if (direct_io && ocfs2_should_use_dio(iocb, from, inode)) { + if (ocfs2_clusters_for_bytes(inode->i_sb, iocb->ki_pos + count) > + ocfs2_clusters_for_bytes(inode->i_sb, i_size_read(inode))) { + /* + * when we are going to alloc extents beyond file size, add the + * inode to orphan dir, so we can recall those spaces when + * system crashed during write. + */ + ret = ocfs2_add_inode_to_orphan(osb, inode); + if (ret < 0) { + mlog_errno(ret); + goto out; + } + dio_flags = IOMAP_DIO_FORCE_WAIT; + } + const struct iomap_dio_ops *dops = rw_level ? + &ocfs2_iomap_dio_ops_w_ex : &ocfs2_iomap_dio_ops_w_pr; + struct iomap_dio *dio; + + dio = __iomap_dio_rw(iocb, from, &ocfs2_iomap_ops, + dops, dio_flags, NULL, 0); + if (dio == NULL) { + /* No I/O issued; rw_lock still held. */ + written = 0; + } else if (IS_ERR(dio)) { + written = PTR_ERR(dio); + /* + * -EIOCBQUEUED: async I/O; end_io will drop the lock on + * completion. Any other error means no I/O started and + * end_io was not called, so the cleanup path below drops + * the lock. + */ + if (written == -EIOCBQUEUED) + rw_level = -1; + } else { + /* + * Synchronous completion. iomap_dio_complete() runs + * end_io, which drops the lock unless the mapping + * bounced us back to buffered I/O (returns 0, lock + * retained). + */ + written = iomap_dio_complete(dio); + if (written != 0) + rw_level = -1; + } + /* + * iomap_dio_rw() returns -ENOTBLK when it could not invalidate + * the page cache for the DIO range (e.g. racing buffered/mmap + * I/O on the same file, as exercised by generic/095). Fall back + * to a buffered write for the remaining data, the same way + * ext4/xfs do, instead of leaking -ENOTBLK to userspace. + */ + if (written == -ENOTBLK) + written = 0; + if (written >= 0 && iov_iter_count(from)) { + iocb->ki_flags &= ~IOCB_DIRECT; + buffered = __generic_file_write_iter(iocb, from); + written = direct_write_fallback(iocb, from, written, buffered); + } + } else { + iocb->ki_flags &= ~IOCB_DIRECT; + written = __generic_file_write_iter(iocb, from); + } - written = __generic_file_write_iter(iocb, from); /* buffered aio wouldn't have proper lock coverage today */ BUG_ON(written == -EIOCBQUEUED && !direct_io); - /* - * deep in g_f_a_w_n()->ocfs2_direct_IO we pass in a ocfs2_dio_end_io - * function pointer which is called when o_direct io completes so that - * it can unlock our rw lock. - * Unfortunately there are error cases which call end_io and others - * that don't. so we don't have to unlock the rw_lock if either an - * async dio is going to do it in the future or an end_io after an - * error has already done it. - */ - if ((written == -EIOCBQUEUED) || (!ocfs2_iocb_is_rw_locked(iocb))) { - rw_level = -1; - } - if (unlikely(written <= 0)) goto out; diff --git a/fs/ocfs2/ocfs2.h b/fs/ocfs2/ocfs2.h index a775c1869293..b228b10b5e71 100644 --- a/fs/ocfs2/ocfs2.h +++ b/fs/ocfs2/ocfs2.h @@ -551,6 +551,8 @@ struct ocfs2_map_block { extern const struct iomap_ops ocfs2_iomap_ops; extern const struct iomap_dio_ops ocfs2_iomap_dio_ops_r_pr; +extern const struct iomap_dio_ops ocfs2_iomap_dio_ops_w_pr; +extern const struct iomap_dio_ops ocfs2_iomap_dio_ops_w_ex; /* Flags used by ocfs2_map_blocks() */ #define OCFS2_GET_BLOCKS_CREATE (0x0001) -- 2.54.0