From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF5D2379C32 for ; Wed, 22 Jul 2026 18:58:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784746741; cv=none; b=u2/z0x473T1q57Lxz6QBPlfQPjOEdSBL8uSod6Vvll7lXW2B7MfhQHht4mUMIeTIJ1ZeyFNjfRc02GQJSx1k/gn6CV/B043ohxS5aj8twq+4MKw6Fy02jEb5/2WFQC8BmB6iOJzs5eneJcST5qcuDskNXL3PLbLhvgwdoqjFD+Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784746741; c=relaxed/simple; bh=1EblrzgLJcRDCn1v3FJEGmGB3FLGsptGxFAPO04kyTs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Ulv6BKkC5VwM8ySmpnvlGGL02gKy7VK8FEDm26AXpZv8kRFh7bCn0KflWL83EWny9WtthJi/IJ5rM0/jDPU3ryOPo1hOchVj0xiOh3QDFtUhi27NNPSyVHt4KstWjeQv3bt4UlbogAle+9pzlA7DWs0DnvoTwcpBGOD3AZ3EBfc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meshstor.io; spf=pass smtp.mailfrom=meshstor.io; dkim=pass (2048-bit key) header.d=meshstor.io header.i=@meshstor.io header.b=nWLDJetw; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meshstor.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meshstor.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meshstor.io header.i=@meshstor.io header.b="nWLDJetw" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-49556f97a9dso28796915e9.1 for ; Wed, 22 Jul 2026 11:58:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meshstor.io; s=google; t=1784746738; x=1785351538; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=Roft6Ikjq+qhKFFk0rGulNK01SEMvk3YKlHglgEZK0o=; b=nWLDJetwc4PT71WlB0yppd3cgwnFSO0G0ShMgQYptDibs4g5PUbfw4QkMVnrzpVrFe XARBIX3yPEa5YAKxzcZFFN8z/GTCctWC7+n7YMDbOb7qKBBB55EQvbhTJtqf04rNvF/2 YkBcG5dyj1XtNdE5BLiGL1DfoNLM6B8ahYPscNf2YqIucoXVtiOELpo/dpnAN4DMr1e+ THEbX1fO/Sd03SBw92H0xzku4j4qiC+O7eq2fcDfZKF0J9WIrwsZPpX2pQlcLTWSlmJ3 d7XJn+/WvfIZ6SSGjxZM1G0E02lLfGGBUFXZAFaQ/zjpnCvLyvFjekwQIbXuwm0wZXnU sZDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784746738; x=1785351538; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Roft6Ikjq+qhKFFk0rGulNK01SEMvk3YKlHglgEZK0o=; b=gXf41l1XRK5Ew4MLFMtLcsDMIRkq8NPq4Cji2kHkiDzPjiaTG41Qew06bCjrEwI/0M nwwSgOztjEM9d+/DyBWFPg7ndThG2GPYHHmFarFN58yqlkh/I0JN5kLBnxDvD3JsGDPO d7+oxBKSxnV/FQhJja2Vrov+/02iGQFcSNzYcwc4fKNizpSIzECuv+0gv5IGcEYBIPx6 tFv+PXZcyf/xIvXHBhRvW6cbq8ePyPuWVOQ0b2zhmV0f2QthfbvqO4nxT46xis4vDTG3 8QxumkaMa+JKm0ogVminy9tywpMajZAQy2UdRG68AsOpdL1b6tOuw/UExnqDeHPimA8u qfHQ== X-Forwarded-Encrypted: i=1; AHgh+RrGSOKwn00+PT/xIrRHJpf7tb3ooodlPR0s4fCPn5ROK5ZsW780ExJ56bHJsOIZBA/yl3STK99KhJGN8Ts=@vger.kernel.org X-Gm-Message-State: AOJu0YysLpgmwxyCpO/LRwtJQFtwxMIcwZFJPiEF3CkbAl0QsNdwiIQz Vh9ga3UaQ0iIsT3hBb8N9S2C1mfZLTIiKDXfEM1ChU89n2p6F814zeJkSjMTqrgg9Q== X-Gm-Gg: AR+sD10Dm11ZQs2u6aif1kQVskj798EahdZl8S/c/Z9rH+YPtwCcfcyuUTkIzI8lquA 5OmlpO5l5T/Xs6Y6uRTMMjOE2OdBWAHLsIBcKgPpkrVwRlPmc6sLx0hqaoz3LH088c7FjClczPI bzFFAOXAxV4DpOjPChkNea1Cd6Mwj2mDOat2UWUdCRM6cqkKoM9m4x6wjrJb8JlbRO2jTVQuv9J kmn1aAWOJKO57lmHeDko7RzoS7p4hnhxudmaIbmg1oFj1gGAf9WHc9YMzgIgSTH115yyzIqGN2T RYzOpxRIs4Pz0d5Gv4GCrNS35yGiwCzL5kXosqx1eI7rfYpi73G6kzfBeHfum2kvq8x0rLNxmSy MyHMzXKXiQflITLnxJY/rbL8GxFBvEDVEWlTyO7AZiIY89kbnRTwrt52LTxdfXhxiv+mrIifKeV pxqPZtD1Hac7FGRAxJ6BjkACPQ16j67BmYjs5/pfO3otOTH74= X-Received: by 2002:a05:600c:1c16:b0:495:5022:71b9 with SMTP id 5b1f17b1804b1-495502272f2mr281514875e9.33.1784746738206; Wed, 22 Jul 2026 11:58:58 -0700 (PDT) Received: from ai-pc ([46.172.241.174]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-495730353d9sm13740055e9.12.2026.07.22.11.58.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jul 2026 11:58:57 -0700 (PDT) From: Mykola Marzhan To: Jens Axboe , Song Liu , Yu Kuai , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-raid@vger.kernel.org, linux-nvme@lists.infradead.org Cc: Li Nan , Xiao Ni , Guoqing Jiang , Leon Romanovsky , Jason Gunthorpe , Kiran Kumar Modukuri , Chaitanya Kulkarni , Logan Gunthorpe , Bjorn Helgaas , Shivaji Kant , Pranjal Shrivastava , Henrique Carvalho , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH v4 0/9] block,md,nvme: correct handling of unsupported P2PDMA transfers Date: Wed, 22 Jul 2026 20:58:32 +0200 Message-ID: <20260722185841.449934-1-mykola@meshstor.io> X-Mailer: git-send-email 2.52.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit md treats an unsupported P2PDMA transfer to a member as success, and nvme-rdma retries one forever. Whether peer memory can be DMA-mapped depends on the PCIe topology between the two devices, and since v6.17 the failure completes as BLK_STS_INVAL: retryable for multipath and deliberately ignored by md for member failures, so mirrors silently diverge. v1-v3 restored BLK_STS_TARGET for these failures and tagged P2PDMA bios in md. Logan Gunthorpe proposed a dedicated BLK_STS_P2PDMA instead, so no consumer needs to know whether an I/O was P2P, and wrote the core patches; v4 is built on them. Patches 1, 5, 7 and 8 are his and keep his authorship (7 and 8 with completion-handler additions). 1 block: add BLK_STS_P2PDMA (stable, v6.17) 2 md: keep REQ_NOMERGE on P2PDMA bios 3 md/raid1: serialize non-write-behind writes on CollisionCheck rdevs (pre-existing bug patch 4 would widen; stable) 4 md/raid1: no write-behind for P2PDMA bios 5 md/raid1,raid10: factor out raid1_write_error() helper 6 md/raid1,raid10: keep REQ_NOMERGE on narrow_write_error() retry clones 7 md/raid1,raid10: skip futile retries on P2PDMA mapping failures 8 md/raid1,raid10: set IO_BLOCKED in case of BLK_STS_P2PDMA 9 nvme-rdma: return BLK_STS_P2PDMA (stable, v7.1; needs patch 1) Routing: patch 1 block, 2-8 md, 9 nvme; 7-9 use the status patch 1 adds. Patch 1 alone already fixes the divergence (md stops ignoring the failure) and the multipath retry storm; the later patches only refine the fallout, so the series bisects safely. The md patches fix a v7.2-rc1 regression (02666132403a) and carry no stable tags. Tested with the QEMU raid1/raid10 error matrix and a ConnectX-4 Lx. After patch 1, block-layer error injection accepts status=P2PDMA, so the md completion paths can be exercised on any member with no P2P hardware. The patches were developed with AI assistance (see the Assisted-by trailers); all code was human-reviewed and tested. Against v7.2-rc4 (Logan's base). Changes in v4: - new BLK_STS_P2PDMA (Logan) replaces restoring BLK_STS_TARGET; the R1BIO_P2PDMA/R10BIO_P2PDMA state bits are gone and the md changes shrink to completion-time status tests - blk_dma_map_iter_start() gets the explicit PCI_P2PDMA_MAP_NOT_SUPPORTED case (Logan's patch; v3 kept default: only) - write completions record the whole failed range as bad blocks instead of narrow_write_error()'s per-block retries; v3's coarse retry is gone -- the new status is deterministic, nothing to re-attempt - read path: bi_status is evaluated before bio_put() Link: https://lore.kernel.org/linux-raid/20260718162547.448892-1-mykola@meshstor.io/ [v1] Link: https://lore.kernel.org/linux-raid/20260719105327.864949-1-mykola@meshstor.io/ [v2] Link: https://lore.kernel.org/linux-raid/20260721174502.111503-1-mykola@meshstor.io/ [v3] Link: https://lore.kernel.org/linux-raid/9f846c85-f427-433a-b87d-8e9e3a4406e7@deltatee.com/ [Logan's proposal] Logan Gunthorpe (4): block: add BLK_STS_P2PDMA for unsupported peer-to-peer transfers md/raid1,raid10: factor out raid1_write_error() helper md/raid1,raid10: skip futile retries on P2PDMA mapping failures md/raid1,raid10: set IO_BLOCKED in case of BLK_STS_P2PDMA Mykola Marzhan (5): md: ensure REQ_NOMERGE is set on P2PDMA bios md/raid1: serialize non-write-behind writes on CollisionCheck rdevs md/raid1: don't use write-behind for P2PDMA bios md/raid1,raid10: keep REQ_NOMERGE on narrow_write_error() retry clones nvme-rdma: return BLK_STS_P2PDMA for unsupported P2P transfers block/blk-core.c | 3 +++ block/blk-mq-dma.c | 3 ++- drivers/md/md.c | 10 ++++++-- drivers/md/md.h | 15 ++++++++++++ drivers/md/raid1-10.c | 16 +++++++++++++ drivers/md/raid1.c | 48 +++++++++++++++++++++++---------------- drivers/md/raid10.c | 26 ++++++++++++--------- drivers/nvme/host/rdma.c | 38 ++++++++++++++++++------------- include/linux/blk_types.h | 10 ++++++++ 9 files changed, 120 insertions(+), 49 deletions(-) base-commit: 1590cf0329716306e948a8fc29f1d3ee87d3989f -- 2.52.0