From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D3CC4B0486 for ; Sun, 9 Aug 2026 03:28:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786246096; cv=none; b=Zxdxccje86dqk/xGDSvZxp2lHytpWNmi+NAdHMP4XchPUSMVtNxpagGQZKsgQr198tMKdh2UwZkphHpPZtkbMFAYwWfGFVp0cJQowOdyfAOOoIPWmkhNbSPfnFvv+ONUIdluUIQ2yX30yXE6UGkND9dmt3KfWLG030kMkRUJvdQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786246096; c=relaxed/simple; bh=OaaO9X8gqrUDj05mGTYFdmi8nWGXU2LWgsF5OZwTkQE=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=luPy/qcg+zAw+deue/s7bpyoJNXzyzofA5ak9BTdvLaKBkQExLCSmMjCLUlndr5dfvvpAPy9PQ+jBD4jfIBZLaqhUKVVDMKXk+VehGj/xKsM7vqs7smXisj4qKDisw2U8aj5ZJTcHxE4KHBlax9sOrPO4X4bBtnKKleEaNnrZ2I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Aq+mk915; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Aq+mk915" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-38e69bdb0fcso665975a91.1 for ; Sat, 08 Aug 2026 20:28:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786246095; x=1786850895; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=PWRK37gCFVYLj4iCEyB0nOW3fLyTc/Qfqd+mtnl3Xik=; b=Aq+mk915TUH3OV3/MItWeSpBDfVVo//3gjvmVNuIclNOur1ILucWU8oPmA9GFVDSTs cTJLol9CpZsxoMB1irnyn8PWqBz3aPbvdKXADAj+3TjA+k03hySWObI0gTtbs4tlYbLO OBEmn9nlwLLjRPRkL/oE4KsFdTa8IY+VreNsRTstIs3qToSn7nO1YfrR3c2rupyxfUG2 jjHenqqnekNKXn/FzgtMHSAza/86uTQ+8gilY5QdtCUSGhwE9MGicnwm3TDGVsqJxRlA aUyMJNvpk/rD3u0Dq7+YNChPML+/wD+9tMck9Ju6F7Uq6Oq4KMK0SV5H+IOCWV4iXyT9 eVRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786246095; x=1786850895; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=PWRK37gCFVYLj4iCEyB0nOW3fLyTc/Qfqd+mtnl3Xik=; b=MtyY9jctBZ4Bp79Ks4+VInoNstzdyryyH5LTjr2iSlc9SCFsohBy87OGibg3F8e2Ve P+gyoMEC1Nu17m6GmHPFv1i0LdiajUavCU90g+uaqTYw8RWTG4har+EcVItYqCY/btVH LqT5QYoEs6PMqtd7qndRzaH5qkGUeD/tfDWx2rUC2kLkuXg7Bmstr00mQf0v/f9h1evh i+uOtjv2eJKbr5iiEjKoHbr6ptIbWTY/epDyMD0ChltrVNkMTkGEGfZDGH3cSCeE2Zez 6eh7EiFgzZYnc3Gv4OffN2xcEJUuXUBhQytqNQQFuxMX7K63sIl5ydjRpv+f1dlYp4Vu 8POA== X-Forwarded-Encrypted: i=1; AHgh+Rr4HESIKBiJwavtBCut5nSNYQihw3lsJhpvJCh2WZ5kq5OkfJTC0bIzMmf58m6nB23Wh3ZrwwZA/R++AMo=@vger.kernel.org X-Gm-Message-State: AOJu0YyK+lj0MAxUWERc52oS3jfuSeXHXwruhj7r6PSluj5uI5l1tRd+ mRkkfM3VD99IySjAZ9dwun7lz/alSqYL4koIIU7reQHFwfSewlTaqFtL X-Gm-Gg: AR+sD10fy7R69Xjo5ZfUIH76m3LqptJgw491b+/wSqpvES/KSOfa4Y3vK4NY6/mZtuM 64miRYryMsoxEXKwg9k/z6qtHnU7JLro8B8EgyAvayJAk6/tZfaMOJCJX3RHG8uPY05hk4+4A4U fbGbNqcEnDIm3+tYTcwv8nGIJpSHKK/R7mLP71p5xisyQwuhdCTDTUsNUv90WKcXfO1lxctyENV sHRiMPAl8pTqU22k8eks+OqATdUNQO+d+CFlTAwYfrDbtzm228BjjLi5Yxc3CP3jOCnHm+3zrVk TDbG4YMTkTnt4rQQy4ZHCrSGXlMTyTTzgckmO8l3L65KzzF3L25rNnSPz8o/DNg7ALCI/hM2C1S cU6HhDgojqAW/s4LJkMSUvibD02U/8BLpCZs/CL9731sPJ4jQOlY50KrTRF0B6Wtl/s/ShT5lqY iAqllEoc2YtC3OMsKVRB/WpyKZWg0cnjJ0qTEKUOGPLToLwHe+lahc8nZknKefHFGVhnr93aZ2O qW5B6jvqgO7JM+lgMeNpHpPk18sDLltWOBUiIIIAnY5qSHdceUFaHq08FZ872c= X-Received: by 2002:a17:90b:2d84:b0:38e:11ba:992c with SMTP id 98e67ed59e1d1-3903c58ece8mr33705159a91.12.1786246094757; Sat, 08 Aug 2026 20:28:14 -0700 (PDT) Received: from homelab1-WRX80-Creator.localdomain (p790255fa.sitmnt01.ap.so-net.ne.jp. [121.2.85.250]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3926254ee45sm7587626a91.2.2026.08.08.20.28.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Aug 2026 20:28:14 -0700 (PDT) From: Shuhei Takeshita To: dennis.dalessandro@cornelisnetworks.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, dean.luick@cornelisnetworks.com, Shuhei Takeshita Subject: [PATCH 0/2] IB/hfi1: fix the PIO_CRED credit-return mmap Date: Sun, 9 Aug 2026 12:27:41 +0900 Message-ID: <20260809032743.2671579-1-jyohuku.alterego@gmail.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The PIO_CRED case of hfi1_file_mmap() hands user space the credit-return page for its send context. On a two-socket host with a translating IOMMU it instead returns a page mapped from a frame above MAXPHYADDR, and the first user read takes "Corrupted page table" / "Oops: Bad pagetable". Correcting only the offset arithmetic replaces the Oops with a silent wrong-page mapping, so every transfer that uses send PIO hangs instead. The failure is intermittent: it depends on which of the credit-return pages the context's entry lands on, which follows the hardware send context index and so varies from boot to boot. Patch 1 fixes the node used to resolve the credit-return buffer, which has been wrong since the driver was merged. Patch 2 fixes the byte offset applied to a typed pointer and the use of dma_mmap_coherent(), both introduced when this case was converted to the DMA API. They are separate patches because they have different Fixes: tags, but only the two together make the mapping correct: patch 2 without patch 1 turns the hang into an -ENXIO from iommu_dma_mmap()'s bounds check, because the cross-node offset exceeds the buffer. Found while debugging psm2_ep_open() Oopsing a Dell T7610 (Xeon E5-2650 v2, Intel IOMMU in DMA-FQ mode) talking to a Threadripper PRO 3995WX, both Omni-Path 100. With the series applied to both hosts, psm2_ep_open() succeeds, send PIO, send DMA and the default mixed mode all work, and MPI over the PSM2 MTL sustains 95 Gb/s. Forcing send PIO only (PSM2_SDMA=0), which hung indefinitely before, now completes. Based on rdma/for-rc at 31b7c700670830a0e8a4cdcd451c88a13cc5dc48. Shuhei Takeshita (2): IB/hfi1: Resolve the credit-return buffer through the send context's node IB/hfi1: Fix the PIO_CRED credit-return mmap drivers/infiniband/hw/hfi1/file_ops.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) -- 2.43.0