From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF585282F03 for ; Wed, 17 Jun 2026 02:02:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781661736; cv=none; b=evkxwtGHp9EY7MR8AzOExZlNUUcdsFH22glR4EEZ1cQkgZh2KFmzfam4XDl2UcG4MCMVVc+buZvooIm8PCqB/g3RdPmRdBSWnm/mvru/+QmtCvb1JpQmvFuucrMm5mShtO9j7Z8pU+7vYytSlMCqj2H7q2F3bVptxx7vIEyX6cY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781661736; c=relaxed/simple; bh=/6qqonAB1rvfwi0nHfBcjTnP0l6KSbTgCUpfm1TzYf8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=i4ls2JMKXcYMI5A9t2hbTxZU479AY8zsEODEiSG0YYpqt9sayhT7bCCpQOfbD5lzjuuohfIpspd/CgNcE1O9wSz1Py6QLoJopEVi6YkcxPx18qbemZjH26aVbQZzeJKOuglLJunkAQQW1ceQT8GryLsxBQz7PRGnyEoMi76bX00= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ankey.net; spf=none smtp.mailfrom=ankey.net; dkim=pass (2048-bit key) header.d=ankey-net.20251104.gappssmtp.com header.i=@ankey-net.20251104.gappssmtp.com header.b=tOQJtQCx; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ankey.net Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ankey.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ankey-net.20251104.gappssmtp.com header.i=@ankey-net.20251104.gappssmtp.com header.b="tOQJtQCx" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2c0b9328c4aso38774115ad.0 for ; Tue, 16 Jun 2026 19:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ankey-net.20251104.gappssmtp.com; s=20251104; t=1781661734; x=1782266534; darn=vger.kernel.org; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z7ztbMiRRaO/C1m+UU2CZ46ifpbNE3xQILmUDMNi364=; b=tOQJtQCxmO8sWi2mM1fh9NbmclYDichvbh8WHbygZw17Zn5GoApHfy2ktyo2wuH7lK jzRF8CgBklu+OV83/NHJxPlGbhm2LbAyipRS/a09aalFDcaaUNESMELmfXxYomfa9aTc vOH3hmx9VeopheqyQtAcElaVq1W40UNnBE0eC47ygMAVkx48Ka0oS825sPy/XB0VwAcK 5ZsrrhtVu66uEd7cqg08yLQUztFx96Co1p4EvcdRA+62kIZRSTt+LwtzKVkvEJ3tmcL6 f2vjjwbNYfW3bB0F1X8F5Xec0FOMONnraiDd/zDk2JNTguRw3xQkuhbWEICBnCFTBXdE 24Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781661734; x=1782266534; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=z7ztbMiRRaO/C1m+UU2CZ46ifpbNE3xQILmUDMNi364=; b=DsvkDqt6tkgodrYiICRE/cHNtwB/prLwYbIxXYbvG49QGlYmxFf+tnPwmeLwRWrYzz u4mnQG7D1GNEVcIPobXkRJMR1DaafruyOyTdiB7DyzV1v+gjspIsAQQolbZQJvgB6JJX GGWLdOTvkmq5VAY4UZDggNgz/493UeVfqnpXGlYw7HARk2mqVsXbQXZ5c4O5FTwp0v2L /fTbQNqGdQYjZXLhY8MRnu+TCVl8enHPecfBTBSpWZGQ0mzbbtveJDJ15L1UvEBykiS8 yzDBaGFNNoAtO9ze6nmMw81fxLppYPYROrFbjIgJUGBUDf2ilSPkMgPRDi5H8XVSeeMt 4Jyw== X-Forwarded-Encrypted: i=1; AFNElJ9BNxmJaNeMVo0fIBRYZ0eGb3hvsVqAQUglnui3w/ww10wQqhubVzmEWWI8As7WBkPgT+yNttoofDuuooA=@vger.kernel.org X-Gm-Message-State: AOJu0YxUACRKjmuY2kKW/KJ9TpncDsO246Qwj9vtFb59OriqSiV+U+UL vopaLfxFL129Kf5RWrarQMLiTll/PBTjm0MOssvBkq4+obA4kxNjIZHLyVWFCVQ9oQ== X-Gm-Gg: AfdE7cnNbTKRlvuNKMuldneVQwWT3irRhstqfg1Znn+YWopiiioweumCzpgxkjQsnst di1HPXSnhc19xKN0w1V5qrkXKnLc+0fgG0r1d9gvdhNc+Ek3kIz0zyyILhjbF+ANaKy+aJ2ElSx /1VikJXpeCM6tujp/DLMy+EOTzhIsTDjqqE8uH5GwZZHZx+K64lsB7ERnkoCa1N49ylRIaiVqry eF59ldVdj1rhujLSV7BlwsarLyJOAdlI45wtXqyqkBHVtenUVI/NVhUYssmCdJFLMVdv8nl0vfI /cM4bmXymckKGx9iosm8XQY32pmNdha33V4aJDH/pFXhZlfLijqLjHbrCTsfhVYW3Ya4KuI3Xy0 ouiLtKRTdcv/czWaPbc4yPtYMKzlT5miwTR6pH3E9WEcLooHjTUE0EnX+Mta5tPSwiIKcemrldv oReHoSfmxUtWkXpXwQ1G9JGIyjcH1Vj+4qRIMxYCiCGCPAxJCL92jlVlHKpomjzEikR3DgxwhMy mKjKVFnRVRcOmW6mqCTwFbvmHY= X-Received: by 2002:a17:902:fc47:b0:2c2:8659:da43 with SMTP id d9443c01a7336-2c6bbf7cdeemr15372715ad.4.1781661733790; Tue, 16 Jun 2026 19:02:13 -0700 (PDT) Received: from atimofey-ld1.linkedin.biz ([20.29.181.177]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c6a3ee1052sm25075775ad.48.2026.06.16.19.02.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2026 19:02:13 -0700 (PDT) From: Alex Timofeyev To: Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Cc: Parav Pandit , Edward Srouji , Vlad Dumitrescu , stable@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH rdma-next v1 0/2] RDMA: fix cross-NIC same-host IPv6 RDMA-CM connect Date: Wed, 17 Jun 2026 02:02:12 +0000 Message-ID: <1781661732.reply1-sashka@ankey.net> In-Reply-To: <1781545579.1-sashka@ankey.net> References: <1781545579.1-sashka@ankey.net> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 > You need to setup policy routing or a VRF so these local routes don't > show up the way they do. We shouldn't be mangling the loopback routes > in the kernel, and removing the check is not correct. You're right, and thanks for pushing back on this. I've been testing VRF all day and it solves the problem completely, with no kernel changes. Putting each NIC in its own VRF moves that NIC's addresses out of the global v6 local table (255), so the cross-NIC same-host destination no longer collapses to lo -- it resolves on-wire to the real egress device: # enp49s0np0 -> vrfdata0 (table 100), enp193s0np0 -> vrfdata1 (101) $ ip -6 route get from oif enp49s0np0 ... dev enp49s0np0 ... # was: local ... dev lo So neither addr_resolve_neigh() nor validate_ipv6_net_dev() ever hits the local shortcut my patches were rewriting. The check stays correct and the loopback routes stay untouched, exactly as you said. I validated it with the actual workload that motivated the series, not just rping: a 3-node DAOS cluster (two engines per host, one per NUMA NIC) on a kernel WITHOUT these two patches -- only the upstream prereqs fa29d1e8877b + c31e4038c97f. All 6 ranks join and stay joined, and a pool created across all of them comes up healthy. That covers both the same-host cross-NIC path and the cross-host path, all over v6 RoCEv2. One observation in case it's useful to others who hit this: VRF self-RPC relies on c31e4038c97f -- once an address is homed in a VRF table, is_dst_local() needs the RTF_LOCAL test rather than the old IFF_LOOPBACK one to still recognize it as local. That commit is in v6.18 but not in linux-6.6.y, which is the stable base we (and presumably other RoCE-on-6.6 users) are on; might be worth a stable backport on its own merits, independent of this series. Given all that, please drop this series -- I'm withdrawing it. The right fix here is configuration, not a kernel patch. Appreciate the review and the steer toward VRF. Thanks, Alex