From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-vs1-f42.google.com (mail-vs1-f42.google.com [209.85.217.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 002B42AEE1 for ; Sat, 10 Jan 2026 03:55:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.217.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768017326; cv=none; b=jIF2jDvx7hBwZjeolzui8jzK8190Ccy0SQ3/YKP9h451XB54Aew/O8aI1/YfVBCx+s1Aa6mZqpMOL2SJLOrFkxDIw0TkbG+6be1rpKgPaoHS2/SibmIttS/vUof+dgRgg0w5MvIBntlVsvU6wLidJW6szP29WLor2qQWdhz+yKQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768017326; c=relaxed/simple; bh=zRMaG2UZoaDAk8JWg4Py4arxiWvpQFKHkL9OU1MsMMQ=; h=From:Subject:Date:Message-Id:MIME-Version:Content-Type:To:Cc; b=qzBnOQ11AEbzQh5VJ+PNGJDW+nTq9P+63DijJ9eE3+7cQeTBXWZvUPLfpIOHofpPIVqvP3+f9XlSfVmhNwkW6t1SjjzkIBK24f7ZwkiLMfdTc0qMTZ3OjZOtbgyLN6//Elz+1W2BI9GqUdxxhkb9z+0IzZId3uCDD+eyW18eNGA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eqbA8uR4; arc=none smtp.client-ip=209.85.217.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eqbA8uR4" Received: by mail-vs1-f42.google.com with SMTP id ada2fe7eead31-5ecddf73062so2527838137.1 for ; Fri, 09 Jan 2026 19:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1768017324; x=1768622124; darn=vger.kernel.org; h=cc:to:content-transfer-encoding:mime-version:message-id:date :subject:from:from:to:cc:subject:date:message-id:reply-to; bh=lzuCZ10ab2Iorj0lcSOmaTCThwhF5YTUtr2Jp++0Xa0=; b=eqbA8uR4X+pxaSf45fi7EOaQdrEeKlurh/PVgiIjdzYrMp1+0Ip49zsGZzWVLdb94l 2zE2iXl3lB0aN5G2z+qFtdaeWzDBMI5Fp44CHSA5Zg91VBYh7qm/agfNZJDMUJqC5oG6 DBFW9FwA8Msuh57XGPCVyoK0ZSKmKg071+McV4anElf7xlaCZAwTKjUlrYjxfD9OYrj2 5WcnPNOsDuCXSFn5Y6GKbUIO0N8duyK7WsADACPA9UEfgFLpdNLPEk0bpHvIFRo188fM bU5Ndw8vDPsbv2lkYbBymqlW5LsBQDbj56HripI1PFjpiZQ5+Jcw4TioKfJinhUeMvXD T3kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768017324; x=1768622124; h=cc:to:content-transfer-encoding:mime-version:message-id:date :subject:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lzuCZ10ab2Iorj0lcSOmaTCThwhF5YTUtr2Jp++0Xa0=; b=B1DC//OF3mmYt361/07LrXq7wnn9nW41g2GhnWcDm8VcNN10KX675CZuuMOUr195+K /xx204a2vq9dI5gWfwbuFnEiYqJGOAJ70vKxumOoNqVh7eYp6bC6BEKbVwN6bvyV7Tou 9ZOLrc4AftUcD6u96UXkQFyv7FJOoChklOBvq2v3lfBq1Fu6ApTyapYggIKmlhPJdZMs Np3Qu3AVY+imNk2dD3e5U0hCHv68RJrQ4buniZkFfFuOPdjZRBjK80A6svAHEiutWvLp NQL9eq9b0RUk1fshZ1VYPHeWMYNdwLIX5hNB+zlgi6ycUoYLEOfgrQ+W0VaNcbZrgbrX KHuQ== X-Forwarded-Encrypted: i=1; AJvYcCUxC1BTZ3tbZackbz2JA/sKC7t/iOyeUlZIyRlqIr6J34LxP6FDsH08sB7gVMqQhWnZGj6Unj8J9fnaOjo=@vger.kernel.org X-Gm-Message-State: AOJu0YwlsLGdPT3GGc53FNMS8v8BSzxYyiwAcFI6E4n1hcSjeK4izkIp 3gKlP6Jxcoi1x1qeZvZwal8KJ6iw4hK1+TJy3LpFcVMupTZfQxzO58RDZ5Dhvw== X-Gm-Gg: AY/fxX7WZ1gD8TaSTYG3FAr32zpEM90p5fRri4QpDuG89Sn0tT7N8C6k3ttreSDKzwL TCCdrZrZ1aWTdNUt8IOowgEXQpU3phDQciESVbs6FHWR+jDfYQ9J5Eov/GwuSh7byb82VyrmyJr 3ZwSdnOEvqfUwFULEZpqeUAz8G700sAkkZpZR7Z9Hclx+BZ6mlbN30BXSDEi2wgksH2P4no+i+Q P8oDR+T4hhd55tSt2uOjB/iM1/6KrFaT3uGQy93moUehUjZ69AQxAbBscGpy8wYbrIB/myZ9f+b qV5gfaJd0oULw4ft9UZ3FWRnLAPZSjwBsWFcWJrIw6GLeq98DX9vYZEEsCX+xw7mvh/zk1h3jbu RVniSLDOUWIUvM1bl7j6nmIrHGWx4IzIuiw08PYTYFoffOmHWF2TWos47w32z9HGfrZEl4y7RuG ImxAyVz6kklA== X-Google-Smtp-Source: AGHT+IEz37L2wQQJA4qX8j5KYwrwN9NH1ecV7Azu8uWOr9xF8kdhENvXLiTeJfu5CUWtgau8Fq/JnA== X-Received: by 2002:a53:d005:0:b0:645:5297:3e5d with SMTP id 956f58d0204a3-6470d31648fmr10030592d50.46.1768011539825; Fri, 09 Jan 2026 18:18:59 -0800 (PST) Received: from localhost ([2a03:2880:25ff:42::]) by smtp.gmail.com with ESMTPSA id 00721157ae682-790ae603282sm43469157b3.13.2026.01.09.18.18.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jan 2026 18:18:59 -0800 (PST) From: Bobby Eshleman Subject: [PATCH net-next v9 0/5] net: devmem: improve cpu cost of RX token management Date: Fri, 09 Jan 2026 18:18:14 -0800 Message-Id: <20260109-scratch-bobbyeshleman-devmem-tcp-token-upstream-v9-0-8042930d00d7@meta.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-B4-Tracking: v=1; b=H4sIAOi2YWkC/5XSS2rEMAyA4asEr0dFfttZ9R6lCz+UJrRJhtgNM wxz90Kgbcgua8H3C6EHK7QMVFjbPNhC61CGeWJt4y8NS32YPgiGzNqGCRQanfBQ0hJq6iHOMd6 p9F80hgkyrSONUNMV6vxJE3xfS10ojCC8iMStyholuzTsulA33LbiG5uowkS3yt4vDeuHUuflv q2y8m2+VT2K09WVA0L2ynDjo9YaX0eq4SXN45ZaxY7n/DwvACE5zFbqmJWWB17ueGHO8xIQ0Km oTMzBue7A6z+eo5DneQ0IyqbodKeF9nTgzT/PUZ3nDSBQ8C51KiuMx+PYHc/PP9RqAYGHmJwyV mqVDrz75Q1ytOd5BwheJG+ckVx5s+Ofz+cPGcVR5jEDAAA= X-Change-ID: 20250829-scratch-bobbyeshleman-devmem-tcp-token-upstream-292be174d503 To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Kuniyuki Iwashima , Willem de Bruijn , Neal Cardwell , David Ahern , Mina Almasry , Arnd Bergmann , Jonathan Corbet , Andrew Lunn , Shuah Khan , Donald Hunter Cc: Stanislav Fomichev , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, asml.silence@gmail.com, matttbe@kernel.org, skhawaja@google.com, Bobby Eshleman X-Mailer: b4 0.14.3 This series improves the CPU cost of RX token management by adding an attribute to NETDEV_CMD_BIND_RX that configures sockets using the binding to avoid the xarray allocator and instead use a per-binding niov array and a uref field in niov. Improvement is ~13% cpu util per RX user thread. Using kperf, the following results were observed: Before: Average RX worker idle %: 13.13, flows 4, test runs 11 After: Average RX worker idle %: 26.32, flows 4, test runs 11 Two other approaches were tested, but with no improvement. Namely, 1) using a hashmap for tokens and 2) keeping an xarray of atomic counters but using RCU so that the hotpath could be mostly lockless. Neither of these approaches proved better than the simple array in terms of CPU. The attribute NETDEV_A_DMABUF_AUTORELEASE is added to toggle the optimization. It is an optional attribute and defaults to 0 (i.e., optimization on). Signed-off-by: Bobby Eshleman Changes in v9: - fixed build with NET_DEVMEM=n - fixed bug in rx bindings count logic - Link to v8: https://lore.kernel.org/r/20260107-scratch-bobbyeshleman-devmem-tcp-token-upstream-v8-0-92c968631496@meta.com Changes in v8: - change static branch logic (only set when enabled, otherwise just always revert back to disabled) - fix missing tests - Link to v7: https://lore.kernel.org/r/20251119-scratch-bobbyeshleman-devmem-tcp-token-upstream-v7-0-1abc8467354c@meta.com Changes in v7: - use netlink instead of sockopt (Stan) - restrict system to only one mode, dmabuf bindings can not co-exist with different modes (Stan) - use static branching to enforce single system-wide mode (Stan) - Link to v6: https://lore.kernel.org/r/20251104-scratch-bobbyeshleman-devmem-tcp-token-upstream-v6-0-ea98cf4d40b3@meta.com Changes in v6: - renamed 'net: devmem: use niov array for token management' to refer to optionality of new config - added documentation and tests - make autorelease flag per-socket sockopt instead of binding field / sysctl - many per-patch changes (see Changes sections per-patch) - Link to v5: https://lore.kernel.org/r/20251023-scratch-bobbyeshleman-devmem-tcp-token-upstream-v5-0-47cb85f5259e@meta.com Changes in v5: - add sysctl to opt-out of performance benefit, back to old token release - Link to v4: https://lore.kernel.org/all/20250926-scratch-bobbyeshleman-devmem-tcp-token-upstream-v4-0-39156563c3ea@meta.com Changes in v4: - rebase to net-next - Link to v3: https://lore.kernel.org/r/20250926-scratch-bobbyeshleman-devmem-tcp-token-upstream-v3-0-084b46bda88f@meta.com Changes in v3: - make urefs per-binding instead of per-socket, reducing memory footprint - fallback to cleaning up references in dmabuf unbind if socket leaked tokens - drop ethtool patch - Link to v2: https://lore.kernel.org/r/20250911-scratch-bobbyeshleman-devmem-tcp-token-upstream-v2-0-c80d735bd453@meta.com Changes in v2: - net: ethtool: prevent user from breaking devmem single-binding rule (Mina) - pre-assign niovs in binding->vec for RX case (Mina) - remove WARNs on invalid user input (Mina) - remove extraneous binding ref get (Mina) - remove WARN for changed binding (Mina) - always use GFP_ZERO for binding->vec (Mina) - fix length of alloc for urefs - use atomic_set(, 0) to initialize sk_user_frags.urefs - Link to v1: https://lore.kernel.org/r/20250902-scratch-bobbyeshleman-devmem-tcp-token-upstream-v1-0-d946169b5550@meta.com --- Bobby Eshleman (5): net: devmem: rename tx_vec to vec in dmabuf binding net: devmem: refactor sock_devmem_dontneed for autorelease split net: devmem: implement autorelease token management net: devmem: document NETDEV_A_DMABUF_AUTORELEASE netlink attribute selftests: drv-net: devmem: add autorelease test Documentation/netlink/specs/netdev.yaml | 12 +++ Documentation/networking/devmem.rst | 70 +++++++++++++ include/net/netmem.h | 1 + include/net/sock.h | 7 +- include/uapi/linux/netdev.h | 1 + net/core/devmem.c | 116 ++++++++++++++++++---- net/core/devmem.h | 29 +++++- net/core/netdev-genl-gen.c | 5 +- net/core/netdev-genl.c | 10 +- net/core/sock.c | 103 ++++++++++++++----- net/ipv4/tcp.c | 76 +++++++++++--- net/ipv4/tcp_ipv4.c | 11 +- net/ipv4/tcp_minisocks.c | 3 +- tools/include/uapi/linux/netdev.h | 1 + tools/testing/selftests/drivers/net/hw/devmem.py | 21 +++- tools/testing/selftests/drivers/net/hw/ncdevmem.c | 19 ++-- 16 files changed, 407 insertions(+), 78 deletions(-) --- base-commit: 6ad078fa0ababa8de2a2b39f476d2abd179a3cf6 change-id: 20250829-scratch-bobbyeshleman-devmem-tcp-token-upstream-292be174d503 Best regards, -- Bobby Eshleman