From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=pass header.oldest-pass=0 smtp.remote-ip=52.101.85.48 ARC-Seal: i=2; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=pass; t=1794336023; b=U2853URGvv+PgA77ufkAQdz0KKQSCo0aPxdmHOXBkw8NttfZp8InVmIbUYKpojjpNNp0 1V53uowloJBHuUC88mu51feKPx4pbIuQFJUrcYZ2Y8dDDcaxh42WZQQR5SkdD6K7la5Xk Nz/f45vzjk+bW0TEkVwWe/Y5ZUFAP9qxF8ANxO4vTSpBGHyOd62fCgKf9hb6nPZ2oNGqm tDVyVxT03NLbJboHv8hXjNsPsIGFIidok8bjZKGcolXCIMZX1tgL70Lh6emoFGAqdln7D ePu49Fsvk9SsRf9b1L6oOjEHE21b1+caoshwkVcEVNFpK5qN2eup7TRYUfigAeQDy/A== ARC-Message-Signature: i=2; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794336023; h=DMARC-Filter:DKIM-Signature:Received:Received: X-MS-Exchange-Authentication-Results:Received-SPF:Received:Received: Received:Received:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:Content-Transfer-Encoding: Content-Type:X-NV-OnPremToCloud:X-EOPAttributedMessage: X-MS-PublicTrafficType:X-MS-TrafficTypeDiagnostic: X-MS-Office365-Filtering-Correlation-Id:X-MS-Exchange-SenderADCheck: X-MS-Exchange-AntiSpam-Relay:X-Microsoft-Antispam: X-Microsoft-Antispam-Message-Info:X-Forefront-Antispam-Report: X-MS-Exchange-AntiSpam-MessageData-ChunkCount: X-MS-Exchange-AntiSpam-MessageData-0:X-OriginatorOrg: X-MS-Exchange-CrossTenant-OriginalArrivalTime: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-Id: X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: X-MS-Exchange-CrossTenant-AuthSource:X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-FromEntityHeader: X-MS-Exchange-Transport-CrossTenantHeadersStamped; bh=5vS+TKwrys6rMPpTfW3rmo+B3J++Piccg2OJKaXMB0k=; b=WtMzukCu2RECyG1SOzOAWOFDYMxcCn2bssK8ZCvBjbBer008b0N3y2wyTTNjp1T0Dmhw HSW8CF31776UcqMLYkkYPGH9zbvALBNJNXv7VBTVS6237ORGNCBwhZH21DHox30HShKKV u4JPD+b3ggwhyS56R0Q8b7mWV9ZeF3HDHAuG6uvtvbRBRsTJJrNTrHwAq29m10ZIaPFj5 WfKbyJ38kfXv59tULQHTkJhPkqqAOyj6cRtyNRP5BLb44vTozo9/liIw3ppjYpul/MjfO qQFVtOVpaGmetEEEo1GuP8ifx1auOE/9kRqDUdOnaHN6VikT7jR3w2XJmoCAuez1MOQ== ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b="f1/J6YBT"; arc=pass header.oldest-pass=0 smtp.remote-ip=52.101.85.48 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010048.outbound.protection.outlook.com [52.101.85.48]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (secp384r1) server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22678606A904; Tue, 10 Nov 2026 18:40:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 22678606A904 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=f1/J6YBT ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MPUU/bQq1U3owVVDpgeSPioCOO0/TFg1hRBaYI9RrBj4Cjm0klUhFx+XqNzV0VFhH5Geh1wH7gWy3HsFY9a0MCoOAP9Dc5BUK+7VLyqOHvVyGhFBzDbpx04EwjuSLPdJOEXwEV0X0X2mC3BXAsuRIyJUNWaEY8MAiuvOi042KS/GupdNj/exHGrUJJ5LymPA+g3rhIJTwY9ihxR8CoDnQcQphxMWk6MR9rgskOnrAjKClddhQ+avQetiL74+BUgvDBv/2WPR3p2h8DQ4QMBg3U9TYGYauuRi/RZEiesWGkQILFPPta2DPu/ytlgaJE/5o9VziuMwQUynu74IAvdiQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5vS+TKwrys6rMPpTfW3rmo+B3J++Piccg2OJKaXMB0k=; b=uaEWGf0+oV5TgRsy6qj584YdSufEIQToYg7KrjPPx5cccJoc8TDAxJJUr6xeObXhrBKDVdwZcIOKKsYXjL/xH8stlLpITDi5KTazh+nN6bxLUdBn5JPXE1NNlEf1ccLMi5IsaleFkxE/JK84rLkVals5i6TrDzrfkaNRJGi1HPhOnKb9V7A3HOTOv7xATS1qPYo71/ncZwHsxLv/tgp1bmeAeHTQvoVKhaQM69D/3rzuntHsdVuNNkI1MBJ+7i54JBwimjJkq7fFJXRKWzeDr3sm0WYhLI0NfTG7k118ipGIU0Y9+35kE1dCT/FvC8XDqNabKAOZb4sZ01kaJFj5sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=lunn.ch smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5vS+TKwrys6rMPpTfW3rmo+B3J++Piccg2OJKaXMB0k=; b=f1/J6YBTPxc093MNqpYukkkRoOmeXUPOEXaeczvX5tpenrukaf6Txe/YyfUYqJwoQmcOfWhq1Z04tJFQaWeJVR3R8Kku6FsK0FXcaA3S6alBpOee4RW1mudHDx/8ed/MfGEAr/1r8p7ol9w+xvD7mXi1mkPY4eg69w1COuGZa4nyvqHpe3K3pRfMyuTk2k3Ph6RJTd1JPDD7OwCBrRTBlUoLX8juyCJGAkT4JXO9toqfOVrIkWw3QGWyrdQy+jUcQwAaAQVgFJKJZ1sZZagkG5/fGhsRzPU9+oocE6wuxqB61pisxEMM2Btlhhb5Wy1Xi97xNz8S7M8yBKhBQSNBhw== Received: from PH8P222CA0022.NAMP222.PROD.OUTLOOK.COM (2603:10b6:510:2d7::33) by BN7PPF2E18BD747.namprd12.prod.outlook.com (2603:10b6:40f:fc02::6ca) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.13; Tue, 10 Nov 2026 18:40:18 +0000 Received: from PH2PEPF00048915.namprd05.prod.outlook.com (2603:10b6:510:2d7:cafe::60) by PH8P222CA0022.outlook.office365.com (2603:10b6:510:2d7::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.8 via Frontend Transport; Tue, 10 Nov 2026 18:40:18 +0000 X-MS-Exchange-Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by PH2PEPF00048915.mail.protection.outlook.com (10.167.243.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.615.4 via Frontend Transport; Tue, 10 Nov 2026 18:40:18 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.56; Tue, 10 Nov 2026 10:39:58 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.56; Tue, 10 Nov 2026 10:39:58 -0800 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.2562.56 via Frontend Transport; Tue, 10 Nov 2026 10:39:50 -0800 From: Tariq Toukan To: Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , , Paolo Abeni CC: Bobby Eshleman , Carolina Jubran , Cosmin Ratiu , Daniel Borkmann , David Wei , Dragos Tatulea , Erfan Sharafzadeh , Gal Pressman , Jacob Keller , Kaifeng Wang , Leon Romanovsky , open list , , Mark Bloch , Matt Fleming , Mina Almasry , Nikolay Aleksandrov , "Saeed Mahameed" , Simon Horman , "Stanislav Fomichev" , Stanislav Fomichev , "Tariq Toukan" , William Tu Subject: [PATCH net-next V2 09/13] net/mlx5e: Use the correct DMA dev when data_direct is enabled Date: Tue, 10 Nov 2026 20:38:22 +0200 Message-ID: <20261110183826.2927274-10-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20261110183826.2927274-1-tariqt@nvidia.com> References: <20261110183826.2927274-1-tariqt@nvidia.com> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH2PEPF00048915:EE_|BN7PPF2E18BD747:EE_ X-MS-Office365-Filtering-Correlation-Id: dcae876e-08c0-49d5-8d24-08df3f5a1193 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|23010399003|82310400026|1800799024|36860700016|11063799006|261022105027099003|261021135927099003|261021121427099003|261026140227099003|56012099006|261021123027099003|261026121527099003|261028144727099003|261009223027099003|261102123227099003|10067099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: IVuFbLQ6tOBsE/PmlT5oHNSoM2LaXqYEF7Yg3vG4ZIIU2EDUD61s/keB8XFPDlpZNcoZEzESsL4Ps0MeMaLv7AVHDbf++Sh3Lls+2UeQGGx6kDx3Xt3xkfIbQ403/K0V7P+1mUd11/IQjADDBkHmtAbAAD1nGxQYIedUqRZP3EfTqBBaGpq1astIBQuodalpcev1p4R/CY+eRsQmPvyFMoHgW3MLzHNF4IxWTDwBFv9WY0+PzaM0glW+yuAFaz3xJo3/QaawhOl+ahO81FLv/Tbb/njQ3hLMYhRXpye4GNFu3SKHTDvIKR+9V65CWvyZJ6CIskgEoSQcvT0WaHBK41TC1zDoR/G3Suz8Kjrw/h/dMHblU5Le4saEzjQjHC6qf9IDvgjzwA4/eN7FSOGjAPQkY+Ace25H9BAS2DGMxjabusbTSP+YF8mypab65ioZ6SddxXowRzcnqJaPD0uEdAHz/oa/+Mxf5rddCiqwLJoLZziNuyC/04JAGSrzS5kwArNPRAz7jFSoMavMs0pTOdwqLsMEplZsg8GK1ulJPXumZdYR8bbRmOU24P6Fdk4oJaHvgMmrjwP+BQUNX7hzTEn2voRcx8C8EKc8bY2hxlhfuGxiy0ZeoRvqQi2Hi8c1Vme5Hpxq8TFHGg36de9C0OCx41fGQDYPRFFl8a5LPje53XnO3z9HLajoyJ8ST1Ks6Mm4wt1Hz9xmeuYcpoCr6g== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(376014)(23010399003)(82310400026)(1800799024)(36860700016)(11063799006)(261022105027099003)(261021135927099003)(261021121427099003)(261026140227099003)(56012099006)(261021123027099003)(261026121527099003)(261028144727099003)(261009223027099003)(261102123227099003)(10067099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DtwjbSizCjq7Iv7ssqAC0Qmgl4756SnwMlGapjsJK8d5lqhu2MyWoi0B/BdXtSz18cOjQl/VFBke+jIKguPIUJ6r2t+I9fwZzR6Qdpy4DPoX1KaQTMQI2CI869yAaa4vJcm6p2Ec6mb75/zYudrMJ8pfGwXuphltDaCMOEMRyyptiMpjZyHg2Es+GeLANzvMhEkOvc5HU7O2pn2CXxdw5Wzp7lWsr7e0IA6f5Jqd3PdfEv2HeIW+EAJVKV9kxlqz7IzWNSLqadmTwPj4hlGvPPggBE8hqe9QXbkQqSrjdWr5wMfBp/IiB9qtd9m7eLwCryUWxYlntHpnSYtZ5cFZt9kHbVK8mMu7FbgfSComQURwfJHR9KCH4Fuxuay5cTXASA8c7/rn+Y2gi/9FdSdzV8s0B6wiFcYUy36ym5hOlu3FdAMhJzjnta26EIyhAbHP X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2026 18:40:18.3417 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dcae876e-08c0-49d5-8d24-08df3f5a1193 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: PH2PEPF00048915.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PPF2E18BD747 From: Dragos Tatulea When the data_direct private flag is enabled, return the data direct device instead of the NIC PF PCI device in .ndo_queue_get_dma_dev. The memory provider maps the payload buffers against that device, so the page_pool of a data direct RQ has to use it as well. Add mlx5e_get_dma_dev() which returns the DMA device of an mlx5_core_dev for either mode and use it in both places. Running .ndo_queue_get_dma_dev while the data direct device is going away returns NULL because the device pointer is already cleared. Opening a data direct RQ fails with -ENODEV for the same reason. Leased queues are not supported in data-direct mode because there is currently no mechanism to revoke leased queues from the leaser. Rename the dev argument to netdev because there are too many confusing "dev"-based names in this function. Signed-off-by: Dragos Tatulea Reviewed-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 49 ++++++++++++++++--- 1 file changed, 42 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 5b99099e4ed0..ebd41f24330d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -962,6 +962,19 @@ static void mlx5e_rq_free_shampo(struct mlx5e_rq *rq) kvfree(shampo); } +static struct device *mlx5e_get_dma_dev(struct mlx5_core_dev *mdev, + bool data_direct) +{ + struct mlx5_data_direct_dev *dd_dev; + + if (!data_direct) + return mlx5_core_dma_dev(mdev); + + dd_dev = mlx5_data_direct_get_dev(mdev->data_direct); + + return dd_dev ? dd_dev->device : NULL; +} + static int mlx5e_alloc_rq(struct mlx5e_params *params, struct mlx5e_rq_param *rq_param, struct mlx5e_rq_opt_param *rqo, @@ -1105,7 +1118,11 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, pp_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV; pp_params.pool_size = pool_size; pp_params.nid = node; - pp_params.dev = rq->pdev; + pp_params.dev = mlx5e_get_dma_dev(mdev, data_direct); + if (!pp_params.dev) { + err = -ENODEV; + goto err_free_by_rq_type; + } pp_params.napi = rq->cq.napi; pp_params.netdev = rq->netdev; pp_params.dma_dir = rq->buff.map_dir; @@ -5897,27 +5914,45 @@ static int mlx5e_queue_start(struct net_device *dev, return 0; } -static struct device *mlx5e_queue_get_dma_dev(struct net_device *dev, +static bool mlx5e_has_leased_rxqs(struct net_device *netdev) +{ + for (u32 i = 0; i < netdev->real_num_rx_queues; i++) + if (READ_ONCE(__netif_get_rx_queue(netdev, i)->lease)) + return true; + + return false; +} + +static struct device *mlx5e_queue_get_dma_dev(struct net_device *netdev, int queue_index) { - struct mlx5e_priv *priv = netdev_priv(dev); + struct mlx5e_priv *priv = netdev_priv(netdev); struct mlx5e_channels *channels; - struct device *pdev = NULL; + struct device *dma_dev = NULL; struct mlx5e_channel *ch; + bool dd; channels = &priv->channels; mutex_lock(&priv->state_lock); - if (queue_index >= channels->num) + /* Allow only opened channels. */ + if (queue_index < 0 || queue_index >= channels->num) + goto out; + + dd = MLX5E_GET_PFLAG(&channels->params, MLX5E_PFLAG_DATA_DIRECT); + if (dd && mlx5e_has_leased_rxqs(netdev)) { + mlx5_core_err_rl(priv->mdev, + "Can't get queue DMA device: leased RX queues are not supported in data-direct mode\n"); goto out; + } ch = channels->c[queue_index]; - pdev = ch->pdev; + dma_dev = mlx5e_get_dma_dev(ch->mdev, dd); out: mutex_unlock(&priv->state_lock); - return pdev; + return dma_dev; } static const struct netdev_queue_mgmt_ops mlx5e_queue_mgmt_ops = { -- 2.44.0