From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E79B7EE4993 for ; Wed, 23 Aug 2023 10:15:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233896AbjHWKPa (ORCPT ); Wed, 23 Aug 2023 06:15:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233168AbjHWKPE (ORCPT ); Wed, 23 Aug 2023 06:15:04 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E12EE10D8; Wed, 23 Aug 2023 03:14:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692785680; x=1724321680; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=dfEtHrBfC4p4m5N8XxOYBKNnCtC5HwP7nhhkR1YE7dY=; b=mBptPcIb2FlJT4tpQv6fWkueBtPSb/bPsMmobc5hQ7nWXZlS8keVScRA jE/FEeNTg5xXH6Tcj/N2v5AOpl6H7CpUOlKg1U4nnk8P2SM1bAdpxY7ce gGb7UrZHYTCN2o9+OizbR7RbmHxjNWoctaBHfIg+B+QkuCCYG7KcHQpfA My7zVFhJXwj7r5Bl0t7/KNhaK9Cs2Er5Ky+guiWzoehrv2gOYNOX6Do6D 8zPojd01UOaWQseclhKujNnDjtp9aTQkk+3B0UePRZhhxheyp/D6ihEou KzMVDu/b9/QD0+nSRzZkqZ7AvuwQj9RBnXHD3rW8kQkLSzeJxUeN/NYUd g==; X-IronPort-AV: E=McAfee;i="6600,9927,10810"; a="364298086" X-IronPort-AV: E=Sophos;i="6.01,195,1684825200"; d="scan'208";a="364298086" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Aug 2023 03:14:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10810"; a="983261330" X-IronPort-AV: E=Sophos;i="6.01,195,1684825200"; d="scan'208";a="983261330" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga006.fm.intel.com with ESMTP; 23 Aug 2023 03:14:38 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 23 Aug 2023 03:14:38 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 23 Aug 2023 03:14:38 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Wed, 23 Aug 2023 03:14:38 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.40) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Wed, 23 Aug 2023 03:14:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PndP6UXqzxHxMej8kpW/m+nuFl0m3wOs9/d3mXDAJbvEBhADrARFUrjEhrUg7dfpQZ9LM6jhANH6JKc5zRUjd9KEEA9hFxd4xwys3S9Nc3CNJnPpKtSjsVrsJflEYu+jkTdesiPNpLjBV2SsHCYNqkYgBNcV9Zi4CFeQyz+Ntr3TI6JAs1S8BKzd1dcfpVfFaI1ENiUD0gVblXmpCnPiPfO9WK0d1+Kpcy8VNR2MIFZ6dMtoKz8L++enXF2AUp/yIw4whXtBsEoNh/sTBjv8ju+DGULUCU/nKxR/FWbcLAeYtALZBc+hhh/Nnlc7R1m5uiF/lJpjZnH92yZ3P/nu5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xvhh+JCvn7TgqJee3evFVXBOiKxgf8dh2DZdXwIU1v0=; b=UqVUm0uwrAkjHxi0/KCbzS+qWfl5Qaa8R2OV55/RFu5TO0JIGC4pMHG4kiFsOqceuanyaSiEi66t3jDZenNYuUfnYhH7u/LmBrKHAtuxIO41hTpNzExQIjBPek1SZdPlyCV5iYZgQfNvfu/7wSQeukVgCL9OXbRaNubb3TLcjoKdh9ufeup6sSNnK/7SeGR71YMDD5cPREO/FQ23Mlgg/CACKpLNJBSf8MRMrG9mWA0zTCZuh2qKhTbZAY+gjGyR6KjOVTCjXXxGUub7F44UZRxuyGhvuLCX09FcfzEXoGZ4BEb44r4y9Nnm09L6uoljUkTdaQHZNdNVkP4B6omRNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM6PR11MB3625.namprd11.prod.outlook.com (2603:10b6:5:13a::21) by CY5PR11MB6185.namprd11.prod.outlook.com (2603:10b6:930:27::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.26; Wed, 23 Aug 2023 10:14:36 +0000 Received: from DM6PR11MB3625.namprd11.prod.outlook.com ([fe80::44ff:6a5:9aa4:124a]) by DM6PR11MB3625.namprd11.prod.outlook.com ([fe80::44ff:6a5:9aa4:124a%7]) with mapi id 15.20.6678.031; Wed, 23 Aug 2023 10:14:35 +0000 Message-ID: <8241b4dd-5d31-3744-9da9-937e43931e35@intel.com> Date: Wed, 23 Aug 2023 12:12:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v2 net] octeontx2-pf: fix page_pool creation fail for rings > 32k Content-Language: en-US To: Ratheesh Kannoth CC: , , , , , , , , , , , , , References: <20230823025325.2499289-1-rkannoth@marvell.com> From: Alexander Lobakin In-Reply-To: <20230823025325.2499289-1-rkannoth@marvell.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: DU7PR01CA0025.eurprd01.prod.exchangelabs.com (2603:10a6:10:50e::15) To DM6PR11MB3625.namprd11.prod.outlook.com (2603:10b6:5:13a::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR11MB3625:EE_|CY5PR11MB6185:EE_ X-MS-Office365-Filtering-Correlation-Id: b7d3ea87-c1c7-4de6-9105-08dba3c1c06d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2mMpXIStJV+WYPqTu62YGNFG5ekmTjE91tzDNnRzBkfgpIDJcxqUwNWLiy9LwtHR5a6jEXAjJBQQHNcQTIUR9nd1vIVxVbW+nL/s3uLjXV7Q+/L96N8PGoibp8slqw0yHfVWUKTnDrFAY6j4PDxQQepxPoBX4RLIt1iFupjcstDI1Nlc9Dki5ZAHni6naX+MQ8/B5/Mj3uoFOJLEh6KGI7hBexNgtrzK3ci9lcbwvZwPtSRFw7AoCCDPs5g5WXWYl/MqUNNT8vj8lQd/ep1PdlvaRT2IrWUWpHlxpYy+jDBGJaqbn8WSxi5zEpanNivqiR7rtiqYbcicGd4Poatyf7/ve3boz+5OgUr6LahZ1VTORXkmcTCSi6SKBm2zspE4MuPeYAyk/ECHD6OULoI6bj97d6++nW++eU7OX8iTypAap+mYHrtnK0Epa35S7UVmCm0yzmfSgJuibbXOIExmVW5DMvmwI/FPIVXiv0I+3i6DhDq35c3V5eQE0zBQJ5VEOp4YhehQLnb/Nf0JNnXzhKebFg3EMAgZO0SzHZso8TtlB4wwBFHqe1TIsgkjhVW6KT+LiKKRhe6a5CZQd4Q7IQFmbBVvY/0RuxCzCMGOyZtx8u1XR8PtP96bWADSLsGWdUoimbvmemSZH8hBO1fC6A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR11MB3625.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(136003)(396003)(366004)(39860400002)(376002)(346002)(186009)(1800799009)(451199024)(6506007)(6486002)(6666004)(6512007)(83380400001)(86362001)(31696002)(38100700002)(82960400001)(36756003)(26005)(2616005)(2906002)(316002)(66946007)(66476007)(66556008)(6916009)(41300700001)(5660300002)(7416002)(31686004)(4326008)(8676002)(8936002)(478600001)(43740500002)(45980500001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SSswbUg2UlJIL3RZK2huaHFBS1pZMXQ2eGRQYzUzclJiMVVzOTJ3NnlFM3hz?= =?utf-8?B?YmZtYjEzUUVXUzBSd3U0dHR6Vk5hYjROV05Ba1hMd2laTHVPbXBMQldIWTBR?= =?utf-8?B?MmpJSDdGL3NaMk4vU0wzK0plMlR3dzFhK3VveDA5eEswdGVsL05MclpkZDRL?= =?utf-8?B?UWtwRzc5Tkk1c2NRbU5oeHN5N2R6UUVBSCtHTkVlRHhNSUJiUkljdzV0TTdE?= =?utf-8?B?YWJNbzhDMXZlQ3JvMk0yR2FFMHIvMW9uNEdIZi8walhGTUMzbGJBaW8yQVRp?= =?utf-8?B?QUVsczQ5TEJvdDJROVVsUWQ5Wm4yeFhRNTAwUU9XbE1UckRnNnBkK2czUWpW?= =?utf-8?B?ejVxTUZlZXJMVlVmc0RVamhKZHRSQjFtcE1lQ0hOeGRxb2ZNdXJlYnpyQmhG?= =?utf-8?B?Ykt4dm5rZ1FCK0wyTTJ1d3ZidDJmMkJxN1p5aDhQZngxdVQ4MDQ5eTIrdmtk?= =?utf-8?B?UVllWTJhWTBEcEpqZEZKVjdCYUVnQUdpbVpBZUh1UlRYTWVpT0ZHN0hrL1JZ?= =?utf-8?B?K1BUQ1ZQVHY0VVJBcEw5amVoT3J6alM1Z0lDRkp4bDM3SHVqUzVZQVNZcjBh?= =?utf-8?B?bDlUdmlXQ3BTbVdSeWovbmxLYk1BRjhQTFkxbXh2cWJaQllkS0hFSjhUOTNs?= =?utf-8?B?MGZqblBPRXA3RjltbXdyREM2V0g0TmZxNWpMd3VEdlVUdzJnM213NEJ5M0Jh?= =?utf-8?B?WFVpbVlISmFkMXlDSnpTb3FNYzE3NWdMTFVrdlBkMjdRcEVhcmFPcE9IVFBH?= =?utf-8?B?WExBaEN1RFFGSFl4SGpoUHQvSHUrMlMzWm9hTCtLbHRlMEdNWW1NRW5ycmN5?= =?utf-8?B?MUc4UTNTZUNnMjdjYXdLdldTNXBtbFdBdGlNUHF6WXhRZ2lpM1hQUTZDRFJ6?= =?utf-8?B?Yi9ZNnRiblpGTkVMTzJHdEEvMG9XYmFGTUFjS0J3eklJYzFVaXEwc2p4WXJ3?= =?utf-8?B?YUFIZmlwYkNRd1hYYU9CSmRRSDkrendidUZjcnpydmFzUmoydVBmUlhTTVJC?= =?utf-8?B?UjRxNkxhdGszdEpMVEdaaGYwaExLQnNzeWo2Z3FtTVE1Q01MM0h3TTZrTlYr?= =?utf-8?B?TU5ESWRQSVphQkpKcGxHMkc1VHBRckFYNDlpZ3lTRlRRbk1yemZnZlErRFFv?= =?utf-8?B?NVhLL2t4LzA4Nmp4bWU5QXRVNmZtYlZUMlB4bWxBU3h1ZjZOODcxUVJML2t1?= =?utf-8?B?ZGJCZWVGYVM4YWZNRTFmeW5xTElGdnlLVEpmZjdlWmdydVN5VGg2SUplRm9j?= =?utf-8?B?eCtkNTA2cFFNSldZQ28xVkY2QnM5UWlIY1o4MkM5Wjk0R2NmekxueXFHS1J2?= =?utf-8?B?ZTBhN3lmbGFuTFJlK2YyeURuQlJqc2RXR3lGNnNZUlFPM1FPQkIyZW80c2Zj?= =?utf-8?B?eXhTOHBzcWdxUDk1RURxSmY3MlV2dkFhdXJrRUowN05QazhTYVZRd0FBMHY3?= =?utf-8?B?d21jaHFEaXRueU5IRDZpTXIzWWZBc2hubld5OUx1TkpGRW9JUXR4MDV3TCtE?= =?utf-8?B?Y2lLMWorRlNrM0R6bnpyS2lab3JrcFo1c1RGY0pOaHNXd2Z6U3NTWS9vbHJs?= =?utf-8?B?RVh1QnZpL3VOQkxOTTltMVJSSnY3MzVPT0Y0Wm5leGw4elpVa1NHZE9CdXl2?= =?utf-8?B?dGlBeHRpeTQ5bWJNR3JybmIvMmdHdkZuNEpXZUVKNzNuZUtxU0d4K2I0SnJV?= =?utf-8?B?L3BYLzBhLzJITnBDZ1ZHSWZEcVdDR3hUdUF0SmtweVJvcm5lTFczNmFsTFRI?= =?utf-8?B?bG9vRm9iblFvQ3NrQXovUTVRL0tDNFJJQ3gwbDVoU2lNZVlWUVBvVFFYYWxt?= =?utf-8?B?ek8zNU5GWUxnYjdjSTVKQkdYVlpkVDlwQk93aWd4aEhDclVzTG9HVEdkdVBo?= =?utf-8?B?alQyK21iT1pFdUlXSzJnQnFpZFN1d2ppMWdsK0p6UEJwbjVSeVgvVWlhM3hR?= =?utf-8?B?aDBlbDgzK1VRbXdaNVIrUWxkNWJqYXVKbGtteE1rdURRaXZJeUpGb3RmTFNv?= =?utf-8?B?K1BXM0Yzc2F2WjVmOXJjK3VJb1ZuaVI5K2EzSEtPSncrclhQWm9uN3VBTmVz?= =?utf-8?B?VXR1c0NNTytEdERCd2Z6RjRwN0F0bGNrcC85SlowVDU3emd0K0VnVm9aM0dJ?= =?utf-8?B?ZXlrVW1XUlExanNxdWF3UDlNdnRjRmEyeUtjNUQyUERrb2Uxd3Z3VWtScFp6?= =?utf-8?B?eGc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: b7d3ea87-c1c7-4de6-9105-08dba3c1c06d X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3625.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Aug 2023 10:14:35.8449 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zWUvbITCEV4wdNvUfnOLgW5pU+6u6jJTzpJnBamjy9LgsFSomjV3dO57vj1ks1kbyTXo5RBU06vSQFY0zOFp0L3JcgCYewcclu6LTUvDxsI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6185 X-OriginatorOrg: intel.com Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ratheesh Kannoth Date: Wed, 23 Aug 2023 08:23:25 +0530 > octeontx2 driver calls page_pool_create() during driver probe() > and fails if queue size > 32k. Page pool infra uses these buffers > as shock absorbers for burst traffic. These pages are pinned down > over time as working sets varies, due to the recycling nature > of page pool, given page pool (currently) don't have a shrinker > mechanism, the pages remain pinned down in ptr_ring. > Instead of clamping page_pool size to 32k at > most, limit it even more to 2k to avoid wasting memory. > > This have been tested on octeontx2 CN10KA hardware. > TCP and UDP tests using iperf shows no performance regressions. See now? I told ya you don't need that many to provide good recycling :> But nice to see it was confirmed 2k is enough. > > Fixes: b2e3406a38f0 ("octeontx2-pf: Add support for page pool") > Suggested-by: Alexander Lobakin > Reviewed-by: Sunil Goutham > Signed-off-by: Ratheesh Kannoth > --- > > ChangeLogs: > > v1->v2: Commit message changes and typo fixes > v0->v1: Commit message changes. > --- > drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c | 2 +- > drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h | 2 ++ > 2 files changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c > index 77c8f650f7ac..3e1c70c74622 100644 > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c > @@ -1432,7 +1432,7 @@ int otx2_pool_init(struct otx2_nic *pfvf, u16 pool_id, > } > > pp_params.flags = PP_FLAG_PAGE_FRAG | PP_FLAG_DMA_MAP; > - pp_params.pool_size = numptrs; > + pp_params.pool_size = min(OTX2_PAGE_POOL_SZ, numptrs); > pp_params.nid = NUMA_NO_NODE; > pp_params.dev = pfvf->dev; > pp_params.dma_dir = DMA_FROM_DEVICE; > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h > index ba8091131ec0..f6fea43617ff 100644 > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h Isn't otx2_txrx.h a better place for this definition? For example, I'd place it somewhere between CGX_CHAN_BASE and OTX2_DATA_ALIGN. Or after MAX_XDP_MTU, doesn't matter much. > @@ -30,6 +30,8 @@ > #include > #include "qos.h" > > +#define OTX2_PAGE_POOL_SZ 2048 Don't forget about alignment. Otx2 code has very inconsistent alignment of definitions -- just one space in some places and proper tab alignment in others. I still believe we prefer the second for netdev code. > + > /* IPv4 flag more fragment bit */ > #define IPV4_FLAG_MORE 0x20 > Thanks, Olek