From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62423189905 for ; Fri, 3 Jan 2025 03:05:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735873544; cv=none; b=FNrT5Tqq1rLD8zLY9U5VIE0AgHAodCcMJjBF+LYb/KDuJ+J4LhBWyhJpkBJPcWXCkBb9WZFB7plUT+UXKs2aLaIENps3S92oG26+N7ZAKkObyDtF9oQwPH5zeMhCmag/CcPOpmmW7oc2NK3pZup+vUp9LqjKWupXDq+kUIW4rFU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735873544; c=relaxed/simple; bh=D8ACTmySjx2qCb2gwARSd7OOJA1snbqkFycABMMJvCk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=sXLQ+dolPi+cwzSa3M+TwfLfssZbzXEJ85VEbJR3+uqGHZE8TaMQFOhNkicAdl1IAMm577zPJnyXd6u+tmPXzcG6TismW/QPihc9txNN2NTcLzesabeJN0sqDX0HESWn0cvogZCQu+ltV2I9ZEqW+JjmPX5QAtOVPE4LRGFmi/g= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com; spf=pass smtp.mailfrom=shopee.com; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b=LZIrt48n; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shopee.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b="LZIrt48n" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-216395e151bso119307895ad.0 for ; Thu, 02 Jan 2025 19:05:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shopee.com; s=shopee.com; t=1735873541; x=1736478341; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=3wBt5uTEWitMwxCv7dEqy1zPrPTtgCQI6rxOHe/aejk=; b=LZIrt48nWWQhPZwcdW25FKeftcqqhR0jY7Qv0PKEREuT/ZCWhU6M5dN99sEhQ6/1jj Z9AAgZ44ZJoTUlZpW0NeuiVTflIwtTF/yeK2Zi4lybNFUuyiKWYceYN5LFkVaUCoyRRK bhu7BGwP/zYYowJHvluhZn9rpXGwFmp3koWKX5L+hy1cnhI8JShgDU6OQyzGIzC6Dw2X nAJHEZL55vFRMeI4hR+9pEU1KY30555vdBXJNycATDCYU+etIXhdGfW/uU54UvsBZbJx cVwmQ7V8IE9Xw1oCOZMdIito6aHYlfaC/VPq8k1lEpYPqrqSA2+n0qDrF+U4M1z/rvrq mHww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735873541; x=1736478341; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=3wBt5uTEWitMwxCv7dEqy1zPrPTtgCQI6rxOHe/aejk=; b=q7b9vK/jlbQmKcor56BgeGq5pVp+CKW4eu4piGUtWZjLmXsynLV8akM0Cg9+zXVwTa DW9NHzeso21xU/8RoCxKgN8bKvVE8fny94gg15VZZ63lqPwY82OTKz4qa4oV4YvGy4im lT+LOg7msN3WtOGpjP/Ih0R2Qj2WKVd7WlmI5bZjNcWAyHs8pCwTYGVSqaceNzWXQQwM P2aecZBmYzl9vxIYpGnth3BiARLtKkgPLtggRQYIUe/v4QzqZapjfIchgkW0Iv5gAU0+ Rh4mtfsLB0dPILt34wim7aOwQITPFx0qDRFhQE2WGx3eQF0bRqYTNZNMd4WFAmCGm7WG kpvA== X-Forwarded-Encrypted: i=1; AJvYcCWLwDRq6eodraTQFjqPpRii1KC1cybghakjaE63e4V92PNLm4P3qYpPvJu+TXVqXzgBdViMPAP3EV1SnYY=@vger.kernel.org X-Gm-Message-State: AOJu0YwynBw/zwsuyPNJtci95MYB7ZU+RR+XfRhos6saH5V2azSwwho1 9uSqqmOn7rtBlQ/VB3H/F7yAdbKYB2k50aVGm/NDOgMPOFfxWP/Yz8ng97nOBLA= X-Gm-Gg: ASbGncs8qz0AfCPUf3brKOdUXPyWRim7l48a2vXrWO63k7KTrlhAMdmCji1sOhgpDDC dr4dSpdrEZOYNMB8xuhwm3p4gtdVRpbTZuTfSiX60vslGFwV9GWZaDs67DaLDpYyDnQD/jRvsE4 vT2IeluZgJ9T0vtF7q6x+E8QamkstXZaSnR9pzPjNWXjFtj4dqdWkStQL3q593E/pSBGZR7/HgP sFHa655xaFxBAnthEdU9JU++HaJn0OkiBzfRl4Rct/fuzx4+wIEfd3JXdVuhG+Kktdizg== X-Google-Smtp-Source: AGHT+IEIrhZaPpLfiwgQfu7MPj7/kD5r+srTbB+HLQv3XgPmHgWQdWvO/uvPNzXwevlemGkbvdnaLA== X-Received: by 2002:a17:902:f689:b0:216:39fa:5c97 with SMTP id d9443c01a7336-219da5ce0b4mr800763245ad.11.1735873541620; Thu, 02 Jan 2025 19:05:41 -0800 (PST) Received: from [10.54.24.59] ([143.92.118.3]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f4fe30f525sm2877782a91.12.2025.01.02.19.05.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 02 Jan 2025 19:05:41 -0800 (PST) Message-ID: <0d98fed8-38e3-4118-82c9-26cefeb5ee7a@shopee.com> Date: Fri, 3 Jan 2025 11:05:36 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: =?UTF-8?Q?Re=3A_=5BQuestion=5D_ixgbe=EF=BC=9AMechanism_of_RSS?= To: Edward Cree , Eric Dumazet Cc: Tony Nguyen , Przemek Kitszel , "David S. Miller" , Jakub Kicinski , Paolo Abeni , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, intel-wired-lan@lists.osuosl.org References: <87e945f6-2811-0ddb-1666-06accd126efb@gmail.com> From: Haifeng Xu In-Reply-To: <87e945f6-2811-0ddb-1666-06accd126efb@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 2025/1/3 00:01, Edward Cree wrote: > On 02/01/2025 11:23, Haifeng Xu wrote: >> We want to make full use of cpu resources to receive packets. So >> we enable 63 rx queues. But we found the rate of interrupt growth >> on cpu 0~15 is faster than other cpus(almost twice). > ... >> I am confused that why ixgbe NIC can dispatch the packets >> to the rx queues that not specified in RSS configuration. > > Hypothesis: it isn't doing so, RX is only happening on cpus (and > queues) 0-15, but the other CPUs are still sending traffic and > thus getting TX completion interrupts from their TX queues. > `ethtool -S` output has per-queue traffic stats which should > confirm this. > I use ethtool -S to check the rx_queus stats and here is the result. According to the below stats, all cpus have new packets received. cpu t1(bytes) t2(bytes) delta(bytes) 0 154155550267550 154156433828875 883561325 1 148748566285840 148749509346247 943060407 2 148874911191685 148875798038140 886846455 3 152483460327704 152484251468998 791141294 4 147790981836915 147791775847804 794010889 5 146047892285722 146048778285682 885999960 6 142880516825921 142881213804363 696978442 7 152016735168735 152017707542774 972374039 8 146019936404393 146020739070311 802665918 9 147448522715540 147449258018186 735302646 10 145865736299432 145866601503106 865203674 11 149548527982122 149549289026453 761044331 12 146848384328236 146849303547769 919219533 13 152942139118542 152942769029253 629910711 14 150884661854828 150885556866976 895012148 15 149222733506734 149223510491115 776984381 16 34150226069524 34150375855113 149785589 17 34115700500819 34115914271025 213770206 18 33906215129998 33906448044501 232914503 19 33983812095357 33983986258546 174163189 20 34156349675011 34156565159083 215484072 21 33574293379024 33574490695725 197316701 22 33438129453422 33438297911151 168457729 23 32967454521585 32967612494711 157973126 24 33507443427266 33507604828468 161401202 25 33413275870121 33413433901940 158031819 26 33852322542796 33852527061150 204518354 27 33131162685385 33131330621474 167936089 28 33407661780251 33407823112381 161332130 29 34256799173845 34256944837757 145663912 30 33814458585183 33814623673528 165088345 31 33848638714862 33848775218038 136503176 32 18683932398308 18684069540891 137142583 33 19454524281229 19454647908293 123627064 34 19717744365436 19717900618222 156252786 35 20295086765202 20295245869666 159104464 36 20501853066588 20502000738936 147672348 37 20954631043374 20954797204375 166161001 38 21102911073326 21103062510369 151437043 39 21376404644179 21376515307288 110663109 40 20935812784743 20935983891491 171106748 41 20721278456831 20721435955715 157498884 42 21268291801465 21268425244578 133443113 43 21661413672829 21661629019091 215346262 44 21696437732484 21696568800049 131067565 45 21027869000890 21028020401214 151400324 46 21707137252644 21707293761990 156509346 47 20655623913790 20655740452889 116539099 48 32692002128477 32692138244468 136115991 49 33548445851486 33548569927672 124076186 50 33197264968787 33197448645817 183677030 51 33379544010500 33379746565576 202555076 52 33503579011721 33503722596159 143584438 53 33145734550468 33145892305819 157755351 54 33422692741858 33422844156764 151414906 55 32750945531107 32751131302251 185771144 56 33404955373530 33405157766253 202392723 57 33701185654471 33701313174725 127520254 58 33014531699810 33014700058409 168358599 59 32948906758429 32949151147605 244389176 60 33470813725985 33470993164755 179438770 61 33803771479735 33803971758441 200278706 62 33509751180818 33509926649969 175469151 Thanks! > (But Eric is right that if you _want_ RX to use every CPU you > should just change the indirection table.)