From: Zhangshaokun <zhangshaokun@hisilicon.com>
To: Eric Dumazet <edumazet@google.com>, Jiri Pirko <jiri@resnulli.us>,
<netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>,
"guoyang (C)" <guoyang2@huawei.com>,
"zhudacai@hisilicon.com" <zhudacai@hisilicon.com>
Subject: [RFC] performance regression with commit-id<adb03115f459> ("net: get rid of an signed integer overflow in ip_idents_reserve()")
Date: Wed, 24 Jul 2019 16:38:30 +0800 [thread overview]
Message-ID: <051e93d4-0206-7416-e639-376b8d2eb98b@hisilicon.com> (raw)
Hi,
I've observed an significant performance regression with the following commit-id <adb03115f459>
("net: get rid of an signed integer overflow in ip_idents_reserve()").
Here are my test scenes:
----Server----
Cmd: iperf3 -s xxx.xxx.xxxx.xxx -p 10000 -i 0 -A 0
Kenel: 4.19.34
Server number: 32
Port: 10000 – 10032
CPU affinity: 0 – 32
CPU architecture: aarch64
NUMA node0 CPU(s): 0-23
NUMA node1 CPU(s): 24-47
----Client----
Cmd: iperf3 -u -c xxx.xxx.xxxx.xxx -p 10000 -l 16 -b 0 -t 0 -i 0 -A 8
Kenel: 4.19.34
Client number: 32
Port: 10000 – 10032
CPU affinity: 0 – 32
CPU architecture: aarch64
NUMA node0 CPU(s): 0-23
NUMA node1 CPU(s): 24-47
Firstly, With patch <adb03115f459> ("net: get rid of an signed integer overflow in ip_idents_reserve()") ,
client’s cpu is 100%, and function ip_idents_reserve() cpu usage is very high, but the result is not good.
03:08:32 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil
03:08:33 AM eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
03:08:33 AM eth1 0.00 3461296.00 0.00 196049.97 0.00 0.00 0.00 0.00
03:08:33 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Secondly, revert that patch, use atomic_add_return() instead, the result is better, as below:
03:23:24 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil
03:23:25 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
03:23:25 AM eth1 0.00 12834590.00 0.00 726959.20 0.00 0.00 0.00 0.00
03:23:25 AM eth0 7.00 11.00 0.40 2.95 0.00 0.00 0.00 0.00
Thirdly, atomic is not used in ip_idents_reserve() completely ,while each cpu core allocates its own ID segment,
Such as: cpu core0 allocate ID 0 – 1023, cpu core1 allocate 1024 – 2047, …,etc
the result is the best:
03:27:06 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil
03:27:07 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
03:27:07 AM eth1 0.00 14275505.00 0.00 808573.53 0.00 0.00 0.00 0.00
03:27:07 AM eth0 0.00 2.00 0.00 0.18 0.00 0.00 0.00 0.00
Because atomic operation performance is bottleneck when cpu core number increase, Can we revert the patch or
use ID segment for each cpu core instead?
Thanks in advance,
Shaokun
next reply other threads:[~2019-07-24 8:38 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-24 8:38 Zhangshaokun [this message]
2019-07-24 8:56 ` Eric Dumazet
2019-07-25 9:05 ` Zhangshaokun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=051e93d4-0206-7416-e639-376b8d2eb98b@hisilicon.com \
--to=zhangshaokun@hisilicon.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=guoyang2@huawei.com \
--cc=jiri@resnulli.us \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=zhudacai@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®