From: Thomas Gleixner <tglx@linutronix.de>
To: Yu Chen <yu.c.chen@intel.com>
Cc: x86@kernel.org, Ingo Molnar <mingo@redhat.com>,
"H. Peter Anvin" <hpa@zytor.com>, Rui Zhang <rui.zhang@intel.com>,
LKML <linux-kernel@vger.kernel.org>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>,
Len Brown <lenb@kernel.org>,
Dan Williams <dan.j.williams@intel.com>,
Christoph Hellwig <hch@lst.de>,
Peter Zijlstra <peterz@infradead.org>,
Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Subject: Re: [PATCH 4/4][RFC v2] x86/apic: Spread the vectors by choosing the idlest CPU
Date: Thu, 7 Sep 2017 07:54:09 +0200 (CEST) [thread overview]
Message-ID: <alpine.DEB.2.20.1709070731110.2433@nanos> (raw)
In-Reply-To: <20170907025212.GA18130@localhost.localdomain>
On Thu, 7 Sep 2017, Yu Chen wrote:
> On Wed, Sep 06, 2017 at 10:03:58AM +0200, Thomas Gleixner wrote:
> > Can you please apply the debug patch below, boot the machine and right
> > after login provide the output of
> >
> > # cat /sys/kernel/debug/tracing/trace
> >
> kworker/0:2-303 [000] .... 9.135467: msi_domain_alloc_irqs: dev: 0000:bb:00.0 nvec 1 virq 34
> kworker/0:2-303 [000] .... 9.135476: msi_domain_alloc_irqs: dev: 0000:bb:00.0 nvec 1 virq 35
> kworker/0:2-303 [000] .... 9.135484: msi_domain_alloc_irqs: dev: 0000:bb:00.0 nvec 1 virq 36
<SNIP>
> kworker/0:2-303 [000] .... 9.762268: msi_domain_alloc_irqs: dev: 0000:bb:00.3 nvec 1 virq 331
> kworker/0:2-303 [000] .... 9.762278: msi_domain_alloc_irqs: dev: 0000:bb:00.3 nvec 1 virq 332
> kworker/0:2-303 [000] .... 9.762288: msi_domain_alloc_irqs: dev: 0000:bb:00.3 nvec 1 virq 333
That's 300 vectors.
> bb:00.[0-3] Ethernet controller: Intel Corporation Device 37d0 (rev 03)
>
> -+-[0000:b2]-+-00.0-[b3-bc]----00.0-[b4-bc]--+-00.0-[b5-b6]----00.0
> | | +-01.0-[b7-b8]----00.0
> | | +-02.0-[b9-ba]----00.0
> | | \-03.0-[bb-bc]--+-00.0
> | | +-00.1
> | | +-00.2
> | | \-00.3
>
> and they are using i40e driver, the vectors should be reserved by:
> i40e_probe() ->
> i40e_init_interrupt_scheme() ->
> i40e_init_msix() ->
> i40e_reserve_msix_vectors() ->
> pci_enable_msix_range()
>
> # ls /sys/kernel/debug/irq/irqs
> 0 10 11 13 142 184 217 259 292 31 33
> 337 339 340 342 344 346 348 350 352 354 356
> 358 360 362 364 366 368 370 372 374 376 378
> 380 382 384 386 388 390 392 394 4 6 7 9
> 1 109 12 14 15 2 24 26 3 32 335
> 338 34 341 343 345 347 349 351 353 355 357
> 359 361 363 365 367 369 371 373 375 377 379
> 381 383 385 387 389 391 393 395 5 67 8
Out of these 300 interrupts exactly 8 randomly selected ones are actively
used. And the other 292 interrupts are just there because it might need
them in the future when the 32 CPU machine gets magically upgraded to 4096
cores at runtime?
Can the i40e people @intel please fix this waste of resources and sanitize
their interrupt allocation scheme?
Please switch it over to managed interrupts so the affinity spreading
happens in a sane way and the interrupts are properly managed on CPU
hotplug.
Thanks,
tglx
next prev parent reply other threads:[~2017-09-07 5:54 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-01 5:03 [PATCH 0/4][RFC v2] x86/irq: Spread vectors on different CPUs Chen Yu
2017-09-01 5:03 ` [PATCH 1/4][RFC v2] x86/apic: Extend the defination for vector_irq Chen Yu
2017-09-01 5:04 ` [PATCH 2/4][RFC v2] x86/apic: Record the number of vectors assigned on a CPU Chen Yu
2017-09-01 5:04 ` [PATCH 3/4] x86/apic: Introduce the per vector cpumask array Chen Yu
2017-09-01 5:04 ` [PATCH 4/4][RFC v2] x86/apic: Spread the vectors by choosing the idlest CPU Chen Yu
2017-09-03 18:17 ` Thomas Gleixner
2017-09-03 19:18 ` RFD: x86: Sanitize the vector allocator Thomas Gleixner
2017-09-05 22:57 ` [PATCH 4/4][RFC v2] x86/apic: Spread the vectors by choosing the idlest CPU Thomas Gleixner
2017-09-06 4:34 ` Yu Chen
2017-09-06 8:03 ` Thomas Gleixner
2017-09-07 2:52 ` Yu Chen
2017-09-07 5:54 ` Thomas Gleixner [this message]
2017-09-07 8:34 ` Yu Chen
2017-09-07 9:45 ` Thomas Gleixner
2017-09-06 4:13 ` Yu Chen
2017-09-06 6:15 ` Christoph Hellwig
2017-09-06 17:46 ` Dan Williams
2017-09-07 2:57 ` Yu Chen
2017-09-07 5:59 ` Thomas Gleixner
2017-09-07 6:23 ` Dan Williams
2017-09-07 6:59 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.20.1709070731110.2433@nanos \
--to=tglx@linutronix.de \
--cc=dan.j.williams@intel.com \
--cc=hch@lst.de \
--cc=hpa@zytor.com \
--cc=jeffrey.t.kirsher@intel.com \
--cc=lenb@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=rui.zhang@intel.com \
--cc=x86@kernel.org \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®