mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Longpeng (Mike, Cloud Infrastructure Service Product Dept.)"  <longpeng2@huawei.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: <bhelgaas@google.com>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <jianjay.zhou@huawei.com>,
	<zhuangshengen@huawei.com>, <arei.gonglei@huawei.com>,
	<yechuan@huawei.com>, <huangzhichao@huawei.com>,
	<xiehong@huawei.com>
Subject: Re: [RFC 0/4] pci/sriov: support VFs dynamic addition
Date: Sun, 13 Nov 2022 21:47:12 +0800	[thread overview]
Message-ID: <0b2202bf-18d3-b288-9605-279208165080@huawei.com> (raw)
In-Reply-To: <Y256ty6xGyUpkFn9@unreal>

Hi leon,

在 2022/11/12 0:39, Leon Romanovsky 写道:
> On Fri, Nov 11, 2022 at 10:27:18PM +0800, Longpeng(Mike) wrote:
>> From: Longpeng <longpeng2@huawei.com>
>>
>> We can enable SRIOV and add VFs by /sys/bus/pci/devices/..../sriov_numvfs, but
>> this operation needs to spend lots of time if there has a large amount of VFs.
>>                                                              
>> For example, if the machine has 10 PFs and 250 VFs per-PF, enable all the VFs
>> concurrently would cost about 200-250ms. However most of them are not need to be
>> used at the moment, so we can enable SRIOV first but add VFs on demand.
> 
> It is unclear what took 200-250ms, is it physical VF creation or bind of
> the driver to these VFs?
>
It is neither. In our test, we already created physical VFs before, so 
we skipped the 100ms waiting when writing PCI_SRIOV_CTRL. And our driver 
only probes PF, it just returns an error if the function is VF.

The hotspot is the sriov_add_vfs (but no driver probe in fact) which is 
a long procedure. Each step costs only a little, but the total cost is 
not acceptable in some time-sensitive cases.

What’s more, the sriov_add_vfs adds the VFs of a PF one by one. So we 
can mostly support 10 concurrent calls if there has 10 PFs.

> If the latter, you can try with sriov_drivers_autoprobe set to true. This is how
> ennoblement of large SR-IOV systems is done.
>  > Also PCI spec declares "VF enable" bit, which is applicable to all VFs
> associated to that PF, see section "9.3.3.3.1 VF Enable".
> 
> Thanks
> 
>>
>> This series introduces two interfaces:
>> 1. sriov_numvfs_no_scan: enable SRIOV without add the VFs.
>> 2. sriov_scan_vf_id: add a specific VF.
>>
>> Longpeng (4):
>>    pci/sriov: extract sriov_numvfs common helper
>>    pci/sriov: add vf_bitmap to mark the vf id allocation
>>    pci/sriov: add sriov_numfs_no_scan interface
>>    pci/sriov: add sriov_scan_vf_id interface
>>
>>   drivers/pci/iov.c | 162 +++++++++++++++++++++++++++++++++++++++++-----
>>   drivers/pci/pci.h |   1 +
>>   2 files changed, 148 insertions(+), 15 deletions(-)
>>
>> -- 
>> 2.23.0
>>
> .

  reply	other threads:[~2022-11-13 13:47 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-11 14:27 Longpeng(Mike)
2022-11-11 14:27 ` [RFC 1/4] pci/sriov: extract sriov_numvfs common helper Longpeng(Mike)
2022-11-11 14:27 ` [RFC 2/4] pci/sriov: add vf_bitmap to mark the vf id allocation Longpeng(Mike)
2022-11-11 14:27 ` [RFC 3/4] pci/sriov: add sriov_numfs_no_scan interface Longpeng(Mike)
2022-11-11 14:27 ` [RFC 4/4] pci/sriov: add sriov_scan_vf_id interface Longpeng(Mike)
2022-11-11 16:39 ` [RFC 0/4] pci/sriov: support VFs dynamic addition Leon Romanovsky
2022-11-13 13:47   ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.) [this message]
2022-11-14  7:04     ` Leon Romanovsky
2022-11-14 12:38       ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-14 13:09         ` Leon Romanovsky
2022-11-14 14:06           ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-14 14:20             ` Leon Romanovsky
2022-11-15  1:38               ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-15  1:50               ` Oliver O'Halloran
2022-11-15  8:32                 ` Leon Romanovsky
2022-11-15  9:36                   ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-15 10:02                     ` Leon Romanovsky
2022-11-15 10:27                       ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-15 12:49                   ` Oliver O'Halloran
2022-11-15  2:06             ` Oliver O'Halloran
2022-11-16  0:52               ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2022-11-11 23:07 ` Bjorn Helgaas
2022-11-13 13:49   ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0b2202bf-18d3-b288-9605-279208165080@huawei.com \
    --to=longpeng2@huawei.com \
    --cc=arei.gonglei@huawei.com \
    --cc=bhelgaas@google.com \
    --cc=huangzhichao@huawei.com \
    --cc=jianjay.zhou@huawei.com \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=xiehong@huawei.com \
    --cc=yechuan@huawei.com \
    --cc=zhuangshengen@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®