Re: [PATCH] PCI/IOV: update num_VFs earlier

[Jakub Kicinski <jakub.kicinski@netronome.com>]

On Thu, 3 Oct 2019 17:10:07 -0500, Bjorn Helgaas wrote:
> On Thu, Oct 03, 2019 at 11:04:45AM +0200, CREGUT Pierre IMT/OLN wrote:
> > Le 02/10/2019 à 01:45, Bjorn Helgaas a écrit :  
> > > On Fri, Apr 26, 2019 at 10:11:54AM +0200, CREGUT Pierre IMT/OLN wrote:  
> > > > I also initially thought that kobject_uevent generated the netlink event
> > > > but this is not the case. This is generated by the specific driver in use.
> > > > For the Intel i40e driver, this is the call to i40e_do_reset_safe in
> > > > i40e_pci_sriov_configure that sends the event.
> > > > It is followed by i40e_pci_sriov_enable that calls i40e_alloc_vfs that
> > > > finally calls the generic pci_enable_sriov function.  
> > > I don't know anything about netlink.  The script from the bugzilla
> > > (https://bugzilla.kernel.org/show_bug.cgi?id=202991) looks like it
> > > runs
> > > 
> > >    ip monitor dev enp9s0f2
> > > 
> > > What are the actual netlink events you see?  Are they related to a
> > > device being removed?  
> > 
> > We have netlink events both when num_vfs goes from 0 to N and from N to 0.
> > Indeed you have to go to 0 before going to M with M != N.  
> 
> Right.

FWIW I think this netlink event is an artefact of i40e implementation,
and is not something the networking stack generates. Hopefully Alex can
correct me if I'm wrong, but I don't think most drivers will generate
such an event.

> commit 0940fc95da45
> Author: Pierre Crégut <pierre.cregut@orange.com>
> Date:   Wed Sep 11 09:27:36 2019 +0200
> 
>     PCI/IOV: Serialize sysfs sriov_numvfs reads vs writes
>     
>     When sriov_numvfs is being updated, drivers may notify about new devices
>     before they are reflected in sriov->num_VFs, so concurrent sysfs reads
>     previously returned stale values.
>     
>     Serialize the sysfs read vs the write so the read returns the correct
>     num_VFs value.
>     
>     Link: https://bugzilla.kernel.org/show_bug.cgi?id=202991
>     Link: https://lore.kernel.org/r/20190911072736.32091-1-pierre.cregut@orange.com
>     Signed-off-by: Pierre Crégut <pierre.cregut@orange.com>
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> 
> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
> index b3f972e8cfed..e77562aabbae 100644
> --- a/drivers/pci/iov.c
> +++ b/drivers/pci/iov.c
> @@ -254,8 +254,14 @@ static ssize_t sriov_numvfs_show(struct device *dev,
>  				 char *buf)
>  {
>  	struct pci_dev *pdev = to_pci_dev(dev);
> +	u16 num_vfs;
> +
> +	/* Serialize vs sriov_numvfs_store() so readers see valid num_VFs */
> +	device_lock(&pdev->dev);
> +	num_vfs = pdev->sriov->num_VFs;
> +	device_lock(&pdev->dev);
>  
> -	return sprintf(buf, "%u\n", pdev->sriov->num_VFs);
> +	return sprintf(buf, "%u\n", num_vfs);
>  }
>  
>  /*

The change makes sense to me!