Re: [EXTERNAL] Re: [net-next PATCH v10 00/11] Introduce RVU representors

[Jiri Pirko <jiri@resnulli.us>]

Fri, Aug 16, 2024 at 03:36:25PM CEST, gakula@marvell.com wrote:
>
>
>>-----Original Message-----
>>From: Jiri Pirko <jiri@resnulli.us>
>>Sent: Thursday, August 8, 2024 9:17 PM
>>To: Geethasowjanya Akula <gakula@marvell.com>
>>Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org; kuba@kernel.org;
>>davem@davemloft.net; pabeni@redhat.com; edumazet@google.com; Sunil
>>Kovvuri Goutham <sgoutham@marvell.com>; Subbaraya Sundeep Bhatta
>><sbhatta@marvell.com>; Hariprasad Kelam <hkelam@marvell.com>
>>Subject: [EXTERNAL] Re: [net-next PATCH v10 00/11] Introduce RVU
>>representors
>>
>>Mon, Aug 05, 2024 at 03: 18: 04PM CEST, gakula@ marvell. com wrote: >This
>>series adds representor support for each rvu devices. >When switchdev mode
>>is enabled, representor netdev is registered >for each rvu device. In
>>implementation of 
>>Mon, Aug 05, 2024 at 03:18:04PM CEST, gakula@marvell.com wrote:
>>>This series adds representor support for each rvu devices.
>>>When switchdev mode is enabled, representor netdev is registered for
>>>each rvu device. In implementation of representor model, one NIX HW LF
>>>with multiple SQ and RQ is reserved, where each RQ and SQ of the LF are
>>>mapped to a representor. A loopback channel is reserved to support
>>>packet path between representors and VFs.
>>>CN10K silicon supports 2 types of MACs, RPM and SDP. This patch set
>>>adds representor support for both RPM and SDP MAC interfaces.
>>>
>>>- Patch 1: Refactors and exports the shared service functions.
>>>- Patch 2: Implements basic representor driver.
>>>- Patch 3: Add devlink support to create representor netdevs that
>>>  can be used to manage VFs.
>>>- Patch 4: Implements basec netdev_ndo_ops.
>>>- Patch 5: Installs tcam rules to route packets between representor and
>>>	   VFs.
>>>- Patch 6: Enables fetching VF stats via representor interface
>>>- Patch 7: Adds support to sync link state between representors and VFs .
>>>- Patch 8: Enables configuring VF MTU via representor netdevs.
>>>- Patch 9: Add representors for sdp MAC.
>>>- Patch 10: Add devlink port support.
>>>
>>>
>>>Command to create PF/VF representor
>>>#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev VF
>>>representors are created for each VF when switch mode is set switchdev
>>>on representor PCI device
>>>
>>>#devlink dev
>>>pci/0002:01:00.0
>>>pci/0002:02:00.0
>>>pci/0002:1c:00.0
>>
>>What are these 3 instances representing? How many PFs do you have? 3?
>>How many physical ports you have?
>The test setup has 3 PFs one for each physical port.
>
>Below example is for the device pci/0002:1c:00.0.

3 port nic, that sounds odd. Is this something shipped already? Care to
paste a link?


>>
>>
>>>
>>>#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev
>>>
>>># ip link show
>>>	eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>>mode
>>>DEFAULT group default qlen 1000 link/ether 7e:58:2d:b6:97:51 brd
>>>ff:ff:ff:ff:ff:ff
>>
>>What is this eth0? Why isn't it connected to any devlink port?
>>
>>>	r0p1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>>mode DEFAULT group default qlen 1000 link/ether 7e:5a:66:ea:fe:d6 brd
>>ff:ff:ff:ff:ff:ff
>>>	r1p1v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>>mode DEFAULT group default qlen 1000 link/ether de:29:be:10:9e:bf brd
>>ff:ff:ff:ff:ff:ff
>>>	r2p1v2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>>mode DEFAULT group default qlen 1000 link/ether 4a:12:c7:a2:66:ad brd
>>ff:ff:ff:ff:ff:ff
>>>	r3p1v3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>>mode
>>>DEFAULT group default qlen 1000 link/ether c2:b8:a8:0e:73:fd brd
>>>ff:ff:ff:ff:ff:ff
>>>
>>>
>>>~# devlink port
>>>pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller 0
>>>pfnum 1 vfnum 0 external false splittable false
>>>pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf controller 0
>>>pfnum 1 vfnum 1 external false splittable false
>>>pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf controller 0
>>>pfnum 1 vfnum 2 external false splittable false
>>>pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf controller 0
>>>pfnum 1 vfnum 3 external false splittable false
>>
>>You are missing physical port devlink instance here? Where is it?
>pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller 0 pfnum 1 vfnum 0 external false splittable false
>This is for the PF.

That is not a physical port. The flavour should be "physical" for them.


>
>Below is the example on a setup with one PF before  3VFs are created.
>
>#devlink dev eswitch set pci/0002:1c:00.0 mode switchdev
>
># ip link show
>	eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
>DEFAULT group default qlen 1000 link/ether 7e:58:2d:b6:97:51 brd ff:ff:ff:ff:ff:ff
>r0p1v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>mode DEFAULT group default qlen 1000 link/ether 7e:5a:66:ea:fe:d6 brd ff:ff:ff:ff:ff:ff
>r1p1v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>mode DEFAULT group default qlen 1000 link/ether de:29:be:10:9e:bf brd
>ff:ff:ff:ff:ff:ff
>r2p1v2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>mode DEFAULT group default qlen 1000 link/ether 4a:12:c7:a2:66:ad brd ff:ff:ff:ff:ff:ff
>r3p1v3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode

How you setup the netdevice names? Do you have custom udev rule? Would
be great to give here the out-of-box udev names instead.



>DEFAULT group default qlen 1000 link/ether c2:b8:a8:0e:73:fd brd ff:ff:ff:ff:ff:ff
>
>Above shows the PF physical port and 4 representors(1 for PF and 3 for VFs).
># devlink port
>pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller  0 pfnum 1 vfnum 0 external false splittable false
>pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf controller 0 pfnum 1 vfnum 1 external false splittable false
>pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf controller 0 pfnum 1 vfnum 2 external false splittable false
>pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf controller 0 pfnum 1 vfnum 3 external false splittable false
>
>>
>>>
>>>
>>>-----------
>>>v1-v2:
>>> -Fixed build warnings.
>>> -Address review comments provided by "Kalesh Anakkur Purayil".
>>>
>>>v2-v3:
>>> - Used extack for error messages.
>>> - As suggested reworked commit messages.
>>> - Fixed sparse warning.
>>>
>>>v3-v4:
>>> - Patch 2 & 3: Fixed coccinelle reported warnings.
>>> - Patch 10: Added devlink port support.
>>
>>
>>When someone reviews your patchset at some version, you put him to cc list
>>from that point. Why didn't you put me to cc list?
>>
>>
>>>
>>>v4-v5:
>>>  - Patch 3: Removed devm_* usage in rvu_rep_create()
>>>  - Patch 3: Fixed build warnings.
>>>
>>>v5-v6:
>>>  - Addressed review comments provided by "Simon Horman".
>>>  - Added review tag.
>>>
>>>v6-v7:
>>>  - Rebased on top net-next branch.
>>>
>>>v7-v8:
>>>   - Implmented offload stats ndo.
>>>   - Added documentation.
>>>
>>>v8-v9:
>>>   - Updated the documentation.
>>>
>>>v9-v10:
>>>  - Fixed build warning w.r.t documentation.
>>>
>>>Geetha sowjanya (11):
>>>  octeontx2-pf: Refactoring RVU driver
>>>  octeontx2-pf: RVU representor driver
>>>  octeontx2-pf: Create representor netdev
>>>  octeontx2-pf: Add basic net_device_ops
>>>  octeontx2-af: Add packet path between representor and VF
>>>  octeontx2-pf: Get VF stats via representor
>>>  octeontx2-pf: Add support to sync link state between representor and
>>>    VFs
>>>  octeontx2-pf: Configure VF mtu via representor
>>>  octeontx2-pf: Add representors for sdp MAC
>>>  octeontx2-pf: Add devlink port support
>>>  octeontx2-pf: Implement offload stats ndo for representors
>>>
>>> .../ethernet/marvell/octeontx2.rst            |  85 ++
>>> .../net/ethernet/marvell/octeontx2/Kconfig    |   8 +
>>> .../ethernet/marvell/octeontx2/af/Makefile    |   3 +-
>>> .../ethernet/marvell/octeontx2/af/common.h    |   2 +
>>> .../net/ethernet/marvell/octeontx2/af/mbox.h  |  74 ++
>>> .../net/ethernet/marvell/octeontx2/af/npc.h   |   1 +
>>> .../net/ethernet/marvell/octeontx2/af/rvu.c   |  11 +
>>> .../net/ethernet/marvell/octeontx2/af/rvu.h   |  30 +-
>>> .../marvell/octeontx2/af/rvu_debugfs.c        |  27 -
>>> .../marvell/octeontx2/af/rvu_devlink.c        |   6 +
>>> .../ethernet/marvell/octeontx2/af/rvu_nix.c   |  81 +-
>>> .../marvell/octeontx2/af/rvu_npc_fs.c         |   5 +
>>> .../ethernet/marvell/octeontx2/af/rvu_reg.h   |   4 +
>>> .../ethernet/marvell/octeontx2/af/rvu_rep.c   | 464 +++++++++++
>>> .../marvell/octeontx2/af/rvu_struct.h         |  26 +
>>> .../marvell/octeontx2/af/rvu_switch.c         |  20 +-
>>> .../ethernet/marvell/octeontx2/nic/Makefile   |   2 +
>>> .../ethernet/marvell/octeontx2/nic/cn10k.c    |   4 +-
>>> .../ethernet/marvell/octeontx2/nic/cn10k.h    |   2 +-
>>> .../marvell/octeontx2/nic/otx2_common.c       |  58 +-
>>> .../marvell/octeontx2/nic/otx2_common.h       |  84 +-
>>> .../marvell/octeontx2/nic/otx2_devlink.c      |  49 ++
>>> .../ethernet/marvell/octeontx2/nic/otx2_pf.c  | 305 +++++---
>>> .../marvell/octeontx2/nic/otx2_txrx.c         |  38 +-
>>> .../marvell/octeontx2/nic/otx2_txrx.h         |   3 +-
>>> .../ethernet/marvell/octeontx2/nic/otx2_vf.c  |  19 +-
>>> .../net/ethernet/marvell/octeontx2/nic/rep.c  | 725 ++++++++++++++++++
>>> .../net/ethernet/marvell/octeontx2/nic/rep.h  |  53 ++
>>> 28 files changed, 1962 insertions(+), 227 deletions(-) create mode
>>> 100644 drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c
>>> create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/rep.c
>>> create mode 100644 drivers/net/ethernet/marvell/octeontx2/nic/rep.h
>>>
>>>--
>>>2.25.1
>>>
>>>