Re: [PATCH net-next 01/15] net: dsa: mt7530: always trap frames to active CPU port on MT7530

["Arınç ÜNAL" <arinc.unal@arinc9.com>]

On 7.12.2023 20:48, Vladimir Oltean wrote:
> On Sat, Dec 02, 2023 at 11:29:18AM +0300, Arınç ÜNAL wrote:
>>>>    struct mt7530_priv {
>>>>    	struct device		*dev;
>>>> @@ -786,6 +787,7 @@ struct mt7530_priv {
>>>>    	struct irq_domain *irq_domain;
>>>>    	u32 irq_enable;
>>>>    	int (*create_sgmii)(struct mt7530_priv *priv, bool dual_sgmii);
>>>> +	unsigned long active_cpu_ports;
>>>
>>> So this will be 32 or 64 bit in size. Presumably you know how many CPU
>>> ports there can be, which looking at this code must be less than 8 as
>>> CPU_PORT_MASK is only 3 bits in size. So, maybe use a u8, and check
>>> that cpu_dp->index <= 7 ?
> 
> We picked "unsigned long" as storage because that's also the size of the
> argument that __ffs() takes. But admittedly, we could have also stored a
> smaller variable and promote it to unsigned long when we pass it to __ffs().
> 
>> Aren't there other mechanisms to check that cpu_dp->index is a valid port?
> 
> cpu_dp->index is guaranteed by DSA to be valid (according to the "reg"
> value from the device tree and smaller than ds->num_ports). It's just a
> question of balancing this kind of optimization with the possibility
> that a future switch appears which has more than MT7530_NUM_PORTS (7) ports.

If this was to happen - as unlikely as I find it - I would suggest adding a
new MTXXXX_NUM_PORTS and set priv->ds->num_ports to it after checking
priv->id. I'm a maintainer here so I'll keep an eye out.

> 
>>
>> 	mt7530_rmw(priv, MT7530_MFC, CPU_EN | CPU_PORT_MASK, val);
>> }
>>
>> struct mt7530_priv {
>> 	[...]
>> 	u8 active_cpu_ports;
>> };
> 
> Actually, looking at the code now, I don't understand why we even keep
> track of the active_cpu_ports mask in the driver. We could read the
> MT7530_MFC register in mt753x_conduit_state_change(), flip the bit
> corresponding just to cpu_dp->index (rather than rmw all of CPU_PORT_MASK),
> and write back the result. And to address Russell's concern, we could test
> whether the resulting CPU_PORT_MASK portion of what we're going to write
> back is all-zeroes or not, and if it is, clear the CPU_EN bit, otherwise
> set it.

Correct me if I'm wrong, we introduced priv->active_cpu_ports because
CPU_PORT_MASK from the MT7530_MFC register is a 3-bit mask, meant to keep a
single port number ranging from 0 to 7. There's no single bit corresponding
to cpu_dp->index on the MT7530_MFC register. So I don't see how what you
suggest here would work.

This is what I've got right now:

static void
mt753x_conduit_state_change(struct dsa_switch *ds,
			    const struct net_device *conduit,
			    bool operational)
{
	struct dsa_port *cpu_dp = conduit->dsa_ptr;
	struct mt7530_priv *priv = ds->priv;
	u8 mask;
	int val = 0;

	/* Set the CPU port to trap frames to for MT7530. Trapped frames will be
	 * forwarded to the numerically smallest CPU port which the DSA conduit
	 * interface its affine to is up.
	 */
	if (priv->id != ID_MT7530 && priv->id != ID_MT7621)
		return;

	mask = BIT(cpu_dp->index);

	if (operational)
		priv->active_cpu_ports |= mask;
	else
		priv->active_cpu_ports &= ~mask;

	if (priv->active_cpu_ports)
		val =
		    CPU_EN |
		    CPU_PORT(__ffs((unsigned long)priv->active_cpu_ports));

	mt7530_rmw(priv, MT7530_MFC, CPU_EN | CPU_PORT_MASK, val);
}

> 
>>
>>>
>>> I would also suggest moving irq_enable after create_sgmii, to avoid
>>> holes in the struct.
>>
>> Sorry, I've got no idea about this. Could you explain why would there
>> possibly be holes in the struct with the current ordering of the members of
>> the mt7530_priv structure?
>>
>> Arınç
> 
> FWIW:
> 
> $ pahole -C mt7530_priv $KBUILD_OUTPUT/drivers/net/dsa/mt7530.o
> struct mt7530_priv {
>          struct device *            dev;                  /*     0     8 */
>          struct dsa_switch *        ds;                   /*     8     8 */
>          struct mii_bus *           bus;                  /*    16     8 */
>          struct regmap *            regmap;               /*    24     8 */
>          struct reset_control *     rstc;                 /*    32     8 */
>          struct regulator *         core_pwr;             /*    40     8 */
>          struct regulator *         io_pwr;               /*    48     8 */
>          struct gpio_desc *         reset;                /*    56     8 */
>          /* --- cacheline 1 boundary (64 bytes) --- */
>          const struct mt753x_info  * info;                /*    64     8 */
>          unsigned int               id;                   /*    72     4 */
>          bool                       mcm;                  /*    76     1 */
> 
>          /* XXX 3 bytes hole, try to pack */
> 
>          phy_interface_t            p6_interface;         /*    80     4 */
>          phy_interface_t            p5_interface;         /*    84     4 */
>          unsigned int               p5_intf_sel;          /*    88     4 */
>          u8                         mirror_rx;            /*    92     1 */
>          u8                         mirror_tx;            /*    93     1 */
> 
>          /* XXX 2 bytes hole, try to pack */
> 
>          struct mt7530_port         ports[7];             /*    96   168 */
>          /* --- cacheline 4 boundary (256 bytes) was 8 bytes ago --- */
>          struct mt753x_pcs          pcs[7];               /*   264   280 */
>          /* --- cacheline 8 boundary (512 bytes) was 32 bytes ago --- */
>          struct mutex               reg_mutex;            /*   544    32 */
>          /* --- cacheline 9 boundary (576 bytes) --- */
>          int                        irq;                  /*   576     4 */
> 
>          /* XXX 4 bytes hole, try to pack */
> 
>          struct irq_domain *        irq_domain;           /*   584     8 */
>          u32                        irq_enable;           /*   592     4 */
> 
>          /* XXX 4 bytes hole, try to pack */
> 
>          int                        (*create_sgmii)(struct mt7530_priv *, bool); /*   600     8 */
>          unsigned long              active_cpu_ports;     /*   608     8 */
> 
>          /* size: 616, cachelines: 10, members: 24 */
>          /* sum members: 603, holes: 4, sum holes: 13 */
>          /* last cacheline: 40 bytes */
> };
> 
> It's not like this makes any practical difference, as struct mt7530_priv
> isn't used from hot paths, but tidying it up is a good sign of clean,
> careful development, and of understanding memory alignment.

Got it, thanks. I've got a few patches that introduce changes to the
mt7530_priv structure. I'll make sure that, in the end, mt7530_priv won't
have any holes.

Arınç