Re: [PATCH net-next v2 3/5] amd-xgbe: add support for new XPCS routines

["Rangoju, Raju" <raju.rangoju@amd.com>]

On 4/30/2025 1:07 AM, Simon Horman wrote:
> On Mon, Apr 28, 2025 at 08:32:33PM +0530, Raju Rangoju wrote:
>> Add the necessary support to enable Crater ethernet device. Since the
>> BAR1 address cannot be used to access the XPCS registers on Crater, use
>> the smn functions.
>>
>> Some of the ethernet add-in-cards have dual PHY but share a single MDIO
>> line (between the ports). In such cases, link inconsistencies are
>> noticed during the heavy traffic and during reboot stress tests. Using
>> smn calls helps avoid such race conditions.
>>
>> Suggested-by: Sudheesh Mavila <sudheesh.mavila@amd.com>
>> Signed-off-by: Raju Rangoju <Raju.Rangoju@amd.com>
>> ---
>> - PCI config accesses can race with other drivers performing SMN accesses
>>    so, fall back to AMD SMN API to avoid race.
>>
>>   drivers/net/ethernet/amd/xgbe/xgbe-dev.c | 81 ++++++++++++++++++++++++
>>   drivers/net/ethernet/amd/xgbe/xgbe-smn.h | 30 +++++++++
>>   drivers/net/ethernet/amd/xgbe/xgbe.h     |  6 ++
>>   3 files changed, 117 insertions(+)
>>   create mode 100644 drivers/net/ethernet/amd/xgbe/xgbe-smn.h
>>
>> diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
>> index 765f20b24722..5f367922e705 100644
>> --- a/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
>> +++ b/drivers/net/ethernet/amd/xgbe/xgbe-dev.c
>> @@ -14,6 +14,7 @@
> 
> Hi Raju,
> 
> I think you need the following about here:
> 
> #include <linux/pci.h>
> 
> To make sure that pci_err(), which is used elsewhere in this patch,
> is always defined. Building allmodconfig for arm and arm64 shows
> that, without this change, pci_err is not defined.

Hi Simon,

Thanks for pointing it out. Sure, will address it in v3.

> 
> Alternatively, perhaps netdev_err can be used instead of pci_err().
> 
>>   
>>   #include "xgbe.h"
>>   #include "xgbe-common.h"
>> +#include "xgbe-smn.h"
>>   
>>   static inline unsigned int xgbe_get_max_frame(struct xgbe_prv_data *pdata)
>>   {
>> @@ -1066,6 +1067,80 @@ static void xgbe_get_pcs_index_and_offset(struct xgbe_prv_data *pdata,
>>   	*offset = pdata->xpcs_window + (mmd_address & pdata->xpcs_window_mask);
>>   }
>>   
>> +static int xgbe_read_mmd_regs_v3(struct xgbe_prv_data *pdata, int prtad,
>> +				 int mmd_reg)
>> +{
>> +	unsigned int mmd_address, index, offset;
>> +	int mmd_data;
>> +	int ret;
>> +
>> +	mmd_address = xgbe_get_mmd_address(pdata, mmd_reg);
>> +
>> +	xgbe_get_pcs_index_and_offset(pdata, mmd_address, &index, &offset);
>> +
>> +	ret = amd_smn_write(0, (pdata->smn_base + pdata->xpcs_window_sel_reg), index);
> 
> nit: Please line wrap to 80 columns wide or less, as is still preferred for
>       Networking code.  Likewise elsewhere in this patch.
> 
>       Flagged by checkpatch.pl --max-line-length=80
> 
>       Also, the inner parentheses seem to be unnecessary.

Sure, will address it in v3.

> 
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = amd_smn_read(0, pdata->smn_base + offset, &mmd_data);
>> +	if (ret)
>> +		return ret;
>> +
>> +	mmd_data = (offset % 4) ? FIELD_GET(XGBE_GEN_HI_MASK, mmd_data) :
>> +				  FIELD_GET(XGBE_GEN_LO_MASK, mmd_data);
>> +
>> +	return mmd_data;
>> +}
>> +
>> +static void xgbe_write_mmd_regs_v3(struct xgbe_prv_data *pdata, int prtad,
>> +				   int mmd_reg, int mmd_data)
>> +{
>> +	unsigned int pci_mmd_data, hi_mask, lo_mask;
>> +	unsigned int mmd_address, index, offset;
>> +	struct pci_dev *dev;
>> +	int ret;
>> +
>> +	dev = pdata->pcidev;
>> +	mmd_address = xgbe_get_mmd_address(pdata, mmd_reg);
>> +
>> +	xgbe_get_pcs_index_and_offset(pdata, mmd_address, &index, &offset);
>> +
>> +	ret = amd_smn_write(0, (pdata->smn_base + pdata->xpcs_window_sel_reg), index);
>> +	if (ret) {
>> +		pci_err(dev, "Failed to write data 0x%x\n", index);
>> +		return;
>> +	}
>> +
>> +	ret = amd_smn_read(0, pdata->smn_base + offset, &pci_mmd_data);
>> +	if (ret) {
>> +		pci_err(dev, "Failed to read data\n");
>> +		return;
>> +	}
>> +
>> +	if (offset % 4) {
>> +		hi_mask = FIELD_PREP(XGBE_GEN_HI_MASK, mmd_data);
>> +		lo_mask = FIELD_GET(XGBE_GEN_LO_MASK, pci_mmd_data);
>> +	} else {
>> +		hi_mask = FIELD_PREP(XGBE_GEN_HI_MASK,
>> +				     FIELD_GET(XGBE_GEN_HI_MASK, pci_mmd_data));
>> +		lo_mask = FIELD_GET(XGBE_GEN_LO_MASK, mmd_data);
>> +	}
>> +
>> +	pci_mmd_data = hi_mask | lo_mask;
>> +
>> +	ret = amd_smn_write(0, (pdata->smn_base + pdata->xpcs_window_sel_reg), index);
>> +	if (ret) {
>> +		pci_err(dev, "Failed to write data 0x%x\n", index);
>> +		return;
>> +	}
>> +
>> +	ret = amd_smn_write(0, (pdata->smn_base + offset), pci_mmd_data);
>> +	if (ret) {
>> +		pci_err(dev, "Failed to write data 0x%x\n", pci_mmd_data);
>> +		return;
>> +	}
>> +}
> 
> ...
> 
>> diff --git a/drivers/net/ethernet/amd/xgbe/xgbe-smn.h b/drivers/net/ethernet/amd/xgbe/xgbe-smn.h
>> new file mode 100644
>> index 000000000000..a1763aa648bd
>> --- /dev/null
>> +++ b/drivers/net/ethernet/amd/xgbe/xgbe-smn.h
>> @@ -0,0 +1,30 @@
>> +// SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-3-Clause)
> 
> Checkpatch says this should be:
> 
> /* SPDX-License-Identifier: (GPL-2.0-or-later OR BSD-3-Clause) */

Sure, will address it in v3.

> 
> 
>> +/*
>> + * Copyright (c) 2014-2025, Advanced Micro Devices, Inc.
>> + * Copyright (c) 2014, Synopsys, Inc.
>> + * All rights reserved
>> + *
>> + * Author: Raju Rangoju <Raju.Rangoju@amd.com>
>> + */
>> +
>> +#ifndef __SMN_H__
>> +#define __SMN_H__
>> +
>> +#ifdef CONFIG_AMD_NB
>> +
>> +#include <asm/amd_nb.h>
>> +
>> +#else
>> +
>> +static inline int amd_smn_write(u16 node, u32 address, u32 value)
>> +{
>> +	return -ENODEV;
>> +}
>> +
>> +static inline int amd_smn_read(u16 node, u32 address, u32 *value)
>> +{
>> +	return -ENODEV;
>> +}
>> +
>> +#endif
>> +#endif
> 
> It feels a little odd to provide these dummy implementation here,
> rather than where the real implementations are declared. But I do
> see where you are coming from with this approach. And I guess it
> is fine so long as this is the only user of this mechanism.

Sure, thank you.

> 
> ...