[PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Marek Vasut]

R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
Max_Payload_Size (MPS). PCIe specification indicates that the MPS
must not exceed minimum MPS of any element along the packet path.
Force limit Max_Payload_Size to at most 256 Bytes for each device
connected to this PCIe controller.

R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
MRRS (Max Read Request Size) states:
  Please set "Max Read Request Size" to 128 bytes or 256 bytes.
  If "Max Read Request Size" is set to anything other than the
  above, the transferred data will not match the expected value.
This limitation also seems the apply to devices issuing MRd TLP.
This limitation can be triggered by using non-HMB NVMe SSD with
Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
Any write into the SSD (MRd TLP issued by the SSD) longer than
256 Bytes wraps around at 256 Byte boundary, and the same data
are written into the SSD starting at offset 0 and at 256 Bytes.
Force limit Max_Read_Request_Size to at most 256 Bytes for each
device connected to this PCIe controller to avoid this behavior.

An non-HMB (Host Memory Buffer) NVMe SSD can be identified using
the following command. Affected SSD reports 'hmpre' field as 0:
"
$ nvme id-ctrl /dev/nvme0 | grep hmpre
hmpre     : 0
"

The symptom is a read from the SSD which wraps around at 256 Byte
boundary. The test for this symptom can be implemented by writing
512 Byte of random data into the SSD and reading the data back. If
the read back data repeat after 256 Bytes, the device is affected.
"
$ dd if=/dev/urandom of=/tmp/data.bin bs=256 count=2 ; \
  dd if=/tmp/data.bin of=/dev/nvme0n1 bs=256 count=2 ; \
  dd if=/dev/nvme0n1 bs=256 count=2 of=/tmp/readback.bin
"

Expected data:
"
$ hexdump -vC /tmp/data.bin
00000000  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca
00000010  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b
...
00000100  7a ce 3c b2 e1 d5 d9 11  88 63 10 59 76 3c dc 32 <-- random
00000110  72 32 2a 7d a3 e1 aa 13  7c da 58 a1 7b 21 11 50 <-- data
"

Faulty readback, collected without this change in place:
"
$ hexdump -vC /tmp/readback.bin
00000000  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca <---.
00000010  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b <-. |
...                                                          | |
00000100  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca <-:-+- repeated
00000110  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b <-+--- data
     ^^^
      |
      '--- Repeat starts at offset 0x100 = 256 Bytes
"

Fixes: 0d0c551011df ("PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode")
Cc: stable@vger.kernel.org
Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
---
Cc: "Krzysztof Wilczyński" <kwilczynski@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Geert Uytterhoeven <geert+renesas@glider.be>
Cc: Koichiro Den <den@valinux.co.jp>
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Manivannan Sadhasivam <mani@kernel.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Cc: linux-renesas-soc@vger.kernel.org
---
V2: - Dispose of the reprogramming of MPS/MRRS altogether
    - Dispose of the entire fixup quirk
    - Replace both with bridge enable_device hook
    - Limit MPS/MRRS along the entire packet path to follow
      MRRS limitation requirement
V3: - Set no_inc_mrrs to prevent periperhal drivers from increasing MRRS
      over the limit set by this PCIe controller driver
    - Warn on MPS > 256 Bytes
    - Recheck updated documentation and update code comments to match
---
 drivers/pci/controller/dwc/pcie-rcar-gen4.c | 51 +++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index fbe465a29068f..e38cde2257bb4 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -411,6 +411,54 @@ static int rcar_gen4_pcie_host_msi_init(struct dw_pcie_rp *pp)
 	return ret;
 }
 
+static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge,
+					struct pci_dev *dev)
+{
+	/*
+	 * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
+	 * Max_Payload_Size (MPS). PCIe specification indicates that the MPS
+	 * must not exceed minimum MPS of any element along the packet path.
+	 * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes
+	 * for header type 0 and 128 Bytes for header type 1. The PCIe core
+	 * will not allow MPS to be set higher than MPSS, warn here in case
+	 * something went very wrong in the core.
+	 *
+	 * For details, refer to chapter "104.1.1 Features" in either of:
+	 * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+	 * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+	 * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+	 */
+	WARN_ON(pcie_get_mps(dev) > 256);
+
+	/*
+	 * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
+	 * MRRS (Max Read Request Size) states:
+	 *   Please set "Max Read Request Size" to 128 bytes or 256 bytes.
+	 *   If "Max Read Request Size" is set to anything other than the
+	 *   above, the transferred data will not match the expected value.
+	 * This limitation also seems the apply to devices issuing MRd TLP.
+	 * This limitation can be triggered by using non-HMB NVMe SSD with
+	 * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
+	 * Any write into the SSD (MRd TLP issued by the SSD) longer than
+	 * 256 Bytes wraps around at 256 Byte boundary, and the same data
+	 * are written into the SSD starting at offset 0 and at 256 Bytes.
+	 * Force limit Max_Read_Request_Size to at most 256 Bytes for each
+	 * device connected to this PCIe controller to avoid this behavior.
+	 *
+	 * For details, refer to aforementioned chapter in either of:
+	 * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+	 * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+	 * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+	 */
+	bridge->no_inc_mrrs = 1;
+	if (pcie_get_readrq(dev) > 256) {
+		pci_info(dev, "Limiting MRRS to 256 bytes\n");
+		pcie_set_readrq(dev, 256);
+	}
+
+	return 0;
+}
+
 /* Host mode */
 static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
 {
@@ -418,6 +466,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
 	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
 	int ret;
 
+	if (pp->bridge)
+		pp->bridge->enable_device = rcar_gen4_pcie_enable_device;
+
 	gpiod_set_value_cansleep(dw->pe_rst, 1);
 
 	ret = rcar_gen4_pcie_common_init(rcar);
-- 
2.53.0

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Manivannan Sadhasivam]

On Fri, 21 Aug 2026 04:05:51 +0200, Marek Vasut wrote:
> R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
> Max_Payload_Size (MPS). PCIe specification indicates that the MPS
> must not exceed minimum MPS of any element along the packet path.
> Force limit Max_Payload_Size to at most 256 Bytes for each device
> connected to this PCIe controller.
> 
> R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
> MRRS (Max Read Request Size) states:
>   Please set "Max Read Request Size" to 128 bytes or 256 bytes.
>   If "Max Read Request Size" is set to anything other than the
>   above, the transferred data will not match the expected value.
> This limitation also seems the apply to devices issuing MRd TLP.
> This limitation can be triggered by using non-HMB NVMe SSD with
> Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
> Any write into the SSD (MRd TLP issued by the SSD) longer than
> 256 Bytes wraps around at 256 Byte boundary, and the same data
> are written into the SSD starting at offset 0 and at 256 Bytes.
> Force limit Max_Read_Request_Size to at most 256 Bytes for each
> device connected to this PCIe controller to avoid this behavior.
> 
> [...]

Applied, thanks!

[1/1] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes
      commit: d1f8ada0815e247b52e5f9d4ff3a8bc9e4080bb9

Best regards,
-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Bjorn Helgaas]

[+cc Ziyao, Rong, Huacai for similar Loongson MRRS issue]

On Fri, Aug 21, 2026 at 04:05:51AM +0200, Marek Vasut wrote:
> R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
> Max_Payload_Size (MPS). PCIe specification indicates that the MPS
> must not exceed minimum MPS of any element along the packet path.
> Force limit Max_Payload_Size to at most 256 Bytes for each device
> connected to this PCIe controller.

IIUC the PCI core already enforces this limit, and what this patch
does is double-check that this limit is observed with this check,
right?

  +	WARN_ON(pcie_get_mps(dev) > 256);

More below.

> R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
> MRRS (Max Read Request Size) states:
>   Please set "Max Read Request Size" to 128 bytes or 256 bytes.
>   If "Max Read Request Size" is set to anything other than the
>   above, the transferred data will not match the expected value.
> This limitation also seems the apply to devices issuing MRd TLP.
> This limitation can be triggered by using non-HMB NVMe SSD with
> Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
> Any write into the SSD (MRd TLP issued by the SSD) longer than
> 256 Bytes wraps around at 256 Byte boundary, and the same data
> are written into the SSD starting at offset 0 and at 256 Bytes.
> Force limit Max_Read_Request_Size to at most 256 Bytes for each
> device connected to this PCIe controller to avoid this behavior.
> 
> An non-HMB (Host Memory Buffer) NVMe SSD can be identified using
> the following command. Affected SSD reports 'hmpre' field as 0:
> "
> $ nvme id-ctrl /dev/nvme0 | grep hmpre
> hmpre     : 0
> "
> 
> The symptom is a read from the SSD which wraps around at 256 Byte
> boundary. The test for this symptom can be implemented by writing
> 512 Byte of random data into the SSD and reading the data back. If
> the read back data repeat after 256 Bytes, the device is affected.
> "
> $ dd if=/dev/urandom of=/tmp/data.bin bs=256 count=2 ; \
>   dd if=/tmp/data.bin of=/dev/nvme0n1 bs=256 count=2 ; \
>   dd if=/dev/nvme0n1 bs=256 count=2 of=/tmp/readback.bin
> "
> 
> Expected data:
> "
> $ hexdump -vC /tmp/data.bin
> 00000000  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca
> 00000010  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b
> ...
> 00000100  7a ce 3c b2 e1 d5 d9 11  88 63 10 59 76 3c dc 32 <-- random
> 00000110  72 32 2a 7d a3 e1 aa 13  7c da 58 a1 7b 21 11 50 <-- data
> "
> 
> Faulty readback, collected without this change in place:
> "
> $ hexdump -vC /tmp/readback.bin
> 00000000  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca <---.
> 00000010  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b <-. |
> ...                                                          | |
> 00000100  97 81 b7 3b 0e 38 2b 4d  a7 d3 e0 47 ff c2 4b ca <-:-+- repeated
> 00000110  c1 85 98 f0 4a ac 03 a0  3b ab f3 19 44 dd 06 8b <-+--- data
>      ^^^
>       |
>       '--- Repeat starts at offset 0x100 = 256 Bytes
> "
> 
> Fixes: 0d0c551011df ("PCI: rcar-gen4: Add R-Car Gen4 PCIe controller support for host mode")
> Cc: stable@vger.kernel.org
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
> ---
> Cc: "Krzysztof Wilczyński" <kwilczynski@kernel.org>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Geert Uytterhoeven <geert+renesas@glider.be>
> Cc: Koichiro Den <den@valinux.co.jp>
> Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
> Cc: Magnus Damm <magnus.damm@gmail.com>
> Cc: Manivannan Sadhasivam <mani@kernel.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-pci@vger.kernel.org
> Cc: linux-renesas-soc@vger.kernel.org
> ---
> V2: - Dispose of the reprogramming of MPS/MRRS altogether
>     - Dispose of the entire fixup quirk
>     - Replace both with bridge enable_device hook
>     - Limit MPS/MRRS along the entire packet path to follow
>       MRRS limitation requirement
> V3: - Set no_inc_mrrs to prevent periperhal drivers from increasing MRRS
>       over the limit set by this PCIe controller driver
>     - Warn on MPS > 256 Bytes
>     - Recheck updated documentation and update code comments to match
> ---
>  drivers/pci/controller/dwc/pcie-rcar-gen4.c | 51 +++++++++++++++++++++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> index fbe465a29068f..e38cde2257bb4 100644
> --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> @@ -411,6 +411,54 @@ static int rcar_gen4_pcie_host_msi_init(struct dw_pcie_rp *pp)
>  	return ret;
>  }
>  
> +static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge,
> +					struct pci_dev *dev)
> +{
> +	/*
> +	 * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
> +	 * Max_Payload_Size (MPS). PCIe specification indicates that the MPS
> +	 * must not exceed minimum MPS of any element along the packet path.
> +	 * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes
> +	 * for header type 0 and 128 Bytes for header type 1. The PCIe core
> +	 * will not allow MPS to be set higher than MPSS, warn here in case
> +	 * something went very wrong in the core.
> +	 *
> +	 * For details, refer to chapter "104.1.1 Features" in either of:
> +	 * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
> +	 * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
> +	 * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
> +	 */
> +	WARN_ON(pcie_get_mps(dev) > 256);
> +
> +	/*
> +	 * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
> +	 * MRRS (Max Read Request Size) states:
> +	 *   Please set "Max Read Request Size" to 128 bytes or 256 bytes.
> +	 *   If "Max Read Request Size" is set to anything other than the
> +	 *   above, the transferred data will not match the expected value.
> +	 * This limitation also seems the apply to devices issuing MRd TLP.
> +	 * This limitation can be triggered by using non-HMB NVMe SSD with
> +	 * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
> +	 * Any write into the SSD (MRd TLP issued by the SSD) longer than
> +	 * 256 Bytes wraps around at 256 Byte boundary, and the same data
> +	 * are written into the SSD starting at offset 0 and at 256 Bytes.
> +	 * Force limit Max_Read_Request_Size to at most 256 Bytes for each
> +	 * device connected to this PCIe controller to avoid this behavior.
> +	 *
> +	 * For details, refer to aforementioned chapter in either of:
> +	 * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
> +	 * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
> +	 * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
> +	 */
> +	bridge->no_inc_mrrs = 1;
> +	if (pcie_get_readrq(dev) > 256) {
> +		pci_info(dev, "Limiting MRRS to 256 bytes\n");
> +		pcie_set_readrq(dev, 256);
> +	}

It would be nice if all the platforms that need no_inc_mrrs could
apply it the same way, but I assume you saw loongson_mrrs_quirk() and
loongson_set_min_mrrs_quirk() in the process of finding no_inc_mrrs,
and chose a different implementation strategy for some reason, e.g.,
this way doesn't have to include device IDs for all the Root Ports?

Maybe we should rework no_inc_mrrs in such a way that drivers could
set a max MRRS in the struct pci_host_bridge and make
pcie_write_mrrs() and pcie_set_readrq() pay attention to it?  That
might let us get rid of the FIXUP approach.

> +	return 0;
> +}
> +
>  /* Host mode */
>  static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
>  {
> @@ -418,6 +466,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
>  	struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
>  	int ret;
>  
> +	if (pp->bridge)
> +		pp->bridge->enable_device = rcar_gen4_pcie_enable_device;
> +
>  	gpiod_set_value_cansleep(dw->pe_rst, 1);
>  
>  	ret = rcar_gen4_pcie_common_init(rcar);
> -- 
> 2.53.0
> 

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Marek Vasut]

Hello Bjorn,

On 9/3/26 7:32 PM, Bjorn Helgaas wrote:
> [+cc Ziyao, Rong, Huacai for similar Loongson MRRS issue]
> 
> On Fri, Aug 21, 2026 at 04:05:51AM +0200, Marek Vasut wrote:
>> R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
>> Max_Payload_Size (MPS). PCIe specification indicates that the MPS
>> must not exceed minimum MPS of any element along the packet path.
>> Force limit Max_Payload_Size to at most 256 Bytes for each device
>> connected to this PCIe controller.
> 
> IIUC the PCI core already enforces this limit, and what this patch
> does is double-check that this limit is observed with this check,
> right?

That is correct, this was changed in V3, I missed the commit message 
update, sorry.

Would you like me to respin the patch one more time with an updated 
commit message, or would you be willing to fix it up in tree ?

>    +	WARN_ON(pcie_get_mps(dev) > 256);
> 
> More below.


[...]

>> +	bridge->no_inc_mrrs = 1;
>> +	if (pcie_get_readrq(dev) > 256) {
>> +		pci_info(dev, "Limiting MRRS to 256 bytes\n");
>> +		pcie_set_readrq(dev, 256);
>> +	}
> 
> It would be nice if all the platforms that need no_inc_mrrs could
> apply it the same way, but I assume you saw loongson_mrrs_quirk() and
> loongson_set_min_mrrs_quirk() in the process of finding no_inc_mrrs,
> and chose a different implementation strategy for some reason, e.g.,
> this way doesn't have to include device IDs for all the Root Ports?

The loongson quirk won't work if the PCIe controller driver is built as 
a module, which the R-Car Gen4 PCIe driver can be, and in fact is often 
built as a module, because it depends on firmware which is loaded from 
filesystem.

If the controller driver is built as a module, then 
DECLARE_PCI_FIXUP_ENABLE() is not applied, the 
DECLARE_PCI_FIXUP_ENABLE() is applied only on boot and therefore only 
for built-in drivers.

I got burnt by DECLARE_PCI_FIXUP_ENABLE() in V1 of this patch.

However, there is also another part to this -- the 
rcar_gen4_pcie_enable_device() is called for every device on the bus and 
applies the MRRS limitation to every device on the bus that is 
downstream of the controller, not only the controller. This is necessary 
on this controller variant, else hardware like PCIe SSDs with MRRS 
higher than the controller break.

> Maybe we should rework no_inc_mrrs in such a way that drivers could
> set a max MRRS in the struct pci_host_bridge and make
> pcie_write_mrrs() and pcie_set_readrq() pay attention to it?  That
> might let us get rid of the FIXUP approach.
In light of the last paragraph above, that the MRRS has to be limited 
also on all devices downstream of this particular controller, I would 
like to ask -- does the Loongson controller have the same limitation or 
not ? If not, then I would argue this quirk should be isolated to this 
controller variant ; else, I am happy to start on the core patches.

[...]

Thank you for your help !

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Bjorn Helgaas]

On Thu, Sep 03, 2026 at 08:51:06PM +0200, Marek Vasut wrote:
> Hello Bjorn,
> 
> On 9/3/26 7:32 PM, Bjorn Helgaas wrote:
> > [+cc Ziyao, Rong, Huacai for similar Loongson MRRS issue]
> > 
> > On Fri, Aug 21, 2026 at 04:05:51AM +0200, Marek Vasut wrote:
> > > R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
> > > Max_Payload_Size (MPS). PCIe specification indicates that the MPS
> > > must not exceed minimum MPS of any element along the packet path.
> > > Force limit Max_Payload_Size to at most 256 Bytes for each device
> > > connected to this PCIe controller.
> > 
> > IIUC the PCI core already enforces this limit, and what this patch
> > does is double-check that this limit is observed with this check,
> > right?
> 
> That is correct, this was changed in V3, I missed the commit message update,
> sorry.
> 
> Would you like me to respin the patch one more time with an updated commit
> message, or would you be willing to fix it up in tree ?

I fixed the commit log, no problem.

> >    +	WARN_ON(pcie_get_mps(dev) > 256);
> > 
> > More below.
> 
> 
> [...]
> 
> > > +	bridge->no_inc_mrrs = 1;
> > > +	if (pcie_get_readrq(dev) > 256) {
> > > +		pci_info(dev, "Limiting MRRS to 256 bytes\n");
> > > +		pcie_set_readrq(dev, 256);
> > > +	}
> > 
> > It would be nice if all the platforms that need no_inc_mrrs could
> > apply it the same way, but I assume you saw loongson_mrrs_quirk() and
> > loongson_set_min_mrrs_quirk() in the process of finding no_inc_mrrs,
> > and chose a different implementation strategy for some reason, e.g.,
> > this way doesn't have to include device IDs for all the Root Ports?
> 
> The loongson quirk won't work if the PCIe controller driver is built as a
> module, which the R-Car Gen4 PCIe driver can be, and in fact is often built
> as a module, because it depends on firmware which is loaded from filesystem.
> 
> If the controller driver is built as a module, then
> DECLARE_PCI_FIXUP_ENABLE() is not applied, the DECLARE_PCI_FIXUP_ENABLE() is
> applied only on boot and therefore only for built-in drivers.
> 
> I got burnt by DECLARE_PCI_FIXUP_ENABLE() in V1 of this patch.

Ouch, that does hurt.

> However, there is also another part to this -- the
> rcar_gen4_pcie_enable_device() is called for every device on the bus and
> applies the MRRS limitation to every device on the bus that is downstream of
> the controller, not only the controller. This is necessary on this
> controller variant, else hardware like PCIe SSDs with MRRS higher than the
> controller break.
> 
> > Maybe we should rework no_inc_mrrs in such a way that drivers could
> > set a max MRRS in the struct pci_host_bridge and make
> > pcie_write_mrrs() and pcie_set_readrq() pay attention to it?  That
> > might let us get rid of the FIXUP approach.
>
> In light of the last paragraph above, that the MRRS has to be limited also
> on all devices downstream of this particular controller, I would like to ask
> -- does the Loongson controller have the same limitation or not ? If not,
> then I would argue this quirk should be isolated to this controller variant
> ; else, I am happy to start on the core patches.

I don't know if we'll get a real answer for Loongson (there's no
maintainer listed for it, hint hint :)), but my guess is that it does
apply to all devices downstream of the Loongson controller.

I think MRRS is mostly interesting for DMA because MMIO from CPUs is
usually small sizes, far below the 128-byte or larger transfers that
devices may do.

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Marek Vasut]

Hello Bjorn,

On 9/3/26 10:27 PM, Bjorn Helgaas wrote:
> On Thu, Sep 03, 2026 at 08:51:06PM +0200, Marek Vasut wrote:
>> Hello Bjorn,
>>
>> On 9/3/26 7:32 PM, Bjorn Helgaas wrote:
>>> [+cc Ziyao, Rong, Huacai for similar Loongson MRRS issue]
>>>
>>> On Fri, Aug 21, 2026 at 04:05:51AM +0200, Marek Vasut wrote:
>>>> R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
>>>> Max_Payload_Size (MPS). PCIe specification indicates that the MPS
>>>> must not exceed minimum MPS of any element along the packet path.
>>>> Force limit Max_Payload_Size to at most 256 Bytes for each device
>>>> connected to this PCIe controller.
>>>
>>> IIUC the PCI core already enforces this limit, and what this patch
>>> does is double-check that this limit is observed with this check,
>>> right?
>>
>> That is correct, this was changed in V3, I missed the commit message update,
>> sorry.
>>
>> Would you like me to respin the patch one more time with an updated commit
>> message, or would you be willing to fix it up in tree ?
> 
> I fixed the commit log, no problem.

Thank you.

>>>     +	WARN_ON(pcie_get_mps(dev) > 256);
>>>
>>> More below.
>>
>>
>> [...]
>>
>>>> +	bridge->no_inc_mrrs = 1;
>>>> +	if (pcie_get_readrq(dev) > 256) {
>>>> +		pci_info(dev, "Limiting MRRS to 256 bytes\n");
>>>> +		pcie_set_readrq(dev, 256);
>>>> +	}
>>>
>>> It would be nice if all the platforms that need no_inc_mrrs could
>>> apply it the same way, but I assume you saw loongson_mrrs_quirk() and
>>> loongson_set_min_mrrs_quirk() in the process of finding no_inc_mrrs,
>>> and chose a different implementation strategy for some reason, e.g.,
>>> this way doesn't have to include device IDs for all the Root Ports?
>>
>> The loongson quirk won't work if the PCIe controller driver is built as a
>> module, which the R-Car Gen4 PCIe driver can be, and in fact is often built
>> as a module, because it depends on firmware which is loaded from filesystem.
>>
>> If the controller driver is built as a module, then
>> DECLARE_PCI_FIXUP_ENABLE() is not applied, the DECLARE_PCI_FIXUP_ENABLE() is
>> applied only on boot and therefore only for built-in drivers.
>>
>> I got burnt by DECLARE_PCI_FIXUP_ENABLE() in V1 of this patch.
> 
> Ouch, that does hurt.

I also mentioned this to TI a while back, because I think the 
pci-keystone.c has the same (module) issue. ( +CC Nishanth here too )

>> However, there is also another part to this -- the
>> rcar_gen4_pcie_enable_device() is called for every device on the bus and
>> applies the MRRS limitation to every device on the bus that is downstream of
>> the controller, not only the controller. This is necessary on this
>> controller variant, else hardware like PCIe SSDs with MRRS higher than the
>> controller break.
>>
>>> Maybe we should rework no_inc_mrrs in such a way that drivers could
>>> set a max MRRS in the struct pci_host_bridge and make
>>> pcie_write_mrrs() and pcie_set_readrq() pay attention to it?  That
>>> might let us get rid of the FIXUP approach.
>>
>> In light of the last paragraph above, that the MRRS has to be limited also
>> on all devices downstream of this particular controller, I would like to ask
>> -- does the Loongson controller have the same limitation or not ? If not,
>> then I would argue this quirk should be isolated to this controller variant
>> ; else, I am happy to start on the core patches.
> 
> I don't know if we'll get a real answer for Loongson (there's no
> maintainer listed for it, hint hint :)), but my guess is that it does
> apply to all devices downstream of the Loongson controller.
> 
> I think MRRS is mostly interesting for DMA because MMIO from CPUs is
> usually small sizes, far below the 128-byte or larger transfers that
> devices may do.

I agree with that, and DMA is what triggers the fault in my case.

Looking at the TI ks_pcie_quirk() FIXUP, I wonder if that might be a 
third instance of the same behavior. TI uses it to work around errata 
i2037 PCIe: PCI-Express May Corrupt Inbound Data [1] page 19 . But I now 
wonder, whether this behavior might be some common behavior of the DWC 
PCIe controller core ? Is there someone from Synopsys who might comment 
on that ?

[1] https://www.ti.com/lit/er/sprz452i/sprz452i.pdf

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Rong Zhang]

Hi Bjorn, Marek,

Thanks for reaching me.

On Thu, 2026-09-03 at 15:27 -0500, Bjorn Helgaas wrote:
> On Thu, Sep 03, 2026 at 08:51:06PM +0200, Marek Vasut wrote:
> > Hello Bjorn,
> > 
> > On 9/3/26 7:32 PM, Bjorn Helgaas wrote:
> > > [+cc Ziyao, Rong, Huacai for similar Loongson MRRS issue]
> > > 
> > > On Fri, Aug 21, 2026 at 04:05:51AM +0200, Marek Vasut wrote:
> > > > R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
> > > > Max_Payload_Size (MPS). PCIe specification indicates that the MPS
> > > > must not exceed minimum MPS of any element along the packet path.
> > > > Force limit Max_Payload_Size to at most 256 Bytes for each device
> > > > connected to this PCIe controller.
> > > 
> > > IIUC the PCI core already enforces this limit, and what this patch
> > > does is double-check that this limit is observed with this check,
> > > right?
> > 
> > That is correct, this was changed in V3, I missed the commit message update,
> > sorry.
> > 
> > Would you like me to respin the patch one more time with an updated commit
> > message, or would you be willing to fix it up in tree ?
> 
> I fixed the commit log, no problem.
> 
> > >    +	WARN_ON(pcie_get_mps(dev) > 256);
> > > 
> > > More below.
> > 
> > 
> > [...]
> > 
> > > > +	bridge->no_inc_mrrs = 1;
> > > > +	if (pcie_get_readrq(dev) > 256) {
> > > > +		pci_info(dev, "Limiting MRRS to 256 bytes\n");
> > > > +		pcie_set_readrq(dev, 256);
> > > > +	}
> > > 
> > > It would be nice if all the platforms that need no_inc_mrrs could
> > > apply it the same way, but I assume you saw loongson_mrrs_quirk() and
> > > loongson_set_min_mrrs_quirk() in the process of finding no_inc_mrrs,
> > > and chose a different implementation strategy for some reason, e.g.,
> > > this way doesn't have to include device IDs for all the Root Ports?
> > 
> > The loongson quirk won't work if the PCIe controller driver is built as a
> > module, which the R-Car Gen4 PCIe driver can be, and in fact is often built
> > as a module, because it depends on firmware which is loaded from filesystem.
> > 
> > If the controller driver is built as a module, then
> > DECLARE_PCI_FIXUP_ENABLE() is not applied, the DECLARE_PCI_FIXUP_ENABLE() is
> > applied only on boot and therefore only for built-in drivers.
> > 
> > I got burnt by DECLARE_PCI_FIXUP_ENABLE() in V1 of this patch.
> 
> Ouch, that does hurt.
> 
> > However, there is also another part to this -- the
> > rcar_gen4_pcie_enable_device() is called for every device on the bus and
> > applies the MRRS limitation to every device on the bus that is downstream of
> > the controller, not only the controller. This is necessary on this
> > controller variant, else hardware like PCIe SSDs with MRRS higher than the
> > controller break.
> > 
> > > Maybe we should rework no_inc_mrrs in such a way that drivers could
> > > set a max MRRS in the struct pci_host_bridge and make
> > > pcie_write_mrrs() and pcie_set_readrq() pay attention to it?  That
> > > might let us get rid of the FIXUP approach.
> > 
> > In light of the last paragraph above, that the MRRS has to be limited also
> > on all devices downstream of this particular controller, I would like to ask
> > -- does the Loongson controller have the same limitation or not ? If not,
> > then I would argue this quirk should be isolated to this controller variant
> > ; else, I am happy to start on the core patches.

ACK. I agreed that it should make our life easier.

> 
> I don't know if we'll get a real answer for Loongson (there's no
> maintainer listed for it, hint hint :)), 
> 

(+CC Jiaxun)

The driver was introduced by Jiaxun without updating MAINTAINERS. I guess
he'd be willing to be listed as a maintainer.

I don't work for Loongson, but I do maintain several MIPS-based Loongson
devices for the Golang community with my colleagues and personally own a
MIPS-based Loongson-LS3A4000-7A1000-NUC-SE mini PC. I do some PCIe
experiments on it from time to time for fun. So I am OK if someone wants
to list me as a maintainer or reviewer :)

> but my guess is that it does
> apply to all devices downstream of the Loongson controller.

I believe this is the case. Maybe Jiaxun can shed a light on it too.

Just checked the kmsg log from April, the firmware seemed to only clamp
MRRS for devices directly connected to the root ports. IOW, it seemed to
only clamp MRRS for the upstream port of a PCIe switch, so
loongson_set_min_mrrs_quirk() had to fix up downstream ports.

If you need more information I can do some more experiments with the PCIe
switch card.

Thanks,
Rong

> 
> I think MRRS is mostly interesting for DMA because MMIO from CPUs is
> usually small sizes, far below the 128-byte or larger transfers that
> devices may do.

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Marek Vasut]

On 9/4/26 7:42 PM, Rong Zhang wrote:

Hello everyone,

>>> In light of the last paragraph above, that the MRRS has to be limited also
>>> on all devices downstream of this particular controller, I would like to ask
>>> -- does the Loongson controller have the same limitation or not ? If not,
>>> then I would argue this quirk should be isolated to this controller variant
>>> ; else, I am happy to start on the core patches.
> 
> ACK. I agreed that it should make our life easier.
> 
>>
>> I don't know if we'll get a real answer for Loongson (there's no
>> maintainer listed for it, hint hint :)),
>>
> 
> (+CC Jiaxun)
> 
> The driver was introduced by Jiaxun without updating MAINTAINERS. I guess
> he'd be willing to be listed as a maintainer.
> 
> I don't work for Loongson, but I do maintain several MIPS-based Loongson
> devices for the Golang community with my colleagues and personally own a
> MIPS-based Loongson-LS3A4000-7A1000-NUC-SE mini PC. I do some PCIe
> experiments on it from time to time for fun. So I am OK if someone wants
> to list me as a maintainer or reviewer :)
> 
>> but my guess is that it does
>> apply to all devices downstream of the Loongson controller.
> 
> I believe this is the case. Maybe Jiaxun can shed a light on it too.
> 
> Just checked the kmsg log from April, the firmware seemed to only clamp
> MRRS for devices directly connected to the root ports. IOW, it seemed to
> only clamp MRRS for the upstream port of a PCIe switch, so
> loongson_set_min_mrrs_quirk() had to fix up downstream ports.
> 
> If you need more information I can do some more experiments with the PCIe
> switch card.

Please look at the commit message [1] , that lists the exact test and 
the PCIe NVMe SSD which triggers the data corruption (let me know if 
something is unclear). I think any PCIe NVMe SSD which is non-HMB with 
MRRS on the SSD size that is > MRRS on the controller side will trigger 
this issue too. If you could reproduce it on the Loongson, then yes, it 
is affected. You might have to disable the current MRRS workaround in 
the Loongson PCIe driver though as it might hide the issue.

Thank you for your help !

+CC NXP, because they have similar controller on MX95 (and others, but 
the MX95 is closest to this I think)

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=37ca1997d2eeec86ad065008d6387fd15d18934c

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Rong Zhang]

Hi Marek,

On Fri, 2026-09-04 at 23:53 +0200, Marek Vasut wrote:
> On 9/4/26 7:42 PM, Rong Zhang wrote:
> 
> Hello everyone,
> 
> > > > In light of the last paragraph above, that the MRRS has to be limited also
> > > > on all devices downstream of this particular controller, I would like to ask
> > > > -- does the Loongson controller have the same limitation or not ? If not,
> > > > then I would argue this quirk should be isolated to this controller variant
> > > > ; else, I am happy to start on the core patches.
> > 
> > ACK. I agreed that it should make our life easier.
> > 
> > > 
> > > I don't know if we'll get a real answer for Loongson (there's no
> > > maintainer listed for it, hint hint :)),
> > > 
> > 
> > (+CC Jiaxun)
> > 
> > The driver was introduced by Jiaxun without updating MAINTAINERS. I guess
> > he'd be willing to be listed as a maintainer.
> > 
> > I don't work for Loongson, but I do maintain several MIPS-based Loongson
> > devices for the Golang community with my colleagues and personally own a
> > MIPS-based Loongson-LS3A4000-7A1000-NUC-SE mini PC. I do some PCIe
> > experiments on it from time to time for fun. So I am OK if someone wants
> > to list me as a maintainer or reviewer :)
> > 
> > > but my guess is that it does
> > > apply to all devices downstream of the Loongson controller.
> > 
> > I believe this is the case. Maybe Jiaxun can shed a light on it too.
> > 
> > Just checked the kmsg log from April, the firmware seemed to only clamp
> > MRRS for devices directly connected to the root ports. IOW, it seemed to
> > only clamp MRRS for the upstream port of a PCIe switch, so
> > loongson_set_min_mrrs_quirk() had to fix up downstream ports.
> > 
> > If you need more information I can do some more experiments with the PCIe
> > switch card.
> 
> Please look at the commit message [1] , that lists the exact test and 
> the PCIe NVMe SSD which triggers the data corruption (let me know if 
> something is unclear). I think any PCIe NVMe SSD which is non-HMB with 
> MRRS on the SSD size that is > MRRS on the controller side will trigger 
> this issue too. 
> 

Unfortunately I don't have any non-HMB NVMe SSD, and it's unlikely I'll
get one anytime soon due to the crazy price. Before the AI supply chain
crisis, I bought several NVMe SSDs, but I chose HMB ones as they generate
less heat and consume less electricity.

> If you could reproduce it on the Loongson, then yes, it 
> is affected. You might have to disable the current MRRS workaround in 
> the Loongson PCIe driver though as it might hide the issue.

Due to the lack of non-HMB NVMe SSDs, I can't reproduce the exact same
issue. Theoretically I may try to mess around with the AHCI controller
(see the mentioned commit below) by manually overriding MRRS, as my
device's firmware already clamps it. We will see...

The firmware of many Loongson devices (except for some MIPS-based ones,
see the commit message of commit ef61a0405742 ("PCI: loongson: Limit MRRS
to 256")) already clamps MRRS for most PCIe devices, which has proven
that the 7A1000/7A2000 chipset does need the MRRS quirk. Therefore, you
can just go ahead with preparing the core patches without the need of
waiting for my test result. It seems that your approach is not going to
change the current fixup effects in a significant way, so it should be
fine.

> 
> Thank you for your help !
> 

Thanks,
Rong

> +CC NXP, because they have similar controller on MX95 (and others, but 
> the MX95 is closest to this I think)
> 
> [1] 
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=37ca1997d2eeec86ad065008d6387fd15d18934c

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Marek Vasut]

On 9/6/26 7:07 PM, Rong Zhang wrote:

Hello Rong,

I apologize for my delayed reply.

>>> If you need more information I can do some more experiments with the PCIe
>>> switch card.
>>
>> Please look at the commit message [1] , that lists the exact test and
>> the PCIe NVMe SSD which triggers the data corruption (let me know if
>> something is unclear). I think any PCIe NVMe SSD which is non-HMB with
>> MRRS on the SSD size that is > MRRS on the controller side will trigger
>> this issue too.
>>
> 
> Unfortunately I don't have any non-HMB NVMe SSD, and it's unlikely I'll
> get one anytime soon due to the crazy price. Before the AI supply chain
> crisis, I bought several NVMe SSDs, but I chose HMB ones as they generate
> less heat and consume less electricity.

Maybe this could be reproduced even with another PCIe device with 
MRRS=512 or more, which posts MRd to the PCIe controller with MRRS=128 
or 256. The AHCI controller might be capable of that. I only have the 
SSD as a reproducer, but there could easily be more.

>> If you could reproduce it on the Loongson, then yes, it
>> is affected. You might have to disable the current MRRS workaround in
>> the Loongson PCIe driver though as it might hide the issue.
> 
> Due to the lack of non-HMB NVMe SSDs, I can't reproduce the exact same
> issue. Theoretically I may try to mess around with the AHCI controller
> (see the mentioned commit below) by manually overriding MRRS, as my
> device's firmware already clamps it. We will see...

Does the AHCI controller have MRRS higher than the PCIe controller ?

> The firmware of many Loongson devices (except for some MIPS-based ones,
> see the commit message of commit ef61a0405742 ("PCI: loongson: Limit MRRS
> to 256")) already clamps MRRS for most PCIe devices, which has proven
> that the 7A1000/7A2000 chipset does need the MRRS quirk.

This still only applies the MRRS limit to the PCIe controller, not to 
all devices on the bus, correct ?

I would like to find out whether all the devices (not only the 
controller) on the PCIe bus on Loongson must have MRRS limited or not.
-> If yes, then I should start preparing core patches
-> If no, then this quirk might specific to this controller

> Therefore, you
> can just go ahead with preparing the core patches without the need of
> waiting for my test result. It seems that your approach is not going to
> change the current fixup effects in a significant way, so it should be
> fine.

[...]

Thank you for your help !

-- 
Best regards,
Marek Vasut

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Rong Zhang]

Hi Marek,

On Wed, 2026-09-09 at 04:34 +0200, Marek Vasut wrote:
> On 9/6/26 7:07 PM, Rong Zhang wrote:
> 
> Hello Rong,
> 
> I apologize for my delayed reply.
> 
> > > > If you need more information I can do some more experiments with the PCIe
> > > > switch card.
> > > 
> > > Please look at the commit message [1] , that lists the exact test and
> > > the PCIe NVMe SSD which triggers the data corruption (let me know if
> > > something is unclear). I think any PCIe NVMe SSD which is non-HMB with
> > > MRRS on the SSD size that is > MRRS on the controller side will trigger
> > > this issue too.
> > > 
> > 
> > Unfortunately I don't have any non-HMB NVMe SSD, and it's unlikely I'll
> > get one anytime soon due to the crazy price. Before the AI supply chain
> > crisis, I bought several NVMe SSDs, but I chose HMB ones as they generate
> > less heat and consume less electricity.
> 
> Maybe this could be reproduced even with another PCIe device with 
> MRRS=512 or more, which posts MRd to the PCIe controller with MRRS=128 
> or 256. The AHCI controller might be capable of that. I only have the 
> SSD as a reproducer, but there could easily be more.
> 
> > > If you could reproduce it on the Loongson, then yes, it
> > > is affected. You might have to disable the current MRRS workaround in
> > > the Loongson PCIe driver though as it might hide the issue.
> > 
> > Due to the lack of non-HMB NVMe SSDs, I can't reproduce the exact same
> > issue. Theoretically I may try to mess around with the AHCI controller
> > (see the mentioned commit below) by manually overriding MRRS, as my
> > device's firmware already clamps it. We will see...
> 
> Does the AHCI controller have MRRS higher than the PCIe controller ?

Oh, I meant "a" controller, not "the" controller. Sorry for the typo.

I don't have the exact same controller as the report, but I do have a
quite modern one. It's from an AMD Promontory 21 chipset add-in card, see
https://oshwhub.com/wesd/b650

I haven't done any MRRS experiment yet. I will find some time this week
or the next. As I've said, you don't need to wait for my result :-)

> 
> > The firmware of many Loongson devices (except for some MIPS-based ones,
> > see the commit message of commit ef61a0405742 ("PCI: loongson: Limit MRRS
> > to 256")) already clamps MRRS for most PCIe devices, which has proven
> > that the 7A1000/7A2000 chipset does need the MRRS quirk.
> 
> This still only applies the MRRS limit to the PCIe controller, not to 
> all devices on the bus, correct ?

This applies to all devices that are the downstream (direct or behind
another bridge) of any root ports (i.e., internal bridges). In detail,
pci_match_id(loongson_internal_bridge_devids, *bridge*) matches the root
port but pcie_set_readrq(*pdev*, 256) sets the downstream PCIe device.

This essentially sets all PCIe devices on the bus (unless firmware has
selected a smaller MRRS), since any other internal endpoints of the
chipset have no PCIe capability (i.e., they are legacy PCI or HT
devices). IOW, only the root ports (i.e.,
loongson_internal_bridge_devids) and their downstream PCIe devices have
MRRS, and they are covered by loongson_set_min_mrrs_quirk().

> 
> I would like to find out whether all the devices (not only the 
> controller) on the PCIe bus on Loongson must have MRRS limited or not.
> -> If yes, then I should start preparing core patches
> -> If no, then this quirk might specific to this controller

My answer is yes.

> 
> > Therefore, you
> > can just go ahead with preparing the core patches without the need of
> > waiting for my test result. It seems that your approach is not going to
> > change the current fixup effects in a significant way, so it should be
> > fine.
> 
> [...]
> 
> Thank you for your help !

Thanks,
Rong

Re: [PATCH v3] PCI: rcar-gen4: Limit Max_Read_Request_Size and Max_Payload_Size to 256 Bytes

[Rong Zhang]

Hi Marek,

On Thu, 2026-09-10 at 00:03 +0800, Rong Zhang wrote:
> Hi Marek,
> 
> On Wed, 2026-09-09 at 04:34 +0200, Marek Vasut wrote:
> > On 9/6/26 7:07 PM, Rong Zhang wrote:
> > 
> > Hello Rong,
> > 
> > I apologize for my delayed reply.
> > 
> > > > > If you need more information I can do some more experiments with the PCIe
> > > > > switch card.
> > > > 
> > > > Please look at the commit message [1] , that lists the exact test and
> > > > the PCIe NVMe SSD which triggers the data corruption (let me know if
> > > > something is unclear). I think any PCIe NVMe SSD which is non-HMB with
> > > > MRRS on the SSD size that is > MRRS on the controller side will trigger
> > > > this issue too.
> > > > 
> > > 
> > > Unfortunately I don't have any non-HMB NVMe SSD, and it's unlikely I'll
> > > get one anytime soon due to the crazy price. Before the AI supply chain
> > > crisis, I bought several NVMe SSDs, but I chose HMB ones as they generate
> > > less heat and consume less electricity.
> > 
> > Maybe this could be reproduced even with another PCIe device with 
> > MRRS=512 or more, which posts MRd to the PCIe controller with MRRS=128 
> > or 256. The AHCI controller might be capable of that. I only have the 
> > SSD as a reproducer, but there could easily be more.
> > 
> > > > If you could reproduce it on the Loongson, then yes, it
> > > > is affected. You might have to disable the current MRRS workaround in
> > > > the Loongson PCIe driver though as it might hide the issue.
> > > 
> > > Due to the lack of non-HMB NVMe SSDs, I can't reproduce the exact same
> > > issue. Theoretically I may try to mess around with the AHCI controller
> > > (see the mentioned commit below) by manually overriding MRRS, as my
> > > device's firmware already clamps it. We will see...
> > 
> > Does the AHCI controller have MRRS higher than the PCIe controller ?
> 
> Oh, I meant "a" controller, not "the" controller. Sorry for the typo.
> 
> I don't have the exact same controller as the report, but I do have a
> quite modern one. It's from an AMD Promontory 21 chipset add-in card, see
> https://oshwhub.com/wesd/b650
> 
> I haven't done any MRRS experiment yet. I will find some time this week
> or the next. As I've said, you don't need to wait for my result :-)

I found some time to do the experiment and successfully reproduced the
MRRS issue.

PCI topology:

   -[0000:00]-+-00.0  Loongson Technology LLC 7A1000 Chipset Hyper Transport Bridge Controller
              +-00.1  Loongson Technology LLC 7A2000 Chipset Hyper Transport Bridge Controller
              +-03.0  Loongson Technology LLC 2K1000/2000 / 7A1000 Chipset Gigabit Ethernet Controller
              +-04.0  Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB OHCI Controller
              +-04.1  Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB EHCI Controller
              +-05.0  Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB OHCI Controller
              +-05.1  Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB EHCI Controller
              +-06.0  Loongson Technology LLC 7A1000 Chipset Vivante GC1000 GPU
              +-06.1  Loongson Technology LLC 2K1000 / 7A1000 Chipset Display Controller
              +-07.0  Loongson Technology LLC 2K1000/2000/3000 / 3B6000M / 7A1000/2000 Chipset HD Audio Controller
              +-08.0  Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller
              +-08.1  Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller
              +-08.2  Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller
              +-0a.0-[01]----00.0  Etron Technology, Inc. EJ188/EJ198 USB 3.0 Host Controller
              +-0f.0-[02-07]----00.0-[03-07]--+-00.0-[04]----00.0  MEDIATEK Corp. MT7925 802.11be 160MHz 2x2 PCIe Wireless Network Adapter [Filogic 360]
              |                               +-08.0-[05]----00.0  Shenzhen Longsys Electronics Co., Ltd. FORESEE XP1000 / Lexar Professional CFexpress Type B Gold series, NM620 PCIe NVME SSD (DRAM-less)
              |                               +-0c.0-[06]----00.0  Advanced Micro Devices, Inc. [AMD] 600 Series Chipset USB 3.2 Controller
              |                               \-0d.0-[07]----00.0  Advanced Micro Devices, Inc. [AMD] 600 Series Chipset SATA Controller
              \-16.0  Loongson Technology LLC 7A1000 Chipset SPI Controller
   
   00:00.0 Host bridge [0600]: Loongson Technology LLC 7A1000 Chipset Hyper Transport Bridge Controller [0014:7a00]
   00:00.1 Non-VGA unclassified device [0000]: Loongson Technology LLC 7A2000 Chipset Hyper Transport Bridge Controller [0014:7a10]
   00:03.0 Ethernet controller [0200]: Loongson Technology LLC 2K1000/2000 / 7A1000 Chipset Gigabit Ethernet Controller [0014:7a03]
   00:04.0 USB controller [0c03]: Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB OHCI Controller [0014:7a24]
   00:04.1 USB controller [0c03]: Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB EHCI Controller [0014:7a14]
   00:05.0 USB controller [0c03]: Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB OHCI Controller [0014:7a24]
   00:05.1 USB controller [0c03]: Loongson Technology LLC 2K1000 / 7A1000/2000 Chipset USB EHCI Controller [0014:7a14]
   00:06.0 Multimedia video controller [0400]: Loongson Technology LLC 7A1000 Chipset Vivante GC1000 GPU [0014:7a15]
   00:06.1 VGA compatible controller [0300]: Loongson Technology LLC 2K1000 / 7A1000 Chipset Display Controller [0014:7a06]
   00:07.0 Audio device [0403]: Loongson Technology LLC 2K1000/2000/3000 / 3B6000M / 7A1000/2000 Chipset HD Audio Controller [0014:7a07]
   00:08.0 SATA controller [0106]: Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller [0014:7a08]
   00:08.1 SATA controller [0106]: Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller [0014:7a08]
   00:08.2 SATA controller [0106]: Loongson Technology LLC 2K1000 / 7A1000 Chipset 3Gb/s SATA AHCI Controller [0014:7a08]
   00:0a.0 PCI bridge [0604]: Loongson Technology LLC 2K1000 / 7A1000 Chipset PCIe x1 Bridge [0014:7a09] (rev 01)
   00:0f.0 PCI bridge [0604]: Loongson Technology LLC 7A1000 Chipset PCIe x8 Bridge [0014:7a29] (rev 01)
   00:16.0 System peripheral [0880]: Loongson Technology LLC 7A1000 Chipset SPI Controller [0014:7a0b]
   01:00.0 USB controller [0c03]: Etron Technology, Inc. EJ188/EJ198 USB 3.0 Host Controller [1b6f:7052]
   02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Upstream Port [1022:43f4] (rev 01)
   03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
   03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
   03:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
   03:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset PCIe Switch Downstream Port [1022:43f5] (rev 01)
   04:00.0 Network controller [0280]: MEDIATEK Corp. MT7925 802.11be 160MHz 2x2 PCIe Wireless Network Adapter [Filogic 360] [14c3:7925]
   05:00.0 Non-Volatile memory controller [0108]: Shenzhen Longsys Electronics Co., Ltd. FORESEE XP1000 / Lexar Professional CFexpress Type B Gold series, NM620 PCIe NVME SSD (DRAM-less) [1d97:5216] (rev 01)
   06:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset USB 3.2 Controller [1022:43f7] (rev 01)
   07:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 600 Series Chipset SATA Controller [1022:43f6] (rev 01)

Instead of messing around with the AHCI controller, I decided to do the
experiment with the xHCI controller (0000:06:00.0) by connecting a USB
3.0 hub with a RTL8153 USB 3.0 to GbE controller. xHCI controllers do
DMA, so they can be used to reproduce the issue easily, without the need
of sacrificing the data of a hard drive.

With the quirk applied, everything worked fine.

I built a kernel image with the quirk removed, manually overrode MRRS to
512B, rebound the driver, and plugged the USB 3.0 hub to the xHCI
controller.

   [  580.527181] xhci_hcd 0000:06:00.0: USB bus 7 deregistered
   [  583.675928] xhci_hcd 0000:06:00.0: xHCI Host Controller
   [  583.689993] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 7
   [  583.774300] xhci_hcd 0000:06:00.0: hcc params 0x0200ef81 hci version 0x110 quirks 0x0000000200000010
   [  583.795399] xhci_hcd 0000:06:00.0: xHCI Host Controller
   [  583.795980] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 8
   [  583.810550] xhci_hcd 0000:06:00.0: Host supports USB 3.2 Enhanced SuperSpeed
   [  583.826573] usb usb7: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 7.01
   [  583.842936] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
   [  583.856206] usb usb7: Product: xHCI Host Controller
   [  583.868244] usb usb7: Manufacturer: Linux 7.1.9-cip-main+ xhci-hcd
   [  583.880346] usb usb7: SerialNumber: 0000:06:00.0
   [  583.899451] hub 7-0:1.0: USB hub found
   [  583.900066] hub 7-0:1.0: 12 ports detected
   [  583.924683] usb usb8: We don't know the algorithms for LPM for this host, disabling LPM.
   [  583.925753] usb usb8: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 7.01
   [  583.946605] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
   [  583.965879] usb usb8: Product: xHCI Host Controller
   [  583.979156] usb usb8: Manufacturer: Linux 7.1.9-cip-main+ xhci-hcd
   [  583.995799] usb usb8: SerialNumber: 0000:06:00.0
   [  584.025088] hub 8-0:1.0: USB hub found
   [  584.026026] hub 8-0:1.0: 5 ports detected
   [  584.249890] usb 7-1: new high-speed USB device number 2 using xhci_hcd
   [  584.452592] usb 7-1: New USB device found, idVendor=2109, idProduct=2817, bcdDevice= 2.14
   [  584.452743] usb 7-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
   [  584.452852] usb 7-1: Product: USB2.0 Hub
   [  584.470092] usb 7-1: Manufacturer: VIA Labs, Inc.
   [  584.495186] hub 7-1:1.0: USB hub found
   [  584.495833] hub 7-1:1.0: 4 ports detected
   [  584.556958] usb 8-1: Device not responding to setup address.
   [  584.774875] usb 8-1: new SuperSpeed USB device number 2 using xhci_hcd
   [  584.821162] usb 8-1: New USB device found, idVendor=2109, idProduct=0817, bcdDevice= 2.14
   [  584.821531] usb 8-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
   [  584.839861] usb 8-1: Product: USB3.0 Hub
   [  584.853146] usb 8-1: Manufacturer: VIA Labs, Inc.
   [  584.884823] hub 8-1:1.0: USB hub found
   [  584.885462] hub 8-1:1.0: 4 ports detected
   [  585.431958] usb 8-1.3: new SuperSpeed USB device number 3 using xhci_hcd
   [  585.466401] usb 8-1.3: Int endpoint with wBytesPerInterval of 8 in config 2 interface 0 altsetting 0 ep 0x83: setting to 16
   [  585.466636] usb 8-1.3: New USB device found, idVendor=0bda, idProduct=8153, bcdDevice=30.00
   [  585.482682] usb 8-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=6
   [  585.495865] usb 8-1.3: Product: USB 10/100/1000 LAN
   [  585.505992] usb 8-1.3: Manufacturer: Realtek
   [  585.514688] usb 8-1.3: SerialNumber: 000001
   [  585.621409] r8152-cfgselector 8-1.3: reset SuperSpeed USB device number 3 using xhci_hcd
   [  591.076733] r8152-cfgselector 8-1.3: Failed to write 512 bytes at 0xf800/0x01ff (-145)
   [  591.077414] r8152 8-1.3:1.0: Write PLA fw fail
   [  593.273770] r8152-cfgselector 8-1.3: reset SuperSpeed USB device number 3 using xhci_hcd
   [  598.570636] r8152-cfgselector 8-1.3: device descriptor read/8, error -145
   [  598.685870] r8152-cfgselector 8-1.3: reset SuperSpeed USB device number 3 using xhci_hcd
   [  615.108738] r8152-cfgselector 8-1.3: device descriptor read/8, error -145
   [  616.256190] usb 8-1-port3: cannot disable (err = -145)
   [  621.920265] hub 8-1:1.0: hub_ext_port_status failed (err = -145)
   [  623.047210] usb 8-1-port3: cannot reset (err = -145)
   [  624.169283] usb 8-1-port3: cannot reset (err = -145)
   [  625.294280] usb 8-1-port3: cannot reset (err = -145)
   [  626.419281] usb 8-1-port3: cannot reset (err = -145)
   [  627.544617] usb 8-1-port3: cannot reset (err = -145)
   [  627.544706] usb 8-1-port3: Cannot enable. Maybe the USB cable is bad?

Oops, the driver "Failed to write 512 bytes" to the RTL8153, and failed
to load the firmware.

I am not an expert in xHCI. IIUC, sending an OUT URB usually needs these
steps:

1. write data to the DMA buffer
2. prepare a TRB
3. write the TRB to the xHCI controller's TRB ring
4. ring the doorbell
5. the xHCI controller reads the data via DMA
6. the xHCI controller delivers the data to the device

The MRRS issue corrupted the data in the fifth step, causing the failures
on firmware loading. The subsequent port reset failures also seemed to be
triggered in the same way. On the other hand, device enumeration itself
succeeded, because IN URBs are delivered to the host via write DMA. 

After that, I unplugged the USB 3.0 hub, manually overrode MRRS to 256B
without rebinding the driver, and replug the hub to the xHCI controller.

   [  741.219489] usb 8-1: USB disconnect, device number 2
   [  741.220582] r8152-cfgselector 8-1.3: USB disconnect, device number 3
   [  741.334226] usb 7-1: USB disconnect, device number 2
   [  772.657795] usb 7-1: new high-speed USB device number 3 using xhci_hcd
   [  772.880341] usb 7-1: New USB device found, idVendor=2109, idProduct=2817, bcdDevice= 2.14
   [  772.880539] usb 7-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
   [  772.892671] usb 7-1: Product: USB2.0 Hub
   [  772.898715] usb 7-1: Manufacturer: VIA Labs, Inc.
   [  772.922778] hub 7-1:1.0: USB hub found
   [  772.923476] hub 7-1:1.0: 4 ports detected
   [  772.976982] usb 8-1: Device not responding to setup address.
   [  773.197271] usb 8-1: new SuperSpeed USB device number 4 using xhci_hcd
   [  773.241771] usb 8-1: New USB device found, idVendor=2109, idProduct=0817, bcdDevice= 2.14
   [  773.243041] usb 8-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
   [  773.256269] usb 8-1: Product: USB3.0 Hub
   [  773.265074] usb 8-1: Manufacturer: VIA Labs, Inc.
   [  773.289073] hub 8-1:1.0: USB hub found
   [  773.289735] hub 8-1:1.0: 4 ports detected
   [  773.831471] usb 8-1.3: new SuperSpeed USB device number 5 using xhci_hcd
   [  773.863051] usb 8-1.3: Int endpoint with wBytesPerInterval of 8 in config 2 interface 0 altsetting 0 ep 0x83: setting to 16
   [  773.863288] usb 8-1.3: New USB device found, idVendor=0bda, idProduct=8153, bcdDevice=30.00
   [  773.880071] usb 8-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=6
   [  773.892286] usb 8-1.3: Product: USB 10/100/1000 LAN
   [  773.902420] usb 8-1.3: Manufacturer: Realtek
   [  773.911120] usb 8-1.3: SerialNumber: 000001
   [  774.011156] r8152-cfgselector 8-1.3: reset SuperSpeed USB device number 5 using xhci_hcd
   [  774.380507] r8152 8-1.3:1.0 enp6s0u1u3: renamed from eth0
   [  777.496526] r8152 8-1.3:1.0 enp6s0u1u3: carrier on

Yay! Everything returned to normal.

> 
> > 
> > > The firmware of many Loongson devices (except for some MIPS-based ones,
> > > see the commit message of commit ef61a0405742 ("PCI: loongson: Limit MRRS
> > > to 256")) already clamps MRRS for most PCIe devices, which has proven
> > > that the 7A1000/7A2000 chipset does need the MRRS quirk.
> > 
> > This still only applies the MRRS limit to the PCIe controller, not to 
> > all devices on the bus, correct ?
> 
> This applies to all devices that are the downstream (direct or behind
> another bridge) of any root ports (i.e., internal bridges). In detail,
> pci_match_id(loongson_internal_bridge_devids, *bridge*) matches the root
> port but pcie_set_readrq(*pdev*, 256) sets the downstream PCIe device.
> 
> This essentially sets all PCIe devices on the bus (unless firmware has
> selected a smaller MRRS), since any other internal endpoints of the
> chipset have no PCIe capability (i.e., they are legacy PCI or HT
> devices). IOW, only the root ports (i.e.,
> loongson_internal_bridge_devids) and their downstream PCIe devices have
> MRRS, and they are covered by loongson_set_min_mrrs_quirk().

In conclusion, the experiment matched with what you've occurred on the R-
Car Gen4 PCIe controller, and proved that the quirk needs to be applied
to every PCIe device [1] on the bus.

[1]: As I've said in the previous reply, the chipset's only PCIe devices
are its PCIe root ports (so-called "internal bridges"), i.e.,
0000:00:0a.0 and 0000:00:0f.0.

Thanks,
Rong

> 
> > 
> > I would like to find out whether all the devices (not only the 
> > controller) on the PCIe bus on Loongson must have MRRS limited or not.
> > -> If yes, then I should start preparing core patches
> > -> If no, then this quirk might specific to this controller
> 
> My answer is yes.
> 
> > 
> > > Therefore, you
> > > can just go ahead with preparing the core patches without the need of
> > > waiting for my test result. It seems that your approach is not going to
> > > change the current fixup effects in a significant way, so it should be
> > > fine.
> > 
> > [...]
> > 
> > Thank you for your help !
> 
> Thanks,
> Rong