Re: [PATCH RFC 0/2] PCI: rockchip-host: Retry training and hot-reset a JMB585 on RK3399

[Igor Velkov <iav@iav.lv>]

Hi Shawn,

I traced the LTSSM on the Helios64 (RK3399 + JMB585, x2, 2.5 GT/s) by
polling PCIE_CLIENT_DEBUG_OUT_0 about every 2 us during link training
and the waits after it. This changes what the RFC said, so here are the
data and a question.

A failing link bring-up looks like this:

- the LTSSM reaches L0 as fast as in a good one;
- then it leaves L0 for Recovery (RcvrLock/RcvrCfg/Idle, about 5 us)
  every 199..204 us, for as long as I watch: 488 times per 100 ms;
- link_st in BASIC_STATUS1 becomes 3 only 15..47 ms after training
  starts, or not within 700 ms; the latter is the "link training gen1
  timeout" with LTSSM 0x10;
- the RC shows nothing: AER and the device status are clear, and a dump
  of the client, core management and root port config registers is
  identical to a good bring-up.

A good bring-up has no Recovery entry after L0. A config read on a bad
link gets no completion; that is the abort I reported. 200 us is the
optional Flow Control Update timeout, so I suspect that one side does
not see the DLLPs of the other.

The state survives Recovery but not Detect: a port re-init or a hot
reset is just one more bring-up, with the same odds. So the hot reset
of patch 2/2 does not cure anything, and the time the link takes to
return is not a reliable sign either.

Unbind/bind cycles with a bad bring-up (a cycle is one training plus
one hot reset):

  Helios64, JMB585, x2                     6 of 104
  same, PHY refclk 100 MHz from NPLL       4 of 33
  same, num-lanes = <1>                    0 of 104
  ROCK Pi 4A, WD SN770 NVMe, x4            0 of 100
  same, num-lanes = <2>                    0 of 100

So it takes both lanes and the JMB585.

Is anything like this known for the RK3399 with a two-lane endpoint,
for example lane-to-lane skew of the transmitter, or a PHY setting that
affects it? And is there a bit in DEBUG_OUT_0/1 that tells which side
directs the Recovery?

Both boards ran the same build: v7.3.1 with the Armbian patches and the
tracing. The Helios64 DT has no ep-gpios there, the ROCK Pi 4A DT has.

Thanks,
Igor