Re: arm64: juno-r2: SSD detect failed on mainline and next

[Robin Murphy <robin.murphy@arm.com>]

On 11/04/2025 8:11 pm, Robin Murphy wrote:
[...]
> OK, I found it, but I'm still not sure what exactly to make of it - it's 
> the pci_request_acs() in of_iommu_configure(), now being called early 
> enough to actually have an effect. Booting with EDK2 already using PCI 
> prior to Linux, here's what I get for `sudo lspci -vv | grep ACSctl` 
> with 6.15-rc1:
> 
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
>          ACSCtl:    SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ 
> UpstreamFwd+ EgressCtrl- DirectTrans-
> 
> whereas with the 6.14 behaviour they are all '-'. I don't have a working 
> root filesystem with the U-Boot setup, but if I boot it with 
> "pci=config_acs=000000@pci:0:0" then the kernel does assign the bridge 
> windows and discover the ethernet/SATA endpoints again. I can spend some 
> time getting NFS working next week, but if you're able to get lspci 
> output off a machine in the "broken" state easily that would be handy to 
> compare.
> 
> So at this point it would seem to be something about how Linux 
> configures ACS when doing it from scratch. What I don't really know is 
> where to go from there. I do know Juno's possibly a bit odd in that the 
> switch supports ACS, but both the root port and endpoints either side of 
> it don't. Could this be tickling some subtle bug in the PCI layer, and 
> what is EDK2 doing that makes it not happen?

Just following up on where I ran out of ideas. I poked at things a 
little more, and from a process of elimination, the culprit appears to 
be is that we enable ACS source validation on the downstream port while 
its secondary bus is still 0, *then* we get to the "bridge configuration 
invalid" bit and reconfigure the bus numbers, but after that, config 
space accesses to the secondary bus still apparently fail to work as 
expected.

What's now beyond me is whether this is just an ACS quirk of this 
particular switch, and/or whether there's something we could or should 
be doing in the PCI layer.

All I can suggest a this point is that you could at least sidestep the 
problem on the LKFT boards by updating them to a less-ancient version of 
U-Boot which supports PCIe for Juno (looks like that landed in 2020.10), 
which should then configure the switch at boot such that the bus 
numbering doesn't need to change when Linux probes it - that appears to 
be the only "magic" thing that EDK2 is doing.

Thanks,
Robin.