Re: [PATCH 2/4 v2] cxl/core: Add helpers to detect Low memory Holes on x86

[Dan Williams <dan.j.williams@intel.com>]

Gregory Price wrote:
> On Tue, Jan 14, 2025 at 09:32:54PM +0100, Fabio M. De Francesco wrote:
> > +/*
> > + * Match CXL Root and Endpoint Decoders by comparing SPA and HPA ranges.
> > + *
> > + * On x86, CFMWS ranges never intersect memory holes while endpoint decoders
> > + * HPA range sizes are always guaranteed aligned to NIW * 256MB; therefore,
> > + * the given endpoint decoder HPA range size is always expected aligned and
> > + * also larger than that of the matching root decoder. If there are LMH's,
> > + * the root decoder range end is always less than SZ_4G.
> > + */
> 
> Is there any reason to limit this memory-hole handling to only low
> memory holes? I have observed systems where the following memory
> hole situation occurs:
> 
> (example, not exact sizes)
> CFMW1:   [  0xc0000000 - 0xdfffffff  ] 512MB range
> Reserved [  0xe0000000 - 0xffffffff  ] 512MB range
> CFMW2:   [ 0x100000000 - 0x15fffffff ] 1.5GB range
> 
> 2 CXL Memory Devices w/ 1GB capacity each (again, an example).
> 
> Note that 1 device has its capacity split across the hole, but
> if the devices are interleaved then both devices have their capacity
> split across the hole.
> 
> It seems with some mild modification, this patch set could be
> re-used to handle this memory hole scenario as well
> (w/ addr translation - Robert's patch set)
> 
> Is there a reason not to handle more than just LMH's in this set?

This discussion was referenced recently on an IM and I wanted to share
my response to it:

The rules for when to apply this memory hole quirk are explicit and
suitable to add to the CXL specification. I want the same standard for
any other quirk and ideally some proof-of-work to get that quirk
recognized by the specification. Otherwise, I worry that generalizing
for all the possible ways that platform BIOS tries to be clever means we
end up with something that has no rules.

The spec is there to allow software to delineate valid configurations vs
mistakes, and this slow drip of "Linux does not understand this platform
configuration" is a spec gap.