Re: [PATCH] x86/xen/mmu: Increase MAX_CONTIG_ORDER

[Jan Beulich <jbeulich@suse.com>]

On 11.12.2024 19:20, Thierry Escande wrote:
> Hi Jan,
> 
> On 09/12/2024 11:04, Jan Beulich wrote:
>> On 04.12.2024 18:14, Thierry Escande wrote:
>>> With change 9f40ec84a797 (xen/swiotlb: add alignment check for dma
>>> buffers), the driver mpt3sas fails to load because it cannot allocate
>>> its DMA pool for an allocation size of ~2,3 MBytes. This is because the
>>> alignement check added by 9f40ec84a797 fails and
>>> xen_swiotlb_alloc_coherent() ends up calling
>>> xen_create_contiguous_region() with a size order of 10 which is too high
>>> for the current max value.
>>>
>>> This patch increases the MAX_CONTIG_ORDER from 9 to 10 (4MB) to allow
>>> such allocations.
>>>
>>> Signed-off-by: Thierry Escande <thierry.escande@vates.tech>
>>> ---
>>>  arch/x86/xen/mmu_pv.c | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
>>> index 55a4996d0c04..7f110740e1a2 100644
>>> --- a/arch/x86/xen/mmu_pv.c
>>> +++ b/arch/x86/xen/mmu_pv.c
>>> @@ -2200,7 +2200,7 @@ void __init xen_init_mmu_ops(void)
>>>  }
>>>  
>>>  /* Protected by xen_reservation_lock. */
>>> -#define MAX_CONTIG_ORDER 9 /* 2MB */
>>> +#define MAX_CONTIG_ORDER 10 /* 4MB */
>>>  static unsigned long discontig_frames[1<<MAX_CONTIG_ORDER];
>>
>> While lacking respective commentary, bumping this value imo also needs to
>> take into account Xen itself, at least commit-message-wise. The bumping is
>> fine for Dom0 in any event. It is also fine for DomU-s with the defaults
>> built into the hypervisor (orders 12 and 10 respectively for x86 and Arm),
>> yet especially for Arm (and in the future PPC and RISC-V) any further
>> bumping would be less straightforward.
> 
> Thanks for pointing this out. On the Xen side, CONFIG_CTLDOM_MAX_ORDER
> and CONFIG_HWDOM_MAX_ORDER seem big enough on all architectures. But I
> see CONFIG_DOMU_MAX_ORDER set to 9 (also all archs). Won't that be a
> problem for drivers trying to allocate more than that from a domU ?

A driver assumes a (physical) device to be in the DomU, at which point it
is CONFIG_PTDOM_MAX_ORDER which applies (PT standing for pass-through).

>> However - does the driver really need this big a contiguous chunk? It
>> would seem far more desirable to me to break that up some, if possible.
> 
> Since this works on bare metal I'm afraid the driver maintainer (mpt
> fusion driver) will just tell me to fix Xen.

Well. The bigger such allocations, the larger the risk that on systems
that have been up for a while such allocations can't be fulfilled anymore
even in the bare metal case.

Jan