[PATCH v2 0/1] drm/ttm: allocate dummy_read_page without DMA32 on fail

[PATCH v2 0/1] drm/ttm: allocate dummy_read_page without DMA32 on fail

[Yangyu Chen]

Some platforms may not have any memory below 4GB address space, but the
kernel defines ZONE_DMA32 on their ISA. Thus, these platforms will have
an empty DMA32 zone, resulting ttm failing when alloc_page with GFP_DMA32
flag. However, we can't directly allocate dummy_read_page without
GFP_DMA32 as some reasons mentioned in the previous patch review [1].

Thus, a solution is to allocate dummy_read_page with GFP_DMA32 first,
if it fails, then allocate it without GFP_DMA32. After this patch, the
amdgpu works on such platforms.

Here is dmesg output on such RISC-V platforms with Radeon RX550 after this
patch:

[    0.000000] Linux version 6.7.0-00001-gd90146c47100-dirty (cyy@cyy-pc) (riscv64-linux-gnu-gcc (Debian 13.2.0-7) 13.2.0, GNU ld (GNU Binutils for Debian) 2.41.50.20231227) #13 SMP Wed Jan 17 02:35:17 CST 2024
[    0.000000] Machine model: 
[    0.000000] SBI specification v2.0 detected
[    0.000000] SBI implementation ID=0x1 Version=0x10004
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] efi: UEFI not found.
[    0.000000] OF: reserved mem: 0x0000002000000000..0x000000200003ffff (256 KiB) nomap non-reusable mmode_resv1@20,0
[    0.000000] OF: reserved mem: 0x0000002000040000..0x000000200005ffff (128 KiB) nomap non-reusable mmode_resv0@20,40000
[    0.000000] Zone ranges:
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000002000000000-0x00000021ffffffff]
...
[   36.425400] [drm] amdgpu kernel modesetting enabled.
[   36.430695] [drm] initializing kernel modesetting (POLARIS12 0x1002:0x699F 0x1043:0x0513 0xC7).
[   36.439436] [drm] register mmio base: 0xA8100000
[   36.444055] [drm] register mmio size: 262144
[   36.448462] [drm] add ip block number 0 <vi_common>
[   36.453348] [drm] add ip block number 1 <gmc_v8_0>
[   36.458150] [drm] add ip block number 2 <tonga_ih>
[   36.458153] [drm] add ip block number 3 <gfx_v8_0>
[   36.458155] [drm] add ip block number 4 <sdma_v3_0>
[   36.458157] [drm] add ip block number 5 <powerplay>
[   36.477491] [drm] add ip block number 6 <dm>
[   36.481764] [drm] add ip block number 7 <uvd_v6_0>
[   36.491409] [drm] add ip block number 8 <vce_v3_0>
[   36.703765] amdgpu 0000:05:00.0: amdgpu: Fetched VBIOS from ROM BAR
[   36.710051] amdgpu: ATOM BIOS: 115-C994PI2-100
[   36.716023] [drm] UVD is enabled in VM mode
[   36.720242] [drm] UVD ENC is enabled in VM mode
[   36.724789] [drm] VCE enabled in VM mode
[   36.728724] amdgpu 0000:05:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported
[   36.728735] amdgpu 0000:05:00.0: amdgpu: PCIE atomic ops is not supported
[   36.743620] [drm] GPU posting now...
[   36.858108] [drm] vm size is 64 GB, 2 levels, block size is 10-bit, fragment size is 9-bit
[   36.867392] amdgpu 0000:05:00.0: amdgpu: VRAM: 4096M 0x000000F400000000 - 0x000000F4FFFFFFFF (4096M used)
[   36.876980] amdgpu 0000:05:00.0: amdgpu: GART: 256M 0x000000FF00000000 - 0x000000FF0FFFFFFF
[   36.885347] [drm] Detected VRAM RAM=4096M, BAR=256M
[   36.890228] [drm] RAM width 128bits GDDR5
[   36.894289] [TTM DEVICE] Using GFP_DMA32 fallback for dummy_read_page
[   36.900907] [drm] amdgpu: 4096M of VRAM memory ready
[   36.905896] [drm] amdgpu: 4007M of GTT memory ready.
[   36.910928] [drm] GART: num cpu pages 65536, num gpu pages 65536
[   36.918185] [drm] PCIE GART of 256M enabled (table at 0x000000F400000000).
[   36.926847] [drm] Chained IB support enabled!
[   36.935727] amdgpu: hwmgr_sw_init smu backed is polaris10_smu
[   36.947466] [drm] Found UVD firmware Version: 1.130 Family ID: 16
[   36.976989] [drm] Found VCE firmware Version: 53.26 Binary ID: 3
[   37.329484] [drm] Display Core v3.2.259 initialized on DCE 11.2
[   37.390981] [drm] UVD and UVD ENC initialized successfully.
[   37.497639] [drm] VCE initialized successfully.
[   37.502935] amdgpu 0000:05:00.0: amdgpu: SE 2, SH per SE 1, CU per SH 5, active_cu_number 8
[   37.516199] amdgpu 0000:05:00.0: amdgpu: Using BACO for runtime pm
[   37.523381] [drm] Initialized amdgpu 3.56.0 20150101 for 0000:05:00.0 on minor 0
[   37.592040] Console: switching to colour frame buffer device 160x45
[   37.614276] amdgpu 0000:05:00.0: [drm] fb0: amdgpudrmfb frame buffer device

[1]. https://lore.kernel.org/lkml/2b715134-9d63-4de1-94e5-37e180aeefd2@amd.com/

v1: https://lore.kernel.org/lkml/tencent_40DF99E09A3681E339EE570C430878232106@qq.com/

changes since v1:
- Add __GFP_NOWARN on first alloc_page to avoid warning on such platforms
- Place comment on the top of the if
- Shorter warning message

Yangyu Chen (1):
  drm/ttm: allocate dummy_read_page without DMA32 on fail

 drivers/gpu/drm/ttm/ttm_device.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

-- 
2.43.0

[PATCH v2 1/1] drm/ttm: allocate dummy_read_page without DMA32 on fail

[Yangyu Chen]

Some platforms may not have any memory in ZONE_DMA32 and use IOMMU to allow
32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page will
fail on such platforms. Retry after fail will get this works on such
platforms.

Signed-off-by: Yangyu Chen <cyy@cyyself.name>
---
 drivers/gpu/drm/ttm/ttm_device.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c
index d48b39132b32..c9fa8561f71f 100644
--- a/drivers/gpu/drm/ttm/ttm_device.c
+++ b/drivers/gpu/drm/ttm/ttm_device.c
@@ -95,11 +95,17 @@ static int ttm_global_init(void)
 	ttm_pool_mgr_init(num_pages);
 	ttm_tt_mgr_init(num_pages, num_dma32);
 
-	glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32);
+	glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32 |
+					   __GFP_NOWARN);
 
+	/* Retry without GFP_DMA32 for platforms DMA32 is not available */
 	if (unlikely(glob->dummy_read_page == NULL)) {
-		ret = -ENOMEM;
-		goto out;
+		glob->dummy_read_page = alloc_page(__GFP_ZERO);
+		if (unlikely(glob->dummy_read_page == NULL)) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		pr_warn("Using GFP_DMA32 fallback for dummy_read_page\n");
 	}
 
 	INIT_LIST_HEAD(&glob->device_list);
-- 
2.43.0

Re: [PATCH v2 1/1] drm/ttm: allocate dummy_read_page without DMA32 on fail

[Christian König]

Am 16.01.24 um 19:50 schrieb Yangyu Chen:
> Some platforms may not have any memory in ZONE_DMA32 and use IOMMU to allow
> 32-bit-DMA-only device to work. Forcing GFP_DMA32 on dummy_read_page will
> fail on such platforms. Retry after fail will get this works on such
> platforms.
>
> Signed-off-by: Yangyu Chen <cyy@cyyself.name>

Reviewed and pushed to drm-misc-fixes. That patch should show up in the 
next rc and stable kernels next week or so.

Thanks,
Christian.


> ---
>   drivers/gpu/drm/ttm/ttm_device.c | 12 +++++++++---
>   1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c
> index d48b39132b32..c9fa8561f71f 100644
> --- a/drivers/gpu/drm/ttm/ttm_device.c
> +++ b/drivers/gpu/drm/ttm/ttm_device.c
> @@ -95,11 +95,17 @@ static int ttm_global_init(void)
>   	ttm_pool_mgr_init(num_pages);
>   	ttm_tt_mgr_init(num_pages, num_dma32);
>   
> -	glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32);
> +	glob->dummy_read_page = alloc_page(__GFP_ZERO | GFP_DMA32 |
> +					   __GFP_NOWARN);
>   
> +	/* Retry without GFP_DMA32 for platforms DMA32 is not available */
>   	if (unlikely(glob->dummy_read_page == NULL)) {
> -		ret = -ENOMEM;
> -		goto out;
> +		glob->dummy_read_page = alloc_page(__GFP_ZERO);
> +		if (unlikely(glob->dummy_read_page == NULL)) {
> +			ret = -ENOMEM;
> +			goto out;
> +		}
> +		pr_warn("Using GFP_DMA32 fallback for dummy_read_page\n");
>   	}
>   
>   	INIT_LIST_HEAD(&glob->device_list);