Re: [PATCH] drm/ast: Fix big-endian support

[Thomas Zimmermann <tzimmermann@suse.de>]

Hi

Am 10.12.25 um 17:56 schrieb René Rebe:
> Hi,
>
> On Wed, 10 Dec 2025 16:41:50 +0100, Thomas Zimmermann <tzimmermann@suse.de> wrote:
>
>> Hi
>>
>> Am 10.12.25 um 16:33 schrieb René Rebe:
>>> Hallo,
>>>
>>> On Wed, 10 Dec 2025 09:55:50 +0100, Thomas Zimmermann
>>> <tzimmermann@suse.de> wrote:
>>> ...
>>>>>> Does that fix the color corruption?
>>>>> Following your suggestions conversion does not want to just work:
>>>>>
>>>>> root@XCODE_SPARC_T4_1:~# dmesg  | tail
>>>>> [  105.444761] ast 0000:0a:00.0: AST 2200 detected
>>>>> [ 105.444947] ast 0000:0a:00.0: [drm] dram MCLK=266 Mhz type=2
>>>>> bus_width=32
>>>>> [  105.444963] ast 0000:0a:00.0: [drm] Using analog VGA
>>>>> [  105.445470] [drm] Initialized ast 0.1.0 for 0000:0a:00.0 on minor 0
>>>>> [ 105.673289] ast 0000:0a:00.0: [drm] format BX24 little-endian
>>>>> (0x34325842) not supported
>>>>> [  105.673302] ast 0000:0a:00.0: [drm] No compatible format found
>>>>> [ 105.673348] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
>>>>> emulation (ret=-22)
>>>>> [ 105.901306] ast 0000:0a:00.0: [drm] format BX24 little-endian
>>>>> (0x34325842) not supported
>>>>> [  105.901319] ast 0000:0a:00.0: [drm] No compatible format found
>>>>> [ 105.901350] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
>>>>> emulation (ret=-22)
>>>> Oh well...
>>>>
>>>> There's a very simple patch attach. Does it fix the problem?
>>> Yes, only leaving the hardcoded swapping from my patch liek this fixes
>>> the byte-swapped output as expected on the sparc64 Sun T4.
>> Great.
>>
>>> How would you like me to go from here? Just use the chip_id to force
>>> swapping and enable hw swapper for pre-AST2400 chips or fix the
>>> generic format selection to work as you had suggested?
>>>
>>> Does the ast_primary_plane_formats need to byte swapped formats for it
>>> to work?
>> I'll send out a full patch that implements the byte swapping. Once
>> reviewed, it can be merged quickly. Can I add your Tested-by tag to
>> the patch?
> I'd be happy to finish my work.

Of course, no problem.

The code for the primary plane should be fine now. But we also need 
something for the cursor plane as well. There's a ast_set_cursor_image() 
with a memcpy_toio() [1] and several additional writes. IIUC they all 
have to be swapped as well.

[1] 
https://gitlab.freedesktop.org/drm/misc/kernel/-/blob/drm-misc-next/drivers/gpu/drm/ast/ast_cursor.c#L101

Best regards
Thomas

>   But if you want to put the last touch
> on it now you can add Co-developed-by, too ... and I'll test the final
> version.
>
>     René
>

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)