Re: Metalanguage for the Linux UAPI

["H. Peter Anvin" <hpa@zytor.com>]

On 5/15/25 13:26, enh wrote:
> On Thu, May 15, 2025 at 4:05 PM H. Peter Anvin <hpa@zytor.com> wrote:
>>
>> OK, so this is something I have been thinking about for quite a while.
>> It would be a quite large project, so I would like to hear people's
>> opinions on it before even starting.
>>
>> We have finally succeeded in divorcing the Linux UAPI from the general
>> kernel headers, but even so, there are a lot of things in the UAPI that
>> means it is not possible for an arbitrary libc to use it directly; for
>> example "struct termios" is not the glibc "struct termios", but
>> redefining it breaks the ioctl numbering unless the ioctl headers are
>> changed as well, and so on. However, other libcs want to use the struct
>> termios as defined in the kernel, or, more likely, struct termios2.
> 
> bionic is a ("the only"?) libc that tries to not duplicate _anything_
> and always defer to the uapi headers. we have quite an extensive list
> of hacks we need to apply to rewrite the uapi headers into something
> directly usable (and a lot of awful python to apply those hacks):
> 
> https://cs.android.com/android/platform/superproject/main/+/main:bionic/libc/kernel/tools/defaults.py
> 

Not "the only".

> a lot are just name collisions ("you say 'class', my c++ compiler says
> wtf?!"), but there are a few "posix and linux disagree"s too. (other
> libcs that weren't linux-only from day one might have more conflicts,
> such as a comically large sigset_t, say :-) )
> 
> but i think most if not all of that could be fixed upstream, given the will?
> 
> (though some c programmers do still get upset if told they shouldn't
> use c++ keywords as identifiers, i note that the uapi headers _were_
> recently fixed to avoid a c extension that's invalid c++. thanks,
> anyone involved in that who's reading this!)
> 
>> Furthermore, I was looking further into how C++ templates could be used
>> to make user pointers inherently safe and probably more efficient, but
>> ran into the problem that you really want to be able to convert a
>> user-tagged structure to a structure with "safe-user-tagged" members
>> (after access_ok), which turned out not to be trivially supportable even
>> after the latest C++ modernizations (without which I don't consider C++
>> viable at all; I would not consider versions of C++ before C++17 worthy
>> of even looking at; C++20 preferred.)
> 
> (/me assumes you're just trolling linus with this.)

I'm not; I posted a long article about why I think it might be an 
alternative worth pursuing. I know, of course, Linus' long time hatred 
of C++, but as I said: I think *very recent* versions of C++ have a lot 
to offer, mainly in the form of metaprogramming (which we currently do 
using some amazingly ugly macros.)

https://lore.kernel.org/lkml/3465e0c6-f5b2-4c42-95eb-29361481f805@zytor.com

>> And it is not just generation of in-kernel versus out-of-kernel headers
>> that is an issue (which we have managed to deal with pretty well.) There
>> generally isn't enough information in C headers alone to do well at
>> creating bindings for other languages, *especially* given how many
>> constants are defined in terms of macros.
> 
> (yeah, while i think the _c_ [and c++] problems could be solved much
> more easily, solving the swift/rust/golang duplication of all that
> stuff is a whole other thing. i'd try to sign up one of those
> languages' library's maintainers before investing too much in having
> another representation of the uapi though...)

Yes, that's one of the reasons for posting this.

>> The use of C also makes it hard to mangle the headers for user space.
>> For example, glibc has to add __extension__ before anonymous struct or
>> union members in order to be able to compile in strict C90 mode.
> 
> (again, that one seems easily fixable upstream.)

Agreed... until it breaks again. And how much

>> I have been considering if it would make sense to create more of a
>> metalanguage for the Linux UAPI. This would be run through a more
>> advanced preprocessor than cpp written in C and yacc/bison. (It could
>> also be done via a gcc plugin or a DWARF parser, but I do not like tying
>> this to compiler internals, and DWARF parsing is probably more complex
>> and less versatile.)
>>
>> It could thus provide things like "true" constants (constexpr for C++11
>> or C23, or enums), bitfield macro explosions and so on, depending on
>> what the backend user would like: namespacing, distributed enumerations,
>> and assembly offset constants, and even possibly syscall stubs.
> 
> (given a clean slate that wouldn't be terrible, but you get a lot of
> #if nonsense. though the `#define foo foo` trick lets you have the
> best of both worlds [at some cost to compile time].)

Again, that would be a choice for the data consumer (backend), which is 
one of the main advantages here.

	-hpa