Re: Allow data races on some read/write operations

[Ralf Jung <post@ralfj.de>]

Hi,

On 05.03.25 19:38, Andreas Hindborg wrote:
> "Ralf Jung" <post@ralfj.de> writes:
> 
>> Hi,
>>
>> On 05.03.25 04:24, Boqun Feng wrote:
>>> On Tue, Mar 04, 2025 at 12:18:28PM -0800, comex wrote:
>>>>
>>>>> On Mar 4, 2025, at 11:03 AM, Ralf Jung <post@ralfj.de> wrote:
>>>>>
>>>>> Those already exist in Rust, albeit only unstably:
>>>>> <https://doc.rust-lang.org/nightly/std/intrinsics/fn.volatile_copy_memory.html>.
>>>>> However, I am not sure how you'd even generate such a call in C? The
>>>>> standard memcpy function is not doing volatile accesses, to my
>>>>> knowledge.
>>>>
>>>> The actual memcpy symbol that exists at runtime is written in
>>>> assembly, and should be valid to treat as performing volatile
>>>> accesses.
>>
>> memcpy is often written in C... and AFAIK compilers understand what that
>> function does and will, for instance, happily eliminate the call if they can
>> prove that the destination memory is not being read from again. So, it doesn't
>> behave like a volatile access at all.
>>
>>>> But both GCC and Clang special-case the memcpy function.  For example,
>>>> if you call memcpy with a small constant as the size, the optimizer
>>>> will transform the call into one or more regular loads/stores, which
>>>> can then be optimized mostly like any other loads/stores (except for
>>>> opting out of alignment and type-based aliasing assumptions).  Even if
>>>> the call isn’t transformed, the optimizer will still make assumptions.
>>>> LLVM will automatically mark memcpy `nosync`, which makes it undefined
>>>> behavior if the function “communicate[s] (synchronize[s]) with another
>>>> thread”, including through “volatile accesses”. [1]
>>
>> The question is more,  what do clang and GCC document / guarantee in a stable
>> way regarding memcpy? I have not seen any indication so far that a memcpy call
>> would ever be considered volatile, so we have to treat it like a non-volatile
>> non-atomic operation.
>>
>>>> However, these optimizations should rarely trigger misbehavior in
>>>> practice, so I wouldn’t be surprised if Linux had some code that
>>>> expected memcpy to act volatile…
>>>>
>>>
>>> Also in this particular case we are discussing [1], it's a memcpy (from
>>> or to) a DMA buffer, which means the device can also read or write the
>>> memory, therefore the content of the memory may be altered outside the
>>> program (the kernel), so we cannot use copy_nonoverlapping() I believe.
>>>
>>> [1]: https://lore.kernel.org/rust-for-linux/87bjuil15w.fsf@kernel.org/
>>
>> Is there actually a potential for races (with reads by hardware, not other
>> threads) on the memcpy'd memory?
> 
> There is another use case for this: copying data to/from a page that is
> mapped into user space. In this case, a user space process can
> potentially modify the data in the mapped page while we are
> reading/writing that data. This would be a misbehaved user space
> process, but it should not be able to cause UB in the kernel anyway.

Yeah that sounds like *the* prototypical case of sharing memory with an 
untrusted third party.

> 
> The C kernel just calls memcpy directly for this use case.
> 
> For this use case, we do not interpret or make control flow decisions
> based on the data we read/write. And _if_ user space decides to do
> concurrent writes to the page, we don't care if the data becomes
> garbage. We just need the UB to be confined to the data moved from that
> page, and not leak into the rest of the kernel.

There is no such thing as "confined UB". Well, there is "poison data", which can 
act a bit like that, but sadly the C standard is extremely ambiguous on that 
subject and has been for decades despite repeated requests for clarifications, 
so it is entirely unclear whether and how "poison data" could exist in C. clang, 
for once, has decided that "poison data" is UB in most situations (including 
just copying it to / returning it from another function), and this is consistent 
with some of the messaging of the standards committee. I don't know enough about 
the internals of gcc to comment on what they do.

Personally, I think that's a mistake; there needs to be some clear way to deal 
with uninitialized memory (which is the typical example of "poison data").

In Rust we have a fairly clear idea of what our rules should be here, and you 
can have "poison data" inside the `MaybeUninit` type. However, neither Rust nor 
C have a way to do reads where data races cause "poison data" rather than UB. 
See my other email I just sent for the rest of this line of discussion.
(I'm not used to the sprawling tree of a discussion that is this mailing list, 
so not sure how to best deal with replies that want to "merge" things said in 
different emails.)

Kind regards,
Ralf