Re: Old O_DIRECT story

[One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>]

> I tried in all ways to implement this with mmap(), but it does not success,
> because I did not find a way to mmap() file as O_WRONLY. Mapping as O_RDWR
> makes kernel to pre-fill mapped memory with partition data. So, kernel and
> DMA actually compete on the RAM area to fill it - one with garbage, one
> with actual data. Kernel wins.
> 
> So, how to implement Linus's advice?

Use O_DIRECT. There are lots of problems with the mmap() model, in
particular with how mmu table changes scale to large numbers of CPU
threads (ie they don't).

You would need to modify the kernel to add an madvise type of
soemthing like ZEROFILL (you can't do WRONLY because the x86 CPU can't
really do write only) but then you'd be stuck with only running on the
latest and greatest kernel. A ZEROFILL madvise would at least mean each
time you touched a new page it took a fault and did a 4K clear not a read.

mmap works well if you don't need to change the page permissions or map
new pages all the time. If you have to keep touching the MMU then it gets
ugly because you tend to need to synchronize btween cores.

Alan