/dev/mem and PCI memory = EFAULT?

/dev/mem and PCI memory = EFAULT?

[Stas Sergeev]

Hello.

For the long time dosemu used /dev/mem for vga pass-through.
Now it appears /dev/mem has this check:
http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
which prevents an accesses to PCI memory regions if the
"high_memory" points low enough. It seems "high_memory"
just points to the end of the physical ram, so depending on
the ram size you either can access PCI devices or you get
EFAULT.
Was it wrong to use /dev/mem for accessing the PCI devices?
How should I do that now?

Note: I haven't properly tested that case. I can see the problem
only under qemu, because all my machines has enough ram for
the things to work. Still the failure under qemu makes me wonder
if I should stop using /dev/mem for accessing PCI memory.

Re: /dev/mem and PCI memory = EFAULT?

[Andy Lutomirski]

On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
> Hello.
>
> For the long time dosemu used /dev/mem for vga pass-through.
> Now it appears /dev/mem has this check:
> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
> which prevents an accesses to PCI memory regions if the
> "high_memory" points low enough. It seems "high_memory"
> just points to the end of the physical ram, so depending on
> the ram size you either can access PCI devices or you get
> EFAULT.
> Was it wrong to use /dev/mem for accessing the PCI devices?
> How should I do that now?
>

What is DOSEMU trying to do here?  Access the framebuffer?

ISTM it would be better to use the DRM or FB layer directly (just map
the framebuffer itself) or, if necessary, use VFIO.

Re: /dev/mem and PCI memory = EFAULT?

[Stas Sergeev]

29.10.2016 00:05, Andy Lutomirski пишет:
> On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>> Hello.
>>
>> For the long time dosemu used /dev/mem for vga pass-through.
>> Now it appears /dev/mem has this check:
>> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
>> which prevents an accesses to PCI memory regions if the
>> "high_memory" points low enough. It seems "high_memory"
>> just points to the end of the physical ram, so depending on
>> the ram size you either can access PCI devices or you get
>> EFAULT.
>> Was it wrong to use /dev/mem for accessing the PCI devices?
>> How should I do that now?
>>
> What is DOSEMU trying to do here?  Access the framebuffer?
>
> ISTM it would be better to use the DRM or FB layer directly (just map
> the framebuffer itself) or, if necessary, use VFIO.
Yes, framebuffer.
Mapping fb directly is not really an option because dosemu does
its own modesetting when you do vga pass-through. So it is
usually started that way with "nomodeset=1" and w/o fb.
Yes, some crazy people try the pass-through even out of fb
console, but that's weird (the problem is most SDL2 builds do
not have directfb backend compiled in, otherwise we could
just use SDL rendering on top of fb).

The thing is, I needed (for testing purposes, unrelated to dosemu)
some quick way to access the PCI memory space, and to my surprise I
couldn't do that with /dev/mem. Was this really disallowed intentionally?

Re: /dev/mem and PCI memory = EFAULT?

[Andy Lutomirski]

On Fri, Oct 28, 2016 at 2:18 PM, Stas Sergeev <stsp@list.ru> wrote:
> 29.10.2016 00:05, Andy Lutomirski пишет:
>
>> On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>
>>> Hello.
>>>
>>> For the long time dosemu used /dev/mem for vga pass-through.
>>> Now it appears /dev/mem has this check:
>>> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
>>> which prevents an accesses to PCI memory regions if the
>>> "high_memory" points low enough. It seems "high_memory"
>>> just points to the end of the physical ram, so depending on
>>> the ram size you either can access PCI devices or you get
>>> EFAULT.
>>> Was it wrong to use /dev/mem for accessing the PCI devices?
>>> How should I do that now?
>>>
>> What is DOSEMU trying to do here?  Access the framebuffer?
>>
>> ISTM it would be better to use the DRM or FB layer directly (just map
>> the framebuffer itself) or, if necessary, use VFIO.
>
> Yes, framebuffer.
> Mapping fb directly is not really an option because dosemu does
> its own modesetting when you do vga pass-through. So it is
> usually started that way with "nomodeset=1" and w/o fb.
> Yes, some crazy people try the pass-through even out of fb
> console, but that's weird (the problem is most SDL2 builds do
> not have directfb backend compiled in, otherwise we could
> just use SDL rendering on top of fb).
>
> The thing is, I needed (for testing purposes, unrelated to dosemu)
> some quick way to access the PCI memory space, and to my surprise I
> couldn't do that with /dev/mem. Was this really disallowed intentionally?

I believe so.

Try the /sys/devices/.../resource? and resource?_wc files.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: /dev/mem and PCI memory = EFAULT (regression?)

[Stas Sergeev]

OK, thanks for the prompt reply, Andy!
I'll try your sysfs suggestion.

Let me just add a bit of CCs to get more people involved.
I believe the ram size involvement makes this smell like a bug,
plus it looks like a regression. Not that I am going to suffer
much (this pass-through stuff always breaks in a million of
different ways, I'll probably just remove it), but its still a very
questionable change IMO.

29.10.2016 00:26, Andy Lutomirski пишет:
> On Fri, Oct 28, 2016 at 2:18 PM, Stas Sergeev <stsp@list.ru> wrote:
>> 29.10.2016 00:05, Andy Lutomirski пишет:
>>
>>> On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>> Hello.
>>>>
>>>> For the long time dosemu used /dev/mem for vga pass-through.
>>>> Now it appears /dev/mem has this check:
>>>> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
>>>> which prevents an accesses to PCI memory regions if the
>>>> "high_memory" points low enough. It seems "high_memory"
>>>> just points to the end of the physical ram, so depending on
>>>> the ram size you either can access PCI devices or you get
>>>> EFAULT.
>>>> Was it wrong to use /dev/mem for accessing the PCI devices?
>>>> How should I do that now?
>>>>
>>> What is DOSEMU trying to do here?  Access the framebuffer?
>>>
>>> ISTM it would be better to use the DRM or FB layer directly (just map
>>> the framebuffer itself) or, if necessary, use VFIO.
>> Yes, framebuffer.
>> Mapping fb directly is not really an option because dosemu does
>> its own modesetting when you do vga pass-through. So it is
>> usually started that way with "nomodeset=1" and w/o fb.
>> Yes, some crazy people try the pass-through even out of fb
>> console, but that's weird (the problem is most SDL2 builds do
>> not have directfb backend compiled in, otherwise we could
>> just use SDL rendering on top of fb).
>>
>> The thing is, I needed (for testing purposes, unrelated to dosemu)
>> some quick way to access the PCI memory space, and to my surprise I
>> couldn't do that with /dev/mem. Was this really disallowed intentionally?
> I believe so.
>
> Try the /sys/devices/.../resource? and resource?_wc files.
>
> --Andy
>

Re: /dev/mem and PCI memory = EFAULT (regression?)

[Linus Torvalds]

On Fri, Oct 28, 2016 at 2:36 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>> On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>
> Hello.
>
> For the long time dosemu used /dev/mem for vga pass-through.
> Now it appears /dev/mem has this check:
> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
> which prevents an accesses to PCI memory regions if the
> "high_memory" points low enough. It seems "high_memory"
> just points to the end of the physical ram, so depending on
> the ram size you either can access PCI devices or you get
> EFAULT.
> Was it wrong to use /dev/mem for accessing the PCI devices?
> How should I do that now?

Has it ever worked for you? That code is ancient, going back in some
form or another at least ten years.

So /dev/mem does not allow "read()/write()" on IO memory, and really
hasn't in a long long time (maybe ever, quite frankly). It does allow
it on regular RAM, but STRICT_DEV_MEM then disallows that too for
security reasons (and realistically, everybody uses STRICT_DEV_MEM
these days).

What people do use /dev/mem for is to mmap() PCI memory, and then you
can access it from user space. That's the traditional model that X.org
used to do etc.

I'm surprised if there is a regression here, because I think
read/write really hasn't been supported in like forever. It's
fundamentally impossible to do on some architectures: the physical
access depends on the _size_ of the access on at least some alpha
CPU's, so you can't just do a random access to PCI address X, you need
to have a real size etc.

But if you can point to a particular commit that broke something (eg
with bisection), I'll certainly take a second look. I might have
missed something.

               Linus

Re: /dev/mem and PCI memory = EFAULT (regression?)

[Stas Sergeev]

29.10.2016 02:26, Linus Torvalds пишет:
> On Fri, Oct 28, 2016 at 2:36 PM, Stas Sergeev <stsp@list.ru> wrote:
>>>>> On Fri, Oct 28, 2016 at 2:03 PM, Stas Sergeev <stsp@list.ru> wrote:
>> Hello.
>>
>> For the long time dosemu used /dev/mem for vga pass-through.
>> Now it appears /dev/mem has this check:
>> http://lxr.free-electrons.com/source/drivers/char/mem.c#L51
>> which prevents an accesses to PCI memory regions if the
>> "high_memory" points low enough. It seems "high_memory"
>> just points to the end of the physical ram, so depending on
>> the ram size you either can access PCI devices or you get
>> EFAULT.
>> Was it wrong to use /dev/mem for accessing the PCI devices?
>> How should I do that now?
> Has it ever worked for you?
Well, yes, even today: if the ram size is large enough to last
beyond the PCI space, then I am not getting EFAULT (i've yet
to check if the results are as expected, but at least no error
returned).
It is exactly that ram size envolvement made me think this
is a bug (and regression).

>   That code is ancient, going back in some
> form or another at least ten years.
Yes, indeed, I can see that on lxr now...

> So /dev/mem does not allow "read()/write()" on IO memory, and really
> hasn't in a long long time (maybe ever, quite frankly). It does allow
> it on regular RAM, but STRICT_DEV_MEM then disallows that too for
> security reasons (and realistically, everybody uses STRICT_DEV_MEM
> these days).
>
> What people do use /dev/mem for is to mmap() PCI memory, and then you
> can access it from user space. That's the traditional model that X.org
> used to do etc.
Yes, thanks, I was confused. mmap() is indeed unaffected,
only read and write are. So the only problem I have, is the test-case
I was trying, that does read/write and works depending on the
ram size (under qemu). But there is no any problem for dosemu.
Perhaps for consistency it would be good to make things independent
of the ram size, but then there is no real problem here.
So sorry for the noise!