Re: Using remap_pfn_range causes system hang on app close in 2.6.15 & up

[Nick Piggin <nickpiggin@yahoo.com.au>]

Sam Abu-Nassar wrote:
> Hello,
> 
> I posted this query a couple of weeks ago regarding a problem with 
> remap_pfn_range.  I was able to resolve the issue and I thought I would 
> post my findings in case it helps someone else or results in a kernel 
> fix.  I will try to keep this short.
> 
> In a nutshell, my driver support user APIs that maps system RAM or a PCI 
> BAR space to user space.  What I did not mention in my original post is 
> that my driver performed a sort of custom protocol when mmap() is 
> called.  Since mmap() really only provides a limited amount of space 
> (the offset field) that I can use to pass additional information to my 
> driver, I had implemented a custom protocol, which works as follows:
> 
> 1.  API calls mmap to obtain a user virtual address
> 2.  Drivers mmap routine stores the VMA in an internal list and returns ok.
> 3.  API then issues a custom IOCTL to driver to complete mapping with 
> additional info
> 4.  Driver retrieves VMA from internal list and performs mapping with 
> io/remap_pfn_range, depending upon whether it's to system RAM or a PCI BAR.
> 
> The mappings always work fine, but starting with 2.6.15, the system 
> freeezes when the file descriptor is closed.  I tried numerous tests and 
> compared my code with existing drivers, such as /dev/mem.  Here is what 
> I found:
> 
> The fix involved moving my calls to io/remap_pfn_range into my 
> Dispatch_mmap() routine.  Once I did this, the system no longer 
> crashed.  I still implement sending some custom information to the 
> driver, but now I use special values in the offset field, remembering 
> that the offset is eventually shifted by PAGE_SIZE by the time it 
> reaches the driver.  My driver code essentially did not change.  In 
> effect, all I really did was move it to the driver's mmap() routine.
> 
> I should mention that my original protocol has worked fine in kernels 
> 2.2, 2.4, and up to 2.6.14.  Some change to the VM subsystem in 2.6.15 
> broke my original code.  I don't believe there should be an issue with 
> calling remap_pfn_range outside of the driver's mmap() routine, but I am 
> not a kernel developer, so I could be wrong in my assumption.  One of my 
> customers posed this question to Nick Piggin, and he seemed to think 
> there should not be a problem with this.

Well, I think I said it shouldn't oops like this... I don't think it
is particularly robust WRT error cases or concurrent page faults
(between mmap and ioctl).

As we established earlier with a debug patch, the reason for the oops
is that VM_PFNMAP has been cleared from your vma->vm_flags for some
reason. This is causing the unmap code to mistakenly try to remove
reverse maps and refcounts from the struct pages.

I don't know why VM_PFNMAP should be getting cleared. But if it
remains set then the oops should go away.

-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com