Re: xen_exit_mmap() questions

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 04/27/2017 12:46 PM, Andy Lutomirski wrote:
> On Thu, Apr 27, 2017 at 6:21 AM, Boris Ostrovsky
> <boris.ostrovsky@oracle.com> wrote:
>>>>>>> Also, this code in drop_other_mm_ref() looks dubious to me:
>>>>>>>
>>>>>>>     /* If this cpu still has a stale cr3 reference, then make sure
>>>>>>>        it has been flushed. */
>>>>>>>     if (this_cpu_read(xen_current_cr3) == __pa(mm->pgd))
>>>>>>>         load_cr3(swapper_pg_dir);
>>>>>>>
>>>>>>> If cr3 hasn't been flushed to the hypervisor because we're in a lazy
>>>>>>> mode, why would load_cr3() help?  Shouldn't this be xen_mc_flush()
>>>>>>> instead?
>>>>>> load_cr3() actually ends with xen_mc_flush() by way of xen_write_cr3()
>>>>>> -> xen_mc_issue().
>>>>> xen_mc_issue() does:
>>>>>
>>>>>         if ((paravirt_get_lazy_mode() & mode) == 0)
>>>>>                 xen_mc_flush();
>>>>>
>>>>> I assume the load_cr3() is intended to deal with the case where we're
>>>>> in lazy mode, but we'll still be in lazy mode, right?  Or does it
>>>>> serve some other purpose?
>>>> Of course. I can't read (I ignored the "== 0" part).
>>>>
>>>> Apparently the early version had an explicit flush but then it disappeared
>>>> (commit 9f79991d4186089e228274196413572cc000143b).
>>>>
>>>> The point of CR3 loading here, I believe, is to make sure the hypervisor
>>>> knows that the (v)CPU is no longer using the the mm's cr3 (we are loading
>>>> swapper_pgdir here).
>>> But that's what leave_mm() does.  To be fair, the x86 lazy TLB
>>> management is a big mess, and this came up because I'm trying to clean
>>> it up without removing it.
>> True. I don't know though if you can guarantee that leave_mm() (or
>> load_cr3() inside it) is actually called if we are in lazy mode.
> The code just before that makes these calls.

Yes, and I was unsure whether we always get to make these calls, based
on mm and cpu_tlbstate. I think we do and with your changes it is made
even more clear.

>
> Anyway, I propose to rewrite the whole thing like this:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/tlbflush_cleanup&id=ff143a54bb3bafaaad6e32145a9cfbc112e8584f

Can you explain xen_pgd_free() change? When do you expect
xen_exit_mmap() to fail unpinning (compared to what we have now)?

-boris