Re: [Xen-devel] [PATCH 1/2] xen: fix race in xen_qlock_wait()

Re: [Xen-devel] [PATCH 1/2] xen: fix race in xen_qlock_wait()

[Juergen Gross]

On 01/10/2018 10:54, Jan Beulich wrote:
>>>> On 01.10.18 at 09:16, <jgross@suse.com> wrote:
>> In the following situation a vcpu waiting for a lock might not be
>> woken up from xen_poll_irq():
>>
>> CPU 1:                CPU 2:                      CPU 3:
>> takes a spinlock
>>                       tries to get lock
>>                       -> xen_qlock_wait()
>>                         -> xen_clear_irq_pending()
> 
> Doesn't the last line above ...
> 
>> frees the lock
>> -> xen_qlock_kick(cpu2)
> 
> ... need to be below here?

You are right, of course!

Thanks for noticing.


Juergen

[PATCH 0/2] xen: fix two issues in Xen pv qspinlock handling

[Juergen Gross]

The Xen specific queue spinlock wait function has two issues which
could result in a hanging system.

They have a similar root cause of clearing a pending wakeup of a
waiting vcpu and later going to sleep waiting for the just cleared
wakeup event, which of course won't ever happen.

Juergen Gross (2):
  xen: fix race in xen_qlock_wait()
  xen: make xen_qlock_wait() nestable

 arch/x86/xen/spinlock.c | 33 ++++++++++++---------------------
 1 file changed, 12 insertions(+), 21 deletions(-)

Cc: Waiman.Long@hp.com
Cc: peterz@infradead.org

-- 
2.16.4

[PATCH 1/2] xen: fix race in xen_qlock_wait()

[Juergen Gross]

In the following situation a vcpu waiting for a lock might not be
woken up from xen_poll_irq():

CPU 1:                CPU 2:                      CPU 3:
takes a spinlock
                      tries to get lock
                      -> xen_qlock_wait()
                        -> xen_clear_irq_pending()
frees the lock
-> xen_qlock_kick(cpu2)

takes lock again
                                                  tries to get lock
                                                  -> *lock = _Q_SLOW_VAL
                        -> *lock == _Q_SLOW_VAL ?
                        -> xen_poll_irq()
frees the lock
-> xen_qlock_kick(cpu3)

And cpu 2 will sleep forever.

This can be avoided easily by modifying xen_qlock_wait() to call
xen_poll_irq() only if the related irq was not pending and to call
xen_clear_irq_pending() only if it was pending.

Cc: stable@vger.kernel.org
Cc: Waiman.Long@hp.com
Cc: peterz@infradead.org
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/spinlock.c | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)

diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c
index 973f10e05211..cd210a4ba7b1 100644
--- a/arch/x86/xen/spinlock.c
+++ b/arch/x86/xen/spinlock.c
@@ -45,17 +45,12 @@ static void xen_qlock_wait(u8 *byte, u8 val)
 	if (irq == -1)
 		return;
 
-	/* clear pending */
-	xen_clear_irq_pending(irq);
-	barrier();
+	/* If irq pending already clear it and return. */
+	if (xen_test_irq_pending(irq)) {
+		xen_clear_irq_pending(irq);
+		return;
+	}
 
-	/*
-	 * We check the byte value after clearing pending IRQ to make sure
-	 * that we won't miss a wakeup event because of the clearing.
-	 *
-	 * The sync_clear_bit() call in xen_clear_irq_pending() is atomic.
-	 * So it is effectively a memory barrier for x86.
-	 */
 	if (READ_ONCE(*byte) != val)
 		return;
 
-- 
2.16.4

Re: [Xen-devel] [PATCH 1/2] xen: fix race in xen_qlock_wait()

[Jan Beulich]

>>> On 01.10.18 at 09:16, <jgross@suse.com> wrote:
> In the following situation a vcpu waiting for a lock might not be
> woken up from xen_poll_irq():
> 
> CPU 1:                CPU 2:                      CPU 3:
> takes a spinlock
>                       tries to get lock
>                       -> xen_qlock_wait()
>                         -> xen_clear_irq_pending()

Doesn't the last line above ...

> frees the lock
> -> xen_qlock_kick(cpu2)

... need to be below here?

> takes lock again
>                                                   tries to get lock
>                                                   -> *lock = _Q_SLOW_VAL
>                         -> *lock == _Q_SLOW_VAL ?
>                         -> xen_poll_irq()
> frees the lock
> -> xen_qlock_kick(cpu3)
> 
> And cpu 2 will sleep forever.
> 
> This can be avoided easily by modifying xen_qlock_wait() to call
> xen_poll_irq() only if the related irq was not pending and to call
> xen_clear_irq_pending() only if it was pending.
> 
> Cc: stable@vger.kernel.org 
> Cc: Waiman.Long@hp.com 
> Cc: peterz@infradead.org 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Patch itself
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan