* [PATCH] KVM: X86: Fix host dr6 miss restore
@ 2017-12-08 9:12 Wanpeng Li
2017-12-08 12:39 ` David Hildenbrand
0 siblings, 1 reply; 6+ messages in thread
From: Wanpeng Li @ 2017-12-08 9:12 UTC (permalink / raw)
To: linux-kernel, kvm
Cc: Paolo Bonzini, Radim Krčmář,
Wanpeng Li, David Hildenbrand, Dmitry Vyukov
From: Wanpeng Li <wanpeng.li@hotmail.com>
Reported by syzkaller:
WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
RIP: 0010:do_debug+0x222/0x250
Call Trace:
<#DB>
debug+0x3e/0x70
RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
</#DB>
_copy_from_user+0x5b/0x90
SyS_timer_create+0x33/0x80
entry_SYSCALL_64_fastpath+0x23/0x9a
The syzkaller will mmap a buffer which is also the struct sigevent parameter of
timer_create(), it will also call perf_event_open() to set a BP for the buffer,
so when the implementation of timer_create() in kernel tries to get the struct
sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
testcase also sets the debug registers for the guest, however, the kvm just
restores host debug registers when we have active breakpoints. I can observe
the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
when running the testcase heavy multithreading. The do_debug() which is triggered
by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
This patch fixes it by restoring host dr6 unconditionally before preempt/irq
enable.
Reported-by: Dmitry Vyukov <dvyukov@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
arch/x86/kvm/x86.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0c5d55c..a6370fd 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
*/
if (hw_breakpoint_active())
hw_breakpoint_restore();
+ else
+ set_debugreg(current->thread.debugreg6, 6);
vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
--
2.7.4
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] KVM: X86: Fix host dr6 miss restore
2017-12-08 9:12 [PATCH] KVM: X86: Fix host dr6 miss restore Wanpeng Li
@ 2017-12-08 12:39 ` David Hildenbrand
2017-12-10 0:44 ` Wanpeng Li
0 siblings, 1 reply; 6+ messages in thread
From: David Hildenbrand @ 2017-12-08 12:39 UTC (permalink / raw)
To: Wanpeng Li, linux-kernel, kvm
Cc: Paolo Bonzini, Radim Krčmář, Wanpeng Li, Dmitry Vyukov
On 08.12.2017 10:12, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
>
> Reported by syzkaller:
>
> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
> RIP: 0010:do_debug+0x222/0x250
> Call Trace:
> <#DB>
> debug+0x3e/0x70
> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
> </#DB>
> _copy_from_user+0x5b/0x90
> SyS_timer_create+0x33/0x80
> entry_SYSCALL_64_fastpath+0x23/0x9a
>
> The syzkaller will mmap a buffer which is also the struct sigevent parameter of
> timer_create(), it will also call perf_event_open() to set a BP for the buffer,
> so when the implementation of timer_create() in kernel tries to get the struct
> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
> testcase also sets the debug registers for the guest, however, the kvm just
> restores host debug registers when we have active breakpoints. I can observe
> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
> when running the testcase heavy multithreading. The do_debug() which is triggered
> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
>
> This patch fixes it by restoring host dr6 unconditionally before preempt/irq
> enable.
>
> Reported-by: Dmitry Vyukov <dvyukov@google.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
> ---
> arch/x86/kvm/x86.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 0c5d55c..a6370fd 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
> */
> if (hw_breakpoint_active())
> hw_breakpoint_restore();
> + else
> + set_debugreg(current->thread.debugreg6, 6);
>
> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>
>
If you haven't seen it, I analyzed this in
https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to
my suggestion/question.
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] KVM: X86: Fix host dr6 miss restore
2017-12-08 12:39 ` David Hildenbrand
@ 2017-12-10 0:44 ` Wanpeng Li
2017-12-11 20:49 ` David Hildenbrand
2017-12-12 15:08 ` Paolo Bonzini
0 siblings, 2 replies; 6+ messages in thread
From: Wanpeng Li @ 2017-12-10 0:44 UTC (permalink / raw)
To: David Hildenbrand
Cc: linux-kernel, kvm, Paolo Bonzini, Radim Krčmář,
Wanpeng Li, Dmitry Vyukov
2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>:
> On 08.12.2017 10:12, Wanpeng Li wrote:
>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>
>> Reported by syzkaller:
>>
>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
>> RIP: 0010:do_debug+0x222/0x250
>> Call Trace:
>> <#DB>
>> debug+0x3e/0x70
>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
>> </#DB>
>> _copy_from_user+0x5b/0x90
>> SyS_timer_create+0x33/0x80
>> entry_SYSCALL_64_fastpath+0x23/0x9a
>>
>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of
>> timer_create(), it will also call perf_event_open() to set a BP for the buffer,
>> so when the implementation of timer_create() in kernel tries to get the struct
>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
>> testcase also sets the debug registers for the guest, however, the kvm just
>> restores host debug registers when we have active breakpoints. I can observe
>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
>> when running the testcase heavy multithreading. The do_debug() which is triggered
>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
>>
>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq
>> enable.
>>
>> Reported-by: Dmitry Vyukov <dvyukov@google.com>
>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>> Cc: David Hildenbrand <david@redhat.com>
>> Cc: Dmitry Vyukov <dvyukov@google.com>
>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
>> ---
>> arch/x86/kvm/x86.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 0c5d55c..a6370fd 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>> */
>> if (hw_breakpoint_active())
>> hw_breakpoint_restore();
>> + else
>> + set_debugreg(current->thread.debugreg6, 6);
>>
>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>>
>>
>
> If you haven't seen it, I analyzed this in
> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to
> my suggestion/question.
I think it's fine to restore dr6 before preempt/irq enable.
Regards,
Wanpeng Li
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] KVM: X86: Fix host dr6 miss restore
2017-12-10 0:44 ` Wanpeng Li
@ 2017-12-11 20:49 ` David Hildenbrand
2017-12-12 15:08 ` Paolo Bonzini
1 sibling, 0 replies; 6+ messages in thread
From: David Hildenbrand @ 2017-12-11 20:49 UTC (permalink / raw)
To: Wanpeng Li
Cc: linux-kernel, kvm, Paolo Bonzini, Radim Krčmář,
Wanpeng Li, Dmitry Vyukov
On 10.12.2017 01:44, Wanpeng Li wrote:
> 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>:
>> On 08.12.2017 10:12, Wanpeng Li wrote:
>>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>>
>>> Reported by syzkaller:
>>>
>>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
>>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
>>> RIP: 0010:do_debug+0x222/0x250
>>> Call Trace:
>>> <#DB>
>>> debug+0x3e/0x70
>>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
>>> </#DB>
>>> _copy_from_user+0x5b/0x90
>>> SyS_timer_create+0x33/0x80
>>> entry_SYSCALL_64_fastpath+0x23/0x9a
>>>
>>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of
>>> timer_create(), it will also call perf_event_open() to set a BP for the buffer,
>>> so when the implementation of timer_create() in kernel tries to get the struct
>>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
>>> testcase also sets the debug registers for the guest, however, the kvm just
>>> restores host debug registers when we have active breakpoints. I can observe
>>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
>>> when running the testcase heavy multithreading. The do_debug() which is triggered
>>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
>>>
>>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq
>>> enable.
>>>
>>> Reported-by: Dmitry Vyukov <dvyukov@google.com>
>>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Dmitry Vyukov <dvyukov@google.com>
>>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
>>> ---
>>> arch/x86/kvm/x86.c | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index 0c5d55c..a6370fd 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>>> */
>>> if (hw_breakpoint_active())
>>> hw_breakpoint_restore();
>>> + else
>>> + set_debugreg(current->thread.debugreg6, 6);
>>>
>>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>>>
>>>
>>
>> If you haven't seen it, I analyzed this in
>> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to
>> my suggestion/question.
>
> I think it's fine to restore dr6 before preempt/irq enable.
That make sense, as I assume this is the first time that a trap would be
delivered.
Reviewed-by: David Hildenbrand <david@redhat.com>
And certainly stable material?
>
> Regards,
> Wanpeng Li
>
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] KVM: X86: Fix host dr6 miss restore
2017-12-10 0:44 ` Wanpeng Li
2017-12-11 20:49 ` David Hildenbrand
@ 2017-12-12 15:08 ` Paolo Bonzini
2017-12-13 3:11 ` Wanpeng Li
1 sibling, 1 reply; 6+ messages in thread
From: Paolo Bonzini @ 2017-12-12 15:08 UTC (permalink / raw)
To: Wanpeng Li, David Hildenbrand
Cc: linux-kernel, kvm, Radim Krčmář,
Wanpeng Li, Dmitry Vyukov
On 10/12/2017 01:44, Wanpeng Li wrote:
> 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>:
>> On 08.12.2017 10:12, Wanpeng Li wrote:
>>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>>
>>> Reported by syzkaller:
>>>
>>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
>>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
>>> RIP: 0010:do_debug+0x222/0x250
>>> Call Trace:
>>> <#DB>
>>> debug+0x3e/0x70
>>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
>>> </#DB>
>>> _copy_from_user+0x5b/0x90
>>> SyS_timer_create+0x33/0x80
>>> entry_SYSCALL_64_fastpath+0x23/0x9a
>>>
>>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of
>>> timer_create(), it will also call perf_event_open() to set a BP for the buffer,
>>> so when the implementation of timer_create() in kernel tries to get the struct
>>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
>>> testcase also sets the debug registers for the guest, however, the kvm just
>>> restores host debug registers when we have active breakpoints. I can observe
>>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
>>> when running the testcase heavy multithreading. The do_debug() which is triggered
>>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
>>>
>>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq
>>> enable.
>>>
>>> Reported-by: Dmitry Vyukov <dvyukov@google.com>
>>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Dmitry Vyukov <dvyukov@google.com>
>>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
>>> ---
>>> arch/x86/kvm/x86.c | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index 0c5d55c..a6370fd 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>>> */
>>> if (hw_breakpoint_active())
>>> hw_breakpoint_restore();
>>> + else
>>> + set_debugreg(current->thread.debugreg6, 6);
>>>
>>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>>>
>>>
>>
>> If you haven't seen it, I analyzed this in
>> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to
>> my suggestion/question.
>
> I think it's fine to restore dr6 before preempt/irq enable.
If no breakpoint is active, you should be able to restore it only in
sched_out.
Paolo
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] KVM: X86: Fix host dr6 miss restore
2017-12-12 15:08 ` Paolo Bonzini
@ 2017-12-13 3:11 ` Wanpeng Li
0 siblings, 0 replies; 6+ messages in thread
From: Wanpeng Li @ 2017-12-13 3:11 UTC (permalink / raw)
To: Paolo Bonzini
Cc: David Hildenbrand, linux-kernel, kvm, Radim Krčmář,
Wanpeng Li, Dmitry Vyukov
2017-12-12 23:08 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
> On 10/12/2017 01:44, Wanpeng Li wrote:
>> 2017-12-08 20:39 GMT+08:00 David Hildenbrand <david@redhat.com>:
>>> On 08.12.2017 10:12, Wanpeng Li wrote:
>>>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>>>
>>>> Reported by syzkaller:
>>>>
>>>> WARNING: CPU: 0 PID: 12927 at arch/x86/kernel/traps.c:780 do_debug+0x222/0x250
>>>> CPU: 0 PID: 12927 Comm: syz-executor Tainted: G OE 4.15.0-rc2+ #16
>>>> RIP: 0010:do_debug+0x222/0x250
>>>> Call Trace:
>>>> <#DB>
>>>> debug+0x3e/0x70
>>>> RIP: 0010:copy_user_enhanced_fast_string+0x10/0x20
>>>> </#DB>
>>>> _copy_from_user+0x5b/0x90
>>>> SyS_timer_create+0x33/0x80
>>>> entry_SYSCALL_64_fastpath+0x23/0x9a
>>>>
>>>> The syzkaller will mmap a buffer which is also the struct sigevent parameter of
>>>> timer_create(), it will also call perf_event_open() to set a BP for the buffer,
>>>> so when the implementation of timer_create() in kernel tries to get the struct
>>>> sigevent parameter by copy_from_user(), rep movsb triggers the BP. The syzkaller
>>>> testcase also sets the debug registers for the guest, however, the kvm just
>>>> restores host debug registers when we have active breakpoints. I can observe
>>>> the dr6 single step bit is set and !hw_breakpoint_active() sporadically by print
>>>> when running the testcase heavy multithreading. The do_debug() which is triggered
>>>> by rep movsb will splash when (dr6 & DR_STEP && !user_mode(regs)).
>>>>
>>>> This patch fixes it by restoring host dr6 unconditionally before preempt/irq
>>>> enable.
>>>>
>>>> Reported-by: Dmitry Vyukov <dvyukov@google.com>
>>>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>>>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>>>> Cc: David Hildenbrand <david@redhat.com>
>>>> Cc: Dmitry Vyukov <dvyukov@google.com>
>>>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
>>>> ---
>>>> arch/x86/kvm/x86.c | 2 ++
>>>> 1 file changed, 2 insertions(+)
>>>>
>>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>>> index 0c5d55c..a6370fd 100644
>>>> --- a/arch/x86/kvm/x86.c
>>>> +++ b/arch/x86/kvm/x86.c
>>>> @@ -7065,6 +7065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
>>>> */
>>>> if (hw_breakpoint_active())
>>>> hw_breakpoint_restore();
>>>> + else
>>>> + set_debugreg(current->thread.debugreg6, 6);
>>>>
>>>> vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>>>>
>>>>
>>>
>>> If you haven't seen it, I analyzed this in
>>> https://lkml.org/lkml/2017/11/7/638 but nobody would respond for now to
>>> my suggestion/question.
>>
>> I think it's fine to restore dr6 before preempt/irq enable.
>
> If no breakpoint is active, you should be able to restore it only in
> sched_out.
Do it in v2. :)
Regards,
Wanpeng Li
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-12-13 3:11 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-12-08 9:12 [PATCH] KVM: X86: Fix host dr6 miss restore Wanpeng Li
2017-12-08 12:39 ` David Hildenbrand
2017-12-10 0:44 ` Wanpeng Li
2017-12-11 20:49 ` David Hildenbrand
2017-12-12 15:08 ` Paolo Bonzini
2017-12-13 3:11 ` Wanpeng Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®