* [GIT pull] smp/urgent for 5.3-rc1
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
@ 2019-07-20 12:50 ` Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] perf/urgent " Thomas Gleixner
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Gleixner @ 2019-07-20 12:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest smp-urgent-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git smp-urgent-for-linus
up to: 19dbdcb8039c: smp: Warn on function calls from softirq context
Add warnings to the smp function calls so callers from wrong contexts get
detected.
Thanks,
tglx
------------------>
Peter Zijlstra (1):
smp: Warn on function calls from softirq context
kernel/smp.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/kernel/smp.c b/kernel/smp.c
index 616d4d114847..7dbcb402c2fc 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -291,6 +291,14 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
WARN_ON_ONCE(cpu_online(this_cpu) && irqs_disabled()
&& !oops_in_progress);
+ /*
+ * When @wait we can deadlock when we interrupt between llist_add() and
+ * arch_send_call_function_ipi*(); when !@wait we can deadlock due to
+ * csd_lock() on because the interrupt context uses the same csd
+ * storage.
+ */
+ WARN_ON_ONCE(!in_task());
+
csd = &csd_stack;
if (!wait) {
csd = this_cpu_ptr(&csd_data);
@@ -416,6 +424,14 @@ void smp_call_function_many(const struct cpumask *mask,
WARN_ON_ONCE(cpu_online(this_cpu) && irqs_disabled()
&& !oops_in_progress && !early_boot_irqs_disabled);
+ /*
+ * When @wait we can deadlock when we interrupt between llist_add() and
+ * arch_send_call_function_ipi*(); when !@wait we can deadlock due to
+ * csd_lock() on because the interrupt context uses the same csd
+ * storage.
+ */
+ WARN_ON_ONCE(!in_task());
+
/* Try to fastpath. So, what's a CPU they want? Ignoring this one. */
cpu = cpumask_first_and(mask, cpu_online_mask);
if (cpu == this_cpu)
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT pull] sched/urgent for 5.3-rc1
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
2019-07-20 12:50 ` [GIT pull] smp/urgent " Thomas Gleixner
2019-07-20 12:50 ` [GIT pull] perf/urgent " Thomas Gleixner
@ 2019-07-20 12:50 ` Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] x86/urgent " Thomas Gleixner
2019-07-20 18:40 ` [GIT pull] core/urgent " pr-tracker-bot
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Gleixner @ 2019-07-20 12:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest sched-urgent-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched-urgent-for-linus
up to: a50a3f4b6a31: sched/rt, Kconfig: Introduce CONFIG_PREEMPT_RT
The real-time preemption patch set exists for almost 15 years now and while
the vast majority of infrastructure and enhancements have found their way
into the mainline kernel, the final integration of RT is still missing.
Over the course of the last few years, we have worked on reducing the
intrusivenness of the RT patches by refactoring kernel infrastructure to be
more real-time friendly. Almost all of these changes were benefitial to
the mainline kernel on their own, so there was no objection to integrate
them.
Though except for the still ongoing printk refactoring, the remaining
changes which are required to make RT a first class mainline citizen are
not longer arguable as immediately beneficial for the mainline kernel. Most
of them are either reordering code flows or adding RT specific functionality.
But this now has hit a wall and turned into a classic hen and egg problem:
Maintainers are rightfully wary vs. these changes as they make only
sense if the final integration of RT into the mainline kernel takes
place.
Adding CONFIG_PREEMPT_RT aims to solve this as a clear sign that RT will be
fully integrated into the mainline kernel. The final integration of the
missing bits and pieces will be of course done with the same careful
approach as we have used in the past.
While I'm aware that you are not entirely enthusiastic about that, I think
that RT should receive the same treatment as any other widely used out of
tree functionality, which we have accepted into mainline over the years.
RT has become the de-facto standard real-time enhancement and is shipped by
enterprise, embedded and community distros. It's in use throughout a wide
range of industries: telecommunications, industrial automation, professional
audio, medical devices, data acquisition, automotive - just to name a
few major use cases.
RT development is backed by a Linuxfoundation project which is supported
by major stakeholders of this technology. The funding will continue over
the actual inclusion into mainline to make sure that the functionality is
neither introducing regressions, regressing itself, nor becomes subject
to bitrot. There is also a lifely user community around RT as well, so
contrary to the grim situation 5 years ago, it's a healthy project.
As RT is still a good vehicle to exercise rarely used code paths and to
detect hard to trigger issues, you could at least view it as a QA tool if
nothing else.
Thanks,
tglx
------------------>
Thomas Gleixner (1):
sched/rt, Kconfig: Introduce CONFIG_PREEMPT_RT
arch/Kconfig | 3 +++
kernel/Kconfig.preempt | 25 +++++++++++++++++++++++--
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index c47b328eada0..ada51f36bd5d 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -801,6 +801,9 @@ config ARCH_NO_COHERENT_DMA_MMAP
config ARCH_NO_PREEMPT
bool
+config ARCH_SUPPORTS_RT
+ bool
+
config CPU_NO_EFFICIENT_FFS
def_bool n
diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
index dc0b682ec2d9..fc020c09b7e8 100644
--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -35,10 +35,10 @@ config PREEMPT_VOLUNTARY
Select this if you are building a kernel for a desktop system.
-config PREEMPT
+config PREEMPT_LL
bool "Preemptible Kernel (Low-Latency Desktop)"
depends on !ARCH_NO_PREEMPT
- select PREEMPT_COUNT
+ select PREEMPT
select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
help
This option reduces the latency of the kernel by making
@@ -55,7 +55,28 @@ config PREEMPT
embedded system with latency requirements in the milliseconds
range.
+config PREEMPT_RT
+ bool "Fully Preemptible Kernel (Real-Time)"
+ depends on EXPERT && ARCH_SUPPORTS_RT
+ select PREEMPT
+ help
+ This option turns the kernel into a real-time kernel by replacing
+ various locking primitives (spinlocks, rwlocks, etc.) with
+ preemptible priority-inheritance aware variants, enforcing
+ interrupt threading and introducing mechanisms to break up long
+ non-preemptible sections. This makes the kernel, except for very
+ low level and critical code pathes (entry code, scheduler, low
+ level interrupt handling) fully preemptible and brings most
+ execution contexts under scheduler control.
+
+ Select this if you are building a kernel for systems which
+ require real-time guarantees.
+
endchoice
config PREEMPT_COUNT
bool
+
+config PREEMPT
+ bool
+ select PREEMPT_COUNT
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT pull] x86/urgent for 5.3-rc1
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
` (2 preceding siblings ...)
2019-07-20 12:50 ` [GIT pull] sched/urgent " Thomas Gleixner
@ 2019-07-20 12:50 ` Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 18:40 ` [GIT pull] core/urgent " pr-tracker-bot
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Gleixner @ 2019-07-20 12:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest x86-urgent-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86-urgent-for-linus
up to: 6879298bd067: x86/entry/64: Prevent clobbering of saved CR2 value
A set of x86 specific fixes and updates:
- The CR2 corruption fixes which store CR2 early in the entry code and
hand the stored address to the fault handlers.
- Revert a forgotten leftover of the dropped FSGSBASE series.
- Plug a memory leak in the boot code.
- Make the Hyper-V assist functionality robust by zeroing the shadow page.
- Remove a useless check for dead processes with LDT
- Update paravirt and VMware maintainers entries.
- A few cleanup patches addressing various compiler warnings.
Thanks,
tglx
------------------>
Andy Lutomirski (1):
Revert "x86/ptrace: Prevent ptrace from clearing the FS/GS selector" and fix the test
Arnd Bergmann (1):
x86: math-emu: Hide clang warnings for 16-bit overflow
David Rientjes (2):
x86/boot: Fix memory leak in default_get_smp_config()
x86/mm: Free sme_early_buffer after init
Dexuan Cui (1):
x86/hyper-v: Zero out the VP ASSIST PAGE on allocation
Jann Horn (1):
x86/process: Delete useless check for dead process with LDT
Peter Zijlstra (5):
x86/paravirt: Make read_cr2() CALLEE_SAVE
x86/entry/32: Simplify common_exception
x86/entry/64: Simplify idtentry a little
x86/entry/64: Update comments and sanity tests for create_gap
x86/mm, tracing: Fix CR2 corruption
Qian Cai (1):
x86/apic: Silence -Wtype-limits compiler warnings
Thomas Gleixner (1):
x86/entry/64: Prevent clobbering of saved CR2 value
Thomas Hellstrom (1):
MAINTAINERS: Update PARAVIRT_OPS_INTERFACE and VMWARE_HYPERVISOR_INTERFACE
Yi Wang (1):
x86/e820: Use proper booleans instead of 0/1
Zhenzhong Duan (3):
x86/boot/efi: Remove unused variables
x86/boot/compressed/64: Remove unused variable
x86, boot: Remove multiple copy of static function sanitize_boot_params()
MAINTAINERS | 6 +-
arch/x86/boot/compressed/eboot.c | 10 +--
arch/x86/boot/compressed/misc.c | 1 +
arch/x86/boot/compressed/misc.h | 1 -
arch/x86/boot/compressed/pgtable_64.c | 1 -
arch/x86/entry/calling.h | 6 ++
arch/x86/entry/entry_32.S | 61 +++++++------
arch/x86/entry/entry_64.S | 155 +++++++++++++++++----------------
arch/x86/hyperv/hv_init.c | 13 ++-
arch/x86/include/asm/apic.h | 2 +-
arch/x86/include/asm/kvm_para.h | 2 +-
arch/x86/include/asm/paravirt.h | 22 +++--
arch/x86/include/asm/paravirt_types.h | 2 +-
arch/x86/include/asm/traps.h | 4 +-
arch/x86/kernel/apic/apic.c | 2 +-
arch/x86/kernel/asm-offsets.c | 1 +
arch/x86/kernel/e820.c | 4 +-
arch/x86/kernel/head_64.S | 4 +-
arch/x86/kernel/kvm.c | 8 +-
arch/x86/kernel/mpparse.c | 10 +--
arch/x86/kernel/paravirt.c | 2 +-
arch/x86/kernel/process_64.c | 12 +--
arch/x86/kernel/ptrace.c | 14 ++-
arch/x86/kernel/traps.c | 6 +-
arch/x86/math-emu/fpu_emu.h | 2 +-
arch/x86/math-emu/reg_constant.c | 2 +-
arch/x86/mm/fault.c | 30 +++----
arch/x86/mm/mem_encrypt.c | 2 +-
arch/x86/xen/enlighten_pv.c | 3 +-
arch/x86/xen/mmu_pv.c | 12 +--
arch/x86/xen/xen-asm.S | 16 ++++
arch/x86/xen/xen-ops.h | 3 +
tools/testing/selftests/x86/fsgsbase.c | 22 +----
33 files changed, 226 insertions(+), 215 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index f5533d1bda2e..80fa7a4a0b56 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12074,7 +12074,8 @@ F: Documentation/parport*.txt
PARAVIRT_OPS INTERFACE
M: Juergen Gross <jgross@suse.com>
-M: Alok Kataria <akataria@vmware.com>
+M: Thomas Hellstrom <thellstrom@vmware.com>
+M: "VMware, Inc." <pv-drivers@vmware.com>
L: virtualization@lists.linux-foundation.org
S: Supported
F: Documentation/virtual/paravirt_ops.txt
@@ -17087,7 +17088,8 @@ S: Maintained
F: drivers/misc/vmw_balloon.c
VMWARE HYPERVISOR INTERFACE
-M: Alok Kataria <akataria@vmware.com>
+M: Thomas Hellstrom <thellstrom@vmware.com>
+M: "VMware, Inc." <pv-drivers@vmware.com>
L: virtualization@lists.linux-foundation.org
S: Supported
F: arch/x86/kernel/cpu/vmware.c
diff --git a/arch/x86/boot/compressed/eboot.c b/arch/x86/boot/compressed/eboot.c
index 220d1279d0e2..d6662fdef300 100644
--- a/arch/x86/boot/compressed/eboot.c
+++ b/arch/x86/boot/compressed/eboot.c
@@ -384,14 +384,11 @@ struct boot_params *make_boot_params(struct efi_config *c)
struct apm_bios_info *bi;
struct setup_header *hdr;
efi_loaded_image_t *image;
- void *options, *handle;
+ void *handle;
efi_guid_t proto = LOADED_IMAGE_PROTOCOL_GUID;
int options_size = 0;
efi_status_t status;
char *cmdline_ptr;
- u16 *s2;
- u8 *s1;
- int i;
unsigned long ramdisk_addr;
unsigned long ramdisk_size;
@@ -494,8 +491,6 @@ static void add_e820ext(struct boot_params *params,
struct setup_data *e820ext, u32 nr_entries)
{
struct setup_data *data;
- efi_status_t status;
- unsigned long size;
e820ext->type = SETUP_E820_EXT;
e820ext->len = nr_entries * sizeof(struct boot_e820_entry);
@@ -677,8 +672,6 @@ static efi_status_t exit_boot_func(efi_system_table_t *sys_table_arg,
void *priv)
{
const char *signature;
- __u32 nr_desc;
- efi_status_t status;
struct exit_boot_struct *p = priv;
signature = efi_is_64bit() ? EFI64_LOADER_SIGNATURE
@@ -747,7 +740,6 @@ struct boot_params *
efi_main(struct efi_config *c, struct boot_params *boot_params)
{
struct desc_ptr *gdt = NULL;
- efi_loaded_image_t *image;
struct setup_header *hdr = &boot_params->hdr;
efi_status_t status;
struct desc_struct *desc;
diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
index 24e65a0f756d..53ac0cb2396d 100644
--- a/arch/x86/boot/compressed/misc.c
+++ b/arch/x86/boot/compressed/misc.c
@@ -17,6 +17,7 @@
#include "pgtable.h"
#include "../string.h"
#include "../voffset.h"
+#include <asm/bootparam_utils.h>
/*
* WARNING!!
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index d2f184165934..c8181392f70d 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -23,7 +23,6 @@
#include <asm/page.h>
#include <asm/boot.h>
#include <asm/bootparam.h>
-#include <asm/bootparam_utils.h>
#define BOOT_CTYPE_H
#include <linux/acpi.h>
diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c
index f8debf7aeb4c..5f2d03067ae5 100644
--- a/arch/x86/boot/compressed/pgtable_64.c
+++ b/arch/x86/boot/compressed/pgtable_64.c
@@ -40,7 +40,6 @@ int cmdline_find_option_bool(const char *option);
static unsigned long find_trampoline_placement(void)
{
unsigned long bios_start = 0, ebda_start = 0;
- unsigned long trampoline_start;
struct boot_e820_entry *entry;
char *signature;
int i;
diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
index 9f1f9e3b8230..830bd984182b 100644
--- a/arch/x86/entry/calling.h
+++ b/arch/x86/entry/calling.h
@@ -343,3 +343,9 @@ For 32-bit we have the following conventions - kernel is built with
.Lafter_call_\@:
#endif
.endm
+
+#ifdef CONFIG_PARAVIRT_XXL
+#define GET_CR2_INTO(reg) GET_CR2_INTO_AX ; _ASM_MOV %_ASM_AX, reg
+#else
+#define GET_CR2_INTO(reg) _ASM_MOV %cr2, reg
+#endif
diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
index 90b473297299..2bb986f305ac 100644
--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -294,9 +294,11 @@
.Lfinished_frame_\@:
.endm
-.macro SAVE_ALL pt_regs_ax=%eax switch_stacks=0
+.macro SAVE_ALL pt_regs_ax=%eax switch_stacks=0 skip_gs=0
cld
+.if \skip_gs == 0
PUSH_GS
+.endif
FIXUP_FRAME
pushl %fs
pushl %es
@@ -313,13 +315,13 @@
movl %edx, %es
movl $(__KERNEL_PERCPU), %edx
movl %edx, %fs
+.if \skip_gs == 0
SET_KERNEL_GS %edx
-
+.endif
/* Switch to kernel stack if necessary */
.if \switch_stacks > 0
SWITCH_TO_KERNEL_STACK
.endif
-
.endm
.macro SAVE_ALL_NMI cr3_reg:req
@@ -1441,39 +1443,46 @@ BUILD_INTERRUPT3(hv_stimer0_callback_vector, HYPERV_STIMER0_VECTOR,
ENTRY(page_fault)
ASM_CLAC
- pushl $do_page_fault
- ALIGN
- jmp common_exception
+ pushl $0; /* %gs's slot on the stack */
+
+ SAVE_ALL switch_stacks=1 skip_gs=1
+
+ ENCODE_FRAME_POINTER
+ UNWIND_ESPFIX_STACK
+
+ /* fixup %gs */
+ GS_TO_REG %ecx
+ REG_TO_PTGS %ecx
+ SET_KERNEL_GS %ecx
+
+ GET_CR2_INTO(%ecx) # might clobber %eax
+
+ /* fixup orig %eax */
+ movl PT_ORIG_EAX(%esp), %edx # get the error code
+ movl $-1, PT_ORIG_EAX(%esp) # no syscall to restart
+
+ TRACE_IRQS_OFF
+ movl %esp, %eax # pt_regs pointer
+ call do_page_fault
+ jmp ret_from_exception
END(page_fault)
common_exception:
/* the function address is in %gs's slot on the stack */
- FIXUP_FRAME
- pushl %fs
- pushl %es
- pushl %ds
- pushl %eax
- movl $(__USER_DS), %eax
- movl %eax, %ds
- movl %eax, %es
- movl $(__KERNEL_PERCPU), %eax
- movl %eax, %fs
- pushl %ebp
- pushl %edi
- pushl %esi
- pushl %edx
- pushl %ecx
- pushl %ebx
- SWITCH_TO_KERNEL_STACK
+ SAVE_ALL switch_stacks=1 skip_gs=1
ENCODE_FRAME_POINTER
- cld
UNWIND_ESPFIX_STACK
+
+ /* fixup %gs */
GS_TO_REG %ecx
movl PT_GS(%esp), %edi # get the function address
- movl PT_ORIG_EAX(%esp), %edx # get the error code
- movl $-1, PT_ORIG_EAX(%esp) # no syscall to restart
REG_TO_PTGS %ecx
SET_KERNEL_GS %ecx
+
+ /* fixup orig %eax */
+ movl PT_ORIG_EAX(%esp), %edx # get the error code
+ movl $-1, PT_ORIG_EAX(%esp) # no syscall to restart
+
TRACE_IRQS_OFF
movl %esp, %eax # pt_regs pointer
CALL_NOSPEC %edi
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 0ea4831a72a4..f7c70c1bee8b 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -864,18 +864,84 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt
*/
#define CPU_TSS_IST(x) PER_CPU_VAR(cpu_tss_rw) + (TSS_ist + (x) * 8)
+.macro idtentry_part do_sym, has_error_code:req, read_cr2:req, paranoid:req, shift_ist=-1, ist_offset=0
+
+ .if \paranoid
+ call paranoid_entry
+ /* returned flag: ebx=0: need swapgs on exit, ebx=1: don't need it */
+ .else
+ call error_entry
+ .endif
+ UNWIND_HINT_REGS
+
+ .if \read_cr2
+ /*
+ * Store CR2 early so subsequent faults cannot clobber it. Use R12 as
+ * intermediate storage as RDX can be clobbered in enter_from_user_mode().
+ * GET_CR2_INTO can clobber RAX.
+ */
+ GET_CR2_INTO(%r12);
+ .endif
+
+ .if \shift_ist != -1
+ TRACE_IRQS_OFF_DEBUG /* reload IDT in case of recursion */
+ .else
+ TRACE_IRQS_OFF
+ .endif
+
+ .if \paranoid == 0
+ testb $3, CS(%rsp)
+ jz .Lfrom_kernel_no_context_tracking_\@
+ CALL_enter_from_user_mode
+.Lfrom_kernel_no_context_tracking_\@:
+ .endif
+
+ movq %rsp, %rdi /* pt_regs pointer */
+
+ .if \has_error_code
+ movq ORIG_RAX(%rsp), %rsi /* get error code */
+ movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */
+ .else
+ xorl %esi, %esi /* no error code */
+ .endif
+
+ .if \shift_ist != -1
+ subq $\ist_offset, CPU_TSS_IST(\shift_ist)
+ .endif
+
+ .if \read_cr2
+ movq %r12, %rdx /* Move CR2 into 3rd argument */
+ .endif
+
+ call \do_sym
+
+ .if \shift_ist != -1
+ addq $\ist_offset, CPU_TSS_IST(\shift_ist)
+ .endif
+
+ .if \paranoid
+ /* this procedure expect "no swapgs" flag in ebx */
+ jmp paranoid_exit
+ .else
+ jmp error_exit
+ .endif
+
+.endm
+
/**
* idtentry - Generate an IDT entry stub
* @sym: Name of the generated entry point
- * @do_sym: C function to be called
- * @has_error_code: True if this IDT vector has an error code on the stack
- * @paranoid: non-zero means that this vector may be invoked from
+ * @do_sym: C function to be called
+ * @has_error_code: True if this IDT vector has an error code on the stack
+ * @paranoid: non-zero means that this vector may be invoked from
* kernel mode with user GSBASE and/or user CR3.
* 2 is special -- see below.
* @shift_ist: Set to an IST index if entries from kernel mode should
- * decrement the IST stack so that nested entries get a
+ * decrement the IST stack so that nested entries get a
* fresh stack. (This is for #DB, which has a nasty habit
- * of recursing.)
+ * of recursing.)
+ * @create_gap: create a 6-word stack gap when coming from kernel mode.
+ * @read_cr2: load CR2 into the 3rd argument; done before calling any C code
*
* idtentry generates an IDT stub that sets up a usable kernel context,
* creates struct pt_regs, and calls @do_sym. The stub has the following
@@ -900,15 +966,19 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt
* @paranoid == 2 is special: the stub will never switch stacks. This is for
* #DF: if the thread stack is somehow unusable, we'll still get a useful OOPS.
*/
-.macro idtentry sym do_sym has_error_code:req paranoid=0 shift_ist=-1 ist_offset=0 create_gap=0
+.macro idtentry sym do_sym has_error_code:req paranoid=0 shift_ist=-1 ist_offset=0 create_gap=0 read_cr2=0
ENTRY(\sym)
UNWIND_HINT_IRET_REGS offset=\has_error_code*8
/* Sanity check */
- .if \shift_ist != -1 && \paranoid == 0
+ .if \shift_ist != -1 && \paranoid != 1
.error "using shift_ist requires paranoid=1"
.endif
+ .if \create_gap && \paranoid
+ .error "using create_gap requires paranoid=0"
+ .endif
+
ASM_CLAC
.if \has_error_code == 0
@@ -934,47 +1004,7 @@ ENTRY(\sym)
.Lfrom_usermode_no_gap_\@:
.endif
- .if \paranoid
- call paranoid_entry
- .else
- call error_entry
- .endif
- UNWIND_HINT_REGS
- /* returned flag: ebx=0: need swapgs on exit, ebx=1: don't need it */
-
- .if \paranoid
- .if \shift_ist != -1
- TRACE_IRQS_OFF_DEBUG /* reload IDT in case of recursion */
- .else
- TRACE_IRQS_OFF
- .endif
- .endif
-
- movq %rsp, %rdi /* pt_regs pointer */
-
- .if \has_error_code
- movq ORIG_RAX(%rsp), %rsi /* get error code */
- movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */
- .else
- xorl %esi, %esi /* no error code */
- .endif
-
- .if \shift_ist != -1
- subq $\ist_offset, CPU_TSS_IST(\shift_ist)
- .endif
-
- call \do_sym
-
- .if \shift_ist != -1
- addq $\ist_offset, CPU_TSS_IST(\shift_ist)
- .endif
-
- /* these procedures expect "no swapgs" flag in ebx */
- .if \paranoid
- jmp paranoid_exit
- .else
- jmp error_exit
- .endif
+ idtentry_part \do_sym, \has_error_code, \read_cr2, \paranoid, \shift_ist, \ist_offset
.if \paranoid == 1
/*
@@ -983,21 +1013,9 @@ ENTRY(\sym)
* run in real process context if user_mode(regs).
*/
.Lfrom_usermode_switch_stack_\@:
- call error_entry
-
- movq %rsp, %rdi /* pt_regs pointer */
-
- .if \has_error_code
- movq ORIG_RAX(%rsp), %rsi /* get error code */
- movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */
- .else
- xorl %esi, %esi /* no error code */
+ idtentry_part \do_sym, \has_error_code, \read_cr2, paranoid=0
.endif
- call \do_sym
-
- jmp error_exit
- .endif
_ASM_NOKPROBE(\sym)
END(\sym)
.endm
@@ -1007,7 +1025,7 @@ idtentry overflow do_overflow has_error_code=0
idtentry bounds do_bounds has_error_code=0
idtentry invalid_op do_invalid_op has_error_code=0
idtentry device_not_available do_device_not_available has_error_code=0
-idtentry double_fault do_double_fault has_error_code=1 paranoid=2
+idtentry double_fault do_double_fault has_error_code=1 paranoid=2 read_cr2=1
idtentry coprocessor_segment_overrun do_coprocessor_segment_overrun has_error_code=0
idtentry invalid_TSS do_invalid_TSS has_error_code=1
idtentry segment_not_present do_segment_not_present has_error_code=1
@@ -1180,10 +1198,10 @@ idtentry xenint3 do_int3 has_error_code=0
#endif
idtentry general_protection do_general_protection has_error_code=1
-idtentry page_fault do_page_fault has_error_code=1
+idtentry page_fault do_page_fault has_error_code=1 read_cr2=1
#ifdef CONFIG_KVM_GUEST
-idtentry async_page_fault do_async_page_fault has_error_code=1
+idtentry async_page_fault do_async_page_fault has_error_code=1 read_cr2=1
#endif
#ifdef CONFIG_X86_MCE
@@ -1282,18 +1300,9 @@ ENTRY(error_entry)
movq %rax, %rsp /* switch stack */
ENCODE_FRAME_POINTER
pushq %r12
-
- /*
- * We need to tell lockdep that IRQs are off. We can't do this until
- * we fix gsbase, and we should do it before enter_from_user_mode
- * (which can take locks).
- */
- TRACE_IRQS_OFF
- CALL_enter_from_user_mode
ret
.Lerror_entry_done:
- TRACE_IRQS_OFF
ret
/*
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 0e033ef11a9f..0d258688c8cf 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -60,8 +60,17 @@ static int hv_cpu_init(unsigned int cpu)
if (!hv_vp_assist_page)
return 0;
- if (!*hvp)
- *hvp = __vmalloc(PAGE_SIZE, GFP_KERNEL, PAGE_KERNEL);
+ /*
+ * The VP ASSIST PAGE is an "overlay" page (see Hyper-V TLFS's Section
+ * 5.2.1 "GPA Overlay Pages"). Here it must be zeroed out to make sure
+ * we always write the EOI MSR in hv_apic_eoi_write() *after* the
+ * EOI optimization is disabled in hv_cpu_die(), otherwise a CPU may
+ * not be stopped in the case of CPU offlining and the VM will hang.
+ */
+ if (!*hvp) {
+ *hvp = __vmalloc(PAGE_SIZE, GFP_KERNEL | __GFP_ZERO,
+ PAGE_KERNEL);
+ }
if (*hvp) {
u64 val;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 050e5f9ebf81..e647aa095867 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -49,7 +49,7 @@ static inline void generic_apic_probe(void)
#ifdef CONFIG_X86_LOCAL_APIC
-extern unsigned int apic_verbosity;
+extern int apic_verbosity;
extern int local_apic_timer_c2_ok;
extern int disable_apic;
diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h
index 5ed3cf1c3934..9b4df6eaa11a 100644
--- a/arch/x86/include/asm/kvm_para.h
+++ b/arch/x86/include/asm/kvm_para.h
@@ -92,7 +92,7 @@ void kvm_async_pf_task_wait(u32 token, int interrupt_kernel);
void kvm_async_pf_task_wake(u32 token);
u32 kvm_read_and_reset_pf_reason(void);
extern void kvm_disable_steal_time(void);
-void do_async_page_fault(struct pt_regs *regs, unsigned long error_code);
+void do_async_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address);
#ifdef CONFIG_PARAVIRT_SPINLOCKS
void __init kvm_spinlock_init(void);
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index c25c38a05c1c..5135282683d4 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -116,7 +116,7 @@ static inline void write_cr0(unsigned long x)
static inline unsigned long read_cr2(void)
{
- return PVOP_CALL0(unsigned long, mmu.read_cr2);
+ return PVOP_CALLEE0(unsigned long, mmu.read_cr2);
}
static inline void write_cr2(unsigned long x)
@@ -909,13 +909,7 @@ extern void default_banner(void);
ANNOTATE_RETPOLINE_SAFE; \
call PARA_INDIRECT(pv_ops+PV_CPU_swapgs); \
)
-#endif
-
-#define GET_CR2_INTO_RAX \
- ANNOTATE_RETPOLINE_SAFE; \
- call PARA_INDIRECT(pv_ops+PV_MMU_read_cr2);
-#ifdef CONFIG_PARAVIRT_XXL
#define USERGS_SYSRET64 \
PARA_SITE(PARA_PATCH(PV_CPU_usergs_sysret64), \
ANNOTATE_RETPOLINE_SAFE; \
@@ -929,9 +923,19 @@ extern void default_banner(void);
call PARA_INDIRECT(pv_ops+PV_IRQ_save_fl); \
PV_RESTORE_REGS(clobbers | CLBR_CALLEE_SAVE);)
#endif
-#endif
+#endif /* CONFIG_PARAVIRT_XXL */
+#endif /* CONFIG_X86_64 */
+
+#ifdef CONFIG_PARAVIRT_XXL
+
+#define GET_CR2_INTO_AX \
+ PARA_SITE(PARA_PATCH(PV_MMU_read_cr2), \
+ ANNOTATE_RETPOLINE_SAFE; \
+ call PARA_INDIRECT(pv_ops+PV_MMU_read_cr2); \
+ )
+
+#endif /* CONFIG_PARAVIRT_XXL */
-#endif /* CONFIG_X86_32 */
#endif /* __ASSEMBLY__ */
#else /* CONFIG_PARAVIRT */
diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h
index 946f8f1f1efc..639b2df445ee 100644
--- a/arch/x86/include/asm/paravirt_types.h
+++ b/arch/x86/include/asm/paravirt_types.h
@@ -220,7 +220,7 @@ struct pv_mmu_ops {
void (*exit_mmap)(struct mm_struct *mm);
#ifdef CONFIG_PARAVIRT_XXL
- unsigned long (*read_cr2)(void);
+ struct paravirt_callee_save read_cr2;
void (*write_cr2)(unsigned long);
unsigned long (*read_cr3)(void);
diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h
index 7d6f3f3fad78..5dd1674ddf4c 100644
--- a/arch/x86/include/asm/traps.h
+++ b/arch/x86/include/asm/traps.h
@@ -74,14 +74,14 @@ dotraplinkage void do_invalid_TSS(struct pt_regs *regs, long error_code);
dotraplinkage void do_segment_not_present(struct pt_regs *regs, long error_code);
dotraplinkage void do_stack_segment(struct pt_regs *regs, long error_code);
#ifdef CONFIG_X86_64
-dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code);
+dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code, unsigned long address);
asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *eregs);
asmlinkage __visible notrace
struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s);
void __init trap_init(void);
#endif
dotraplinkage void do_general_protection(struct pt_regs *regs, long error_code);
-dotraplinkage void do_page_fault(struct pt_regs *regs, unsigned long error_code);
+dotraplinkage void do_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address);
dotraplinkage void do_spurious_interrupt_bug(struct pt_regs *regs, long error_code);
dotraplinkage void do_coprocessor_error(struct pt_regs *regs, long error_code);
dotraplinkage void do_alignment_check(struct pt_regs *regs, long error_code);
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 1bd91cb7b320..f5291362da1a 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -183,7 +183,7 @@ EXPORT_SYMBOL_GPL(local_apic_timer_c2_ok);
/*
* Debug level, exported for io_apic.c
*/
-unsigned int apic_verbosity;
+int apic_verbosity;
int pic_mode;
diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c
index da64452584b0..5c7ee3df4d0b 100644
--- a/arch/x86/kernel/asm-offsets.c
+++ b/arch/x86/kernel/asm-offsets.c
@@ -76,6 +76,7 @@ static void __used common(void)
BLANK();
OFFSET(XEN_vcpu_info_mask, vcpu_info, evtchn_upcall_mask);
OFFSET(XEN_vcpu_info_pending, vcpu_info, evtchn_upcall_pending);
+ OFFSET(XEN_vcpu_info_arch_cr2, vcpu_info, arch.cr2);
#endif
BLANK();
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index e69408bf664b..7da2bcd2b8eb 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -86,9 +86,9 @@ static bool _e820__mapped_any(struct e820_table *table,
continue;
if (entry->addr >= end || entry->addr + entry->size <= start)
continue;
- return 1;
+ return true;
}
- return 0;
+ return false;
}
bool e820__mapped_raw_any(u64 start, u64 end, enum e820_type type)
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index bcd206c8ac90..0e2d72929a8c 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -29,9 +29,7 @@
#ifdef CONFIG_PARAVIRT_XXL
#include <asm/asm-offsets.h>
#include <asm/paravirt.h>
-#define GET_CR2_INTO(reg) GET_CR2_INTO_RAX ; movq %rax, reg
#else
-#define GET_CR2_INTO(reg) movq %cr2, reg
#define INTERRUPT_RETURN iretq
#endif
@@ -323,7 +321,7 @@ early_idt_handler_common:
cmpq $14,%rsi /* Page fault? */
jnz 10f
- GET_CR2_INTO(%rdi) /* Can clobber any volatile register if pv */
+ GET_CR2_INTO(%rdi) /* can clobber %rax if pv */
call early_make_pgtable
andl %eax,%eax
jz 20f /* All good */
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 82caf01b63dd..3231440d6253 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -242,23 +242,23 @@ EXPORT_SYMBOL_GPL(kvm_read_and_reset_pf_reason);
NOKPROBE_SYMBOL(kvm_read_and_reset_pf_reason);
dotraplinkage void
-do_async_page_fault(struct pt_regs *regs, unsigned long error_code)
+do_async_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
enum ctx_state prev_state;
switch (kvm_read_and_reset_pf_reason()) {
default:
- do_page_fault(regs, error_code);
+ do_page_fault(regs, error_code, address);
break;
case KVM_PV_REASON_PAGE_NOT_PRESENT:
/* page is swapped out by the host. */
prev_state = exception_enter();
- kvm_async_pf_task_wait((u32)read_cr2(), !user_mode(regs));
+ kvm_async_pf_task_wait((u32)address, !user_mode(regs));
exception_exit(prev_state);
break;
case KVM_PV_REASON_PAGE_READY:
rcu_irq_enter();
- kvm_async_pf_task_wake((u32)read_cr2());
+ kvm_async_pf_task_wake((u32)address);
rcu_irq_exit();
break;
}
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index 1bfe5c6e6cfe..afac7ccce72f 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -546,17 +546,15 @@ void __init default_get_smp_config(unsigned int early)
* local APIC has default address
*/
mp_lapic_addr = APIC_DEFAULT_PHYS_BASE;
- return;
+ goto out;
}
pr_info("Default MP configuration #%d\n", mpf->feature1);
construct_default_ISA_mptable(mpf->feature1);
} else if (mpf->physptr) {
- if (check_physptr(mpf, early)) {
- early_memunmap(mpf, sizeof(*mpf));
- return;
- }
+ if (check_physptr(mpf, early))
+ goto out;
} else
BUG();
@@ -565,7 +563,7 @@ void __init default_get_smp_config(unsigned int early)
/*
* Only use the first configuration found.
*/
-
+out:
early_memunmap(mpf, sizeof(*mpf));
}
diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
index 98039d7fb998..0aa6256eedd8 100644
--- a/arch/x86/kernel/paravirt.c
+++ b/arch/x86/kernel/paravirt.c
@@ -370,7 +370,7 @@ struct paravirt_patch_template pv_ops = {
.mmu.exit_mmap = paravirt_nop,
#ifdef CONFIG_PARAVIRT_XXL
- .mmu.read_cr2 = native_read_cr2,
+ .mmu.read_cr2 = __PV_IS_CALLEE_SAVE(native_read_cr2),
.mmu.write_cr2 = native_write_cr2,
.mmu.read_cr3 = __native_read_cr3,
.mmu.write_cr3 = native_write_cr3,
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 250e4c4ac6d9..af64519b2695 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -143,17 +143,7 @@ void __show_regs(struct pt_regs *regs, enum show_regs_mode mode)
void release_thread(struct task_struct *dead_task)
{
- if (dead_task->mm) {
-#ifdef CONFIG_MODIFY_LDT_SYSCALL
- if (dead_task->mm->context.ldt) {
- pr_warn("WARNING: dead process %s still has LDT? <%p/%d>\n",
- dead_task->comm,
- dead_task->mm->context.ldt->entries,
- dead_task->mm->context.ldt->nr_entries);
- BUG();
- }
-#endif
- }
+ WARN_ON(dead_task->mm);
}
enum which_selector {
diff --git a/arch/x86/kernel/ptrace.c b/arch/x86/kernel/ptrace.c
index 71691a8310e7..0fdbe89d0754 100644
--- a/arch/x86/kernel/ptrace.c
+++ b/arch/x86/kernel/ptrace.c
@@ -369,12 +369,22 @@ static int putreg(struct task_struct *child,
case offsetof(struct user_regs_struct,fs_base):
if (value >= TASK_SIZE_MAX)
return -EIO;
- x86_fsbase_write_task(child, value);
+ /*
+ * When changing the FS base, use do_arch_prctl_64()
+ * to set the index to zero and to set the base
+ * as requested.
+ */
+ if (child->thread.fsbase != value)
+ return do_arch_prctl_64(child, ARCH_SET_FS, value);
return 0;
case offsetof(struct user_regs_struct,gs_base):
+ /*
+ * Exactly the same here as the %fs handling above.
+ */
if (value >= TASK_SIZE_MAX)
return -EIO;
- x86_gsbase_write_task(child, value);
+ if (child->thread.gsbase != value)
+ return do_arch_prctl_64(child, ARCH_SET_GS, value);
return 0;
#endif
}
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 87095a477154..4bb0f8447112 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -313,13 +313,10 @@ __visible void __noreturn handle_stack_overflow(const char *message,
#ifdef CONFIG_X86_64
/* Runs on IST stack */
-dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
+dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code, unsigned long cr2)
{
static const char str[] = "double fault";
struct task_struct *tsk = current;
-#ifdef CONFIG_VMAP_STACK
- unsigned long cr2;
-#endif
#ifdef CONFIG_X86_ESPFIX64
extern unsigned char native_irq_return_iret[];
@@ -415,7 +412,6 @@ dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
* stack even if the actual trigger for the double fault was
* something else.
*/
- cr2 = read_cr2();
if ((unsigned long)task_stack_page(tsk) - 1 - cr2 < PAGE_SIZE)
handle_stack_overflow("kernel stack overflow (double-fault)", regs, cr2);
#endif
diff --git a/arch/x86/math-emu/fpu_emu.h b/arch/x86/math-emu/fpu_emu.h
index a5a41ec58072..0c122226ca56 100644
--- a/arch/x86/math-emu/fpu_emu.h
+++ b/arch/x86/math-emu/fpu_emu.h
@@ -177,7 +177,7 @@ static inline void reg_copy(FPU_REG const *x, FPU_REG *y)
#define setexponentpos(x,y) { (*(short *)&((x)->exp)) = \
((y) + EXTENDED_Ebias) & 0x7fff; }
#define exponent16(x) (*(short *)&((x)->exp))
-#define setexponent16(x,y) { (*(short *)&((x)->exp)) = (y); }
+#define setexponent16(x,y) { (*(short *)&((x)->exp)) = (u16)(y); }
#define addexponent(x,y) { (*(short *)&((x)->exp)) += (y); }
#define stdexp(x) { (*(short *)&((x)->exp)) += EXTENDED_Ebias; }
diff --git a/arch/x86/math-emu/reg_constant.c b/arch/x86/math-emu/reg_constant.c
index 8dc9095bab22..742619e94bdf 100644
--- a/arch/x86/math-emu/reg_constant.c
+++ b/arch/x86/math-emu/reg_constant.c
@@ -18,7 +18,7 @@
#include "control_w.h"
#define MAKE_REG(s, e, l, h) { l, h, \
- ((EXTENDED_Ebias+(e)) | ((SIGN_##s != 0)*0x8000)) }
+ (u16)((EXTENDED_Ebias+(e)) | ((SIGN_##s != 0)*0x8000)) }
FPU_REG const CONST_1 = MAKE_REG(POS, 0, 0x00000000, 0x80000000);
#if 0
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 794f364cb882..0799cc79efd3 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1507,9 +1507,8 @@ void do_user_addr_fault(struct pt_regs *regs,
NOKPROBE_SYMBOL(do_user_addr_fault);
/*
- * This routine handles page faults. It determines the address,
- * and the problem, and then passes it off to one of the appropriate
- * routines.
+ * Explicitly marked noinline such that the function tracer sees this as the
+ * page_fault entry point.
*/
static noinline void
__do_page_fault(struct pt_regs *regs, unsigned long hw_error_code,
@@ -1528,33 +1527,26 @@ __do_page_fault(struct pt_regs *regs, unsigned long hw_error_code,
}
NOKPROBE_SYMBOL(__do_page_fault);
-static nokprobe_inline void
-trace_page_fault_entries(unsigned long address, struct pt_regs *regs,
- unsigned long error_code)
+static __always_inline void
+trace_page_fault_entries(struct pt_regs *regs, unsigned long error_code,
+ unsigned long address)
{
+ if (!trace_pagefault_enabled())
+ return;
+
if (user_mode(regs))
trace_page_fault_user(address, regs, error_code);
else
trace_page_fault_kernel(address, regs, error_code);
}
-/*
- * We must have this function blacklisted from kprobes, tagged with notrace
- * and call read_cr2() before calling anything else. To avoid calling any
- * kind of tracing machinery before we've observed the CR2 value.
- *
- * exception_{enter,exit}() contains all sorts of tracepoints.
- */
-dotraplinkage void notrace
-do_page_fault(struct pt_regs *regs, unsigned long error_code)
+dotraplinkage void
+do_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
- unsigned long address = read_cr2(); /* Get the faulting address */
enum ctx_state prev_state;
prev_state = exception_enter();
- if (trace_pagefault_enabled())
- trace_page_fault_entries(address, regs, error_code);
-
+ trace_page_fault_entries(regs, error_code, address);
__do_page_fault(regs, error_code, address);
exception_exit(prev_state);
}
diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c
index e0df96fdfe46..e94e0a62ba92 100644
--- a/arch/x86/mm/mem_encrypt.c
+++ b/arch/x86/mm/mem_encrypt.c
@@ -41,7 +41,7 @@ EXPORT_SYMBOL_GPL(sev_enable_key);
bool sev_enabled __section(.data);
/* Buffer used for early in-place encryption by BSP, no locking needed */
-static char sme_early_buffer[PAGE_SIZE] __aligned(PAGE_SIZE);
+static char sme_early_buffer[PAGE_SIZE] __initdata __aligned(PAGE_SIZE);
/*
* This routine does not change the underlying encryption setting of the
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 4722ba2966ac..26b63d051bda 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -998,7 +998,8 @@ void __init xen_setup_vcpu_info_placement(void)
__PV_IS_CALLEE_SAVE(xen_irq_disable_direct);
pv_ops.irq.irq_enable =
__PV_IS_CALLEE_SAVE(xen_irq_enable_direct);
- pv_ops.mmu.read_cr2 = xen_read_cr2_direct;
+ pv_ops.mmu.read_cr2 =
+ __PV_IS_CALLEE_SAVE(xen_read_cr2_direct);
}
}
diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c
index f6e5eeecfc69..26e8b326966d 100644
--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1307,16 +1307,6 @@ static void xen_write_cr2(unsigned long cr2)
this_cpu_read(xen_vcpu)->arch.cr2 = cr2;
}
-static unsigned long xen_read_cr2(void)
-{
- return this_cpu_read(xen_vcpu)->arch.cr2;
-}
-
-unsigned long xen_read_cr2_direct(void)
-{
- return this_cpu_read(xen_vcpu_info.arch.cr2);
-}
-
static noinline void xen_flush_tlb(void)
{
struct mmuext_op *op;
@@ -2397,7 +2387,7 @@ static void xen_leave_lazy_mmu(void)
}
static const struct pv_mmu_ops xen_mmu_ops __initconst = {
- .read_cr2 = xen_read_cr2,
+ .read_cr2 = __PV_IS_CALLEE_SAVE(xen_read_cr2),
.write_cr2 = xen_write_cr2,
.read_cr3 = xen_read_cr3,
diff --git a/arch/x86/xen/xen-asm.S b/arch/x86/xen/xen-asm.S
index 8019edd0125c..be104eef80be 100644
--- a/arch/x86/xen/xen-asm.S
+++ b/arch/x86/xen/xen-asm.S
@@ -10,6 +10,7 @@
#include <asm/percpu.h>
#include <asm/processor-flags.h>
#include <asm/frame.h>
+#include <asm/asm.h>
#include <linux/linkage.h>
@@ -135,3 +136,18 @@ ENTRY(check_events)
FRAME_END
ret
ENDPROC(check_events)
+
+ENTRY(xen_read_cr2)
+ FRAME_BEGIN
+ _ASM_MOV PER_CPU_VAR(xen_vcpu), %_ASM_AX
+ _ASM_MOV XEN_vcpu_info_arch_cr2(%_ASM_AX), %_ASM_AX
+ FRAME_END
+ ret
+ ENDPROC(xen_read_cr2);
+
+ENTRY(xen_read_cr2_direct)
+ FRAME_BEGIN
+ _ASM_MOV PER_CPU_VAR(xen_vcpu_info) + XEN_vcpu_info_arch_cr2, %_ASM_AX
+ FRAME_END
+ ret
+ ENDPROC(xen_read_cr2_direct);
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 2f111f47ba98..45a441c33d6d 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -134,6 +134,9 @@ __visible void xen_irq_disable_direct(void);
__visible unsigned long xen_save_fl_direct(void);
__visible void xen_restore_fl_direct(unsigned long);
+__visible unsigned long xen_read_cr2(void);
+__visible unsigned long xen_read_cr2_direct(void);
+
/* These are not functions, and cannot be called normally */
__visible void xen_iret(void);
__visible void xen_sysret32(void);
diff --git a/tools/testing/selftests/x86/fsgsbase.c b/tools/testing/selftests/x86/fsgsbase.c
index 5ab4c60c100e..15a329da59fa 100644
--- a/tools/testing/selftests/x86/fsgsbase.c
+++ b/tools/testing/selftests/x86/fsgsbase.c
@@ -489,25 +489,11 @@ static void test_ptrace_write_gsbase(void)
* selector value is changed or not by the GSBASE write in
* a ptracer.
*/
- if (gs != *shared_scratch) {
- nerrs++;
- printf("[FAIL]\tGS changed to %lx\n", gs);
-
- /*
- * On older kernels, poking a nonzero value into the
- * base would zero the selector. On newer kernels,
- * this behavior has changed -- poking the base
- * changes only the base and, if FSGSBASE is not
- * available, this may have no effect.
- */
- if (gs == 0)
- printf("\tNote: this is expected behavior on older kernels.\n");
- } else if (have_fsgsbase && (base != 0xFF)) {
- nerrs++;
- printf("[FAIL]\tGSBASE changed to %lx\n", base);
+ if (gs == 0 && base == 0xFF) {
+ printf("[OK]\tGS was reset as expected\n");
} else {
- printf("[OK]\tGS remained 0x%hx%s", *shared_scratch, have_fsgsbase ? " and GSBASE changed to 0xFF" : "");
- printf("\n");
+ nerrs++;
+ printf("[FAIL]\tGS=0x%lx, GSBASE=0x%lx (should be 0, 0xFF)\n", gs, base);
}
}
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT pull] core/urgent for 5.3-rc1
@ 2019-07-20 12:50 Thomas Gleixner
2019-07-20 12:50 ` [GIT pull] smp/urgent " Thomas Gleixner
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Thomas Gleixner @ 2019-07-20 12:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest core-urgent-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-urgent-for-linus
up to: b68b9907069a: objtool: Support conditional retpolines
This update contains:
- A collection of objtool fixes which address recent fallout partially
exposed by newer toolchains, clang, BPF and general code changes.
- Force USER_DS for user stack traces
Thanks,
tglx
------------------>
Jann Horn (1):
objtool: Support repeated uses of the same C jump table
Josh Poimboeuf (23):
objtool: Add support for C jump tables
bpf: Fix ORC unwinding in non-JIT BPF code
x86/paravirt: Fix callee-saved function ELF sizes
x86/kvm: Fix fastop function ELF metadata
x86/kvm: Replace vmx_vmenter()'s call to kvm_spurious_fault() with UD2
x86/kvm: Don't call kvm_spurious_fault() from .fixup
x86/entry: Fix thunk function ELF sizes
x86/head/64: Annotate start_cpu0() as non-callable
x86/uaccess: Remove ELF function annotation from copy_user_handle_tail()
x86/uaccess: Don't leak AC flag into fentry from mcsafe_handle_tail()
x86/uaccess: Remove redundant CLACs in getuser/putuser error paths
bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
objtool: Add mcsafe_handle_tail() to the uaccess safe list
objtool: Track original function across branches
objtool: Refactor function alias logic
objtool: Warn on zero-length functions
objtool: Change dead_end_function() to return boolean
objtool: Do frame pointer check before dead end check
objtool: Refactor sibling call detection logic
objtool: Refactor jump table code
objtool: Fix seg fault on bad switch table entry
objtool: Convert insn type to enum
objtool: Support conditional retpolines
Michael Forney (2):
objtool: Use Elf_Scn typedef instead of assuming struct name
objtool: Rename elf_open() to prevent conflict with libelf from elftoolchain
Peter Zijlstra (1):
stacktrace: Force USER_DS for stack_trace_save_user()
arch/x86/entry/thunk_64.S | 5 +-
arch/x86/include/asm/kvm_host.h | 34 ++--
arch/x86/include/asm/paravirt.h | 1 +
arch/x86/kernel/head_64.S | 4 +-
arch/x86/kernel/kvm.c | 1 +
arch/x86/kvm/emulate.c | 44 ++++--
arch/x86/kvm/vmx/vmenter.S | 6 +-
arch/x86/lib/copy_user_64.S | 2 +-
arch/x86/lib/getuser.S | 20 +--
arch/x86/lib/putuser.S | 29 ++--
arch/x86/lib/usercopy_64.c | 2 +-
include/linux/compiler-gcc.h | 2 +
include/linux/compiler.h | 5 +
include/linux/compiler_types.h | 4 +
kernel/bpf/core.c | 5 +-
kernel/stacktrace.c | 5 +
tools/objtool/arch.h | 36 +++--
tools/objtool/arch/x86/decode.c | 2 +-
tools/objtool/check.c | 333 +++++++++++++++++++++-------------------
tools/objtool/check.h | 3 +-
tools/objtool/elf.c | 8 +-
tools/objtool/elf.h | 5 +-
22 files changed, 311 insertions(+), 245 deletions(-)
diff --git a/arch/x86/entry/thunk_64.S b/arch/x86/entry/thunk_64.S
index cfdca8b42c70..cc20465b2867 100644
--- a/arch/x86/entry/thunk_64.S
+++ b/arch/x86/entry/thunk_64.S
@@ -12,9 +12,7 @@
/* rdi: arg1 ... normal C conventions. rax is saved/restored. */
.macro THUNK name, func, put_ret_addr_in_rdi=0
- .globl \name
- .type \name, @function
-\name:
+ ENTRY(\name)
pushq %rbp
movq %rsp, %rbp
@@ -35,6 +33,7 @@
call \func
jmp .L_restore
+ ENDPROC(\name)
_ASM_NOKPROBE(\name)
.endm
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 0cc5b611a113..8282b8d41209 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1496,25 +1496,29 @@ enum {
#define kvm_arch_vcpu_memslots_id(vcpu) ((vcpu)->arch.hflags & HF_SMM_MASK ? 1 : 0)
#define kvm_memslots_for_spte_role(kvm, role) __kvm_memslots(kvm, (role).smm)
+asmlinkage void __noreturn kvm_spurious_fault(void);
+
/*
* Hardware virtualization extension instructions may fault if a
* reboot turns off virtualization while processes are running.
- * Trap the fault and ignore the instruction if that happens.
+ * Usually after catching the fault we just panic; during reboot
+ * instead the instruction is ignored.
*/
-asmlinkage void kvm_spurious_fault(void);
-
-#define ____kvm_handle_fault_on_reboot(insn, cleanup_insn) \
- "666: " insn "\n\t" \
- "668: \n\t" \
- ".pushsection .fixup, \"ax\" \n" \
- "667: \n\t" \
- cleanup_insn "\n\t" \
- "cmpb $0, kvm_rebooting \n\t" \
- "jne 668b \n\t" \
- __ASM_SIZE(push) " $666b \n\t" \
- "jmp kvm_spurious_fault \n\t" \
- ".popsection \n\t" \
- _ASM_EXTABLE(666b, 667b)
+#define ____kvm_handle_fault_on_reboot(insn, cleanup_insn) \
+ "666: \n\t" \
+ insn "\n\t" \
+ "jmp 668f \n\t" \
+ "667: \n\t" \
+ "call kvm_spurious_fault \n\t" \
+ "668: \n\t" \
+ ".pushsection .fixup, \"ax\" \n\t" \
+ "700: \n\t" \
+ cleanup_insn "\n\t" \
+ "cmpb $0, kvm_rebooting\n\t" \
+ "je 667b \n\t" \
+ "jmp 668b \n\t" \
+ ".popsection \n\t" \
+ _ASM_EXTABLE(666b, 700b)
#define __kvm_handle_fault_on_reboot(insn) \
____kvm_handle_fault_on_reboot(insn, "")
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h
index c25c38a05c1c..d6f5ae2c79ab 100644
--- a/arch/x86/include/asm/paravirt.h
+++ b/arch/x86/include/asm/paravirt.h
@@ -746,6 +746,7 @@ bool __raw_callee_save___native_vcpu_is_preempted(long cpu);
PV_RESTORE_ALL_CALLER_REGS \
FRAME_END \
"ret;" \
+ ".size " PV_THUNK_NAME(func) ", .-" PV_THUNK_NAME(func) ";" \
".popsection")
/* Get a reference to a callee-save function */
diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index bcd206c8ac90..66b4a7757397 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -253,10 +253,10 @@ END(secondary_startup_64)
* start_secondary() via .Ljump_to_C_code.
*/
ENTRY(start_cpu0)
- movq initial_stack(%rip), %rsp
UNWIND_HINT_EMPTY
+ movq initial_stack(%rip), %rsp
jmp .Ljump_to_C_code
-ENDPROC(start_cpu0)
+END(start_cpu0)
#endif
/* Both SMP bootup and ACPI suspend change these variables */
diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 82caf01b63dd..6661bd2f08a6 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -838,6 +838,7 @@ asm(
"cmpb $0, " __stringify(KVM_STEAL_TIME_preempted) "+steal_time(%rax);"
"setne %al;"
"ret;"
+".size __raw_callee_save___kvm_vcpu_is_preempted, .-__raw_callee_save___kvm_vcpu_is_preempted;"
".popsection");
#endif
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 8e409ad448f9..718f7d9afedc 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -312,29 +312,42 @@ static void invalidate_registers(struct x86_emulate_ctxt *ctxt)
static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
-#define FOP_FUNC(name) \
+#define __FOP_FUNC(name) \
".align " __stringify(FASTOP_SIZE) " \n\t" \
".type " name ", @function \n\t" \
name ":\n\t"
-#define FOP_RET "ret \n\t"
+#define FOP_FUNC(name) \
+ __FOP_FUNC(#name)
+
+#define __FOP_RET(name) \
+ "ret \n\t" \
+ ".size " name ", .-" name "\n\t"
+
+#define FOP_RET(name) \
+ __FOP_RET(#name)
#define FOP_START(op) \
extern void em_##op(struct fastop *fake); \
asm(".pushsection .text, \"ax\" \n\t" \
".global em_" #op " \n\t" \
- FOP_FUNC("em_" #op)
+ ".align " __stringify(FASTOP_SIZE) " \n\t" \
+ "em_" #op ":\n\t"
#define FOP_END \
".popsection")
+#define __FOPNOP(name) \
+ __FOP_FUNC(name) \
+ __FOP_RET(name)
+
#define FOPNOP() \
- FOP_FUNC(__stringify(__UNIQUE_ID(nop))) \
- FOP_RET
+ __FOPNOP(__stringify(__UNIQUE_ID(nop)))
#define FOP1E(op, dst) \
- FOP_FUNC(#op "_" #dst) \
- "10: " #op " %" #dst " \n\t" FOP_RET
+ __FOP_FUNC(#op "_" #dst) \
+ "10: " #op " %" #dst " \n\t" \
+ __FOP_RET(#op "_" #dst)
#define FOP1EEX(op, dst) \
FOP1E(op, dst) _ASM_EXTABLE(10b, kvm_fastop_exception)
@@ -366,8 +379,9 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
FOP_END
#define FOP2E(op, dst, src) \
- FOP_FUNC(#op "_" #dst "_" #src) \
- #op " %" #src ", %" #dst " \n\t" FOP_RET
+ __FOP_FUNC(#op "_" #dst "_" #src) \
+ #op " %" #src ", %" #dst " \n\t" \
+ __FOP_RET(#op "_" #dst "_" #src)
#define FASTOP2(op) \
FOP_START(op) \
@@ -405,8 +419,9 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
FOP_END
#define FOP3E(op, dst, src, src2) \
- FOP_FUNC(#op "_" #dst "_" #src "_" #src2) \
- #op " %" #src2 ", %" #src ", %" #dst " \n\t" FOP_RET
+ __FOP_FUNC(#op "_" #dst "_" #src "_" #src2) \
+ #op " %" #src2 ", %" #src ", %" #dst " \n\t"\
+ __FOP_RET(#op "_" #dst "_" #src "_" #src2)
/* 3-operand, word-only, src2=cl */
#define FASTOP3WCL(op) \
@@ -423,7 +438,7 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
".type " #op ", @function \n\t" \
#op ": \n\t" \
#op " %al \n\t" \
- FOP_RET
+ __FOP_RET(#op)
asm(".pushsection .fixup, \"ax\"\n"
".global kvm_fastop_exception \n"
@@ -449,7 +464,10 @@ FOP_SETCC(setle)
FOP_SETCC(setnle)
FOP_END;
-FOP_START(salc) "pushf; sbb %al, %al; popf \n\t" FOP_RET
+FOP_START(salc)
+FOP_FUNC(salc)
+"pushf; sbb %al, %al; popf \n\t"
+FOP_RET(salc)
FOP_END;
/*
diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S
index d4cb1945b2e3..4010d519eb8c 100644
--- a/arch/x86/kvm/vmx/vmenter.S
+++ b/arch/x86/kvm/vmx/vmenter.S
@@ -54,9 +54,9 @@ ENTRY(vmx_vmenter)
ret
3: cmpb $0, kvm_rebooting
- jne 4f
- call kvm_spurious_fault
-4: ret
+ je 4f
+ ret
+4: ud2
.pushsection .fixup, "ax"
5: jmp 3b
diff --git a/arch/x86/lib/copy_user_64.S b/arch/x86/lib/copy_user_64.S
index 378a1f70ae7d..4fe1601dbc5d 100644
--- a/arch/x86/lib/copy_user_64.S
+++ b/arch/x86/lib/copy_user_64.S
@@ -239,7 +239,7 @@ copy_user_handle_tail:
ret
_ASM_EXTABLE_UA(1b, 2b)
-ENDPROC(copy_user_handle_tail)
+END(copy_user_handle_tail)
/*
* copy_user_nocache - Uncached memory copy with exception handling
diff --git a/arch/x86/lib/getuser.S b/arch/x86/lib/getuser.S
index 74fdff968ea3..304f958c27b2 100644
--- a/arch/x86/lib/getuser.S
+++ b/arch/x86/lib/getuser.S
@@ -115,29 +115,29 @@ ENDPROC(__get_user_8)
EXPORT_SYMBOL(__get_user_8)
+bad_get_user_clac:
+ ASM_CLAC
bad_get_user:
xor %edx,%edx
mov $(-EFAULT),%_ASM_AX
- ASM_CLAC
ret
-END(bad_get_user)
#ifdef CONFIG_X86_32
+bad_get_user_8_clac:
+ ASM_CLAC
bad_get_user_8:
xor %edx,%edx
xor %ecx,%ecx
mov $(-EFAULT),%_ASM_AX
- ASM_CLAC
ret
-END(bad_get_user_8)
#endif
- _ASM_EXTABLE_UA(1b, bad_get_user)
- _ASM_EXTABLE_UA(2b, bad_get_user)
- _ASM_EXTABLE_UA(3b, bad_get_user)
+ _ASM_EXTABLE_UA(1b, bad_get_user_clac)
+ _ASM_EXTABLE_UA(2b, bad_get_user_clac)
+ _ASM_EXTABLE_UA(3b, bad_get_user_clac)
#ifdef CONFIG_X86_64
- _ASM_EXTABLE_UA(4b, bad_get_user)
+ _ASM_EXTABLE_UA(4b, bad_get_user_clac)
#else
- _ASM_EXTABLE_UA(4b, bad_get_user_8)
- _ASM_EXTABLE_UA(5b, bad_get_user_8)
+ _ASM_EXTABLE_UA(4b, bad_get_user_8_clac)
+ _ASM_EXTABLE_UA(5b, bad_get_user_8_clac)
#endif
diff --git a/arch/x86/lib/putuser.S b/arch/x86/lib/putuser.S
index d2e5c9c39601..14bf78341d3c 100644
--- a/arch/x86/lib/putuser.S
+++ b/arch/x86/lib/putuser.S
@@ -32,8 +32,6 @@
*/
#define ENTER mov PER_CPU_VAR(current_task), %_ASM_BX
-#define EXIT ASM_CLAC ; \
- ret
.text
ENTRY(__put_user_1)
@@ -43,7 +41,8 @@ ENTRY(__put_user_1)
ASM_STAC
1: movb %al,(%_ASM_CX)
xor %eax,%eax
- EXIT
+ ASM_CLAC
+ ret
ENDPROC(__put_user_1)
EXPORT_SYMBOL(__put_user_1)
@@ -56,7 +55,8 @@ ENTRY(__put_user_2)
ASM_STAC
2: movw %ax,(%_ASM_CX)
xor %eax,%eax
- EXIT
+ ASM_CLAC
+ ret
ENDPROC(__put_user_2)
EXPORT_SYMBOL(__put_user_2)
@@ -69,7 +69,8 @@ ENTRY(__put_user_4)
ASM_STAC
3: movl %eax,(%_ASM_CX)
xor %eax,%eax
- EXIT
+ ASM_CLAC
+ ret
ENDPROC(__put_user_4)
EXPORT_SYMBOL(__put_user_4)
@@ -85,19 +86,21 @@ ENTRY(__put_user_8)
5: movl %edx,4(%_ASM_CX)
#endif
xor %eax,%eax
- EXIT
+ ASM_CLAC
+ RET
ENDPROC(__put_user_8)
EXPORT_SYMBOL(__put_user_8)
+bad_put_user_clac:
+ ASM_CLAC
bad_put_user:
movl $-EFAULT,%eax
- EXIT
-END(bad_put_user)
+ RET
- _ASM_EXTABLE_UA(1b, bad_put_user)
- _ASM_EXTABLE_UA(2b, bad_put_user)
- _ASM_EXTABLE_UA(3b, bad_put_user)
- _ASM_EXTABLE_UA(4b, bad_put_user)
+ _ASM_EXTABLE_UA(1b, bad_put_user_clac)
+ _ASM_EXTABLE_UA(2b, bad_put_user_clac)
+ _ASM_EXTABLE_UA(3b, bad_put_user_clac)
+ _ASM_EXTABLE_UA(4b, bad_put_user_clac)
#ifdef CONFIG_X86_32
- _ASM_EXTABLE_UA(5b, bad_put_user)
+ _ASM_EXTABLE_UA(5b, bad_put_user_clac)
#endif
diff --git a/arch/x86/lib/usercopy_64.c b/arch/x86/lib/usercopy_64.c
index e0e006f1624e..fff28c6f73a2 100644
--- a/arch/x86/lib/usercopy_64.c
+++ b/arch/x86/lib/usercopy_64.c
@@ -60,7 +60,7 @@ EXPORT_SYMBOL(clear_user);
* but reuse __memcpy_mcsafe in case a new read error is encountered.
* clac() is handled in _copy_to_iter_mcsafe().
*/
-__visible unsigned long
+__visible notrace unsigned long
mcsafe_handle_tail(char *to, char *from, unsigned len)
{
for (; len; --len, to++, from++) {
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index e8579412ad21..d7ee4c6bad48 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -170,3 +170,5 @@
#else
#define __diag_GCC_8(s)
#endif
+
+#define __no_fgcse __attribute__((optimize("-fno-gcse")))
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 8aaf7cd026b0..f0fd5636fddb 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -116,9 +116,14 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
".pushsection .discard.unreachable\n\t" \
".long 999b - .\n\t" \
".popsection\n\t"
+
+/* Annotate a C jump table to allow objtool to follow the code flow */
+#define __annotate_jump_table __section(".rodata..c_jump_table")
+
#else
#define annotate_reachable()
#define annotate_unreachable()
+#define __annotate_jump_table
#endif
#ifndef ASM_UNREACHABLE
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index 095d55c3834d..599c27b56c29 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -189,6 +189,10 @@ struct ftrace_likely_data {
#define asm_volatile_goto(x...) asm goto(x)
#endif
+#ifndef __no_fgcse
+# define __no_fgcse
+#endif
+
/* Are two types/vars the same type (ignoring qualifiers)? */
#define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 16079550db6d..8191a7db2777 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1295,11 +1295,11 @@ bool bpf_opcode_in_insntable(u8 code)
*
* Decode and execute eBPF instructions.
*/
-static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
+static u64 __no_fgcse ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
{
#define BPF_INSN_2_LBL(x, y) [BPF_##x | BPF_##y] = &&x##_##y
#define BPF_INSN_3_LBL(x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
- static const void *jumptable[256] = {
+ static const void * const jumptable[256] __annotate_jump_table = {
[0 ... 255] = &&default_label,
/* Now overwrite non-defaults ... */
BPF_INSN_MAP(BPF_INSN_2_LBL, BPF_INSN_3_LBL),
@@ -1558,7 +1558,6 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
BUG_ON(1);
return 0;
}
-STACK_FRAME_NON_STANDARD(___bpf_prog_run); /* jump table */
#define PROG_NAME(stack_size) __bpf_prog_run##stack_size
#define DEFINE_BPF_PROG_RUN(stack_size) \
diff --git a/kernel/stacktrace.c b/kernel/stacktrace.c
index e6a02b274b73..f5440abb7532 100644
--- a/kernel/stacktrace.c
+++ b/kernel/stacktrace.c
@@ -226,12 +226,17 @@ unsigned int stack_trace_save_user(unsigned long *store, unsigned int size)
.store = store,
.size = size,
};
+ mm_segment_t fs;
/* Trace user stack if not a kernel thread */
if (current->flags & PF_KTHREAD)
return 0;
+ fs = get_fs();
+ set_fs(USER_DS);
arch_stack_walk_user(consume_entry, &c, task_pt_regs(current));
+ set_fs(fs);
+
return c.len;
}
#endif
diff --git a/tools/objtool/arch.h b/tools/objtool/arch.h
index 580e344db3dd..ced3765c4f44 100644
--- a/tools/objtool/arch.h
+++ b/tools/objtool/arch.h
@@ -11,22 +11,24 @@
#include "elf.h"
#include "cfi.h"
-#define INSN_JUMP_CONDITIONAL 1
-#define INSN_JUMP_UNCONDITIONAL 2
-#define INSN_JUMP_DYNAMIC 3
-#define INSN_CALL 4
-#define INSN_CALL_DYNAMIC 5
-#define INSN_RETURN 6
-#define INSN_CONTEXT_SWITCH 7
-#define INSN_STACK 8
-#define INSN_BUG 9
-#define INSN_NOP 10
-#define INSN_STAC 11
-#define INSN_CLAC 12
-#define INSN_STD 13
-#define INSN_CLD 14
-#define INSN_OTHER 15
-#define INSN_LAST INSN_OTHER
+enum insn_type {
+ INSN_JUMP_CONDITIONAL,
+ INSN_JUMP_UNCONDITIONAL,
+ INSN_JUMP_DYNAMIC,
+ INSN_JUMP_DYNAMIC_CONDITIONAL,
+ INSN_CALL,
+ INSN_CALL_DYNAMIC,
+ INSN_RETURN,
+ INSN_CONTEXT_SWITCH,
+ INSN_STACK,
+ INSN_BUG,
+ INSN_NOP,
+ INSN_STAC,
+ INSN_CLAC,
+ INSN_STD,
+ INSN_CLD,
+ INSN_OTHER,
+};
enum op_dest_type {
OP_DEST_REG,
@@ -68,7 +70,7 @@ void arch_initial_func_cfi_state(struct cfi_state *state);
int arch_decode_instruction(struct elf *elf, struct section *sec,
unsigned long offset, unsigned int maxlen,
- unsigned int *len, unsigned char *type,
+ unsigned int *len, enum insn_type *type,
unsigned long *immediate, struct stack_op *op);
bool arch_callee_saved_reg(unsigned char reg);
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index 584568f27a83..0567c47a91b1 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -68,7 +68,7 @@ bool arch_callee_saved_reg(unsigned char reg)
int arch_decode_instruction(struct elf *elf, struct section *sec,
unsigned long offset, unsigned int maxlen,
- unsigned int *len, unsigned char *type,
+ unsigned int *len, enum insn_type *type,
unsigned long *immediate, struct stack_op *op)
{
struct insn insn;
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 172f99195726..5f26620f13f5 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -18,6 +18,8 @@
#define FAKE_JUMP_OFFSET -1
+#define C_JUMP_TABLE_SECTION ".rodata..c_jump_table"
+
struct alternative {
struct list_head list;
struct instruction *insn;
@@ -95,6 +97,20 @@ static struct instruction *next_insn_same_func(struct objtool_file *file,
for (insn = next_insn_same_sec(file, insn); insn; \
insn = next_insn_same_sec(file, insn))
+static bool is_sibling_call(struct instruction *insn)
+{
+ /* An indirect jump is either a sibling call or a jump to a table. */
+ if (insn->type == INSN_JUMP_DYNAMIC)
+ return list_empty(&insn->alts);
+
+ if (insn->type != INSN_JUMP_CONDITIONAL &&
+ insn->type != INSN_JUMP_UNCONDITIONAL)
+ return false;
+
+ /* add_jump_destinations() sets insn->call_dest for sibling calls. */
+ return !!insn->call_dest;
+}
+
/*
* This checks to see if the given function is a "noreturn" function.
*
@@ -103,14 +119,9 @@ static struct instruction *next_insn_same_func(struct objtool_file *file,
*
* For local functions, we have to detect them manually by simply looking for
* the lack of a return instruction.
- *
- * Returns:
- * -1: error
- * 0: no dead end
- * 1: dead end
*/
-static int __dead_end_function(struct objtool_file *file, struct symbol *func,
- int recursion)
+static bool __dead_end_function(struct objtool_file *file, struct symbol *func,
+ int recursion)
{
int i;
struct instruction *insn;
@@ -136,30 +147,33 @@ static int __dead_end_function(struct objtool_file *file, struct symbol *func,
"rewind_stack_do_exit",
};
+ if (!func)
+ return false;
+
if (func->bind == STB_WEAK)
- return 0;
+ return false;
if (func->bind == STB_GLOBAL)
for (i = 0; i < ARRAY_SIZE(global_noreturns); i++)
if (!strcmp(func->name, global_noreturns[i]))
- return 1;
+ return true;
if (!func->len)
- return 0;
+ return false;
insn = find_insn(file, func->sec, func->offset);
if (!insn->func)
- return 0;
+ return false;
func_for_each_insn_all(file, func, insn) {
empty = false;
if (insn->type == INSN_RETURN)
- return 0;
+ return false;
}
if (empty)
- return 0;
+ return false;
/*
* A function can have a sibling call instead of a return. In that
@@ -167,40 +181,31 @@ static int __dead_end_function(struct objtool_file *file, struct symbol *func,
* of the sibling call returns.
*/
func_for_each_insn_all(file, func, insn) {
- if (insn->type == INSN_JUMP_UNCONDITIONAL) {
+ if (is_sibling_call(insn)) {
struct instruction *dest = insn->jump_dest;
if (!dest)
/* sibling call to another file */
- return 0;
-
- if (dest->func && dest->func->pfunc != insn->func->pfunc) {
+ return false;
- /* local sibling call */
- if (recursion == 5) {
- /*
- * Infinite recursion: two functions
- * have sibling calls to each other.
- * This is a very rare case. It means
- * they aren't dead ends.
- */
- return 0;
- }
-
- return __dead_end_function(file, dest->func,
- recursion + 1);
+ /* local sibling call */
+ if (recursion == 5) {
+ /*
+ * Infinite recursion: two functions have
+ * sibling calls to each other. This is a very
+ * rare case. It means they aren't dead ends.
+ */
+ return false;
}
- }
- if (insn->type == INSN_JUMP_DYNAMIC && list_empty(&insn->alts))
- /* sibling call */
- return 0;
+ return __dead_end_function(file, dest->func, recursion+1);
+ }
}
- return 1;
+ return true;
}
-static int dead_end_function(struct objtool_file *file, struct symbol *func)
+static bool dead_end_function(struct objtool_file *file, struct symbol *func)
{
return __dead_end_function(file, func, 0);
}
@@ -262,19 +267,12 @@ static int decode_instructions(struct objtool_file *file)
if (ret)
goto err;
- if (!insn->type || insn->type > INSN_LAST) {
- WARN_FUNC("invalid instruction type %d",
- insn->sec, insn->offset, insn->type);
- ret = -1;
- goto err;
- }
-
hash_add(file->insn_hash, &insn->hash, insn->offset);
list_add_tail(&insn->list, &file->insn_list);
}
list_for_each_entry(func, &sec->symbol_list, list) {
- if (func->type != STT_FUNC)
+ if (func->type != STT_FUNC || func->alias != func)
continue;
if (!find_insn(file, sec, func->offset)) {
@@ -284,8 +282,7 @@ static int decode_instructions(struct objtool_file *file)
}
func_for_each_insn(file, func, insn)
- if (!insn->func)
- insn->func = func;
+ insn->func = func;
}
}
@@ -488,6 +485,7 @@ static const char *uaccess_safe_builtin[] = {
/* misc */
"csum_partial_copy_generic",
"__memcpy_mcsafe",
+ "mcsafe_handle_tail",
"ftrace_likely_update", /* CONFIG_TRACE_BRANCH_PROFILING */
NULL
};
@@ -505,7 +503,7 @@ static void add_uaccess_safe(struct objtool_file *file)
if (!func)
continue;
- func->alias->uaccess_safe = true;
+ func->uaccess_safe = true;
}
}
@@ -577,13 +575,16 @@ static int add_jump_destinations(struct objtool_file *file)
* Retpoline jumps are really dynamic jumps in
* disguise, so convert them accordingly.
*/
- insn->type = INSN_JUMP_DYNAMIC;
+ if (insn->type == INSN_JUMP_UNCONDITIONAL)
+ insn->type = INSN_JUMP_DYNAMIC;
+ else
+ insn->type = INSN_JUMP_DYNAMIC_CONDITIONAL;
+
insn->retpoline_safe = true;
continue;
} else {
- /* sibling call */
+ /* external sibling call */
insn->call_dest = rela->sym;
- insn->jump_dest = NULL;
continue;
}
@@ -623,7 +624,7 @@ static int add_jump_destinations(struct objtool_file *file)
* However this code can't completely replace the
* read_symbols() code because this doesn't detect the
* case where the parent function's only reference to a
- * subfunction is through a switch table.
+ * subfunction is through a jump table.
*/
if (!strstr(insn->func->name, ".cold.") &&
strstr(insn->jump_dest->func->name, ".cold.")) {
@@ -633,9 +634,8 @@ static int add_jump_destinations(struct objtool_file *file)
} else if (insn->jump_dest->func->pfunc != insn->func->pfunc &&
insn->jump_dest->offset == insn->jump_dest->func->offset) {
- /* sibling class */
+ /* internal sibling call */
insn->call_dest = insn->jump_dest->func;
- insn->jump_dest = NULL;
}
}
}
@@ -896,20 +896,26 @@ static int add_special_section_alts(struct objtool_file *file)
return ret;
}
-static int add_switch_table(struct objtool_file *file, struct instruction *insn,
- struct rela *table, struct rela *next_table)
+static int add_jump_table(struct objtool_file *file, struct instruction *insn,
+ struct rela *table)
{
struct rela *rela = table;
- struct instruction *alt_insn;
+ struct instruction *dest_insn;
struct alternative *alt;
struct symbol *pfunc = insn->func->pfunc;
unsigned int prev_offset = 0;
- list_for_each_entry_from(rela, &table->rela_sec->rela_list, list) {
- if (rela == next_table)
+ /*
+ * Each @rela is a switch table relocation which points to the target
+ * instruction.
+ */
+ list_for_each_entry_from(rela, &table->sec->rela_list, list) {
+
+ /* Check for the end of the table: */
+ if (rela != table && rela->jump_table_start)
break;
- /* Make sure the switch table entries are consecutive: */
+ /* Make sure the table entries are consecutive: */
if (prev_offset && rela->offset != prev_offset + 8)
break;
@@ -918,12 +924,12 @@ static int add_switch_table(struct objtool_file *file, struct instruction *insn,
rela->addend == pfunc->offset)
break;
- alt_insn = find_insn(file, rela->sym->sec, rela->addend);
- if (!alt_insn)
+ dest_insn = find_insn(file, rela->sym->sec, rela->addend);
+ if (!dest_insn)
break;
- /* Make sure the jmp dest is in the function or subfunction: */
- if (alt_insn->func->pfunc != pfunc)
+ /* Make sure the destination is in the same function: */
+ if (!dest_insn->func || dest_insn->func->pfunc != pfunc)
break;
alt = malloc(sizeof(*alt));
@@ -932,7 +938,7 @@ static int add_switch_table(struct objtool_file *file, struct instruction *insn,
return -1;
}
- alt->insn = alt_insn;
+ alt->insn = dest_insn;
list_add_tail(&alt->list, &insn->alts);
prev_offset = rela->offset;
}
@@ -947,7 +953,7 @@ static int add_switch_table(struct objtool_file *file, struct instruction *insn,
}
/*
- * find_switch_table() - Given a dynamic jump, find the switch jump table in
+ * find_jump_table() - Given a dynamic jump, find the switch jump table in
* .rodata associated with it.
*
* There are 3 basic patterns:
@@ -989,13 +995,13 @@ static int add_switch_table(struct objtool_file *file, struct instruction *insn,
*
* NOTE: RETPOLINE made it harder still to decode dynamic jumps.
*/
-static struct rela *find_switch_table(struct objtool_file *file,
+static struct rela *find_jump_table(struct objtool_file *file,
struct symbol *func,
struct instruction *insn)
{
- struct rela *text_rela, *rodata_rela;
+ struct rela *text_rela, *table_rela;
struct instruction *orig_insn = insn;
- struct section *rodata_sec;
+ struct section *table_sec;
unsigned long table_offset;
/*
@@ -1028,42 +1034,52 @@ static struct rela *find_switch_table(struct objtool_file *file,
continue;
table_offset = text_rela->addend;
- rodata_sec = text_rela->sym->sec;
+ table_sec = text_rela->sym->sec;
if (text_rela->type == R_X86_64_PC32)
table_offset += 4;
/*
* Make sure the .rodata address isn't associated with a
- * symbol. gcc jump tables are anonymous data.
+ * symbol. GCC jump tables are anonymous data.
+ *
+ * Also support C jump tables which are in the same format as
+ * switch jump tables. For objtool to recognize them, they
+ * need to be placed in the C_JUMP_TABLE_SECTION section. They
+ * have symbols associated with them.
*/
- if (find_symbol_containing(rodata_sec, table_offset))
+ if (find_symbol_containing(table_sec, table_offset) &&
+ strcmp(table_sec->name, C_JUMP_TABLE_SECTION))
continue;
- rodata_rela = find_rela_by_dest(rodata_sec, table_offset);
- if (rodata_rela) {
- /*
- * Use of RIP-relative switch jumps is quite rare, and
- * indicates a rare GCC quirk/bug which can leave dead
- * code behind.
- */
- if (text_rela->type == R_X86_64_PC32)
- file->ignore_unreachables = true;
+ /* Each table entry has a rela associated with it. */
+ table_rela = find_rela_by_dest(table_sec, table_offset);
+ if (!table_rela)
+ continue;
- return rodata_rela;
- }
+ /*
+ * Use of RIP-relative switch jumps is quite rare, and
+ * indicates a rare GCC quirk/bug which can leave dead code
+ * behind.
+ */
+ if (text_rela->type == R_X86_64_PC32)
+ file->ignore_unreachables = true;
+
+ return table_rela;
}
return NULL;
}
-
-static int add_func_switch_tables(struct objtool_file *file,
- struct symbol *func)
+/*
+ * First pass: Mark the head of each jump table so that in the next pass,
+ * we know when a given jump table ends and the next one starts.
+ */
+static void mark_func_jump_tables(struct objtool_file *file,
+ struct symbol *func)
{
- struct instruction *insn, *last = NULL, *prev_jump = NULL;
- struct rela *rela, *prev_rela = NULL;
- int ret;
+ struct instruction *insn, *last = NULL;
+ struct rela *rela;
func_for_each_insn_all(file, func, insn) {
if (!last)
@@ -1071,7 +1087,7 @@ static int add_func_switch_tables(struct objtool_file *file,
/*
* Store back-pointers for unconditional forward jumps such
- * that find_switch_table() can back-track using those and
+ * that find_jump_table() can back-track using those and
* avoid some potentially confusing code.
*/
if (insn->type == INSN_JUMP_UNCONDITIONAL && insn->jump_dest &&
@@ -1086,27 +1102,25 @@ static int add_func_switch_tables(struct objtool_file *file,
if (insn->type != INSN_JUMP_DYNAMIC)
continue;
- rela = find_switch_table(file, func, insn);
- if (!rela)
- continue;
-
- /*
- * We found a switch table, but we don't know yet how big it
- * is. Don't add it until we reach the end of the function or
- * the beginning of another switch table in the same function.
- */
- if (prev_jump) {
- ret = add_switch_table(file, prev_jump, prev_rela, rela);
- if (ret)
- return ret;
+ rela = find_jump_table(file, func, insn);
+ if (rela) {
+ rela->jump_table_start = true;
+ insn->jump_table = rela;
}
-
- prev_jump = insn;
- prev_rela = rela;
}
+}
+
+static int add_func_jump_tables(struct objtool_file *file,
+ struct symbol *func)
+{
+ struct instruction *insn;
+ int ret;
- if (prev_jump) {
- ret = add_switch_table(file, prev_jump, prev_rela, NULL);
+ func_for_each_insn_all(file, func, insn) {
+ if (!insn->jump_table)
+ continue;
+
+ ret = add_jump_table(file, insn, insn->jump_table);
if (ret)
return ret;
}
@@ -1119,7 +1133,7 @@ static int add_func_switch_tables(struct objtool_file *file,
* section which contains a list of addresses within the function to jump to.
* This finds these jump tables and adds them to the insn->alts lists.
*/
-static int add_switch_table_alts(struct objtool_file *file)
+static int add_jump_table_alts(struct objtool_file *file)
{
struct section *sec;
struct symbol *func;
@@ -1133,7 +1147,8 @@ static int add_switch_table_alts(struct objtool_file *file)
if (func->type != STT_FUNC)
continue;
- ret = add_func_switch_tables(file, func);
+ mark_func_jump_tables(file, func);
+ ret = add_func_jump_tables(file, func);
if (ret)
return ret;
}
@@ -1277,13 +1292,18 @@ static void mark_rodata(struct objtool_file *file)
bool found = false;
/*
- * This searches for the .rodata section or multiple .rodata.func_name
- * sections if -fdata-sections is being used. The .str.1.1 and .str.1.8
- * rodata sections are ignored as they don't contain jump tables.
+ * Search for the following rodata sections, each of which can
+ * potentially contain jump tables:
+ *
+ * - .rodata: can contain GCC switch tables
+ * - .rodata.<func>: same, if -fdata-sections is being used
+ * - .rodata..c_jump_table: contains C annotated jump tables
+ *
+ * .rodata.str1.* sections are ignored; they don't contain jump tables.
*/
for_each_sec(file, sec) {
- if (!strncmp(sec->name, ".rodata", 7) &&
- !strstr(sec->name, ".str1.")) {
+ if ((!strncmp(sec->name, ".rodata", 7) && !strstr(sec->name, ".str1.")) ||
+ !strcmp(sec->name, C_JUMP_TABLE_SECTION)) {
sec->rodata = true;
found = true;
}
@@ -1325,7 +1345,7 @@ static int decode_sections(struct objtool_file *file)
if (ret)
return ret;
- ret = add_switch_table_alts(file);
+ ret = add_jump_table_alts(file);
if (ret)
return ret;
@@ -1873,12 +1893,12 @@ static bool insn_state_match(struct instruction *insn, struct insn_state *state)
static inline bool func_uaccess_safe(struct symbol *func)
{
if (func)
- return func->alias->uaccess_safe;
+ return func->uaccess_safe;
return false;
}
-static inline const char *insn_dest_name(struct instruction *insn)
+static inline const char *call_dest_name(struct instruction *insn)
{
if (insn->call_dest)
return insn->call_dest->name;
@@ -1890,13 +1910,13 @@ static int validate_call(struct instruction *insn, struct insn_state *state)
{
if (state->uaccess && !func_uaccess_safe(insn->call_dest)) {
WARN_FUNC("call to %s() with UACCESS enabled",
- insn->sec, insn->offset, insn_dest_name(insn));
+ insn->sec, insn->offset, call_dest_name(insn));
return 1;
}
if (state->df) {
WARN_FUNC("call to %s() with DF set",
- insn->sec, insn->offset, insn_dest_name(insn));
+ insn->sec, insn->offset, call_dest_name(insn));
return 1;
}
@@ -1920,13 +1940,12 @@ static int validate_sibling_call(struct instruction *insn, struct insn_state *st
* each instruction and validate all the rules described in
* tools/objtool/Documentation/stack-validation.txt.
*/
-static int validate_branch(struct objtool_file *file, struct instruction *first,
- struct insn_state state)
+static int validate_branch(struct objtool_file *file, struct symbol *func,
+ struct instruction *first, struct insn_state state)
{
struct alternative *alt;
struct instruction *insn, *next_insn;
struct section *sec;
- struct symbol *func = NULL;
int ret;
insn = first;
@@ -1947,9 +1966,6 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
return 1;
}
- if (insn->func)
- func = insn->func->pfunc;
-
if (func && insn->ignore) {
WARN_FUNC("BUG: why am I validating an ignored function?",
sec, insn->offset);
@@ -1971,7 +1987,7 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
i = insn;
save_insn = NULL;
- func_for_each_insn_continue_reverse(file, insn->func, i) {
+ func_for_each_insn_continue_reverse(file, func, i) {
if (i->save) {
save_insn = i;
break;
@@ -2017,7 +2033,7 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
if (alt->skip_orig)
skip_orig = true;
- ret = validate_branch(file, alt->insn, state);
+ ret = validate_branch(file, func, alt->insn, state);
if (ret) {
if (backtrace)
BT_FUNC("(alt)", insn);
@@ -2055,7 +2071,7 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
if (state.bp_scratch) {
WARN("%s uses BP as a scratch register",
- insn->func->name);
+ func->name);
return 1;
}
@@ -2067,36 +2083,28 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
if (ret)
return ret;
- if (insn->type == INSN_CALL) {
- if (is_fentry_call(insn))
- break;
-
- ret = dead_end_function(file, insn->call_dest);
- if (ret == 1)
- return 0;
- if (ret == -1)
- return 1;
- }
-
- if (!no_fp && func && !has_valid_stack_frame(&state)) {
+ if (!no_fp && func && !is_fentry_call(insn) &&
+ !has_valid_stack_frame(&state)) {
WARN_FUNC("call without frame pointer save/setup",
sec, insn->offset);
return 1;
}
+
+ if (dead_end_function(file, insn->call_dest))
+ return 0;
+
break;
case INSN_JUMP_CONDITIONAL:
case INSN_JUMP_UNCONDITIONAL:
- if (func && !insn->jump_dest) {
+ if (func && is_sibling_call(insn)) {
ret = validate_sibling_call(insn, &state);
if (ret)
return ret;
- } else if (insn->jump_dest &&
- (!func || !insn->jump_dest->func ||
- insn->jump_dest->func->pfunc == func)) {
- ret = validate_branch(file, insn->jump_dest,
- state);
+ } else if (insn->jump_dest) {
+ ret = validate_branch(file, func,
+ insn->jump_dest, state);
if (ret) {
if (backtrace)
BT_FUNC("(branch)", insn);
@@ -2110,13 +2118,17 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
break;
case INSN_JUMP_DYNAMIC:
- if (func && list_empty(&insn->alts)) {
+ case INSN_JUMP_DYNAMIC_CONDITIONAL:
+ if (func && is_sibling_call(insn)) {
ret = validate_sibling_call(insn, &state);
if (ret)
return ret;
}
- return 0;
+ if (insn->type == INSN_JUMP_DYNAMIC)
+ return 0;
+
+ break;
case INSN_CONTEXT_SWITCH:
if (func && (!next_insn || !next_insn->hint)) {
@@ -2162,7 +2174,7 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
break;
case INSN_CLAC:
- if (!state.uaccess && insn->func) {
+ if (!state.uaccess && func) {
WARN_FUNC("redundant UACCESS disable", sec, insn->offset);
return 1;
}
@@ -2183,7 +2195,7 @@ static int validate_branch(struct objtool_file *file, struct instruction *first,
break;
case INSN_CLD:
- if (!state.df && insn->func)
+ if (!state.df && func)
WARN_FUNC("redundant CLD", sec, insn->offset);
state.df = false;
@@ -2222,7 +2234,7 @@ static int validate_unwind_hints(struct objtool_file *file)
for_each_insn(file, insn) {
if (insn->hint && !insn->visited) {
- ret = validate_branch(file, insn, state);
+ ret = validate_branch(file, insn->func, insn, state);
if (ret && backtrace)
BT_FUNC("<=== (hint)", insn);
warnings += ret;
@@ -2345,16 +2357,25 @@ static int validate_functions(struct objtool_file *file)
for_each_sec(file, sec) {
list_for_each_entry(func, &sec->symbol_list, list) {
- if (func->type != STT_FUNC || func->pfunc != func)
+ if (func->type != STT_FUNC)
+ continue;
+
+ if (!func->len) {
+ WARN("%s() is missing an ELF size annotation",
+ func->name);
+ warnings++;
+ }
+
+ if (func->pfunc != func || func->alias != func)
continue;
insn = find_insn(file, sec, func->offset);
- if (!insn || insn->ignore)
+ if (!insn || insn->ignore || insn->visited)
continue;
- state.uaccess = func->alias->uaccess_safe;
+ state.uaccess = func->uaccess_safe;
- ret = validate_branch(file, insn, state);
+ ret = validate_branch(file, func, insn, state);
if (ret && backtrace)
BT_FUNC("<=== (func)", insn);
warnings += ret;
@@ -2407,7 +2428,7 @@ int check(const char *_objname, bool orc)
objname = _objname;
- file.elf = elf_open(objname, orc ? O_RDWR : O_RDONLY);
+ file.elf = elf_read(objname, orc ? O_RDWR : O_RDONLY);
if (!file.elf)
return 1;
diff --git a/tools/objtool/check.h b/tools/objtool/check.h
index cb60b9acf5cf..b881fafcf55d 100644
--- a/tools/objtool/check.h
+++ b/tools/objtool/check.h
@@ -31,13 +31,14 @@ struct instruction {
struct section *sec;
unsigned long offset;
unsigned int len;
- unsigned char type;
+ enum insn_type type;
unsigned long immediate;
bool alt_group, visited, dead_end, ignore, hint, save, restore, ignore_alts;
bool retpoline_safe;
struct symbol *call_dest;
struct instruction *jump_dest;
struct instruction *first_jump_src;
+ struct rela *jump_table;
struct list_head alts;
struct symbol *func;
struct stack_op stack_op;
diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index e99e1be19ad9..edba4745f25a 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -278,7 +278,7 @@ static int read_symbols(struct elf *elf)
}
if (sym->offset == s->offset) {
- if (sym->len == s->len && alias == sym)
+ if (sym->len && sym->len == s->len && alias == sym)
alias = s;
if (sym->len >= s->len) {
@@ -385,7 +385,7 @@ static int read_relas(struct elf *elf)
rela->offset = rela->rela.r_offset;
symndx = GELF_R_SYM(rela->rela.r_info);
rela->sym = find_symbol_by_index(elf, symndx);
- rela->rela_sec = sec;
+ rela->sec = sec;
if (!rela->sym) {
WARN("can't find rela entry symbol %d for %s",
symndx, sec->name);
@@ -401,7 +401,7 @@ static int read_relas(struct elf *elf)
return 0;
}
-struct elf *elf_open(const char *name, int flags)
+struct elf *elf_read(const char *name, int flags)
{
struct elf *elf;
Elf_Cmd cmd;
@@ -463,7 +463,7 @@ struct section *elf_create_section(struct elf *elf, const char *name,
{
struct section *sec, *shstrtab;
size_t size = entsize * nr;
- struct Elf_Scn *s;
+ Elf_Scn *s;
Elf_Data *data;
sec = malloc(sizeof(*sec));
diff --git a/tools/objtool/elf.h b/tools/objtool/elf.h
index e44ca5d51871..44150204db4d 100644
--- a/tools/objtool/elf.h
+++ b/tools/objtool/elf.h
@@ -57,11 +57,12 @@ struct rela {
struct list_head list;
struct hlist_node hash;
GElf_Rela rela;
- struct section *rela_sec;
+ struct section *sec;
struct symbol *sym;
unsigned int type;
unsigned long offset;
int addend;
+ bool jump_table_start;
};
struct elf {
@@ -74,7 +75,7 @@ struct elf {
};
-struct elf *elf_open(const char *name, int flags);
+struct elf *elf_read(const char *name, int flags);
struct section *find_section_by_name(struct elf *elf, const char *name);
struct symbol *find_symbol_by_offset(struct section *sec, unsigned long offset);
struct symbol *find_symbol_by_name(struct elf *elf, const char *name);
^ permalink raw reply [flat|nested] 10+ messages in thread
* [GIT pull] perf/urgent for 5.3-rc1
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
2019-07-20 12:50 ` [GIT pull] smp/urgent " Thomas Gleixner
@ 2019-07-20 12:50 ` Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] sched/urgent " Thomas Gleixner
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Thomas Gleixner @ 2019-07-20 12:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest perf-urgent-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf-urgent-for-linus
up to: e0c5c5e308ee: Merge tag 'perf-core-for-mingo-5.3-20190715' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent
A set of perf improvements and fixes:
perf db-export:
- Improvements in how COMM details are exported to databases for
post processing and use in the sql-viewer.py UI.
- Export switch events to the database.
BPF:
- Bump rlimit(MEMLOCK) for 'perf test bpf' and 'perf trace', just like
selftests/bpf/bpf_rlimit.h do, which makes errors due to exhaustion of
this limit, which are kinda cryptic (EPERM sometimes) less frequent.
perf version:
- Fix segfault due to missing OPT_END(), noticed on PowerPC.
perf vendor events:
- Add JSON files for IBM s/390 machine type 8561.
perf cs-etm (ARM):
- Fix two cases of error returns not bing done properly: Invalid ERR_PTR() use
and loss of propagation error codes.
Thanks,
tglx
------------------>
Adrian Hunter (21):
perf db-export: Get rid of db_export__deferred()
perf db-export: Rename db_export__comm() to db_export__exec_comm()
perf db-export: Pass main_thread to db_export__thread()
perf db-export: Export main_thread in db_export__sample()
perf db-export: Export comm before exporting thread
perf db-export: Move export__comm_thread into db_export__sample()
perf db-export: Fix a white space issue in db_export__sample()
perf db-export: Export comm details
perf scripts python: export-to-sqlite.py: Export comm details
perf scripts python: export-to-postgresql.py: Export comm details
perf db-export: Factor out db_export__comm()
perf db-export: Also export thread's current comm
perf scripts python: export-to-sqlite.py: Add has_calls column to comms table
perf scripts python: export-to-postgresql.py: Add has_calls column to comms table
perf scripts python: exported-sql-viewer.py: Remove redundant semi-colons
perf scripts python: exported-sql-viewer.py: Use new 'has_calls' column
perf script: Add scripting operation process_switch()
perf db-export: Factor out db_export__threads()
perf db-export: Export switch events
perf scripts python: export-to-sqlite.py: Export switch events
perf scripts python: export-to-postgresql.py: Export switch events
Arnaldo Carvalho de Melo (3):
perf tools: Introduce rlimit__bump_memlock() helper
perf test: Auto bump rlimit(MEMLOCK) for BPF test sake
perf trace: Auto bump rlimit(MEMLOCK) for eBPF maps sake
Ravi Bangoria (1):
perf version: Fix segfault due to missing OPT_END()
Thomas Richter (1):
perf vendor events s390: Add JSON files for machine type 8561
YueHaibing (2):
perf cs-etm: Remove errnoeous ERR_PTR() usage in cs_etm__process_auxtrace_info
perf cs-etm: Return errcode in cs_etm__process_auxtrace_info()
tools/perf/builtin-script.c | 8 +-
tools/perf/builtin-trace.c | 10 +
tools/perf/builtin-version.c | 1 +
.../perf/pmu-events/arch/s390/cf_m8561/basic.json | 58 ++++
.../perf/pmu-events/arch/s390/cf_m8561/crypto.json | 114 +++++++
.../pmu-events/arch/s390/cf_m8561/crypto6.json | 30 ++
.../pmu-events/arch/s390/cf_m8561/extended.json | 373 +++++++++++++++++++++
tools/perf/pmu-events/arch/s390/mapfile.csv | 1 +
tools/perf/scripts/python/export-to-postgresql.py | 68 +++-
tools/perf/scripts/python/export-to-sqlite.py | 54 ++-
tools/perf/scripts/python/exported-sql-viewer.py | 34 +-
tools/perf/tests/builtin-test.c | 6 +
tools/perf/util/Build | 1 +
tools/perf/util/cs-etm.c | 12 +-
tools/perf/util/db-export.c | 291 ++++++++++------
tools/perf/util/db-export.h | 19 +-
tools/perf/util/rlimit.c | 29 ++
tools/perf/util/rlimit.h | 6 +
.../util/scripting-engines/trace-event-python.c | 53 ++-
tools/perf/util/trace-event.h | 3 +
20 files changed, 1029 insertions(+), 142 deletions(-)
create mode 100644 tools/perf/pmu-events/arch/s390/cf_m8561/basic.json
create mode 100644 tools/perf/pmu-events/arch/s390/cf_m8561/crypto.json
create mode 100644 tools/perf/pmu-events/arch/s390/cf_m8561/crypto6.json
create mode 100644 tools/perf/pmu-events/arch/s390/cf_m8561/extended.json
create mode 100644 tools/perf/util/rlimit.c
create mode 100644 tools/perf/util/rlimit.h
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 79367087bd18..8f24865596af 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -2289,6 +2289,12 @@ static int process_switch_event(struct perf_tool *tool,
if (perf_event__process_switch(tool, event, sample, machine) < 0)
return -1;
+ if (scripting_ops && scripting_ops->process_switch)
+ scripting_ops->process_switch(event, sample, machine);
+
+ if (!script->show_switch_events)
+ return 0;
+
thread = machine__findnew_thread(machine, sample->pid,
sample->tid);
if (thread == NULL) {
@@ -2467,7 +2473,7 @@ static int __cmd_script(struct perf_script *script)
script->tool.mmap = process_mmap_event;
script->tool.mmap2 = process_mmap2_event;
}
- if (script->show_switch_events)
+ if (script->show_switch_events || (scripting_ops && scripting_ops->process_switch))
script->tool.context_switch = process_switch_event;
if (script->show_namespace_events)
script->tool.namespaces = process_namespaces_event;
diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index 1aa2ed096f65..4f0bbffee05f 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -19,6 +19,7 @@
#include <api/fs/tracing_path.h>
#include <bpf/bpf.h>
#include "util/bpf_map.h"
+#include "util/rlimit.h"
#include "builtin.h"
#include "util/cgroup.h"
#include "util/color.h"
@@ -3864,6 +3865,15 @@ int cmd_trace(int argc, const char **argv)
goto out;
}
+ /*
+ * Parsing .perfconfig may entail creating a BPF event, that may need
+ * to create BPF maps, so bump RLIM_MEMLOCK as the default 64K setting
+ * is too small. This affects just this process, not touching the
+ * global setting. If it fails we'll get something in 'perf trace -v'
+ * to help diagnose the problem.
+ */
+ rlimit__bump_memlock();
+
err = perf_config(trace__config, &trace);
if (err)
goto out;
diff --git a/tools/perf/builtin-version.c b/tools/perf/builtin-version.c
index f470144d1a70..bf114ca9ca87 100644
--- a/tools/perf/builtin-version.c
+++ b/tools/perf/builtin-version.c
@@ -19,6 +19,7 @@ static struct version version;
static struct option version_options[] = {
OPT_BOOLEAN(0, "build-options", &version.build_options,
"display the build options"),
+ OPT_END(),
};
static const char * const version_usage[] = {
diff --git a/tools/perf/pmu-events/arch/s390/cf_m8561/basic.json b/tools/perf/pmu-events/arch/s390/cf_m8561/basic.json
new file mode 100644
index 000000000000..17fb5241928b
--- /dev/null
+++ b/tools/perf/pmu-events/arch/s390/cf_m8561/basic.json
@@ -0,0 +1,58 @@
+[
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "0",
+ "EventName": "CPU_CYCLES",
+ "BriefDescription": "CPU Cycles",
+ "PublicDescription": "Cycle Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "1",
+ "EventName": "INSTRUCTIONS",
+ "BriefDescription": "Instructions",
+ "PublicDescription": "Instruction Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "2",
+ "EventName": "L1I_DIR_WRITES",
+ "BriefDescription": "L1I Directory Writes",
+ "PublicDescription": "Level-1 I-Cache Directory Write Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "3",
+ "EventName": "L1I_PENALTY_CYCLES",
+ "BriefDescription": "L1I Penalty Cycles",
+ "PublicDescription": "Level-1 I-Cache Penalty Cycle Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "4",
+ "EventName": "L1D_DIR_WRITES",
+ "BriefDescription": "L1D Directory Writes",
+ "PublicDescription": "Level-1 D-Cache Directory Write Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "5",
+ "EventName": "L1D_PENALTY_CYCLES",
+ "BriefDescription": "L1D Penalty Cycles",
+ "PublicDescription": "Level-1 D-Cache Penalty Cycle Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "32",
+ "EventName": "PROBLEM_STATE_CPU_CYCLES",
+ "BriefDescription": "Problem-State CPU Cycles",
+ "PublicDescription": "Problem-State Cycle Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "33",
+ "EventName": "PROBLEM_STATE_INSTRUCTIONS",
+ "BriefDescription": "Problem-State Instructions",
+ "PublicDescription": "Problem-State Instruction Count"
+ },
+]
diff --git a/tools/perf/pmu-events/arch/s390/cf_m8561/crypto.json b/tools/perf/pmu-events/arch/s390/cf_m8561/crypto.json
new file mode 100644
index 000000000000..db286f19e7b6
--- /dev/null
+++ b/tools/perf/pmu-events/arch/s390/cf_m8561/crypto.json
@@ -0,0 +1,114 @@
+[
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "64",
+ "EventName": "PRNG_FUNCTIONS",
+ "BriefDescription": "PRNG Functions",
+ "PublicDescription": "Total number of the PRNG functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "65",
+ "EventName": "PRNG_CYCLES",
+ "BriefDescription": "PRNG Cycles",
+ "PublicDescription": "Total number of CPU cycles when the DEA/AES coprocessor is busy performing PRNG functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "66",
+ "EventName": "PRNG_BLOCKED_FUNCTIONS",
+ "BriefDescription": "PRNG Blocked Functions",
+ "PublicDescription": "Total number of the PRNG functions that are issued by the CPU and are blocked because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "67",
+ "EventName": "PRNG_BLOCKED_CYCLES",
+ "BriefDescription": "PRNG Blocked Cycles",
+ "PublicDescription": "Total number of CPU cycles blocked for the PRNG functions issued by the CPU because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "68",
+ "EventName": "SHA_FUNCTIONS",
+ "BriefDescription": "SHA Functions",
+ "PublicDescription": "Total number of SHA functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "69",
+ "EventName": "SHA_CYCLES",
+ "BriefDescription": "SHA Cycles",
+ "PublicDescription": "Total number of CPU cycles when the SHA coprocessor is busy performing the SHA functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "70",
+ "EventName": "SHA_BLOCKED_FUNCTIONS",
+ "BriefDescription": "SHA Blocked Functions",
+ "PublicDescription": "Total number of the SHA functions that are issued by the CPU and are blocked because the SHA coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "71",
+ "EventName": "SHA_BLOCKED_CYCLES",
+ "BriefDescription": "SHA Bloced Cycles",
+ "PublicDescription": "Total number of CPU cycles blocked for the SHA functions issued by the CPU because the SHA coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "72",
+ "EventName": "DEA_FUNCTIONS",
+ "BriefDescription": "DEA Functions",
+ "PublicDescription": "Total number of the DEA functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "73",
+ "EventName": "DEA_CYCLES",
+ "BriefDescription": "DEA Cycles",
+ "PublicDescription": "Total number of CPU cycles when the DEA/AES coprocessor is busy performing the DEA functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "74",
+ "EventName": "DEA_BLOCKED_FUNCTIONS",
+ "BriefDescription": "DEA Blocked Functions",
+ "PublicDescription": "Total number of the DEA functions that are issued by the CPU and are blocked because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "75",
+ "EventName": "DEA_BLOCKED_CYCLES",
+ "BriefDescription": "DEA Blocked Cycles",
+ "PublicDescription": "Total number of CPU cycles blocked for the DEA functions issued by the CPU because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "76",
+ "EventName": "AES_FUNCTIONS",
+ "BriefDescription": "AES Functions",
+ "PublicDescription": "Total number of AES functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "77",
+ "EventName": "AES_CYCLES",
+ "BriefDescription": "AES Cycles",
+ "PublicDescription": "Total number of CPU cycles when the DEA/AES coprocessor is busy performing the AES functions issued by the CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "78",
+ "EventName": "AES_BLOCKED_FUNCTIONS",
+ "BriefDescription": "AES Blocked Functions",
+ "PublicDescription": "Total number of AES functions that are issued by the CPU and are blocked because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "79",
+ "EventName": "AES_BLOCKED_CYCLES",
+ "BriefDescription": "AES Blocked Cycles",
+ "PublicDescription": "Total number of CPU cycles blocked for the AES functions issued by the CPU because the DEA/AES coprocessor is busy performing a function issued by another CPU"
+ },
+]
diff --git a/tools/perf/pmu-events/arch/s390/cf_m8561/crypto6.json b/tools/perf/pmu-events/arch/s390/cf_m8561/crypto6.json
new file mode 100644
index 000000000000..5e36bc2468d0
--- /dev/null
+++ b/tools/perf/pmu-events/arch/s390/cf_m8561/crypto6.json
@@ -0,0 +1,30 @@
+[
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "80",
+ "EventName": "ECC_FUNCTION_COUNT",
+ "BriefDescription": "ECC Function Count",
+ "PublicDescription": "Long ECC function Count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "81",
+ "EventName": "ECC_CYCLES_COUNT",
+ "BriefDescription": "ECC Cycles Count",
+ "PublicDescription": "Long ECC Function cycles count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "82",
+ "EventName": "ECC_BLOCKED_FUNCTION_COUNT",
+ "BriefDescription": "Ecc Blocked Function Count",
+ "PublicDescription": "Long ECC blocked function count"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "83",
+ "EventName": "ECC_BLOCKED_CYCLES_COUNT",
+ "BriefDescription": "ECC Blocked Cycles Count",
+ "PublicDescription": "Long ECC blocked cycles count"
+ },
+]
diff --git a/tools/perf/pmu-events/arch/s390/cf_m8561/extended.json b/tools/perf/pmu-events/arch/s390/cf_m8561/extended.json
new file mode 100644
index 000000000000..89e070727e1b
--- /dev/null
+++ b/tools/perf/pmu-events/arch/s390/cf_m8561/extended.json
@@ -0,0 +1,373 @@
+[
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "128",
+ "EventName": "L1D_RO_EXCL_WRITES",
+ "BriefDescription": "L1D Read-only Exclusive Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache where the line was originally in a Read-Only state in the cache but has been updated to be in the Exclusive state that allows stores to the cache line"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "129",
+ "EventName": "DTLB2_WRITES",
+ "BriefDescription": "DTLB2 Writes",
+ "PublicDescription": "A translation has been written into The Translation Lookaside Buffer 2 (TLB2) and the request was made by the data cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "130",
+ "EventName": "DTLB2_MISSES",
+ "BriefDescription": "DTLB2 Misses",
+ "PublicDescription": "A TLB2 miss is in progress for a request made by the data cache. Incremented by one for every TLB2 miss in progress for the Level-1 Data cache on this cycle"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "131",
+ "EventName": "DTLB2_HPAGE_WRITES",
+ "BriefDescription": "DTLB2 One-Megabyte Page Writes",
+ "PublicDescription": "A translation entry was written into the Combined Region and Segment Table Entry array in the Level-2 TLB for a one-megabyte page or a Last Host Translation was done"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "132",
+ "EventName": "DTLB2_GPAGE_WRITES",
+ "BriefDescription": "DTLB2 Two-Gigabyte Page Writes",
+ "PublicDescription": "A translation entry for a two-gigabyte page was written into the Level-2 TLB"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "133",
+ "EventName": "L1D_L2D_SOURCED_WRITES",
+ "BriefDescription": "L1D L2D Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from the Level-2 Data cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "134",
+ "EventName": "ITLB2_WRITES",
+ "BriefDescription": "ITLB2 Writes",
+ "PublicDescription": "A translation entry has been written into the Translation Lookaside Buffer 2 (TLB2) and the request was made by the instruction cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "135",
+ "EventName": "ITLB2_MISSES",
+ "BriefDescription": "ITLB2 Misses",
+ "PublicDescription": "A TLB2 miss is in progress for a request made by the instruction cache. Incremented by one for every TLB2 miss in progress for the Level-1 Instruction cache in a cycle"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "136",
+ "EventName": "L1I_L2I_SOURCED_WRITES",
+ "BriefDescription": "L1I L2I Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from the Level-2 Instruction cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "137",
+ "EventName": "TLB2_PTE_WRITES",
+ "BriefDescription": "TLB2 PTE Writes",
+ "PublicDescription": "A translation entry was written into the Page Table Entry array in the Level-2 TLB"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "138",
+ "EventName": "TLB2_CRSTE_WRITES",
+ "BriefDescription": "TLB2 CRSTE Writes",
+ "PublicDescription": "Translation entries were written into the Combined Region and Segment Table Entry array and the Page Table Entry array in the Level-2 TLB"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "139",
+ "EventName": "TLB2_ENGINES_BUSY",
+ "BriefDescription": "TLB2 Engines Busy",
+ "PublicDescription": "The number of Level-2 TLB translation engines busy in a cycle"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "140",
+ "EventName": "TX_C_TEND",
+ "BriefDescription": "Completed TEND instructions in constrained TX mode",
+ "PublicDescription": "A TEND instruction has completed in a constrained transactional-execution mode"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "141",
+ "EventName": "TX_NC_TEND",
+ "BriefDescription": "Completed TEND instructions in non-constrained TX mode",
+ "PublicDescription": "A TEND instruction has completed in a non-constrained transactional-execution mode"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "143",
+ "EventName": "L1C_TLB2_MISSES",
+ "BriefDescription": "L1C TLB2 Misses",
+ "PublicDescription": "Increments by one for any cycle where a level-1 cache or level-2 TLB miss is in progress"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "144",
+ "EventName": "L1D_ONCHIP_L3_SOURCED_WRITES",
+ "BriefDescription": "L1D On-Chip L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an On-Chip Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "145",
+ "EventName": "L1D_ONCHIP_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1D On-Chip Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from On-Chip memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "146",
+ "EventName": "L1D_ONCHIP_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1D On-Chip L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an On-Chip Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "147",
+ "EventName": "L1D_ONCLUSTER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1D On-Cluster L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from On-Cluster Level-3 cache withountervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "148",
+ "EventName": "L1D_ONCLUSTER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1D On-Cluster Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an On-Cluster memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "149",
+ "EventName": "L1D_ONCLUSTER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1D On-Cluster L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an On-Cluster Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "150",
+ "EventName": "L1D_OFFCLUSTER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1D Off-Cluster L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an Off-Cluster Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "151",
+ "EventName": "L1D_OFFCLUSTER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1D Off-Cluster Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from Off-Cluster memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "152",
+ "EventName": "L1D_OFFCLUSTER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1D Off-Cluster L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an Off-Cluster Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "153",
+ "EventName": "L1D_OFFDRAWER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1D Off-Drawer L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an Off-Drawer Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "154",
+ "EventName": "L1D_OFFDRAWER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1D Off-Drawer Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from Off-Drawer memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "155",
+ "EventName": "L1D_OFFDRAWER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1D Off-Drawer L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from an Off-Drawer Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "156",
+ "EventName": "L1D_ONDRAWER_L4_SOURCED_WRITES",
+ "BriefDescription": "L1D On-Drawer L4 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from On-Drawer Level-4 cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "157",
+ "EventName": "L1D_OFFDRAWER_L4_SOURCED_WRITES",
+ "BriefDescription": "L1D Off-Drawer L4 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from Off-Drawer Level-4 cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "158",
+ "EventName": "L1D_ONCHIP_L3_SOURCED_WRITES_RO",
+ "BriefDescription": "L1D On-Chip L3 Sourced Writes read-only",
+ "PublicDescription": "A directory write to the Level-1 Data cache directory where the returned cache line was sourced from On-Chip L3 but a read-only invalidate was done to remove other copies of the cache line"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "162",
+ "EventName": "L1I_ONCHIP_L3_SOURCED_WRITES",
+ "BriefDescription": "L1I On-Chip L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache ine was sourced from an On-Chip Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "163",
+ "EventName": "L1I_ONCHIP_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1I On-Chip Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache ine was sourced from On-Chip memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "164",
+ "EventName": "L1I_ONCHIP_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1I On-Chip L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache ine was sourced from an On-Chip Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "165",
+ "EventName": "L1I_ONCLUSTER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1I On-Cluster L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an On-Cluster Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "166",
+ "EventName": "L1I_ONCLUSTER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1I On-Cluster Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an On-Cluster memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "167",
+ "EventName": "L1I_ONCLUSTER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1I On-Cluster L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from On-Cluster Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "168",
+ "EventName": "L1I_OFFCLUSTER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1I Off-Cluster L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an Off-Cluster Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "169",
+ "EventName": "L1I_OFFCLUSTER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1I Off-Cluster Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from Off-Cluster memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "170",
+ "EventName": "L1I_OFFCLUSTER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1I Off-Cluster L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an Off-Cluster Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "171",
+ "EventName": "L1I_OFFDRAWER_L3_SOURCED_WRITES",
+ "BriefDescription": "L1I Off-Drawer L3 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an Off-Drawer Level-3 cache without intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "172",
+ "EventName": "L1I_OFFDRAWER_MEMORY_SOURCED_WRITES",
+ "BriefDescription": "L1I Off-Drawer Memory Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from Off-Drawer memory"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "173",
+ "EventName": "L1I_OFFDRAWER_L3_SOURCED_WRITES_IV",
+ "BriefDescription": "L1I Off-Drawer L3 Sourced Writes with Intervention",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from an Off-Drawer Level-3 cache with intervention"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "174",
+ "EventName": "L1I_ONDRAWER_L4_SOURCED_WRITES",
+ "BriefDescription": "L1I On-Drawer L4 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from On-Drawer Level-4 cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "175",
+ "EventName": "L1I_OFFDRAWER_L4_SOURCED_WRITES",
+ "BriefDescription": "L1I Off-Drawer L4 Sourced Writes",
+ "PublicDescription": "A directory write to the Level-1 Instruction cache directory where the returned cache line was sourced from Off-Drawer Level-4 cache"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "224",
+ "EventName": "BCD_DFP_EXECUTION_SLOTS",
+ "BriefDescription": "BCD DFP Execution Slots",
+ "PublicDescription": "Count of floating point execution slots used for finished Binary Coded Decimal to Decimal Floating Point conversions. Instructions: CDZT, CXZT, CZDT, CZXT"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "225",
+ "EventName": "VX_BCD_EXECUTION_SLOTS",
+ "BriefDescription": "VX BCD Execution Slots",
+ "PublicDescription": "Count of floating point execution slots used for finished vector arithmetic Binary Coded Decimal instructions. Instructions: VAP, VSP, VMPVMSP, VDP, VSDP, VRP, VLIP, VSRP, VPSOPVCP, VTP, VPKZ, VUPKZ, VCVB, VCVBG, VCVDVCVDG"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "226",
+ "EventName": "DECIMAL_INSTRUCTIONS",
+ "BriefDescription": "Decimal Instructions",
+ "PublicDescription": "Decimal instructions dispatched. Instructions: CVB, CVD, AP, CP, DP, ED, EDMK, MP, SRP, SP, ZAP"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "232",
+ "EventName": "LAST_HOST_TRANSLATIONS",
+ "BriefDescription": "Last host translation done",
+ "PublicDescription": "Last Host Translation done"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "243",
+ "EventName": "TX_NC_TABORT",
+ "BriefDescription": "Aborted transactions in non-constrained TX mode",
+ "PublicDescription": "A transaction abort has occurred in a non-constrained transactional-execution mode"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "244",
+ "EventName": "TX_C_TABORT_NO_SPECIAL",
+ "BriefDescription": "Aborted transactions in constrained TX mode not using special completion logic",
+ "PublicDescription": "A transaction abort has occurred in a constrained transactional-execution mode and the CPU is not using any special logic to allow the transaction to complete"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "245",
+ "EventName": "TX_C_TABORT_SPECIAL",
+ "BriefDescription": "Aborted transactions in constrained TX mode using special completion logic",
+ "PublicDescription": "A transaction abort has occurred in a constrained transactional-execution mode and the CPU is using special logic to allow the transaction to complete"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "448",
+ "EventName": "MT_DIAG_CYCLES_ONE_THR_ACTIVE",
+ "BriefDescription": "Cycle count with one thread active",
+ "PublicDescription": "Cycle count with one thread active"
+ },
+ {
+ "Unit": "CPU-M-CF",
+ "EventCode": "449",
+ "EventName": "MT_DIAG_CYCLES_TWO_THR_ACTIVE",
+ "BriefDescription": "Cycle count with two threads active",
+ "PublicDescription": "Cycle count with two threads active"
+ },
+]
diff --git a/tools/perf/pmu-events/arch/s390/mapfile.csv b/tools/perf/pmu-events/arch/s390/mapfile.csv
index 78bcf7f8e206..bd3fc577139c 100644
--- a/tools/perf/pmu-events/arch/s390/mapfile.csv
+++ b/tools/perf/pmu-events/arch/s390/mapfile.csv
@@ -4,3 +4,4 @@ Family-model,Version,Filename,EventType
^IBM.282[78].*[13]\.[1-5].[[:xdigit:]]+$,1,cf_zec12,core
^IBM.296[45].*[13]\.[1-5].[[:xdigit:]]+$,1,cf_z13,core
^IBM.390[67].*[13]\.[1-5].[[:xdigit:]]+$,3,cf_z14,core
+^IBM.856[12].*3\.6.[[:xdigit:]]+$,3,cf_m8561,core
diff --git a/tools/perf/scripts/python/export-to-postgresql.py b/tools/perf/scripts/python/export-to-postgresql.py
index 92713d93e956..7bd73a904b4e 100644
--- a/tools/perf/scripts/python/export-to-postgresql.py
+++ b/tools/perf/scripts/python/export-to-postgresql.py
@@ -353,7 +353,10 @@ do_query(query, 'CREATE TABLE threads ('
'tid integer)')
do_query(query, 'CREATE TABLE comms ('
'id bigint NOT NULL,'
- 'comm varchar(16))')
+ 'comm varchar(16),'
+ 'c_thread_id bigint,'
+ 'c_time bigint,'
+ 'exec_flag boolean)')
do_query(query, 'CREATE TABLE comm_threads ('
'id bigint NOT NULL,'
'comm_id bigint,'
@@ -479,6 +482,17 @@ do_query(query, 'CREATE TABLE pwrx ('
'last_cstate integer,'
'wake_reason integer)')
+do_query(query, 'CREATE TABLE context_switches ('
+ 'id bigint NOT NULL,'
+ 'machine_id bigint,'
+ 'time bigint,'
+ 'cpu integer,'
+ 'thread_out_id bigint,'
+ 'comm_out_id bigint,'
+ 'thread_in_id bigint,'
+ 'comm_in_id bigint,'
+ 'flags integer)')
+
do_query(query, 'CREATE VIEW machines_view AS '
'SELECT '
'id,'
@@ -692,6 +706,29 @@ do_query(query, 'CREATE VIEW power_events_view AS '
' INNER JOIN selected_events ON selected_events.id = samples.evsel_id'
' ORDER BY samples.id')
+do_query(query, 'CREATE VIEW context_switches_view AS '
+ 'SELECT '
+ 'context_switches.id,'
+ 'context_switches.machine_id,'
+ 'context_switches.time,'
+ 'context_switches.cpu,'
+ 'th_out.pid AS pid_out,'
+ 'th_out.tid AS tid_out,'
+ 'comm_out.comm AS comm_out,'
+ 'th_in.pid AS pid_in,'
+ 'th_in.tid AS tid_in,'
+ 'comm_in.comm AS comm_in,'
+ 'CASE WHEN context_switches.flags = 0 THEN \'in\''
+ ' WHEN context_switches.flags = 1 THEN \'out\''
+ ' WHEN context_switches.flags = 3 THEN \'out preempt\''
+ ' ELSE CAST ( context_switches.flags AS VARCHAR(11) )'
+ 'END AS flags'
+ ' FROM context_switches'
+ ' INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id'
+ ' INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id'
+ ' INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id'
+ ' INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id')
+
file_header = struct.pack("!11sii", b"PGCOPY\n\377\r\n\0", 0, 0)
file_trailer = b"\377\377"
@@ -756,6 +793,7 @@ mwait_file = open_output_file("mwait_table.bin")
pwre_file = open_output_file("pwre_table.bin")
exstop_file = open_output_file("exstop_table.bin")
pwrx_file = open_output_file("pwrx_table.bin")
+context_switches_file = open_output_file("context_switches_table.bin")
def trace_begin():
printdate("Writing to intermediate files...")
@@ -763,7 +801,7 @@ def trace_begin():
evsel_table(0, "unknown")
machine_table(0, 0, "unknown")
thread_table(0, 0, 0, -1, -1)
- comm_table(0, "unknown")
+ comm_table(0, "unknown", 0, 0, 0)
dso_table(0, 0, "unknown", "unknown", "")
symbol_table(0, 0, 0, 0, 0, "unknown")
sample_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
@@ -804,6 +842,7 @@ def trace_end():
copy_output_file(pwre_file, "pwre")
copy_output_file(exstop_file, "exstop")
copy_output_file(pwrx_file, "pwrx")
+ copy_output_file(context_switches_file, "context_switches")
printdate("Removing intermediate files...")
remove_output_file(evsel_file)
@@ -825,6 +864,7 @@ def trace_end():
remove_output_file(pwre_file)
remove_output_file(exstop_file)
remove_output_file(pwrx_file)
+ remove_output_file(context_switches_file)
os.rmdir(output_dir_name)
printdate("Adding primary keys")
do_query(query, 'ALTER TABLE selected_events ADD PRIMARY KEY (id)')
@@ -846,11 +886,14 @@ def trace_end():
do_query(query, 'ALTER TABLE pwre ADD PRIMARY KEY (id)')
do_query(query, 'ALTER TABLE exstop ADD PRIMARY KEY (id)')
do_query(query, 'ALTER TABLE pwrx ADD PRIMARY KEY (id)')
+ do_query(query, 'ALTER TABLE context_switches ADD PRIMARY KEY (id)')
printdate("Adding foreign keys")
do_query(query, 'ALTER TABLE threads '
'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
'ADD CONSTRAINT processfk FOREIGN KEY (process_id) REFERENCES threads (id)')
+ do_query(query, 'ALTER TABLE comms '
+ 'ADD CONSTRAINT threadfk FOREIGN KEY (c_thread_id) REFERENCES threads (id)')
do_query(query, 'ALTER TABLE comm_threads '
'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id)')
@@ -881,6 +924,8 @@ def trace_end():
'ADD CONSTRAINT parent_call_pathfk FOREIGN KEY (parent_call_path_id) REFERENCES call_paths (id)')
do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)')
do_query(query, 'CREATE INDEX pid_idx ON calls (parent_id)')
+ do_query(query, 'ALTER TABLE comms ADD has_calls boolean')
+ do_query(query, 'UPDATE comms SET has_calls = TRUE WHERE comms.id IN (SELECT DISTINCT comm_id FROM calls)')
do_query(query, 'ALTER TABLE ptwrite '
'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
do_query(query, 'ALTER TABLE cbr '
@@ -893,6 +938,12 @@ def trace_end():
'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
do_query(query, 'ALTER TABLE pwrx '
'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ do_query(query, 'ALTER TABLE context_switches '
+ 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
+ 'ADD CONSTRAINT toutfk FOREIGN KEY (thread_out_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT tinfk FOREIGN KEY (thread_in_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT coutfk FOREIGN KEY (comm_out_id) REFERENCES comms (id),'
+ 'ADD CONSTRAINT cinfk FOREIGN KEY (comm_in_id) REFERENCES comms (id)')
printdate("Dropping unused tables")
if is_table_empty("ptwrite"):
@@ -905,6 +956,8 @@ def trace_end():
drop("pwrx")
if is_table_empty("cbr"):
drop("cbr")
+ if is_table_empty("context_switches"):
+ drop("context_switches")
if (unhandled_count):
printdate("Warning: ", unhandled_count, " unhandled events")
@@ -935,11 +988,11 @@ def thread_table(thread_id, machine_id, process_id, pid, tid, *x):
value = struct.pack("!hiqiqiqiiii", 5, 8, thread_id, 8, machine_id, 8, process_id, 4, pid, 4, tid)
thread_file.write(value)
-def comm_table(comm_id, comm_str, *x):
+def comm_table(comm_id, comm_str, thread_id, time, exec_flag, *x):
comm_str = toserverstr(comm_str)
n = len(comm_str)
- fmt = "!hiqi" + str(n) + "s"
- value = struct.pack(fmt, 2, 8, comm_id, n, comm_str)
+ fmt = "!hiqi" + str(n) + "s" + "iqiqiB"
+ value = struct.pack(fmt, 5, 8, comm_id, n, comm_str, 8, thread_id, 8, time, 1, exec_flag)
comm_file.write(value)
def comm_thread_table(comm_thread_id, comm_id, thread_id, *x):
@@ -1051,3 +1104,8 @@ def synth_data(id, config, raw_buf, *x):
pwrx(id, raw_buf)
elif config == 5:
cbr(id, raw_buf)
+
+def context_switch_table(id, machine_id, time, cpu, thread_out_id, comm_out_id, thread_in_id, comm_in_id, flags, *x):
+ fmt = "!hiqiqiqiiiqiqiqiqii"
+ value = struct.pack(fmt, 9, 8, id, 8, machine_id, 8, time, 4, cpu, 8, thread_out_id, 8, comm_out_id, 8, thread_in_id, 8, comm_in_id, 4, flags)
+ context_switches_file.write(value)
diff --git a/tools/perf/scripts/python/export-to-sqlite.py b/tools/perf/scripts/python/export-to-sqlite.py
index 021326c46285..8043a7272a56 100644
--- a/tools/perf/scripts/python/export-to-sqlite.py
+++ b/tools/perf/scripts/python/export-to-sqlite.py
@@ -177,7 +177,10 @@ do_query(query, 'CREATE TABLE threads ('
'tid integer)')
do_query(query, 'CREATE TABLE comms ('
'id integer NOT NULL PRIMARY KEY,'
- 'comm varchar(16))')
+ 'comm varchar(16),'
+ 'c_thread_id bigint,'
+ 'c_time bigint,'
+ 'exec_flag boolean)')
do_query(query, 'CREATE TABLE comm_threads ('
'id integer NOT NULL PRIMARY KEY,'
'comm_id bigint,'
@@ -303,6 +306,17 @@ do_query(query, 'CREATE TABLE pwrx ('
'last_cstate integer,'
'wake_reason integer)')
+do_query(query, 'CREATE TABLE context_switches ('
+ 'id integer NOT NULL PRIMARY KEY,'
+ 'machine_id bigint,'
+ 'time bigint,'
+ 'cpu integer,'
+ 'thread_out_id bigint,'
+ 'comm_out_id bigint,'
+ 'thread_in_id bigint,'
+ 'comm_in_id bigint,'
+ 'flags integer)')
+
# printf was added to sqlite in version 3.8.3
sqlite_has_printf = False
try:
@@ -527,6 +541,29 @@ do_query(query, 'CREATE VIEW power_events_view AS '
' INNER JOIN selected_events ON selected_events.id = evsel_id'
' WHERE selected_events.name IN (\'cbr\',\'mwait\',\'exstop\',\'pwre\',\'pwrx\')')
+do_query(query, 'CREATE VIEW context_switches_view AS '
+ 'SELECT '
+ 'context_switches.id,'
+ 'context_switches.machine_id,'
+ 'context_switches.time,'
+ 'context_switches.cpu,'
+ 'th_out.pid AS pid_out,'
+ 'th_out.tid AS tid_out,'
+ 'comm_out.comm AS comm_out,'
+ 'th_in.pid AS pid_in,'
+ 'th_in.tid AS tid_in,'
+ 'comm_in.comm AS comm_in,'
+ 'CASE WHEN context_switches.flags = 0 THEN \'in\''
+ ' WHEN context_switches.flags = 1 THEN \'out\''
+ ' WHEN context_switches.flags = 3 THEN \'out preempt\''
+ ' ELSE context_switches.flags '
+ 'END AS flags'
+ ' FROM context_switches'
+ ' INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id'
+ ' INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id'
+ ' INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id'
+ ' INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id')
+
do_query(query, 'END TRANSACTION')
evsel_query = QSqlQuery(db)
@@ -536,7 +573,7 @@ machine_query.prepare("INSERT INTO machines VALUES (?, ?, ?)")
thread_query = QSqlQuery(db)
thread_query.prepare("INSERT INTO threads VALUES (?, ?, ?, ?, ?)")
comm_query = QSqlQuery(db)
-comm_query.prepare("INSERT INTO comms VALUES (?, ?)")
+comm_query.prepare("INSERT INTO comms VALUES (?, ?, ?, ?, ?)")
comm_thread_query = QSqlQuery(db)
comm_thread_query.prepare("INSERT INTO comm_threads VALUES (?, ?, ?)")
dso_query = QSqlQuery(db)
@@ -568,6 +605,8 @@ exstop_query = QSqlQuery(db)
exstop_query.prepare("INSERT INTO exstop VALUES (?, ?)")
pwrx_query = QSqlQuery(db)
pwrx_query.prepare("INSERT INTO pwrx VALUES (?, ?, ?, ?)")
+context_switch_query = QSqlQuery(db)
+context_switch_query.prepare("INSERT INTO context_switches VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)")
def trace_begin():
printdate("Writing records...")
@@ -576,7 +615,7 @@ def trace_begin():
evsel_table(0, "unknown")
machine_table(0, 0, "unknown")
thread_table(0, 0, 0, -1, -1)
- comm_table(0, "unknown")
+ comm_table(0, "unknown", 0, 0, 0)
dso_table(0, 0, "unknown", "unknown", "")
symbol_table(0, 0, 0, 0, 0, "unknown")
sample_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
@@ -603,6 +642,8 @@ def trace_end():
if perf_db_export_calls:
do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)')
do_query(query, 'CREATE INDEX pid_idx ON calls (parent_id)')
+ do_query(query, 'ALTER TABLE comms ADD has_calls boolean')
+ do_query(query, 'UPDATE comms SET has_calls = 1 WHERE comms.id IN (SELECT DISTINCT comm_id FROM calls)')
printdate("Dropping unused tables")
if is_table_empty("ptwrite"):
@@ -615,6 +656,8 @@ def trace_end():
drop("pwrx")
if is_table_empty("cbr"):
drop("cbr")
+ if is_table_empty("context_switches"):
+ drop("context_switches")
if (unhandled_count):
printdate("Warning: ", unhandled_count, " unhandled events")
@@ -642,7 +685,7 @@ def thread_table(*x):
bind_exec(thread_query, 5, x)
def comm_table(*x):
- bind_exec(comm_query, 2, x)
+ bind_exec(comm_query, 5, x)
def comm_thread_table(*x):
bind_exec(comm_thread_query, 3, x)
@@ -748,3 +791,6 @@ def synth_data(id, config, raw_buf, *x):
pwrx(id, raw_buf)
elif config == 5:
cbr(id, raw_buf)
+
+def context_switch_table(*x):
+ bind_exec(context_switch_query, 9, x)
diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
index 6e7934f2ac9a..61b3911d91e6 100755
--- a/tools/perf/scripts/python/exported-sql-viewer.py
+++ b/tools/perf/scripts/python/exported-sql-viewer.py
@@ -392,7 +392,7 @@ class FindBar():
self.hbox.addWidget(self.close_button)
self.bar = QWidget()
- self.bar.setLayout(self.hbox);
+ self.bar.setLayout(self.hbox)
self.bar.hide()
def Widget(self):
@@ -470,7 +470,7 @@ class CallGraphLevelItemBase(object):
self.params = params
self.row = row
self.parent_item = parent_item
- self.query_done = False;
+ self.query_done = False
self.child_count = 0
self.child_items = []
if parent_item:
@@ -517,7 +517,7 @@ class CallGraphLevelTwoPlusItemBase(CallGraphLevelItemBase):
self.time = time
def Select(self):
- self.query_done = True;
+ self.query_done = True
query = QSqlQuery(self.glb.db)
if self.params.have_ipc:
ipc_str = ", SUM(insn_count), SUM(cyc_count)"
@@ -604,7 +604,7 @@ class CallGraphLevelOneItem(CallGraphLevelItemBase):
self.dbid = comm_id
def Select(self):
- self.query_done = True;
+ self.query_done = True
query = QSqlQuery(self.glb.db)
QueryExec(query, "SELECT thread_id, pid, tid"
" FROM comm_threads"
@@ -622,9 +622,12 @@ class CallGraphRootItem(CallGraphLevelItemBase):
def __init__(self, glb, params):
super(CallGraphRootItem, self).__init__(glb, params, 0, None)
self.dbid = 0
- self.query_done = True;
+ self.query_done = True
+ if_has_calls = ""
+ if IsSelectable(glb.db, "comms", columns = "has_calls"):
+ if_has_calls = " WHERE has_calls = TRUE"
query = QSqlQuery(glb.db)
- QueryExec(query, "SELECT id, comm FROM comms")
+ QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
while query.next():
if not query.value(0):
continue
@@ -793,7 +796,7 @@ class CallTreeLevelTwoPlusItemBase(CallGraphLevelItemBase):
self.time = time
def Select(self):
- self.query_done = True;
+ self.query_done = True
if self.calls_id == 0:
comm_thread = " AND comm_id = " + str(self.comm_id) + " AND thread_id = " + str(self.thread_id)
else:
@@ -881,7 +884,7 @@ class CallTreeLevelOneItem(CallGraphLevelItemBase):
self.dbid = comm_id
def Select(self):
- self.query_done = True;
+ self.query_done = True
query = QSqlQuery(self.glb.db)
QueryExec(query, "SELECT thread_id, pid, tid"
" FROM comm_threads"
@@ -899,9 +902,12 @@ class CallTreeRootItem(CallGraphLevelItemBase):
def __init__(self, glb, params):
super(CallTreeRootItem, self).__init__(glb, params, 0, None)
self.dbid = 0
- self.query_done = True;
+ self.query_done = True
+ if_has_calls = ""
+ if IsSelectable(glb.db, "comms", columns = "has_calls"):
+ if_has_calls = " WHERE has_calls = TRUE"
query = QSqlQuery(glb.db)
- QueryExec(query, "SELECT id, comm FROM comms")
+ QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
while query.next():
if not query.value(0):
continue
@@ -971,7 +977,7 @@ class VBox():
def __init__(self, w1, w2, w3=None):
self.vbox = QWidget()
- self.vbox.setLayout(QVBoxLayout());
+ self.vbox.setLayout(QVBoxLayout())
self.vbox.layout().setContentsMargins(0, 0, 0, 0)
@@ -1391,7 +1397,7 @@ class FetchMoreRecordsBar():
self.hbox.addWidget(self.close_button)
self.bar = QWidget()
- self.bar.setLayout(self.hbox);
+ self.bar.setLayout(self.hbox)
self.bar.show()
self.in_progress = False
@@ -2206,7 +2212,7 @@ class ReportDialogBase(QDialog):
self.vbox.addLayout(self.grid)
self.vbox.addLayout(self.hbox)
- self.setLayout(self.vbox);
+ self.setLayout(self.vbox)
def Ok(self):
vars = self.report_vars
@@ -3139,7 +3145,7 @@ class AboutDialog(QDialog):
self.vbox = QVBoxLayout()
self.vbox.addWidget(self.text)
- self.setLayout(self.vbox);
+ self.setLayout(self.vbox)
# Font resize
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 66a82badc1d1..c3bec9d2c201 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -21,6 +21,7 @@
#include <subcmd/parse-options.h>
#include "string2.h"
#include "symbol.h"
+#include "util/rlimit.h"
#include <linux/kernel.h>
#include <linux/string.h>
#include <subcmd/exec-cmd.h>
@@ -727,6 +728,11 @@ int cmd_test(int argc, const char **argv)
if (skip != NULL)
skiplist = intlist__new(skip);
+ /*
+ * Tests that create BPF maps, for instance, need more than the 64K
+ * default:
+ */
+ rlimit__bump_memlock();
return __cmd_test(argc, argv, skiplist);
}
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index d7e3b008a613..14f812bb07a7 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -20,6 +20,7 @@ perf-y += parse-events.o
perf-y += perf_regs.o
perf-y += path.o
perf-y += print_binary.o
+perf-y += rlimit.o
perf-y += argv_split.o
perf-y += rbtree.o
perf-y += libstring.o
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c
index 67b88b599a53..3d1c34fc4d68 100644
--- a/tools/perf/util/cs-etm.c
+++ b/tools/perf/util/cs-etm.c
@@ -2460,7 +2460,7 @@ int cs_etm__process_auxtrace_info(union perf_event *event,
/* Something went wrong, no need to continue */
if (!inode) {
- err = PTR_ERR(inode);
+ err = -ENOMEM;
goto err_free_metadata;
}
@@ -2517,8 +2517,10 @@ int cs_etm__process_auxtrace_info(union perf_event *event,
session->auxtrace = &etm->auxtrace;
etm->unknown_thread = thread__new(999999999, 999999999);
- if (!etm->unknown_thread)
+ if (!etm->unknown_thread) {
+ err = -ENOMEM;
goto err_free_queues;
+ }
/*
* Initialize list node so that at thread__zput() we can avoid
@@ -2530,8 +2532,10 @@ int cs_etm__process_auxtrace_info(union perf_event *event,
if (err)
goto err_delete_thread;
- if (thread__init_map_groups(etm->unknown_thread, etm->machine))
+ if (thread__init_map_groups(etm->unknown_thread, etm->machine)) {
+ err = -ENOMEM;
goto err_delete_thread;
+ }
if (dump_trace) {
cs_etm__print_auxtrace_info(auxtrace_info->priv, num_cpu);
@@ -2575,5 +2579,5 @@ int cs_etm__process_auxtrace_info(union perf_event *event,
err_free_hdr:
zfree(&hdr);
- return -EINVAL;
+ return err;
}
diff --git a/tools/perf/util/db-export.c b/tools/perf/util/db-export.c
index 2394c7506abe..ffbb3e7d3288 100644
--- a/tools/perf/util/db-export.c
+++ b/tools/perf/util/db-export.c
@@ -20,70 +20,14 @@
#include "db-export.h"
#include <linux/zalloc.h>
-struct deferred_export {
- struct list_head node;
- struct comm *comm;
-};
-
-static int db_export__deferred(struct db_export *dbe)
-{
- struct deferred_export *de;
- int err;
-
- while (!list_empty(&dbe->deferred)) {
- de = list_entry(dbe->deferred.next, struct deferred_export,
- node);
- err = dbe->export_comm(dbe, de->comm);
- list_del_init(&de->node);
- free(de);
- if (err)
- return err;
- }
-
- return 0;
-}
-
-static void db_export__free_deferred(struct db_export *dbe)
-{
- struct deferred_export *de;
-
- while (!list_empty(&dbe->deferred)) {
- de = list_entry(dbe->deferred.next, struct deferred_export,
- node);
- list_del_init(&de->node);
- free(de);
- }
-}
-
-static int db_export__defer_comm(struct db_export *dbe, struct comm *comm)
-{
- struct deferred_export *de;
-
- de = zalloc(sizeof(struct deferred_export));
- if (!de)
- return -ENOMEM;
-
- de->comm = comm;
- list_add_tail(&de->node, &dbe->deferred);
-
- return 0;
-}
-
int db_export__init(struct db_export *dbe)
{
memset(dbe, 0, sizeof(struct db_export));
- INIT_LIST_HEAD(&dbe->deferred);
return 0;
}
-int db_export__flush(struct db_export *dbe)
-{
- return db_export__deferred(dbe);
-}
-
void db_export__exit(struct db_export *dbe)
{
- db_export__free_deferred(dbe);
call_return_processor__free(dbe->crp);
dbe->crp = NULL;
}
@@ -115,71 +59,73 @@ int db_export__machine(struct db_export *dbe, struct machine *machine)
}
int db_export__thread(struct db_export *dbe, struct thread *thread,
- struct machine *machine, struct comm *comm)
+ struct machine *machine, struct thread *main_thread)
{
- struct thread *main_thread;
u64 main_thread_db_id = 0;
- int err;
if (thread->db_id)
return 0;
thread->db_id = ++dbe->thread_last_db_id;
- if (thread->pid_ != -1) {
- if (thread->pid_ == thread->tid) {
- main_thread = thread;
- } else {
- main_thread = machine__findnew_thread(machine,
- thread->pid_,
- thread->pid_);
- if (!main_thread)
- return -ENOMEM;
- err = db_export__thread(dbe, main_thread, machine,
- comm);
- if (err)
- goto out_put;
- if (comm) {
- err = db_export__comm_thread(dbe, comm, thread);
- if (err)
- goto out_put;
- }
- }
+ if (main_thread)
main_thread_db_id = main_thread->db_id;
- if (main_thread != thread)
- thread__put(main_thread);
- }
if (dbe->export_thread)
return dbe->export_thread(dbe, thread, main_thread_db_id,
machine);
return 0;
+}
-out_put:
- thread__put(main_thread);
- return err;
+static int __db_export__comm(struct db_export *dbe, struct comm *comm,
+ struct thread *thread)
+{
+ comm->db_id = ++dbe->comm_last_db_id;
+
+ if (dbe->export_comm)
+ return dbe->export_comm(dbe, comm, thread);
+
+ return 0;
}
int db_export__comm(struct db_export *dbe, struct comm *comm,
- struct thread *main_thread)
+ struct thread *thread)
+{
+ if (comm->db_id)
+ return 0;
+
+ return __db_export__comm(dbe, comm, thread);
+}
+
+/*
+ * Export the "exec" comm. The "exec" comm is the program / application command
+ * name at the time it first executes. It is used to group threads for the same
+ * program. Note that the main thread pid (or thread group id tgid) cannot be
+ * used because it does not change when a new program is exec'ed.
+ */
+int db_export__exec_comm(struct db_export *dbe, struct comm *comm,
+ struct thread *main_thread)
{
int err;
if (comm->db_id)
return 0;
- comm->db_id = ++dbe->comm_last_db_id;
-
- if (dbe->export_comm) {
- if (main_thread->comm_set)
- err = dbe->export_comm(dbe, comm);
- else
- err = db_export__defer_comm(dbe, comm);
- if (err)
- return err;
- }
+ err = __db_export__comm(dbe, comm, main_thread);
+ if (err)
+ return err;
+ /*
+ * Record the main thread for this comm. Note that the main thread can
+ * have many "exec" comms because there will be a new one every time it
+ * exec's. An "exec" comm however will only ever have 1 main thread.
+ * That is different to any other threads for that same program because
+ * exec() will effectively kill them, so the relationship between the
+ * "exec" comm and non-main threads is 1-to-1. That is why
+ * db_export__comm_thread() is called here for the main thread, but it
+ * is called for non-main threads when they are exported.
+ */
return db_export__comm_thread(dbe, comm, main_thread);
}
@@ -340,11 +286,65 @@ int db_export__branch_type(struct db_export *dbe, u32 branch_type,
return 0;
}
+static int db_export__threads(struct db_export *dbe, struct thread *thread,
+ struct thread *main_thread,
+ struct machine *machine, struct comm **comm_ptr)
+{
+ struct comm *comm = NULL;
+ struct comm *curr_comm;
+ int err;
+
+ if (main_thread) {
+ /*
+ * A thread has a reference to the main thread, so export the
+ * main thread first.
+ */
+ err = db_export__thread(dbe, main_thread, machine, main_thread);
+ if (err)
+ return err;
+ /*
+ * Export comm before exporting the non-main thread because
+ * db_export__comm_thread() can be called further below.
+ */
+ comm = machine__thread_exec_comm(machine, main_thread);
+ if (comm) {
+ err = db_export__exec_comm(dbe, comm, main_thread);
+ if (err)
+ return err;
+ *comm_ptr = comm;
+ }
+ }
+
+ if (thread != main_thread) {
+ /*
+ * For a non-main thread, db_export__comm_thread() must be
+ * called only if thread has not previously been exported.
+ */
+ bool export_comm_thread = comm && !thread->db_id;
+
+ err = db_export__thread(dbe, thread, machine, main_thread);
+ if (err)
+ return err;
+
+ if (export_comm_thread) {
+ err = db_export__comm_thread(dbe, comm, thread);
+ if (err)
+ return err;
+ }
+ }
+
+ curr_comm = thread__comm(thread);
+ if (curr_comm)
+ return db_export__comm(dbe, curr_comm, thread);
+
+ return 0;
+}
+
int db_export__sample(struct db_export *dbe, union perf_event *event,
struct perf_sample *sample, struct perf_evsel *evsel,
struct addr_location *al)
{
- struct thread* thread = al->thread;
+ struct thread *thread = al->thread;
struct export_sample es = {
.event = event,
.sample = sample,
@@ -364,19 +364,13 @@ int db_export__sample(struct db_export *dbe, union perf_event *event,
return err;
main_thread = thread__main_thread(al->machine, thread);
- if (main_thread)
- comm = machine__thread_exec_comm(al->machine, main_thread);
- err = db_export__thread(dbe, thread, al->machine, comm);
+ err = db_export__threads(dbe, thread, main_thread, al->machine, &comm);
if (err)
goto out_put;
- if (comm) {
- err = db_export__comm(dbe, comm, main_thread);
- if (err)
- goto out_put;
+ if (comm)
es.comm_db_id = comm->db_id;
- }
es.db_id = ++dbe->sample_last_db_id;
@@ -525,3 +519,92 @@ int db_export__call_return(struct db_export *dbe, struct call_return *cr,
return 0;
}
+
+static int db_export__pid_tid(struct db_export *dbe, struct machine *machine,
+ pid_t pid, pid_t tid, u64 *db_id,
+ struct comm **comm_ptr, bool *is_idle)
+{
+ struct thread *thread = machine__find_thread(machine, pid, tid);
+ struct thread *main_thread;
+ int err = 0;
+
+ if (!thread || !thread->comm_set)
+ goto out_put;
+
+ *is_idle = !thread->pid_ && !thread->tid;
+
+ main_thread = thread__main_thread(machine, thread);
+
+ err = db_export__threads(dbe, thread, main_thread, machine, comm_ptr);
+
+ *db_id = thread->db_id;
+
+ thread__put(main_thread);
+out_put:
+ thread__put(thread);
+
+ return err;
+}
+
+int db_export__switch(struct db_export *dbe, union perf_event *event,
+ struct perf_sample *sample, struct machine *machine)
+{
+ bool out = event->header.misc & PERF_RECORD_MISC_SWITCH_OUT;
+ bool out_preempt = out &&
+ (event->header.misc & PERF_RECORD_MISC_SWITCH_OUT_PREEMPT);
+ int flags = out | (out_preempt << 1);
+ bool is_idle_a = false, is_idle_b = false;
+ u64 th_a_id = 0, th_b_id = 0;
+ u64 comm_out_id, comm_in_id;
+ struct comm *comm_a = NULL;
+ struct comm *comm_b = NULL;
+ u64 th_out_id, th_in_id;
+ u64 db_id;
+ int err;
+
+ err = db_export__machine(dbe, machine);
+ if (err)
+ return err;
+
+ err = db_export__pid_tid(dbe, machine, sample->pid, sample->tid,
+ &th_a_id, &comm_a, &is_idle_a);
+ if (err)
+ return err;
+
+ if (event->header.type == PERF_RECORD_SWITCH_CPU_WIDE) {
+ pid_t pid = event->context_switch.next_prev_pid;
+ pid_t tid = event->context_switch.next_prev_tid;
+
+ err = db_export__pid_tid(dbe, machine, pid, tid, &th_b_id,
+ &comm_b, &is_idle_b);
+ if (err)
+ return err;
+ }
+
+ /*
+ * Do not export if both threads are unknown (i.e. not being traced),
+ * or one is unknown and the other is the idle task.
+ */
+ if ((!th_a_id || is_idle_a) && (!th_b_id || is_idle_b))
+ return 0;
+
+ db_id = ++dbe->context_switch_last_db_id;
+
+ if (out) {
+ th_out_id = th_a_id;
+ th_in_id = th_b_id;
+ comm_out_id = comm_a ? comm_a->db_id : 0;
+ comm_in_id = comm_b ? comm_b->db_id : 0;
+ } else {
+ th_out_id = th_b_id;
+ th_in_id = th_a_id;
+ comm_out_id = comm_b ? comm_b->db_id : 0;
+ comm_in_id = comm_a ? comm_a->db_id : 0;
+ }
+
+ if (dbe->export_context_switch)
+ return dbe->export_context_switch(dbe, db_id, machine, sample,
+ th_out_id, comm_out_id,
+ th_in_id, comm_in_id, flags);
+ return 0;
+}
diff --git a/tools/perf/util/db-export.h b/tools/perf/util/db-export.h
index e8a64028a386..ba1f62a5fe10 100644
--- a/tools/perf/util/db-export.h
+++ b/tools/perf/util/db-export.h
@@ -43,7 +43,8 @@ struct db_export {
int (*export_machine)(struct db_export *dbe, struct machine *machine);
int (*export_thread)(struct db_export *dbe, struct thread *thread,
u64 main_thread_db_id, struct machine *machine);
- int (*export_comm)(struct db_export *dbe, struct comm *comm);
+ int (*export_comm)(struct db_export *dbe, struct comm *comm,
+ struct thread *thread);
int (*export_comm_thread)(struct db_export *dbe, u64 db_id,
struct comm *comm, struct thread *thread);
int (*export_dso)(struct db_export *dbe, struct dso *dso,
@@ -56,6 +57,11 @@ struct db_export {
int (*export_call_path)(struct db_export *dbe, struct call_path *cp);
int (*export_call_return)(struct db_export *dbe,
struct call_return *cr);
+ int (*export_context_switch)(struct db_export *dbe, u64 db_id,
+ struct machine *machine,
+ struct perf_sample *sample,
+ u64 th_out_id, u64 comm_out_id,
+ u64 th_in_id, u64 comm_in_id, int flags);
struct call_return_processor *crp;
struct call_path_root *cpr;
u64 evsel_last_db_id;
@@ -68,18 +74,19 @@ struct db_export {
u64 sample_last_db_id;
u64 call_path_last_db_id;
u64 call_return_last_db_id;
- struct list_head deferred;
+ u64 context_switch_last_db_id;
};
int db_export__init(struct db_export *dbe);
-int db_export__flush(struct db_export *dbe);
void db_export__exit(struct db_export *dbe);
int db_export__evsel(struct db_export *dbe, struct perf_evsel *evsel);
int db_export__machine(struct db_export *dbe, struct machine *machine);
int db_export__thread(struct db_export *dbe, struct thread *thread,
- struct machine *machine, struct comm *comm);
+ struct machine *machine, struct thread *main_thread);
int db_export__comm(struct db_export *dbe, struct comm *comm,
- struct thread *main_thread);
+ struct thread *thread);
+int db_export__exec_comm(struct db_export *dbe, struct comm *comm,
+ struct thread *main_thread);
int db_export__comm_thread(struct db_export *dbe, struct comm *comm,
struct thread *thread);
int db_export__dso(struct db_export *dbe, struct dso *dso,
@@ -97,5 +104,7 @@ int db_export__branch_types(struct db_export *dbe);
int db_export__call_path(struct db_export *dbe, struct call_path *cp);
int db_export__call_return(struct db_export *dbe, struct call_return *cr,
u64 *parent_db_id);
+int db_export__switch(struct db_export *dbe, union perf_event *event,
+ struct perf_sample *sample, struct machine *machine);
#endif
diff --git a/tools/perf/util/rlimit.c b/tools/perf/util/rlimit.c
new file mode 100644
index 000000000000..13521d392a22
--- /dev/null
+++ b/tools/perf/util/rlimit.c
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: LGPL-2.1 */
+
+#include "util/debug.h"
+#include "util/rlimit.h"
+#include <sys/time.h>
+#include <sys/resource.h>
+
+/*
+ * Bump the memlock so that we can get bpf maps of a reasonable size,
+ * like the ones used with 'perf trace' and with 'perf test bpf',
+ * improve this to some specific request if needed.
+ */
+void rlimit__bump_memlock(void)
+{
+ struct rlimit rlim;
+
+ if (getrlimit(RLIMIT_MEMLOCK, &rlim) == 0) {
+ rlim.rlim_cur *= 4;
+ rlim.rlim_max *= 4;
+
+ if (setrlimit(RLIMIT_MEMLOCK, &rlim) < 0) {
+ rlim.rlim_cur /= 2;
+ rlim.rlim_max /= 2;
+
+ if (setrlimit(RLIMIT_MEMLOCK, &rlim) < 0)
+ pr_debug("Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc\n");
+ }
+ }
+}
diff --git a/tools/perf/util/rlimit.h b/tools/perf/util/rlimit.h
new file mode 100644
index 000000000000..9f59d8e710a3
--- /dev/null
+++ b/tools/perf/util/rlimit.h
@@ -0,0 +1,6 @@
+#ifndef __PERF_RLIMIT_H_
+#define __PERF_RLIMIT_H_
+/* SPDX-License-Identifier: LGPL-2.1 */
+
+void rlimit__bump_memlock(void);
+#endif // __PERF_RLIMIT_H_
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 112bed65232f..25dc1d765553 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -113,6 +113,7 @@ struct tables {
PyObject *call_path_handler;
PyObject *call_return_handler;
PyObject *synth_handler;
+ PyObject *context_switch_handler;
bool db_export_mode;
};
@@ -1011,15 +1012,19 @@ static int python_export_thread(struct db_export *dbe, struct thread *thread,
return 0;
}
-static int python_export_comm(struct db_export *dbe, struct comm *comm)
+static int python_export_comm(struct db_export *dbe, struct comm *comm,
+ struct thread *thread)
{
struct tables *tables = container_of(dbe, struct tables, dbe);
PyObject *t;
- t = tuple_new(2);
+ t = tuple_new(5);
tuple_set_u64(t, 0, comm->db_id);
tuple_set_string(t, 1, comm__str(comm));
+ tuple_set_u64(t, 2, thread->db_id);
+ tuple_set_u64(t, 3, comm->start);
+ tuple_set_s32(t, 4, comm->exec);
call_object(tables->comm_handler, t, "comm_table");
@@ -1233,6 +1238,34 @@ static int python_export_call_return(struct db_export *dbe,
return 0;
}
+static int python_export_context_switch(struct db_export *dbe, u64 db_id,
+ struct machine *machine,
+ struct perf_sample *sample,
+ u64 th_out_id, u64 comm_out_id,
+ u64 th_in_id, u64 comm_in_id, int flags)
+{
+ struct tables *tables = container_of(dbe, struct tables, dbe);
+ PyObject *t;
+
+ t = tuple_new(9);
+
+ tuple_set_u64(t, 0, db_id);
+ tuple_set_u64(t, 1, machine->db_id);
+ tuple_set_u64(t, 2, sample->time);
+ tuple_set_s32(t, 3, sample->cpu);
+ tuple_set_u64(t, 4, th_out_id);
+ tuple_set_u64(t, 5, comm_out_id);
+ tuple_set_u64(t, 6, th_in_id);
+ tuple_set_u64(t, 7, comm_in_id);
+ tuple_set_s32(t, 8, flags);
+
+ call_object(tables->context_switch_handler, t, "context_switch");
+
+ Py_DECREF(t);
+
+ return 0;
+}
+
static int python_process_call_return(struct call_return *cr, u64 *parent_db_id,
void *data)
{
@@ -1296,6 +1329,16 @@ static void python_process_event(union perf_event *event,
}
}
+static void python_process_switch(union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct tables *tables = &tables_global;
+
+ if (tables->db_export_mode)
+ db_export__switch(&tables->dbe, event, sample, machine);
+}
+
static void get_handler_name(char *str, size_t size,
struct perf_evsel *evsel)
{
@@ -1511,6 +1554,7 @@ static void set_table_handlers(struct tables *tables)
SET_TABLE_HANDLER(sample);
SET_TABLE_HANDLER(call_path);
SET_TABLE_HANDLER(call_return);
+ SET_TABLE_HANDLER(context_switch);
/*
* Synthesized events are samples but with architecture-specific data
@@ -1620,9 +1664,7 @@ static int python_start_script(const char *script, int argc, const char **argv)
static int python_flush_script(void)
{
- struct tables *tables = &tables_global;
-
- return db_export__flush(&tables->dbe);
+ return 0;
}
/*
@@ -1831,6 +1873,7 @@ struct scripting_ops python_scripting_ops = {
.flush_script = python_flush_script,
.stop_script = python_stop_script,
.process_event = python_process_event,
+ .process_switch = python_process_switch,
.process_stat = python_process_stat,
.process_stat_interval = python_process_stat_interval,
.generate_script = python_generate_script,
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index d9b0a942090a..c7002fe11673 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -81,6 +81,9 @@ struct scripting_ops {
struct perf_sample *sample,
struct perf_evsel *evsel,
struct addr_location *al);
+ void (*process_switch)(union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine);
void (*process_stat)(struct perf_stat_config *config,
struct perf_evsel *evsel, u64 tstamp);
void (*process_stat_interval)(u64 tstamp);
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [GIT pull] smp/urgent for 5.3-rc1
2019-07-20 12:50 ` [GIT pull] smp/urgent " Thomas Gleixner
@ 2019-07-20 18:40 ` pr-tracker-bot
0 siblings, 0 replies; 10+ messages in thread
From: pr-tracker-bot @ 2019-07-20 18:40 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Sat, 20 Jul 2019 12:50:00 -0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git smp-urgent-for-linus
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/4b01f5a4c9c4f0c502875c1fb31dcd5d0df86475
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [GIT pull] x86/urgent for 5.3-rc1
2019-07-20 12:50 ` [GIT pull] x86/urgent " Thomas Gleixner
@ 2019-07-20 18:40 ` pr-tracker-bot
0 siblings, 0 replies; 10+ messages in thread
From: pr-tracker-bot @ 2019-07-20 18:40 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Sat, 20 Jul 2019 12:50:00 -0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86-urgent-for-linus
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/c6dd78fcb8eefa15dd861889e0f59d301cb5230c
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [GIT pull] sched/urgent for 5.3-rc1
2019-07-20 12:50 ` [GIT pull] sched/urgent " Thomas Gleixner
@ 2019-07-20 18:40 ` pr-tracker-bot
0 siblings, 0 replies; 10+ messages in thread
From: pr-tracker-bot @ 2019-07-20 18:40 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Sat, 20 Jul 2019 12:50:00 -0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched-urgent-for-linus
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/70e6e1b971e46f5c1c2d72217ba62401a2edc22b
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [GIT pull] core/urgent for 5.3-rc1
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
` (3 preceding siblings ...)
2019-07-20 12:50 ` [GIT pull] x86/urgent " Thomas Gleixner
@ 2019-07-20 18:40 ` pr-tracker-bot
4 siblings, 0 replies; 10+ messages in thread
From: pr-tracker-bot @ 2019-07-20 18:40 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Sat, 20 Jul 2019 12:50:00 -0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-urgent-for-linus
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/e6023adc5c6af79ac8ac5b17939f58091fa0d870
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [GIT pull] perf/urgent for 5.3-rc1
2019-07-20 12:50 ` [GIT pull] perf/urgent " Thomas Gleixner
@ 2019-07-20 18:40 ` pr-tracker-bot
0 siblings, 0 replies; 10+ messages in thread
From: pr-tracker-bot @ 2019-07-20 18:40 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Sat, 20 Jul 2019 12:50:00 -0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf-urgent-for-linus
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/46f5c0cc3af0ecb76224a91d2997d74e35ff7821
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-07-20 18:40 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-20 12:50 [GIT pull] core/urgent for 5.3-rc1 Thomas Gleixner
2019-07-20 12:50 ` [GIT pull] smp/urgent " Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] perf/urgent " Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] sched/urgent " Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 12:50 ` [GIT pull] x86/urgent " Thomas Gleixner
2019-07-20 18:40 ` pr-tracker-bot
2019-07-20 18:40 ` [GIT pull] core/urgent " pr-tracker-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®