* [GIT pull] irq/core for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
@ 2024-09-16 8:38 ` Thomas Gleixner
2024-09-17 6:35 ` pr-tracker-bot
2024-09-16 8:38 ` [GIT pull] smp/core " Thomas Gleixner
` (5 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: Thomas Gleixner @ 2024-09-16 8:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest irq/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq-core-2024-09-16
up to: a6fe30d1e365: genirq: Use cpumask_intersects()
Updates for the interrupt subsystem:
- Core:
- Remove a global lock in the affinity setting code
The lock protects a cpumask for intermediate results and the lock
causes a bottleneck on simultaneous start of multiple virtual
machines. Replace the lock and the static cpumask with a per CPU
cpumask which is nicely serialized by raw spinlock held when
executing this code.
- Provide support for giving a suffix to interrupt domain names.
That's required to support devices with subfunctions so that the
domain names are distinct even if they originate from the same
device node.
- The usual set of cleanups and enhancements all over the place
- Drivers:
- Support for longarch AVEC interrupt chip
- Refurbishment of the Armada driver so it can be extended for new
variants.
- The usual set of cleanups and enhancements all over the place
Thanks,
tglx
------------------>
Andy Shevchenko (2):
irqdomain: Clarify checks for bus_token
irqdomain: Remove stray '-' in the domain name
Anshuman Khandual (1):
irqchip/gic-v4.1: Replace bare number with ID_AA64PFR0_EL1_GIC_V4P1
Caleb Sander Mateos (1):
softirq: Remove unused 'action' parameter from action callback
Costa Shulyupin (3):
genirq: Fix typo in struct comment
genirq/cpuhotplug: Use cpumask_intersects()
genirq: Use cpumask_intersects()
Hongbo Li (1):
irqdomain: Use IS_ERR_OR_NULL() in irq_domain_trim_hierarchy()
Huacai Chen (4):
irqchip/loongson-pch-msi: Switch to MSI parent domains
LoongArch: Move irqchip function prototypes to irq-loongson.h
LoongArch: Architectural preparation for AVEC irqchip
irqchip/loongson-eiointc: Rename CPUHP_AP_IRQ_LOONGARCH_STARTING
Jeff Xie (2):
genirq/proc: Correctly set file permissions for affinity control files
genirq/proc: Change the return value for set affinity permission error
Jinjie Ruan (3):
irqchip: Remove asmlinkage for handlers registered with set_handle_irq()
genirq/proc: Use irq_move_pending() in show_irq_affinity()
genirq/msi: Use kmemdup_array() instead of kmemdup()
Jiri Slaby (SUSE) (3):
genirq/irq_sim: Remove unused irq_sim_work_ctx:: Irq_base
genirq: Remove unused irq_chip_generic:: {type,polarity}_cache
genirq: Remove irq_chip_regs:: Polarity
Konrad Dybcio (1):
irqchip/apple-aic: Only access system registers on SoCs which provide them
Marc Zyngier (1):
genirq: Get rid of global lock in irq_do_set_affinity()
Marek Behún (41):
irqchip/armada-370-xp: Drop _OFFS suffix from some register constants
irqchip/armada-370-xp: Change register constant suffix from _MSK to _MASK
irqchip/armada-370-xp: Change spaces to tabs
irqchip/armada-370-xp: Use BIT() and GENMASK() macros
irqchip/armada-370-xp: Cosmetic fix parentheses in register constant definitions
irqchip/armada-370-xp: Change register constants prefix to MPIC_
irqchip/armada-370-xp: Use correct type for cpu variable
irqchip/armada-370-xp: Simplify is_percpu_irq() code
irqchip/armada-370-xp: Change to SPDX license identifier
irqchip/armada-370-xp: Declare iterators in for loop
irqchip/armada-370-xp: Rename variable for consistency
irqchip/armada-370-xp: Use unsigned int type for virqs
irqchip/armada-370-xp: Use !virq instead of virq == 0 in condition
irqchip/armada-370-xp: Simplify ipi_resume() code
irqchip/armada-370-xp: Improve indentation
irqchip/armada-370-xp: Change symbol prefixes to mpic
irqchip/armada-370-xp: Don't read number of supported interrupts multiple times
irqchip/armada-370-xp: Use FIELD_GET() and named register constant
irqchip/armada-370-xp: Refactor mpic_handle_msi_irq() code
irqchip/armada-370-xp: Refactor handling IPI interrupts
irqchip/armada-370-xp: Use consistent variable names for hwirqs
irqchip/armada-370-xp: Use consistent types when iterating interrupts
irqchip/armada-370-xp: Use consistent name for struct irq_data variables
irqchip/armada-370-xp: Simplify mpic_reenable_percpu() and mpic_resume()
irqchip/armada-370-xp: Drop redundant continue
irqchip/armada-370-xp: Rename variable for consistency
irqchip/armada-370-xp: Use u32 type instead of unsigned long where possieble
irqchip/armada-370-xp: Refactor initial memory regions mapping
irqchip/armada-370-xp: Print error and return error code on initialization failure
irqchip/armada-370-xp: Drop IPI_DOORBELL_START and rename IPI_DOORBELL_END
irqchip/armada-370-xp: Drop msi_doorbell_end()
irqchip/armada-370-xp: Add the __init attribute to mpic_msi_init()
irqchip/armada-370-xp: Put __init attribute after return type in mpic_ipi_init()
irqchip/armada-370-xp: Put static variables into driver private structure
irqchip/armada-370-xp: Put MSI doorbell limits into the mpic structure
irqchip/armada-370-xp: Pass around the driver private structure
irqchip/armada-370-xp: Dynamically allocate the driver private structure
irqchip/armada-370-xp: Fix reenabling last per-CPU interrupt
irqchip/armada-370-xp: Iterate only valid bits of the per-CPU interrupt cause register
irqchip/armada-370-xp: Allow mapping only per-CPU interrupts
irqchip/armada-370-xp: Use mpic_is_ipi_available() in mpic_of_init()
Matti Vaittinen (3):
irqdomain: Simplify simple and legacy domain creation
irqdomain: Allow giving name suffix for domain
irqdomain: Always associate interrupts for legacy domains
Nick Chan (3):
dt-bindings: apple,aic: Document A7-A11 compatibles
irqchip/apple-aic: Skip unnecessary enabling of use_fast_ipi
irqchip/apple-aic: Add a new "Global fast IPIs only" feature level
Thomas Gleixner (1):
irqdomain: Cleanup domain name allocation
Tianyang Zhang (2):
irqchip/loongson-pch-msi: Prepare get_pch_msi_handle() for AVECINTC
irqchip/loongarch-avec: Add AVEC irqchip support
Zhang Zekun (1):
irqchip/mbigen: Simplify code logic with for_each_child_of_node_scoped()
.../bindings/interrupt-controller/apple,aic.yaml | 14 +-
arch/loongarch/Kconfig | 1 +
arch/loongarch/include/asm/cpu-features.h | 1 +
arch/loongarch/include/asm/cpu.h | 2 +
arch/loongarch/include/asm/hardirq.h | 3 +-
arch/loongarch/include/asm/irq.h | 31 +-
arch/loongarch/include/asm/loongarch.h | 18 +-
arch/loongarch/include/asm/smp.h | 2 +
arch/loongarch/kernel/cpu-probe.c | 3 +-
arch/loongarch/kernel/irq.c | 12 +
arch/loongarch/kernel/paravirt.c | 5 +
arch/loongarch/kernel/smp.c | 6 +
block/blk-mq.c | 2 +-
drivers/irqchip/Kconfig | 1 +
drivers/irqchip/Makefile | 2 +-
drivers/irqchip/irq-apple-aic.c | 59 +-
drivers/irqchip/irq-armada-370-xp.c | 861 +++++++++++----------
drivers/irqchip/irq-atmel-aic.c | 3 +-
drivers/irqchip/irq-atmel-aic5.c | 3 +-
drivers/irqchip/irq-clps711x.c | 2 +-
drivers/irqchip/irq-davinci-cp-intc.c | 3 +-
drivers/irqchip/irq-ftintc010.c | 2 +-
drivers/irqchip/irq-gic-v3.c | 2 +-
drivers/irqchip/irq-gic-v4.c | 2 +-
drivers/irqchip/irq-ixp4xx.c | 3 +-
drivers/irqchip/irq-loongarch-avec.c | 425 ++++++++++
drivers/irqchip/irq-loongarch-cpu.c | 7 +-
drivers/irqchip/irq-loongson-eiointc.c | 9 +-
drivers/irqchip/irq-loongson-htvec.c | 2 +
drivers/irqchip/irq-loongson-liointc.c | 2 +
drivers/irqchip/irq-loongson-pch-lpc.c | 2 +
drivers/irqchip/irq-loongson-pch-msi.c | 83 +-
drivers/irqchip/irq-loongson-pch-pic.c | 2 +
drivers/irqchip/irq-loongson.h | 27 +
drivers/irqchip/irq-mbigen.c | 24 +-
drivers/irqchip/irq-omap-intc.c | 3 +-
drivers/irqchip/irq-sa11x0.c | 3 +-
drivers/irqchip/irq-versatile-fpga.c | 2 +-
include/linux/cpuhotplug.h | 3 +-
include/linux/interrupt.h | 6 +-
include/linux/irq.h | 6 -
include/linux/irqdomain.h | 8 +
kernel/irq/chip.c | 2 +-
kernel/irq/cpuhotplug.c | 4 +-
kernel/irq/irq_sim.c | 1 -
kernel/irq/irqdomain.c | 210 +++--
kernel/irq/manage.c | 21 +-
kernel/irq/migration.c | 4 +-
kernel/irq/msi.c | 2 +-
kernel/irq/proc.c | 17 +-
kernel/rcu/tiny.c | 2 +-
kernel/rcu/tree.c | 2 +-
kernel/sched/fair.c | 2 +-
kernel/softirq.c | 15 +-
kernel/time/hrtimer.c | 2 +-
kernel/time/timer.c | 2 +-
lib/irq_poll.c | 2 +-
net/core/dev.c | 4 +-
58 files changed, 1267 insertions(+), 682 deletions(-)
create mode 100644 drivers/irqchip/irq-loongarch-avec.c
create mode 100644 drivers/irqchip/irq-loongson.h
diff --git a/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml b/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
index 698588e9aa86..4be9b596a790 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
+++ b/Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
@@ -31,13 +31,25 @@ description: |
This device also represents the FIQ interrupt sources on platforms using AIC,
which do not go through a discrete interrupt controller.
+ IPIs may be performed via MMIO registers on all variants of AIC. Starting
+ from A11, system registers may also be used for "fast" IPIs. Starting from
+ M1, even faster IPIs within the same cluster may be achieved by writing to
+ a "local" fast IPI register as opposed to using the "global" fast IPI
+ register.
+
allOf:
- $ref: /schemas/interrupt-controller.yaml#
properties:
compatible:
items:
- - const: apple,t8103-aic
+ - enum:
+ - apple,s5l8960x-aic
+ - apple,t7000-aic
+ - apple,s8000-aic
+ - apple,t8010-aic
+ - apple,t8015-aic
+ - apple,t8103-aic
- const: apple,aic
interrupt-controller: true
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index 70f169210b52..0e3abf7b0bd3 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -85,6 +85,7 @@ config LOONGARCH
select GENERIC_ENTRY
select GENERIC_GETTIMEOFDAY
select GENERIC_IOREMAP if !ARCH_IOREMAP
+ select GENERIC_IRQ_MATRIX_ALLOCATOR
select GENERIC_IRQ_MULTI_HANDLER
select GENERIC_IRQ_PROBE
select GENERIC_IRQ_SHOW
diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h
index 2eafe6a6aca8..16a716f88a5c 100644
--- a/arch/loongarch/include/asm/cpu-features.h
+++ b/arch/loongarch/include/asm/cpu-features.h
@@ -65,5 +65,6 @@
#define cpu_has_guestid cpu_opt(LOONGARCH_CPU_GUESTID)
#define cpu_has_hypervisor cpu_opt(LOONGARCH_CPU_HYPERVISOR)
#define cpu_has_ptw cpu_opt(LOONGARCH_CPU_PTW)
+#define cpu_has_avecint cpu_opt(LOONGARCH_CPU_AVECINT)
#endif /* __ASM_CPU_FEATURES_H */
diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h
index 48b9f7168bcc..843f9c4ec980 100644
--- a/arch/loongarch/include/asm/cpu.h
+++ b/arch/loongarch/include/asm/cpu.h
@@ -99,6 +99,7 @@ enum cpu_type_enum {
#define CPU_FEATURE_GUESTID 24 /* CPU has GuestID feature */
#define CPU_FEATURE_HYPERVISOR 25 /* CPU has hypervisor (running in VM) */
#define CPU_FEATURE_PTW 26 /* CPU has hardware page table walker */
+#define CPU_FEATURE_AVECINT 27 /* CPU has avec interrupt */
#define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG)
#define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM)
@@ -127,5 +128,6 @@ enum cpu_type_enum {
#define LOONGARCH_CPU_GUESTID BIT_ULL(CPU_FEATURE_GUESTID)
#define LOONGARCH_CPU_HYPERVISOR BIT_ULL(CPU_FEATURE_HYPERVISOR)
#define LOONGARCH_CPU_PTW BIT_ULL(CPU_FEATURE_PTW)
+#define LOONGARCH_CPU_AVECINT BIT_ULL(CPU_FEATURE_AVECINT)
#endif /* _ASM_CPU_H */
diff --git a/arch/loongarch/include/asm/hardirq.h b/arch/loongarch/include/asm/hardirq.h
index 1d7feb719515..10da8d6961cb 100644
--- a/arch/loongarch/include/asm/hardirq.h
+++ b/arch/loongarch/include/asm/hardirq.h
@@ -12,12 +12,13 @@
extern void ack_bad_irq(unsigned int irq);
#define ack_bad_irq ack_bad_irq
-#define NR_IPI 3
+#define NR_IPI 4
enum ipi_msg_type {
IPI_RESCHEDULE,
IPI_CALL_FUNCTION,
IPI_IRQ_WORK,
+ IPI_CLEAR_VECTOR,
};
typedef struct {
diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
index 480418bc5071..9c2ca785faa9 100644
--- a/arch/loongarch/include/asm/irq.h
+++ b/arch/loongarch/include/asm/irq.h
@@ -39,11 +39,22 @@ void spurious_interrupt(void);
#define NR_IRQS_LEGACY 16
+/*
+ * 256 Vectors Mapping for AVECINTC:
+ *
+ * 0 - 15: Mapping classic IPs, e.g. IP0-12.
+ * 16 - 255: Mapping vectors for external IRQ.
+ *
+ */
+#define NR_VECTORS 256
+#define NR_LEGACY_VECTORS 16
+#define IRQ_MATRIX_BITS NR_VECTORS
+
#define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace
void arch_trigger_cpumask_backtrace(const struct cpumask *mask, int exclude_cpu);
#define MAX_IO_PICS 2
-#define NR_IRQS (64 + (256 * MAX_IO_PICS))
+#define NR_IRQS (64 + NR_VECTORS * (NR_CPUS + MAX_IO_PICS))
struct acpi_vector_group {
int node;
@@ -65,7 +76,7 @@ extern struct acpi_vector_group msi_group[MAX_IO_PICS];
#define LOONGSON_LPC_LAST_IRQ (LOONGSON_LPC_IRQ_BASE + 15)
#define LOONGSON_CPU_IRQ_BASE 16
-#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 14)
+#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 15)
#define LOONGSON_PCH_IRQ_BASE 64
#define LOONGSON_PCH_ACPI_IRQ (LOONGSON_PCH_IRQ_BASE + 47)
@@ -88,20 +99,8 @@ struct acpi_madt_bio_pic;
struct acpi_madt_msi_pic;
struct acpi_madt_lpc_pic;
-int liointc_acpi_init(struct irq_domain *parent,
- struct acpi_madt_lio_pic *acpi_liointc);
-int eiointc_acpi_init(struct irq_domain *parent,
- struct acpi_madt_eio_pic *acpi_eiointc);
-
-int htvec_acpi_init(struct irq_domain *parent,
- struct acpi_madt_ht_pic *acpi_htvec);
-int pch_lpc_acpi_init(struct irq_domain *parent,
- struct acpi_madt_lpc_pic *acpi_pchlpc);
-int pch_msi_acpi_init(struct irq_domain *parent,
- struct acpi_madt_msi_pic *acpi_pchmsi);
-int pch_pic_acpi_init(struct irq_domain *parent,
- struct acpi_madt_bio_pic *acpi_pchpic);
-int find_pch_pic(u32 gsi);
+void complete_irq_moving(void);
+
struct fwnode_handle *get_pch_msi_handle(int pci_segment);
extern struct acpi_madt_lio_pic *acpi_liointc;
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 04a78010fc72..631d249b3ef2 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -253,8 +253,8 @@
#define CSR_ESTAT_EXC_WIDTH 6
#define CSR_ESTAT_EXC (_ULCAST_(0x3f) << CSR_ESTAT_EXC_SHIFT)
#define CSR_ESTAT_IS_SHIFT 0
-#define CSR_ESTAT_IS_WIDTH 14
-#define CSR_ESTAT_IS (_ULCAST_(0x3fff) << CSR_ESTAT_IS_SHIFT)
+#define CSR_ESTAT_IS_WIDTH 15
+#define CSR_ESTAT_IS (_ULCAST_(0x7fff) << CSR_ESTAT_IS_SHIFT)
#define LOONGARCH_CSR_ERA 0x6 /* ERA */
@@ -649,6 +649,13 @@
#define LOONGARCH_CSR_CTAG 0x98 /* TagLo + TagHi */
+#define LOONGARCH_CSR_ISR0 0xa0
+#define LOONGARCH_CSR_ISR1 0xa1
+#define LOONGARCH_CSR_ISR2 0xa2
+#define LOONGARCH_CSR_ISR3 0xa3
+
+#define LOONGARCH_CSR_IRR 0xa4
+
#define LOONGARCH_CSR_PRID 0xc0
/* Shadow MCSR : 0xc0 ~ 0xff */
@@ -1011,7 +1018,7 @@
/*
* CSR_ECFG IM
*/
-#define ECFG0_IM 0x00001fff
+#define ECFG0_IM 0x00005fff
#define ECFGB_SIP0 0
#define ECFGF_SIP0 (_ULCAST_(1) << ECFGB_SIP0)
#define ECFGB_SIP1 1
@@ -1054,6 +1061,7 @@
#define IOCSRF_EIODECODE BIT_ULL(9)
#define IOCSRF_FLATMODE BIT_ULL(10)
#define IOCSRF_VM BIT_ULL(11)
+#define IOCSRF_AVEC BIT_ULL(15)
#define LOONGARCH_IOCSR_VENDOR 0x10
@@ -1065,6 +1073,7 @@
#define IOCSR_MISC_FUNC_SOFT_INT BIT_ULL(10)
#define IOCSR_MISC_FUNC_TIMER_RESET BIT_ULL(21)
#define IOCSR_MISC_FUNC_EXT_IOI_EN BIT_ULL(48)
+#define IOCSR_MISC_FUNC_AVEC_EN BIT_ULL(51)
#define LOONGARCH_IOCSR_CPUTEMP 0x428
@@ -1387,9 +1396,10 @@ __BUILD_CSR_OP(tlbidx)
#define INT_TI 11 /* Timer */
#define INT_IPI 12
#define INT_NMI 13
+#define INT_AVEC 14
/* ExcCodes corresponding to interrupts */
-#define EXCCODE_INT_NUM (INT_NMI + 1)
+#define EXCCODE_INT_NUM (INT_AVEC + 1)
#define EXCCODE_INT_START 64
#define EXCCODE_INT_END (EXCCODE_INT_START + EXCCODE_INT_NUM - 1)
diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h
index 50db503f44e3..3383c9d24e94 100644
--- a/arch/loongarch/include/asm/smp.h
+++ b/arch/loongarch/include/asm/smp.h
@@ -70,10 +70,12 @@ extern int __cpu_logical_map[NR_CPUS];
#define ACTION_RESCHEDULE 1
#define ACTION_CALL_FUNCTION 2
#define ACTION_IRQ_WORK 3
+#define ACTION_CLEAR_VECTOR 4
#define SMP_BOOT_CPU BIT(ACTION_BOOT_CPU)
#define SMP_RESCHEDULE BIT(ACTION_RESCHEDULE)
#define SMP_CALL_FUNCTION BIT(ACTION_CALL_FUNCTION)
#define SMP_IRQ_WORK BIT(ACTION_IRQ_WORK)
+#define SMP_CLEAR_VECTOR BIT(ACTION_CLEAR_VECTOR)
struct secondary_data {
unsigned long stack;
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
index 55320813ee08..14f0449f5452 100644
--- a/arch/loongarch/kernel/cpu-probe.c
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -106,7 +106,6 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c)
elf_hwcap |= HWCAP_LOONGARCH_CRC32;
}
-
config = read_cpucfg(LOONGARCH_CPUCFG2);
if (config & CPUCFG2_LAM) {
c->options |= LOONGARCH_CPU_LAM;
@@ -174,6 +173,8 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c)
c->options |= LOONGARCH_CPU_FLATMODE;
if (config & IOCSRF_EIODECODE)
c->options |= LOONGARCH_CPU_EIODECODE;
+ if (config & IOCSRF_AVEC)
+ c->options |= LOONGARCH_CPU_AVECINT;
if (config & IOCSRF_VM)
c->options |= LOONGARCH_CPU_HYPERVISOR;
diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c
index f4991c03514f..414f5249d70a 100644
--- a/arch/loongarch/kernel/irq.c
+++ b/arch/loongarch/kernel/irq.c
@@ -87,6 +87,18 @@ static void __init init_vec_parent_group(void)
acpi_table_parse(ACPI_SIG_MCFG, early_pci_mcfg_parse);
}
+int __init arch_probe_nr_irqs(void)
+{
+ int nr_io_pics = bitmap_weight(loongson_sysconf.cores_io_master, NR_CPUS);
+
+ if (!cpu_has_avecint)
+ nr_irqs = (64 + NR_VECTORS * nr_io_pics);
+ else
+ nr_irqs = (64 + NR_VECTORS * (nr_cpu_ids + nr_io_pics));
+
+ return NR_IRQS_LEGACY;
+}
+
void __init init_IRQ(void)
{
int i;
diff --git a/arch/loongarch/kernel/paravirt.c b/arch/loongarch/kernel/paravirt.c
index 9c9b75b76f62..4d736a4e488d 100644
--- a/arch/loongarch/kernel/paravirt.c
+++ b/arch/loongarch/kernel/paravirt.c
@@ -134,6 +134,11 @@ static irqreturn_t pv_ipi_interrupt(int irq, void *dev)
info->ipi_irqs[IPI_IRQ_WORK]++;
}
+ if (action & SMP_CLEAR_VECTOR) {
+ complete_irq_moving();
+ info->ipi_irqs[IPI_CLEAR_VECTOR]++;
+ }
+
return IRQ_HANDLED;
}
diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c
index ca405ab86aae..4adbbef3450a 100644
--- a/arch/loongarch/kernel/smp.c
+++ b/arch/loongarch/kernel/smp.c
@@ -72,6 +72,7 @@ static const char *ipi_types[NR_IPI] __tracepoint_string = {
[IPI_RESCHEDULE] = "Rescheduling interrupts",
[IPI_CALL_FUNCTION] = "Function call interrupts",
[IPI_IRQ_WORK] = "IRQ work interrupts",
+ [IPI_CLEAR_VECTOR] = "Clear vector interrupts",
};
void show_ipi_list(struct seq_file *p, int prec)
@@ -248,6 +249,11 @@ static irqreturn_t loongson_ipi_interrupt(int irq, void *dev)
per_cpu(irq_stat, cpu).ipi_irqs[IPI_IRQ_WORK]++;
}
+ if (action & SMP_CLEAR_VECTOR) {
+ complete_irq_moving();
+ per_cpu(irq_stat, cpu).ipi_irqs[IPI_CLEAR_VECTOR]++;
+ }
+
return IRQ_HANDLED;
}
diff --git a/block/blk-mq.c b/block/blk-mq.c
index e3c3c0c21b55..aa28157b1aaf 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1128,7 +1128,7 @@ static void blk_complete_reqs(struct llist_head *list)
rq->q->mq_ops->complete(rq);
}
-static __latent_entropy void blk_done_softirq(struct softirq_action *h)
+static __latent_entropy void blk_done_softirq(void)
{
blk_complete_reqs(this_cpu_ptr(&blk_cpu_done));
}
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index d078bdc48c38..341cd9ca5a05 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -685,6 +685,7 @@ config LOONGSON_PCH_MSI
depends on PCI
default MACH_LOONGSON64
select IRQ_DOMAIN_HIERARCHY
+ select IRQ_MSI_LIB
select PCI_MSI
help
Support for the Loongson PCH MSI Controller.
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 15635812b2d6..e3679ec2b9f7 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -110,7 +110,7 @@ obj-$(CONFIG_LS1X_IRQ) += irq-ls1x.o
obj-$(CONFIG_TI_SCI_INTR_IRQCHIP) += irq-ti-sci-intr.o
obj-$(CONFIG_TI_SCI_INTA_IRQCHIP) += irq-ti-sci-inta.o
obj-$(CONFIG_TI_PRUSS_INTC) += irq-pruss-intc.o
-obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o
+obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o irq-loongarch-avec.o
obj-$(CONFIG_LOONGSON_LIOINTC) += irq-loongson-liointc.o
obj-$(CONFIG_LOONGSON_EIOINTC) += irq-loongson-eiointc.o
obj-$(CONFIG_LOONGSON_HTPIC) += irq-loongson-htpic.o
diff --git a/drivers/irqchip/irq-apple-aic.c b/drivers/irqchip/irq-apple-aic.c
index 5c534d9fd2b0..da5250f0155c 100644
--- a/drivers/irqchip/irq-apple-aic.c
+++ b/drivers/irqchip/irq-apple-aic.c
@@ -234,7 +234,10 @@ enum fiq_hwirq {
AIC_NR_FIQ
};
+/* True if UNCORE/UNCORE2 and Sn_... IPI registers are present and used (A11+) */
static DEFINE_STATIC_KEY_TRUE(use_fast_ipi);
+/* True if SYS_IMP_APL_IPI_RR_LOCAL_EL1 exists for local fast IPIs (M1+) */
+static DEFINE_STATIC_KEY_TRUE(use_local_fast_ipi);
struct aic_info {
int version;
@@ -252,6 +255,7 @@ struct aic_info {
/* Features */
bool fast_ipi;
+ bool local_fast_ipi;
};
static const struct aic_info aic1_info __initconst = {
@@ -270,17 +274,32 @@ static const struct aic_info aic1_fipi_info __initconst = {
.fast_ipi = true,
};
+static const struct aic_info aic1_local_fipi_info __initconst = {
+ .version = 1,
+
+ .event = AIC_EVENT,
+ .target_cpu = AIC_TARGET_CPU,
+
+ .fast_ipi = true,
+ .local_fast_ipi = true,
+};
+
static const struct aic_info aic2_info __initconst = {
.version = 2,
.irq_cfg = AIC2_IRQ_CFG,
.fast_ipi = true,
+ .local_fast_ipi = true,
};
static const struct of_device_id aic_info_match[] = {
{
.compatible = "apple,t8103-aic",
+ .data = &aic1_local_fipi_info,
+ },
+ {
+ .compatible = "apple,t8015-aic",
.data = &aic1_fipi_info,
},
{
@@ -532,14 +551,9 @@ static void __exception_irq_entry aic_handle_fiq(struct pt_regs *regs)
* we check for everything here, even things we don't support yet.
*/
- if (read_sysreg_s(SYS_IMP_APL_IPI_SR_EL1) & IPI_SR_PENDING) {
- if (static_branch_likely(&use_fast_ipi)) {
- aic_handle_ipi(regs);
- } else {
- pr_err_ratelimited("Fast IPI fired. Acking.\n");
- write_sysreg_s(IPI_SR_PENDING, SYS_IMP_APL_IPI_SR_EL1);
- }
- }
+ if (static_branch_likely(&use_fast_ipi) &&
+ (read_sysreg_s(SYS_IMP_APL_IPI_SR_EL1) & IPI_SR_PENDING))
+ aic_handle_ipi(regs);
if (TIMER_FIRING(read_sysreg(cntp_ctl_el0)))
generic_handle_domain_irq(aic_irqc->hw_domain,
@@ -574,8 +588,9 @@ static void __exception_irq_entry aic_handle_fiq(struct pt_regs *regs)
AIC_FIQ_HWIRQ(irq));
}
- if (FIELD_GET(UPMCR0_IMODE, read_sysreg_s(SYS_IMP_APL_UPMCR0_EL1)) == UPMCR0_IMODE_FIQ &&
- (read_sysreg_s(SYS_IMP_APL_UPMSR_EL1) & UPMSR_IACT)) {
+ if (static_branch_likely(&use_fast_ipi) &&
+ (FIELD_GET(UPMCR0_IMODE, read_sysreg_s(SYS_IMP_APL_UPMCR0_EL1)) == UPMCR0_IMODE_FIQ) &&
+ (read_sysreg_s(SYS_IMP_APL_UPMSR_EL1) & UPMSR_IACT)) {
/* Same story with uncore PMCs */
pr_err_ratelimited("Uncore PMC FIQ fired. Masking.\n");
sysreg_clear_set_s(SYS_IMP_APL_UPMCR0_EL1, UPMCR0_IMODE,
@@ -750,12 +765,12 @@ static void aic_ipi_send_fast(int cpu)
u64 cluster = MPIDR_CLUSTER(mpidr);
u64 idx = MPIDR_CPU(mpidr);
- if (MPIDR_CLUSTER(my_mpidr) == cluster)
- write_sysreg_s(FIELD_PREP(IPI_RR_CPU, idx),
- SYS_IMP_APL_IPI_RR_LOCAL_EL1);
- else
+ if (static_branch_likely(&use_local_fast_ipi) && MPIDR_CLUSTER(my_mpidr) == cluster) {
+ write_sysreg_s(FIELD_PREP(IPI_RR_CPU, idx), SYS_IMP_APL_IPI_RR_LOCAL_EL1);
+ } else {
write_sysreg_s(FIELD_PREP(IPI_RR_CPU, idx) | FIELD_PREP(IPI_RR_CLUSTER, cluster),
SYS_IMP_APL_IPI_RR_GLOBAL_EL1);
+ }
isb();
}
@@ -811,7 +826,8 @@ static int aic_init_cpu(unsigned int cpu)
/* Mask all hard-wired per-CPU IRQ/FIQ sources */
/* Pending Fast IPI FIQs */
- write_sysreg_s(IPI_SR_PENDING, SYS_IMP_APL_IPI_SR_EL1);
+ if (static_branch_likely(&use_fast_ipi))
+ write_sysreg_s(IPI_SR_PENDING, SYS_IMP_APL_IPI_SR_EL1);
/* Timer FIQs */
sysreg_clear_set(cntp_ctl_el0, 0, ARCH_TIMER_CTRL_IT_MASK);
@@ -832,8 +848,10 @@ static int aic_init_cpu(unsigned int cpu)
FIELD_PREP(PMCR0_IMODE, PMCR0_IMODE_OFF));
/* Uncore PMC FIQ */
- sysreg_clear_set_s(SYS_IMP_APL_UPMCR0_EL1, UPMCR0_IMODE,
- FIELD_PREP(UPMCR0_IMODE, UPMCR0_IMODE_OFF));
+ if (static_branch_likely(&use_fast_ipi)) {
+ sysreg_clear_set_s(SYS_IMP_APL_UPMCR0_EL1, UPMCR0_IMODE,
+ FIELD_PREP(UPMCR0_IMODE, UPMCR0_IMODE_OFF));
+ }
/* Commit all of the above */
isb();
@@ -987,11 +1005,12 @@ static int __init aic_of_ic_init(struct device_node *node, struct device_node *p
off += sizeof(u32) * (irqc->max_irq >> 5); /* MASK_CLR */
off += sizeof(u32) * (irqc->max_irq >> 5); /* HW_STATE */
- if (irqc->info.fast_ipi)
- static_branch_enable(&use_fast_ipi);
- else
+ if (!irqc->info.fast_ipi)
static_branch_disable(&use_fast_ipi);
+ if (!irqc->info.local_fast_ipi)
+ static_branch_disable(&use_local_fast_ipi);
+
irqc->info.die_stride = off - start_off;
irqc->hw_domain = irq_domain_create_tree(of_node_to_fwnode(node),
diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c
index dce2b80bf439..d7c5ef248474 100644
--- a/drivers/irqchip/irq-armada-370-xp.c
+++ b/drivers/irqchip/irq-armada-370-xp.c
@@ -1,3 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0-only
/*
* Marvell Armada 370 and Armada XP SoC IRQ handling
*
@@ -7,13 +8,11 @@
* Gregory CLEMENT <gregory.clement@free-electrons.com>
* Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
* Ben Dooks <ben.dooks@codethink.co.uk>
- *
- * This file is licensed under the terms of the GNU General Public
- * License version 2. This program is licensed "as is" without any
- * warranty of any kind, whether express or implied.
*/
+#include <linux/bitfield.h>
#include <linux/bits.h>
+#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
@@ -66,19 +65,17 @@
* device
*
* The "global interrupt mask/unmask" is modified using the
- * ARMADA_370_XP_INT_SET_ENABLE_OFFS and
- * ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS registers, which are relative
- * to "main_int_base".
+ * MPIC_INT_SET_ENABLE and MPIC_INT_CLEAR_ENABLE
+ * registers, which are relative to "mpic->base".
*
- * The "per-CPU mask/unmask" is modified using the
- * ARMADA_370_XP_INT_SET_MASK_OFFS and
- * ARMADA_370_XP_INT_CLEAR_MASK_OFFS registers, which are relative to
- * "per_cpu_int_base". This base address points to a special address,
+ * The "per-CPU mask/unmask" is modified using the MPIC_INT_SET_MASK
+ * and MPIC_INT_CLEAR_MASK registers, which are relative to
+ * "mpic->per_cpu". This base address points to a special address,
* which automatically accesses the registers of the current CPU.
*
* The per-CPU mask/unmask can also be adjusted using the global
- * per-interrupt ARMADA_370_XP_INT_SOURCE_CTL register, which we use
- * to configure interrupt affinity.
+ * per-interrupt MPIC_INT_SOURCE_CTL register, which we use to
+ * configure interrupt affinity.
*
* Due to this model, all interrupts need to be mask/unmasked at two
* different levels: at the global level and at the per-CPU level.
@@ -92,9 +89,8 @@
* the current CPU, running the ->map() code. This allows to have
* the interrupt unmasked at this level in non-SMP
* configurations. In SMP configurations, the ->set_affinity()
- * callback is called, which using the
- * ARMADA_370_XP_INT_SOURCE_CTL() readjusts the per-CPU mask/unmask
- * for the interrupt.
+ * callback is called, which using the MPIC_INT_SOURCE_CTL()
+ * readjusts the per-CPU mask/unmask for the interrupt.
*
* The ->mask() and ->unmask() operations only mask/unmask the
* interrupt at the "global" level.
@@ -116,58 +112,84 @@
* at the per-CPU level.
*/
-/* Registers relative to main_int_base */
-#define ARMADA_370_XP_INT_CONTROL (0x00)
-#define ARMADA_370_XP_SW_TRIG_INT_OFFS (0x04)
-#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)
-#define ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS (0x34)
-#define ARMADA_370_XP_INT_SOURCE_CTL(irq) (0x100 + irq*4)
-#define ARMADA_370_XP_INT_SOURCE_CPU_MASK 0xF
-#define ARMADA_370_XP_INT_IRQ_FIQ_MASK(cpuid) ((BIT(0) | BIT(8)) << cpuid)
-
-/* Registers relative to per_cpu_int_base */
-#define ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS (0x08)
-#define ARMADA_370_XP_IN_DRBEL_MSK_OFFS (0x0c)
-#define ARMADA_375_PPI_CAUSE (0x10)
-#define ARMADA_370_XP_CPU_INTACK_OFFS (0x44)
-#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)
-#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C)
-#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)
-#define ARMADA_370_XP_INT_CAUSE_PERF(cpu) (1 << cpu)
-
-#define ARMADA_370_XP_MAX_PER_CPU_IRQS (28)
+/* Registers relative to mpic->base */
+#define MPIC_INT_CONTROL 0x00
+#define MPIC_INT_CONTROL_NUMINT_MASK GENMASK(12, 2)
+#define MPIC_SW_TRIG_INT 0x04
+#define MPIC_INT_SET_ENABLE 0x30
+#define MPIC_INT_CLEAR_ENABLE 0x34
+#define MPIC_INT_SOURCE_CTL(hwirq) (0x100 + (hwirq) * 4)
+#define MPIC_INT_SOURCE_CPU_MASK GENMASK(3, 0)
+#define MPIC_INT_IRQ_FIQ_MASK(cpuid) ((BIT(0) | BIT(8)) << (cpuid))
+
+/* Registers relative to mpic->per_cpu */
+#define MPIC_IN_DRBEL_CAUSE 0x08
+#define MPIC_IN_DRBEL_MASK 0x0c
+#define MPIC_PPI_CAUSE 0x10
+#define MPIC_CPU_INTACK 0x44
+#define MPIC_CPU_INTACK_IID_MASK GENMASK(9, 0)
+#define MPIC_INT_SET_MASK 0x48
+#define MPIC_INT_CLEAR_MASK 0x4C
+#define MPIC_INT_FABRIC_MASK 0x54
+#define MPIC_INT_CAUSE_PERF(cpu) BIT(cpu)
+
+#define MPIC_PER_CPU_IRQS_NR 29
/* IPI and MSI interrupt definitions for IPI platforms */
-#define IPI_DOORBELL_START (0)
-#define IPI_DOORBELL_END (8)
-#define IPI_DOORBELL_MASK 0xFF
-#define PCI_MSI_DOORBELL_START (16)
-#define PCI_MSI_DOORBELL_NR (16)
-#define PCI_MSI_DOORBELL_END (32)
-#define PCI_MSI_DOORBELL_MASK 0xFFFF0000
+#define IPI_DOORBELL_NR 8
+#define IPI_DOORBELL_MASK GENMASK(7, 0)
+#define PCI_MSI_DOORBELL_START 16
+#define PCI_MSI_DOORBELL_NR 16
+#define PCI_MSI_DOORBELL_MASK GENMASK(31, 16)
/* MSI interrupt definitions for non-IPI platforms */
#define PCI_MSI_FULL_DOORBELL_START 0
#define PCI_MSI_FULL_DOORBELL_NR 32
-#define PCI_MSI_FULL_DOORBELL_END 32
#define PCI_MSI_FULL_DOORBELL_MASK GENMASK(31, 0)
#define PCI_MSI_FULL_DOORBELL_SRC0_MASK GENMASK(15, 0)
#define PCI_MSI_FULL_DOORBELL_SRC1_MASK GENMASK(31, 16)
-static void __iomem *per_cpu_int_base;
-static void __iomem *main_int_base;
-static struct irq_domain *armada_370_xp_mpic_domain;
-static u32 doorbell_mask_reg;
-static int parent_irq;
+/**
+ * struct mpic - MPIC private data structure
+ * @base: MPIC registers base address
+ * @per_cpu: per-CPU registers base address
+ * @parent_irq: parent IRQ if MPIC is not top-level interrupt controller
+ * @domain: MPIC main interrupt domain
+ * @ipi_domain: IPI domain
+ * @msi_domain: MSI domain
+ * @msi_inner_domain: MSI inner domain
+ * @msi_used: bitmap of used MSI numbers
+ * @msi_lock: mutex serializing access to @msi_used
+ * @msi_doorbell_addr: physical address of MSI doorbell register
+ * @msi_doorbell_mask: mask of available doorbell bits for MSIs (either PCI_MSI_DOORBELL_MASK or
+ * PCI_MSI_FULL_DOORBELL_MASK)
+ * @msi_doorbell_start: first set bit in @msi_doorbell_mask
+ * @msi_doorbell_size: number of set bits in @msi_doorbell_mask
+ * @doorbell_mask: doorbell mask of MSIs and IPIs, stored on suspend, restored on resume
+ */
+struct mpic {
+ void __iomem *base;
+ void __iomem *per_cpu;
+ int parent_irq;
+ struct irq_domain *domain;
+#ifdef CONFIG_SMP
+ struct irq_domain *ipi_domain;
+#endif
#ifdef CONFIG_PCI_MSI
-static struct irq_domain *armada_370_xp_msi_domain;
-static struct irq_domain *armada_370_xp_msi_inner_domain;
-static DECLARE_BITMAP(msi_used, PCI_MSI_FULL_DOORBELL_NR);
-static DEFINE_MUTEX(msi_used_lock);
-static phys_addr_t msi_doorbell_addr;
+ struct irq_domain *msi_domain;
+ struct irq_domain *msi_inner_domain;
+ DECLARE_BITMAP(msi_used, PCI_MSI_FULL_DOORBELL_NR);
+ struct mutex msi_lock;
+ phys_addr_t msi_doorbell_addr;
+ u32 msi_doorbell_mask;
+ unsigned int msi_doorbell_start, msi_doorbell_size;
#endif
+ u32 doorbell_mask;
+};
+
+static struct mpic *mpic_data __ro_after_init;
-static inline bool is_ipi_available(void)
+static inline bool mpic_is_ipi_available(struct mpic *mpic)
{
/*
* We distinguish IPI availability in the IC by the IC not having a
@@ -175,39 +197,12 @@ static inline bool is_ipi_available(void)
* interrupt controller (e.g. GIC) that takes care of inter-processor
* interrupts.
*/
- return parent_irq <= 0;
-}
-
-static inline u32 msi_doorbell_mask(void)
-{
- return is_ipi_available() ? PCI_MSI_DOORBELL_MASK :
- PCI_MSI_FULL_DOORBELL_MASK;
-}
-
-static inline unsigned int msi_doorbell_start(void)
-{
- return is_ipi_available() ? PCI_MSI_DOORBELL_START :
- PCI_MSI_FULL_DOORBELL_START;
+ return mpic->parent_irq <= 0;
}
-static inline unsigned int msi_doorbell_size(void)
+static inline bool mpic_is_percpu_irq(irq_hw_number_t hwirq)
{
- return is_ipi_available() ? PCI_MSI_DOORBELL_NR :
- PCI_MSI_FULL_DOORBELL_NR;
-}
-
-static inline unsigned int msi_doorbell_end(void)
-{
- return is_ipi_available() ? PCI_MSI_DOORBELL_END :
- PCI_MSI_FULL_DOORBELL_END;
-}
-
-static inline bool is_percpu_irq(irq_hw_number_t irq)
-{
- if (irq <= ARMADA_370_XP_MAX_PER_CPU_IRQS)
- return true;
-
- return false;
+ return hwirq < MPIC_PER_CPU_IRQS_NR;
}
/*
@@ -215,55 +210,53 @@ static inline bool is_percpu_irq(irq_hw_number_t irq)
* For shared global interrupts, mask/unmask global enable bit
* For CPU interrupts, mask/unmask the calling CPU's bit
*/
-static void armada_370_xp_irq_mask(struct irq_data *d)
+static void mpic_irq_mask(struct irq_data *d)
{
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
irq_hw_number_t hwirq = irqd_to_hwirq(d);
- if (!is_percpu_irq(hwirq))
- writel(hwirq, main_int_base +
- ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS);
+ if (!mpic_is_percpu_irq(hwirq))
+ writel(hwirq, mpic->base + MPIC_INT_CLEAR_ENABLE);
else
- writel(hwirq, per_cpu_int_base +
- ARMADA_370_XP_INT_SET_MASK_OFFS);
+ writel(hwirq, mpic->per_cpu + MPIC_INT_SET_MASK);
}
-static void armada_370_xp_irq_unmask(struct irq_data *d)
+static void mpic_irq_unmask(struct irq_data *d)
{
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
irq_hw_number_t hwirq = irqd_to_hwirq(d);
- if (!is_percpu_irq(hwirq))
- writel(hwirq, main_int_base +
- ARMADA_370_XP_INT_SET_ENABLE_OFFS);
+ if (!mpic_is_percpu_irq(hwirq))
+ writel(hwirq, mpic->base + MPIC_INT_SET_ENABLE);
else
- writel(hwirq, per_cpu_int_base +
- ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ writel(hwirq, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
}
#ifdef CONFIG_PCI_MSI
-static struct irq_chip armada_370_xp_msi_irq_chip = {
- .name = "MPIC MSI",
- .irq_mask = pci_msi_mask_irq,
- .irq_unmask = pci_msi_unmask_irq,
+static struct irq_chip mpic_msi_irq_chip = {
+ .name = "MPIC MSI",
+ .irq_mask = pci_msi_mask_irq,
+ .irq_unmask = pci_msi_unmask_irq,
};
-static struct msi_domain_info armada_370_xp_msi_domain_info = {
+static struct msi_domain_info mpic_msi_domain_info = {
.flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX),
- .chip = &armada_370_xp_msi_irq_chip,
+ .chip = &mpic_msi_irq_chip,
};
-static void armada_370_xp_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
+static void mpic_compose_msi_msg(struct irq_data *d, struct msi_msg *msg)
{
- unsigned int cpu = cpumask_first(irq_data_get_effective_affinity_mask(data));
+ unsigned int cpu = cpumask_first(irq_data_get_effective_affinity_mask(d));
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
- msg->address_lo = lower_32_bits(msi_doorbell_addr);
- msg->address_hi = upper_32_bits(msi_doorbell_addr);
- msg->data = BIT(cpu + 8) | (data->hwirq + msi_doorbell_start());
+ msg->address_lo = lower_32_bits(mpic->msi_doorbell_addr);
+ msg->address_hi = upper_32_bits(mpic->msi_doorbell_addr);
+ msg->data = BIT(cpu + 8) | (d->hwirq + mpic->msi_doorbell_start);
}
-static int armada_370_xp_msi_set_affinity(struct irq_data *irq_data,
- const struct cpumask *mask, bool force)
+static int mpic_msi_set_affinity(struct irq_data *d, const struct cpumask *mask, bool force)
{
unsigned int cpu;
@@ -275,33 +268,34 @@ static int armada_370_xp_msi_set_affinity(struct irq_data *irq_data,
if (cpu >= nr_cpu_ids)
return -EINVAL;
- irq_data_update_effective_affinity(irq_data, cpumask_of(cpu));
+ irq_data_update_effective_affinity(d, cpumask_of(cpu));
return IRQ_SET_MASK_OK;
}
-static struct irq_chip armada_370_xp_msi_bottom_irq_chip = {
+static struct irq_chip mpic_msi_bottom_irq_chip = {
.name = "MPIC MSI",
- .irq_compose_msi_msg = armada_370_xp_compose_msi_msg,
- .irq_set_affinity = armada_370_xp_msi_set_affinity,
+ .irq_compose_msi_msg = mpic_compose_msi_msg,
+ .irq_set_affinity = mpic_msi_set_affinity,
};
-static int armada_370_xp_msi_alloc(struct irq_domain *domain, unsigned int virq,
- unsigned int nr_irqs, void *args)
+static int mpic_msi_alloc(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs,
+ void *args)
{
- int hwirq, i;
+ struct mpic *mpic = domain->host_data;
+ int hwirq;
- mutex_lock(&msi_used_lock);
- hwirq = bitmap_find_free_region(msi_used, msi_doorbell_size(),
+ mutex_lock(&mpic->msi_lock);
+ hwirq = bitmap_find_free_region(mpic->msi_used, mpic->msi_doorbell_size,
order_base_2(nr_irqs));
- mutex_unlock(&msi_used_lock);
+ mutex_unlock(&mpic->msi_lock);
if (hwirq < 0)
return -ENOSPC;
- for (i = 0; i < nr_irqs; i++) {
+ for (unsigned int i = 0; i < nr_irqs; i++) {
irq_domain_set_info(domain, virq + i, hwirq + i,
- &armada_370_xp_msi_bottom_irq_chip,
+ &mpic_msi_bottom_irq_chip,
domain->host_data, handle_simple_irq,
NULL, NULL);
}
@@ -309,76 +303,84 @@ static int armada_370_xp_msi_alloc(struct irq_domain *domain, unsigned int virq,
return 0;
}
-static void armada_370_xp_msi_free(struct irq_domain *domain,
- unsigned int virq, unsigned int nr_irqs)
+static void mpic_msi_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs)
{
struct irq_data *d = irq_domain_get_irq_data(domain, virq);
+ struct mpic *mpic = domain->host_data;
- mutex_lock(&msi_used_lock);
- bitmap_release_region(msi_used, d->hwirq, order_base_2(nr_irqs));
- mutex_unlock(&msi_used_lock);
+ mutex_lock(&mpic->msi_lock);
+ bitmap_release_region(mpic->msi_used, d->hwirq, order_base_2(nr_irqs));
+ mutex_unlock(&mpic->msi_lock);
}
-static const struct irq_domain_ops armada_370_xp_msi_domain_ops = {
- .alloc = armada_370_xp_msi_alloc,
- .free = armada_370_xp_msi_free,
+static const struct irq_domain_ops mpic_msi_domain_ops = {
+ .alloc = mpic_msi_alloc,
+ .free = mpic_msi_free,
};
-static void armada_370_xp_msi_reenable_percpu(void)
+static void mpic_msi_reenable_percpu(struct mpic *mpic)
{
u32 reg;
/* Enable MSI doorbell mask and combined cpu local interrupt */
- reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
- reg |= msi_doorbell_mask();
- writel(reg, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ reg = readl(mpic->per_cpu + MPIC_IN_DRBEL_MASK);
+ reg |= mpic->msi_doorbell_mask;
+ writel(reg, mpic->per_cpu + MPIC_IN_DRBEL_MASK);
/* Unmask local doorbell interrupt */
- writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ writel(1, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
}
-static int armada_370_xp_msi_init(struct device_node *node,
- phys_addr_t main_int_phys_base)
+static int __init mpic_msi_init(struct mpic *mpic, struct device_node *node,
+ phys_addr_t main_int_phys_base)
{
- msi_doorbell_addr = main_int_phys_base +
- ARMADA_370_XP_SW_TRIG_INT_OFFS;
+ mpic->msi_doorbell_addr = main_int_phys_base + MPIC_SW_TRIG_INT;
+
+ mutex_init(&mpic->msi_lock);
+
+ if (mpic_is_ipi_available(mpic)) {
+ mpic->msi_doorbell_start = PCI_MSI_DOORBELL_START;
+ mpic->msi_doorbell_size = PCI_MSI_DOORBELL_NR;
+ mpic->msi_doorbell_mask = PCI_MSI_DOORBELL_MASK;
+ } else {
+ mpic->msi_doorbell_start = PCI_MSI_FULL_DOORBELL_START;
+ mpic->msi_doorbell_size = PCI_MSI_FULL_DOORBELL_NR;
+ mpic->msi_doorbell_mask = PCI_MSI_FULL_DOORBELL_MASK;
+ }
- armada_370_xp_msi_inner_domain =
- irq_domain_add_linear(NULL, msi_doorbell_size(),
- &armada_370_xp_msi_domain_ops, NULL);
- if (!armada_370_xp_msi_inner_domain)
+ mpic->msi_inner_domain = irq_domain_add_linear(NULL, mpic->msi_doorbell_size,
+ &mpic_msi_domain_ops, mpic);
+ if (!mpic->msi_inner_domain)
return -ENOMEM;
- armada_370_xp_msi_domain =
- pci_msi_create_irq_domain(of_node_to_fwnode(node),
- &armada_370_xp_msi_domain_info,
- armada_370_xp_msi_inner_domain);
- if (!armada_370_xp_msi_domain) {
- irq_domain_remove(armada_370_xp_msi_inner_domain);
+ mpic->msi_domain = pci_msi_create_irq_domain(of_node_to_fwnode(node), &mpic_msi_domain_info,
+ mpic->msi_inner_domain);
+ if (!mpic->msi_domain) {
+ irq_domain_remove(mpic->msi_inner_domain);
return -ENOMEM;
}
- armada_370_xp_msi_reenable_percpu();
+ mpic_msi_reenable_percpu(mpic);
/* Unmask low 16 MSI irqs on non-IPI platforms */
- if (!is_ipi_available())
- writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ if (!mpic_is_ipi_available(mpic))
+ writel(0, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
return 0;
}
#else
-static __maybe_unused void armada_370_xp_msi_reenable_percpu(void) {}
+static __maybe_unused void mpic_msi_reenable_percpu(struct mpic *mpic) {}
-static inline int armada_370_xp_msi_init(struct device_node *node,
- phys_addr_t main_int_phys_base)
+static inline int mpic_msi_init(struct mpic *mpic, struct device_node *node,
+ phys_addr_t main_int_phys_base)
{
return 0;
}
#endif
-static void armada_xp_mpic_perf_init(void)
+static void mpic_perf_init(struct mpic *mpic)
{
- unsigned long cpuid;
+ u32 cpuid;
/*
* This Performance Counter Overflow interrupt is specific for
@@ -390,38 +392,39 @@ static void armada_xp_mpic_perf_init(void)
cpuid = cpu_logical_map(smp_processor_id());
/* Enable Performance Counter Overflow interrupts */
- writel(ARMADA_370_XP_INT_CAUSE_PERF(cpuid),
- per_cpu_int_base + ARMADA_370_XP_INT_FABRIC_MASK_OFFS);
+ writel(MPIC_INT_CAUSE_PERF(cpuid), mpic->per_cpu + MPIC_INT_FABRIC_MASK);
}
#ifdef CONFIG_SMP
-static struct irq_domain *ipi_domain;
-
-static void armada_370_xp_ipi_mask(struct irq_data *d)
+static void mpic_ipi_mask(struct irq_data *d)
{
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
u32 reg;
- reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+
+ reg = readl(mpic->per_cpu + MPIC_IN_DRBEL_MASK);
reg &= ~BIT(d->hwirq);
- writel(reg, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ writel(reg, mpic->per_cpu + MPIC_IN_DRBEL_MASK);
}
-static void armada_370_xp_ipi_unmask(struct irq_data *d)
+static void mpic_ipi_unmask(struct irq_data *d)
{
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
u32 reg;
- reg = readl(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+
+ reg = readl(mpic->per_cpu + MPIC_IN_DRBEL_MASK);
reg |= BIT(d->hwirq);
- writel(reg, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ writel(reg, mpic->per_cpu + MPIC_IN_DRBEL_MASK);
}
-static void armada_370_xp_ipi_send_mask(struct irq_data *d,
- const struct cpumask *mask)
+static void mpic_ipi_send_mask(struct irq_data *d, const struct cpumask *mask)
{
- unsigned long map = 0;
- int cpu;
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
+ unsigned int cpu;
+ u32 map = 0;
/* Convert our logical CPU mask into a physical one. */
for_each_cpu(cpu, mask)
- map |= 1 << cpu_logical_map(cpu);
+ map |= BIT(cpu_logical_map(cpu));
/*
* Ensure that stores to Normal memory are visible to the
@@ -430,451 +433,465 @@ static void armada_370_xp_ipi_send_mask(struct irq_data *d,
dsb();
/* submit softirq */
- writel((map << 8) | d->hwirq, main_int_base +
- ARMADA_370_XP_SW_TRIG_INT_OFFS);
+ writel((map << 8) | d->hwirq, mpic->base + MPIC_SW_TRIG_INT);
}
-static void armada_370_xp_ipi_ack(struct irq_data *d)
+static void mpic_ipi_ack(struct irq_data *d)
{
- writel(~BIT(d->hwirq), per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
+
+ writel(~BIT(d->hwirq), mpic->per_cpu + MPIC_IN_DRBEL_CAUSE);
}
-static struct irq_chip ipi_irqchip = {
+static struct irq_chip mpic_ipi_irqchip = {
.name = "IPI",
- .irq_ack = armada_370_xp_ipi_ack,
- .irq_mask = armada_370_xp_ipi_mask,
- .irq_unmask = armada_370_xp_ipi_unmask,
- .ipi_send_mask = armada_370_xp_ipi_send_mask,
+ .irq_ack = mpic_ipi_ack,
+ .irq_mask = mpic_ipi_mask,
+ .irq_unmask = mpic_ipi_unmask,
+ .ipi_send_mask = mpic_ipi_send_mask,
};
-static int armada_370_xp_ipi_alloc(struct irq_domain *d,
- unsigned int virq,
- unsigned int nr_irqs, void *args)
+static int mpic_ipi_alloc(struct irq_domain *d, unsigned int virq,
+ unsigned int nr_irqs, void *args)
{
- int i;
-
- for (i = 0; i < nr_irqs; i++) {
+ for (unsigned int i = 0; i < nr_irqs; i++) {
irq_set_percpu_devid(virq + i);
- irq_domain_set_info(d, virq + i, i, &ipi_irqchip,
- d->host_data,
- handle_percpu_devid_irq,
- NULL, NULL);
+ irq_domain_set_info(d, virq + i, i, &mpic_ipi_irqchip, d->host_data,
+ handle_percpu_devid_irq, NULL, NULL);
}
return 0;
}
-static void armada_370_xp_ipi_free(struct irq_domain *d,
- unsigned int virq,
- unsigned int nr_irqs)
+static void mpic_ipi_free(struct irq_domain *d, unsigned int virq,
+ unsigned int nr_irqs)
{
/* Not freeing IPIs */
}
-static const struct irq_domain_ops ipi_domain_ops = {
- .alloc = armada_370_xp_ipi_alloc,
- .free = armada_370_xp_ipi_free,
+static const struct irq_domain_ops mpic_ipi_domain_ops = {
+ .alloc = mpic_ipi_alloc,
+ .free = mpic_ipi_free,
};
-static void ipi_resume(void)
+static void mpic_ipi_resume(struct mpic *mpic)
{
- int i;
-
- for (i = 0; i < IPI_DOORBELL_END; i++) {
- int irq;
+ for (irq_hw_number_t i = 0; i < IPI_DOORBELL_NR; i++) {
+ unsigned int virq = irq_find_mapping(mpic->ipi_domain, i);
+ struct irq_data *d;
- irq = irq_find_mapping(ipi_domain, i);
- if (irq <= 0)
+ if (!virq || !irq_percpu_is_enabled(virq))
continue;
- if (irq_percpu_is_enabled(irq)) {
- struct irq_data *d;
- d = irq_domain_get_irq_data(ipi_domain, irq);
- armada_370_xp_ipi_unmask(d);
- }
+
+ d = irq_domain_get_irq_data(mpic->ipi_domain, virq);
+ mpic_ipi_unmask(d);
}
}
-static __init void armada_xp_ipi_init(struct device_node *node)
+static int __init mpic_ipi_init(struct mpic *mpic, struct device_node *node)
{
int base_ipi;
- ipi_domain = irq_domain_create_linear(of_node_to_fwnode(node),
- IPI_DOORBELL_END,
- &ipi_domain_ops, NULL);
- if (WARN_ON(!ipi_domain))
- return;
+ mpic->ipi_domain = irq_domain_create_linear(of_node_to_fwnode(node), IPI_DOORBELL_NR,
+ &mpic_ipi_domain_ops, mpic);
+ if (WARN_ON(!mpic->ipi_domain))
+ return -ENOMEM;
- irq_domain_update_bus_token(ipi_domain, DOMAIN_BUS_IPI);
- base_ipi = irq_domain_alloc_irqs(ipi_domain, IPI_DOORBELL_END, NUMA_NO_NODE, NULL);
+ irq_domain_update_bus_token(mpic->ipi_domain, DOMAIN_BUS_IPI);
+ base_ipi = irq_domain_alloc_irqs(mpic->ipi_domain, IPI_DOORBELL_NR, NUMA_NO_NODE, NULL);
if (WARN_ON(!base_ipi))
- return;
+ return -ENOMEM;
+
+ set_smp_ipi_range(base_ipi, IPI_DOORBELL_NR);
- set_smp_ipi_range(base_ipi, IPI_DOORBELL_END);
+ return 0;
}
-static int armada_xp_set_affinity(struct irq_data *d,
- const struct cpumask *mask_val, bool force)
+static int mpic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, bool force)
{
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
irq_hw_number_t hwirq = irqd_to_hwirq(d);
- int cpu;
+ unsigned int cpu;
/* Select a single core from the affinity mask which is online */
cpu = cpumask_any_and(mask_val, cpu_online_mask);
- atomic_io_modify(main_int_base + ARMADA_370_XP_INT_SOURCE_CTL(hwirq),
- ARMADA_370_XP_INT_SOURCE_CPU_MASK,
- BIT(cpu_logical_map(cpu)));
+ atomic_io_modify(mpic->base + MPIC_INT_SOURCE_CTL(hwirq),
+ MPIC_INT_SOURCE_CPU_MASK, BIT(cpu_logical_map(cpu)));
irq_data_update_effective_affinity(d, cpumask_of(cpu));
return IRQ_SET_MASK_OK;
}
-static void armada_xp_mpic_smp_cpu_init(void)
+static void mpic_smp_cpu_init(struct mpic *mpic)
{
- u32 control;
- int nr_irqs, i;
-
- control = readl(main_int_base + ARMADA_370_XP_INT_CONTROL);
- nr_irqs = (control >> 2) & 0x3ff;
+ for (irq_hw_number_t i = 0; i < mpic->domain->hwirq_max; i++)
+ writel(i, mpic->per_cpu + MPIC_INT_SET_MASK);
- for (i = 0; i < nr_irqs; i++)
- writel(i, per_cpu_int_base + ARMADA_370_XP_INT_SET_MASK_OFFS);
-
- if (!is_ipi_available())
+ if (!mpic_is_ipi_available(mpic))
return;
/* Disable all IPIs */
- writel(0, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ writel(0, mpic->per_cpu + MPIC_IN_DRBEL_MASK);
/* Clear pending IPIs */
- writel(0, per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+ writel(0, mpic->per_cpu + MPIC_IN_DRBEL_CAUSE);
/* Unmask IPI interrupt */
- writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ writel(0, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
}
-static void armada_xp_mpic_reenable_percpu(void)
+static void mpic_reenable_percpu(struct mpic *mpic)
{
- unsigned int irq;
-
/* Re-enable per-CPU interrupts that were enabled before suspend */
- for (irq = 0; irq < ARMADA_370_XP_MAX_PER_CPU_IRQS; irq++) {
- struct irq_data *data;
- int virq;
-
- virq = irq_linear_revmap(armada_370_xp_mpic_domain, irq);
- if (virq == 0)
- continue;
-
- data = irq_get_irq_data(virq);
+ for (irq_hw_number_t i = 0; i < MPIC_PER_CPU_IRQS_NR; i++) {
+ unsigned int virq = irq_linear_revmap(mpic->domain, i);
+ struct irq_data *d;
- if (!irq_percpu_is_enabled(virq))
+ if (!virq || !irq_percpu_is_enabled(virq))
continue;
- armada_370_xp_irq_unmask(data);
+ d = irq_get_irq_data(virq);
+ mpic_irq_unmask(d);
}
- if (is_ipi_available())
- ipi_resume();
+ if (mpic_is_ipi_available(mpic))
+ mpic_ipi_resume(mpic);
- armada_370_xp_msi_reenable_percpu();
+ mpic_msi_reenable_percpu(mpic);
}
-static int armada_xp_mpic_starting_cpu(unsigned int cpu)
+static int mpic_starting_cpu(unsigned int cpu)
{
- armada_xp_mpic_perf_init();
- armada_xp_mpic_smp_cpu_init();
- armada_xp_mpic_reenable_percpu();
+ struct mpic *mpic = irq_get_default_host()->host_data;
+
+ mpic_perf_init(mpic);
+ mpic_smp_cpu_init(mpic);
+ mpic_reenable_percpu(mpic);
+
return 0;
}
static int mpic_cascaded_starting_cpu(unsigned int cpu)
{
- armada_xp_mpic_perf_init();
- armada_xp_mpic_reenable_percpu();
- enable_percpu_irq(parent_irq, IRQ_TYPE_NONE);
+ struct mpic *mpic = mpic_data;
+
+ mpic_perf_init(mpic);
+ mpic_reenable_percpu(mpic);
+ enable_percpu_irq(mpic->parent_irq, IRQ_TYPE_NONE);
+
return 0;
}
#else
-static void armada_xp_mpic_smp_cpu_init(void) {}
-static void ipi_resume(void) {}
+static void mpic_smp_cpu_init(struct mpic *mpic) {}
+static void mpic_ipi_resume(struct mpic *mpic) {}
#endif
-static struct irq_chip armada_370_xp_irq_chip = {
+static struct irq_chip mpic_irq_chip = {
.name = "MPIC",
- .irq_mask = armada_370_xp_irq_mask,
- .irq_mask_ack = armada_370_xp_irq_mask,
- .irq_unmask = armada_370_xp_irq_unmask,
+ .irq_mask = mpic_irq_mask,
+ .irq_mask_ack = mpic_irq_mask,
+ .irq_unmask = mpic_irq_unmask,
#ifdef CONFIG_SMP
- .irq_set_affinity = armada_xp_set_affinity,
+ .irq_set_affinity = mpic_set_affinity,
#endif
.flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND,
};
-static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
- unsigned int virq, irq_hw_number_t hw)
+static int mpic_irq_map(struct irq_domain *domain, unsigned int virq, irq_hw_number_t hwirq)
{
+ struct mpic *mpic = domain->host_data;
+
/* IRQs 0 and 1 cannot be mapped, they are handled internally */
- if (hw <= 1)
+ if (hwirq <= 1)
return -EINVAL;
- armada_370_xp_irq_mask(irq_get_irq_data(virq));
- if (!is_percpu_irq(hw))
- writel(hw, per_cpu_int_base +
- ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ irq_set_chip_data(virq, mpic);
+
+ mpic_irq_mask(irq_get_irq_data(virq));
+ if (!mpic_is_percpu_irq(hwirq))
+ writel(hwirq, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
else
- writel(hw, main_int_base + ARMADA_370_XP_INT_SET_ENABLE_OFFS);
+ writel(hwirq, mpic->base + MPIC_INT_SET_ENABLE);
irq_set_status_flags(virq, IRQ_LEVEL);
- if (is_percpu_irq(hw)) {
+ if (mpic_is_percpu_irq(hwirq)) {
irq_set_percpu_devid(virq);
- irq_set_chip_and_handler(virq, &armada_370_xp_irq_chip,
- handle_percpu_devid_irq);
+ irq_set_chip_and_handler(virq, &mpic_irq_chip, handle_percpu_devid_irq);
} else {
- irq_set_chip_and_handler(virq, &armada_370_xp_irq_chip,
- handle_level_irq);
+ irq_set_chip_and_handler(virq, &mpic_irq_chip, handle_level_irq);
irqd_set_single_target(irq_desc_get_irq_data(irq_to_desc(virq)));
}
irq_set_probe(virq);
-
return 0;
}
-static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = {
- .map = armada_370_xp_mpic_irq_map,
- .xlate = irq_domain_xlate_onecell,
+static const struct irq_domain_ops mpic_irq_ops = {
+ .map = mpic_irq_map,
+ .xlate = irq_domain_xlate_onecell,
};
#ifdef CONFIG_PCI_MSI
-static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained)
+static void mpic_handle_msi_irq(struct mpic *mpic)
{
- u32 msimask, msinr;
-
- msimask = readl_relaxed(per_cpu_int_base +
- ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
- msimask &= msi_doorbell_mask();
+ unsigned long cause;
+ unsigned int i;
- writel(~msimask, per_cpu_int_base +
- ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS);
+ cause = readl_relaxed(mpic->per_cpu + MPIC_IN_DRBEL_CAUSE);
+ cause &= mpic->msi_doorbell_mask;
+ writel(~cause, mpic->per_cpu + MPIC_IN_DRBEL_CAUSE);
- for (msinr = msi_doorbell_start();
- msinr < msi_doorbell_end(); msinr++) {
- unsigned int irq;
+ for_each_set_bit(i, &cause, BITS_PER_LONG)
+ generic_handle_domain_irq(mpic->msi_inner_domain, i - mpic->msi_doorbell_start);
+}
+#else
+static void mpic_handle_msi_irq(struct mpic *mpic) {}
+#endif
- if (!(msimask & BIT(msinr)))
- continue;
+#ifdef CONFIG_SMP
+static void mpic_handle_ipi_irq(struct mpic *mpic)
+{
+ unsigned long cause;
+ irq_hw_number_t i;
- irq = msinr - msi_doorbell_start();
+ cause = readl_relaxed(mpic->per_cpu + MPIC_IN_DRBEL_CAUSE);
+ cause &= IPI_DOORBELL_MASK;
- generic_handle_domain_irq(armada_370_xp_msi_inner_domain, irq);
- }
+ for_each_set_bit(i, &cause, IPI_DOORBELL_NR)
+ generic_handle_domain_irq(mpic->ipi_domain, i);
}
#else
-static void armada_370_xp_handle_msi_irq(struct pt_regs *r, bool b) {}
+static inline void mpic_handle_ipi_irq(struct mpic *mpic) {}
#endif
-static void armada_370_xp_mpic_handle_cascade_irq(struct irq_desc *desc)
+static void mpic_handle_cascade_irq(struct irq_desc *desc)
{
+ struct mpic *mpic = irq_desc_get_handler_data(desc);
struct irq_chip *chip = irq_desc_get_chip(desc);
- unsigned long irqmap, irqn, irqsrc, cpuid;
+ unsigned long cause;
+ u32 irqsrc, cpuid;
+ irq_hw_number_t i;
chained_irq_enter(chip, desc);
- irqmap = readl_relaxed(per_cpu_int_base + ARMADA_375_PPI_CAUSE);
+ cause = readl_relaxed(mpic->per_cpu + MPIC_PPI_CAUSE);
cpuid = cpu_logical_map(smp_processor_id());
- for_each_set_bit(irqn, &irqmap, BITS_PER_LONG) {
- irqsrc = readl_relaxed(main_int_base +
- ARMADA_370_XP_INT_SOURCE_CTL(irqn));
+ for_each_set_bit(i, &cause, MPIC_PER_CPU_IRQS_NR) {
+ irqsrc = readl_relaxed(mpic->base + MPIC_INT_SOURCE_CTL(i));
/* Check if the interrupt is not masked on current CPU.
* Test IRQ (0-1) and FIQ (8-9) mask bits.
*/
- if (!(irqsrc & ARMADA_370_XP_INT_IRQ_FIQ_MASK(cpuid)))
+ if (!(irqsrc & MPIC_INT_IRQ_FIQ_MASK(cpuid)))
continue;
- if (irqn == 0 || irqn == 1) {
- armada_370_xp_handle_msi_irq(NULL, true);
+ if (i == 0 || i == 1) {
+ mpic_handle_msi_irq(mpic);
continue;
}
- generic_handle_domain_irq(armada_370_xp_mpic_domain, irqn);
+ generic_handle_domain_irq(mpic->domain, i);
}
chained_irq_exit(chip, desc);
}
-static void __exception_irq_entry
-armada_370_xp_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry mpic_handle_irq(struct pt_regs *regs)
{
- u32 irqstat, irqnr;
+ struct mpic *mpic = irq_get_default_host()->host_data;
+ irq_hw_number_t i;
+ u32 irqstat;
do {
- irqstat = readl_relaxed(per_cpu_int_base +
- ARMADA_370_XP_CPU_INTACK_OFFS);
- irqnr = irqstat & 0x3FF;
+ irqstat = readl_relaxed(mpic->per_cpu + MPIC_CPU_INTACK);
+ i = FIELD_GET(MPIC_CPU_INTACK_IID_MASK, irqstat);
- if (irqnr > 1022)
+ if (i > 1022)
break;
- if (irqnr > 1) {
- generic_handle_domain_irq(armada_370_xp_mpic_domain,
- irqnr);
- continue;
- }
+ if (i > 1)
+ generic_handle_domain_irq(mpic->domain, i);
/* MSI handling */
- if (irqnr == 1)
- armada_370_xp_handle_msi_irq(regs, false);
+ if (i == 1)
+ mpic_handle_msi_irq(mpic);
-#ifdef CONFIG_SMP
/* IPI Handling */
- if (irqnr == 0) {
- unsigned long ipimask;
- int ipi;
-
- ipimask = readl_relaxed(per_cpu_int_base +
- ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS)
- & IPI_DOORBELL_MASK;
-
- for_each_set_bit(ipi, &ipimask, IPI_DOORBELL_END)
- generic_handle_domain_irq(ipi_domain, ipi);
- }
-#endif
-
+ if (i == 0)
+ mpic_handle_ipi_irq(mpic);
} while (1);
}
-static int armada_370_xp_mpic_suspend(void)
+static int mpic_suspend(void)
{
- doorbell_mask_reg = readl(per_cpu_int_base +
- ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ struct mpic *mpic = mpic_data;
+
+ mpic->doorbell_mask = readl(mpic->per_cpu + MPIC_IN_DRBEL_MASK);
+
return 0;
}
-static void armada_370_xp_mpic_resume(void)
+static void mpic_resume(void)
{
+ struct mpic *mpic = mpic_data;
bool src0, src1;
- int nirqs;
- irq_hw_number_t irq;
/* Re-enable interrupts */
- nirqs = (readl(main_int_base + ARMADA_370_XP_INT_CONTROL) >> 2) & 0x3ff;
- for (irq = 0; irq < nirqs; irq++) {
- struct irq_data *data;
- int virq;
+ for (irq_hw_number_t i = 0; i < mpic->domain->hwirq_max; i++) {
+ unsigned int virq = irq_linear_revmap(mpic->domain, i);
+ struct irq_data *d;
- virq = irq_linear_revmap(armada_370_xp_mpic_domain, irq);
- if (virq == 0)
+ if (!virq)
continue;
- data = irq_get_irq_data(virq);
+ d = irq_get_irq_data(virq);
- if (!is_percpu_irq(irq)) {
+ if (!mpic_is_percpu_irq(i)) {
/* Non per-CPU interrupts */
- writel(irq, per_cpu_int_base +
- ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
- if (!irqd_irq_disabled(data))
- armada_370_xp_irq_unmask(data);
+ writel(i, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
+ if (!irqd_irq_disabled(d))
+ mpic_irq_unmask(d);
} else {
/* Per-CPU interrupts */
- writel(irq, main_int_base +
- ARMADA_370_XP_INT_SET_ENABLE_OFFS);
+ writel(i, mpic->base + MPIC_INT_SET_ENABLE);
/*
- * Re-enable on the current CPU,
- * armada_xp_mpic_reenable_percpu() will take
- * care of secondary CPUs when they come up.
+ * Re-enable on the current CPU, mpic_reenable_percpu()
+ * will take care of secondary CPUs when they come up.
*/
if (irq_percpu_is_enabled(virq))
- armada_370_xp_irq_unmask(data);
+ mpic_irq_unmask(d);
}
}
/* Reconfigure doorbells for IPIs and MSIs */
- writel(doorbell_mask_reg,
- per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS);
+ writel(mpic->doorbell_mask, mpic->per_cpu + MPIC_IN_DRBEL_MASK);
- if (is_ipi_available()) {
- src0 = doorbell_mask_reg & IPI_DOORBELL_MASK;
- src1 = doorbell_mask_reg & PCI_MSI_DOORBELL_MASK;
+ if (mpic_is_ipi_available(mpic)) {
+ src0 = mpic->doorbell_mask & IPI_DOORBELL_MASK;
+ src1 = mpic->doorbell_mask & PCI_MSI_DOORBELL_MASK;
} else {
- src0 = doorbell_mask_reg & PCI_MSI_FULL_DOORBELL_SRC0_MASK;
- src1 = doorbell_mask_reg & PCI_MSI_FULL_DOORBELL_SRC1_MASK;
+ src0 = mpic->doorbell_mask & PCI_MSI_FULL_DOORBELL_SRC0_MASK;
+ src1 = mpic->doorbell_mask & PCI_MSI_FULL_DOORBELL_SRC1_MASK;
}
if (src0)
- writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ writel(0, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
if (src1)
- writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
+ writel(1, mpic->per_cpu + MPIC_INT_CLEAR_MASK);
- if (is_ipi_available())
- ipi_resume();
+ if (mpic_is_ipi_available(mpic))
+ mpic_ipi_resume(mpic);
}
-static struct syscore_ops armada_370_xp_mpic_syscore_ops = {
- .suspend = armada_370_xp_mpic_suspend,
- .resume = armada_370_xp_mpic_resume,
+static struct syscore_ops mpic_syscore_ops = {
+ .suspend = mpic_suspend,
+ .resume = mpic_resume,
};
-static int __init armada_370_xp_mpic_of_init(struct device_node *node,
- struct device_node *parent)
+static int __init mpic_map_region(struct device_node *np, int index,
+ void __iomem **base, phys_addr_t *phys_base)
{
- struct resource main_int_res, per_cpu_int_res;
- int nr_irqs, i;
- u32 control;
+ struct resource res;
+ int err;
+
+ err = of_address_to_resource(np, index, &res);
+ if (WARN_ON(err))
+ goto fail;
+
+ if (WARN_ON(!request_mem_region(res.start, resource_size(&res), np->full_name))) {
+ err = -EBUSY;
+ goto fail;
+ }
+
+ *base = ioremap(res.start, resource_size(&res));
+ if (WARN_ON(!*base)) {
+ err = -ENOMEM;
+ goto fail;
+ }
- BUG_ON(of_address_to_resource(node, 0, &main_int_res));
- BUG_ON(of_address_to_resource(node, 1, &per_cpu_int_res));
+ if (phys_base)
+ *phys_base = res.start;
- BUG_ON(!request_mem_region(main_int_res.start,
- resource_size(&main_int_res),
- node->full_name));
- BUG_ON(!request_mem_region(per_cpu_int_res.start,
- resource_size(&per_cpu_int_res),
- node->full_name));
+ return 0;
+
+fail:
+ pr_err("%pOF: Unable to map resource %d: %pE\n", np, index, ERR_PTR(err));
+ return err;
+}
- main_int_base = ioremap(main_int_res.start,
- resource_size(&main_int_res));
- BUG_ON(!main_int_base);
+static int __init mpic_of_init(struct device_node *node, struct device_node *parent)
+{
+ phys_addr_t phys_base;
+ unsigned int nr_irqs;
+ struct mpic *mpic;
+ int err;
+
+ mpic = kzalloc(sizeof(*mpic), GFP_KERNEL);
+ if (WARN_ON(!mpic))
+ return -ENOMEM;
- per_cpu_int_base = ioremap(per_cpu_int_res.start,
- resource_size(&per_cpu_int_res));
- BUG_ON(!per_cpu_int_base);
+ mpic_data = mpic;
- control = readl(main_int_base + ARMADA_370_XP_INT_CONTROL);
- nr_irqs = (control >> 2) & 0x3ff;
+ err = mpic_map_region(node, 0, &mpic->base, &phys_base);
+ if (err)
+ return err;
- for (i = 0; i < nr_irqs; i++)
- writel(i, main_int_base + ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS);
+ err = mpic_map_region(node, 1, &mpic->per_cpu, NULL);
+ if (err)
+ return err;
- armada_370_xp_mpic_domain =
- irq_domain_add_linear(node, nr_irqs,
- &armada_370_xp_mpic_irq_ops, NULL);
- BUG_ON(!armada_370_xp_mpic_domain);
- irq_domain_update_bus_token(armada_370_xp_mpic_domain, DOMAIN_BUS_WIRED);
+ nr_irqs = FIELD_GET(MPIC_INT_CONTROL_NUMINT_MASK, readl(mpic->base + MPIC_INT_CONTROL));
+
+ for (irq_hw_number_t i = 0; i < nr_irqs; i++)
+ writel(i, mpic->base + MPIC_INT_CLEAR_ENABLE);
+
+ /*
+ * Initialize mpic->parent_irq before calling any other functions, since
+ * it is used to distinguish between IPI and non-IPI platforms.
+ */
+ mpic->parent_irq = irq_of_parse_and_map(node, 0);
/*
- * Initialize parent_irq before calling any other functions, since it is
- * used to distinguish between IPI and non-IPI platforms.
+ * On non-IPI platforms the driver currently supports only the per-CPU
+ * interrupts (the first 29 interrupts). See mpic_handle_cascade_irq().
*/
- parent_irq = irq_of_parse_and_map(node, 0);
+ if (!mpic_is_ipi_available(mpic))
+ nr_irqs = MPIC_PER_CPU_IRQS_NR;
+
+ mpic->domain = irq_domain_add_linear(node, nr_irqs, &mpic_irq_ops, mpic);
+ if (!mpic->domain) {
+ pr_err("%pOF: Unable to add IRQ domain\n", node);
+ return -ENOMEM;
+ }
+
+ irq_domain_update_bus_token(mpic->domain, DOMAIN_BUS_WIRED);
/* Setup for the boot CPU */
- armada_xp_mpic_perf_init();
- armada_xp_mpic_smp_cpu_init();
+ mpic_perf_init(mpic);
+ mpic_smp_cpu_init(mpic);
- armada_370_xp_msi_init(node, main_int_res.start);
+ err = mpic_msi_init(mpic, node, phys_base);
+ if (err) {
+ pr_err("%pOF: Unable to initialize MSI domain\n", node);
+ return err;
+ }
- if (parent_irq <= 0) {
- irq_set_default_host(armada_370_xp_mpic_domain);
- set_handle_irq(armada_370_xp_handle_irq);
+ if (mpic_is_ipi_available(mpic)) {
+ irq_set_default_host(mpic->domain);
+ set_handle_irq(mpic_handle_irq);
#ifdef CONFIG_SMP
- armada_xp_ipi_init(node);
+ err = mpic_ipi_init(mpic, node);
+ if (err) {
+ pr_err("%pOF: Unable to initialize IPI domain\n", node);
+ return err;
+ }
+
cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_ARMADA_XP_STARTING,
"irqchip/armada/ipi:starting",
- armada_xp_mpic_starting_cpu, NULL);
+ mpic_starting_cpu, NULL);
#endif
} else {
#ifdef CONFIG_SMP
@@ -882,13 +899,13 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node,
"irqchip/armada/cascade:starting",
mpic_cascaded_starting_cpu, NULL);
#endif
- irq_set_chained_handler(parent_irq,
- armada_370_xp_mpic_handle_cascade_irq);
+ irq_set_chained_handler_and_data(mpic->parent_irq,
+ mpic_handle_cascade_irq, mpic);
}
- register_syscore_ops(&armada_370_xp_mpic_syscore_ops);
+ register_syscore_ops(&mpic_syscore_ops);
return 0;
}
-IRQCHIP_DECLARE(armada_370_xp_mpic, "marvell,mpic", armada_370_xp_mpic_of_init);
+IRQCHIP_DECLARE(marvell_mpic, "marvell,mpic", mpic_of_init);
diff --git a/drivers/irqchip/irq-atmel-aic.c b/drivers/irqchip/irq-atmel-aic.c
index 4631f6847953..3839ad79ad31 100644
--- a/drivers/irqchip/irq-atmel-aic.c
+++ b/drivers/irqchip/irq-atmel-aic.c
@@ -57,8 +57,7 @@
static struct irq_domain *aic_domain;
-static asmlinkage void __exception_irq_entry
-aic_handle(struct pt_regs *regs)
+static void __exception_irq_entry aic_handle(struct pt_regs *regs)
{
struct irq_domain_chip_generic *dgc = aic_domain->gc;
struct irq_chip_generic *gc = dgc->gc[0];
diff --git a/drivers/irqchip/irq-atmel-aic5.c b/drivers/irqchip/irq-atmel-aic5.c
index 145535bd7560..c0f55dc7b050 100644
--- a/drivers/irqchip/irq-atmel-aic5.c
+++ b/drivers/irqchip/irq-atmel-aic5.c
@@ -67,8 +67,7 @@
static struct irq_domain *aic5_domain;
-static asmlinkage void __exception_irq_entry
-aic5_handle(struct pt_regs *regs)
+static void __exception_irq_entry aic5_handle(struct pt_regs *regs)
{
struct irq_chip_generic *bgc = irq_get_domain_generic_chip(aic5_domain, 0);
u32 irqnr;
diff --git a/drivers/irqchip/irq-clps711x.c b/drivers/irqchip/irq-clps711x.c
index e731e0784f7e..806ebb1de201 100644
--- a/drivers/irqchip/irq-clps711x.c
+++ b/drivers/irqchip/irq-clps711x.c
@@ -69,7 +69,7 @@ static struct {
struct irq_domain_ops ops;
} *clps711x_intc;
-static asmlinkage void __exception_irq_entry clps711x_irqh(struct pt_regs *regs)
+static void __exception_irq_entry clps711x_irqh(struct pt_regs *regs)
{
u32 irqstat;
diff --git a/drivers/irqchip/irq-davinci-cp-intc.c b/drivers/irqchip/irq-davinci-cp-intc.c
index 7482c8ed34b2..f4f8e9fadbbf 100644
--- a/drivers/irqchip/irq-davinci-cp-intc.c
+++ b/drivers/irqchip/irq-davinci-cp-intc.c
@@ -116,8 +116,7 @@ static struct irq_chip davinci_cp_intc_irq_chip = {
.flags = IRQCHIP_SKIP_SET_WAKE,
};
-static asmlinkage void __exception_irq_entry
-davinci_cp_intc_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry davinci_cp_intc_handle_irq(struct pt_regs *regs)
{
int gpir, irqnr, none;
diff --git a/drivers/irqchip/irq-ftintc010.c b/drivers/irqchip/irq-ftintc010.c
index 359efc1d1be7..b91c358ea6db 100644
--- a/drivers/irqchip/irq-ftintc010.c
+++ b/drivers/irqchip/irq-ftintc010.c
@@ -125,7 +125,7 @@ static struct irq_chip ft010_irq_chip = {
/* Local static for the IRQ entry call */
static struct ft010_irq_data firq;
-static asmlinkage void __exception_irq_entry ft010_irqchip_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry ft010_irqchip_handle_irq(struct pt_regs *regs)
{
struct ft010_irq_data *f = &firq;
int irq;
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index c19083bfb943..0efa3443c323 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -930,7 +930,7 @@ static void __gic_handle_irq_from_irqsoff(struct pt_regs *regs)
__gic_handle_nmi(irqnr, regs);
}
-static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
{
if (unlikely(gic_supports_nmi() && !interrupts_enabled(regs)))
__gic_handle_irq_from_irqsoff(regs);
diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
index ca32ac19d284..58c28895f8c4 100644
--- a/drivers/irqchip/irq-gic-v4.c
+++ b/drivers/irqchip/irq-gic-v4.c
@@ -97,7 +97,7 @@ bool gic_cpuif_has_vsgi(void)
fld = cpuid_feature_extract_unsigned_field(reg, ID_AA64PFR0_EL1_GIC_SHIFT);
- return fld >= 0x3;
+ return fld >= ID_AA64PFR0_EL1_GIC_V4P1;
}
#else
bool gic_cpuif_has_vsgi(void)
diff --git a/drivers/irqchip/irq-ixp4xx.c b/drivers/irqchip/irq-ixp4xx.c
index 5fba907b9052..f23b02f62a5c 100644
--- a/drivers/irqchip/irq-ixp4xx.c
+++ b/drivers/irqchip/irq-ixp4xx.c
@@ -105,8 +105,7 @@ static void ixp4xx_irq_unmask(struct irq_data *d)
}
}
-static asmlinkage void __exception_irq_entry
-ixp4xx_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry ixp4xx_handle_irq(struct pt_regs *regs)
{
struct ixp4xx_irq *ixi = &ixirq;
unsigned long status;
diff --git a/drivers/irqchip/irq-loongarch-avec.c b/drivers/irqchip/irq-loongarch-avec.c
new file mode 100644
index 000000000000..0f6e465dd309
--- /dev/null
+++ b/drivers/irqchip/irq-loongarch-avec.c
@@ -0,0 +1,425 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020-2024 Loongson Technologies, Inc.
+ */
+
+#include <linux/cpuhotplug.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqdomain.h>
+#include <linux/kernel.h>
+#include <linux/msi.h>
+#include <linux/radix-tree.h>
+#include <linux/spinlock.h>
+
+#include <asm/loongarch.h>
+#include <asm/setup.h>
+
+#include "irq-msi-lib.h"
+#include "irq-loongson.h"
+
+#define VECTORS_PER_REG 64
+#define IRR_VECTOR_MASK 0xffUL
+#define IRR_INVALID_MASK 0x80000000UL
+#define AVEC_MSG_OFFSET 0x100000
+
+#ifdef CONFIG_SMP
+struct pending_list {
+ struct list_head head;
+};
+
+static struct cpumask intersect_mask;
+static DEFINE_PER_CPU(struct pending_list, pending_list);
+#endif
+
+static DEFINE_PER_CPU(struct irq_desc * [NR_VECTORS], irq_map);
+
+struct avecintc_chip {
+ raw_spinlock_t lock;
+ struct fwnode_handle *fwnode;
+ struct irq_domain *domain;
+ struct irq_matrix *vector_matrix;
+ phys_addr_t msi_base_addr;
+};
+
+static struct avecintc_chip loongarch_avec;
+
+struct avecintc_data {
+ struct list_head entry;
+ unsigned int cpu;
+ unsigned int vec;
+ unsigned int prev_cpu;
+ unsigned int prev_vec;
+ unsigned int moving;
+};
+
+static inline void avecintc_ack_irq(struct irq_data *d)
+{
+}
+
+static inline void avecintc_mask_irq(struct irq_data *d)
+{
+}
+
+static inline void avecintc_unmask_irq(struct irq_data *d)
+{
+}
+
+#ifdef CONFIG_SMP
+static inline void pending_list_init(int cpu)
+{
+ struct pending_list *plist = per_cpu_ptr(&pending_list, cpu);
+
+ INIT_LIST_HEAD(&plist->head);
+}
+
+static void avecintc_sync(struct avecintc_data *adata)
+{
+ struct pending_list *plist;
+
+ if (cpu_online(adata->prev_cpu)) {
+ plist = per_cpu_ptr(&pending_list, adata->prev_cpu);
+ list_add_tail(&adata->entry, &plist->head);
+ adata->moving = 1;
+ mp_ops.send_ipi_single(adata->prev_cpu, ACTION_CLEAR_VECTOR);
+ }
+}
+
+static int avecintc_set_affinity(struct irq_data *data, const struct cpumask *dest, bool force)
+{
+ int cpu, ret, vector;
+ struct avecintc_data *adata;
+
+ scoped_guard(raw_spinlock, &loongarch_avec.lock) {
+ adata = irq_data_get_irq_chip_data(data);
+
+ if (adata->moving)
+ return -EBUSY;
+
+ if (cpu_online(adata->cpu) && cpumask_test_cpu(adata->cpu, dest))
+ return 0;
+
+ cpumask_and(&intersect_mask, dest, cpu_online_mask);
+
+ ret = irq_matrix_alloc(loongarch_avec.vector_matrix, &intersect_mask, false, &cpu);
+ if (ret < 0)
+ return ret;
+
+ vector = ret;
+ adata->cpu = cpu;
+ adata->vec = vector;
+ per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(data);
+ avecintc_sync(adata);
+ }
+
+ irq_data_update_effective_affinity(data, cpumask_of(cpu));
+
+ return IRQ_SET_MASK_OK;
+}
+
+static int avecintc_cpu_online(unsigned int cpu)
+{
+ if (!loongarch_avec.vector_matrix)
+ return 0;
+
+ guard(raw_spinlock)(&loongarch_avec.lock);
+
+ irq_matrix_online(loongarch_avec.vector_matrix);
+
+ pending_list_init(cpu);
+
+ return 0;
+}
+
+static int avecintc_cpu_offline(unsigned int cpu)
+{
+ struct pending_list *plist = per_cpu_ptr(&pending_list, cpu);
+
+ if (!loongarch_avec.vector_matrix)
+ return 0;
+
+ guard(raw_spinlock)(&loongarch_avec.lock);
+
+ if (!list_empty(&plist->head))
+ pr_warn("CPU#%d vector is busy\n", cpu);
+
+ irq_matrix_offline(loongarch_avec.vector_matrix);
+
+ return 0;
+}
+
+void complete_irq_moving(void)
+{
+ struct pending_list *plist = this_cpu_ptr(&pending_list);
+ struct avecintc_data *adata, *tdata;
+ int cpu, vector, bias;
+ uint64_t isr;
+
+ guard(raw_spinlock)(&loongarch_avec.lock);
+
+ list_for_each_entry_safe(adata, tdata, &plist->head, entry) {
+ cpu = adata->prev_cpu;
+ vector = adata->prev_vec;
+ bias = vector / VECTORS_PER_REG;
+ switch (bias) {
+ case 0:
+ isr = csr_read64(LOONGARCH_CSR_ISR0);
+ break;
+ case 1:
+ isr = csr_read64(LOONGARCH_CSR_ISR1);
+ break;
+ case 2:
+ isr = csr_read64(LOONGARCH_CSR_ISR2);
+ break;
+ case 3:
+ isr = csr_read64(LOONGARCH_CSR_ISR3);
+ break;
+ }
+
+ if (isr & (1UL << (vector % VECTORS_PER_REG))) {
+ mp_ops.send_ipi_single(cpu, ACTION_CLEAR_VECTOR);
+ continue;
+ }
+ list_del(&adata->entry);
+ irq_matrix_free(loongarch_avec.vector_matrix, cpu, vector, false);
+ this_cpu_write(irq_map[vector], NULL);
+ adata->moving = 0;
+ adata->prev_cpu = adata->cpu;
+ adata->prev_vec = adata->vec;
+ }
+}
+#endif
+
+static void avecintc_compose_msi_msg(struct irq_data *d, struct msi_msg *msg)
+{
+ struct avecintc_data *adata = irq_data_get_irq_chip_data(d);
+
+ msg->address_hi = 0x0;
+ msg->address_lo = (loongarch_avec.msi_base_addr | (adata->vec & 0xff) << 4)
+ | ((cpu_logical_map(adata->cpu & 0xffff)) << 12);
+ msg->data = 0x0;
+}
+
+static struct irq_chip avec_irq_controller = {
+ .name = "AVECINTC",
+ .irq_ack = avecintc_ack_irq,
+ .irq_mask = avecintc_mask_irq,
+ .irq_unmask = avecintc_unmask_irq,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = avecintc_set_affinity,
+#endif
+ .irq_compose_msi_msg = avecintc_compose_msi_msg,
+};
+
+static void avecintc_irq_dispatch(struct irq_desc *desc)
+{
+ struct irq_chip *chip = irq_desc_get_chip(desc);
+ struct irq_desc *d;
+
+ chained_irq_enter(chip, desc);
+
+ while (true) {
+ unsigned long vector = csr_read64(LOONGARCH_CSR_IRR);
+ if (vector & IRR_INVALID_MASK)
+ break;
+
+ vector &= IRR_VECTOR_MASK;
+
+ d = this_cpu_read(irq_map[vector]);
+ if (d) {
+ generic_handle_irq_desc(d);
+ } else {
+ spurious_interrupt();
+ pr_warn("Unexpected IRQ occurs on CPU#%d [vector %ld]\n", smp_processor_id(), vector);
+ }
+ }
+
+ chained_irq_exit(chip, desc);
+}
+
+static int avecintc_alloc_vector(struct irq_data *irqd, struct avecintc_data *adata)
+{
+ int cpu, ret;
+
+ guard(raw_spinlock_irqsave)(&loongarch_avec.lock);
+
+ ret = irq_matrix_alloc(loongarch_avec.vector_matrix, cpu_online_mask, false, &cpu);
+ if (ret < 0)
+ return ret;
+
+ adata->prev_cpu = adata->cpu = cpu;
+ adata->prev_vec = adata->vec = ret;
+ per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(irqd);
+
+ return 0;
+}
+
+static int avecintc_domain_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *arg)
+{
+ for (unsigned int i = 0; i < nr_irqs; i++) {
+ struct irq_data *irqd = irq_domain_get_irq_data(domain, virq + i);
+ struct avecintc_data *adata = kzalloc(sizeof(*adata), GFP_KERNEL);
+ int ret;
+
+ if (!adata)
+ return -ENOMEM;
+
+ ret = avecintc_alloc_vector(irqd, adata);
+ if (ret < 0) {
+ kfree(adata);
+ return ret;
+ }
+
+ irq_domain_set_info(domain, virq + i, virq + i, &avec_irq_controller,
+ adata, handle_edge_irq, NULL, NULL);
+ irqd_set_single_target(irqd);
+ irqd_set_affinity_on_activate(irqd);
+ }
+
+ return 0;
+}
+
+static void avecintc_free_vector(struct irq_data *irqd, struct avecintc_data *adata)
+{
+ guard(raw_spinlock_irqsave)(&loongarch_avec.lock);
+
+ per_cpu(irq_map, adata->cpu)[adata->vec] = NULL;
+ irq_matrix_free(loongarch_avec.vector_matrix, adata->cpu, adata->vec, false);
+
+#ifdef CONFIG_SMP
+ if (!adata->moving)
+ return;
+
+ per_cpu(irq_map, adata->prev_cpu)[adata->prev_vec] = NULL;
+ irq_matrix_free(loongarch_avec.vector_matrix, adata->prev_cpu, adata->prev_vec, false);
+ list_del_init(&adata->entry);
+#endif
+}
+
+static void avecintc_domain_free(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs)
+{
+ for (unsigned int i = 0; i < nr_irqs; i++) {
+ struct irq_data *d = irq_domain_get_irq_data(domain, virq + i);
+
+ if (d) {
+ struct avecintc_data *adata = irq_data_get_irq_chip_data(d);
+
+ avecintc_free_vector(d, adata);
+ irq_domain_reset_irq_data(d);
+ kfree(adata);
+ }
+ }
+}
+
+static const struct irq_domain_ops avecintc_domain_ops = {
+ .alloc = avecintc_domain_alloc,
+ .free = avecintc_domain_free,
+ .select = msi_lib_irq_domain_select,
+};
+
+static int __init irq_matrix_init(void)
+{
+ loongarch_avec.vector_matrix = irq_alloc_matrix(NR_VECTORS, 0, NR_VECTORS);
+ if (!loongarch_avec.vector_matrix)
+ return -ENOMEM;
+
+ for (int i = 0; i < NR_LEGACY_VECTORS; i++)
+ irq_matrix_assign_system(loongarch_avec.vector_matrix, i, false);
+
+ irq_matrix_online(loongarch_avec.vector_matrix);
+
+ return 0;
+}
+
+static int __init avecintc_init(struct irq_domain *parent)
+{
+ int ret, parent_irq;
+ unsigned long value;
+
+ raw_spin_lock_init(&loongarch_avec.lock);
+
+ loongarch_avec.fwnode = irq_domain_alloc_named_fwnode("AVECINTC");
+ if (!loongarch_avec.fwnode) {
+ pr_err("Unable to allocate domain handle\n");
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ loongarch_avec.domain = irq_domain_create_tree(loongarch_avec.fwnode,
+ &avecintc_domain_ops, NULL);
+ if (!loongarch_avec.domain) {
+ pr_err("Unable to create IRQ domain\n");
+ ret = -ENOMEM;
+ goto out_free_handle;
+ }
+
+ parent_irq = irq_create_mapping(parent, INT_AVEC);
+ if (!parent_irq) {
+ pr_err("Failed to mapping hwirq\n");
+ ret = -EINVAL;
+ goto out_remove_domain;
+ }
+
+ ret = irq_matrix_init();
+ if (ret < 0) {
+ pr_err("Failed to init irq matrix\n");
+ goto out_remove_domain;
+ }
+ irq_set_chained_handler_and_data(parent_irq, avecintc_irq_dispatch, NULL);
+
+#ifdef CONFIG_SMP
+ pending_list_init(0);
+ cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_AVECINTC_STARTING,
+ "irqchip/loongarch/avecintc:starting",
+ avecintc_cpu_online, avecintc_cpu_offline);
+#endif
+ value = iocsr_read64(LOONGARCH_IOCSR_MISC_FUNC);
+ value |= IOCSR_MISC_FUNC_AVEC_EN;
+ iocsr_write64(value, LOONGARCH_IOCSR_MISC_FUNC);
+
+ return ret;
+
+out_remove_domain:
+ irq_domain_remove(loongarch_avec.domain);
+out_free_handle:
+ irq_domain_free_fwnode(loongarch_avec.fwnode);
+out:
+ return ret;
+}
+
+static int __init pch_msi_parse_madt(union acpi_subtable_headers *header,
+ const unsigned long end)
+{
+ struct acpi_madt_msi_pic *pchmsi_entry = (struct acpi_madt_msi_pic *)header;
+
+ loongarch_avec.msi_base_addr = pchmsi_entry->msg_address - AVEC_MSG_OFFSET;
+
+ return pch_msi_acpi_init_avec(loongarch_avec.domain);
+}
+
+static inline int __init acpi_cascade_irqdomain_init(void)
+{
+ return acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1);
+}
+
+int __init avecintc_acpi_init(struct irq_domain *parent)
+{
+ int ret = avecintc_init(parent);
+ if (ret < 0) {
+ pr_err("Failed to init IRQ domain\n");
+ return ret;
+ }
+
+ ret = acpi_cascade_irqdomain_init();
+ if (ret < 0) {
+ pr_err("Failed to init cascade IRQ domain\n");
+ return ret;
+ }
+
+ return ret;
+}
diff --git a/drivers/irqchip/irq-loongarch-cpu.c b/drivers/irqchip/irq-loongarch-cpu.c
index 9d8f2c406043..bcbd7fd33178 100644
--- a/drivers/irqchip/irq-loongarch-cpu.c
+++ b/drivers/irqchip/irq-loongarch-cpu.c
@@ -13,6 +13,8 @@
#include <asm/loongarch.h>
#include <asm/setup.h>
+#include "irq-loongson.h"
+
static struct irq_domain *irq_domain;
struct fwnode_handle *cpuintc_handle;
@@ -138,7 +140,10 @@ static int __init acpi_cascade_irqdomain_init(void)
if (r < 0)
return r;
- return 0;
+ if (cpu_has_avecint)
+ r = avecintc_acpi_init(irq_domain);
+
+ return r;
}
static int __init cpuintc_acpi_init(union acpi_subtable_headers *header,
diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c
index b1f2080be2be..e24db71a8783 100644
--- a/drivers/irqchip/irq-loongson-eiointc.c
+++ b/drivers/irqchip/irq-loongson-eiointc.c
@@ -17,6 +17,8 @@
#include <linux/syscore_ops.h>
#include <asm/numa.h>
+#include "irq-loongson.h"
+
#define EIOINTC_REG_NODEMAP 0x14a0
#define EIOINTC_REG_IPMAP 0x14c0
#define EIOINTC_REG_ENABLE 0x1600
@@ -360,6 +362,9 @@ static int __init acpi_cascade_irqdomain_init(void)
if (r < 0)
return r;
+ if (cpu_has_avecint)
+ return 0;
+
r = acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1);
if (r < 0)
return r;
@@ -396,8 +401,8 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
if (nr_pics == 1) {
register_syscore_ops(&eiointc_syscore_ops);
- cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_LOONGARCH_STARTING,
- "irqchip/loongarch/intc:starting",
+ cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_EIOINTC_STARTING,
+ "irqchip/loongarch/eiointc:starting",
eiointc_router_init, NULL);
}
diff --git a/drivers/irqchip/irq-loongson-htvec.c b/drivers/irqchip/irq-loongson-htvec.c
index 0bff728b25e3..5da02c7ad0b3 100644
--- a/drivers/irqchip/irq-loongson-htvec.c
+++ b/drivers/irqchip/irq-loongson-htvec.c
@@ -17,6 +17,8 @@
#include <linux/of_irq.h>
#include <linux/syscore_ops.h>
+#include "irq-loongson.h"
+
/* Registers */
#define HTVEC_EN_OFF 0x20
#define HTVEC_MAX_PARENT_IRQ 8
diff --git a/drivers/irqchip/irq-loongson-liointc.c b/drivers/irqchip/irq-loongson-liointc.c
index 7c4fe7ab4b83..2b1bd4a96665 100644
--- a/drivers/irqchip/irq-loongson-liointc.c
+++ b/drivers/irqchip/irq-loongson-liointc.c
@@ -22,6 +22,8 @@
#include <asm/loongson.h>
#endif
+#include "irq-loongson.h"
+
#define LIOINTC_CHIP_IRQ 32
#define LIOINTC_NUM_PARENT 4
#define LIOINTC_NUM_CORES 4
diff --git a/drivers/irqchip/irq-loongson-pch-lpc.c b/drivers/irqchip/irq-loongson-pch-lpc.c
index 9b35492fb6be..2d4c3ec128b8 100644
--- a/drivers/irqchip/irq-loongson-pch-lpc.c
+++ b/drivers/irqchip/irq-loongson-pch-lpc.c
@@ -15,6 +15,8 @@
#include <linux/kernel.h>
#include <linux/syscore_ops.h>
+#include "irq-loongson.h"
+
/* Registers */
#define LPC_INT_CTL 0x00
#define LPC_INT_ENA 0x04
diff --git a/drivers/irqchip/irq-loongson-pch-msi.c b/drivers/irqchip/irq-loongson-pch-msi.c
index dd4d699170f4..bd337ecddb40 100644
--- a/drivers/irqchip/irq-loongson-pch-msi.c
+++ b/drivers/irqchip/irq-loongson-pch-msi.c
@@ -15,6 +15,9 @@
#include <linux/pci.h>
#include <linux/slab.h>
+#include "irq-msi-lib.h"
+#include "irq-loongson.h"
+
static int nr_pics;
struct pch_msi_data {
@@ -27,26 +30,6 @@ struct pch_msi_data {
static struct fwnode_handle *pch_msi_handle[MAX_IO_PICS];
-static void pch_msi_mask_msi_irq(struct irq_data *d)
-{
- pci_msi_mask_irq(d);
- irq_chip_mask_parent(d);
-}
-
-static void pch_msi_unmask_msi_irq(struct irq_data *d)
-{
- irq_chip_unmask_parent(d);
- pci_msi_unmask_irq(d);
-}
-
-static struct irq_chip pch_msi_irq_chip = {
- .name = "PCH PCI MSI",
- .irq_mask = pch_msi_mask_msi_irq,
- .irq_unmask = pch_msi_unmask_msi_irq,
- .irq_ack = irq_chip_ack_parent,
- .irq_set_affinity = irq_chip_set_affinity_parent,
-};
-
static int pch_msi_allocate_hwirq(struct pch_msi_data *priv, int num_req)
{
int first;
@@ -85,12 +68,6 @@ static void pch_msi_compose_msi_msg(struct irq_data *data,
msg->data = data->hwirq;
}
-static struct msi_domain_info pch_msi_domain_info = {
- .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
- MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX,
- .chip = &pch_msi_irq_chip,
-};
-
static struct irq_chip middle_irq_chip = {
.name = "PCH MSI",
.irq_mask = irq_chip_mask_parent,
@@ -155,13 +132,31 @@ static void pch_msi_middle_domain_free(struct irq_domain *domain,
static const struct irq_domain_ops pch_msi_middle_domain_ops = {
.alloc = pch_msi_middle_domain_alloc,
.free = pch_msi_middle_domain_free,
+ .select = msi_lib_irq_domain_select,
+};
+
+#define PCH_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS | \
+ MSI_FLAG_USE_DEF_CHIP_OPS | \
+ MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+#define PCH_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK | \
+ MSI_FLAG_PCI_MSIX | \
+ MSI_FLAG_MULTI_PCI_MSI)
+
+static struct msi_parent_ops pch_msi_parent_ops = {
+ .required_flags = PCH_MSI_FLAGS_REQUIRED,
+ .supported_flags = PCH_MSI_FLAGS_SUPPORTED,
+ .bus_select_mask = MATCH_PCI_MSI,
+ .bus_select_token = DOMAIN_BUS_NEXUS,
+ .prefix = "PCH-",
+ .init_dev_msi_info = msi_lib_init_dev_msi_info,
};
static int pch_msi_init_domains(struct pch_msi_data *priv,
struct irq_domain *parent,
struct fwnode_handle *domain_handle)
{
- struct irq_domain *middle_domain, *msi_domain;
+ struct irq_domain *middle_domain;
middle_domain = irq_domain_create_hierarchy(parent, 0, priv->num_irqs,
domain_handle,
@@ -174,14 +169,8 @@ static int pch_msi_init_domains(struct pch_msi_data *priv,
irq_domain_update_bus_token(middle_domain, DOMAIN_BUS_NEXUS);
- msi_domain = pci_msi_create_irq_domain(domain_handle,
- &pch_msi_domain_info,
- middle_domain);
- if (!msi_domain) {
- pr_err("Failed to create PCI MSI domain\n");
- irq_domain_remove(middle_domain);
- return -ENOMEM;
- }
+ middle_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+ middle_domain->msi_parent_ops = &pch_msi_parent_ops;
return 0;
}
@@ -266,17 +255,17 @@ IRQCHIP_DECLARE(pch_msi, "loongson,pch-msi-1.0", pch_msi_of_init);
#ifdef CONFIG_ACPI
struct fwnode_handle *get_pch_msi_handle(int pci_segment)
{
- int i;
+ if (cpu_has_avecint)
+ return pch_msi_handle[0];
- for (i = 0; i < MAX_IO_PICS; i++) {
+ for (int i = 0; i < MAX_IO_PICS; i++) {
if (msi_group[i].pci_segment == pci_segment)
return pch_msi_handle[i];
}
- return NULL;
+ return pch_msi_handle[0];
}
-int __init pch_msi_acpi_init(struct irq_domain *parent,
- struct acpi_madt_msi_pic *acpi_pchmsi)
+int __init pch_msi_acpi_init(struct irq_domain *parent, struct acpi_madt_msi_pic *acpi_pchmsi)
{
int ret;
struct fwnode_handle *domain_handle;
@@ -289,4 +278,18 @@ int __init pch_msi_acpi_init(struct irq_domain *parent,
return ret;
}
+
+int __init pch_msi_acpi_init_avec(struct irq_domain *parent)
+{
+ if (pch_msi_handle[0])
+ return 0;
+
+ pch_msi_handle[0] = parent->fwnode;
+ irq_domain_update_bus_token(parent, DOMAIN_BUS_NEXUS);
+
+ parent->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+ parent->msi_parent_ops = &pch_msi_parent_ops;
+
+ return 0;
+}
#endif
diff --git a/drivers/irqchip/irq-loongson-pch-pic.c b/drivers/irqchip/irq-loongson-pch-pic.c
index cbaef65e804c..69efda35a8e7 100644
--- a/drivers/irqchip/irq-loongson-pch-pic.c
+++ b/drivers/irqchip/irq-loongson-pch-pic.c
@@ -17,6 +17,8 @@
#include <linux/of_irq.h>
#include <linux/syscore_ops.h>
+#include "irq-loongson.h"
+
/* Registers */
#define PCH_PIC_MASK 0x20
#define PCH_PIC_HTMSI_EN 0x40
diff --git a/drivers/irqchip/irq-loongson.h b/drivers/irqchip/irq-loongson.h
new file mode 100644
index 000000000000..11fa138d1f44
--- /dev/null
+++ b/drivers/irqchip/irq-loongson.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
+ */
+
+#ifndef _DRIVERS_IRQCHIP_IRQ_LOONGSON_H
+#define _DRIVERS_IRQCHIP_IRQ_LOONGSON_H
+
+int find_pch_pic(u32 gsi);
+
+int liointc_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_lio_pic *acpi_liointc);
+int eiointc_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_eio_pic *acpi_eiointc);
+int avecintc_acpi_init(struct irq_domain *parent);
+
+int htvec_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_ht_pic *acpi_htvec);
+int pch_lpc_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_lpc_pic *acpi_pchlpc);
+int pch_pic_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_bio_pic *acpi_pchpic);
+int pch_msi_acpi_init(struct irq_domain *parent,
+ struct acpi_madt_msi_pic *acpi_pchmsi);
+int pch_msi_acpi_init_avec(struct irq_domain *parent);
+
+#endif /* _DRIVERS_IRQCHIP_IRQ_LOONGSON_H */
diff --git a/drivers/irqchip/irq-mbigen.c b/drivers/irqchip/irq-mbigen.c
index 093fd42893a7..12919836dadb 100644
--- a/drivers/irqchip/irq-mbigen.c
+++ b/drivers/irqchip/irq-mbigen.c
@@ -222,37 +222,27 @@ static int mbigen_of_create_domain(struct platform_device *pdev,
struct mbigen_device *mgn_chip)
{
struct platform_device *child;
- struct device_node *np;
u32 num_pins;
- int ret = 0;
- for_each_child_of_node(pdev->dev.of_node, np) {
+ for_each_child_of_node_scoped(pdev->dev.of_node, np) {
if (!of_property_read_bool(np, "interrupt-controller"))
continue;
child = of_platform_device_create(np, NULL, NULL);
- if (!child) {
- ret = -ENOMEM;
- break;
- }
+ if (!child)
+ return -ENOMEM;
if (of_property_read_u32(child->dev.of_node, "num-pins",
&num_pins) < 0) {
dev_err(&pdev->dev, "No num-pins property\n");
- ret = -EINVAL;
- break;
+ return -EINVAL;
}
- if (!mbigen_create_device_domain(&child->dev, num_pins, mgn_chip)) {
- ret = -ENOMEM;
- break;
- }
+ if (!mbigen_create_device_domain(&child->dev, num_pins, mgn_chip))
+ return -ENOMEM;
}
- if (ret)
- of_node_put(np);
-
- return ret;
+ return 0;
}
#ifdef CONFIG_ACPI
diff --git a/drivers/irqchip/irq-omap-intc.c b/drivers/irqchip/irq-omap-intc.c
index dc82162ba763..ad84a2f03368 100644
--- a/drivers/irqchip/irq-omap-intc.c
+++ b/drivers/irqchip/irq-omap-intc.c
@@ -325,8 +325,7 @@ static int __init omap_init_irq(u32 base, struct device_node *node)
return ret;
}
-static asmlinkage void __exception_irq_entry
-omap_intc_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry omap_intc_handle_irq(struct pt_regs *regs)
{
extern unsigned long irq_err_count;
u32 irqnr;
diff --git a/drivers/irqchip/irq-sa11x0.c b/drivers/irqchip/irq-sa11x0.c
index 31c202a1ae62..9d0b80271949 100644
--- a/drivers/irqchip/irq-sa11x0.c
+++ b/drivers/irqchip/irq-sa11x0.c
@@ -127,8 +127,7 @@ static int __init sa1100irq_init_devicefs(void)
device_initcall(sa1100irq_init_devicefs);
-static asmlinkage void __exception_irq_entry
-sa1100_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry sa1100_handle_irq(struct pt_regs *regs)
{
uint32_t icip, icmr, mask;
diff --git a/drivers/irqchip/irq-versatile-fpga.c b/drivers/irqchip/irq-versatile-fpga.c
index 5018a06060e6..ca471c6fee99 100644
--- a/drivers/irqchip/irq-versatile-fpga.c
+++ b/drivers/irqchip/irq-versatile-fpga.c
@@ -128,7 +128,7 @@ static int handle_one_fpga(struct fpga_irq_data *f, struct pt_regs *regs)
* Keep iterating over all registered FPGA IRQ controllers until there are
* no pending interrupts.
*/
-static asmlinkage void __exception_irq_entry fpga_handle_irq(struct pt_regs *regs)
+static void __exception_irq_entry fpga_handle_irq(struct pt_regs *regs)
{
int i, handled;
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 51ba681b915a..55a726d317d4 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -145,7 +145,8 @@ enum cpuhp_state {
CPUHP_AP_IRQ_ARMADA_XP_STARTING,
CPUHP_AP_IRQ_BCM2836_STARTING,
CPUHP_AP_IRQ_MIPS_GIC_STARTING,
- CPUHP_AP_IRQ_LOONGARCH_STARTING,
+ CPUHP_AP_IRQ_EIOINTC_STARTING,
+ CPUHP_AP_IRQ_AVECINTC_STARTING,
CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
CPUHP_AP_IRQ_RISCV_IMSIC_STARTING,
CPUHP_AP_ARM_MVEBU_COHERENCY,
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index 3f30c88e0b4c..457151f9f263 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -276,7 +276,7 @@ struct irq_affinity_notify {
#define IRQ_AFFINITY_MAX_SETS 4
/**
- * struct irq_affinity - Description for automatic irq affinity assignements
+ * struct irq_affinity - Description for automatic irq affinity assignments
* @pre_vectors: Don't apply affinity to @pre_vectors at beginning of
* the MSI(-X) vector space
* @post_vectors: Don't apply affinity to @post_vectors at end of
@@ -594,7 +594,7 @@ extern const char * const softirq_to_name[NR_SOFTIRQS];
struct softirq_action
{
- void (*action)(struct softirq_action *);
+ void (*action)(void);
};
asmlinkage void do_softirq(void);
@@ -609,7 +609,7 @@ static inline void do_softirq_post_smp_call_flush(unsigned int unused)
}
#endif
-extern void open_softirq(int nr, void (*action)(struct softirq_action *));
+extern void open_softirq(int nr, void (*action)(void));
extern void softirq_init(void);
extern void __raise_softirq_irqoff(unsigned int nr);
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 1f5dbf1f92c9..fa711f80957b 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -991,7 +991,6 @@ void irq_init_desc(unsigned int irq);
* @ack: Ack register offset to reg_base
* @eoi: Eoi register offset to reg_base
* @type: Type configuration register offset to reg_base
- * @polarity: Polarity configuration register offset to reg_base
*/
struct irq_chip_regs {
unsigned long enable;
@@ -1000,7 +999,6 @@ struct irq_chip_regs {
unsigned long ack;
unsigned long eoi;
unsigned long type;
- unsigned long polarity;
};
/**
@@ -1040,8 +1038,6 @@ struct irq_chip_type {
* @irq_base: Interrupt base nr for this chip
* @irq_cnt: Number of interrupts handled by this chip
* @mask_cache: Cached mask register shared between all chip types
- * @type_cache: Cached type register
- * @polarity_cache: Cached polarity register
* @wake_enabled: Interrupt can wakeup from suspend
* @wake_active: Interrupt is marked as an wakeup from suspend source
* @num_ct: Number of available irq_chip_type instances (usually 1)
@@ -1068,8 +1064,6 @@ struct irq_chip_generic {
unsigned int irq_base;
unsigned int irq_cnt;
u32 mask_cache;
- u32 type_cache;
- u32 polarity_cache;
u32 wake_enabled;
u32 wake_active;
unsigned int num_ct;
diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index de6105f68fec..e432b6a12a32 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -291,7 +291,12 @@ struct irq_domain_chip_generic_info;
* @hwirq_max: Maximum number of interrupts supported by controller
* @direct_max: Maximum value of direct maps;
* Use ~0 for no limit; 0 for no direct mapping
+ * @hwirq_base: The first hardware interrupt number (legacy domains only)
+ * @virq_base: The first Linux interrupt number for legacy domains to
+ * immediately associate the interrupts after domain creation
* @bus_token: Domain bus token
+ * @name_suffix: Optional name suffix to avoid collisions when multiple
+ * domains are added using same fwnode
* @ops: Domain operation callbacks
* @host_data: Controller private data pointer
* @dgc_info: Geneneric chip information structure pointer used to
@@ -307,7 +312,10 @@ struct irq_domain_info {
unsigned int size;
irq_hw_number_t hwirq_max;
int direct_max;
+ unsigned int hwirq_base;
+ unsigned int virq_base;
enum irq_domain_bus_token bus_token;
+ const char *name_suffix;
const struct irq_domain_ops *ops;
void *host_data;
#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index dc94e0bf2c94..271e9139de77 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -198,7 +198,7 @@ __irq_startup_managed(struct irq_desc *desc, const struct cpumask *aff,
irqd_clr_managed_shutdown(d);
- if (cpumask_any_and(aff, cpu_online_mask) >= nr_cpu_ids) {
+ if (!cpumask_intersects(aff, cpu_online_mask)) {
/*
* Catch code which fiddles with enable_irq() on a managed
* and potentially shutdown IRQ. Chained interrupt
diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c
index eb8628390156..15a7654eff68 100644
--- a/kernel/irq/cpuhotplug.c
+++ b/kernel/irq/cpuhotplug.c
@@ -37,7 +37,7 @@ static inline bool irq_needs_fixup(struct irq_data *d)
* has been removed from the online mask already.
*/
if (cpumask_any_but(m, cpu) < nr_cpu_ids &&
- cpumask_any_and(m, cpu_online_mask) >= nr_cpu_ids) {
+ !cpumask_intersects(m, cpu_online_mask)) {
/*
* If this happens then there was a missed IRQ fixup at some
* point. Warn about it and enforce fixup.
@@ -110,7 +110,7 @@ static bool migrate_one_irq(struct irq_desc *desc)
if (maskchip && chip->irq_mask)
chip->irq_mask(d);
- if (cpumask_any_and(affinity, cpu_online_mask) >= nr_cpu_ids) {
+ if (!cpumask_intersects(affinity, cpu_online_mask)) {
/*
* If the interrupt is managed, then shut it down and leave
* the affinity untouched.
diff --git a/kernel/irq/irq_sim.c b/kernel/irq/irq_sim.c
index 3d4036db15ac..1a3d483548e2 100644
--- a/kernel/irq/irq_sim.c
+++ b/kernel/irq/irq_sim.c
@@ -13,7 +13,6 @@
struct irq_sim_work_ctx {
struct irq_work work;
- int irq_base;
unsigned int irq_count;
unsigned long *pending;
struct irq_domain *domain;
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index cea8f6874b1f..e0bff21f30e0 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -128,72 +128,98 @@ void irq_domain_free_fwnode(struct fwnode_handle *fwnode)
}
EXPORT_SYMBOL_GPL(irq_domain_free_fwnode);
-static int irq_domain_set_name(struct irq_domain *domain,
- const struct fwnode_handle *fwnode,
- enum irq_domain_bus_token bus_token)
+static int alloc_name(struct irq_domain *domain, char *base, enum irq_domain_bus_token bus_token)
+{
+ if (bus_token == DOMAIN_BUS_ANY)
+ domain->name = kasprintf(GFP_KERNEL, "%s", base);
+ else
+ domain->name = kasprintf(GFP_KERNEL, "%s-%d", base, bus_token);
+ if (!domain->name)
+ return -ENOMEM;
+
+ domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
+ return 0;
+}
+
+static int alloc_fwnode_name(struct irq_domain *domain, const struct fwnode_handle *fwnode,
+ enum irq_domain_bus_token bus_token, const char *suffix)
+{
+ const char *sep = suffix ? "-" : "";
+ const char *suf = suffix ? : "";
+ char *name;
+
+ if (bus_token == DOMAIN_BUS_ANY)
+ name = kasprintf(GFP_KERNEL, "%pfw%s%s", fwnode, sep, suf);
+ else
+ name = kasprintf(GFP_KERNEL, "%pfw%s%s-%d", fwnode, sep, suf, bus_token);
+ if (!name)
+ return -ENOMEM;
+
+ /*
+ * fwnode paths contain '/', which debugfs is legitimately unhappy
+ * about. Replace them with ':', which does the trick and is not as
+ * offensive as '\'...
+ */
+ domain->name = strreplace(name, '/', ':');
+ domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
+ return 0;
+}
+
+static int alloc_unknown_name(struct irq_domain *domain, enum irq_domain_bus_token bus_token)
{
static atomic_t unknown_domains;
- struct irqchip_fwid *fwid;
+ int id = atomic_inc_return(&unknown_domains);
+
+ if (bus_token == DOMAIN_BUS_ANY)
+ domain->name = kasprintf(GFP_KERNEL, "unknown-%d", id);
+ else
+ domain->name = kasprintf(GFP_KERNEL, "unknown-%d-%d", id, bus_token);
+ if (!domain->name)
+ return -ENOMEM;
+
+ domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
+ return 0;
+}
+
+static int irq_domain_set_name(struct irq_domain *domain, const struct irq_domain_info *info)
+{
+ enum irq_domain_bus_token bus_token = info->bus_token;
+ const struct fwnode_handle *fwnode = info->fwnode;
if (is_fwnode_irqchip(fwnode)) {
- fwid = container_of(fwnode, struct irqchip_fwid, fwnode);
+ struct irqchip_fwid *fwid = container_of(fwnode, struct irqchip_fwid, fwnode);
+
+ /*
+ * The name_suffix is only intended to be used to avoid a name
+ * collision when multiple domains are created for a single
+ * device and the name is picked using a real device node.
+ * (Typical use-case is regmap-IRQ controllers for devices
+ * providing more than one physical IRQ.) There should be no
+ * need to use name_suffix with irqchip-fwnode.
+ */
+ if (info->name_suffix)
+ return -EINVAL;
switch (fwid->type) {
case IRQCHIP_FWNODE_NAMED:
case IRQCHIP_FWNODE_NAMED_ID:
- domain->name = bus_token ?
- kasprintf(GFP_KERNEL, "%s-%d",
- fwid->name, bus_token) :
- kstrdup(fwid->name, GFP_KERNEL);
- if (!domain->name)
- return -ENOMEM;
- domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
- break;
+ return alloc_name(domain, fwid->name, bus_token);
default:
domain->name = fwid->name;
- if (bus_token) {
- domain->name = kasprintf(GFP_KERNEL, "%s-%d",
- fwid->name, bus_token);
- if (!domain->name)
- return -ENOMEM;
- domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
- }
- break;
+ if (bus_token != DOMAIN_BUS_ANY)
+ return alloc_name(domain, fwid->name, bus_token);
}
- } else if (is_of_node(fwnode) || is_acpi_device_node(fwnode) ||
- is_software_node(fwnode)) {
- char *name;
-
- /*
- * fwnode paths contain '/', which debugfs is legitimately
- * unhappy about. Replace them with ':', which does
- * the trick and is not as offensive as '\'...
- */
- name = bus_token ?
- kasprintf(GFP_KERNEL, "%pfw-%d", fwnode, bus_token) :
- kasprintf(GFP_KERNEL, "%pfw", fwnode);
- if (!name)
- return -ENOMEM;
- domain->name = strreplace(name, '/', ':');
- domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
+ } else if (is_of_node(fwnode) || is_acpi_device_node(fwnode) || is_software_node(fwnode)) {
+ return alloc_fwnode_name(domain, fwnode, bus_token, info->name_suffix);
}
- if (!domain->name) {
- if (fwnode)
- pr_err("Invalid fwnode type for irqdomain\n");
- domain->name = bus_token ?
- kasprintf(GFP_KERNEL, "unknown-%d-%d",
- atomic_inc_return(&unknown_domains),
- bus_token) :
- kasprintf(GFP_KERNEL, "unknown-%d",
- atomic_inc_return(&unknown_domains));
- if (!domain->name)
- return -ENOMEM;
- domain->flags |= IRQ_DOMAIN_NAME_ALLOCATED;
- }
+ if (domain->name)
+ return 0;
- return 0;
+ if (fwnode)
+ pr_err("Invalid fwnode type for irqdomain\n");
+ return alloc_unknown_name(domain, bus_token);
}
static struct irq_domain *__irq_domain_create(const struct irq_domain_info *info)
@@ -211,7 +237,7 @@ static struct irq_domain *__irq_domain_create(const struct irq_domain_info *info
if (!domain)
return ERR_PTR(-ENOMEM);
- err = irq_domain_set_name(domain, info->fwnode, info->bus_token);
+ err = irq_domain_set_name(domain, info);
if (err) {
kfree(domain);
return ERR_PTR(err);
@@ -267,13 +293,20 @@ static void irq_domain_free(struct irq_domain *domain)
kfree(domain);
}
-/**
- * irq_domain_instantiate() - Instantiate a new irq domain data structure
- * @info: Domain information pointer pointing to the information for this domain
- *
- * Return: A pointer to the instantiated irq domain or an ERR_PTR value.
- */
-struct irq_domain *irq_domain_instantiate(const struct irq_domain_info *info)
+static void irq_domain_instantiate_descs(const struct irq_domain_info *info)
+{
+ if (!IS_ENABLED(CONFIG_SPARSE_IRQ))
+ return;
+
+ if (irq_alloc_descs(info->virq_base, info->virq_base, info->size,
+ of_node_to_nid(to_of_node(info->fwnode))) < 0) {
+ pr_info("Cannot allocate irq_descs @ IRQ%d, assuming pre-allocated\n",
+ info->virq_base);
+ }
+}
+
+static struct irq_domain *__irq_domain_instantiate(const struct irq_domain_info *info,
+ bool cond_alloc_descs, bool force_associate)
{
struct irq_domain *domain;
int err;
@@ -306,6 +339,19 @@ struct irq_domain *irq_domain_instantiate(const struct irq_domain_info *info)
__irq_domain_publish(domain);
+ if (cond_alloc_descs && info->virq_base > 0)
+ irq_domain_instantiate_descs(info);
+
+ /*
+ * Legacy interrupt domains have a fixed Linux interrupt number
+ * associated. Other interrupt domains can request association by
+ * providing a Linux interrupt number > 0.
+ */
+ if (force_associate || info->virq_base > 0) {
+ irq_domain_associate_many(domain, info->virq_base, info->hwirq_base,
+ info->size - info->hwirq_base);
+ }
+
return domain;
err_domain_gc_remove:
@@ -315,6 +361,17 @@ struct irq_domain *irq_domain_instantiate(const struct irq_domain_info *info)
irq_domain_free(domain);
return ERR_PTR(err);
}
+
+/**
+ * irq_domain_instantiate() - Instantiate a new irq domain data structure
+ * @info: Domain information pointer pointing to the information for this domain
+ *
+ * Return: A pointer to the instantiated irq domain or an ERR_PTR value.
+ */
+struct irq_domain *irq_domain_instantiate(const struct irq_domain_info *info)
+{
+ return __irq_domain_instantiate(info, false, false);
+}
EXPORT_SYMBOL_GPL(irq_domain_instantiate);
/**
@@ -413,28 +470,13 @@ struct irq_domain *irq_domain_create_simple(struct fwnode_handle *fwnode,
.fwnode = fwnode,
.size = size,
.hwirq_max = size,
+ .virq_base = first_irq,
.ops = ops,
.host_data = host_data,
};
- struct irq_domain *domain;
-
- domain = irq_domain_instantiate(&info);
- if (IS_ERR(domain))
- return NULL;
+ struct irq_domain *domain = __irq_domain_instantiate(&info, true, false);
- if (first_irq > 0) {
- if (IS_ENABLED(CONFIG_SPARSE_IRQ)) {
- /* attempt to allocated irq_descs */
- int rc = irq_alloc_descs(first_irq, first_irq, size,
- of_node_to_nid(to_of_node(fwnode)));
- if (rc < 0)
- pr_info("Cannot allocate irq_descs @ IRQ%d, assuming pre-allocated\n",
- first_irq);
- }
- irq_domain_associate_many(domain, first_irq, 0, size);
- }
-
- return domain;
+ return IS_ERR(domain) ? NULL : domain;
}
EXPORT_SYMBOL_GPL(irq_domain_create_simple);
@@ -476,18 +518,14 @@ struct irq_domain *irq_domain_create_legacy(struct fwnode_handle *fwnode,
.fwnode = fwnode,
.size = first_hwirq + size,
.hwirq_max = first_hwirq + size,
+ .hwirq_base = first_hwirq,
+ .virq_base = first_irq,
.ops = ops,
.host_data = host_data,
};
- struct irq_domain *domain;
+ struct irq_domain *domain = __irq_domain_instantiate(&info, false, true);
- domain = irq_domain_instantiate(&info);
- if (IS_ERR(domain))
- return NULL;
-
- irq_domain_associate_many(domain, first_irq, first_hwirq, size);
-
- return domain;
+ return IS_ERR(domain) ? NULL : domain;
}
EXPORT_SYMBOL_GPL(irq_domain_create_legacy);
@@ -1365,7 +1403,7 @@ static int irq_domain_trim_hierarchy(unsigned int virq)
tail = NULL;
/* The first entry must have a valid irqchip */
- if (!irq_data->chip || IS_ERR(irq_data->chip))
+ if (IS_ERR_OR_NULL(irq_data->chip))
return -EINVAL;
/*
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index dd53298ef1a5..f0803d6bd296 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -218,21 +218,20 @@ static void irq_validate_effective_affinity(struct irq_data *data)
static inline void irq_validate_effective_affinity(struct irq_data *data) { }
#endif
+static DEFINE_PER_CPU(struct cpumask, __tmp_mask);
+
int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask,
bool force)
{
+ struct cpumask *tmp_mask = this_cpu_ptr(&__tmp_mask);
struct irq_desc *desc = irq_data_to_desc(data);
struct irq_chip *chip = irq_data_get_irq_chip(data);
const struct cpumask *prog_mask;
int ret;
- static DEFINE_RAW_SPINLOCK(tmp_mask_lock);
- static struct cpumask tmp_mask;
-
if (!chip || !chip->irq_set_affinity)
return -EINVAL;
- raw_spin_lock(&tmp_mask_lock);
/*
* If this is a managed interrupt and housekeeping is enabled on
* it check whether the requested affinity mask intersects with
@@ -258,11 +257,11 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask,
hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
- cpumask_and(&tmp_mask, mask, hk_mask);
- if (!cpumask_intersects(&tmp_mask, cpu_online_mask))
+ cpumask_and(tmp_mask, mask, hk_mask);
+ if (!cpumask_intersects(tmp_mask, cpu_online_mask))
prog_mask = mask;
else
- prog_mask = &tmp_mask;
+ prog_mask = tmp_mask;
} else {
prog_mask = mask;
}
@@ -272,16 +271,14 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask,
* unless we are being asked to force the affinity (in which
* case we do as we are told).
*/
- cpumask_and(&tmp_mask, prog_mask, cpu_online_mask);
- if (!force && !cpumask_empty(&tmp_mask))
- ret = chip->irq_set_affinity(data, &tmp_mask, force);
+ cpumask_and(tmp_mask, prog_mask, cpu_online_mask);
+ if (!force && !cpumask_empty(tmp_mask))
+ ret = chip->irq_set_affinity(data, tmp_mask, force);
else if (force)
ret = chip->irq_set_affinity(data, mask, force);
else
ret = -EINVAL;
- raw_spin_unlock(&tmp_mask_lock);
-
switch (ret) {
case IRQ_SET_MASK_OK:
case IRQ_SET_MASK_OK_DONE:
diff --git a/kernel/irq/migration.c b/kernel/irq/migration.c
index 61ca924ef4b4..eb150afd671f 100644
--- a/kernel/irq/migration.c
+++ b/kernel/irq/migration.c
@@ -26,7 +26,7 @@ bool irq_fixup_move_pending(struct irq_desc *desc, bool force_clear)
* The outgoing CPU might be the last online target in a pending
* interrupt move. If that's the case clear the pending move bit.
*/
- if (cpumask_any_and(desc->pending_mask, cpu_online_mask) >= nr_cpu_ids) {
+ if (!cpumask_intersects(desc->pending_mask, cpu_online_mask)) {
irqd_clr_move_pending(data);
return false;
}
@@ -74,7 +74,7 @@ void irq_move_masked_irq(struct irq_data *idata)
* For correct operation this depends on the caller
* masking the irqs.
*/
- if (cpumask_any_and(desc->pending_mask, cpu_online_mask) < nr_cpu_ids) {
+ if (cpumask_intersects(desc->pending_mask, cpu_online_mask)) {
int ret;
ret = irq_do_set_affinity(data, desc->pending_mask, false);
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 5fa0547ece0c..1c7e5159064c 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -82,7 +82,7 @@ static struct msi_desc *msi_alloc_desc(struct device *dev, int nvec,
desc->dev = dev;
desc->nvec_used = nvec;
if (affinity) {
- desc->affinity = kmemdup(affinity, nvec * sizeof(*desc->affinity), GFP_KERNEL);
+ desc->affinity = kmemdup_array(affinity, nvec, sizeof(*desc->affinity), GFP_KERNEL);
if (!desc->affinity) {
kfree(desc);
return NULL;
diff --git a/kernel/irq/proc.c b/kernel/irq/proc.c
index 8cccdf40725a..9081ada81c3d 100644
--- a/kernel/irq/proc.c
+++ b/kernel/irq/proc.c
@@ -52,10 +52,8 @@ static int show_irq_affinity(int type, struct seq_file *m)
case AFFINITY:
case AFFINITY_LIST:
mask = desc->irq_common_data.affinity;
-#ifdef CONFIG_GENERIC_PENDING_IRQ
- if (irqd_is_setaffinity_pending(&desc->irq_data))
- mask = desc->pending_mask;
-#endif
+ if (irq_move_pending(&desc->irq_data))
+ mask = irq_desc_get_pending_mask(desc);
break;
case EFFECTIVE:
case EFFECTIVE_LIST:
@@ -142,7 +140,7 @@ static ssize_t write_irq_affinity(int type, struct file *file,
int err;
if (!irq_can_set_affinity_usr(irq) || no_irq_affinity)
- return -EIO;
+ return -EPERM;
if (!zalloc_cpumask_var(&new_value, GFP_KERNEL))
return -ENOMEM;
@@ -362,8 +360,13 @@ void register_irq_proc(unsigned int irq, struct irq_desc *desc)
goto out_unlock;
#ifdef CONFIG_SMP
+ umode_t umode = S_IRUGO;
+
+ if (irq_can_set_affinity_usr(desc->irq_data.irq))
+ umode |= S_IWUSR;
+
/* create /proc/irq/<irq>/smp_affinity */
- proc_create_data("smp_affinity", 0644, desc->dir,
+ proc_create_data("smp_affinity", umode, desc->dir,
&irq_affinity_proc_ops, irqp);
/* create /proc/irq/<irq>/affinity_hint */
@@ -371,7 +374,7 @@ void register_irq_proc(unsigned int irq, struct irq_desc *desc)
irq_affinity_hint_proc_show, irqp);
/* create /proc/irq/<irq>/smp_affinity_list */
- proc_create_data("smp_affinity_list", 0644, desc->dir,
+ proc_create_data("smp_affinity_list", umode, desc->dir,
&irq_affinity_list_proc_ops, irqp);
proc_create_single_data("node", 0444, desc->dir, irq_node_proc_show,
diff --git a/kernel/rcu/tiny.c b/kernel/rcu/tiny.c
index 4402d6f5f857..b3b3ce34df63 100644
--- a/kernel/rcu/tiny.c
+++ b/kernel/rcu/tiny.c
@@ -105,7 +105,7 @@ static inline bool rcu_reclaim_tiny(struct rcu_head *head)
}
/* Invoke the RCU callbacks whose grace period has elapsed. */
-static __latent_entropy void rcu_process_callbacks(struct softirq_action *unused)
+static __latent_entropy void rcu_process_callbacks(void)
{
struct rcu_head *next, *list;
unsigned long flags;
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index e641cc681901..93bd665637c0 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2855,7 +2855,7 @@ static __latent_entropy void rcu_core(void)
queue_work_on(rdp->cpu, rcu_gp_wq, &rdp->strict_work);
}
-static void rcu_core_si(struct softirq_action *h)
+static void rcu_core_si(void)
{
rcu_core();
}
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 9057584ec06d..8dc9385f6da4 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12483,7 +12483,7 @@ static int sched_balance_newidle(struct rq *this_rq, struct rq_flags *rf)
* - indirectly from a remote scheduler_tick() for NOHZ idle balancing
* through the SMP cross-call nohz_csd_func()
*/
-static __latent_entropy void sched_balance_softirq(struct softirq_action *h)
+static __latent_entropy void sched_balance_softirq(void)
{
struct rq *this_rq = this_rq();
enum cpu_idle_type idle = this_rq->idle_balance;
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 02582017759a..d082e7840f88 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -551,7 +551,7 @@ static void handle_softirqs(bool ksirqd)
kstat_incr_softirqs_this_cpu(vec_nr);
trace_softirq_entry(vec_nr);
- h->action(h);
+ h->action();
trace_softirq_exit(vec_nr);
if (unlikely(prev_count != preempt_count())) {
pr_err("huh, entered softirq %u %s %p with preempt_count %08x, exited with %08x?\n",
@@ -700,7 +700,7 @@ void __raise_softirq_irqoff(unsigned int nr)
or_softirq_pending(1UL << nr);
}
-void open_softirq(int nr, void (*action)(struct softirq_action *))
+void open_softirq(int nr, void (*action)(void))
{
softirq_vec[nr].action = action;
}
@@ -760,8 +760,7 @@ static bool tasklet_clear_sched(struct tasklet_struct *t)
return false;
}
-static void tasklet_action_common(struct softirq_action *a,
- struct tasklet_head *tl_head,
+static void tasklet_action_common(struct tasklet_head *tl_head,
unsigned int softirq_nr)
{
struct tasklet_struct *list;
@@ -805,16 +804,16 @@ static void tasklet_action_common(struct softirq_action *a,
}
}
-static __latent_entropy void tasklet_action(struct softirq_action *a)
+static __latent_entropy void tasklet_action(void)
{
workqueue_softirq_action(false);
- tasklet_action_common(a, this_cpu_ptr(&tasklet_vec), TASKLET_SOFTIRQ);
+ tasklet_action_common(this_cpu_ptr(&tasklet_vec), TASKLET_SOFTIRQ);
}
-static __latent_entropy void tasklet_hi_action(struct softirq_action *a)
+static __latent_entropy void tasklet_hi_action(void)
{
workqueue_softirq_action(true);
- tasklet_action_common(a, this_cpu_ptr(&tasklet_hi_vec), HI_SOFTIRQ);
+ tasklet_action_common(this_cpu_ptr(&tasklet_hi_vec), HI_SOFTIRQ);
}
void tasklet_setup(struct tasklet_struct *t,
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index b8ee320208d4..836157e09e25 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1757,7 +1757,7 @@ static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
}
}
-static __latent_entropy void hrtimer_run_softirq(struct softirq_action *h)
+static __latent_entropy void hrtimer_run_softirq(void)
{
struct hrtimer_cpu_base *cpu_base = this_cpu_ptr(&hrtimer_bases);
unsigned long flags;
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 64b0d8a0aa0f..760bbeb1f331 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -2440,7 +2440,7 @@ static void run_timer_base(int index)
/*
* This function runs timers and the timer-tq in bottom half context.
*/
-static __latent_entropy void run_timer_softirq(struct softirq_action *h)
+static __latent_entropy void run_timer_softirq(void)
{
run_timer_base(BASE_LOCAL);
if (IS_ENABLED(CONFIG_NO_HZ_COMMON)) {
diff --git a/lib/irq_poll.c b/lib/irq_poll.c
index 2d5329a42105..08b242bbdbdf 100644
--- a/lib/irq_poll.c
+++ b/lib/irq_poll.c
@@ -75,7 +75,7 @@ void irq_poll_complete(struct irq_poll *iop)
}
EXPORT_SYMBOL(irq_poll_complete);
-static void __latent_entropy irq_poll_softirq(struct softirq_action *h)
+static void __latent_entropy irq_poll_softirq(void)
{
struct list_head *list = this_cpu_ptr(&blk_cpu_iopoll);
int rearm = 0, budget = irq_poll_budget;
diff --git a/net/core/dev.c b/net/core/dev.c
index 6ea1d20676fb..e24a3bcb496d 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5247,7 +5247,7 @@ int netif_rx(struct sk_buff *skb)
}
EXPORT_SYMBOL(netif_rx);
-static __latent_entropy void net_tx_action(struct softirq_action *h)
+static __latent_entropy void net_tx_action(void)
{
struct softnet_data *sd = this_cpu_ptr(&softnet_data);
@@ -6920,7 +6920,7 @@ static int napi_threaded_poll(void *data)
return 0;
}
-static __latent_entropy void net_rx_action(struct softirq_action *h)
+static __latent_entropy void net_rx_action(void)
{
struct softnet_data *sd = this_cpu_ptr(&softnet_data);
unsigned long time_limit = jiffies +
^ permalink raw reply [flat|nested] 12+ messages in thread* [GIT pull] smp/core for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
2024-09-16 8:38 ` [GIT pull] irq/core " Thomas Gleixner
@ 2024-09-16 8:38 ` Thomas Gleixner
2024-09-17 6:35 ` pr-tracker-bot
2024-09-16 8:38 ` [GIT pull] timers/clocksource " Thomas Gleixner
` (4 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: Thomas Gleixner @ 2024-09-16 8:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest smp/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git smp-core-2024-09-16
up to: 1d07085402d1: smp: Mark smp_prepare_boot_cpu() __init
A set of updates for CPU hotplug:
- Prepare the core for supporting parallel hotplug on loongarch
- A small set of cleanups and enhancements
Thanks,
tglx
------------------>
Bibo Mao (1):
smp: Mark smp_prepare_boot_cpu() __init
Jiaxun Yang (2):
cpu/hotplug: Make HOTPLUG_PARALLEL independent of HOTPLUG_SMT
cpu/hotplug: Provide weak fallback for arch_cpuhp_init_parallel_bringup()
Thorsten Blum (1):
cpu: Fix W=1 build kernel-doc warning
arch/loongarch/kernel/smp.c | 2 +-
arch/mips/kernel/smp.c | 2 +-
arch/powerpc/kernel/smp.c | 2 +-
include/linux/smp.h | 2 +-
kernel/cpu.c | 20 +++++++++++++++++---
5 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c
index ca405ab86aae..be2655c4c414 100644
--- a/arch/loongarch/kernel/smp.c
+++ b/arch/loongarch/kernel/smp.c
@@ -476,7 +476,7 @@ core_initcall(ipi_pm_init);
#endif
/* Preload SMP state for boot cpu */
-void smp_prepare_boot_cpu(void)
+void __init smp_prepare_boot_cpu(void)
{
unsigned int cpu, node, rr_node;
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 0362fc5df7b0..39e193cad2b9 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -439,7 +439,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
}
/* preload SMP state for boot cpu */
-void smp_prepare_boot_cpu(void)
+void __init smp_prepare_boot_cpu(void)
{
if (mp_ops->prepare_boot_cpu)
mp_ops->prepare_boot_cpu();
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 46e6d2cd7a2d..4ab9b8cee77a 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -1166,7 +1166,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
cpu_smt_set_num_threads(num_threads, threads_per_core);
}
-void smp_prepare_boot_cpu(void)
+void __init smp_prepare_boot_cpu(void)
{
BUG_ON(smp_processor_id() != boot_cpuid);
#ifdef CONFIG_PPC64
diff --git a/include/linux/smp.h b/include/linux/smp.h
index fcd61dfe2af3..6a0813c905d0 100644
--- a/include/linux/smp.h
+++ b/include/linux/smp.h
@@ -109,7 +109,7 @@ static inline void on_each_cpu_cond(smp_cond_func_t cond_func,
* Architecture specific boot CPU setup. Defined as empty weak function in
* init/main.c. Architectures can override it.
*/
-void smp_prepare_boot_cpu(void);
+void __init smp_prepare_boot_cpu(void);
#ifdef CONFIG_SMP
diff --git a/kernel/cpu.c b/kernel/cpu.c
index b1fd2a3db91a..0c9c5dfc8ddd 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1808,6 +1808,7 @@ static int __init parallel_bringup_parse_param(char *arg)
}
early_param("cpuhp.parallel", parallel_bringup_parse_param);
+#ifdef CONFIG_HOTPLUG_SMT
static inline bool cpuhp_smt_aware(void)
{
return cpu_smt_max_threads > 1;
@@ -1817,6 +1818,21 @@ static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
{
return cpu_primary_thread_mask;
}
+#else
+static inline bool cpuhp_smt_aware(void)
+{
+ return false;
+}
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_none_mask;
+}
+#endif
+
+bool __weak arch_cpuhp_init_parallel_bringup(void)
+{
+ return true;
+}
/*
* On architectures which have enabled parallel bringup this invokes all BP
@@ -2689,9 +2705,7 @@ int cpuhp_smt_disable(enum cpuhp_smt_control ctrlval)
return ret;
}
-/**
- * Check if the core a CPU belongs to is online
- */
+/* Check if the core a CPU belongs to is online */
#if !defined(topology_is_core_online)
static inline bool topology_is_core_online(unsigned int cpu)
{
^ permalink raw reply [flat|nested] 12+ messages in thread* [GIT pull] timers/clocksource for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
2024-09-16 8:38 ` [GIT pull] irq/core " Thomas Gleixner
2024-09-16 8:38 ` [GIT pull] smp/core " Thomas Gleixner
@ 2024-09-16 8:38 ` Thomas Gleixner
2024-09-17 6:35 ` pr-tracker-bot
2024-09-16 8:38 ` [GIT pull] timers/core " Thomas Gleixner
` (3 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: Thomas Gleixner @ 2024-09-16 8:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/clocksource branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-clocksource-2024-09-16
up to: 4ac1dd3245b9: clocksource: Set cs_watchdog_read() checks based on .uncertainty_margin
Updates for the clocksource watchdog:
- Make the uncertainty margin handling more robust to prevent false
positives
- Clarify comments
Thanks,
tglx
------------------>
Borislav Petkov (1):
clocksource: Improve comments for watchdog skew bounds
Paul E. McKenney (2):
clocksource: Fix comments on WATCHDOG_THRESHOLD & WATCHDOG_MAX_SKEW
clocksource: Set cs_watchdog_read() checks based on .uncertainty_margin
kernel/time/clocksource.c | 45 ++++++++++++++++++++++++++++++++-------------
1 file changed, 32 insertions(+), 13 deletions(-)
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index d0538a75f4c6..23336eecb4f4 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -113,7 +113,6 @@ static u64 suspend_start;
/*
* Threshold: 0.0312s, when doubled: 0.0625s.
- * Also a default for cs->uncertainty_margin when registering clocks.
*/
#define WATCHDOG_THRESHOLD (NSEC_PER_SEC >> 5)
@@ -125,6 +124,13 @@ static u64 suspend_start;
*
* The default of 500 parts per million is based on NTP's limits.
* If a clocksource is good enough for NTP, it is good enough for us!
+ *
+ * In other words, by default, even if a clocksource is extremely
+ * precise (for example, with a sub-nanosecond period), the maximum
+ * permissible skew between the clocksource watchdog and the clocksource
+ * under test is not permitted to go below the 500ppm minimum defined
+ * by MAX_SKEW_USEC. This 500ppm minimum may be overridden using the
+ * CLOCKSOURCE_WATCHDOG_MAX_SKEW_US Kconfig option.
*/
#ifdef CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US
#define MAX_SKEW_USEC CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US
@@ -132,6 +138,13 @@ static u64 suspend_start;
#define MAX_SKEW_USEC (125 * WATCHDOG_INTERVAL / HZ)
#endif
+/*
+ * Default for maximum permissible skew when cs->uncertainty_margin is
+ * not specified, and the lower bound even when cs->uncertainty_margin
+ * is specified. This is also the default that is used when registering
+ * clocks with unspecifed cs->uncertainty_margin, so this macro is used
+ * even in CONFIG_CLOCKSOURCE_WATCHDOG=n kernels.
+ */
#define WATCHDOG_MAX_SKEW (MAX_SKEW_USEC * NSEC_PER_USEC)
#ifdef CONFIG_CLOCKSOURCE_WATCHDOG
@@ -231,6 +244,7 @@ enum wd_read_status {
static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, u64 *wdnow)
{
+ int64_t md = 2 * watchdog->uncertainty_margin;
unsigned int nretries, max_retries;
int64_t wd_delay, wd_seq_delay;
u64 wd_end, wd_end2;
@@ -245,7 +259,7 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow,
local_irq_enable();
wd_delay = cycles_to_nsec_safe(watchdog, *wdnow, wd_end);
- if (wd_delay <= WATCHDOG_MAX_SKEW) {
+ if (wd_delay <= md + cs->uncertainty_margin) {
if (nretries > 1 && nretries >= max_retries) {
pr_warn("timekeeping watchdog on CPU%d: %s retried %d times before success\n",
smp_processor_id(), watchdog->name, nretries);
@@ -258,12 +272,12 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow,
* there is too much external interferences that cause
* significant delay in reading both clocksource and watchdog.
*
- * If consecutive WD read-back delay > WATCHDOG_MAX_SKEW/2,
- * report system busy, reinit the watchdog and skip the current
+ * If consecutive WD read-back delay > md, report
+ * system busy, reinit the watchdog and skip the current
* watchdog test.
*/
wd_seq_delay = cycles_to_nsec_safe(watchdog, wd_end, wd_end2);
- if (wd_seq_delay > WATCHDOG_MAX_SKEW/2)
+ if (wd_seq_delay > md)
goto skip_test;
}
@@ -1146,14 +1160,19 @@ void __clocksource_update_freq_scale(struct clocksource *cs, u32 scale, u32 freq
}
/*
- * If the uncertainty margin is not specified, calculate it.
- * If both scale and freq are non-zero, calculate the clock
- * period, but bound below at 2*WATCHDOG_MAX_SKEW. However,
- * if either of scale or freq is zero, be very conservative and
- * take the tens-of-milliseconds WATCHDOG_THRESHOLD value for the
- * uncertainty margin. Allow stupidly small uncertainty margins
- * to be specified by the caller for testing purposes, but warn
- * to discourage production use of this capability.
+ * If the uncertainty margin is not specified, calculate it. If
+ * both scale and freq are non-zero, calculate the clock period, but
+ * bound below at 2*WATCHDOG_MAX_SKEW, that is, 500ppm by default.
+ * However, if either of scale or freq is zero, be very conservative
+ * and take the tens-of-milliseconds WATCHDOG_THRESHOLD value
+ * for the uncertainty margin. Allow stupidly small uncertainty
+ * margins to be specified by the caller for testing purposes,
+ * but warn to discourage production use of this capability.
+ *
+ * Bottom line: The sum of the uncertainty margins of the
+ * watchdog clocksource and the clocksource under test will be at
+ * least 500ppm by default. For more information, please see the
+ * comment preceding CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US above.
*/
if (scale && freq && !cs->uncertainty_margin) {
cs->uncertainty_margin = NSEC_PER_SEC / (scale * freq);
^ permalink raw reply [flat|nested] 12+ messages in thread* [GIT pull] timers/core for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
` (2 preceding siblings ...)
2024-09-16 8:38 ` [GIT pull] timers/clocksource " Thomas Gleixner
@ 2024-09-16 8:38 ` Thomas Gleixner
2024-09-17 6:35 ` pr-tracker-bot
2024-09-16 8:44 ` [GIT pull] core/debugobjects " Thomas Gleixner
` (2 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: Thomas Gleixner @ 2024-09-16 8:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest timers/core branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers-core-2024-09-16
up to: 35b603f8a78b: ntp: Make sure RTC is synchronized when time goes backwards
Updates for timers and timekeeping:
- Core:
- Overhaul of posix-timers in preparation of removing the
workaround for periodic timers which have signal delivery
ignored.
- Remove the historical extra jiffie in msleep()
msleep() adds an extra jiffie to the timeout value to ensure
minimal sleep time. The timer wheel ensures minimal sleep
time since the large rewrite to a non-cascading wheel, but the
extra jiffie in msleep() remained unnoticed. Remove it.
- Make the timer slack handling correct for realtime tasks.
The procfs interface is inconsistent and does neither reflect
reality nor conforms to the man page. Show the correct 0 slack
for real time tasks and enforce it at the core level instead of
having inconsistent individual checks in various timer setup
functions.
- The usual set of updates and enhancements all over the place.
- Drivers:
- Allow the ACPI PM timer to be turned off during suspend
- No new drivers
- The usual updates and enhancements in various drivers
Thanks,
tglx
------------------>
Ankit Agrawal (1):
clocksource/drivers/qcom: Add missing iounmap() on errors in msm_dt_timer_init()
Anna-Maria Behnsen (5):
timers: Remove historical extra jiffie for timeout in msleep()
timers: Annotate possible non critical data race of next_expiry
timers: Rename next_expiry_recalc() to be unique
cpu: Use already existing usleep_range()
treewide: Fix wrong singular form of jiffies in comments
Benjamin ROBIN (1):
ntp: Make sure RTC is synchronized when time goes backwards
Detlev Casanova (1):
dt-bindings: timer: rockchip: Add rk3576 compatible
Felix Moessbauer (1):
hrtimer: Use and report correct timerslack values for realtime tasks
Gaosheng Cui (2):
clocksource/drivers/asm9260: Add missing clk_disable_unprepare in asm9260_timer_init
clocksource/drivers/cadence-ttc: Add missing clk_disable_unprepare in ttc_setup_clockevent
Huan Yang (1):
clocksource/drivers/ingenic: Use devm_clk_get_enabled() helpers
Marek Maslanka (3):
clocksource: acpi_pm: Add external callback for suspend/resume
platform/x86:intel/pmc: Enable the ACPI PM Timer to be turned off when suspended
platform/x86:intel/pmc: Fix comment for the pmc_core_acpi_pm_timer_suspend_resume function
Sebastian Andrzej Siewior (2):
timers: Add sparse annotation for timer_sync_wait_running().
hrtimer: Annotate hrtimer_cpu_base_.*_expiry() for sparse.
Thomas Gleixner (24):
selftests/timers/posix_timers: Simplify error handling
selftests/timers/posix_timers: Add SIG_IGN test
selftests/timers/posix_timers: Validate signal rules
selftests/timers/posix-timers: Validate SIGEV_NONE
selftests/timers/posix-timers: Validate timer_gettime()
selftests/timers/posix-timers: Validate overrun after unblock
posix-cpu-timers: Split up posix_cpu_timer_get()
posix-cpu-timers: Save interval only for armed timers
posix-cpu-timers: Handle interval timers correctly in timer_get()
posix-cpu-timers: Handle SIGEV_NONE timers correctly in timer_get()
posix-cpu-timers: Handle SIGEV_NONE timers correctly in timer_set()
posix-cpu-timers: Replace old expiry retrieval in posix_cpu_timer_set()
posix-cpu-timers: Do not arm SIGEV_NONE timers
posix-cpu-timers: Use @now instead of @val for clarity
posix-cpu-timers: Remove incorrect comment in posix_cpu_timer_set()
posix-cpu-timers: Simplify posix_cpu_timer_set()
posix-timers: Retrieve interval in common timer_settime() code
posix-timers: Clear overrun in common_timer_set()
posix-timers: Convert timer list to hlist
posix-timers: Consolidate timer setup
posix-cpu-timers: Make k_itimer::it_active consistent
posix-timers: Consolidate signal queueing
signal: Remove task argument from dequeue_signal()
signal: Replace BUG_ON()s
Uros Bizjak (1):
clocksource/drivers/jcore: Use request_percpu_irq()
Zhang Zekun (1):
clocksource/drivers/arm_arch_timer: Using for_each_available_child_of_node_scoped()
Documentation/admin-guide/media/vivid.rst | 2 +-
.../bindings/timer/rockchip,rk-timer.yaml | 1 +
Documentation/timers/timers-howto.rst | 2 +-
.../sp_SP/scheduler/sched-design-CFS.rst | 2 +-
arch/arm/mach-versatile/spc.c | 2 +-
arch/m68k/q40/q40ints.c | 2 +-
arch/x86/kernel/cpu/mce/dev-mcelog.c | 2 +-
drivers/char/ipmi/ipmi_ssif.c | 2 +-
drivers/clocksource/acpi_pm.c | 32 ++
drivers/clocksource/arm_arch_timer.c | 11 +-
drivers/clocksource/asm9260_timer.c | 1 +
drivers/clocksource/ingenic-ost.c | 7 +-
drivers/clocksource/jcore-pit.c | 7 +-
drivers/clocksource/timer-cadence-ttc.c | 6 +-
drivers/clocksource/timer-qcom.c | 7 +-
drivers/dma-buf/st-dma-fence.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_wait.c | 2 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 4 +-
drivers/gpu/drm/i915/i915_utils.c | 2 +-
drivers/gpu/drm/v3d/v3d_bo.c | 2 +-
drivers/isdn/mISDN/dsp_cmx.c | 2 +-
drivers/net/ethernet/marvell/mvmdio.c | 2 +-
drivers/platform/x86/intel/pmc/adl.c | 2 +
drivers/platform/x86/intel/pmc/cnp.c | 2 +
drivers/platform/x86/intel/pmc/core.c | 45 ++
drivers/platform/x86/intel/pmc/core.h | 8 +
drivers/platform/x86/intel/pmc/icl.c | 2 +
drivers/platform/x86/intel/pmc/mtl.c | 2 +
drivers/platform/x86/intel/pmc/spt.c | 2 +
drivers/platform/x86/intel/pmc/tgl.c | 2 +
fs/proc/base.c | 15 +-
fs/select.c | 11 +-
fs/signalfd.c | 4 +-
fs/xfs/xfs_buf.h | 2 +-
include/linux/acpi_pmtmr.h | 13 +
include/linux/jiffies.h | 2 +-
include/linux/posix-timers.h | 2 +-
include/linux/sched/signal.h | 7 +-
include/linux/timekeeper_internal.h | 2 +-
init/init_task.c | 2 +-
kernel/cpu.c | 2 +-
kernel/fork.c | 2 +-
kernel/sched/syscalls.c | 8 +
kernel/signal.c | 34 +-
kernel/sys.c | 2 +
kernel/time/alarmtimer.c | 9 +-
kernel/time/clockevents.c | 2 +-
kernel/time/hrtimer.c | 22 +-
kernel/time/ntp.c | 10 +-
kernel/time/ntp_internal.h | 4 +-
kernel/time/posix-cpu-timers.c | 207 ++++----
kernel/time/posix-timers.c | 73 +--
kernel/time/posix-timers.h | 3 +-
kernel/time/timekeeping.c | 4 +-
kernel/time/timer.c | 62 ++-
lib/Kconfig.debug | 2 +-
net/batman-adv/types.h | 2 +-
tools/testing/selftests/timers/posix_timers.c | 550 +++++++++++++++++----
58 files changed, 848 insertions(+), 377 deletions(-)
diff --git a/Documentation/admin-guide/media/vivid.rst b/Documentation/admin-guide/media/vivid.rst
index 1306f19ecb5a..c9d301ab46a3 100644
--- a/Documentation/admin-guide/media/vivid.rst
+++ b/Documentation/admin-guide/media/vivid.rst
@@ -328,7 +328,7 @@ and an HDMI input, one input for each input type. Those are described in more
detail below.
Special attention has been given to the rate at which new frames become
-available. The jitter will be around 1 jiffie (that depends on the HZ
+available. The jitter will be around 1 jiffy (that depends on the HZ
configuration of your kernel, so usually 1/100, 1/250 or 1/1000 of a second),
but the long-term behavior is exactly following the framerate. So a
framerate of 59.94 Hz is really different from 60 Hz. If the framerate
diff --git a/Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml b/Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml
index 19e56b7577a0..6d0eb0014eee 100644
--- a/Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml
+++ b/Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml
@@ -24,6 +24,7 @@ properties:
- rockchip,rk3228-timer
- rockchip,rk3229-timer
- rockchip,rk3368-timer
+ - rockchip,rk3576-timer
- rockchip,rk3588-timer
- rockchip,px30-timer
- const: rockchip,rk3288-timer
diff --git a/Documentation/timers/timers-howto.rst b/Documentation/timers/timers-howto.rst
index 5c169e3d29a8..ef7a4652ccc9 100644
--- a/Documentation/timers/timers-howto.rst
+++ b/Documentation/timers/timers-howto.rst
@@ -19,7 +19,7 @@ it really need to delay in atomic context?" If so...
ATOMIC CONTEXT:
You must use the `*delay` family of functions. These
- functions use the jiffie estimation of clock speed
+ functions use the jiffy estimation of clock speed
and will busy wait for enough loop cycles to achieve
the desired delay:
diff --git a/Documentation/translations/sp_SP/scheduler/sched-design-CFS.rst b/Documentation/translations/sp_SP/scheduler/sched-design-CFS.rst
index 90a153cad4e8..731c266beb1a 100644
--- a/Documentation/translations/sp_SP/scheduler/sched-design-CFS.rst
+++ b/Documentation/translations/sp_SP/scheduler/sched-design-CFS.rst
@@ -109,7 +109,7 @@ para que se ejecute, y la tarea en ejecución es interrumpida.
==================================
CFS usa una granularidad de nanosegundos y no depende de ningún
-jiffie o detalles como HZ. De este modo, el gestor de tareas CFS no tiene
+jiffy o detalles como HZ. De este modo, el gestor de tareas CFS no tiene
noción de "ventanas de tiempo" de la forma en que tenía el gestor de
tareas previo, y tampoco tiene heurísticos. Únicamente hay un parámetro
central ajustable (se ha de cambiar en CONFIG_SCHED_DEBUG):
diff --git a/arch/arm/mach-versatile/spc.c b/arch/arm/mach-versatile/spc.c
index 5e44170e1a9a..790092734cf6 100644
--- a/arch/arm/mach-versatile/spc.c
+++ b/arch/arm/mach-versatile/spc.c
@@ -73,7 +73,7 @@
/*
* Even though the SPC takes max 3-5 ms to complete any OPP/COMMS
- * operation, the operation could start just before jiffie is about
+ * operation, the operation could start just before jiffy is about
* to be incremented. So setting timeout value of 20ms = 2jiffies@100Hz
*/
#define TIMEOUT_US 20000
diff --git a/arch/m68k/q40/q40ints.c b/arch/m68k/q40/q40ints.c
index 10f1f294e91f..14b774b9d308 100644
--- a/arch/m68k/q40/q40ints.c
+++ b/arch/m68k/q40/q40ints.c
@@ -106,7 +106,7 @@ void __init q40_init_IRQ(void)
* this stuff doesn't really belong here..
*/
-int ql_ticks; /* 200Hz ticks since last jiffie */
+int ql_ticks; /* 200Hz ticks since last jiffy */
static int sound_ticks;
#define SVOL 45
diff --git a/arch/x86/kernel/cpu/mce/dev-mcelog.c b/arch/x86/kernel/cpu/mce/dev-mcelog.c
index a05ac0716ecf..a3aa0199222e 100644
--- a/arch/x86/kernel/cpu/mce/dev-mcelog.c
+++ b/arch/x86/kernel/cpu/mce/dev-mcelog.c
@@ -314,7 +314,7 @@ static ssize_t mce_chrdev_write(struct file *filp, const char __user *ubuf,
/*
* Need to give user space some time to set everything up,
- * so do it a jiffie or two later everywhere.
+ * so do it a jiffy or two later everywhere.
*/
schedule_timeout(2);
diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c
index 96ad571d041a..e093028391af 100644
--- a/drivers/char/ipmi/ipmi_ssif.c
+++ b/drivers/char/ipmi/ipmi_ssif.c
@@ -980,7 +980,7 @@ static void msg_written_handler(struct ssif_info *ssif_info, int result,
ipmi_ssif_unlock_cond(ssif_info, flags);
start_get(ssif_info);
} else {
- /* Wait a jiffie then request the next message */
+ /* Wait a jiffy then request the next message */
ssif_info->waiting_alert = true;
ssif_info->retries_left = SSIF_RECV_RETRIES;
if (!ssif_info->stopping)
diff --git a/drivers/clocksource/acpi_pm.c b/drivers/clocksource/acpi_pm.c
index 82338773602c..b4330a01a566 100644
--- a/drivers/clocksource/acpi_pm.c
+++ b/drivers/clocksource/acpi_pm.c
@@ -25,6 +25,10 @@
#include <asm/io.h>
#include <asm/time.h>
+static void *suspend_resume_cb_data;
+
+static void (*suspend_resume_callback)(void *data, bool suspend);
+
/*
* The I/O port the PMTMR resides at.
* The location is detected during setup_arch(),
@@ -58,6 +62,32 @@ u32 acpi_pm_read_verified(void)
return v2;
}
+void acpi_pmtmr_register_suspend_resume_callback(void (*cb)(void *data, bool suspend), void *data)
+{
+ suspend_resume_callback = cb;
+ suspend_resume_cb_data = data;
+}
+EXPORT_SYMBOL_GPL(acpi_pmtmr_register_suspend_resume_callback);
+
+void acpi_pmtmr_unregister_suspend_resume_callback(void)
+{
+ suspend_resume_callback = NULL;
+ suspend_resume_cb_data = NULL;
+}
+EXPORT_SYMBOL_GPL(acpi_pmtmr_unregister_suspend_resume_callback);
+
+static void acpi_pm_suspend(struct clocksource *cs)
+{
+ if (suspend_resume_callback)
+ suspend_resume_callback(suspend_resume_cb_data, true);
+}
+
+static void acpi_pm_resume(struct clocksource *cs)
+{
+ if (suspend_resume_callback)
+ suspend_resume_callback(suspend_resume_cb_data, false);
+}
+
static u64 acpi_pm_read(struct clocksource *cs)
{
return (u64)read_pmtmr();
@@ -69,6 +99,8 @@ static struct clocksource clocksource_acpi_pm = {
.read = acpi_pm_read,
.mask = (u64)ACPI_PM_MASK,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
+ .suspend = acpi_pm_suspend,
+ .resume = acpi_pm_resume,
};
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index aeafc74181f0..03733101e231 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -1594,7 +1594,6 @@ static int __init arch_timer_mem_of_init(struct device_node *np)
{
struct arch_timer_mem *timer_mem;
struct arch_timer_mem_frame *frame;
- struct device_node *frame_node;
struct resource res;
int ret = -EINVAL;
u32 rate;
@@ -1608,33 +1607,29 @@ static int __init arch_timer_mem_of_init(struct device_node *np)
timer_mem->cntctlbase = res.start;
timer_mem->size = resource_size(&res);
- for_each_available_child_of_node(np, frame_node) {
+ for_each_available_child_of_node_scoped(np, frame_node) {
u32 n;
struct arch_timer_mem_frame *frame;
if (of_property_read_u32(frame_node, "frame-number", &n)) {
pr_err(FW_BUG "Missing frame-number.\n");
- of_node_put(frame_node);
goto out;
}
if (n >= ARCH_TIMER_MEM_MAX_FRAMES) {
pr_err(FW_BUG "Wrong frame-number, only 0-%u are permitted.\n",
ARCH_TIMER_MEM_MAX_FRAMES - 1);
- of_node_put(frame_node);
goto out;
}
frame = &timer_mem->frame[n];
if (frame->valid) {
pr_err(FW_BUG "Duplicated frame-number.\n");
- of_node_put(frame_node);
goto out;
}
- if (of_address_to_resource(frame_node, 0, &res)) {
- of_node_put(frame_node);
+ if (of_address_to_resource(frame_node, 0, &res))
goto out;
- }
+
frame->cntbase = res.start;
frame->size = resource_size(&res);
diff --git a/drivers/clocksource/asm9260_timer.c b/drivers/clocksource/asm9260_timer.c
index 5b39d3701fa3..8f97ab0b01ec 100644
--- a/drivers/clocksource/asm9260_timer.c
+++ b/drivers/clocksource/asm9260_timer.c
@@ -210,6 +210,7 @@ static int __init asm9260_timer_init(struct device_node *np)
DRIVER_NAME, &event_dev);
if (ret) {
pr_err("Failed to setup irq!\n");
+ clk_disable_unprepare(clk);
return ret;
}
diff --git a/drivers/clocksource/ingenic-ost.c b/drivers/clocksource/ingenic-ost.c
index 9f7c280a1336..e0ec33307c84 100644
--- a/drivers/clocksource/ingenic-ost.c
+++ b/drivers/clocksource/ingenic-ost.c
@@ -93,14 +93,10 @@ static int __init ingenic_ost_probe(struct platform_device *pdev)
return PTR_ERR(map);
}
- ost->clk = devm_clk_get(dev, "ost");
+ ost->clk = devm_clk_get_enabled(dev, "ost");
if (IS_ERR(ost->clk))
return PTR_ERR(ost->clk);
- err = clk_prepare_enable(ost->clk);
- if (err)
- return err;
-
/* Clear counter high/low registers */
if (soc_info->is64bit)
regmap_write(map, TCU_REG_OST_CNTL, 0);
@@ -129,7 +125,6 @@ static int __init ingenic_ost_probe(struct platform_device *pdev)
err = clocksource_register_hz(cs, rate);
if (err) {
dev_err(dev, "clocksource registration failed");
- clk_disable_unprepare(ost->clk);
return err;
}
diff --git a/drivers/clocksource/jcore-pit.c b/drivers/clocksource/jcore-pit.c
index a4a991101fa3..a3fe98cd3838 100644
--- a/drivers/clocksource/jcore-pit.c
+++ b/drivers/clocksource/jcore-pit.c
@@ -120,7 +120,7 @@ static int jcore_pit_local_init(unsigned cpu)
static irqreturn_t jcore_timer_interrupt(int irq, void *dev_id)
{
- struct jcore_pit *pit = this_cpu_ptr(dev_id);
+ struct jcore_pit *pit = dev_id;
if (clockevent_state_oneshot(&pit->ced))
jcore_pit_disable(pit);
@@ -168,9 +168,8 @@ static int __init jcore_pit_init(struct device_node *node)
return -ENOMEM;
}
- err = request_irq(pit_irq, jcore_timer_interrupt,
- IRQF_TIMER | IRQF_PERCPU,
- "jcore_pit", jcore_pit_percpu);
+ err = request_percpu_irq(pit_irq, jcore_timer_interrupt,
+ "jcore_pit", jcore_pit_percpu);
if (err) {
pr_err("pit irq request failed: %d\n", err);
free_percpu(jcore_pit_percpu);
diff --git a/drivers/clocksource/timer-cadence-ttc.c b/drivers/clocksource/timer-cadence-ttc.c
index ca7a06489c40..b8a1cf59b9d6 100644
--- a/drivers/clocksource/timer-cadence-ttc.c
+++ b/drivers/clocksource/timer-cadence-ttc.c
@@ -435,7 +435,7 @@ static int __init ttc_setup_clockevent(struct clk *clk,
&ttcce->ttc.clk_rate_change_nb);
if (err) {
pr_warn("Unable to register clock notifier.\n");
- goto out_kfree;
+ goto out_clk_unprepare;
}
ttcce->ttc.freq = clk_get_rate(ttcce->ttc.clk);
@@ -465,13 +465,15 @@ static int __init ttc_setup_clockevent(struct clk *clk,
err = request_irq(irq, ttc_clock_event_interrupt,
IRQF_TIMER, ttcce->ce.name, ttcce);
if (err)
- goto out_kfree;
+ goto out_clk_unprepare;
clockevents_config_and_register(&ttcce->ce,
ttcce->ttc.freq / PRESCALE, 1, 0xfffe);
return 0;
+out_clk_unprepare:
+ clk_disable_unprepare(ttcce->ttc.clk);
out_kfree:
kfree(ttcce);
return err;
diff --git a/drivers/clocksource/timer-qcom.c b/drivers/clocksource/timer-qcom.c
index b4afe3a67583..eac4c95c6127 100644
--- a/drivers/clocksource/timer-qcom.c
+++ b/drivers/clocksource/timer-qcom.c
@@ -233,6 +233,7 @@ static int __init msm_dt_timer_init(struct device_node *np)
}
if (of_property_read_u32(np, "clock-frequency", &freq)) {
+ iounmap(cpu0_base);
pr_err("Unknown frequency\n");
return -EINVAL;
}
@@ -243,7 +244,11 @@ static int __init msm_dt_timer_init(struct device_node *np)
freq /= 4;
writel_relaxed(DGT_CLK_CTL_DIV_4, source_base + DGT_CLK_CTL);
- return msm_timer_init(freq, 32, irq, !!percpu_offset);
+ ret = msm_timer_init(freq, 32, irq, !!percpu_offset);
+ if (ret)
+ iounmap(cpu0_base);
+
+ return ret;
}
TIMER_OF_DECLARE(kpss_timer, "qcom,kpss-timer", msm_dt_timer_init);
TIMER_OF_DECLARE(scss_timer, "qcom,scss-timer", msm_dt_timer_init);
diff --git a/drivers/dma-buf/st-dma-fence.c b/drivers/dma-buf/st-dma-fence.c
index 6a1bfcd0cc21..cf2ce3744ce6 100644
--- a/drivers/dma-buf/st-dma-fence.c
+++ b/drivers/dma-buf/st-dma-fence.c
@@ -402,7 +402,7 @@ static int test_wait_timeout(void *arg)
if (dma_fence_wait_timeout(wt.f, false, 2) == -ETIME) {
if (timer_pending(&wt.timer)) {
- pr_notice("Timer did not fire within the jiffie!\n");
+ pr_notice("Timer did not fire within the jiffy!\n");
err = 0; /* not our fault! */
} else {
pr_err("Wait reported incomplete after timeout\n");
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_wait.c b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
index d4b918fb11ce..1f55e62044a4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_wait.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_wait.c
@@ -266,7 +266,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
if (ret == -ETIME && !nsecs_to_jiffies(args->timeout_ns))
args->timeout_ns = 0;
- /* Asked to wait beyond the jiffie/scheduler precision? */
+ /* Asked to wait beyond the jiffy/scheduler precision? */
if (ret == -ETIME && args->timeout_ns)
ret = -EAGAIN;
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 4202df5b8c12..222ca7c44951 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -93,7 +93,7 @@ static int wait_for_reset(struct intel_engine_cs *engine,
return -EINVAL;
}
- /* Give the request a jiffie to complete after flushing the worker */
+ /* Give the request a jiffy to complete after flushing the worker */
if (i915_request_wait(rq, 0,
max(0l, (long)(timeout - jiffies)) + 1) < 0) {
pr_err("%s: hanging request %llx:%lld did not complete\n",
@@ -3426,7 +3426,7 @@ static int live_preempt_timeout(void *arg)
cpu_relax();
saved_timeout = engine->props.preempt_timeout_ms;
- engine->props.preempt_timeout_ms = 1; /* in ms, -> 1 jiffie */
+ engine->props.preempt_timeout_ms = 1; /* in ms, -> 1 jiffy */
i915_request_get(rq);
i915_request_add(rq);
diff --git a/drivers/gpu/drm/i915/i915_utils.c b/drivers/gpu/drm/i915/i915_utils.c
index 6f9e7b354b54..f2ba51c20e97 100644
--- a/drivers/gpu/drm/i915/i915_utils.c
+++ b/drivers/gpu/drm/i915/i915_utils.c
@@ -110,7 +110,7 @@ void set_timer_ms(struct timer_list *t, unsigned long timeout)
* Paranoia to make sure the compiler computes the timeout before
* loading 'jiffies' as jiffies is volatile and may be updated in
* the background by a timer tick. All to reduce the complexity
- * of the addition and reduce the risk of losing a jiffie.
+ * of the addition and reduce the risk of losing a jiffy.
*/
barrier();
diff --git a/drivers/gpu/drm/v3d/v3d_bo.c b/drivers/gpu/drm/v3d/v3d_bo.c
index a165cbcdd27b..9eafe53a8f41 100644
--- a/drivers/gpu/drm/v3d/v3d_bo.c
+++ b/drivers/gpu/drm/v3d/v3d_bo.c
@@ -279,7 +279,7 @@ v3d_wait_bo_ioctl(struct drm_device *dev, void *data,
else
args->timeout_ns = 0;
- /* Asked to wait beyond the jiffie/scheduler precision? */
+ /* Asked to wait beyond the jiffy/scheduler precision? */
if (ret == -ETIME && args->timeout_ns)
ret = -EAGAIN;
diff --git a/drivers/isdn/mISDN/dsp_cmx.c b/drivers/isdn/mISDN/dsp_cmx.c
index 61cb45c5d0d8..53fad9487574 100644
--- a/drivers/isdn/mISDN/dsp_cmx.c
+++ b/drivers/isdn/mISDN/dsp_cmx.c
@@ -82,7 +82,7 @@
* - has multiple clocks.
* - has no usable clock due to jitter or packet loss (VoIP).
* In this case the system's clock is used. The clock resolution depends on
- * the jiffie resolution.
+ * the jiffy resolution.
*
* If a member joins a conference:
*
diff --git a/drivers/net/ethernet/marvell/mvmdio.c b/drivers/net/ethernet/marvell/mvmdio.c
index 9190eff6c0bb..e1d003fdbc2e 100644
--- a/drivers/net/ethernet/marvell/mvmdio.c
+++ b/drivers/net/ethernet/marvell/mvmdio.c
@@ -104,7 +104,7 @@ static int orion_mdio_wait_ready(const struct orion_mdio_ops *ops,
return 0;
} else {
/* wait_event_timeout does not guarantee a delay of at
- * least one whole jiffie, so timeout must be no less
+ * least one whole jiffy, so timeout must be no less
* than two.
*/
timeout = max(usecs_to_jiffies(MVMDIO_SMI_TIMEOUT), 2);
diff --git a/drivers/platform/x86/intel/pmc/adl.c b/drivers/platform/x86/intel/pmc/adl.c
index e7878558fd90..9d9c07f44ff6 100644
--- a/drivers/platform/x86/intel/pmc/adl.c
+++ b/drivers/platform/x86/intel/pmc/adl.c
@@ -295,6 +295,8 @@ const struct pmc_reg_map adl_reg_map = {
.ppfear_buckets = CNP_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = CNP_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = CNP_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.ltr_ignore_max = ADL_NUM_IP_IGN_ALLOWED,
.lpm_num_modes = ADL_LPM_NUM_MODES,
.lpm_num_maps = ADL_LPM_NUM_MAPS,
diff --git a/drivers/platform/x86/intel/pmc/cnp.c b/drivers/platform/x86/intel/pmc/cnp.c
index dd72974bf71e..513c02670c5a 100644
--- a/drivers/platform/x86/intel/pmc/cnp.c
+++ b/drivers/platform/x86/intel/pmc/cnp.c
@@ -200,6 +200,8 @@ const struct pmc_reg_map cnp_reg_map = {
.ppfear_buckets = CNP_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = CNP_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = CNP_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.ltr_ignore_max = CNP_NUM_IP_IGN_ALLOWED,
.etr3_offset = ETR3_OFFSET,
};
diff --git a/drivers/platform/x86/intel/pmc/core.c b/drivers/platform/x86/intel/pmc/core.c
index 01ae71c6df59..e2b4c74ce7f6 100644
--- a/drivers/platform/x86/intel/pmc/core.c
+++ b/drivers/platform/x86/intel/pmc/core.c
@@ -11,6 +11,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/acpi_pmtmr.h>
#include <linux/bitfield.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
@@ -1208,6 +1209,38 @@ static bool pmc_core_is_pson_residency_enabled(struct pmc_dev *pmcdev)
return val == 1;
}
+/*
+ * Enable or disable ACPI PM Timer
+ *
+ * This function is intended to be a callback for ACPI PM suspend/resume event.
+ * The ACPI PM Timer is enabled on resume only if it was enabled during suspend.
+ */
+static void pmc_core_acpi_pm_timer_suspend_resume(void *data, bool suspend)
+{
+ struct pmc_dev *pmcdev = data;
+ struct pmc *pmc = pmcdev->pmcs[PMC_IDX_MAIN];
+ const struct pmc_reg_map *map = pmc->map;
+ bool enabled;
+ u32 reg;
+
+ if (!map->acpi_pm_tmr_ctl_offset)
+ return;
+
+ guard(mutex)(&pmcdev->lock);
+
+ if (!suspend && !pmcdev->enable_acpi_pm_timer_on_resume)
+ return;
+
+ reg = pmc_core_reg_read(pmc, map->acpi_pm_tmr_ctl_offset);
+ enabled = !(reg & map->acpi_pm_tmr_disable_bit);
+ if (suspend)
+ reg |= map->acpi_pm_tmr_disable_bit;
+ else
+ reg &= ~map->acpi_pm_tmr_disable_bit;
+ pmc_core_reg_write(pmc, map->acpi_pm_tmr_ctl_offset, reg);
+
+ pmcdev->enable_acpi_pm_timer_on_resume = suspend && enabled;
+}
static void pmc_core_dbgfs_unregister(struct pmc_dev *pmcdev)
{
@@ -1404,6 +1437,7 @@ static int pmc_core_probe(struct platform_device *pdev)
struct pmc_dev *pmcdev;
const struct x86_cpu_id *cpu_id;
int (*core_init)(struct pmc_dev *pmcdev);
+ const struct pmc_reg_map *map;
struct pmc *primary_pmc;
int ret;
@@ -1462,6 +1496,11 @@ static int pmc_core_probe(struct platform_device *pdev)
pm_report_max_hw_sleep(FIELD_MAX(SLP_S0_RES_COUNTER_MASK) *
pmc_core_adjust_slp_s0_step(primary_pmc, 1));
+ map = primary_pmc->map;
+ if (map->acpi_pm_tmr_ctl_offset)
+ acpi_pmtmr_register_suspend_resume_callback(pmc_core_acpi_pm_timer_suspend_resume,
+ pmcdev);
+
device_initialized = true;
dev_info(&pdev->dev, " initialized\n");
@@ -1471,6 +1510,12 @@ static int pmc_core_probe(struct platform_device *pdev)
static void pmc_core_remove(struct platform_device *pdev)
{
struct pmc_dev *pmcdev = platform_get_drvdata(pdev);
+ const struct pmc *pmc = pmcdev->pmcs[PMC_IDX_MAIN];
+ const struct pmc_reg_map *map = pmc->map;
+
+ if (map->acpi_pm_tmr_ctl_offset)
+ acpi_pmtmr_unregister_suspend_resume_callback();
+
pmc_core_dbgfs_unregister(pmcdev);
pmc_core_clean_structure(pdev);
}
diff --git a/drivers/platform/x86/intel/pmc/core.h b/drivers/platform/x86/intel/pmc/core.h
index ea04de7eb9e8..4d37ef7113d7 100644
--- a/drivers/platform/x86/intel/pmc/core.h
+++ b/drivers/platform/x86/intel/pmc/core.h
@@ -68,6 +68,8 @@ struct telem_endpoint;
#define SPT_PMC_LTR_SCC 0x3A0
#define SPT_PMC_LTR_ISH 0x3A4
+#define SPT_PMC_ACPI_PM_TMR_CTL_OFFSET 0x18FC
+
/* Sunrise Point: PGD PFET Enable Ack Status Registers */
enum ppfear_regs {
SPT_PMC_XRAM_PPFEAR0A = 0x590,
@@ -148,6 +150,8 @@ enum ppfear_regs {
#define SPT_PMC_VRIC1_SLPS0LVEN BIT(13)
#define SPT_PMC_VRIC1_XTALSDQDIS BIT(22)
+#define SPT_PMC_BIT_ACPI_PM_TMR_DISABLE BIT(1)
+
/* Cannonlake Power Management Controller register offsets */
#define CNP_PMC_SLPS0_DBG_OFFSET 0x10B4
#define CNP_PMC_PM_CFG_OFFSET 0x1818
@@ -351,6 +355,8 @@ struct pmc_reg_map {
const u8 *lpm_reg_index;
const u32 pson_residency_offset;
const u32 pson_residency_counter_step;
+ const u32 acpi_pm_tmr_ctl_offset;
+ const u32 acpi_pm_tmr_disable_bit;
};
/**
@@ -424,6 +430,8 @@ struct pmc_dev {
u32 die_c6_offset;
struct telem_endpoint *punit_ep;
struct pmc_info *regmap_list;
+
+ bool enable_acpi_pm_timer_on_resume;
};
enum pmc_index {
diff --git a/drivers/platform/x86/intel/pmc/icl.c b/drivers/platform/x86/intel/pmc/icl.c
index 71b0fd6cb7d8..cbbd44054468 100644
--- a/drivers/platform/x86/intel/pmc/icl.c
+++ b/drivers/platform/x86/intel/pmc/icl.c
@@ -46,6 +46,8 @@ const struct pmc_reg_map icl_reg_map = {
.ppfear_buckets = ICL_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = CNP_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = CNP_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.ltr_ignore_max = ICL_NUM_IP_IGN_ALLOWED,
.etr3_offset = ETR3_OFFSET,
};
diff --git a/drivers/platform/x86/intel/pmc/mtl.c b/drivers/platform/x86/intel/pmc/mtl.c
index c7d15d864039..91f2fa728f5c 100644
--- a/drivers/platform/x86/intel/pmc/mtl.c
+++ b/drivers/platform/x86/intel/pmc/mtl.c
@@ -462,6 +462,8 @@ const struct pmc_reg_map mtl_socm_reg_map = {
.ppfear_buckets = MTL_SOCM_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = CNP_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = CNP_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.lpm_num_maps = ADL_LPM_NUM_MAPS,
.ltr_ignore_max = MTL_SOCM_NUM_IP_IGN_ALLOWED,
.lpm_res_counter_step_x2 = TGL_PMC_LPM_RES_COUNTER_STEP_X2,
diff --git a/drivers/platform/x86/intel/pmc/spt.c b/drivers/platform/x86/intel/pmc/spt.c
index ab993a69e33e..2cd2b3c68e46 100644
--- a/drivers/platform/x86/intel/pmc/spt.c
+++ b/drivers/platform/x86/intel/pmc/spt.c
@@ -130,6 +130,8 @@ const struct pmc_reg_map spt_reg_map = {
.ppfear_buckets = SPT_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = SPT_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = SPT_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.ltr_ignore_max = SPT_NUM_IP_IGN_ALLOWED,
.pm_vric1_offset = SPT_PMC_VRIC1_OFFSET,
};
diff --git a/drivers/platform/x86/intel/pmc/tgl.c b/drivers/platform/x86/intel/pmc/tgl.c
index e0580de18077..371b4e30f142 100644
--- a/drivers/platform/x86/intel/pmc/tgl.c
+++ b/drivers/platform/x86/intel/pmc/tgl.c
@@ -197,6 +197,8 @@ const struct pmc_reg_map tgl_reg_map = {
.ppfear_buckets = ICL_PPFEAR_NUM_ENTRIES,
.pm_cfg_offset = CNP_PMC_PM_CFG_OFFSET,
.pm_read_disable_bit = CNP_PMC_READ_DISABLE_BIT,
+ .acpi_pm_tmr_ctl_offset = SPT_PMC_ACPI_PM_TMR_CTL_OFFSET,
+ .acpi_pm_tmr_disable_bit = SPT_PMC_BIT_ACPI_PM_TMR_DISABLE,
.ltr_ignore_max = TGL_NUM_IP_IGN_ALLOWED,
.lpm_num_maps = TGL_LPM_NUM_MAPS,
.lpm_res_counter_step_x2 = TGL_PMC_LPM_RES_COUNTER_STEP_X2,
diff --git a/fs/proc/base.c b/fs/proc/base.c
index 72a1acd03675..afe573e8d8f1 100644
--- a/fs/proc/base.c
+++ b/fs/proc/base.c
@@ -2456,13 +2456,13 @@ static void *timers_start(struct seq_file *m, loff_t *pos)
if (!tp->sighand)
return ERR_PTR(-ESRCH);
- return seq_list_start(&tp->task->signal->posix_timers, *pos);
+ return seq_hlist_start(&tp->task->signal->posix_timers, *pos);
}
static void *timers_next(struct seq_file *m, void *v, loff_t *pos)
{
struct timers_private *tp = m->private;
- return seq_list_next(v, &tp->task->signal->posix_timers, pos);
+ return seq_hlist_next(v, &tp->task->signal->posix_timers, pos);
}
static void timers_stop(struct seq_file *m, void *v)
@@ -2491,7 +2491,7 @@ static int show_timer(struct seq_file *m, void *v)
[SIGEV_THREAD] = "thread",
};
- timer = list_entry((struct list_head *)v, struct k_itimer, list);
+ timer = hlist_entry((struct hlist_node *)v, struct k_itimer, list);
notify = timer->it_sigev_notify;
seq_printf(m, "ID: %d\n", timer->it_id);
@@ -2569,10 +2569,11 @@ static ssize_t timerslack_ns_write(struct file *file, const char __user *buf,
}
task_lock(p);
- if (slack_ns == 0)
- p->timer_slack_ns = p->default_timer_slack_ns;
- else
- p->timer_slack_ns = slack_ns;
+ if (task_is_realtime(p))
+ slack_ns = 0;
+ else if (slack_ns == 0)
+ slack_ns = p->default_timer_slack_ns;
+ p->timer_slack_ns = slack_ns;
task_unlock(p);
out:
diff --git a/fs/select.c b/fs/select.c
index 9515c3fa1a03..ad171b7a5c11 100644
--- a/fs/select.c
+++ b/fs/select.c
@@ -77,19 +77,16 @@ u64 select_estimate_accuracy(struct timespec64 *tv)
{
u64 ret;
struct timespec64 now;
+ u64 slack = current->timer_slack_ns;
- /*
- * Realtime tasks get a slack of 0 for obvious reasons.
- */
-
- if (rt_task(current))
+ if (slack == 0)
return 0;
ktime_get_ts64(&now);
now = timespec64_sub(*tv, now);
ret = __estimate_accuracy(&now);
- if (ret < current->timer_slack_ns)
- return current->timer_slack_ns;
+ if (ret < slack)
+ return slack;
return ret;
}
diff --git a/fs/signalfd.c b/fs/signalfd.c
index ec7b2da2477a..d0333bce015e 100644
--- a/fs/signalfd.c
+++ b/fs/signalfd.c
@@ -159,7 +159,7 @@ static ssize_t signalfd_dequeue(struct signalfd_ctx *ctx, kernel_siginfo_t *info
DECLARE_WAITQUEUE(wait, current);
spin_lock_irq(¤t->sighand->siglock);
- ret = dequeue_signal(current, &ctx->sigmask, info, &type);
+ ret = dequeue_signal(&ctx->sigmask, info, &type);
switch (ret) {
case 0:
if (!nonblock)
@@ -174,7 +174,7 @@ static ssize_t signalfd_dequeue(struct signalfd_ctx *ctx, kernel_siginfo_t *info
add_wait_queue(¤t->sighand->signalfd_wqh, &wait);
for (;;) {
set_current_state(TASK_INTERRUPTIBLE);
- ret = dequeue_signal(current, &ctx->sigmask, info, &type);
+ ret = dequeue_signal(&ctx->sigmask, info, &type);
if (ret != 0)
break;
if (signal_pending(current)) {
diff --git a/fs/xfs/xfs_buf.h b/fs/xfs/xfs_buf.h
index b1580644501f..209a389f2abc 100644
--- a/fs/xfs/xfs_buf.h
+++ b/fs/xfs/xfs_buf.h
@@ -210,7 +210,7 @@ struct xfs_buf {
* success the write is considered to be failed permanently and the
* iodone handler will take appropriate action.
*
- * For retry timeouts, we record the jiffie of the first failure. This
+ * For retry timeouts, we record the jiffy of the first failure. This
* means that we can change the retry timeout for buffers already under
* I/O and thus avoid getting stuck in a retry loop with a long timeout.
*
diff --git a/include/linux/acpi_pmtmr.h b/include/linux/acpi_pmtmr.h
index 50d88bf1498d..0ded9220d379 100644
--- a/include/linux/acpi_pmtmr.h
+++ b/include/linux/acpi_pmtmr.h
@@ -26,6 +26,19 @@ static inline u32 acpi_pm_read_early(void)
return acpi_pm_read_verified() & ACPI_PM_MASK;
}
+/**
+ * Register callback for suspend and resume event
+ *
+ * @cb Callback triggered on suspend and resume
+ * @data Data passed with the callback
+ */
+void acpi_pmtmr_register_suspend_resume_callback(void (*cb)(void *data, bool suspend), void *data);
+
+/**
+ * Remove registered callback for suspend and resume event
+ */
+void acpi_pmtmr_unregister_suspend_resume_callback(void);
+
#else
static inline u32 acpi_pm_read_early(void)
diff --git a/include/linux/jiffies.h b/include/linux/jiffies.h
index d9f1435a5a13..1220f0fbe5bf 100644
--- a/include/linux/jiffies.h
+++ b/include/linux/jiffies.h
@@ -418,7 +418,7 @@ extern unsigned long preset_lpj;
#define NSEC_CONVERSION ((unsigned long)((((u64)1 << NSEC_JIFFIE_SC) +\
TICK_NSEC -1) / (u64)TICK_NSEC))
/*
- * The maximum jiffie value is (MAX_INT >> 1). Here we translate that
+ * The maximum jiffy value is (MAX_INT >> 1). Here we translate that
* into seconds. The 64-bit case will overflow if we are not careful,
* so use the messy SH_DIV macro to do it. Still all constants.
*/
diff --git a/include/linux/posix-timers.h b/include/linux/posix-timers.h
index dc7b738de299..453691710839 100644
--- a/include/linux/posix-timers.h
+++ b/include/linux/posix-timers.h
@@ -158,7 +158,7 @@ static inline void posix_cputimers_init_work(void) { }
* @rcu: RCU head for freeing the timer.
*/
struct k_itimer {
- struct list_head list;
+ struct hlist_node list;
struct hlist_node t_hash;
spinlock_t it_lock;
const struct k_clock *kclock;
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index 0a0e23c45406..c8ed09ac29ac 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -137,7 +137,7 @@ struct signal_struct {
/* POSIX.1b Interval Timers */
unsigned int next_posix_timer_id;
- struct list_head posix_timers;
+ struct hlist_head posix_timers;
/* ITIMER_REAL timer for the process */
struct hrtimer real_timer;
@@ -276,8 +276,7 @@ static inline void signal_set_stop_flags(struct signal_struct *sig,
extern void flush_signals(struct task_struct *);
extern void ignore_signals(struct task_struct *);
extern void flush_signal_handlers(struct task_struct *, int force_default);
-extern int dequeue_signal(struct task_struct *task, sigset_t *mask,
- kernel_siginfo_t *info, enum pid_type *type);
+extern int dequeue_signal(sigset_t *mask, kernel_siginfo_t *info, enum pid_type *type);
static inline int kernel_dequeue_signal(void)
{
@@ -287,7 +286,7 @@ static inline int kernel_dequeue_signal(void)
int ret;
spin_lock_irq(&task->sighand->siglock);
- ret = dequeue_signal(task, &task->blocked, &__info, &__type);
+ ret = dequeue_signal(&task->blocked, &__info, &__type);
spin_unlock_irq(&task->sighand->siglock);
return ret;
diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h
index 84ff2844df2a..902c20ef495a 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -73,7 +73,7 @@ struct tk_read_base {
* @overflow_seen: Overflow warning flag (DEBUG_TIMEKEEPING)
*
* Note: For timespec(64) based interfaces wall_to_monotonic is what
- * we need to add to xtime (or xtime corrected for sub jiffie times)
+ * we need to add to xtime (or xtime corrected for sub jiffy times)
* to get to monotonic time. Monotonic is pegged at zero at system
* boot time, so wall_to_monotonic will be negative, however, we will
* ALWAYS keep the tv_nsec part positive so we can use the usual
diff --git a/init/init_task.c b/init/init_task.c
index eeb110c65fe2..5d0399bc8d2f 100644
--- a/init/init_task.c
+++ b/init/init_task.c
@@ -29,7 +29,7 @@ static struct signal_struct init_signals = {
.cred_guard_mutex = __MUTEX_INITIALIZER(init_signals.cred_guard_mutex),
.exec_update_lock = __RWSEM_INITIALIZER(init_signals.exec_update_lock),
#ifdef CONFIG_POSIX_TIMERS
- .posix_timers = LIST_HEAD_INIT(init_signals.posix_timers),
+ .posix_timers = HLIST_HEAD_INIT,
.cputimer = {
.cputime_atomic = INIT_CPUTIME_ATOMIC,
},
diff --git a/kernel/cpu.c b/kernel/cpu.c
index b1fd2a3db91a..5bfc2fdd7da0 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -330,7 +330,7 @@ static bool cpuhp_wait_for_sync_state(unsigned int cpu, enum cpuhp_sync_state st
/* Poll for one millisecond */
arch_cpuhp_sync_state_poll();
} else {
- usleep_range_state(USEC_PER_MSEC, 2 * USEC_PER_MSEC, TASK_UNINTERRUPTIBLE);
+ usleep_range(USEC_PER_MSEC, 2 * USEC_PER_MSEC);
}
sync = atomic_read(st);
}
diff --git a/kernel/fork.c b/kernel/fork.c
index cc760491f201..c1b343cba560 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1861,7 +1861,7 @@ static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
prev_cputime_init(&sig->prev_cputime);
#ifdef CONFIG_POSIX_TIMERS
- INIT_LIST_HEAD(&sig->posix_timers);
+ INIT_HLIST_HEAD(&sig->posix_timers);
hrtimer_init(&sig->real_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
sig->real_timer.function = it_real_fn;
#endif
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index ae1b42775ef9..195d2f2834a9 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -406,6 +406,14 @@ static void __setscheduler_params(struct task_struct *p,
else if (fair_policy(policy))
p->static_prio = NICE_TO_PRIO(attr->sched_nice);
+ /* rt-policy tasks do not have a timerslack */
+ if (task_is_realtime(p)) {
+ p->timer_slack_ns = 0;
+ } else if (p->timer_slack_ns == 0) {
+ /* when switching back to non-rt policy, restore timerslack */
+ p->timer_slack_ns = p->default_timer_slack_ns;
+ }
+
/*
* __sched_setscheduler() ensures attr->sched_priority == 0 when
* !rt_policy. Always setting this ensures that things like
diff --git a/kernel/signal.c b/kernel/signal.c
index 60c737e423a1..6f3a5aa39b09 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -618,20 +618,18 @@ static int __dequeue_signal(struct sigpending *pending, sigset_t *mask,
}
/*
- * Dequeue a signal and return the element to the caller, which is
- * expected to free it.
- *
- * All callers have to hold the siglock.
+ * Try to dequeue a signal. If a deliverable signal is found fill in the
+ * caller provided siginfo and return the signal number. Otherwise return
+ * 0.
*/
-int dequeue_signal(struct task_struct *tsk, sigset_t *mask,
- kernel_siginfo_t *info, enum pid_type *type)
+int dequeue_signal(sigset_t *mask, kernel_siginfo_t *info, enum pid_type *type)
{
+ struct task_struct *tsk = current;
bool resched_timer = false;
int signr;
- /* We only dequeue private signals from ourselves, we don't let
- * signalfd steal them
- */
+ lockdep_assert_held(&tsk->sighand->siglock);
+
*type = PIDTYPE_PID;
signr = __dequeue_signal(&tsk->pending, mask, info, &resched_timer);
if (!signr) {
@@ -1940,10 +1938,11 @@ struct sigqueue *sigqueue_alloc(void)
void sigqueue_free(struct sigqueue *q)
{
- unsigned long flags;
spinlock_t *lock = ¤t->sighand->siglock;
+ unsigned long flags;
- BUG_ON(!(q->flags & SIGQUEUE_PREALLOC));
+ if (WARN_ON_ONCE(!(q->flags & SIGQUEUE_PREALLOC)))
+ return;
/*
* We must hold ->siglock while testing q->list
* to serialize with collect_signal() or with
@@ -1971,7 +1970,10 @@ int send_sigqueue(struct sigqueue *q, struct pid *pid, enum pid_type type)
unsigned long flags;
int ret, result;
- BUG_ON(!(q->flags & SIGQUEUE_PREALLOC));
+ if (WARN_ON_ONCE(!(q->flags & SIGQUEUE_PREALLOC)))
+ return 0;
+ if (WARN_ON_ONCE(q->info.si_code != SI_TIMER))
+ return 0;
ret = -1;
rcu_read_lock();
@@ -2006,7 +2008,6 @@ int send_sigqueue(struct sigqueue *q, struct pid *pid, enum pid_type type)
* If an SI_TIMER entry is already queue just increment
* the overrun count.
*/
- BUG_ON(q->info.si_code != SI_TIMER);
q->info.si_overrun++;
result = TRACE_SIGNAL_ALREADY_PENDING;
goto out;
@@ -2793,8 +2794,7 @@ bool get_signal(struct ksignal *ksig)
type = PIDTYPE_PID;
signr = dequeue_synchronous_signal(&ksig->info);
if (!signr)
- signr = dequeue_signal(current, ¤t->blocked,
- &ksig->info, &type);
+ signr = dequeue_signal(¤t->blocked, &ksig->info, &type);
if (!signr)
break; /* will return 0 */
@@ -3648,7 +3648,7 @@ static int do_sigtimedwait(const sigset_t *which, kernel_siginfo_t *info,
signotset(&mask);
spin_lock_irq(&tsk->sighand->siglock);
- sig = dequeue_signal(tsk, &mask, info, &type);
+ sig = dequeue_signal(&mask, info, &type);
if (!sig && timeout) {
/*
* None ready, temporarily unblock those we're interested
@@ -3667,7 +3667,7 @@ static int do_sigtimedwait(const sigset_t *which, kernel_siginfo_t *info,
spin_lock_irq(&tsk->sighand->siglock);
__set_task_blocked(tsk, &tsk->real_blocked);
sigemptyset(&tsk->real_blocked);
- sig = dequeue_signal(tsk, &mask, info, &type);
+ sig = dequeue_signal(&mask, info, &type);
}
spin_unlock_irq(&tsk->sighand->siglock);
diff --git a/kernel/sys.c b/kernel/sys.c
index 3a2df1bd9f64..e3c4cffb520c 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -2557,6 +2557,8 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
error = current->timer_slack_ns;
break;
case PR_SET_TIMERSLACK:
+ if (task_is_realtime(current))
+ break;
if (arg2 <= 0)
current->timer_slack_ns =
current->default_timer_slack_ns;
diff --git a/kernel/time/alarmtimer.c b/kernel/time/alarmtimer.c
index 5abfa4390673..8bf888641694 100644
--- a/kernel/time/alarmtimer.c
+++ b/kernel/time/alarmtimer.c
@@ -493,7 +493,7 @@ static u64 __alarm_forward_now(struct alarm *alarm, ktime_t interval, bool throt
* promised in the context of posix_timer_fn() never
* materialized, but someone should really work on it.
*
- * To prevent DOS fake @now to be 1 jiffie out which keeps
+ * To prevent DOS fake @now to be 1 jiffy out which keeps
* the overrun accounting correct but creates an
* inconsistency vs. timer_gettime(2).
*/
@@ -574,15 +574,10 @@ static enum alarmtimer_restart alarm_handle_timer(struct alarm *alarm,
it.alarm.alarmtimer);
enum alarmtimer_restart result = ALARMTIMER_NORESTART;
unsigned long flags;
- int si_private = 0;
spin_lock_irqsave(&ptr->it_lock, flags);
- ptr->it_active = 0;
- if (ptr->it_interval)
- si_private = ++ptr->it_requeue_pending;
-
- if (posix_timer_event(ptr, si_private) && ptr->it_interval) {
+ if (posix_timer_queue_signal(ptr) && ptr->it_interval) {
/*
* Handle ignored signals and rearm the timer. This will go
* away once we handle ignored signals proper. Ensure that
diff --git a/kernel/time/clockevents.c b/kernel/time/clockevents.c
index 60a6484831b1..78c7bd64d0dd 100644
--- a/kernel/time/clockevents.c
+++ b/kernel/time/clockevents.c
@@ -190,7 +190,7 @@ int clockevents_tick_resume(struct clock_event_device *dev)
#ifdef CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST
-/* Limit min_delta to a jiffie */
+/* Limit min_delta to a jiffy */
#define MIN_DELTA_LIMIT (NSEC_PER_SEC / HZ)
/**
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index b8ee320208d4..e834b2bd83df 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -1177,7 +1177,7 @@ static inline ktime_t hrtimer_update_lowres(struct hrtimer *timer, ktime_t tim,
/*
* CONFIG_TIME_LOW_RES indicates that the system has no way to return
* granular time values. For relative timers we add hrtimer_resolution
- * (i.e. one jiffie) to prevent short timeouts.
+ * (i.e. one jiffy) to prevent short timeouts.
*/
timer->is_rel = mode & HRTIMER_MODE_REL;
if (timer->is_rel)
@@ -1351,11 +1351,13 @@ static void hrtimer_cpu_base_init_expiry_lock(struct hrtimer_cpu_base *base)
}
static void hrtimer_cpu_base_lock_expiry(struct hrtimer_cpu_base *base)
+ __acquires(&base->softirq_expiry_lock)
{
spin_lock(&base->softirq_expiry_lock);
}
static void hrtimer_cpu_base_unlock_expiry(struct hrtimer_cpu_base *base)
+ __releases(&base->softirq_expiry_lock)
{
spin_unlock(&base->softirq_expiry_lock);
}
@@ -2072,14 +2074,9 @@ long hrtimer_nanosleep(ktime_t rqtp, const enum hrtimer_mode mode,
struct restart_block *restart;
struct hrtimer_sleeper t;
int ret = 0;
- u64 slack;
-
- slack = current->timer_slack_ns;
- if (rt_task(current))
- slack = 0;
hrtimer_init_sleeper_on_stack(&t, clockid, mode);
- hrtimer_set_expires_range_ns(&t.timer, rqtp, slack);
+ hrtimer_set_expires_range_ns(&t.timer, rqtp, current->timer_slack_ns);
ret = do_nanosleep(&t, mode);
if (ret != -ERESTART_RESTARTBLOCK)
goto out;
@@ -2249,7 +2246,7 @@ void __init hrtimers_init(void)
/**
* schedule_hrtimeout_range_clock - sleep until timeout
* @expires: timeout value (ktime_t)
- * @delta: slack in expires timeout (ktime_t) for SCHED_OTHER tasks
+ * @delta: slack in expires timeout (ktime_t)
* @mode: timer mode
* @clock_id: timer clock to be used
*/
@@ -2276,13 +2273,6 @@ schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
return -EINTR;
}
- /*
- * Override any slack passed by the user if under
- * rt contraints.
- */
- if (rt_task(current))
- delta = 0;
-
hrtimer_init_sleeper_on_stack(&t, clock_id, mode);
hrtimer_set_expires_range_ns(&t.timer, *expires, delta);
hrtimer_sleeper_start_expires(&t, mode);
@@ -2302,7 +2292,7 @@ EXPORT_SYMBOL_GPL(schedule_hrtimeout_range_clock);
/**
* schedule_hrtimeout_range - sleep until timeout
* @expires: timeout value (ktime_t)
- * @delta: slack in expires timeout (ktime_t) for SCHED_OTHER tasks
+ * @delta: slack in expires timeout (ktime_t)
* @mode: timer mode
*
* Make the current task sleep until the given expiry time has
diff --git a/kernel/time/ntp.c b/kernel/time/ntp.c
index 8d2dd214ec68..802b336f4b8c 100644
--- a/kernel/time/ntp.c
+++ b/kernel/time/ntp.c
@@ -660,8 +660,16 @@ static void sync_hw_clock(struct work_struct *work)
sched_sync_hw_clock(offset_nsec, res != 0);
}
-void ntp_notify_cmos_timer(void)
+void ntp_notify_cmos_timer(bool offset_set)
{
+ /*
+ * If the time jumped (using ADJ_SETOFFSET) cancels sync timer,
+ * which may have been running if the time was synchronized
+ * prior to the ADJ_SETOFFSET call.
+ */
+ if (offset_set)
+ hrtimer_cancel(&sync_hrtimer);
+
/*
* When the work is currently executed but has not yet the timer
* rearmed this queues the work immediately again. No big issue,
diff --git a/kernel/time/ntp_internal.h b/kernel/time/ntp_internal.h
index 23d1b74c3065..5a633dce9057 100644
--- a/kernel/time/ntp_internal.h
+++ b/kernel/time/ntp_internal.h
@@ -14,9 +14,9 @@ extern int __do_adjtimex(struct __kernel_timex *txc,
extern void __hardpps(const struct timespec64 *phase_ts, const struct timespec64 *raw_ts);
#if defined(CONFIG_GENERIC_CMOS_UPDATE) || defined(CONFIG_RTC_SYSTOHC)
-extern void ntp_notify_cmos_timer(void);
+extern void ntp_notify_cmos_timer(bool offset_set);
#else
-static inline void ntp_notify_cmos_timer(void) { }
+static inline void ntp_notify_cmos_timer(bool offset_set) { }
#endif
#endif /* _LINUX_NTP_INTERNAL_H */
diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c
index e9c6f9d0e42c..6bcee4704059 100644
--- a/kernel/time/posix-cpu-timers.c
+++ b/kernel/time/posix-cpu-timers.c
@@ -453,6 +453,7 @@ static void disarm_timer(struct k_itimer *timer, struct task_struct *p)
struct cpu_timer *ctmr = &timer->it.cpu;
struct posix_cputimer_base *base;
+ timer->it_active = 0;
if (!cpu_timer_dequeue(ctmr))
return;
@@ -559,6 +560,7 @@ static void arm_timer(struct k_itimer *timer, struct task_struct *p)
struct cpu_timer *ctmr = &timer->it.cpu;
u64 newexp = cpu_timer_getexpires(ctmr);
+ timer->it_active = 1;
if (!cpu_timer_enqueue(&base->tqhead, ctmr))
return;
@@ -584,12 +586,8 @@ static void cpu_timer_fire(struct k_itimer *timer)
{
struct cpu_timer *ctmr = &timer->it.cpu;
- if ((timer->it_sigev_notify & ~SIGEV_THREAD_ID) == SIGEV_NONE) {
- /*
- * User don't want any signal.
- */
- cpu_timer_setexpires(ctmr, 0);
- } else if (unlikely(timer->sigq == NULL)) {
+ timer->it_active = 0;
+ if (unlikely(timer->sigq == NULL)) {
/*
* This a special case for clock_nanosleep,
* not a normal timer from sys_timer_create.
@@ -600,9 +598,9 @@ static void cpu_timer_fire(struct k_itimer *timer)
/*
* One-shot timer. Clear it as soon as it's fired.
*/
- posix_timer_event(timer, 0);
+ posix_timer_queue_signal(timer);
cpu_timer_setexpires(ctmr, 0);
- } else if (posix_timer_event(timer, ++timer->it_requeue_pending)) {
+ } else if (posix_timer_queue_signal(timer)) {
/*
* The signal did not get queued because the signal
* was ignored, so we won't get any callback to
@@ -614,6 +612,8 @@ static void cpu_timer_fire(struct k_itimer *timer)
}
}
+static void __posix_cpu_timer_get(struct k_itimer *timer, struct itimerspec64 *itp, u64 now);
+
/*
* Guts of sys_timer_settime for CPU timers.
* This is called with the timer locked and interrupts disabled.
@@ -623,9 +623,10 @@ static void cpu_timer_fire(struct k_itimer *timer)
static int posix_cpu_timer_set(struct k_itimer *timer, int timer_flags,
struct itimerspec64 *new, struct itimerspec64 *old)
{
+ bool sigev_none = timer->it_sigev_notify == SIGEV_NONE;
clockid_t clkid = CPUCLOCK_WHICH(timer->it_clock);
- u64 old_expires, new_expires, old_incr, val;
struct cpu_timer *ctmr = &timer->it.cpu;
+ u64 old_expires, new_expires, now;
struct sighand_struct *sighand;
struct task_struct *p;
unsigned long flags;
@@ -662,10 +663,7 @@ static int posix_cpu_timer_set(struct k_itimer *timer, int timer_flags,
return -ESRCH;
}
- /*
- * Disarm any old timer after extracting its expiry time.
- */
- old_incr = timer->it_interval;
+ /* Retrieve the current expiry time before disarming the timer */
old_expires = cpu_timer_getexpires(ctmr);
if (unlikely(timer->it.cpu.firing)) {
@@ -673,157 +671,122 @@ static int posix_cpu_timer_set(struct k_itimer *timer, int timer_flags,
ret = TIMER_RETRY;
} else {
cpu_timer_dequeue(ctmr);
+ timer->it_active = 0;
}
/*
- * We need to sample the current value to convert the new
- * value from to relative and absolute, and to convert the
- * old value from absolute to relative. To set a process
- * timer, we need a sample to balance the thread expiry
- * times (in arm_timer). With an absolute time, we must
- * check if it's already passed. In short, we need a sample.
+ * Sample the current clock for saving the previous setting
+ * and for rearming the timer.
*/
if (CPUCLOCK_PERTHREAD(timer->it_clock))
- val = cpu_clock_sample(clkid, p);
+ now = cpu_clock_sample(clkid, p);
else
- val = cpu_clock_sample_group(clkid, p, true);
+ now = cpu_clock_sample_group(clkid, p, !sigev_none);
+ /* Retrieve the previous expiry value if requested. */
if (old) {
- if (old_expires == 0) {
- old->it_value.tv_sec = 0;
- old->it_value.tv_nsec = 0;
- } else {
- /*
- * Update the timer in case it has overrun already.
- * If it has, we'll report it as having overrun and
- * with the next reloaded timer already ticking,
- * though we are swallowing that pending
- * notification here to install the new setting.
- */
- u64 exp = bump_cpu_timer(timer, val);
-
- if (val < exp) {
- old_expires = exp - val;
- old->it_value = ns_to_timespec64(old_expires);
- } else {
- old->it_value.tv_nsec = 1;
- old->it_value.tv_sec = 0;
- }
- }
+ old->it_value = (struct timespec64){ };
+ if (old_expires)
+ __posix_cpu_timer_get(timer, old, now);
}
+ /* Retry if the timer expiry is running concurrently */
if (unlikely(ret)) {
- /*
- * We are colliding with the timer actually firing.
- * Punt after filling in the timer's old value, and
- * disable this firing since we are already reporting
- * it as an overrun (thanks to bump_cpu_timer above).
- */
unlock_task_sighand(p, &flags);
goto out;
}
- if (new_expires != 0 && !(timer_flags & TIMER_ABSTIME)) {
- new_expires += val;
- }
+ /* Convert relative expiry time to absolute */
+ if (new_expires && !(timer_flags & TIMER_ABSTIME))
+ new_expires += now;
+
+ /* Set the new expiry time (might be 0) */
+ cpu_timer_setexpires(ctmr, new_expires);
/*
- * Install the new expiry time (or zero).
- * For a timer with no notification action, we don't actually
- * arm the timer (we'll just fake it for timer_gettime).
+ * Arm the timer if it is not disabled, the new expiry value has
+ * not yet expired and the timer requires signal delivery.
+ * SIGEV_NONE timers are never armed. In case the timer is not
+ * armed, enforce the reevaluation of the timer base so that the
+ * process wide cputime counter can be disabled eventually.
*/
- cpu_timer_setexpires(ctmr, new_expires);
- if (new_expires != 0 && val < new_expires) {
- arm_timer(timer, p);
+ if (likely(!sigev_none)) {
+ if (new_expires && now < new_expires)
+ arm_timer(timer, p);
+ else
+ trigger_base_recalc_expires(timer, p);
}
unlock_task_sighand(p, &flags);
+
+ posix_timer_set_common(timer, new);
+
/*
- * Install the new reload setting, and
- * set up the signal and overrun bookkeeping.
+ * If the new expiry time was already in the past the timer was not
+ * queued. Fire it immediately even if the thread never runs to
+ * accumulate more time on this clock.
*/
- timer->it_interval = timespec64_to_ktime(new->it_interval);
+ if (!sigev_none && new_expires && now >= new_expires)
+ cpu_timer_fire(timer);
+out:
+ rcu_read_unlock();
+ return ret;
+}
+
+static void __posix_cpu_timer_get(struct k_itimer *timer, struct itimerspec64 *itp, u64 now)
+{
+ bool sigev_none = timer->it_sigev_notify == SIGEV_NONE;
+ u64 expires, iv = timer->it_interval;
/*
- * This acts as a modification timestamp for the timer,
- * so any automatic reload attempt will punt on seeing
- * that we have reset the timer manually.
+ * Make sure that interval timers are moved forward for the
+ * following cases:
+ * - SIGEV_NONE timers which are never armed
+ * - Timers which expired, but the signal has not yet been
+ * delivered
*/
- timer->it_requeue_pending = (timer->it_requeue_pending + 2) &
- ~REQUEUE_PENDING;
- timer->it_overrun_last = 0;
- timer->it_overrun = -1;
-
- if (val >= new_expires) {
- if (new_expires != 0) {
- /*
- * The designated time already passed, so we notify
- * immediately, even if the thread never runs to
- * accumulate more time on this clock.
- */
- cpu_timer_fire(timer);
- }
+ if (iv && ((timer->it_requeue_pending & REQUEUE_PENDING) || sigev_none))
+ expires = bump_cpu_timer(timer, now);
+ else
+ expires = cpu_timer_getexpires(&timer->it.cpu);
+ /*
+ * Expired interval timers cannot have a remaining time <= 0.
+ * The kernel has to move them forward so that the next
+ * timer expiry is > @now.
+ */
+ if (now < expires) {
+ itp->it_value = ns_to_timespec64(expires - now);
+ } else {
/*
- * Make sure we don't keep around the process wide cputime
- * counter or the tick dependency if they are not necessary.
+ * A single shot SIGEV_NONE timer must return 0, when it is
+ * expired! Timers which have a real signal delivery mode
+ * must return a remaining time greater than 0 because the
+ * signal has not yet been delivered.
*/
- sighand = lock_task_sighand(p, &flags);
- if (!sighand)
- goto out;
-
- if (!cpu_timer_queued(ctmr))
- trigger_base_recalc_expires(timer, p);
-
- unlock_task_sighand(p, &flags);
+ if (!sigev_none)
+ itp->it_value.tv_nsec = 1;
}
- out:
- rcu_read_unlock();
- if (old)
- old->it_interval = ns_to_timespec64(old_incr);
-
- return ret;
}
static void posix_cpu_timer_get(struct k_itimer *timer, struct itimerspec64 *itp)
{
clockid_t clkid = CPUCLOCK_WHICH(timer->it_clock);
- struct cpu_timer *ctmr = &timer->it.cpu;
- u64 now, expires = cpu_timer_getexpires(ctmr);
struct task_struct *p;
+ u64 now;
rcu_read_lock();
p = cpu_timer_task_rcu(timer);
- if (!p)
- goto out;
+ if (p && cpu_timer_getexpires(&timer->it.cpu)) {
+ itp->it_interval = ktime_to_timespec64(timer->it_interval);
- /*
- * Easy part: convert the reload time.
- */
- itp->it_interval = ktime_to_timespec64(timer->it_interval);
-
- if (!expires)
- goto out;
-
- /*
- * Sample the clock to take the difference with the expiry time.
- */
- if (CPUCLOCK_PERTHREAD(timer->it_clock))
- now = cpu_clock_sample(clkid, p);
- else
- now = cpu_clock_sample_group(clkid, p, false);
+ if (CPUCLOCK_PERTHREAD(timer->it_clock))
+ now = cpu_clock_sample(clkid, p);
+ else
+ now = cpu_clock_sample_group(clkid, p, false);
- if (now < expires) {
- itp->it_value = ns_to_timespec64(expires - now);
- } else {
- /*
- * The timer should have expired already, but the firing
- * hasn't taken place yet. Say it's just about to expire.
- */
- itp->it_value.tv_nsec = 1;
- itp->it_value.tv_sec = 0;
+ __posix_cpu_timer_get(timer, itp, now);
}
-out:
rcu_read_unlock();
}
diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
index b924f0f096fa..4576aaed13b2 100644
--- a/kernel/time/posix-timers.c
+++ b/kernel/time/posix-timers.c
@@ -277,10 +277,17 @@ void posixtimer_rearm(struct kernel_siginfo *info)
unlock_timer(timr, flags);
}
-int posix_timer_event(struct k_itimer *timr, int si_private)
+int posix_timer_queue_signal(struct k_itimer *timr)
{
+ int ret, si_private = 0;
enum pid_type type;
- int ret;
+
+ lockdep_assert_held(&timr->it_lock);
+
+ timr->it_active = 0;
+ if (timr->it_interval)
+ si_private = ++timr->it_requeue_pending;
+
/*
* FIXME: if ->sigq is queued we can race with
* dequeue_signal()->posixtimer_rearm().
@@ -309,19 +316,13 @@ int posix_timer_event(struct k_itimer *timr, int si_private)
*/
static enum hrtimer_restart posix_timer_fn(struct hrtimer *timer)
{
+ struct k_itimer *timr = container_of(timer, struct k_itimer, it.real.timer);
enum hrtimer_restart ret = HRTIMER_NORESTART;
- struct k_itimer *timr;
unsigned long flags;
- int si_private = 0;
- timr = container_of(timer, struct k_itimer, it.real.timer);
spin_lock_irqsave(&timr->it_lock, flags);
- timr->it_active = 0;
- if (timr->it_interval != 0)
- si_private = ++timr->it_requeue_pending;
-
- if (posix_timer_event(timr, si_private)) {
+ if (posix_timer_queue_signal(timr)) {
/*
* The signal was not queued due to SIG_IGN. As a
* consequence the timer is not going to be rearmed from
@@ -338,14 +339,14 @@ static enum hrtimer_restart posix_timer_fn(struct hrtimer *timer)
* change to the signal handling code.
*
* For now let timers with an interval less than a
- * jiffie expire every jiffie and recheck for a
+ * jiffy expire every jiffy and recheck for a
* valid signal handler.
*
* This avoids interrupt starvation in case of a
* very small interval, which would expire the
* timer immediately again.
*
- * Moving now ahead of time by one jiffie tricks
+ * Moving now ahead of time by one jiffy tricks
* hrtimer_forward() to expire the timer later,
* while it still maintains the overrun accuracy
* for the price of a slight inconsistency in the
@@ -515,7 +516,7 @@ static int do_timer_create(clockid_t which_clock, struct sigevent *event,
spin_lock_irq(¤t->sighand->siglock);
/* This makes the timer valid in the hash table */
WRITE_ONCE(new_timer->it_signal, current->signal);
- list_add(&new_timer->list, ¤t->signal->posix_timers);
+ hlist_add_head(&new_timer->list, ¤t->signal->posix_timers);
spin_unlock_irq(¤t->sighand->siglock);
/*
* After unlocking sighand::siglock @new_timer is subject to
@@ -856,6 +857,23 @@ static struct k_itimer *timer_wait_running(struct k_itimer *timer,
return lock_timer(timer_id, flags);
}
+/*
+ * Set up the new interval and reset the signal delivery data
+ */
+void posix_timer_set_common(struct k_itimer *timer, struct itimerspec64 *new_setting)
+{
+ if (new_setting->it_value.tv_sec || new_setting->it_value.tv_nsec)
+ timer->it_interval = timespec64_to_ktime(new_setting->it_interval);
+ else
+ timer->it_interval = 0;
+
+ /* Prevent reloading in case there is a signal pending */
+ timer->it_requeue_pending = (timer->it_requeue_pending + 2) & ~REQUEUE_PENDING;
+ /* Reset overrun accounting */
+ timer->it_overrun_last = 0;
+ timer->it_overrun = -1LL;
+}
+
/* Set a POSIX.1b interval timer. */
int common_timer_set(struct k_itimer *timr, int flags,
struct itimerspec64 *new_setting,
@@ -878,15 +896,12 @@ int common_timer_set(struct k_itimer *timr, int flags,
return TIMER_RETRY;
timr->it_active = 0;
- timr->it_requeue_pending = (timr->it_requeue_pending + 2) &
- ~REQUEUE_PENDING;
- timr->it_overrun_last = 0;
+ posix_timer_set_common(timr, new_setting);
- /* Switch off the timer when it_value is zero */
+ /* Keep timer disarmed when it_value is zero */
if (!new_setting->it_value.tv_sec && !new_setting->it_value.tv_nsec)
return 0;
- timr->it_interval = timespec64_to_ktime(new_setting->it_interval);
expires = timespec64_to_ktime(new_setting->it_value);
if (flags & TIMER_ABSTIME)
expires = timens_ktime_to_host(timr->it_clock, expires);
@@ -904,7 +919,7 @@ static int do_timer_settime(timer_t timer_id, int tmr_flags,
const struct k_clock *kc;
struct k_itimer *timr;
unsigned long flags;
- int error = 0;
+ int error;
if (!timespec64_valid(&new_spec64->it_interval) ||
!timespec64_valid(&new_spec64->it_value))
@@ -918,6 +933,9 @@ static int do_timer_settime(timer_t timer_id, int tmr_flags,
if (!timr)
return -EINVAL;
+ if (old_spec64)
+ old_spec64->it_interval = ktime_to_timespec64(timr->it_interval);
+
kc = timr->kclock;
if (WARN_ON_ONCE(!kc || !kc->timer_set))
error = -EINVAL;
@@ -1021,7 +1039,7 @@ SYSCALL_DEFINE1(timer_delete, timer_t, timer_id)
}
spin_lock(¤t->sighand->siglock);
- list_del(&timer->list);
+ hlist_del(&timer->list);
spin_unlock(¤t->sighand->siglock);
/*
* A concurrent lookup could check timer::it_signal lockless. It
@@ -1071,7 +1089,7 @@ static void itimer_delete(struct k_itimer *timer)
goto retry_delete;
}
- list_del(&timer->list);
+ hlist_del(&timer->list);
/*
* Setting timer::it_signal to NULL is technically not required
@@ -1092,22 +1110,19 @@ static void itimer_delete(struct k_itimer *timer)
*/
void exit_itimers(struct task_struct *tsk)
{
- struct list_head timers;
- struct k_itimer *tmr;
+ struct hlist_head timers;
- if (list_empty(&tsk->signal->posix_timers))
+ if (hlist_empty(&tsk->signal->posix_timers))
return;
/* Protect against concurrent read via /proc/$PID/timers */
spin_lock_irq(&tsk->sighand->siglock);
- list_replace_init(&tsk->signal->posix_timers, &timers);
+ hlist_move_list(&tsk->signal->posix_timers, &timers);
spin_unlock_irq(&tsk->sighand->siglock);
/* The timers are not longer accessible via tsk::signal */
- while (!list_empty(&timers)) {
- tmr = list_first_entry(&timers, struct k_itimer, list);
- itimer_delete(tmr);
- }
+ while (!hlist_empty(&timers))
+ itimer_delete(hlist_entry(timers.first, struct k_itimer, list));
}
SYSCALL_DEFINE2(clock_settime, const clockid_t, which_clock,
diff --git a/kernel/time/posix-timers.h b/kernel/time/posix-timers.h
index f32a2ebba9b8..4784ea65f685 100644
--- a/kernel/time/posix-timers.h
+++ b/kernel/time/posix-timers.h
@@ -36,10 +36,11 @@ extern const struct k_clock clock_process;
extern const struct k_clock clock_thread;
extern const struct k_clock alarm_clock;
-int posix_timer_event(struct k_itimer *timr, int si_private);
+int posix_timer_queue_signal(struct k_itimer *timr);
void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting);
int common_timer_set(struct k_itimer *timr, int flags,
struct itimerspec64 *new_setting,
struct itimerspec64 *old_setting);
+void posix_timer_set_common(struct k_itimer *timer, struct itimerspec64 *new_setting);
int common_timer_del(struct k_itimer *timer);
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 5391e4167d60..7e6f409bf311 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -2553,6 +2553,7 @@ int do_adjtimex(struct __kernel_timex *txc)
{
struct timekeeper *tk = &tk_core.timekeeper;
struct audit_ntp_data ad;
+ bool offset_set = false;
bool clock_set = false;
struct timespec64 ts;
unsigned long flags;
@@ -2575,6 +2576,7 @@ int do_adjtimex(struct __kernel_timex *txc)
if (ret)
return ret;
+ offset_set = delta.tv_sec != 0;
audit_tk_injoffset(delta);
}
@@ -2608,7 +2610,7 @@ int do_adjtimex(struct __kernel_timex *txc)
if (clock_set)
clock_was_set(CLOCK_SET_WALL);
- ntp_notify_cmos_timer();
+ ntp_notify_cmos_timer(offset_set);
return ret;
}
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 64b0d8a0aa0f..2b38f3035a3e 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -365,7 +365,7 @@ static unsigned long round_jiffies_common(unsigned long j, int cpu,
rem = j % HZ;
/*
- * If the target jiffie is just after a whole second (which can happen
+ * If the target jiffy is just after a whole second (which can happen
* due to delays of the timer irq, long irq off times etc etc) then
* we should round down to the whole second, not up. Use 1/4th second
* as cutoff for this rounding as an extreme upper bound for this.
@@ -672,7 +672,7 @@ static void enqueue_timer(struct timer_base *base, struct timer_list *timer,
* Set the next expiry time and kick the CPU so it
* can reevaluate the wheel:
*/
- base->next_expiry = bucket_expiry;
+ WRITE_ONCE(base->next_expiry, bucket_expiry);
base->timers_pending = true;
base->next_expiry_recalc = false;
trigger_dyntick_cpu(base, timer);
@@ -1561,6 +1561,8 @@ static inline void timer_base_unlock_expiry(struct timer_base *base)
* the waiter to acquire the lock and make progress.
*/
static void timer_sync_wait_running(struct timer_base *base)
+ __releases(&base->lock) __releases(&base->expiry_lock)
+ __acquires(&base->expiry_lock) __acquires(&base->lock)
{
if (atomic_read(&base->timer_waiters)) {
raw_spin_unlock_irq(&base->lock);
@@ -1898,7 +1900,7 @@ static int next_pending_bucket(struct timer_base *base, unsigned offset,
*
* Store next expiry time in base->next_expiry.
*/
-static void next_expiry_recalc(struct timer_base *base)
+static void timer_recalc_next_expiry(struct timer_base *base)
{
unsigned long clk, next, adj;
unsigned lvl, offset = 0;
@@ -1928,7 +1930,7 @@ static void next_expiry_recalc(struct timer_base *base)
* bits are zero, we look at the next level as is. If not we
* need to advance it by one because that's going to be the
* next expiring bucket in that level. base->clk is the next
- * expiring jiffie. So in case of:
+ * expiring jiffy. So in case of:
*
* LVL5 LVL4 LVL3 LVL2 LVL1 LVL0
* 0 0 0 0 0 0
@@ -1964,7 +1966,7 @@ static void next_expiry_recalc(struct timer_base *base)
clk += adj;
}
- base->next_expiry = next;
+ WRITE_ONCE(base->next_expiry, next);
base->next_expiry_recalc = false;
base->timers_pending = !(next == base->clk + NEXT_TIMER_MAX_DELTA);
}
@@ -1993,7 +1995,7 @@ static u64 cmp_next_hrtimer_event(u64 basem, u64 expires)
return basem;
/*
- * Round up to the next jiffie. High resolution timers are
+ * Round up to the next jiffy. High resolution timers are
* off, so the hrtimers are expired in the tick and we need to
* make sure that this tick really expires the timer to avoid
* a ping pong of the nohz stop code.
@@ -2007,7 +2009,7 @@ static unsigned long next_timer_interrupt(struct timer_base *base,
unsigned long basej)
{
if (base->next_expiry_recalc)
- next_expiry_recalc(base);
+ timer_recalc_next_expiry(base);
/*
* Move next_expiry for the empty base into the future to prevent an
@@ -2018,7 +2020,7 @@ static unsigned long next_timer_interrupt(struct timer_base *base,
* easy comparable to find out which base holds the first pending timer.
*/
if (!base->timers_pending)
- base->next_expiry = basej + NEXT_TIMER_MAX_DELTA;
+ WRITE_ONCE(base->next_expiry, basej + NEXT_TIMER_MAX_DELTA);
return base->next_expiry;
}
@@ -2252,7 +2254,7 @@ static inline u64 __get_next_timer_interrupt(unsigned long basej, u64 basem,
base_global, &tevt);
/*
- * If the next event is only one jiffie ahead there is no need to call
+ * If the next event is only one jiffy ahead there is no need to call
* timer migration hierarchy related functions. The value for the next
* global timer in @tevt struct equals then KTIME_MAX. This is also
* true, when the timer base is idle.
@@ -2411,7 +2413,7 @@ static inline void __run_timers(struct timer_base *base)
* jiffies to avoid endless requeuing to current jiffies.
*/
base->clk++;
- next_expiry_recalc(base);
+ timer_recalc_next_expiry(base);
while (levels--)
expire_timers(base, heads + levels);
@@ -2462,8 +2464,40 @@ static void run_local_timers(void)
hrtimer_run_queues();
for (int i = 0; i < NR_BASES; i++, base++) {
- /* Raise the softirq only if required. */
- if (time_after_eq(jiffies, base->next_expiry) ||
+ /*
+ * Raise the softirq only if required.
+ *
+ * timer_base::next_expiry can be written by a remote CPU while
+ * holding the lock. If this write happens at the same time than
+ * the lockless local read, sanity checker could complain about
+ * data corruption.
+ *
+ * There are two possible situations where
+ * timer_base::next_expiry is written by a remote CPU:
+ *
+ * 1. Remote CPU expires global timers of this CPU and updates
+ * timer_base::next_expiry of BASE_GLOBAL afterwards in
+ * next_timer_interrupt() or timer_recalc_next_expiry(). The
+ * worst outcome is a superfluous raise of the timer softirq
+ * when the not yet updated value is read.
+ *
+ * 2. A new first pinned timer is enqueued by a remote CPU
+ * and therefore timer_base::next_expiry of BASE_LOCAL is
+ * updated. When this update is missed, this isn't a
+ * problem, as an IPI is executed nevertheless when the CPU
+ * was idle before. When the CPU wasn't idle but the update
+ * is missed, then the timer would expire one jiffy late -
+ * bad luck.
+ *
+ * Those unlikely corner cases where the worst outcome is only a
+ * one jiffy delay or a superfluous raise of the softirq are
+ * not that expensive as doing the check always while holding
+ * the lock.
+ *
+ * Possible remote writers are using WRITE_ONCE(). Local reader
+ * uses therefore READ_ONCE().
+ */
+ if (time_after_eq(jiffies, READ_ONCE(base->next_expiry)) ||
(i == BASE_DEF && tmigr_requires_handle_remote())) {
raise_softirq(TIMER_SOFTIRQ);
return;
@@ -2730,7 +2764,7 @@ void __init init_timers(void)
*/
void msleep(unsigned int msecs)
{
- unsigned long timeout = msecs_to_jiffies(msecs) + 1;
+ unsigned long timeout = msecs_to_jiffies(msecs);
while (timeout)
timeout = schedule_timeout_uninterruptible(timeout);
@@ -2744,7 +2778,7 @@ EXPORT_SYMBOL(msleep);
*/
unsigned long msleep_interruptible(unsigned int msecs)
{
- unsigned long timeout = msecs_to_jiffies(msecs) + 1;
+ unsigned long timeout = msecs_to_jiffies(msecs);
while (timeout && !signal_pending(current))
timeout = schedule_timeout_interruptible(timeout);
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index a30c03a66172..a40aa606cd04 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -97,7 +97,7 @@ config BOOT_PRINTK_DELAY
using "boot_delay=N".
It is likely that you would also need to use "lpj=M" to preset
- the "loops per jiffie" value.
+ the "loops per jiffy" value.
See a previous boot log for the "lpj" value to use for your
system, and then set "lpj=M" before setting "boot_delay=N".
NOTE: Using this option may adversely affect SMP systems.
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 00840d5784fe..04f6398b3a40 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -287,7 +287,7 @@ struct batadv_frag_table_entry {
/** @lock: lock to protect the list of fragments */
spinlock_t lock;
- /** @timestamp: time (jiffie) of last received fragment */
+ /** @timestamp: time (jiffy) of last received fragment */
unsigned long timestamp;
/** @seqno: sequence number of the fragments in the list */
diff --git a/tools/testing/selftests/timers/posix_timers.c b/tools/testing/selftests/timers/posix_timers.c
index 07c81c0093c0..16bd49492efa 100644
--- a/tools/testing/selftests/timers/posix_timers.c
+++ b/tools/testing/selftests/timers/posix_timers.c
@@ -6,10 +6,13 @@
*
* Kernel loop code stolen from Steven Rostedt <srostedt@redhat.com>
*/
-
+#define _GNU_SOURCE
#include <sys/time.h>
+#include <sys/types.h>
#include <stdio.h>
#include <signal.h>
+#include <stdint.h>
+#include <string.h>
#include <unistd.h>
#include <time.h>
#include <pthread.h>
@@ -18,6 +21,21 @@
#define DELAY 2
#define USECS_PER_SEC 1000000
+#define NSECS_PER_SEC 1000000000
+
+static void __fatal_error(const char *test, const char *name, const char *what)
+{
+ char buf[64];
+
+ strerror_r(errno, buf, sizeof(buf));
+
+ if (name && strlen(name))
+ ksft_exit_fail_msg("%s %s %s %s\n", test, name, what, buf);
+ else
+ ksft_exit_fail_msg("%s %s %s\n", test, what, buf);
+}
+
+#define fatal_error(name, what) __fatal_error(__func__, name, what)
static volatile int done;
@@ -74,24 +92,13 @@ static int check_diff(struct timeval start, struct timeval end)
return 0;
}
-static int check_itimer(int which)
+static void check_itimer(int which, const char *name)
{
- const char *name;
- int err;
struct timeval start, end;
struct itimerval val = {
.it_value.tv_sec = DELAY,
};
- if (which == ITIMER_VIRTUAL)
- name = "ITIMER_VIRTUAL";
- else if (which == ITIMER_PROF)
- name = "ITIMER_PROF";
- else if (which == ITIMER_REAL)
- name = "ITIMER_REAL";
- else
- return -1;
-
done = 0;
if (which == ITIMER_VIRTUAL)
@@ -101,17 +108,11 @@ static int check_itimer(int which)
else if (which == ITIMER_REAL)
signal(SIGALRM, sig_handler);
- err = gettimeofday(&start, NULL);
- if (err < 0) {
- ksft_perror("Can't call gettimeofday()");
- return -1;
- }
+ if (gettimeofday(&start, NULL) < 0)
+ fatal_error(name, "gettimeofday()");
- err = setitimer(which, &val, NULL);
- if (err < 0) {
- ksft_perror("Can't set timer");
- return -1;
- }
+ if (setitimer(which, &val, NULL) < 0)
+ fatal_error(name, "setitimer()");
if (which == ITIMER_VIRTUAL)
user_loop();
@@ -120,68 +121,41 @@ static int check_itimer(int which)
else if (which == ITIMER_REAL)
idle_loop();
- err = gettimeofday(&end, NULL);
- if (err < 0) {
- ksft_perror("Can't call gettimeofday()");
- return -1;
- }
+ if (gettimeofday(&end, NULL) < 0)
+ fatal_error(name, "gettimeofday()");
ksft_test_result(check_diff(start, end) == 0, "%s\n", name);
-
- return 0;
}
-static int check_timer_create(int which)
+static void check_timer_create(int which, const char *name)
{
- const char *type;
- int err;
- timer_t id;
struct timeval start, end;
struct itimerspec val = {
.it_value.tv_sec = DELAY,
};
-
- if (which == CLOCK_THREAD_CPUTIME_ID) {
- type = "thread";
- } else if (which == CLOCK_PROCESS_CPUTIME_ID) {
- type = "process";
- } else {
- ksft_print_msg("Unknown timer_create() type %d\n", which);
- return -1;
- }
+ timer_t id;
done = 0;
- err = timer_create(which, NULL, &id);
- if (err < 0) {
- ksft_perror("Can't create timer");
- return -1;
- }
- signal(SIGALRM, sig_handler);
- err = gettimeofday(&start, NULL);
- if (err < 0) {
- ksft_perror("Can't call gettimeofday()");
- return -1;
- }
+ if (timer_create(which, NULL, &id) < 0)
+ fatal_error(name, "timer_create()");
- err = timer_settime(id, 0, &val, NULL);
- if (err < 0) {
- ksft_perror("Can't set timer");
- return -1;
- }
+ if (signal(SIGALRM, sig_handler) == SIG_ERR)
+ fatal_error(name, "signal()");
+
+ if (gettimeofday(&start, NULL) < 0)
+ fatal_error(name, "gettimeofday()");
+
+ if (timer_settime(id, 0, &val, NULL) < 0)
+ fatal_error(name, "timer_settime()");
user_loop();
- err = gettimeofday(&end, NULL);
- if (err < 0) {
- ksft_perror("Can't call gettimeofday()");
- return -1;
- }
+ if (gettimeofday(&end, NULL) < 0)
+ fatal_error(name, "gettimeofday()");
ksft_test_result(check_diff(start, end) == 0,
- "timer_create() per %s\n", type);
-
- return 0;
+ "timer_create() per %s\n", name);
}
static pthread_t ctd_thread;
@@ -209,15 +183,14 @@ static void *ctd_thread_func(void *arg)
ctd_count = 100;
if (timer_create(CLOCK_PROCESS_CPUTIME_ID, NULL, &id))
- return "Can't create timer\n";
+ fatal_error(NULL, "timer_create()");
if (timer_settime(id, 0, &val, NULL))
- return "Can't set timer\n";
-
+ fatal_error(NULL, "timer_settime()");
while (ctd_count > 0 && !ctd_failed)
;
if (timer_delete(id))
- return "Can't delete timer\n";
+ fatal_error(NULL, "timer_delete()");
return NULL;
}
@@ -225,19 +198,16 @@ static void *ctd_thread_func(void *arg)
/*
* Test that only the running thread receives the timer signal.
*/
-static int check_timer_distribution(void)
+static void check_timer_distribution(void)
{
- const char *errmsg;
+ if (signal(SIGALRM, ctd_sighandler) == SIG_ERR)
+ fatal_error(NULL, "signal()");
- signal(SIGALRM, ctd_sighandler);
-
- errmsg = "Can't create thread\n";
if (pthread_create(&ctd_thread, NULL, ctd_thread_func, NULL))
- goto err;
+ fatal_error(NULL, "pthread_create()");
- errmsg = "Can't join thread\n";
- if (pthread_join(ctd_thread, (void **)&errmsg) || errmsg)
- goto err;
+ if (pthread_join(ctd_thread, NULL))
+ fatal_error(NULL, "pthread_join()");
if (!ctd_failed)
ksft_test_result_pass("check signal distribution\n");
@@ -245,31 +215,399 @@ static int check_timer_distribution(void)
ksft_test_result_fail("check signal distribution\n");
else
ksft_test_result_skip("check signal distribution (old kernel)\n");
- return 0;
-err:
- ksft_print_msg("%s", errmsg);
- return -1;
+}
+
+struct tmrsig {
+ int signals;
+ int overruns;
+};
+
+static void siginfo_handler(int sig, siginfo_t *si, void *uc)
+{
+ struct tmrsig *tsig = si ? si->si_ptr : NULL;
+
+ if (tsig) {
+ tsig->signals++;
+ tsig->overruns += si->si_overrun;
+ }
+}
+
+static void *ignore_thread(void *arg)
+{
+ unsigned int *tid = arg;
+ sigset_t set;
+
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_BLOCK)");
+
+ *tid = gettid();
+ sleep(100);
+
+ if (sigprocmask(SIG_UNBLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_UNBLOCK)");
+ return NULL;
+}
+
+static void check_sig_ign(int thread)
+{
+ struct tmrsig tsig = { };
+ struct itimerspec its;
+ unsigned int tid = 0;
+ struct sigaction sa;
+ struct sigevent sev;
+ pthread_t pthread;
+ timer_t timerid;
+ sigset_t set;
+
+ if (thread) {
+ if (pthread_create(&pthread, NULL, ignore_thread, &tid))
+ fatal_error(NULL, "pthread_create()");
+ sleep(1);
+ }
+
+ sa.sa_flags = SA_SIGINFO;
+ sa.sa_sigaction = siginfo_handler;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(SIGUSR1, &sa, NULL))
+ fatal_error(NULL, "sigaction()");
+
+ /* Block the signal */
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_BLOCK)");
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_SIGNAL;
+ sev.sigev_signo = SIGUSR1;
+ sev.sigev_value.sival_ptr = &tsig;
+ if (thread) {
+ sev.sigev_notify = SIGEV_THREAD_ID;
+ sev._sigev_un._tid = tid;
+ }
+
+ if (timer_create(CLOCK_MONOTONIC, &sev, &timerid))
+ fatal_error(NULL, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ timer_settime(timerid, 0, &its, NULL);
+
+ sleep(1);
+
+ /* Set the signal to be ignored */
+ if (signal(SIGUSR1, SIG_IGN) == SIG_ERR)
+ fatal_error(NULL, "signal(SIG_IGN)");
+
+ sleep(1);
+
+ if (thread) {
+ /* Stop the thread first. No signal should be delivered to it */
+ if (pthread_cancel(pthread))
+ fatal_error(NULL, "pthread_cancel()");
+ if (pthread_join(pthread, NULL))
+ fatal_error(NULL, "pthread_join()");
+ }
+
+ /* Restore the handler */
+ if (sigaction(SIGUSR1, &sa, NULL))
+ fatal_error(NULL, "sigaction()");
+
+ sleep(1);
+
+ /* Unblock it, which should deliver the signal in the !thread case*/
+ if (sigprocmask(SIG_UNBLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_UNBLOCK)");
+
+ if (timer_delete(timerid))
+ fatal_error(NULL, "timer_delete()");
+
+ if (!thread) {
+ ksft_test_result(tsig.signals == 1 && tsig.overruns == 29,
+ "check_sig_ign SIGEV_SIGNAL\n");
+ } else {
+ ksft_test_result(tsig.signals == 0 && tsig.overruns == 0,
+ "check_sig_ign SIGEV_THREAD_ID\n");
+ }
+}
+
+static void check_rearm(void)
+{
+ struct tmrsig tsig = { };
+ struct itimerspec its;
+ struct sigaction sa;
+ struct sigevent sev;
+ timer_t timerid;
+ sigset_t set;
+
+ sa.sa_flags = SA_SIGINFO;
+ sa.sa_sigaction = siginfo_handler;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(SIGUSR1, &sa, NULL))
+ fatal_error(NULL, "sigaction()");
+
+ /* Block the signal */
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_BLOCK)");
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_SIGNAL;
+ sev.sigev_signo = SIGUSR1;
+ sev.sigev_value.sival_ptr = &tsig;
+ if (timer_create(CLOCK_MONOTONIC, &sev, &timerid))
+ fatal_error(NULL, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ if (timer_settime(timerid, 0, &its, NULL))
+ fatal_error(NULL, "timer_settime()");
+
+ sleep(1);
+
+ /* Reprogram the timer to single shot */
+ its.it_value.tv_sec = 10;
+ its.it_value.tv_nsec = 0;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 0;
+ if (timer_settime(timerid, 0, &its, NULL))
+ fatal_error(NULL, "timer_settime()");
+
+ /* Unblock it, which should not deliver a signal */
+ if (sigprocmask(SIG_UNBLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_UNBLOCK)");
+
+ if (timer_delete(timerid))
+ fatal_error(NULL, "timer_delete()");
+
+ ksft_test_result(!tsig.signals, "check_rearm\n");
+}
+
+static void check_delete(void)
+{
+ struct tmrsig tsig = { };
+ struct itimerspec its;
+ struct sigaction sa;
+ struct sigevent sev;
+ timer_t timerid;
+ sigset_t set;
+
+ sa.sa_flags = SA_SIGINFO;
+ sa.sa_sigaction = siginfo_handler;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(SIGUSR1, &sa, NULL))
+ fatal_error(NULL, "sigaction()");
+
+ /* Block the signal */
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_BLOCK)");
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_SIGNAL;
+ sev.sigev_signo = SIGUSR1;
+ sev.sigev_value.sival_ptr = &tsig;
+ if (timer_create(CLOCK_MONOTONIC, &sev, &timerid))
+ fatal_error(NULL, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ if (timer_settime(timerid, 0, &its, NULL))
+ fatal_error(NULL, "timer_settime()");
+
+ sleep(1);
+
+ if (timer_delete(timerid))
+ fatal_error(NULL, "timer_delete()");
+
+ /* Unblock it, which should not deliver a signal */
+ if (sigprocmask(SIG_UNBLOCK, &set, NULL))
+ fatal_error(NULL, "sigprocmask(SIG_UNBLOCK)");
+
+ ksft_test_result(!tsig.signals, "check_delete\n");
+}
+
+static inline int64_t calcdiff_ns(struct timespec t1, struct timespec t2)
+{
+ int64_t diff;
+
+ diff = NSECS_PER_SEC * (int64_t)((int) t1.tv_sec - (int) t2.tv_sec);
+ diff += ((int) t1.tv_nsec - (int) t2.tv_nsec);
+ return diff;
+}
+
+static void check_sigev_none(int which, const char *name)
+{
+ struct timespec start, now;
+ struct itimerspec its;
+ struct sigevent sev;
+ timer_t timerid;
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_NONE;
+
+ if (timer_create(which, &sev, &timerid))
+ fatal_error(name, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ timer_settime(timerid, 0, &its, NULL);
+
+ if (clock_gettime(which, &start))
+ fatal_error(name, "clock_gettime()");
+
+ do {
+ if (clock_gettime(which, &now))
+ fatal_error(name, "clock_gettime()");
+ } while (calcdiff_ns(now, start) < NSECS_PER_SEC);
+
+ if (timer_gettime(timerid, &its))
+ fatal_error(name, "timer_gettime()");
+
+ if (timer_delete(timerid))
+ fatal_error(name, "timer_delete()");
+
+ ksft_test_result(its.it_value.tv_sec || its.it_value.tv_nsec,
+ "check_sigev_none %s\n", name);
+}
+
+static void check_gettime(int which, const char *name)
+{
+ struct itimerspec its, prev;
+ struct timespec start, now;
+ struct sigevent sev;
+ timer_t timerid;
+ int wraps = 0;
+ sigset_t set;
+
+ /* Block the signal */
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(name, "sigprocmask(SIG_BLOCK)");
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_SIGNAL;
+ sev.sigev_signo = SIGUSR1;
+
+ if (timer_create(which, &sev, &timerid))
+ fatal_error(name, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ if (timer_settime(timerid, 0, &its, NULL))
+ fatal_error(name, "timer_settime()");
+
+ if (timer_gettime(timerid, &prev))
+ fatal_error(name, "timer_gettime()");
+
+ if (clock_gettime(which, &start))
+ fatal_error(name, "clock_gettime()");
+
+ do {
+ if (clock_gettime(which, &now))
+ fatal_error(name, "clock_gettime()");
+ if (timer_gettime(timerid, &its))
+ fatal_error(name, "timer_gettime()");
+ if (its.it_value.tv_nsec > prev.it_value.tv_nsec)
+ wraps++;
+ prev = its;
+
+ } while (calcdiff_ns(now, start) < NSECS_PER_SEC);
+
+ if (timer_delete(timerid))
+ fatal_error(name, "timer_delete()");
+
+ ksft_test_result(wraps > 1, "check_gettime %s\n", name);
+}
+
+static void check_overrun(int which, const char *name)
+{
+ struct timespec start, now;
+ struct tmrsig tsig = { };
+ struct itimerspec its;
+ struct sigaction sa;
+ struct sigevent sev;
+ timer_t timerid;
+ sigset_t set;
+
+ sa.sa_flags = SA_SIGINFO;
+ sa.sa_sigaction = siginfo_handler;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(SIGUSR1, &sa, NULL))
+ fatal_error(name, "sigaction()");
+
+ /* Block the signal */
+ sigemptyset(&set);
+ sigaddset(&set, SIGUSR1);
+ if (sigprocmask(SIG_BLOCK, &set, NULL))
+ fatal_error(name, "sigprocmask(SIG_BLOCK)");
+
+ memset(&sev, 0, sizeof(sev));
+ sev.sigev_notify = SIGEV_SIGNAL;
+ sev.sigev_signo = SIGUSR1;
+ sev.sigev_value.sival_ptr = &tsig;
+ if (timer_create(which, &sev, &timerid))
+ fatal_error(name, "timer_create()");
+
+ /* Start the timer to expire in 100ms and 100ms intervals */
+ its.it_value.tv_sec = 0;
+ its.it_value.tv_nsec = 100000000;
+ its.it_interval.tv_sec = 0;
+ its.it_interval.tv_nsec = 100000000;
+ if (timer_settime(timerid, 0, &its, NULL))
+ fatal_error(name, "timer_settime()");
+
+ if (clock_gettime(which, &start))
+ fatal_error(name, "clock_gettime()");
+
+ do {
+ if (clock_gettime(which, &now))
+ fatal_error(name, "clock_gettime()");
+ } while (calcdiff_ns(now, start) < NSECS_PER_SEC);
+
+ /* Unblock it, which should deliver a signal */
+ if (sigprocmask(SIG_UNBLOCK, &set, NULL))
+ fatal_error(name, "sigprocmask(SIG_UNBLOCK)");
+
+ if (timer_delete(timerid))
+ fatal_error(name, "timer_delete()");
+
+ ksft_test_result(tsig.signals == 1 && tsig.overruns == 9,
+ "check_overrun %s\n", name);
}
int main(int argc, char **argv)
{
ksft_print_header();
- ksft_set_plan(6);
+ ksft_set_plan(18);
ksft_print_msg("Testing posix timers. False negative may happen on CPU execution \n");
ksft_print_msg("based timers if other threads run on the CPU...\n");
- if (check_itimer(ITIMER_VIRTUAL) < 0)
- ksft_exit_fail();
-
- if (check_itimer(ITIMER_PROF) < 0)
- ksft_exit_fail();
-
- if (check_itimer(ITIMER_REAL) < 0)
- ksft_exit_fail();
-
- if (check_timer_create(CLOCK_THREAD_CPUTIME_ID) < 0)
- ksft_exit_fail();
+ check_itimer(ITIMER_VIRTUAL, "ITIMER_VIRTUAL");
+ check_itimer(ITIMER_PROF, "ITIMER_PROF");
+ check_itimer(ITIMER_REAL, "ITIMER_REAL");
+ check_timer_create(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
/*
* It's unfortunately hard to reliably test a timer expiration
@@ -280,11 +618,21 @@ int main(int argc, char **argv)
* to ensure true parallelism. So test only one thread until we
* find a better solution.
*/
- if (check_timer_create(CLOCK_PROCESS_CPUTIME_ID) < 0)
- ksft_exit_fail();
-
- if (check_timer_distribution() < 0)
- ksft_exit_fail();
+ check_timer_create(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
+ check_timer_distribution();
+
+ check_sig_ign(0);
+ check_sig_ign(1);
+ check_rearm();
+ check_delete();
+ check_sigev_none(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
+ check_sigev_none(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
+ check_gettime(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
+ check_gettime(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
+ check_gettime(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
+ check_overrun(CLOCK_MONOTONIC, "CLOCK_MONOTONIC");
+ check_overrun(CLOCK_PROCESS_CPUTIME_ID, "CLOCK_PROCESS_CPUTIME_ID");
+ check_overrun(CLOCK_THREAD_CPUTIME_ID, "CLOCK_THREAD_CPUTIME_ID");
ksft_finished();
}
^ permalink raw reply [flat|nested] 12+ messages in thread* [GIT pull] core/debugobjects for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
` (3 preceding siblings ...)
2024-09-16 8:38 ` [GIT pull] timers/core " Thomas Gleixner
@ 2024-09-16 8:44 ` Thomas Gleixner
2024-09-17 5:13 ` Linus Torvalds
2024-09-17 6:35 ` pr-tracker-bot
6 siblings, 0 replies; 12+ messages in thread
From: Thomas Gleixner @ 2024-09-16 8:44 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, x86
Linus,
please pull the latest core/debugobjects branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-debugobjects-2024-09-16
up to: 63a4a9b52c3c: debugobjects: Remove redundant checks in fill_pool()
A set of updates for debugobjects:
- Use the threshold to check for the pool refill condition and not the
run time recorded all time low fill value, which is lower than the
threshold and therefore causes refills to be delayed.
- KCSAN annotation updates and simplification of the fill_pool() code.
Thanks,
tglx
------------------>
Zhen Lei (3):
debugobjects: Fix the compilation attributes of some global variables
debugobjects: Fix conditions in fill_pool()
debugobjects: Remove redundant checks in fill_pool()
lib/debugobjects.c | 27 +++++++++++++--------------
1 file changed, 13 insertions(+), 14 deletions(-)
diff --git a/lib/debugobjects.c b/lib/debugobjects.c
index 7cea91e193a8..5ce473ad499b 100644
--- a/lib/debugobjects.c
+++ b/lib/debugobjects.c
@@ -70,10 +70,10 @@ static HLIST_HEAD(obj_to_free);
* made at debug_stats_show(). Both obj_pool_min_free and obj_pool_max_used
* can be off.
*/
-static int obj_pool_min_free = ODEBUG_POOL_SIZE;
-static int obj_pool_free = ODEBUG_POOL_SIZE;
+static int __data_racy obj_pool_min_free = ODEBUG_POOL_SIZE;
+static int __data_racy obj_pool_free = ODEBUG_POOL_SIZE;
static int obj_pool_used;
-static int obj_pool_max_used;
+static int __data_racy obj_pool_max_used;
static bool obj_freeing;
/* The number of objs on the global free list */
static int obj_nr_tofree;
@@ -84,9 +84,9 @@ static int __data_racy debug_objects_fixups __read_mostly;
static int __data_racy debug_objects_warnings __read_mostly;
static int __data_racy debug_objects_enabled __read_mostly
= CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT;
-static int __data_racy debug_objects_pool_size __read_mostly
+static int debug_objects_pool_size __ro_after_init
= ODEBUG_POOL_SIZE;
-static int __data_racy debug_objects_pool_min_level __read_mostly
+static int debug_objects_pool_min_level __ro_after_init
= ODEBUG_POOL_MIN_LEVEL;
static const struct debug_obj_descr *descr_test __read_mostly;
@@ -95,8 +95,8 @@ static struct kmem_cache *obj_cache __ro_after_init;
/*
* Track numbers of kmem_cache_alloc()/free() calls done.
*/
-static int debug_objects_allocated;
-static int debug_objects_freed;
+static int __data_racy debug_objects_allocated;
+static int __data_racy debug_objects_freed;
static void free_obj_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(debug_obj_work, free_obj_work);
@@ -135,20 +135,19 @@ static void fill_pool(void)
return;
/*
- * Reuse objs from the global free list; they will be reinitialized
- * when allocating.
+ * Reuse objs from the global obj_to_free list; they will be
+ * reinitialized when allocating.
*
- * Both obj_nr_tofree and obj_pool_free are checked locklessly; the
- * READ_ONCE()s pair with the WRITE_ONCE()s in pool_lock critical
- * sections.
+ * obj_nr_tofree is checked locklessly; the READ_ONCE() pairs with
+ * the WRITE_ONCE() in pool_lock critical sections.
*/
- while (READ_ONCE(obj_nr_tofree) && (READ_ONCE(obj_pool_free) < obj_pool_min_free)) {
+ if (READ_ONCE(obj_nr_tofree)) {
raw_spin_lock_irqsave(&pool_lock, flags);
/*
* Recheck with the lock held as the worker thread might have
* won the race and freed the global free list already.
*/
- while (obj_nr_tofree && (obj_pool_free < obj_pool_min_free)) {
+ while (obj_nr_tofree && (obj_pool_free < debug_objects_pool_min_level)) {
obj = hlist_entry(obj_to_free.first, typeof(*obj), node);
hlist_del(&obj->node);
WRITE_ONCE(obj_nr_tofree, obj_nr_tofree - 1);
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [GIT pull] core/debugobjects for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
` (4 preceding siblings ...)
2024-09-16 8:44 ` [GIT pull] core/debugobjects " Thomas Gleixner
@ 2024-09-17 5:13 ` Linus Torvalds
2024-09-17 6:35 ` pr-tracker-bot
6 siblings, 0 replies; 12+ messages in thread
From: Linus Torvalds @ 2024-09-17 5:13 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: linux-kernel, x86
On Mon, 16 Sept 2024 at 10:44, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> please pull the latest core/debugobjects branch from: [...]
You sent this twice.
Which is fine, but makes me wonder if perhaps you *meant* to send a
different pull request the second time, and I'm missing some pull?
I have the cpu hotplug one, the clocksource watchdog, and the irq and
timer ones (and obviously the debugobjects pull twice). Anything
missing from that list?
Linus
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [GIT pull] core/debugobjects for v6.12-rc1
2024-09-16 8:38 [GIT pull] core/debugobjects for v6.12-rc1 Thomas Gleixner
` (5 preceding siblings ...)
2024-09-17 5:13 ` Linus Torvalds
@ 2024-09-17 6:35 ` pr-tracker-bot
6 siblings, 0 replies; 12+ messages in thread
From: pr-tracker-bot @ 2024-09-17 6:35 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Linus Torvalds, linux-kernel, x86
The pull request you sent on Mon, 16 Sep 2024 10:38:34 +0200 (CEST):
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-debugobjects-2024-09-16
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/daa394f0f9d3cb002c72e2d3db99972e2ee42862
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 12+ messages in thread