* Re: Linux 7.3.4
2026-11-10 18:28 Linux 7.3.4 Greg Kroah-Hartman
@ 2026-11-10 18:28 ` Greg Kroah-Hartman
0 siblings, 0 replies; 2+ messages in thread
From: Greg Kroah-Hartman @ 2026-11-10 18:28 UTC (permalink / raw)
To: linux-kernel, akpm, torvalds, stable; +Cc: lwn, jslaby, Greg Kroah-Hartman
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index e75344f4e0cd..4e64219c8da4 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4802,6 +4802,16 @@ Kernel parameters
Example:
nvme.quirks=7710:2267:bogus_nid,^identify_cns-9900:7711:broken_msi
+ nvme_tcp.tls_handshake_timeout=
+ [NVME] NVMe/TCP host TLS handshake timeout in seconds.
+ Valid values range from 0 to the smaller of
+ UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10
+
+ nvmet_tcp.tls_handshake_timeout=
+ [NVME] NVMe/TCP target TLS handshake timeout in seconds.
+ Valid values range from 0 to the smaller of
+ UINT_MAX / 1000 and MAX_JIFFY_OFFSET / HZ. Default: 10
+
ohci1394_dma=early [HW,EARLY] enable debugging via the ohci1394 driver.
See Documentation/core-api/debugging-via-ohci1394.rst for more
info.
diff --git a/Documentation/block/ublk.rst b/Documentation/block/ublk.rst
index 28300fee22bf..ed3285e361ad 100644
--- a/Documentation/block/ublk.rst
+++ b/Documentation/block/ublk.rst
@@ -126,6 +126,13 @@ managing and controlling ublk devices with help of several control commands:
ublk server will release resources (such as destroying I/O handler threads &
io_uring).
+ If a server has ``/dev/ublkc*`` open, its fetched I/O commands complete with
+ ``UBLK_IO_RES_ABORT``. Until it closes ``/dev/ublkc*``, new
+ ``UBLK_IO_FETCH_REQ`` and ``UBLK_U_IO_PREP_IO_CMDS`` commands complete with
+ ``UBLK_IO_RES_ABORT`` too, and ``UBLK_CMD_START_DEV`` fails with ``-EBUSY``,
+ even if the server hadn't fetched anything. The device can be started again
+ by a server that opens ``/dev/ublkc*`` after that.
+
- ``UBLK_CMD_DEL_DEV``
Remove ``/dev/ublkc*``. When this command returns, the allocated ublk device
diff --git a/Documentation/devicetree/bindings/clock/nuvoton,ma35d1-clk.yaml b/Documentation/devicetree/bindings/clock/nuvoton,ma35d1-clk.yaml
index 8f0c43683729..9649feda8d7a 100644
--- a/Documentation/devicetree/bindings/clock/nuvoton,ma35d1-clk.yaml
+++ b/Documentation/devicetree/bindings/clock/nuvoton,ma35d1-clk.yaml
@@ -29,7 +29,18 @@ properties:
const: 1
clocks:
- maxItems: 1
+ minItems: 1
+ maxItems: 2
+ description:
+ The controller takes 2 crystal oscillator inputs from the board.
+ HXT is a 24 MHz crystal feeding the internal PLLs. LXT is a 32.768
+ kHz crystal, for RTC and low power operations.
+
+ clock-names:
+ minItems: 1
+ items:
+ - const: hxt
+ - const: lxt
nuvoton,pll-mode:
description:
@@ -58,6 +69,7 @@ examples:
compatible = "nuvoton,ma35d1-clk";
reg = <0x40460200 0x100>;
#clock-cells = <1>;
- clocks = <&clk_hxt>;
+ clocks = <&clk_hxt>, <&clk_lxt>;
+ clock-names = "hxt", "lxt";
};
...
diff --git a/Documentation/filesystems/fsverity.rst b/Documentation/filesystems/fsverity.rst
index 22b49b295d1f..e5d38439bf58 100644
--- a/Documentation/filesystems/fsverity.rst
+++ b/Documentation/filesystems/fsverity.rst
@@ -170,6 +170,7 @@ fatal signal), no changes are made to the file.
FS_IOC_ENABLE_VERITY can fail with the following errors:
- ``EACCES``: the process does not have write access to the file
+- ``EBADF``: the file descriptor is not open for reading
- ``EBADMSG``: the builtin signature is malformed
- ``EBUSY``: this ioctl is already running on the file
- ``EEXIST``: the file already has verity enabled
@@ -232,6 +233,7 @@ regardless of the size of the file.
FS_IOC_MEASURE_VERITY can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``ENODATA``: the file is not a verity file
- ``ENOTTY``: this type of filesystem does not implement fs-verity
@@ -310,6 +312,7 @@ blocks from disk without actually verifying the path to the root node.
FS_IOC_READ_VERITY_METADATA can fail with the following errors:
+- ``EBADF``: the file descriptor is not open for reading
- ``EFAULT``: the caller provided inaccessible memory
- ``EINTR``: the ioctl was interrupted before any data was read
- ``EINVAL``: reserved fields were set, or ``offset + length``
diff --git a/Documentation/hwmon/coretemp.rst b/Documentation/hwmon/coretemp.rst
index 349301683381..09f885a847e3 100644
--- a/Documentation/hwmon/coretemp.rst
+++ b/Documentation/hwmon/coretemp.rst
@@ -50,6 +50,9 @@ All sysfs entries are named with their core_id (represented here by 'X').
================= ========================================================
tempX_input Core temperature (in millidegrees Celsius).
+ Reading fails with -ENODATA while the CPU reports the
+ temperature as invalid. Some CPUs do this when the
+ temperature is too low to be measured.
tempX_max Maximum recommended operating temperature (Tcontrol).
All cooling devices should be turned on.
tempX_crit Maximum junction temperature (in millidegrees Celsius).
diff --git a/Documentation/hwmon/prom21-xhci.rst b/Documentation/hwmon/prom21-xhci.rst
index 7984fb187bd8..6c0b6e736892 100644
--- a/Documentation/hwmon/prom21-xhci.rst
+++ b/Documentation/hwmon/prom21-xhci.rst
@@ -9,7 +9,7 @@ Supported chips:
Prefix: 'prom21_xhci'
- PCI IDs: 1022:43fc, 1022:43fd
+ PCI IDs: 1022:43f7, 1022:43f9, 1022:43fa, 1022:43fc, 1022:43fd
Author:
diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst
index f7af0286341c..66a54c510693 100644
--- a/Documentation/networking/ip-sysctl.rst
+++ b/Documentation/networking/ip-sysctl.rst
@@ -1128,6 +1128,8 @@ tcp_tso_rtt_log - INTEGER
If you want to use the old autosizing, set this sysctl to 0.
+ Possible values are [0, 31], inclusive.
+
Default: 9 (2^9 = 512 usec)
tcp_pacing_ss_ratio - INTEGER
@@ -3823,5 +3825,6 @@ l3mdev_accept - BOOLEAN
max_dgram_qlen - INTEGER
The maximum length of dgram socket receive queue
- Default: 10
+ Possible values: 0 - INT_MAX
+ Default: 10
diff --git a/Documentation/userspace-api/check_exec.rst b/Documentation/userspace-api/check_exec.rst
index 05dfe3b56f71..0904810e1d8d 100644
--- a/Documentation/userspace-api/check_exec.rst
+++ b/Documentation/userspace-api/check_exec.rst
@@ -99,6 +99,10 @@ This secure bit may be set by user session managers, service managers,
container runtimes, sandboxer tools... Except for test environments, the
related ``SECBIT_EXEC_DENY_INTERACTIVE_LOCKED`` bit should also be set.
+Both secure bits and their locks are inherited across :manpage:`fork(2)` and
+:manpage:`execve(2)`, and they are kept when a task creates or joins a user
+namespace.
+
Here is the expected behavior for a script interpreter according to combination
of any exec securebits:
diff --git a/Documentation/virt/hyperv/vmbus.rst b/Documentation/virt/hyperv/vmbus.rst
index 654bb4849972..c546e821ce6b 100644
--- a/Documentation/virt/hyperv/vmbus.rst
+++ b/Documentation/virt/hyperv/vmbus.rst
@@ -141,14 +141,15 @@ Three functions exist to send VMBus channel messages:
1. vmbus_sendpacket(): Control-only messages and messages with
embedded data -- no GPAs
-2. vmbus_sendpacket_pagebuffer(): Message with list of GPAs
- identifying data to transfer. An offset and length is
- associated with each GPA so that multiple discontinuous areas
- of guest memory can be targeted.
-3. vmbus_sendpacket_mpb_desc(): Message with list of GPAs
- identifying data to transfer. A single offset and length is
- associated with a list of GPAs. The GPAs must describe a
- single logical area of guest memory to be targeted.
+2. vmbus_sendpacket_getid(): Similar to vmbus_sendpacket() but returns
+ a transaction ID that is matched against a reply from the Hyper-V
+ host. Useful when the struct completion (or other rendezvous data)
+ must reside on the caller's stack. By invalidating the transaction
+ ID before exiting the function due to a timeout or other error, a
+ late arriving interrupt is assured to not trash the stack.
+3. vmbus_sendpacket_mpb_desc(): Message with one or more data ranges
+ to target. Each range consists of a single offset and length plus
+ a list of GPAs that describe a logical area of guest memory.
Historically, Linux guests have trusted Hyper-V to send well-formed
and valid messages, and Linux drivers for synthetic devices did not
diff --git a/Makefile b/Makefile
index a8f35bdca5d4..05c5b682ad50 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 7
PATCHLEVEL = 3
-SUBLEVEL = 3
+SUBLEVEL = 4
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/alpha/include/asm/mmu_context.h b/arch/alpha/include/asm/mmu_context.h
index eee8fe836a59..09f356934b90 100644
--- a/arch/alpha/include/asm/mmu_context.h
+++ b/arch/alpha/include/asm/mmu_context.h
@@ -130,6 +130,8 @@ __get_new_mm_context(struct mm_struct *mm, long cpu)
return next;
}
+extern void __load_new_mm_context(struct mm_struct *);
+
__EXTERN_INLINE void
ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
struct task_struct *next)
@@ -139,6 +141,16 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
unsigned long mmc;
long cpu = smp_processor_id();
+ /*
+ * kthread_use_mm(), sched_force_init_mm() and do_shoot_lazy_tlb()
+ * switch current's mm without alpha_switch_to(), which loads the
+ * context. The latter two require options Alpha does not select.
+ */
+ if (next == current) {
+ __load_new_mm_context(next_mm);
+ return;
+ }
+
#ifdef CONFIG_SMP
cpu_data[cpu].asn_lock = 1;
barrier();
@@ -147,11 +159,16 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
mmc = next_mm->context[cpu];
if ((mmc ^ asn) & ~HARDWARE_ASN_MASK) {
mmc = __get_new_mm_context(next_mm, cpu);
- next_mm->context[cpu] = mmc;
+ /*
+ * Ordered before any user access through the mm by the
+ * mb() in arch_spin_unlock(), when finish_lock_switch()
+ * drops the rq lock.
+ */
+ WRITE_ONCE(next_mm->context[cpu], mmc);
}
#ifdef CONFIG_SMP
- else
- cpu_data[cpu].need_new_asn = 1;
+ /* A deferred shootdown can also invalidate a newly allocated ASN. */
+ cpu_data[cpu].need_new_asn = 1;
#endif
/* Always update the PCB ASN. Another thread may have allocated
@@ -160,7 +177,6 @@ ev5_switch_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm,
task_thread_info(next)->pcb.asn = mmc & HARDWARE_ASN_MASK;
}
-extern void __load_new_mm_context(struct mm_struct *);
asmlinkage void do_page_fault(unsigned long address, unsigned long mmcsr,
long cause, struct pt_regs *regs);
@@ -181,6 +197,17 @@ do { \
#define check_mmu_context() do { } while(0)
#endif
+/*
+ * Also called from kthread_use_mm() with preemption enabled. Alpha has
+ * no kernel preemption, so the task is still on the CPU that ran
+ * switch_mm().
+ */
+#define finish_arch_post_lock_switch finish_arch_post_lock_switch
+static inline void finish_arch_post_lock_switch(void)
+{
+ check_mmu_context();
+}
+
__EXTERN_INLINE void
ev5_activate_mm(struct mm_struct *prev_mm, struct mm_struct *next_mm)
{
diff --git a/arch/alpha/include/asm/sfp-machine.h b/arch/alpha/include/asm/sfp-machine.h
index 5fe63afbd474..bff1ad963c68 100644
--- a/arch/alpha/include/asm/sfp-machine.h
+++ b/arch/alpha/include/asm/sfp-machine.h
@@ -59,6 +59,10 @@
R##_c = FP_CLS_NAN; \
} while (0)
+/* Alpha determines tininess after rounding, so the emulation must do the
+ same as the hardware does for the results it produces itself. */
+#define _FP_TININESS_AFTER_ROUNDING 1
+
/* Obtain the current rounding mode. */
#define FP_ROUNDMODE mode
#define FP_RND_NEAREST (FPCR_DYN_NORMAL >> FPCR_DYN_SHIFT)
diff --git a/arch/alpha/include/asm/switch_to.h b/arch/alpha/include/asm/switch_to.h
index 762b7f975310..35c4b2c9d992 100644
--- a/arch/alpha/include/asm/switch_to.h
+++ b/arch/alpha/include/asm/switch_to.h
@@ -9,7 +9,6 @@ extern struct task_struct *alpha_switch_to(unsigned long, struct task_struct *);
#define switch_to(P,N,L) \
do { \
(L) = alpha_switch_to(virt_to_phys(&task_thread_info(N)->pcb), (P)); \
- check_mmu_context(); \
} while (0)
#endif /* __ALPHA_SWITCH_TO_H */
diff --git a/arch/alpha/include/asm/tlbflush.h b/arch/alpha/include/asm/tlbflush.h
index 0c8529997f54..9ae1902faf8e 100644
--- a/arch/alpha/include/asm/tlbflush.h
+++ b/arch/alpha/include/asm/tlbflush.h
@@ -87,7 +87,8 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
{
struct mm_struct *mm = vma->vm_mm;
- if (mm == current->active_mm)
+ /* A targeted tbi() needs a thread of MM to be current. */
+ if (mm == current->mm)
flush_tlb_current_page(mm, vma, addr);
else
flush_tlb_other(mm);
diff --git a/arch/alpha/kernel/entry.S b/arch/alpha/kernel/entry.S
index 9f2608de2544..fe88cd598492 100644
--- a/arch/alpha/kernel/entry.S
+++ b/arch/alpha/kernel/entry.S
@@ -603,6 +603,14 @@ ret_to_kernel:
.align 3
$syscall_error:
+ /*
+ * force_successful_syscall_return() zeroes the saved r0 (the
+ * syscall number): the negative return value is data, not an
+ * error number.
+ */
+ ldq $18, 0($sp)
+ beq $18, $ret_success
+
/* Restart syscall nr comes from saved r2 (preserved even if r0 overwritten). */
ldq $18, 16($sp) /* old syscall nr for restart */
@@ -700,9 +708,11 @@ ret_from_straced:
$strace_success:
stq $31, 72($sp) /* a3=0 => no error */
stq $0, 0($sp) /* save return value */
+ br $strace_leave
$strace_skip_call:
SYSCALL_SKIP_RETURN_RESTART_GATE
+$strace_leave:
DO_SWITCH_STACK
jsr $26, syscall_trace_leave
UNDO_SWITCH_STACK
@@ -710,6 +720,10 @@ $strace_skip_call:
.align 3
$strace_error:
+ /* See $syscall_error. */
+ ldq $18, 0($sp)
+ beq $18, $strace_success
+
ldq $18, 16($sp) /* restart syscall nr */
ldq $19, 72($sp) /* .. and this a3 */
diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index ed06367ece57..11768aa292a4 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -625,12 +625,30 @@ static void
ipi_flush_tlb_mm(void *x)
{
struct mm_struct *mm = x;
- if (mm == current->active_mm && !asn_locked())
+ if (mm == current->mm && !asn_locked())
flush_tlb_current(mm);
else
flush_tlb_other(mm);
}
+/* True if a CPU other than this one holds an ASN for MM. */
+static bool
+mm_context_elsewhere(struct mm_struct *mm)
+{
+ int cpu, this_cpu = smp_processor_id();
+
+ /* Pairs with the barrier the publishing CPU issues before using MM. */
+ smp_mb();
+
+ for_each_online_cpu(cpu) {
+ if (cpu == this_cpu)
+ continue;
+ if (READ_ONCE(mm->context[cpu]))
+ return true;
+ }
+ return false;
+}
+
void
flush_tlb_mm(struct mm_struct *mm)
{
@@ -638,17 +656,14 @@ flush_tlb_mm(struct mm_struct *mm)
if (mm == current->active_mm) {
flush_tlb_current(mm);
- if (atomic_read(&mm->mm_users) <= 1) {
- int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
- if (!cpu_online(cpu) || cpu == this_cpu)
- continue;
- if (mm->context[cpu])
- mm->context[cpu] = 0;
- }
+ if (atomic_read(&mm->mm_users) <= 1 &&
+ !mm_context_elsewhere(mm)) {
preempt_enable();
return;
}
+ } else {
+ /* smp_call_function() does not call back into this CPU. */
+ flush_tlb_other(mm);
}
smp_call_function(ipi_flush_tlb_mm, mm, 1);
@@ -669,7 +684,8 @@ ipi_flush_tlb_page(void *x)
struct flush_tlb_page_struct *data = x;
struct mm_struct * mm = data->mm;
- if (mm == current->active_mm && !asn_locked())
+ /* A targeted tbi() needs a thread of MM to be current. */
+ if (mm == current->mm && !asn_locked())
flush_tlb_current_page(mm, data->vma, data->addr);
else
flush_tlb_other(mm);
@@ -683,19 +699,17 @@ flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
preempt_disable();
- if (mm == current->active_mm) {
+ /* As in ipi_flush_tlb_page(): a targeted tbi() needs MM current. */
+ if (mm == current->mm) {
flush_tlb_current_page(mm, vma, addr);
- if (atomic_read(&mm->mm_users) <= 1) {
- int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
- if (!cpu_online(cpu) || cpu == this_cpu)
- continue;
- if (mm->context[cpu])
- mm->context[cpu] = 0;
- }
+ if (atomic_read(&mm->mm_users) <= 1 &&
+ !mm_context_elsewhere(mm)) {
preempt_enable();
return;
}
+ } else {
+ /* smp_call_function() does not call back into this CPU. */
+ flush_tlb_other(mm);
}
data.vma = vma;
@@ -720,7 +734,7 @@ static void
ipi_flush_icache_page(void *x)
{
struct mm_struct *mm = (struct mm_struct *) x;
- if (mm == current->active_mm && !asn_locked())
+ if (mm == current->mm && !asn_locked())
__load_new_mm_context(mm);
else
flush_tlb_other(mm);
@@ -739,17 +753,14 @@ flush_icache_user_page(struct vm_area_struct *vma, struct page *page,
if (mm == current->active_mm) {
__load_new_mm_context(mm);
- if (atomic_read(&mm->mm_users) <= 1) {
- int cpu, this_cpu = smp_processor_id();
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
- if (!cpu_online(cpu) || cpu == this_cpu)
- continue;
- if (mm->context[cpu])
- mm->context[cpu] = 0;
- }
+ if (atomic_read(&mm->mm_users) <= 1 &&
+ !mm_context_elsewhere(mm)) {
preempt_enable();
return;
}
+ } else {
+ /* smp_call_function() does not call back into this CPU. */
+ flush_tlb_other(mm);
}
smp_call_function(ipi_flush_icache_page, mm, 1);
diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c
index a9816bbc9f34..53e170e96591 100644
--- a/arch/alpha/mm/fault.c
+++ b/arch/alpha/mm/fault.c
@@ -8,6 +8,7 @@
#include <linux/sched/signal.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/hugetlb.h>
#include <asm/io.h>
#define __EXTERN_INLINE inline
@@ -45,7 +46,9 @@ __load_new_mm_context(struct mm_struct *next_mm)
struct pcb_struct *pcb;
mmc = __get_new_mm_context(next_mm, smp_processor_id());
- next_mm->context[smp_processor_id()] = mmc;
+ WRITE_ONCE(next_mm->context[smp_processor_id()], mmc);
+ /* Publish the context before this CPU can access the mm. */
+ smp_mb();
pcb = ¤t_thread_info()->pcb;
pcb->asn = mmc & HARDWARE_ASN_MASK;
@@ -89,6 +92,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
struct mm_struct *mm = current->mm;
const struct exception_table_entry *fixup;
int si_code = SEGV_MAPERR;
+ unsigned int lsb;
vm_fault_t fault;
unsigned int flags = FAULT_FLAG_DEFAULT;
@@ -161,6 +165,8 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
goto bad_area;
else if (fault & VM_FAULT_SIGBUS)
goto do_sigbus;
+ else if (fault & (VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE))
+ goto do_sigbus_mceerr;
BUG();
}
@@ -222,6 +228,21 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
goto no_context;
return;
+ do_sigbus_mceerr:
+ mmap_read_unlock(mm);
+ if (!user_mode(regs))
+ goto no_context;
+ /*
+ * Report the size of the poisoned area, which for a hugetlb fault
+ * is the size of the huge page that could not be mapped.
+ */
+ lsb = PAGE_SHIFT;
+ if (fault & VM_FAULT_HWPOISON_LARGE)
+ lsb = hstate_index_to_shift(VM_FAULT_GET_HINDEX(fault));
+ show_signal_msg(regs, address, SIGBUS, "hardware memory error");
+ force_sig_mceerr(BUS_MCEERR_AR, (void __user *) address, lsb);
+ return;
+
do_sigsegv:
force_sig_fault(SIGSEGV, si_code, (void __user *) address);
return;
diff --git a/arch/alpha/mm/tlbflush.c b/arch/alpha/mm/tlbflush.c
index ccbc317b9a34..a2aef90e6234 100644
--- a/arch/alpha/mm/tlbflush.c
+++ b/arch/alpha/mm/tlbflush.c
@@ -66,7 +66,7 @@ static void ipi_flush_mm_and_page(void *x)
struct tlb_mm_and_addr *d = x;
/* Part 1: mm context side (Alpha uses ASN/context as a key mechanism). */
- if (d->mm == current->active_mm && !asn_locked())
+ if (d->mm == current->mm && !asn_locked())
__load_new_mm_context(d->mm);
else
flush_tlb_other(d->mm);
@@ -90,22 +90,6 @@ void migrate_flush_tlb_page(struct vm_area_struct *vma, unsigned long addr)
*/
preempt_disable();
on_each_cpu(ipi_flush_mm_and_page, &d, 1);
-
- /*
- * mimic flush_tlb_mm()'s mm_users<=1 optimization.
- */
- if (atomic_read(&mm->mm_users) <= 1) {
-
- int cpu, this_cpu;
- this_cpu = smp_processor_id();
-
- for (cpu = 0; cpu < NR_CPUS; cpu++) {
- if (!cpu_online(cpu) || cpu == this_cpu)
- continue;
- if (READ_ONCE(mm->context[cpu]))
- WRITE_ONCE(mm->context[cpu], 0);
- }
- }
preempt_enable();
}
diff --git a/arch/arm/include/asm/linkage.h b/arch/arm/include/asm/linkage.h
index c4670694ada7..bba992f5d7a8 100644
--- a/arch/arm/include/asm/linkage.h
+++ b/arch/arm/include/asm/linkage.h
@@ -9,4 +9,33 @@
.type name, %function; \
END(name)
+#ifdef __ASSEMBLY__
+
+/*
+ * Variants of SYM_TYPED_START/SYM_TYPED_FUNC_START that align the
+ * function entry itself instead of the kCFI type hash preceding it,
+ * for functions whose entry point must meet an alignment requirement,
+ * such as the 8-byte alignment fncpy() demands of its source.
+ */
+#ifdef CONFIG_CFI
+
+#define SYM_TYPED_START_ALIGNED(name, linkage, align) \
+ linkage(name) ASM_NL \
+ .balign align ASM_NL \
+ .fill (align) - 4, 1, 0 ASM_NL \
+ __CFI_TYPE(name) ASM_NL \
+ name:
+
+#else /* CONFIG_CFI */
+
+#define SYM_TYPED_START_ALIGNED(name, linkage, align) \
+ SYM_START(name, linkage, .balign align)
+
+#endif /* CONFIG_CFI */
+
+#define SYM_TYPED_FUNC_START_ALIGNED(name, align) \
+ SYM_TYPED_START_ALIGNED(name, SYM_L_GLOBAL, align)
+
+#endif /* __ASSEMBLY__ */
+
#endif
diff --git a/arch/arm/include/asm/smp.h b/arch/arm/include/asm/smp.h
index 8c05a7f374d8..84aa6a663e81 100644
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -43,7 +43,7 @@ extern void set_smp_ipi_range(int ipi_base, int nr_ipi);
* Called from platform specific assembly code, this is the
* secondary CPU entry point.
*/
-asmlinkage void secondary_start_kernel(struct task_struct *task);
+asmlinkage void secondary_start_kernel(void);
/*
diff --git a/arch/arm/kernel/cacheinfo.c b/arch/arm/kernel/cacheinfo.c
index e1469b641780..993c8a134786 100644
--- a/arch/arm/kernel/cacheinfo.c
+++ b/arch/arm/kernel/cacheinfo.c
@@ -80,19 +80,21 @@ static void ci_leaf_init(struct cacheinfo *this_leaf,
this_leaf->type = type;
}
-static int detect_cache_level(unsigned int *level_p, unsigned int *leaves_p)
+static bool clidr_present(void)
{
- unsigned int ctype, level, leaves;
- u32 ctr, format;
-
/* CLIDR is not present before ARMv7/v7m */
if (cpu_architecture() < CPU_ARCH_ARMv7)
- return -EOPNOTSUPP;
+ return false;
/* Don't try reading CLIDR if CTR declares old format */
- ctr = read_cpuid_cachetype();
- format = FIELD_GET(CTR_FORMAT_MASK, ctr);
- if (format != CTR_FORMAT_ARMV7)
+ return FIELD_GET(CTR_FORMAT_MASK, read_cpuid_cachetype()) == CTR_FORMAT_ARMV7;
+}
+
+static int detect_cache_level(unsigned int *level_p, unsigned int *leaves_p)
+{
+ unsigned int ctype, level, leaves;
+
+ if (!clidr_present())
return -EOPNOTSUPP;
for (level = 1, leaves = 0; level <= MAX_CACHE_LEVEL; level++) {
@@ -111,13 +113,6 @@ static int detect_cache_level(unsigned int *level_p, unsigned int *leaves_p)
return 0;
}
-int early_cache_level(unsigned int cpu)
-{
- struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
-
- return detect_cache_level(&this_cpu_ci->num_levels, &this_cpu_ci->num_leaves);
-}
-
int init_cache_level(unsigned int cpu)
{
unsigned int level, leaves;
@@ -146,26 +141,32 @@ int init_cache_level(unsigned int cpu)
return 0;
}
+int early_cache_level(unsigned int cpu)
+{
+ return init_cache_level(cpu);
+}
+
int populate_cache_leaves(unsigned int cpu)
{
unsigned int level, idx;
enum cache_type type;
struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
- struct cacheinfo *this_leaf = this_cpu_ci->info_list;
- unsigned int arch = cpu_architecture();
+ struct cacheinfo *infos = this_cpu_ci->info_list;
- /* CLIDR is not present before ARMv7/v7m */
- if (arch < CPU_ARCH_ARMv7)
- return -EOPNOTSUPP;
+ /* The device tree can describe caches CLIDR cannot fill in. */
+ if (!clidr_present())
+ return -ENOENT;
for (idx = 0, level = 1; level <= this_cpu_ci->num_levels &&
- idx < this_cpu_ci->num_leaves; idx++, level++) {
+ idx < this_cpu_ci->num_leaves; level++) {
type = get_cache_type(level);
if (type == CACHE_TYPE_SEPARATE) {
- ci_leaf_init(this_leaf++, CACHE_TYPE_DATA, level);
- ci_leaf_init(this_leaf++, CACHE_TYPE_INST, level);
+ if (idx + 1 >= this_cpu_ci->num_leaves)
+ break;
+ ci_leaf_init(&infos[idx++], CACHE_TYPE_DATA, level);
+ ci_leaf_init(&infos[idx++], CACHE_TYPE_INST, level);
} else {
- ci_leaf_init(this_leaf++, type, level);
+ ci_leaf_init(&infos[idx++], type, level);
}
}
diff --git a/arch/arm/kernel/head-nommu.S b/arch/arm/kernel/head-nommu.S
index b9d6818f1ee1..f80bbfbd3a6f 100644
--- a/arch/arm/kernel/head-nommu.S
+++ b/arch/arm/kernel/head-nommu.S
@@ -116,6 +116,7 @@ ENTRY(secondary_startup)
ldr r7, __secondary_data @ reload r7
ldr sp, [r7, #12] @ set up the stack pointer
ldr r0, [r7, #16] @ set up task pointer
+ set_current r0, r1
mov fp, #0
b secondary_start_kernel
ENDPROC(secondary_startup)
diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index f22c50d4bd41..a22403a629ef 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -442,6 +442,7 @@ ENTRY(__secondary_switched)
adr_l r7, secondary_data + 12 @ get secondary_data.stack
ldr sp, [r7]
ldr r0, [r7, #4] @ get secondary_data.task
+ set_current r0, r1
mov fp, #0
b secondary_start_kernel
ENDPROC(__secondary_switched)
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index b5fb4697bc3f..a9f4600fe4d0 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -371,12 +371,9 @@ void __noreturn arch_cpu_idle_dead(void)
*/
__asm__("mov sp, %0\n"
" mov fp, #0\n"
- " mov r0, %1\n"
" b secondary_start_kernel"
:
- : "r" (task_stack_page(current) + THREAD_SIZE - 8),
- "r" (current)
- : "r0");
+ : "r" (task_stack_page(current) + THREAD_SIZE - 8));
unreachable();
}
@@ -397,23 +394,15 @@ static void smp_store_cpu_info(unsigned int cpuid)
check_cpu_icache_size(cpuid);
}
-static void set_current(struct task_struct *cur)
-{
- /* Set TPIDRURO */
- asm("mcr p15, 0, %0, c13, c0, 3" :: "r"(cur) : "memory");
-}
-
/*
* This is the secondary CPU boot entry. We're using this CPUs
* idle thread stack, but a set of temporary page tables.
*/
-asmlinkage void secondary_start_kernel(struct task_struct *task)
+asmlinkage void secondary_start_kernel(void)
{
struct mm_struct *mm = &init_mm;
unsigned int cpu;
- set_current(task);
-
secondary_biglittle_init();
/*
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 2336ee2aa44a..e6b50cd43ce5 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -12,6 +12,7 @@
*/
#include <linux/arch_topology.h>
+#include <linux/cacheinfo.h>
#include <linux/cpu.h>
#include <linux/cpufreq.h>
#include <linux/cpumask.h>
@@ -238,8 +239,20 @@ void store_cpu_topology(unsigned int cpuid)
*/
void __init init_cpu_topology(void)
{
+ int cpu, ret;
+
reset_cpu_topology();
smp_wmb();
parse_dt_topology();
+
+ for_each_possible_cpu(cpu) {
+ ret = fetch_cache_info(cpu);
+ if (!ret)
+ continue;
+ /* CPUs without a usable CLIDR return -EOPNOTSUPP. */
+ if (ret != -ENOENT && ret != -EOPNOTSUPP)
+ pr_err("Early cacheinfo failed, ret = %d\n", ret);
+ return;
+ }
}
diff --git a/arch/arm/mach-imx/pm-imx6.c b/arch/arm/mach-imx/pm-imx6.c
index a671ca498f88..3c73e2cdb8ce 100644
--- a/arch/arm/mach-imx/pm-imx6.c
+++ b/arch/arm/mach-imx/pm-imx6.c
@@ -4,6 +4,8 @@
* Copyright 2011 Linaro Ltd.
*/
+#include <linux/build_bug.h>
+#include <linux/cfi.h>
#include <linux/clk/imx.h>
#include <linux/delay.h>
#include <linux/init.h>
@@ -18,6 +20,7 @@
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
+#include <linux/stddef.h>
#include <linux/suspend.h>
#include <asm/cacheflush.h>
#include <asm/fncpy.h>
@@ -61,9 +64,9 @@
#define MX6Q_SUSPEND_OCRAM_SIZE 0x1000
#define MX6_MAX_MMDC_IO_NUM 33
-static void __iomem *ccm_base;
-static void __iomem *suspend_ocram_base;
-static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase);
+static void __iomem *ccm_base __ro_after_init;
+static void __iomem *suspend_ocram_base __ro_after_init;
+static void (*imx6_suspend_in_ocram_fn)(void __iomem *ocram_vbase) __ro_after_init;
/*
* suspend ocram space layout:
@@ -229,8 +232,18 @@ struct imx6_cpu_pm_info {
struct imx6_pm_base l2_base;
u32 mmdc_io_num; /* Number of MMDC IOs which need saved/restored. */
u32 mmdc_io_val[MX6_MAX_MMDC_IO_NUM][2]; /* To save offset and value */
+ u32 cfi_type; /* kCFI type hash of imx6_suspend() */
} __aligned(8);
+/*
+ * The ocram copy of imx6_suspend() starts right after struct imx6_cpu_pm_info,
+ * and the CFI check on the indirect call reads the kCFI type hash from the
+ * four bytes preceding the function entry, so cfi_type must occupy the last
+ * four bytes of the struct, i.e. fit into its tail padding.
+ */
+static_assert(offsetofend(struct imx6_cpu_pm_info, cfi_type) ==
+ sizeof(struct imx6_cpu_pm_info));
+
void imx6_set_int_mem_clk_lpm(bool enable)
{
u32 val = readl_relaxed(ccm_base + CGPR);
@@ -568,6 +581,13 @@ static int __init imx6q_suspend_init(const struct imx6_pm_socdata *socdata)
mmdc_offset_array[i]);
}
+ /*
+ * Mask out the Thumb bit, as cfi_get_func_hash() expects the
+ * function's actual start address. Returns 0 if CONFIG_CFI=n.
+ */
+ pm_info->cfi_type =
+ cfi_get_func_hash((void *)((uintptr_t)&imx6_suspend & ~1UL));
+
imx6_suspend_in_ocram_fn = fncpy(
suspend_ocram_base + sizeof(*pm_info),
&imx6_suspend,
diff --git a/arch/arm/mach-imx/suspend-imx6.S b/arch/arm/mach-imx/suspend-imx6.S
index 63ccc2d0e920..95d5f723ea0e 100644
--- a/arch/arm/mach-imx/suspend-imx6.S
+++ b/arch/arm/mach-imx/suspend-imx6.S
@@ -3,6 +3,7 @@
* Copyright 2014 Freescale Semiconductor, Inc.
*/
+#include <linux/cfi_types.h>
#include <linux/linkage.h>
#include <asm/assembler.h>
#include <asm/asm-offsets.h>
@@ -148,7 +149,8 @@
.endm
-ENTRY(imx6_suspend)
+/* fncpy() requires an 8-byte-aligned entry; see arch/arm/include/asm/fncpy.h */
+SYM_TYPED_FUNC_START_ALIGNED(imx6_suspend, 8)
ldr r1, [r0, #PM_INFO_PBASE_OFFSET]
ldr r2, [r0, #PM_INFO_RESUME_ADDR_OFFSET]
ldr r3, [r0, #PM_INFO_DDR_TYPE_OFFSET]
@@ -329,4 +331,4 @@ resume:
resume_mmdc
ret lr
-ENDPROC(imx6_suspend)
+SYM_FUNC_END(imx6_suspend)
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 812380f30ae3..2b470a419dda 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -341,6 +341,8 @@ static int
do_alignment_ldrhstrh(unsigned long addr, u32 instr, struct pt_regs *regs)
{
unsigned int rd = RD_BITS(instr);
+ unsigned int ua_flags;
+ bool ua_saved = false;
ai_half += 1;
@@ -364,10 +366,11 @@ do_alignment_ldrhstrh(unsigned long addr, u32 instr, struct pt_regs *regs)
user:
if (LDST_L_BIT(instr)) {
unsigned long val;
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
get16t_unaligned_check(val, addr);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
/* signed half-word? */
if (instr & 0x40)
@@ -375,14 +378,17 @@ do_alignment_ldrhstrh(unsigned long addr, u32 instr, struct pt_regs *regs)
regs->uregs[rd] = val;
} else {
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
put16t_unaligned_check(regs->uregs[rd], addr);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
}
return TYPE_LDST;
fault:
+ if (ua_saved)
+ uaccess_restore(ua_flags);
return TYPE_FAULT;
}
@@ -391,6 +397,8 @@ do_alignment_ldrdstrd(unsigned long addr, u32 instr, struct pt_regs *regs)
{
unsigned int rd = RD_BITS(instr);
unsigned int rd2;
+ unsigned int ua_flags;
+ bool ua_saved = false;
int load;
if ((instr & 0xfe000000) == 0xe8000000) {
@@ -425,26 +433,30 @@ do_alignment_ldrdstrd(unsigned long addr, u32 instr, struct pt_regs *regs)
user:
if (load) {
unsigned long val, val2;
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
get32t_unaligned_check(val, addr);
get32t_unaligned_check(val2, addr + 4);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
regs->uregs[rd] = val;
regs->uregs[rd2] = val2;
} else {
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
put32t_unaligned_check(regs->uregs[rd], addr);
put32t_unaligned_check(regs->uregs[rd2], addr + 4);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
}
return TYPE_LDST;
bad:
return TYPE_ERROR;
fault:
+ if (ua_saved)
+ uaccess_restore(ua_flags);
return TYPE_FAULT;
}
@@ -452,6 +464,8 @@ static int
do_alignment_ldrstr(unsigned long addr, u32 instr, struct pt_regs *regs)
{
unsigned int rd = RD_BITS(instr);
+ unsigned int ua_flags;
+ bool ua_saved = false;
ai_word += 1;
@@ -469,18 +483,22 @@ do_alignment_ldrstr(unsigned long addr, u32 instr, struct pt_regs *regs)
trans:
if (LDST_L_BIT(instr)) {
unsigned int val;
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
get32t_unaligned_check(val, addr);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
regs->uregs[rd] = val;
} else {
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
put32t_unaligned_check(regs->uregs[rd], addr);
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
}
return TYPE_LDST;
fault:
+ if (ua_saved)
+ uaccess_restore(ua_flags);
return TYPE_FAULT;
}
@@ -502,6 +520,8 @@ do_alignment_ldmstm(unsigned long addr, u32 instr, struct pt_regs *regs)
{
unsigned int rd, rn, correction, nr_regs, regbits;
unsigned long eaddr, newaddr;
+ unsigned int ua_flags;
+ bool ua_saved = false;
if (LDM_S_BIT(instr))
goto bad;
@@ -547,7 +567,8 @@ do_alignment_ldmstm(unsigned long addr, u32 instr, struct pt_regs *regs)
#endif
if (user_mode(regs)) {
- unsigned int __ua_flags = uaccess_save_and_enable();
+ ua_flags = uaccess_save_and_enable();
+ ua_saved = true;
for (regbits = REGMASK_BITS(instr), rd = 0; regbits;
regbits >>= 1, rd += 1)
if (regbits & 1) {
@@ -559,7 +580,7 @@ do_alignment_ldmstm(unsigned long addr, u32 instr, struct pt_regs *regs)
put32t_unaligned_check(regs->uregs[rd], eaddr);
eaddr += 4;
}
- uaccess_restore(__ua_flags);
+ uaccess_restore(ua_flags);
} else {
for (regbits = REGMASK_BITS(instr), rd = 0; regbits;
regbits >>= 1, rd += 1)
@@ -581,6 +602,8 @@ do_alignment_ldmstm(unsigned long addr, u32 instr, struct pt_regs *regs)
return TYPE_DONE;
fault:
+ if (ua_saved)
+ uaccess_restore(ua_flags);
regs->ARM_pc -= correction;
return TYPE_FAULT;
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index cab741a695f5..70994918fab5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -198,6 +198,7 @@ config ARM64
select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_BUILDTIME_MCOUNT_SORT
select HAVE_EFFICIENT_UNALIGNED_ACCESS
+ select HAVE_EXIT_THREAD
select HAVE_GUP_FAST
select HAVE_FTRACE_GRAPH_FUNC
select HAVE_FUNCTION_TRACER
diff --git a/arch/arm64/boot/dts/freescale/imx8mm-verdin-ivy.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-verdin-ivy.dtsi
index 29075ff5eda6..b64751aa31d5 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-verdin-ivy.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-verdin-ivy.dtsi
@@ -270,8 +270,8 @@ &gpio3 {
"",
"",
"",
- "DIG_1", /* SODIMM 56 */
- "DIG_2", /* SODIMM 58 */
+ "DIGI_1", /* SODIMM 56 */
+ "DIGI_2", /* SODIMM 58 */
"REL1", /* SODIMM 60 */
"REL2", /* SODIMM 62 */
"", /* 10 */
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-toradex-smarc.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-toradex-smarc.dtsi
index 0348da385f23..eb8b9db7e3cd 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-toradex-smarc.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-toradex-smarc.dtsi
@@ -323,7 +323,7 @@ &gpio3 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_lvds_dsi_sel>;
gpio-line-names = "ETH_0_INT#", /* 0 */
- "SLEEP#",
+ "SMARC_SLEEP#",
"",
"",
"",
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-verdin-ivy.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-verdin-ivy.dtsi
index db1b4ee7728c..bf021e85cfc9 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-verdin-ivy.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-verdin-ivy.dtsi
@@ -333,8 +333,8 @@ &gpio3 {
"",
"",
"",
- "DIG_1", /* SODIMM 56 */
- "DIG_2", /* SODIMM 58 */
+ "DIGI_1", /* SODIMM 56 */
+ "DIGI_2", /* SODIMM 58 */
"REL1", /* SODIMM 60 */
"REL2", /* SODIMM 62 */
"", /* 10 */
diff --git a/arch/arm64/boot/dts/freescale/imx8mp-verdin-yavia.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-verdin-yavia.dtsi
index f3d28e23ba6c..2a90f312fb27 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-verdin-yavia.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-verdin-yavia.dtsi
@@ -117,6 +117,11 @@ &flexcan1 {
status = "okay";
};
+&gpio3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_gpios_ext_yavia>;
+};
+
&gpio4 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_ctrl_sleep_moci>;
@@ -266,7 +271,7 @@ MX8MP_IOMUXC_NAND_DATA03__GPIO3_IO09 0x106 /* SODIMM 62 */
pinctrl_gpios_ext_yavia: gpiosextyaviagrp {
fsl,pins = <
- MX8MP_IOMUXC_NAND_CE1_B__GPIO3_IO02 0x106 /* SODIMM 64 */
+ MX8MP_IOMUXC_NAND_READY_B__GPIO3_IO16 0x106 /* SODIMM 64 */
MX8MP_IOMUXC_NAND_DQS__GPIO3_IO14 0x106 /* SODIMM 66 */
>;
};
diff --git a/arch/arm64/boot/dts/freescale/imx95-aquila-clover.dts b/arch/arm64/boot/dts/freescale/imx95-aquila-clover.dts
index fd9304331446..435567b25708 100644
--- a/arch/arm64/boot/dts/freescale/imx95-aquila-clover.dts
+++ b/arch/arm64/boot/dts/freescale/imx95-aquila-clover.dts
@@ -188,6 +188,11 @@ &lpi2c5 {
status = "okay";
};
+/* Aquila I2C_3_DSI1 */
+&lpi2c8 {
+ status = "okay";
+};
+
/* Aquila SPI_1 */
&lpspi6 {
pinctrl-names = "default";
diff --git a/arch/arm64/boot/dts/freescale/imx95-aquila.dtsi b/arch/arm64/boot/dts/freescale/imx95-aquila.dtsi
index 69dc962a24a1..4a809e1a840e 100644
--- a/arch/arm64/boot/dts/freescale/imx95-aquila.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx95-aquila.dtsi
@@ -406,10 +406,10 @@ som_gpio_expander_1: gpio@21 {
#gpio-cells = <2>;
gpio-controller;
gpio-line-names =
- "AQUILA_C1", /* 0 */
- "AQUILA_C2",
- "AQUILA_C3",
- "AQUILA_C4",
+ "AQUILA_C01", /* 0 */
+ "AQUILA_C02",
+ "AQUILA_C03",
+ "AQUILA_C04",
"AQUILA_C36",
"AQUILA_B74",
"AQUILA_B75",
diff --git a/arch/arm64/boot/dts/freescale/imx95-toradex-smarc.dtsi b/arch/arm64/boot/dts/freescale/imx95-toradex-smarc.dtsi
index a6c5398a81e3..4c102f213214 100644
--- a/arch/arm64/boot/dts/freescale/imx95-toradex-smarc.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx95-toradex-smarc.dtsi
@@ -839,6 +839,7 @@ &usdhc3 {
keep-power-in-suspend;
non-removable;
vmmc-supply = <®_wifi_en>;
+ status = "okay";
};
&scmi_bbm {
diff --git a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
index 9ebc196107e5..318639a7c64e 100644
--- a/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
+++ b/arch/arm64/boot/dts/mediatek/mt7986a.dtsi
@@ -657,6 +657,7 @@ cpu_trip_active_low: active-low {
timer {
compatible = "arm,armv8-timer";
interrupt-parent = <&gic>;
+ clock-frequency = <12986200>;
interrupts = <GIC_PPI 13 IRQ_TYPE_LEVEL_LOW>,
<GIC_PPI 14 IRQ_TYPE_LEVEL_LOW>,
<GIC_PPI 11 IRQ_TYPE_LEVEL_LOW>,
diff --git a/arch/arm64/boot/dts/qcom/ipq9574.dtsi b/arch/arm64/boot/dts/qcom/ipq9574.dtsi
index 5a5bda5f21a1..dd1dcd955c95 100644
--- a/arch/arm64/boot/dts/qcom/ipq9574.dtsi
+++ b/arch/arm64/boot/dts/qcom/ipq9574.dtsi
@@ -428,7 +428,7 @@ gcc: clock-controller@1800000 {
<&pcie1_phy>,
<&pcie2_phy>,
<&pcie3_phy>,
- <0>;
+ <&usb_0_qmpphy>;
#clock-cells = <1>;
#reset-cells = <1>;
#interconnect-cells = <1>;
diff --git a/arch/arm64/boot/dts/qcom/sdm845-google-common.dtsi b/arch/arm64/boot/dts/qcom/sdm845-google-common.dtsi
index 9e952f9862f2..f73d4ea57705 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-google-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845-google-common.dtsi
@@ -392,6 +392,8 @@ &gpu_zap_shader {
};
&i2c3 {
+ clock-frequency = <400000>;
+
status = "okay";
nfc@28 {
diff --git a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
index 080f5294988d..35babe8c5f75 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-beryllium-common.dtsi
@@ -27,6 +27,7 @@
/delete-node/ &slpi_mem;
/delete-node/ &spss_mem;
/delete-node/ &rmtfs_mem;
+/delete-node/ &xbl_mem;
/ {
chassis-type = "handset";
@@ -71,6 +72,11 @@ key-vol-up {
/* Reserved memory changes from downstream */
reserved-memory {
+ xbl-mem@85d00000 {
+ reg = <0 0x85d00000 0 0x200000>;
+ no-map;
+ };
+
tz_mem: memory@86200000 {
reg = <0 0x86200000 0 0x4900000>;
no-map;
diff --git a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-polaris.dts b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-polaris.dts
index d3162e444453..a782023d0dcb 100644
--- a/arch/arm64/boot/dts/qcom/sdm845-xiaomi-polaris.dts
+++ b/arch/arm64/boot/dts/qcom/sdm845-xiaomi-polaris.dts
@@ -32,6 +32,7 @@
/delete-node/ &mba_region;
/delete-node/ &slpi_mem;
/delete-node/ &spss_mem;
+/delete-node/ &xbl_mem;
/ {
model = "Xiaomi Mi MIX 2S";
@@ -63,6 +64,11 @@ key-vol-up {
};
reserved-memory {
+ xbl-mem@85d00000 {
+ reg = <0 0x85d00000 0 0x200000>;
+ no-map;
+ };
+
adsp_mem: memory@8c500000 {
reg = <0 0x8c500000 0 0x1e00000>;
no-map;
diff --git a/arch/arm64/boot/dts/qcom/shikra.dtsi b/arch/arm64/boot/dts/qcom/shikra.dtsi
index 92d22e3b6931..65cc94b9a624 100644
--- a/arch/arm64/boot/dts/qcom/shikra.dtsi
+++ b/arch/arm64/boot/dts/qcom/shikra.dtsi
@@ -1948,12 +1948,6 @@ compute-cb@6 {
reg = <6>;
iommus = <&apps_smmu 0x0206 0x0000>;
};
-
- compute-cb@9 {
- compatible = "qcom,fastrpc-compute-cb";
- reg = <9>;
- iommus = <&apps_smmu 0x0209 0x0000>;
- };
};
};
};
diff --git a/arch/arm64/boot/dts/ti/Makefile b/arch/arm64/boot/dts/ti/Makefile
index 795978e962ee..ec03d594f23a 100644
--- a/arch/arm64/boot/dts/ti/Makefile
+++ b/arch/arm64/boot/dts/ti/Makefile
@@ -91,6 +91,7 @@ dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-nonwifi-dev.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-nonwifi-ivy.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-nonwifi-mallow.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-nonwifi-yavia.dtb
+dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-nonwifi-zinnia.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-ov5640-24mhz.dtbo
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-ov5640.dtbo
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-dahlia.dtb
@@ -98,6 +99,7 @@ dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-dev.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-ivy.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-mallow.dtb
dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-yavia.dtb
+dtb-$(CONFIG_ARCH_K3) += k3-am62p5-verdin-wifi-zinnia.dtb
# Common overlays for SK-AM62* family of boards
dtb-$(CONFIG_ARCH_K3) += k3-am62x-sk-csi2-ov5640.dtbo
diff --git a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
index dfd65c00f5b9..7fd508f4eddf 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62-lp-sk.dts
@@ -86,7 +86,7 @@ main_mmc0_pins_default: main-mmc0-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (V3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (Y1) MMC0_CLK */
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (Y1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (V2) MMC0_DAT0 */
AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (V1) MMC0_DAT1 */
AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (W2) MMC0_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62-phycore-som.dtsi b/arch/arm64/boot/dts/ti/k3-am62-phycore-som.dtsi
index 3baa653257bb..ce3e5165bd31 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-phycore-som.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62-phycore-som.dtsi
@@ -153,7 +153,7 @@ AM62X_IOPAD(0x15c, PIN_INPUT, 0) /* (AB22) MDIO0_MDIO */
main_mmc0_pins_default: main-mmc0-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x220, PIN_INPUT_PULLUP, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT_PULLDOWN, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x218, PIN_OUTPUT_PULLDOWN, 0) /* (AB1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT_PULLUP, 0) /* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62-pocketbeagle2.dts b/arch/arm64/boot/dts/ti/k3-am62-pocketbeagle2.dts
index 7a4cffc27bda..1c691629d4ad 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-pocketbeagle2.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62-pocketbeagle2.dts
@@ -235,7 +235,7 @@ AM62X_IOPAD(0x0020, PIN_OUTPUT, 3) /* (J25) OSPI0_D5.UART6_TXD */
main_mmc1_pins_default: main-mmc1-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x23c, PIN_INPUT, 0) /* (A21/C18) MMC1_CMD */
- AM62X_IOPAD(0x234, PIN_INPUT, 0) /* (B22/A20) MMC1_CLK */
+ AM62X_IOPAD(0x234, PIN_OUTPUT, 0) /* (B22/A20) MMC1_CLK */
AM62X_IOPAD(0x230, PIN_INPUT, 0) /* (A22/A19) MMC1_DAT0 */
AM62X_IOPAD(0x22c, PIN_INPUT, 0) /* (B21/B19) MMC1_DAT1 */
AM62X_IOPAD(0x228, PIN_INPUT, 0) /* (C21/B20) MMC1_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi b/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
index e97b2b047d10..895b1b7dcac7 100644
--- a/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62-verdin.dtsi
@@ -573,7 +573,7 @@ AM62X_IOPAD(0x15c, PIN_INPUT, 0) /* (AB22) MDIO0_MDIO */ /* ETH_1_MDIO, SODIMM
pinctrl_sdhci0: main-mmc0-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
@@ -589,7 +589,7 @@ AM62X_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */
pinctrl_sdhci1: main-mmc1-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x23c, PIN_INPUT, 0) /* (A21) MMC1_CMD */ /* SODIMM 74 */
- AM62X_IOPAD(0x234, PIN_INPUT, 0) /* (B22) MMC1_CLK */ /* SODIMM 78 */
+ AM62X_IOPAD(0x234, PIN_OUTPUT, 0) /* (B22) MMC1_CLK */ /* SODIMM 78 */
AM62X_IOPAD(0x230, PIN_INPUT, 0) /* (A22) MMC1_DAT0 */ /* SODIMM 80 */
AM62X_IOPAD(0x22c, PIN_INPUT, 0) /* (B21) MMC1_DAT1 */ /* SODIMM 82 */
AM62X_IOPAD(0x228, PIN_INPUT, 0) /* (C21) MMC1_DAT2 */ /* SODIMM 70 */
@@ -601,7 +601,7 @@ AM62X_IOPAD(0x224, PIN_INPUT, 0) /* (D22) MMC1_DAT3 */ /* SODIMM 72 */
pinctrl_sdhci2: main-mmc2-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x120, PIN_INPUT, 0) /* (C24) MMC2_CMD */ /* WiFi_SDIO_CMD */
- AM62X_IOPAD(0x118, PIN_INPUT, 0) /* (D25) MMC2_CLK */ /* WiFi_SDIO_CLK */
+ AM62X_IOPAD(0x118, PIN_OUTPUT,0) /* (D25) MMC2_CLK */ /* WiFi_SDIO_CLK */
AM62X_IOPAD(0x114, PIN_INPUT, 0) /* (B24) MMC2_DAT0 */ /* WiFi_SDIO_DATA0 */
AM62X_IOPAD(0x110, PIN_INPUT, 0) /* (C25) MMC2_DAT1 */ /* WiFi_SDIO_DATA1 */
AM62X_IOPAD(0x10c, PIN_INPUT, 0) /* (E23) MMC2_DAT2 */ /* WiFi_SDIO_DATA2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-beagleplay.dts b/arch/arm64/boot/dts/ti/k3-am625-beagleplay.dts
index c468b9c5fc09..898e4ca3fd87 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-beagleplay.dts
+++ b/arch/arm64/boot/dts/ti/k3-am625-beagleplay.dts
@@ -323,7 +323,7 @@ emmc_pins_default: emmc-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x0220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x0218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x0218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
AM62X_IOPAD(0x0214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x0210, PIN_INPUT, 0) /* (AA1) MMC0_DAT1 */
AM62X_IOPAD(0x020c, PIN_INPUT, 0) /* (AA3) MMC0_DAT2 */
@@ -346,7 +346,7 @@ sd_pins_default: sd-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x023c, PIN_INPUT, 0) /* (A21) MMC1_CMD */
- AM62X_IOPAD(0x0234, PIN_INPUT, 0) /* (B22) MMC1_CLK */
+ AM62X_IOPAD(0x0234, PIN_OUTPUT, 0) /* (B22) MMC1_CLK */
AM62X_IOPAD(0x0230, PIN_INPUT, 0) /* (A22) MMC1_DAT0 */
AM62X_IOPAD(0x022c, PIN_INPUT, 0) /* (B21) MMC1_DAT1 */
AM62X_IOPAD(0x0228, PIN_INPUT, 0) /* (C21) MMC1_DAT2 */
@@ -358,7 +358,7 @@ AM62X_IOPAD(0x0240, PIN_INPUT, 7) /* (D17) MMC1_SDCD.GPIO1_48 */
wifi_pins_default: wifi-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x0120, PIN_INPUT, 0) /* (C24) MMC2_CMD */
- AM62X_IOPAD(0x0118, PIN_INPUT, 0) /* (D25) MMC2_CLK */
+ AM62X_IOPAD(0x0118, PIN_OUTPUT, 0) /* (D25) MMC2_CLK */
AM62X_IOPAD(0x0114, PIN_INPUT, 0) /* (B24) MMC2_DAT0 */
AM62X_IOPAD(0x0110, PIN_INPUT, 0) /* (C25) MMC2_DAT1 */
AM62X_IOPAD(0x010c, PIN_INPUT, 0) /* (E23) MMC2_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi
index 9c8362682645..085b067d96fb 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am625-sk-common.dtsi
@@ -98,7 +98,7 @@ main_mmc0_pins_default: main-mmc0-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
AM62X_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-tqma62xx-mba62xx.dts b/arch/arm64/boot/dts/ti/k3-am625-tqma62xx-mba62xx.dts
index efc00d2a860a..3859512c4262 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-tqma62xx-mba62xx.dts
+++ b/arch/arm64/boot/dts/ti/k3-am625-tqma62xx-mba62xx.dts
@@ -735,7 +735,7 @@ main_mmc1_pins: main-mmc1-pins {
/* (A21) MMC1_CMD */
AM62X_IOPAD(0x23c, PIN_INPUT, 0)
/* (B22) MMC1_CLK */
- AM62X_IOPAD(0x234, PIN_INPUT, 0)
+ AM62X_IOPAD(0x234, PIN_OUTPUT, 0)
/* (A22) MMC1_DAT0 */
AM62X_IOPAD(0x230, PIN_INPUT, 0)
/* (B21) MMC1_DAT1 */
@@ -755,7 +755,7 @@ main_mmc2_pins: main-mmc2-pins {
/* (C24) MMC2_CMD */
AM62X_IOPAD(0x120, PIN_INPUT, 0)
/* (D25) MMC2_CLK */
- AM62X_IOPAD(0x118, PIN_INPUT, 0)
+ AM62X_IOPAD(0x118, PIN_OUTPUT, 0)
/* (B24) MMC2_DAT0 */
AM62X_IOPAD(0x114, PIN_INPUT, 0)
/* (C25) MMC2_DAT1 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-tqma62xx.dtsi b/arch/arm64/boot/dts/ti/k3-am625-tqma62xx.dtsi
index 72288678cd01..a200a2afe16f 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-tqma62xx.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am625-tqma62xx.dtsi
@@ -299,7 +299,7 @@ main_mmc0_pins: main-mmc0-pins {
/* (Y3) MMC0_CMD */
AM62X_IOPAD(0x220, PIN_INPUT, 0)
/* (AB1) MMC0_CLK */
- AM62X_IOPAD(0x218, PIN_INPUT, 0)
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0)
/* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x214, PIN_INPUT, 0)
/* (AA1) MMC0_DAT1 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-var-som-symphony.dts b/arch/arm64/boot/dts/ti/k3-am625-var-som-symphony.dts
index 7951d7ec952b..082bf81143dc 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-var-som-symphony.dts
+++ b/arch/arm64/boot/dts/ti/k3-am625-var-som-symphony.dts
@@ -391,7 +391,7 @@ pinctrl_mmc1: main-mmc1-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x23c, PIN_INPUT, 0) /* (A21) MMC1_CMD */
- AM62X_IOPAD(0x234, PIN_INPUT, 0) /* (B22) MMC1_CLK */
+ AM62X_IOPAD(0x234, PIN_OUTPUT, 0) /* (B22) MMC1_CLK */
AM62X_IOPAD(0x230, PIN_INPUT, 0) /* (A22) MMC1_DAT0 */
AM62X_IOPAD(0x22c, PIN_INPUT, 0) /* (B21) MMC1_DAT1 */
AM62X_IOPAD(0x228, PIN_INPUT, 0) /* (C21) MMC1_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am625-var-som.dtsi b/arch/arm64/boot/dts/ti/k3-am625-var-som.dtsi
index c2daa56c35e1..c23b6a90b8b1 100644
--- a/arch/arm64/boot/dts/ti/k3-am625-var-som.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am625-var-som.dtsi
@@ -277,7 +277,7 @@ pinctrl_mmc0: main-mmc0-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62X_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
+ AM62X_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
AM62X_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
AM62X_IOPAD(0x210, PIN_INPUT, 0) /* (AA1) MMC0_DAT1 */
AM62X_IOPAD(0x20c, PIN_INPUT, 0) /* (AA3) MMC0_DAT2 */
@@ -292,7 +292,7 @@ AM62X_IOPAD(0x1f8, PIN_INPUT, 0) /* (AC2) MMC0_DAT7 */
pinctrl_mmc2: main-mmc2-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x120, PIN_INPUT_PULLUP, 0) /* (C24) MMC2_CMD */
- AM62X_IOPAD(0x118, PIN_INPUT_PULLDOWN, 0) /* (D25) MMC2_CLK */
+ AM62X_IOPAD(0x118, PIN_OUTPUT_PULLDOWN, 0) /* (D25) MMC2_CLK */
AM62X_IOPAD(0x114, PIN_INPUT_PULLUP, 0) /* (B24) MMC2_DAT0 */
AM62X_IOPAD(0x110, PIN_INPUT_PULLUP, 0) /* (C25) MMC2_DAT1 */
AM62X_IOPAD(0x10c, PIN_INPUT_PULLUP, 0) /* (E23) MMC2_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62a-phycore-som.dtsi b/arch/arm64/boot/dts/ti/k3-am62a-phycore-som.dtsi
index de4048a3564b..5ad93be3d336 100644
--- a/arch/arm64/boot/dts/ti/k3-am62a-phycore-som.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62a-phycore-som.dtsi
@@ -120,7 +120,7 @@ AM62AX_IOPAD(0x15c, PIN_INPUT, 0) /* (V13) MDIO0_MDIO */
main_mmc0_pins_default: main-mmc0-default-pins {
pinctrl-single,pins = <
AM62AX_IOPAD(0x220, PIN_INPUT_PULLUP, 0) /* (Y6) MMC0_CMD */
- AM62AX_IOPAD(0x218, PIN_INPUT_PULLDOWN, 0) /* (AB7) MMC0_CLK */
+ AM62AX_IOPAD(0x218, PIN_OUTPUT_PULLDOWN, 0) /* (AB7) MMC0_CLK */
AM62AX_IOPAD(0x214, PIN_INPUT_PULLUP, 0) /* (AA6) MMC0_DAT0 */
AM62AX_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AB6) MMC0_DAT1 */
AM62AX_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (Y7) MMC0_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts b/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
index d3b3675e7a8f..e63c5200a237 100644
--- a/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
+++ b/arch/arm64/boot/dts/ti/k3-am62a7-sk.dts
@@ -328,8 +328,7 @@ AM62AX_IOPAD(0x0b4, PIN_INPUT_PULLUP, 1) /* (K24) GPMC0_CSn3.I2C2_SDA */
main_mmc0_pins_default: main-mmc0-default-pins {
pinctrl-single,pins = <
AM62AX_IOPAD(0x220, PIN_INPUT, 0) /* (Y3) MMC0_CMD */
- AM62AX_IOPAD(0x218, PIN_INPUT, 0) /* (AB1) MMC0_CLKLB */
- AM62AX_IOPAD(0x21c, PIN_INPUT, 0) /* (AB1) MMC0_CLK */
+ AM62AX_IOPAD(0x218, PIN_OUTPUT, 0) /* (AB1) MMC0_CLK */
AM62AX_IOPAD(0x214, PIN_INPUT, 0) /* (AA2) MMC0_DAT0 */
AM62AX_IOPAD(0x210, PIN_INPUT_PULLUP, 0) /* (AA1) MMC0_DAT1 */
AM62AX_IOPAD(0x20c, PIN_INPUT_PULLUP, 0) /* (AA3) MMC0_DAT2 */
@@ -345,7 +344,7 @@ AM62AX_IOPAD(0x1f8, PIN_INPUT_PULLUP, 0) /* (AC2) MMC0_DAT7 */
main_mmc1_pins_default: main-mmc1-default-pins {
pinctrl-single,pins = <
AM62AX_IOPAD(0x23c, PIN_INPUT, 0) /* (A21) MMC1_CMD */
- AM62AX_IOPAD(0x234, PIN_INPUT, 0) /* (B22) MMC1_CLK */
+ AM62AX_IOPAD(0x234, PIN_OUTPUT, 0) /* (B22) MMC1_CLK */
AM62AX_IOPAD(0x230, PIN_INPUT, 0) /* (A22) MMC1_DAT0 */
AM62AX_IOPAD(0x22c, PIN_INPUT, 0) /* (B21) MMC1_DAT1 */
AM62AX_IOPAD(0x228, PIN_INPUT, 0) /* (C21) MMC1_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62x-phyboard-lyra.dtsi b/arch/arm64/boot/dts/ti/k3-am62x-phyboard-lyra.dtsi
index d6e70ee15938..c9b3e0518faf 100644
--- a/arch/arm64/boot/dts/ti/k3-am62x-phyboard-lyra.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62x-phyboard-lyra.dtsi
@@ -227,7 +227,7 @@ AM62X_IOPAD(0x068, PIN_INPUT, 3) /* (R21) GPMC0_AD11.MCASP2_AXR3 */
main_mmc1_pins_default: main-mmc1-default-pins {
pinctrl-single,pins = <
AM62X_IOPAD(0x23c, PIN_INPUT_PULLUP, 0) /* (A21) MMC1_CMD */
- AM62X_IOPAD(0x234, PIN_INPUT_PULLDOWN, 0) /* (B22) MMC1_CLK */
+ AM62X_IOPAD(0x234, PIN_OUTPUT_PULLDOWN, 0) /* (B22) MMC1_CLK */
AM62X_IOPAD(0x230, PIN_INPUT_PULLUP, 0) /* (A22) MMC1_DAT0 */
AM62X_IOPAD(0x22c, PIN_INPUT_PULLUP, 0) /* (B21) MMC1_DAT1 */
AM62X_IOPAD(0x228, PIN_INPUT_PULLUP, 0) /* (C21) MMC1_DAT2 */
diff --git a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
index ab9e58c2d225..d2f41e2ed623 100644
--- a/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am62x-sk-common.dtsi
@@ -194,7 +194,7 @@ main_mmc1_pins_default: main-mmc1-default-pins {
bootph-all;
pinctrl-single,pins = <
AM62X_IOPAD(0x23c, PIN_INPUT, 0) /* (A21/C18) MMC1_CMD */
- AM62X_IOPAD(0x234, PIN_INPUT, 0) /* (B22/A20) MMC1_CLK */
+ AM62X_IOPAD(0x234, PIN_OUTPUT, 0) /* (B22/A20) MMC1_CLK */
AM62X_IOPAD(0x230, PIN_INPUT, 0) /* (A22/A19) MMC1_DAT0 */
AM62X_IOPAD(0x22c, PIN_INPUT, 0) /* (B21/B19) MMC1_DAT1 */
AM62X_IOPAD(0x228, PIN_INPUT, 0) /* (C21/B20) MMC1_DAT2 */
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 4bfbd827c5aa..b3e9dd302730 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -98,7 +98,7 @@
* TID1: Trap REVIDR_EL1, AIDR_EL1, and SMIDR_EL1
*/
#define HCR_GUEST_FLAGS (HCR_TSC | HCR_TSW | HCR_TWE | HCR_TWI | HCR_VM | \
- HCR_BSU_IS | HCR_FB | HCR_TACR | \
+ SYS_FIELD_PREP_ENUM(HCR_EL2, BSU, IS) | HCR_EL2_FB | HCR_EL2_TACR | \
HCR_AMO | HCR_SWIO | HCR_TIDCP | HCR_RW | HCR_TLOR | \
HCR_FMO | HCR_IMO | HCR_PTW | HCR_TID3 | HCR_TID1)
#define HCR_HOST_NVHE_FLAGS (HCR_RW | HCR_API | HCR_APK)
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index cd9b9d2462f9..d7caf8c4a11e 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -889,11 +889,10 @@ struct kvm_vcpu_arch {
/*
* Don't run the guest (internal implementation need).
*
- * Contrary to the flags above, this is set/cleared outside of
- * a vcpu context, and thus cannot be mixed with the flags
- * themselves (or the flag accesses need to be made atomic).
+ * Contrary to the flags above, this is updated outside of
+ * a vcpu context, and thus cannot be mixed with the flags.
*/
- bool pause;
+ atomic_t pause;
/*
* We maintain more than a single set of debug registers to support
@@ -1659,6 +1658,7 @@ static inline bool kvm_arch_has_irq_bypass(void)
return true;
}
+void kvm_idregs_enforce_dependencies(struct kvm *kvm);
void compute_fgu(struct kvm *kvm, enum fgt_group_id fgt);
struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg);
void check_feature_map(void);
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6eae7e7e2a68..ac2b0637692a 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -392,12 +392,10 @@ static inline bool kvm_supports_cacheable_pfnmap(void)
#ifdef CONFIG_PTDUMP_STAGE2_DEBUGFS
void kvm_s2_ptdump_create_debugfs(struct kvm *kvm);
-void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu);
-void kvm_nested_s2_ptdump_remove_debugfs(struct kvm_s2_mmu *mmu);
+void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu, int idx);
#else
static inline void kvm_s2_ptdump_create_debugfs(struct kvm *kvm) {}
-static inline void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu) {}
-static inline void kvm_nested_s2_ptdump_remove_debugfs(struct kvm_s2_mmu *mmu) {}
+static inline void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu, int idx) {}
#endif /* CONFIG_PTDUMP_STAGE2_DEBUGFS */
#endif /* __ASSEMBLER__ */
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 6000905a2e86..6c81a3d22d94 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -1651,6 +1651,9 @@ extern void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned long ad
* The contpte APIs are used to transparently manage the contiguous bit in ptes
* where it is possible and makes sense to do so. The PTE_CONT bit is considered
* a private implementation detail of the public ptep API (see below).
+ * Single-PTE transparent contpte helpers use pte_valid_cont() to identify
+ * grouped PTEs; explicit contiguous-encoding users such as hugetlb manage
+ * their entries separately.
*/
extern void __contpte_try_fold(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte);
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index 0ec90fd1754e..6f34aef3d04e 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -117,8 +117,14 @@ static int __init set_permissions(pte_t *ptep, unsigned long addr, void *data)
const efi_memory_desc_t *md = spd->md;
pte_t pte = __ptep_get(ptep);
+ /*
+ * Clear PTE_WRITE as well: it doubles as the DBM bit, and leaving it
+ * set would permit the hardware to make the mapping writable again
+ * when TCR_EL1.HD is set.
+ */
if (md->attribute & EFI_MEMORY_RO)
- pte = set_pte_bit(pte, __pgprot(PTE_RDONLY));
+ pte = set_pte_bit(clear_pte_bit(pte, __pgprot(PTE_WRITE)),
+ __pgprot(PTE_RDONLY));
if (md->attribute & EFI_MEMORY_XP)
pte = set_pte_bit(pte, __pgprot(PTE_PXN));
else if (system_supports_bti_kernel() && spd->has_bti)
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 581f80e9b9b7..42b15b0c069f 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -320,9 +320,18 @@ static int copy_thread_gcs(struct task_struct *p,
return 0;
}
+static void gcs_reset_thread(struct task_struct *task)
+{
+ task->thread.gcs_base = 0;
+ task->thread.gcs_size = 0;
+ task->thread.gcspr_el0 = 0;
+}
+
#else
static void flush_gcs(void) { }
+static void gcs_reset_thread(struct task_struct *task) { }
+
static int copy_thread_gcs(struct task_struct *p,
const struct kernel_clone_args *args)
{
@@ -341,6 +350,12 @@ void flush_thread(void)
flush_gcs();
}
+void exit_thread(struct task_struct *tsk)
+{
+ /* Failed fork after copy_thread(); deactivate_mm() does not run. */
+ gcs_free(tsk);
+}
+
void arch_release_task_struct(struct task_struct *tsk)
{
fpsimd_release_task(tsk);
@@ -503,6 +518,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.cpu_context.x19 = (unsigned long)args->fn;
p->thread.cpu_context.x20 = (unsigned long)args->fn_arg;
+ /* Kernel threads sharing the parent's mm must not free its GCS. */
+ gcs_reset_thread(p);
if (system_supports_poe())
p->thread.por_el0 = POR_EL0_INIT;
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 806373aac3db..b8baba281166 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -426,7 +426,7 @@ int counters_read_on_cpu(int cpu, smp_call_func_t func, void *val)
return -EPERM;
func(val);
} else {
- smp_call_function_single(cpu, func, val, 1);
+ return smp_call_function_single(cpu, func, val, 1);
}
return 0;
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 449154f9a485..e179b8ad79e8 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -74,7 +74,7 @@ if NVHE_EL2_DEBUG
config NVHE_EL2_TRACING
bool
- depends on TRACING && FTRACE
+ depends on FTRACE
select TRACE_REMOTE
default y
diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 6ac3321f4c57..9278b5383c04 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -528,17 +528,11 @@ static void timer_save_state(struct arch_timer_context *ctx)
goto out;
switch (index) {
- u64 cval;
-
case TIMER_VTIMER:
case TIMER_HVTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTV_CTL));
- cval = read_sysreg_el0(SYS_CNTV_CVAL);
-
- if (has_broken_cntvoff())
- cval -= timer_get_offset(ctx);
-
- timer_set_cval(ctx, cval);
+ if (!has_broken_cntvoff() || !timer_get_offset(ctx))
+ timer_set_cval(ctx, read_sysreg_el0(SYS_CNTV_CVAL));
/* Disable the timer */
write_sysreg_el0(0, SYS_CNTV_CTL);
@@ -564,11 +558,12 @@ static void timer_save_state(struct arch_timer_context *ctx)
case TIMER_PTIMER:
case TIMER_HPTIMER:
timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL));
- cval = read_sysreg_el0(SYS_CNTP_CVAL);
-
- cval -= timer_get_offset(ctx);
-
- timer_set_cval(ctx, cval);
+ /*
+ * With an offset, memory already holds the guest's CVAL (the
+ * trap handler or __deactivate_traps() wrote it).
+ */
+ if (!timer_get_offset(ctx))
+ timer_set_cval(ctx, read_sysreg_el0(SYS_CNTP_CVAL));
/* Disable the timer */
write_sysreg_el0(0, SYS_CNTP_CTL);
@@ -647,7 +642,8 @@ static void timer_restore_state(struct arch_timer_context *ctx)
offset = timer_get_offset(ctx);
if (has_broken_cntvoff()) {
set_cntvoff(0);
- cval += offset;
+ if (offset)
+ cval = timer_apply_offset(cval, offset, kvm_phys_timer_read());
} else {
set_cntvoff(offset);
}
@@ -660,7 +656,8 @@ static void timer_restore_state(struct arch_timer_context *ctx)
cval = timer_get_cval(ctx);
offset = timer_get_offset(ctx);
set_cntpoff(offset);
- cval += offset;
+ if (offset)
+ cval = timer_apply_offset(cval, offset, kvm_phys_timer_read());
write_sysreg_el0(cval, SYS_CNTP_CVAL);
isb();
write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTP_CTL);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 0576c2022ef5..9b773ab0b51d 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -565,6 +565,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
kvm_arm_pvtime_vcpu_init(&vcpu->arch);
vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
+ atomic_set(&vcpu->arch.pause, 0);
/*
* This vCPU may have been created after mpidr_data was initialized.
@@ -839,6 +840,11 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
return ret;
}
+static bool vcpu_can_run(struct kvm_vcpu *vcpu)
+{
+ return !kvm_arm_vcpu_stopped(vcpu) && !atomic_read(&vcpu->arch.pause);
+}
+
/**
* kvm_arch_vcpu_runnable - determine if the vcpu can be scheduled
* @v: The VCPU pointer
@@ -854,8 +860,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v)
(kvm_timer_should_notify_user(v) ||
kvm_pmu_should_notify_user(v)));
- return ((irq_lines || kvm_vgic_vcpu_pending_irq(v))
- && !kvm_arm_vcpu_stopped(v) && !v->arch.pause);
+ return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) && vcpu_can_run(v));
}
bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu)
@@ -1018,7 +1023,7 @@ void kvm_arm_halt_guest(struct kvm *kvm)
struct kvm_vcpu *vcpu;
kvm_for_each_vcpu(i, vcpu, kvm)
- vcpu->arch.pause = true;
+ atomic_inc(&vcpu->arch.pause);
kvm_make_all_cpus_request(kvm, KVM_REQ_SLEEP);
}
@@ -1028,8 +1033,8 @@ void kvm_arm_resume_guest(struct kvm *kvm)
struct kvm_vcpu *vcpu;
kvm_for_each_vcpu(i, vcpu, kvm) {
- vcpu->arch.pause = false;
- __kvm_vcpu_wake_up(vcpu);
+ if (atomic_dec_and_test(&vcpu->arch.pause))
+ __kvm_vcpu_wake_up(vcpu);
}
}
@@ -1037,11 +1042,9 @@ static void kvm_vcpu_sleep(struct kvm_vcpu *vcpu)
{
struct rcuwait *wait = kvm_arch_vcpu_get_wait(vcpu);
- rcuwait_wait_event(wait,
- (!kvm_arm_vcpu_stopped(vcpu)) && (!vcpu->arch.pause),
- TASK_INTERRUPTIBLE);
+ rcuwait_wait_event(wait, vcpu_can_run(vcpu), TASK_INTERRUPTIBLE);
- if (kvm_arm_vcpu_stopped(vcpu) || vcpu->arch.pause) {
+ if (!vcpu_can_run(vcpu)) {
/* Awaken to handle a signal, request we sleep again later. */
kvm_make_request(KVM_REQ_SLEEP, vcpu);
}
@@ -1314,38 +1317,40 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
*/
preempt_disable();
- kvm_nested_flush_hwstate(vcpu);
-
if (kvm_vcpu_has_pmu(vcpu))
kvm_pmu_flush_hwstate(vcpu);
- local_irq_disable();
-
- kvm_vgic_flush_hwstate(vcpu);
+ kvm_nested_flush_hwstate(vcpu);
- kvm_pmu_update_vcpu_events(vcpu);
+ local_irq_disable();
/*
* Ensure we set mode to IN_GUEST_MODE after we disable
* interrupts and before the final VCPU requests check.
* See the comment in kvm_vcpu_exiting_guest_mode() and
* Documentation/virt/kvm/vcpu-requests.rst
+ *
+ * Requests to self must be posted before this point, or
+ * won't be taken into account until the next exit. No state
+ * loading other than what happens at vcpu_load() time can
+ * be done before we have checked for pending requests.
*/
smp_store_mb(vcpu->mode, IN_GUEST_MODE);
if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) {
- vcpu->mode = OUTSIDE_GUEST_MODE;
- isb(); /* Ensure work in x_flush_hwstate is committed */
- if (kvm_vcpu_has_pmu(vcpu))
- kvm_pmu_sync_hwstate(vcpu);
if (unlikely(!irqchip_in_kernel(vcpu->kvm)))
kvm_timer_sync_user(vcpu);
- kvm_vgic_sync_hwstate(vcpu);
+ smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
local_irq_enable();
preempt_enable();
continue;
}
+ /* From here onwards, we are commited to entering the guest */
+ kvm_vgic_flush_hwstate(vcpu);
+
+ kvm_pmu_update_vcpu_events(vcpu);
+
kvm_arch_vcpu_ctxflush_fp(vcpu);
/**************************************************************
@@ -1356,7 +1361,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
ret = kvm_arm_vcpu_enter_exit(vcpu);
- vcpu->mode = OUTSIDE_GUEST_MODE;
vcpu->stat.exits++;
/*
* Back from guest
@@ -1390,6 +1394,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
kvm_arch_vcpu_ctxsync_fp(vcpu);
+ /*
+ * The state that was pushed into the CPU on entering
+ * the guest has been synchronised back, let's
+ * advertise we're outside of the guest.
+ */
+ smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE);
+
/*
* We must ensure that any pending interrupts are taken before
* we exit guest timing so that timer ticks are accounted as
diff --git a/arch/arm64/kvm/config.c b/arch/arm64/kvm/config.c
index 1053676551af..cfdb6c713320 100644
--- a/arch/arm64/kvm/config.c
+++ b/arch/arm64/kvm/config.c
@@ -59,6 +59,7 @@ struct reg_feat_map_desc {
const struct reg_bits_to_feat_map feat_map;
const struct reg_bits_to_feat_map *bit_feat_map;
const unsigned int bit_feat_map_sz;
+ const unsigned int idreg_idx;
};
#define __NEEDS_FEAT_3(m, f, w, id, fld, lim) \
@@ -130,6 +131,21 @@ struct reg_feat_map_desc {
.bit_feat_map_sz = ARRAY_SIZE(m), \
}
+/*
+ * Declare the dependency between the features advertised in an id
+ * register, and the set of other features they depend on. This
+ * generates a struct reg_feat_map_desc.
+ */
+#define DECLARE_IDREG_FEAT_MAP(n, r, m, f) \
+ struct reg_feat_map_desc n = { \
+ .name = #r, \
+ .feat_map = NEEDS_FEAT(~(r##_RES0 | \
+ r##_RES1), f), \
+ .bit_feat_map = m, \
+ .bit_feat_map_sz = ARRAY_SIZE(m), \
+ .idreg_idx = IDREG_IDX(SYS_##r), \
+ }
+
#define FEAT_SPE ID_AA64DFR0_EL1, PMSVer, IMP
#define FEAT_BRBE ID_AA64DFR0_EL1, BRBE, IMP
#define FEAT_TRC_SR ID_AA64DFR0_EL1, TraceVer, IMP
@@ -205,6 +221,7 @@ struct reg_feat_map_desc {
#define FEAT_MEC ID_AA64MMFR3_EL1, MEC, IMP
#define FEAT_HAFT ID_AA64MMFR1_EL1, HAFDBS, HAFT
#define FEAT_HDBSS ID_AA64MMFR1_EL1, HAFDBS, HDBSS
+#define FEAT_HPDS ID_AA64MMFR1_EL1, HPDS, IMP
#define FEAT_HPDS2 ID_AA64MMFR1_EL1, HPDS, HPDS2
#define FEAT_BTI ID_AA64PFR1_EL1, BT, IMP
#define FEAT_ExS ID_AA64MMFR0_EL1, EXS, IMP
@@ -280,6 +297,25 @@ static bool feat_anerr(struct kvm *kvm)
kvm_has_feat(kvm, ID_AA64MMFR3_EL1, SNERR, FEAT_ANERR));
}
+/*
+ * S1PIE and S1POE imply TCR2 and ATS1A, and S1POE also implies HPDS, but
+ * userspace can set them independently.
+ */
+static bool feat_s1pie(struct kvm *kvm)
+{
+ return (kvm_has_feat(kvm, FEAT_S1PIE) &&
+ kvm_has_feat(kvm, FEAT_TCR2) &&
+ kvm_has_feat(kvm, FEAT_ATS1A));
+}
+
+static bool feat_s1poe(struct kvm *kvm)
+{
+ return (kvm_has_feat(kvm, FEAT_S1POE) &&
+ kvm_has_feat(kvm, FEAT_TCR2) &&
+ kvm_has_feat(kvm, FEAT_ATS1A) &&
+ kvm_has_feat(kvm, FEAT_HPDS));
+}
+
static bool feat_sme_smps(struct kvm *kvm)
{
/*
@@ -1299,6 +1335,14 @@ static const struct reg_bits_to_feat_map vtcr_el2_feat_map[] = {
static const DECLARE_FEAT_MAP(vtcr_el2_desc, VTCR_EL2,
vtcr_el2_feat_map, FEAT_AA64EL2);
+static const struct reg_bits_to_feat_map id_aa64mmfr3_el1_feat_map[] = {
+ NEEDS_FEAT(ID_AA64MMFR3_EL1_S1PIE, feat_s1pie),
+ NEEDS_FEAT(ID_AA64MMFR3_EL1_S1POE, feat_s1poe),
+};
+
+static const DECLARE_IDREG_FEAT_MAP(id_aa64mmfr3_el1_desc, ID_AA64MMFR3_EL1,
+ id_aa64mmfr3_el1_feat_map, FEAT_AA64EL1);
+
static const struct reg_bits_to_feat_map ich_hfgrtr_feat_map[] = {
NEEDS_FEAT(ICH_HFGRTR_EL2_ICC_APR_EL1 |
ICH_HFGRTR_EL2_ICC_IDRn_EL1 |
@@ -1408,6 +1452,40 @@ void __init check_feature_map(void)
check_reg_desc(&ich_hfgitr_desc);
}
+static const struct reg_feat_map_desc *idreg_feat_maps[] = {
+ &id_aa64mmfr3_el1_desc,
+};
+
+static bool idreg_apply_deps(struct kvm *kvm, const struct reg_feat_map_desc *r)
+{
+ struct resx resx = {};
+ u64 val;
+
+ for (int i = 0; i < r->bit_feat_map_sz; i++) {
+ if ((r->bit_feat_map[i].flags & CALL_FUNC) &&
+ !r->bit_feat_map[i].match(kvm))
+ resx.res0 |= reg_feat_map_bits(&r->bit_feat_map[i]);
+ }
+
+ val = kvm->arch.id_regs[r->idreg_idx];
+ kvm->arch.id_regs[r->idreg_idx] &= ~resx.res0;
+ kvm->arch.id_regs[r->idreg_idx] |= resx.res1;
+
+ return val != kvm->arch.id_regs[r->idreg_idx];
+}
+
+void kvm_idregs_enforce_dependencies(struct kvm *kvm)
+{
+ bool again;
+
+ do {
+ again = false;
+
+ for (int i = 0; i < ARRAY_SIZE(idreg_feat_maps); i++)
+ again |= idreg_apply_deps(kvm, idreg_feat_maps[i]);
+ } while (again);
+}
+
static bool idreg_feat_match(struct kvm *kvm, const struct reg_bits_to_feat_map *map)
{
u64 regval = kvm->arch.id_regs[map->regidx];
diff --git a/arch/arm64/kvm/hyp/exception.c b/arch/arm64/kvm/hyp/exception.c
index 754e2dc1df54..6e60d890afa4 100644
--- a/arch/arm64/kvm/hyp/exception.c
+++ b/arch/arm64/kvm/hyp/exception.c
@@ -70,6 +70,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
enum exception_type type)
{
unsigned long sctlr, vbar, old, new, mode;
+ struct kvm *kvm;
u64 exc_offset;
mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
@@ -109,8 +110,10 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,
new |= (old & PSR_C_BIT);
new |= (old & PSR_V_BIT);
- if (kvm_has_mte(kern_hyp_va(vcpu->kvm)))
+ kvm = vcpu_get_kvm(vcpu);
+ if (kvm && kvm_has_mte(kvm))
new |= PSR_TCO_BIT;
+ vcpu_put_kvm(vcpu, kvm);
new |= (old & PSR_DIT_BIT);
diff --git a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
index 4fdfeabefeb4..f55950ee2a7e 100644
--- a/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
+++ b/arch/arm64/kvm/hyp/include/hyp/adjust_pc.h
@@ -13,6 +13,34 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
+#ifdef __KVM_NVHE_HYPERVISOR__
+#include <nvhe/pkvm.h>
+
+/* Under pKVM a host vCPU's ->kvm is host-writable. */
+static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ if (is_protected_kvm_enabled())
+ return pkvm_vcpu_get_kvm(vcpu);
+
+ return kern_hyp_va(vcpu->kvm);
+}
+
+static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+ if (is_protected_kvm_enabled())
+ pkvm_vcpu_put_kvm(vcpu, kvm);
+}
+#else
+static inline struct kvm *vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ return vcpu->kvm;
+}
+
+static inline void vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+}
+#endif
+
static inline void kvm_skip_instr(struct kvm_vcpu *vcpu)
{
if (vcpu_mode_is_32bit(vcpu)) {
diff --git a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
index 5c050f21066a..5cd8b2769cba 100644
--- a/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
+++ b/arch/arm64/kvm/hyp/include/nvhe/pkvm.h
@@ -92,6 +92,8 @@ struct pkvm_hyp_vcpu *pkvm_load_hyp_vcpu(pkvm_handle_t handle,
unsigned int vcpu_idx);
void pkvm_put_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu);
struct pkvm_hyp_vcpu *pkvm_get_loaded_hyp_vcpu(void);
+struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu);
+void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm);
struct pkvm_hyp_vm *get_pkvm_hyp_vm(pkvm_handle_t handle);
struct pkvm_hyp_vm *get_np_pkvm_hyp_vm(pkvm_handle_t handle);
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
index ac64a036b0a9..c8e6e50d2f35 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
@@ -543,8 +543,27 @@ static void handle___pkvm_host_mkyoung_guest(struct kvm_cpu_context *host_ctxt)
static void handle___kvm_adjust_pc(struct kvm_cpu_context *host_ctxt)
{
DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1);
+ struct pkvm_hyp_vcpu *hyp_vcpu;
+ struct kvm_vcpu *host_vcpu;
+
+ host_vcpu = __get_host_hyp_vcpus(vcpu, &hyp_vcpu);
+ if (host_vcpu) {
+ __kvm_adjust_pc(host_vcpu);
+ return;
+ }
+
+ /*
+ * With no hyp vCPU loaded for it, the host vCPU may be unpinned,
+ * and so unmapped at EL2: its first run pins it. A pin fails only
+ * for memory the host isn't sharing, a bad pointer, so the request
+ * is dropped.
+ */
+ host_vcpu = kern_hyp_va(vcpu);
+ if (hyp_pin_shared_mem(host_vcpu, host_vcpu + 1))
+ return;
- __kvm_adjust_pc(kern_hyp_va(vcpu));
+ __kvm_adjust_pc(host_vcpu);
+ hyp_unpin_shared_mem(host_vcpu, host_vcpu + 1);
}
static void handle___kvm_flush_vm_context(struct kvm_cpu_context *host_ctxt)
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 6290c4b62659..1a17ab23ddfe 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -311,6 +311,43 @@ struct pkvm_hyp_vcpu *pkvm_get_loaded_hyp_vcpu(void)
}
+static struct pkvm_hyp_vm *pkvm_get_loaded_hyp_vm(struct kvm_vcpu *vcpu)
+{
+ struct pkvm_hyp_vcpu *hyp_vcpu = pkvm_get_loaded_hyp_vcpu();
+
+ if (hyp_vcpu &&
+ (vcpu == &hyp_vcpu->vcpu || vcpu == hyp_vcpu->host_vcpu))
+ return pkvm_hyp_vcpu_to_hyp_vm(hyp_vcpu);
+
+ return NULL;
+}
+
+/*
+ * A loaded vCPU's VM is the hyp VM. An unloaded host vCPU's is mapped at
+ * EL2 only while pinned, so it's read once and pinned; the pin fails for
+ * memory the host isn't sharing.
+ */
+struct kvm *pkvm_vcpu_get_kvm(struct kvm_vcpu *vcpu)
+{
+ struct pkvm_hyp_vm *hyp_vm = pkvm_get_loaded_hyp_vm(vcpu);
+ struct kvm *kvm;
+
+ if (hyp_vm)
+ return &hyp_vm->kvm;
+
+ kvm = kern_hyp_va(READ_ONCE(vcpu->kvm));
+ if (hyp_pin_shared_mem(kvm, kvm + 1))
+ return NULL;
+
+ return kvm;
+}
+
+void pkvm_vcpu_put_kvm(struct kvm_vcpu *vcpu, struct kvm *kvm)
+{
+ if (kvm && !pkvm_get_loaded_hyp_vm(vcpu))
+ hyp_unpin_shared_mem(kvm, kvm + 1);
+}
+
struct pkvm_hyp_vm *get_pkvm_hyp_vm(pkvm_handle_t handle)
{
struct pkvm_hyp_vm *hyp_vm;
diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
index 8758c6801776..ec469d96516a 100644
--- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c
+++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
@@ -121,6 +121,9 @@ static const struct pvm_ftr_bits pvmid_aa64mmfr0[] = {
MAX_FEAT(ID_AA64MMFR0_EL1, SNSMEM, IMP),
MAX_FEAT(ID_AA64MMFR0_EL1, BIGENDEL0, IMP),
MAX_FEAT(ID_AA64MMFR0_EL1, EXS, IMP),
+ MAX_FEAT(ID_AA64MMFR0_EL1, TGRAN4, IMP),
+ MAX_FEAT(ID_AA64MMFR0_EL1, TGRAN16, IMP),
+ MAX_FEAT(ID_AA64MMFR0_EL1, TGRAN64, IMP),
FEAT_END
};
diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
index 7875911c0506..14aada311bac 100644
--- a/arch/arm64/kvm/hyp/vhe/switch.c
+++ b/arch/arm64/kvm/hyp/vhe/switch.c
@@ -175,7 +175,8 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
offset = read_sysreg_s(SYS_CNTPOFF_EL2);
if (map.direct_ptimer && offset) {
- write_sysreg_el0(val + offset, SYS_CNTP_CVAL);
+ val = timer_apply_offset(val, offset, arch_timer_read_cntpct_el0());
+ write_sysreg_el0(val, SYS_CNTP_CVAL);
isb();
}
}
@@ -296,10 +297,10 @@ static bool kvm_hyp_handle_timer(struct kvm_vcpu *vcpu, u64 *exit_code)
break;
case SYS_CNTP_CVAL_EL0:
if (vcpu_el2_e2h_is_set(vcpu)) {
- val = read_sysreg_el0(SYS_CNTP_CVAL);
-
- if (!has_cntpoff())
- val -= timer_get_offset(vcpu_hptimer(vcpu));
+ if (!has_cntpoff() && timer_get_offset(vcpu_hptimer(vcpu)))
+ val = __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
+ else
+ val = read_sysreg_el0(SYS_CNTP_CVAL);
} else {
val = __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0);
}
diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c
index 9644c424819b..c60d7dfbf5e2 100644
--- a/arch/arm64/kvm/hyp_trace.c
+++ b/arch/arm64/kvm/hyp_trace.c
@@ -59,14 +59,8 @@ static void __hyp_clock_work(struct work_struct *work)
if (hyp_clock->mult) {
u64 err, cur = delta_cycles;
- if (WARN_ON_ONCE(cur >= hyp_clock->cyc_overflow64)) {
- __uint128_t tmp = (__uint128_t)cur * hyp_clock->mult;
-
- cur = tmp >> hyp_clock->shift;
- } else {
- cur *= hyp_clock->mult;
- cur >>= hyp_clock->shift;
- }
+ WARN_ON_ONCE(cur >= hyp_clock->cyc_overflow64);
+ cur = mul_u64_u32_shr(cur, hyp_clock->mult, hyp_clock->shift);
cur += hyp_clock->boot;
err = abs_diff(cur, boot);
@@ -84,7 +78,7 @@ static void __hyp_clock_work(struct work_struct *work)
err / NSEC_PER_USEC);
}
- rate = div64_u64(delta_cycles * NSEC_PER_SEC, delta_boot);
+ rate = mul_u64_u64_div_u64(delta_cycles, NSEC_PER_SEC, delta_boot);
clocks_calc_mult_shift(&hyp_clock->mult, &hyp_clock->shift,
rate, NSEC_PER_SEC, CLOCK_MAX_CONVERSION_S);
diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c
index b191365d97cc..bbd7042082b6 100644
--- a/arch/arm64/kvm/nested.c
+++ b/arch/arm64/kvm/nested.c
@@ -124,6 +124,9 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
return ret;
}
+ for (i = 0; i < S2_MMU_PER_VCPU; i++)
+ kvm_nested_s2_ptdump_create_debugfs(&tmp[i], i + kvm->arch.nested_mmus_size);
+
guard(write_lock)(&kvm->mmu_lock);
for (i = 0; i < S2_MMU_PER_VCPU; i++)
@@ -837,10 +840,8 @@ static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
kvm->arch.nested_mmus_next = (i + 1) % kvm->arch.nested_mmus_size;
/* Make sure we don't forget to do the laundry */
- if (kvm_s2_mmu_valid(s2_mmu)) {
- kvm_nested_s2_ptdump_remove_debugfs(s2_mmu);
+ if (kvm_s2_mmu_valid(s2_mmu))
s2_mmu->pending_unmap = true;
- }
/*
* The virtual VMID (modulo CnP) will be used as a key when matching
@@ -854,8 +855,6 @@ static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
s2_mmu->tlb_vtcr = vcpu_read_sys_reg(vcpu, VTCR_EL2);
s2_mmu->nested_stage2_enabled = vcpu_read_sys_reg(vcpu, HCR_EL2) & HCR_VM;
- kvm_nested_s2_ptdump_create_debugfs(s2_mmu);
-
out:
atomic_inc(&s2_mmu->refcnt);
@@ -2046,6 +2045,9 @@ void check_nested_vcpu_requests(struct kvm_vcpu *vcpu)
*
* What follows is some slop to (1) react to exception routing / masking and (2)
* preserve the pending SError state across translation regimes.
+ *
+ * Also generate the maintenance interrupt state, and finally check that if
+ * we need to take an interrupt targetting the guest hypervisor.
*/
void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
{
@@ -2054,6 +2056,11 @@ void kvm_nested_flush_hwstate(struct kvm_vcpu *vcpu)
if (unlikely(vcpu_test_and_clear_flag(vcpu, NESTED_SERROR_PENDING)))
kvm_inject_serror_esr(vcpu, vcpu_get_vsesr(vcpu));
+
+ vgic_v3_nested_update_mi(vcpu);
+
+ if (vgic_state_is_nested(vcpu) && kvm_vgic_vcpu_pending_irq(vcpu))
+ kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
}
void kvm_nested_sync_hwstate(struct kvm_vcpu *vcpu)
diff --git a/arch/arm64/kvm/ptdump.c b/arch/arm64/kvm/ptdump.c
index 69899797dbad..7829fe376385 100644
--- a/arch/arm64/kvm/ptdump.c
+++ b/arch/arm64/kvm/ptdump.c
@@ -17,7 +17,7 @@
#define MARKERS_LEN 2
#define KVM_PGTABLE_MAX_LEVELS (KVM_PGTABLE_LAST_LEVEL + 1)
-#define S2FNAMESZ sizeof("0x0123456789abcdef-0x0123456789abcdef-s2-disabled")
+#define S2FNAMESZ sizeof("nested_mmu0000")
struct kvm_ptdump_guest_state {
struct kvm_s2_mmu *mmu;
@@ -115,15 +115,28 @@ static int kvm_ptdump_build_levels(struct ptdump_pg_level *level, u32 start_lvl)
static struct kvm_ptdump_guest_state *kvm_ptdump_parser_create(struct kvm_s2_mmu *mmu)
{
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
struct kvm_ptdump_guest_state *st;
- struct kvm_pgtable *pgtable = mmu->pgt;
+ s8 start_level;
int ret;
+ /*
+ * We only need the pgt start level to initialize the ptdump, get it
+ * while holding the mmu_lock. It's fine if the pgt gets freed
+ * afterwards, we'll check again when doing the actual dump.
+ */
+ scoped_guard(read_lock, &kvm->mmu_lock) {
+ if (mmu->pgt)
+ start_level = mmu->pgt->start_level;
+ else
+ return ERR_PTR(-ENOENT);
+ }
+
st = kzalloc_obj(struct kvm_ptdump_guest_state, GFP_KERNEL_ACCOUNT);
if (!st)
return ERR_PTR(-ENOMEM);
- ret = kvm_ptdump_build_levels(&st->level[0], pgtable->start_level);
+ ret = kvm_ptdump_build_levels(&st->level[0], start_level);
if (ret) {
kfree(st);
return ERR_PTR(ret);
@@ -149,6 +162,9 @@ static int kvm_ptdump_guest_show(struct seq_file *m, void *unused)
};
guard(write_lock)(&kvm->mmu_lock);
+ if (!mmu->pgt)
+ return 0;
+
st->parser_state = (struct ptdump_pg_state) {
.marker = &st->ipa_marker[0],
.end_address = BIT(mmu->pgt->ia_bits),
@@ -157,6 +173,15 @@ static int kvm_ptdump_guest_show(struct seq_file *m, void *unused)
.seq = m,
};
+ if (kvm_is_nested_s2_mmu(kvm, mmu)) {
+ if (kvm_s2_mmu_valid(mmu))
+ seq_printf(m, "VTCR: 0x%016llx VTTBR: 0x%016llx s2: %s\n",
+ mmu->tlb_vtcr, mmu->tlb_vttbr,
+ mmu->nested_stage2_enabled ? "enabled" : "disabled");
+ else
+ return 0;
+ }
+
ret = kvm_pgtable_walk(mmu->pgt, 0, BIT(mmu->pgt->ia_bits), &walker);
if (ret)
return ret;
@@ -211,17 +236,27 @@ static const struct file_operations kvm_ptdump_guest_fops = {
static int kvm_pgtable_range_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%2u\n", mmu->pgt->ia_bits);
- seq_printf(m, "%2u\n", pgtable->ia_bits);
return 0;
}
static int kvm_pgtable_levels_show(struct seq_file *m, void *unused)
{
- struct kvm_pgtable *pgtable = m->private;
+ struct kvm_s2_mmu *mmu = m->private;
+ struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
+
+ guard(read_lock)(&kvm->mmu_lock);
+
+ if (mmu->pgt)
+ seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - mmu->pgt->start_level);
- seq_printf(m, "%1d\n", KVM_PGTABLE_MAX_LEVELS - pgtable->start_level);
return 0;
}
@@ -230,15 +265,12 @@ static int kvm_pgtable_debugfs_open(struct inode *m, struct file *file,
{
struct kvm_s2_mmu *mmu = m->i_private;
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
- struct kvm_pgtable *pgtable;
int ret;
if (!kvm_get_kvm_safe(kvm))
return -ENOENT;
- pgtable = mmu->pgt;
-
- ret = single_open(file, show, pgtable);
+ ret = single_open(file, show, mmu);
if (ret < 0)
kvm_put_kvm(kvm);
return ret;
@@ -276,26 +308,15 @@ static const struct file_operations kvm_pgtable_levels_fops = {
.release = kvm_pgtable_debugfs_close,
};
-void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu)
+void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu, int idx)
{
- struct dentry *dent;
char file_name[S2FNAMESZ];
- snprintf(file_name, sizeof(file_name), "0x%016llx-0x%016llx-s2-%sabled",
- mmu->tlb_vttbr,
- mmu->tlb_vtcr,
- mmu->nested_stage2_enabled ? "en" : "dis");
+ snprintf(file_name, sizeof(file_name), "nested_mmu%d", idx);
- dent = debugfs_create_file(file_name, 0400,
- mmu->arch->debugfs_nv_dentry, mmu,
- &kvm_ptdump_guest_fops);
-
- mmu->shadow_pt_debugfs_dentry = dent;
-}
-
-void kvm_nested_s2_ptdump_remove_debugfs(struct kvm_s2_mmu *mmu)
-{
- debugfs_remove(mmu->shadow_pt_debugfs_dentry);
+ debugfs_create_file(file_name, 0400,
+ mmu->arch->debugfs_nv_dentry, mmu,
+ &kvm_ptdump_guest_fops);
}
void kvm_s2_ptdump_create_debugfs(struct kvm *kvm)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index a2f4e769a428..cac62ed43767 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -5884,6 +5884,8 @@ int kvm_finalize_sys_regs(struct kvm_vcpu *vcpu)
if (kvm_vm_has_ran_once(kvm))
return 0;
+ kvm_idregs_enforce_dependencies(kvm);
+
/*
* This hacks into the ID registers, so only perform it when the
* first vcpu runs, or the kvm_set_vm_id_reg() helper will scream.
diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c
index c10a11373e10..642dd6d0eb33 100644
--- a/arch/arm64/kvm/vgic/vgic-its.c
+++ b/arch/arm64/kvm/vgic/vgic-its.c
@@ -776,6 +776,7 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
its_unmap_vlpi(ite->irq->host_irq);
irq->hw = false;
+ irq->mapped = false;
}
vgic_put_irq(kvm, ite->irq);
@@ -784,6 +785,23 @@ static void its_free_ite(struct kvm *kvm, struct its_ite *ite)
kfree(ite);
}
+/* Mapping one LPI from two events is UNPREDICTABLE: the first ITE keeps it. */
+static bool vgic_its_map_lpi(struct kvm *kvm, struct its_ite *ite,
+ struct vgic_irq *irq, struct kvm_vcpu *vcpu)
+{
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+ if (!irq->mapped) {
+ irq->mapped = true;
+ irq->target_vcpu = vcpu;
+ ite->irq = irq;
+ return true;
+ }
+ }
+
+ vgic_put_irq(kvm, irq);
+ return false;
+}
+
static u64 its_cmd_mask_field(u64 *its_cmd, int word, int shift, int size)
{
return (le64_to_cpu(its_cmd[word]) >> shift) & (BIT_ULL(size) - 1);
@@ -1093,13 +1111,14 @@ static int vgic_its_cmd_handle_mapi(struct kvm *kvm, struct vgic_its *its,
vcpu = collection_to_vcpu(kvm, collection);
irq = vgic_add_lpi(kvm, lpi_nr, vcpu);
+ if (!IS_ERR(irq) && !vgic_its_map_lpi(kvm, ite, irq, vcpu))
+ irq = ERR_PTR(-EEXIST);
if (IS_ERR(irq)) {
if (new_coll)
vgic_its_free_collection(its, coll_id);
its_free_ite(kvm, ite);
return PTR_ERR(irq);
}
- ite->irq = irq;
return 0;
}
@@ -2208,7 +2227,10 @@ static int vgic_its_restore_ite(struct vgic_its *its, u32 event_id,
its_free_ite(kvm, ite);
return PTR_ERR(irq);
}
- ite->irq = irq;
+
+ /* Drop an ITE whose LPI another ITE maps, as MAPTI does. */
+ if (!vgic_its_map_lpi(kvm, ite, irq, vcpu))
+ its_free_ite(kvm, ite);
return offset;
}
diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
index 5913a20d8301..5ab789accc9d 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c
@@ -818,16 +818,21 @@ int vgic_register_redist_iodev(struct kvm_vcpu *vcpu)
rd_dev->nr_regions = ARRAY_SIZE(vgic_v3_rd_registers);
rd_dev->redist_vcpu = vcpu;
+ /* Protected by slots_lock */
+ rdreg->free_index++;
+
mutex_unlock(&kvm->arch.config_lock);
ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, rd_base,
2 * SZ_64K, &rd_dev->dev);
- if (ret)
- return ret;
+ if (ret) {
+ guard(mutex)(&vcpu->kvm->arch.config_lock);
+ vgic_cpu->rdreg->free_index--;
+ vgic_cpu->rdreg = NULL;
+ vgic_cpu->rd_iodev.base_addr = VGIC_ADDR_UNDEF;
+ }
- /* Protected by slots_lock */
- rdreg->free_index++;
- return 0;
+ return ret;
out_unlock:
mutex_unlock(&kvm->arch.config_lock);
@@ -841,7 +846,8 @@ void vgic_unregister_redist_iodev(struct kvm_vcpu *vcpu)
kvm_io_bus_unregister_dev(vcpu->kvm, KVM_MMIO_BUS, &rd_dev->dev);
}
-static int vgic_register_all_redist_iodevs(struct kvm *kvm)
+static int vgic_register_all_redist_iodevs(struct kvm *kvm,
+ struct vgic_redist_region *rdreg)
{
struct kvm_vcpu *vcpu;
unsigned long c;
@@ -861,7 +867,14 @@ static int vgic_register_all_redist_iodevs(struct kvm *kvm)
for (i = 0; i < c; i++) {
vcpu = kvm_get_vcpu(kvm, i);
+ if (vcpu->arch.vgic_cpu.rdreg != rdreg)
+ continue;
+
vgic_unregister_redist_iodev(vcpu);
+
+ guard(mutex)(&vcpu->kvm->arch.config_lock);
+ vcpu->arch.vgic_cpu.rdreg = NULL;
+ vcpu->arch.vgic_cpu.rd_iodev.base_addr = VGIC_ADDR_UNDEF;
}
}
@@ -970,10 +983,13 @@ void vgic_v3_free_redist_region(struct kvm *kvm, struct vgic_redist_region *rdre
int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)
{
+ struct vgic_redist_region *rdreg;
int ret;
mutex_lock(&kvm->arch.config_lock);
ret = vgic_v3_alloc_redist_region(kvm, index, addr, count);
+ if (!ret)
+ rdreg = vgic_v3_rdist_region_from_index(kvm, index);
mutex_unlock(&kvm->arch.config_lock);
if (ret)
return ret;
@@ -982,12 +998,9 @@ int vgic_v3_set_redist_base(struct kvm *kvm, u32 index, u64 addr, u32 count)
* Register iodevs for each existing VCPU. Adding more VCPUs
* afterwards will register the iodevs when needed.
*/
- ret = vgic_register_all_redist_iodevs(kvm);
+ ret = vgic_register_all_redist_iodevs(kvm, rdreg);
if (ret) {
- struct vgic_redist_region *rdreg;
-
mutex_lock(&kvm->arch.config_lock);
- rdreg = vgic_v3_rdist_region_from_index(kvm, index);
vgic_v3_free_redist_region(kvm, rdreg);
mutex_unlock(&kvm->arch.config_lock);
return ret;
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index 5cf5a1ef86cd..7061b4ab206c 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -1152,7 +1152,7 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
*
* We do that by placing a request to ourselves which will
* abort the entry procedure and inject the exception at the
- * beginning of the run loop.
+ * beginning of the run loop (see kvm_nested_flush_hwstate()).
*
* - Otherwise, do exactly *NOTHING* apart from enabling the virtual
* CPU interface. The guest state is already loaded, and we can
@@ -1162,16 +1162,10 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
* maintenance interrupt state, as it may fire.
*/
if (vgic_state_is_nested(vcpu)) {
- if (kvm_vgic_vcpu_pending_irq(vcpu))
- kvm_make_request(KVM_REQ_GUEST_HYP_IRQ_PENDING, vcpu);
-
vgic_v3_flush_nested(vcpu);
return;
}
- if (vcpu_has_nv(vcpu))
- vgic_v3_nested_update_mi(vcpu);
-
DEBUG_SPINLOCK_BUG_ON(!irqs_disabled());
vgic_flush_state(vcpu);
diff --git a/arch/arm64/mm/contpte.c b/arch/arm64/mm/contpte.c
index 0acab179fc1a..be78d8602e51 100644
--- a/arch/arm64/mm/contpte.c
+++ b/arch/arm64/mm/contpte.c
@@ -460,8 +460,10 @@ void contpte_set_ptes(struct mm_struct *mm, unsigned long addr,
*/
VM_WARN_ON(nr == 1);
- if (!mm_is_user(mm))
+ if (!mm_is_user(mm) || !pte_valid(pte)) {
+ VM_WARN_ON_ONCE(!pte_valid(pte) && pte_cont(pte));
return __set_ptes(mm, addr, ptep, pte, nr);
+ }
end = addr + (nr << PAGE_SHIFT);
pfn = pte_pfn(pte);
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 75c3e463df2e..357be4c9106d 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -620,9 +620,10 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
/*
* If we're in an interrupt or have no user context, we must not take
- * the fault.
+ * the fault. EFI runtime services may execute with preemption enabled
+ * and mm == &efi_mm, but faults need to be treated as kernel faults.
*/
- if (faulthandler_disabled() || !mm)
+ if (faulthandler_disabled() || !mm || mm_is_efi(mm))
goto no_context;
if (user_mode(regs))
diff --git a/arch/loongarch/include/asm/barrier.h b/arch/loongarch/include/asm/barrier.h
index 4b663f197706..84c6d739119d 100644
--- a/arch/loongarch/include/asm/barrier.h
+++ b/arch/loongarch/include/asm/barrier.h
@@ -57,9 +57,6 @@
#define __WEAK_LLSC_MB " \n"
#endif
-#define __smp_mb__before_atomic() barrier()
-#define __smp_mb__after_atomic() barrier()
-
/**
* array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise
* @index: array element index
diff --git a/arch/loongarch/include/asm/hw_breakpoint.h b/arch/loongarch/include/asm/hw_breakpoint.h
index 5faa97a87a9e..f4478936787d 100644
--- a/arch/loongarch/include/asm/hw_breakpoint.h
+++ b/arch/loongarch/include/asm/hw_breakpoint.h
@@ -116,17 +116,17 @@ extern void arch_uninstall_hw_breakpoint(struct perf_event *bp);
extern int hw_breakpoint_slots(int type);
extern void hw_breakpoint_pmu_read(struct perf_event *bp);
-void breakpoint_handler(struct pt_regs *regs);
-void watchpoint_handler(struct pt_regs *regs);
+bool breakpoint_handler(struct pt_regs *regs);
+bool watchpoint_handler(struct pt_regs *regs);
#ifdef CONFIG_HAVE_HW_BREAKPOINT
extern void ptrace_hw_copy_thread(struct task_struct *task);
-extern void hw_breakpoint_thread_switch(struct task_struct *next);
+extern void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next);
#else
static inline void ptrace_hw_copy_thread(struct task_struct *task)
{
}
-static inline void hw_breakpoint_thread_switch(struct task_struct *next)
+static inline void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
}
#endif
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 2a6bc99177d8..28eff8b49d1b 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -1130,8 +1130,11 @@
#define LOONGARCH_CSR_DERA 0x501 /* debug era */
#define LOONGARCH_CSR_DESAVE 0x502 /* debug save */
-#define CSR_FWPC_SKIP_SHIFT 16
-#define CSR_FWPC_SKIP (_ULCAST_(1) << CSR_FWPC_SKIP_SHIFT)
+#define CSR_FWPS_SKIP_SHIFT 16
+#define CSR_FWPS_SKIP (_ULCAST_(1) << CSR_FWPS_SKIP_SHIFT)
+
+#define CSR_MWPS_SKIP_SHIFT 16
+#define CSR_MWPS_SKIP (_ULCAST_(1) << CSR_MWPS_SKIP_SHIFT)
/*
* CSR_ECFG IM
diff --git a/arch/loongarch/include/asm/numa.h b/arch/loongarch/include/asm/numa.h
index bbf9f70bd25f..c9654ff900a3 100644
--- a/arch/loongarch/include/asm/numa.h
+++ b/arch/loongarch/include/asm/numa.h
@@ -14,7 +14,7 @@
#define NODE_ADDRSPACE_SHIFT 44
#define pa_to_nid(addr) (((addr) & 0xf00000000000) >> NODE_ADDRSPACE_SHIFT)
-#define nid_to_addrbase(nid) (_ULCAST_(nid) << NODE_ADDRSPACE_SHIFT)
+#define nid_to_addrbase(nid) (_U64CAST_(nid) << NODE_ADDRSPACE_SHIFT)
#ifdef CONFIG_NUMA
diff --git a/arch/loongarch/include/asm/processor.h b/arch/loongarch/include/asm/processor.h
index ce8b953f8c79..a884f964ec98 100644
--- a/arch/loongarch/include/asm/processor.h
+++ b/arch/loongarch/include/asm/processor.h
@@ -138,6 +138,7 @@ struct thread_struct {
struct loongarch_lbt lbt; /* Also conditionally copied */
/* Hardware breakpoints pinned to this task. */
+ bool hbp_break_skip, hbp_watch_skip;
struct perf_event *hbp_break[LOONGARCH_MAX_BRP];
struct perf_event *hbp_watch[LOONGARCH_MAX_WRP];
};
diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h
index 27acbf913774..ab9e0292c7f3 100644
--- a/arch/loongarch/include/asm/switch_to.h
+++ b/arch/loongarch/include/asm/switch_to.h
@@ -34,7 +34,7 @@ extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev,
do { \
lose_fpu_inatomic(1, prev); \
lose_lbt_inatomic(1, prev); \
- hw_breakpoint_thread_switch(next); \
+ hw_breakpoint_thread_switch(prev, next); \
set_current(next); \
(last) = __switch_to(prev, next, \
__builtin_return_address(0), __builtin_frame_address(0)); \
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
index 74d31f260dfd..629c976c33c3 100644
--- a/arch/loongarch/kernel/cpu-probe.c
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -93,7 +93,7 @@ static int __init cpu_final_simd(void)
return 0;
}
-arch_initcall(cpu_final_simd);
+pure_initcall(cpu_final_simd);
static inline void set_elf_platform(int cpu, const char *plat)
{
diff --git a/arch/loongarch/kernel/hw_breakpoint.c b/arch/loongarch/kernel/hw_breakpoint.c
index c35f9bf38033..df149ff06dd5 100644
--- a/arch/loongarch/kernel/hw_breakpoint.c
+++ b/arch/loongarch/kernel/hw_breakpoint.c
@@ -156,6 +156,9 @@ static int hw_breakpoint_slot_setup(struct perf_event **slots, int max_slots,
void ptrace_hw_copy_thread(struct task_struct *tsk)
{
+ tsk->thread.hbp_break_skip = 0;
+ tsk->thread.hbp_watch_skip = 0;
+
memset(tsk->thread.hbp_break, 0, sizeof(tsk->thread.hbp_break));
memset(tsk->thread.hbp_watch, 0, sizeof(tsk->thread.hbp_watch));
}
@@ -233,7 +236,7 @@ static int hw_breakpoint_control(struct perf_event *bp,
}
enable = csr_read64(LOONGARCH_CSR_CRMD);
csr_write64(CSR_CRMD_WE | enable, LOONGARCH_CSR_CRMD);
- if (bp->hw.target && test_tsk_thread_flag(bp->hw.target, TIF_LOAD_WATCH))
+ if (bp->hw.target)
regs->csr_prmd |= CSR_PRMD_PWE;
break;
case HW_BREAKPOINT_UNINSTALL:
@@ -453,7 +456,6 @@ static void update_bp_registers(struct pt_regs *regs, int enable, int type)
if (enable) {
if ((info->ctrl.type == LOONGARCH_BREAKPOINT_EXECUTE) && (type == 0)) {
write_wb_reg(CSR_CFG_CTRL, i, 0, CTRL_PLV_ENABLE);
- write_wb_reg(CSR_CFG_CTRL, i, 0, CTRL_PLV_ENABLE);
} else {
ctrl = read_wb_reg(CSR_CFG_CTRL, i, 1);
if (info->ctrl.type == LOONGARCH_BREAKPOINT_LOAD)
@@ -469,9 +471,9 @@ static void update_bp_registers(struct pt_regs *regs, int enable, int type)
} else {
ctrl = read_wb_reg(CSR_CFG_CTRL, i, 1);
if (info->ctrl.type == LOONGARCH_BREAKPOINT_LOAD)
- ctrl &= ~0x1 << MWPnCFG3_LoadEn;
+ ctrl &= ~(0x1 << MWPnCFG3_LoadEn);
if (info->ctrl.type == LOONGARCH_BREAKPOINT_STORE)
- ctrl &= ~0x1 << MWPnCFG3_StoreEn;
+ ctrl &= ~(0x1 << MWPnCFG3_StoreEn);
write_wb_reg(CSR_CFG_CTRL, i, 1, ctrl);
}
regs->csr_prmd &= ~CSR_PRMD_PWE;
@@ -483,9 +485,11 @@ NOKPROBE_SYMBOL(update_bp_registers);
/*
* Debug exception handlers.
*/
-void breakpoint_handler(struct pt_regs *regs)
+bool breakpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
+ bool need_sigtrap = false;
struct perf_event *bp, **slots;
slots = this_cpu_ptr(bp_on_reg);
@@ -495,17 +499,27 @@ void breakpoint_handler(struct pt_regs *regs)
bp = slots[i];
if (bp == NULL)
continue;
+
perf_bp_event(bp, regs);
- csr_write32(0x1 << i, LOONGARCH_CSR_FWPS);
- update_bp_registers(regs, 0, 0);
+ if (bp->attr.sigtrap)
+ need_sigtrap = true;
+
+ clear_mask |= (0x1 << i);
}
}
+
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+
+ return need_sigtrap;
}
NOKPROBE_SYMBOL(breakpoint_handler);
-void watchpoint_handler(struct pt_regs *regs)
+bool watchpoint_handler(struct pt_regs *regs)
{
int i;
+ unsigned int clear_mask = 0;
+ bool need_sigtrap = false;
struct perf_event *wp, **slots;
slots = this_cpu_ptr(wp_on_reg);
@@ -515,11 +529,19 @@ void watchpoint_handler(struct pt_regs *regs)
wp = slots[i];
if (wp == NULL)
continue;
+
perf_bp_event(wp, regs);
- csr_write32(0x1 << i, LOONGARCH_CSR_MWPS);
- update_bp_registers(regs, 0, 1);
+ if (wp->attr.sigtrap)
+ need_sigtrap = true;
+
+ clear_mask |= (0x1 << i);
}
}
+
+ if (clear_mask)
+ csr_write32(clear_mask | CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+
+ return need_sigtrap;
}
NOKPROBE_SYMBOL(watchpoint_handler);
@@ -542,7 +564,7 @@ static int __init arch_hw_breakpoint_init(void)
}
arch_initcall(arch_hw_breakpoint_init);
-void hw_breakpoint_thread_switch(struct task_struct *next)
+void hw_breakpoint_thread_switch(struct task_struct *prev, struct task_struct *next)
{
u64 addr, mask;
struct pt_regs *regs = task_pt_regs(next);
@@ -551,9 +573,29 @@ void hw_breakpoint_thread_switch(struct task_struct *next)
addr = read_wb_reg(CSR_CFG_ADDR, 0, 0);
mask = read_wb_reg(CSR_CFG_MASK, 0, 0);
if (!((regs->csr_era ^ addr) & ~mask))
- csr_write32(CSR_FWPC_SKIP, LOONGARCH_CSR_FWPS);
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
regs->csr_prmd |= CSR_PRMD_PWE;
} else {
+ unsigned int fwps = csr_read32(LOONGARCH_CSR_FWPS);
+ unsigned int mwps = csr_read32(LOONGARCH_CSR_MWPS);
+
+ prev->thread.hbp_break_skip = !!(fwps & CSR_FWPS_SKIP);
+ prev->thread.hbp_watch_skip = !!(mwps & CSR_MWPS_SKIP);
+
+ if (!next->thread.hbp_break_skip)
+ csr_write32(0, LOONGARCH_CSR_FWPS);
+ else {
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
+ next->thread.hbp_break_skip = 0;
+ }
+
+ if (!next->thread.hbp_watch_skip)
+ csr_write32(0, LOONGARCH_CSR_MWPS);
+ else {
+ csr_write32(CSR_MWPS_SKIP, LOONGARCH_CSR_MWPS);
+ next->thread.hbp_watch_skip = 0;
+ }
+
/* Update breakpoints */
update_bp_registers(regs, 1, 0);
/* Update watchpoints */
diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c
index 7bf68a7a5f4b..29e0f2baaa8c 100644
--- a/arch/loongarch/kernel/irq.c
+++ b/arch/loongarch/kernel/irq.c
@@ -20,6 +20,7 @@
#include <asm/irq.h>
#include <asm/loongson.h>
+#include <asm/numa.h>
#include <asm/setup.h>
DEFINE_PER_CPU(unsigned long, irq_stack);
@@ -69,7 +70,7 @@ static int __init early_pci_mcfg_parse(struct acpi_table_header *header)
for (i = 0; i < n; i++, mptr++) {
msi_group[i].pci_segment = mptr->pci_segment;
- pch_group[i].node = msi_group[i].node = (mptr->address >> 44) & 0xf;
+ pch_group[i].node = msi_group[i].node = pa_to_nid(mptr->address);
}
return 0;
diff --git a/arch/loongarch/kernel/numa.c b/arch/loongarch/kernel/numa.c
index c96c53623715..7dedc134ddfe 100644
--- a/arch/loongarch/kernel/numa.c
+++ b/arch/loongarch/kernel/numa.c
@@ -216,6 +216,8 @@ static int __init fake_numa_init(void)
phys_addr_t start = memblock_start_of_DRAM();
phys_addr_t end = memblock_end_of_DRAM() - 1;
+ set_cpuid_to_node(0, 0);
+ early_numa_add_cpu(0, 0);
pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end);
return numa_add_memblk(0, start, end + 1);
@@ -244,8 +246,10 @@ int __init init_numa_memory(void)
return -EINVAL;
for_each_node_mask(node, node_possible_map) {
- node_mem_init(node);
- node_set_online(node);
+ if (!cpumask_empty(&cpus_on_node[node])) {
+ node_mem_init(node);
+ node_set_online(node);
+ }
}
max_pfn = PFN_DOWN(memblock_end_of_DRAM());
max_low_pfn = min(PFN_DOWN(HIGHMEM_START), max_pfn);
diff --git a/arch/loongarch/kernel/traps.c b/arch/loongarch/kernel/traps.c
index 776523747ea3..20ce2c8d77a1 100644
--- a/arch/loongarch/kernel/traps.c
+++ b/arch/loongarch/kernel/traps.c
@@ -815,6 +815,8 @@ asmlinkage void noinstr do_watch(struct pt_regs *regs)
#ifndef CONFIG_HAVE_HW_BREAKPOINT
pr_warn("Hardware watch point handler not implemented!\n");
#else
+ bool need_sigtrap = !!current->ptrace;
+
if (kgdb_breakpoint_handler(regs))
goto out;
@@ -829,7 +831,7 @@ asmlinkage void noinstr do_watch(struct pt_regs *regs)
* instruction. So don't clear llbit and reset CSR.FWPS.Skip until
* the llsc execution is completed.
*/
- csr_write32(CSR_FWPC_SKIP, LOONGARCH_CSR_FWPS);
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
csr_write32(CSR_LLBCTL_KLO, LOONGARCH_CSR_LLBCTL);
goto out;
}
@@ -846,16 +848,17 @@ asmlinkage void noinstr do_watch(struct pt_regs *regs)
* current pc, If yes, then we should not set the CSR.FWPS.SKIP
* bit to break the original instruction stream.
*/
- csr_write32(CSR_FWPC_SKIP, LOONGARCH_CSR_FWPS);
+ csr_write32(CSR_FWPS_SKIP, LOONGARCH_CSR_FWPS);
goto out;
}
}
} else {
- breakpoint_handler(regs);
- watchpoint_handler(regs);
+ need_sigtrap |= breakpoint_handler(regs);
+ need_sigtrap |= watchpoint_handler(regs);
}
- force_sig(SIGTRAP);
+ if (need_sigtrap)
+ force_sig(SIGTRAP);
out:
#endif
irqentry_exit(regs, state);
diff --git a/arch/loongarch/pci/pci.c b/arch/loongarch/pci/pci.c
index f33c7ea1443d..0b99b32110a5 100644
--- a/arch/loongarch/pci/pci.c
+++ b/arch/loongarch/pci/pci.c
@@ -12,6 +12,7 @@
#include <linux/io-64-nonatomic-lo-hi.h>
#include <asm/cacheflush.h>
#include <asm/loongson.h>
+#include <asm/numa.h>
#define PCI_DEVICE_ID_LOONGSON_HOST 0x7a00
#define PCI_DEVICE_ID_LOONGSON_DC1 0x7a06
@@ -43,7 +44,7 @@ int raw_pci_write(unsigned int domain, unsigned int bus, unsigned int devfn,
phys_addr_t mcfg_addr_init(int node)
{
- return (((u64)node << 44) | MCFG_EXT_PCICFG_BASE);
+ return (nid_to_addrbase(node) | MCFG_EXT_PCICFG_BASE);
}
static int __init pcibios_init(void)
diff --git a/arch/m68k/68000/entry.S b/arch/m68k/68000/entry.S
index c257cc415c47..0d3cbededc43 100644
--- a/arch/m68k/68000/entry.S
+++ b/arch/m68k/68000/entry.S
@@ -79,7 +79,8 @@ ENTRY(system_call)
/* Doing a trace ? */
getthreadinfo
- btst #(TIF_SYSCALL_TRACE%8),%a2@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ moveb %a2@(TINFO_FLAGS+2),%d1
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d1
jne do_trace
cmpl #NR_syscalls,%d0
jcc badsys
diff --git a/arch/m68k/coldfire/entry.S b/arch/m68k/coldfire/entry.S
index 4ea08336e2fb..d658abdc5207 100644
--- a/arch/m68k/coldfire/entry.S
+++ b/arch/m68k/coldfire/entry.S
@@ -72,7 +72,8 @@ ENTRY(system_call)
movel %d2,%a0
movel %a0@,%a1 /* save top of frame */
movel %sp,%a1@(TASK_THREAD+THREAD_ESP0)
- btst #(TIF_SYSCALL_TRACE%8),%a0@(TINFO_FLAGS+(31-TIF_SYSCALL_TRACE)/8)
+ moveb %a0@(TINFO_FLAGS+2),%d2
+ andl #((_TIF_SYSCALL_TRACE + _TIF_SECCOMP) >> 8),%d2
bnes 1f
movel %d3,%a0
diff --git a/arch/microblaze/include/asm/entry.h b/arch/microblaze/include/asm/entry.h
index 9efadf12397c..05acf4d7bc4b 100644
--- a/arch/microblaze/include/asm/entry.h
+++ b/arch/microblaze/include/asm/entry.h
@@ -21,6 +21,19 @@
#define PER_CPU(var) var
+/*
+ * The MicroBlaze ABI has the caller reserve an argument home area:
+ * REG_PARM_STACK_SPACE is 24 and OUTGOING_REG_PARM_STACK_SPACE is 1, so a
+ * callee may write to [caller_sp + 4, caller_sp + 28). The kernel calls C
+ * with r1 at the frame base, so reserve that area below pt_regs and reach
+ * the saved registers through PTO. This restores what 6e83557c38b4
+ * ("microblaze: Remove r0_ram pointer and PTO alignment") removed; the old
+ * value of 24 was one word short and left the last argument slot
+ * overlapping pt_regs' r0.
+ */
+#define PTO 28
+#define STATE_SAVE_SIZE (PT_SIZE + PTO)
+
# ifndef __ASSEMBLER__
DECLARE_PER_CPU(unsigned int, KSP); /* Saved kernel stack pointer */
DECLARE_PER_CPU(unsigned int, KM); /* Kernel/user mode */
diff --git a/arch/microblaze/include/asm/processor.h b/arch/microblaze/include/asm/processor.h
index d59bdfffca7c..595c24db2728 100644
--- a/arch/microblaze/include/asm/processor.h
+++ b/arch/microblaze/include/asm/processor.h
@@ -73,7 +73,7 @@ unsigned long __get_wchan(struct task_struct *p);
# define task_regs(task) ((struct pt_regs *)task_tos(task) - 1)
# define task_pt_regs_plus_args(tsk) \
- ((void *)task_pt_regs(tsk))
+ (((void *)task_pt_regs(tsk)) - PTO)
# define task_sp(task) (task_regs(task)->r1)
# define task_pc(task) (task_regs(task)->pc)
diff --git a/arch/microblaze/kernel/entry.S b/arch/microblaze/kernel/entry.S
index 582d7256d815..ebc8bcb98c52 100644
--- a/arch/microblaze/kernel/entry.S
+++ b/arch/microblaze/kernel/entry.S
@@ -177,78 +177,78 @@ syscall_debug_table:
1:
#define SAVE_REGS \
- swi r2, r1, PT_R2; /* Save SDA */ \
- swi r3, r1, PT_R3; \
- swi r4, r1, PT_R4; \
- swi r5, r1, PT_R5; \
- swi r6, r1, PT_R6; \
- swi r7, r1, PT_R7; \
- swi r8, r1, PT_R8; \
- swi r9, r1, PT_R9; \
- swi r10, r1, PT_R10; \
- swi r11, r1, PT_R11; /* save clobbered regs after rval */\
- swi r12, r1, PT_R12; \
- swi r13, r1, PT_R13; /* Save SDA2 */ \
- swi r14, r1, PT_PC; /* PC, before IRQ/trap */ \
- swi r15, r1, PT_R15; /* Save LP */ \
- swi r16, r1, PT_R16; \
- swi r17, r1, PT_R17; \
- swi r18, r1, PT_R18; /* Save asm scratch reg */ \
- swi r19, r1, PT_R19; \
- swi r20, r1, PT_R20; \
- swi r21, r1, PT_R21; \
- swi r22, r1, PT_R22; \
- swi r23, r1, PT_R23; \
- swi r24, r1, PT_R24; \
- swi r25, r1, PT_R25; \
- swi r26, r1, PT_R26; \
- swi r27, r1, PT_R27; \
- swi r28, r1, PT_R28; \
- swi r29, r1, PT_R29; \
- swi r30, r1, PT_R30; \
- swi r31, r1, PT_R31; /* Save current task reg */ \
+ swi r2, r1, PTO+PT_R2; /* Save SDA */ \
+ swi r3, r1, PTO+PT_R3; \
+ swi r4, r1, PTO+PT_R4; \
+ swi r5, r1, PTO+PT_R5; \
+ swi r6, r1, PTO+PT_R6; \
+ swi r7, r1, PTO+PT_R7; \
+ swi r8, r1, PTO+PT_R8; \
+ swi r9, r1, PTO+PT_R9; \
+ swi r10, r1, PTO+PT_R10; \
+ swi r11, r1, PTO+PT_R11; /* save clobbered regs after rval */\
+ swi r12, r1, PTO+PT_R12; \
+ swi r13, r1, PTO+PT_R13; /* Save SDA2 */ \
+ swi r14, r1, PTO+PT_PC; /* PC, before IRQ/trap */ \
+ swi r15, r1, PTO+PT_R15; /* Save LP */ \
+ swi r16, r1, PTO+PT_R16; \
+ swi r17, r1, PTO+PT_R17; \
+ swi r18, r1, PTO+PT_R18; /* Save asm scratch reg */ \
+ swi r19, r1, PTO+PT_R19; \
+ swi r20, r1, PTO+PT_R20; \
+ swi r21, r1, PTO+PT_R21; \
+ swi r22, r1, PTO+PT_R22; \
+ swi r23, r1, PTO+PT_R23; \
+ swi r24, r1, PTO+PT_R24; \
+ swi r25, r1, PTO+PT_R25; \
+ swi r26, r1, PTO+PT_R26; \
+ swi r27, r1, PTO+PT_R27; \
+ swi r28, r1, PTO+PT_R28; \
+ swi r29, r1, PTO+PT_R29; \
+ swi r30, r1, PTO+PT_R30; \
+ swi r31, r1, PTO+PT_R31; /* Save current task reg */ \
mfs r11, rmsr; /* save MSR */ \
- swi r11, r1, PT_MSR;
+ swi r11, r1, PTO+PT_MSR;
#define RESTORE_REGS_GP \
- lwi r2, r1, PT_R2; /* restore SDA */ \
- lwi r3, r1, PT_R3; \
- lwi r4, r1, PT_R4; \
- lwi r5, r1, PT_R5; \
- lwi r6, r1, PT_R6; \
- lwi r7, r1, PT_R7; \
- lwi r8, r1, PT_R8; \
- lwi r9, r1, PT_R9; \
- lwi r10, r1, PT_R10; \
- lwi r11, r1, PT_R11; /* restore clobbered regs after rval */\
- lwi r12, r1, PT_R12; \
- lwi r13, r1, PT_R13; /* restore SDA2 */ \
- lwi r14, r1, PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\
- lwi r15, r1, PT_R15; /* restore LP */ \
- lwi r16, r1, PT_R16; \
- lwi r17, r1, PT_R17; \
- lwi r18, r1, PT_R18; /* restore asm scratch reg */ \
- lwi r19, r1, PT_R19; \
- lwi r20, r1, PT_R20; \
- lwi r21, r1, PT_R21; \
- lwi r22, r1, PT_R22; \
- lwi r23, r1, PT_R23; \
- lwi r24, r1, PT_R24; \
- lwi r25, r1, PT_R25; \
- lwi r26, r1, PT_R26; \
- lwi r27, r1, PT_R27; \
- lwi r28, r1, PT_R28; \
- lwi r29, r1, PT_R29; \
- lwi r30, r1, PT_R30; \
- lwi r31, r1, PT_R31; /* Restore cur task reg */
+ lwi r2, r1, PTO+PT_R2; /* restore SDA */ \
+ lwi r3, r1, PTO+PT_R3; \
+ lwi r4, r1, PTO+PT_R4; \
+ lwi r5, r1, PTO+PT_R5; \
+ lwi r6, r1, PTO+PT_R6; \
+ lwi r7, r1, PTO+PT_R7; \
+ lwi r8, r1, PTO+PT_R8; \
+ lwi r9, r1, PTO+PT_R9; \
+ lwi r10, r1, PTO+PT_R10; \
+ lwi r11, r1, PTO+PT_R11; /* restore clobbered regs after rval */\
+ lwi r12, r1, PTO+PT_R12; \
+ lwi r13, r1, PTO+PT_R13; /* restore SDA2 */ \
+ lwi r14, r1, PTO+PT_PC; /* RESTORE_LINK PC, before IRQ/trap */\
+ lwi r15, r1, PTO+PT_R15; /* restore LP */ \
+ lwi r16, r1, PTO+PT_R16; \
+ lwi r17, r1, PTO+PT_R17; \
+ lwi r18, r1, PTO+PT_R18; /* restore asm scratch reg */ \
+ lwi r19, r1, PTO+PT_R19; \
+ lwi r20, r1, PTO+PT_R20; \
+ lwi r21, r1, PTO+PT_R21; \
+ lwi r22, r1, PTO+PT_R22; \
+ lwi r23, r1, PTO+PT_R23; \
+ lwi r24, r1, PTO+PT_R24; \
+ lwi r25, r1, PTO+PT_R25; \
+ lwi r26, r1, PTO+PT_R26; \
+ lwi r27, r1, PTO+PT_R27; \
+ lwi r28, r1, PTO+PT_R28; \
+ lwi r29, r1, PTO+PT_R29; \
+ lwi r30, r1, PTO+PT_R30; \
+ lwi r31, r1, PTO+PT_R31; /* Restore cur task reg */
#define RESTORE_REGS \
- lwi r11, r1, PT_MSR; \
+ lwi r11, r1, PTO+PT_MSR; \
mts rmsr , r11; \
RESTORE_REGS_GP
#define RESTORE_REGS_RTBD \
- lwi r11, r1, PT_MSR; \
+ lwi r11, r1, PTO+PT_MSR; \
andni r11, r11, MSR_EIP; /* clear EIP */ \
ori r11, r11, MSR_EE | MSR_BIP; /* set EE and BIP */ \
mts rmsr , r11; \
@@ -265,11 +265,11 @@ syscall_debug_table:
lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); \
/* FIXME: I can add these two lines to one */ \
/* tophys(r1,r1); */ \
- /* addik r1, r1, -PT_SIZE; */ \
- addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \
+ /* addik r1, r1, -STATE_SAVE_SIZE; */ \
+ addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \
SAVE_REGS \
brid 2f; \
- swi r1, r1, PT_MODE; \
+ swi r1, r1, PTO+PT_MODE; \
1: /* User-mode state save. */ \
lwi r1, r0, TOPHYS(PER_CPU(CURRENT_SAVE)); /* get saved current */\
tophys(r1,r1); \
@@ -277,12 +277,13 @@ syscall_debug_table:
/* MS these three instructions can be added to one */ \
/* addik r1, r1, THREAD_SIZE; */ \
/* tophys(r1,r1); */ \
- /* addik r1, r1, -PT_SIZE; */ \
- addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE; \
+ /* addik r1, r1, -STATE_SAVE_SIZE; */ \
+ addik r1, r1, THREAD_SIZE + CONFIG_KERNEL_BASE_ADDR \
+ - CONFIG_KERNEL_START - STATE_SAVE_SIZE; \
SAVE_REGS \
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP)); \
- swi r11, r1, PT_R1; /* Store user SP. */ \
- swi r0, r1, PT_MODE; /* Was in user-mode. */ \
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */ \
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */ \
/* MS: I am clearing UMS even in case when I come from kernel space */ \
clear_ums; \
2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
@@ -292,7 +293,7 @@ syscall_debug_table:
.extern cpuinfo
C_ENTRY(mb_flush_dcache):
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
addik r3, r0, cpuinfo
@@ -305,12 +306,12 @@ C_ENTRY(mb_flush_dcache):
addk r9, r9, r8
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8
nop
C_ENTRY(mb_invalidate_icache):
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
addik r3, r0, cpuinfo
@@ -323,7 +324,7 @@ C_ENTRY(mb_invalidate_icache):
addk r9, r9, r8
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8
nop
@@ -350,18 +351,18 @@ C_ENTRY(_user_exception):
addik r1, r1, THREAD_SIZE;
tophys(r1,r1);
- addik r1, r1, -PT_SIZE; /* Make room on the stack. */
+ addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */
SAVE_REGS
- swi r0, r1, PT_R3
- swi r0, r1, PT_R4
+ swi r0, r1, PTO+PT_R3
+ swi r0, r1, PTO+PT_R4
- swi r0, r1, PT_MODE; /* Was in user-mode. */
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1; /* Store user SP. */
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */
clear_ums;
2: lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
/* Save away the syscall number. */
- swi r12, r1, PT_R0;
+ swi r12, r1, PTO+PT_R0;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8*/
@@ -380,18 +381,18 @@ C_ENTRY(_user_exception):
beqi r11, 4f
addik r3, r0, -ENOSYS
- swi r3, r1, PT_R3
+ swi r3, r1, PTO+PT_R3
brlid r15, do_syscall_trace_enter
- addik r5, r1, PT_R0
+ addik r5, r1, PTO+PT_R0
# do_syscall_trace_enter returns the new syscall nr.
addk r12, r0, r3
- lwi r5, r1, PT_R5;
- lwi r6, r1, PT_R6;
- lwi r7, r1, PT_R7;
- lwi r8, r1, PT_R8;
- lwi r9, r1, PT_R9;
- lwi r10, r1, PT_R10;
+ lwi r5, r1, PTO+PT_R5;
+ lwi r6, r1, PTO+PT_R6;
+ lwi r7, r1, PTO+PT_R7;
+ lwi r8, r1, PTO+PT_R8;
+ lwi r9, r1, PTO+PT_R9;
+ lwi r10, r1, PTO+PT_R10;
4:
/* Jump to the appropriate function for the system call number in r12
* (r12 is not preserved), or return an error if r12 is not valid.
@@ -432,10 +433,16 @@ C_ENTRY(_user_exception):
/* Entry point used to return from a syscall/trap */
/* We re-enable BIP bit before state restore */
C_ENTRY(ret_from_trap):
- swi r3, r1, PT_R3
- swi r4, r1, PT_R4
-
- lwi r11, r1, PT_MODE;
+ swi r3, r1, PTO+PT_R3
+ swi r4, r1, PTO+PT_R4
+/*
+ * Entry point for returns that must not store r3/r4 back into pt_regs,
+ * i.e. rt_sigreturn, which has already restored them from the signal
+ * context. Reached as "rtsd r15, 8" with r15 set to this label minus 8,
+ * so it stays correct if the number of stores above ever changes.
+ */
+ret_from_trap_no_rval:
+ lwi r11, r1, PTO+PT_MODE;
/* See if returning to kernel mode, if so, skip resched &c. */
bnei r11, 2f;
/* We're returning to user mode, so check for various conditions that
@@ -447,7 +454,7 @@ C_ENTRY(ret_from_trap):
beqi r11, 1f
brlid r15, do_syscall_trace_leave
- addik r5, r1, PT_R0
+ addik r5, r1, PTO+PT_R0
1:
/* We're returning to user mode, so check for various conditions that
* trigger rescheduling. */
@@ -466,7 +473,7 @@ C_ENTRY(ret_from_trap):
andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqi r11, 4f; /* Signals to handle, handle them */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
add r6, r30, r0; /* Arg 2: int in_syscall */
add r30, r0, r0 /* no more restarts */
@@ -478,7 +485,11 @@ C_ENTRY(ret_from_trap):
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
+ /*
+ * Deliberately PT_SIZE, not STATE_SAVE_SIZE: r1 is back at the stack
+ * top, so PTO cancels and the saved SP sits at PT_R1 - PT_SIZE.
+ */
lwi r1, r1, PT_R1 - PT_SIZE;/* Restore user stack pointer. */
bri 6f;
@@ -487,7 +498,7 @@ C_ENTRY(ret_from_trap):
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
tovirt(r1,r1);
6:
TRAP_return: /* Make global symbol for debugging */
@@ -518,8 +529,17 @@ C_ENTRY(ret_from_kernel_thread):
C_ENTRY(sys_rt_sigreturn_wrapper):
addik r30, r0, 0 /* no restarts */
+ /*
+ * rt_sigreturn restores the full register set from the signal
+ * context, so it must skip the r3/r4 syscall-return stores at the
+ * head of ret_from_trap which would otherwise overwrite the
+ * just-restored r3/r4. Every C function returns with "rtsd r15, 8"
+ * -- the ABI return, where the 8 skips the caller's branch and its
+ * delay slot -- so bias r15 by -8 to land on ret_from_trap_no_rval.
+ */
+ addik r15, r0, ret_from_trap_no_rval - 8
brid sys_rt_sigreturn /* Do real work */
- addik r5, r1, 0; /* add user context as 1st arg */
+ addik r5, r1, PTO; /* add user context as 1st arg */
/*
* HW EXCEPTION rutine start
@@ -530,7 +550,7 @@ C_ENTRY(full_exception_trap):
addik r17, r17, -4
SAVE_STATE /* Save registers */
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* FIXME this can be store directly in PT_ESR reg.
* I tested it but there is a fault */
@@ -540,7 +560,7 @@ C_ENTRY(full_exception_trap):
mfs r7, rfsr; /* save FSR */
mts rfsr, r0; /* Clear sticky fsr */
rted r0, full_exception
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
/*
* Unaligned data trap.
@@ -566,14 +586,14 @@ C_ENTRY(unaligned_data_trap):
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r3, resr /* ESR */
mfs r4, rear /* EAR */
rtbd r0, _unaligned_data_exception
- addik r7, r1, 0 /* parameter struct pt_regs * regs */
+ addik r7, r1, PTO /* parameter struct pt_regs * regs */
/*
* Page fault traps.
@@ -596,30 +616,30 @@ C_ENTRY(unaligned_data_trap):
C_ENTRY(page_fault_data_trap):
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r6, rear /* parameter unsigned long address */
mfs r7, resr /* parameter unsigned long error_code */
rted r0, do_page_fault
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
C_ENTRY(page_fault_instr_trap):
SAVE_STATE /* Save registers.*/
/* PC, before IRQ/trap - this is one instruction above */
- swi r17, r1, PT_PC;
+ swi r17, r1, PTO+PT_PC;
tovirt(r1,r1)
/* where the trap should return need -8 to adjust for rtsd r15, 8 */
addik r15, r0, ret_from_exc-8
mfs r6, rear /* parameter unsigned long address */
ori r7, r0, 0 /* parameter unsigned long error_code */
rted r0, do_page_fault
- addik r5, r1, 0 /* parameter struct pt_regs * regs */
+ addik r5, r1, PTO /* parameter struct pt_regs * regs */
/* Entry point used to return from an exception. */
C_ENTRY(ret_from_exc):
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f; /* See if returning to kernel mode, */
/* ... if so, skip resched &c. */
@@ -651,7 +671,7 @@ C_ENTRY(ret_from_exc):
* complete register state. Here we save anything not saved by
* the normal entry sequence, so that it may be safely restored
* (in a possibly modified form) after do_notify_resume returns. */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -663,7 +683,7 @@ C_ENTRY(ret_from_exc):
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer. */
bri 6f;
@@ -672,7 +692,7 @@ C_ENTRY(ret_from_exc):
VM_OFF;
tophys(r1,r1);
RESTORE_REGS_RTBD;
- addik r1, r1, PT_SIZE /* Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space. */
tovirt(r1,r1);
6:
@@ -705,10 +725,10 @@ C_ENTRY(_interrupt):
tophys(r1,r1); /* MS: I have in r1 physical address where stack is */
/* save registers */
/* MS: Make room on the stack -> activation record */
- addik r1, r1, -PT_SIZE;
+ addik r1, r1, -STATE_SAVE_SIZE;
SAVE_REGS
brid 2f;
- swi r1, r1, PT_MODE; /* 0 - user mode, 1 - kernel mode */
+ swi r1, r1, PTO+PT_MODE; /* 0 - user mode, 1 - kernel mode */
1:
/* User-mode state save. */
/* MS: get the saved current */
@@ -718,23 +738,23 @@ C_ENTRY(_interrupt):
addik r1, r1, THREAD_SIZE;
tophys(r1,r1);
/* save registers */
- addik r1, r1, -PT_SIZE;
+ addik r1, r1, -STATE_SAVE_SIZE;
SAVE_REGS
/* calculate mode */
- swi r0, r1, PT_MODE;
+ swi r0, r1, PTO+PT_MODE;
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1;
+ swi r11, r1, PTO+PT_R1;
clear_ums;
2:
lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
tovirt(r1,r1)
addik r15, r0, irq_call;
irq_call:rtbd r0, do_IRQ;
- addik r5, r1, 0;
+ addik r5, r1, PTO;
/* MS: we are in virtual mode */
ret_from_irq:
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f;
1:
@@ -750,7 +770,7 @@ ret_from_irq:
5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqid r11, no_intr_resched
/* Handle a signal return; Pending signals should be in r18. */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -763,7 +783,7 @@ no_intr_resched:
VM_OFF;
tophys(r1,r1);
RESTORE_REGS
- addik r1, r1, PT_SIZE /* MS: Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */
lwi r1, r1, PT_R1 - PT_SIZE;
bri 6f;
/* MS: Return to kernel state. */
@@ -786,7 +806,7 @@ restore:
VM_OFF /* MS: turn off MMU */
tophys(r1,r1)
RESTORE_REGS
- addik r1, r1, PT_SIZE /* MS: Clean up stack space. */
+ addik r1, r1, STATE_SAVE_SIZE /* MS: Clean up stack space. */
tovirt(r1,r1);
6:
IRQ_return: /* MS: Make global symbol for debugging */
@@ -805,29 +825,29 @@ C_ENTRY(_xtmr_manager_reset):
lwi r1, r0, xmb_manager_stackpointer
/* Restore MSR */
- lwi r2, r1, PT_MSR
+ lwi r2, r1, PTO+PT_MSR
mts rmsr, r2
bri 4
/* restore Special purpose registers */
- lwi r2, r1, PT_PID
+ lwi r2, r1, PTO+PT_PID
mts rpid, r2
- lwi r2, r1, PT_TLBI
+ lwi r2, r1, PTO+PT_TLBI
mts rtlbx, r2
- lwi r2, r1, PT_ZPR
+ lwi r2, r1, PTO+PT_ZPR
mts rzpr, r2
#if CONFIG_XILINX_MICROBLAZE0_USE_FPU
- lwi r2, r1, PT_FSR
+ lwi r2, r1, PTO+PT_FSR
mts rfsr, r2
#endif
/* restore all the tlb's */
addik r3, r0, TOPHYS(tlb_skip)
- addik r6, r0, PT_TLBL0
- addik r7, r0, PT_TLBH0
+ addik r6, r0, PTO+PT_TLBL0
+ addik r7, r0, PTO+PT_TLBH0
restore_tlb:
add r6, r6, r1
add r7, r7, r1
@@ -853,9 +873,9 @@ ret_from_reset:
VM_OFF
/* MS: Restore all regs */
RESTORE_REGS
- lwi r14, r1, PT_R14
- lwi r16, r1, PT_PC
- addik r1, r1, PT_SIZE + 36
+ lwi r14, r1, PTO+PT_R14
+ lwi r16, r1, PTO+PT_PC
+ addik r1, r1, STATE_SAVE_SIZE + 36
rtbd r16, 0
nop
@@ -870,11 +890,11 @@ C_ENTRY(_xmb_manager_break):
* Reserve memory in the stack for context store/restore
* (which includes memory for storing tlbs (max two tlbs))
*/
- addik r1, r1, -PT_SIZE - 36
+ addik r1, r1, -STATE_SAVE_SIZE - 36
swi r1, r0, xmb_manager_stackpointer
SAVE_REGS
- swi r14, r1, PT_R14 /* rewrite saved R14 value */
- swi r16, r1, PT_PC; /* PC and r16 are the same */
+ swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */
+ swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */
lwi r6, r0, TOPHYS(xmb_manager_baseaddr)
lwi r7, r0, TOPHYS(xmb_manager_crval)
@@ -888,25 +908,25 @@ C_ENTRY(_xmb_manager_break):
/* Save the special purpose registers */
mfs r2, rpid
- swi r2, r1, PT_PID
+ swi r2, r1, PTO+PT_PID
mfs r2, rtlbx
- swi r2, r1, PT_TLBI
+ swi r2, r1, PTO+PT_TLBI
mfs r2, rzpr
- swi r2, r1, PT_ZPR
+ swi r2, r1, PTO+PT_ZPR
#if CONFIG_XILINX_MICROBLAZE0_USE_FPU
mfs r2, rfsr
- swi r2, r1, PT_FSR
+ swi r2, r1, PTO+PT_FSR
#endif
mfs r2, rmsr
- swi r2, r1, PT_MSR
+ swi r2, r1, PTO+PT_MSR
/* Save all the tlb's */
addik r3, r0, TOPHYS(tlb_skip)
- addik r6, r0, PT_TLBL0
- addik r7, r0, PT_TLBH0
+ addik r6, r0, PTO+PT_TLBL0
+ addik r7, r0, PTO+PT_TLBH0
save_tlb:
add r6, r6, r1
add r7, r7, r1
@@ -943,7 +963,7 @@ ret_from_break:
mbar 1
mbar 2
bri 4
- suspend
+ mbar 24 /* suspend */
nop
#endif
@@ -963,28 +983,28 @@ C_ENTRY(_debug_exception):
lwi r1, r0, TOPHYS(PER_CPU(ENTRY_SP)); /* Reload kernel stack-ptr*/
/* BIP bit is set on entry, no interrupts can occur */
- addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - PT_SIZE;
+ addik r1, r1, CONFIG_KERNEL_BASE_ADDR - CONFIG_KERNEL_START - STATE_SAVE_SIZE;
SAVE_REGS;
/* save all regs to pt_reg structure */
- swi r0, r1, PT_R0; /* R0 must be saved too */
- swi r14, r1, PT_R14 /* rewrite saved R14 value */
- swi r16, r1, PT_PC; /* PC and r16 are the same */
+ swi r0, r1, PTO+PT_R0; /* R0 must be saved too */
+ swi r14, r1, PTO+PT_R14 /* rewrite saved R14 value */
+ swi r16, r1, PTO+PT_PC; /* PC and r16 are the same */
/* save special purpose registers to pt_regs */
mfs r11, rear;
- swi r11, r1, PT_EAR;
+ swi r11, r1, PTO+PT_EAR;
mfs r11, resr;
- swi r11, r1, PT_ESR;
+ swi r11, r1, PTO+PT_ESR;
mfs r11, rfsr;
- swi r11, r1, PT_FSR;
+ swi r11, r1, PTO+PT_FSR;
/* stack pointer is in physical address at it is decrease
- * by PT_SIZE but we need to get correct R1 value */
- addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + PT_SIZE;
- swi r11, r1, PT_R1
+ * by STATE_SAVE_SIZE but we need to get correct R1 value */
+ addik r11, r1, CONFIG_KERNEL_START - CONFIG_KERNEL_BASE_ADDR + STATE_SAVE_SIZE;
+ swi r11, r1, PTO+PT_R1
/* MS: r31 - current pointer isn't changed */
tovirt(r1,r1)
#ifdef CONFIG_KGDB
- addi r5, r1, 0 /* pass pt_reg address as the first arg */
+ addi r5, r1, PTO /* pass pt_reg address as the first arg */
addik r15, r0, dbtrap_call; /* return address */
rtbd r0, microblaze_kgdb_break
nop;
@@ -1000,16 +1020,16 @@ C_ENTRY(_debug_exception):
addik r1, r1, THREAD_SIZE; /* calculate kernel stack pointer */
tophys(r1,r1);
- addik r1, r1, -PT_SIZE; /* Make room on the stack. */
+ addik r1, r1, -STATE_SAVE_SIZE; /* Make room on the stack. */
SAVE_REGS;
- swi r16, r1, PT_PC; /* Save LP */
- swi r0, r1, PT_MODE; /* Was in user-mode. */
+ swi r16, r1, PTO+PT_PC; /* Save LP */
+ swi r0, r1, PTO+PT_MODE; /* Was in user-mode. */
lwi r11, r0, TOPHYS(PER_CPU(ENTRY_SP));
- swi r11, r1, PT_R1; /* Store user SP. */
+ swi r11, r1, PTO+PT_R1; /* Store user SP. */
lwi CURRENT_TASK, r0, TOPHYS(PER_CPU(CURRENT_SAVE));
tovirt(r1,r1)
set_vms;
- addik r5, r1, 0;
+ addik r5, r1, PTO;
addik r15, r0, dbtrap_call;
dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
rtbd r0, sw_exception
@@ -1017,7 +1037,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
/* MS: The first instruction for the second part of the gdb/kgdb */
set_bip; /* Ints masked for state restore */
- lwi r11, r1, PT_MODE;
+ lwi r11, r1, PTO+PT_MODE;
bnei r11, 2f;
/* MS: Return to user space - gdb */
1:
@@ -1036,7 +1056,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
5: andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqi r11, 4f; /* Signals to handle, handle them */
- addik r5, r1, 0; /* Arg 1: struct pt_regs *regs */
+ addik r5, r1, PTO; /* Arg 1: struct pt_regs *regs */
bralid r15, do_notify_resume; /* Handle any signals */
addi r6, r0, 0; /* Arg 2: int in_syscall */
bri 1b
@@ -1047,7 +1067,7 @@ dbtrap_call: /* Return point for kernel/user entry + 8 because of rtsd r15, 8 */
tophys(r1,r1);
/* MS: Restore all regs */
RESTORE_REGS_RTBD
- addik r1, r1, PT_SIZE /* Clean up stack space */
+ addik r1, r1, STATE_SAVE_SIZE /* Clean up stack space */
lwi r1, r1, PT_R1 - PT_SIZE; /* Restore user stack pointer */
DBTRAP_return_user: /* MS: Make global symbol for debugging */
rtbd r16, 0; /* MS: Instructions to return from a debug trap */
@@ -1058,9 +1078,9 @@ DBTRAP_return_user: /* MS: Make global symbol for debugging */
tophys(r1,r1);
/* MS: Restore all regs */
RESTORE_REGS_RTBD
- lwi r14, r1, PT_R14;
- lwi r16, r1, PT_PC;
- addik r1, r1, PT_SIZE; /* MS: Clean up stack space */
+ lwi r14, r1, PTO+PT_R14;
+ lwi r16, r1, PTO+PT_PC;
+ addik r1, r1, STATE_SAVE_SIZE; /* MS: Clean up stack space */
tovirt(r1,r1);
DBTRAP_return_kernel: /* MS: Make global symbol for debugging */
rtbd r16, 0; /* MS: Instructions to return from a debug trap */
@@ -1158,7 +1178,7 @@ ENTRY(_switch_to)
.ent xmb_inject_err
.type xmb_inject_err, @function
xmb_inject_err:
- addik r1, r1, -PT_SIZE
+ addik r1, r1, -STATE_SAVE_SIZE
SAVE_REGS
/* Switch to real mode */
@@ -1182,7 +1202,7 @@ xmb_inject_err:
nop;
1:
RESTORE_REGS
- addik r1, r1, PT_SIZE
+ addik r1, r1, STATE_SAVE_SIZE
rtsd r15, 8;
nop;
.end xmb_inject_err
diff --git a/arch/microblaze/kernel/hw_exception_handler.S b/arch/microblaze/kernel/hw_exception_handler.S
index 07ea23965f81..4693916be526 100644
--- a/arch/microblaze/kernel/hw_exception_handler.S
+++ b/arch/microblaze/kernel/hw_exception_handler.S
@@ -313,6 +313,11 @@ _MB_HW_ExceptionVectorTable:
.align 4
.ent _hw_exception_handler
_hw_exception_handler:
+ /*
+ * This handler saves into the static pt_pool_space buffer, not into a
+ * stack frame, so the plain PT_* offsets are right here -- do not add
+ * the PTO of entry.S's frames.
+ */
swi r1, r0, TOPHYS(pt_pool_space + PT_R1); /* GET_SP */
/* Save date to kernel memory. Here is the problem
* when you came from user space */
diff --git a/arch/microblaze/kernel/process.c b/arch/microblaze/kernel/process.c
index 6cbf642d7b80..090e1697ea7a 100644
--- a/arch/microblaze/kernel/process.c
+++ b/arch/microblaze/kernel/process.c
@@ -19,6 +19,7 @@
#include <linux/bitops.h>
#include <linux/ptrace.h>
#include <asm/cacheflush.h>
+#include <asm/entry.h>
void show_regs(struct pt_regs *regs)
{
@@ -65,7 +66,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
* the registers. That's OK for a brand new thread.*/
memset(childregs, 0, sizeof(struct pt_regs));
memset(&ti->cpu_context, 0, sizeof(struct cpu_context));
- ti->cpu_context.r1 = (unsigned long)childregs;
+ ti->cpu_context.r1 = (unsigned long)childregs - PTO;
ti->cpu_context.r20 = (unsigned long)args->fn;
ti->cpu_context.r19 = (unsigned long)args->fn_arg;
childregs->pt_mode = 1;
@@ -79,7 +80,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
childregs->r1 = usp;
memset(&ti->cpu_context, 0, sizeof(struct cpu_context));
- ti->cpu_context.r1 = (unsigned long)childregs;
+ ti->cpu_context.r1 = (unsigned long)childregs - PTO;
childregs->msr |= MSR_UMS;
/* we should consider the fact that childregs is a copy of the parent
diff --git a/arch/microblaze/kernel/signal.c b/arch/microblaze/kernel/signal.c
index c78a0ff48066..6bbc16f98d5e 100644
--- a/arch/microblaze/kernel/signal.c
+++ b/arch/microblaze/kernel/signal.c
@@ -33,6 +33,7 @@
#include <linux/linkage.h>
#include <linux/resume_user_mode.h>
#include <asm/entry.h>
+#include <asm/registers.h>
#include <asm/ucontext.h>
#include <linux/uaccess.h>
#include <linux/syscalls.h>
@@ -49,6 +50,14 @@ struct sigframe {
};
struct rt_sigframe {
+ /*
+ * Home area for the handler's register arguments: the MicroBlaze
+ * ABI reserves [r1+0] for the return address and lets the callee
+ * store r5..r10 at [r1+4]..[r1+24], and r1 points at this frame
+ * when the handler is entered. Without the gap those stores
+ * corrupt frame->info.
+ */
+ unsigned long abi_gap[7];
struct siginfo info;
struct ucontext uc;
unsigned long tramp[2]; /* signal trampoline */
@@ -73,6 +82,22 @@ static int restore_sigcontext(struct pt_regs *regs,
COPY(pc); COPY(ear); COPY(esr); COPY(fsr);
#undef COPY
+ /*
+ * The frame is user-writable, so restore only the user-writable
+ * status flags (carry) and keep the kernel-controlled MSR bits
+ * (UMS/VMS/IE/EE/...) from regs->msr: rtbd derives the resumed mode
+ * from UMS/VMS, so a verbatim restore would hand userspace the
+ * privileged return state. Same idea as x86 masking the restored
+ * EFLAGS to FIX_EFLAGS.
+ */
+ {
+ unsigned long msr;
+
+ err |= __get_user(msr, &sc->regs.msr);
+ regs->msr = (regs->msr & ~(MSR_C | MSR_CC)) |
+ (msr & (MSR_C | MSR_CC));
+ }
+
*rval_p = regs->r3;
return err;
@@ -132,6 +157,7 @@ setup_sigcontext(struct sigcontext __user *sc, struct pt_regs *regs,
COPY(r26); COPY(r27); COPY(r28); COPY(r29);
COPY(r30); COPY(r31);
COPY(pc); COPY(ear); COPY(esr); COPY(fsr);
+ COPY(msr); /* restore_sigcontext() accepts only carry state */
#undef COPY
err |= __put_user(mask, &sc->oldmask);
diff --git a/arch/mips/mm/mmap.c b/arch/mips/mm/mmap.c
index 5d2a1225785b..119edaa928f3 100644
--- a/arch/mips/mm/mmap.c
+++ b/arch/mips/mm/mmap.c
@@ -9,6 +9,7 @@
#include <linux/compiler.h>
#include <linux/elf-randomize.h>
#include <linux/errno.h>
+#include <linux/hugetlb.h>
#include <linux/mm.h>
#include <linux/mman.h>
#include <linux/export.h>
@@ -72,8 +73,11 @@ static unsigned long arch_get_unmapped_area_common(struct file *filp,
}
info.length = len;
- info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
info.align_offset = pgoff << PAGE_SHIFT;
+ if (filp && is_file_hugepages(filp))
+ info.align_mask = huge_page_mask_align(filp);
+ else
+ info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
if (dir == DOWN) {
info.flags = VM_UNMAPPED_AREA_TOPDOWN;
diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
index 24fe85fa169d..78bafa317fa6 100644
--- a/arch/mips/mm/tlb-r4k.c
+++ b/arch/mips/mm/tlb-r4k.c
@@ -332,6 +332,19 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
/* this could be a huge page */
if (pmd_leaf(*pmdp)) {
unsigned long lo;
+
+ /*
+ * The probe above only covers the 8K pair at @address, and
+ * a huge mapping is installed with the huge-aligned address
+ * while the refill that started the fault left a 4K entry
+ * for the faulting page elsewhere in the range. Writing the
+ * huge entry to a random index would leave two entries
+ * matching the faulting address; leave it to the refill and
+ * TLBL/TLBS handlers, which probe the faulting address.
+ */
+ if (idx < 0)
+ goto out;
+
write_c0_pagemask(PM_HUGE_MASK);
ptep = (pte_t *)pmdp;
lo = pte_to_entrylo(pte_val(*ptep));
@@ -339,10 +352,7 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
write_c0_entrylo1(lo + (HPAGE_SIZE >> 7));
mtc0_tlbw_hazard();
- if (idx < 0)
- tlb_write_random();
- else
- tlb_write_indexed();
+ tlb_write_indexed();
tlbw_use_hazard();
write_c0_pagemask(PM_DEFAULT_MASK);
} else
@@ -380,6 +390,9 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
tlb_write_indexed();
}
tlbw_use_hazard();
+#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
+out:
+#endif
htw_start();
flush_micro_tlb_vm(vma);
diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index f32de8704d4d..44fa414dc5a7 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -14,6 +14,13 @@
#include <asm/ppc-opcode.h>
#include <linux/build_bug.h>
+/*
+ * We need at least 2 passes for proper code generation, and may need
+ * additional passes if code size changes between passes.
+ */
+#define CODEGEN_MIN_PASSES 2
+#define CODEGEN_MAX_PASSES 3
+
#ifdef CONFIG_PPC64_ELF_ABI_V1
#define FUNCTION_DESCR_SIZE 24
#else
@@ -214,10 +221,11 @@ int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *
int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct codegen_context *ctx,
u32 *addrs, int pass, bool extra_pass);
void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx);
-void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx);
-void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx);
+void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx);
+void bpf_jit_build_fentry_stubs(u32 *image, u32 *fimage, struct codegen_context *ctx);
void bpf_jit_realloc_regs(struct codegen_context *ctx);
-int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr);
+int bpf_jit_emit_exit_insn(u32 *image, u32 *fimage, struct codegen_context *ctx, int tmp_reg,
+ long exit_addr);
void prepare_for_fsession_fentry(u32 *image, struct codegen_context *ctx, int cookie_cnt,
int cookie_off, int retval_off);
void store_func_meta(u32 *image, struct codegen_context *ctx, u64 func_meta, int func_meta_off);
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 7a612688e7a2..127240933612 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -49,11 +49,35 @@ asm (
" .popsection ;"
);
-void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx)
+void bpf_jit_build_fentry_stubs(u32 *image, u32 *fimage, struct codegen_context *ctx)
{
int ool_stub_idx, long_branch_stub_idx;
+ int ool_stub_sz;
/*
+ * Align the mis-aligned dummy_tramp_addr field in the fimage.
+ * The alignment NOP must appear before OOL stub, to make
+ * ool_stub_idx & long_branch_stub_idx constant from end.
+ *
+ * The fimage can be non 8-byte aligned, so final alignment depends
+ * on start of fimage and the stub's instruction count offset. The
+ * OOL stub size is 4 instructions (with CONFIG_PPC_FTRACE_OUT_OF_LINE)
+ * or 3 instructions (without) before dummy_tramp_addr.
+ *
+ * Emit a NOP here if address is not SZL aligned.
+ *
+ * In pass=0 when image==NULL, conservatively account for space
+ * required to accommodate alignment NOP. In case final pass skips
+ * emitting alignment NOP, the image buffer have 4 spare bytes and
+ * jited_len signifies correct program size.
+ */
+
+ ool_stub_sz = IS_ENABLED(CONFIG_PPC_FTRACE_OUT_OF_LINE) ? 16 : 12;
+ if (!image || !IS_ALIGNED((unsigned long)fimage + ctx->idx*4 + ool_stub_sz, SZL))
+ EMIT(PPC_RAW_NOP());
+
+ /*
+ * nop // optional, for alignment of dummy_tramp_addr
* Out-of-line stub:
* mflr r0
* [b|bl] tramp
@@ -70,7 +94,7 @@ void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx)
/*
* Long branch stub:
- * .long <dummy_tramp_addr>
+ * .long <dummy_tramp_addr> // 8-byte aligned
* mflr r11
* bcl 20,31,$+4
* mflr r12
@@ -81,6 +105,7 @@ void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx)
*/
if (image)
*((unsigned long *)&image[ctx->idx]) = (unsigned long)dummy_tramp;
+
ctx->idx += SZL / 4;
long_branch_stub_idx = ctx->idx;
EMIT(PPC_RAW_MFLR(_R11));
@@ -97,17 +122,17 @@ void bpf_jit_build_fentry_stubs(u32 *image, struct codegen_context *ctx)
}
}
-int bpf_jit_emit_exit_insn(u32 *image, struct codegen_context *ctx, int tmp_reg, long exit_addr)
+int bpf_jit_emit_exit_insn(u32 *image, u32 *fimage, struct codegen_context *ctx,
+ int tmp_reg, long exit_addr)
{
- if (!exit_addr || is_offset_in_branch_range(exit_addr - (ctx->idx * 4))) {
+ if (exit_addr && is_offset_in_branch_range(exit_addr - (long)(ctx->idx * 4))) {
PPC_JMP(exit_addr);
- } else if (ctx->alt_exit_addr) {
- if (WARN_ON(!is_offset_in_branch_range((long)ctx->alt_exit_addr - (ctx->idx * 4))))
- return -1;
+ } else if (ctx->alt_exit_addr && is_offset_in_branch_range(
+ (long)(ctx->alt_exit_addr) - (long)(ctx->idx * 4))) {
PPC_JMP(ctx->alt_exit_addr);
} else {
ctx->alt_exit_addr = ctx->idx * 4;
- bpf_jit_build_epilogue(image, ctx);
+ bpf_jit_build_epilogue(image, fimage, ctx);
}
return 0;
@@ -274,6 +299,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
*/
if (cgctx.seen & SEEN_TAILCALL || !is_offset_in_branch_range((long)cgctx.idx * 4)) {
cgctx.idx = 0;
+ cgctx.alt_exit_addr = 0;
if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false))
goto out_err;
}
@@ -286,7 +312,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
*/
bpf_jit_build_prologue(NULL, &cgctx);
addrs[fp->len] = cgctx.idx * 4;
- bpf_jit_build_epilogue(NULL, &cgctx);
+ bpf_jit_build_epilogue(NULL, NULL, &cgctx);
fixup_len = fp->aux->num_exentries * BPF_FIXUP_LEN * 4;
extable_len = fp->aux->num_exentries * sizeof(struct exception_table_entry);
@@ -306,10 +332,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
code_base = (u32 *)(image + FUNCTION_DESCR_SIZE);
fcode_base = (u32 *)(fimage + FUNCTION_DESCR_SIZE);
- /* Code generation passes 1-2 */
- for (pass = 1; pass < 3; pass++) {
+ /* Code generation passes 1-2+, loop until program size converges. */
+ for (pass = 1; pass <= CODEGEN_MAX_PASSES; pass++) {
+ u32 prev_proglen = proglen;
+
/* Now build the prologue, body code & epilogue for real. */
cgctx.idx = 0;
+ cgctx.exentry_idx = 0;
cgctx.alt_exit_addr = 0;
bpf_jit_build_prologue(code_base, &cgctx);
if (bpf_jit_build_body(fp, code_base, fcode_base, &cgctx, addrs, pass,
@@ -318,11 +347,26 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
bpf_jit_binary_pack_free(fhdr, hdr);
goto out_err;
}
- bpf_jit_build_epilogue(code_base, &cgctx);
+ addrs[fp->len] = cgctx.idx * 4;
+ bpf_jit_build_epilogue(code_base, fcode_base, &cgctx);
+
+ proglen = cgctx.idx * 4;
if (bpf_jit_enable > 1)
pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass,
- proglen - (cgctx.idx * 4), cgctx.seen);
+ prev_proglen - proglen, cgctx.seen);
+
+ /* Check if program size has converged, but ensure minimum passes */
+ if (pass >= CODEGEN_MIN_PASSES && proglen == prev_proglen)
+ break;
+
+ if (pass == CODEGEN_MAX_PASSES && proglen != prev_proglen) {
+ pr_err("BPF JIT: Program did not converge after %d passes\n",
+ CODEGEN_MAX_PASSES);
+ bpf_arch_text_copy(&fhdr->size, &hdr->size, sizeof(hdr->size));
+ bpf_jit_binary_pack_free(fhdr, hdr);
+ goto out_err;
+ }
}
if (bpf_jit_enable > 1)
@@ -357,7 +401,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
(void *)fimage + FUNCTION_DESCR_SIZE);
out_addrs:
- if (!image && priv_stack_ptr) {
+ if (!fp->jited && priv_stack_ptr) {
fp->aux->priv_stack_ptr = NULL;
free_percpu(priv_stack_ptr);
}
@@ -399,7 +443,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass
u32 *fixup;
/* Populate extable entries only in the last pass */
- if (pass != 2)
+ if (pass < CODEGEN_MIN_PASSES)
return 0;
if (!fp->aux->extable ||
diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c
index bfdc50740da8..f5b9441cf46a 100644
--- a/arch/powerpc/net/bpf_jit_comp32.c
+++ b/arch/powerpc/net/bpf_jit_comp32.c
@@ -229,7 +229,7 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx
}
-void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
+void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx)
{
EMIT(PPC_RAW_MR(_R3, bpf_to_ppc(BPF_REG_0)));
@@ -237,7 +237,7 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
EMIT(PPC_RAW_BLR());
- bpf_jit_build_fentry_stubs(image, ctx);
+ bpf_jit_build_fentry_stubs(image, fimage, ctx);
}
/* Relative offset needs to be calculated based on final image location */
@@ -1149,7 +1149,8 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
* we'll just fall through to the epilogue.
*/
if (i != flen - 1) {
- ret = bpf_jit_emit_exit_insn(image, ctx, _R0, exit_addr);
+ ret = bpf_jit_emit_exit_insn(image, fimage,
+ ctx, _R0, exit_addr);
if (ret)
return ret;
}
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index fc9db691e820..667e2d4d2c61 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -398,7 +398,7 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx
}
}
-void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
+void bpf_jit_build_epilogue(u32 *image, u32 *fimage, struct codegen_context *ctx)
{
bpf_jit_emit_common_epilogue(image, ctx);
@@ -407,7 +407,7 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
EMIT(PPC_RAW_BLR());
- bpf_jit_build_fentry_stubs(image, ctx);
+ bpf_jit_build_fentry_stubs(image, fimage, ctx);
}
/*
@@ -1737,7 +1737,8 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, u32 *fimage, struct code
* we'll just fall through to the epilogue.
*/
if (i != flen - 1) {
- ret = bpf_jit_emit_exit_insn(image, ctx, tmp1_reg, exit_addr);
+ ret = bpf_jit_emit_exit_insn(image, fimage, ctx,
+ tmp1_reg, exit_addr);
if (ret)
return ret;
}
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 272e9aab66d4..32d414c2cb84 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -69,6 +69,8 @@ static struct iommu_table *iommu_pseries_alloc_table(int node)
return tbl;
}
+static phys_addr_t pseries_ddw_max_ram __ro_after_init;
+
#ifdef CONFIG_IOMMU_API
static struct iommu_table_group_ops spapr_tce_table_group_ops;
#endif
@@ -1284,15 +1286,19 @@ struct failed_ddw_pdn {
static LIST_HEAD(failed_ddw_pdn_list);
-static phys_addr_t ddw_memory_hotplug_max(void)
+static phys_addr_t __init ddw_memory_hotplug_max(void)
{
- resource_size_t max_addr;
+ resource_size_t max_addr = memory_hotplug_max();
+ struct device_node *memory;
-#if defined(CONFIG_NUMA) && defined(CONFIG_MEMORY_HOTPLUG)
- max_addr = hot_add_drconf_memory_max();
-#else
- max_addr = memblock_end_of_DRAM();
-#endif
+ for_each_node_by_type(memory, "memory") {
+ struct resource res;
+
+ if (of_address_to_resource(memory, 0, &res))
+ continue;
+
+ max_addr = max_t(resource_size_t, max_addr, res.end + 1);
+ }
return max_addr;
}
@@ -1447,7 +1453,7 @@ static struct property *ddw_property_create(const char *propname, u32 liobn, u64
static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mask)
{
int len = 0, ret;
- int max_ram_len = order_base_2(ddw_memory_hotplug_max());
+ int max_ram_len = order_base_2(pseries_ddw_max_ram);
struct ddw_query_response query;
struct ddw_create_response create;
int page_shift;
@@ -1617,12 +1623,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (default_win_removed && pmem_present && !direct_mapping) {
/* DDW is big enough to be split */
if ((1ULL << ddw_sz) >=
- MIN_DDW_VPMEM_DMA_WINDOW + (1ULL << max_ram_len)) {
+ MIN_DDW_VPMEM_DMA_WINDOW + pseries_ddw_max_ram) {
direct_mapping = true;
/* offset of the Dynamic part of DDW */
- dynamic_offset = 1ULL << max_ram_len;
+ dynamic_offset = pseries_ddw_max_ram;
}
/* DDW will at least have dynamic allocation */
@@ -1669,7 +1675,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
if (direct_mapping) {
/* DDW maps the whole partition, so enable direct DMA mapping */
- ret = walk_system_ram_range(0, ddw_memory_hotplug_max() >> PAGE_SHIFT,
+ ret = walk_system_ram_range(0, pseries_ddw_max_ram >> PAGE_SHIFT,
win64->value, tce_setrange_multi_pSeriesLP_walk);
if (ret) {
dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n",
@@ -1772,8 +1778,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn, u64 dma_mas
/* For pre-mapped memory, set bus_dma_limit to the max RAM */
if (direct_mapping)
- dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset +
- (1ULL << max_ram_len);
+ dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + pseries_ddw_max_ram;
dev_info(&dev->dev, "lsa_required: %x, lsa_enabled: %x, direct mapping: %x\n",
limited_addr_req, limited_addr_enabled, direct_mapping);
@@ -2420,23 +2425,35 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
{
struct dma_win *window;
struct memory_notify *arg = data;
+ unsigned long limit = arg->nr_pages;
+ unsigned long max_ram_pages = pseries_ddw_max_ram >> PAGE_SHIFT;
int ret = 0;
/* This notifier can get called when onlining persistent memory as well.
* TCEs are not pre-mapped for persistent memory. Persistent memory will
- * always be above ddw_memory_hotplug_max()
+ * always be above pseries_ddw_max_ram
*/
+ if (arg->start_pfn >= max_ram_pages)
+ return NOTIFY_OK;
+
+ /* RAM is being DLPAR'ed. The range should never exceed max ram.
+ * Just in case, clamp the range and throw a warning.
+ */
+ if (arg->start_pfn + limit > max_ram_pages) {
+ limit = max_ram_pages - arg->start_pfn;
+ WARN_ONCE(1, "Limiting Page Range %lx - %lx to Max Mem Pages: %lx\n",
+ arg->start_pfn, arg->start_pfn + arg->nr_pages,
+ max_ram_pages);
+ }
switch (action) {
case MEM_GOING_ONLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_setrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2444,12 +2461,10 @@ static int iommu_mem_notifier(struct notifier_block *nb, unsigned long action,
case MEM_OFFLINE:
spin_lock(&dma_win_list_lock);
list_for_each_entry(window, &dma_win_list, list) {
- if (window->direct && (arg->start_pfn << PAGE_SHIFT) <
- ddw_memory_hotplug_max()) {
+ if (window->direct) {
ret |= tce_clearrange_multi_pSeriesLP(arg->start_pfn,
- arg->nr_pages, window->prop);
+ limit, window->prop);
}
- /* XXX log error */
}
spin_unlock(&dma_win_list_lock);
break;
@@ -2533,6 +2548,14 @@ void __init iommu_init_early_pSeries(void)
register_memory_notifier(&iommu_mem_nb);
set_pci_dma_ops(&dma_iommu_ops);
+
+ /* During init determine the max memory an LPAR can have and set it. This
+ * will be used for pre-mapping RAM in DDW. memblock_end_of_DRAM() can
+ * change during the running of LPAR - daxctl can add pmemory as
+ * "system-ram". This memory range should not be pre-mapped in DDW since
+ * the address of pmemory can be much higher than the DDW size.
+ */
+ pseries_ddw_max_ram = ddw_memory_hotplug_max();
}
static int __init disable_multitce(char *str)
diff --git a/arch/powerpc/platforms/pseries/plpks-sysfs.c b/arch/powerpc/platforms/pseries/plpks-sysfs.c
index c2ebcbb41ae3..f2436229f323 100644
--- a/arch/powerpc/platforms/pseries/plpks-sysfs.c
+++ b/arch/powerpc/platforms/pseries/plpks-sysfs.c
@@ -45,6 +45,16 @@ static const struct attribute *config_attrs[] = {
static struct kobject *plpks_kobj, *plpks_config_kobj;
+static umode_t plpks_config_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int n)
+{
+ if (attr == &attr_wrapping_features.attr &&
+ !plpks_wrapping_is_supported())
+ return 0;
+
+ return attr->mode;
+}
+
int plpks_config_create_softlink(struct kobject *from)
{
if (!plpks_config_kobj)
@@ -52,13 +62,13 @@ int plpks_config_create_softlink(struct kobject *from)
return sysfs_create_link(from, plpks_config_kobj, "config");
}
+static const struct attribute_group config_group = {
+ .attrs = (struct attribute **)config_attrs,
+ .is_visible = plpks_config_attr_is_visible,
+};
+
static __init int plpks_sysfs_config(struct kobject *kobj)
{
- struct attribute_group config_group = {
- .name = NULL,
- .attrs = (struct attribute **)config_attrs,
- };
-
return sysfs_create_group(kobj, &config_group);
}
diff --git a/arch/powerpc/platforms/pseries/plpks.c b/arch/powerpc/platforms/pseries/plpks.c
index 23e4e2a922fc..896eda3eb65f 100644
--- a/arch/powerpc/platforms/pseries/plpks.c
+++ b/arch/powerpc/platforms/pseries/plpks.c
@@ -637,6 +637,9 @@ int plpks_signed_update_var(struct plpks_var *var, u64 flags)
u64 continuetoken = 0;
u64 timeout = 0;
+ if (!var)
+ return -EINVAL;
+
if (!var->data || var->datalen <= 0 || var->namelen > PLPKS_MAX_NAME_SIZE)
return -EINVAL;
@@ -682,7 +685,7 @@ int plpks_signed_update_var(struct plpks_var *var, u64 flags)
kfree(label);
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
@@ -751,7 +754,7 @@ int plpks_write_var(struct plpks_var var)
rc = pseries_status_to_err(rc);
kfree(label);
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
@@ -809,7 +812,7 @@ int plpks_remove_var(char *component, u8 varos, struct plpks_var_name vname)
rc = pseries_status_to_err(rc);
kfree(label);
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
@@ -822,11 +825,14 @@ static int plpks_read_var(u8 consumer, struct plpks_var *var)
u8 *output;
int rc;
+ if (!var)
+ return -EINVAL;
+
if (var->namelen > PLPKS_MAX_NAME_SIZE)
return -EINVAL;
if (var->policy & PLPKS_WRAPPINGKEY)
- return -EINVAL;
+ return -EPERM;
auth = construct_auth(consumer);
if (IS_ERR(auth))
@@ -856,28 +862,27 @@ static int plpks_read_var(u8 consumer, struct plpks_var *var)
virt_to_phys(var->name), var->namelen, virt_to_phys(output),
maxobjsize);
-
if (rc != H_SUCCESS) {
rc = pseries_status_to_err(rc);
- goto out_free_output;
+ if (rc != -EPERM || !retbuf[1])
+ goto out_free_output;
+ goto out_copy_policy;
}
if (!var->data || var->datalen > retbuf[0])
var->datalen = retbuf[0];
- var->policy = retbuf[1];
-
if (var->data)
memcpy(var->data, output, var->datalen);
- rc = 0;
-
+out_copy_policy:
+ var->policy = retbuf[1];
out_free_output:
- kfree(output);
+ kfree_sensitive(output);
out_free_label:
kfree(label);
out_free_auth:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
@@ -934,7 +939,7 @@ int plpks_gen_wrapping_key(void)
int rc = 0, pseries_status = 0;
struct plpks_var var = {
.name = PLPKS_WRAPKEY_NAME,
- .namelen = strlen(var.name),
+ .namelen = sizeof(PLPKS_WRAPKEY_NAME) - 1,
.policy = PLPKS_WRAPPINGKEY,
.os = PLPKS_VAR_LINUX,
.component = PLPKS_WRAPKEY_COMPONENT
@@ -970,7 +975,7 @@ int plpks_gen_wrapping_key(void)
kfree(label);
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
EXPORT_SYMBOL_GPL(plpks_gen_wrapping_key);
@@ -1029,7 +1034,7 @@ int plpks_wrap_object(u8 **input_buf, u32 input_len, u16 wrap_flags,
bool sb_enforce_bit = wrap_flags & BIT(1);
struct plpks_var var = {
.name = PLPKS_WRAPKEY_NAME,
- .namelen = strlen(var.name),
+ .namelen = sizeof(PLPKS_WRAPKEY_NAME) - 1,
.os = PLPKS_VAR_LINUX,
.component = PLPKS_WRAPKEY_COMPONENT
};
@@ -1095,7 +1100,7 @@ int plpks_wrap_object(u8 **input_buf, u32 input_len, u16 wrap_flags,
out_free_label:
kfree(label);
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
EXPORT_SYMBOL_GPL(plpks_wrap_object);
@@ -1172,14 +1177,14 @@ int plpks_unwrap_object(u8 **input_buf, u32 input_len, u8 **output_buf,
if (rc) {
pr_err("H_PKS_UNWRAP_OBJECT failed. pseries_status=%d, rc=%d",
pseries_status, rc);
- kfree(*output_buf);
+ kfree_sensitive(*output_buf);
*output_buf = NULL;
} else {
*output_len = retbuf[1];
}
out:
- kfree(auth);
+ kfree_sensitive(auth);
return rc;
}
EXPORT_SYMBOL_GPL(plpks_unwrap_object);
diff --git a/arch/powerpc/platforms/pseries/plpks_sed_ops.c b/arch/powerpc/platforms/pseries/plpks_sed_ops.c
index 7c873c9589ef..ac3ec2ca7fbe 100644
--- a/arch/powerpc/platforms/pseries/plpks_sed_ops.c
+++ b/arch/powerpc/platforms/pseries/plpks_sed_ops.c
@@ -84,14 +84,16 @@ int sed_read_key(char *keyname, char *key, u_int *keylen)
ret = plpks_read_os_var(&var);
if (ret != 0)
- return ret;
+ goto out;
len = min_t(u16, be32_to_cpu(data.key_len), var.datalen);
memcpy(key, data.key, len);
key[len] = '\0';
*keylen = len;
- return 0;
+out:
+ memzero_explicit(&data, sizeof(data));
+ return ret;
}
/*
@@ -102,6 +104,7 @@ int sed_write_key(char *keyname, char *key, u_int keylen)
struct plpks_var var;
struct plpks_sed_object_data data;
struct plpks_var_name vname;
+ int ret;
plpks_init_var(&var, keyname);
@@ -127,5 +130,8 @@ int sed_write_key(char *keyname, char *key, u_int keylen)
vname.name = var.name;
plpks_remove_var(var.component, var.os, vname);
- return plpks_write_var(var);
+ ret = plpks_write_var(var);
+ memzero_explicit(&data, sizeof(data));
+
+ return ret;
}
diff --git a/arch/riscv/boot/dts/spacemit/k3.dtsi b/arch/riscv/boot/dts/spacemit/k3.dtsi
index c3f2dce0969c..f9c2ffedeae8 100644
--- a/arch/riscv/boot/dts/spacemit/k3.dtsi
+++ b/arch/riscv/boot/dts/spacemit/k3.dtsi
@@ -1248,7 +1248,7 @@ simsic: interrupt-controller@e0400000 {
msi-controller;
riscv,guest-index-bits = <6>;
riscv,hart-index-bits = <4>;
- riscv,num-guest-ids = <511>;
+ riscv,num-guest-ids = <63>;
riscv,num-ids = <511>;
};
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 40b1ed4f3ea8..bda7c021b429 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -993,7 +993,14 @@ static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long
static inline bool pmd_user_accessible_page(struct mm_struct *mm, unsigned long addr, pmd_t pmd)
{
- return pmd_leaf(pmd) && pmd_user(pmd);
+ /*
+ * page_table_check() must ignore THP split invalidation entries created by
+ * pmd_mkinvalid(). These retain _PAGE_LEAF so pmd_present()/pmd_leaf() stay
+ * true during the split, but they no longer describe a user-accessible
+ * mapping once both _PAGE_PRESENT and _PAGE_PROT_NONE are cleared.
+ */
+ return (pmd_val(pmd) & (_PAGE_PRESENT | _PAGE_PROT_NONE)) &&
+ (pmd_val(pmd) & _PAGE_LEAF) && pmd_user(pmd);
}
static inline bool pud_user_accessible_page(struct mm_struct *mm, unsigned long addr, pud_t pud)
diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c
index dec0ba5eff5e..86141e5bab82 100644
--- a/arch/riscv/kernel/usercfi.c
+++ b/arch/riscv/kernel/usercfi.c
@@ -15,6 +15,7 @@
#include <linux/syscalls.h>
#include <linux/prctl.h>
#include <asm/csr.h>
+#include <asm/switch_to.h>
#include <asm/usercfi.h>
unsigned long riscv_nousercfi __read_mostly;
@@ -66,12 +67,7 @@ void set_shstk_status(struct task_struct *task, bool enable)
task->thread_info.user_cfi_state.ubcfi_en = enable ? 1 : 0;
- if (enable)
- task->thread.envcfg |= ENVCFG_SSE;
- else
- task->thread.envcfg &= ~ENVCFG_SSE;
-
- csr_write(CSR_ENVCFG, task->thread.envcfg);
+ envcfg_update_bits(task, ENVCFG_SSE, enable ? ENVCFG_SSE : 0);
}
void set_shstk_lock(struct task_struct *task, bool lock)
@@ -96,12 +92,7 @@ void set_indir_lp_status(struct task_struct *task, bool enable)
task->thread_info.user_cfi_state.ufcfi_en = enable ? 1 : 0;
- if (enable)
- task->thread.envcfg |= ENVCFG_LPE;
- else
- task->thread.envcfg &= ~ENVCFG_LPE;
-
- csr_write(CSR_ENVCFG, task->thread.envcfg);
+ envcfg_update_bits(task, ENVCFG_LPE, enable ? ENVCFG_LPE : 0);
}
void set_indir_lp_lock(struct task_struct *task, bool lock)
@@ -206,10 +197,13 @@ int save_user_shstk(struct task_struct *tsk, unsigned long *saved_shstk_ptr)
*/
int restore_user_shstk(struct task_struct *tsk, unsigned long shstk_ptr)
{
+ unsigned long __user *token_ptr = (unsigned long __user *)shstk_ptr;
unsigned long token = 0;
- token = amo_user_shstk((unsigned long __user *)shstk_ptr, 0);
+ if (!access_ok(token_ptr, sizeof(token)))
+ return -EFAULT;
+ token = amo_user_shstk(token_ptr, 0);
if (token == -1)
return -EFAULT;
diff --git a/arch/riscv/kvm/vcpu_sbi_replace.c b/arch/riscv/kvm/vcpu_sbi_replace.c
index 506a510b6bff..c2f1f7f26442 100644
--- a/arch/riscv/kvm/vcpu_sbi_replace.c
+++ b/arch/riscv/kvm/vcpu_sbi_replace.c
@@ -64,8 +64,17 @@ static int kvm_sbi_ext_ipi_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (tmp->vcpu_id < hbase)
continue;
hart_bit = tmp->vcpu_id - hbase;
+ /*
+ * kvm_for_each_vcpu() walks kvm->vcpus[] by
+ * vcpu_idx, i.e. the creation order, which has
+ * nothing to do with the vcpu_id (hart id) space
+ * that the SBI IPI operates on. vcpu_ids need not
+ * increase along the iteration, so harts outside
+ * the hart_mask window must be skipped instead of
+ * aborting the loop.
+ */
if (hart_bit >= __riscv_xlen)
- goto done;
+ continue;
if (!(hmask & (1UL << hart_bit)))
continue;
}
@@ -76,7 +85,6 @@ static int kvm_sbi_ext_ipi_handler(struct kvm_vcpu *vcpu, struct kvm_run *run,
kvm_riscv_vcpu_pmu_incr_fw(tmp, SBI_PMU_FW_IPI_RCVD);
}
-done:
if (hbase != -1UL && (hmask ^ sentmask))
retdata->err_val = SBI_ERR_INVALID_PARAM;
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index e713b5818c25..fc3e4f0e3a43 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -431,6 +431,7 @@
#define X86_FEATURE_CPPC_PERF_PRIO (17*32+ 2) /* CPPC Floor Perf support */
#define X86_FEATURE_SMCA (17*32+ 3) /* "smca" Scalable MCA */
#define X86_FEATURE_BTB_CTX_ISOLATION (17*32+ 4) /* AMD: Branch predictions contexts isolated */
+#define X86_FEATURE_TSA_FILL_NO (17*32+ 5) /* AMD CPU not vulnerable to TSA-FILL */
/* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */
#define X86_FEATURE_AVX512_4VNNIW (18*32+ 2) /* "avx512_4vnniw" AVX-512 Neural Network Instructions */
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 40463bad1078..9edc08b867ca 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -416,9 +416,14 @@ static void tsa_init(struct cpuinfo_x86 *c)
else
pr_debug("%s: current revision: 0x%x\n", __func__, c->microcode);
} else {
- setup_force_cpu_cap(X86_FEATURE_TSA_SQ_NO);
- setup_force_cpu_cap(X86_FEATURE_TSA_L1_NO);
+ if (!cpu_has(c, X86_FEATURE_ZEN5)) {
+ setup_force_cpu_cap(X86_FEATURE_TSA_SQ_NO);
+ setup_force_cpu_cap(X86_FEATURE_TSA_L1_NO);
+ }
}
+
+ if (!cpu_has(c, X86_FEATURE_ZEN5))
+ setup_force_cpu_cap(X86_FEATURE_TSA_FILL_NO);
}
static void bsp_init_amd(struct cpuinfo_x86 *c)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 1b2381da4d83..000ebee2d247 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1958,7 +1958,7 @@ early_param("spectre_v2", spectre_v2_parse_cmdline);
static enum spectre_v2_mitigation __init spectre_v2_select_retpoline(void)
{
if (!IS_ENABLED(CONFIG_MITIGATION_RETPOLINE)) {
- pr_err("Kernel not compiled with retpoline; no mitigation available!");
+ pr_err("Kernel not compiled with retpoline");
return SPECTRE_V2_NONE;
}
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index c7352827f491..a9b3308f48a8 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1341,7 +1341,7 @@ static const struct x86_cpu_id cpu_vuln_blacklist[] __initconst = {
VULNBL_AMD(0x17, RETBLEED | SMT_RSB | SRSO | VMSCAPE),
VULNBL_HYGON(0x18, RETBLEED | SMT_RSB | SRSO | VMSCAPE),
VULNBL_AMD(0x19, SRSO | TSA | VMSCAPE),
- VULNBL_AMD(0x1a, SRSO | VMSCAPE),
+ VULNBL_AMD(0x1a, SRSO | TSA | VMSCAPE),
{}
};
@@ -1606,7 +1606,8 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
if (c->x86_vendor == X86_VENDOR_AMD) {
if (!cpu_has(c, X86_FEATURE_TSA_SQ_NO) ||
- !cpu_has(c, X86_FEATURE_TSA_L1_NO)) {
+ !cpu_has(c, X86_FEATURE_TSA_L1_NO) ||
+ !cpu_has(c, X86_FEATURE_TSA_FILL_NO)) {
if (cpu_matches(cpu_vuln_blacklist, TSA) ||
/* Enable bug on Zen guests to allow for live migration. */
(cpu_has(c, X86_FEATURE_HYPERVISOR) && cpu_has(c, X86_FEATURE_ZEN)))
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 41b4880be7ed..c8e216633bf5 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -64,6 +64,7 @@ static const struct cpuid_bit cpuid_bits[] = {
{ X86_FEATURE_AMD_WORKLOAD_CLASS, CPUID_EAX, 22, 0x80000021, 0 },
{ X86_FEATURE_TSA_SQ_NO, CPUID_ECX, 1, 0x80000021, 0 },
{ X86_FEATURE_TSA_L1_NO, CPUID_ECX, 2, 0x80000021, 0 },
+ { X86_FEATURE_TSA_FILL_NO, CPUID_ECX, 7, 0x80000021, 0 },
{ X86_FEATURE_BTB_CTX_ISOLATION, CPUID_ECX, 8, 0x80000021, 0 },
{ X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 },
{ X86_FEATURE_AMD_LBR_V2, CPUID_EAX, 1, 0x80000022, 0 },
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 851f151efb35..b73ec77cd84b 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1263,6 +1263,7 @@ void kvm_initialize_cpu_caps(void)
kvm_cpu_cap_init(CPUID_8000_0021_ECX,
SYNTHESIZED_F(TSA_SQ_NO),
SYNTHESIZED_F(TSA_L1_NO),
+ SYNTHESIZED_F(TSA_FILL_NO),
);
kvm_cpu_cap_init(CPUID_8000_0022_EAX,
diff --git a/arch/x86/kvm/reverse_cpuid.h b/arch/x86/kvm/reverse_cpuid.h
index 2ad25781cefb..22c346332b38 100644
--- a/arch/x86/kvm/reverse_cpuid.h
+++ b/arch/x86/kvm/reverse_cpuid.h
@@ -77,6 +77,7 @@
/* CPUID level 0x80000021 (ECX) */
#define KVM_X86_FEATURE_TSA_SQ_NO KVM_X86_FEATURE(CPUID_8000_0021_ECX, 1)
#define KVM_X86_FEATURE_TSA_L1_NO KVM_X86_FEATURE(CPUID_8000_0021_ECX, 2)
+#define KVM_X86_FEATURE_TSA_FILL_NO KVM_X86_FEATURE(CPUID_8000_0021_ECX, 7)
struct cpuid_reg {
u32 function;
@@ -153,6 +154,7 @@ static __always_inline u32 __feature_translate(int x86_feature)
KVM_X86_TRANSLATE_FEATURE(TSA_SQ_NO);
KVM_X86_TRANSLATE_FEATURE(TSA_L1_NO);
KVM_X86_TRANSLATE_FEATURE(MSR_IMM);
+ KVM_X86_TRANSLATE_FEATURE(TSA_FILL_NO);
default:
return x86_feature;
}
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 226660973d51..c71f6774a2f3 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -719,6 +719,21 @@ static inline pgprot_t verify_rwx(pgprot_t old, pgprot_t new, unsigned long star
return new;
end = start + npg * PAGE_SIZE - 1;
+
+ /*
+ * If the kernel text is still RWX, gently complain, this could be a
+ * false positive.
+ * Once debug_checkwx() becomes mandatory for CONFIG_STRICT_KERNEL_RWX,
+ * the warning can be removed completely.
+ */
+ if (!kernel_set_to_readonly) {
+ pr_warn_once("CPA detected W^X violation: %016llx -> %016llx range: 0x%016lx - 0x%016lx PFN %lx\n",
+ (unsigned long long)pgprot_val(old),
+ (unsigned long long)pgprot_val(new),
+ start, end, pfn);
+ return new;
+ }
+
WARN_ONCE(1, "CPA detected W^X violation: %016llx -> %016llx range: 0x%016lx - 0x%016lx PFN %lx\n",
(unsigned long long)pgprot_val(old),
(unsigned long long)pgprot_val(new),
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 2745bffcd5ee..7b8af094ab37 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -1956,6 +1956,7 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now)
pr_cont(": active=%u donating=%u after=%u\n",
iocg->hweight_active, iocg->hweight_donating,
iocg->hweight_after_donation);
+ goto out;
}
}
@@ -2054,6 +2055,7 @@ static void transfer_surpluses(struct list_head *surpluses, struct ioc_now *now)
__propagate_weights(iocg, iocg->active, inuse, true, now);
}
+out:
/* walk list should be dissolved after use */
list_for_each_entry_safe(iocg, tiocg, &inner_walk, walk_list)
list_del_init(&iocg->walk_list);
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 025a799f35b9..5a5dc66fc3b1 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1397,6 +1397,7 @@ static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq)
(!blk_queue_nomerges(rq->q) &&
blk_rq_bytes(last) >= BLK_PLUG_FLUSH_SIZE)) {
blk_mq_flush_plug_list(plug, false);
+ blk_plug_invalidate_ts();
last = NULL;
trace_block_plug(rq->q);
}
@@ -4002,6 +4003,7 @@ static int blk_mq_init_hctx(struct request_queue *q,
goto fail;
hctx->queue_num = hctx_idx;
+ clear_bit(BLK_MQ_S_INACTIVE, &hctx->state);
hctx->tags = set->tags[hctx_idx];
diff --git a/block/bsg.c b/block/bsg.c
index 82aaf3cee582..b296fa373292 100644
--- a/block/bsg.c
+++ b/block/bsg.c
@@ -132,7 +132,10 @@ static long bsg_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
case SG_SET_TIMEOUT:
if (get_user(val, intp))
return -EFAULT;
- bd->timeout = clock_t_to_jiffies(val);
+ if (val < 0)
+ return -EIO;
+ bd->timeout = min(clock_t_to_jiffies(val),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
return 0;
case SG_GET_TIMEOUT:
return jiffies_to_clock_t(bd->timeout);
diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c
index 1e61fe6e24b9..291dbd698932 100644
--- a/crypto/algif_skcipher.c
+++ b/crypto/algif_skcipher.c
@@ -135,12 +135,16 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg,
* full block size buffers.
*/
if (ctx->more || len < ctx->used) {
- if (len < bs) {
+ size_t excess = len % bs;
+
+ len -= excess;
+ iov_iter_revert(&msg->msg_iter, excess);
+
+ if (!len) {
err = -EINVAL;
goto free;
}
- len -= len % bs;
cflags |= CRYPTO_SKCIPHER_REQ_NOTFINAL;
}
diff --git a/crypto/ecc.c b/crypto/ecc.c
index 3250a464b852..c7366eb7ce1f 100644
--- a/crypto/ecc.c
+++ b/crypto/ecc.c
@@ -1423,9 +1423,22 @@ static void ecc_point_add(const struct ecc_point *result,
vli_set(result->x, q->x, ndigits);
vli_set(result->y, q->y, ndigits);
vli_mod_sub(z, result->x, p->x, curve->p, ndigits);
+ if (vli_is_zero(z, ndigits)) {
+ if (vli_cmp(p->y, q->y, ndigits)) {
+ /* P + (-P) is the point at infinity. */
+ vli_clear(result->x, ndigits);
+ vli_clear(result->y, ndigits);
+ return;
+ }
+ /* The co-Z addition formula does not handle P == Q. */
+ z[0] = 1;
+ ecc_point_double_jacobian(result->x, result->y, z, curve);
+ goto out;
+ }
vli_set(px, p->x, ndigits);
vli_set(py, p->y, ndigits);
xycz_add(px, py, result->x, result->y, curve);
+out:
vli_mod_inv(z, z, curve->p, ndigits);
apply_z(result->x, result->y, z, curve);
}
@@ -1465,22 +1478,38 @@ void ecc_point_mult_shamir(const struct ecc_point *result,
vli_set(rx, point->x, ndigits);
vli_set(ry, point->y, ndigits);
vli_clear(z + 1, ndigits - 1);
- z[0] = 1;
+ z[0] = !ecc_point_is_zero(point);
for (--i; i >= 0; i--) {
ecc_point_double_jacobian(rx, ry, z, curve);
idx = !!vli_test_bit(u1, i);
idx |= (!!vli_test_bit(u2, i)) << 1;
point = points[idx];
- if (point) {
+ if (point && !ecc_point_is_zero(point)) {
u64 tx[ECC_MAX_DIGITS];
u64 ty[ECC_MAX_DIGITS];
u64 tz[ECC_MAX_DIGITS];
+ if (vli_is_zero(z, ndigits)) {
+ /* Adding to infinity starts a new accumulator. */
+ vli_set(rx, point->x, ndigits);
+ vli_set(ry, point->y, ndigits);
+ z[0] = 1;
+ continue;
+ }
vli_set(tx, point->x, ndigits);
vli_set(ty, point->y, ndigits);
apply_z(tx, ty, z, curve);
vli_mod_sub(tz, rx, tx, curve->p, ndigits);
+ if (vli_is_zero(tz, ndigits)) {
+ if (!vli_cmp(ry, ty, ndigits))
+ ecc_point_double_jacobian(rx, ry, z,
+ curve);
+ else
+ /* Adding opposite points yields infinity. */
+ vli_clear(z, ndigits);
+ continue;
+ }
xycz_add(tx, ty, rx, ry, curve);
vli_mod_mult_fast(z, z, tz, curve);
}
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 4958211fbfa9..9419d61d0602 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -3958,16 +3958,17 @@ static int test_sig_one(struct crypto_sig *tfm, const struct sig_testvec *vecs)
*/
err = crypto_sig_verify(tfm, vecs->c, vecs->c_size,
vecs->m, vecs->m_size);
- if (err) {
- pr_err("alg: sig: verify test failed: err %d\n", err);
- return err;
+ if (err != vecs->verify_error) {
+ pr_err("alg: sig: verify test failed: expected %d, got %d\n",
+ vecs->verify_error, err);
+ return err ?: -EINVAL;
}
/*
* Don't invoke sign test (which requires a private key)
- * for vectors with only a public key.
+ * for vectors with only a public key or an invalid signature.
*/
- if (vecs->public_key_vec)
+ if (vecs->public_key_vec || vecs->verify_error)
return 0;
sig_size = crypto_sig_maxsize(tfm);
diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index c4a15e714ecd..0ee15a8fae27 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -161,6 +161,7 @@ struct sig_testvec {
unsigned int param_len;
unsigned int m_size;
unsigned int c_size;
+ int verify_error;
bool public_key_vec;
enum OID algo;
};
@@ -1443,6 +1444,140 @@ static const struct sig_testvec x962_ecdsa_nist_p192_tv_template[] = {
};
static const struct sig_testvec x962_ecdsa_nist_p256_tv_template[] = {
+ /* Invalid signature: Q = G, r = s, e = 3r mod n. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e"
+ "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51"
+ "\xf5",
+ .key_len = 65,
+ .m =
+ "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15"
+ "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a",
+ .m_size = 32,
+ .c =
+ "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2"
+ "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e"
+ "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e"
+ "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3"
+ "\x10\x60\x9e\x7e\xfa\x58\x39\xd9",
+ .c_size = 72,
+ .verify_error = -EKEYREJECTED,
+ .public_key_vec = true,
+ },
+ /* Invalid signature: Q = -G, r = s, e = 3r mod n. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61"
+ "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae"
+ "\x0a",
+ .key_len = 65,
+ .m =
+ "\xb2\xc3\x09\x72\x03\x17\x9d\xaa\x5c\x4a\xd8\xff\x02\xfe\xcb\x15"
+ "\x21\x53\x81\x77\x45\xec\x45\x24\x3d\x68\x10\xb9\xf2\xa5\x88\x3a",
+ .m_size = 32,
+ .c =
+ "\x30\x46\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e\x74\x18\xf2"
+ "\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3\x10\x60\x9e"
+ "\x7e\xfa\x58\x39\xd9\x02\x21\x00\x90\xeb\xad\xd0\x56\x5d\x34\x8e"
+ "\x74\x18\xf2\xff\xab\xaa\x43\xb1\x9f\x68\xd4\x0c\x4f\x01\x4b\xe3"
+ "\x10\x60\x9e\x7e\xfa\x58\x39\xd9",
+ .c_size = 72,
+ .verify_error = -EKEYREJECTED,
+ .public_key_vec = true,
+ },
+ /* Wycheproof secp256r1/SHA-256 tcId 60. */
+ {
+ .key =
+ "\x04\x29\x27\xb1\x05\x12\xba\xe3\xed\xdc\xfe\x46\x78\x28\x12\x8b"
+ "\xad\x29\x03\x26\x99\x19\xf7\x08\x60\x69\xc8\xc4\xdf\x6c\x73\x28"
+ "\x38\xc7\x78\x79\x64\xea\xac\x00\xe5\x92\x1f\xb1\x49\x8a\x60\xf4"
+ "\x60\x67\x66\xb3\xd9\x68\x50\x01\x55\x8d\x1a\x97\x4e\x73\x41\x51"
+ "\x3e",
+ .key_len = 65,
+ .m =
+ "\x70\x23\x9d\xd8\x77\xf7\xc9\x44\xc4\x22\xf4\x4d\xea\x4e\xd1\xa5"
+ "\x2f\x26\x27\x41\x6f\xaf\x2f\x07\x2f\xa5\x0c\x77\x2e\xd6\xf8\x07",
+ .m_size = 32,
+ .c =
+ "\x30\x44\x02\x20\x64\xa1\xaa\xb5\x00\x0d\x0e\x80\x4f\x3e\x2f\xc0"
+ "\x2b\xde\xe9\xbe\x8f\xf3\x12\x33\x4e\x2b\xa1\x6d\x11\x54\x7c\x97"
+ "\x71\x1c\x89\x8e\x02\x20\x6a\xf0\x15\x97\x1c\xc3\x0b\xe6\xd1\xa2"
+ "\x06\xd4\xe0\x13\xe0\x99\x77\x72\xa2\xf9\x1d\x73\x28\x6f\xfd\x68"
+ "\x3b\x9b\xb2\xcf\x4f\x1b",
+ .c_size = 70,
+ .public_key_vec = true,
+ },
+ /* Wycheproof secp256r1/SHA-256 tcId 210. */
+ {
+ .key =
+ "\x04\xc6\xa7\x71\x52\x70\x24\x22\x77\x92\x17\x0a\x6f\x8e\xee\x73"
+ "\x5b\xf3\x2b\x7f\x98\xaf\x66\x9e\xad\x29\x98\x02\xe3\x2d\x7c\x31"
+ "\x07\xbc\x3b\x4b\x5e\x65\xab\x88\x7b\xbd\x34\x35\x72\xb3\xe5\x61"
+ "\x92\x61\xfe\x3a\x07\x3e\x2f\xfd\x78\x41\x2f\x72\x68\x67\xdb\x58"
+ "\x9e",
+ .key_len = 65,
+ .m =
+ "\xbb\x5a\x52\xf4\x2f\x9c\x92\x61\xed\x43\x61\xf5\x94\x22\xa1\xe3"
+ "\x00\x36\xe7\xc3\x2b\x27\x0c\x88\x07\xa4\x19\xfe\xca\x60\x50\x23",
+ .m_size = 32,
+ .c =
+ "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x21\x00\xb6\xdb\x6d\xb6\x24\x92\x49\x25\x49"
+ "\x24\x92\x49\x24\x92\x49\x24\x62\x5b\xd7\xa0\x9b\xec\x4c\xa8\x1b"
+ "\xcd\xd9\xf8\xfd\x6b\x63\xcc",
+ .c_size = 71,
+ .public_key_vec = true,
+ },
+ /* Q = G, d = 1, k = 2. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\x4f\xe3\x42\xe2\xfe\x1a\x7f\x9b\x8e\xe7\xeb\x4a\x7c\x0f\x9e"
+ "\x16\x2b\xce\x33\x57\x6b\x31\x5e\xce\xcb\xb6\x40\x68\x37\xbf\x51"
+ "\xf5",
+ .key_len = 65,
+ .m =
+ "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10"
+ "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc",
+ .m_size = 32,
+ .c =
+ "\x30\x44\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x20\x59\x47\xc9\x21\x23\x0e\xd1\x34\x0b\x03"
+ "\xe6\xe3\x45\x6e\xab\x69\xf8\x66\xff\x7e\x8c\x7d\x1e\x13\x23\x17"
+ "\x06\x8a\x84\xec\x1d\x22",
+ .c_size = 70,
+ .public_key_vec = true,
+ },
+ /* Q = -G, d = n - 1, k = 2. */
+ {
+ .key =
+ "\x04\x6b\x17\xd1\xf2\xe1\x2c\x42\x47\xf8\xbc\xe6\xe5\x63\xa4\x40"
+ "\xf2\x77\x03\x7d\x81\x2d\xeb\x33\xa0\xf4\xa1\x39\x45\xd8\x98\xc2"
+ "\x96\xb0\x1c\xbd\x1c\x01\xe5\x80\x65\x71\x18\x14\xb5\x83\xf0\x61"
+ "\xe9\xd4\x31\xcc\xa9\x94\xce\xa1\x31\x34\x49\xbf\x97\xc8\x40\xae"
+ "\x0a",
+ .key_len = 65,
+ .m =
+ "\x35\x9d\x17\x29\xb9\x1a\x52\xe9\x8b\xb5\x95\xc3\x86\x28\x3c\x10"
+ "\x30\x44\x95\x1a\xa1\x08\x20\xf0\xa0\x22\xc4\x18\xc2\x71\xa0\xcc",
+ .m_size = 32,
+ .c =
+ "\x30\x45\x02\x20\x7c\xf2\x7b\x18\x8d\x03\x4f\x7e\x8a\x52\x38\x03"
+ "\x04\xb5\x1a\xc3\xc0\x89\x69\xe2\x77\xf2\x1b\x35\xa6\x0b\x48\xfc"
+ "\x47\x66\x99\x78\x02\x21\x00\xdc\x55\x4e\x07\x96\x0b\x81\xb6\x80"
+ "\xb1\xae\xe0\x40\xb9\x90\xa5\xf4\xc4\x90\x49\xbb\xa2\xa1\x62\x70"
+ "\xc5\x88\x51\x39\xe8\xa8\xfb",
+ .c_size = 71,
+ .public_key_vec = true,
+ },
{
.key = /* secp256r1(sha1) */
"\x04\xb9\x7b\xbb\xd7\x17\x64\xd2\x7e\xfc\x81\x5d\x87\x06\x83\x41"
@@ -1827,7 +1962,8 @@ static const struct sig_testvec p1363_ecdsa_nist_p256_tv_template[] = {
};
/*
- * EC-RDSA test vectors are generated by gost-engine.
+ * EC-RDSA test vectors generated by gost-engine, followed by constructed
+ * exceptional-point regression vectors.
*/
static const struct sig_testvec ecrdsa_tv_template[] = {
{
@@ -1973,6 +2109,195 @@ static const struct sig_testvec ecrdsa_tv_template[] = {
.m_size = 64,
.public_key_vec = true,
},
+ /*
+ * Constructed EC-RDSA exceptional-point vectors.
+ * With Q = dG, choose k = z1 + d*z2 (mod n), r = x(kG) mod n,
+ * e = -r/z2 (mod n), and s = r*d + k*e (mod n).
+ * The input digest is e, encoded little-endian.
+ */
+ /* cp256a: Q=G, z1 = 3, z2 = 1. */
+ {
+ .key =
+ "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f"
+ "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4"
+ "\x91\x8d",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x1a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x4a\xed\x44\xae\xd4\x49"
+ "\x32\x67\xe0\x26\x16\xfd\xb7\xaf\xa4\xd7\x3e\x61\xd4\xf9\x7b\x94"
+ "\xf7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7"
+ "\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe7\x06\x3e\x70\x63\xe4\xb7",
+ .c_size = 64,
+ .m =
+ "\xdc\xd3\xfd\x46\xcb\x14\x9d\xe1\x8f\x92\x54\xb2\x0c\xa0\x22\x66"
+ "\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x18\x9c\x8f\xc1\xf9\x08",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ },
+ /* cp256a: Q=-G, z1 = 7, z2 = 6. */
+ {
+ .key =
+ "\x04\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x83\xdf\x60\x61\x63\x36\x53\xdd\x4e\x1c\xdc\x20\xd2\xb0"
+ "\xd6\xca\x89\xd4\xc0\xba\xa5\xaf\x20\xd8\x25\x63\x67\x1f\x8e\x1b"
+ "\x6e\x72",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x2a\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa"
+ "\x92\x10\x2d\x68\x19\x8f\x22\xd5\x60\xeb\x59\xd6\xf3\xe5\x9e\xc2"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",
+ .c_size = 64,
+ .m =
+ "\xc3\x9e\xe5\xf3\xd6\x59\xeb\x60\xd5\x22\x8f\x19\x68\x2d\x10\x92"
+ "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\x2a",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ },
+ /* cp256a: Q=-2G, z1 = 5, z2 = 2. */
+ {
+ .key =
+ "\x04\x40\x95\xfd\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\x14\x1e\x9f\x9e\x9c\xc9\xac\x22\xb1\xe3\x23\xdf\x2d\x4f"
+ "\x29\x35\x76\x2b\x3f\x45\x5a\x50\xdf\x27\xda\x9c\x98\xe0\x71\xe4"
+ "\x91\x8d",
+ .key_len = 66,
+ .params = /* OID_gostCPSignA */
+ "\x30\x13\x06\x07\x2a\x85\x03\x02\x02\x23\x01\x06\x08\x2a\x85\x03"
+ "\x07\x01\x01\x02\x02",
+ .param_len = 21,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xb6\x30\x88\x38\x4c\xad\x68\x80\x22\xc2\x0d\x84\xdb\xb0\xdc\x47"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",
+ .c_size = 64,
+ .m =
+ "\x49\xdc\xb0\xdb\x84\x0d\xc2\x22\x80\x68\xad\x4c\x38\x88\x30\xb6"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 32,
+ .algo = OID_gost2012PKey256,
+ .public_key_vec = true,
+ },
+ /* tc512a: Q=G, z1 = 3, z2 = 1. */
+ {
+ .key =
+ "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\xa4\xf2\x15\x52\xcb\x89\xa5\x89\xb8\xf5\x35\xc2\x5f"
+ "\xfe\x28\x80\xe9\x41\x3a\x0e\xa5\xe6\x75\x3d\xe9\x36\xd0\x4f\xbe"
+ "\x26\x16\xdf\x21\xa9\xef\xcb\xfd\x64\x80\x77\xc1\xab\xf1\xac\x93"
+ "\x1c\x5e\xce\xe6\x50\x54\xe2\x16\x88\x1b\xa6\xe3\x6a\x83\x7a\xe8"
+ "\xcf\x03\x75",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\xd8\xc0\xd7\xe3\xfb\xa6\xca\xff\x6a\xd1\xb4\xb4\x82\x1d\x15\x6f"
+ "\xa7\xc7\xbd\x63\x95\xb8\x29\x0d\xd4\xfa\xb1\x1c\x36\xbd\x32\x8d"
+ "\x1a\xba\xf1\x2e\xbc\x7b\xf2\x02\x96\xf8\xef\x2a\x5e\x72\x64\x86"
+ "\x67\x62\xfb\x62\x8e\x83\x53\x63\xb7\xe6\x78\x3c\x42\xd7\x0d\x1d"
+ "\xb7\xbf\xb8\x09\x56\xc8\x67\x00\x31\xba\x19\x19\x29\xf6\x4e\x30"
+ "\x1d\x68\x16\x34\x23\x6d\x47\xa6\x0e\x57\x1a\x4b\xed\xc0\xef\x25"
+ "\x74\x52\xef\x78\xb5\xb9\x8d\xbb\x3d\x9f\x31\x29\xd9\x34\x94\x33"
+ "\xce\x2a\x3a\x35\xcb\x51\x9c\x91\xe2\xd6\x33\xd7\xb3\x73\xae\x16",
+ .c_size = 128,
+ .m =
+ "\x5f\x04\x9d\x6b\x69\x7d\xf7\xe7\xcb\x1b\x81\x2f\x76\xfe\x20\xcd"
+ "\x2c\xcc\xd0\x74\x63\xfa\x52\x32\x56\xfb\xd3\x3e\xba\xa5\x93\xb3"
+ "\xd9\x10\x3f\x12\xb4\xe5\xa8\xf1\x59\xb8\x92\xdc\xcb\xe9\x97\xe2"
+ "\xcf\xb1\x09\xd6\xe6\xe6\x45\xce\xff\x98\x37\xa9\xf6\x47\x40\x48",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ },
+ /* tc512a: Q=-G, z1 = 7, z2 = 6. */
+ {
+ .key =
+ "\x04\x81\x80\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x23\x0b\xea\xad\x34\x76\x5a\x76\x47\x0a\xca\x3d\xa0"
+ "\x01\xd7\x7f\x16\xbe\xc5\xf1\x5a\x19\x8a\xc2\x16\xc9\x2f\xb0\x41"
+ "\xd9\xe9\x20\xde\x56\x10\x34\x02\x9b\x7f\x88\x3e\x54\x0e\x53\x6c"
+ "\xe3\xa1\x31\x19\xaf\xab\x1d\xe9\x77\xe4\x59\x1c\x95\x7c\x85\x17"
+ "\x30\xfc\x8a",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30"
+ "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x37"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03",
+ .c_size = 128,
+ .m =
+ "\x3a\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d"
+ "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ },
+ /* tc512a: Q=-2G, z1 = 5, z2 = 2. */
+ {
+ .key =
+ "\x04\x81\x80\x7c\x13\xea\xd2\xd9\xaf\x82\xb9\x1f\xb2\xfd\xe1\x8d"
+ "\xf4\x5b\x37\xe4\xf2\x04\x6a\x2a\x1d\x15\x16\x95\x2a\x5d\x40\xcd"
+ "\xbb\x34\x44\x2f\x2b\x13\x0a\x47\xb0\xf6\xe5\xa5\x58\x3a\xf4\x54"
+ "\x09\xb0\x1d\xf5\x5c\xf1\xbf\x9d\x86\xa5\x97\xab\x96\x29\x62\xfc"
+ "\xdc\x89\x3b\x76\x3d\x50\x6e\x8a\xcf\x02\x4c\x74\x0e\x6e\xe0\x1f"
+ "\x28\xab\x27\x9b\x20\x02\xc8\xc4\x12\xcb\x8a\xe3\xa4\x27\xce\x39"
+ "\x62\xdf\x35\x6d\xc1\x65\x7f\x7b\x48\x90\x11\x8f\x19\x7c\x1e\x67"
+ "\x91\x97\xeb\x90\x85\x25\xfc\x86\xb3\x88\x6e\x5c\x57\x1f\x1d\x1d"
+ "\x3b\xec\x37",
+ .key_len = 131,
+ .params = /* OID_gostTC26Sign512A */
+ "\x30\x0b\x06\x09\x2a\x85\x03\x07\x01\x02\x01\x02\x01",
+ .param_len = 13,
+ .c =
+ "\x7f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\x93\xf3\x4a\x99\x7a\x46\xc4\x88\xb7\xf9\x15\xc6\xa7\x02\xb0\x30"
+ "\x4d\xa5\x9c\x55\xfd\x69\x5c\x2e\xe5\x66\xd8\xa0\x8f\x88\x59\x33"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
+ "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03",
+ .c_size = 128,
+ .m =
+ "\x39\x59\x88\x8f\xa0\xd8\x66\xe5\x2e\x5c\x69\xfd\x55\x9c\xa5\x4d"
+ "\x30\xb0\x02\xa7\xc6\x15\xf9\xb7\x88\xc4\x46\x7a\x99\x4a\xf3\x93"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"
+ "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x7f",
+ .m_size = 64,
+ .algo = OID_gost2012PKey512,
+ .public_key_vec = true,
+ },
};
/*
diff --git a/drivers/accel/qaic/sahara.c b/drivers/accel/qaic/sahara.c
index c7c0b3eb4b65..6333c45ef7d2 100644
--- a/drivers/accel/qaic/sahara.c
+++ b/drivers/accel/qaic/sahara.c
@@ -880,6 +880,7 @@ static void sahara_mhi_remove(struct mhi_device *mhi_dev)
cancel_work_sync(&context->fw_work);
cancel_work_sync(&context->dump_work);
+ cancel_work_sync(&context->read_data_work);
vfree(context->mem_dump);
sahara_release_image(context);
mhi_unprepare_from_transfer(mhi_dev);
diff --git a/drivers/accessibility/speakup/selection.c b/drivers/accessibility/speakup/selection.c
index 1713ce4e0ba5..a04a7e8c307b 100644
--- a/drivers/accessibility/speakup/selection.c
+++ b/drivers/accessibility/speakup/selection.c
@@ -61,6 +61,9 @@ static struct speakup_selection_work speakup_sel_work = {
int speakup_set_selection(struct tty_struct *tty)
{
+ if (!tty)
+ return -ENODEV;
+
/* we get kref here first in order to avoid a subtle race when
* cancelling selection work. getting kref first establishes the
* invariant that if speakup_sel_work.tty is not NULL when
@@ -120,6 +123,9 @@ static struct speakup_selection_work speakup_paste_work = {
int speakup_paste_selection(struct tty_struct *tty)
{
+ if (!tty)
+ return -ENODEV;
+
tty_kref_get(tty);
if (cmpxchg(&speakup_paste_work.tty, NULL, tty)) {
tty_kref_put(tty);
diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
index fe10ab0e02f6..ec2d5ca51db0 100644
--- a/drivers/acpi/apei/ghes.c
+++ b/drivers/acpi/apei/ghes.c
@@ -603,7 +603,7 @@ static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata,
* and don't filter out 'corrected' error here.
*/
if (is_cache && has_pa) {
- queued = ghes_do_memory_failure(err_info->physical_fault_addr, flags);
+ queued |= ghes_do_memory_failure(err_info->physical_fault_addr, flags);
p += err_info->length;
continue;
}
@@ -951,11 +951,11 @@ static void ghes_do_proc(struct ghes *ghes,
atomic_notifier_call_chain(&ghes_report_chain, sev, mem_err);
arch_apei_report_mem_error(sev, mem_err);
- queued = ghes_handle_memory_failure(gdata, sev, sync);
+ queued |= ghes_handle_memory_failure(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_PCIE)) {
ghes_handle_aer(gdata);
} else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) {
- queued = ghes_handle_arm_hw_error(gdata, sev, sync);
+ queued |= ghes_handle_arm_hw_error(gdata, sev, sync);
} else if (guid_equal(sec_type, &CPER_SEC_CXL_PROT_ERR)) {
struct cxl_cper_sec_prot_err *prot_err = acpi_hest_get_payload(gdata);
diff --git a/drivers/acpi/pfr_update.c b/drivers/acpi/pfr_update.c
index 9afd2c52fdbd..98ace679601b 100644
--- a/drivers/acpi/pfr_update.c
+++ b/drivers/acpi/pfr_update.c
@@ -422,7 +422,7 @@ static int start_update(int action, struct pfru_device *pfru_dev)
static long pfru_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
- struct pfru_update_cap_info cap_hdr;
+ struct pfru_update_cap_info cap_hdr = {};
struct pfru_device *pfru_dev = to_pfru_dev(file);
void __user *p = (void __user *)arg;
u32 rev;
diff --git a/drivers/acpi/riscv/cppc.c b/drivers/acpi/riscv/cppc.c
index 42c1a9052470..4ea4ddedd91f 100644
--- a/drivers/acpi/riscv/cppc.c
+++ b/drivers/acpi/riscv/cppc.c
@@ -97,6 +97,7 @@ bool cpc_ffh_supported(void)
int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -107,19 +108,27 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
- smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return sbi_err_map_linux_errno(data.ret.error);
*val = data.ret.value;
- return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
+ return 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
- smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_read, &data, 1);
+ if (ret)
+ return ret;
+ if (data.ret.error)
+ return data.ret.error;
*val = data.ret.value;
- return data.ret.error;
+ return 0;
}
return -EINVAL;
@@ -128,6 +137,7 @@ int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val)
int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
{
struct sbi_cppc_data data;
+ int ret;
if (WARN_ON_ONCE(irqs_disabled()))
return -EPERM;
@@ -139,14 +149,18 @@ int cpc_write_ffh(int cpu, struct cpc_reg *reg, u64 val)
data.reg = FFH_CPPC_SBI_REG(reg->address);
data.val = val;
- smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ ret = smp_call_function_single(cpu, sbi_cppc_write, &data, 1);
+ if (ret)
+ return ret;
return (data.ret.error) ? sbi_err_map_linux_errno(data.ret.error) : 0;
} else if (FFH_CPPC_TYPE(reg->address) == FFH_CPPC_CSR) {
data.reg = FFH_CPPC_CSR_NUM(reg->address);
data.val = val;
- smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ ret = smp_call_function_single(cpu, cppc_ffh_csr_write, &data, 1);
+ if (ret)
+ return ret;
return data.ret.error;
}
diff --git a/drivers/acpi/sbs.c b/drivers/acpi/sbs.c
index 86b7c7975852..4b84b3cd11e3 100644
--- a/drivers/acpi/sbs.c
+++ b/drivers/acpi/sbs.c
@@ -639,10 +639,8 @@ static int acpi_sbs_probe(struct platform_device *pdev)
return -ENODEV;
sbs = kzalloc_obj(struct acpi_sbs);
- if (!sbs) {
- result = -ENOMEM;
- goto end;
- }
+ if (!sbs)
+ return -ENOMEM;
mutex_init(&sbs->lock);
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 28ca856ecc75..8ef9398b704f 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -328,13 +328,6 @@ config SATA_AHCI_SEATTLE
If unsure, say N
-config SATA_INIC162X
- tristate "Initio 162x SATA support (Very Experimental)"
- depends on PCI
- select SATA_HOST
- help
- This option enables support for Initio 162x Serial ATA.
-
config SATA_ACARD_AHCI
tristate "ACard AHCI variant (ATP 8620)"
depends on PCI
diff --git a/drivers/ata/Makefile b/drivers/ata/Makefile
index b96025abd45e..55b89c09dd74 100644
--- a/drivers/ata/Makefile
+++ b/drivers/ata/Makefile
@@ -11,7 +11,6 @@ obj-$(CONFIG_SATA_AHCI_SEATTLE) += ahci_seattle.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_AHCI_PLATFORM) += ahci_platform.o libahci.o libahci_platform.o
obj-$(CONFIG_SATA_FSL) += sata_fsl.o
obj-$(CONFIG_SATA_GEMINI) += sata_gemini.o
-obj-$(CONFIG_SATA_INIC162X) += sata_inic162x.o
obj-$(CONFIG_SATA_SIL24) += sata_sil24.o
obj-$(CONFIG_SATA_DWC) += sata_dwc_460ex.o
obj-$(CONFIG_SATA_HIGHBANK) += sata_highbank.o libahci.o
diff --git a/drivers/ata/ahci_st.c b/drivers/ata/ahci_st.c
index 4336c8a6e208..eea75961038c 100644
--- a/drivers/ata/ahci_st.c
+++ b/drivers/ata/ahci_st.c
@@ -83,18 +83,27 @@ static int st_ahci_deassert_resets(struct ahci_host_priv *hpriv,
return 0;
}
-static void st_ahci_host_stop(struct ata_host *host)
+static int st_ahci_assert_pwrdwn(struct ahci_host_priv *hpriv,
+ struct device *dev)
{
- struct ahci_host_priv *hpriv = host->private_data;
struct st_ahci_drv_data *drv_data = hpriv->plat_data;
- struct device *dev = host->dev;
int err;
- if (drv_data->pwr) {
- err = reset_control_assert(drv_data->pwr);
- if (err)
- dev_err(dev, "unable to pwrdwn\n");
- }
+ if (!drv_data->pwr)
+ return 0;
+
+ err = reset_control_assert(drv_data->pwr);
+ if (err)
+ dev_err(dev, "unable to pwrdwn\n");
+
+ return err;
+}
+
+static void st_ahci_host_stop(struct ata_host *host)
+{
+ struct ahci_host_priv *hpriv = host->private_data;
+
+ st_ahci_assert_pwrdwn(hpriv, host->dev);
ahci_platform_disable_resources(hpriv);
}
@@ -104,23 +113,20 @@ static int st_ahci_probe_resets(struct ahci_host_priv *hpriv,
{
struct st_ahci_drv_data *drv_data = hpriv->plat_data;
- drv_data->pwr = devm_reset_control_get(dev, "pwr-dwn");
- if (IS_ERR(drv_data->pwr)) {
- dev_info(dev, "power reset control not defined\n");
- drv_data->pwr = NULL;
- }
+ drv_data->pwr = devm_reset_control_get_optional(dev, "pwr-dwn");
+ if (IS_ERR(drv_data->pwr))
+ return dev_err_probe(dev, PTR_ERR(drv_data->pwr),
+ "failed to get pwr-dwn reset\n");
- drv_data->sw_rst = devm_reset_control_get(dev, "sw-rst");
- if (IS_ERR(drv_data->sw_rst)) {
- dev_info(dev, "soft reset control not defined\n");
- drv_data->sw_rst = NULL;
- }
+ drv_data->sw_rst = devm_reset_control_get_optional(dev, "sw-rst");
+ if (IS_ERR(drv_data->sw_rst))
+ return dev_err_probe(dev, PTR_ERR(drv_data->sw_rst),
+ "failed to get sw-rst reset\n");
- drv_data->pwr_rst = devm_reset_control_get(dev, "pwr-rst");
- if (IS_ERR(drv_data->pwr_rst)) {
- dev_dbg(dev, "power soft reset control not defined\n");
- drv_data->pwr_rst = NULL;
- }
+ drv_data->pwr_rst = devm_reset_control_get_optional(dev, "pwr-rst");
+ if (IS_ERR(drv_data->pwr_rst))
+ return dev_err_probe(dev, PTR_ERR(drv_data->pwr_rst),
+ "failed to get pwr-rst reset\n");
return st_ahci_deassert_resets(hpriv, dev);
}
@@ -162,38 +168,38 @@ static int st_ahci_probe(struct platform_device *pdev)
err = ahci_platform_enable_resources(hpriv);
if (err)
- return err;
+ goto assert_pwrdwn;
st_ahci_configure_oob(hpriv->mmio);
err = ahci_platform_init_host(pdev, hpriv, &st_ahci_port_info,
&ahci_platform_sht);
- if (err) {
- ahci_platform_disable_resources(hpriv);
- return err;
- }
+ if (err)
+ goto disable_resources;
return 0;
+
+disable_resources:
+ ahci_platform_disable_resources(hpriv);
+assert_pwrdwn:
+ st_ahci_assert_pwrdwn(hpriv, &pdev->dev);
+
+ return err;
}
static int st_ahci_suspend(struct device *dev)
{
struct ata_host *host = dev_get_drvdata(dev);
struct ahci_host_priv *hpriv = host->private_data;
- struct st_ahci_drv_data *drv_data = hpriv->plat_data;
int err;
err = ahci_platform_suspend_host(dev);
if (err)
return err;
- if (drv_data->pwr) {
- err = reset_control_assert(drv_data->pwr);
- if (err) {
- dev_err(dev, "unable to pwrdwn");
- return err;
- }
- }
+ err = st_ahci_assert_pwrdwn(hpriv, dev);
+ if (err)
+ return err;
ahci_platform_disable_resources(hpriv);
diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index 08ab56e95566..f3fd56f15334 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -2736,11 +2736,35 @@ static int ahci_host_activate_multi_irqs(struct ata_host *host,
0, pp->irq_desc, host->ports[i]);
if (rc)
- return rc;
+ goto free_irqs;
ata_port_desc_misc(host->ports[i], irq);
}
- return ata_host_register(host, sht);
+ rc = ata_host_register(host, sht);
+ if (rc)
+ goto free_irqs;
+
+ return 0;
+
+free_irqs:
+ /*
+ * Free the IRQs which have been requested, so that the handlers can no
+ * longer access the MMIO of the host once we return, e.g. after the
+ * caller has disabled the clocks of the host.
+ */
+ while (--i >= 0) {
+ struct ahci_port_priv *pp = host->ports[i]->private_data;
+
+ if (!pp)
+ continue;
+
+ devm_free_irq(host->dev, hpriv->get_irq_vector(host, i),
+ host->ports[i]);
+ }
+
+ ata_host_undo_start(host);
+
+ return rc;
}
/**
diff --git a/drivers/ata/libahci_platform.c b/drivers/ata/libahci_platform.c
index f6524135ea11..ae342080556a 100644
--- a/drivers/ata/libahci_platform.c
+++ b/drivers/ata/libahci_platform.c
@@ -689,6 +689,10 @@ EXPORT_SYMBOL_GPL(ahci_platform_get_resources);
* ahci-platform host, note any necessary resources (ie clks, phys, etc.)
* must be initialized / enabled before calling this.
*
+ * On failure, ->host_stop() is not called, so the caller has to release the
+ * resources it enabled (clocks, regulators, resets, PHYs) in its probe()
+ * error path.
+ *
* RETURNS:
* 0 on success otherwise a negative error code
*/
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 92233fb25051..f218773f187f 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -6128,6 +6128,46 @@ int ata_host_start(struct ata_host *host)
}
EXPORT_SYMBOL_GPL(ata_host_start);
+/**
+ * ata_host_undo_start - undo ata_host_start()
+ * @host: ATA host to undo_start
+ *
+ * Stop the ports of @host and drop the devres action registered by
+ * ata_host_start(), without calling ->host_stop(). Nothing is done if
+ * @host has not been started.
+ *
+ * This gives the release of the host resources back to the caller, which
+ * is what a driver whose probe() error path releases those resources
+ * itself needs when starting or activating the host fails.
+ *
+ * LOCKING:
+ * Inherited from calling layer (may sleep).
+ */
+void ata_host_undo_start(struct ata_host *host)
+{
+ int i;
+
+ if (!(host->flags & ATA_HOST_STARTED))
+ return;
+
+ for (i = 0; i < host->n_ports; i++) {
+ struct ata_port *ap = host->ports[i];
+
+ if (ap->ops->port_stop)
+ ap->ops->port_stop(ap);
+ }
+
+ /*
+ * Drop the action added by ata_host_start() without calling it.
+ * It does not exist if neither ->port_stop() nor ->host_stop() is
+ * implemented, in which case there is nothing to drop.
+ */
+ devres_destroy(host->dev, ata_host_stop, NULL, NULL);
+
+ host->flags &= ~ATA_HOST_STARTED;
+}
+EXPORT_SYMBOL_GPL(ata_host_undo_start);
+
/**
* ata_host_init - Initialize a host struct for sas (ipr, libsas)
* @host: host to initialize
@@ -6202,6 +6242,12 @@ static void async_port_probe(void *data, async_cookie_t cookie)
* starts ports, registers @host with ATA and SCSI layers and
* probe registered devices.
*
+ * On failure, the host remains started, i.e. the devres action
+ * registered by ata_host_start() is kept, so ->host_stop() is called by
+ * the driver core when probe() fails. A caller which releases the host
+ * resources in its own probe() error path must therefore call
+ * ata_host_undo_start().
+ *
* LOCKING:
* Inherited from calling layer (may sleep).
*
@@ -6295,6 +6341,10 @@ EXPORT_SYMBOL_GPL(ata_host_register);
* have set polling mode on the port. In this case, @irq_handler
* should be NULL.
*
+ * On failure, the ports are stopped again and the devres action
+ * registered by ata_host_start() is dropped without calling
+ * ->host_stop(), so releasing the host resources is left to the caller.
+ *
* LOCKING:
* Inherited from calling layer (may sleep).
*
@@ -6315,27 +6365,40 @@ int ata_host_activate(struct ata_host *host, int irq,
/* Special case for polling mode */
if (!irq) {
WARN_ON(irq_handler);
- return ata_host_register(host, sht);
+ rc = ata_host_register(host, sht);
+ if (rc)
+ goto undo_start;
+
+ return 0;
}
irq_desc = devm_kasprintf(host->dev, GFP_KERNEL, "%s[%s]",
dev_driver_string(host->dev),
dev_name(host->dev));
- if (!irq_desc)
- return -ENOMEM;
+ if (!irq_desc) {
+ rc = -ENOMEM;
+ goto undo_start;
+ }
rc = devm_request_irq(host->dev, irq, irq_handler, irq_flags,
irq_desc, host);
if (rc)
- return rc;
+ goto undo_start;
for (i = 0; i < host->n_ports; i++)
ata_port_desc_misc(host->ports[i], irq);
rc = ata_host_register(host, sht);
- /* if failed, just free the IRQ and leave ports alone */
- if (rc)
+ if (rc) {
+ /* Free the IRQ, so that the handler can no longer run */
devm_free_irq(host->dev, irq, host);
+ goto undo_start;
+ }
+
+ return 0;
+
+undo_start:
+ ata_host_undo_start(host);
return rc;
}
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index fb5517d51095..b86ac50f16b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -3517,6 +3517,11 @@ static unsigned int ata_scsi_pass_thru(struct ata_queued_cmd *qc)
if (is_multi_taskfile(tf)) {
unsigned int multi_count = 1 << (cdb[1] >> 5);
+ if (!dev->multi_count) {
+ ata_scsi_set_sense(dev, scmd, ABORTED_COMMAND, 0, 0);
+ return 1;
+ }
+
/* compare the passed through multi_count
* with the cached multi_count of libata
*/
diff --git a/drivers/ata/libata-sff.c b/drivers/ata/libata-sff.c
index 976e4e160494..c2aad525010c 100644
--- a/drivers/ata/libata-sff.c
+++ b/drivers/ata/libata-sff.c
@@ -1196,8 +1196,6 @@ void ata_sff_flush_pio_task(struct ata_port *ap)
{
trace_ata_sff_flush_pio_task(ap);
- cancel_delayed_work_sync(&ap->sff_pio_task);
-
/*
* We wanna reset the HSM state to IDLE. If we do so without
* grabbing the port lock, critical sections protected by it which
@@ -1205,11 +1203,26 @@ void ata_sff_flush_pio_task(struct ata_port *ap)
* example, we may set IDLE in between the time
* __ata_sff_port_intr() checks for HSM_ST_IDLE and before it calls
* ata_sff_hsm_move() causing ata_sff_hsm_move() to BUG().
+ *
+ * Reset the state before cancelling the work item: once the state
+ * is IDLE, __ata_sff_port_intr() can no longer enter
+ * ata_sff_hsm_move(), which prevents the interrupt handler from
+ * queueing sff_pio_task when deferring the completion of a PIO
+ * data-in command. The SCSI error handler cannot dispatch new
+ * commands while error handling is in progress, so issue-time
+ * queuing is not possible either.
*/
spin_lock_irq(ap->lock);
ap->hsm_task_state = HSM_ST_IDLE;
spin_unlock_irq(ap->lock);
+ /*
+ * Now that the interrupt handler can no longer queue the pio task,
+ * cancel it. This also drains a work item that the interrupt
+ * handler managed to queue before the HSM state was reset.
+ */
+ cancel_delayed_work_sync(&ap->sff_pio_task);
+
ap->sff_pio_task_link = NULL;
}
@@ -1233,7 +1246,13 @@ static void ata_sff_pio_task(struct work_struct *work)
}
fsm_start:
- WARN_ON_ONCE(ap->hsm_task_state == HSM_ST_IDLE);
+ /*
+ * ata_sff_flush_pio_task() may have marked the HSM idle after this
+ * work item was queued or while it was running. There is nothing
+ * left to do.
+ */
+ if (ap->hsm_task_state == HSM_ST_IDLE)
+ goto out_unlock;
wait_mask = ATA_BUSY;
if (ap->hsm_task_state == HSM_ST_LAST)
@@ -1252,6 +1271,13 @@ static void ata_sff_pio_task(struct work_struct *work)
ata_msleep(ap, 2);
spin_lock_irq(ap->lock);
+ /*
+ * ata_sff_flush_pio_task() may have marked the HSM idle
+ * while the port lock was dropped for ata_msleep().
+ */
+ if (ap->hsm_task_state == HSM_ST_IDLE)
+ goto out_unlock;
+
status = ata_sff_busy_wait(ap, wait_mask, 10);
if (status & wait_mask) {
ata_sff_queue_pio_task(link, ATA_SHORT_PAUSE);
@@ -2267,6 +2293,11 @@ EXPORT_SYMBOL_GPL(ata_pci_sff_prepare_host);
* hosts. This separate helper is necessary because SFF hosts
* use two separate interrupts in legacy mode.
*
+ * Note that, unlike ata_host_activate(), the devres action registered
+ * by ata_host_start() is kept on failure, i.e. ->host_stop() is called
+ * by the driver core when probe() fails. All callers of this function
+ * rely on that, none of them releases the host resources itself.
+ *
* LOCKING:
* Inherited from calling layer (may sleep).
*
diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c
index c6df55886fe1..987d8acf1486 100644
--- a/drivers/ata/sata_fsl.c
+++ b/drivers/ata/sata_fsl.c
@@ -1490,8 +1490,10 @@ static int sata_fsl_probe(struct platform_device *ofdev)
* device discovery process, invoking our port_start() handler &
* error_handler() to execute a dummy Softreset EH session
*/
- ata_host_activate(host, irq, sata_fsl_interrupt, SATA_FSL_IRQ_FLAG,
- &sata_fsl_sht);
+ retval = ata_host_activate(host, irq, sata_fsl_interrupt,
+ SATA_FSL_IRQ_FLAG, &sata_fsl_sht);
+ if (retval)
+ goto error_exit_with_cleanup;
host_priv->intr_coalescing.show = fsl_sata_intr_coalescing_show;
host_priv->intr_coalescing.store = fsl_sata_intr_coalescing_store;
diff --git a/drivers/ata/sata_inic162x.c b/drivers/ata/sata_inic162x.c
deleted file mode 100644
index 065408975f5b..000000000000
--- a/drivers/ata/sata_inic162x.c
+++ /dev/null
@@ -1,903 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * sata_inic162x.c - Driver for Initio 162x SATA controllers
- *
- * Copyright 2006 SUSE Linux Products GmbH
- * Copyright 2006 Tejun Heo <teheo@novell.com>
- *
- * **** WARNING ****
- *
- * This driver never worked properly and unfortunately data corruption is
- * relatively common. There isn't anyone working on the driver and there's
- * no support from the vendor. Do not use this driver in any production
- * environment.
- *
- * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491
- * https://bugzilla.kernel.org/show_bug.cgi?id=60565
- *
- * *****************
- *
- * This controller is eccentric and easily locks up if something isn't
- * right. Documentation is available at initio's website but it only
- * documents registers (not programming model).
- *
- * This driver has interesting history. The first version was written
- * from the documentation and a 2.4 IDE driver posted on a Taiwan
- * company, which didn't use any IDMA features and couldn't handle
- * LBA48. The resulting driver couldn't handle LBA48 devices either
- * making it pretty useless.
- *
- * After a while, initio picked the driver up, renamed it to
- * sata_initio162x, updated it to use IDMA for ATA DMA commands and
- * posted it on their website. It only used ATA_PROT_DMA for IDMA and
- * attaching both devices and issuing IDMA and !IDMA commands
- * simultaneously broke it due to PIRQ masking interaction but it did
- * show how to use the IDMA (ADMA + some initio specific twists)
- * engine.
- *
- * Then, I picked up their changes again and here's the usable driver
- * which uses IDMA for everything. Everything works now including
- * LBA48, CD/DVD burning, suspend/resume and hotplug. There are some
- * issues tho. Result Tf is not resported properly, NCQ isn't
- * supported yet and CD/DVD writing works with DMA assisted PIO
- * protocol (which, for native SATA devices, shouldn't cause any
- * noticeable difference).
- *
- * Anyways, so, here's finally a working driver for inic162x. Enjoy!
- *
- * initio: If you guys wanna improve the driver regarding result TF
- * access and other stuff, please feel free to contact me. I'll be
- * happy to assist.
- */
-
-#include <linux/gfp.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/pci.h>
-#include <scsi/scsi_host.h>
-#include <linux/libata.h>
-#include <linux/blkdev.h>
-#include <scsi/scsi_device.h>
-
-#define DRV_NAME "sata_inic162x"
-#define DRV_VERSION "0.4"
-
-enum {
- MMIO_BAR_PCI = 5,
- MMIO_BAR_CARDBUS = 1,
-
- NR_PORTS = 2,
-
- IDMA_CPB_TBL_SIZE = 4 * 32,
-
- INIC_DMA_BOUNDARY = 0xffffff,
-
- HOST_ACTRL = 0x08,
- HOST_CTL = 0x7c,
- HOST_STAT = 0x7e,
- HOST_IRQ_STAT = 0xbc,
- HOST_IRQ_MASK = 0xbe,
-
- PORT_SIZE = 0x40,
-
- /* registers for ATA TF operation */
- PORT_TF_DATA = 0x00,
- PORT_TF_FEATURE = 0x01,
- PORT_TF_NSECT = 0x02,
- PORT_TF_LBAL = 0x03,
- PORT_TF_LBAM = 0x04,
- PORT_TF_LBAH = 0x05,
- PORT_TF_DEVICE = 0x06,
- PORT_TF_COMMAND = 0x07,
- PORT_TF_ALT_STAT = 0x08,
- PORT_IRQ_STAT = 0x09,
- PORT_IRQ_MASK = 0x0a,
- PORT_PRD_CTL = 0x0b,
- PORT_PRD_ADDR = 0x0c,
- PORT_PRD_XFERLEN = 0x10,
- PORT_CPB_CPBLAR = 0x18,
- PORT_CPB_PTQFIFO = 0x1c,
-
- /* IDMA register */
- PORT_IDMA_CTL = 0x14,
- PORT_IDMA_STAT = 0x16,
-
- PORT_RPQ_FIFO = 0x1e,
- PORT_RPQ_CNT = 0x1f,
-
- PORT_SCR = 0x20,
-
- /* HOST_CTL bits */
- HCTL_LEDEN = (1 << 3), /* enable LED operation */
- HCTL_IRQOFF = (1 << 8), /* global IRQ off */
- HCTL_FTHD0 = (1 << 10), /* fifo threshold 0 */
- HCTL_FTHD1 = (1 << 11), /* fifo threshold 1*/
- HCTL_PWRDWN = (1 << 12), /* power down PHYs */
- HCTL_SOFTRST = (1 << 13), /* global reset (no phy reset) */
- HCTL_RPGSEL = (1 << 15), /* register page select */
-
- HCTL_KNOWN_BITS = HCTL_IRQOFF | HCTL_PWRDWN | HCTL_SOFTRST |
- HCTL_RPGSEL,
-
- /* HOST_IRQ_(STAT|MASK) bits */
- HIRQ_PORT0 = (1 << 0),
- HIRQ_PORT1 = (1 << 1),
- HIRQ_SOFT = (1 << 14),
- HIRQ_GLOBAL = (1 << 15), /* STAT only */
-
- /* PORT_IRQ_(STAT|MASK) bits */
- PIRQ_OFFLINE = (1 << 0), /* device unplugged */
- PIRQ_ONLINE = (1 << 1), /* device plugged */
- PIRQ_COMPLETE = (1 << 2), /* completion interrupt */
- PIRQ_FATAL = (1 << 3), /* fatal error */
- PIRQ_ATA = (1 << 4), /* ATA interrupt */
- PIRQ_REPLY = (1 << 5), /* reply FIFO not empty */
- PIRQ_PENDING = (1 << 7), /* port IRQ pending (STAT only) */
-
- PIRQ_ERR = PIRQ_OFFLINE | PIRQ_ONLINE | PIRQ_FATAL,
- PIRQ_MASK_DEFAULT = PIRQ_REPLY | PIRQ_ATA,
- PIRQ_MASK_FREEZE = 0xff,
-
- /* PORT_PRD_CTL bits */
- PRD_CTL_START = (1 << 0),
- PRD_CTL_WR = (1 << 3),
- PRD_CTL_DMAEN = (1 << 7), /* DMA enable */
-
- /* PORT_IDMA_CTL bits */
- IDMA_CTL_RST_ATA = (1 << 2), /* hardreset ATA bus */
- IDMA_CTL_RST_IDMA = (1 << 5), /* reset IDMA machinery */
- IDMA_CTL_GO = (1 << 7), /* IDMA mode go */
- IDMA_CTL_ATA_NIEN = (1 << 8), /* ATA IRQ disable */
-
- /* PORT_IDMA_STAT bits */
- IDMA_STAT_PERR = (1 << 0), /* PCI ERROR MODE */
- IDMA_STAT_CPBERR = (1 << 1), /* ADMA CPB error */
- IDMA_STAT_LGCY = (1 << 3), /* ADMA legacy */
- IDMA_STAT_UIRQ = (1 << 4), /* ADMA unsolicited irq */
- IDMA_STAT_STPD = (1 << 5), /* ADMA stopped */
- IDMA_STAT_PSD = (1 << 6), /* ADMA pause */
- IDMA_STAT_DONE = (1 << 7), /* ADMA done */
-
- IDMA_STAT_ERR = IDMA_STAT_PERR | IDMA_STAT_CPBERR,
-
- /* CPB Control Flags*/
- CPB_CTL_VALID = (1 << 0), /* CPB valid */
- CPB_CTL_QUEUED = (1 << 1), /* queued command */
- CPB_CTL_DATA = (1 << 2), /* data, rsvd in datasheet */
- CPB_CTL_IEN = (1 << 3), /* PCI interrupt enable */
- CPB_CTL_DEVDIR = (1 << 4), /* device direction control */
-
- /* CPB Response Flags */
- CPB_RESP_DONE = (1 << 0), /* ATA command complete */
- CPB_RESP_REL = (1 << 1), /* ATA release */
- CPB_RESP_IGNORED = (1 << 2), /* CPB ignored */
- CPB_RESP_ATA_ERR = (1 << 3), /* ATA command error */
- CPB_RESP_SPURIOUS = (1 << 4), /* ATA spurious interrupt error */
- CPB_RESP_UNDERFLOW = (1 << 5), /* APRD deficiency length error */
- CPB_RESP_OVERFLOW = (1 << 6), /* APRD exccess length error */
- CPB_RESP_CPB_ERR = (1 << 7), /* CPB error flag */
-
- /* PRD Control Flags */
- PRD_DRAIN = (1 << 1), /* ignore data excess */
- PRD_CDB = (1 << 2), /* atapi packet command pointer */
- PRD_DIRECT_INTR = (1 << 3), /* direct interrupt */
- PRD_DMA = (1 << 4), /* data transfer method */
- PRD_WRITE = (1 << 5), /* data dir, rsvd in datasheet */
- PRD_IOM = (1 << 6), /* io/memory transfer */
- PRD_END = (1 << 7), /* APRD chain end */
-};
-
-/* Comman Parameter Block */
-struct inic_cpb {
- u8 resp_flags; /* Response Flags */
- u8 error; /* ATA Error */
- u8 status; /* ATA Status */
- u8 ctl_flags; /* Control Flags */
- __le32 len; /* Total Transfer Length */
- __le32 prd; /* First PRD pointer */
- u8 rsvd[4];
- /* 16 bytes */
- u8 feature; /* ATA Feature */
- u8 hob_feature; /* ATA Ex. Feature */
- u8 device; /* ATA Device/Head */
- u8 mirctl; /* Mirror Control */
- u8 nsect; /* ATA Sector Count */
- u8 hob_nsect; /* ATA Ex. Sector Count */
- u8 lbal; /* ATA Sector Number */
- u8 hob_lbal; /* ATA Ex. Sector Number */
- u8 lbam; /* ATA Cylinder Low */
- u8 hob_lbam; /* ATA Ex. Cylinder Low */
- u8 lbah; /* ATA Cylinder High */
- u8 hob_lbah; /* ATA Ex. Cylinder High */
- u8 command; /* ATA Command */
- u8 ctl; /* ATA Control */
- u8 slave_error; /* Slave ATA Error */
- u8 slave_status; /* Slave ATA Status */
- /* 32 bytes */
-} __packed;
-
-/* Physical Region Descriptor */
-struct inic_prd {
- __le32 mad; /* Physical Memory Address */
- __le16 len; /* Transfer Length */
- u8 rsvd;
- u8 flags; /* Control Flags */
-} __packed;
-
-struct inic_pkt {
- struct inic_cpb cpb;
- struct inic_prd prd[LIBATA_MAX_PRD + 1]; /* + 1 for cdb */
- u8 cdb[ATAPI_CDB_LEN];
-} __packed;
-
-struct inic_host_priv {
- void __iomem *mmio_base;
- u16 cached_hctl;
-};
-
-struct inic_port_priv {
- struct inic_pkt *pkt;
- dma_addr_t pkt_dma;
- u32 *cpb_tbl;
- dma_addr_t cpb_tbl_dma;
-};
-
-static const struct scsi_host_template inic_sht = {
- ATA_BASE_SHT(DRV_NAME),
- .sg_tablesize = LIBATA_MAX_PRD, /* maybe it can be larger? */
-
- /*
- * This controller is braindamaged. dma_boundary is 0xffff like others
- * but it will lock up the whole machine HARD if 65536 byte PRD entry
- * is fed. Reduce maximum segment size.
- */
- .dma_boundary = INIC_DMA_BOUNDARY,
- .max_segment_size = 65536 - 512,
-};
-
-static const int scr_map[] = {
- [SCR_STATUS] = 0,
- [SCR_ERROR] = 1,
- [SCR_CONTROL] = 2,
-};
-
-static void __iomem *inic_port_base(struct ata_port *ap)
-{
- struct inic_host_priv *hpriv = ap->host->private_data;
-
- return hpriv->mmio_base + ap->port_no * PORT_SIZE;
-}
-
-static void inic_reset_port(void __iomem *port_base)
-{
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
-
- /* stop IDMA engine */
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* mask IRQ and assert reset */
- writew(IDMA_CTL_RST_IDMA, idma_ctl);
- readw(idma_ctl); /* flush */
- msleep(1);
-
- /* release reset */
- writew(0, idma_ctl);
-
- /* clear irq */
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static int inic_scr_read(struct ata_link *link, unsigned sc_reg, u32 *val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- *val = readl(scr_addr + scr_map[sc_reg] * 4);
-
- /* this controller has stuck DIAG.N, ignore it */
- if (sc_reg == SCR_ERROR)
- *val &= ~SERR_PHYRDY_CHG;
- return 0;
-}
-
-static int inic_scr_write(struct ata_link *link, unsigned sc_reg, u32 val)
-{
- void __iomem *scr_addr = inic_port_base(link->ap) + PORT_SCR;
-
- if (unlikely(sc_reg >= ARRAY_SIZE(scr_map)))
- return -EINVAL;
-
- writel(val, scr_addr + scr_map[sc_reg] * 4);
- return 0;
-}
-
-static void inic_stop_idma(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- readb(port_base + PORT_RPQ_FIFO);
- readb(port_base + PORT_RPQ_CNT);
- writew(0, port_base + PORT_IDMA_CTL);
-}
-
-static void inic_host_err_intr(struct ata_port *ap, u8 irq_stat, u16 idma_stat)
-{
- struct ata_eh_info *ehi = &ap->link.eh_info;
- struct inic_port_priv *pp = ap->private_data;
- struct inic_cpb *cpb = &pp->pkt->cpb;
- bool freeze = false;
-
- ata_ehi_clear_desc(ehi);
- ata_ehi_push_desc(ehi, "irq_stat=0x%x idma_stat=0x%x",
- irq_stat, idma_stat);
-
- inic_stop_idma(ap);
-
- if (irq_stat & (PIRQ_OFFLINE | PIRQ_ONLINE)) {
- ata_ehi_push_desc(ehi, "hotplug");
- ata_ehi_hotplugged(ehi);
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_PERR) {
- ata_ehi_push_desc(ehi, "PCI error");
- freeze = true;
- }
-
- if (idma_stat & IDMA_STAT_CPBERR) {
- ata_ehi_push_desc(ehi, "CPB error");
-
- if (cpb->resp_flags & CPB_RESP_IGNORED) {
- __ata_ehi_push_desc(ehi, " ignored");
- ehi->err_mask |= AC_ERR_INVALID;
- freeze = true;
- }
-
- if (cpb->resp_flags & CPB_RESP_ATA_ERR)
- ehi->err_mask |= AC_ERR_DEV;
-
- if (cpb->resp_flags & CPB_RESP_SPURIOUS) {
- __ata_ehi_push_desc(ehi, " spurious-intr");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
-
- if (cpb->resp_flags &
- (CPB_RESP_UNDERFLOW | CPB_RESP_OVERFLOW)) {
- __ata_ehi_push_desc(ehi, " data-over/underflow");
- ehi->err_mask |= AC_ERR_HSM;
- freeze = true;
- }
- }
-
- if (freeze)
- ata_port_freeze(ap);
- else
- ata_port_abort(ap);
-}
-
-static void inic_host_intr(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct ata_queued_cmd *qc = ata_qc_from_tag(ap, ap->link.active_tag);
- u8 irq_stat;
- u16 idma_stat;
-
- /* read and clear IRQ status */
- irq_stat = readb(port_base + PORT_IRQ_STAT);
- writeb(irq_stat, port_base + PORT_IRQ_STAT);
- idma_stat = readw(port_base + PORT_IDMA_STAT);
-
- if (unlikely((irq_stat & PIRQ_ERR) || (idma_stat & IDMA_STAT_ERR)))
- inic_host_err_intr(ap, irq_stat, idma_stat);
-
- if (unlikely(!qc))
- goto spurious;
-
- if (likely(idma_stat & IDMA_STAT_DONE)) {
- inic_stop_idma(ap);
-
- /* Depending on circumstances, device error
- * isn't reported by IDMA, check it explicitly.
- */
- if (unlikely(readb(port_base + PORT_TF_COMMAND) &
- (ATA_DF | ATA_ERR)))
- qc->err_mask |= AC_ERR_DEV;
-
- ata_qc_complete(qc);
- return;
- }
-
- spurious:
- ata_port_warn(ap, "unhandled interrupt: cmd=0x%x irq_stat=0x%x idma_stat=0x%x\n",
- qc ? qc->tf.command : 0xff, irq_stat, idma_stat);
-}
-
-static irqreturn_t inic_interrupt(int irq, void *dev_instance)
-{
- struct ata_host *host = dev_instance;
- struct inic_host_priv *hpriv = host->private_data;
- u16 host_irq_stat;
- int i, handled = 0;
-
- host_irq_stat = readw(hpriv->mmio_base + HOST_IRQ_STAT);
-
- if (unlikely(!(host_irq_stat & HIRQ_GLOBAL)))
- goto out;
-
- spin_lock(&host->lock);
-
- for (i = 0; i < NR_PORTS; i++)
- if (host_irq_stat & (HIRQ_PORT0 << i)) {
- inic_host_intr(host->ports[i]);
- handled++;
- }
-
- spin_unlock(&host->lock);
-
- out:
- return IRQ_RETVAL(handled);
-}
-
-static int inic_check_atapi_dma(struct ata_queued_cmd *qc)
-{
- /* For some reason ATAPI_PROT_DMA doesn't work for some
- * commands including writes and other misc ops. Use PIO
- * protocol instead, which BTW is driven by the DMA engine
- * anyway, so it shouldn't make much difference for native
- * SATA devices.
- */
- if (atapi_cmd_type(qc->cdb[0]) == READ)
- return 0;
- return 1;
-}
-
-static void inic_fill_sg(struct inic_prd *prd, struct ata_queued_cmd *qc)
-{
- struct scatterlist *sg;
- unsigned int si;
- u8 flags = 0;
-
- if (qc->tf.flags & ATA_TFLAG_WRITE)
- flags |= PRD_WRITE;
-
- if (ata_is_dma(qc->tf.protocol))
- flags |= PRD_DMA;
-
- for_each_sg(qc->sg, sg, qc->n_elem, si) {
- prd->mad = cpu_to_le32(sg_dma_address(sg));
- prd->len = cpu_to_le16(sg_dma_len(sg));
- prd->flags = flags;
- prd++;
- }
-
- WARN_ON(!si);
- prd[-1].flags |= PRD_END;
-}
-
-static enum ata_completion_errors inic_qc_prep(struct ata_queued_cmd *qc)
-{
- struct inic_port_priv *pp = qc->ap->private_data;
- struct inic_pkt *pkt = pp->pkt;
- struct inic_cpb *cpb = &pkt->cpb;
- struct inic_prd *prd = pkt->prd;
- bool is_atapi = ata_is_atapi(qc->tf.protocol);
- bool is_data = ata_is_data(qc->tf.protocol);
- unsigned int cdb_len = 0;
-
- if (is_atapi)
- cdb_len = qc->dev->cdb_len;
-
- /* prepare packet, based on initio driver */
- memset(pkt, 0, sizeof(struct inic_pkt));
-
- cpb->ctl_flags = CPB_CTL_VALID | CPB_CTL_IEN;
- if (is_atapi || is_data)
- cpb->ctl_flags |= CPB_CTL_DATA;
-
- cpb->len = cpu_to_le32(qc->nbytes + cdb_len);
- cpb->prd = cpu_to_le32(pp->pkt_dma + offsetof(struct inic_pkt, prd));
-
- cpb->device = qc->tf.device;
- cpb->feature = qc->tf.feature;
- cpb->nsect = qc->tf.nsect;
- cpb->lbal = qc->tf.lbal;
- cpb->lbam = qc->tf.lbam;
- cpb->lbah = qc->tf.lbah;
-
- if (qc->tf.flags & ATA_TFLAG_LBA48) {
- cpb->hob_feature = qc->tf.hob_feature;
- cpb->hob_nsect = qc->tf.hob_nsect;
- cpb->hob_lbal = qc->tf.hob_lbal;
- cpb->hob_lbam = qc->tf.hob_lbam;
- cpb->hob_lbah = qc->tf.hob_lbah;
- }
-
- cpb->command = qc->tf.command;
- /* don't load ctl - dunno why. it's like that in the initio driver */
-
- /* setup PRD for CDB */
- if (is_atapi) {
- memcpy(pkt->cdb, qc->cdb, ATAPI_CDB_LEN);
- prd->mad = cpu_to_le32(pp->pkt_dma +
- offsetof(struct inic_pkt, cdb));
- prd->len = cpu_to_le16(cdb_len);
- prd->flags = PRD_CDB | PRD_WRITE;
- if (!is_data)
- prd->flags |= PRD_END;
- prd++;
- }
-
- /* setup sg table */
- if (is_data)
- inic_fill_sg(prd, qc);
-
- pp->cpb_tbl[0] = pp->pkt_dma;
-
- return AC_ERR_OK;
-}
-
-static unsigned int inic_qc_issue(struct ata_queued_cmd *qc)
-{
- struct ata_port *ap = qc->ap;
- void __iomem *port_base = inic_port_base(ap);
-
- /* fire up the ADMA engine */
- writew(HCTL_FTHD0 | HCTL_LEDEN, port_base + HOST_CTL);
- writew(IDMA_CTL_GO, port_base + PORT_IDMA_CTL);
- writeb(0, port_base + PORT_CPB_PTQFIFO);
-
- return 0;
-}
-
-static void inic_tf_read(struct ata_port *ap, struct ata_taskfile *tf)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- tf->error = readb(port_base + PORT_TF_FEATURE);
- tf->nsect = readb(port_base + PORT_TF_NSECT);
- tf->lbal = readb(port_base + PORT_TF_LBAL);
- tf->lbam = readb(port_base + PORT_TF_LBAM);
- tf->lbah = readb(port_base + PORT_TF_LBAH);
- tf->device = readb(port_base + PORT_TF_DEVICE);
- tf->status = readb(port_base + PORT_TF_COMMAND);
-}
-
-static void inic_qc_fill_rtf(struct ata_queued_cmd *qc)
-{
- struct ata_taskfile *rtf = &qc->result_tf;
- struct ata_taskfile tf;
-
- /* FIXME: Except for status and error, result TF access
- * doesn't work. I tried reading from BAR0/2, CPB and BAR5.
- * None works regardless of which command interface is used.
- * For now return true iff status indicates device error.
- * This means that we're reporting bogus sector for RW
- * failures. Eeekk....
- */
- inic_tf_read(qc->ap, &tf);
-
- if (tf.status & ATA_ERR) {
- rtf->status = tf.status;
- rtf->error = tf.error;
- }
-}
-
-static void inic_freeze(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(PIRQ_MASK_FREEZE, port_base + PORT_IRQ_MASK);
- writeb(0xff, port_base + PORT_IRQ_STAT);
-}
-
-static void inic_thaw(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- writeb(0xff, port_base + PORT_IRQ_STAT);
- writeb(PIRQ_MASK_DEFAULT, port_base + PORT_IRQ_MASK);
-}
-
-static int inic_check_ready(struct ata_link *link)
-{
- void __iomem *port_base = inic_port_base(link->ap);
-
- return ata_check_ready(readb(port_base + PORT_TF_COMMAND));
-}
-
-/*
- * SRST and SControl hardreset don't give valid signature on this
- * controller. Only controller specific hardreset mechanism works.
- */
-static int inic_hardreset(struct ata_link *link, unsigned int *class,
- unsigned long deadline)
-{
- struct ata_port *ap = link->ap;
- void __iomem *port_base = inic_port_base(ap);
- void __iomem *idma_ctl = port_base + PORT_IDMA_CTL;
- const unsigned int *timing = sata_ehc_deb_timing(&link->eh_context);
- int rc;
-
- /* hammer it into sane state */
- inic_reset_port(port_base);
-
- writew(IDMA_CTL_RST_ATA, idma_ctl);
- readw(idma_ctl); /* flush */
- ata_msleep(ap, 1);
- writew(0, idma_ctl);
-
- rc = sata_link_resume(link, timing, deadline);
- if (rc) {
- ata_link_warn(link,
- "failed to resume link after reset (errno=%d)\n",
- rc);
- return rc;
- }
-
- *class = ATA_DEV_NONE;
- if (ata_link_online(link)) {
- struct ata_taskfile tf;
-
- /* wait for link to become ready */
- rc = ata_wait_after_reset(link, deadline, inic_check_ready);
- /* link occupied, -ENODEV too is an error */
- if (rc) {
- ata_link_warn(link,
- "device not ready after hardreset (errno=%d)\n",
- rc);
- return rc;
- }
-
- inic_tf_read(ap, &tf);
- *class = ata_port_classify(ap, &tf);
- }
-
- return 0;
-}
-
-static void inic_error_handler(struct ata_port *ap)
- __must_hold(&ap->host->eh_mutex)
-{
- void __iomem *port_base = inic_port_base(ap);
-
- inic_reset_port(port_base);
- ata_std_error_handler(ap);
-}
-
-static void inic_post_internal_cmd(struct ata_queued_cmd *qc)
-{
- /* make DMA engine forget about the failed command */
- if (qc->flags & ATA_QCFLAG_EH)
- inic_reset_port(inic_port_base(qc->ap));
-}
-
-static void init_port(struct ata_port *ap)
-{
- void __iomem *port_base = inic_port_base(ap);
- struct inic_port_priv *pp = ap->private_data;
-
- /* clear packet and CPB table */
- memset(pp->pkt, 0, sizeof(struct inic_pkt));
- memset(pp->cpb_tbl, 0, IDMA_CPB_TBL_SIZE);
-
- /* setup CPB lookup table addresses */
- writel(pp->cpb_tbl_dma, port_base + PORT_CPB_CPBLAR);
-}
-
-static int inic_port_resume(struct ata_port *ap)
-{
- init_port(ap);
- return 0;
-}
-
-static int inic_port_start(struct ata_port *ap)
-{
- struct device *dev = ap->host->dev;
- struct inic_port_priv *pp;
-
- /* alloc and initialize private data */
- pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL);
- if (!pp)
- return -ENOMEM;
- ap->private_data = pp;
-
- /* Alloc resources */
- pp->pkt = dmam_alloc_coherent(dev, sizeof(struct inic_pkt),
- &pp->pkt_dma, GFP_KERNEL);
- if (!pp->pkt)
- return -ENOMEM;
-
- pp->cpb_tbl = dmam_alloc_coherent(dev, IDMA_CPB_TBL_SIZE,
- &pp->cpb_tbl_dma, GFP_KERNEL);
- if (!pp->cpb_tbl)
- return -ENOMEM;
-
- init_port(ap);
-
- return 0;
-}
-
-static struct ata_port_operations inic_port_ops = {
- .inherits = &sata_port_ops,
-
- .check_atapi_dma = inic_check_atapi_dma,
- .qc_prep = inic_qc_prep,
- .qc_issue = inic_qc_issue,
- .qc_fill_rtf = inic_qc_fill_rtf,
-
- .freeze = inic_freeze,
- .thaw = inic_thaw,
- .reset.hardreset = inic_hardreset,
- .error_handler = inic_error_handler,
- .post_internal_cmd = inic_post_internal_cmd,
-
- .scr_read = inic_scr_read,
- .scr_write = inic_scr_write,
-
- .port_resume = inic_port_resume,
- .port_start = inic_port_start,
-};
-
-static const struct ata_port_info inic_port_info = {
- .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA,
- .pio_mask = ATA_PIO4,
- .mwdma_mask = ATA_MWDMA2,
- .udma_mask = ATA_UDMA6,
- .port_ops = &inic_port_ops
-};
-
-static int init_controller(void __iomem *mmio_base, u16 hctl)
-{
- int i;
- u16 val;
-
- hctl &= ~HCTL_KNOWN_BITS;
-
- /* Soft reset whole controller. Spec says reset duration is 3
- * PCI clocks, be generous and give it 10ms.
- */
- writew(hctl | HCTL_SOFTRST, mmio_base + HOST_CTL);
- readw(mmio_base + HOST_CTL); /* flush */
-
- for (i = 0; i < 10; i++) {
- msleep(1);
- val = readw(mmio_base + HOST_CTL);
- if (!(val & HCTL_SOFTRST))
- break;
- }
-
- if (val & HCTL_SOFTRST)
- return -EIO;
-
- /* mask all interrupts and reset ports */
- for (i = 0; i < NR_PORTS; i++) {
- void __iomem *port_base = mmio_base + i * PORT_SIZE;
-
- writeb(0xff, port_base + PORT_IRQ_MASK);
- inic_reset_port(port_base);
- }
-
- /* port IRQ is masked now, unmask global IRQ */
- writew(hctl & ~HCTL_IRQOFF, mmio_base + HOST_CTL);
- val = readw(mmio_base + HOST_IRQ_MASK);
- val &= ~(HIRQ_PORT0 | HIRQ_PORT1);
- writew(val, mmio_base + HOST_IRQ_MASK);
-
- return 0;
-}
-
-#ifdef CONFIG_PM_SLEEP
-static int inic_pci_device_resume(struct pci_dev *pdev)
-{
- struct ata_host *host = pci_get_drvdata(pdev);
- struct inic_host_priv *hpriv = host->private_data;
- int rc;
-
- rc = ata_pci_device_do_resume(pdev);
- if (rc)
- return rc;
-
- if (pdev->dev.power.power_state.event == PM_EVENT_SUSPEND) {
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc)
- return rc;
- }
-
- ata_host_resume(host);
-
- return 0;
-}
-#endif
-
-static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- const struct ata_port_info *ppi[] = { &inic_port_info, NULL };
- struct ata_host *host;
- struct inic_host_priv *hpriv;
- void __iomem * const *iomap;
- int mmio_bar;
- int i, rc;
-
- ata_print_version_once(&pdev->dev, DRV_VERSION);
-
- dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact linux-ide@vger.kernel.org if in production use\n");
-
- /* alloc host */
- host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS);
- hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);
- if (!host || !hpriv)
- return -ENOMEM;
-
- host->private_data = hpriv;
-
- /* Acquire resources and fill host. Note that PCI and cardbus
- * use different BARs.
- */
- rc = pcim_enable_device(pdev);
- if (rc)
- return rc;
-
- if (pci_resource_flags(pdev, MMIO_BAR_PCI) & IORESOURCE_MEM)
- mmio_bar = MMIO_BAR_PCI;
- else
- mmio_bar = MMIO_BAR_CARDBUS;
-
- rc = pcim_iomap_regions(pdev, 1 << mmio_bar, DRV_NAME);
- if (rc)
- return rc;
- host->iomap = iomap = pcim_iomap_table(pdev);
- hpriv->mmio_base = iomap[mmio_bar];
- hpriv->cached_hctl = readw(hpriv->mmio_base + HOST_CTL);
-
- for (i = 0; i < NR_PORTS; i++) {
- struct ata_port *ap = host->ports[i];
-
- ata_port_pbar_desc(ap, mmio_bar, -1, "mmio");
- ata_port_pbar_desc(ap, mmio_bar, i * PORT_SIZE, "port");
- }
-
- /* Set dma_mask. This devices doesn't support 64bit addressing. */
- rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (rc) {
- dev_err(&pdev->dev, "32-bit DMA enable failed\n");
- return rc;
- }
-
- rc = init_controller(hpriv->mmio_base, hpriv->cached_hctl);
- if (rc) {
- dev_err(&pdev->dev, "failed to initialize controller\n");
- return rc;
- }
-
- pci_set_master(pdev);
- return ata_host_activate(host, pdev->irq, inic_interrupt, IRQF_SHARED,
- &inic_sht);
-}
-
-static const struct pci_device_id inic_pci_tbl[] = {
- { PCI_VDEVICE(INIT, 0x1622), },
- { },
-};
-
-static struct pci_driver inic_pci_driver = {
- .name = DRV_NAME,
- .id_table = inic_pci_tbl,
-#ifdef CONFIG_PM_SLEEP
- .suspend = ata_pci_device_suspend,
- .resume = inic_pci_device_resume,
-#endif
- .probe = inic_init_one,
- .remove = ata_pci_remove_one,
-};
-
-module_pci_driver(inic_pci_driver);
-
-MODULE_AUTHOR("Tejun Heo");
-MODULE_DESCRIPTION("low-level driver for Initio 162x SATA");
-MODULE_LICENSE("GPL v2");
-MODULE_DEVICE_TABLE(pci, inic_pci_tbl);
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/ata/sata_qstor.c b/drivers/ata/sata_qstor.c
index 4e7f5b2ff3f6..f38310b84f51 100644
--- a/drivers/ata/sata_qstor.c
+++ b/drivers/ata/sata_qstor.c
@@ -584,8 +584,12 @@ static int qs_ata_init_one(struct pci_dev *pdev,
qs_host_init(host, board_idx);
pci_set_master(pdev);
- return ata_host_activate(host, pdev->irq, qs_intr, IRQF_SHARED,
- &qs_ata_sht);
+ rc = ata_host_activate(host, pdev->irq, qs_intr, IRQF_SHARED,
+ &qs_ata_sht);
+ if (rc)
+ qs_host_stop(host);
+
+ return rc;
}
module_pci_driver(qs_ata_pci_driver);
diff --git a/drivers/base/devtmpfs.c b/drivers/base/devtmpfs.c
index aef0fcc6aba1..11c70888f38b 100644
--- a/drivers/base/devtmpfs.c
+++ b/drivers/base/devtmpfs.c
@@ -72,39 +72,90 @@ static struct file_system_type internal_fs_type = {
.kill_sb = kill_anon_super,
};
-/* Simply take a ref on the existing mount */
+struct devtmpfs_context {
+ struct fs_context *fc;
+};
+
+static void devtmpfs_free(struct fs_context *fc)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ if (ctx) {
+ put_fs_context(ctx->fc);
+ kfree(ctx);
+ }
+}
+
+static int devtmpfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ return ctx->fc->ops->parse_param(ctx->fc, param);
+}
+
+static int devtmpfs_parse_monolithic(struct fs_context *fc, void *data)
+{
+ struct devtmpfs_context *ctx = fc->fs_private;
+
+ if (ctx->fc->ops->parse_monolithic)
+ return ctx->fc->ops->parse_monolithic(ctx->fc, data);
+ return generic_parse_monolithic(ctx->fc, data);
+}
+
static int devtmpfs_get_tree(struct fs_context *fc)
{
+ struct devtmpfs_context *ctx = fc->fs_private;
struct super_block *sb = mnt->mnt_sb;
+ int err;
atomic_inc(&sb->s_active);
down_write(&sb->s_umount);
+
+ if (ctx->fc->ops->reconfigure) {
+ err = ctx->fc->ops->reconfigure(ctx->fc);
+ if (err) {
+ deactivate_locked_super(sb);
+ return err;
+ }
+ }
+
fc->root = dget(sb->s_root);
return 0;
}
-/* Ops are filled in during init depending on underlying shmem or ramfs type */
-static struct fs_context_operations devtmpfs_context_ops = {};
+static const struct fs_context_operations devtmpfs_context_ops = {
+ .free = devtmpfs_free,
+ .parse_param = devtmpfs_parse_param,
+ .parse_monolithic = devtmpfs_parse_monolithic,
+ .get_tree = devtmpfs_get_tree,
+};
-/* Call the underlying initialization and set to our ops */
static int devtmpfs_init_fs_context(struct fs_context *fc)
{
- int ret;
-#ifdef CONFIG_TMPFS
- ret = shmem_init_fs_context(fc);
-#else
- ret = ramfs_init_fs_context(fc);
-#endif
- if (ret < 0)
- return ret;
+ struct devtmpfs_context *ctx;
+ int err;
+
+ ctx = kzalloc_obj(struct devtmpfs_context);
+ if (!ctx)
+ return -ENOMEM;
+
+ /* Each mount will reconfigure the shared superblock w/ new options */
+ ctx->fc = fs_context_for_reconfigure(mnt->mnt_root,
+ mnt->mnt_sb->s_flags, MS_RMT_MASK);
+ if (IS_ERR(ctx->fc)) {
+ err = PTR_ERR(ctx->fc);
+ kfree(ctx);
+ return err;
+ }
+ fc->fs_private = ctx;
fc->ops = &devtmpfs_context_ops;
return 0;
}
static struct file_system_type dev_fs_type = {
- .name = "devtmpfs",
+ .name = "devtmpfs",
.init_fs_context = devtmpfs_init_fs_context,
};
@@ -442,31 +493,6 @@ static int __ref devtmpfsd(void *p)
return 0;
}
-/*
- * Get the underlying (shmem/ramfs) context ops to build ours
- */
-static int devtmpfs_configure_context(void)
-{
- struct fs_context *fc;
-
- fc = fs_context_for_reconfigure(mnt->mnt_root, mnt->mnt_sb->s_flags,
- MS_RMT_MASK);
- if (IS_ERR(fc))
- return PTR_ERR(fc);
-
- /* Set up devtmpfs_context_ops based on underlying type */
- devtmpfs_context_ops.free = fc->ops->free;
- devtmpfs_context_ops.dup = fc->ops->dup;
- devtmpfs_context_ops.parse_param = fc->ops->parse_param;
- devtmpfs_context_ops.parse_monolithic = fc->ops->parse_monolithic;
- devtmpfs_context_ops.get_tree = &devtmpfs_get_tree;
- devtmpfs_context_ops.reconfigure = fc->ops->reconfigure;
-
- put_fs_context(fc);
-
- return 0;
-}
-
/*
* Create devtmpfs instance, driver-core devices will add their device
* nodes here.
@@ -482,12 +508,6 @@ int __init devtmpfs_init(void)
return PTR_ERR(mnt);
}
- err = devtmpfs_configure_context();
- if (err) {
- pr_err("unable to configure devtmpfs type %d\n", err);
- return err;
- }
-
err = register_filesystem(&dev_fs_type);
if (err) {
pr_err("unable to register devtmpfs type %d\n", err);
diff --git a/drivers/block/amiflop.c b/drivers/block/amiflop.c
index 2932b6653b6f..a477ada6b1ef 100644
--- a/drivers/block/amiflop.c
+++ b/drivers/block/amiflop.c
@@ -265,6 +265,8 @@ static struct fd_data_type data_types[] = {
static struct amiga_floppy_struct unit[FD_MAX_UNITS];
static struct timer_list flush_track_timer[FD_MAX_UNITS];
+static DEFINE_SPINLOCK(flush_track_timer_lock);
+static bool flush_track_timer_stopped[FD_MAX_UNITS];
static struct timer_list post_write_timer;
static unsigned long post_write_timer_drive;
static struct timer_list motor_on_timer;
@@ -1370,11 +1372,22 @@ static void flush_track_callback(struct timer_list *timer)
sizeof(flush_track_timer[0]);
nr&=3;
+ spin_lock(&flush_track_timer_lock);
+ if (flush_track_timer_stopped[nr]) {
+ spin_unlock(&flush_track_timer_lock);
+ return;
+ }
+ spin_unlock(&flush_track_timer_lock);
+
writefromint = 1;
if (!try_fdc(nr)) {
/* we might block in an interrupt, so try again later */
- flush_track_timer[nr].expires = jiffies + 1;
- add_timer(flush_track_timer + nr);
+ spin_lock(&flush_track_timer_lock);
+ if (!flush_track_timer_stopped[nr]) {
+ flush_track_timer[nr].expires = jiffies + 1;
+ add_timer(flush_track_timer + nr);
+ }
+ spin_unlock(&flush_track_timer_lock);
return;
}
get_fdc(nr);
@@ -1421,6 +1434,35 @@ static int non_int_flush_track (unsigned long nr)
return 1;
}
+static void flush_track_timer_stop(unsigned int drive)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&flush_track_timer_lock, flags);
+ flush_track_timer_stopped[drive] = true;
+ spin_unlock_irqrestore(&flush_track_timer_lock, flags);
+
+ timer_delete_sync(flush_track_timer + drive);
+}
+
+static void flush_track_timer_start(unsigned int drive)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&flush_track_timer_lock, flags);
+ flush_track_timer_stopped[drive] = false;
+ if (unit[drive].dirty)
+ mod_timer(flush_track_timer + drive, jiffies + 1);
+ spin_unlock_irqrestore(&flush_track_timer_lock, flags);
+}
+
+static void flush_track_timer_sync(unsigned int drive)
+{
+ flush_track_timer_stop(drive);
+ non_int_flush_track(drive);
+ flush_track_timer_start(drive);
+}
+
static int get_track(int drive, int track)
{
int error, errcnt;
@@ -1435,8 +1477,7 @@ static int get_track(int drive, int track)
}
if (unit[drive].dirty == 1) {
- timer_delete(flush_track_timer + drive);
- non_int_flush_track (drive);
+ flush_track_timer_sync(drive);
}
errcnt = 0;
while (errcnt < MAX_ERRORS) {
@@ -1562,6 +1603,12 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
case FDFMTTRK:
if (param < p->type->tracks * p->type->heads)
{
+ flush_track_timer_stop(drive);
+ if (unit[drive].dirty == 1 &&
+ !non_int_flush_track(drive)) {
+ flush_track_timer_start(drive);
+ return -EIO;
+ }
get_fdc(drive);
if (fd_seek(drive,param) != 0){
memset(p->trackbuf, FD_FILL_BYTE,
@@ -1570,6 +1617,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
}
floppy_off(drive);
rel_fdc();
+ flush_track_timer_start(drive);
}
else
return -EINVAL;
@@ -1591,8 +1639,7 @@ static int fd_locked_ioctl(struct block_device *bdev, blk_mode_t mode,
case FDDEFPRM:
return -EINVAL;
case FDFLUSH: /* unconditionally, even if not needed */
- timer_delete(flush_track_timer + drive);
- non_int_flush_track(drive);
+ flush_track_timer_sync(drive);
break;
#ifdef RAW_IOCTL
case IOCTL_RAW_TRACK:
@@ -1611,9 +1658,15 @@ static int fd_ioctl(struct block_device *bdev, blk_mode_t mode,
unsigned int cmd, unsigned long param)
{
int ret;
+ unsigned int memflags = 0;
+ bool freeze_queue = cmd == FDFMTTRK || cmd == FDFLUSH;
mutex_lock(&amiflop_mutex);
+ if (freeze_queue)
+ memflags = blk_mq_freeze_queue(bdev->bd_disk->queue);
ret = fd_locked_ioctl(bdev, mode, cmd, param);
+ if (freeze_queue)
+ blk_mq_unfreeze_queue(bdev->bd_disk->queue, memflags);
mutex_unlock(&amiflop_mutex);
return ret;
@@ -1711,11 +1764,12 @@ static void floppy_release(struct gendisk *disk)
{
struct amiga_floppy_struct *p = disk->private_data;
int drive = p - unit;
+ unsigned int memflags;
mutex_lock(&amiflop_mutex);
+ memflags = blk_mq_freeze_queue(disk->queue);
if (unit[drive].dirty == 1) {
- timer_delete(flush_track_timer + drive);
- non_int_flush_track (drive);
+ flush_track_timer_sync(drive);
}
if (!fd_ref[drive]--) {
@@ -1725,6 +1779,7 @@ static void floppy_release(struct gendisk *disk)
#ifdef MODULE
floppy_off (drive);
#endif
+ blk_mq_unfreeze_queue(disk->queue, memflags);
mutex_unlock(&amiflop_mutex);
}
diff --git a/drivers/block/aoe/aoenet.c b/drivers/block/aoe/aoenet.c
index 66e617664c14..cd7c735bb43b 100644
--- a/drivers/block/aoe/aoenet.c
+++ b/drivers/block/aoe/aoenet.c
@@ -131,9 +131,7 @@ static int
aoenet_rcv(struct sk_buff *skb, struct net_device *ifp, struct packet_type *pt, struct net_device *orig_dev)
{
struct aoe_hdr *h;
- struct aoe_atahdr *ah;
u32 n;
- int sn;
if (dev_net(ifp) != &init_net)
goto exit;
@@ -144,12 +142,8 @@ aoenet_rcv(struct sk_buff *skb, struct net_device *ifp, struct packet_type *pt,
if (!is_aoe_netif(ifp))
goto exit;
skb_push(skb, ETH_HLEN); /* (1) */
- sn = sizeof(*h) + sizeof(*ah);
- if (skb->len >= sn) {
- sn -= skb_headlen(skb);
- if (sn > 0 && !__pskb_pull_tail(skb, sn))
- goto exit;
- }
+ if (!pskb_may_pull(skb, sizeof(*h)))
+ goto exit;
h = (struct aoe_hdr *) skb->data;
n = get_unaligned_be32(&h->tag);
if ((h->verfl & AOEFL_RSP) == 0 || (n & 1<<31))
@@ -171,10 +165,14 @@ aoenet_rcv(struct sk_buff *skb, struct net_device *ifp, struct packet_type *pt,
switch (h->cmd) {
case AOECMD_ATA:
+ if (!pskb_may_pull(skb, sizeof(*h) + sizeof(struct aoe_atahdr)))
+ goto exit;
/* ata_rsp may keep skb for later processing or give it back */
skb = aoecmd_ata_rsp(skb);
break;
case AOECMD_CFG:
+ if (!pskb_may_pull(skb, sizeof(*h) + sizeof(struct aoe_cfghdr)))
+ goto exit;
aoecmd_cfg_rsp(skb);
break;
default:
diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c
index f04397b8e381..7465e10f36d4 100644
--- a/drivers/block/floppy.c
+++ b/drivers/block/floppy.c
@@ -2276,11 +2276,10 @@ static void request_done(int uptodate)
{
struct request *req = current_req;
int block;
- char msg[sizeof("request done ") + sizeof(int) * 3];
probing = 0;
- snprintf(msg, sizeof(msg), "request done %d", uptodate);
- reschedule_timeout(MAXTIMEOUT, msg);
+ reschedule_timeout(MAXTIMEOUT,
+ uptodate ? "request done 1" : "request done 0");
if (!req) {
pr_info("floppy.c: no request in request_done\n");
@@ -3601,7 +3600,7 @@ struct compat_floppy_drive_params {
};
struct compat_floppy_drive_struct {
- signed char flags;
+ compat_ulong_t flags;
compat_ulong_t spinup_date;
compat_ulong_t select_date;
compat_ulong_t first_read_date;
@@ -3785,6 +3784,7 @@ static int compat_getdrvstat(int drive, bool poll,
goto Eintr;
process_fd_request();
}
+ v.flags = drive_state[drive].flags;
v.spinup_date = drive_state[drive].spinup_date;
v.select_date = drive_state[drive].select_date;
v.first_read_date = drive_state[drive].first_read_date;
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 758c20678bf6..f15a5cae5c63 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -1708,8 +1708,10 @@ static int lo_compat_ioctl(struct block_device *bdev, blk_mode_t mode,
switch(cmd) {
case LOOP_SET_STATUS:
- err = loop_set_status_compat(lo,
- (const struct compat_loop_info __user *)arg);
+ err = -EPERM;
+ if ((mode & BLK_OPEN_WRITE) || capable(CAP_SYS_ADMIN))
+ err = loop_set_status_compat(lo,
+ (const struct compat_loop_info __user *)arg);
break;
case LOOP_GET_STATUS:
err = loop_get_status_compat(lo,
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 66eb55e7162e..1ef01ce18f4f 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -321,6 +321,8 @@ struct ublk_device {
#define UB_STATE_OPEN 0
#define UB_STATE_USED 1
#define UB_STATE_DELETED 2
+/* STOP_DEV canceled the server's commands, until its release */
+#define UB_STATE_STOPPING 3
unsigned long state;
int ub_number;
@@ -334,6 +336,13 @@ struct ublk_device {
u16 nr_queue_ready;
bool unprivileged_daemons;
struct mutex cancel_mutex;
+ /* the open /dev/ublkcN, protected by cancel_mutex */
+ struct file *ch_file;
+ /*
+ * A cancel started in this FETCH round. Set by ublk_set_canceling(),
+ * cleared only by ublk_reset_ch_dev() when a new round starts. While
+ * it is set, no queue clears its ->canceling.
+ */
bool canceling;
pid_t ublksrv_tgid;
struct delayed_work exit_work;
@@ -810,6 +819,8 @@ ublk_batch_alloc_fcmd(struct io_uring_cmd *cmd)
if (fcmd) {
fcmd->cmd = cmd;
fcmd->buf_group = READ_ONCE(cmd->sqe->buf_index);
+ /* a cancel may look at it before it is linked */
+ INIT_LIST_HEAD(&fcmd->node);
}
return fcmd;
}
@@ -2399,6 +2410,9 @@ static int ublk_ch_open(struct inode *inode, struct file *filp)
return -EBUSY;
filp->private_data = ub;
ub->ublksrv_tgid = current->tgid;
+ mutex_lock(&ub->cancel_mutex);
+ ub->ch_file = filp;
+ mutex_unlock(&ub->cancel_mutex);
return 0;
}
@@ -2415,11 +2429,17 @@ static void ublk_reset_ch_dev(struct ublk_device *ub)
spin_unlock(&ubq->cancel_lock);
}
+ /* a new FETCH round starts, the queues stay canceling until ready */
+ mutex_lock(&ub->cancel_mutex);
+ ub->canceling = false;
+ mutex_unlock(&ub->cancel_mutex);
+
/* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */
ub->mm = NULL;
ub->nr_queue_ready = 0;
ub->unprivileged_daemons = false;
ub->ublksrv_tgid = -1;
+ clear_bit(UB_STATE_STOPPING, &ub->state);
}
static struct gendisk *ublk_get_disk(struct ublk_device *ub)
@@ -2539,12 +2559,15 @@ static void ublk_ch_release_work_fn(struct work_struct *work)
}
/*
- * disk isn't attached yet, either device isn't live, or it has
- * been removed already, so we needn't to do anything
+ * No disk: the device isn't live, or it has been removed already.
+ * There are no requests to abort, but the round still has to be
+ * reset, so that a new server can fetch and start the device.
*/
disk = ublk_get_disk(ub);
- if (!disk)
- goto out;
+ if (!disk) {
+ mutex_lock(&ub->mutex);
+ goto reset;
+ }
/*
* All uring_cmd are done now, so abort any request outstanding to
@@ -2576,7 +2599,7 @@ static void ublk_ch_release_work_fn(struct work_struct *work)
/* double check after grabbing lock */
if (!ub->ub_disk)
- goto unlock;
+ goto reset;
/*
* Transition the device to the nosrv state. What exactly this
@@ -2602,29 +2625,36 @@ static void ublk_ch_release_work_fn(struct work_struct *work)
WRITE_ONCE(ublk_get_queue(ub, i)->fail_io, true);
}
}
-unlock:
+reset:
+ /*
+ * All uring_cmd has been done now, reset device & ubq. Under
+ * ub->mutex, so START_DEV sees the round either ready or reset.
+ */
+ ublk_reset_ch_dev(ub);
mutex_unlock(&ub->mutex);
ublk_put_disk(disk);
-
- /* all uring_cmd has been done now, reset device & ubq */
- ublk_reset_ch_dev(ub);
-out:
clear_bit(UB_STATE_OPEN, &ub->state);
/* put the reference grabbed in ublk_ch_release() */
ublk_put_device(ub);
+ module_put(THIS_MODULE);
}
static int ublk_ch_release(struct inode *inode, struct file *filp)
{
struct ublk_device *ub = filp->private_data;
+ mutex_lock(&ub->cancel_mutex);
+ ub->ch_file = NULL;
+ mutex_unlock(&ub->cancel_mutex);
/*
* Grab ublk device reference, so it won't be gone until we are
* really released from work function.
*/
ublk_get_device(ub);
+ /* Keep the worker's callback and final device release code resident. */
+ __module_get(THIS_MODULE);
INIT_DELAYED_WORK(&ub->exit_work, ublk_ch_release_work_fn);
schedule_delayed_work(&ub->exit_work, 0);
return 0;
@@ -2790,7 +2820,8 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, u16 tag,
done = !!(io->flags & UBLK_IO_FLAG_CANCELED);
if (!done) {
io->flags |= UBLK_IO_FLAG_CANCELED;
- cmd = io->cmd;
+ /* dependency ordered against smp_wmb() in ublk_prep_cancel() */
+ cmd = READ_ONCE(io->cmd);
io->cmd = NULL;
}
spin_unlock(&ubq->cancel_lock);
@@ -3006,13 +3037,54 @@ static void ublk_stop_dev_unlocked(struct ublk_device *ub)
put_disk(disk);
}
+static struct file *ublk_get_ch_file(struct ublk_device *ub)
+{
+ struct file *file;
+
+ mutex_lock(&ub->cancel_mutex);
+ file = ub->ch_file;
+ if (file && !file_ref_get(&file->f_ref))
+ file = NULL;
+ mutex_unlock(&ub->cancel_mutex);
+ return file;
+}
+
static void ublk_stop_dev(struct ublk_device *ub)
{
+ struct file *file;
+
+ /*
+ * FETCH, PREP and START_DEV take ub->mutex. If a server has
+ * /dev/ublkcN open, set STOPPING, which turns it away, and hold a
+ * reference on the file: the server's release, whose reset clears
+ * STOPPING and lets a new server attach, can't run before we drop it.
+ * So the cancel can run after the unlock and only meets this server's
+ * commands; it has to: io_uring_cmd_done() may take uring_lock, under
+ * which FETCH takes ub->mutex.
+ */
mutex_lock(&ub->mutex);
ublk_stop_dev_unlocked(ub);
- mutex_unlock(&ub->mutex);
cancel_work_sync(&ub->partition_scan_work);
+ file = ublk_get_ch_file(ub);
+ if (file) {
+ /*
+ * The server has to close /dev/ublkcN before this device can
+ * be started again: the reset in its release clears STOPPING.
+ */
+ set_bit(UB_STATE_STOPPING, &ub->state);
+ /* for wake_up_var() below, see wake_up_bit() */
+ smp_mb__after_atomic();
+ }
+ mutex_unlock(&ub->mutex);
+
+ /* no server: nothing to cancel */
+ if (!file)
+ return;
+
+ /* wake a START_DEV waiting for the device to get ready */
+ wake_up_var(&ub->nr_queue_ready);
ublk_cancel_dev(ub);
+ __fput_sync(file);
}
static void ublk_reset_io_flags(struct ublk_queue *ubq, struct ublk_io *io)
@@ -3024,11 +3096,19 @@ static void ublk_reset_io_flags(struct ublk_queue *ubq, struct ublk_io *io)
}
/* reset per-queue io flags */
-static void ublk_queue_reset_io_flags(struct ublk_queue *ubq)
+static void ublk_queue_reset_io_flags(struct ublk_device *ub,
+ struct ublk_queue *ubq)
{
- spin_lock(&ubq->cancel_lock);
- ubq->canceling = false;
- spin_unlock(&ubq->cancel_lock);
+ /*
+ * A cancel in this FETCH round took a command which still counts as
+ * ready, so the queue has to stay canceling. ub->canceling is set
+ * under cancel_mutex before any command is taken: either we see it
+ * here, or the cancel marks this queue again later.
+ */
+ mutex_lock(&ub->cancel_mutex);
+ if (!ub->canceling)
+ ubq->canceling = false;
+ mutex_unlock(&ub->cancel_mutex);
ubq->fail_io = false;
ubq->force_abort = false;
}
@@ -3051,24 +3131,17 @@ static void ublk_mark_io_ready(struct ublk_device *ub, u16 q_id,
ub->nr_queue_ready++;
/*
- * Reset queue flags as soon as this queue is ready.
- * This clears the canceling flag, allowing batch FETCH commands
- * to succeed during recovery without waiting for all queues.
+ * Reset queue flags as soon as this queue is ready. Unless
+ * this round saw a cancel, this clears the canceling flag,
+ * allowing batch FETCH commands to succeed during recovery
+ * without waiting for all queues.
*/
- ublk_queue_reset_io_flags(ubq);
+ ublk_queue_reset_io_flags(ub, ubq);
}
- /* Check if all queues are ready */
- if (ublk_dev_ready(ub)) {
- /*
- * All queues ready - clear device-level canceling flag
- * and wake ublk_dev_ready() waiters.
- */
- mutex_lock(&ub->cancel_mutex);
- ub->canceling = false;
- mutex_unlock(&ub->cancel_mutex);
+ /* All queues ready - wake ublk_dev_ready() waiters */
+ if (ublk_dev_ready(ub))
wake_up_var(&ub->nr_queue_ready);
- }
}
static inline int ublk_check_cmd_op(u32 cmd_op)
@@ -3152,6 +3225,12 @@ ublk_fill_io_cmd(struct ublk_io *io, struct io_uring_cmd *cmd)
return req;
}
+/*
+ * Call before ublk_fill_io_cmd() publishes @cmd in io->cmd: a control-path
+ * cancel may complete any command found there, and io_uring_cmd_done() only
+ * takes it off the cancelable list if it is marked already. The handlers
+ * hold uring_lock, so marking takes no lock.
+ */
static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
unsigned int issue_flags,
struct ublk_queue *ubq, u16 tag)
@@ -3165,6 +3244,8 @@ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd,
pdu->ubq = ubq;
pdu->tag = tag;
io_uring_cmd_mark_cancelable(cmd, issue_flags);
+ /* pairs with the cancel loading cmd from io->cmd, then cmd->flags */
+ smp_wmb();
}
static void ublk_io_release(void *priv)
@@ -3269,6 +3350,9 @@ static int ublk_check_fetch_buf(const struct ublk_device *ub, __u64 buf_addr)
static int __ublk_fetch(struct io_uring_cmd *cmd, struct ublk_device *ub,
struct ublk_io *io, u16 q_id)
{
+ if (test_bit(UB_STATE_STOPPING, &ub->state))
+ return UBLK_IO_RES_ABORT;
+
/* UBLK_IO_FETCH_REQ is only allowed before dev is setup */
if (ublk_dev_ready(ub))
return -EBUSY;
@@ -3412,11 +3496,11 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
ret = ublk_check_fetch_buf(ub, addr);
if (ret)
goto out;
+ /* before ublk_fetch() publishes io->cmd, see ublk_prep_cancel() */
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
ret = ublk_fetch(cmd, ub, io, addr, q_id);
if (ret)
- goto out;
-
- ublk_prep_cancel(cmd, issue_flags, ubq, tag);
+ goto out_done;
return -EIOCBQUEUED;
}
@@ -3460,6 +3544,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
if (ret)
goto out;
io->res = result;
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
req = ublk_fill_io_cmd(io, cmd);
ublk_apply_io_buf(ub, io, cmd, addr, &auto_buf, &buf_idx);
if (buf_idx != UBLK_INVALID_BUF_IDX)
@@ -3478,19 +3563,24 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
* uring_cmd active first and prepare for handling new requeued
* request
*/
+ ublk_prep_cancel(cmd, issue_flags, ubq, tag);
req = ublk_fill_io_cmd(io, cmd);
io->buf.addr = addr;
if (likely(ublk_get_data(ubq, io, req))) {
__ublk_prep_compl_io_cmd(io, req);
- return UBLK_IO_RES_OK;
+ ret = UBLK_IO_RES_OK;
+ goto out_done;
}
break;
default:
goto out;
}
- ublk_prep_cancel(cmd, issue_flags, ubq, tag);
return -EIOCBQUEUED;
+ out_done:
+ /* marked cancelable: complete through io_uring_cmd_done() */
+ io_uring_cmd_done(cmd, ret, issue_flags);
+ return -EIOCBQUEUED;
out:
pr_devel("%s: complete: cmd op %d, tag %d ret %x io_flags %x\n",
__func__, cmd_op, tag, ret, io ? io->flags : 0);
@@ -3889,6 +3979,15 @@ static int ublk_batch_attach(struct ublk_queue *ubq,
bool free = false;
struct ublk_uring_cmd_pdu *pdu = ublk_get_uring_cmd_pdu(data->cmd);
+ /*
+ * Mark it cancelable before linking it into fcmd_head, where a cancel
+ * from the control path can take and complete it: see
+ * ublk_prep_cancel(). evts_lock orders the mark before the link.
+ */
+ pdu->ubq = ubq;
+ pdu->fcmd = fcmd;
+ io_uring_cmd_mark_cancelable(fcmd->cmd, data->issue_flags);
+
spin_lock(&ubq->evts_lock);
if (unlikely(ubq->force_abort || ubq->canceling)) {
free = true;
@@ -3899,14 +3998,12 @@ static int ublk_batch_attach(struct ublk_queue *ubq,
spin_unlock(&ubq->evts_lock);
if (unlikely(free)) {
+ /* off the cancelable list first, then nothing can see fcmd */
+ io_uring_cmd_done(data->cmd, -ENODEV, data->issue_flags);
ublk_batch_free_fcmd(fcmd);
- return -ENODEV;
+ return -EIOCBQUEUED;
}
- pdu->ubq = ubq;
- pdu->fcmd = fcmd;
- io_uring_cmd_mark_cancelable(fcmd->cmd, data->issue_flags);
-
if (!new_fcmd)
goto out;
@@ -3914,9 +4011,12 @@ static int ublk_batch_attach(struct ublk_queue *ubq,
* If the two fetch commands are originated from same io_ring_ctx,
* run batch dispatch directly. Otherwise, schedule task work for
* doing it.
+ *
+ * Use data->cmd, not fcmd->cmd: once fcmd is linked and not active,
+ * a cancel from the control path may complete and free it.
*/
if (io_uring_cmd_ctx_handle(new_fcmd->cmd) ==
- io_uring_cmd_ctx_handle(fcmd->cmd)) {
+ io_uring_cmd_ctx_handle(data->cmd)) {
data->cmd = new_fcmd->cmd;
ublk_batch_dispatch(ubq, data, new_fcmd);
} else {
@@ -4431,21 +4531,27 @@ static bool ublk_validate_user_pid(struct ublk_device *ub, pid_t ublksrv_pid)
return ub->ublksrv_tgid == ublksrv_pid;
}
+static bool ublk_dev_ready_or_stopping(const struct ublk_device *ub)
+{
+ return ublk_dev_ready(ub) || test_bit(UB_STATE_STOPPING, &ub->state);
+}
+
/*
- * Wait until all queues have fetched their I/O commands, and return with
- * ub->mutex held and readiness guaranteed: then every queue's ->canceling
- * is cleared. Ready may regress between wakeup and mutex_lock() (F_BATCH
- * UNPREP, daemon death), so re-check it under the mutex and wait again.
+ * Wait until all queues have fetched their I/O commands, or STOP_DEV set
+ * UB_STATE_STOPPING, and return with ub->mutex held. The queues stay
+ * canceling if this round saw a cancel, see ublk_queue_reset_io_flags().
+ * Ready may regress between wakeup and mutex_lock() (F_BATCH UNPREP,
+ * daemon death), so re-check it under the mutex and wait again.
*/
static int ublk_wait_dev_ready_and_lock(struct ublk_device *ub)
{
while (true) {
if (wait_var_event_interruptible(&ub->nr_queue_ready,
- ublk_dev_ready(ub)))
+ ublk_dev_ready_or_stopping(ub)))
return -EINTR;
mutex_lock(&ub->mutex);
- if (ublk_dev_ready(ub))
+ if (ublk_dev_ready_or_stopping(ub))
return 0;
mutex_unlock(&ub->mutex);
}
@@ -4545,6 +4651,10 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub,
ret = -EEXIST;
goto out_unlock;
}
+ if (test_bit(UB_STATE_STOPPING, &ub->state)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
disk = blk_mq_alloc_disk(&ub->tag_set, &lim, NULL);
if (IS_ERR(disk)) {
diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index 86f5dd3aced1..ad0cb42c1019 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -2142,10 +2142,8 @@ static void blkfront_closing(struct blkfront_info *info)
return;
/* No more blkif_request(). */
- if (info->rq && info->gd) {
- blk_mq_stop_hw_queues(info->rq);
+ if (info->gd)
blk_mark_disk_dead(info->gd);
- }
for_each_rinfo(info, rinfo, i) {
/* No more gnttab callback work. */
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index a9b3bb1d3bef..4ba0f77b2abd 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -415,7 +415,7 @@ static ssize_t mem_used_max_store(struct device *dev,
* Mark all pages which are older than or equal to cutoff as IDLE.
* Callers should hold the zram init lock in read mode
*/
-static void mark_idle(struct zram *zram, ktime_t cutoff)
+static void mark_idle(struct zram *zram, time64_t cutoff)
{
int is_idle = 1;
unsigned long nr_pages = zram->disksize >> PAGE_SHIFT;
@@ -439,7 +439,7 @@ static void mark_idle(struct zram *zram, ktime_t cutoff)
#ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME
is_idle = !cutoff ||
- ktime_after(cutoff, zram->table[index].attr.ac_time);
+ cutoff > zram->table[index].attr.ac_time;
#endif
if (is_idle)
set_slot_flag(zram, index, ZRAM_IDLE);
@@ -453,21 +453,26 @@ static ssize_t idle_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t len)
{
struct zram *zram = dev_to_zram(dev);
- ktime_t cutoff = 0;
+ time64_t cutoff = 0;
if (!sysfs_streq(buf, "all")) {
/*
* If it did not parse as 'all' try to treat it as an integer
* when we have memory tracking enabled.
*/
+ time64_t uptime;
u32 age_sec;
- if (IS_ENABLED(CONFIG_ZRAM_TRACK_ENTRY_ACTIME) &&
- !kstrtouint(buf, 0, &age_sec))
- cutoff = ktime_sub((u32)ktime_get_boottime_seconds(),
- age_sec);
- else
+ if (!IS_ENABLED(CONFIG_ZRAM_TRACK_ENTRY_ACTIME) ||
+ kstrtouint(buf, 0, &age_sec))
return -EINVAL;
+
+ /* No slot can be older than the system uptime */
+ uptime = ktime_get_boottime_seconds();
+ if (age_sec >= uptime)
+ return len;
+
+ cutoff = uptime - age_sec;
}
guard(rwsem_read)(&zram->dev_lock);
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index dc7191bf4234..ece1fd426f12 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -878,6 +878,8 @@ static const struct usb_device_id quirks_table[] = {
BTUSB_WIDEBAND_SPEECH },
{ USB_DEVICE(0x37ad, 0x0600), .driver_info = BTUSB_REALTEK |
BTUSB_WIDEBAND_SPEECH },
+ { USB_DEVICE(0x14b0, 0x047a), .driver_info = BTUSB_REALTEK |
+ BTUSB_WIDEBAND_SPEECH },
/* Additional Realtek 8761CU Bluetooth devices */
{ USB_DEVICE(0x0b05, 0x1bef), .driver_info = BTUSB_REALTEK |
diff --git a/drivers/char/hw_random/amd-rng.c b/drivers/char/hw_random/amd-rng.c
index dff80daae587..ac2d257793bf 100644
--- a/drivers/char/hw_random/amd-rng.c
+++ b/drivers/char/hw_random/amd-rng.c
@@ -80,7 +80,7 @@ static int amd_rng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
if (timeout-- == 0)
return read;
} else {
- return 0;
+ return read;
}
} else {
*data = ioread32(priv->iobase + RNGDATA);
diff --git a/drivers/char/hw_random/ba431-rng.c b/drivers/char/hw_random/ba431-rng.c
index b0a39032c610..810c8cd5629a 100644
--- a/drivers/char/hw_random/ba431-rng.c
+++ b/drivers/char/hw_random/ba431-rng.c
@@ -147,7 +147,7 @@ static int ba431_trng_read(struct hwrng *rng, void *buf, size_t max, bool wait)
}
}
- n *= sizeof(data);
+ n *= sizeof(*data);
return (n || !wait) ? n : -EIO;
}
diff --git a/drivers/char/hw_random/core.c b/drivers/char/hw_random/core.c
index 57eeefe90da9..1f9e31cdc7f0 100644
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -376,6 +376,7 @@ static ssize_t rng_current_store(struct device *dev,
cur_rng_set_by_user = 1;
drop_current_rng();
} else {
+ err = -ENODEV;
list_for_each_entry(rng, &rng_list, list) {
if (sysfs_streq(rng->name, buf)) {
err = set_current_rng(rng);
diff --git a/drivers/char/tpm/tpm-dev-common.c b/drivers/char/tpm/tpm-dev-common.c
index f942c0c8e402..6569212dc6b8 100644
--- a/drivers/char/tpm/tpm-dev-common.c
+++ b/drivers/char/tpm/tpm-dev-common.c
@@ -134,6 +134,9 @@ ssize_t tpm_common_read(struct file *file, char __user *buf,
ssize_t ret_size = 0;
int rc;
+ if (!size)
+ return 0;
+
mutex_lock(&priv->buffer_mutex);
if (priv->response_length) {
diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
index ae22295df798..119c73a603e6 100644
--- a/drivers/char/tpm/tpm2-cmd.c
+++ b/drivers/char/tpm/tpm2-cmd.c
@@ -483,8 +483,9 @@ int tpm2_probe(struct tpm_chip *chip)
out = (struct tpm_header *)buf->data;
if (be16_to_cpu(out->tag) == TPM2_ST_NO_SESSIONS)
chip->flags |= TPM_CHIP_FLAG_TPM2;
+ return 0;
}
- return 0;
+ return rc;
}
EXPORT_SYMBOL_GPL(tpm2_probe);
diff --git a/drivers/char/tpm/tpm_nsc.c b/drivers/char/tpm/tpm_nsc.c
index 879ac88f5783..bcfa2a1a208a 100644
--- a/drivers/char/tpm/tpm_nsc.c
+++ b/drivers/char/tpm/tpm_nsc.c
@@ -259,7 +259,12 @@ static struct platform_device *pdev = NULL;
static void tpm_nsc_remove(struct device *dev)
{
struct tpm_chip *chip = dev_get_drvdata(dev);
- struct tpm_nsc_priv *priv = dev_get_drvdata(&chip->dev);
+ struct tpm_nsc_priv *priv;
+
+ if (!chip)
+ return;
+
+ priv = dev_get_drvdata(&chip->dev);
tpm_chip_unregister(chip);
release_region(priv->base, 2);
@@ -322,7 +327,6 @@ static int __init init_nsc(void)
pdev->num_resources = 0;
pdev->dev.driver = &nsc_drv.driver;
- pdev->dev.release = tpm_nsc_remove;
if ((rc = platform_device_add(pdev)) < 0)
goto err_put_dev;
diff --git a/drivers/clk/imx/clk.c b/drivers/clk/imx/clk.c
index f38506830ccf..49e51e949f71 100644
--- a/drivers/clk/imx/clk.c
+++ b/drivers/clk/imx/clk.c
@@ -95,11 +95,16 @@ struct clk_hw *imx_obtain_fixed_clock_hw(
const char *name, unsigned long rate)
{
struct clk *clk;
+ struct clk_hw *hw;
clk = imx_obtain_fixed_clock_from_dt(name);
if (IS_ERR(clk))
- clk = imx_clk_fixed(name, rate);
- return __clk_get_hw(clk);
+ return __clk_get_hw(imx_clk_fixed(name, rate));
+
+ hw = __clk_get_hw(clk);
+ clk_put(clk);
+
+ return hw;
}
struct clk_hw *imx_obtain_fixed_of_clock(struct device_node *np,
@@ -119,12 +124,16 @@ struct clk_hw *imx_obtain_fixed_of_clock(struct device_node *np,
struct clk_hw *imx_get_clk_hw_by_name(struct device_node *np, const char *name)
{
struct clk *clk;
+ struct clk_hw *hw;
clk = of_clk_get_by_name(np, name);
if (IS_ERR(clk))
return ERR_PTR(-ENOENT);
- return __clk_get_hw(clk);
+ hw = __clk_get_hw(clk);
+ clk_put(clk);
+
+ return hw;
}
EXPORT_SYMBOL_GPL(imx_get_clk_hw_by_name);
diff --git a/drivers/clk/mediatek/Kconfig b/drivers/clk/mediatek/Kconfig
index b059d44577f1..ba1ca9c90dbf 100644
--- a/drivers/clk/mediatek/Kconfig
+++ b/drivers/clk/mediatek/Kconfig
@@ -541,6 +541,7 @@ config COMMON_CLK_MT8173_MFGTOP
tristate "Clock and power driver for MediaTek MT8173 mfgtop"
depends on COMMON_CLK_MT8173
depends on PM
+ depends on OF
default COMMON_CLK_MT8173
select PM_GENERIC_DOMAINS
select PM_GENERIC_DOMAINS_OF
diff --git a/drivers/clk/mstar/clk-msc313-mpll.c b/drivers/clk/mstar/clk-msc313-mpll.c
index 61beb4e87525..bd45aa0aec01 100644
--- a/drivers/clk/mstar/clk-msc313-mpll.c
+++ b/drivers/clk/mstar/clk-msc313-mpll.c
@@ -105,7 +105,7 @@ static int msc313_mpll_probe(struct platform_device *pdev)
return PTR_ERR(mpll->loop_div_second);
mpll->clk_data = devm_kzalloc(dev, struct_size(mpll->clk_data, hws,
- ARRAY_SIZE(output_dividers)), GFP_KERNEL);
+ NUMOUTPUTS), GFP_KERNEL);
if (!mpll->clk_data)
return -ENOMEM;
diff --git a/drivers/clk/mvebu/ap-cpu-clk.c b/drivers/clk/mvebu/ap-cpu-clk.c
index 1e44ace7d951..ed3ca59e0ee9 100644
--- a/drivers/clk/mvebu/ap-cpu-clk.c
+++ b/drivers/clk/mvebu/ap-cpu-clk.c
@@ -284,6 +284,8 @@ static int ap_cpu_clock_probe(struct platform_device *pdev)
if (!ap_cpu_data)
return -ENOMEM;
+ ap_cpu_data->num = nclusters;
+
for_each_of_cpu_node(dn) {
char *clk_name = "cpu-cluster-0";
struct clk_init_data init;
@@ -333,8 +335,6 @@ static int ap_cpu_clock_probe(struct platform_device *pdev)
ap_cpu_data->hws[cluster_index] = &ap_cpu_clk[cluster_index].hw;
}
- ap_cpu_data->num = cluster_index + 1;
-
ret = of_clk_add_hw_provider(np, of_clk_hw_onecell_get, ap_cpu_data);
if (ret)
dev_err(dev, "failed to register OF clock provider\n");
diff --git a/drivers/clk/qcom/gcc-ipq5018.c b/drivers/clk/qcom/gcc-ipq5018.c
index 594dae3bac4c..add6a53d4df5 100644
--- a/drivers/clk/qcom/gcc-ipq5018.c
+++ b/drivers/clk/qcom/gcc-ipq5018.c
@@ -68,6 +68,15 @@ static struct clk_alpha_pll gpll0_main = {
.parent_data = gcc_xo_data,
.num_parents = ARRAY_SIZE(gcc_xo_data),
.ops = &clk_alpha_pll_stromer_ops,
+ /*
+ * During system boot, this PLL feeds the CPUs.
+ * Mark it as critical to ensure that CCF does
+ * not disable it, even if there are no active
+ * consumers. This is needed to avoid a system
+ * hang caused by turning off the clock driving
+ * the CPUs.
+ */
+ .flags = CLK_IS_CRITICAL,
},
},
};
@@ -1260,6 +1269,7 @@ static struct clk_rcg2 usb0_master_clk_src = {
};
static const struct freq_tbl ftbl_usb0_mock_utmi_clk_src[] = {
+ F(24000000, P_XO, 1, 0, 0),
F(60000000, P_GPLL4, 10, 1, 2),
{ }
};
diff --git a/drivers/clk/qcom/gcc-ipq5210.c b/drivers/clk/qcom/gcc-ipq5210.c
index 3a786a21bdff..76adfa8bf2ad 100644
--- a/drivers/clk/qcom/gcc-ipq5210.c
+++ b/drivers/clk/qcom/gcc-ipq5210.c
@@ -716,6 +716,7 @@ static struct clk_rcg2 gcc_usb0_master_clk_src = {
};
static const struct freq_tbl ftbl_gcc_usb0_mock_utmi_clk_src[] = {
+ F(24000000, P_XO, 1, 0, 0),
F(60000000, P_GPLL4_OUT_AUX, 10, 1, 2),
{ }
};
diff --git a/drivers/clk/qcom/gcc-ipq5332.c b/drivers/clk/qcom/gcc-ipq5332.c
index ecd9ebeed754..4625b447c235 100644
--- a/drivers/clk/qcom/gcc-ipq5332.c
+++ b/drivers/clk/qcom/gcc-ipq5332.c
@@ -1038,6 +1038,7 @@ static struct clk_rcg2 gcc_usb0_master_clk_src = {
};
static const struct freq_tbl ftbl_gcc_usb0_mock_utmi_clk_src[] = {
+ F(24000000, P_XO, 1, 0, 0),
F(60000000, P_GPLL4_OUT_AUX, 10, 1, 2),
{ }
};
diff --git a/drivers/clk/qcom/ipq-cmn-pll.c b/drivers/clk/qcom/ipq-cmn-pll.c
index dafe8c1738df..a9abad9ff4e5 100644
--- a/drivers/clk/qcom/ipq-cmn-pll.c
+++ b/drivers/clk/qcom/ipq-cmn-pll.c
@@ -380,6 +380,8 @@ static int ipq_cmn_pll_register_clks(struct platform_device *pdev)
if (!hw_data)
return -ENOMEM;
+ hw_data->num = num_clks + 1;
+
/*
* Register the CMN PLL clock, which is the parent clock of
* the fixed rate output clocks.
@@ -406,7 +408,6 @@ static int ipq_cmn_pll_register_clks(struct platform_device *pdev)
* is configured to 12 GHZ by DT property assigned-clock-rates-u64.
*/
hw_data->hws[CMN_PLL_CLK] = cmn_pll_hw;
- hw_data->num = num_clks + 1;
ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get, hw_data);
if (ret)
diff --git a/drivers/clk/starfive/clk-starfive-jh7110-isp.c b/drivers/clk/starfive/clk-starfive-jh7110-isp.c
index f3fa069db193..c02c8b29a123 100644
--- a/drivers/clk/starfive/clk-starfive-jh7110-isp.c
+++ b/drivers/clk/starfive/clk-starfive-jh7110-isp.c
@@ -130,7 +130,7 @@ static int jh7110_ispcrg_probe(struct platform_device *pdev)
/* enable power domain and clocks */
pm_runtime_enable(priv->dev);
- ret = pm_runtime_get_sync(priv->dev);
+ ret = pm_runtime_resume_and_get(priv->dev);
if (ret < 0)
return dev_err_probe(priv->dev, ret, "failed to turn on power\n");
diff --git a/drivers/clk/tenstorrent/atlantis-prcm.c b/drivers/clk/tenstorrent/atlantis-prcm.c
index 6d4386eeb7da..bf244b1fbfc1 100644
--- a/drivers/clk/tenstorrent/atlantis-prcm.c
+++ b/drivers/clk/tenstorrent/atlantis-prcm.c
@@ -397,22 +397,21 @@ static int atlantis_clk_gate_shared_enable(struct clk_hw *hw)
scoped_guard(spinlock_irqsave, gate->config.refcount_lock)
{
- need_enable = (*gate->config.share_count)++ == 0;
+ need_enable = (*gate->config.share_count) == 0;
if (need_enable) {
regmap_set_bits(gate->common.regmap,
gate->config.reg_offset,
gate->config.enable);
- }
- }
- if (need_enable) {
- if (!regmap_test_bits(gate->common.regmap,
- gate->config.reg_offset,
- gate->config.enable)) {
- pr_warn("%s: gate enable %d failed to enable\n",
- clk_hw_get_name(hw), gate->config.enable);
- return -EIO;
+ if (!regmap_test_bits(gate->common.regmap,
+ gate->config.reg_offset,
+ gate->config.enable)) {
+ pr_warn("%s: gate enable %d failed to enable\n",
+ clk_hw_get_name(hw), gate->config.enable);
+ return -EIO;
+ }
}
+ (*gate->config.share_count)++;
}
return 0;
@@ -789,13 +788,14 @@ static int atlantis_prcm_clocks_register(struct device *dev,
{
struct clk_hw_onecell_data *clk_data;
int i, ret;
- size_t num_clks = data->num;
clk_data = devm_kzalloc(dev, struct_size(clk_data, hws, data->num),
GFP_KERNEL);
if (!clk_data)
return -ENOMEM;
+ clk_data->num = data->num;
+
for (i = 0; i < data->num; i++) {
struct clk_hw *hw = data->hws[i];
struct atlantis_clk_common *common =
@@ -809,8 +809,6 @@ static int atlantis_prcm_clocks_register(struct device *dev,
clk_data->hws[common->clkid] = hw;
}
- clk_data->num = num_clks;
-
return devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get, clk_data);
}
diff --git a/drivers/clocksource/timer-digicolor.c b/drivers/clocksource/timer-digicolor.c
index 559aa96089c3..46e0c329957a 100644
--- a/drivers/clocksource/timer-digicolor.c
+++ b/drivers/clocksource/timer-digicolor.c
@@ -190,6 +190,8 @@ static int __init digicolor_timer_init(struct device_node *node)
&dc_timer_dev.ce);
if (ret) {
pr_warn("request of timer irq %d failed (%d)\n", irq, ret);
+ clk_disable_unprepare(clk);
+ clk_put(clk);
return ret;
}
diff --git a/drivers/clocksource/timer-fttmr010.c b/drivers/clocksource/timer-fttmr010.c
index 308f94bfbaed..c2553851e8d6 100644
--- a/drivers/clocksource/timer-fttmr010.c
+++ b/drivers/clocksource/timer-fttmr010.c
@@ -292,6 +292,7 @@ static int __init fttmr010_common_init(struct device_node *np,
ret = clk_prepare_enable(clk);
if (ret) {
pr_err("failed to enable PCLK\n");
+ clk_put(clk);
return ret;
}
@@ -432,6 +433,7 @@ static int __init fttmr010_common_init(struct device_node *np,
kfree(fttmr010);
out_disable_clock:
clk_disable_unprepare(clk);
+ clk_put(clk);
return ret;
}
diff --git a/drivers/cpufreq/Kconfig b/drivers/cpufreq/Kconfig
index db83f3365698..edc1299098d4 100644
--- a/drivers/cpufreq/Kconfig
+++ b/drivers/cpufreq/Kconfig
@@ -288,6 +288,7 @@ endif
if LOONGARCH
config LOONGSON3_CPUFREQ
tristate "Loongson3 CPUFreq Driver"
+ depends on MACH_LOONGSON64
help
This option adds a CPUFreq driver for Loongson processors which
support software configurable cpu frequency.
diff --git a/drivers/cpufreq/airoha-cpufreq.c b/drivers/cpufreq/airoha-cpufreq.c
index 3e7770860d13..8283c528d0c1 100644
--- a/drivers/cpufreq/airoha-cpufreq.c
+++ b/drivers/cpufreq/airoha-cpufreq.c
@@ -64,7 +64,7 @@ static int airoha_cpufreq_probe(struct platform_device *pdev)
/* Attach PM for OPP */
ret = dev_pm_domain_attach_list(cpu_dev, &attach_data,
&priv->pd_list);
- if (ret)
+ if (ret < 0)
goto clear_opp_config;
cpufreq_dt = platform_device_register_simple("cpufreq-dt", -1, NULL, 0);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index 8bfd46d60843..6e00d200920a 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -524,8 +524,10 @@ static int shmem_init_perf(struct amd_cpudata *cpudata)
return 0;
}
- ret = cppc_set_auto_sel(cpudata->cpu,
- (cppc_state == AMD_PSTATE_PASSIVE) ? 0 : 1);
+ if ((cppc_state != AMD_PSTATE_PASSIVE) == auto_sel)
+ return 0;
+
+ ret = cppc_set_auto_sel(cpudata->cpu, !auto_sel);
if (ret)
pr_warn("failed to set auto_sel, ret: %d\n", ret);
diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 80893844353c..5f20c4a7ba2f 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -18,6 +18,7 @@
#include <linux/cpufreq.h>
#include <linux/irq_work.h>
#include <linux/kthread.h>
+#include <linux/mutex.h>
#include <linux/time.h>
#include <linux/vmalloc.h>
#include <uapi/linux/sched/types.h>
@@ -49,10 +50,12 @@ struct cppc_freq_invariance {
static DEFINE_PER_CPU(struct cppc_freq_invariance, cppc_freq_inv);
static struct kthread_worker *kworker_fie;
+static DEFINE_MUTEX(cppc_fie_lock);
static int cppc_perf_from_fbctrs(u64 reference_perf,
struct cppc_perf_fb_ctrs *fb_ctrs_t0,
struct cppc_perf_fb_ctrs *fb_ctrs_t1);
+static int cppc_fie_kworker_init(void);
/**
* __cppc_scale_freq_tick - CPPC arch_freq_scale updater for frequency invariance
@@ -161,6 +164,9 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
cppc_fi->cpu = cpu;
cppc_fi->cpu_data = policy->driver_data;
if (cppc_perf_ctrs_in_pcc_cpu(cpu)) {
+ if (cppc_fie_kworker_init())
+ return;
+
kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
sftd = &cppc_sftd_pcc;
@@ -211,8 +217,9 @@ static void cppc_cpufreq_cpu_fie_exit(struct cpufreq_policy *policy)
}
}
-static void cppc_fie_kworker_init(void)
+static int cppc_fie_kworker_init(void)
{
+ struct kthread_worker *worker;
struct sched_attr attr = {
.size = sizeof(struct sched_attr),
.sched_policy = SCHED_DEADLINE,
@@ -228,23 +235,28 @@ static void cppc_fie_kworker_init(void)
};
int ret;
- kworker_fie = kthread_run_worker(0, "cppc_fie");
- if (IS_ERR(kworker_fie)) {
+ guard(mutex)(&cppc_fie_lock);
+
+ if (kworker_fie)
+ return 0;
+
+ worker = kthread_run_worker(0, "cppc_fie");
+ if (IS_ERR(worker)) {
pr_warn("%s: failed to create kworker_fie: %ld\n", __func__,
- PTR_ERR(kworker_fie));
- fie_disabled = FIE_DISABLED;
- kworker_fie = NULL;
- return;
+ PTR_ERR(worker));
+ return PTR_ERR(worker);
}
- ret = sched_setattr_nocheck(kworker_fie->task, &attr);
+ ret = sched_setattr_nocheck(worker->task, &attr);
if (ret) {
pr_warn("%s: failed to set SCHED_DEADLINE: %d\n", __func__,
ret);
- kthread_destroy_worker(kworker_fie);
- fie_disabled = FIE_DISABLED;
- kworker_fie = NULL;
+ kthread_destroy_worker(worker);
+ return ret;
}
+
+ kworker_fie = worker;
+ return 0;
}
static void __init cppc_freq_invariance_init(void)
@@ -259,11 +271,6 @@ static void __init cppc_freq_invariance_init(void)
fie_disabled = FIE_ENABLED;
}
}
-
- if (fie_disabled || !perf_ctrs_in_pcc)
- return;
-
- cppc_fie_kworker_init();
}
static void cppc_freq_invariance_exit(void)
diff --git a/drivers/cpufreq/cpufreq_conservative.c b/drivers/cpufreq/cpufreq_conservative.c
index 0b32ae28ec85..02bfd46543e9 100644
--- a/drivers/cpufreq/cpufreq_conservative.c
+++ b/drivers/cpufreq/cpufreq_conservative.c
@@ -85,10 +85,12 @@ static unsigned int cs_dbs_update(struct cpufreq_policy *policy)
freq_step = get_freq_step(cs_tuners, policy);
/*
- * Decrease requested_freq one freq_step for each idle period that
- * we didn't update the frequency.
+ * Apply deferred down steps only when the policy sample load is
+ * below down_threshold. Otherwise, multiple deferred down steps may
+ * cause a net frequency decrease outside the downscaling region.
*/
- if (policy_dbs->idle_periods < UINT_MAX) {
+ if (policy_dbs->max_sample_load < cs_tuners->down_threshold &&
+ policy_dbs->idle_periods < UINT_MAX) {
unsigned int freq_steps = policy_dbs->idle_periods * freq_step;
if (requested_freq > policy->min + freq_steps)
diff --git a/drivers/cpufreq/cpufreq_governor.c b/drivers/cpufreq/cpufreq_governor.c
index 710d93ec89b5..f3bd3dbe7c41 100644
--- a/drivers/cpufreq/cpufreq_governor.c
+++ b/drivers/cpufreq/cpufreq_governor.c
@@ -124,7 +124,8 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
struct policy_dbs_info *policy_dbs = policy->governor_data;
struct dbs_data *dbs_data = policy_dbs->dbs_data;
unsigned int ignore_nice = dbs_data->ignore_nice_load;
- unsigned int max_load = 0, idle_periods = UINT_MAX;
+ unsigned int max_load = 0, max_sample_load = 0;
+ unsigned int idle_periods = UINT_MAX;
unsigned int sampling_rate, io_busy, j;
u64 cur_nice;
@@ -147,7 +148,7 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
struct cpu_dbs_info *j_cdbs = &per_cpu(cpu_dbs, j);
u64 update_time, cur_idle_time;
unsigned int idle_time, time_elapsed;
- unsigned int load;
+ unsigned int load, sample_load;
cur_idle_time = get_cpu_idle_time(j, &update_time, io_busy);
@@ -186,6 +187,20 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
j_cdbs->prev_cpu_nice = cur_nice;
+ /*
+ * Compute the sample load separately from the prev_load value
+ * that may be reused after a long idle interval. The conservative
+ * governor uses it to decide whether to apply deferred down steps.
+ * If no time has elapsed, retain the existing behavior and use
+ * prev_load.
+ */
+ if (unlikely(!time_elapsed))
+ sample_load = j_cdbs->prev_load;
+ else if (time_elapsed > idle_time)
+ sample_load = 100 * (time_elapsed - idle_time) / time_elapsed;
+ else
+ sample_load = 0;
+
if (unlikely(!time_elapsed)) {
/*
* That can only happen when this function is called
@@ -220,11 +235,7 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
load = j_cdbs->prev_load;
j_cdbs->prev_load = 0;
} else {
- if (time_elapsed > idle_time)
- load = 100 * (time_elapsed - idle_time) / time_elapsed;
- else
- load = 0;
-
+ load = sample_load;
j_cdbs->prev_load = load;
}
@@ -237,9 +248,13 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
if (load > max_load)
max_load = load;
+
+ if (sample_load > max_sample_load)
+ max_sample_load = sample_load;
}
policy_dbs->idle_periods = idle_periods;
+ policy_dbs->max_sample_load = max_sample_load;
return max_load;
}
diff --git a/drivers/cpufreq/cpufreq_governor.h b/drivers/cpufreq/cpufreq_governor.h
index 73b8ed7cfaae..806d8fb4dff1 100644
--- a/drivers/cpufreq/cpufreq_governor.h
+++ b/drivers/cpufreq/cpufreq_governor.h
@@ -95,6 +95,8 @@ struct policy_dbs_info {
/* Multiplier for increasing sample delay temporarily. */
unsigned int rate_mult;
unsigned int idle_periods; /* For conservative */
+ /* Maximum load from the current policy sample. */
+ unsigned int max_sample_load;
/* Status indicators */
bool is_shared; /* This object is used by multiple CPUs */
bool work_in_progress; /* Work is being queued up or in progress */
diff --git a/drivers/cpufreq/cpufreq_stats.c b/drivers/cpufreq/cpufreq_stats.c
index 670a10c3ecfa..e7ec16bcb67e 100644
--- a/drivers/cpufreq/cpufreq_stats.c
+++ b/drivers/cpufreq/cpufreq_stats.c
@@ -52,6 +52,10 @@ static void cpufreq_stats_reset_table(struct cpufreq_stats *stats)
* reset_pending accesses in cpufreq_stats_record_transition().
*/
smp_rmb();
+
+ if (stats->last_index == -1)
+ return;
+
cpufreq_stats_update(stats, READ_ONCE(stats->reset_time));
}
diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 17158cb25dd8..0e6ae4cdda5d 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -3436,6 +3436,8 @@ static void intel_pstate_driver_cleanup(void)
if (intel_pstate_driver == &intel_pstate)
intel_pstate_clear_update_util_hook(cpu);
+ intel_pstate_disable_hwp_interrupt(all_cpu_data[cpu]);
+
kfree(all_cpu_data[cpu]);
WRITE_ONCE(all_cpu_data[cpu], NULL);
}
diff --git a/drivers/cpufreq/loongson3_cpufreq.c b/drivers/cpufreq/loongson3_cpufreq.c
index 1e8715ea1b77..aea7c31e155a 100644
--- a/drivers/cpufreq/loongson3_cpufreq.c
+++ b/drivers/cpufreq/loongson3_cpufreq.c
@@ -16,14 +16,16 @@
#include <asm/idle.h>
#include <asm/loongarch.h>
#include <asm/loongson.h>
+#include <asm/smp.h>
+#include <asm/numa.h>
/* Message */
union smc_message {
u32 value;
struct {
- u32 id : 4;
+ u32 id : 8;
u32 info : 4;
- u32 val : 16;
+ u32 val : 12;
u32 cmd : 6;
u32 extra : 1;
u32 complete : 1;
@@ -164,27 +166,47 @@ union smc_message {
#define FREQ_MAX_LEVEL 16
+#define MMIO_SMCMBX(node) \
+ ((void __iomem *)(nid_to_addrbase(node) | IO_BASE | LOONGSON_REG_BASE | LOONGARCH_IOCSR_SMCMBX))
+
+#define MMIO_MISC_FUNC(node) \
+ ((void __iomem *)(nid_to_addrbase(node) | IO_BASE | LOONGSON_REG_BASE | LOONGARCH_IOCSR_MISC_FUNC))
+
struct loongson3_freq_data {
unsigned int def_freq_level;
struct cpufreq_frequency_table table[];
};
-static struct mutex cpufreq_mutex[MAX_PACKAGES];
+static struct mutex cpufreq_mutex[MAX_NUMNODES];
static struct cpufreq_driver loongson3_cpufreq_driver;
static DEFINE_PER_CPU(struct loongson3_freq_data *, freq_data);
static inline int do_service_request(u32 id, u32 info, u32 cmd, u32 val, u32 extra)
{
int retries;
- unsigned int cpu = raw_smp_processor_id();
- unsigned int package = cpu_data[cpu].package;
+ unsigned int cpu = 0, nid;
union smc_message msg, last;
- mutex_lock(&cpufreq_mutex[package]);
+#ifdef CONFIG_SMP
+ switch (cmd) {
+ case CMD_GET_FREQ_INFO:
+ case CMD_SET_FREQ_INFO:
+ case CMD_GET_FREQ_LEVEL_NUM:
+ case CMD_GET_FREQ_LEVEL_INFO:
+ case CMD_GET_FREQ_BOOST_LEVEL:
+ cpu = cpu_number_map(id);
+ break;
+ default:
+ cpu = raw_smp_processor_id();
+ }
+#endif
+ nid = cpu_to_node(cpu);
- last.value = iocsr_read32(LOONGARCH_IOCSR_SMCMBX);
+ mutex_lock(&cpufreq_mutex[nid]);
+
+ last.value = readl(MMIO_SMCMBX(nid));
if (!last.complete) {
- mutex_unlock(&cpufreq_mutex[package]);
+ mutex_unlock(&cpufreq_mutex[nid]);
return -EPERM;
}
@@ -195,12 +217,11 @@ static inline int do_service_request(u32 id, u32 info, u32 cmd, u32 val, u32 ext
msg.extra = extra;
msg.complete = 0;
- iocsr_write32(msg.value, LOONGARCH_IOCSR_SMCMBX);
- iocsr_write32(iocsr_read32(LOONGARCH_IOCSR_MISC_FUNC) | IOCSR_MISC_FUNC_SOFT_INT,
- LOONGARCH_IOCSR_MISC_FUNC);
+ writel(msg.value, MMIO_SMCMBX(nid));
+ writel(readl(MMIO_MISC_FUNC(nid)) | IOCSR_MISC_FUNC_SOFT_INT, MMIO_MISC_FUNC(nid));
for (retries = 0; retries < 10000; retries++) {
- msg.value = iocsr_read32(LOONGARCH_IOCSR_SMCMBX);
+ msg.value = readl(MMIO_SMCMBX(nid));
if (msg.complete)
break;
@@ -208,49 +229,51 @@ static inline int do_service_request(u32 id, u32 info, u32 cmd, u32 val, u32 ext
}
if (!msg.complete || msg.cmd != CMD_OK) {
- mutex_unlock(&cpufreq_mutex[package]);
+ mutex_unlock(&cpufreq_mutex[nid]);
return -EPERM;
}
- mutex_unlock(&cpufreq_mutex[package]);
+ mutex_unlock(&cpufreq_mutex[nid]);
return msg.val;
}
static unsigned int loongson3_cpufreq_get(unsigned int cpu)
{
- int ret;
+ int ret, core = cpu_logical_map(cpu);
- ret = do_service_request(cpu, FREQ_INFO_TYPE_FREQ, CMD_GET_FREQ_INFO, 0, 0);
+ ret = do_service_request(core, FREQ_INFO_TYPE_FREQ, CMD_GET_FREQ_INFO, 0, 0);
- return ret * KILO;
+ return (ret < 0) ? 0 : ret * KILO;
}
static int loongson3_cpufreq_target(struct cpufreq_policy *policy, unsigned int index)
{
- int ret;
+ int ret, core = cpu_logical_map(policy->cpu);
- ret = do_service_request(cpu_data[policy->cpu].core,
- FREQ_INFO_TYPE_LEVEL, CMD_SET_FREQ_INFO, index, 0);
+ ret = do_service_request(core, FREQ_INFO_TYPE_LEVEL, CMD_SET_FREQ_INFO,
+ index, 0);
return (ret >= 0) ? 0 : ret;
}
static int configure_freq_table(int cpu)
{
- int i, ret, boost_level, max_level, freq_level;
+ int i, ret, core, boost_level, max_level, freq_level;
struct platform_device *pdev = cpufreq_get_driver_data();
struct loongson3_freq_data *data;
if (per_cpu(freq_data, cpu))
return 0;
- ret = do_service_request(cpu, 0, CMD_GET_FREQ_LEVEL_NUM, 0, 0);
+ core = cpu_logical_map(cpu);
+
+ ret = do_service_request(core, 0, CMD_GET_FREQ_LEVEL_NUM, 0, 0);
if (ret < 0)
return ret;
max_level = ret;
- ret = do_service_request(cpu, 0, CMD_GET_FREQ_BOOST_LEVEL, 0, 0);
+ ret = do_service_request(core, 0, CMD_GET_FREQ_BOOST_LEVEL, 0, 0);
if (ret < 0)
return ret;
boost_level = ret;
@@ -263,7 +286,7 @@ static int configure_freq_table(int cpu)
data->def_freq_level = boost_level - 1;
for (i = 0; i < freq_level; i++) {
- ret = do_service_request(cpu, FREQ_INFO_TYPE_FREQ, CMD_GET_FREQ_LEVEL_INFO, i, 0);
+ ret = do_service_request(core, FREQ_INFO_TYPE_FREQ, CMD_GET_FREQ_LEVEL_INFO, i, 0);
if (ret < 0) {
devm_kfree(&pdev->dev, data);
return ret;
@@ -337,7 +360,7 @@ static int loongson3_cpufreq_probe(struct platform_device *pdev)
{
int i, ret;
- for (i = 0; i < MAX_PACKAGES; i++) {
+ for (i = 0; i < MAX_NUMNODES; i++) {
ret = devm_mutex_init(&pdev->dev, &cpufreq_mutex[i]);
if (ret)
return ret;
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 169c6eeb51e5..09d3ccd92b80 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -37,12 +37,13 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
}
- dev_err(mc->dev, "%s %d Got irq for flow %d but ctrl is empty\n", __func__, irq, flow);
+ dev_err_ratelimited(mc->dev, "%s %d Got irq for flow %d but ctrl is empty\n", __func__, irq, flow);
+ return IRQ_NONE;
}
}
- dev_err(mc->dev, "%s %d from unknown irq\n", __func__, irq);
- return IRQ_HANDLED;
+ dev_err_ratelimited(mc->dev, "%s %d from unknown irq\n", __func__, irq);
+ return IRQ_NONE;
}
static struct meson_alg_template mc_algs[] = {
diff --git a/drivers/crypto/atmel-aes.c b/drivers/crypto/atmel-aes.c
index 50d97cef75b8..8e4de859cadc 100644
--- a/drivers/crypto/atmel-aes.c
+++ b/drivers/crypto/atmel-aes.c
@@ -1267,7 +1267,7 @@ static int atmel_aes_gcm_ghash_init(struct atmel_aes_dev *dd)
struct atmel_aes_gcm_ctx *ctx = atmel_aes_gcm_ctx_cast(dd->ctx);
/* Set the data length. */
- atmel_aes_write(dd, AES_AADLENR, dd->total);
+ atmel_aes_write(dd, AES_AADLENR, dd->datalen);
atmel_aes_write(dd, AES_CLENR, 0);
/* If needed, overwrite the GCM Intermediate Hash Word Registers */
diff --git a/drivers/crypto/atmel-tdes.c b/drivers/crypto/atmel-tdes.c
index ffea1db74651..ed80423b4209 100644
--- a/drivers/crypto/atmel-tdes.c
+++ b/drivers/crypto/atmel-tdes.c
@@ -370,6 +370,8 @@ static int atmel_tdes_crypt_pdc(struct atmel_tdes_dev *dd,
if (!(dd->flags & TDES_FLAGS_FAST)) {
dma_sync_single_for_device(dd->dev, dma_addr_in, length,
DMA_TO_DEVICE);
+ dma_sync_single_for_device(dd->dev, dma_addr_out, length,
+ DMA_FROM_DEVICE);
}
len32 = DIV_ROUND_UP(length, sizeof(u32));
@@ -402,6 +404,8 @@ static int atmel_tdes_crypt_dma(struct atmel_tdes_dev *dd,
if (!(dd->flags & TDES_FLAGS_FAST)) {
dma_sync_single_for_device(dd->dev, dma_addr_in, length,
DMA_TO_DEVICE);
+ dma_sync_single_for_device(dd->dev, dma_addr_out, length,
+ DMA_FROM_DEVICE);
}
addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
@@ -933,7 +937,7 @@ static int atmel_tdes_probe(struct platform_device *pdev)
struct resource *tdes_res;
int err;
- tdes_dd = devm_kmalloc(&pdev->dev, sizeof(*tdes_dd), GFP_KERNEL);
+ tdes_dd = devm_kzalloc(&pdev->dev, sizeof(*tdes_dd), GFP_KERNEL);
if (!tdes_dd)
return -ENOMEM;
diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
index cb001aa1de66..ac515ed8c575 100644
--- a/drivers/crypto/caam/caampkc.c
+++ b/drivers/crypto/caam/caampkc.c
@@ -1033,16 +1033,22 @@ static int caam_rsa_set_priv_key_form(struct caam_rsa_ctx *ctx,
free_dq:
kfree_sensitive(rsa_key->dq);
+ rsa_key->dq = NULL;
free_dp:
kfree_sensitive(rsa_key->dp);
+ rsa_key->dp = NULL;
free_tmp2:
kfree_sensitive(rsa_key->tmp2);
+ rsa_key->tmp2 = NULL;
free_tmp1:
kfree_sensitive(rsa_key->tmp1);
+ rsa_key->tmp1 = NULL;
free_q:
kfree_sensitive(rsa_key->q);
+ rsa_key->q = NULL;
free_p:
kfree_sensitive(rsa_key->p);
+ rsa_key->p = NULL;
return -ENOMEM;
}
diff --git a/drivers/crypto/ccp/dbc.c b/drivers/crypto/ccp/dbc.c
index 410084a9039c..4db49a14550c 100644
--- a/drivers/crypto/ccp/dbc.c
+++ b/drivers/crypto/ccp/dbc.c
@@ -236,6 +236,8 @@ int dbc_dev_init(struct psp_device *psp)
goto cleanup_mbox;
}
+ mutex_init(&dbc_dev->ioctl_mutex);
+
dbc_dev->char_dev.minor = MISC_DYNAMIC_MINOR;
dbc_dev->char_dev.name = "dbc";
dbc_dev->char_dev.fops = &dbc_fops;
@@ -244,8 +246,6 @@ int dbc_dev_init(struct psp_device *psp)
if (ret)
goto cleanup_mbox;
- mutex_init(&dbc_dev->ioctl_mutex);
-
return 0;
cleanup_mbox:
diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c
index 09077abbf6ad..0861f06da8ee 100644
--- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c
+++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c
@@ -138,6 +138,7 @@ struct hpre_asym_request {
int err;
hpre_cb cb;
struct timespec64 req_time;
+ u32 flags;
};
static inline unsigned int hpre_align_sz(void)
@@ -186,6 +187,7 @@ static int hpre_prepare_dma_buf(struct hpre_asym_request *hpre_req,
struct scatterlist *data, unsigned int len,
int is_src, dma_addr_t *tmp)
{
+ gfp_t gfp = hpre_req->flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC;
struct hpre_ctx *ctx = hpre_req->ctx;
struct device *dev = ctx->dev;
void *ptr;
@@ -195,7 +197,7 @@ static int hpre_prepare_dma_buf(struct hpre_asym_request *hpre_req,
if (unlikely(shift < 0))
return -EINVAL;
- ptr = dma_alloc_coherent(dev, ctx->key_sz, tmp, GFP_ATOMIC);
+ ptr = dma_alloc_coherent(dev, ctx->key_sz, tmp, gfp);
if (unlikely(!ptr))
return -ENOMEM;
@@ -419,6 +421,7 @@ static int hpre_msg_request_set(struct hpre_ctx *ctx, void *req, bool is_rsa)
h_req = PTR_ALIGN(tmp, hpre_align_sz());
h_req->cb = hpre_rsa_cb;
h_req->areq.rsa = akreq;
+ h_req->flags = akreq->base.flags;
msg = &h_req->req;
memset(msg, 0, sizeof(*msg));
} else {
@@ -433,6 +436,7 @@ static int hpre_msg_request_set(struct hpre_ctx *ctx, void *req, bool is_rsa)
h_req = PTR_ALIGN(tmp, hpre_align_sz());
h_req->cb = hpre_dh_cb;
h_req->areq.dh = kreq;
+ h_req->flags = kreq->base.flags;
msg = &h_req->req;
memset(msg, 0, sizeof(*msg));
msg->key = cpu_to_le64(ctx->dh.dma_xa_p);
@@ -720,7 +724,7 @@ static void hpre_dh_exit_tfm(struct crypto_kpp *tfm)
static void hpre_rsa_drop_leading_zeros(const char **ptr, size_t *len)
{
- while (!**ptr && *len) {
+ while (*len && !**ptr) {
(*ptr)++;
(*len)--;
}
@@ -1452,13 +1456,14 @@ static int hpre_ecdh_msg_request_set(struct hpre_ctx *ctx,
if (req->dst_len < ctx->key_sz << 1) {
req->dst_len = ctx->key_sz << 1;
- return -EINVAL;
+ return -EOVERFLOW;
}
tmp = kpp_request_ctx(req);
h_req = PTR_ALIGN(tmp, hpre_align_sz());
h_req->cb = hpre_ecdh_cb;
h_req->areq.ecdh = req;
+ h_req->flags = req->base.flags;
msg = &h_req->req;
memset(msg, 0, sizeof(*msg));
msg->in = cpu_to_le64(DMA_MAPPING_ERROR);
@@ -1477,6 +1482,7 @@ static int hpre_ecdh_msg_request_set(struct hpre_ctx *ctx,
static int hpre_ecdh_src_data_init(struct hpre_asym_request *hpre_req,
struct scatterlist *data, unsigned int len)
{
+ gfp_t gfp = hpre_req->flags & CRYPTO_TFM_REQ_MAY_SLEEP ? GFP_KERNEL : GFP_ATOMIC;
struct hpre_sqe *msg = &hpre_req->req;
struct hpre_ctx *ctx = hpre_req->ctx;
struct device *dev = ctx->dev;
@@ -1490,7 +1496,7 @@ static int hpre_ecdh_src_data_init(struct hpre_asym_request *hpre_req,
if (unlikely(shift < 0))
return -EINVAL;
- ptr = dma_alloc_coherent(dev, ctx->key_sz << 2, &dma, GFP_KERNEL);
+ ptr = dma_alloc_coherent(dev, ctx->key_sz << 2, &dma, gfp);
if (unlikely(!ptr))
return -ENOMEM;
@@ -1512,13 +1518,13 @@ static int hpre_ecdh_dst_data_init(struct hpre_asym_request *hpre_req,
struct device *dev = ctx->dev;
dma_addr_t dma;
- if (unlikely(!data || !sg_is_last(data) || len != ctx->key_sz << 1)) {
- dev_err(dev, "data or data length is illegal!\n");
+ if (unlikely(!data || !sg_is_last(data))) {
+ dev_err(dev, "data is illegal!\n");
return -EINVAL;
}
hpre_req->dst = NULL;
- dma = dma_map_single(dev, sg_virt(data), len, DMA_FROM_DEVICE);
+ dma = dma_map_single(dev, sg_virt(data), ctx->key_sz << 1, DMA_FROM_DEVICE);
if (unlikely(dma_mapping_error(dev, dma))) {
dev_err(dev, "dma map data err!\n");
return -ENOMEM;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
index 161863841a52..4e6d860c9a60 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_dev_mgr.c
@@ -137,6 +137,13 @@ int adf_devmgr_add_dev(struct adf_accel_dev *accel_dev,
mutex_lock(&table_lock);
atomic_set(&accel_dev->ref_count, 0);
+ mutex_init(&accel_dev->state_lock);
+ /*
+ * The VF2PF lock is used to send messages to the PF outside of the
+ * device up/down cycle, so it must live as long as the accel device.
+ */
+ if (accel_dev->is_vf)
+ mutex_init(&accel_dev->vf.vf2pf_lock);
/* PF on host or VF on guest - optimized to remove redundant is_vf */
if (!accel_dev->is_vf || !pf) {
@@ -210,7 +217,6 @@ int adf_devmgr_add_dev(struct adf_accel_dev *accel_dev,
map->attached = true;
list_add_tail(&map->list, &vfs_table);
}
- mutex_init(&accel_dev->state_lock);
unlock:
mutex_unlock(&table_lock);
return ret;
@@ -257,6 +263,8 @@ void adf_devmgr_rm_dev(struct adf_accel_dev *accel_dev,
}
}
unlock:
+ if (accel_dev->is_vf)
+ mutex_destroy(&accel_dev->vf.vf2pf_lock);
mutex_destroy(&accel_dev->state_lock);
list_del(&accel_dev->list);
mutex_unlock(&table_lock);
diff --git a/drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c b/drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
index bf2bae78a40d..d8b961b2f2f7 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_gen4_vf_mig.c
@@ -837,6 +837,9 @@ static int adf_gen4_vfmig_save_state(struct qat_mig_dev *mdev)
return ret;
}
+ memset(mdev->state + mdev->setup_size, 0,
+ mdev->state_size - mdev->setup_size);
+
adf_mstate_mgr_init(vfmig->mstate_mgr, mdev->state + mdev->setup_size,
mdev->state_size - mdev->setup_size);
if (!adf_mstate_preamble_add(vfmig->mstate_mgr))
@@ -934,6 +937,8 @@ static int adf_gen4_vfmig_save_setup(struct qat_mig_dev *mdev)
if (mdev->setup_size)
return 0;
+ memset(mdev->state, 0, mdev->state_size);
+
adf_mstate_mgr_init(vfmig->mstate_mgr, mdev->state, mdev->state_size);
if (!adf_mstate_preamble_add(vfmig->mstate_mgr))
return -EINVAL;
diff --git a/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.c b/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.c
index 967253082a98..902c4b1a23e3 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.c
@@ -323,14 +323,26 @@ static void adf_handle_ti_cd_par_sts(struct adf_accel_dev *accel_dev, void __iom
u32 ti_cd_par_sts;
ti_cd_par_sts = ADF_CSR_RD(csr, ADF_GEN6_TI_CD_PAR_STS);
- ti_cd_par_sts &= ADF_GEN6_TI_CD_PAR_STS_MASK;
+ ti_cd_par_sts &= ADF_GEN6_TI_CD_PAR_STS_TI_INT_ERR_DEVHALT_MASK;
if (ti_cd_par_sts) {
dev_err(&GET_DEV(accel_dev), "TI CD parity error: %#x\n", ti_cd_par_sts);
- ADF_RAS_ERR_CTR_INC(accel_dev->ras_errors, ADF_RAS_UNCORR);
ADF_CSR_WR(csr, ADF_GEN6_TI_CD_PAR_STS, ti_cd_par_sts);
}
}
+static void adf_handle_ti_misc_cd_par_sts(struct adf_accel_dev *accel_dev, void __iomem *csr)
+{
+ u32 ti_misc_cd_par_sts;
+
+ ti_misc_cd_par_sts = ADF_CSR_RD(csr, ADF_GEN6_TI_CD_PAR_STS);
+ ti_misc_cd_par_sts &= ADF_GEN6_TI_CD_PAR_STS_TI_MISC_MASK;
+ if (ti_misc_cd_par_sts) {
+ dev_err(&GET_DEV(accel_dev), "TI MISC CD parity error: %#x\n", ti_misc_cd_par_sts);
+ ADF_RAS_ERR_CTR_INC(accel_dev->ras_errors, ADF_RAS_UNCORR);
+ ADF_CSR_WR(csr, ADF_GEN6_TI_CD_PAR_STS, ti_misc_cd_par_sts);
+ }
+}
+
static void adf_handle_ti_trnsb_par_sts(struct adf_accel_dev *accel_dev, void __iomem *csr)
{
u32 ti_trnsb_par_sts;
@@ -367,7 +379,6 @@ static void adf_handle_ti_err(struct adf_accel_dev *accel_dev, void __iomem *csr
adf_handle_ti_ci_par_sts(accel_dev, csr);
adf_handle_ti_pullfub_par_sts(accel_dev, csr);
adf_handle_ti_pushfub_par_sts(accel_dev, csr);
- adf_handle_ti_cd_par_sts(accel_dev, csr);
adf_handle_ti_trnsb_par_sts(accel_dev, csr);
adf_handle_iosfp_cmd_parerr(accel_dev, csr);
}
@@ -591,6 +602,8 @@ static void adf_handle_timiscsts(struct adf_accel_dev *accel_dev, void __iomem *
if (!(errsou & ADF_GEN6_ERRSOU3_TIMISCSTS_BIT))
return;
+ adf_handle_ti_misc_cd_par_sts(accel_dev, csr);
+
timiscsts = ADF_CSR_RD(csr, ADF_GEN6_TIMISCSTS);
if (timiscsts) {
dev_err(&GET_DEV(accel_dev), "Fatal error in transmit interface: %#x\n",
@@ -730,6 +743,7 @@ static void adf_handle_ti_int_err_devhalt(struct adf_accel_dev *accel_dev, void
if (!(errsou & ADF_GEN6_ERRSOU3_TI_INT_ERR_DEVHALT_BIT))
return;
+ adf_handle_ti_cd_par_sts(accel_dev, csr);
dev_err(&GET_DEV(accel_dev), "DEVHALT due to a TI internal memory error\n");
ADF_RAS_ERR_CTR_INC(accel_dev->ras_errors, ADF_RAS_FATAL);
}
diff --git a/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.h b/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.h
index 66ced271d173..11f7b2986f56 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.h
+++ b/drivers/crypto/intel/qat/qat_common/adf_gen6_ras.h
@@ -170,23 +170,25 @@ struct adf_ras_ops;
#define ADF_GEN6_TI_CD_PAR_ERR_MASK 0x500638
/*
- * TI CD parity status mask
+ * TI CD parity status mask - TI_INT_ERR_DEVHALT bits
* BIT(0) - BIT(15) - CtxMdRam[0:15]_sts parity status
* BIT(16) - Leaf2ClusterRam_sts parity status
* BIT(17) - BIT(18) - Ring2LeafRam[0:1]_sts parity status
- * BIT(19) - VirtualQ_sts parity status
- * BIT(20) - DtRdQ_sts parity status
- * BIT(21) - DtWrQ_sts parity status
- * BIT(22) - RiCmdQ_sts parity status
- * BIT(23) - BypassQ_sts parity status
- * BIT(24) - DtRdQ_sc_sts parity status
- * BIT(25) - DtWrQ_sc_sts parity status
*/
-#define ADF_GEN6_TI_CD_PAR_STS_MASK \
+#define ADF_GEN6_TI_CD_PAR_STS_TI_INT_ERR_DEVHALT_MASK \
(BIT(0) | BIT(1) | BIT(2) | BIT(3) | BIT(4) | BIT(5) | BIT(6) | \
BIT(7) | BIT(8) | BIT(9) | BIT(10) | BIT(11) | BIT(12) | BIT(13) | \
- BIT(14) | BIT(15) | BIT(16) | BIT(17) | BIT(18) | BIT(19) | BIT(20) | \
- BIT(21) | BIT(22) | BIT(23) | BIT(24) | BIT(25))
+ BIT(14) | BIT(15) | BIT(16) | BIT(17) | BIT(18))
+
+/*
+ * TI CD parity status mask - TIMISCSTS bit
+ * BIT(19) - VirtualQ_sts parity status
+ */
+#define ADF_GEN6_TI_CD_PAR_STS_TI_MISC_MASK BIT(19)
+
+#define ADF_GEN6_TI_CD_PAR_STS_MASK \
+ (ADF_GEN6_TI_CD_PAR_STS_TI_INT_ERR_DEVHALT_MASK | \
+ ADF_GEN6_TI_CD_PAR_STS_TI_MISC_MASK)
/* TI TRNSB parity status */
#define ADF_GEN6_TI_TRNSB_PAR_STS 0x500648
diff --git a/drivers/crypto/intel/qat/qat_common/adf_vf_isr.c b/drivers/crypto/intel/qat/qat_common/adf_vf_isr.c
index 9ffaa56def67..1f28655ec051 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_vf_isr.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_vf_isr.c
@@ -118,7 +118,6 @@ static int adf_setup_pf2vf_bh(struct adf_accel_dev *accel_dev)
tasklet_init(&accel_dev->vf.pf2vf_bh_tasklet,
(void *)adf_pf2vf_bh_handler, (unsigned long)accel_dev);
- mutex_init(&accel_dev->vf.vf2pf_lock);
return 0;
}
@@ -126,7 +125,6 @@ static void adf_cleanup_pf2vf_bh(struct adf_accel_dev *accel_dev)
{
tasklet_disable(&accel_dev->vf.pf2vf_bh_tasklet);
tasklet_kill(&accel_dev->vf.pf2vf_bh_tasklet);
- mutex_destroy(&accel_dev->vf.vf2pf_lock);
}
static irqreturn_t adf_isr(int irq, void *privdata)
diff --git a/drivers/crypto/nx/nx-842.c b/drivers/crypto/nx/nx-842.c
index a61208cbcd27..b4404a138962 100644
--- a/drivers/crypto/nx/nx-842.c
+++ b/drivers/crypto/nx/nx-842.c
@@ -421,6 +421,8 @@ static int decompress(struct nx842_crypto_ctx *ctx,
slen -= spadding;
+ if (ignore > dlen)
+ return -EINVAL;
dlen -= ignore;
if (ignore)
pr_debug("ignoring last %x bytes\n", ignore);
diff --git a/drivers/crypto/omap-des.c b/drivers/crypto/omap-des.c
index 3b7104cb4487..4b5a8d13f505 100644
--- a/drivers/crypto/omap-des.c
+++ b/drivers/crypto/omap-des.c
@@ -1082,6 +1082,8 @@ static void omap_des_remove(struct platform_device *pdev)
omap_des_unregister_algs(dd->pdata);
+ crypto_engine_exit(dd->engine);
+
cancel_work_sync(&dd->done_task);
omap_des_dma_cleanup(dd);
pm_runtime_disable(dd->dev);
diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c
index 6d2dd0b88f15..207fc7680c8f 100644
--- a/drivers/crypto/omap-sham.c
+++ b/drivers/crypto/omap-sham.c
@@ -2129,6 +2129,7 @@ static void omap_sham_remove(struct platform_device *pdev)
list_del(&dd->list);
spin_unlock_bh(&sham.lock);
omap_sham_unregister_algs(dd->pdata);
+ crypto_engine_exit(dd->engine);
cancel_work_sync(&dd->done_task);
pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/crypto/sa2ul.c b/drivers/crypto/sa2ul.c
index 9846cbeb3449..255334b6b0db 100644
--- a/drivers/crypto/sa2ul.c
+++ b/drivers/crypto/sa2ul.c
@@ -125,9 +125,7 @@ struct sa_cmdl_cfg {
* @mci_enc: Mode Control Instruction for Encryption algorithm
* @mci_dec: Mode Control Instruction for Decryption
* @inv_key: Whether the encryption algorithm demands key inversion
- * @ctx: Pointer to the algorithm context
* @keyed_mac: Whether the authentication algorithm has key
- * @prep_iopad: Function pointer to generate intermediate ipad/opad
*/
struct algo_data {
struct sa_eng_info enc_eng;
@@ -141,10 +139,7 @@ struct algo_data {
u8 *mci_enc;
u8 *mci_dec;
bool inv_key;
- struct sa_tfm_ctx *ctx;
bool keyed_mac;
- void (*prep_iopad)(struct algo_data *algo, const u8 *key,
- u16 key_sz, __be32 *ipad, __be32 *opad);
};
/**
@@ -381,86 +376,36 @@ static void sa_swiz_128(u8 *in, u16 len)
}
}
-/* Prepare the ipad and opad from key as per SHA algorithm step 1*/
-static void prepare_kipad(u8 *k_ipad, const u8 *key, u16 key_sz)
+static void sa_prepare_iopads(struct algo_data *data, const u8 *key, u16 key_sz,
+ __be32 *ipad, __be32 *opad)
{
- int i;
-
- for (i = 0; i < key_sz; i++)
- k_ipad[i] = key[i] ^ 0x36;
-
- /* Instead of XOR with 0 */
- for (; i < SHA1_BLOCK_SIZE; i++)
- k_ipad[i] = 0x36;
-}
-
-static void prepare_kopad(u8 *k_opad, const u8 *key, u16 key_sz)
-{
- int i;
-
- for (i = 0; i < key_sz; i++)
- k_opad[i] = key[i] ^ 0x5c;
-
- /* Instead of XOR with 0 */
- for (; i < SHA1_BLOCK_SIZE; i++)
- k_opad[i] = 0x5c;
-}
-
-static void sa_export_shash(void *state, struct shash_desc *hash,
- int digest_size, __be32 *out)
-{
- struct sha1_state *sha1;
- struct sha256_state *sha256;
- u32 *result;
-
- switch (digest_size) {
- case SHA1_DIGEST_SIZE:
- sha1 = state;
- result = sha1->state;
+ union {
+ struct hmac_sha1_key sha1;
+ struct hmac_sha256_key sha256;
+ } k;
+
+ switch (data->aalg_id) {
+ case SA_AALG_ID_HMAC_SHA1:
+ hmac_sha1_preparekey(&k.sha1, key, key_sz);
+ cpu_to_be32_array(ipad, k.sha1.istate.h,
+ ARRAY_SIZE(k.sha1.istate.h));
+ cpu_to_be32_array(opad, k.sha1.ostate.h,
+ ARRAY_SIZE(k.sha1.ostate.h));
break;
- case SHA256_DIGEST_SIZE:
- sha256 = state;
- result = sha256->state;
+ case SA_AALG_ID_HMAC_SHA2_256:
+ hmac_sha256_preparekey(&k.sha256, key, key_sz);
+ cpu_to_be32_array(ipad, k.sha256.key.istate.h,
+ ARRAY_SIZE(k.sha256.key.istate.h));
+ cpu_to_be32_array(opad, k.sha256.key.ostate.h,
+ ARRAY_SIZE(k.sha256.key.ostate.h));
break;
default:
- dev_err(sa_k3_dev, "%s: bad digest_size=%d\n", __func__,
- digest_size);
+ dev_err(sa_k3_dev, "%s: bad aalg_id=%d\n", __func__,
+ data->aalg_id);
return;
}
- crypto_shash_export(hash, state);
-
- cpu_to_be32_array(out, result, digest_size / 4);
-}
-
-static void sa_prepare_iopads(struct algo_data *data, const u8 *key,
- u16 key_sz, __be32 *ipad, __be32 *opad)
-{
- SHASH_DESC_ON_STACK(shash, data->ctx->shash);
- int block_size = crypto_shash_blocksize(data->ctx->shash);
- int digest_size = crypto_shash_digestsize(data->ctx->shash);
- union {
- struct sha1_state sha1;
- struct sha256_state sha256;
- u8 k_pad[SHA1_BLOCK_SIZE];
- } sha;
-
- shash->tfm = data->ctx->shash;
-
- prepare_kipad(sha.k_pad, key, key_sz);
-
- crypto_shash_init(shash);
- crypto_shash_update(shash, sha.k_pad, block_size);
- sa_export_shash(&sha, shash, digest_size, ipad);
-
- prepare_kopad(sha.k_pad, key, key_sz);
-
- crypto_shash_init(shash);
- crypto_shash_update(shash, sha.k_pad, block_size);
-
- sa_export_shash(&sha, shash, digest_size, opad);
-
- memzero_explicit(&sha, sizeof(sha));
+ memzero_explicit(&k, sizeof(k));
}
/* Derive the inverse key used in AES-CBC decryption operation */
@@ -544,7 +489,7 @@ static void sa_set_sc_auth(struct algo_data *ad, const u8 *key, u16 key_sz,
/* Copy the keys or ipad/opad */
if (ad->keyed_mac)
- ad->prep_iopad(ad, key, key_sz, ipad, opad);
+ sa_prepare_iopads(ad, key, key_sz, ipad, opad);
else {
/* basic hash */
sc_buf[1] |= SA_BASIC_HASH;
@@ -1697,8 +1642,7 @@ static void sa_aead_dma_in_callback(void *data)
aead_request_complete(req, err);
}
-static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
- const char *fallback)
+static int sa_cra_init_aead(struct crypto_aead *tfm, const char *fallback)
{
struct sa_tfm_ctx *ctx = crypto_aead_ctx(tfm);
struct sa_crypto_data *data = dev_get_drvdata(sa_k3_dev);
@@ -1707,12 +1651,6 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
memzero_explicit(ctx, sizeof(*ctx));
ctx->dev_data = data;
- ctx->shash = crypto_alloc_shash(hash, 0, CRYPTO_ALG_NEED_FALLBACK);
- if (IS_ERR(ctx->shash)) {
- dev_err(sa_k3_dev, "base driver %s couldn't be loaded\n", hash);
- return PTR_ERR(ctx->shash);
- }
-
ctx->fallback.aead = crypto_alloc_aead(fallback, 0,
CRYPTO_ALG_NEED_FALLBACK);
@@ -1744,14 +1682,12 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
static int sa_cra_init_aead_sha1(struct crypto_aead *tfm)
{
- return sa_cra_init_aead(tfm, "sha1",
- "authenc(hmac(sha1),cbc(aes))");
+ return sa_cra_init_aead(tfm, "authenc(hmac(sha1),cbc(aes))");
}
static int sa_cra_init_aead_sha256(struct crypto_aead *tfm)
{
- return sa_cra_init_aead(tfm, "sha256",
- "authenc(hmac(sha256),cbc(aes))");
+ return sa_cra_init_aead(tfm, "authenc(hmac(sha256),cbc(aes))");
}
static void sa_exit_tfm_aead(struct crypto_aead *tfm)
@@ -1759,7 +1695,6 @@ static void sa_exit_tfm_aead(struct crypto_aead *tfm)
struct sa_tfm_ctx *ctx = crypto_aead_ctx(tfm);
struct sa_crypto_data *data = dev_get_drvdata(sa_k3_dev);
- crypto_free_shash(ctx->shash);
crypto_free_aead(ctx->fallback.aead);
sa_free_ctx_info(&ctx->enc, data);
@@ -1785,7 +1720,6 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
if (key_idx >= 3)
return -EINVAL;
- ad->ctx = ctx;
ad->enc_eng.eng_id = SA_ENG_ID_EM1;
ad->enc_eng.sc_size = SA_CTX_ENC_TYPE1_SZ;
ad->auth_eng.eng_id = SA_ENG_ID_AM1;
@@ -1795,7 +1729,6 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
ad->inv_key = true;
ad->keyed_mac = true;
ad->ealg_id = SA_EALG_ID_AES_CBC;
- ad->prep_iopad = sa_prepare_iopads;
memset(&cfg, 0, sizeof(cfg));
cfg.enc = true;
diff --git a/drivers/crypto/xilinx/zynqmp-aes-gcm.c b/drivers/crypto/xilinx/zynqmp-aes-gcm.c
index d54c268dfe34..ebee37dddb1f 100644
--- a/drivers/crypto/xilinx/zynqmp-aes-gcm.c
+++ b/drivers/crypto/xilinx/zynqmp-aes-gcm.c
@@ -766,7 +766,7 @@ static void xilinx_aes_aead_exit(struct crypto_aead *aead)
struct xilinx_aead_tfm_ctx *tfm_ctx = crypto_tfm_ctx(tfm);
dma_unmap_single(tfm_ctx->dev, tfm_ctx->key_dma_addr, AES_KEYSIZE_256, DMA_TO_DEVICE);
- kfree(tfm_ctx->key);
+ kfree_sensitive(tfm_ctx->key);
if (tfm_ctx->fbk_cipher) {
crypto_free_aead(tfm_ctx->fbk_cipher);
tfm_ctx->fbk_cipher = NULL;
diff --git a/drivers/crypto/xilinx/zynqmp-sha.c b/drivers/crypto/xilinx/zynqmp-sha.c
index 5813017b6b79..cd051e1f4275 100644
--- a/drivers/crypto/xilinx/zynqmp-sha.c
+++ b/drivers/crypto/xilinx/zynqmp-sha.c
@@ -59,7 +59,7 @@ static int zynqmp_sha_init_tfm(struct crypto_shash *hash)
return PTR_ERR(fallback_tfm);
if (crypto_shash_descsize(hash) <
- crypto_shash_statesize(tfm_ctx->fbk_tfm)) {
+ crypto_shash_statesize(fallback_tfm)) {
crypto_free_shash(fallback_tfm);
return -EINVAL;
}
diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
index 3b818adbd38b..b7ce6b59c680 100644
--- a/drivers/cxl/acpi.c
+++ b/drivers/cxl/acpi.c
@@ -231,9 +231,10 @@ cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, struct access_coordinate *coord,
int entries, int *qos_class)
{
union acpi_object *out_obj, *out_buf, *obj;
+ /* QTG _DSM expects latencies in picoseconds. */
union acpi_object in_array[4] = {
- [0].integer = { ACPI_TYPE_INTEGER, coord->read_latency },
- [1].integer = { ACPI_TYPE_INTEGER, coord->write_latency },
+ [0].integer = { ACPI_TYPE_INTEGER, coord->read_latency * 1000ULL },
+ [1].integer = { ACPI_TYPE_INTEGER, coord->write_latency * 1000ULL },
[2].integer = { ACPI_TYPE_INTEGER, coord->read_bandwidth },
[3].integer = { ACPI_TYPE_INTEGER, coord->write_bandwidth },
};
@@ -251,6 +252,11 @@ cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, struct access_coordinate *coord,
if (!entries)
return -EINVAL;
+ /* Latency inputs must fit in the DWORD fields defined by the spec. */
+ if (coord->read_latency > U32_MAX / 1000 ||
+ coord->write_latency > U32_MAX / 1000)
+ return -ERANGE;
+
out_obj = acpi_evaluate_dsm(handle, &acpi_cxl_qtg_id_guid, 1, 1, &in_obj);
if (!out_obj)
return -ENXIO;
@@ -885,13 +891,17 @@ static int pair_cxl_resource(struct device *dev, void *data)
static int cxl_acpi_probe(struct platform_device *pdev)
{
- int rc;
+ struct cxl_cfmws_context ctx;
+ struct acpi_device *adev;
struct resource *cxl_res;
struct cxl_root *cxl_root;
struct cxl_port *root_port;
struct device *host = &pdev->dev;
- struct acpi_device *adev = ACPI_COMPANION(host);
- struct cxl_cfmws_context ctx;
+ int rc;
+
+ adev = ACPI_COMPANION(host);
+ if (!adev)
+ return -ENODEV;
device_lock_set_class(&pdev->dev, &cxl_root_key);
rc = devm_add_action_or_reset(&pdev->dev, cxl_acpi_lock_reset_class,
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 55828a836c01..01341f524a93 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -942,11 +942,11 @@ void cxl_event_trace_record(struct cxl_memdev *cxlmd,
if (evt->gen_media.media_hdr.descriptor &
CXL_GMER_EVT_DESC_THRESHOLD_EVENT)
- WARN_ON_ONCE((evt->gen_media.media_hdr.type &
+ WARN_ON_ONCE((evt->gen_media.media_hdr.type ==
CXL_GMER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE) &&
!get_unaligned_le24(evt->gen_media.cme_count));
else
- WARN_ON_ONCE(evt->gen_media.media_hdr.type &
+ WARN_ON_ONCE(evt->gen_media.media_hdr.type ==
CXL_GMER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE);
trace_cxl_general_media(cxlmd, type, cxlr, hpa,
@@ -957,11 +957,11 @@ void cxl_event_trace_record(struct cxl_memdev *cxlmd,
if (evt->dram.media_hdr.descriptor &
CXL_GMER_EVT_DESC_THRESHOLD_EVENT)
- WARN_ON_ONCE((evt->dram.media_hdr.type &
+ WARN_ON_ONCE((evt->dram.media_hdr.type ==
CXL_DER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE) &&
!get_unaligned_le24(evt->dram.cvme_count));
else
- WARN_ON_ONCE(evt->dram.media_hdr.type &
+ WARN_ON_ONCE(evt->dram.media_hdr.type ==
CXL_DER_MEM_EVT_TYPE_AP_CME_COUNTER_EXPIRE);
trace_cxl_dram(cxlmd, type, cxlr, hpa, hpa_alias,
diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c
index 9d807c1a002c..bb6ad0753a42 100644
--- a/drivers/cxl/core/pci.c
+++ b/drivers/cxl/core/pci.c
@@ -660,8 +660,8 @@ long cxl_pci_get_latency(struct pci_dev *pdev)
{
long bw;
- bw = pcie_link_speed_mbps(pdev);
- if (bw < 0)
+ bw = pcie_link_bandwidth_mbps(pdev);
+ if (bw <= 0)
return 0;
bw /= BITS_PER_BYTE;
@@ -756,18 +756,12 @@ EXPORT_SYMBOL_NS_GPL(cxl_pci_setup_regs, "CXL");
int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c)
{
- int speed, bw;
- u16 lnksta;
- u32 width;
-
- speed = pcie_link_speed_mbps(pdev);
- if (speed < 0)
- return speed;
- speed /= BITS_PER_BYTE;
+ int bw;
- pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta);
- width = FIELD_GET(PCI_EXP_LNKSTA_NLW, lnksta);
- bw = speed * width;
+ bw = pcie_link_bandwidth_mbps(pdev);
+ if (bw < 0)
+ return bw;
+ bw /= BITS_PER_BYTE;
for (int i = 0; i < ACCESS_COORDINATE_MAX; i++) {
c[i].read_bandwidth = bw;
@@ -804,7 +798,8 @@ u16 cxl_gpf_get_dvsec(struct device *dev)
return 0;
pdev = to_pci_dev(dev);
- if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ENDPOINT)
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ENDPOINT ||
+ is_cxl_restricted(pdev))
is_port = false;
dvsec = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
@@ -839,7 +834,7 @@ static int update_gpf_port_dvsec(struct pci_dev *pdev, int dvsec, int phase)
rc = pci_read_config_word(pdev, dvsec + offset, &ctrl);
if (rc)
- return rc;
+ return pcibios_err_to_errno(rc);
if (FIELD_GET(base, ctrl) == GPF_TIMEOUT_BASE_MAX &&
FIELD_GET(scale, ctrl) == GPF_TIMEOUT_SCALE_MAX)
@@ -849,11 +844,17 @@ static int update_gpf_port_dvsec(struct pci_dev *pdev, int dvsec, int phase)
ctrl |= FIELD_PREP(scale, GPF_TIMEOUT_SCALE_MAX);
rc = pci_write_config_word(pdev, dvsec + offset, ctrl);
- if (!rc)
- pci_dbg(pdev, "Port GPF phase %d timeout: %d0 secs\n",
- phase, GPF_TIMEOUT_BASE_MAX);
+ if (rc) {
+ rc = pcibios_err_to_errno(rc);
+ pci_warn(pdev, "Port GPF phase %d timeout write failed: %d\n",
+ phase, rc);
+ return rc;
+ }
+
+ pci_dbg(pdev, "Port GPF phase %d timeout: %d0 secs\n",
+ phase, GPF_TIMEOUT_BASE_MAX);
- return rc;
+ return 0;
}
int cxl_gpf_port_setup(struct cxl_dport *dport)
@@ -863,16 +864,22 @@ int cxl_gpf_port_setup(struct cxl_dport *dport)
if (!dport->gpf_dvsec) {
struct pci_dev *pdev;
- int dvsec;
+ int dvsec, rc;
dvsec = cxl_gpf_get_dvsec(dport->dport_dev);
if (!dvsec)
return -EINVAL;
- dport->gpf_dvsec = dvsec;
pdev = to_pci_dev(dport->dport_dev);
- update_gpf_port_dvsec(pdev, dport->gpf_dvsec, 1);
- update_gpf_port_dvsec(pdev, dport->gpf_dvsec, 2);
+ rc = update_gpf_port_dvsec(pdev, dvsec, 1);
+ if (rc)
+ return rc;
+
+ rc = update_gpf_port_dvsec(pdev, dvsec, 2);
+ if (rc)
+ return rc;
+
+ dport->gpf_dvsec = dvsec;
}
return 0;
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index 625e4aa427db..cd9e58d8144f 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -1089,11 +1089,40 @@ static void cond_cxl_root_unlock(struct cxl_port *port)
device_unlock(&port->dev);
}
+static int clear_decoder_target(struct device *dev, void *data)
+{
+ struct cxl_dport *dport = data;
+ struct cxl_switch_decoder *cxlsd;
+
+ if (!is_switch_decoder(dev))
+ return 0;
+
+ cxlsd = to_cxl_switch_decoder(dev);
+ guard(rwsem_write)(&cxl_rwsem.region);
+
+ /*
+ * A dport can occupy more than one position of an interleave, so
+ * scan the whole target list rather than stopping at the first hit.
+ */
+ for (int i = 0; i < cxlsd->nr_targets; i++) {
+ if (cxlsd->target[i] != dport)
+ continue;
+ cxlsd->target[i] = NULL;
+ dev_dbg(dev, "dport%d removed from target list, index %d\n",
+ dport->port_id, i);
+ }
+
+ return 0;
+}
+
static void cxl_dport_remove(void *data)
{
struct cxl_dport *dport = data;
struct cxl_port *port = dport->port;
+ /* counterpart of cxl_port_update_decoder_targets() */
+ device_for_each_child(&port->dev, dport, clear_decoder_target);
+
port->nr_dports--;
xa_erase(&port->dports, (unsigned long) dport->dport_dev);
put_device(dport->dport_dev);
@@ -2320,6 +2349,9 @@ EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, "CXL");
static void add_latency(struct access_coordinate *c, long latency)
{
+ /* Convert link latency from picoseconds to nanoseconds. */
+ latency = DIV_ROUND_UP(latency, 1000);
+
for (int i = 0; i < ACCESS_COORDINATE_MAX; i++) {
c[i].write_latency += latency;
c[i].read_latency += latency;
diff --git a/drivers/dax/super.c b/drivers/dax/super.c
index 45f84b0eb909..210f4af479f0 100644
--- a/drivers/dax/super.c
+++ b/drivers/dax/super.c
@@ -738,7 +738,7 @@ static int __init dax_core_init(void)
unregister_chrdev_region(dax_devt, MINORMASK+1);
err_chrdev:
dax_fs_exit();
- return 0;
+ return rc;
}
static void __exit dax_core_exit(void)
diff --git a/drivers/dma/altera-msgdma.c b/drivers/dma/altera-msgdma.c
index e23e5b441a24..412f4bce1d4a 100644
--- a/drivers/dma/altera-msgdma.c
+++ b/drivers/dma/altera-msgdma.c
@@ -847,11 +847,6 @@ static int msgdma_probe(struct platform_device *pdev)
if (mdev->irq < 0)
return -ENXIO;
- ret = devm_request_irq(&pdev->dev, mdev->irq, msgdma_irq_handler,
- 0, dev_name(&pdev->dev), mdev);
- if (ret)
- return ret;
-
tasklet_setup(&mdev->irq_tasklet, msgdma_tasklet);
dma_cookie_init(&mdev->dmachan);
@@ -863,6 +858,11 @@ static int msgdma_probe(struct platform_device *pdev)
INIT_LIST_HEAD(&mdev->done_list);
INIT_LIST_HEAD(&mdev->free_list);
+ ret = devm_request_irq(&pdev->dev, mdev->irq, msgdma_irq_handler,
+ 0, dev_name(&pdev->dev), mdev);
+ if (ret)
+ return ret;
+
dma_dev = &mdev->dmadev;
/* Set DMA capabilities */
diff --git a/drivers/dma/loongson/loongson2-apb-cmc-dma.c b/drivers/dma/loongson/loongson2-apb-cmc-dma.c
index 1c9a542edc85..60eb4d961610 100644
--- a/drivers/dma/loongson/loongson2-apb-cmc-dma.c
+++ b/drivers/dma/loongson/loongson2-apb-cmc-dma.c
@@ -404,8 +404,10 @@ loongson2_cmc_dma_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl,
for_each_sg(sgl, sg, sg_len, i) {
ret = loongson2_cmc_dma_set_xfer_param(lchan, direction, &buswidth, sg_dma_len(sg));
- if (ret)
+ if (ret) {
+ kfree(desc);
return ERR_PTR(ret);
+ }
num_items = DIV_ROUND_UP(sg_dma_len(sg), buswidth);
if (num_items >= LOONSON2_CMCDMA_MAX_DATA_ITEMS) {
diff --git a/drivers/dma/sh/rz-dmac.c b/drivers/dma/sh/rz-dmac.c
index ca76f1bb45c4..fdd7131e890a 100644
--- a/drivers/dma/sh/rz-dmac.c
+++ b/drivers/dma/sh/rz-dmac.c
@@ -937,7 +937,7 @@ static u32 rz_dmac_calculate_residue_bytes_in_vd(struct rz_dmac_chan *channel,
struct rz_dmac *dmac = to_rz_dmac(chan->device);
u32 residue = 0, i = 0;
- while (lmdesc->nxla != crla) {
+ while (rz_dmac_lmdesc_addr(channel, lmdesc) != crla) {
lmdesc = rz_dmac_get_next_lmdesc(channel->lmdesc.base, lmdesc);
if (++i >= DMAC_NR_LMDESC)
return 0;
@@ -948,13 +948,13 @@ static u32 rz_dmac_calculate_residue_bytes_in_vd(struct rz_dmac_chan *channel,
u32 start_lmdesc_addr = rz_dmac_lmdesc_addr(channel, desc->start_lmdesc);
while (lmdesc->nxla != start_lmdesc_addr) {
- residue += lmdesc->tb;
lmdesc = rz_dmac_get_next_lmdesc(channel->lmdesc.base, lmdesc);
+ residue += lmdesc->tb;
}
} else {
while (lmdesc->chcfg & CHCFG_DEM) {
- residue += lmdesc->tb;
lmdesc = rz_dmac_get_next_lmdesc(channel->lmdesc.base, lmdesc);
+ residue += lmdesc->tb;
}
}
diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c
index fa1853f252b0..12e1949fcf44 100644
--- a/drivers/edac/i10nm_base.c
+++ b/drivers/edac/i10nm_base.c
@@ -634,6 +634,7 @@ static int i10nm_get_ddr_munits(void)
mbase = ioremap(base + off, size);
if (!mbase) {
+ pci_dev_put(mdev);
i10nm_printk(KERN_ERR, "Failed to ioremap 0x%llx\n",
base + off);
return -ENODEV;
diff --git a/drivers/firewire/ohci-serdes-test.c b/drivers/firewire/ohci-serdes-test.c
index 258f668619ef..87b136e041de 100644
--- a/drivers/firewire/ohci-serdes-test.c
+++ b/drivers/firewire/ohci-serdes-test.c
@@ -43,7 +43,7 @@ static void test_self_id_receive_buffer_deserialization(struct kunit *test)
static void test_at_data_serdes(struct kunit *test)
{
static const __le32 expected[] = {
- cpu_to_le32(0x00020e80),
+ cpu_to_le32(0x00820e80),
cpu_to_le32(0xffc2ffff),
cpu_to_le32(0xe0000000),
};
@@ -56,7 +56,7 @@ static void test_at_data_serdes(struct kunit *test)
unsigned int destination_id = ohci1394_at_data_get_destination_id(expected);
u64 destination_offset = ohci1394_at_data_get_destination_offset(expected);
- KUNIT_EXPECT_FALSE(test, has_src_bus_id);
+ KUNIT_EXPECT_TRUE(test, has_src_bus_id);
KUNIT_EXPECT_EQ(test, 0x02, speed);
KUNIT_EXPECT_EQ(test, 0x03, tlabel);
KUNIT_EXPECT_EQ(test, 0x02, retry);
diff --git a/drivers/firewire/ohci.h b/drivers/firewire/ohci.h
index 218666cfe14a..63ccf2f44504 100644
--- a/drivers/firewire/ohci.h
+++ b/drivers/firewire/ohci.h
@@ -178,7 +178,7 @@
static inline bool ohci1394_at_data_get_src_bus_id(const __le32 *data)
{
- return !!((data[0] & OHCI1394_AT_DATA_Q0_srcBusID_MASK) >> OHCI1394_AT_DATA_Q0_srcBusID_SHIFT);
+ return !!((le32_to_cpu(data[0]) & OHCI1394_AT_DATA_Q0_srcBusID_MASK) >> OHCI1394_AT_DATA_Q0_srcBusID_SHIFT);
}
static inline void ohci1394_at_data_set_src_bus_id(__le32 *data, bool src_bus_id)
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index 77a2b2d74f3f..48982d05c441 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -21,7 +21,7 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ $(CC_FLAGS_DIALECT) \
# arm64 uses the full KBUILD_CFLAGS so it's necessary to explicitly
# disable the stackleak plugin
-cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_KSTACK_ERASE) \
+cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_KSTACK_ERASE) -ffixed-x18 \
-fno-unwind-tables -fno-asynchronous-unwind-tables
cflags-$(CONFIG_ARM) += -DEFI_HAVE_STRLEN -DEFI_HAVE_STRNLEN \
-DEFI_HAVE_MEMCHR -DEFI_HAVE_STRRCHR \
diff --git a/drivers/firmware/efi/libstub/efi-stub.c b/drivers/firmware/efi/libstub/efi-stub.c
index 42d6073bcd06..a4cb39656b03 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -83,6 +83,14 @@ static void install_memreserve_table(void)
efi_guid_t memreserve_table_guid = LINUX_EFI_MEMRESERVE_TABLE_GUID;
efi_status_t status;
+ /*
+ * If an earlier boot stage installed a memreserve table already, e.g.,
+ * a kexec loader passing on the persistent reservations made by the
+ * previous kernel, use it as is: installing a new one would replace it.
+ */
+ if (get_efi_config_table(memreserve_table_guid))
+ return;
+
status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, sizeof(*rsv),
(void **)&rsv);
if (status != EFI_SUCCESS) {
diff --git a/drivers/fpga/fpga-mgr.c b/drivers/fpga/fpga-mgr.c
index 2672070ced02..5a0b7780b335 100644
--- a/drivers/fpga/fpga-mgr.c
+++ b/drivers/fpga/fpga-mgr.c
@@ -744,7 +744,7 @@ EXPORT_SYMBOL_GPL(fpga_mgr_put);
* @mgr: fpga manager
*
* Given a pointer to FPGA Manager (from fpga_mgr_get() or
- * of_fpga_mgr_put()) attempt to get the mutex. The user should call
+ * of_fpga_mgr_get()) attempt to get the mutex. The user should call
* fpga_mgr_lock() and verify that it returns 0 before attempting to
* program the FPGA. Likewise, the user should call fpga_mgr_unlock
* when done programming the FPGA.
diff --git a/drivers/gpio/gpio-davinci.c b/drivers/gpio/gpio-davinci.c
index cccbaea1dec2..fc2b9bef7efc 100644
--- a/drivers/gpio/gpio-davinci.c
+++ b/drivers/gpio/gpio-davinci.c
@@ -611,10 +611,10 @@ static void davinci_gpio_save_context(struct davinci_gpio_controller *chips,
context->set_data = readl_relaxed(&g->set_data);
context->set_rising = readl_relaxed(&g->set_rising);
context->set_falling = readl_relaxed(&g->set_falling);
- }
- /* Clear all interrupt status registers */
- writel_relaxed(GENMASK(31, 0), &g->intstat);
+ /* Clear stale interrupt status for this bank. */
+ writel_relaxed(GENMASK(31, 0), &g->intstat);
+ }
}
static void davinci_gpio_restore_context(struct davinci_gpio_controller *chips,
diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c
index aac9ea7520b6..afec5ec23696 100644
--- a/drivers/gpio/gpio-pca953x.c
+++ b/drivers/gpio/gpio-pca953x.c
@@ -838,13 +838,18 @@ static int pca953x_irq_set_wake(struct irq_data *d, unsigned int on)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
struct pca953x_chip *chip = gpiochip_get_data(gc);
+ int ret;
+
+ ret = irq_set_irq_wake(chip->client->irq, on);
+ if (ret)
+ return ret;
if (on)
atomic_inc(&chip->wakeup_path);
else
atomic_dec(&chip->wakeup_path);
- return irq_set_irq_wake(chip->client->irq, on);
+ return 0;
}
static void pca953x_irq_bus_lock(struct irq_data *d)
@@ -862,6 +867,7 @@ static void pca953x_irq_bus_sync_unlock(struct irq_data *d)
DECLARE_BITMAP(irq_mask, MAX_LINE);
DECLARE_BITMAP(reg_direction, MAX_LINE);
int level;
+ int ret;
guard(mutex)(&chip->i2c_lock);
@@ -880,7 +886,9 @@ static void pca953x_irq_bus_sync_unlock(struct irq_data *d)
}
/* Switch direction to input if needed */
- pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ ret = pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ if (ret)
+ goto out;
bitmap_or(irq_mask, chip->irq_trig_fall, chip->irq_trig_raise, gc->ngpio);
bitmap_or(irq_mask, irq_mask, chip->irq_trig_level_high, gc->ngpio);
@@ -890,6 +898,7 @@ static void pca953x_irq_bus_sync_unlock(struct irq_data *d)
for_each_andnot_bit(level, irq_mask, reg_direction, gc->ngpio)
pca953x_gpio_direction_input_unlocked(&chip->gpio_chip, level);
+out:
mutex_unlock(&chip->irq_lock);
}
@@ -984,7 +993,9 @@ static bool pca953x_irq_pending(struct pca953x_chip *chip, unsigned long *pendin
}
/* Remove output pins from the equation */
- pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ ret = pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ if (ret)
+ return false;
bitmap_copy(old_stat, chip->irq_stat, gc->ngpio);
@@ -1084,7 +1095,9 @@ static int pca953x_irq_setup(struct pca953x_chip *chip, int irq_base)
* interrupt. We have to rely on the previous read for
* this purpose.
*/
- pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ ret = pca953x_read_regs(chip, chip->regs->direction, reg_direction);
+ if (ret)
+ return ret;
bitmap_and(chip->irq_stat, irq_stat, reg_direction, gc->ngpio);
mutex_init(&chip->irq_lock);
diff --git a/drivers/gpio/gpiolib-cdev.c b/drivers/gpio/gpiolib-cdev.c
index 5d53bfcdf726..f4d78113ade2 100644
--- a/drivers/gpio/gpiolib-cdev.c
+++ b/drivers/gpio/gpiolib-cdev.c
@@ -1229,11 +1229,6 @@ static void gpio_v2_line_config_flags_to_desc_flags(u64 lflags,
assign_bit(GPIOD_FLAG_ACTIVE_LOW, &flags,
lflags & GPIO_V2_LINE_FLAG_ACTIVE_LOW);
- if (lflags & GPIO_V2_LINE_FLAG_OUTPUT)
- set_bit(GPIOD_FLAG_IS_OUT, &flags);
- else if (lflags & GPIO_V2_LINE_FLAG_INPUT)
- clear_bit(GPIOD_FLAG_IS_OUT, &flags);
-
assign_bit(GPIOD_FLAG_EDGE_RISING, &flags,
lflags & GPIO_V2_LINE_FLAG_EDGE_RISING);
assign_bit(GPIOD_FLAG_EDGE_FALLING, &flags,
diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index c0394c7d6279..226ed5b8d444 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -2627,10 +2627,13 @@ int gpiod_request_commit(struct gpio_desc *desc, const char *label)
ret = desc_set_label(desc, label ? : "?");
if (ret)
- goto out_clear_bit;
+ goto out_free_gpio;
return 0;
+out_free_gpio:
+ if (guard.gc->free)
+ guard.gc->free(guard.gc, offset);
out_clear_bit:
clear_bit(GPIOD_FLAG_REQUESTED, &desc->flags);
return ret;
@@ -2716,6 +2719,7 @@ void gpiod_free(struct gpio_desc *desc)
char *gpiochip_dup_line_label(struct gpio_chip *gc, unsigned int offset)
{
struct gpio_desc *desc;
+ const char *name;
char *label;
desc = gpiochip_get_desc(gc, offset);
@@ -2727,7 +2731,11 @@ char *gpiochip_dup_line_label(struct gpio_chip *gc, unsigned int offset)
guard(srcu)(&desc->gdev->desc_srcu);
- label = kstrdup(gpiod_get_label(desc), GFP_KERNEL);
+ name = gpiod_get_label(desc);
+ if (!name)
+ return NULL;
+
+ label = kstrdup(name, GFP_KERNEL);
if (!label)
return ERR_PTR(-ENOMEM);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index a9c6f5d4a639..a63404adc2d8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -834,8 +834,17 @@ struct amdgpu_device {
struct mutex mn_lock;
DECLARE_HASHTABLE(mn_hash, 7);
- /* tracking pinned memory */
+ /* tracking pinned memory
+ *
+ * vram_pin_size = sum(vram_pin_size_xcp[]) + vram_pin_size_common
+ *
+ * vram_pin_size total pin size in device
+ * vram_pin_size_xcp[i] pins owned by partition i (bo->xcp_id = i, i >= 0)
+ * vram_pin_size_common pins not owned by any partition, bo->xcp_id = -1
+ */
atomic64_t vram_pin_size;
+ atomic64_t vram_pin_size_xcp[MAX_XCP];
+ atomic64_t vram_pin_size_common;
atomic64_t visible_pin_size;
atomic64_t gart_pin_size;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 1b7dc0d3963b..f462b40220eb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -42,6 +42,12 @@
extern uint64_t amdgpu_amdkfd_total_mem_size;
+/*
+ * Align VRAM availability to 2MB to avoid fragmentation caused by 4K
+ * allocations in the tail 2MB BO chunk.
+ */
+#define VRAM_AVAILABLITY_ALIGN (1 << 21)
+
enum TLB_FLUSH_TYPE {
TLB_FLUSH_LEGACY = 0,
TLB_FLUSH_LIGHTWEIGHT,
@@ -106,6 +112,7 @@ struct amdgpu_kfd_dev {
struct kfd_dev *dev;
int64_t vram_used[MAX_XCP];
uint64_t vram_used_aligned[MAX_XCP];
+ atomic64_t vram_pinned[MAX_XCP];
bool init_complete;
struct work_struct reset_work;
@@ -390,6 +397,8 @@ void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev,
* Allows KFD to release its resources associated with the GEM object.
*/
void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo);
+void amdgpu_amdkfd_account_vram_pin(struct amdgpu_bo *bo, int xcp_id);
+void amdgpu_amdkfd_account_vram_unpin(struct amdgpu_bo *bo, int xcp_id);
void amdgpu_amdkfd_reserve_system_mem(uint64_t size);
#else
static inline
@@ -407,6 +416,16 @@ static inline
void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo)
{
}
+
+static inline
+void amdgpu_amdkfd_account_vram_pin(struct amdgpu_bo *bo, int xcp_id)
+{
+}
+
+static inline
+void amdgpu_amdkfd_account_vram_unpin(struct amdgpu_bo *bo, int xcp_id)
+{
+}
#endif
#if IS_ENABLED(CONFIG_HSA_AMD_SVM)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v12_1.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v12_1.c
index 070001fd34b0..7b04258c3893 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v12_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v12_1.c
@@ -77,25 +77,21 @@ static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev,
unsigned int engine_id,
unsigned int queue_id)
{
- uint32_t sdma_engine_reg_base = 0;
- uint32_t sdma_rlc_reg_offset;
uint32_t dev_inst = GET_INST(SDMA0, engine_id);
+ int xcc_id = adev->sdma.instance[engine_id].xcc_id;
+ int xcc_inst = dev_inst % adev->sdma.num_inst_per_xcc;
+ uint32_t sdma_engine_reg_base;
+ uint32_t sdma_rlc_reg_offset;
- switch (dev_inst % adev->sdma.num_inst_per_xcc) {
- case 0:
- sdma_engine_reg_base = SOC15_REG_OFFSET(SDMA0,
- dev_inst / adev->sdma.num_inst_per_xcc,
- regSDMA0_SDMA_QUEUE0_RB_CNTL) - regSDMA0_SDMA_QUEUE0_RB_CNTL;
- break;
- case 1:
- sdma_engine_reg_base = SOC15_REG_OFFSET(SDMA1,
- dev_inst / adev->sdma.num_inst_per_xcc,
- regSDMA1_SDMA_QUEUE0_RB_CNTL) - regSDMA0_SDMA_QUEUE0_RB_CNTL;
- break;
- default:
- WARN(1, "Invalid SDMA engine id %d\n", engine_id);
- break;
- }
+ /*
+ * On SDMA v7 (gc_12_x) all SDMA queue registers live in the GC_HWIP
+ * aperture. The second SDMA per XCC is reached by adding a fixed
+ * offset (regSDMA1_* - regSDMA0_* == SDMA1_REG_OFFSET
+ */
+ sdma_engine_reg_base = adev->reg_offset[GC_HWIP][xcc_id][0];
+ if (xcc_inst != 0)
+ sdma_engine_reg_base +=
+ regSDMA1_SDMA_QUEUE0_RB_CNTL - regSDMA0_SDMA_QUEUE0_RB_CNTL;
sdma_rlc_reg_offset = sdma_engine_reg_base
+ queue_id * (regSDMA0_SDMA_QUEUE1_RB_CNTL - regSDMA0_SDMA_QUEUE0_RB_CNTL);
@@ -141,8 +137,7 @@ static int hqd_sdma_dump_v12_1(struct amdgpu_device *adev,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
- uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev,
- engine_id, queue_id);
+ uint32_t sdma_rlc_reg_offset;
uint32_t i = 0, reg;
const uint32_t first_reg = regSDMA0_SDMA_QUEUE0_RB_CNTL;
@@ -150,6 +145,17 @@ static int hqd_sdma_dump_v12_1(struct amdgpu_device *adev,
#undef HQD_N_REGS
#define HQD_N_REGS (last_reg - first_reg + 1)
+ /*
+ * dqm_debugfs_hqds() iterates engine_id over
+ * node_id * get_num_all_sdma_engines()..+get_num_all_sdma_engines().
+ * On partitioned devices that range can address a logical SDMA engine
+ * beyond the instances that actually exist on this node; reject it
+ */
+ if (engine_id >= adev->sdma.num_instances)
+ return -EINVAL;
+
+ sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, engine_id, queue_id);
+
*dump = kmalloc(HQD_N_REGS*2*sizeof(uint32_t), GFP_KERNEL);
if (*dump == NULL)
return -ENOMEM;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d66881684ee5..c5c2a73ddd94 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -46,12 +46,6 @@
#define AMDGPU_USERPTR_RESTORE_DELAY_MS 1
#define AMDGPU_RESERVE_MEM_LIMIT (3UL << 29)
-/*
- * Align VRAM availability to 2MB to avoid fragmentation caused by 4K allocations in the tail 2MB
- * BO chunk
- */
-#define VRAM_AVAILABLITY_ALIGN (1 << 21)
-
/* Impose limit on how much memory KFD can use */
static struct {
uint64_t max_system_mem_limit;
@@ -73,6 +67,8 @@ static const char * const domain_bit_to_string[] = {
#define domain_string(domain) domain_bit_to_string[ffs(domain)-1]
static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work);
+static size_t __get_available_memory(struct amdgpu_device *adev,
+ uint8_t xcp_id, int64_t vram_used);
static bool kfd_mem_is_attached(struct amdgpu_vm *avm,
struct kgd_mem *mem)
@@ -169,13 +165,8 @@ void amdgpu_amdkfd_reserve_system_mem(uint64_t size)
int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
uint64_t size, u32 alloc_flag, int8_t xcp_id)
{
- uint64_t reserved_for_pt =
- ESTIMATE_PT_SIZE(amdgpu_amdkfd_total_mem_size);
- struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- uint64_t reserved_for_ras = (con ? con->reserved_pages_in_bytes : 0);
size_t system_mem_needed, ttm_mem_needed, vram_needed;
int ret = 0;
- uint64_t vram_size = 0;
system_mem_needed = 0;
ttm_mem_needed = 0;
@@ -196,7 +187,6 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
if (WARN_ONCE(xcp_id < 0, "invalid XCP ID %d", xcp_id))
return -EINVAL;
- vram_size = KFD_XCP_MEMORY_SIZE(adev, xcp_id);
if (adev->apu_prefer_gtt) {
system_mem_needed = size;
ttm_mem_needed = size;
@@ -232,14 +222,11 @@ int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
* VRAM check since ttm_mem_limit check already cover this allocation
*/
- if (adev && xcp_id >= 0 && (!adev->apu_prefer_gtt || adev->gmc.is_app_apu)) {
- uint64_t vram_available =
- vram_size - reserved_for_pt - reserved_for_ras -
- atomic64_read(&adev->vram_pin_size);
- if (adev->kfd.vram_used[xcp_id] + vram_needed > vram_available) {
- ret = -ENOMEM;
- goto release;
- }
+ if (adev && xcp_id >= 0 && (!adev->apu_prefer_gtt || adev->gmc.is_app_apu) &&
+ vram_needed > __get_available_memory(adev, xcp_id,
+ adev->kfd.vram_used[xcp_id])) {
+ ret = -ENOMEM;
+ goto release;
}
/* Update memory accounting by decreasing available system
@@ -318,6 +305,30 @@ void amdgpu_amdkfd_release_notify(struct amdgpu_bo *bo)
kfree(bo->kfd_bo);
}
+void amdgpu_amdkfd_account_vram_pin(struct amdgpu_bo *bo, int xcp_id)
+{
+ if (!bo->kfd_bo)
+ return;
+
+ if (WARN_ONCE(xcp_id < 0, "KFD BO pinned in VRAM with no partition"))
+ return;
+
+ atomic64_add(amdgpu_bo_size(bo),
+ &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned[xcp_id]);
+}
+
+void amdgpu_amdkfd_account_vram_unpin(struct amdgpu_bo *bo, int xcp_id)
+{
+ if (!bo->kfd_bo)
+ return;
+
+ if (WARN_ONCE(xcp_id < 0, "KFD BO unpinned in VRAM with no partition"))
+ return;
+
+ atomic64_sub(amdgpu_bo_size(bo),
+ &amdgpu_ttm_adev(bo->tbo.bdev)->kfd.vram_pinned[xcp_id]);
+}
+
/**
* create_dmamap_sg_bo() - Creates a amdgpu_bo object to reflect information
* about USERPTR or DOOREBELL or MMIO BO.
@@ -1536,6 +1547,7 @@ static int amdgpu_amdkfd_gpuvm_pin_bo(struct amdgpu_bo *bo, u32 domain)
pr_err("Error in Pinning BO to domain: %d\n", domain);
amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, false);
+
out:
amdgpu_bo_unreserve(bo);
return ret;
@@ -1558,6 +1570,7 @@ static void amdgpu_amdkfd_gpuvm_unpin_bo(struct amdgpu_bo *bo)
return;
amdgpu_bo_unpin(bo);
+
amdgpu_bo_unreserve(bo);
}
@@ -1661,8 +1674,9 @@ int amdgpu_amdkfd_criu_resume(void *p)
return ret;
}
-size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev,
- uint8_t xcp_id)
+/* Caller must hold kfd_mem_limit.mem_limit_lock. */
+static size_t __get_available_memory(struct amdgpu_device *adev,
+ uint8_t xcp_id, int64_t vram_used)
{
uint64_t reserved_for_pt =
ESTIMATE_PT_SIZE(amdgpu_amdkfd_total_mem_size);
@@ -1671,14 +1685,17 @@ size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev,
ssize_t available;
uint64_t vram_available, system_mem_available, ttm_mem_available;
- spin_lock(&kfd_mem_limit.mem_limit_lock);
+ lockdep_assert_held(&kfd_mem_limit.mem_limit_lock);
+
if (adev->apu_prefer_gtt && !adev->gmc.is_app_apu)
vram_available = KFD_XCP_MEMORY_SIZE(adev, xcp_id)
- - adev->kfd.vram_used_aligned[xcp_id];
+ - vram_used;
else
vram_available = KFD_XCP_MEMORY_SIZE(adev, xcp_id)
- - adev->kfd.vram_used_aligned[xcp_id]
- - atomic64_read(&adev->vram_pin_size)
+ - vram_used
+ - atomic64_read(&adev->vram_pin_size_xcp[xcp_id])
+ - atomic64_read(&adev->vram_pin_size_common)
+ + atomic64_read(&adev->kfd.vram_pinned[xcp_id])
- reserved_for_pt
- reserved_for_ras;
@@ -1693,19 +1710,29 @@ size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev,
available = min3(system_mem_available, ttm_mem_available,
vram_available);
- available = ALIGN_DOWN(available, PAGE_SIZE);
} else {
- available = ALIGN_DOWN(vram_available, VRAM_AVAILABLITY_ALIGN);
+ available = vram_available;
}
- spin_unlock(&kfd_mem_limit.mem_limit_lock);
-
if (available < 0)
available = 0;
return available;
}
+size_t amdgpu_amdkfd_get_available_memory(struct amdgpu_device *adev,
+ uint8_t xcp_id)
+{
+ size_t available;
+
+ spin_lock(&kfd_mem_limit.mem_limit_lock);
+ available = __get_available_memory(adev, xcp_id,
+ adev->kfd.vram_used_aligned[xcp_id]);
+ spin_unlock(&kfd_mem_limit.mem_limit_lock);
+
+ return available;
+}
+
int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
struct amdgpu_device *adev, uint64_t va, uint64_t size,
void *drm_priv, struct kgd_mem **mem,
@@ -2827,6 +2854,7 @@ static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work)
struct task_struct *usertask;
struct mm_struct *mm;
uint32_t evicted_bos;
+ int ret;
mutex_lock(&process_info->notifier_lock);
evicted_bos = process_info->evicted_bos;
@@ -2846,14 +2874,16 @@ static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work)
mutex_lock(&process_info->lock);
- if (update_invalid_user_pages(process_info, mm))
+ ret = update_invalid_user_pages(process_info, mm);
+ if (ret)
goto unlock_out;
/* userptr_inval_list can be empty if all evicted userptr BOs
* have been freed. In that case there is nothing to validate
* and we can just restart the queues.
*/
if (!list_empty(&process_info->userptr_inval_list)) {
- if (validate_invalid_user_pages(process_info))
+ ret = validate_invalid_user_pages(process_info);
+ if (ret)
goto unlock_out;
}
/* Final check for concurrent evicton and atomic update. If
@@ -2862,11 +2892,14 @@ static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work)
* reference counting inside KFD will handle this case.
*/
mutex_lock(&process_info->notifier_lock);
- if (process_info->evicted_bos != evicted_bos)
+ if (process_info->evicted_bos != evicted_bos) {
+ ret = -EAGAIN;
goto unlock_notifier_out;
+ }
if (confirm_valid_user_pages_locked(process_info)) {
WARN(1, "User pages unexpectedly invalid");
+ ret = -EINVAL;
goto unlock_notifier_out;
}
@@ -2884,8 +2917,10 @@ static void amdgpu_amdkfd_restore_userptr_worker(struct work_struct *work)
unlock_out:
mutex_unlock(&process_info->lock);
- /* If validation failed, reschedule another attempt */
- if (evicted_bos) {
+ /* If validation failed, reschedule another attempt, unless the
+ * device is gone for good
+ */
+ if (evicted_bos && ret != -ENODEV) {
queue_delayed_work(system_freezable_wq,
&process_info->restore_userptr_work,
msecs_to_jiffies(AMDGPU_USERPTR_RESTORE_DELAY_MS));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index ef8a61ce4b1d..86a3dc3655b3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -217,6 +217,12 @@ static int amdgpu_cs_pass1(struct amdgpu_cs_parser *p,
size = p->chunks[i].length_dw;
+ if (size > INT_MAX / sizeof(uint32_t)) {
+ ret = -EINVAL;
+ i--;
+ goto free_partial_kdata;
+ }
+
p->chunks[i].kdata = vmemdup_array_user(u64_to_user_ptr(user_chunk.chunk_data),
size,
sizeof(uint32_t));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 9269e780feb7..5b72e4afb650 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -583,10 +583,8 @@ bool amdgpu_device_supports_boco(struct amdgpu_device *adev)
if (!IS_ENABLED(CONFIG_HOTPLUG_PCI_PCIE))
return false;
- if (adev->has_pr3 ||
- ((adev->flags & AMD_IS_PX) && amdgpu_is_atpx_hybrid()))
- return true;
- return false;
+ /* BOCO needs a platform power resource (_PR3) to cut the power */
+ return adev->has_pr3;
}
/**
@@ -2772,6 +2770,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
+ r = amdgpu_dpm_reapply_umd_profile(adev);
+ if (r) {
+ dev_err(adev->dev, "Failed to restore UMD profile state: %d\n", r);
+ return r;
+ }
+
amdgpu_device_fill_reset_magic(adev);
r = amdgpu_device_enable_mgpu_fan_boost();
@@ -4315,8 +4319,11 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
* before ip_fini_early to prevent kfd locking refcount issues by calling
* amdgpu_amdkfd_suspend()
*/
- if (pci_dev_is_disconnected(adev->pdev))
+ if (pci_dev_is_disconnected(adev->pdev)) {
+ /* Drain KFD processes while kfd.dev is still valid */
+ amdgpu_amdkfd_teardown_processes(adev);
amdgpu_amdkfd_device_fini_sw(adev);
+ }
amdgpu_coredump_fini(adev);
amdgpu_device_ip_fini_early(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index d9d794a7db96..69278026c935 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -2929,19 +2929,6 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
return 0;
}
-static void amdgpu_restore_umd_profile_pstate_after_runpm(struct amdgpu_device *adev)
-{
- enum amd_dpm_forced_level level;
- uint32_t profile_mode_mask = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD |
- AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK |
- AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK |
- AMD_DPM_FORCED_LEVEL_PROFILE_PEAK;
-
- level = amdgpu_dpm_get_performance_level(adev);
- if (level & profile_mode_mask)
- amdgpu_asic_update_umd_stable_pstate(adev, true);
-}
-
static int amdgpu_pmops_runtime_resume(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
@@ -2987,7 +2974,6 @@ static int amdgpu_pmops_runtime_resume(struct device *dev)
if (adev->pm.rpm_mode == AMDGPU_RUNPM_PX)
drm_dev->switch_power_state = DRM_SWITCH_POWER_ON;
- amdgpu_restore_umd_profile_pstate_after_runpm(adev);
adev->in_runpm = false;
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 1dedaf1698dc..92323d0bdb4a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -313,22 +313,7 @@ void amdgpu_job_free_resources(struct amdgpu_job *job)
static void amdgpu_job_free_cb(struct drm_sched_job *s_job)
{
- struct amdgpu_job *job = to_amdgpu_job(s_job);
-
- drm_sched_job_cleanup(s_job);
-
- amdgpu_sync_free(&job->explicit_sync);
-
- if (dma_fence_was_initialized(&job->hw_fence->base))
- dma_fence_put(&job->hw_fence->base);
- else
- kfree(job->hw_fence);
- if (dma_fence_was_initialized(&job->hw_vm_fence->base))
- dma_fence_put(&job->hw_vm_fence->base);
- else
- kfree(job->hw_vm_fence);
-
- kfree(job);
+ amdgpu_job_free(to_amdgpu_job(s_job));
}
void amdgpu_job_set_gang_leader(struct amdgpu_job *job,
@@ -349,13 +334,14 @@ void amdgpu_job_set_gang_leader(struct amdgpu_job *job,
void amdgpu_job_free(struct amdgpu_job *job)
{
- if (job->base.entity)
+ if (job->gang_submit != &job->base.s_fence->scheduled)
+ dma_fence_put(job->gang_submit);
+
+ if (job->base.s_fence)
drm_sched_job_cleanup(&job->base);
amdgpu_job_free_resources(job);
amdgpu_sync_free(&job->explicit_sync);
- if (job->gang_submit != &job->base.s_fence->scheduled)
- dma_fence_put(job->gang_submit);
if (dma_fence_was_initialized(&job->hw_fence->base))
dma_fence_put(&job->hw_fence->base);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 1b6c32a177fb..ee366d729abf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -915,6 +915,19 @@ void amdgpu_bo_unref(struct amdgpu_bo **bo)
*bo = NULL;
}
+/* Map a BO onto vram_pin_size_xcp[]. bo->xcp_id < 0 means "any partition":
+ * such pins are not owned by a single partition and go to
+ * vram_pin_size_common, which every partition subtracts.
+ */
+static int amdgpu_bo_get_xcp(struct amdgpu_bo *bo)
+{
+ s8 xcp = bo->xcp_id;
+
+ if (xcp >= 0 && xcp < MAX_XCP)
+ return xcp;
+ return -1;
+}
+
/**
* amdgpu_bo_pin - pin an &amdgpu_bo buffer object
* @bo: &amdgpu_bo buffer object to be pinned
@@ -995,7 +1008,17 @@ int amdgpu_bo_pin(struct amdgpu_bo *bo, u32 domain)
ttm_bo_pin(&bo->tbo);
if (bo->tbo.resource->mem_type == TTM_PL_VRAM) {
+ int xcp = amdgpu_bo_get_xcp(bo);
+
atomic64_add(amdgpu_bo_size(bo), &adev->vram_pin_size);
+ if (xcp >= 0) {
+ atomic64_add(amdgpu_bo_size(bo),
+ &adev->vram_pin_size_xcp[xcp]);
+ } else {
+ atomic64_add(amdgpu_bo_size(bo),
+ &adev->vram_pin_size_common);
+ }
+ amdgpu_amdkfd_account_vram_pin(bo, xcp);
atomic64_add(amdgpu_vram_mgr_bo_visible_size(bo),
&adev->visible_pin_size);
} else if (bo->tbo.resource->mem_type == TTM_PL_TT) {
@@ -1028,7 +1051,17 @@ void amdgpu_bo_unpin(struct amdgpu_bo *bo)
dma_buf_unpin(bo->tbo.base.import_attach);
if (bo->tbo.resource->mem_type == TTM_PL_VRAM) {
+ int xcp = amdgpu_bo_get_xcp(bo);
+
atomic64_sub(amdgpu_bo_size(bo), &adev->vram_pin_size);
+ if (xcp >= 0) {
+ atomic64_sub(amdgpu_bo_size(bo),
+ &adev->vram_pin_size_xcp[xcp]);
+ } else {
+ atomic64_sub(amdgpu_bo_size(bo),
+ &adev->vram_pin_size_common);
+ }
+ amdgpu_amdkfd_account_vram_unpin(bo, xcp);
atomic64_sub(amdgpu_vram_mgr_bo_visible_size(bo),
&adev->visible_pin_size);
} else if (bo->tbo.resource->mem_type == TTM_PL_TT) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 83968886c29a..364be96c08df 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -777,7 +777,8 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
goto out_unlock;
}
if (unlikely((gtt->userflags & AMDGPU_GEM_USERPTR_ANONONLY) &&
- vma->vm_file)) {
+ (vma->vm_file ||
+ vma->vm_end < start + (u64)ttm->num_pages * PAGE_SIZE))) {
r = -EPERM;
goto out_unlock;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index ecd7caa95d4b..cd35ff7ca864 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -377,7 +377,8 @@ int amdgpu_uvd_sw_fini(struct amdgpu_device *adev)
void *addr = amdgpu_bo_kptr(adev->uvd.ib_bo);
int i, j;
- drm_sched_entity_destroy(&adev->uvd.entity);
+ if (adev->uvd.entity.rq)
+ drm_sched_entity_destroy(&adev->uvd.entity);
for (j = 0; j < adev->uvd.num_uvd_inst; ++j) {
if (adev->uvd.harvest_config & (1 << j))
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index 1f7a98cdce5a..3952af9ae115 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -253,7 +253,8 @@ int amdgpu_vce_sw_fini(struct amdgpu_device *adev)
if (adev->vce.vcpu_bo == NULL)
return 0;
- drm_sched_entity_destroy(&adev->vce.entity);
+ if (adev->vce.entity.rq)
+ drm_sched_entity_destroy(&adev->vce.entity);
for (i = 0; i < adev->vce.num_rings; i++)
amdgpu_ring_fini(&adev->vce.ring[i]);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 4a63b472f68e..fc3a8d38d514 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2725,10 +2725,7 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm)
if (vm->use_cpu_for_update) {
/* Sync with last SDMA update/clear before switching to CPU */
- r = amdgpu_bo_sync_wait(vm->root.bo,
- AMDGPU_FENCE_OWNER_UNDEFINED, true);
- if (r)
- goto unreserve_bo;
+ dma_fence_wait(vm->last_update, false);
vm->update_funcs = &amdgpu_vm_cpu_funcs;
r = amdgpu_vm_pt_map_tables(adev, vm);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index e43a60d09808..cef4171b4ad0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -422,7 +422,7 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
if (r)
goto exit;
- r = vm->update_funcs->commit(¶ms, NULL);
+ r = vm->update_funcs->commit(¶ms, &vm->last_update);
exit:
drm_dev_exit(idx);
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index 50cc0779c340..eb8cf0d9d9c5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -254,6 +254,11 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
AMDGPU_KERNEL_JOB_ID_VM_UPDATE);
if (r)
return r;
+
+ /* The estimate above describes the job which was just
+ * submitted, take the budget of the newly allocated one.
+ */
+ ndw = p->num_dw_left;
}
if (!p->pages_addr) {
diff --git a/drivers/gpu/drm/amd/amdgpu/cik.c b/drivers/gpu/drm/amd/amdgpu/cik.c
index e037c08c8547..71eeaf42fd33 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik.c
@@ -1927,6 +1927,16 @@ static bool cik_need_reset_on_init(struct amdgpu_device *adev)
{
u32 clock_cntl, pc;
+ /*
+ * MacBookPros with switchable graphics put VRAM at 0 when the iGPU
+ * is enabled. Any later post (S3 resume, GPU reset) moves it back to
+ * the vbios default, so reset and post on load to start with the
+ * vbios layout.
+ */
+ if (adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE &&
+ !(RREG32(mmMC_VM_FB_LOCATION) & MC_VM_FB_LOCATION__FB_BASE_MASK))
+ return true;
+
if (adev->flags & AMD_IS_APU)
return false;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index 119268273530..bb0d013e1fa1 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -2172,10 +2172,22 @@ static int gmc_v9_0_hw_init(struct amdgpu_ip_block *ip_block)
* (type 2), which flushes both. Due to a race condition with
* concurrent memory accesses using the same TLB cache line, we still
* need a second TLB flush after this.
+ *
+ * GC 9.4.3, 9.4.4 and 9.5.0 hit a similar race.
*/
- adev->gmc.flush_tlb_needs_extra_type_2 =
- amdgpu_ip_version(adev, GC_HWIP, 0) == IP_VERSION(9, 4, 0) &&
- adev->gmc.xgmi.num_physical_nodes;
+ switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
+ case IP_VERSION(9, 4, 0):
+ adev->gmc.flush_tlb_needs_extra_type_2 =
+ adev->gmc.xgmi.num_physical_nodes;
+ break;
+ case IP_VERSION(9, 4, 3):
+ case IP_VERSION(9, 4, 4):
+ case IP_VERSION(9, 5, 0):
+ adev->gmc.flush_tlb_needs_extra_type_2 = true;
+ break;
+ default:
+ break;
+ }
/* The sequence of these two function calls matters.*/
gmc_v9_0_init_golden_registers(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c
index 7100f30ffeff..a765371ea4bc 100644
--- a/drivers/gpu/drm/amd/amdgpu/si.c
+++ b/drivers/gpu/drm/amd/amdgpu/si.c
@@ -1512,6 +1512,16 @@ static void si_invalidate_hdp(struct amdgpu_device *adev,
static bool si_need_reset_on_init(struct amdgpu_device *adev)
{
+ /*
+ * MacBookPros with switchable graphics put VRAM at 0 when the iGPU
+ * is enabled. Any later post (S3 resume, GPU reset) moves it back to
+ * the vbios default, so reset and post on load to start with the
+ * vbios layout.
+ */
+ if (adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE &&
+ !(RREG32(mmMC_VM_FB_LOCATION) & MC_VM_FB_LOCATION__FB_BASE_MASK))
+ return true;
+
return false;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c
index e926d09128e0..50ba2f140efb 100644
--- a/drivers/gpu/drm/amd/amdgpu/vi.c
+++ b/drivers/gpu/drm/amd/amdgpu/vi.c
@@ -1405,6 +1405,16 @@ static bool vi_need_reset_on_init(struct amdgpu_device *adev)
unsigned int i;
u32 clock_cntl, pc;
+ /*
+ * MacBookPros with switchable graphics put VRAM at 0 when the iGPU
+ * is enabled. Any later post (S3 resume, GPU reset) moves it back to
+ * the vbios default, so reset and post on load to start with the
+ * vbios layout.
+ */
+ if (adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE &&
+ !(RREG32(mmMC_VM_FB_LOCATION) & MC_VM_FB_LOCATION__FB_BASE_MASK))
+ return true;
+
for (i = 0; i < ARRAY_SIZE(vi_reset_quirks); i++)
if (adev->pdev->device == vi_reset_quirks[i].device &&
adev->pdev->subsystem_vendor == vi_reset_quirks[i].subsystem_vendor &&
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 344da6c0e96a..d65ae53c81a5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -427,12 +427,10 @@ static int kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p,
}
}
- if (!pdd->qpd.proc_doorbells) {
- err = kfd_alloc_process_doorbells(dev->kfd, pdd);
- if (err) {
- pr_debug("failed to allocate process doorbells\n");
- goto err_bind_process;
- }
+ err = kfd_alloc_process_doorbells(dev->kfd, pdd);
+ if (err) {
+ pr_debug("failed to allocate process doorbells\n");
+ goto err_bind_process;
}
err = kfd_queue_acquire_buffers(pdd, &q_properties);
@@ -1112,11 +1110,19 @@ static int kfd_ioctl_get_available_memory(struct file *filep,
{
struct kfd_ioctl_get_available_memory_args *args = data;
struct kfd_process_device *pdd = kfd_lock_pdd_by_id(p, args->gpu_id);
+ struct amdgpu_device *adev;
+ size_t available;
if (!pdd)
return -EINVAL;
- args->available = amdgpu_amdkfd_get_available_memory(pdd->dev->adev,
- pdd->dev->node_id);
+
+ adev = pdd->dev->adev;
+ available = amdgpu_amdkfd_get_available_memory(adev, pdd->dev->node_id);
+ if (adev->apu_prefer_gtt)
+ args->available = ALIGN_DOWN(available, PAGE_SIZE);
+ else
+ args->available = ALIGN_DOWN(available, VRAM_AVAILABLITY_ALIGN);
+
kfd_unlock_pdd(pdd);
return 0;
}
@@ -1305,6 +1311,9 @@ static int kfd_ioctl_free_memory_of_gpu(struct file *filep,
goto err_unlock;
}
+ /* Queues a reset could not stop may still access the buffer */
+ kfd_process_wait_gpu_reset_complete(p);
+
ret = amdgpu_amdkfd_gpuvm_free_memory_of_gpu(pdd->dev->adev,
(struct kgd_mem *)mem, pdd->drm_priv, &size);
@@ -2455,11 +2464,9 @@ static int criu_restore_devices(struct kfd_process *p,
goto exit;
}
- if (!pdd->qpd.proc_doorbells) {
- ret = kfd_alloc_process_doorbells(dev->kfd, pdd);
- if (ret)
- goto exit;
- }
+ ret = kfd_alloc_process_doorbells(dev->kfd, pdd);
+ if (ret)
+ goto exit;
}
/*
@@ -2869,6 +2876,7 @@ static int criu_process_info(struct file *filep,
struct kfd_process *p,
struct kfd_ioctl_criu_args *args)
{
+ bool paused_here = false;
int ret = 0;
mutex_lock(&p->mutex);
@@ -2879,11 +2887,15 @@ static int criu_process_info(struct file *filep,
goto err_unlock;
}
- ret = kfd_process_evict_queues(p, KFD_QUEUE_EVICTION_CRIU_CHECKPOINT);
- if (ret)
- goto err_unlock;
+ /* UNPAUSE restores the queues once, so evict them only once */
+ if (!p->queues_paused) {
+ ret = kfd_process_evict_queues(p, KFD_QUEUE_EVICTION_CRIU_CHECKPOINT);
+ if (ret)
+ goto err_unlock;
- p->queues_paused = true;
+ p->queues_paused = true;
+ paused_here = true;
+ }
args->pid = task_pid_nr_ns(p->lead_thread,
task_active_pid_ns(p->lead_thread));
@@ -2898,7 +2910,7 @@ static int criu_process_info(struct file *filep,
args->priv_data_size);
err_unlock:
- if (ret) {
+ if (ret && paused_here) {
kfd_process_restore_queues(p);
p->queues_paused = false;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
index 6e0df685503d..32d7573dffc9 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
@@ -2056,27 +2056,33 @@ static void kfd_find_numa_node_in_srat(struct kfd_node *kdev)
return;
}
+ if (table_header->length < sizeof(struct acpi_table_srat))
+ goto invalid_srat;
+
table_end = (unsigned long)table_header + table_header->length;
/* Parse all entries looking for a match. */
sub_header = (struct acpi_subtable_header *)
((unsigned long)table_header +
sizeof(struct acpi_table_srat));
- subtable_len = sub_header->length;
- while (((unsigned long)sub_header) + subtable_len < table_end) {
- /*
- * If length is 0, break from this loop to avoid
- * infinite loop.
- */
- if (subtable_len == 0) {
- pr_err("SRAT invalid zero length\n");
- break;
- }
+ while ((unsigned long)sub_header < table_end) {
+ unsigned long start = (unsigned long)sub_header;
+
+ if (table_end - start < sizeof(*sub_header))
+ goto invalid_srat;
+
+ subtable_len = sub_header->length;
+ if (subtable_len < sizeof(*sub_header) ||
+ subtable_len > table_end - start)
+ goto invalid_srat;
switch (sub_header->type) {
case ACPI_SRAT_TYPE_CPU_AFFINITY:
cpu = (struct acpi_srat_cpu_affinity *)sub_header;
+ if (subtable_len < sizeof(*cpu))
+ goto invalid_srat;
+
pxm = *((u32 *)cpu->proximity_domain_hi) << 8 |
cpu->proximity_domain_lo;
if (pxm > max_pxm)
@@ -2084,6 +2090,9 @@ static void kfd_find_numa_node_in_srat(struct kfd_node *kdev)
break;
case ACPI_SRAT_TYPE_GENERIC_AFFINITY:
gpu = (struct acpi_srat_generic_affinity *)sub_header;
+ if (subtable_len < sizeof(*gpu))
+ goto invalid_srat;
+
bdf = *((u16 *)(&gpu->device_handle[0])) << 16 |
*((u16 *)(&gpu->device_handle[2]));
if (bdf == pci_id) {
@@ -2100,9 +2109,13 @@ static void kfd_find_numa_node_in_srat(struct kfd_node *kdev)
sub_header = (struct acpi_subtable_header *)
((unsigned long)sub_header + subtable_len);
- subtable_len = sub_header->length;
}
+ goto out;
+
+invalid_srat:
+ pr_err("SRAT contains an invalid subtable\n");
+out:
acpi_put_table(table_header);
/* Workaround bad cpu-gpu binding case */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 586e640f13dc..43029b2022c0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -1801,7 +1801,14 @@ void kgd2kfd_teardown_processes(struct amdgpu_device *adev)
mutex_unlock(&kfd_processes_mutex);
- /* wait all kfd processes use adev terminate */
+ /* kfd.dev may already be freed on surprise removal */
+ if (!adev->kfd.dev)
+ return;
+
+ /*
+ * Wait for all KFD processes using adev to terminate. Must be
+ * unbounded: device state is freed once they are gone.
+ */
while (!!atomic_read(&adev->kfd.dev->kfd_processes_count))
cond_resched();
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 4594824985e1..dd2c8ac7cb9c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -4046,6 +4046,15 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data)
queue++) {
r = dqm->dev->kfd2kgd->hqd_sdma_dump(
dqm->dev->adev, pipe, queue, &dump, &n_regs);
+ if (r == -EINVAL) {
+ /*
+ * SDMA engine not present/mapped on this
+ * device; skip it instead of aborting the
+ * whole dump.
+ */
+ r = 0;
+ break;
+ }
if (r)
break;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
index fdcf7f2d1b5b..6af268a031f2 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c
@@ -239,11 +239,9 @@ phys_addr_t kfd_get_process_doorbells(struct kfd_process_device *pdd)
struct amdgpu_device *adev = pdd->dev->adev;
uint32_t first_db_index;
- if (!pdd->qpd.proc_doorbells) {
- if (kfd_alloc_process_doorbells(pdd->dev->kfd, pdd))
- /* phys_addr_t 0 is error */
- return 0;
- }
+ if (kfd_alloc_process_doorbells(pdd->dev->kfd, pdd))
+ /* phys_addr_t 0 is error */
+ return 0;
first_db_index = amdgpu_doorbell_index_on_bar(adev,
pdd->qpd.proc_doorbells,
@@ -257,12 +255,21 @@ int kfd_alloc_process_doorbells(struct kfd_dev *kfd, struct kfd_process_device *
int r;
struct qcm_process_device *qpd = &pdd->qpd;
+ mutex_lock(&pdd->doorbell_mutex);
+
+ /* Another thread may have won the check-and-allocate race */
+ if (qpd->proc_doorbells) {
+ mutex_unlock(&pdd->doorbell_mutex);
+ return 0;
+ }
+
/* Allocate bitmap for dynamic doorbell allocation */
qpd->doorbell_bitmap = bitmap_zalloc(KFD_MAX_NUM_OF_QUEUES_PER_PROCESS,
GFP_KERNEL);
if (!qpd->doorbell_bitmap) {
DRM_ERROR("Failed to allocate process doorbell bitmap\n");
- return -ENOMEM;
+ r = -ENOMEM;
+ goto unlock;
}
r = init_doorbell_bitmap(&pdd->qpd, kfd);
@@ -284,11 +291,15 @@ int kfd_alloc_process_doorbells(struct kfd_dev *kfd, struct kfd_process_device *
DRM_ERROR("Failed to allocate process doorbells\n");
goto err;
}
+
+ mutex_unlock(&pdd->doorbell_mutex);
return 0;
err:
bitmap_free(qpd->doorbell_bitmap);
qpd->doorbell_bitmap = NULL;
+unlock:
+ mutex_unlock(&pdd->doorbell_mutex);
return r;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
index 38b46b696243..d63f7e9fae30 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
@@ -138,6 +138,15 @@ struct mqd_manager {
uint32_t ctl_stack_size;
};
+/* Keep offset <= size <= wave state offset, as init_mqd() and CP do */
+static inline void kfd_mqd_restore_ctl_stack(u32 *size, u32 *offset, u32 *wg_offset,
+ u32 ctl_stack_size)
+{
+ *size = ctl_stack_size;
+ *offset = min(*offset, ctl_stack_size);
+ *wg_offset = max(*wg_offset, ctl_stack_size);
+}
+
struct mqd_user_context_save_area_header {
/* Byte offset from start of user context
* save area to the last saved top (lowest
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
index 4f8a8a1a6186..6c9bafa3962e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
@@ -305,6 +305,8 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd,
addr = mqd_mem_obj->gpu_addr;
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
*mqd = m;
if (gart_addr)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
index bf015dc5b868..5709949de2eb 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v11.c
@@ -340,6 +340,8 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd,
addr = mqd_mem_obj->gpu_addr;
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
*mqd = m;
if (gart_addr)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
index 6ea09b031caf..aea80266d254 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12.c
@@ -393,6 +393,8 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd,
memset(m, 0, AMDGPU_MQD_SIZE_ALIGN(mm->mqd_size));
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
/* Update MQD base address to the newly allocated location */
m->cp_mqd_base_addr_lo = lower_32_bits(addr);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12_1.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12_1.c
index c709db0210ce..3c335d24492a 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12_1.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v12_1.c
@@ -663,6 +663,8 @@ static void restore_mqd_v12_1(struct mqd_manager *mm, void **mqd,
memset(m, 0, AMDGPU_MQD_SIZE_ALIGN(mm->mqd_size) *
NUM_XCC(mm->dev->xcc_mask));
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
/* Update MQD base address to the newly allocated location */
m->cp_mqd_base_addr_lo = lower_32_bits(addr);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index 6e6bc1ec0b64..fca43017ee7e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -462,6 +462,8 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd,
addr = mqd_mem_obj->gpu_addr;
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
*mqd = m;
if (gart_addr)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
index 029572548c14..5ead0d5613b6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
@@ -317,6 +317,8 @@ static void restore_mqd(struct mqd_manager *mm, void **mqd,
addr = mqd_mem_obj->gpu_addr;
memcpy(m, mqd_src, sizeof(*m));
+ kfd_mqd_restore_ctl_stack(&m->cp_hqd_cntl_stack_size, &m->cp_hqd_cntl_stack_offset,
+ &m->cp_hqd_wg_state_offset, qp->ctl_stack_size);
*mqd = m;
if (gart_addr)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 435bf2692968..f405f476ed18 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -786,6 +786,9 @@ struct kfd_process_device {
/* per-process-per device QCM data structure */
struct qcm_process_device qpd;
+ /* Serializes process doorbell allocation for this device */
+ struct mutex doorbell_mutex;
+
/*Apertures*/
uint64_t lds_base;
uint64_t lds_limit;
@@ -1132,6 +1135,7 @@ void kfd_unref_process(struct kfd_process *p);
int kfd_process_evict_queues(struct kfd_process *p, uint32_t trigger);
int kfd_process_restore_queues(struct kfd_process *p);
void kfd_suspend_all_processes(void);
+void kfd_process_wait_gpu_reset_complete(struct kfd_process *p);
int kfd_resume_all_processes(void);
struct kfd_process_device *kfd_process_device_data_by_id(struct kfd_process *process,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 0a7c1900da95..586a6dd53883 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -1231,6 +1231,7 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
}
atomic_dec(&pdd->dev->kfd->kfd_processes_count);
+ mutex_destroy(&pdd->doorbell_mutex);
kfree(pdd);
p->pdds[i] = NULL;
@@ -1291,7 +1292,7 @@ static void kfd_process_remove_sysfs(struct kfd_process *p)
/*
* If any GPU is ongoing reset, wait for reset complete.
*/
-static void kfd_process_wait_gpu_reset_complete(struct kfd_process *p)
+void kfd_process_wait_gpu_reset_complete(struct kfd_process *p)
{
int i;
@@ -1788,6 +1789,7 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_node *dev,
pdd->dev = dev;
INIT_LIST_HEAD(&pdd->qpd.queues_list);
INIT_LIST_HEAD(&pdd->qpd.priv_queue_list);
+ mutex_init(&pdd->doorbell_mutex);
pdd->qpd.dqm = dev->dqm;
pdd->qpd.pqm = &p->pqm;
pdd->qpd.evicted = 0;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index 4fe40e9fcfc8..87fb42d3c713 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -26,6 +26,7 @@
#include <linux/list.h>
#include "kfd_device_queue_manager.h"
#include "kfd_priv.h"
+#include "kfd_topology.h"
#include "kfd_kernel_queue.h"
#include "amdgpu_amdkfd.h"
#include "amdgpu_reset.h"
@@ -981,7 +982,7 @@ int kfd_criu_checkpoint_queues(struct kfd_process *p,
}
static void set_queue_properties_from_criu(struct queue_properties *qp,
- struct kfd_criu_queue_priv_data *q_data, uint32_t num_xcc)
+ struct kfd_criu_queue_priv_data *q_data)
{
qp->is_interop = false;
qp->queue_percent = q_data->q_percent;
@@ -994,11 +995,6 @@ static void set_queue_properties_from_criu(struct queue_properties *qp,
qp->eop_ring_buffer_size = q_data->eop_ring_buffer_size;
qp->ctx_save_restore_area_address = q_data->ctx_save_restore_area_address;
qp->ctx_save_restore_area_size = q_data->ctx_save_restore_area_size;
- if (q_data->type == KFD_QUEUE_TYPE_COMPUTE)
- qp->ctl_stack_size = q_data->ctl_stack_size / num_xcc;
- else
- qp->ctl_stack_size = q_data->ctl_stack_size;
-
qp->type = q_data->type;
qp->format = q_data->format;
}
@@ -1010,6 +1006,7 @@ int kfd_criu_restore_queue(struct kfd_process *p,
{
uint8_t *mqd, *ctl_stack, *q_extra_data = NULL;
struct kfd_criu_queue_priv_data *q_data;
+ struct kfd_topology_device *topo_dev;
struct kfd_process_device *pdd;
uint64_t q_extra_data_size;
struct queue_properties qp;
@@ -1046,6 +1043,24 @@ int kfd_criu_restore_queue(struct kfd_process *p,
goto exit;
}
+ memset(&qp, 0, sizeof(qp));
+ set_queue_properties_from_criu(&qp, q_data);
+
+ /* Per-XCC control stack size from topology, as create-queue requires */
+ if (qp.type == KFD_QUEUE_TYPE_COMPUTE && pdd->dev->kfd->cwsr_enabled) {
+ topo_dev = kfd_topology_device_by_id(pdd->dev->id);
+ if (!topo_dev) {
+ ret = -EINVAL;
+ goto exit;
+ }
+ qp.ctl_stack_size = topo_dev->node_props.ctl_stack_size;
+ }
+
+ if (q_data->ctl_stack_size > (u64)qp.ctl_stack_size * NUM_XCC(pdd->dev->xcc_mask)) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
*priv_data_offset += sizeof(*q_data);
q_extra_data_size = (uint64_t)q_data->ctl_stack_size + q_data->mqd_size;
@@ -1075,9 +1090,6 @@ int kfd_criu_restore_queue(struct kfd_process *p,
mqd = q_extra_data;
ctl_stack = mqd + q_data->mqd_size;
- memset(&qp, 0, sizeof(qp));
- set_queue_properties_from_criu(&qp, q_data, NUM_XCC(pdd->dev->xcc_mask));
-
print_queue_properties(&qp);
ret = pqm_create_queue(&p->pqm, pdd->dev, &qp, &queue_id, q_data, mqd, ctl_stack, NULL);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
index 15eeaaebbbcd..910d695bb962 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_queue.c
@@ -230,6 +230,15 @@ void kfd_queue_buffer_put(struct amdgpu_bo **bo)
amdgpu_bo_unref(bo);
}
+static void kfd_queue_put_bos(struct queue_properties *properties)
+{
+ kfd_queue_buffer_put(&properties->wptr_bo);
+ kfd_queue_buffer_put(&properties->rptr_bo);
+ kfd_queue_buffer_put(&properties->ring_bo);
+ kfd_queue_buffer_put(&properties->eop_buf_bo);
+ kfd_queue_buffer_put(&properties->cwsr_bo);
+}
+
int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_properties *properties)
{
struct kfd_topology_device *topo_dev;
@@ -343,7 +352,13 @@ int kfd_queue_acquire_buffers(struct kfd_process_device *pdd, struct queue_prope
* dropping the VM reservation.
*/
kfd_queue_unref_bo_vas(pdd, properties);
- kfd_queue_release_buffers(pdd, properties);
+ /*
+ * The SVM CWSR range is the last thing acquired and a failed
+ * kfd_queue_buffer_svm_get() takes nothing, so only the BOs are held
+ * here. Dropping the SVM reference would drop the one of another queue
+ * using the same CWSR address.
+ */
+ kfd_queue_put_bos(properties);
return err;
}
@@ -352,11 +367,7 @@ int kfd_queue_release_buffers(struct kfd_process_device *pdd, struct queue_prope
struct kfd_topology_device *topo_dev;
u64 total_cwsr_size;
- kfd_queue_buffer_put(&properties->wptr_bo);
- kfd_queue_buffer_put(&properties->rptr_bo);
- kfd_queue_buffer_put(&properties->ring_bo);
- kfd_queue_buffer_put(&properties->eop_buf_bo);
- kfd_queue_buffer_put(&properties->cwsr_bo);
+ kfd_queue_put_bos(properties);
topo_dev = kfd_topology_device_by_id(pdd->dev->id);
if (!topo_dev)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
index 6a7b4d959541..c87370a9f5f0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
@@ -40,7 +40,7 @@ struct kfd_smi_client {
struct kfd_node *dev;
spinlock_t lock;
struct rcu_head rcu;
- pid_t pid;
+ pid_t tgid;
bool suser;
};
@@ -160,18 +160,18 @@ static int kfd_smi_ev_release(struct inode *inode, struct file *filep)
return 0;
}
-static bool kfd_smi_ev_enabled(pid_t pid, struct kfd_smi_client *client,
+static bool kfd_smi_ev_enabled(pid_t tgid, struct kfd_smi_client *client,
unsigned int event)
{
uint64_t events = READ_ONCE(client->events);
- if (pid && client->pid != pid && !client->suser)
+ if (tgid && client->tgid != tgid && !client->suser)
return false;
return events & KFD_SMI_EVENT_MASK_FROM_INDEX(event);
}
-static void add_event_to_kfifo(pid_t pid, struct kfd_node *dev,
+static void add_event_to_kfifo(pid_t tgid, struct kfd_node *dev,
unsigned int smi_event, char *event_msg, int len)
{
struct kfd_smi_client *client;
@@ -179,7 +179,7 @@ static void add_event_to_kfifo(pid_t pid, struct kfd_node *dev,
rcu_read_lock();
list_for_each_entry_rcu(client, &dev->smi_clients, list) {
- if (!kfd_smi_ev_enabled(pid, client, smi_event))
+ if (!kfd_smi_ev_enabled(tgid, client, smi_event))
continue;
spin_lock(&client->lock);
if (kfifo_avail(&client->fifo) >= len) {
@@ -210,27 +210,33 @@ static inline pid_t kfd_smi_task_to_pid(struct task_struct *task)
return task ? task_tgid_nr_ns(task, task_active_pid_ns(task)) : 0;
}
+/*
+ * Clients are matched by global TGID, which is unique across PID namespaces
+ * and stays valid after the task exited. Events show kfd_smi_task_to_pid().
+ */
+static inline pid_t kfd_smi_task_tgid(struct task_struct *task)
+{
+ return task ? task_tgid_nr(task) : 0;
+}
+
__printf(4, 5)
-static void kfd_smi_event_add(struct task_struct *task, struct kfd_node *dev,
+static void kfd_smi_event_add(pid_t tgid, struct kfd_node *dev,
unsigned int event, char *fmt, ...)
{
char fifo_in[KFD_SMI_EVENT_MSG_SIZE];
int len;
va_list args;
- pid_t pid;
if (list_empty(&dev->smi_clients))
return;
- pid = kfd_smi_task_to_pid(task);
-
len = scnprintf(fifo_in, sizeof(fifo_in), "%x ", event);
va_start(args, fmt);
len += vscnprintf(fifo_in + len, sizeof(fifo_in) - len, fmt, args);
va_end(args);
- add_event_to_kfifo(pid, dev, event, fifo_in, len);
+ add_event_to_kfifo(tgid, dev, event, fifo_in, len);
}
void kfd_smi_event_update_gpu_reset(struct kfd_node *dev, bool post_reset,
@@ -252,14 +258,14 @@ void kfd_smi_event_update_gpu_reset(struct kfd_node *dev, bool post_reset,
amdgpu_reset_get_desc(reset_context, reset_cause,
sizeof(reset_cause));
- kfd_smi_event_add(NULL, dev, event, KFD_EVENT_FMT_UPDATE_GPU_RESET(
+ kfd_smi_event_add(0, dev, event, KFD_EVENT_FMT_UPDATE_GPU_RESET(
dev->reset_seq_num, reset_cause));
}
void kfd_smi_event_update_thermal_throttling(struct kfd_node *dev,
uint64_t throttle_bitmask)
{
- kfd_smi_event_add(NULL, dev, KFD_SMI_EVENT_THERMAL_THROTTLE,
+ kfd_smi_event_add(0, dev, KFD_SMI_EVENT_THERMAL_THROTTLE,
KFD_EVENT_FMT_THERMAL_THROTTLING(
throttle_bitmask,
amdgpu_dpm_get_thermal_throttling_counter(dev->adev)));
@@ -271,10 +277,14 @@ void kfd_smi_event_update_vmfault(struct kfd_node *dev, uint16_t pasid)
task_info = amdgpu_vm_get_task_info_pasid(dev->adev, pasid);
if (task_info) {
- /* Report VM faults from user applications, not retry from kernel */
+ /*
+ * Report VM faults from user applications, not retry from
+ * kernel, and only to clients allowed to see that process.
+ */
if (task_info->task.pid)
- kfd_smi_event_add(NULL, dev, KFD_SMI_EVENT_VMFAULT, KFD_EVENT_FMT_VMFAULT(
- task_info->task.pid, task_info->task.comm));
+ kfd_smi_event_add(task_info->tgid, dev, KFD_SMI_EVENT_VMFAULT,
+ KFD_EVENT_FMT_VMFAULT(task_info->task.pid,
+ task_info->task.comm));
amdgpu_vm_put_task_info(task_info);
}
}
@@ -283,7 +293,7 @@ void kfd_smi_event_page_fault_start(struct kfd_node *node, struct task_struct *t
unsigned long address, bool write_fault,
ktime_t ts)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_PAGE_FAULT_START,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_PAGE_FAULT_START,
KFD_EVENT_FMT_PAGEFAULT_START(ktime_to_ns(ts),
kfd_smi_task_to_pid(task), address, node->id,
write_fault ? 'W' : 'R'));
@@ -292,7 +302,7 @@ void kfd_smi_event_page_fault_start(struct kfd_node *node, struct task_struct *t
void kfd_smi_event_page_fault_end(struct kfd_node *node, struct task_struct *task,
unsigned long address, bool migration)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_PAGE_FAULT_END,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_PAGE_FAULT_END,
KFD_EVENT_FMT_PAGEFAULT_END(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), address, node->id,
migration ? 'M' : 'U'));
@@ -304,7 +314,7 @@ void kfd_smi_event_migration_start(struct kfd_node *node, struct task_struct *ta
uint32_t prefetch_loc, uint32_t preferred_loc,
uint32_t trigger)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_MIGRATE_START,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_MIGRATE_START,
KFD_EVENT_FMT_MIGRATE_START(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), start, end - start, from,
to, prefetch_loc, preferred_loc, trigger));
@@ -315,7 +325,7 @@ void kfd_smi_event_migration_end(struct kfd_node *node, struct task_struct *task
uint32_t from, uint32_t to, uint32_t trigger,
int error_code)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_MIGRATE_END,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_MIGRATE_END,
KFD_EVENT_FMT_MIGRATE_END(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), start, end - start, from,
to, trigger, error_code));
@@ -324,14 +334,14 @@ void kfd_smi_event_migration_end(struct kfd_node *node, struct task_struct *task
void kfd_smi_event_queue_eviction(struct kfd_node *node, struct task_struct *task,
uint32_t trigger)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_QUEUE_EVICTION,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_QUEUE_EVICTION,
KFD_EVENT_FMT_QUEUE_EVICTION(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), node->id, trigger));
}
void kfd_smi_event_queue_restore(struct kfd_node *node, struct task_struct *task)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_QUEUE_RESTORE,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_QUEUE_RESTORE,
KFD_EVENT_FMT_QUEUE_RESTORE(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), node->id, '0'));
}
@@ -348,7 +358,7 @@ void kfd_smi_event_queue_restore_rescheduled(struct mm_struct *mm)
for (i = 0; i < p->n_pdds; i++) {
struct kfd_process_device *pdd = p->pdds[i];
- kfd_smi_event_add(p->lead_thread, pdd->dev,
+ kfd_smi_event_add(kfd_smi_task_tgid(p->lead_thread), pdd->dev,
KFD_SMI_EVENT_QUEUE_RESTORE,
KFD_EVENT_FMT_QUEUE_RESTORE(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(p->lead_thread),
@@ -361,7 +371,7 @@ void kfd_smi_event_unmap_from_gpu(struct kfd_node *node, struct task_struct *tas
unsigned long address, unsigned long last,
uint32_t trigger)
{
- kfd_smi_event_add(task, node, KFD_SMI_EVENT_UNMAP_FROM_GPU,
+ kfd_smi_event_add(kfd_smi_task_tgid(task), node, KFD_SMI_EVENT_UNMAP_FROM_GPU,
KFD_EVENT_FMT_UNMAP_FROM_GPU(ktime_get_boottime_ns(),
kfd_smi_task_to_pid(task), address,
last - address + 1, node->id, trigger));
@@ -379,7 +389,7 @@ void kfd_smi_event_process(struct kfd_process_device *pdd, bool start)
task_info = amdgpu_vm_get_task_info_vm(avm);
if (task_info) {
- kfd_smi_event_add(NULL, pdd->dev,
+ kfd_smi_event_add(kfd_smi_task_tgid(pdd->process->lead_thread), pdd->dev,
start ? KFD_SMI_EVENT_PROCESS_START :
KFD_SMI_EVENT_PROCESS_END,
KFD_EVENT_FMT_PROCESS(task_info->task.pid,
@@ -408,7 +418,7 @@ int kfd_smi_event_open(struct kfd_node *dev, uint32_t *fd)
spin_lock_init(&client->lock);
client->events = 0;
client->dev = dev;
- client->pid = kfd_smi_task_to_pid(current);
+ client->tgid = kfd_smi_task_tgid(current);
client->suser = capable(CAP_SYS_ADMIN);
spin_lock(&dev->smi_lock);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index d7f2ff5c8f02..ff7a895bdd7c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -1019,9 +1019,12 @@ svm_range_split_pages(struct svm_range *new, struct svm_range *old,
uint64_t start, uint64_t last)
{
uint64_t npages = last - start + 1;
+ bool has_dma_addr = false;
int i, r;
for (i = 0; i < MAX_GPU_INSTANCE; i++) {
+ if (old->dma_addr[i])
+ has_dma_addr = true;
r = svm_range_split_array(&new->dma_addr[i], &old->dma_addr[i],
sizeof(*old->dma_addr[i]), old->start,
npages, new->start, new->npages,
@@ -1029,8 +1032,16 @@ svm_range_split_pages(struct svm_range *new, struct svm_range *old,
if (r)
return r;
}
- if (old->actual_loc)
+ if (old->actual_loc) {
+ /*
+ * Pages migrated by prefetch have no dma_addr until they are
+ * mapped on a GPU. A fully resident range still splits exactly,
+ * old->npages isn't adjusted yet here.
+ */
+ if (!has_dma_addr && old->vram_pages == old->npages)
+ new->vram_pages = new->npages;
old->vram_pages -= new->vram_pages;
+ }
return 0;
}
@@ -1942,7 +1953,7 @@ static void svm_range_restore_work(struct work_struct *work)
struct mm_struct *mm;
int evicted_ranges;
int invalid;
- int r;
+ int r = 0;
svms = container_of(dwork, struct svm_range_list, restore_work);
evicted_ranges = atomic_read(&svms->evicted_ranges);
@@ -2016,8 +2027,10 @@ static void svm_range_restore_work(struct work_struct *work)
mmap_write_unlock(mm);
mutex_unlock(&process_info->lock);
- /* If validation failed, reschedule another attempt */
- if (evicted_ranges) {
+ /* If validation failed, reschedule another attempt, unless the
+ * device is gone for good
+ */
+ if (evicted_ranges && r != -ENODEV) {
pr_debug("reschedule to restore svm range\n");
queue_delayed_work(system_freezable_wq, &svms->restore_work,
msecs_to_jiffies(AMDGPU_SVM_RANGE_RESTORE_DELAY_MS));
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 47a0ff2ed035..f3551bd0b0a4 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -1979,10 +1979,6 @@ static int dm_resume(struct amdgpu_ip_block *ip_block)
/* On resume we need to rewrite the MSTM control bits to enable MST*/
s3_handle_mst(ddev, false);
- /* Exit IPS before the detection loop's first AUX/DDC access. */
- scoped_guard(mutex, &dm->dc_lock)
- dc_exit_ips_for_hw_access(dm->dc);
-
/* Do detection*/
drm_connector_list_iter_begin(ddev, &iter);
drm_for_each_connector_iter(connector, &iter) {
@@ -4597,8 +4593,6 @@ static void amdgpu_dm_commit_streams(struct drm_atomic_commit *state,
if (dm_old_crtc_state->stream)
remove_stream(adev, acrtc, dm_old_crtc_state->stream);
- pm_runtime_get_noresume(dev->dev);
-
acrtc->enabled = true;
acrtc->hw_mode = new_crtc_state->mode;
crtc->hwmode = new_crtc_state->mode;
@@ -5178,8 +5172,13 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_commit *state)
#ifdef CONFIG_DEBUG_FS
enum amdgpu_dm_pipe_crc_source cur_crc_src;
#endif
- /* Count number of newly disabled CRTCs for dropping PM refs later. */
- if (old_crtc_state->active && !new_crtc_state->active)
+ /*
+ * One runtime PM reference per active CRTC: take it when the
+ * CRTC turns on, count it when it turns off for dropping later.
+ */
+ if (!old_crtc_state->active && new_crtc_state->active)
+ pm_runtime_get_noresume(dev->dev);
+ else if (old_crtc_state->active && !new_crtc_state->active)
crtc_disable_count++;
dm_new_crtc_state = to_dm_crtc_state(new_crtc_state);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index ced8b3d2d762..673b3cede7dd 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -283,7 +283,7 @@ fill_dc_mst_payload_table_from_drm(struct dc_link *link,
break;
}
}
- ASSERT(i != copy_of_link_table.stream_count);
+ /* Payload absent from HW table when never allocated: nothing to remove. */
}
/* Fill payload info*/
@@ -1218,6 +1218,26 @@ void populate_hdmi_info_from_connector(bool enable_frl, struct drm_hdmi_info *hd
}
EXPORT_IF_KUNIT(populate_hdmi_info_from_connector);
+static void dm_helpers_wait_for_edid_ready(struct dc_link *link,
+ struct i2c_adapter *ddc)
+{
+ const unsigned int retry_count = 50;
+ const unsigned int retry_interval_ms = 20;
+ unsigned int retry;
+
+ if (!ddc || link->dpcd_caps.dongle_type !=
+ DISPLAY_DONGLE_DP_DVI_DONGLE)
+ return;
+
+ for (retry = 0; retry < retry_count; retry++) {
+ if (drm_probe_ddc(ddc))
+ return;
+
+ if (retry + 1 < retry_count)
+ msleep(retry_interval_ms);
+ }
+}
+
enum dc_edid_status dm_helpers_read_local_edid(
struct dc_context *ctx,
struct dc_link *link,
@@ -1242,6 +1262,8 @@ enum dc_edid_status dm_helpers_read_local_edid(
if (link->dc->hwss.prepare_ddc)
link->dc->hwss.prepare_ddc(link);
+ dm_helpers_wait_for_edid_ready(link, ddc);
+
/* some dongles read edid incorrectly the first time,
* do check sum and retry to make sure read correct edid.
*/
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
index b8fc27a5361e..657173d8081c 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dsc/rc_calc_fpu.c
@@ -46,7 +46,7 @@ static int median3(int a, int b, int c)
if (b > c)
swap(b, c);
if (a > b)
- swap(b, c);
+ swap(a, b);
return b;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.c b/drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.c
index 72acf12e9251..0ee385e8ba5f 100644
--- a/drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.c
+++ b/drivers/gpu/drm/amd/display/dc/dml2_0/dml21/src/dml2_core/dml2_core_dcn4_calcs.c
@@ -11074,6 +11074,8 @@ static bool dml_core_mode_programming(struct dml2_core_calcs_mode_programming_ex
calculate_tdlut_setting_params->cursor_buffer_size = mode_lib->ip.cursor_buffer_size;
calculate_tdlut_setting_params->gpuvm_enable = display_cfg->gpuvm_enable;
calculate_tdlut_setting_params->gpuvm_page_size_kbytes = display_cfg->plane_descriptors[k].overrides.gpuvm_min_page_size_kbytes;
+ calculate_tdlut_setting_params->tdlut_mpc_width_flag = display_cfg->plane_descriptors[k].tdlut.tdlut_mpc_width_flag;
+ calculate_tdlut_setting_params->is_gfx11 = dml_get_gfx_version(display_cfg->plane_descriptors[k].surface.tiling);
// output
calculate_tdlut_setting_params->tdlut_pte_bytes_per_frame = &s->tdlut_pte_bytes_per_frame[k];
diff --git a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
index 03d88e78165d..5786ef5787ac 100644
--- a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
+++ b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
@@ -1696,7 +1696,8 @@ bool mod_color_calculate_degamma_params(struct dc_color_caps *dc_caps,
return true;
if (dc_caps != NULL &&
- dc_caps->dpp.dcn_arch == 1) {
+ dc_caps->dpp.dcn_arch == 1 &&
+ !map_user_ramp) {
if (input_tf->tf == TRANSFER_FUNCTION_PQ &&
dc_caps->dpp.dgam_rom_caps.pq == 1)
diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
index 606d48115301..4ec330dd8ead 100644
--- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h
+++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h
@@ -433,6 +433,8 @@ struct amd_pm_funcs {
int (*print_clock_levels)(void *handle, enum pp_clock_type type, char *buf);
int (*emit_clock_levels)(void *handle, enum pp_clock_type type, char *buf, int *offset);
int (*force_performance_level)(void *handle, enum amd_dpm_forced_level level);
+ int (*reapply_performance_level)(void *handle,
+ enum amd_dpm_forced_level level);
int (*get_sclk_od)(void *handle);
int (*set_sclk_od)(void *handle, uint32_t value);
int (*get_mclk_od)(void *handle);
diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
index 808be6c425bf..5615af97a349 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
@@ -1085,6 +1085,91 @@ static void amdgpu_dpm_exit_umd_state(struct amdgpu_device *adev)
AMD_PG_STATE_GATE);
}
+/**
+ * amdgpu_dpm_reapply_umd_profile - Restore the active UMD profile
+ * @adev: amdgpu device
+ *
+ * Resume and reset retain the selected DPM level in software, but the
+ * underlying GFX and PMFW state is lost. Reapply a retained UMD profile
+ * after the common resume path has restored its default CG and PG state.
+ */
+int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev)
+{
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
+ enum amd_dpm_forced_level level;
+ int ret, fallback_ret;
+
+ if (!pp_funcs || !pp_funcs->reapply_performance_level ||
+ !pp_funcs->force_performance_level)
+ return 0;
+
+ /* Serialize the retained level with the GFX and PMFW reprogramming. */
+ mutex_lock(&adev->pm.mutex);
+ if (adev->pm.dpm.thermal_active) {
+ dev_warn(adev->dev,
+ "Cannot restore UMD profile while thermal control is active\n");
+ ret = 0;
+ goto unlock;
+ }
+
+ if (pp_funcs->get_performance_level)
+ level = pp_funcs->get_performance_level(adev->powerplay.pp_handle);
+ else
+ level = adev->pm.dpm.forced_level;
+
+ switch (level) {
+ case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+ break;
+ default:
+ ret = 0;
+ goto unlock;
+ }
+
+ /* The active profile already owns the GFX PG reference. */
+ ret = amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_UNGATE);
+ if (ret)
+ goto fallback_auto;
+
+ ret = amdgpu_asic_update_umd_stable_pstate(adev, true);
+ if (ret)
+ goto fallback_auto;
+
+ ret = pp_funcs->reapply_performance_level(adev->powerplay.pp_handle,
+ level);
+ if (ret)
+ goto fallback_auto;
+
+unlock:
+ mutex_unlock(&adev->pm.mutex);
+
+ return ret;
+
+fallback_auto:
+ dev_warn(adev->dev, "Failed to restore UMD profile state: %d\n", ret);
+
+ /* Restore normal CG before releasing the profile's PG reference. */
+ amdgpu_asic_update_umd_stable_pstate(adev, false);
+ amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_GFX,
+ AMD_CG_STATE_GATE);
+ mutex_unlock(&adev->pm.mutex);
+
+ mutex_lock(&adev->pm.stable_pstate_ctx_lock);
+ fallback_ret = amdgpu_dpm_force_performance_level(adev,
+ AMD_DPM_FORCED_LEVEL_AUTO);
+ if (fallback_ret)
+ dev_warn(adev->dev, "Failed to fall back to AUTO performance level: %d\n",
+ fallback_ret);
+ else
+ adev->pm.stable_pstate_ctx = NULL;
+ mutex_unlock(&adev->pm.stable_pstate_ctx_lock);
+
+ return 0;
+}
+
int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
enum amd_dpm_forced_level level)
{
diff --git a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
index 8e8a052e1d53..3016f0d19f74 100644
--- a/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
+++ b/drivers/gpu/drm/amd/pm/inc/amdgpu_dpm.h
@@ -482,6 +482,7 @@ void amdgpu_dpm_set_power_state(struct amdgpu_device *adev,
enum amd_dpm_forced_level amdgpu_dpm_get_performance_level(struct amdgpu_device *adev);
int amdgpu_dpm_force_performance_level(struct amdgpu_device *adev,
enum amd_dpm_forced_level level);
+int amdgpu_dpm_reapply_umd_profile(struct amdgpu_device *adev);
int amdgpu_dpm_get_pp_num_states(struct amdgpu_device *adev,
struct pp_states_info *states);
int amdgpu_dpm_dispatch_task(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
index 7c70e228a5ba..59494959682b 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c
@@ -364,24 +364,46 @@ static void pp_dpm_en_umd_pstate(struct pp_hwmgr *hwmgr,
}
}
-static int pp_dpm_force_performance_level(void *handle,
- enum amd_dpm_forced_level level)
+static int pp_dpm_apply_performance_level(struct pp_hwmgr *hwmgr,
+ enum amd_dpm_forced_level level,
+ bool reapply)
{
- struct pp_hwmgr *hwmgr = handle;
+ int ret;
if (!hwmgr || !hwmgr->pm_en)
return -EINVAL;
- if (level == hwmgr->dpm_level)
+ if (!reapply && level == hwmgr->dpm_level)
return 0;
pp_dpm_en_umd_pstate(hwmgr, &level);
hwmgr->request_dpm_level = level;
- hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL);
+ ret = hwmgr_handle_task(hwmgr, AMD_PP_TASK_READJUST_POWER_STATE, NULL);
+ if (ret)
+ return ret;
+
+ ret = phm_force_dpm_levels(hwmgr, level);
+ if (ret)
+ return ret;
+
+ hwmgr->dpm_level = level;
return 0;
}
+static int pp_dpm_force_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ return pp_dpm_apply_performance_level(handle, level, false);
+}
+
+static int
+pp_dpm_reapply_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ return pp_dpm_apply_performance_level(handle, level, true);
+}
+
static enum amd_dpm_forced_level pp_dpm_get_performance_level(
void *handle)
{
@@ -1542,6 +1564,7 @@ static const struct amd_pm_funcs pp_dpm_funcs = {
.load_firmware = pp_dpm_load_fw,
.wait_for_fw_loading_complete = pp_dpm_fw_loading_complete,
.force_performance_level = pp_dpm_force_performance_level,
+ .reapply_performance_level = pp_dpm_reapply_performance_level,
.get_performance_level = pp_dpm_get_performance_level,
.get_current_power_state = pp_dpm_get_current_power_state,
.dispatch_tasks = pp_dpm_dispatch_tasks,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index cca32f74abf0..9e70a0f3ae30 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -2353,6 +2353,13 @@ static int smu_set_powergating_state(struct amdgpu_ip_block *ip_block,
return 0;
}
+static void smu_enter_umd_pstate(struct smu_context *smu)
+{
+ smu_gpo_control(smu, false);
+ smu_gfx_ulv_control(smu, false);
+ smu_deep_sleep_control(smu, false);
+}
+
static int smu_enable_umd_pstate(void *handle,
enum amd_dpm_forced_level *level)
{
@@ -2371,9 +2378,7 @@ static int smu_enable_umd_pstate(void *handle,
/* enter umd pstate, save current level, disable gfx cg*/
if (*level & profile_mode_mask) {
smu_dpm_ctx->saved_dpm_level = smu_dpm_ctx->dpm_level;
- smu_gpo_control(smu, false);
- smu_gfx_ulv_control(smu, false);
- smu_deep_sleep_control(smu, false);
+ smu_enter_umd_pstate(smu);
amdgpu_asic_update_umd_stable_pstate(smu->adev, true);
}
} else {
@@ -2432,7 +2437,8 @@ static void smu_power_profile_mode_put(struct smu_context *smu,
static int smu_adjust_power_state_dynamic(struct smu_context *smu,
enum amd_dpm_forced_level level,
- bool skip_display_settings)
+ bool skip_display_settings,
+ bool force_performance_level)
{
int ret = 0;
struct smu_dpm_context *smu_dpm_ctx = &(smu->smu_dpm);
@@ -2459,7 +2465,7 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu,
}
}
- if (smu_dpm_ctx->dpm_level != level) {
+ if (force_performance_level || smu_dpm_ctx->dpm_level != level) {
ret = smu_asic_set_performance_level(smu, level);
if (ret) {
if (ret == -EOPNOTSUPP)
@@ -2482,6 +2488,39 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu,
return ret;
}
+static int
+smu_reapply_power_state_dynamic(struct smu_context *smu,
+ enum amd_dpm_forced_level level)
+{
+ struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
+ int ret;
+
+ ret = smu_apply_clocks_adjust_rules(smu);
+ if (ret) {
+ dev_err(smu->adev->dev, "Failed to apply clocks adjust rules!");
+ return ret;
+ }
+
+ ret = smu_asic_set_performance_level(smu, level);
+ if (ret) {
+ if (ret == -EOPNOTSUPP)
+ dev_info(smu->adev->dev, "set performance level %d not supported",
+ level);
+ else
+ dev_err(smu->adev->dev, "Failed to set performance level %d",
+ level);
+ return ret;
+ }
+
+ smu_dpm_ctx->dpm_level = level;
+
+ if (smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL ||
+ smu_dpm_ctx->dpm_level == AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM)
+ return 0;
+
+ return smu_bump_power_profile_mode(smu, NULL, 0);
+}
+
static int smu_handle_task(struct smu_context *smu,
enum amd_dpm_forced_level level,
enum amd_pp_task task_id)
@@ -2496,13 +2535,13 @@ static int smu_handle_task(struct smu_context *smu,
ret = smu_pre_display_config_changed(smu);
if (ret)
return ret;
- ret = smu_adjust_power_state_dynamic(smu, level, false);
+ ret = smu_adjust_power_state_dynamic(smu, level, false, false);
break;
case AMD_PP_TASK_COMPLETE_INIT:
- ret = smu_adjust_power_state_dynamic(smu, level, true);
+ ret = smu_adjust_power_state_dynamic(smu, level, true, false);
break;
case AMD_PP_TASK_READJUST_POWER_STATE:
- ret = smu_adjust_power_state_dynamic(smu, level, true);
+ ret = smu_adjust_power_state_dynamic(smu, level, true, false);
break;
default:
break;
@@ -2631,6 +2670,37 @@ static int smu_force_performance_level(void *handle,
return ret;
}
+static int
+smu_reapply_performance_level(void *handle,
+ enum amd_dpm_forced_level level)
+{
+ struct smu_context *smu = handle;
+ struct smu_dpm_context *smu_dpm_ctx = &smu->smu_dpm;
+
+ if (!smu->pm_enabled || !smu->adev->pm.dpm_enabled)
+ return -EOPNOTSUPP;
+
+ if (!smu->is_apu && !smu_dpm_ctx->dpm_context)
+ return -EINVAL;
+
+ switch (level) {
+ case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
+ case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (smu_dpm_ctx->dpm_level != level)
+ return -EINVAL;
+
+ smu_enter_umd_pstate(smu);
+
+ return smu_reapply_power_state_dynamic(smu, level);
+}
+
static int smu_set_display_count(void *handle, uint32_t count)
{
struct smu_context *smu = handle;
@@ -3988,6 +4058,7 @@ static const struct amd_pm_funcs swsmu_pm_funcs = {
.force_clock_level = smu_force_ppclk_levels,
.emit_clock_levels = smu_emit_ppclk_levels,
.force_performance_level = smu_force_performance_level,
+ .reapply_performance_level = smu_reapply_performance_level,
.read_sensor = smu_read_sensor,
.get_apu_thermal_limit = smu_get_apu_thermal_limit,
.set_apu_thermal_limit = smu_set_apu_thermal_limit,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
index 3ec65630ab99..e354abfef2eb 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
+++ b/drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
@@ -433,6 +433,8 @@ struct smu_table_context {
void *boot_overdrive_table;
void *user_overdrive_table;
+ uint32_t user_od_feature_mask;
+
struct smu_driver_table driver_tables[SMU_DRIVER_TABLE_COUNT];
};
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
index a29e76b03476..aaafe07ffdad 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
@@ -63,6 +63,9 @@ static void smu_v13_0_0_get_od_setting_limits(struct smu_context *smu,
int od_feature_bit,
int32_t *min, int32_t *max);
static int smu_v13_0_0_init_ppt_limits(struct smu_context *smu);
+static int
+smu_v13_0_0_upload_overdrive_table(struct smu_context *smu,
+ OverDriveTableExternal_t *od_table);
static const struct smu_feature_bits smu_v13_0_0_dpm_features = {
.bits = {
@@ -1079,6 +1082,93 @@ static bool smu_v13_0_0_is_od_feature_supported(struct smu_context *smu,
return feature_enabled;
}
+static u32
+smu_v13_0_0_filter_od_feature_mask(struct smu_context *smu, u32 feature_mask)
+{
+ PPTable_t *pptable = smu->smu_table.driver_pptable;
+ u32 supported_mask =
+ pptable->SkuTable.OverDriveLimitsBasicMax.FeatureCtrlMask;
+
+ return feature_mask & supported_mask;
+}
+
+static bool smu_v13_0_0_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return !memcmp(od->FanLinearPwmPoints, boot->FanLinearPwmPoints,
+ sizeof(od->FanLinearPwmPoints)) &&
+ !memcmp(od->FanLinearTempPoints, boot->FanLinearTempPoints,
+ sizeof(od->FanLinearTempPoints)) &&
+ od->FanMinimumPwm == boot->FanMinimumPwm &&
+ od->AcousticTargetRpmThreshold == boot->AcousticTargetRpmThreshold &&
+ od->AcousticLimitRpmThreshold == boot->AcousticLimitRpmThreshold &&
+ od->FanTargetTemperature == boot->FanTargetTemperature &&
+ od->FanMode == boot->FanMode;
+}
+
+static bool smu_v13_0_0_zero_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return od->FanZeroRpmEnable == boot->FanZeroRpmEnable &&
+ od->FanZeroRpmStopTemp == boot->FanZeroRpmStopTemp;
+}
+
+static u32 smu_v13_0_0_normalize_feature_mask(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot,
+ u32 feature_mask)
+{
+ if (od->GfxclkFmin == boot->GfxclkFmin &&
+ od->GfxclkFmax == boot->GfxclkFmax)
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFXCLK_BIT);
+ if (od->UclkFmin == boot->UclkFmin &&
+ od->UclkFmax == boot->UclkFmax)
+ feature_mask &= ~BIT(PP_OD_FEATURE_UCLK_BIT);
+ if (!memcmp(od->VoltageOffsetPerZoneBoundary,
+ boot->VoltageOffsetPerZoneBoundary,
+ sizeof(od->VoltageOffsetPerZoneBoundary)))
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT);
+ if (smu_v13_0_0_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ if (smu_v13_0_0_zero_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ if (!od->Ppt)
+ feature_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+
+ return feature_mask;
+}
+
+static int
+smu_v13_0_0_upload_ppt_overdrive_table(struct smu_context *smu, s16 ppt)
+{
+ struct smu_table_context *table_context = &smu->smu_table;
+ OverDriveTableExternal_t *od_table = table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask = table_context->user_od_feature_mask;
+ int ret;
+
+ /* Do not commit OD settings staged after the last c operation. */
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.Ppt = ppt;
+ if (ppt)
+ next_mask |= BIT(PP_OD_FEATURE_PPT_BIT);
+ else
+ next_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+ upload_table.OverDriveTable.FeatureCtrlMask =
+ smu_v13_0_0_filter_od_feature_mask(smu, next_mask);
+
+ ret = smu_v13_0_0_upload_overdrive_table(smu, &upload_table);
+ if (ret)
+ return ret;
+
+ table_context->user_od_feature_mask =
+ upload_table.OverDriveTable.FeatureCtrlMask;
+ od_table->OverDriveTable.Ppt = ppt;
+ user_od_table->OverDriveTable.Ppt = ppt;
+
+ return 0;
+}
+
static void smu_v13_0_0_get_od_setting_limits(struct smu_context *smu,
int od_feature_bit,
int32_t *min,
@@ -1514,41 +1604,30 @@ static int smu_v13_0_0_od_restore_table_single(struct smu_context *smu, long inp
boot_overdrive_table->OverDriveTable.FanLinearPwmPoints[i];
}
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_LIMIT:
od_table->OverDriveTable.AcousticLimitRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticLimitRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_TARGET:
od_table->OverDriveTable.AcousticTargetRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticTargetRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_TARGET_TEMPERATURE:
od_table->OverDriveTable.FanTargetTemperature =
boot_overdrive_table->OverDriveTable.FanTargetTemperature;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_MINIMUM_PWM:
od_table->OverDriveTable.FanMinimumPwm =
boot_overdrive_table->OverDriveTable.FanMinimumPwm;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_ENABLE:
od_table->OverDriveTable.FanZeroRpmEnable =
boot_overdrive_table->OverDriveTable.FanZeroRpmEnable;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_STOP_TEMP:
od_table->OverDriveTable.FanZeroRpmStopTemp =
boot_overdrive_table->OverDriveTable.FanZeroRpmStopTemp;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
default:
dev_info(adev->dev, "Invalid table index: %ld\n", input);
@@ -1566,10 +1645,17 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table =
(OverDriveTableExternal_t *)table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table =
+ (OverDriveTableExternal_t *)table_context->user_overdrive_table;
+ OverDriveTableExternal_t *boot_od_table =
+ (OverDriveTableExternal_t *)table_context->boot_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ OverDriveTable_t *od = &od_table->OverDriveTable;
+ OverDriveTable_t *boot = &boot_od_table->OverDriveTable;
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
+ u32 next_mask;
int i, ret = 0;
switch (type) {
@@ -1891,44 +1977,43 @@ static int smu_v13_0_0_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
- sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
- if (memcmp((u8 *)od_table + offset_of_voltageoffset,
- table_context->user_overdrive_table + offset_of_voltageoffset,
- sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset)) {
- smu_v13_0_0_dump_od_table(smu, od_table);
+ next_mask = smu_v13_0_0_filter_od_feature_mask(smu,
+ table_context->user_od_feature_mask |
+ od->FeatureCtrlMask);
+ next_mask = smu_v13_0_0_normalize_feature_mask(od, boot,
+ next_mask);
- ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
+ if (memcmp((u8 *)od_table + offset_of_voltageoffset,
+ (u8 *)user_od_table + offset_of_voltageoffset,
+ sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset) ||
+ next_mask != table_context->user_od_feature_mask) {
+ memcpy(&upload_table, od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ smu_v13_0_0_dump_od_table(smu, &upload_table);
+
+ ret = smu_v13_0_0_upload_overdrive_table(smu, &upload_table);
if (ret) {
dev_err(adev->dev, "Failed to upload overdrive table!\n");
return ret;
}
+ table_context->user_od_feature_mask = next_mask;
od_table->OverDriveTable.FeatureCtrlMask = 0;
- memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
+ memcpy((u8 *)user_od_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (!memcmp(table_context->user_overdrive_table,
- table_context->boot_overdrive_table,
- sizeof(OverDriveTableExternal_t)))
- smu->user_dpm_profile.user_od = false;
- else
- smu->user_dpm_profile.user_od = true;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
}
break;
@@ -2191,6 +2276,7 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu)
memcpy(od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
/*
* For S3/S4/Runpm resume, we need to setup those overdrive tables again,
@@ -2200,7 +2286,9 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu)
memcpy(user_od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2237,6 +2325,14 @@ static int smu_v13_0_0_set_default_od_settings(struct smu_context *smu)
user_od_table_bak.OverDriveTable.FanZeroRpmEnable;
user_od_table->OverDriveTable.FanZeroRpmStopTemp =
user_od_table_bak.OverDriveTable.FanZeroRpmStopTemp;
+ user_od_table->OverDriveTable.FanMode =
+ user_od_table_bak.OverDriveTable.FanMode;
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
+ if (smu->adev->in_suspend) {
+ user_od_table->OverDriveTable.Ppt = 0;
+ smu->smu_table.user_od_feature_mask &=
+ ~BIT(PP_OD_FEATURE_PPT_BIT);
}
smu_v13_0_0_set_supported_od_feature_mask(smu);
@@ -2249,17 +2345,22 @@ static int smu_v13_0_0_restore_user_od_settings(struct smu_context *smu)
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table = table_context->overdrive_table;
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
- res = smu_v13_0_0_upload_overdrive_table(smu, user_od_table);
- user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (res == 0)
+ next_mask = smu_v13_0_0_filter_od_feature_mask(smu,
+ table_context->user_od_feature_mask);
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ res = smu_v13_0_0_upload_overdrive_table(smu, &upload_table);
+ if (res == 0) {
+ table_context->user_od_feature_mask = next_mask;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
memcpy(od_table, user_od_table, sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
return res;
}
@@ -3053,10 +3154,8 @@ static int smu_v13_0_0_set_ppt_limit(struct smu_context *smu,
SkuTable_t *skutable = &pptable->SkuTable;
uint32_t msg_limit = skutable->MsgLimits.Power
[PPT_THROTTLER_PPT0][POWER_SOURCE_AC];
- struct smu_table_context *table_context = &smu->smu_table;
- OverDriveTableExternal_t *od_table =
- (OverDriveTableExternal_t *)table_context->overdrive_table;
OverDriveTableExternal_t current_od_table;
+ s16 ppt;
int ret = 0;
if (limit_type != SMU_PPT_LIMIT_PPT0)
@@ -3068,10 +3167,7 @@ static int smu_v13_0_0_set_ppt_limit(struct smu_context *smu,
return ret;
if (current_od_table.OverDriveTable.Ppt) {
- od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
+ ret = smu_v13_0_0_upload_ppt_overdrive_table(smu, 0);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
@@ -3083,10 +3179,8 @@ static int smu_v13_0_0_set_ppt_limit(struct smu_context *smu,
if (ret)
return ret;
- od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v13_0_0_upload_overdrive_table(smu, od_table);
+ ppt = (limit * 100) / msg_limit - 100;
+ ret = smu_v13_0_0_upload_ppt_overdrive_table(smu, ppt);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
index 5fe409a23772..8285d4a91589 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
@@ -63,6 +63,9 @@ static void smu_v13_0_7_get_od_setting_limits(struct smu_context *smu,
int od_feature_bit,
int32_t *min, int32_t *max);
static int smu_v13_0_7_init_ppt_limits(struct smu_context *smu);
+static int
+smu_v13_0_7_upload_overdrive_table(struct smu_context *smu,
+ OverDriveTableExternal_t *od_table);
static const struct smu_feature_bits smu_v13_0_7_dpm_features = {
.bits = {
@@ -94,6 +97,7 @@ static const struct smu_feature_bits smu_v13_0_7_dpm_features = {
#define PP_OD_FEATURE_GFXCLK_FMAX 1
#define PP_OD_FEATURE_UCLK_FMIN 2
#define PP_OD_FEATURE_UCLK_FMAX 3
+
#define PP_OD_FEATURE_GFX_VF_CURVE 4
#define PP_OD_FEATURE_FAN_CURVE_TEMP 5
#define PP_OD_FEATURE_FAN_CURVE_PWM 6
@@ -1089,6 +1093,93 @@ static bool smu_v13_0_7_is_od_feature_supported(struct smu_context *smu,
return feature_enabled;
}
+static u32
+smu_v13_0_7_filter_od_feature_mask(struct smu_context *smu, u32 feature_mask)
+{
+ PPTable_t *pptable = smu->smu_table.driver_pptable;
+ u32 supported_mask =
+ pptable->SkuTable.OverDriveLimitsBasicMax.FeatureCtrlMask;
+
+ return feature_mask & supported_mask;
+}
+
+static bool smu_v13_0_7_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return !memcmp(od->FanLinearPwmPoints, boot->FanLinearPwmPoints,
+ sizeof(od->FanLinearPwmPoints)) &&
+ !memcmp(od->FanLinearTempPoints, boot->FanLinearTempPoints,
+ sizeof(od->FanLinearTempPoints)) &&
+ od->FanMinimumPwm == boot->FanMinimumPwm &&
+ od->AcousticTargetRpmThreshold == boot->AcousticTargetRpmThreshold &&
+ od->AcousticLimitRpmThreshold == boot->AcousticLimitRpmThreshold &&
+ od->FanTargetTemperature == boot->FanTargetTemperature &&
+ od->FanMode == boot->FanMode;
+}
+
+static bool smu_v13_0_7_zero_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return od->FanZeroRpmEnable == boot->FanZeroRpmEnable &&
+ od->FanZeroRpmStopTemp == boot->FanZeroRpmStopTemp;
+}
+
+static u32 smu_v13_0_7_normalize_feature_mask(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot,
+ u32 feature_mask)
+{
+ if (od->GfxclkFmin == boot->GfxclkFmin &&
+ od->GfxclkFmax == boot->GfxclkFmax)
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFXCLK_BIT);
+ if (od->UclkFmin == boot->UclkFmin &&
+ od->UclkFmax == boot->UclkFmax)
+ feature_mask &= ~BIT(PP_OD_FEATURE_UCLK_BIT);
+ if (!memcmp(od->VoltageOffsetPerZoneBoundary,
+ boot->VoltageOffsetPerZoneBoundary,
+ sizeof(od->VoltageOffsetPerZoneBoundary)))
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT);
+ if (smu_v13_0_7_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ if (smu_v13_0_7_zero_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ if (!od->Ppt)
+ feature_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+
+ return feature_mask;
+}
+
+static int
+smu_v13_0_7_upload_ppt_overdrive_table(struct smu_context *smu, s16 ppt)
+{
+ struct smu_table_context *table_context = &smu->smu_table;
+ OverDriveTableExternal_t *od_table = table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask = table_context->user_od_feature_mask;
+ int ret;
+
+ /* Do not commit OD settings staged after the last c operation. */
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.Ppt = ppt;
+ if (ppt)
+ next_mask |= BIT(PP_OD_FEATURE_PPT_BIT);
+ else
+ next_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+ upload_table.OverDriveTable.FeatureCtrlMask =
+ smu_v13_0_7_filter_od_feature_mask(smu, next_mask);
+
+ ret = smu_v13_0_7_upload_overdrive_table(smu, &upload_table);
+ if (ret)
+ return ret;
+
+ table_context->user_od_feature_mask =
+ upload_table.OverDriveTable.FeatureCtrlMask;
+ od_table->OverDriveTable.Ppt = ppt;
+ user_od_table->OverDriveTable.Ppt = ppt;
+
+ return 0;
+}
+
static void smu_v13_0_7_get_od_setting_limits(struct smu_context *smu,
int od_feature_bit,
int32_t *min,
@@ -1523,41 +1614,30 @@ static int smu_v13_0_7_od_restore_table_single(struct smu_context *smu, long inp
boot_overdrive_table->OverDriveTable.FanLinearPwmPoints[i];
}
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_LIMIT:
od_table->OverDriveTable.AcousticLimitRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticLimitRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_TARGET:
od_table->OverDriveTable.AcousticTargetRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticTargetRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_TARGET_TEMPERATURE:
od_table->OverDriveTable.FanTargetTemperature =
boot_overdrive_table->OverDriveTable.FanTargetTemperature;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_MINIMUM_PWM:
od_table->OverDriveTable.FanMinimumPwm =
boot_overdrive_table->OverDriveTable.FanMinimumPwm;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_ENABLE:
od_table->OverDriveTable.FanZeroRpmEnable =
boot_overdrive_table->OverDriveTable.FanZeroRpmEnable;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_STOP_TEMP:
od_table->OverDriveTable.FanZeroRpmStopTemp =
boot_overdrive_table->OverDriveTable.FanZeroRpmStopTemp;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
default:
dev_info(adev->dev, "Invalid table index: %ld\n", input);
@@ -1575,10 +1655,17 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table =
(OverDriveTableExternal_t *)table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table =
+ (OverDriveTableExternal_t *)table_context->user_overdrive_table;
+ OverDriveTableExternal_t *boot_od_table =
+ (OverDriveTableExternal_t *)table_context->boot_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ OverDriveTable_t *od = &od_table->OverDriveTable;
+ OverDriveTable_t *boot = &boot_od_table->OverDriveTable;
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
+ u32 next_mask;
int i, ret = 0;
switch (type) {
@@ -1900,45 +1987,44 @@ static int smu_v13_0_7_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
- if (memcmp((u8 *)od_table + offset_of_voltageoffset,
- table_context->user_overdrive_table + offset_of_voltageoffset,
- sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset)) {
- smu_v13_0_7_dump_od_table(smu, od_table);
+ next_mask = smu_v13_0_7_filter_od_feature_mask(smu,
+ table_context->user_od_feature_mask |
+ od->FeatureCtrlMask);
+ next_mask = smu_v13_0_7_normalize_feature_mask(od, boot,
+ next_mask);
- ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
+ if (memcmp((u8 *)od_table + offset_of_voltageoffset,
+ (u8 *)user_od_table + offset_of_voltageoffset,
+ sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset) ||
+ next_mask != table_context->user_od_feature_mask) {
+ memcpy(&upload_table, od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ smu_v13_0_7_dump_od_table(smu, &upload_table);
+
+ ret = smu_v13_0_7_upload_overdrive_table(smu, &upload_table);
if (ret) {
dev_err(adev->dev, "Failed to upload overdrive table!\n");
return ret;
}
+ table_context->user_od_feature_mask = next_mask;
od_table->OverDriveTable.FeatureCtrlMask = 0;
- memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
+ memcpy((u8 *)user_od_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (!memcmp(table_context->user_overdrive_table,
- table_context->boot_overdrive_table,
- sizeof(OverDriveTableExternal_t)))
- smu->user_dpm_profile.user_od = false;
- else
- smu->user_dpm_profile.user_od = true;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
}
break;
@@ -2193,6 +2279,7 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu)
memcpy(od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
/*
* For S3/S4/Runpm resume, we need to setup those overdrive tables again,
@@ -2202,7 +2289,9 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu)
memcpy(user_od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2239,6 +2328,14 @@ static int smu_v13_0_7_set_default_od_settings(struct smu_context *smu)
user_od_table_bak.OverDriveTable.FanZeroRpmEnable;
user_od_table->OverDriveTable.FanZeroRpmStopTemp =
user_od_table_bak.OverDriveTable.FanZeroRpmStopTemp;
+ user_od_table->OverDriveTable.FanMode =
+ user_od_table_bak.OverDriveTable.FanMode;
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
+ if (smu->adev->in_suspend) {
+ user_od_table->OverDriveTable.Ppt = 0;
+ smu->smu_table.user_od_feature_mask &=
+ ~BIT(PP_OD_FEATURE_PPT_BIT);
}
smu_v13_0_7_set_supported_od_feature_mask(smu);
@@ -2251,17 +2348,22 @@ static int smu_v13_0_7_restore_user_od_settings(struct smu_context *smu)
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table = table_context->overdrive_table;
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
- res = smu_v13_0_7_upload_overdrive_table(smu, user_od_table);
- user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (res == 0)
+ next_mask = smu_v13_0_7_filter_od_feature_mask(smu,
+ table_context->user_od_feature_mask);
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ res = smu_v13_0_7_upload_overdrive_table(smu, &upload_table);
+ if (res == 0) {
+ table_context->user_od_feature_mask = next_mask;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
memcpy(od_table, user_od_table, sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
return res;
}
@@ -2673,10 +2775,8 @@ static int smu_v13_0_7_set_ppt_limit(struct smu_context *smu,
SkuTable_t *skutable = &pptable->SkuTable;
uint32_t msg_limit = skutable->MsgLimits.Power
[PPT_THROTTLER_PPT0][POWER_SOURCE_AC];
- struct smu_table_context *table_context = &smu->smu_table;
- OverDriveTableExternal_t *od_table =
- (OverDriveTableExternal_t *)table_context->overdrive_table;
OverDriveTableExternal_t current_od_table;
+ s16 ppt;
int ret = 0;
if (limit_type != SMU_PPT_LIMIT_PPT0)
@@ -2688,10 +2788,7 @@ static int smu_v13_0_7_set_ppt_limit(struct smu_context *smu,
return ret;
if (current_od_table.OverDriveTable.Ppt) {
- od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
+ ret = smu_v13_0_7_upload_ppt_overdrive_table(smu, 0);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
@@ -2703,10 +2800,8 @@ static int smu_v13_0_7_set_ppt_limit(struct smu_context *smu,
if (ret)
return ret;
- od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v13_0_7_upload_overdrive_table(smu, od_table);
+ ppt = (limit * 100) / msg_limit - 100;
+ ret = smu_v13_0_7_upload_ppt_overdrive_table(smu, ppt);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
diff --git a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
index 6621e50de965..6a12d6f8c9dd 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/smu14/smu_v14_0_2_ppt.c
@@ -91,6 +91,10 @@ static const struct smu_feature_bits smu_v14_0_2_dpm_features = {
#define PP_OD_FEATURE_FAN_ZERO_RPM_ENABLE 11
#define PP_OD_FEATURE_FAN_ZERO_RPM_STOP_TEMP 12
+#define SMU_14_0_2_OD_FAN_FEATURE_MASK \
+ (BIT(PP_OD_FEATURE_FAN_CURVE_BIT) | \
+ BIT(PP_OD_FEATURE_FAN_LEGACY_BIT))
+
static struct cmn2asic_msg_mapping smu_v14_0_2_message_map[SMU_MSG_MAX_COUNT] = {
MSG_MAP(TestMessage, PPSMC_MSG_TestMessage, 1),
MSG_MAP(GetSmuVersion, PPSMC_MSG_GetSmuVersion, 1),
@@ -963,6 +967,84 @@ static bool smu_v14_0_2_is_od_feature_supported(struct smu_context *smu,
return feature_enabled;
}
+static u32
+smu_v14_0_2_filter_od_feature_mask(struct smu_context *smu, u32 feature_mask)
+{
+ PPTable_t *pptable = smu->smu_table.driver_pptable;
+ u32 supported_mask =
+ pptable->SkuTable.OverDriveLimitsBasicMax.FeatureCtrlMask;
+
+ return feature_mask & supported_mask;
+}
+
+static u32
+smu_v14_0_2_get_od_feature_mask(struct smu_context *smu, u32 pending_mask)
+{
+ u32 supported_mask = smu_v14_0_2_filter_od_feature_mask(smu, ~0U);
+ u32 fan_mask = supported_mask & SMU_14_0_2_OD_FAN_FEATURE_MASK;
+ u32 mask = smu->smu_table.user_od_feature_mask;
+
+ if (pending_mask & fan_mask)
+ mask &= ~fan_mask;
+ mask |= pending_mask;
+
+ return smu_v14_0_2_filter_od_feature_mask(smu, mask);
+}
+
+static bool smu_v14_0_2_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return !memcmp(od->FanLinearPwmPoints, boot->FanLinearPwmPoints,
+ sizeof(od->FanLinearPwmPoints)) &&
+ !memcmp(od->FanLinearTempPoints, boot->FanLinearTempPoints,
+ sizeof(od->FanLinearTempPoints)) &&
+ od->AcousticTargetRpmThreshold == boot->AcousticTargetRpmThreshold &&
+ od->AcousticLimitRpmThreshold == boot->AcousticLimitRpmThreshold &&
+ od->FanTargetTemperature == boot->FanTargetTemperature &&
+ od->FanMode == boot->FanMode;
+}
+
+static bool smu_v14_0_2_legacy_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return od->FanMinimumPwm == boot->FanMinimumPwm &&
+ od->FanMode == boot->FanMode;
+}
+
+static bool smu_v14_0_2_zero_fan_is_default(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot)
+{
+ return od->FanZeroRpmEnable == boot->FanZeroRpmEnable &&
+ od->FanZeroRpmStopTemp == boot->FanZeroRpmStopTemp;
+}
+
+static u32 smu_v14_0_2_normalize_feature_mask(const OverDriveTable_t *od,
+ const OverDriveTable_t *boot,
+ u32 feature_mask)
+{
+ if (od->GfxclkFoffset == boot->GfxclkFoffset)
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFXCLK_BIT);
+ if (od->UclkFmin == boot->UclkFmin &&
+ od->UclkFmax == boot->UclkFmax)
+ feature_mask &= ~BIT(PP_OD_FEATURE_UCLK_BIT);
+ if (!memcmp(od->VoltageOffsetPerZoneBoundary,
+ boot->VoltageOffsetPerZoneBoundary,
+ sizeof(od->VoltageOffsetPerZoneBoundary)))
+ feature_mask &= ~BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT);
+ if ((feature_mask & BIT(PP_OD_FEATURE_FAN_CURVE_BIT)) &&
+ smu_v14_0_2_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ if ((feature_mask & BIT(PP_OD_FEATURE_FAN_LEGACY_BIT)) &&
+ smu_v14_0_2_legacy_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
+ if (smu_v14_0_2_zero_fan_is_default(od, boot))
+ feature_mask &= ~BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
+ if (!od->Ppt)
+ feature_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+
+ return feature_mask;
+}
+
static void smu_v14_0_2_get_od_setting_limits(struct smu_context *smu,
int od_feature_bit,
int32_t *min,
@@ -2317,6 +2399,38 @@ static int smu_v14_0_2_upload_overdrive_table(struct smu_context *smu,
return ret;
}
+static int
+smu_v14_0_2_upload_ppt_overdrive_table(struct smu_context *smu, s16 ppt)
+{
+ struct smu_table_context *table_context = &smu->smu_table;
+ OverDriveTableExternal_t *od_table = table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask = table_context->user_od_feature_mask;
+ int ret;
+
+ /* Do not commit OD settings staged after the last c operation. */
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.Ppt = ppt;
+ if (ppt)
+ next_mask |= BIT(PP_OD_FEATURE_PPT_BIT);
+ else
+ next_mask &= ~BIT(PP_OD_FEATURE_PPT_BIT);
+ upload_table.OverDriveTable.FeatureCtrlMask =
+ smu_v14_0_2_filter_od_feature_mask(smu, next_mask);
+
+ ret = smu_v14_0_2_upload_overdrive_table(smu, &upload_table);
+ if (ret)
+ return ret;
+
+ table_context->user_od_feature_mask =
+ upload_table.OverDriveTable.FeatureCtrlMask;
+ od_table->OverDriveTable.Ppt = ppt;
+ user_od_table->OverDriveTable.Ppt = ppt;
+
+ return 0;
+}
+
static void smu_v14_0_2_set_supported_od_feature_mask(struct smu_context *smu)
{
struct amdgpu_device *adev = smu->adev;
@@ -2375,6 +2489,7 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu)
memcpy(od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
/*
* For S3/S4/Runpm resume, we need to setup those overdrive tables again,
@@ -2384,7 +2499,9 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu)
memcpy(user_od_table,
boot_od_table,
sizeof(OverDriveTableExternal_t));
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
smu->user_dpm_profile.user_od = false;
+ smu->smu_table.user_od_feature_mask = 0;
} else if (smu->user_dpm_profile.user_od) {
memcpy(&user_od_table_bak,
user_od_table,
@@ -2419,6 +2536,14 @@ static int smu_v14_0_2_set_default_od_settings(struct smu_context *smu)
user_od_table_bak.OverDriveTable.FanZeroRpmEnable;
user_od_table->OverDriveTable.FanZeroRpmStopTemp =
user_od_table_bak.OverDriveTable.FanZeroRpmStopTemp;
+ user_od_table->OverDriveTable.FanMode =
+ user_od_table_bak.OverDriveTable.FanMode;
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
+ if (smu->adev->in_suspend) {
+ user_od_table->OverDriveTable.Ppt = 0;
+ smu->smu_table.user_od_feature_mask &=
+ ~BIT(PP_OD_FEATURE_PPT_BIT);
}
smu_v14_0_2_set_supported_od_feature_mask(smu);
@@ -2431,17 +2556,22 @@ static int smu_v14_0_2_restore_user_od_settings(struct smu_context *smu)
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table = table_context->overdrive_table;
OverDriveTableExternal_t *user_od_table = table_context->user_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ u32 next_mask;
int res;
- user_od_table->OverDriveTable.FeatureCtrlMask = BIT(PP_OD_FEATURE_GFXCLK_BIT) |
- BIT(PP_OD_FEATURE_UCLK_BIT) |
- BIT(PP_OD_FEATURE_GFX_VF_CURVE_BIT) |
- BIT(PP_OD_FEATURE_FAN_CURVE_BIT) |
- BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
- res = smu_v14_0_2_upload_overdrive_table(smu, user_od_table);
- user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (res == 0)
+ next_mask = smu_v14_0_2_filter_od_feature_mask(smu,
+ table_context->user_od_feature_mask);
+ memcpy(&upload_table, user_od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ res = smu_v14_0_2_upload_overdrive_table(smu, &upload_table);
+ if (res == 0) {
+ table_context->user_od_feature_mask = next_mask;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
memcpy(od_table, user_od_table, sizeof(OverDriveTableExternal_t));
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
+ }
return res;
}
@@ -2465,43 +2595,30 @@ static int smu_v14_0_2_od_restore_table_single(struct smu_context *smu, long inp
boot_overdrive_table->OverDriveTable.FanLinearPwmPoints[i];
}
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
- od_table->OverDriveTable.FeatureCtrlMask &= ~BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_ENABLE:
od_table->OverDriveTable.FanZeroRpmEnable =
boot_overdrive_table->OverDriveTable.FanZeroRpmEnable;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
case PP_OD_EDIT_FAN_ZERO_RPM_STOP_TEMP:
od_table->OverDriveTable.FanZeroRpmStopTemp =
boot_overdrive_table->OverDriveTable.FanZeroRpmStopTemp;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_ZERO_FAN_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_LIMIT:
od_table->OverDriveTable.AcousticLimitRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticLimitRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_TARGET:
od_table->OverDriveTable.AcousticTargetRpmThreshold =
boot_overdrive_table->OverDriveTable.AcousticTargetRpmThreshold;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_TARGET_TEMPERATURE:
od_table->OverDriveTable.FanTargetTemperature =
boot_overdrive_table->OverDriveTable.FanTargetTemperature;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
case PP_OD_EDIT_FAN_MINIMUM_PWM:
od_table->OverDriveTable.FanMinimumPwm =
boot_overdrive_table->OverDriveTable.FanMinimumPwm;
- od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
- od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
- od_table->OverDriveTable.FeatureCtrlMask &= ~BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
break;
default:
dev_info(adev->dev, "Invalid table index: %ld\n", input);
@@ -2519,10 +2636,17 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
struct smu_table_context *table_context = &smu->smu_table;
OverDriveTableExternal_t *od_table =
(OverDriveTableExternal_t *)table_context->overdrive_table;
+ OverDriveTableExternal_t *user_od_table =
+ (OverDriveTableExternal_t *)table_context->user_overdrive_table;
+ OverDriveTableExternal_t *boot_od_table =
+ (OverDriveTableExternal_t *)table_context->boot_overdrive_table;
+ OverDriveTableExternal_t upload_table;
+ OverDriveTable_t *od = &od_table->OverDriveTable;
+ OverDriveTable_t *boot = &boot_od_table->OverDriveTable;
struct amdgpu_device *adev = smu->adev;
uint32_t offset_of_voltageoffset;
int32_t minimum, maximum;
- uint32_t feature_ctrlmask;
+ u32 next_mask;
int i, ret = 0;
switch (type) {
@@ -2694,6 +2818,7 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
od_table->OverDriveTable.AcousticLimitRpmThreshold = input[0];
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask &= ~BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
break;
case PP_OD_EDIT_ACOUSTIC_TARGET:
@@ -2716,6 +2841,7 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
od_table->OverDriveTable.AcousticTargetRpmThreshold = input[0];
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask &= ~BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
break;
case PP_OD_EDIT_FAN_TARGET_TEMPERATURE:
@@ -2738,6 +2864,7 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
od_table->OverDriveTable.FanTargetTemperature = input[0];
od_table->OverDriveTable.FanMode = FAN_MODE_AUTO;
od_table->OverDriveTable.FeatureCtrlMask |= BIT(PP_OD_FEATURE_FAN_CURVE_BIT);
+ od_table->OverDriveTable.FeatureCtrlMask &= ~BIT(PP_OD_FEATURE_FAN_LEGACY_BIT);
break;
case PP_OD_EDIT_FAN_MINIMUM_PWM:
@@ -2811,44 +2938,42 @@ static int smu_v14_0_2_od_edit_dpm_table(struct smu_context *smu,
if (ret)
return ret;
} else {
- feature_ctrlmask = od_table->OverDriveTable.FeatureCtrlMask;
memcpy(od_table,
table_context->boot_overdrive_table,
sizeof(OverDriveTableExternal_t));
- od_table->OverDriveTable.FeatureCtrlMask = feature_ctrlmask;
+ od_table->OverDriveTable.FeatureCtrlMask = 0;
}
fallthrough;
case PP_OD_COMMIT_DPM_TABLE:
- /*
- * The member below instructs PMFW the settings focused in
- * this single operation.
- * `uint32_t FeatureCtrlMask;`
- * It does not contain actual informations about user's custom
- * settings. Thus we do not cache it.
- */
offset_of_voltageoffset = offsetof(OverDriveTable_t, VoltageOffsetPerZoneBoundary);
- if (memcmp((u8 *)od_table + offset_of_voltageoffset,
- table_context->user_overdrive_table + offset_of_voltageoffset,
- sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset)) {
- smu_v14_0_2_dump_od_table(smu, od_table);
+ next_mask = smu_v14_0_2_get_od_feature_mask(smu,
+ od->FeatureCtrlMask);
+ next_mask = smu_v14_0_2_normalize_feature_mask(od, boot,
+ next_mask);
- ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
+ if (memcmp((u8 *)od_table + offset_of_voltageoffset,
+ (u8 *)user_od_table + offset_of_voltageoffset,
+ sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset) ||
+ next_mask != table_context->user_od_feature_mask) {
+ memcpy(&upload_table, od_table, sizeof(upload_table));
+ upload_table.OverDriveTable.FeatureCtrlMask = next_mask;
+ smu_v14_0_2_dump_od_table(smu, &upload_table);
+
+ ret = smu_v14_0_2_upload_overdrive_table(smu, &upload_table);
if (ret) {
dev_err(adev->dev, "Failed to upload overdrive table!\n");
return ret;
}
+ table_context->user_od_feature_mask = next_mask;
od_table->OverDriveTable.FeatureCtrlMask = 0;
- memcpy(table_context->user_overdrive_table + offset_of_voltageoffset,
+ memcpy((u8 *)user_od_table + offset_of_voltageoffset,
(u8 *)od_table + offset_of_voltageoffset,
sizeof(OverDriveTableExternal_t) - offset_of_voltageoffset);
+ user_od_table->OverDriveTable.FeatureCtrlMask = 0;
- if (!memcmp(table_context->user_overdrive_table,
- table_context->boot_overdrive_table,
- sizeof(OverDriveTableExternal_t)))
- smu->user_dpm_profile.user_od = false;
- else
- smu->user_dpm_profile.user_od = true;
+ smu->user_dpm_profile.user_od =
+ !!(next_mask & ~BIT(PP_OD_FEATURE_PPT_BIT));
}
break;
@@ -2866,10 +2991,8 @@ static int smu_v14_0_2_set_ppt_limit(struct smu_context *smu,
PPTable_t *pptable = smu->smu_table.driver_pptable;
uint32_t msg_limit = pptable->SkuTable.MsgLimits.Power
[PPT_THROTTLER_PPT0][POWER_SOURCE_AC];
- struct smu_table_context *table_context = &smu->smu_table;
- OverDriveTableExternal_t *od_table =
- (OverDriveTableExternal_t *)table_context->overdrive_table;
OverDriveTableExternal_t current_od_table;
+ s16 ppt;
int ret = 0;
if (limit_type != SMU_PPT_LIMIT_PPT0)
@@ -2881,10 +3004,7 @@ static int smu_v14_0_2_set_ppt_limit(struct smu_context *smu,
return ret;
if (current_od_table.OverDriveTable.Ppt) {
- od_table->OverDriveTable.Ppt = 0;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
+ ret = smu_v14_0_2_upload_ppt_overdrive_table(smu, 0);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
@@ -2896,10 +3016,8 @@ static int smu_v14_0_2_set_ppt_limit(struct smu_context *smu,
if (ret)
return ret;
- od_table->OverDriveTable.Ppt = (limit * 100) / msg_limit - 100;
- od_table->OverDriveTable.FeatureCtrlMask |= 1U << PP_OD_FEATURE_PPT_BIT;
-
- ret = smu_v14_0_2_upload_overdrive_table(smu, od_table);
+ ppt = (limit * 100) / msg_limit - 100;
+ ret = smu_v14_0_2_upload_ppt_overdrive_table(smu, ppt);
if (ret) {
dev_err(smu->adev->dev, "Failed to upload overdrive table!\n");
return ret;
diff --git a/drivers/gpu/drm/arm/malidp_crtc.c b/drivers/gpu/drm/arm/malidp_crtc.c
index ebe8e1078777..af62bc102764 100644
--- a/drivers/gpu/drm/arm/malidp_crtc.c
+++ b/drivers/gpu/drm/arm/malidp_crtc.c
@@ -56,6 +56,7 @@ static void malidp_crtc_atomic_enable(struct drm_crtc *crtc,
if (err < 0) {
DRM_DEBUG_DRIVER("Failed to enable runtime power management: %d\n", err);
+ pm_runtime_put_noidle(crtc->dev->dev);
return;
}
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c
index 30c0c0b41d5a..4213cd68d5a0 100644
--- a/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c
@@ -1248,6 +1248,7 @@ static int analogix_dp_dt_parse_pdata(struct analogix_dp_device *dp)
{
struct device_node *dp_node = dp->dev->of_node;
struct video_info *video_info = &dp->video_info;
+ u32 val;
switch (dp->plat_data->dev_type) {
case RK3288_DP:
@@ -1269,10 +1270,14 @@ static int analogix_dp_dt_parse_pdata(struct analogix_dp_device *dp)
* NOTE: those property parseing code is used for
* providing backward compatibility for samsung platform.
*/
- of_property_read_u32(dp_node, "samsung,link-rate",
- &video_info->max_link_rate);
- of_property_read_u32(dp_node, "samsung,lane-count",
- &video_info->max_lane_count);
+ if (of_property_read_u32(dp_node, "samsung,link-rate", &val))
+ return dev_err_probe(dp->dev, -EINVAL,
+ "Failed to get samsung,link-rate\n");
+ video_info->max_link_rate = val;
+ if (of_property_read_u32(dp_node, "samsung,lane-count", &val))
+ return dev_err_probe(dp->dev, -EINVAL,
+ "Failed to get samsung,lane-count\n");
+ video_info->max_lane_count = val;
break;
}
diff --git a/drivers/gpu/drm/drm_panic.c b/drivers/gpu/drm/drm_panic.c
index e576c4791861..d28ce8878e05 100644
--- a/drivers/gpu/drm/drm_panic.c
+++ b/drivers/gpu/drm/drm_panic.c
@@ -178,6 +178,8 @@ static void drm_panic_write_pixel24_xpage(void *vaddr, struct page *next_page,
u8 *p = vaddr + offset;
vaddr2 = kmap_local_page_try_from_panic(next_page);
+ if (!vaddr2)
+ return;
*p++ = color & 0xff;
color >>= 8;
@@ -658,7 +660,8 @@ static int drm_panic_get_qr_code_url(u8 **qr_image)
char *kmsg;
int max_qr_data_size, url_len;
- url_len = snprintf(url, sizeof(url), CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL "?a=%s&v=%s&z=",
+ url_len = snprintf(url, sizeof(url), "%s?a=%s&v=%s&z=",
+ CONFIG_DRM_PANIC_SCREEN_QR_CODE_URL,
utsname()->machine, utsname()->release);
max_qr_data_size = drm_panic_qr_max_data_size(panic_qr_version, url_len);
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index b292dfd266d1..d92659d80f17 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -1254,6 +1254,8 @@ int dsi_tx_buf_alloc_6g(struct msm_dsi_host *msm_host, int size)
if (IS_ERR(data)) {
msm_host->tx_gem_obj = NULL;
+ drm_gpuvm_put(msm_host->vm);
+ msm_host->vm = NULL;
return PTR_ERR(data);
}
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c b/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
index 157f19bd90b4..1860605850c9 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi_bridge.c
@@ -13,14 +13,16 @@
#include "msm_kms.h"
#include "hdmi.h"
-static void msm_hdmi_power_on(struct drm_bridge *bridge)
+static int msm_hdmi_power_on(struct drm_bridge *bridge)
{
struct drm_device *dev = bridge->dev;
struct hdmi_bridge *hdmi_bridge = to_hdmi_bridge(bridge);
struct hdmi *hdmi = hdmi_bridge->hdmi;
int ret;
- pm_runtime_resume_and_get(&hdmi->pdev->dev);
+ ret = pm_runtime_resume_and_get(&hdmi->pdev->dev);
+ if (ret)
+ return ret;
if (hdmi->extp_clk) {
DBG("pixclock: %lu", hdmi->pixclock);
@@ -29,9 +31,14 @@ static void msm_hdmi_power_on(struct drm_bridge *bridge)
DRM_DEV_ERROR(dev->dev, "failed to set extp clk rate: %d\n", ret);
ret = clk_prepare_enable(hdmi->extp_clk);
- if (ret)
+ if (ret) {
DRM_DEV_ERROR(dev->dev, "failed to enable extp clk: %d\n", ret);
+ pm_runtime_put(&hdmi->pdev->dev);
+ return ret;
+ }
}
+
+ return 0;
}
static void power_off(struct drm_bridge *bridge)
@@ -287,13 +294,18 @@ static void msm_hdmi_bridge_atomic_pre_enable(struct drm_bridge *bridge,
msm_hdmi_set_timings(hdmi, &crtc_state->adjusted_mode);
- mutex_lock(&hdmi->state_mutex);
- if (!hdmi->power_on) {
- msm_hdmi_phy_resource_enable(phy);
- msm_hdmi_power_on(bridge);
- hdmi->power_on = true;
+ scoped_guard(mutex, &hdmi->state_mutex) {
+ if (!hdmi->power_on) {
+ if (msm_hdmi_phy_resource_enable(phy))
+ return;
+
+ if (msm_hdmi_power_on(bridge)) {
+ msm_hdmi_phy_resource_disable(phy);
+ return;
+ }
+ hdmi->power_on = true;
+ }
}
- mutex_unlock(&hdmi->state_mutex);
if (connector->display_info.is_hdmi)
msm_hdmi_audio_update(hdmi);
@@ -327,10 +339,10 @@ static void msm_hdmi_bridge_atomic_post_disable(struct drm_bridge *bridge,
msm_hdmi_phy_powerdown(phy);
if (hdmi->power_on) {
- power_off(bridge);
hdmi->power_on = false;
if (hdmi->connector->display_info.is_hdmi)
msm_hdmi_audio_update(hdmi);
+ power_off(bridge);
msm_hdmi_phy_resource_disable(phy);
}
mutex_unlock(&hdmi->state_mutex);
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_phy.c b/drivers/gpu/drm/msm/hdmi/hdmi_phy.c
index 77dce35cd45e..ebb600d77cd8 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi_phy.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi_phy.c
@@ -67,16 +67,28 @@ int msm_hdmi_phy_resource_enable(struct hdmi_phy *phy)
ret = regulator_bulk_enable(cfg->num_regs, phy->regs);
if (ret) {
DRM_DEV_ERROR(dev, "failed to enable regulators: (%d)\n", ret);
- return ret;
+ goto err_pm_put;
}
for (i = 0; i < cfg->num_clks; i++) {
ret = clk_prepare_enable(phy->clks[i]);
- if (ret)
+ if (ret) {
DRM_DEV_ERROR(dev, "failed to enable clock: %s (%d)\n",
cfg->clk_names[i], ret);
+ goto err_clk_unprepare;
+ }
}
+ return 0;
+
+err_clk_unprepare:
+ for (; i > 0; i--)
+ clk_disable_unprepare(phy->clks[i - 1]);
+ regulator_bulk_disable(cfg->num_regs, phy->regs);
+
+err_pm_put:
+ pm_runtime_put_sync(dev);
+
return ret;
}
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
index 112b674ed9c8..0172e0d200f8 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/pci.h
@@ -46,6 +46,7 @@ int nv4c_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct n
int g84_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int g92_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int g94_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
+int mcp79_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gf100_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gf106_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
int gk104_pci_new(struct nvkm_device *, enum nvkm_subdev_type, int inst, struct nvkm_pci **);
diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c
index b776030460f7..e876f154963d 100644
--- a/drivers/gpu/drm/nouveau/nouveau_abi16.c
+++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c
@@ -666,6 +666,10 @@ nouveau_abi16_ioctl_notifierobj_alloc(ABI16_IOCTL_ARGS)
if (unlikely(device->info.family >= NV_DEVICE_INFO_V0_FERMI))
return nouveau_abi16_put(abi16, -EINVAL);
+ /* zero size yields a zero-length node, underflowing args.limit */
+ if (unlikely(!info->size))
+ return nouveau_abi16_put(abi16, -EINVAL);
+
chan = nouveau_abi16_chan(abi16, info->channel);
if (!chan)
return nouveau_abi16_put(abi16, -ENOENT);
diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.c b/drivers/gpu/drm/nouveau/nouveau_connector.c
index 4cfc9c7c2ae0..7c4f0a29b05b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_connector.c
+++ b/drivers/gpu/drm/nouveau/nouveau_connector.c
@@ -397,6 +397,7 @@ nouveau_connector_destroy(struct drm_connector *connector)
struct nouveau_connector *nv_connector = nouveau_connector(connector);
nvif_event_dtor(&nv_connector->irq);
nvif_event_dtor(&nv_connector->hpd);
+ cancel_work_sync(&nv_connector->irq_work);
kfree(nv_connector->edid);
drm_connector_unregister(connector);
drm_connector_cleanup(connector);
diff --git a/drivers/gpu/drm/nouveau/nouveau_dp.c b/drivers/gpu/drm/nouveau/nouveau_dp.c
index 55691ec44aba..738802358d85 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dp.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dp.c
@@ -486,7 +486,7 @@ nouveau_dp_irq(struct work_struct *work)
container_of(work, typeof(*nv_connector), irq_work);
struct drm_connector *connector = &nv_connector->base;
struct nouveau_encoder *outp = find_encoder(connector, DCB_OUTPUT_DP);
- struct nouveau_drm *drm = nouveau_drm(outp->base.base.dev);
+ struct nouveau_drm *drm;
struct nv50_mstm *mstm;
u64 hpd = 0;
int ret;
@@ -494,6 +494,8 @@ nouveau_dp_irq(struct work_struct *work)
if (!outp)
return;
+ drm = nouveau_drm(outp->base.base.dev);
+
mstm = outp->dp.mstm;
NV_DEBUG(drm, "service %s\n", connector->name);
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
index edbe9e08ba0f..11e95c37ce50 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
@@ -96,7 +96,7 @@ nouveau_fence_context_kill(struct nouveau_fence_chan *fctx, int error)
void
nouveau_fence_context_del(struct nouveau_fence_chan *fctx)
{
- cancel_work_sync(&fctx->uevent_work);
+ disable_work_sync(&fctx->uevent_work);
nouveau_fence_context_kill(fctx, 0);
nvif_event_dtor(&fctx->event);
fctx->dead = 1;
diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c
index 58735446d783..c519c28164fa 100644
--- a/drivers/gpu/drm/nouveau/nouveau_svm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_svm.c
@@ -304,6 +304,19 @@ nouveau_svmm_fini(struct nouveau_svmm **psvmm)
{
struct nouveau_svmm *svmm = *psvmm;
if (svmm) {
+ struct nouveau_svm *svm = svmm->vmm->cli->drm->svm;
+
+ /* The fault handler caches svmm pointers under svm->mutex and
+ * dereferences them after dropping it, across blocking faults.
+ * The instance is already unlinked (nouveau_svmm_part), so drain
+ * the handler before the free to release any in-flight reference.
+ *
+ * On device teardown nouveau_svm_fini() ran first and freed
+ * drm->svm (NULL) after draining the buffer; nothing to flush.
+ */
+ if (svm)
+ flush_work(&svm->buffer[0].work);
+
mutex_lock(&svmm->mutex);
svmm->vmm = NULL;
mutex_unlock(&svmm->mutex);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
index 96c8a5b29999..a2ed8c4fe52a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
@@ -1237,7 +1237,7 @@ nvac_chipset = {
.mc = { 0x00000001, g98_mc_new },
.mmu = { 0x00000001, mcp77_mmu_new },
.mxm = { 0x00000001, nv50_mxm_new },
- .pci = { 0x00000001, g94_pci_new },
+ .pci = { 0x00000001, mcp79_pci_new },
.therm = { 0x00000001, g84_therm_new },
.timer = { 0x00000001, nv41_timer_new },
.volt = { 0x00000001, nv40_volt_new },
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
index a14ea0f7b1c8..90f03baeb22a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/Kbuild
@@ -9,6 +9,7 @@ nvkm-y += nvkm/subdev/pci/nv4c.o
nvkm-y += nvkm/subdev/pci/g84.o
nvkm-y += nvkm/subdev/pci/g92.o
nvkm-y += nvkm/subdev/pci/g94.o
+nvkm-y += nvkm/subdev/pci/mcp79.o
nvkm-y += nvkm/subdev/pci/gf100.o
nvkm-y += nvkm/subdev/pci/gf106.o
nvkm-y += nvkm/subdev/pci/gk104.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c
new file mode 100644
index 000000000000..e2ae242f1aa7
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/mcp79.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2026 Marek Czernohous
+ *
+ * MCP79/MCP7A (NVAC): like g94, but MSI re-arm goes through real PCI
+ * config space. The MMIO-mirror re-arm is unreliable on this IGP and a
+ * missed re-arm kills the interrupt line (see the nv46 comment; g92
+ * already re-arms through config space for the same reason).
+ */
+#include "priv.h"
+
+static const struct nvkm_pci_func
+mcp79_pci_func = {
+ .cfg = { .addr = 0x088000, .size = 0x1000 },
+
+ .init = g84_pci_init,
+ .msi_rearm = nv46_pci_msi_rearm,
+
+ .pcie.init = g84_pcie_init,
+ .pcie.set_link = g84_pcie_set_link,
+
+ .pcie.max_speed = g84_pcie_max_speed,
+ .pcie.cur_speed = g84_pcie_cur_speed,
+
+ .pcie.set_version = g84_pcie_set_version,
+ .pcie.version = g84_pcie_version,
+ .pcie.version_supported = g92_pcie_version_supported,
+};
+
+int
+mcp79_pci_new(struct nvkm_device *device, enum nvkm_subdev_type type, int inst,
+ struct nvkm_pci **ppci)
+{
+ return nvkm_pci_new_(&mcp79_pci_func, device, type, inst, ppci);
+}
diff --git a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
index fb9f9f42be4f..1ab09c50a630 100644
--- a/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
+++ b/drivers/gpu/drm/panel/panel-ebbg-ft8719.c
@@ -198,6 +198,8 @@ static int ebbg_ft8719_probe(struct mipi_dsi_device *dsi)
dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
MIPI_DSI_CLOCK_NON_CONTINUOUS;
+ ctx->panel.prepare_prev_first = true;
+
ret = drm_panel_of_backlight(&ctx->panel);
if (ret)
return dev_err_probe(dev, ret, "Failed to get backlight\n");
diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c
index c95964ba25a9..70e37ff12c9e 100644
--- a/drivers/gpu/drm/panel/panel-simple.c
+++ b/drivers/gpu/drm/panel/panel-simple.c
@@ -4354,6 +4354,9 @@ static const struct panel_desc raystar_rff500f_awh_dnn = {
.width = 108,
.height = 65,
},
+ .delay = {
+ .enable = 250,
+ },
.bus_format = MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
.connector_type = DRM_MODE_CONNECTOR_LVDS,
};
diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c
index bd0b23ac2cc4..f9458a4d5a09 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_util.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_util.c
@@ -356,6 +356,10 @@ static int ttm_bo_kmap_ttm(struct ttm_buffer_object *bo,
if (ret)
return ret;
+ /* Imported sg tables only come with DMA addresses, no pages to map. */
+ if (!ttm->pages)
+ return -EINVAL;
+
if (num_pages == 1 && ttm->caching == ttm_cached &&
!(man->use_tt && (ttm->page_flags & TTM_TT_FLAG_DECRYPTED))) {
/*
@@ -419,7 +423,7 @@ EXPORT_SYMBOL(ttm_bo_kmap_try_from_panic);
*
* Returns
* -ENOMEM: Out of memory.
- * -EINVAL: Invalid range.
+ * -EINVAL: Invalid range or the BO has no pages to map.
*/
int ttm_bo_kmap(struct ttm_buffer_object *bo,
unsigned long start_page, unsigned long num_pages,
@@ -493,7 +497,7 @@ EXPORT_SYMBOL(ttm_bo_kunmap);
*
* Returns
* -ENOMEM: Out of memory.
- * -EINVAL: Invalid range.
+ * -EINVAL: Invalid range or the BO has no pages to map.
*/
int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map)
{
@@ -537,6 +541,9 @@ int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map)
if (ret)
return ret;
+ if (!ttm->pages)
+ return -EINVAL;
+
/*
* We need to use vmap to get the desired page protection
* or to make the buffer object look contiguous.
diff --git a/drivers/gpu/drm/xe/xe_guc.c b/drivers/gpu/drm/xe/xe_guc.c
index 4286bd05c686..c988a2a3b7bd 100644
--- a/drivers/gpu/drm/xe/xe_guc.c
+++ b/drivers/gpu/drm/xe/xe_guc.c
@@ -1468,6 +1468,22 @@ int xe_guc_suspend(struct xe_guc *guc)
return 0;
}
+/**
+ * xe_guc_bo_wa_flags - Extra BO flags for memory shared with the GuC
+ * @gt: the &xe_gt whose GuC the buffer will be shared with
+ *
+ * Wa_22016122933: on the standalone media GT, memory shared between the
+ * CPU and the GuC must not be mapped cached on the CPU side, otherwise
+ * the CPU can read stale data written by the GuC (e.g. G2H CTB writes)
+ * for multiple seconds.
+ *
+ * Return: additional XE_BO_FLAG_* to use when allocating GuC-shared memory
+ */
+u32 xe_guc_bo_wa_flags(struct xe_gt *gt)
+{
+ return XE_GT_WA(gt, 22016122933) ? XE_BO_FLAG_NEEDS_UC : 0;
+}
+
void xe_guc_notify(struct xe_guc *guc)
{
struct xe_gt *gt = guc_to_gt(guc);
diff --git a/drivers/gpu/drm/xe/xe_guc.h b/drivers/gpu/drm/xe/xe_guc.h
index 61e3ee19a59b..c4eca40d69cf 100644
--- a/drivers/gpu/drm/xe/xe_guc.h
+++ b/drivers/gpu/drm/xe/xe_guc.h
@@ -30,6 +30,7 @@
xe_guc_fw_version_at_least((guc), MAKE_GUC_VER_ARGS(ver))
struct drm_printer;
+struct xe_gt;
void xe_guc_comm_init_early(struct xe_guc *guc);
int xe_guc_init_noalloc(struct xe_guc *guc);
@@ -45,6 +46,7 @@ void xe_guc_runtime_suspend(struct xe_guc *guc);
void xe_guc_runtime_resume(struct xe_guc *guc);
int xe_guc_suspend(struct xe_guc *guc);
int xe_guc_softreset(struct xe_guc *guc);
+u32 xe_guc_bo_wa_flags(struct xe_gt *gt);
void xe_guc_notify(struct xe_guc *guc);
int xe_guc_auth_huc(struct xe_guc *guc, u32 rsa_addr);
int xe_guc_mmio_send(struct xe_guc *guc, const u32 *request, u32 len);
diff --git a/drivers/gpu/drm/xe/xe_guc_ads.c b/drivers/gpu/drm/xe/xe_guc_ads.c
index 58a3f6293ce4..9ceb662ca40e 100644
--- a/drivers/gpu/drm/xe/xe_guc_ads.c
+++ b/drivers/gpu/drm/xe/xe_guc_ads.c
@@ -435,7 +435,8 @@ int xe_guc_ads_init(struct xe_guc_ads *ads)
XE_BO_FLAG_SYSTEM |
XE_BO_FLAG_GGTT |
XE_BO_FLAG_GGTT_INVALIDATE |
- XE_BO_FLAG_PINNED_NORESTORE);
+ XE_BO_FLAG_PINNED_NORESTORE |
+ xe_guc_bo_wa_flags(gt));
if (IS_ERR(bo))
return PTR_ERR(bo);
diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
index fe70c0fd85c5..1ad96e4b4f89 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.c
+++ b/drivers/gpu/drm/xe/xe_guc_ct.c
@@ -376,7 +376,8 @@ int xe_guc_ct_init(struct xe_guc_ct *ct)
XE_BO_FLAG_SYSTEM |
XE_BO_FLAG_GGTT |
XE_BO_FLAG_GGTT_INVALIDATE |
- XE_BO_FLAG_PINNED_NORESTORE);
+ XE_BO_FLAG_PINNED_NORESTORE |
+ xe_guc_bo_wa_flags(gt));
if (IS_ERR(bo))
return PTR_ERR(bo);
@@ -386,7 +387,8 @@ int xe_guc_ct_init(struct xe_guc_ct *ct)
XE_BO_FLAG_SYSTEM |
XE_BO_FLAG_GGTT |
XE_BO_FLAG_GGTT_INVALIDATE |
- XE_BO_FLAG_PINNED_NORESTORE);
+ XE_BO_FLAG_PINNED_NORESTORE |
+ xe_guc_bo_wa_flags(gt));
if (IS_ERR(bo))
return PTR_ERR(bo);
diff --git a/drivers/gpu/drm/xe/xe_guc_engine_activity.c b/drivers/gpu/drm/xe/xe_guc_engine_activity.c
index a782be57caad..729ce8ac140b 100644
--- a/drivers/gpu/drm/xe/xe_guc_engine_activity.c
+++ b/drivers/gpu/drm/xe/xe_guc_engine_activity.c
@@ -97,7 +97,8 @@ static int allocate_engine_activity_buffers(struct xe_guc *guc,
metadata_bo = xe_bo_create_pin_map_novm(gt_to_xe(gt), tile, PAGE_ALIGN(metadata_size),
ttm_bo_type_kernel, XE_BO_FLAG_SYSTEM |
- XE_BO_FLAG_GGTT | XE_BO_FLAG_GGTT_INVALIDATE,
+ XE_BO_FLAG_GGTT | XE_BO_FLAG_GGTT_INVALIDATE |
+ xe_guc_bo_wa_flags(gt),
false);
if (IS_ERR(metadata_bo))
@@ -105,7 +106,8 @@ static int allocate_engine_activity_buffers(struct xe_guc *guc,
bo = xe_bo_create_pin_map_novm(gt_to_xe(gt), tile, PAGE_ALIGN(size),
ttm_bo_type_kernel, XE_BO_FLAG_VRAM_IF_DGFX(tile) |
- XE_BO_FLAG_GGTT | XE_BO_FLAG_GGTT_INVALIDATE, false);
+ XE_BO_FLAG_GGTT | XE_BO_FLAG_GGTT_INVALIDATE |
+ xe_guc_bo_wa_flags(gt), false);
if (IS_ERR(bo)) {
xe_bo_unpin_map_no_vm(metadata_bo);
diff --git a/drivers/gpu/drm/xe/xe_guc_log.c b/drivers/gpu/drm/xe/xe_guc_log.c
index 538d4df0f7aa..7d006268ce91 100644
--- a/drivers/gpu/drm/xe/xe_guc_log.c
+++ b/drivers/gpu/drm/xe/xe_guc_log.c
@@ -17,6 +17,7 @@
#include "xe_force_wake.h"
#include "xe_gt_printk.h"
#include "xe_gt_types.h"
+#include "xe_guc.h"
#include "xe_map.h"
#include "xe_mmio.h"
#include "xe_module.h"
@@ -624,14 +625,16 @@ void xe_guc_log_print_lfd(struct xe_guc_log *log, struct drm_printer *p)
int xe_guc_log_init(struct xe_guc_log *log)
{
struct xe_device *xe = log_to_xe(log);
- struct xe_tile *tile = gt_to_tile(log_to_gt(log));
+ struct xe_gt *gt = log_to_gt(log);
+ struct xe_tile *tile = gt_to_tile(gt);
struct xe_bo *bo;
bo = xe_managed_bo_create_pin_map(xe, tile, GUC_LOG_SIZE,
XE_BO_FLAG_SYSTEM |
XE_BO_FLAG_GGTT |
XE_BO_FLAG_GGTT_INVALIDATE |
- XE_BO_FLAG_PINNED_NORESTORE);
+ XE_BO_FLAG_PINNED_NORESTORE |
+ xe_guc_bo_wa_flags(gt));
if (IS_ERR(bo))
return PTR_ERR(bo);
diff --git a/drivers/gpu/drm/xe/xe_guc_pc.c b/drivers/gpu/drm/xe/xe_guc_pc.c
index 7cf8f4858598..58e41b2319bc 100644
--- a/drivers/gpu/drm/xe/xe_guc_pc.c
+++ b/drivers/gpu/drm/xe/xe_guc_pc.c
@@ -1378,7 +1378,8 @@ int xe_guc_pc_init(struct xe_guc_pc *pc)
XE_BO_FLAG_VRAM_IF_DGFX(tile) |
XE_BO_FLAG_GGTT |
XE_BO_FLAG_GGTT_INVALIDATE |
- XE_BO_FLAG_PINNED_NORESTORE);
+ XE_BO_FLAG_PINNED_NORESTORE |
+ xe_guc_bo_wa_flags(gt));
if (IS_ERR(bo))
return PTR_ERR(bo);
diff --git a/drivers/gpu/drm/xe/xe_wa_oob.rules b/drivers/gpu/drm/xe/xe_wa_oob.rules
index 3001155f8d09..6662987effd7 100644
--- a/drivers/gpu/drm/xe/xe_wa_oob.rules
+++ b/drivers/gpu/drm/xe/xe_wa_oob.rules
@@ -14,6 +14,7 @@
16017236439 PLATFORM(PVC)
14019821291 MEDIA_VERSION_RANGE(1300, 2000)
14015076503 MEDIA_VERSION(1300)
+22016122933 MEDIA_VERSION(1300)
14018913170 GRAPHICS_VERSION_RANGE(1270, 1274)
MEDIA_VERSION(1300)
PLATFORM(DG2)
diff --git a/drivers/hte/hte-tegra194-test.c b/drivers/hte/hte-tegra194-test.c
index 32907d951ec9..cca33868fb03 100644
--- a/drivers/hte/hte-tegra194-test.c
+++ b/drivers/hte/hte-tegra194-test.c
@@ -217,10 +217,10 @@ static void tegra_hte_test_remove(struct platform_device *pdev)
{
(void)pdev;
+ timer_shutdown_sync(&hte.timer);
free_irq(hte.gpio_in_irq, &hte);
gpiod_put(hte.gpio_in);
gpiod_put(hte.gpio_out);
- timer_delete_sync(&hte.timer);
}
static struct platform_driver tegra_hte_test_driver = {
diff --git a/drivers/hte/hte-tegra194.c b/drivers/hte/hte-tegra194.c
index 4a7702b32b24..1612812abe8f 100644
--- a/drivers/hte/hte-tegra194.c
+++ b/drivers/hte/hte-tegra194.c
@@ -910,17 +910,17 @@ static int tegra_hte_probe(struct platform_device *pdev)
hte_dev->chip = gc;
+ for (i = 0; i < slices; i++) {
+ hte_dev->sl[i].flags = 0;
+ spin_lock_init(&hte_dev->sl[i].s_lock);
+ }
+
ret = devm_hte_register_chip(hte_dev->chip);
if (ret) {
dev_err(gc->dev, "hte chip register failed");
return ret;
}
- for (i = 0; i < slices; i++) {
- hte_dev->sl[i].flags = 0;
- spin_lock_init(&hte_dev->sl[i].s_lock);
- }
-
val = HTE_TECTRL_ENABLE_ENABLE |
(HTE_TECTRL_INTR_ENABLE << HTE_TECTRL_INTR_SHIFT) |
(hte_dev->itr_thrshld << HTE_TECTRL_OCCU_SHIFT);
diff --git a/drivers/hv/mshv_vtl_main.c b/drivers/hv/mshv_vtl_main.c
index c6f1e2157d25..9133646b5b66 100644
--- a/drivers/hv/mshv_vtl_main.c
+++ b/drivers/hv/mshv_vtl_main.c
@@ -73,6 +73,7 @@ struct mshv_vtl_poll_file {
wait_queue_entry_t wait;
wait_queue_head_t *wqh;
poll_table pt;
+ int error;
int cpu;
};
@@ -482,7 +483,18 @@ static void mshv_vtl_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh
{
struct mshv_vtl_poll_file *poll_file = container_of(pt, struct mshv_vtl_poll_file, pt);
- WARN_ON(poll_file->wqh);
+ if (poll_file->error)
+ return;
+
+ if (poll_file->wqh) {
+ if (poll_file->wqh == wqh)
+ return;
+
+ mshv_vtl_poll_file_remove_wait_queue(poll_file);
+ poll_file->error = -EINVAL;
+ return;
+ }
+
poll_file->wqh = wqh;
add_wait_queue(wqh, &poll_file->wait);
}
@@ -492,6 +504,7 @@ static int mshv_vtl_ioctl_set_poll_file(struct mshv_vtl_set_poll_file __user *us
struct file *file, *old_file;
struct mshv_vtl_poll_file *poll_file;
struct mshv_vtl_set_poll_file input;
+ int ret;
if (copy_from_user(&input, user_input, sizeof(input)))
return -EFAULT;
@@ -518,6 +531,7 @@ static int mshv_vtl_ioctl_set_poll_file(struct mshv_vtl_set_poll_file __user *us
old_file = poll_file->file;
poll_file->file = file;
+ poll_file->error = 0;
poll_file->cpu = input.cpu;
if (file) {
@@ -525,13 +539,18 @@ static int mshv_vtl_ioctl_set_poll_file(struct mshv_vtl_set_poll_file __user *us
init_poll_funcptr(&poll_file->pt, mshv_vtl_ptable_queue_proc);
vfs_poll(file, &poll_file->pt);
}
+ ret = poll_file->error;
+ if (ret)
+ poll_file->file = NULL;
mutex_unlock(&mshv_vtl_poll_file_lock);
if (old_file)
fput(old_file);
+ if (ret)
+ fput(file);
- return 0;
+ return ret;
}
/* Static table mapping register names to their corresponding actions */
diff --git a/drivers/hwmon/acpi_power_meter.c b/drivers/hwmon/acpi_power_meter.c
index 8a539e8d1334..d3886adddf40 100644
--- a/drivers/hwmon/acpi_power_meter.c
+++ b/drivers/hwmon/acpi_power_meter.c
@@ -10,6 +10,7 @@
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
#include <linux/jiffies.h>
+#include <linux/limits.h>
#include <linux/mutex.h>
#include <linux/dmi.h>
#include <linux/slab.h>
@@ -30,7 +31,12 @@
#define POWER_METER_CAN_NOTIFY (1 << 3)
#define POWER_METER_IS_BATTERY (1 << 8)
#define UNKNOWN_HYSTERESIS 0xFFFFFFFF
-#define UNKNOWN_POWER 0xFFFFFFFF
+/*
+ * ACPI integers can either be 32-bit or 64-bit wide, so
+ * we have to check for both values.
+ */
+#define UNKNOWN_POWER_32 U32_MAX
+#define UNKNOWN_POWER_64 U64_MAX
#define METER_NOTIFY_CONFIG 0x80
#define METER_NOTIFY_TRIP 0x81
@@ -452,7 +458,7 @@ static int power_meter_read(struct device *dev, enum hwmon_sensor_types type,
ret = update_meter(res);
if (ret)
return ret;
- if (res->power == UNKNOWN_POWER)
+ if (res->power == UNKNOWN_POWER_32 || res->power == UNKNOWN_POWER_64)
return -ENODATA;
*val = res->power * 1000;
break;
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 5095eb057680..9ed21c8c093a 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -60,6 +60,8 @@ enum coretemp_attr_index {
/*
* Per-Core Temperature Data
+ * @temp_valid: Whether the CPU reported @temp as valid. @temp is not
+ * reported when this is false.
* @tjmax: The static tjmax value when tjmax cannot be retrieved from
* IA32_TEMPERATURE_TARGET MSR.
* @last_updated: The time when the current temperature value was updated
@@ -72,6 +74,7 @@ enum coretemp_attr_index {
*/
struct temp_data {
int temp;
+ bool temp_valid;
int tjmax;
unsigned long last_updated;
unsigned int cpu;
@@ -398,7 +401,8 @@ static ssize_t show_temp(struct device *dev,
{
struct msr val;
struct temp_data *tdata = container_of(devattr, struct temp_data, sd_attrs[ATTR_TEMP]);
- int tjmax;
+ int tjmax, temp;
+ bool valid;
mutex_lock(&tdata->update_lock);
@@ -407,17 +411,22 @@ static ssize_t show_temp(struct device *dev,
if (time_after(jiffies, tdata->last_updated + HZ)) {
rdmsrq_on_cpu(tdata->cpu, tdata->status_reg, &val.q);
/*
- * Ignore the valid bit. In all observed cases the register
- * value is either low or zero if the valid bit is 0.
- * Return it instead of reporting an error which doesn't
- * really help at all.
+ * Bit 31 indicates whether the reading is valid. Some CPUs
+ * clear it while the temperature is too low to be measured.
+ * Such a reading is not a temperature. Do not report it.
*/
+ tdata->temp_valid = (val.l >> 31) & 1;
tdata->temp = tjmax - ((val.l >> 16) & 0xff) * 1000;
tdata->last_updated = jiffies;
}
+ valid = tdata->temp_valid;
+ temp = tdata->temp;
mutex_unlock(&tdata->update_lock);
- return sprintf(buf, "%d\n", tdata->temp);
+
+ if (!valid)
+ return -ENODATA;
+ return sprintf(buf, "%d\n", temp);
}
static int create_core_attrs(struct temp_data *tdata, struct device *dev)
@@ -511,6 +520,11 @@ init_temp_data(struct platform_data *pdata, unsigned int cpu, int pkg_flag)
tdata->cpu = cpu;
tdata->cpu_core_id = topology_core_id(cpu);
tdata->attr_size = MAX_CORE_ATTRS;
+ /*
+ * A zero timestamp does not look stale on 32-bit, where jiffies
+ * starts just short of wrapping. Backdate it instead.
+ */
+ tdata->last_updated = jiffies - HZ - 1;
mutex_init(&tdata->update_lock);
return tdata;
}
@@ -554,9 +568,6 @@ static int create_core_data(struct platform_device *pdev, unsigned int cpu,
u64 val;
int err;
- if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
- return 0;
-
tdata = init_temp_data(pdata, cpu, pkg_flag);
if (!tdata)
return -ENOMEM;
@@ -676,6 +687,14 @@ static int coretemp_cpu_online(unsigned int cpu)
if (!cpu_has(c, X86_FEATURE_DTHERM))
return -ENODEV;
+ /*
+ * Keep isolated CPUs out of pdata->cpumask. Besides tracking the CPUs
+ * available to this driver, the mask is used to select the CPU on which
+ * to read the temperature MSRs.
+ */
+ if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
+ return 0;
+
pdata = platform_get_drvdata(pdev);
if (!pdata->hwmon_dev) {
struct device *hwmon;
@@ -705,8 +724,8 @@ static int coretemp_cpu_online(unsigned int cpu)
}
/*
- * Check whether a thread sibling is already online. If not add the
- * interface for this CPU core.
+ * Check whether a housekeeping thread sibling is already online. If
+ * not, add the interface for this CPU core.
*/
if (!cpumask_intersects(&pdata->cpumask, topology_sibling_cpumask(cpu)))
coretemp_add_core(pdev, cpu, 0);
@@ -736,9 +755,9 @@ static int coretemp_cpu_offline(unsigned int cpu)
cpumask_clear_cpu(cpu, &pd->cpumask);
/*
- * If this is the last thread sibling, remove the CPU core
- * interface, If there is still a sibling online, transfer the
- * target cpu of that core interface to it.
+ * If this is the last housekeeping thread sibling, remove the CPU core
+ * interface. If there is still a housekeeping sibling online, transfer
+ * the target CPU of that core interface to it.
*/
target = cpumask_any_and(&pd->cpumask, topology_sibling_cpumask(cpu));
if (target >= nr_cpu_ids) {
@@ -750,7 +769,8 @@ static int coretemp_cpu_offline(unsigned int cpu)
}
/*
- * If all cores in this pkg are offline, remove the interface.
+ * If all housekeeping cores in this pkg are offline, remove the
+ * interface.
*/
tdata = get_temp_data(pd, -1);
if (cpumask_empty(&pd->cpumask)) {
@@ -763,7 +783,7 @@ static int coretemp_cpu_offline(unsigned int cpu)
/*
* Check whether this core is the target for the package
- * interface. We need to assign it to some other cpu.
+ * interface. We need to assign it to some other housekeeping CPU.
*/
if (tdata && tdata->cpu == cpu) {
target = cpumask_first(&pd->cpumask);
diff --git a/drivers/hwmon/corsair-cpro.c b/drivers/hwmon/corsair-cpro.c
index c09645152613..e6d31d367698 100644
--- a/drivers/hwmon/corsair-cpro.c
+++ b/drivers/hwmon/corsair-cpro.c
@@ -135,9 +135,9 @@ static int send_usb_cmd(struct ccp_device *ccp, u8 command, u8 byte1, u8 byte2,
* the raw event parsing and marked the ccp->wait_input_report
* completion as done.
*/
- spin_lock_bh(&ccp->wait_input_report_lock);
+ spin_lock_irq(&ccp->wait_input_report_lock);
reinit_completion(&ccp->wait_input_report);
- spin_unlock_bh(&ccp->wait_input_report_lock);
+ spin_unlock_irq(&ccp->wait_input_report_lock);
ret = hid_hw_output_report(ccp->hdev, ccp->cmd_buffer, OUT_BUFFER_SIZE);
if (ret < 0)
@@ -156,15 +156,16 @@ static int send_usb_cmd(struct ccp_device *ccp, u8 command, u8 byte1, u8 byte2,
static int ccp_raw_event(struct hid_device *hdev, struct hid_report *report, u8 *data, int size)
{
struct ccp_device *ccp = hid_get_drvdata(hdev);
+ unsigned long flags;
/* only copy buffer when requested */
- spin_lock(&ccp->wait_input_report_lock);
+ spin_lock_irqsave(&ccp->wait_input_report_lock, flags);
if (!completion_done(&ccp->wait_input_report)) {
memcpy(ccp->buffer, data, min(IN_BUFFER_SIZE, size));
ccp->buffer_recv_size = size;
complete_all(&ccp->wait_input_report);
}
- spin_unlock(&ccp->wait_input_report_lock);
+ spin_unlock_irqrestore(&ccp->wait_input_report_lock, flags);
return 0;
}
diff --git a/drivers/hwmon/gigabyte_waterforce.c b/drivers/hwmon/gigabyte_waterforce.c
index 4eea05f8b569..f4d0376dd68e 100644
--- a/drivers/hwmon/gigabyte_waterforce.c
+++ b/drivers/hwmon/gigabyte_waterforce.c
@@ -257,6 +257,9 @@ static int waterforce_raw_event(struct hid_device *hdev, struct hid_report *repo
{
struct waterforce_data *priv = hid_get_drvdata(hdev);
+ if (size <= FIRMWARE_VER_START_OFFSET_2)
+ return 0;
+
if (data[0] == get_firmware_ver_cmd[0] && data[1] == get_firmware_ver_cmd[1]) {
/* Received a firmware version report */
priv->firmware_version =
@@ -269,6 +272,8 @@ static int waterforce_raw_event(struct hid_device *hdev, struct hid_report *repo
if (data[0] != get_status_cmd[0] || data[1] != get_status_cmd[1])
return 0;
+ if (size <= WATERFORCE_TEMP_SENSOR)
+ return 0;
priv->temp_input[0] = data[WATERFORCE_TEMP_SENSOR] * 1000;
priv->speed_input[0] = get_unaligned_le16(data + WATERFORCE_FAN_SPEED);
diff --git a/drivers/hwmon/lm70.c b/drivers/hwmon/lm70.c
index 8b525725fef4..b29bac7a4cc1 100644
--- a/drivers/hwmon/lm70.c
+++ b/drivers/hwmon/lm70.c
@@ -96,21 +96,21 @@ static ssize_t temp1_input_show(struct device *dev,
*/
switch (p_lm70->chip) {
case LM70_CHIP_LM70:
- val = ((int)raw / 32) * 250;
+ val = ((int)raw >> 5) * 250;
break;
case LM70_CHIP_TMP121:
case LM70_CHIP_TMP122:
case LM70_CHIP_LM74:
- val = ((int)raw / 8) * 625 / 10;
+ val = ((int)raw >> 3) * 625 / 10;
break;
case LM70_CHIP_LM71:
- val = ((int)raw / 4) * 3125 / 100;
+ val = ((int)raw >> 2) * 3125 / 100;
break;
case LM70_CHIP_TMP125:
- val = (sign_extend32(raw, 14) / 32) * 250;
+ val = (sign_extend32(raw, 14) >> 5) * 250;
break;
}
diff --git a/drivers/hwmon/lm95245.c b/drivers/hwmon/lm95245.c
index 11553391f54b..925574b8b0ca 100644
--- a/drivers/hwmon/lm95245.c
+++ b/drivers/hwmon/lm95245.c
@@ -96,9 +96,9 @@ static int temp_from_reg_unsigned(u8 val_h, u8 val_l)
static int temp_from_reg_signed(u8 val_h, u8 val_l)
{
- if (val_h & 0x80)
- return (val_h - 0x100) * 1000;
- return temp_from_reg_unsigned(val_h, val_l);
+ s16 val_hl = (val_h << 8) | val_l;
+
+ return val_hl * 1000 / 256;
}
static int lm95245_read_conversion_rate(struct lm95245_data *data)
diff --git a/drivers/hwmon/mcp9982.c b/drivers/hwmon/mcp9982.c
index 3918dc36c946..2cd41dcbd9af 100644
--- a/drivers/hwmon/mcp9982.c
+++ b/drivers/hwmon/mcp9982.c
@@ -624,6 +624,7 @@ static int mcp9982_write(struct device *dev, enum hwmon_sensor_types type, u32 a
{
struct mcp9982_priv *priv = dev_get_drvdata(dev);
unsigned int idx;
+ int ret;
u8 addr;
switch (type) {
@@ -640,9 +641,13 @@ static int mcp9982_write(struct device *dev, enum hwmon_sensor_types type, u32 a
idx = find_closest_descending(val, mcp9982_update_interval,
ARRAY_SIZE(mcp9982_update_interval));
+ ret = regmap_write(priv->regmap, MCP9982_CONV_ADDR, idx);
+ if (ret)
+ return ret;
+
priv->interval_idx = idx;
- return regmap_write(priv->regmap, MCP9982_CONV_ADDR, idx);
+ return 0;
default:
return -EINVAL;
}
diff --git a/drivers/hwmon/sht3x.c b/drivers/hwmon/sht3x.c
index b46de050e20a..a0f5ba5c888d 100644
--- a/drivers/hwmon/sht3x.c
+++ b/drivers/hwmon/sht3x.c
@@ -543,7 +543,10 @@ static ssize_t heater_enable_store(struct device *dev,
mutex_unlock(&data->i2c_lock);
- return ret;
+ if (ret != SHT3X_CMD_LENGTH)
+ return ret < 0 ? ret : -EIO;
+
+ return count;
}
static int update_interval_read(struct device *dev)
diff --git a/drivers/hwmon/sht4x.c b/drivers/hwmon/sht4x.c
index a97dda9e92dc..0bb9227674a8 100644
--- a/drivers/hwmon/sht4x.c
+++ b/drivers/hwmon/sht4x.c
@@ -67,7 +67,7 @@ DECLARE_CRC8_TABLE(sht4x_crc8_table);
*/
struct sht4x_data {
struct i2c_client *client;
- unsigned long heating_complete; /* in jiffies */
+ u64 heating_complete; /* in jiffies */
bool data_pending;
u32 heater_power; /* in milli-watts */
u32 heater_time; /* in milli-seconds */
@@ -92,14 +92,14 @@ static int sht4x_read_values(struct sht4x_data *data)
u8 crc;
u8 cmd[SHT4X_CMD_LEN] = {SHT4X_CMD_MEASURE_HPM};
u8 raw_data[SHT4X_RESPONSE_LENGTH];
- unsigned long curr_jiffies;
+ u64 curr_jiffies;
- curr_jiffies = jiffies;
- if (time_before(curr_jiffies, data->heating_complete))
+ curr_jiffies = get_jiffies_64();
+ if (time_before64(curr_jiffies, data->heating_complete))
msleep(jiffies_to_msecs(data->heating_complete - curr_jiffies));
if (data->data_pending &&
- time_before(jiffies, data->heating_complete + data->update_interval)) {
+ time_before64(get_jiffies_64(), data->heating_complete + data->update_interval)) {
data->data_pending = false;
} else {
next_update = data->last_updated +
@@ -237,7 +237,9 @@ static ssize_t heater_enable_show(struct device *dev,
{
struct sht4x_data *data = dev_get_drvdata(dev);
- return sysfs_emit(buf, "%u\n", time_before(jiffies, data->heating_complete));
+ guard(hwmon_lock)(dev);
+
+ return sysfs_emit(buf, "%u\n", time_before64(get_jiffies_64(), data->heating_complete));
}
static ssize_t heater_enable_store(struct device *dev,
@@ -279,14 +281,16 @@ static ssize_t heater_enable_store(struct device *dev,
guard(hwmon_lock)(dev);
- if (time_before(jiffies, data->heating_complete))
+ if (time_before64(get_jiffies_64(), data->heating_complete))
return -EBUSY;
ret = i2c_master_send(data->client, &cmd, SHT4X_CMD_LEN);
if (ret < 0)
return ret;
+ if (ret != SHT4X_CMD_LEN)
+ return -EIO;
- data->heating_complete = jiffies + msecs_to_jiffies(heating_time_bound);
+ data->heating_complete = get_jiffies_64() + msecs_to_jiffies(heating_time_bound);
data->data_pending = true;
return count;
}
@@ -410,7 +414,7 @@ static int sht4x_probe(struct i2c_client *client)
data->client = client;
data->heater_power = 200;
data->heater_time = 1000;
- data->heating_complete = jiffies;
+ data->heating_complete = get_jiffies_64();
crc8_populate_msb(sht4x_crc8_table, SHT4X_CRC8_POLYNOMIAL);
diff --git a/drivers/hwmon/tmp102.c b/drivers/hwmon/tmp102.c
index 50a8b050d8da..8c8463ddb6c4 100644
--- a/drivers/hwmon/tmp102.c
+++ b/drivers/hwmon/tmp102.c
@@ -58,7 +58,7 @@ struct tmp102 {
const char *label;
struct regmap *regmap;
u16 config_orig;
- unsigned long ready_time;
+ u64 ready_time;
u16 sample_time;
};
@@ -106,7 +106,7 @@ static int tmp102_read_temp(struct device *dev, u32 attr, long *val)
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp102->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp102->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n", __func__);
return -EAGAIN;
}
@@ -354,7 +354,7 @@ static int tmp102_probe(struct i2c_client *client)
* Mark that we are not ready with data until the first
* conversion is complete
*/
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
hwmon_dev = devm_hwmon_device_register_with_info(dev, client->name,
tmp102,
@@ -387,7 +387,7 @@ static int tmp102_resume(struct device *dev)
err = regmap_update_bits(tmp102->regmap, TMP102_CONF_REG,
TMP102_CONF_SD, 0);
- tmp102->ready_time = jiffies + msecs_to_jiffies(CONVERSION_TIME_MS);
+ tmp102->ready_time = get_jiffies_64() + msecs_to_jiffies(CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/hwmon/tmp108.c b/drivers/hwmon/tmp108.c
index 9fa31bd66ff6..91d20a1ce112 100644
--- a/drivers/hwmon/tmp108.c
+++ b/drivers/hwmon/tmp108.c
@@ -78,7 +78,7 @@
struct tmp108 {
struct regmap *regmap;
u16 orig_config;
- unsigned long ready_time;
+ u64 ready_time;
const struct tmp108_params *params;
};
@@ -138,7 +138,7 @@ static int tmp108_read(struct device *dev, enum hwmon_sensor_types type,
switch (attr) {
case hwmon_temp_input:
/* Is it too early to return a conversion ? */
- if (time_before(jiffies, tmp108->ready_time)) {
+ if (time_before64(get_jiffies_64(), tmp108->ready_time)) {
dev_dbg(dev, "%s: Conversion not ready yet..\n",
__func__);
return -EAGAIN;
@@ -477,7 +477,7 @@ static int tmp108_common_probe(struct device *dev, struct regmap *regmap, char *
return err;
}
- tmp108->ready_time = jiffies;
+ tmp108->ready_time = get_jiffies_64();
if ((tmp108->orig_config & TMP108_CONF_MODE_MASK) ==
TMP108_MODE_SHUTDOWN)
tmp108->ready_time +=
@@ -528,7 +528,7 @@ static int tmp108_resume(struct device *dev)
err = regmap_update_bits(tmp108->regmap, TMP108_REG_CONF,
TMP108_CONF_MODE_MASK, TMP108_MODE_CONTINUOUS);
- tmp108->ready_time = jiffies +
+ tmp108->ready_time = get_jiffies_64() +
msecs_to_jiffies(TMP108_CONVERSION_TIME_MS);
return err;
}
diff --git a/drivers/i2c/busses/i2c-designware-master.c b/drivers/i2c/busses/i2c-designware-master.c
index a1bcc3797e4f..cd4f57b4e2b5 100644
--- a/drivers/i2c/busses/i2c-designware-master.c
+++ b/drivers/i2c/busses/i2c-designware-master.c
@@ -441,6 +441,14 @@ i2c_dw_xfer_msg(struct dw_i2c_dev *dev)
if (dev->rx_outstanding >= dev->rx_fifo_depth)
break;
+ /*
+ * For block reads, queue only the length
+ * byte until i2c_dw_recv_len() runs.
+ */
+ if ((flags & I2C_M_RECV_LEN) &&
+ buf_len < msgs[dev->msg_write_idx].len)
+ break;
+
regmap_write(dev->map, DW_IC_DATA_CMD,
cmd | 0x100);
rx_limit--;
diff --git a/drivers/i2c/busses/i2c-mt7621.c b/drivers/i2c/busses/i2c-mt7621.c
index 0a288c998419..9ea5e3f9a763 100644
--- a/drivers/i2c/busses/i2c-mt7621.c
+++ b/drivers/i2c/busses/i2c-mt7621.c
@@ -144,7 +144,6 @@ static int mtk_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
u16 addr;
int i, j, ret, len, page_len;
u32 cmd;
- u32 data[2];
i2c = i2c_get_adapdata(adap);
@@ -186,6 +185,8 @@ static int mtk_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
/* transfer data */
for (len = pmsg->len, j = 0; len > 0; len -= 8, j += 8) {
+ u32 data[2] = { };
+
page_len = (len >= 8) ? 8 : len;
if (pmsg->flags & I2C_M_RD) {
@@ -194,7 +195,8 @@ static int mtk_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
} else {
memcpy(data, &pmsg->buf[j], page_len);
iowrite32(data[0], i2c->base + REG_SM0D0_REG);
- iowrite32(data[1], i2c->base + REG_SM0D1_REG);
+ if (page_len > 4)
+ iowrite32(data[1], i2c->base + REG_SM0D1_REG);
cmd = SM0CTL1_WRITE;
}
@@ -204,7 +206,8 @@ static int mtk_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
if (pmsg->flags & I2C_M_RD) {
data[0] = ioread32(i2c->base + REG_SM0D0_REG);
- data[1] = ioread32(i2c->base + REG_SM0D1_REG);
+ if (page_len > 4)
+ data[1] = ioread32(i2c->base + REG_SM0D1_REG);
memcpy(&pmsg->buf[j], data, page_len);
} else {
if (!(pmsg->flags & I2C_M_IGNORE_NAK)) {
diff --git a/drivers/i2c/i2c-core-acpi.c b/drivers/i2c/i2c-core-acpi.c
index 8f3bdd50186e..37d1309e93b5 100644
--- a/drivers/i2c/i2c-core-acpi.c
+++ b/drivers/i2c/i2c-core-acpi.c
@@ -547,6 +547,7 @@ struct i2c_client *i2c_acpi_new_device_by_fwnode(struct fwnode_handle *fwnode,
{
struct i2c_acpi_lookup lookup;
struct i2c_adapter *adapter;
+ struct i2c_client *client;
struct acpi_device *adev;
LIST_HEAD(resource_list);
int ret;
@@ -574,7 +575,10 @@ struct i2c_client *i2c_acpi_new_device_by_fwnode(struct fwnode_handle *fwnode,
if (!adapter)
return ERR_PTR(-EPROBE_DEFER);
- return i2c_new_client_device(adapter, info);
+ client = i2c_new_client_device(adapter, info);
+ put_device(&adapter->dev);
+
+ return client;
}
EXPORT_SYMBOL_GPL(i2c_acpi_new_device_by_fwnode);
diff --git a/drivers/i3c/master.c b/drivers/i3c/master.c
index afcd7a21a3e6..f9a6c8560fab 100644
--- a/drivers/i3c/master.c
+++ b/drivers/i3c/master.c
@@ -2216,12 +2216,11 @@ struct i3c_dma *i3c_master_dma_map_single(struct device *dev, void *buf,
if (force_bounce) {
dma_xfer->map_len = ALIGN(len, cache_line_size());
- if (dir == DMA_FROM_DEVICE)
- bounce = kzalloc(dma_xfer->map_len, GFP_KERNEL);
- else
- bounce = kmemdup(buf, dma_xfer->map_len, GFP_KERNEL);
+ bounce = kzalloc(dma_xfer->map_len, GFP_KERNEL);
if (!bounce)
return NULL;
+ if (dir != DMA_FROM_DEVICE)
+ memcpy(bounce, buf, len);
dma_buf = bounce;
}
diff --git a/drivers/i3c/master/adi-i3c-master.c b/drivers/i3c/master/adi-i3c-master.c
index b35386260350..ad57b7500678 100644
--- a/drivers/i3c/master/adi-i3c-master.c
+++ b/drivers/i3c/master/adi-i3c-master.c
@@ -115,6 +115,7 @@ struct adi_i3c_master {
spinlock_t lock; /* Protect transfer */
} xferqueue;
void __iomem *regs;
+ int irq;
struct clk *clk;
unsigned long i3c_scl_lim;
struct {
@@ -957,6 +958,7 @@ static int adi_i3c_master_probe(struct platform_device *pdev)
irq = platform_get_irq(pdev, 0);
if (irq < 0)
return irq;
+ master->irq = irq;
version = readl(master->regs + ADI_AXI_REG_VERSION);
if (ADI_AXI_PCORE_VER_MAJOR(version) != 1)
@@ -1001,6 +1003,8 @@ static void adi_i3c_master_remove(struct platform_device *pdev)
writel(0x00, master->regs + REG_IRQ_MASK);
writel(0x01, master->regs + REG_ENABLE);
+ devm_free_irq(&pdev->dev, master->irq, master);
+
i3c_master_unregister(&master->base);
}
diff --git a/drivers/i3c/master/mipi-i3c-hci/dma.c b/drivers/i3c/master/mipi-i3c-hci/dma.c
index 7c2b20474130..5b195978f376 100644
--- a/drivers/i3c/master/mipi-i3c-hci/dma.c
+++ b/drivers/i3c/master/mipi-i3c-hci/dma.c
@@ -428,7 +428,7 @@ static void hci_dma_unmap_xfer(struct i3c_hci *hci,
static struct i3c_dma *hci_dma_map_xfer(struct device *dev, struct hci_xfer *xfer)
{
enum dma_data_direction dir = xfer->rnw ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
- bool need_bounce = device_iommu_mapped(dev) && xfer->rnw && (xfer->data_len & 3);
+ bool need_bounce = xfer->rnw && (xfer->data_len & 3);
return i3c_master_dma_map_single(dev, xfer->data, xfer->data_len, need_bounce, dir);
}
diff --git a/drivers/i3c/master/mipi-i3c-hci/mipi-i3c-hci-pci.c b/drivers/i3c/master/mipi-i3c-hci/mipi-i3c-hci-pci.c
index ab595661db1e..af9595009b03 100644
--- a/drivers/i3c/master/mipi-i3c-hci/mipi-i3c-hci-pci.c
+++ b/drivers/i3c/master/mipi-i3c-hci/mipi-i3c-hci-pci.c
@@ -445,6 +445,8 @@ static int mipi_i3c_hci_pci_probe(struct pci_dev *pci,
if (ret < 0)
return ret;
+ pci_set_drvdata(pci, hci);
+
hci->info = (const struct mipi_i3c_hci_pci_info *)id->driver_data;
ret = hci->info->init ? hci->info->init(hci) : 0;
@@ -455,8 +457,6 @@ static int mipi_i3c_hci_pci_probe(struct pci_dev *pci,
if (ret)
goto err_exit;
- pci_set_drvdata(pci, hci);
-
mipi_i3c_hci_pci_rpm_allow(&pci->dev);
return 0;
diff --git a/drivers/iio/accel/adis16201.c b/drivers/iio/accel/adis16201.c
index ba0f97944c6d..834714fb94a5 100644
--- a/drivers/iio/accel/adis16201.c
+++ b/drivers/iio/accel/adis16201.c
@@ -202,19 +202,19 @@ static int adis16201_write_raw(struct iio_dev *indio_dev,
}
static const struct iio_chan_spec adis16201_channels[] = {
- ADIS_SUPPLY_CHAN(ADIS16201_SUPPLY_OUT_REG, ADIS16201_SCAN_SUPPLY, 0,
- 12),
- ADIS_TEMP_CHAN(ADIS16201_TEMP_OUT_REG, ADIS16201_SCAN_TEMP, 0, 12),
ADIS_ACCEL_CHAN(X, ADIS16201_XACCL_OUT_REG, ADIS16201_SCAN_ACC_X,
BIT(IIO_CHAN_INFO_CALIBBIAS), 0, 14),
ADIS_ACCEL_CHAN(Y, ADIS16201_YACCL_OUT_REG, ADIS16201_SCAN_ACC_Y,
BIT(IIO_CHAN_INFO_CALIBBIAS), 0, 14),
- ADIS_AUX_ADC_CHAN(ADIS16201_AUX_ADC_REG, ADIS16201_SCAN_AUX_ADC, 0, 12),
ADIS_INCLI_CHAN(X, ADIS16201_XINCL_OUT_REG, ADIS16201_SCAN_INCLI_X,
BIT(IIO_CHAN_INFO_CALIBBIAS), 0, 12),
ADIS_INCLI_CHAN(Y, ADIS16201_YINCL_OUT_REG, ADIS16201_SCAN_INCLI_Y,
BIT(IIO_CHAN_INFO_CALIBBIAS), 0, 12),
- IIO_CHAN_SOFT_TIMESTAMP(7)
+ ADIS_SUPPLY_CHAN(ADIS16201_SUPPLY_OUT_REG, ADIS16201_SCAN_SUPPLY, 0,
+ 12),
+ ADIS_AUX_ADC_CHAN(ADIS16201_AUX_ADC_REG, ADIS16201_SCAN_AUX_ADC, 0, 12),
+ ADIS_TEMP_CHAN(ADIS16201_TEMP_OUT_REG, ADIS16201_SCAN_TEMP, 0, 12),
+ IIO_CHAN_SOFT_TIMESTAMP(7),
};
static const struct iio_info adis16201_info = {
diff --git a/drivers/iio/accel/adxl313_spi.c b/drivers/iio/accel/adxl313_spi.c
index a61295de104f..0f0da44461f7 100644
--- a/drivers/iio/accel/adxl313_spi.c
+++ b/drivers/iio/accel/adxl313_spi.c
@@ -79,6 +79,8 @@ static int adxl313_spi_probe(struct spi_device *spi)
return ret;
chip_data = spi_get_device_match_data(spi);
+ if (!chip_data)
+ return -ENODATA;
regmap = devm_regmap_init_spi(spi,
&adxl31x_spi_regmap_config[chip_data->type]);
diff --git a/drivers/iio/accel/adxl372_spi.c b/drivers/iio/accel/adxl372_spi.c
index 128e148c4e96..d54544d6df6f 100644
--- a/drivers/iio/accel/adxl372_spi.c
+++ b/drivers/iio/accel/adxl372_spi.c
@@ -25,6 +25,8 @@ static int adxl372_spi_probe(struct spi_device *spi)
struct regmap *regmap;
chip_info = spi_get_device_match_data(spi);
+ if (!chip_info)
+ return -ENODATA;
regmap = devm_regmap_init_spi(spi, &adxl372_spi_regmap_config);
if (IS_ERR(regmap))
diff --git a/drivers/iio/accel/adxl380_spi.c b/drivers/iio/accel/adxl380_spi.c
index ae8467f4e6c0..8ffb3a8690a6 100644
--- a/drivers/iio/accel/adxl380_spi.c
+++ b/drivers/iio/accel/adxl380_spi.c
@@ -25,6 +25,8 @@ static int adxl380_spi_probe(struct spi_device *spi)
struct regmap *regmap;
chip_data = spi_get_device_match_data(spi);
+ if (!chip_data)
+ return -ENODATA;
regmap = devm_regmap_init_spi(spi, &adxl380_spi_regmap_config);
if (IS_ERR(regmap))
diff --git a/drivers/iio/accel/kxcjk-1013.c b/drivers/iio/accel/kxcjk-1013.c
index 8994c9e8e048..b3d753bd9476 100644
--- a/drivers/iio/accel/kxcjk-1013.c
+++ b/drivers/iio/accel/kxcjk-1013.c
@@ -1547,6 +1547,7 @@ static void kxcjk1013_remove(struct i2c_client *client)
iio_device_unregister(indio_dev);
+ pm_runtime_dont_use_autosuspend(&client->dev);
pm_runtime_disable(&client->dev);
pm_runtime_set_suspended(&client->dev);
diff --git a/drivers/iio/accel/mma9551.c b/drivers/iio/accel/mma9551.c
index 7e01427fd9c4..182f488edab1 100644
--- a/drivers/iio/accel/mma9551.c
+++ b/drivers/iio/accel/mma9551.c
@@ -500,6 +500,7 @@ static void mma9551_remove(struct i2c_client *client)
iio_device_unregister(indio_dev);
+ pm_runtime_dont_use_autosuspend(&client->dev);
pm_runtime_disable(&client->dev);
pm_runtime_set_suspended(&client->dev);
diff --git a/drivers/iio/accel/mma9553.c b/drivers/iio/accel/mma9553.c
index 8e7aaac89d47..5a36d415a837 100644
--- a/drivers/iio/accel/mma9553.c
+++ b/drivers/iio/accel/mma9553.c
@@ -1136,6 +1136,7 @@ static void mma9553_remove(struct i2c_client *client)
iio_device_unregister(indio_dev);
+ pm_runtime_dont_use_autosuspend(&client->dev);
pm_runtime_disable(&client->dev);
pm_runtime_set_suspended(&client->dev);
diff --git a/drivers/iio/accel/sca3000.c b/drivers/iio/accel/sca3000.c
index 8f1ac1d3305e..acd0890c78a5 100644
--- a/drivers/iio/accel/sca3000.c
+++ b/drivers/iio/accel/sca3000.c
@@ -1462,6 +1462,8 @@ static int sca3000_probe(struct spi_device *spi)
st->us = spi;
mutex_init(&st->lock);
st->info = spi_get_device_match_data(spi);
+ if (!st->info)
+ return -ENODATA;
indio_dev->name = st->info->name;
indio_dev->info = &sca3000_info;
diff --git a/drivers/iio/adc/ad4080.c b/drivers/iio/adc/ad4080.c
index 04cd6628ebff..7b3b338886b4 100644
--- a/drivers/iio/adc/ad4080.c
+++ b/drivers/iio/adc/ad4080.c
@@ -685,7 +685,8 @@ static int ad4080_setup_channel(struct ad4080_state *st, unsigned int ch)
return ret;
ret = regmap_write(st->regmap[ch], AD4080_REG_INTERFACE_CONFIG_A,
- AD4080_INTERFACE_CONFIG_A_SDO_ENABLE);
+ AD4080_INTERFACE_CONFIG_A_SDO_ENABLE |
+ AD4080_INTERFACE_CONFIG_A_ADDR_ASC);
if (ret)
return ret;
diff --git a/drivers/iio/adc/ad7192.c b/drivers/iio/adc/ad7192.c
index d8f5886fcf69..8bd810546311 100644
--- a/drivers/iio/adc/ad7192.c
+++ b/drivers/iio/adc/ad7192.c
@@ -1406,6 +1406,9 @@ static int ad7192_probe(struct spi_device *spi)
st->int_vref_mv = ret == -ENODEV ? avdd_mv : ret / MILLI;
st->chip_info = spi_get_device_match_data(spi);
+ if (!st->chip_info)
+ return -ENODATA;
+
indio_dev->name = st->chip_info->name;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = st->chip_info->info;
diff --git a/drivers/iio/adc/ad7606_spi.c b/drivers/iio/adc/ad7606_spi.c
index 1ebdfb8580ab..8c479fa50a23 100644
--- a/drivers/iio/adc/ad7606_spi.c
+++ b/drivers/iio/adc/ad7606_spi.c
@@ -456,7 +456,11 @@ static const struct ad7606_bus_info ad7616_bus_info = {
static int ad7606_spi_probe(struct spi_device *spi)
{
- const struct ad7606_bus_info *bus_info = spi_get_device_match_data(spi);
+ const struct ad7606_bus_info *bus_info;
+
+ bus_info = spi_get_device_match_data(spi);
+ if (!bus_info)
+ return -ENODATA;
return ad7606_probe(&spi->dev, spi->irq, NULL,
bus_info->chip_info, bus_info->bops);
diff --git a/drivers/iio/adc/ad7768-1.c b/drivers/iio/adc/ad7768-1.c
index e9060c1bbe6f..3972821c295a 100644
--- a/drivers/iio/adc/ad7768-1.c
+++ b/drivers/iio/adc/ad7768-1.c
@@ -1829,6 +1829,9 @@ static int ad7768_probe(struct spi_device *spi)
}
st->chip = spi_get_device_match_data(spi);
+ if (!st->chip)
+ return -ENODATA;
+
st->spi = spi;
st->regmap = devm_regmap_init_spi(spi, &ad7768_regmap_config);
diff --git a/drivers/iio/adc/at91-sama5d2_adc.c b/drivers/iio/adc/at91-sama5d2_adc.c
index e8a5285bb6d4..e68632e628c7 100644
--- a/drivers/iio/adc/at91-sama5d2_adc.c
+++ b/drivers/iio/adc/at91-sama5d2_adc.c
@@ -2487,6 +2487,7 @@ static void at91_adc_remove(struct platform_device *pdev)
pm_runtime_disable(st->dev);
pm_runtime_set_suspended(st->dev);
+ pm_runtime_dont_use_autosuspend(st->dev);
clk_disable_unprepare(st->per_clk);
regulator_disable(st->vref);
diff --git a/drivers/iio/adc/max11205.c b/drivers/iio/adc/max11205.c
index 63b6561dcf47..53ac2f4f383e 100644
--- a/drivers/iio/adc/max11205.c
+++ b/drivers/iio/adc/max11205.c
@@ -117,6 +117,8 @@ static int max11205_probe(struct spi_device *spi)
ad_sd_init(&st->sd, indio_dev, spi, &max11205_sigma_delta_info);
st->chip_info = spi_get_device_match_data(spi);
+ if (!st->chip_info)
+ return -ENODATA;
indio_dev->name = st->chip_info->name;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/adc/max14001.c b/drivers/iio/adc/max14001.c
index 58adeac62dde..7694fbb6a9fd 100644
--- a/drivers/iio/adc/max14001.c
+++ b/drivers/iio/adc/max14001.c
@@ -366,6 +366,7 @@ static const struct spi_device_id max14001_id_table[] = {
{ .name = "max14002", .driver_data = (kernel_ulong_t)&max14002_chip_info },
{ }
};
+MODULE_DEVICE_TABLE(spi, max14001_id_table);
static const struct of_device_id max14001_of_match[] = {
{ .compatible = "adi,max14001", .data = &max14001_chip_info },
diff --git a/drivers/iio/adc/mcp3911.c b/drivers/iio/adc/mcp3911.c
index 797424f59f74..90167930f913 100644
--- a/drivers/iio/adc/mcp3911.c
+++ b/drivers/iio/adc/mcp3911.c
@@ -721,6 +721,8 @@ static int mcp3911_probe(struct spi_device *spi)
adc = iio_priv(indio_dev);
adc->spi = spi;
adc->chip = spi_get_device_match_data(spi);
+ if (!adc->chip)
+ return -ENODATA;
ret = devm_regulator_get_enable_read_voltage(dev, "vref");
if (ret < 0 && ret != -ENODEV)
diff --git a/drivers/iio/adc/stm32-adc.c b/drivers/iio/adc/stm32-adc.c
index 90f0e257e6d1..b024326775ae 100644
--- a/drivers/iio/adc/stm32-adc.c
+++ b/drivers/iio/adc/stm32-adc.c
@@ -2734,7 +2734,7 @@ static const struct stm32_adc_cfg stm32h7_adc_cfg = {
.set_ovs = stm32h7_adc_set_ovs,
};
-static const unsigned int stm32_adc_min_ts_mp1[] = { 100, 100, 100, 4300, 9800 };
+static const unsigned int stm32_adc_min_ts_mp1[] = { 100, 0, 0, 4300, 9800 };
static_assert(ARRAY_SIZE(stm32_adc_min_ts_mp1) == STM32_ADC_INT_CH_NB);
static const struct stm32_adc_cfg stm32mp1_adc_cfg = {
@@ -2756,7 +2756,7 @@ static const struct stm32_adc_cfg stm32mp1_adc_cfg = {
.set_ovs = stm32h7_adc_set_ovs,
};
-static const unsigned int stm32_adc_min_ts_mp13[] = { 100, 0, 0, 4300, 9800 };
+static const unsigned int stm32_adc_min_ts_mp13[] = { 1000, 1000, 1000, 4300, 9800 };
static_assert(ARRAY_SIZE(stm32_adc_min_ts_mp13) == STM32_ADC_INT_CH_NB);
static const struct stm32_adc_cfg stm32mp13_adc_cfg = {
diff --git a/drivers/iio/adc/ti-adc128s052.c b/drivers/iio/adc/ti-adc128s052.c
index 1899813c1aee..f9a995ff403e 100644
--- a/drivers/iio/adc/ti-adc128s052.c
+++ b/drivers/iio/adc/ti-adc128s052.c
@@ -195,12 +195,14 @@ static int adc128_probe(struct spi_device *spi)
adc = iio_priv(indio_dev);
adc->spi = spi;
- indio_dev->name = spi_get_device_id(spi)->name;
+ config = spi_get_device_match_data(spi);
+ if (!config)
+ return -ENODATA;
+
+ indio_dev->name = spi->modalias;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &adc128_info;
- config = spi_get_device_match_data(spi);
-
indio_dev->channels = config->channels;
indio_dev->num_channels = config->num_channels;
diff --git a/drivers/iio/adc/ti-ads1018.c b/drivers/iio/adc/ti-ads1018.c
index d9ec4a8d7ce5..8f2d03eee77b 100644
--- a/drivers/iio/adc/ti-ads1018.c
+++ b/drivers/iio/adc/ti-ads1018.c
@@ -624,12 +624,16 @@ static int ads1018_trigger_setup(struct iio_dev *indio_dev)
static int ads1018_spi_probe(struct spi_device *spi)
{
- const struct ads1018_chip_info *info = spi_get_device_match_data(spi);
+ const struct ads1018_chip_info *info;
struct device *dev = &spi->dev;
struct iio_dev *indio_dev;
struct ads1018 *ads1018;
int ret;
+ info = spi_get_device_match_data(spi);
+ if (!info)
+ return -ENODATA;
+
indio_dev = devm_iio_device_alloc(dev, sizeof(*ads1018));
if (!indio_dev)
return -ENOMEM;
diff --git a/drivers/iio/adc/ti-ads1100.c b/drivers/iio/adc/ti-ads1100.c
index 9fe8d54cce83..03096d7e1091 100644
--- a/drivers/iio/adc/ti-ads1100.c
+++ b/drivers/iio/adc/ti-ads1100.c
@@ -15,10 +15,12 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/i2c.h>
+#include <linux/iopoll.h>
#include <linux/mutex.h>
#include <linux/property.h>
#include <linux/pm_runtime.h>
#include <linux/regulator/consumer.h>
+#include <linux/time.h>
#include <linux/units.h>
#include <linux/iio/iio.h>
@@ -43,6 +45,9 @@
static const int ads1100_data_rate[] = { 128, 32, 16, 8 };
static const int ads1100_data_rate_bits[] = { 12, 14, 15, 16 };
+/* Timeout based on the minimum sample rate of 8 SPS (7500ms) */
+#define ADS1100_MAX_DRDY_TIMEOUT_US (7500 * USEC_PER_MSEC)
+
struct ads1100_data {
struct i2c_client *client;
struct regulator *reg_vdd;
@@ -123,10 +128,86 @@ static int ads1100_get_adc_result(struct ads1100_data *data, int chan, int *val)
return 0;
}
+static int ads1100_conversion_busy(struct ads1100_data *data)
+{
+ u8 buffer[3];
+ int ret;
+
+ ret = i2c_master_recv(data->client, (char *)&buffer, sizeof(buffer));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C read fail: %d\n", ret);
+ return ret;
+ }
+
+ return FIELD_GET(ADS1100_CFG_ST_BSY, buffer[2]);
+}
+
+static int ads1100_wait_single_conversion(struct ads1100_data *data)
+{
+ int data_rate_index = FIELD_GET(ADS1100_DR_MASK, data->config);
+ int data_rate_Hz = ads1100_data_rate[data_rate_index];
+ unsigned long poll_us = DIV_ROUND_CLOSEST(USEC_PER_SEC, data_rate_Hz) / 4;
+ int busy;
+ int ret;
+
+ ret = readx_poll_timeout(ads1100_conversion_busy, data,
+ busy, busy <= 0,
+ poll_us, ADS1100_MAX_DRDY_TIMEOUT_US);
+ if (busy < 0)
+ return busy;
+
+ return ret;
+}
+
+static int ads1100_start_single_conversion(struct ads1100_data *data)
+{
+ u8 config = data->config | ADS1100_CFG_SC;
+ int ret;
+
+ ret = i2c_master_send(data->client, &config, sizeof(config));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C write fail: %d\n", ret);
+ return ret;
+ }
+ /* Need to wait because of change from continuous to single mode */
+ ret = ads1100_wait_single_conversion(data);
+ if (ret)
+ return ret;
+
+ config |= ADS1100_CFG_ST_BSY;
+
+ ret = i2c_master_send(data->client, &config, sizeof(config));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "I2C write fail: %d\n", ret);
+ return ret;
+ }
+
+ /* No need to cache it, it's status bit */
+ data->config = config & ~ADS1100_CFG_ST_BSY;
+
+ return 0;
+}
+
+static int ads1100_poll_data_ready(struct ads1100_data *data)
+{
+ int ret;
+
+ ret = ads1100_start_single_conversion(data);
+ if (ret)
+ return ret;
+
+ ret = ads1100_wait_single_conversion(data);
+ if (ret)
+ return ret;
+
+ return ads1100_set_config_bits(data, ADS1100_CFG_SC, ADS1100_CONTINUOUS);
+}
+
static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
{
int microvolts;
int gain;
+ int ret;
/* With Vdd between 2.7 and 5V, the scale is always below 1 */
if (val)
@@ -135,6 +216,11 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
if (!val2)
return -EINVAL;
+ PM_RUNTIME_ACQUIRE_IF_ENABLED_AUTOSUSPEND(&data->client->dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
+ return ret;
+
microvolts = regulator_get_voltage(data->reg_vdd);
/*
* val2 is in 'micro' units, n = val2 / 1000000
@@ -149,22 +235,35 @@ static int ads1100_set_scale(struct ads1100_data *data, int val, int val2)
ads1100_set_config_bits(data, ADS1100_PGA_MASK, ffs(gain) - 1);
- return 0;
+ return ads1100_poll_data_ready(data);
}
static int ads1100_set_data_rate(struct ads1100_data *data, int chan, int rate)
{
unsigned int i;
unsigned int size;
+ int ret;
size = data->supports_data_rate ? ARRAY_SIZE(ads1100_data_rate) : 1;
for (i = 0; i < size; i++) {
if (ads1100_data_rate[i] == rate)
- return ads1100_set_config_bits(data, ADS1100_DR_MASK,
- FIELD_PREP(ADS1100_DR_MASK, i));
+ break;
}
- return -EINVAL;
+ if (i == size)
+ return -EINVAL;
+
+ PM_RUNTIME_ACQUIRE_IF_ENABLED_AUTOSUSPEND(&data->client->dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
+ return ret;
+
+ ret = ads1100_set_config_bits(data, ADS1100_DR_MASK,
+ FIELD_PREP(ADS1100_DR_MASK, i));
+ if (ret)
+ return ret;
+
+ return ads1100_poll_data_ready(data);
}
static int ads1100_get_vdd_millivolts(struct ads1100_data *data)
diff --git a/drivers/iio/adc/ti-ads131m02.c b/drivers/iio/adc/ti-ads131m02.c
index 2f8f75c8216b..7ecfb27fd7ef 100644
--- a/drivers/iio/adc/ti-ads131m02.c
+++ b/drivers/iio/adc/ti-ads131m02.c
@@ -897,6 +897,8 @@ static int ads131m_probe(struct spi_device *spi)
indio_dev->info = &ads131m_info;
config = spi_get_device_match_data(spi);
+ if (!config)
+ return -ENODATA;
priv->config = config;
indio_dev->name = config->name;
diff --git a/drivers/iio/adc/ti-ads7950.c b/drivers/iio/adc/ti-ads7950.c
index d27c1f2885fd..0d5708e0b831 100644
--- a/drivers/iio/adc/ti-ads7950.c
+++ b/drivers/iio/adc/ti-ads7950.c
@@ -527,8 +527,10 @@ static int ti_ads7950_probe(struct spi_device *spi)
st->spi = spi;
info = spi_get_device_match_data(spi);
+ if (!info)
+ return -ENODATA;
- indio_dev->name = spi_get_device_id(spi)->name;
+ indio_dev->name = spi->modalias;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->channels = info->channels;
indio_dev->num_channels = info->num_channels;
diff --git a/drivers/iio/amplifiers/ad8366.c b/drivers/iio/amplifiers/ad8366.c
index e12f8604f59f..9adff120de1a 100644
--- a/drivers/iio/amplifiers/ad8366.c
+++ b/drivers/iio/amplifiers/ad8366.c
@@ -326,6 +326,8 @@ static int ad8366_probe(struct spi_device *spi)
st->spi = spi;
st->info = spi_get_device_match_data(spi);
+ if (!st->info)
+ return -ENODATA;
enable_gpio = devm_gpiod_get_optional(dev, "enable", GPIOD_OUT_HIGH);
if (IS_ERR(enable_gpio))
diff --git a/drivers/iio/dac/ad5755.c b/drivers/iio/dac/ad5755.c
index 0828b9909099..730289dd8ef6 100644
--- a/drivers/iio/dac/ad5755.c
+++ b/drivers/iio/dac/ad5755.c
@@ -819,6 +819,9 @@ static int ad5755_probe(struct spi_device *spi)
spi_set_drvdata(spi, indio_dev);
st->chip_info = spi_get_device_match_data(spi);
+ if (!st->chip_info)
+ return -ENODATA;
+
st->spi = spi;
st->pwr_down = 0xf;
diff --git a/drivers/iio/dac/mcp47feb02.c b/drivers/iio/dac/mcp47feb02.c
index 8640b0ef4433..baeb1be455c6 100644
--- a/drivers/iio/dac/mcp47feb02.c
+++ b/drivers/iio/dac/mcp47feb02.c
@@ -36,7 +36,6 @@
#define MCP47FEB02_DAC_CTRL_MASK GENMASK(1, 0)
#define MCP47FEB02_GAIN_CTRL_STATUS_REG_ADDR 0x50
-#define MCP47FEB02_GAIN_BIT_MASK BIT(0)
#define MCP47FEB02_GAIN_BIT_STATUS_EEWA_MASK BIT(6)
#define MCP47FEB02_GAIN_BITS_MASK GENMASK(15, 8)
@@ -448,7 +447,7 @@ static int mcp47feb02_write_to_eeprom(struct mcp47feb02_data *data, unsigned int
ret = regmap_read_poll_timeout(data->regmap, MCP47FEB02_GAIN_CTRL_STATUS_REG_ADDR,
eewa_val,
!(eewa_val & MCP47FEB02_GAIN_BIT_STATUS_EEWA_MASK),
- USEC_PER_MSEC, USEC_PER_MSEC * 5);
+ 1 * USEC_PER_MSEC, 150 * USEC_PER_MSEC);
if (ret)
return ret;
@@ -468,7 +467,7 @@ static ssize_t store_eeprom_store(struct device *dev, struct device_attribute *a
return ret;
if (!state)
- return 0;
+ return len;
/*
* Verify DAC Wiper and DAC Configuration are unlocked. If both are disabled,
@@ -508,7 +507,7 @@ static ssize_t store_eeprom_store(struct device *dev, struct device_attribute *a
ret = regmap_read_poll_timeout(data->regmap, MCP47FEB02_GAIN_CTRL_STATUS_REG_ADDR, eewa_val,
!(eewa_val & MCP47FEB02_GAIN_BIT_STATUS_EEWA_MASK),
- USEC_PER_MSEC, USEC_PER_MSEC * 5);
+ 1 * USEC_PER_MSEC, 150 * USEC_PER_MSEC);
if (ret)
return ret;
@@ -922,7 +921,10 @@ static int mcp47feb02_read_label(struct iio_dev *indio_dev, struct iio_chan_spec
{
struct mcp47feb02_data *data = iio_priv(indio_dev);
- return sysfs_emit(label, "%s\n", data->labels[ch->address]);
+ if (data->labels[ch->address])
+ return sysfs_emit(label, "%s\n", data->labels[ch->address]);
+
+ return -ENOENT;
}
static const struct iio_info mcp47feb02_info = {
@@ -975,10 +977,11 @@ static int mcp47feb02_parse_fw(struct iio_dev *indio_dev,
set_bit(reg, &data->active_channels_mask);
- ret = fwnode_property_read_string(child, "label", &data->labels[reg]);
- if (ret)
- return dev_err_probe(dev, ret, "%pfw: invalid label\n",
- fwnode_get_name(child));
+ if (fwnode_property_present(child, "label")) {
+ ret = fwnode_property_read_string(child, "label", &data->labels[reg]);
+ if (ret)
+ dev_warn_probe(dev, ret, "%pfw: invalid label\n", child);
+ }
chanspec.address = reg;
chanspec.channel = reg;
@@ -1016,13 +1019,17 @@ static int mcp47feb02_init_ctrl_regs(struct mcp47feb02_data *data)
if (ret)
return ret;
- gain_ch = gain_ch & MCP47FEB02_GAIN_BITS_MASK;
for_each_set_bit(i, &data->active_channels_mask, data->phys_channels) {
struct device *dev = regmap_get_device(data->regmap);
- unsigned int pd_tmp;
+ unsigned int pd_tmp, dac_val;
+
+ ret = regmap_read(data->regmap, REG_ADDR(i), &dac_val);
+ if (ret)
+ return ret;
+ data->chdata[i].dac_data = dac_val;
data->chdata[i].ref_mode = (vref_ch >> (2 * i)) & MCP47FEB02_DAC_CTRL_MASK;
- data->chdata[i].use_2x_gain = (gain_ch >> i) & MCP47FEB02_GAIN_BIT_MASK;
+ data->chdata[i].use_2x_gain = field_get(DAC_GAIN_MASK(i), gain_ch);
/*
* Inform the user that the current voltage reference read from the volatile
diff --git a/drivers/iio/dac/mcp4821.c b/drivers/iio/dac/mcp4821.c
index 8d8c00cd60e9..59defb1b5e62 100644
--- a/drivers/iio/dac/mcp4821.c
+++ b/drivers/iio/dac/mcp4821.c
@@ -255,9 +255,13 @@ static const struct iio_info mcp4821_info = {
static int mcp4821_probe(struct spi_device *spi)
{
+ const struct mcp4821_chip_info *info;
struct iio_dev *indio_dev;
struct mcp4821_state *state;
- const struct mcp4821_chip_info *info;
+
+ info = spi_get_device_match_data(spi);
+ if (!info)
+ return -ENODATA;
indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*state));
if (indio_dev == NULL)
@@ -268,7 +272,6 @@ static int mcp4821_probe(struct spi_device *spi)
/* default gain is 2x */
state->gain = 2;
- info = spi_get_device_match_data(spi);
indio_dev->name = info->name;
indio_dev->info = &mcp4821_info;
indio_dev->modes = INDIO_DIRECT_MODE;
diff --git a/drivers/iio/frequency/adf4377.c b/drivers/iio/frequency/adf4377.c
index 7385be62fa6a..52266f83877a 100644
--- a/drivers/iio/frequency/adf4377.c
+++ b/drivers/iio/frequency/adf4377.c
@@ -1060,6 +1060,9 @@ static int adf4377_probe(struct spi_device *spi)
st->regmap = regmap;
st->spi = spi;
st->chip_info = spi_get_device_match_data(spi);
+ if (!st->chip_info)
+ return -ENODATA;
+
mutex_init(&st->lock);
ret = adf4377_properties_parse(st);
diff --git a/drivers/iio/imu/adis16400.c b/drivers/iio/imu/adis16400.c
index 8f664d055e9d..512a8da63cdc 100644
--- a/drivers/iio/imu/adis16400.c
+++ b/drivers/iio/imu/adis16400.c
@@ -1150,6 +1150,9 @@ static int adis16400_probe(struct spi_device *spi)
/* setup the industrialio driver allocated elements */
st->variant = spi_get_device_match_data(spi);
+ if (!st->variant)
+ return -ENODATA;
+
indio_dev->name = spi_get_device_id(spi)->name;
indio_dev->channels = st->variant->channels;
indio_dev->num_channels = st->variant->num_channels;
diff --git a/drivers/iio/imu/inv_icm42600/inv_icm42600.h b/drivers/iio/imu/inv_icm42600/inv_icm42600.h
index b55d993f0264..15fc8395da52 100644
--- a/drivers/iio/imu/inv_icm42600/inv_icm42600.h
+++ b/drivers/iio/imu/inv_icm42600/inv_icm42600.h
@@ -179,6 +179,7 @@ struct inv_icm42600_state {
struct {
s64 gyro;
s64 accel;
+ bool accel_force_odr;
} timestamp;
struct inv_icm42600_apex apex;
struct inv_icm42600_fifo fifo;
diff --git a/drivers/iio/imu/inv_icm42600/inv_icm42600_accel.c b/drivers/iio/imu/inv_icm42600/inv_icm42600_accel.c
index 4b0e3cd8a506..aea9bbc188da 100644
--- a/drivers/iio/imu/inv_icm42600/inv_icm42600_accel.c
+++ b/drivers/iio/imu/inv_icm42600/inv_icm42600_accel.c
@@ -1227,14 +1227,16 @@ int inv_icm42600_accel_parse_fifo(struct iio_dev *indio_dev)
if (size <= 0)
return size;
- /* skip packet if no accel data or data is invalid */
- if (accel == NULL || !inv_icm42600_fifo_is_data_valid(accel))
- continue;
-
/* update odr */
- if (odr & INV_ICM42600_SENSOR_ACCEL)
+ if (odr & INV_ICM42600_SENSOR_ACCEL) {
inv_sensors_timestamp_apply_odr(ts, st->fifo.period,
st->fifo.nb.total, no);
+ st->timestamp.accel_force_odr = false;
+ }
+
+ /* skip packet if no accel data or data is invalid */
+ if (accel == NULL || !inv_icm42600_fifo_is_data_valid(accel))
+ continue;
memcpy(&buffer.accel, accel, sizeof(buffer.accel));
/* convert 8 bits FIFO temperature in high resolution format */
@@ -1243,5 +1245,10 @@ int inv_icm42600_accel_parse_fifo(struct iio_dev *indio_dev)
iio_push_to_buffers_with_timestamp(indio_dev, &buffer, ts_val);
}
+ if (st->timestamp.accel_force_odr) {
+ inv_sensors_timestamp_apply_odr(ts, 0, 0, 0);
+ st->timestamp.accel_force_odr = false;
+ }
+
return 0;
}
diff --git a/drivers/iio/imu/inv_icm42600/inv_icm42600_buffer.c b/drivers/iio/imu/inv_icm42600/inv_icm42600_buffer.c
index 998d312f7bde..c871981d2dad 100644
--- a/drivers/iio/imu/inv_icm42600/inv_icm42600_buffer.c
+++ b/drivers/iio/imu/inv_icm42600/inv_icm42600_buffer.c
@@ -399,6 +399,8 @@ static int inv_icm42600_buffer_postdisable(struct iio_dev *indio_dev)
mutex_lock(&st->lock);
inv_sensors_timestamp_apply_odr(ts, 0, 0, 0);
+ if (sensor == INV_ICM42600_SENSOR_ACCEL)
+ st->timestamp.accel_force_odr = false;
ret = inv_icm42600_buffer_set_fifo_en(st, st->fifo.en & ~sensor);
if (ret)
diff --git a/drivers/iio/imu/inv_icm42600/inv_icm42600_core.c b/drivers/iio/imu/inv_icm42600/inv_icm42600_core.c
index dc97d8a274e3..e28f012e62b1 100644
--- a/drivers/iio/imu/inv_icm42600/inv_icm42600_core.c
+++ b/drivers/iio/imu/inv_icm42600/inv_icm42600_core.c
@@ -324,6 +324,15 @@ int inv_icm42600_set_accel_conf(struct inv_icm42600_state *st,
break;
}
+ /*
+ * If we change mode from low-power to low-noise because of odr,
+ * we need to force odr change.
+ */
+ if (oldconf->mode == INV_ICM42600_SENSOR_MODE_LOW_POWER &&
+ conf->mode == INV_ICM42600_SENSOR_MODE_LOW_NOISE &&
+ conf->odr != oldconf->odr)
+ st->timestamp.accel_force_odr = true;
+
/* set ACCEL_CONFIG0 register (accel fullscale & odr) */
if (conf->fs != oldconf->fs || conf->odr != oldconf->odr) {
val = INV_ICM42600_ACCEL_CONFIG0_FS(conf->fs) |
diff --git a/drivers/iio/imu/inv_icm42600/inv_icm42600_gyro.c b/drivers/iio/imu/inv_icm42600/inv_icm42600_gyro.c
index 253bf571439d..41e46276e6f2 100644
--- a/drivers/iio/imu/inv_icm42600/inv_icm42600_gyro.c
+++ b/drivers/iio/imu/inv_icm42600/inv_icm42600_gyro.c
@@ -808,15 +808,15 @@ int inv_icm42600_gyro_parse_fifo(struct iio_dev *indio_dev)
if (size <= 0)
return size;
- /* skip packet if no gyro data or data is invalid */
- if (gyro == NULL || !inv_icm42600_fifo_is_data_valid(gyro))
- continue;
-
/* update odr */
if (odr & INV_ICM42600_SENSOR_GYRO)
inv_sensors_timestamp_apply_odr(ts, st->fifo.period,
st->fifo.nb.total, no);
+ /* skip packet if no gyro data or data is invalid */
+ if (gyro == NULL || !inv_icm42600_fifo_is_data_valid(gyro))
+ continue;
+
memcpy(&buffer.gyro, gyro, sizeof(buffer.gyro));
/* convert 8 bits FIFO temperature in high resolution format */
buffer.temp = temp ? (*temp * 64) : 0;
diff --git a/drivers/iio/light/apds9306.c b/drivers/iio/light/apds9306.c
index f681b02e4922..7dd41136734f 100644
--- a/drivers/iio/light/apds9306.c
+++ b/drivers/iio/light/apds9306.c
@@ -61,7 +61,7 @@
#define APDS9306_NUM_REPEAT_RATES 7
#define APDS9306_INT_SRC_CLEAR 0
#define APDS9306_INT_SRC_ALS 1
-#define APDS9306_SAMP_FREQ_10HZ 0
+#define APDS9306_SAMP_FREQ_10HZ 2
/**
* struct part_id_gts_multiplier - Part no. and corresponding gts multiplier
diff --git a/drivers/iio/light/isl29028.c b/drivers/iio/light/isl29028.c
index 33deb1726689..e481ac908fc1 100644
--- a/drivers/iio/light/isl29028.c
+++ b/drivers/iio/light/isl29028.c
@@ -342,8 +342,9 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
struct device *dev = regmap_get_device(chip->regmap);
int ret;
- ret = pm_runtime_resume_and_get(dev);
- if (ret < 0)
+ PM_RUNTIME_ACQUIRE_AUTOSUSPEND(dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
return ret;
mutex_lock(&chip->lock);
@@ -392,14 +393,7 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
-
- ret = pm_runtime_put_autosuspend(dev);
- if (ret < 0)
- return ret;
-
- return 0;
+ return ret;
}
static int isl29028_read_raw(struct iio_dev *indio_dev,
@@ -408,10 +402,11 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
{
struct isl29028_chip *chip = iio_priv(indio_dev);
struct device *dev = regmap_get_device(chip->regmap);
- int ret, pm_ret;
+ int ret;
- ret = pm_runtime_resume_and_get(dev);
- if (ret < 0)
+ PM_RUNTIME_ACQUIRE_AUTOSUSPEND(dev, pm);
+ ret = PM_RUNTIME_ACQUIRE_ERR(&pm);
+ if (ret)
return ret;
mutex_lock(&chip->lock);
@@ -461,18 +456,6 @@ static int isl29028_read_raw(struct iio_dev *indio_dev,
mutex_unlock(&chip->lock);
- if (ret < 0)
- return ret;
-
- /**
- * Preserve the ret variable if the call to
- * pm_runtime_put_autosuspend() is successful so the reading
- * (if applicable) is returned to user space.
- */
- pm_ret = pm_runtime_put_autosuspend(dev);
- if (pm_ret < 0)
- return pm_ret;
-
return ret;
}
diff --git a/drivers/iio/light/ltr501.c b/drivers/iio/light/ltr501.c
index fa0d16890c8b..b36bd10a0262 100644
--- a/drivers/iio/light/ltr501.c
+++ b/drivers/iio/light/ltr501.c
@@ -1538,7 +1538,7 @@ static int ltr501_probe(struct i2c_client *client)
"ltr501_thresh_event",
indio_dev);
if (ret)
- return ret;
+ goto powerdown_on_error;
} else {
indio_dev->info = data->chip_info->info_no_irq;
}
diff --git a/drivers/iio/potentiometer/x9250.c b/drivers/iio/potentiometer/x9250.c
index 974e4447fc2c..675b78677baa 100644
--- a/drivers/iio/potentiometer/x9250.c
+++ b/drivers/iio/potentiometer/x9250.c
@@ -155,10 +155,15 @@ static const char *const x9250_regulator_names[] = {
static int x9250_probe(struct spi_device *spi)
{
+ const struct x9250_cfg *cfg;
struct iio_dev *indio_dev;
struct x9250 *x9250;
int ret;
+ cfg = spi_get_device_match_data(spi);
+ if (!cfg)
+ return -ENODATA;
+
ret = devm_regulator_bulk_get_enable(&spi->dev, ARRAY_SIZE(x9250_regulator_names),
x9250_regulator_names);
if (ret)
@@ -176,7 +181,7 @@ static int x9250_probe(struct spi_device *spi)
x9250 = iio_priv(indio_dev);
x9250->spi = spi;
- x9250->cfg = spi_get_device_match_data(spi);
+ x9250->cfg = cfg;
x9250->wp_gpio = devm_gpiod_get_optional(&spi->dev, "wp", GPIOD_OUT_LOW);
if (IS_ERR(x9250->wp_gpio))
return dev_err_probe(&spi->dev, PTR_ERR(x9250->wp_gpio),
diff --git a/drivers/iio/pressure/bmp280-spi.c b/drivers/iio/pressure/bmp280-spi.c
index a02a621b8015..95d9635f4711 100644
--- a/drivers/iio/pressure/bmp280-spi.c
+++ b/drivers/iio/pressure/bmp280-spi.c
@@ -83,6 +83,8 @@ static int bmp280_spi_probe(struct spi_device *spi)
struct regmap *regmap;
chip_info = spi_get_device_match_data(spi);
+ if (!chip_info)
+ return -ENODATA;
if (chip_info->spi_read_extra_byte)
bmp_regmap_bus = &bmp380_regmap_bus;
diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c
index 45bdb8c7670f..35260b399390 100644
--- a/drivers/iio/pressure/dps310.c
+++ b/drivers/iio/pressure/dps310.c
@@ -50,9 +50,9 @@
#define DPS310_CFG_REG 0x09
#define DPS310_INT_HL BIT(7)
#define DPS310_TMP_SHIFT_EN BIT(3)
-#define DPS310_PRS_SHIFT_EN BIT(4)
-#define DPS310_FIFO_EN BIT(5)
-#define DPS310_SPI_EN BIT(6)
+#define DPS310_PRS_SHIFT_EN BIT(2)
+#define DPS310_FIFO_EN BIT(1)
+#define DPS310_SPI_EN BIT(0)
#define DPS310_RESET 0x0c
#define DPS310_RESET_MAGIC 0x09
#define DPS310_COEF_BASE 0x10
diff --git a/drivers/infiniband/core/uverbs_std_types_dmabuf.c b/drivers/infiniband/core/uverbs_std_types_dmabuf.c
index 33bafadc695d..8d8c997a4883 100644
--- a/drivers/infiniband/core/uverbs_std_types_dmabuf.c
+++ b/drivers/infiniband/core/uverbs_std_types_dmabuf.c
@@ -68,9 +68,10 @@ static void uverbs_dmabuf_release(struct dma_buf *dmabuf)
struct ib_uverbs_dmabuf_file *priv = dmabuf->priv;
/*
- * This can only happen if the fput came from alloc_abort_fd_uobject()
+ * This can only happen if the fput came from the error unwind of
+ * UVERBS_METHOD_DMABUF_ALLOC. The abort cleans up the uobject.
*/
- if (!priv->uobj.context)
+ if (!priv)
return;
/*
@@ -161,6 +162,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_DMABUF_ALLOC)(
return 0;
err_revoked:
+ /* Tell uverbs_dmabuf_release() that the uobject is being aborted */
+ uverbs_dmabuf->dmabuf->priv = NULL;
dma_buf_put(uverbs_dmabuf->dmabuf);
err:
rdma_user_mmap_entry_put(mmap_entry);
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index 592e330e74bf..faa724ca33bd 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -5567,7 +5567,7 @@ static int init_rcverr(struct hfi1_devdata *dd)
static void free_rcverr(struct hfi1_devdata *dd)
{
if (dd->rcverr_timer.function)
- timer_delete_sync(&dd->rcverr_timer);
+ timer_shutdown_sync(&dd->rcverr_timer);
}
static void handle_rxe_err(struct hfi1_devdata *dd, u32 unused, u64 reg)
@@ -12300,7 +12300,7 @@ static void free_cntrs(struct hfi1_devdata *dd)
int i;
if (dd->synth_stats_timer.function)
- timer_delete_sync(&dd->synth_stats_timer);
+ timer_shutdown_sync(&dd->synth_stats_timer);
cancel_work_sync(&dd->update_cntr_work);
ppd = (struct hfi1_pportdata *)(dd + 1);
for (i = 0; i < dd->num_pports; i++, ppd++) {
diff --git a/drivers/infiniband/hw/hfi1/fault.c b/drivers/infiniband/hw/hfi1/fault.c
index 4ab72ef03ba1..941a0b96590b 100644
--- a/drivers/infiniband/hw/hfi1/fault.c
+++ b/drivers/infiniband/hw/hfi1/fault.c
@@ -216,7 +216,6 @@ int hfi1_fault_init_debugfs(struct hfi1_ibdev *ibd)
ibd->fault->attr.interval = 1;
ibd->fault->attr.require_end = ULONG_MAX;
ibd->fault->attr.stacktrace_depth = 32;
- ibd->fault->attr.dname = NULL;
ibd->fault->attr.verbose = 0;
ibd->fault->enable = false;
ibd->fault->opcode = false;
diff --git a/drivers/input/input.c b/drivers/input/input.c
index 01c91fec9b7e..6c45a158047f 100644
--- a/drivers/input/input.c
+++ b/drivers/input/input.c
@@ -391,7 +391,7 @@ void input_handle_event(struct input_dev *dev,
void input_event(struct input_dev *dev,
unsigned int type, unsigned int code, int value)
{
- if (is_event_supported(type, dev->evbit, EV_MAX)) {
+ if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) {
guard(spinlock_irqsave)(&dev->event_lock);
input_handle_event(dev, type, code, value);
}
@@ -415,7 +415,7 @@ void input_inject_event(struct input_handle *handle,
struct input_dev *dev = handle->dev;
struct input_handle *grab;
- if (is_event_supported(type, dev->evbit, EV_MAX)) {
+ if (type == EV_SYN || is_event_supported(type, dev->evbit, EV_MAX)) {
guard(spinlock_irqsave)(&dev->event_lock);
guard(rcu)();
diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c
index 810e52782ae8..d2eb48d98fea 100644
--- a/drivers/input/joystick/xpad.c
+++ b/drivers/input/joystick/xpad.c
@@ -408,6 +408,7 @@ static const struct xpad_device {
{ 0x3537, 0x1004, "GameSir T4 Kaleid", 0, XTYPE_XBOX360 },
{ 0x3537, 0x100f, "GameSir Nova 2 Lite", 0, XTYPE_XBOX360 },
{ 0x3537, 0x1010, "GameSir G7 SE", 0, XTYPE_XBOXONE },
+ { 0x3537, 0x1040, "GameSir Nova Lite", 0, XTYPE_XBOX360 },
{ 0x3651, 0x1000, "CRKD SG", 0, XTYPE_XBOX360 },
{ 0x366c, 0x0005, "ByoWave Proteus Controller", MAP_SHARE_BUTTON, XTYPE_XBOXONE, FLAG_DELAY_INIT },
{ 0x3767, 0x0101, "Fanatec Speedster 3 Forceshock Wheel", 0, XTYPE_XBOX },
diff --git a/drivers/input/keyboard/matrix_keypad.c b/drivers/input/keyboard/matrix_keypad.c
index 8863b741d1a3..cda2888b2e56 100644
--- a/drivers/input/keyboard/matrix_keypad.c
+++ b/drivers/input/keyboard/matrix_keypad.c
@@ -134,9 +134,6 @@ static void matrix_keypad_scan(struct work_struct *work)
memset(new_state, 0, sizeof(new_state));
- for (row = 0; row < keypad->num_row_gpios; row++)
- gpiod_direction_input(keypad->row_gpios[row]);
-
/* assert each column and read the row status out */
for (col = 0; col < keypad->num_col_gpios; col++) {
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index edcc187b8f14..0506617b27c2 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -390,7 +390,13 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
BUG_ON(iommu->mmio_base == NULL);
- if (is_kdump_kernel())
+ /*
+ * A kdump kernel that reuses the previous kernel's device table must
+ * leave the base register alone. It already points at that table, and
+ * with SNP enabled the register is locked. If reuse failed, program
+ * the freshly allocated table like a normal boot.
+ */
+ if (iommu->pci_seg->old_dev_tbl_cpy)
return;
entry = iommu_virt_to_phys(dev_table);
@@ -623,8 +629,19 @@ static int __init find_last_devid_acpi(struct acpi_table_header *table, u16 pci_
/* Allocate per PCI segment device table */
static inline int __init alloc_dev_table(struct amd_iommu_pci_seg *pci_seg)
{
- pci_seg->dev_table = iommu_alloc_pages_sz(GFP_KERNEL | GFP_DMA32,
- pci_seg->dev_table_size);
+ gfp_t gfp = GFP_KERNEL;
+
+ /*
+ * Keep normal kernel device tables below 4G because a later kdump
+ * kernel only trusts and reuses old tables below that limit. A kdump
+ * kernel's new table is either discarded when reuse succeeds or
+ * programmed after the IOMMU is disabled, so the table does not
+ * need DMA32 memory (which the crashkernel may not have).
+ */
+ if (!is_kdump_kernel())
+ gfp |= GFP_DMA32;
+
+ pci_seg->dev_table = iommu_alloc_pages_sz(gfp, pci_seg->dev_table_size);
if (!pci_seg->dev_table)
return -ENOMEM;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
index 25982bdbcbd9..747b9d47ff7e 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-iommufd.c
@@ -322,6 +322,106 @@ struct arm_vsmmu_invalidation_cmd {
};
};
+/* Reject the range field values that the spec defines as Reserved */
+static int arm_vsmmu_validate_range(struct arm_smmu_device *smmu, u64 data[2])
+{
+ bool range = !!(data[0] & (CMDQ_TLBI_0_NUM | CMDQ_TLBI_0_SCALE));
+ u8 ttl = FIELD_GET(CMDQ_TLBI_1_TTL, data[1]);
+ u8 tg = FIELD_GET(CMDQ_TLBI_1_TG, data[1]);
+
+ /* NUM, SCALE and TTL are RES0 when TG == 0 */
+ if (!tg)
+ return (range || ttl) ? -EIO : 0;
+ /* TTL == 0b01 with a 16KB TG requires SMMU_IDR5.DS */
+ if (tg == 2 && ttl == 1 && !(smmu->features & ARM_SMMU_FEAT_DS))
+ return -EIO;
+ /* NUM == 0, SCALE == 0 with TTL == 0 is a reserved combination */
+ if (!range && !ttl)
+ return -EIO;
+ return 0;
+}
+
+static int arm_vsmmu_validate_user_cmd(struct arm_vsmmu *vsmmu, u64 data[2])
+{
+ struct arm_smmu_device *smmu = vsmmu->smmu;
+ u64 allowed[2] = { CMDQ_0_OP };
+
+ /* Collect the fields userspace is allowed to set for each opcode */
+ switch (data[0] & CMDQ_0_OP) {
+ case CMDQ_OP_TLBI_NSNH_ALL:
+ break;
+ case CMDQ_OP_TLBI_NH_VA:
+ /* An SMMU with 8-bit ASIDs treats the upper 8 bits as RES0 */
+ allowed[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID,
+ GENMASK(smmu->asid_bits - 1, 0));
+ fallthrough;
+ case CMDQ_OP_TLBI_NH_VAA:
+ /* NUM/SCALE/TG/TTL are range fields gated on FEAT_RANGE_INV */
+ if (smmu->features & ARM_SMMU_FEAT_RANGE_INV) {
+ if (arm_vsmmu_validate_range(smmu, data))
+ return -EIO;
+ allowed[0] |= CMDQ_TLBI_0_NUM | CMDQ_TLBI_0_SCALE;
+ allowed[1] |= CMDQ_TLBI_1_TG | CMDQ_TLBI_1_TTL;
+ /* SCALE bit 25 (values above 31) is RES0 without DS */
+ if (!(smmu->features & ARM_SMMU_FEAT_DS))
+ allowed[0] &= ~FIELD_PREP(CMDQ_TLBI_0_SCALE,
+ BIT(5));
+ }
+ allowed[0] |= CMDQ_TLBI_0_VMID;
+ allowed[1] |= CMDQ_TLBI_1_LEAF | CMDQ_TLBI_1_VA_MASK;
+ break;
+ case CMDQ_OP_TLBI_NH_ASID:
+ /* An SMMU with 8-bit ASIDs treats the upper 8 bits as RES0 */
+ allowed[0] |= FIELD_PREP(CMDQ_TLBI_0_ASID,
+ GENMASK(smmu->asid_bits - 1, 0));
+ fallthrough;
+ case CMDQ_OP_TLBI_NH_ALL:
+ allowed[0] |= CMDQ_TLBI_0_VMID;
+ break;
+ case CMDQ_OP_ATC_INV:
+ /* ATC_INV is illegal unless the SMMU implements ATS */
+ if (!(smmu->features & ARM_SMMU_FEAT_ATS))
+ return -EIO;
+ /* A Size above 52 (invalidate-all) may raise a CERROR_ILL */
+ if (FIELD_GET(CMDQ_ATC_1_SIZE, data[1]) > ATC_INV_SIZE_ALL)
+ return -EIO;
+ /*
+ * SSV/SSID/Global need substream support. SSID and Global are
+ * IGNORED (not RES0) when SSV == 0, so they need no SSV check.
+ */
+ if (smmu->ssid_bits)
+ allowed[0] |= CMDQ_0_SSV | CMDQ_ATC_0_SSID |
+ CMDQ_ATC_0_GLOBAL;
+ allowed[0] |= CMDQ_ATC_0_SID;
+ allowed[1] |= CMDQ_ATC_1_SIZE | CMDQ_ATC_1_ADDR_MASK;
+ break;
+ case CMDQ_OP_CFGI_CD:
+ /* No SSV for CFGI_CD; SSID requires substream support */
+ if (smmu->ssid_bits)
+ allowed[0] |= CMDQ_CFGI_0_SSID;
+ allowed[1] |= CMDQ_CFGI_1_LEAF;
+ fallthrough;
+ case CMDQ_OP_CFGI_CD_ALL:
+ allowed[0] |= CMDQ_CFGI_0_SID;
+ break;
+ default:
+ return -EIO;
+ }
+
+ /*
+ * Reject any other bit, e.g. a RES0 bit or a Secure bit, before the
+ * command reaches the trusted main cmdq, so a guest cannot wedge the
+ * shared queue for every device with a CERROR_ILL.
+ *
+ * By contrast, an out-of-range address or ID value does not need a
+ * check: the spec defines it as CONSTRAINED UNPREDICTABLE, which is
+ * scoped to the guest itself and does not raise a CERROR_ILL.
+ */
+ if ((data[0] & ~allowed[0]) || (data[1] & ~allowed[1]))
+ return -EIO;
+ return 0;
+}
+
/*
* Convert, in place, the raw invalidation command into an internal format that
* can be passed to arm_smmu_cmdq_issue_cmdlist(). Internally commands are
@@ -332,33 +432,40 @@ struct arm_vsmmu_invalidation_cmd {
static int arm_vsmmu_convert_user_cmd(struct arm_vsmmu *vsmmu,
struct arm_vsmmu_invalidation_cmd *cmd)
{
+ u64 *data = cmd->cmd.data;
+ int ret;
+
/* Commands are le64 stored in u64 */
- cmd->cmd.data[0] = le64_to_cpu(cmd->ucmd.cmd[0]);
- cmd->cmd.data[1] = le64_to_cpu(cmd->ucmd.cmd[1]);
+ data[0] = le64_to_cpu(cmd->ucmd.cmd[0]);
+ data[1] = le64_to_cpu(cmd->ucmd.cmd[1]);
+
+ ret = arm_vsmmu_validate_user_cmd(vsmmu, data);
+ if (ret)
+ return ret;
- switch (cmd->cmd.data[0] & CMDQ_0_OP) {
+ switch (data[0] & CMDQ_0_OP) {
case CMDQ_OP_TLBI_NSNH_ALL:
/* Convert to NH_ALL */
- cmd->cmd.data[0] = CMDQ_OP_TLBI_NH_ALL |
- FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid);
- cmd->cmd.data[1] = 0;
+ data[0] = CMDQ_OP_TLBI_NH_ALL |
+ FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid);
+ data[1] = 0;
break;
case CMDQ_OP_TLBI_NH_VA:
case CMDQ_OP_TLBI_NH_VAA:
case CMDQ_OP_TLBI_NH_ALL:
case CMDQ_OP_TLBI_NH_ASID:
- cmd->cmd.data[0] &= ~CMDQ_TLBI_0_VMID;
- cmd->cmd.data[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid);
+ data[0] &= ~CMDQ_TLBI_0_VMID;
+ data[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, vsmmu->vmid);
break;
case CMDQ_OP_ATC_INV:
case CMDQ_OP_CFGI_CD:
case CMDQ_OP_CFGI_CD_ALL: {
- u32 sid, vsid = FIELD_GET(CMDQ_CFGI_0_SID, cmd->cmd.data[0]);
+ u32 sid, vsid = FIELD_GET(CMDQ_CFGI_0_SID, data[0]);
if (arm_vsmmu_vsid_to_sid(vsmmu, vsid, &sid))
return -EIO;
- cmd->cmd.data[0] &= ~CMDQ_CFGI_0_SID;
- cmd->cmd.data[0] |= FIELD_PREP(CMDQ_CFGI_0_SID, sid);
+ data[0] &= ~CMDQ_CFGI_0_SID;
+ data[0] |= FIELD_PREP(CMDQ_CFGI_0_SID, sid);
break;
}
default:
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 68ee57800fef..795a0d6248ce 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -1477,6 +1477,11 @@ int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
sg_dma_len(s) = s_length;
s->offset -= s_iova_off;
s_length = iova_align(iovad, s_length + s_iova_off);
+ if (overflows_type(s_length, s->length)) {
+ ret = -EOVERFLOW;
+
+ goto out_restore_sg;
+ }
s->length = s_length;
/*
@@ -1493,7 +1498,20 @@ int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
* time through here (i.e. before it has a meaningful value).
*/
if (pad_len && pad_len < s_length - 1) {
- prev->length += pad_len;
+ unsigned int new_prev_len;
+
+ /*
+ * For large mappings spanning multiple GBs we
+ * may not be able to fit all needed padding into
+ * sg->length.
+ */
+ if (check_add_overflow(prev->length, pad_len, &new_prev_len)) {
+ ret = -EOVERFLOW;
+
+ goto out_restore_sg;
+ }
+
+ prev->length = new_prev_len;
iova_len += pad_len;
}
diff --git a/drivers/iommu/generic_pt/iommu_pt.h b/drivers/iommu/generic_pt/iommu_pt.h
index 07ec2b3ab986..837d835e4fad 100644
--- a/drivers/iommu/generic_pt/iommu_pt.h
+++ b/drivers/iommu/generic_pt/iommu_pt.h
@@ -240,27 +240,14 @@ struct pt_iommu_dirty_args {
};
static void record_dirty(struct pt_state *pts,
- struct pt_iommu_dirty_args *dirty,
- unsigned int num_contig_lg2)
+ struct pt_iommu_dirty_args *dirty)
{
- pt_vaddr_t dirty_len;
-
- if (num_contig_lg2 != ilog2(1)) {
- unsigned int index = pts->index;
- unsigned int end_index = log2_set_mod_max_t(
- unsigned int, pts->index, num_contig_lg2);
-
- /* Adjust for being contained inside a contiguous page */
- end_index = min(end_index, pts->end_index);
- dirty_len = (end_index - index) *
- log2_to_int(pt_table_item_lg2sz(pts));
- } else {
- dirty_len = log2_to_int(pt_table_item_lg2sz(pts));
- }
+ unsigned int lg2sz = pt_entry_oa_lg2sz(pts);
+ pt_vaddr_t start_va = log2_set_mod(pts->range->va, 0, lg2sz);
+ pt_vaddr_t dirty_len = log2_to_int(lg2sz);
if (dirty->dirty->bitmap)
- iova_bitmap_set(dirty->dirty->bitmap, pts->range->va,
- dirty_len);
+ iova_bitmap_set(dirty->dirty->bitmap, start_va, dirty_len);
if (!(dirty->flags & IOMMU_DIRTY_NO_CLEAR)) {
/*
@@ -268,8 +255,8 @@ static void record_dirty(struct pt_state *pts,
* dirty tracking bits can't work together
*/
pt_entry_make_write_clean(pts);
- iommu_iotlb_gather_add_range(dirty->dirty->gather,
- pts->range->va, dirty_len);
+ iommu_iotlb_gather_add_range(dirty->dirty->gather, start_va,
+ dirty_len);
}
}
@@ -289,8 +276,7 @@ static inline int __read_and_clear_dirty(struct pt_range *range, void *arg,
continue;
}
if (pts.type == PT_ENTRY_OA && pt_entry_is_write_dirty(&pts))
- record_dirty(&pts, dirty,
- pt_entry_num_contig_lg2(&pts));
+ record_dirty(&pts, dirty);
}
return 0;
}
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 23dbe6c24439..5b3d234ae265 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1105,12 +1105,20 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
unsigned int size_order,
struct qi_desc *desc)
{
- unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
-
desc->qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
QI_DEV_IOTLB_PFSID(pfsid);
+ /*
+ * The widest range the descriptor can express is a full flush, encoded
+ * by making bit 63 the least significant zero bit of ADDR, that is
+ * @size_order == 64 - VTD_PAGE_SHIFT. Bound @size_order there so that
+ * a caller passing something larger cannot produce an out-of-range
+ * shift below.
+ */
+ if (size_order > 64 - VTD_PAGE_SHIFT)
+ size_order = 64 - VTD_PAGE_SHIFT;
+
/*
* If S bit is 0, we only flush a single page. If S bit is set,
* The least significant zero bit indicates the invalidation address
@@ -1120,7 +1128,8 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
* Max Invs Pending (MIP) is set to 0 for now until we have DIT in
* ECAP.
*/
- if (!IS_ALIGNED(addr, VTD_PAGE_SIZE << size_order))
+ if (size_order < 64 - VTD_PAGE_SHIFT &&
+ !IS_ALIGNED(addr, BIT_ULL(VTD_PAGE_SHIFT + size_order)))
pr_warn_ratelimited("Invalidate non-aligned address %llx, order %d\n",
addr, size_order);
@@ -1136,7 +1145,7 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
desc->qw1 |= GENMASK_ULL(size_order + VTD_PAGE_SHIFT - 1,
VTD_PAGE_SHIFT);
/* Clear size_order bit to indicate size */
- desc->qw1 &= ~mask;
+ desc->qw1 &= ~BIT_ULL(VTD_PAGE_SHIFT + size_order - 1);
/* Set the S bit to indicate flushing more than 1 page */
desc->qw1 |= QI_DEV_EIOTLB_SIZE;
}
diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 4e447ce74cf6..283ab372e8da 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -605,6 +605,9 @@ int iommufd_check_iova_range(struct io_pagetable *iopt,
size_t iommu_pgsize = iopt->iova_alignment;
u64 last_iova;
+ if (!bitmap->length)
+ return -EINVAL;
+
if (check_add_overflow(bitmap->iova, bitmap->length - 1, &last_iova))
return -EOVERFLOW;
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 404f31d8f729..2f6153108add 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1178,6 +1178,15 @@ static int pfn_reader_fill_span(struct pfn_reader *pfns)
WARN_ON(span->last_used < start_index))
return -EINVAL;
+ /*
+ * Always read a dmabuf from its phys, even where a domain already
+ * maps it: a domain can only report CPU memory PFNs, losing
+ * BATCH_MMIO. last_hole aliases last_used, so this covers any span.
+ */
+ if (iopt_is_dmabuf(pfns->pages))
+ return pfn_reader_fill_dmabuf(&pfns->dmabuf, &pfns->batch,
+ start_index, span->last_hole);
+
if (span->is_used == 1) {
batch_from_xarray(&pfns->batch, &pfns->pages->pinned_pfns,
start_index, span->last_used);
@@ -1201,10 +1210,6 @@ static int pfn_reader_fill_span(struct pfn_reader *pfns)
return 0;
}
- if (iopt_is_dmabuf(pfns->pages))
- return pfn_reader_fill_dmabuf(&pfns->dmabuf, &pfns->batch,
- start_index, span->last_hole);
-
user = &pfns->user;
if (start_index >= user->upages_end) {
rc = pfn_reader_user_pin(user, pfns->pages, start_index,
diff --git a/drivers/irqchip/irq-atmel-aic.c b/drivers/irqchip/irq-atmel-aic.c
index 1dcc52760eca..4a7935e75e4d 100644
--- a/drivers/irqchip/irq-atmel-aic.c
+++ b/drivers/irqchip/irq-atmel-aic.c
@@ -123,7 +123,7 @@ static void aic_pm_shutdown(struct irq_data *d)
{
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
- guard(raw_spinlock)(&gc->lock);
+ guard(raw_spinlock_irqsave)(&gc->lock);
irq_reg_writel(gc, 0xffffffff, AT91_AIC_IDCR);
irq_reg_writel(gc, 0xffffffff, AT91_AIC_ICCR);
}
diff --git a/drivers/irqchip/irq-atmel-aic5.c b/drivers/irqchip/irq-atmel-aic5.c
index 1f14b401f71d..07a317ed06ef 100644
--- a/drivers/irqchip/irq-atmel-aic5.c
+++ b/drivers/irqchip/irq-atmel-aic5.c
@@ -217,7 +217,7 @@ static void aic5_pm_shutdown(struct irq_data *d)
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
int i;
- guard(raw_spinlock)(&bgc->lock);
+ guard(raw_spinlock_irqsave)(&bgc->lock);
for (i = 0; i < dgc->irqs_per_chip; i++) {
irq_reg_writel(bgc, i + gc->irq_base, AT91_AIC5_SSR);
irq_reg_writel(bgc, 1, AT91_AIC5_IDCR);
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index e9807af23537..81a96149f915 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -3484,7 +3484,7 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id,
* Also honor the ITS's own EID limit.
*/
id_bits = FIELD_GET(GITS_TYPER_IDBITS, its->typer) + 1;
- nvecs = min_t(unsigned int, nvecs, BIT(id_bits));
+ nvecs = min_t(u64, nvecs, BIT_ULL(id_bits));
nr_ites = max(2, nvecs);
sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1);
sz = max(sz, ITS_ITT_ALIGN);
diff --git a/drivers/irqchip/irq-imx-intmux.c b/drivers/irqchip/irq-imx-intmux.c
index 47c2681d138a..de2a8f98f625 100644
--- a/drivers/irqchip/irq-imx-intmux.c
+++ b/drivers/irqchip/irq-imx-intmux.c
@@ -288,12 +288,16 @@ static int imx_intmux_probe(struct platform_device *pdev)
static void imx_intmux_remove(struct platform_device *pdev)
{
struct intmux_data *data = platform_get_drvdata(pdev);
- int i;
+ int i, ret;
+
+ ret = pm_runtime_resume_and_get(&pdev->dev);
+ if (ret < 0)
+ dev_warn(&pdev->dev, "failed to resume device: %d\n", ret);
for (i = 0; i < data->channum; i++) {
/* disable all interrupt sources of this channel */
- writel_relaxed(0, data->regs + CHANIER(i));
-
+ if (ret >= 0)
+ writel_relaxed(0, data->regs + CHANIER(i));
irq_set_chained_handler_and_data(data->irqchip_data[i].irq,
NULL, NULL);
@@ -301,6 +305,11 @@ static void imx_intmux_remove(struct platform_device *pdev)
}
pm_runtime_disable(&pdev->dev);
+ if (ret >= 0) {
+ pm_runtime_put_noidle(&pdev->dev);
+ clk_disable_unprepare(data->ipg_clk);
+ pm_runtime_set_suspended(&pdev->dev);
+ }
}
#ifdef CONFIG_PM
diff --git a/drivers/irqchip/irq-jcore-aic.c b/drivers/irqchip/irq-jcore-aic.c
index 94c05cf974be..b4fee02997ee 100644
--- a/drivers/irqchip/irq-jcore-aic.c
+++ b/drivers/irqchip/irq-jcore-aic.c
@@ -37,8 +37,10 @@ static struct irq_chip jcore_aic;
static void handle_jcore_irq(struct irq_desc *desc)
{
- if (irqd_is_per_cpu(irq_desc_get_irq_data(desc)))
+ if (irq_is_percpu_devid(irq_desc_get_irq(desc)))
handle_percpu_devid_irq(desc);
+ else if (irqd_is_per_cpu(irq_desc_get_irq_data(desc)))
+ handle_percpu_irq(desc);
else
handle_simple_irq(desc);
}
diff --git a/drivers/irqchip/irq-mchp-eic.c b/drivers/irqchip/irq-mchp-eic.c
index be007d325e67..5250e6baff63 100644
--- a/drivers/irqchip/irq-mchp-eic.c
+++ b/drivers/irqchip/irq-mchp-eic.c
@@ -233,7 +233,7 @@ static int mchp_eic_probe(struct platform_device *pdev, struct device_node *pare
ret = clk_prepare_enable(eic->clk);
if (ret)
- goto unmap;
+ goto clk_put;
for (i = 0; i < MCHP_EIC_NIRQ; i++) {
struct of_phandle_args irq;
@@ -270,6 +270,8 @@ static int mchp_eic_probe(struct platform_device *pdev, struct device_node *pare
clk_unprepare:
clk_disable_unprepare(eic->clk);
+clk_put:
+ clk_put(eic->clk);
unmap:
iounmap(eic->base);
free:
diff --git a/drivers/irqchip/irq-msi-lib.c b/drivers/irqchip/irq-msi-lib.c
index 45e0ed3134ce..a1e12aa820e7 100644
--- a/drivers/irqchip/irq-msi-lib.c
+++ b/drivers/irqchip/irq-msi-lib.c
@@ -75,6 +75,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
fallthrough;
case DOMAIN_BUS_WIRED_TO_MSI:
+ info->flags |= MSI_FLAG_FREE_MSI_DESCS;
/* Remove PCI specific flags */
required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
break;
diff --git a/drivers/irqchip/irq-renesas-rzv2h.c b/drivers/irqchip/irq-renesas-rzv2h.c
index 1a4703a353e9..485c3eea7632 100644
--- a/drivers/irqchip/irq-renesas-rzv2h.c
+++ b/drivers/irqchip/irq-renesas-rzv2h.c
@@ -420,9 +420,9 @@ static void rzv2h_clear_tint_int(struct rzv2h_icu_priv *priv, unsigned int hwirq
static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
{
u32 titsr, titsr_k, titsel_n, tien;
+ u32 titsr_cur, tssr_cur, tien_cur;
struct rzv2h_icu_priv *priv;
u32 tssr, tssr_k, tssel_n;
- u32 titsr_cur, tssr_cur;
unsigned int hwirq;
u32 tint, sense;
int tint_nr;
@@ -477,6 +477,8 @@ static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
if (tssr_cur == tint && titsr_cur == sense)
goto set_parent_type;
+ tien_cur = tssr & tien;
+
tssr &= ~(ICU_TSSR_TSSEL_MASK(tssel_n, priv->info->field_width) | tien);
tssr |= ICU_TSSR_TSSEL_PREP(tint, tssel_n, priv->info->field_width);
@@ -489,7 +491,7 @@ static int rzv2h_tint_set_type(struct irq_data *d, unsigned int type)
rzv2h_clear_tint_int(priv, hwirq);
- writel_relaxed(tssr | tien, priv->base + priv->info->t_offs + ICU_TSSR(tssr_k));
+ writel_relaxed(tssr | tien_cur, priv->base + priv->info->t_offs + ICU_TSSR(tssr_k));
}
set_parent_type:
return irq_chip_set_type_parent(d, IRQ_TYPE_LEVEL_HIGH);
diff --git a/drivers/irqchip/irq-riscv-imsic-platform.c b/drivers/irqchip/irq-riscv-imsic-platform.c
index 643c8e459611..52673ff9edf4 100644
--- a/drivers/irqchip/irq-riscv-imsic-platform.c
+++ b/drivers/irqchip/irq-riscv-imsic-platform.c
@@ -61,7 +61,7 @@ static int imsic_irq_retrigger(struct irq_data *d)
local = per_cpu_ptr(imsic->global.local, vec->cpu);
writel_relaxed(vec->local_id, local->msi_va);
- return 0;
+ return 1;
}
static void imsic_irq_ack(struct irq_data *d)
diff --git a/drivers/leds/flash/leds-ktd2692.c b/drivers/leds/flash/leds-ktd2692.c
index 22fbfccd4873..43d4d55503ab 100644
--- a/drivers/leds/flash/leds-ktd2692.c
+++ b/drivers/leds/flash/leds-ktd2692.c
@@ -69,6 +69,8 @@ static const struct expresswire_timing ktd2692_timing = {
};
struct ktd2692_context {
+ struct device *dev;
+
/* Common ExpressWire properties (ctrl GPIO and timing) */
struct expresswire_common_props props;
@@ -196,13 +198,12 @@ static void ktd2692_setup(struct ktd2692_context *led)
static void regulator_disable_action(void *_data)
{
- struct device *dev = _data;
- struct ktd2692_context *led = dev_get_drvdata(dev);
+ struct ktd2692_context *led = _data;
int ret;
ret = regulator_disable(led->regulator);
if (ret)
- dev_err(dev, "Failed to disable supply: %d\n", ret);
+ dev_err(led->dev, "Failed to disable supply: %d\n", ret);
}
static int ktd2692_parse_dt(struct ktd2692_context *led, struct device *dev,
@@ -233,7 +234,7 @@ static int ktd2692_parse_dt(struct ktd2692_context *led, struct device *dev,
dev_err(dev, "Failed to enable supply: %d\n", ret);
} else {
ret = devm_add_action_or_reset(dev,
- regulator_disable_action, dev);
+ regulator_disable_action, led);
if (ret)
return ret;
}
@@ -290,6 +291,7 @@ static int ktd2692_probe(struct platform_device *pdev)
if (!led)
return -ENOMEM;
+ led->dev = &pdev->dev;
fled_cdev = &led->fled_cdev;
led_cdev = &fled_cdev->led_cdev;
led->props.timing = ktd2692_timing;
diff --git a/drivers/mailbox/bcm74110-mailbox.c b/drivers/mailbox/bcm74110-mailbox.c
index c8709d509912..bec138d76914 100644
--- a/drivers/mailbox/bcm74110-mailbox.c
+++ b/drivers/mailbox/bcm74110-mailbox.c
@@ -97,6 +97,8 @@ struct bcm74110_mbox_chan {
int type;
};
+#define BCM74110_INIT_MSG_MAX 16
+
struct bcm74110_mbox {
struct platform_device *pdev;
void __iomem *base;
@@ -105,6 +107,7 @@ struct bcm74110_mbox {
int rx_chan;
int rx_irq;
struct list_head rx_svc_init_list;
+ unsigned int rx_svc_init_count;
spinlock_t rx_svc_list_lock;
struct mbox_controller controller;
@@ -148,7 +151,15 @@ static void bcm74110_rx_push_init_msg(struct bcm74110_mbox *mbox, u32 val)
msg->msg = val;
spin_lock(&mbox->rx_svc_list_lock);
+ if (mbox->rx_svc_init_count >= BCM74110_INIT_MSG_MAX) {
+ spin_unlock(&mbox->rx_svc_list_lock);
+ kfree(msg);
+ dev_warn_ratelimited(&mbox->pdev->dev,
+ "PMC INIT msg list full, dropping message\n");
+ return;
+ }
list_add_tail(&msg->list_entry, &mbox->rx_svc_init_list);
+ mbox->rx_svc_init_count++;
spin_unlock(&mbox->rx_svc_list_lock);
}
@@ -225,6 +236,7 @@ static int bcm74110_rx_pop_init_msg(struct bcm74110_mbox *mbox, u32 func_type,
list_entry) {
if (BCM_MSG_GET_FIELD(msg->msg, FUNC) == func_type) {
list_del(&msg->list_entry);
+ mbox->rx_svc_init_count--;
found = true;
break;
}
@@ -248,6 +260,7 @@ static void bcm74110_rx_flush_msg(struct bcm74110_mbox *mbox)
spin_lock_irqsave(&mbox->rx_svc_list_lock, flags);
list_splice_init(&mbox->rx_svc_init_list, &list_temp);
+ mbox->rx_svc_init_count = 0;
spin_unlock_irqrestore(&mbox->rx_svc_list_lock, flags);
list_for_each_entry_safe(msg, msg_tmp, &list_temp, list_entry) {
diff --git a/drivers/mailbox/mailbox-altera.c b/drivers/mailbox/mailbox-altera.c
index 17278c2571d3..7cb91413ef7f 100644
--- a/drivers/mailbox/mailbox-altera.c
+++ b/drivers/mailbox/mailbox-altera.c
@@ -270,7 +270,7 @@ static void altera_mbox_shutdown(struct mbox_chan *chan)
writel_relaxed(~0, mbox->mbox_base + MAILBOX_INTMASK_REG);
free_irq(mbox->irq, chan);
} else if (!mbox->is_sender) {
- timer_delete_sync(&mbox->rxpoll_timer);
+ timer_shutdown_sync(&mbox->rxpoll_timer);
}
}
diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c b/drivers/mailbox/mtk-cmdq-mailbox.c
index e523c84b4808..96462ca84471 100644
--- a/drivers/mailbox/mtk-cmdq-mailbox.c
+++ b/drivers/mailbox/mtk-cmdq-mailbox.c
@@ -565,8 +565,10 @@ static int cmdq_mbox_flush(struct mbox_chan *chan, unsigned long timeout)
int ret;
ret = pm_runtime_get_sync(cmdq->mbox.dev);
- if (ret < 0)
+ if (ret < 0) {
+ pm_runtime_put_noidle(cmdq->mbox.dev);
return ret;
+ }
spin_lock_irqsave(&thread->chan->lock, flags);
if (list_empty(&thread->task_busy_list))
@@ -603,6 +605,9 @@ static int cmdq_mbox_flush(struct mbox_chan *chan, unsigned long timeout)
dev_err(cmdq->mbox.dev, "Fail to wait GCE thread 0x%x done\n",
(u32)(thread->base - cmdq->base));
+ pm_runtime_mark_last_busy(cmdq->mbox.dev);
+ pm_runtime_put_autosuspend(cmdq->mbox.dev);
+
return -EFAULT;
}
pm_runtime_mark_last_busy(cmdq->mbox.dev);
diff --git a/drivers/mailbox/qcom-ipcc.c b/drivers/mailbox/qcom-ipcc.c
index 185b63f724d4..387aa72aeedf 100644
--- a/drivers/mailbox/qcom-ipcc.c
+++ b/drivers/mailbox/qcom-ipcc.c
@@ -10,6 +10,7 @@
#include <linux/mailbox_controller.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/spinlock.h>
#include <dt-bindings/mailbox/qcom-ipcc.h>
@@ -151,6 +152,9 @@ static int qcom_ipcc_mbox_send_data(struct mbox_chan *chan, void *data)
struct qcom_ipcc_chan_info *mchan = chan->con_priv;
u32 hwirq;
+ if (!mchan)
+ return -EINVAL;
+
hwirq = qcom_ipcc_get_hwirq(mchan->client_id, mchan->signal_id);
writel(hwirq, ipcc->base + IPCC_REG_SEND_ID);
@@ -159,6 +163,8 @@ static int qcom_ipcc_mbox_send_data(struct mbox_chan *chan, void *data)
static void qcom_ipcc_mbox_shutdown(struct mbox_chan *chan)
{
+ guard(spinlock_irqsave)(&chan->lock);
+
chan->con_priv = NULL;
}
diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index d58eed96a8ff..5f0226270761 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -2426,8 +2426,10 @@ static void __scan(struct dm_bufio_client *c)
for (l = 0; l < LIST_SIZE; l++) {
while (true) {
- if (count - freed <= retain_target)
+ if (count - freed <= retain_target) {
atomic_long_set(&c->need_shrink, 0);
+ break;
+ }
if (!atomic_long_read(&c->need_shrink))
break;
@@ -2545,6 +2547,13 @@ struct dm_bufio_client *dm_bufio_client_create(struct block_device *bdev, unsign
init_waitqueue_head(&c->free_buffer_wait);
c->async_write_error = 0;
+ /*
+ * A client with no buffers yet is not holding the oldest buffer in
+ * the system; leaving this zeroed would claim exactly that and keep
+ * the client pinned at the front of dm_bufio_all_clients.
+ */
+ c->oldest_buffer = jiffies;
+
c->dm_io = dm_io_client_create();
if (IS_ERR(c->dm_io)) {
r = PTR_ERR(c->dm_io);
@@ -2779,9 +2788,23 @@ static unsigned long __evict_a_few(unsigned long nr_buffers)
dm_bufio_unlock(c);
- if (count)
+ if (count) {
c->oldest_buffer = oldest_buffer;
- __insert_client(c);
+ __insert_client(c);
+ } else {
+ /*
+ * We evicted nothing, so 'oldest_buffer' tells us nothing new
+ * and the client keeps its old, possibly very stale, value.
+ * Requeue it at the back rather than by that value, otherwise
+ * __insert_client() would put it straight back at the front
+ * and every subsequent call would pop this same client again,
+ * never reaching any of the others.
+ *
+ * This leaves the list slightly out of order, which is fine -
+ * it is only ever an approximation.
+ */
+ list_add_tail(&c->client_list, &dm_bufio_all_clients);
+ }
return count;
}
@@ -2806,11 +2829,23 @@ static void evict_old(void)
{
unsigned long threshold = dm_bufio_cache_size -
dm_bufio_cache_size / DM_BUFIO_LOW_WATERMARK_RATIO;
+ int unproductive = 0;
mutex_lock(&dm_bufio_clients_lock);
while (dm_bufio_current_allocated > threshold) {
- if (!__evict_a_few(64))
- break;
+ if (__evict_a_few(64)) {
+ unproductive = 0;
+ } else {
+ /*
+ * This client had nothing to give. Try the others
+ * before giving up: one idle client, or one whose
+ * buffers are all held, must not stop us evicting
+ * from everybody else. Stop once a whole sweep of
+ * the client list has freed nothing.
+ */
+ if (++unproductive >= dm_bufio_client_count)
+ break;
+ }
cond_resched();
}
mutex_unlock(&dm_bufio_clients_lock);
diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c
index e86d929b0e95..b587ee733056 100644
--- a/drivers/md/dm-snap-persistent.c
+++ b/drivers/md/dm-snap-persistent.c
@@ -325,8 +325,11 @@ static int read_header(struct pstore *ps, int *new_snapshot)
}
ps->io_client = dm_io_client_create();
- if (IS_ERR(ps->io_client))
- return PTR_ERR(ps->io_client);
+ if (IS_ERR(ps->io_client)) {
+ r = PTR_ERR(ps->io_client);
+ ps->io_client = NULL;
+ return r;
+ }
r = alloc_area(ps);
if (r)
diff --git a/drivers/md/dm-vdo/indexer/chapter-index.c b/drivers/md/dm-vdo/indexer/chapter-index.c
index bb3b0ab5d50d..9fb9e59238d4 100644
--- a/drivers/md/dm-vdo/indexer/chapter-index.c
+++ b/drivers/md/dm-vdo/indexer/chapter-index.c
@@ -284,10 +284,17 @@ int uds_search_chapter_index_page(struct delta_index_page *index_page,
if (result != UDS_SUCCESS)
return result;
- if (was_entry_found(&entry, address))
- *record_page_ptr = uds_get_delta_entry_value(&entry);
- else
+ if (!was_entry_found(&entry, address)) {
*record_page_ptr = NO_CHAPTER_INDEX_ENTRY;
+ return UDS_SUCCESS;
+ }
- return UDS_SUCCESS;
+ *record_page_ptr = uds_get_delta_entry_value(&entry);
+ result = VDO_ASSERT(*record_page_ptr < geometry->record_pages_per_chapter,
+ "0 <= %d < %u", *record_page_ptr,
+ geometry->record_pages_per_chapter);
+ if (result != VDO_SUCCESS)
+ *record_page_ptr = NO_CHAPTER_INDEX_ENTRY;
+
+ return result;
}
diff --git a/drivers/md/dm-vdo/indexer/delta-index.c b/drivers/md/dm-vdo/indexer/delta-index.c
index b288749067de..257612b81a9d 100644
--- a/drivers/md/dm-vdo/indexer/delta-index.c
+++ b/drivers/md/dm-vdo/indexer/delta-index.c
@@ -800,7 +800,7 @@ static void compute_new_list_offsets(struct delta_zone *delta_zone, u32 growing_
(delta_zone->size * BITS_PER_BYTE - delta_lists[tail_guard_index].size);
}
-static void rebalance_lists(struct delta_zone *delta_zone)
+static int rebalance_lists(struct delta_zone *delta_zone)
{
struct delta_list *delta_lists;
u32 i;
@@ -811,9 +811,17 @@ static void rebalance_lists(struct delta_zone *delta_zone)
for (i = 0; i <= delta_zone->list_count + 1; i++)
used_space += get_delta_list_byte_size(&delta_lists[i]);
+ if (delta_zone->size < used_space) {
+ return vdo_log_warning_strerror(UDS_CORRUPT_DATA,
+ "delta zone lists overflow zone size %zu",
+ delta_zone->size);
+ }
+
compute_new_list_offsets(delta_zone, 0, 0, used_space);
for (i = 1; i <= delta_zone->list_count + 1; i++)
delta_lists[i].start = delta_zone->new_offsets[i];
+
+ return UDS_SUCCESS;
}
/* Start restoring a delta index from multiple input streams. */
@@ -941,8 +949,11 @@ int uds_start_restoring_delta_index(struct delta_index *delta_index,
}
/* Prepare each zone to start receiving the delta list data. */
- for (z = 0; z < delta_index->zone_count; z++)
- rebalance_lists(&delta_index->delta_zones[z]);
+ for (z = 0; z < delta_index->zone_count; z++) {
+ result = rebalance_lists(&delta_index->delta_zones[z]);
+ if (result != UDS_SUCCESS)
+ return result;
+ }
return UDS_SUCCESS;
}
diff --git a/drivers/md/dm-vdo/indexer/volume.c b/drivers/md/dm-vdo/indexer/volume.c
index 78c3729f1f65..38e51fa1cbd0 100644
--- a/drivers/md/dm-vdo/indexer/volume.c
+++ b/drivers/md/dm-vdo/indexer/volume.c
@@ -891,12 +891,6 @@ int uds_search_cached_record_page(struct volume *volume, struct uds_request *req
if (record_page_number == NO_CHAPTER_INDEX_ENTRY)
return UDS_SUCCESS;
- result = VDO_ASSERT(record_page_number < geometry->record_pages_per_chapter,
- "0 <= %d < %u", record_page_number,
- geometry->record_pages_per_chapter);
- if (result != VDO_SUCCESS)
- return result;
-
page_number = geometry->index_pages_per_chapter + record_page_number;
physical_page = map_to_physical_page(&volume->geometry, chapter, page_number);
diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c
index 978646f71f99..75287f1667fd 100644
--- a/drivers/md/dm-writecache.c
+++ b/drivers/md/dm-writecache.c
@@ -194,6 +194,7 @@ struct dm_writecache {
struct task_struct *endio_thread;
struct task_struct *flush_thread;
+ spinlock_t flush_list_lock;
struct bio_list flush_list;
struct dm_kcopyd_client *dm_kcopyd;
@@ -899,8 +900,6 @@ static void writecache_discard(struct dm_writecache *wc, sector_t start, sector_
}
discarded_something = true;
}
- if (!writecache_entry_is_committed(wc, e))
- wc->uncommitted_blocks--;
writecache_free_entry(wc, e);
}
@@ -1290,9 +1289,11 @@ static int writecache_flush_thread(void *data)
struct bio *bio;
wc_lock(wc);
+ spin_lock_irq(&wc->flush_list_lock);
bio = bio_list_pop(&wc->flush_list);
if (!bio) {
set_current_state(TASK_INTERRUPTIBLE);
+ spin_unlock_irq(&wc->flush_list_lock);
wc_unlock(wc);
if (unlikely(kthread_should_stop())) {
@@ -1303,6 +1304,7 @@ static int writecache_flush_thread(void *data)
schedule();
continue;
}
+ spin_unlock_irq(&wc->flush_list_lock);
if (bio_op(bio) == REQ_OP_DISCARD) {
writecache_discard(wc, bio->bi_iter.bi_sector,
@@ -1324,9 +1326,12 @@ static int writecache_flush_thread(void *data)
static void writecache_offload_bio(struct dm_writecache *wc, struct bio *bio)
{
+ unsigned long flags;
+ spin_lock_irqsave(&wc->flush_list_lock, flags);
if (bio_list_empty(&wc->flush_list))
wake_up_process(wc->flush_thread);
bio_list_add(&wc->flush_list, bio);
+ spin_unlock_irqrestore(&wc->flush_list_lock, flags);
}
enum wc_map_op {
@@ -1424,7 +1429,8 @@ static void writecache_bio_copy_ssd(struct dm_writecache *wc, struct bio *bio,
dm_accept_partial_bio(bio, bio_size >> SECTOR_SHIFT);
wc->stats.writes += bio->bi_iter.bi_size >> wc->block_size_bits;
- wc->stats.writes_allocate += (bio->bi_iter.bi_size - wc->block_size) >> wc->block_size_bits;
+ if (!search_used)
+ wc->stats.writes_allocate += (bio->bi_iter.bi_size - wc->block_size) >> wc->block_size_bits;
if (unlikely(wc->uncommitted_blocks >= wc->autocommit_blocks)) {
wc->uncommitted_blocks = 0;
@@ -1437,6 +1443,8 @@ static void writecache_bio_copy_ssd(struct dm_writecache *wc, struct bio *bio,
static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio *bio)
{
struct wc_entry *e;
+ enum wc_map_op ret = WC_MAP_SUBMIT;
+ bool need_flush = false;
do {
bool found_entry = false;
@@ -1473,7 +1481,8 @@ static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio
writecache_map_remap_origin(wc, bio, e);
wc->stats.writes_around += bio->bi_iter.bi_size >> wc->block_size_bits;
wc->stats.writes += bio->bi_iter.bi_size >> wc->block_size_bits;
- return WC_MAP_REMAP_ORIGIN;
+ ret = WC_MAP_REMAP_ORIGIN;
+ goto flush_pmem_and_ret;
}
wc->stats.writes_blocked_on_freelist++;
writecache_wait_on_freelist(wc);
@@ -1487,18 +1496,25 @@ static enum wc_map_op writecache_map_write(struct dm_writecache *wc, struct bio
if (WC_MODE_PMEM(wc)) {
bio_copy_block(wc, bio, memory_data(wc, e));
wc->stats.writes++;
+ need_flush = true;
} else {
writecache_bio_copy_ssd(wc, bio, e, search_used);
return WC_MAP_REMAP;
}
} while (bio->bi_iter.bi_size);
- if (unlikely(bio->bi_opf & REQ_FUA || wc->uncommitted_blocks >= wc->autocommit_blocks))
- writecache_flush(wc);
- else
- writecache_schedule_autocommit(wc);
+flush_pmem_and_ret:
+ if (need_flush) {
+ if (unlikely(bio->bi_opf & REQ_FUA || wc->uncommitted_blocks >= wc->autocommit_blocks)) {
+ writecache_flush(wc);
+ if (writecache_has_error(wc))
+ ret = WC_MAP_ERROR;
+ } else {
+ writecache_schedule_autocommit(wc);
+ }
+ }
- return WC_MAP_SUBMIT;
+ return ret;
}
static enum wc_map_op writecache_map_flush(struct dm_writecache *wc, struct bio *bio)
@@ -1588,7 +1604,10 @@ static int writecache_map(struct dm_target *ti, struct bio *bio)
case WC_MAP_REMAP:
/* make sure that writecache_end_io decrements bio_in_progress: */
- bio->bi_private = (void *)1;
+ if (unlikely(bio->bi_opf & REQ_FUA) && bio_op(bio) == REQ_OP_WRITE)
+ bio->bi_private = (void *)3;
+ else
+ bio->bi_private = (void *)1;
atomic_inc(&wc->bio_in_progress[bio_data_dir(bio)]);
wc_unlock(wc);
return DM_MAPIO_REMAPPED;
@@ -1618,12 +1637,19 @@ static int writecache_end_io(struct dm_target *ti, struct bio *bio, blk_status_t
{
struct dm_writecache *wc = ti->private;
- if (bio->bi_private == (void *)1) {
+ if (bio->bi_private == (void *)1 || bio->bi_private == (void *)3) {
int dir = bio_data_dir(bio);
+ bool fua = bio->bi_private == (void *)3;
if (atomic_dec_and_test(&wc->bio_in_progress[dir]))
if (unlikely(waitqueue_active(&wc->bio_in_progress_wait[dir])))
wake_up(&wc->bio_in_progress_wait[dir]);
+
+ if (fua && likely(*status == BLK_STS_OK)) {
+ bio->bi_private = NULL;
+ writecache_offload_bio(wc, bio);
+ return DM_ENDIO_INCOMPLETE;
+ }
} else if (bio->bi_private == (void *)2) {
dm_iot_io_end(&wc->iot, 1);
}
@@ -2498,6 +2524,11 @@ static int writecache_ctr(struct dm_target *ti, unsigned int argc, char **argv)
}
if (WC_MODE_PMEM(wc)) {
+ if (!wc->ssd_dev->dax_dev) {
+ r = -EOPNOTSUPP;
+ ti->error = "Cache device is not a DAX device";
+ goto bad;
+ }
if (!dax_synchronous(wc->ssd_dev->dax_dev)) {
r = -EOPNOTSUPP;
ti->error = "Asynchronous persistent memory not supported as pmem cache";
@@ -2515,6 +2546,7 @@ static int writecache_ctr(struct dm_target *ti, unsigned int argc, char **argv)
wc->memory_map_size -= (uint64_t)wc->start_sector << SECTOR_SHIFT;
+ spin_lock_init(&wc->flush_list_lock);
bio_list_init(&wc->flush_list);
wc->flush_thread = kthread_run(writecache_flush_thread, wc, "dm_writecache_flush");
if (IS_ERR(wc->flush_thread)) {
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index eb9e32995b79..1881d86111fb 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -1350,7 +1350,7 @@ void dm_accept_partial_bio(struct bio *bio, unsigned int n_sectors)
* for accounting but it is being reduced so update accordingly.
*/
dm_io_set_flag(io, DM_IO_WAS_SPLIT);
- io->sectors = n_sectors;
+ io->sectors = *tio->len_ptr;
io->sector_offset = bio_sectors(io->orig_bio);
}
EXPORT_SYMBOL_GPL(dm_accept_partial_bio);
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 680b34a63cb3..d8ecf76d4a5f 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -1934,8 +1934,11 @@ static int super_1_load(struct md_rdev *rdev, struct md_rdev *refdev, int minor_
if (!badblocks_set(&rdev->badblocks, sector, count, 1))
return -EINVAL;
}
- } else if (sb->bblog_offset != 0)
+ } else if (sb->bblog_offset != 0) {
+ if (le16_to_cpu(sb->bblog_size) > (PAGE_SIZE / 512))
+ return -EINVAL;
rdev->badblocks.shift = 0;
+ }
if ((le32_to_cpu(sb->feature_map) &
(MD_FEATURE_PPL | MD_FEATURE_MULTIPLE_PPLS))) {
@@ -6385,12 +6388,22 @@ struct mddev *md_alloc(dev_t dev, char *name)
disk->events |= DISK_EVENT_MEDIA_CHANGE;
mddev->gendisk = disk;
+
+ /*
+ * add_disk() makes the array openable before mddev->kobj exists.
+ * Hold reconfig_mutex until kobject_add() is done so that ioctls
+ * issued in between wait instead of using an uninitialised kobj.
+ */
+ mddev_lock_nointr(mddev);
error = add_disk(disk);
- if (error)
+ if (error) {
+ mddev_unlock(mddev);
goto out_put_disk;
+ }
kobject_init(&mddev->kobj, &md_ktype);
error = kobject_add(&mddev->kobj, &disk_to_dev(disk)->kobj, "%s", "md");
+ mddev_unlock(mddev);
if (error) {
/*
* The disk is already live at this point. Clear the hold flag
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 7b7546bfa21f..62460e231f02 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -2001,27 +2001,36 @@ r5l_recovery_verify_data_checksum_for_mb(struct r5l_log *log,
if (!page)
return -ENOMEM;
- while (mb_offset < le32_to_cpu(mb->meta_size)) {
+ while (mb_offset + sizeof(struct r5l_payload_header) <= le32_to_cpu(mb->meta_size)) {
+ u32 payload_size;
sector_t payload_len;
payload = (void *)mb + mb_offset;
payload_flush = (void *)mb + mb_offset;
if (le16_to_cpu(payload->header.type) == R5LOG_PAYLOAD_DATA) {
+ if (mb_offset + sizeof(struct r5l_payload_data_parity)
+ > le32_to_cpu(mb->meta_size))
+ goto mismatch;
+ payload_size = le32_to_cpu(payload->size) >> (PAGE_SHIFT - 9);
payload_len = sizeof(struct r5l_payload_data_parity) +
- (sector_t)sizeof(__le32) *
- (le32_to_cpu(payload->size) >> (PAGE_SHIFT - 9));
- if (mb_offset + payload_len > le32_to_cpu(mb->meta_size))
+ (sector_t)sizeof(__le32) * payload_size;
+ if (mb_offset + payload_len > le32_to_cpu(mb->meta_size) ||
+ payload_size < 1)
goto mismatch;
if (r5l_recovery_verify_data_checksum(
log, ctx, page, log_offset,
payload->checksum[0]) < 0)
goto mismatch;
} else if (le16_to_cpu(payload->header.type) == R5LOG_PAYLOAD_PARITY) {
+ if (mb_offset + sizeof(struct r5l_payload_data_parity)
+ > le32_to_cpu(mb->meta_size))
+ goto mismatch;
+ payload_size = le32_to_cpu(payload->size) >> (PAGE_SHIFT - 9);
payload_len = sizeof(struct r5l_payload_data_parity) +
- (sector_t)sizeof(__le32) *
- (le32_to_cpu(payload->size) >> (PAGE_SHIFT - 9));
- if (mb_offset + payload_len > le32_to_cpu(mb->meta_size))
+ (sector_t)sizeof(__le32) * payload_size;
+ if (mb_offset + payload_len > le32_to_cpu(mb->meta_size) ||
+ payload_size < conf->max_degraded)
goto mismatch;
if (r5l_recovery_verify_data_checksum(
log, ctx, page, log_offset,
@@ -2035,6 +2044,9 @@ r5l_recovery_verify_data_checksum_for_mb(struct r5l_log *log,
payload->checksum[1]) < 0)
goto mismatch;
} else if (le16_to_cpu(payload->header.type) == R5LOG_PAYLOAD_FLUSH) {
+ if (mb_offset + sizeof(struct r5l_payload_flush)
+ > le32_to_cpu(mb->meta_size))
+ goto mismatch;
payload_len = sizeof(struct r5l_payload_flush) +
(sector_t)le32_to_cpu(payload_flush->size);
if (mb_offset + payload_len > le32_to_cpu(mb->meta_size))
@@ -2096,7 +2108,7 @@ r5c_recovery_analyze_meta_block(struct r5l_log *log,
mb_offset = sizeof(struct r5l_meta_block);
log_offset = r5l_ring_add(log, ctx->pos, BLOCK_SECTORS);
- while (mb_offset < le32_to_cpu(mb->meta_size)) {
+ while (mb_offset + sizeof(struct r5l_payload_header) < le32_to_cpu(mb->meta_size)) {
sector_t payload_len;
int dd;
@@ -2106,6 +2118,9 @@ r5c_recovery_analyze_meta_block(struct r5l_log *log,
if (le16_to_cpu(payload->header.type) == R5LOG_PAYLOAD_FLUSH) {
int i, count;
+ if (mb_offset + sizeof(struct r5l_payload_flush) >
+ le32_to_cpu(mb->meta_size))
+ return -EINVAL;
payload_len = sizeof(struct r5l_payload_flush) +
(sector_t)le32_to_cpu(payload_flush->size);
if (mb_offset + payload_len >
@@ -2130,6 +2145,9 @@ r5c_recovery_analyze_meta_block(struct r5l_log *log,
}
/* DATA or PARITY payload */
+ if (mb_offset + sizeof(struct r5l_payload_data_parity) >
+ le32_to_cpu(mb->meta_size))
+ return -EINVAL;
payload_len = sizeof(struct r5l_payload_data_parity) +
(sector_t)sizeof(__le32) *
(le32_to_cpu(payload->size) >> (PAGE_SHIFT - 9));
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index b91545ce090d..bc5f4b048eed 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -2642,6 +2642,7 @@ static int resize_stripes(struct r5conf *conf, int newsize)
} else {
kfree(conf->disks);
conf->disks = ndisks;
+ conf->disks_cnt = newsize;
}
} else
err = -ENOMEM;
@@ -7538,7 +7539,8 @@ static void raid5_free_percpu(struct r5conf *conf)
if (!conf->percpu)
return;
- cpuhp_state_remove_instance(CPUHP_MD_RAID5_PREPARE, &conf->node);
+ if (!hlist_unhashed_lockless(&conf->node))
+ cpuhp_state_remove_instance(CPUHP_MD_RAID5_PREPARE, &conf->node);
free_percpu(conf->percpu);
}
@@ -7552,7 +7554,7 @@ static void free_conf(struct r5conf *conf)
free_thread_groups(conf);
shrink_stripes(conf);
raid5_free_percpu(conf);
- for (i = 0; i < conf->pool_size; i++)
+ for (i = 0; i < conf->disks_cnt; i++)
if (conf->disks[i].extra_page)
put_page(conf->disks[i].extra_page);
kfree(conf->disks);
@@ -7733,7 +7735,7 @@ static struct r5conf *setup_conf(struct mddev *mddev)
if (!conf->disks)
goto abort;
-
+ conf->disks_cnt = max_disks;
for (i = 0; i < max_disks; i++) {
conf->disks[i].extra_page = alloc_page(GFP_KERNEL);
if (!conf->disks[i].extra_page)
diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h
index cb5feae04db2..94b3e2c947ba 100644
--- a/drivers/md/raid5.h
+++ b/drivers/md/raid5.h
@@ -667,6 +667,7 @@ struct r5conf {
unsigned long cache_state;
struct shrinker *shrinker;
int pool_size; /* number of disks in stripeheads in pool */
+ int disks_cnt; /* number of entries in disks[] array */
spinlock_t device_lock;
struct disk_info *disks;
struct bio_set bio_split;
diff --git a/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c b/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
index 3c6ce6f3d93e..1f8ed7ff31e5 100644
--- a/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
+++ b/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
@@ -847,7 +847,9 @@ static irqreturn_t extron_interrupt(struct serio *serio, unsigned char data,
return IRQ_HANDLED;
memcpy(extron->data, extron->buf, extron->idx);
extron->len = extron->idx;
- extron->data[extron->len] = 0;
+ /* Keep fixed-offset response tests from using stale tail bytes. */
+ memset(extron->data + extron->len, 0,
+ min_t(size_t, 10, sizeof(extron->data) - extron->len));
if (debug)
dev_info(extron->dev, "received %s\n", extron->data);
extron->idx = 0;
diff --git a/drivers/media/common/cypress_firmware.c b/drivers/media/common/cypress_firmware.c
index 66274fdf5243..d0f66ed01c4b 100644
--- a/drivers/media/common/cypress_firmware.c
+++ b/drivers/media/common/cypress_firmware.c
@@ -59,6 +59,8 @@ static int cypress_get_hexline(const struct firmware *fw,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data
* field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
}
diff --git a/drivers/media/common/saa7146/saa7146_core.c b/drivers/media/common/saa7146/saa7146_core.c
index c297d019f2b3..70d6ca00513f 100644
--- a/drivers/media/common/saa7146/saa7146_core.c
+++ b/drivers/media/common/saa7146/saa7146_core.c
@@ -340,6 +340,8 @@ static int saa7146_init_one(struct pci_dev *pci, const struct pci_device_id *ent
goto out;
}
+ spin_lock_init(&dev->int_slock);
+
/* create a nice device name */
sprintf(dev->name, "saa7146 (%d)", saa7146_num);
@@ -425,7 +427,6 @@ static int saa7146_init_one(struct pci_dev *pci, const struct pci_device_id *ent
dev->ext = ext;
mutex_init(&dev->v4l2_lock);
- spin_lock_init(&dev->int_slock);
spin_lock_init(&dev->slock);
mutex_init(&dev->i2c_lock);
diff --git a/drivers/media/dvb-frontends/stv0900_core.c b/drivers/media/dvb-frontends/stv0900_core.c
index d15c55de2723..0ca6b6d81273 100644
--- a/drivers/media/dvb-frontends/stv0900_core.c
+++ b/drivers/media/dvb-frontends/stv0900_core.c
@@ -1773,6 +1773,8 @@ static int stv0900_recv_slave_reply(struct dvb_frontend *fe,
if (stv0900_get_bits(intp, RX_END)) {
reply->msg_len = stv0900_get_bits(intp, FIFO_BYTENBR);
+ if (reply->msg_len > sizeof(reply->msg))
+ reply->msg_len = sizeof(reply->msg);
for (i = 0; i < reply->msg_len; i++)
reply->msg[i] = stv0900_read_reg(intp, DISRXDATA);
diff --git a/drivers/media/dvb-frontends/stv090x.c b/drivers/media/dvb-frontends/stv090x.c
index 932bbed5497a..63be9ee104ab 100644
--- a/drivers/media/dvb-frontends/stv090x.c
+++ b/drivers/media/dvb-frontends/stv090x.c
@@ -3902,6 +3902,8 @@ static int stv090x_recv_slave_reply(struct dvb_frontend *fe, struct dvb_diseqc_s
if (rx_end) {
reply->msg_len = STV090x_GETFIELD_Px(reg, FIFO_BYTENBR_FIELD);
+ if (reply->msg_len > sizeof(reply->msg))
+ reply->msg_len = sizeof(reply->msg);
for (i = 0; i < reply->msg_len; i++)
reply->msg[i] = STV090x_READ_DEMOD(state, DISRXDATA);
}
diff --git a/drivers/media/i2c/adv7180.c b/drivers/media/i2c/adv7180.c
index a1c7f68225b4..8c2cf3f53978 100644
--- a/drivers/media/i2c/adv7180.c
+++ b/drivers/media/i2c/adv7180.c
@@ -1454,6 +1454,7 @@ static int adv7180_probe(struct i2c_client *client)
if (state == NULL)
return -ENOMEM;
+ mutex_init(&state->mutex);
state->client = client;
state->field = V4L2_FIELD_ALTERNATE;
state->chip_info = i2c_get_match_data(client);
@@ -1497,7 +1498,6 @@ static int adv7180_probe(struct i2c_client *client)
}
state->irq = client->irq;
- mutex_init(&state->mutex);
state->curr_norm = V4L2_STD_NTSC;
state->input = 0;
diff --git a/drivers/media/i2c/adv748x/adv748x-core.c b/drivers/media/i2c/adv748x/adv748x-core.c
index 3eb6d5e8f082..70594d001804 100644
--- a/drivers/media/i2c/adv748x/adv748x-core.c
+++ b/drivers/media/i2c/adv748x/adv748x-core.c
@@ -675,9 +675,6 @@ static int adv748x_parse_dt(struct adv748x_state *state)
continue;
}
- of_node_get(ep_np);
- state->endpoints[ep.port] = ep_np;
-
/*
* At least one input endpoint and one output endpoint shall
* be defined.
@@ -689,8 +686,13 @@ static int adv748x_parse_dt(struct adv748x_state *state)
/* Store number of CSI-2 lanes used for TXA and TXB. */
ret = adv748x_parse_csi2_lanes(state, ep.port, ep_np);
- if (ret)
+ if (ret) {
+ of_node_put(ep_np);
return ret;
+ }
+
+ of_node_get(ep_np);
+ state->endpoints[ep.port] = ep_np;
}
return in_found && out_found ? 0 : -ENODEV;
@@ -739,7 +741,7 @@ static int adv748x_probe(struct i2c_client *client)
ret = adv748x_parse_dt(state);
if (ret) {
adv_err(state, "Failed to parse device tree");
- goto err_free_mutex;
+ goto err_cleanup_dt;
}
/* Configure IO Regmap region */
@@ -809,7 +811,6 @@ static int adv748x_probe(struct i2c_client *client)
adv748x_unregister_clients(state);
err_cleanup_dt:
adv748x_dt_cleanup(state);
-err_free_mutex:
mutex_destroy(&state->mutex);
return ret;
diff --git a/drivers/media/i2c/adv7604.c b/drivers/media/i2c/adv7604.c
index ac9c69ce438f..231edd3797ab 100644
--- a/drivers/media/i2c/adv7604.c
+++ b/drivers/media/i2c/adv7604.c
@@ -2641,8 +2641,9 @@ static int adv76xx_log_status(struct v4l2_subdev *sd)
"(16-235)" : "(0-255)",
(reg_io_0x02 & 0x08) ? "enabled" : "disabled");
}
+ ret = cp_read(sd, info->cp_csc) >> 4;
v4l2_info(sd, "Color space conversion: %s\n",
- csc_coeff_sel_rb[cp_read(sd, info->cp_csc) >> 4]);
+ ret < 0 ? "" : csc_coeff_sel_rb[ret]);
if (!is_digital_input(sd))
return 0;
diff --git a/drivers/media/i2c/alvium-csi2.c b/drivers/media/i2c/alvium-csi2.c
index f51f9b987759..f218999e0ed9 100644
--- a/drivers/media/i2c/alvium-csi2.c
+++ b/drivers/media/i2c/alvium-csi2.c
@@ -2523,6 +2523,7 @@ static void alvium_remove(struct i2c_client *client)
* make sure to turn power off manually.
*/
pm_runtime_disable(dev);
+ pm_runtime_put_noidle(dev);
if (!pm_runtime_status_suspended(dev))
alvium_set_power(alvium, false);
pm_runtime_set_suspended(dev);
diff --git a/drivers/media/i2c/cvs/core.c b/drivers/media/i2c/cvs/core.c
index d4a3b9c3bab1..8d857bbd8ab5 100644
--- a/drivers/media/i2c/cvs/core.c
+++ b/drivers/media/i2c/cvs/core.c
@@ -725,8 +725,6 @@ static int cvs_core_probe(struct device *dev, struct i2c_client *i2c)
}
if (ctx->res == ICVS_FULLCAP) {
- struct gpio_desc *wake;
-
ctx->rst = devm_gpiod_get(dev, "rst", GPIOD_OUT_HIGH);
if (IS_ERR(ctx->rst)) {
ret = dev_err_probe(dev, PTR_ERR(ctx->rst),
@@ -734,14 +732,12 @@ static int cvs_core_probe(struct device *dev, struct i2c_client *i2c)
goto err_put_ipu;
}
- wake = devm_gpiod_get(dev, "wake", GPIOD_IN);
- if (IS_ERR(wake)) {
- ret = dev_err_probe(dev, PTR_ERR(wake),
- "failed to get wake GPIO\n");
- goto err_put_ipu;
- }
-
- ctx->irq = gpiod_to_irq(wake);
+ /*
+ * Do not request the line: another device's _CRS may list
+ * the same pin, and its driver would then fail with -EBUSY.
+ */
+ ctx->irq = acpi_dev_gpio_irq_get_by(ACPI_COMPANION(dev),
+ "wake", 0);
if (ctx->irq < 0) {
ret = dev_err_probe(dev, ctx->irq,
"failed to get wake IRQ\n");
diff --git a/drivers/media/i2c/cvs/v4l2.c b/drivers/media/i2c/cvs/v4l2.c
index 9fadca7a3bee..863112cde266 100644
--- a/drivers/media/i2c/cvs/v4l2.c
+++ b/drivers/media/i2c/cvs/v4l2.c
@@ -13,6 +13,7 @@
#include <media/v4l2-async.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
#include <media/v4l2-fwnode.h>
#include <media/v4l2-mc.h>
@@ -400,7 +401,7 @@ static int cvs_csi_notify_bound(struct v4l2_async_notifier *notifier,
struct v4l2_async_connection *asc)
{
struct icvs *ctx = notifier_to_csi(notifier);
- int pad;
+ int pad, ret;
pad = media_entity_get_fwnode_pad(&sd->entity, asc->match.fwnode,
MEDIA_PAD_FL_SOURCE);
@@ -409,9 +410,13 @@ static int cvs_csi_notify_bound(struct v4l2_async_notifier *notifier,
ctx->remote = &sd->entity.pads[pad];
- return media_create_pad_link(&sd->entity, pad, &ctx->subdev.entity,
- ICVS_CSI_PAD_SINK, MEDIA_LNK_FL_ENABLED |
- MEDIA_LNK_FL_IMMUTABLE);
+ ret = media_create_pad_link(&sd->entity, pad, &ctx->subdev.entity,
+ ICVS_CSI_PAD_SINK, MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret)
+ return ret;
+
+ return v4l2_device_register_subdev_nodes(sd->v4l2_dev);
}
/**
diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 9571f3622d2d..05717b1e3224 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -78,6 +78,7 @@
#define IMX219_LLP_MIN 0x0d78
#define IMX219_BINNED_LLP_MIN 0x0de8
#define IMX219_LLP_MAX 0x7ff0
+#define IMX219_LLP_STEP 8
#define IMX219_REG_X_ADD_STA_A CCI_REG16(0x0164)
#define IMX219_REG_X_ADD_END_A CCI_REG16(0x0166)
@@ -340,13 +341,13 @@ static const struct imx219_mode supported_modes[] = {
/* 2x2 binned 60fps mode */
.width = 1640,
.height = 1232,
- .fll_def = 1707,
+ .fll_def = 1706,
},
{
/* 640x480 60fps mode */
.width = 640,
.height = 480,
- .fll_def = 1707,
+ .fll_def = 1706,
},
};
@@ -466,14 +467,14 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
int exposure_max, exposure_def;
/* Update max exposure while meeting expected vblanking */
- exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
+ exposure_max = format->height + ctrl->val -
+ IMX219_EXPOSURE_OFFSET * rate_factor;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
exposure_max : IMX219_EXPOSURE_DEFAULT;
ret = __v4l2_ctrl_modify_range(imx219->exposure,
imx219->exposure->minimum,
exposure_max,
- imx219->exposure->step,
- exposure_def);
+ rate_factor, exposure_def);
if (ret)
return ret;
@@ -593,7 +594,8 @@ static int imx219_init_controls(struct imx219 *imx219)
imx219->hblank = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
V4L2_CID_HBLANK,
IMX219_LLP_MIN - mode->width,
- IMX219_LLP_MAX - mode->width, 1,
+ IMX219_LLP_MAX - mode->width,
+ IMX219_LLP_STEP,
IMX219_LLP_MIN - mode->width);
exposure_max = mode->fll_def - IMX219_EXPOSURE_OFFSET;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
@@ -893,6 +895,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
+ int rate_factor = imx219_get_rate_factor(state);
int exposure_max;
int exposure_def;
int llp_min;
@@ -900,7 +903,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
/* Update limits and set FPS to default */
ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
- IMX219_FLL_MAX - mode->height, 1,
+ IMX219_FLL_MAX - mode->height,
+ rate_factor,
mode->fll_def - mode->height);
if (ret)
return ret;
@@ -911,14 +915,14 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
return ret;
/* Update max exposure while meeting expected vblanking */
- exposure_max = mode->fll_def - IMX219_EXPOSURE_OFFSET;
+ exposure_max = mode->fll_def -
+ IMX219_EXPOSURE_OFFSET * rate_factor;
exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
exposure_max : IMX219_EXPOSURE_DEFAULT;
ret = __v4l2_ctrl_modify_range(imx219->exposure,
imx219->exposure->minimum,
exposure_max,
- imx219->exposure->step,
- exposure_def);
+ rate_factor, exposure_def);
if (ret)
return ret;
@@ -933,7 +937,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
IMX219_BINNED_LLP_MIN : IMX219_LLP_MIN;
ret = __v4l2_ctrl_modify_range(imx219->hblank,
llp_min - mode->width,
- IMX219_LLP_MAX - mode->width, 1,
+ IMX219_LLP_MAX - mode->width,
+ IMX219_LLP_STEP,
llp_min - mode->width);
if (ret)
return ret;
@@ -943,8 +948,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
return ret;
/* Scale the pixel rate based on the mode specific factor */
- pixel_rate = imx219_get_pixel_rate(imx219) *
- imx219_get_rate_factor(state);
+ pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
pixel_rate, 1, pixel_rate);
if (ret)
diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c
index 1675f0460969..1cece90c59ee 100644
--- a/drivers/media/i2c/og01a1b.c
+++ b/drivers/media/i2c/og01a1b.c
@@ -956,6 +956,11 @@ static void og01a1b_remove(struct i2c_client *client)
media_entity_cleanup(&sd->entity);
v4l2_ctrl_handler_free(sd->ctrl_handler);
pm_runtime_disable(og01a1b->dev);
+
+ if (!pm_runtime_status_suspended(og01a1b->dev)) {
+ og01a1b_power_off(og01a1b->dev);
+ pm_runtime_set_suspended(og01a1b->dev);
+ }
}
static int og01a1b_probe(struct i2c_client *client)
diff --git a/drivers/media/i2c/ov2735.c b/drivers/media/i2c/ov2735.c
index dcb1add1fd9f..2defcaa8b795 100644
--- a/drivers/media/i2c/ov2735.c
+++ b/drivers/media/i2c/ov2735.c
@@ -1081,6 +1081,7 @@ static void ov2735_remove(struct i2c_client *client)
v4l2_subdev_cleanup(&ov2735->sd);
media_entity_cleanup(&sd->entity);
v4l2_ctrl_handler_free(ov2735->sd.ctrl_handler);
+ ov2735_power_off(ov2735->dev);
}
static DEFINE_RUNTIME_DEV_PM_OPS(ov2735_pm_ops,
diff --git a/drivers/media/i2c/ov5647.c b/drivers/media/i2c/ov5647.c
index 3facf92b3841..3b3e5653a0c6 100644
--- a/drivers/media/i2c/ov5647.c
+++ b/drivers/media/i2c/ov5647.c
@@ -1271,6 +1271,10 @@ static void ov5647_remove(struct i2c_client *client)
v4l2_ctrl_handler_free(&sensor->ctrls);
v4l2_device_unregister_subdev(sd);
pm_runtime_disable(&client->dev);
+ if (!pm_runtime_status_suspended(&client->dev)) {
+ ov5647_power_off(&client->dev);
+ pm_runtime_set_suspended(&client->dev);
+ }
}
static const struct dev_pm_ops ov5647_pm_ops = {
diff --git a/drivers/media/i2c/ov8856.c b/drivers/media/i2c/ov8856.c
index 8bedb47cd7cf..6011233a218d 100644
--- a/drivers/media/i2c/ov8856.c
+++ b/drivers/media/i2c/ov8856.c
@@ -2082,9 +2082,6 @@ static int ov8856_power_on(struct device *dev)
struct ov8856 *ov8856 = to_ov8856(sd);
int ret;
- if (is_acpi_node(dev_fwnode(dev)))
- return 0;
-
ret = clk_prepare_enable(ov8856->xvclk);
if (ret < 0) {
dev_err(dev, "failed to enable xvclk\n");
@@ -2120,9 +2117,6 @@ static int ov8856_power_off(struct device *dev)
struct v4l2_subdev *sd = dev_get_drvdata(dev);
struct ov8856 *ov8856 = to_ov8856(sd);
- if (is_acpi_node(dev_fwnode(dev)))
- return 0;
-
gpiod_set_value_cansleep(ov8856->reset_gpio, 1);
regulator_bulk_disable(ARRAY_SIZE(ov8856_supply_names),
ov8856->supplies);
@@ -2293,21 +2287,18 @@ static int ov8856_get_hwcfg(struct ov8856 *ov8856)
dev_warn(dev, "external clock rate %u is unsupported",
xvclk_rate);
- if (!is_acpi_node(fwnode)) {
- ov8856->reset_gpio = devm_gpiod_get_optional(dev, "reset",
- GPIOD_OUT_LOW);
- if (IS_ERR(ov8856->reset_gpio))
- return PTR_ERR(ov8856->reset_gpio);
+ ov8856->reset_gpio = devm_gpiod_get_optional(dev, "reset",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(ov8856->reset_gpio))
+ return PTR_ERR(ov8856->reset_gpio);
- for (i = 0; i < ARRAY_SIZE(ov8856_supply_names); i++)
- ov8856->supplies[i].supply = ov8856_supply_names[i];
+ for (i = 0; i < ARRAY_SIZE(ov8856_supply_names); i++)
+ ov8856->supplies[i].supply = ov8856_supply_names[i];
- ret = devm_regulator_bulk_get(dev,
- ARRAY_SIZE(ov8856_supply_names),
- ov8856->supplies);
- if (ret)
- return ret;
- }
+ ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(ov8856_supply_names),
+ ov8856->supplies);
+ if (ret)
+ return ret;
ep = fwnode_graph_get_next_endpoint(fwnode, NULL);
if (!ep)
@@ -2327,7 +2318,8 @@ static int ov8856_get_hwcfg(struct ov8856 *ov8856)
goto check_hwcfg_error;
}
- dev_dbg(dev, "Using %u data lanes\n", ov8856->cur_mode->data_lanes);
+ dev_dbg(dev, "Using %u data lanes\n",
+ bus_cfg.bus.mipi_csi2.num_data_lanes);
if (bus_cfg.bus.mipi_csi2.num_data_lanes == 2)
ov8856->priv_lane = &lane_cfg_2;
diff --git a/drivers/media/i2c/tc358743.c b/drivers/media/i2c/tc358743.c
index fbd38bbfee03..ce75812cbf72 100644
--- a/drivers/media/i2c/tc358743.c
+++ b/drivers/media/i2c/tc358743.c
@@ -1373,7 +1373,8 @@ static int tc358743_log_status(struct v4l2_subdev *sd)
static const char * const input_color_space[] = {
"RGB", "YCbCr 601", "opRGB", "YCbCr 709", "NA (4)",
"xvYCC 601", "NA(6)", "xvYCC 709", "NA(8)", "sYCC601",
- "NA(10)", "NA(11)", "NA(12)", "opYCC 601"};
+ "NA(10)", "NA(11)", "NA(12)", "opYCC 601", "NA(14)",
+ "NA(15)"};
v4l2_info(sd, "-----Chip status-----\n");
v4l2_info(sd, "Chip ID: 0x%02x\n",
diff --git a/drivers/media/pci/cx18/cx18-driver.c b/drivers/media/pci/cx18/cx18-driver.c
index 214fac7af61e..ac90ce6cd044 100644
--- a/drivers/media/pci/cx18/cx18-driver.c
+++ b/drivers/media/pci/cx18/cx18-driver.c
@@ -805,12 +805,12 @@ static int cx18_setup_pci(struct cx18 *cx, struct pci_dev *pci_dev,
}
if (dma_set_mask(&pci_dev->dev, DMA_BIT_MASK(32))) {
CX18_ERR("No suitable DMA available, card %d\n", cx->instance);
- return -EIO;
+ goto err_disable_device;
}
if (!request_mem_region(cx->base_addr, CX18_MEM_SIZE, "cx18 encoder")) {
CX18_ERR("Cannot request encoder memory region, card %d\n",
cx->instance);
- return -EIO;
+ goto err_disable_device;
}
/* Enable bus mastering and memory mapped IO for the CX23418 */
@@ -834,6 +834,10 @@ static int cx18_setup_pci(struct cx18 *cx, struct pci_dev *pci_dev,
cx->pci_dev->irq, pci_latency, (u64)cx->base_addr);
return 0;
+
+err_disable_device:
+ pci_disable_device(pci_dev);
+ return -EIO;
}
static void cx18_init_subdevs(struct cx18 *cx)
@@ -1120,6 +1124,7 @@ static int cx18_probe(struct pci_dev *pci_dev,
cx18_iounmap(cx);
free_mem:
release_mem_region(cx->base_addr, CX18_MEM_SIZE);
+ pci_disable_device(pci_dev);
free_workqueues:
destroy_workqueue(cx->in_work_queue);
err:
diff --git a/drivers/media/pci/cx88/cx88-input.c b/drivers/media/pci/cx88/cx88-input.c
index 5d9ce4f9af01..c96d289e5ab7 100644
--- a/drivers/media/pci/cx88/cx88-input.c
+++ b/drivers/media/pci/cx88/cx88-input.c
@@ -259,7 +259,7 @@ static void cx88_ir_close(struct rc_dev *rc)
int cx88_ir_init(struct cx88_core *core, struct pci_dev *pci)
{
struct cx88_IR *ir;
- struct rc_dev *dev;
+ struct rc_dev *dev = NULL;
char *ir_codes = NULL;
u64 rc_proto = RC_PROTO_BIT_OTHER;
int err = -ENOMEM;
@@ -268,11 +268,8 @@ int cx88_ir_init(struct cx88_core *core, struct pci_dev *pci)
*/
ir = kzalloc_obj(*ir);
- dev = rc_allocate_device(RC_DRIVER_IR_RAW);
- if (!ir || !dev)
- goto err_out_free;
-
- ir->dev = dev;
+ if (!ir)
+ return -ENOMEM;
/* detect & configure */
switch (core->boardnr) {
@@ -439,6 +436,13 @@ int cx88_ir_init(struct cx88_core *core, struct pci_dev *pci)
goto err_out_free;
}
+ dev = rc_allocate_device(ir->sampling ?
+ RC_DRIVER_IR_RAW : RC_DRIVER_SCANCODE);
+ if (!dev)
+ goto err_out_free;
+
+ ir->dev = dev;
+
/*
* The usage of mask_keycode were very convenient, due to several
* reasons. Among others, the scancode tables were using the scancode
@@ -477,12 +481,10 @@ int cx88_ir_init(struct cx88_core *core, struct pci_dev *pci)
dev->close = cx88_ir_close;
dev->scancode_mask = hardware_mask;
- if (ir->sampling) {
+ if (ir->sampling)
dev->timeout = MS_TO_US(10); /* 10 ms */
- } else {
- dev->driver_type = RC_DRIVER_SCANCODE;
+ else
dev->allowed_protocols = rc_proto;
- }
ir->core = core;
core->ir = ir;
diff --git a/drivers/media/pci/hws/hws.h b/drivers/media/pci/hws/hws.h
index 8fbe1fe27844..d87d52674b69 100644
--- a/drivers/media/pci/hws/hws.h
+++ b/drivers/media/pci/hws/hws.h
@@ -33,8 +33,8 @@ struct hws_pix_state {
u32 sizeimage; /* full frame */
enum v4l2_field field; /* V4L2_FIELD_NONE or INTERLACED */
enum v4l2_colorspace colorspace; /* e.g., REC709 */
- enum v4l2_ycbcr_encoding ycbcr_enc; /* V4L2_YCBCR_ENC_DEFAULT */
- enum v4l2_quantization quantization; /* V4L2_QUANTIZATION_LIM_RANGE */
+ enum v4l2_ycbcr_encoding ycbcr_enc; /* V4L2_YCBCR_ENC_601 */
+ enum v4l2_quantization quantization; /* V4L2_QUANTIZATION_FULL_RANGE */
enum v4l2_xfer_func xfer_func; /* V4L2_XFER_FUNC_DEFAULT */
bool interlaced; /* cached hardware state */
u32 half_size; /* hardware half-frame size */
diff --git a/drivers/media/pci/hws/hws_pci.c b/drivers/media/pci/hws/hws_pci.c
index 30bb7d34465b..87ba27877f29 100644
--- a/drivers/media/pci/hws/hws_pci.c
+++ b/drivers/media/pci/hws/hws_pci.c
@@ -441,7 +441,7 @@ static int hws_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id)
hws_init_video_sys(hws, false);
/* 5) Init channels (video state, locks, vb2, ctrls) */
- for (i = 0; i < hws->max_channels; i++) {
+ for (i = 0; i < hws->cur_max_video_ch; i++) {
ret = hws_video_init_channel(hws, i);
if (ret) {
dev_err(&pdev->dev, "video channel init failed (ch=%d)\n", i);
diff --git a/drivers/media/pci/hws/hws_reg.h b/drivers/media/pci/hws/hws_reg.h
index e4fb4af44434..f6027a16a193 100644
--- a/drivers/media/pci/hws/hws_reg.h
+++ b/drivers/media/pci/hws/hws_reg.h
@@ -70,7 +70,8 @@
#define HWS_CTL_IRQ_ENABLE_BIT BIT(0) /* Global interrupt enable bit */
/* Write 0x00 to fully reset decoder,
* set bit 31=1 to "start run",
- * low byte=0x13 selects YUYV/BT.709/etc,
+ * low byte=0x13 is the vendor baseline capture-mode value; its individual
+ * bit meanings are not documented,
* in ReadChipId() we also write 0x00 and 0x10 here for chip-ID sequencing.
*/
diff --git a/drivers/media/pci/hws/hws_v4l2_ioctl.c b/drivers/media/pci/hws/hws_v4l2_ioctl.c
index 0303e311ee4a..ce396b7225d2 100644
--- a/drivers/media/pci/hws/hws_v4l2_ioctl.c
+++ b/drivers/media/pci/hws/hws_v4l2_ioctl.c
@@ -511,8 +511,9 @@ static inline void hws_set_colorimetry_state(struct hws_pix_state *p)
{
bool sd = p->height <= 576;
+ /* Captured samples use full-range BT.601 Y'CbCr at every resolution. */
p->colorspace = sd ? V4L2_COLORSPACE_SMPTE170M : V4L2_COLORSPACE_REC709;
- p->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ p->ycbcr_enc = V4L2_YCBCR_ENC_601;
p->quantization = V4L2_QUANTIZATION_FULL_RANGE;
p->xfer_func = V4L2_XFER_FUNC_DEFAULT;
}
@@ -736,7 +737,7 @@ static inline void hws_set_colorimetry_fmt(struct v4l2_pix_format *p)
bool sd = p->height <= 576;
p->colorspace = sd ? V4L2_COLORSPACE_SMPTE170M : V4L2_COLORSPACE_REC709;
- p->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ p->ycbcr_enc = V4L2_YCBCR_ENC_601;
p->quantization = V4L2_QUANTIZATION_FULL_RANGE;
p->xfer_func = V4L2_XFER_FUNC_DEFAULT;
}
diff --git a/drivers/media/pci/hws/hws_video.c b/drivers/media/pci/hws/hws_video.c
index 18e4bc6901d3..dbe0fc2a66b5 100644
--- a/drivers/media/pci/hws/hws_video.c
+++ b/drivers/media/pci/hws/hws_video.c
@@ -327,7 +327,7 @@ int hws_video_init_channel(struct hws_pcie_dev *pdev, int ch)
vid->pix.sizeimage = vid->pix.bytesperline * vid->pix.height;
vid->pix.field = V4L2_FIELD_NONE;
vid->pix.colorspace = V4L2_COLORSPACE_REC709;
- vid->pix.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ vid->pix.ycbcr_enc = V4L2_YCBCR_ENC_601;
vid->pix.quantization = V4L2_QUANTIZATION_FULL_RANGE;
vid->pix.xfer_func = V4L2_XFER_FUNC_DEFAULT;
vid->pix.interlaced = false;
@@ -895,6 +895,8 @@ static void hws_video_apply_mode_change(struct hws_pcie_dev *pdx,
v->pix.width = w;
v->pix.height = h;
v->pix.interlaced = interlaced;
+ v->pix.colorspace = h <= 576 ? V4L2_COLORSPACE_SMPTE170M :
+ V4L2_COLORSPACE_REC709;
hws_set_current_dv_timings(v, w, h, interlaced);
v->current_fps = fps;
@@ -1292,6 +1294,8 @@ static void hws_stop_streaming(struct vb2_queue *q)
WRITE_ONCE(v->stop_requested, true);
hws_enable_video_capture(v->parent, v->channel_index, false);
+ if (hws->irq >= 0)
+ synchronize_irq(hws->irq);
/* 2) Collect in-flight + queued under the IRQ lock */
spin_lock_irqsave(&v->irq_lock, flags);
diff --git a/drivers/media/pci/saa7134/saa7134-input.c b/drivers/media/pci/saa7134/saa7134-input.c
index 7f6680de3156..b2a2372d149a 100644
--- a/drivers/media/pci/saa7134/saa7134-input.c
+++ b/drivers/media/pci/saa7134/saa7134-input.c
@@ -769,7 +769,8 @@ int saa7134_input_init1(struct saa7134_dev *dev)
}
ir = kzalloc_obj(*ir);
- rc = rc_allocate_device(RC_DRIVER_SCANCODE);
+ rc = rc_allocate_device(raw_decode ?
+ RC_DRIVER_IR_RAW : RC_DRIVER_SCANCODE);
if (!ir || !rc) {
err = -ENOMEM;
goto err_out_free;
@@ -792,10 +793,8 @@ int saa7134_input_init1(struct saa7134_dev *dev)
rc->priv = dev;
rc->open = saa7134_ir_open;
rc->close = saa7134_ir_close;
- if (raw_decode) {
- rc->driver_type = RC_DRIVER_IR_RAW;
+ if (raw_decode)
rc->allowed_protocols = RC_PROTO_BIT_ALL_IR_DECODER;
- }
rc->device_name = saa7134_boards[dev->board].name;
rc->input_phys = ir->phys;
diff --git a/drivers/media/pci/tw68/tw68-core.c b/drivers/media/pci/tw68/tw68-core.c
index 509d7ddec150..7abb1bcff55f 100644
--- a/drivers/media/pci/tw68/tw68-core.c
+++ b/drivers/media/pci/tw68/tw68-core.c
@@ -228,7 +228,7 @@ static int tw68_initdev(struct pci_dev *pci_dev,
/* pci init */
dev->pci = pci_dev;
- if (pci_enable_device(pci_dev)) {
+ if (pcim_enable_device(pci_dev)) {
err = -EIO;
goto fail1;
}
diff --git a/drivers/media/pci/tw686x/tw686x-core.c b/drivers/media/pci/tw686x/tw686x-core.c
index 5a4ab329c06e..1e3079bb8bb4 100644
--- a/drivers/media/pci/tw686x/tw686x-core.c
+++ b/drivers/media/pci/tw686x/tw686x-core.c
@@ -241,6 +241,7 @@ static int tw686x_probe(struct pci_dev *pci_dev,
const struct pci_device_id *pci_id)
{
struct tw686x_dev *dev;
+ unsigned long flags;
int err;
dev = kzalloc_obj(*dev);
@@ -352,6 +353,15 @@ static int tw686x_probe(struct pci_dev *pci_dev,
pci_release_regions(pci_dev);
disable_pci:
pci_disable_device(pci_dev);
+ if (dev->v4l2_dev.dev) {
+ spin_lock_irqsave(&dev->lock, flags);
+ dev->pci_dev = NULL;
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ v4l2_device_put(&dev->v4l2_dev);
+ return err;
+ }
+
free_audio:
kfree(dev->audio_channels);
free_video:
diff --git a/drivers/media/platform/amphion/vpu_core.c b/drivers/media/platform/amphion/vpu_core.c
index 85cc4a14f8ed..ad54f014d8f6 100644
--- a/drivers/media/platform/amphion/vpu_core.c
+++ b/drivers/media/platform/amphion/vpu_core.c
@@ -299,22 +299,23 @@ static void vpu_core_put_vpu(struct vpu_core *core)
core->vpu->put_vpu(core->vpu);
}
+static void vpu_core_cancel_work(struct vpu_core *core);
+
static int vpu_core_unregister(struct device *dev, struct vpu_core *core)
{
list_del_init(&core->list);
- vpu_core_put_vpu(core);
- core->vpu = NULL;
- kfree(core->msg_buffer);
- core->msg_buffer = NULL;
-
if (core->workqueue) {
- cancel_work_sync(&core->msg_work);
- cancel_delayed_work_sync(&core->msg_delayed_work);
+ vpu_core_cancel_work(core);
destroy_workqueue(core->workqueue);
core->workqueue = NULL;
}
+ vpu_core_put_vpu(core);
+ core->vpu = NULL;
+ kfree(core->msg_buffer);
+ core->msg_buffer = NULL;
+
return 0;
}
@@ -666,7 +667,6 @@ static int vpu_core_probe(struct platform_device *pdev)
pm_runtime_enable(dev);
ret = pm_runtime_resume_and_get(dev);
if (ret) {
- pm_runtime_put_noidle(dev);
pm_runtime_set_suspended(dev);
goto err_runtime_disable;
}
@@ -695,6 +695,8 @@ static void vpu_core_remove(struct platform_device *pdev)
struct vpu_core *core = platform_get_drvdata(pdev);
int ret;
+ WARN_ON(!list_empty(&core->instances));
+
vpu_core_remove_dbgfs_file(core);
ret = pm_runtime_resume_and_get(dev);
WARN_ON(ret < 0);
@@ -845,6 +847,7 @@ static struct platform_driver amphion_vpu_core_driver = {
.remove = vpu_core_remove,
.driver = {
.name = "amphion-vpu-core",
+ .suppress_bind_attrs = true,
.of_match_table = vpu_core_dt_match,
.pm = &vpu_core_pm_ops,
},
diff --git a/drivers/media/platform/amphion/vpu_drv.c b/drivers/media/platform/amphion/vpu_drv.c
index 2cca61f41bea..9c6f32c5eae9 100644
--- a/drivers/media/platform/amphion/vpu_drv.c
+++ b/drivers/media/platform/amphion/vpu_drv.c
@@ -205,6 +205,7 @@ static struct platform_driver amphion_vpu_driver = {
.remove = vpu_remove,
.driver = {
.name = "amphion-vpu",
+ .suppress_bind_attrs = true,
.of_match_table = vpu_dt_match,
},
};
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-core.c b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
index f28e9f4354ac..a14c6042345b 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-core.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-core.c
@@ -510,11 +510,16 @@ bool mali_c55_pipeline_ready(struct mali_c55 *mali_c55)
struct mali_c55_params *params = &mali_c55->params;
struct mali_c55_stats *stats = &mali_c55->stats;
- return vb2_start_streaming_called(&fr->queue) &&
+ /* Only wait for queues connected through enabled media links. */
+ return (!media_pad_remote_pad_first(&fr->pad) ||
+ vb2_start_streaming_called(&fr->queue)) &&
(!(mali_c55->capabilities & MALI_C55_GPS_DS_PIPE_FITTED) ||
+ !media_pad_remote_pad_first(&ds->pad) ||
vb2_start_streaming_called(&ds->queue)) &&
- vb2_start_streaming_called(¶ms->queue) &&
- vb2_start_streaming_called(&stats->queue);
+ (!media_pad_remote_pad_first(¶ms->pad) ||
+ vb2_start_streaming_called(¶ms->queue)) &&
+ (!media_pad_remote_pad_first(&stats->pad) ||
+ vb2_start_streaming_called(&stats->queue));
}
static int mali_c55_check_hwcfg(struct mali_c55 *mali_c55)
@@ -730,8 +735,13 @@ static int __mali_c55_power_on(struct mali_c55 *mali_c55)
/* Set safe stop to ensure we're in a non-streaming state */
mali_c55_write(mali_c55, MALI_C55_REG_INPUT_MODE_REQUEST,
MALI_C55_INPUT_SAFE_STOP);
- readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
- val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
+ ret = readl_poll_timeout(mali_c55->base + MALI_C55_REG_MODE_STATUS,
+ val, !val, 10 * USEC_PER_MSEC, 250 * USEC_PER_MSEC);
+ if (ret) {
+ dev_err(mali_c55->dev, "safe stop timed out\n");
+ __mali_c55_power_off(mali_c55);
+ return ret;
+ }
return 0;
}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
index e128adf6ee37..c45a3760e084 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
@@ -641,6 +641,7 @@ void mali_c55_unregister_isp(struct mali_c55 *mali_c55)
mutex_destroy(&isp->capture_lock);
v4l2_device_unregister_subdev(&isp->sd);
+ v4l2_ctrl_handler_free(&isp->handler);
v4l2_subdev_cleanup(&isp->sd);
media_entity_cleanup(&isp->sd.entity);
}
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-params.c b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
index 70106276b7e4..16e895e3488e 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-params.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-params.c
@@ -710,6 +710,9 @@ static int mali_c55_params_buf_prepare(struct vb2_buffer *vb)
struct mali_c55 *mali_c55 = params->mali_c55;
int ret;
+ if (!config)
+ return -EFAULT;
+
ret = v4l2_isp_params_validate_buffer_size(mali_c55->dev, vb,
v4l2_isp_buffer_size(MALI_C55_PARAMS_MAX_SIZE));
if (ret)
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
index 655e52a5f288..a9c28b090b2a 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-stats.c
@@ -200,9 +200,9 @@ static const struct vb2_ops mali_c55_stats_vb2_ops = {
.stop_streaming = mali_c55_stats_stop_streaming,
};
-static void mali_c55_stats_cpu_read(struct mali_c55_stats *stats,
- struct mali_c55_stats_buf *buf,
- enum mali_c55_config_spaces cfg_space)
+static int mali_c55_stats_cpu_read(struct mali_c55_stats *stats,
+ struct mali_c55_stats_buf *buf,
+ enum mali_c55_config_spaces cfg_space)
{
struct mali_c55 *mali_c55 = stats->mali_c55;
const void __iomem *src;
@@ -211,12 +211,18 @@ static void mali_c55_stats_cpu_read(struct mali_c55_stats *stats,
src = mali_c55->base + MALI_C55_REG_1024BIN_HIST;
dst = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
+
+ if (!dst)
+ return -EFAULT;
+
memcpy_fromio(dst, src, MALI_C55_1024BIN_HIST_SIZE);
src = mali_c55->base + metering_space_addrs[cfg_space];
dst += MALI_C55_1024BIN_HIST_SIZE;
length = sizeof(struct mali_c55_stats_buffer) - MALI_C55_1024BIN_HIST_SIZE;
memcpy_fromio(dst, src, length);
+
+ return 0;
}
void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
@@ -224,6 +230,7 @@ void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
{
struct mali_c55_stats *stats = &mali_c55->stats;
struct mali_c55_stats_buf *buf = NULL;
+ int ret;
spin_lock(&stats->buffers.lock);
if (!list_empty(&stats->buffers.queue)) {
@@ -239,8 +246,9 @@ void mali_c55_stats_fill_buffer(struct mali_c55 *mali_c55,
buf->vb.sequence = mali_c55->isp.frame_sequence;
buf->vb.vb2_buf.timestamp = ktime_get_boottime_ns();
- mali_c55_stats_cpu_read(stats, buf, cfg_space);
- vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+ ret = mali_c55_stats_cpu_read(stats, buf, cfg_space);
+ vb2_buffer_done(&buf->vb.vb2_buf,
+ ret ? VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
}
void mali_c55_unregister_stats(struct mali_c55 *mali_c55)
diff --git a/drivers/media/platform/atmel/atmel-isi.c b/drivers/media/platform/atmel/atmel-isi.c
index a05a744cbb75..6a7e240c5b18 100644
--- a/drivers/media/platform/atmel/atmel-isi.c
+++ b/drivers/media/platform/atmel/atmel-isi.c
@@ -1121,10 +1121,13 @@ static void isi_graph_notify_unbind(struct v4l2_async_notifier *notifier,
{
struct atmel_isi *isi = notifier_to_isi(notifier);
+ if (!video_is_registered(isi->vdev))
+ return;
+
dev_dbg(isi->dev, "Removing %s\n", video_device_node_name(isi->vdev));
- /* Checks internally if vdev have been init or not */
video_unregister_device(isi->vdev);
+ isi->vdev = NULL;
}
static int isi_graph_notify_bound(struct v4l2_async_notifier *notifier,
@@ -1323,6 +1326,8 @@ static void atmel_isi_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
v4l2_async_nf_unregister(&isi->notifier);
v4l2_async_nf_cleanup(&isi->notifier);
+ if (isi->vdev)
+ video_device_release(isi->vdev);
v4l2_device_unregister(&isi->v4l2_dev);
}
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
index 6564cf3ec739..86bc36ee17fd 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
@@ -1302,8 +1302,14 @@ static void wave5_vpu_dec_buf_queue_dst(struct vb2_buffer *vb)
struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
struct vpu_instance *inst = vb2_get_drv_priv(vb->vb2_queue);
struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
+ int ret;
+
+ ret = pm_runtime_resume_and_get(inst->dev->dev);
+ if (ret < 0) {
+ vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
+ return;
+ }
- pm_runtime_resume_and_get(inst->dev->dev);
vbuf->sequence = inst->queued_dst_buf_num++;
if (inst->state == VPU_INST_STATE_PIC_RUN) {
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.c b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
index 76d57c6b636a..37df270b5f99 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
@@ -340,6 +340,13 @@ static int wave5_vpu_probe(struct platform_device *pdev)
dev_err(&pdev->dev, "failed to get irq resource, falling back to polling\n");
sema_init(&dev->irq_sem, 1);
dev->irq_thread = kthread_run(irq_thread, dev, "irq thread");
+ if (IS_ERR(dev->irq_thread)) {
+ ret = PTR_ERR(dev->irq_thread);
+ dev_err(&pdev->dev, "failed to create irq thread\n");
+ dev->irq_thread = NULL;
+ goto err_vdi_release;
+ }
+
hrtimer_setup(&dev->hrtimer, &wave5_vpu_timer_callback, CLOCK_MONOTONIC,
HRTIMER_MODE_REL_PINNED);
dev->worker = kthread_run_worker(0, "vpu_irq_thread");
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
index f77abd5e122a..4c1842f02c88 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
@@ -754,7 +754,7 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
if (ret) {
- pm_runtime_resume_and_get(inst->dev->dev);
+ pm_runtime_put_sync(inst->dev->dev);
return ret;
}
@@ -762,16 +762,13 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
ret = wave5_vpu_enc_finish_seq(inst, fail_res);
if (ret < 0 && *fail_res != WAVE5_SYSERR_VPU_STILL_RUNNING) {
dev_warn(inst->dev->dev, "enc_finish_seq timed out\n");
- pm_runtime_resume_and_get(inst->dev->dev);
- mutex_unlock(&vpu_dev->hw_lock);
- return ret;
+ goto unlock_and_return;
}
if (*fail_res == WAVE5_SYSERR_VPU_STILL_RUNNING &&
retry++ >= MAX_FIRMWARE_CALL_RETRY) {
- pm_runtime_resume_and_get(inst->dev->dev);
- mutex_unlock(&vpu_dev->hw_lock);
- return -ETIMEDOUT;
+ ret = -ETIMEDOUT;
+ goto unlock_and_return;
}
} while (ret != 0);
@@ -787,10 +784,12 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
}
wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
+
+unlock_and_return:
mutex_unlock(&vpu_dev->hw_lock);
pm_runtime_put_sync(inst->dev->dev);
- return 0;
+ return ret;
}
int wave5_vpu_enc_register_frame_buffer(struct vpu_instance *inst, unsigned int num,
diff --git a/drivers/media/platform/imagination/e5010-jpeg-enc.c b/drivers/media/platform/imagination/e5010-jpeg-enc.c
index 42ad9ee3993b..420cb8bfdb12 100644
--- a/drivers/media/platform/imagination/e5010-jpeg-enc.c
+++ b/drivers/media/platform/imagination/e5010-jpeg-enc.c
@@ -175,12 +175,12 @@ static void calculate_qp_tables(struct e5010_context *ctx)
long long delta = v4l2_jpeg_ref_table_chroma_qt[i] * contrast + luminosity;
int val = (int)(v4l2_jpeg_ref_table_chroma_qt[i] + delta);
- clamp(val, 1, 255);
+ val = clamp(val, 1, 255);
ctx->chroma_qp[i] = quality == -50 ? 1 : val;
delta = v4l2_jpeg_ref_table_luma_qt[i] * contrast + luminosity;
val = (int)(v4l2_jpeg_ref_table_luma_qt[i] + delta);
- clamp(val, 1, 255);
+ val = clamp(val, 1, 255);
ctx->luma_qp[i] = quality == -50 ? 1 : val;
}
diff --git a/drivers/media/platform/m2m-deinterlace.c b/drivers/media/platform/m2m-deinterlace.c
index 1d0e70eaea3d..9dcc4bd6cbdd 100644
--- a/drivers/media/platform/m2m-deinterlace.c
+++ b/drivers/media/platform/m2m-deinterlace.c
@@ -822,7 +822,7 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,
q_data[V4L2_M2M_DST].width = 640;
q_data[V4L2_M2M_DST].height = 480;
q_data[V4L2_M2M_DST].sizeimage = (640 * 480 * 3) / 2;
- q_data[V4L2_M2M_SRC].field = V4L2_FIELD_INTERLACED_TB;
+ q_data[V4L2_M2M_DST].field = V4L2_FIELD_INTERLACED_TB;
return vb2_queue_init(dst_vq);
}
diff --git a/drivers/media/platform/marvell/Kconfig b/drivers/media/platform/marvell/Kconfig
index d31f4730f2a3..ab58175e433c 100644
--- a/drivers/media/platform/marvell/Kconfig
+++ b/drivers/media/platform/marvell/Kconfig
@@ -22,6 +22,7 @@ config VIDEO_MMP_CAMERA
depends on V4L_PLATFORM_DRIVERS
depends on I2C && VIDEO_DEV
depends on ARCH_MMP || COMPILE_TEST
+ depends on GPIOLIB || COMPILE_TEST
depends on COMMON_CLK
select VIDEO_OV7670 if MEDIA_SUBDRV_AUTOSELECT && VIDEO_CAMERA_SENSOR
select I2C_GPIO
diff --git a/drivers/media/platform/marvell/mmp-driver.c b/drivers/media/platform/marvell/mmp-driver.c
index d3da7ebb4a2b..4c31b62751a1 100644
--- a/drivers/media/platform/marvell/mmp-driver.c
+++ b/drivers/media/platform/marvell/mmp-driver.c
@@ -25,6 +25,7 @@
#include <linux/list.h>
#include <linux/pm.h>
#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include "mcam-core.h"
@@ -269,8 +270,8 @@ static int mmpcam_probe(struct platform_device *pdev)
/*
* Add OF clock provider.
*/
- ret = of_clk_add_provider(pdev->dev.of_node, of_clk_src_simple_get,
- mcam->mclk);
+ ret = devm_of_clk_add_hw_provider(&pdev->dev, of_clk_hw_simple_get,
+ &mcam->mclk_hw);
if (ret) {
dev_err(&pdev->dev, "can't add DT clock provider\n");
goto out;
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index d147ec483081..cc375309e4f3 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1574,7 +1574,7 @@ static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
@@ -1623,7 +1623,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
v4l2_m2m_buf_copy_metadata(src_buf, dst_buf);
mtk_jpegenc_set_hw_param(ctx, hw_id, src_buf, dst_buf);
- ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
+ ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
if (ret < 0) {
dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail !!!\n",
__func__, __LINE__);
@@ -1634,6 +1634,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
if (ret) {
dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
__func__, __LINE__);
+ pm_runtime_put(comp_jpeg[hw_id]->dev);
goto enc_end;
}
@@ -1697,9 +1698,15 @@ static void mtk_jpegdec_worker(struct work_struct *work)
ret = wait_event_interruptible_timeout(jpeg->hw_wq,
atomic_read(&jpeg->hw_rdy) > 0,
MTK_JPEG_HW_TIMEOUT_MSEC);
- if (ret != 0 || (i++ > MTK_JPEG_MAX_RETRY_TIME)) {
- dev_err(jpeg->dev, "%s : %d, all HW are busy\n",
- __func__, __LINE__);
+ if (ret < 0) {
+ dev_err(jpeg->dev, "decoder HW wait interrupted: %d\n",
+ ret);
+ v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
+ return;
+ }
+
+ if (!ret && i++ > MTK_JPEG_MAX_RETRY_TIME) {
+ dev_err(jpeg->dev, "all decoder HW are busy\n");
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
return;
}
@@ -1733,7 +1740,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
jpeg_dst_buf->frame_num = ctx->total_frame_num;
mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
- ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
+ ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
if (ret < 0) {
dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail !!!\n",
__func__, __LINE__);
@@ -1855,7 +1862,7 @@ static irqreturn_t mtk_jpeg_dec_irq(int irq, void *priv)
v4l2_m2m_buf_done(src_buf, buf_state);
v4l2_m2m_buf_done(dst_buf, buf_state);
v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
- pm_runtime_put(ctx->jpeg->dev);
+ pm_runtime_put(jpeg->dev);
return IRQ_HANDLED;
}
diff --git a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-comp.c b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-comp.c
index 7fcb2fbdd64e..acb2758a7b42 100644
--- a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-comp.c
+++ b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-comp.c
@@ -1730,6 +1730,7 @@ static int mdp_comp_init(struct mdp_dev *mdp, struct device_node *node,
struct device *dev = &mdp->pdev->dev;
struct platform_device *pdev_c;
int clk_ofst;
+ int ret;
int i;
s32 event;
@@ -1756,8 +1757,10 @@ static int mdp_comp_init(struct mdp_dev *mdp, struct device_node *node,
comp->clk_num = mdp->mdp_data->comp_data[id].info.clk_num;
comp->clks = devm_kzalloc(dev, sizeof(struct clk *) * comp->clk_num,
GFP_KERNEL);
- if (!comp->clks)
- return -ENOMEM;
+ if (!comp->clks) {
+ ret = -ENOMEM;
+ goto err_put_device;
+ }
clk_ofst = mdp->mdp_data->comp_data[id].info.clk_ofst;
@@ -1782,7 +1785,8 @@ static int mdp_comp_init(struct mdp_dev *mdp, struct device_node *node,
if (of_property_read_u32_index(node, "mediatek,gce-events",
MDP_GCE_EVENT_EOF, &event)) {
dev_err(dev, "Component id %d has no EOF\n", id);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_put_device;
}
} else {
event = MDP_GCE_NO_EVENT;
@@ -1791,6 +1795,10 @@ static int mdp_comp_init(struct mdp_dev *mdp, struct device_node *node,
comp->gce_event[MDP_GCE_EVENT_EOF] = event;
return 0;
+
+err_put_device:
+ put_device(&pdev_c->dev);
+ return ret;
}
static void mdp_comp_deinit(struct mdp_comp *comp)
diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.c
index e936ed8dffba..426d060d937c 100644
--- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.c
@@ -570,6 +570,8 @@ static void mtk_vcodec_dec_remove(struct platform_device *pdev)
struct mtk_vcodec_dec_dev *dev = platform_get_drvdata(pdev);
destroy_workqueue(dev->decode_workqueue);
+ if (IS_VDEC_LAT_ARCH(dev->vdec_pdata->hw_arch))
+ destroy_workqueue(dev->core_workqueue);
if (media_devnode_is_registered(dev->mdev_dec.devnode)) {
media_device_unregister(&dev->mdev_dec);
diff --git a/drivers/media/platform/mediatek/vpu/mtk_vpu.c b/drivers/media/platform/mediatek/vpu/mtk_vpu.c
index 8d8319f0cd22..9dff1d971d37 100644
--- a/drivers/media/platform/mediatek/vpu/mtk_vpu.c
+++ b/drivers/media/platform/mediatek/vpu/mtk_vpu.c
@@ -962,10 +962,12 @@ static void mtk_vpu_remove(struct platform_device *pdev)
#ifdef CONFIG_DEBUG_FS
debugfs_remove(vpu_debugfs);
#endif
+ devm_free_irq(&pdev->dev, vpu->reg.irq, vpu);
if (vpu->wdt.wq)
destroy_workqueue(vpu->wdt.wq);
vpu_free_ext_mem(vpu, P_FW);
vpu_free_ext_mem(vpu, D_FW);
+ of_reserved_mem_device_release(&pdev->dev);
mutex_destroy(&vpu->vpu_mutex);
clk_unprepare(vpu->clk);
}
diff --git a/drivers/media/platform/microchip/microchip-isc-base.c b/drivers/media/platform/microchip/microchip-isc-base.c
index 32bdeae04ad0..21d108eb93c7 100644
--- a/drivers/media/platform/microchip/microchip-isc-base.c
+++ b/drivers/media/platform/microchip/microchip-isc-base.c
@@ -289,6 +289,7 @@ static int isc_configure(struct isc_device *isc)
struct regmap *regmap = isc->regmap;
u32 pfe_cfg0, dcfg, mask, pipeline;
struct isc_subdev_entity *subdev = isc->current_subdev;
+ int ret;
pfe_cfg0 = isc->config.sd_format->pfe_cfg0_bps;
pipeline = isc->config.bits_pipeline;
@@ -321,7 +322,15 @@ static int isc_configure(struct isc_device *isc)
isc_set_histogram(isc, false);
/* Update profile */
- return isc_update_profile(isc);
+ ret = isc_update_profile(isc);
+ if (ret) {
+ /* flush the histogram work before the clocks are gated */
+ isc_set_histogram(isc, false);
+ synchronize_irq(isc->irq);
+ cancel_work_sync(&isc->awb_work);
+ }
+
+ return ret;
}
static int isc_prepare_streaming(struct vb2_queue *vq)
@@ -425,6 +434,13 @@ static void isc_stop_streaming(struct vb2_queue *vq)
/* Disable DMA interrupt */
regmap_write(isc->regmap, ISC_INTDIS, ISC_INT_DDONE);
+ isc_set_histogram(isc, false);
+
+ /* let a running IRQ handler finish before the clock is disabled */
+ synchronize_irq(isc->irq);
+
+ cancel_work_sync(&isc->awb_work);
+
pm_runtime_put_sync(isc->dev);
/* Disable stream on the sub device */
@@ -1416,7 +1432,7 @@ static void isc_awb_work(struct work_struct *w)
/* streaming is not active anymore */
if (isc->stop) {
mutex_unlock(&isc->awb_mutex);
- return;
+ goto out_pm_put;
}
isc_update_profile(isc);
@@ -1427,6 +1443,7 @@ static void isc_awb_work(struct work_struct *w)
if (ctrls->awb)
regmap_write(regmap, ISC_CTRLEN, ISC_CTRL_HISREQ);
+out_pm_put:
pm_runtime_put_sync(isc->dev);
}
@@ -1495,20 +1512,24 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
if (ctrl->cluster[ISC_CTRL_GB_OFF]->is_new)
ctrls->offset[ISC_HIS_CFG_MODE_GB] = isc->gb_off_ctrl->val;
- isc_update_awb_ctrls(isc);
-
mutex_lock(&isc->awb_mutex);
- if (vb2_is_streaming(&isc->vb2_vidq)) {
+ if (vb2_is_streaming(&isc->vb2_vidq) && !isc->stop) {
+ unsigned long flags;
+
/*
- * If we are streaming, we can update profile to
- * have the new settings in place.
+ * awb_lock keeps the DMA done IRQ from latching a
+ * partially written WB pipeline.
*/
+ spin_lock_irqsave(&isc->awb_lock, flags);
+ isc_update_awb_ctrls(isc);
+ spin_unlock_irqrestore(&isc->awb_lock, flags);
+
isc_update_profile(isc);
} else {
/*
- * The auto cluster will activate automatically this
- * control. This has to be deactivated when not
- * streaming.
+ * Not streaming: keep the cached values for the next
+ * stream start and deactivate the cluster-activated
+ * do_white_balance button.
*/
v4l2_ctrl_activate(isc->do_wb_ctrl, false);
}
@@ -1703,7 +1724,6 @@ static void isc_async_unbind(struct v4l2_async_notifier *notifier,
{
struct isc_device *isc = container_of(notifier->v4l2_dev,
struct isc_device, v4l2_dev);
- mutex_destroy(&isc->awb_mutex);
cancel_work_sync(&isc->awb_work);
video_unregister_device(&isc->video_dev);
v4l2_ctrl_handler_free(&isc->ctrls.handler);
@@ -1767,8 +1787,6 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
isc->current_subdev = container_of(notifier,
struct isc_subdev_entity, notifier);
- mutex_init(&isc->lock);
- mutex_init(&isc->awb_mutex);
init_completion(&isc->comp);
@@ -1787,7 +1805,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
ret = vb2_queue_init(q);
if (ret < 0) {
dev_err(isc->dev, "vb2_queue_init() failed: %d\n", ret);
- goto isc_async_complete_err;
+ return ret;
}
/* Init video dma queues */
@@ -1798,13 +1816,13 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
ret = isc_set_default_fmt(isc);
if (ret) {
dev_err(isc->dev, "Could not set default format\n");
- goto isc_async_complete_err;
+ return ret;
}
ret = isc_ctrl_init(isc);
if (ret) {
dev_err(isc->dev, "Init isc ctrols failed: %d\n", ret);
- goto isc_async_complete_err;
+ return ret;
}
/* Register video device */
@@ -1824,7 +1842,7 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret < 0) {
dev_err(isc->dev, "video_register_device failed: %d\n", ret);
- goto isc_async_complete_err;
+ return ret;
}
ret = isc_scaler_link(isc);
@@ -1839,10 +1857,6 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
isc_async_complete_unregister_device:
video_unregister_device(vdev);
-
-isc_async_complete_err:
- mutex_destroy(&isc->awb_mutex);
- mutex_destroy(&isc->lock);
return ret;
}
diff --git a/drivers/media/platform/microchip/microchip-isc-clk.c b/drivers/media/platform/microchip/microchip-isc-clk.c
index 24358d804e75..66dc522a6190 100644
--- a/drivers/media/platform/microchip/microchip-isc-clk.c
+++ b/drivers/media/platform/microchip/microchip-isc-clk.c
@@ -98,15 +98,14 @@ static int isc_clk_is_enabled(struct clk_hw *hw)
{
struct isc_clk *isc_clk = to_isc_clk(hw);
u32 status;
- int ret;
- ret = pm_runtime_resume_and_get(isc_clk->dev);
- if (ret < 0)
+ /* Runs in atomic context, so must not sleep to resume the ISC. */
+ if (pm_runtime_get_if_active(isc_clk->dev) <= 0)
return 0;
regmap_read(isc_clk->regmap, ISC_CLKSR, &status);
- pm_runtime_put_sync(isc_clk->dev);
+ pm_runtime_put(isc_clk->dev);
return status & ISC_CLK(isc_clk->id) ? 1 : 0;
}
diff --git a/drivers/media/platform/microchip/microchip-isc.h b/drivers/media/platform/microchip/microchip-isc.h
index ad4e98a1dd8f..f5e322c2e36b 100644
--- a/drivers/media/platform/microchip/microchip-isc.h
+++ b/drivers/media/platform/microchip/microchip-isc.h
@@ -287,6 +287,7 @@ struct isc_device {
u32 dcfg;
struct device *dev;
+ int irq;
struct v4l2_device v4l2_dev;
struct video_device video_dev;
diff --git a/drivers/media/platform/microchip/microchip-sama5d2-isc.c b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
index 88e6d8e1c6c9..5bd109e93a8f 100644
--- a/drivers/media/platform/microchip/microchip-sama5d2-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama5d2-isc.c
@@ -30,6 +30,7 @@
#include <linux/interrupt.h>
#include <linux/math64.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/of.h>
#include <linux/of_graph.h>
#include <linux/platform_device.h>
@@ -424,6 +425,14 @@ static int microchip_isc_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, isc);
isc->dev = dev;
+ ret = devm_mutex_init(dev, &isc->lock);
+ if (ret)
+ return ret;
+
+ ret = devm_mutex_init(dev, &isc->awb_mutex);
+ if (ret)
+ return ret;
+
io_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(io_base))
return PTR_ERR(io_base);
@@ -439,6 +448,8 @@ static int microchip_isc_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
+ isc->irq = irq;
+
ret = devm_request_irq(dev, irq, microchip_isc_interrupt, 0,
"microchip-sama5d2-isc", isc);
if (ret < 0) {
diff --git a/drivers/media/platform/microchip/microchip-sama7g5-isc.c b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
index 77c6c183451b..d978ce448a4d 100644
--- a/drivers/media/platform/microchip/microchip-sama7g5-isc.c
+++ b/drivers/media/platform/microchip/microchip-sama7g5-isc.c
@@ -33,6 +33,7 @@
#include <linux/interrupt.h>
#include <linux/math64.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/of.h>
#include <linux/of_graph.h>
#include <linux/platform_device.h>
@@ -413,6 +414,14 @@ static int microchip_xisc_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, isc);
isc->dev = dev;
+ ret = devm_mutex_init(dev, &isc->lock);
+ if (ret)
+ return ret;
+
+ ret = devm_mutex_init(dev, &isc->awb_mutex);
+ if (ret)
+ return ret;
+
io_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(io_base))
return PTR_ERR(io_base);
@@ -428,6 +437,8 @@ static int microchip_xisc_probe(struct platform_device *pdev)
if (irq < 0)
return irq;
+ isc->irq = irq;
+
ret = devm_request_irq(dev, irq, microchip_isc_interrupt, 0,
"microchip-sama7g5-xisc", isc);
if (ret < 0) {
diff --git a/drivers/media/platform/nuvoton/npcm-video.c b/drivers/media/platform/nuvoton/npcm-video.c
index 52505af35c08..c28d9d7edd83 100644
--- a/drivers/media/platform/nuvoton/npcm-video.c
+++ b/drivers/media/platform/nuvoton/npcm-video.c
@@ -120,6 +120,7 @@ struct npcm_video {
struct list_head buffers;
struct mutex buffer_lock; /* buffer list lock */
+ int irq;
unsigned long flags;
unsigned int sequence;
@@ -1486,6 +1487,7 @@ static int npcm_video_start_streaming(struct vb2_queue *q, unsigned int count)
}
set_bit(VIDEO_STREAMING, &video->flags);
+ enable_irq(video->irq);
return 0;
}
@@ -1494,6 +1496,7 @@ static void npcm_video_stop_streaming(struct vb2_queue *q)
struct npcm_video *video = vb2_get_drv_priv(q);
struct regmap *vcd = video->vcd_regmap;
+ disable_irq(video->irq);
clear_bit(VIDEO_STREAMING, &video->flags);
regmap_write(vcd, VCD_INTE, 0);
regmap_write(vcd, VCD_STAT, VCD_STAT_CLEAR);
@@ -1707,9 +1710,10 @@ static int npcm_video_init(struct npcm_video *video)
dev_err(dev, "Failed to find VCD IRQ\n");
return -ENODEV;
}
+ video->irq = irq;
rc = devm_request_threaded_irq(dev, irq, NULL, npcm_video_irq,
- IRQF_ONESHOT, DEVICE_NAME, video);
+ IRQF_ONESHOT | IRQF_NO_AUTOEN, DEVICE_NAME, video);
if (rc < 0) {
dev_err(dev, "Failed to request IRQ %d\n", irq);
return rc;
@@ -1807,8 +1811,7 @@ static void npcm_video_remove(struct platform_device *pdev)
struct v4l2_device *v4l2_dev = dev_get_drvdata(dev);
struct npcm_video *video = to_npcm_video(v4l2_dev);
- video_unregister_device(&video->vdev);
- vb2_queue_release(&video->queue);
+ vb2_video_unregister_device(&video->vdev);
v4l2_ctrl_handler_free(&video->ctrl_handler);
v4l2_device_unregister(v4l2_dev);
if (video->ece.enable)
diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c
index ea2fad5dd646..e17085935435 100644
--- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c
+++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c
@@ -1590,15 +1590,15 @@ static void mxc_jpeg_device_run(void *priv)
dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
if (!src_buf || !dst_buf) {
dev_err(dev, "Null src or dst buf\n");
- goto end;
+ goto job_finish;
}
q_data_cap = mxc_jpeg_get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
if (!q_data_cap)
- goto end;
+ goto buf_finish;
q_data_out = mxc_jpeg_get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
if (!q_data_out)
- goto end;
+ goto buf_finish;
src_buf->sequence = q_data_out->sequence++;
dst_buf->sequence = q_data_cap->sequence++;
@@ -1636,11 +1636,11 @@ static void mxc_jpeg_device_run(void *priv)
ctx->slot = mxc_get_free_slot(&jpeg->slot_data);
if (ctx->slot < 0) {
dev_err(dev, "No more free slots\n");
- goto end;
+ goto buf_finish;
}
if (!mxc_jpeg_alloc_slot_data(jpeg)) {
dev_err(dev, "Cannot allocate slot data\n");
- goto end;
+ goto buf_finish;
}
mxc_jpeg_enable_slot(reg, ctx->slot);
@@ -1660,8 +1660,17 @@ static void mxc_jpeg_device_run(void *priv)
mxc_jpeg_dec_mode_go(dev, reg);
}
schedule_delayed_work(&ctx->task_timer, msecs_to_jiffies(hw_timeout));
-end:
+
spin_unlock_irqrestore(&ctx->mxc_jpeg->hw_lock, flags);
+ return;
+buf_finish:
+ v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+job_finish:
+ spin_unlock_irqrestore(&ctx->mxc_jpeg->hw_lock, flags);
+ v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
}
static int mxc_jpeg_decoder_cmd(struct file *file, void *priv,
diff --git a/drivers/media/platform/qcom/camss/camss-vfe-17x.c b/drivers/media/platform/qcom/camss/camss-vfe-17x.c
index e5ee7e717b3b..62c2405359a0 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe-17x.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe-17x.c
@@ -364,9 +364,8 @@ static irqreturn_t vfe_isr(int irq, void *dev)
vfe->isr_ops.comp_done(vfe, i);
for (wm = 0; wm < MSM_VFE_IMAGE_MASTERS_NUM; wm++)
- if (status0 & BIT(9))
- if (vfe_bus_status[1] & STATUS1_WM_CLIENT_BUF_DONE(wm))
- vfe->isr_ops.wm_done(vfe, wm);
+ if (vfe_bus_status[1] & STATUS1_WM_CLIENT_BUF_DONE(wm))
+ vfe->isr_ops.wm_done(vfe, wm);
return IRQ_HANDLED;
}
@@ -383,45 +382,6 @@ static int vfe_halt(struct vfe_device *vfe)
return 0;
}
-static int vfe_get_output(struct vfe_line *line)
-{
- struct vfe_device *vfe = to_vfe(line);
- struct vfe_output *output;
- unsigned long flags;
- int wm_idx;
-
- spin_lock_irqsave(&vfe->output_lock, flags);
-
- output = &line->output;
- if (output->state > VFE_OUTPUT_RESERVED) {
- dev_err(vfe->camss->dev, "Output is running\n");
- goto error;
- }
-
- output->wm_num = 1;
-
- wm_idx = vfe_reserve_wm(vfe, line->id);
- if (wm_idx < 0) {
- dev_err(vfe->camss->dev, "Can not reserve wm\n");
- goto error_get_wm;
- }
- output->wm_idx[0] = wm_idx;
-
- output->drop_update_idx = 0;
-
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return 0;
-
-error_get_wm:
- vfe_release_wm(vfe, output->wm_idx[0]);
- output->state = VFE_OUTPUT_OFF;
-error:
- spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return -EINVAL;
-}
-
/*
* vfe_enable - Enable streaming on VFE line
* @line: VFE line
@@ -442,7 +402,7 @@ static int vfe_enable(struct vfe_line *line)
mutex_unlock(&vfe->stream_lock);
- ret = vfe_get_output(line);
+ ret = vfe_get_output_v2(line);
if (ret < 0)
goto error_get_output;
diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c
index 319d19158988..55c0a0ee398f 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe.c
@@ -808,7 +808,7 @@ int vfe_put_output(struct vfe_line *line)
return 0;
}
-static int vfe_disable_output(struct vfe_line *line)
+static void vfe_disable_output(struct vfe_line *line)
{
struct vfe_device *vfe = to_vfe(line);
struct vfe_output *output = &line->output;
@@ -820,8 +820,6 @@ static int vfe_disable_output(struct vfe_line *line)
vfe->res->hw_ops->vfe_wm_stop(vfe, output->wm_idx[i]);
output->gen2.active_num = 0;
spin_unlock_irqrestore(&vfe->output_lock, flags);
-
- return vfe_reset(vfe);
}
/*
@@ -833,21 +831,19 @@ static int vfe_disable_output(struct vfe_line *line)
int vfe_disable(struct vfe_line *line)
{
struct vfe_device *vfe = to_vfe(line);
- int ret;
+ int ret = 0;
- ret = vfe_disable_output(line);
- if (ret)
- goto error;
+ vfe_disable_output(line);
vfe_put_output(line);
mutex_lock(&vfe->stream_lock);
- vfe->stream_count--;
+ if (--vfe->stream_count == 0)
+ ret = vfe_reset(vfe);
mutex_unlock(&vfe->stream_lock);
-error:
return ret;
}
diff --git a/drivers/media/platform/qcom/camss/camss.c b/drivers/media/platform/qcom/camss/camss.c
index 2123f6388e3d..68491b16d647 100644
--- a/drivers/media/platform/qcom/camss/camss.c
+++ b/drivers/media/platform/qcom/camss/camss.c
@@ -4037,10 +4037,11 @@ static const struct camss_subdev_resources vfe_res_8775p[] = {
/* VFE2 (lite) */
{
.regulators = {},
- .clock = { "cpas_ahb", "cpas_vfe_lite", "vfe_lite_ahb",
- "vfe_lite_csid", "vfe_lite_cphy_rx",
+ .clock = { "cpas_ahb", "gcc_axi_hf", "cpas_vfe_lite",
+ "vfe_lite_ahb", "vfe_lite_csid", "vfe_lite_cphy_rx",
"vfe_lite", "camnoc_axi"},
.clock_rate = {
+ { 0 },
{ 0 },
{ 0 },
{ 300000000, 400000000, 400000000, 400000000 },
@@ -4062,10 +4063,11 @@ static const struct camss_subdev_resources vfe_res_8775p[] = {
/* VFE3 (lite) */
{
.regulators = {},
- .clock = { "cpas_ahb", "cpas_vfe_lite", "vfe_lite_ahb",
- "vfe_lite_csid", "vfe_lite_cphy_rx",
+ .clock = { "cpas_ahb", "gcc_axi_hf", "cpas_vfe_lite",
+ "vfe_lite_ahb", "vfe_lite_csid", "vfe_lite_cphy_rx",
"vfe_lite", "camnoc_axi"},
.clock_rate = {
+ { 0 },
{ 0 },
{ 0 },
{ 300000000, 400000000, 400000000, 400000000 },
@@ -4087,10 +4089,11 @@ static const struct camss_subdev_resources vfe_res_8775p[] = {
/* VFE4 (lite) */
{
.regulators = {},
- .clock = { "cpas_ahb", "cpas_vfe_lite", "vfe_lite_ahb",
- "vfe_lite_csid", "vfe_lite_cphy_rx",
+ .clock = { "cpas_ahb", "gcc_axi_hf", "cpas_vfe_lite",
+ "vfe_lite_ahb", "vfe_lite_csid", "vfe_lite_cphy_rx",
"vfe_lite", "camnoc_axi"},
.clock_rate = {
+ { 0 },
{ 0 },
{ 0 },
{ 300000000, 400000000, 400000000, 400000000 },
@@ -4112,10 +4115,11 @@ static const struct camss_subdev_resources vfe_res_8775p[] = {
/* VFE5 (lite) */
{
.regulators = {},
- .clock = { "cpas_ahb", "cpas_vfe_lite", "vfe_lite_ahb",
- "vfe_lite_csid", "vfe_lite_cphy_rx",
+ .clock = { "cpas_ahb", "gcc_axi_hf", "cpas_vfe_lite",
+ "vfe_lite_ahb", "vfe_lite_csid", "vfe_lite_cphy_rx",
"vfe_lite", "camnoc_axi"},
.clock_rate = {
+ { 0 },
{ 0 },
{ 0 },
{ 300000000, 400000000, 400000000, 400000000 },
@@ -4137,10 +4141,11 @@ static const struct camss_subdev_resources vfe_res_8775p[] = {
/* VFE6 (lite) */
{
.regulators = {},
- .clock = { "cpas_ahb", "cpas_vfe_lite", "vfe_lite_ahb",
- "vfe_lite_csid", "vfe_lite_cphy_rx",
+ .clock = { "cpas_ahb", "gcc_axi_hf", "cpas_vfe_lite",
+ "vfe_lite_ahb", "vfe_lite_csid", "vfe_lite_cphy_rx",
"vfe_lite", "camnoc_axi"},
.clock_rate = {
+ { 0 },
{ 0 },
{ 0 },
{ 300000000, 400000000, 400000000, 400000000 },
diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h
index 24da60448cf2..3e4cf652bf39 100644
--- a/drivers/media/platform/qcom/iris/iris_core.h
+++ b/drivers/media/platform/qcom/iris/iris_core.h
@@ -7,6 +7,7 @@
#define __IRIS_CORE_H__
#include <linux/types.h>
+#include <linux/dma-mapping.h>
#include <linux/pm_domain.h>
#include <media/v4l2-device.h>
@@ -25,6 +26,9 @@ struct icc_info {
#define IRIS_FW_VERSION_LENGTH 128
#define IFACEQ_CORE_PKT_SIZE (1024 * 4)
+#define IRIS_NP_RESERVE_IOVA_START 0x0
+#define IRIS_NP_RESERVE_IOVA_SIZE 0x25800000
+
enum domain_type {
ENCODER = BIT(0),
DECODER = BIT(1),
@@ -77,6 +81,7 @@ struct qcom_ubwc_cfg_data;
* @instances: a list_head of all instances
* @inst_fw_caps_dec: an array of supported instance capabilities by decoder
* @inst_fw_caps_enc: an array of supported instance capabilities by encoder
+ * @iova_state: a pointer to an array of dma_iova_state entries reserved for restricted IOVA region
*/
struct iris_core {
@@ -123,6 +128,7 @@ struct iris_core {
/* encoder and decoder have overlapping caps, so two different arrays are required */
struct platform_inst_fw_cap inst_fw_caps_dec[INST_FW_CAP_MAX];
struct platform_inst_fw_cap inst_fw_caps_enc[INST_FW_CAP_MAX];
+ struct dma_iova_state *iova_state;
};
int iris_core_init(struct iris_core *core);
diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c
index e4acf4a74f94..6581a969fe3f 100644
--- a/drivers/media/platform/qcom/iris/iris_probe.c
+++ b/drivers/media/platform/qcom/iris/iris_probe.c
@@ -150,6 +150,57 @@ static int iris_init_resources(struct iris_core *core)
return iris_init_resets(core);
}
+static void iris_unreserve_iova_region(struct device *dev, struct dma_iova_state *iova_state)
+{
+ unsigned int i;
+
+ for (i = 0; dma_iova_size(&iova_state[i]); i++)
+ dma_iova_free(dev, &iova_state[i]);
+}
+
+static int iris_reserve_iova_region(struct device *dev, struct dma_iova_state **iova_state,
+ unsigned long start, unsigned long size)
+{
+ unsigned long dma_limit = dev->bus_dma_limit;
+ unsigned long end, rem, chunk;
+ struct dma_iova_state *state;
+ unsigned int count = 0;
+ int ret = -ENOMEM;
+
+ state = devm_kcalloc(dev, BITS_PER_TYPE(dma_addr_t) + 1, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return ret;
+
+ end = start + size;
+ rem = end - max(start, PAGE_SIZE);
+ dev->bus_dma_limit = end - 1;
+
+ while (rem) {
+ chunk = min(end & -end, (u64)1 << (fls64(rem) - 1));
+
+ if (!dma_iova_try_alloc(dev, &state[count], 0, chunk))
+ goto err_free_iova;
+
+ if (state[count].addr != end - chunk || state[count].__size != chunk)
+ goto err_free_iova;
+
+ rem -= chunk;
+ end -= chunk;
+ count++;
+ }
+
+ *iova_state = state;
+ dev->bus_dma_limit = dma_limit;
+
+ return 0;
+
+err_free_iova:
+ iris_unreserve_iova_region(dev, state);
+ dev->bus_dma_limit = dma_limit;
+
+ return ret;
+}
+
static int iris_register_video_device(struct iris_core *core, enum domain_type type)
{
struct video_device *vdev;
@@ -207,6 +258,8 @@ static void iris_remove(struct platform_device *pdev)
v4l2_device_unregister(&core->v4l2_dev);
+ iris_unreserve_iova_region(core->dev, core->iova_state);
+
mutex_destroy(&core->lock);
}
@@ -269,10 +322,15 @@ static int iris_probe(struct platform_device *pdev)
if (ret)
return ret;
- ret = v4l2_device_register(dev, &core->v4l2_dev);
+ ret = iris_reserve_iova_region(dev, &core->iova_state, IRIS_NP_RESERVE_IOVA_START,
+ IRIS_NP_RESERVE_IOVA_SIZE);
if (ret)
return ret;
+ ret = v4l2_device_register(dev, &core->v4l2_dev);
+ if (ret)
+ goto err_unresv_iova_region;
+
ret = iris_register_video_device(core, DECODER);
if (ret)
goto err_v4l2_unreg;
@@ -306,6 +364,8 @@ static int iris_probe(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
err_v4l2_unreg:
v4l2_device_unregister(&core->v4l2_dev);
+err_unresv_iova_region:
+ iris_unreserve_iova_region(dev, core->iova_state);
return ret;
}
diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c
index 9e228b70420e..7f89e745a4b1 100644
--- a/drivers/media/platform/qcom/iris/iris_vdec.c
+++ b/drivers/media/platform/qcom/iris/iris_vdec.c
@@ -95,16 +95,6 @@ static bool check_format(struct iris_inst *inst, u32 pixfmt, u32 type)
if (i == size)
return false;
- if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
- if (iris_fmt_is_8bit(pixfmt) &&
- inst->fw_caps[BIT_DEPTH].value == BIT_DEPTH_10)
- return false;
-
- if (iris_fmt_is_10bit(pixfmt) &&
- inst->fw_caps[BIT_DEPTH].value != BIT_DEPTH_10)
- return false;
- }
-
return true;
}
diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c
index fcbc60016bee..f1d7f186002d 100644
--- a/drivers/media/platform/qcom/iris/iris_vidc.c
+++ b/drivers/media/platform/qcom/iris/iris_vidc.c
@@ -438,14 +438,14 @@ static int iris_enum_frameintervals(struct file *filp, void *fh,
mbpf = NUM_MBS_PER_FRAME(fival->height, fival->width);
fps = DIV_ROUND_UP(core->iris_platform_data->max_core_mbps, mbpf);
- fival->type = V4L2_FRMIVAL_TYPE_STEPWISE;
+ fival->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;
fival->stepwise.min.numerator = 1;
fival->stepwise.min.denominator =
min_t(u32, fps, MAXIMUM_FPS);
fival->stepwise.max.numerator = 1;
fival->stepwise.max.denominator = 1;
fival->stepwise.step.numerator = 1;
- fival->stepwise.step.denominator = MAXIMUM_FPS;
+ fival->stepwise.step.denominator = 1;
return 0;
}
@@ -466,7 +466,7 @@ static int iris_querycap(struct file *filp, void *fh, struct v4l2_capability *ca
info = "enc";
}
snprintf(cap->bus_info, sizeof(cap->bus_info),
- "plat:%s:%s", dev_name(core->dev), info);
+ "platform:%s:%s", dev_name(core->dev), info);
return 0;
}
diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c b/drivers/media/platform/qcom/iris/iris_vpu_common.c
index e4847c107709..c9f5dc851d9f 100644
--- a/drivers/media/platform/qcom/iris/iris_vpu_common.c
+++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c
@@ -223,10 +223,10 @@ int iris_vpu_power_off_controller(struct iris_core *core)
void iris_vpu_power_off_hw(struct iris_core *core)
{
dev_pm_genpd_set_hwmode(core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN], false);
- iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]);
iris_disable_unprepare_clock(core, IRIS_BSE_HW_CLK);
iris_disable_unprepare_clock(core, IRIS_HW_AHB_CLK);
iris_disable_unprepare_clock(core, IRIS_HW_CLK);
+ iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]);
}
void iris_vpu_power_off(struct iris_core *core)
@@ -311,6 +311,11 @@ int iris_vpu_power_on_hw(struct iris_core *core)
int iris_vpu_set_hwmode(struct iris_core *core)
{
+ writel(CORE_BRIDGE_SW_RESET | CORE_BRIDGE_HW_RESET_DISABLE,
+ core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET);
+ writel(CORE_BRIDGE_HW_RESET_DISABLE, core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET);
+ writel(0x0, core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET);
+
return dev_pm_genpd_set_hwmode(core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN], true);
}
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 243e342b0ae7..2a0ab553c451 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -377,6 +377,57 @@ static int venus_add_dynamic_nodes(struct venus_core *core)
static void venus_remove_dynamic_nodes(struct venus_core *core) {}
#endif
+static void venus_unreserve_iova_region(struct device *dev, struct dma_iova_state *state)
+{
+ unsigned int i;
+
+ for (i = 0; dma_iova_size(&state[i]); i++)
+ dma_iova_free(dev, &state[i]);
+}
+
+static int venus_reserve_iova_region(struct device *dev, struct dma_iova_state **iova_state,
+ unsigned long start, unsigned long size)
+{
+ unsigned long dma_limit = dev->bus_dma_limit;
+ unsigned long end, rem, chunk;
+ struct dma_iova_state *state;
+ unsigned int count = 0;
+ int ret = -ENOMEM;
+
+ state = devm_kcalloc(dev, BITS_PER_TYPE(dma_addr_t) + 1, sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return ret;
+
+ end = start + size;
+ rem = end - max(start, PAGE_SIZE);
+ dev->bus_dma_limit = end - 1;
+
+ while (rem) {
+ chunk = min(end & -end, (u64)1 << (fls64(rem) - 1));
+
+ if (!dma_iova_try_alloc(dev, &state[count], 0, chunk))
+ goto err_free_iova;
+
+ if (state[count].addr != end - chunk || state[count].__size != chunk)
+ goto err_free_iova;
+
+ rem -= chunk;
+ end -= chunk;
+ count++;
+ }
+
+ *iova_state = state;
+ dev->bus_dma_limit = dma_limit;
+
+ return 0;
+
+err_free_iova:
+ venus_unreserve_iova_region(dev, state);
+ dev->bus_dma_limit = dma_limit;
+
+ return ret;
+}
+
static int venus_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -421,10 +472,15 @@ static int venus_probe(struct platform_device *pdev)
return ret;
}
- ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
+ ret = venus_reserve_iova_region(dev, &core->iova_state, VENUS_NP_RESERVE_IOVA_START,
+ VENUS_NP_RESERVE_IOVA_SIZE);
if (ret)
goto err_core_put;
+ ret = dma_set_mask_and_coherent(dev, core->res->dma_mask);
+ if (ret)
+ goto err_unresv_iova_region;
+
dma_set_max_seg_size(dev, UINT_MAX);
INIT_LIST_HEAD(&core->instances);
@@ -434,13 +490,13 @@ static int venus_probe(struct platform_device *pdev)
ret = hfi_create(core, &venus_core_ops);
if (ret)
- goto err_core_put;
+ goto err_unresv_iova_region;
ret = devm_request_threaded_irq(dev, core->irq, hfi_isr, venus_isr_thread,
IRQF_TRIGGER_HIGH | IRQF_ONESHOT,
"venus", core);
if (ret)
- goto err_core_put;
+ goto err_unresv_iova_region;
venus_assign_register_offsets(core);
@@ -525,6 +581,8 @@ static int venus_probe(struct platform_device *pdev)
v4l2_device_unregister(&core->v4l2_dev);
err_hfi_destroy:
hfi_destroy(core);
+err_unresv_iova_region:
+ venus_unreserve_iova_region(dev, core->iova_state);
err_core_put:
if (core->pm_ops->core_put)
core->pm_ops->core_put(core);
@@ -562,6 +620,8 @@ static void venus_remove(struct platform_device *pdev)
hfi_destroy(core);
+ venus_unreserve_iova_region(dev, core->iova_state);
+
mutex_destroy(&core->pm_lock);
mutex_destroy(&core->lock);
venus_dbgfs_deinit(core);
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 46705a666776..93fa77f05834 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -8,6 +8,7 @@
#define __VENUS_CORE_H_
#include <linux/bitops.h>
+#include <linux/dma-mapping.h>
#include <linux/list.h>
#include <media/videobuf2-v4l2.h>
#include <media/v4l2-ctrls.h>
@@ -30,6 +31,9 @@
#define VENUS_MAX_FPS 240
+#define VENUS_NP_RESERVE_IOVA_START 0x0
+#define VENUS_NP_RESERVE_IOVA_SIZE 0x25800000
+
extern int venus_fw_debug;
struct freq_tbl {
@@ -189,6 +193,7 @@ struct venus_format {
* @dump_core: a flag indicating that a core dump is required
* @ocs: OF changeset pointer
* @hwmode_dev: a flag indicating that HW_CTRL_TRIGGER is used in clock driver
+ * @iova_state: a pointer to an array of dma_iova_state entries reserved for restricted IOVA region
*/
struct venus_core {
void __iomem *base;
@@ -250,6 +255,7 @@ struct venus_core {
unsigned long dump_core;
struct of_changeset *ocs;
bool hwmode_dev;
+ struct dma_iova_state *iova_state;
};
struct vdec_controls {
diff --git a/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-dev.c b/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-dev.c
index 355842abb24b..48f9fd818bff 100644
--- a/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-dev.c
+++ b/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-dev.c
@@ -202,7 +202,9 @@ static int rzv2h_ivc_probe(struct platform_device *pdev)
pm_runtime_set_autosuspend_delay(dev, 2000);
pm_runtime_use_autosuspend(dev);
- pm_runtime_enable(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ return ret;
ivc->irqnum = platform_get_irq(pdev, 0);
if (ivc->irqnum < 0)
@@ -210,14 +212,9 @@ static int rzv2h_ivc_probe(struct platform_device *pdev)
ret = rzv2h_ivc_initialise_subdevice(ivc);
if (ret)
- goto err_disable_pm_runtime;
+ return ret;
return 0;
-
-err_disable_pm_runtime:
- pm_runtime_disable(dev);
-
- return ret;
}
static void rzv2h_ivc_remove(struct platform_device *pdev)
diff --git a/drivers/media/platform/rockchip/rga/rga-hw.c b/drivers/media/platform/rockchip/rga/rga-hw.c
index be1bc8ddbd03..2e7ec8b7c20f 100644
--- a/drivers/media/platform/rockchip/rga/rga-hw.c
+++ b/drivers/media/platform/rockchip/rga/rga-hw.c
@@ -158,7 +158,6 @@ static void rga_cmd_set_dst_addr(struct rga_ctx *ctx, dma_addr_t dma_addr)
static void rga_cmd_set_trans_info(struct rga_ctx *ctx)
{
- struct rockchip_rga *rga = ctx->rga;
u32 *dest = ctx->cmdbuf_virt;
unsigned int scale_dst_w, scale_dst_h;
unsigned int src_h, src_w, dst_h, dst_w;
@@ -251,13 +250,6 @@ static void rga_cmd_set_trans_info(struct rga_ctx *ctx)
*/
if (src_info.data.rot_mode == RGA_SRC_ROT_MODE_90_DEGREE ||
src_info.data.rot_mode == RGA_SRC_ROT_MODE_270_DEGREE) {
- if (rga->version.major == 0 || rga->version.minor == 0) {
- if (dst_w == src_h)
- src_h -= 8;
- if (abs(src_w - dst_h) < 16)
- src_w -= 16;
- }
-
scale_dst_h = dst_w;
scale_dst_w = dst_h;
} else {
diff --git a/drivers/media/platform/rockchip/rga/rga.c b/drivers/media/platform/rockchip/rga/rga.c
index b3cb6bf8eb86..fbc99462dce2 100644
--- a/drivers/media/platform/rockchip/rga/rga.c
+++ b/drivers/media/platform/rockchip/rga/rga.c
@@ -797,6 +797,7 @@ static int rga_probe(struct platform_device *pdev)
ret = irq;
goto err_put_clk;
}
+ rga->irq = irq;
ret = devm_request_irq(rga->dev, irq, rga_isr,
rga_has_internal_iommu(rga) ? 0 : IRQF_SHARED,
@@ -876,8 +877,9 @@ static void rga_remove(struct platform_device *pdev)
v4l2_info(&rga->v4l2_dev, "Removing\n");
- v4l2_m2m_release(rga->m2m_dev);
video_unregister_device(rga->vfd);
+ devm_free_irq(rga->dev, rga->irq, rga);
+ v4l2_m2m_release(rga->m2m_dev);
v4l2_device_unregister(&rga->v4l2_dev);
pm_runtime_disable(rga->dev);
diff --git a/drivers/media/platform/rockchip/rga/rga.h b/drivers/media/platform/rockchip/rga/rga.h
index bd431534d0d3..7bcdf36c11b4 100644
--- a/drivers/media/platform/rockchip/rga/rga.h
+++ b/drivers/media/platform/rockchip/rga/rga.h
@@ -73,6 +73,7 @@ struct rockchip_rga {
struct device *dev;
struct regmap *grf;
void __iomem *regs;
+ int irq;
struct clk_bulk_data *clks;
int num_clks;
struct rockchip_rga_version version;
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
index 061281f903f3..4541ef9ee3b9 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
@@ -1857,9 +1857,6 @@ static int rkvdec_probe(struct platform_device *pdev)
pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_disable(&pdev->dev);
- if (rkvdec->sram_pool)
- gen_pool_destroy(rkvdec->sram_pool);
-
return ret;
}
diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-lite.c b/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
index 8be20fd32d1c..ce3634909c8c 100644
--- a/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
@@ -1209,9 +1209,8 @@ static int fimc_lite_subdev_s_stream(struct v4l2_subdev *sd, int on)
spin_unlock_irqrestore(&fimc->slock, flags);
}
} else {
- set_bit(ST_FLITE_OFF, &fimc->state);
-
spin_lock_irqsave(&fimc->slock, flags);
+ set_bit(ST_FLITE_OFF, &fimc->state);
flite_hw_capture_stop(fimc);
spin_unlock_irqrestore(&fimc->slock, flags);
diff --git a/drivers/media/platform/st/sti/delta/delta-mjpeg-hdr.c b/drivers/media/platform/st/sti/delta/delta-mjpeg-hdr.c
index 90e5b2f72c82..849a5e118aec 100644
--- a/drivers/media/platform/st/sti/delta/delta-mjpeg-hdr.c
+++ b/drivers/media/platform/st/sti/delta/delta-mjpeg-hdr.c
@@ -4,6 +4,7 @@
* Author: Hugues Fruchet <hugues.fruchet@st.com> for STMicroelectronics.
*/
+#include <linux/unaligned.h>
#include "delta.h"
#include "delta-mjpeg.h"
@@ -48,13 +49,13 @@ static int delta_mjpeg_read_sof(struct delta_ctx *pctx,
goto err_no_more;
memset(header, 0, sizeof(*header));
- header->length = be16_to_cpu(*(__be16 *)(data + offset));
+ header->length = get_unaligned_be16(data + offset);
offset += sizeof(u16);
header->sample_precision = *(u8 *)(data + offset);
offset += sizeof(u8);
- header->frame_height = be16_to_cpu(*(__be16 *)(data + offset));
+ header->frame_height = get_unaligned_be16(data + offset);
offset += sizeof(u16);
- header->frame_width = be16_to_cpu(*(__be16 *)(data + offset));
+ header->frame_width = get_unaligned_be16(data + offset);
offset += sizeof(u16);
header->nb_of_components = *(u8 *)(data + offset);
offset += sizeof(u8);
diff --git a/drivers/media/platform/st/stm32/stm32-csi.c b/drivers/media/platform/st/stm32/stm32-csi.c
index fd2b6dfbd44c..78c68cf90ffc 100644
--- a/drivers/media/platform/st/stm32/stm32-csi.c
+++ b/drivers/media/platform/st/stm32/stm32-csi.c
@@ -825,7 +825,7 @@ static int stm32_csi_async_bound(struct v4l2_async_notifier *notifier,
int remote_pad;
remote_pad = media_entity_get_fwnode_pad(&s_subdev->entity,
- s_subdev->fwnode,
+ asd->match.fwnode,
MEDIA_PAD_FL_SOURCE);
if (remote_pad < 0) {
dev_err(csidev->dev, "Couldn't find output pad for subdev %s\n",
@@ -1059,6 +1059,7 @@ static int stm32_csi_probe(struct platform_device *pdev)
return 0;
err_cleanup:
+ v4l2_async_nf_unregister(&csidev->notifier);
v4l2_async_nf_cleanup(&csidev->notifier);
return ret;
}
@@ -1067,6 +1068,8 @@ static void stm32_csi_remove(struct platform_device *pdev)
{
struct stm32_csi_dev *csidev = platform_get_drvdata(pdev);
+ v4l2_async_nf_unregister(&csidev->notifier);
+ v4l2_async_nf_cleanup(&csidev->notifier);
v4l2_async_unregister_subdev(&csidev->sd);
pm_runtime_disable(&pdev->dev);
diff --git a/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c b/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c
index 25f8ca0d6d94..3b9d78e69aac 100644
--- a/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c
+++ b/drivers/media/platform/synopsys/hdmirx/snps_hdmirx.c
@@ -156,7 +156,6 @@ struct snps_hdmirx_dev {
int num_clks;
u32 edid_blocks_written;
u32 cur_fmt_fourcc;
- u32 color_depth;
spinlock_t rst_lock; /* to lock register access */
u8 edid[EDID_NUM_BLOCKS_MAX * EDID_BLOCK_SIZE];
};
@@ -380,6 +379,38 @@ static void hdmirx_toggle_polarity(struct snps_hdmirx_dev *hdmirx_dev)
VPROC_HSYNC_POL_OVR_EN, 0);
}
+static u32 hdmirx_get_colordepth(struct snps_hdmirx_dev *hdmirx_dev)
+{
+ struct v4l2_device *v4l2_dev = &hdmirx_dev->v4l2_dev;
+ u32 val, color_depth_reg, color_depth;
+
+ val = hdmirx_readl(hdmirx_dev, DMA_STATUS11);
+ color_depth_reg = (val & HDMIRX_COLOR_DEPTH_MASK) >> 3;
+
+ switch (color_depth_reg) {
+ case 0x4:
+ color_depth = 24;
+ break;
+ case 0x5:
+ color_depth = 30;
+ break;
+ case 0x6:
+ color_depth = 36;
+ break;
+ case 0x7:
+ color_depth = 48;
+ break;
+ default:
+ color_depth = 24;
+ break;
+ }
+
+ v4l2_dbg(1, debug, v4l2_dev, "%s: color_depth: %d, reg_val:%d\n",
+ __func__, color_depth, color_depth_reg);
+
+ return color_depth;
+}
+
/*
* When querying DV timings during preview, if the DMA's timing is stable,
* we retrieve the timings directly from the DMA. However, if the current
@@ -393,7 +424,7 @@ static int hdmirx_get_detected_timings(struct snps_hdmirx_dev *hdmirx_dev,
struct v4l2_bt_timings *bt = &timings->bt;
u32 val, tmdsqpclk_freq, pix_clk;
unsigned int num_retries = 0;
- u32 field_type, deframer_st;
+ u32 field_type, deframer_st, color_depth;
u64 tmp_data, tmds_clk;
bool is_dvi_mode;
int ret;
@@ -414,10 +445,11 @@ static int hdmirx_get_detected_timings(struct snps_hdmirx_dev *hdmirx_dev,
deframer_st = hdmirx_readl(hdmirx_dev, DEFRAMER_STATUS);
is_dvi_mode = !(deframer_st & OPMODE_STS_MASK);
+ color_depth = hdmirx_get_colordepth(hdmirx_dev);
tmdsqpclk_freq = hdmirx_readl(hdmirx_dev, CMU_TMDSQPCLK_FREQ);
tmds_clk = tmdsqpclk_freq * 4 * 1000;
tmp_data = tmds_clk * 24;
- do_div(tmp_data, hdmirx_dev->color_depth);
+ do_div(tmp_data, color_depth);
pix_clk = tmp_data;
bt->pixelclock = pix_clk;
@@ -429,7 +461,7 @@ static int hdmirx_get_detected_timings(struct snps_hdmirx_dev *hdmirx_dev,
v4l2_dbg(2, debug, v4l2_dev, "tmds_clk:%llu, pix_clk:%d\n", tmds_clk, pix_clk);
v4l2_dbg(1, debug, v4l2_dev, "interlace:%d, fmt:%d, color:%d, mode:%s\n",
bt->interlaced, hdmirx_dev->pix_fmt,
- hdmirx_dev->color_depth,
+ color_depth,
is_dvi_mode ? "dvi" : "hdmi");
v4l2_dbg(2, debug, v4l2_dev, "deframer_st:%#x\n", deframer_st);
@@ -988,36 +1020,6 @@ static void hdmirx_controller_init(struct snps_hdmirx_dev *hdmirx_dev)
VS_REMAPFILTER_EN_QST | VS_FILTER_ORDER_QST(0x3));
}
-static void hdmirx_get_colordepth(struct snps_hdmirx_dev *hdmirx_dev)
-{
- struct v4l2_device *v4l2_dev = &hdmirx_dev->v4l2_dev;
- u32 val, color_depth_reg;
-
- val = hdmirx_readl(hdmirx_dev, DMA_STATUS11);
- color_depth_reg = (val & HDMIRX_COLOR_DEPTH_MASK) >> 3;
-
- switch (color_depth_reg) {
- case 0x4:
- hdmirx_dev->color_depth = 24;
- break;
- case 0x5:
- hdmirx_dev->color_depth = 30;
- break;
- case 0x6:
- hdmirx_dev->color_depth = 36;
- break;
- case 0x7:
- hdmirx_dev->color_depth = 48;
- break;
- default:
- hdmirx_dev->color_depth = 24;
- break;
- }
-
- v4l2_dbg(1, debug, v4l2_dev, "%s: color_depth: %d, reg_val:%d\n",
- __func__, hdmirx_dev->color_depth, color_depth_reg);
-}
-
static void hdmirx_get_pix_fmt(struct snps_hdmirx_dev *hdmirx_dev)
{
struct v4l2_device *v4l2_dev = &hdmirx_dev->v4l2_dev;
@@ -1128,7 +1130,6 @@ static void hdmirx_format_change(struct snps_hdmirx_dev *hdmirx_dev)
};
hdmirx_get_pix_fmt(hdmirx_dev);
- hdmirx_get_colordepth(hdmirx_dev);
hdmirx_get_avi_infoframe(hdmirx_dev);
v4l2_dbg(1, debug, v4l2_dev, "%s: queue res_chg_event\n", __func__);
@@ -2609,10 +2610,11 @@ static int hdmirx_setup_irq(struct snps_hdmirx_dev *hdmirx_dev,
irq_set_status_flags(irq, IRQ_NOAUTOEN);
hdmirx_dev->det_irq = irq;
- ret = devm_request_irq(dev, irq, hdmirx_5v_det_irq_handler,
- IRQF_TRIGGER_FALLING | IRQF_TRIGGER_RISING,
- "rk_hdmirx-5v", hdmirx_dev);
- if (ret) {
+ ret = devm_request_any_context_irq(dev, irq, hdmirx_5v_det_irq_handler,
+ IRQF_TRIGGER_FALLING |
+ IRQF_TRIGGER_RISING,
+ "rk_hdmirx-5v", hdmirx_dev);
+ if (ret < 0) {
dev_err_probe(dev, ret, "failed to request hdmirx-5v irq\n");
return ret;
}
diff --git a/drivers/media/platform/ti/davinci/vpif_capture.c b/drivers/media/platform/ti/davinci/vpif_capture.c
index 91cb6223561a..397f3115af5b 100644
--- a/drivers/media/platform/ti/davinci/vpif_capture.c
+++ b/drivers/media/platform/ti/davinci/vpif_capture.c
@@ -1602,7 +1602,7 @@ vpif_capture_get_pdata(struct platform_device *pdev,
static int vpif_probe(struct platform_device *pdev)
{
struct vpif_subdev_info *subdevdata;
- struct i2c_adapter *i2c_adap;
+ struct i2c_adapter *i2c_adap = NULL;
int subdev_count;
int res_idx = 0;
int i, err;
@@ -1692,9 +1692,12 @@ static int vpif_probe(struct platform_device *pdev)
}
}
+ i2c_put_adapter(i2c_adap);
+
return 0;
probe_subdev_out:
+ i2c_put_adapter(i2c_adap);
v4l2_async_nf_cleanup(&vpif_obj.notifier);
/* free sub devices memory */
kfree(vpif_obj.sd);
diff --git a/drivers/media/platform/ti/davinci/vpif_display.c b/drivers/media/platform/ti/davinci/vpif_display.c
index 08877ae1513a..8bea1de6ccc8 100644
--- a/drivers/media/platform/ti/davinci/vpif_display.c
+++ b/drivers/media/platform/ti/davinci/vpif_display.c
@@ -1289,9 +1289,12 @@ static int vpif_probe(struct platform_device *pdev)
if (err)
goto probe_subdev_out;
+ i2c_put_adapter(i2c_adap);
+
return 0;
probe_subdev_out:
+ i2c_put_adapter(i2c_adap);
kfree(vpif_obj.sd);
vpif_unregister:
v4l2_device_unregister(&vpif_obj.v4l2_dev);
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index ad71c0402ef3..66699865f65a 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -1218,7 +1218,7 @@ static int hantro_probe(struct platform_device *pdev)
ret = clk_bulk_prepare(vpu->variant->num_clocks, vpu->clocks);
if (ret) {
dev_err(&pdev->dev, "Failed to prepare clocks\n");
- goto err_rst_assert;
+ goto err_pm_disable_assert_reset;
}
ret = v4l2_device_register(&pdev->dev, &vpu->v4l2_dev);
@@ -1272,8 +1272,11 @@ static int hantro_probe(struct platform_device *pdev)
v4l2_device_unregister(&vpu->v4l2_dev);
err_clk_unprepare:
clk_bulk_unprepare(vpu->variant->num_clocks, vpu->clocks);
-err_rst_assert:
+err_pm_disable_assert_reset:
+ pm_runtime_dont_use_autosuspend(vpu->dev);
+ pm_runtime_disable(vpu->dev);
reset_control_assert(vpu->resets);
+ return ret;
err_pm_disable:
pm_runtime_dont_use_autosuspend(vpu->dev);
pm_runtime_disable(vpu->dev);
@@ -1293,9 +1296,9 @@ static void hantro_remove(struct platform_device *pdev)
v4l2_m2m_put(vpu->m2m_dev);
v4l2_device_unregister(&vpu->v4l2_dev);
clk_bulk_unprepare(vpu->variant->num_clocks, vpu->clocks);
- reset_control_assert(vpu->resets);
pm_runtime_dont_use_autosuspend(vpu->dev);
pm_runtime_disable(vpu->dev);
+ reset_control_assert(vpu->resets);
}
#ifdef CONFIG_PM
diff --git a/drivers/media/platform/verisilicon/hantro_g1_mpeg2_dec.c b/drivers/media/platform/verisilicon/hantro_g1_mpeg2_dec.c
index e0d6bd0a6e44..6a942f4b8a2d 100644
--- a/drivers/media/platform/verisilicon/hantro_g1_mpeg2_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g1_mpeg2_dec.c
@@ -161,8 +161,13 @@ int hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx)
seq = hantro_get_ctrl(ctx,
V4L2_CID_STATELESS_MPEG2_SEQUENCE);
+ if (WARN_ON(!seq))
+ return -EINVAL;
+
pic = hantro_get_ctrl(ctx,
V4L2_CID_STATELESS_MPEG2_PICTURE);
+ if (WARN_ON(!pic))
+ return -EINVAL;
reg = G1_REG_DEC_AXI_RD_ID(0) |
G1_REG_DEC_TIMEOUT_E(1) |
diff --git a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
index 00aa566a4ccd..2a145433e6d1 100644
--- a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
+++ b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
@@ -369,6 +369,7 @@ void rockchip_vpu981_av1_dec_exit(struct hantro_ctx *ctx)
int rockchip_vpu981_av1_dec_init(struct hantro_ctx *ctx)
{
+ int ret = 0;
struct hantro_dev *vpu = ctx->dev;
struct hantro_av1_dec_hw_ctx *av1_dec = &ctx->av1_dec;
@@ -377,39 +378,54 @@ int rockchip_vpu981_av1_dec_init(struct hantro_ctx *ctx)
av1_dec->global_model.cpu = dma_alloc_coherent(vpu->dev, GLOBAL_MODEL_SIZE,
&av1_dec->global_model.dma,
GFP_KERNEL);
- if (!av1_dec->global_model.cpu)
- return -ENOMEM;
+ if (!av1_dec->global_model.cpu) {
+ ret = -ENOMEM;
+ goto global_model_cpu_err;
+ }
+
av1_dec->global_model.size = GLOBAL_MODEL_SIZE;
av1_dec->tile_info.cpu = dma_alloc_coherent(vpu->dev, AV1_TILE_INFO_SIZE,
&av1_dec->tile_info.dma,
GFP_KERNEL);
- if (!av1_dec->tile_info.cpu)
- return -ENOMEM;
+ if (!av1_dec->tile_info.cpu) {
+ ret = -ENOMEM;
+ goto tile_info_cpu_err;
+ }
+
av1_dec->tile_info.size = AV1_TILE_INFO_SIZE;
av1_dec->film_grain.cpu = dma_alloc_coherent(vpu->dev,
ALIGN(sizeof(struct rockchip_av1_film_grain), 2048),
&av1_dec->film_grain.dma,
GFP_KERNEL);
- if (!av1_dec->film_grain.cpu)
- return -ENOMEM;
+ if (!av1_dec->film_grain.cpu) {
+ ret = -ENOMEM;
+ goto film_grain_cpu_err;
+ }
+
av1_dec->film_grain.size = ALIGN(sizeof(struct rockchip_av1_film_grain), 2048);
av1_dec->prob_tbl.cpu = dma_alloc_coherent(vpu->dev,
ALIGN(sizeof(struct av1cdfs), 2048),
&av1_dec->prob_tbl.dma,
GFP_KERNEL);
- if (!av1_dec->prob_tbl.cpu)
- return -ENOMEM;
+ if (!av1_dec->prob_tbl.cpu) {
+ ret = -ENOMEM;
+ goto prob_tbl_cpu_err;
+ }
+
av1_dec->prob_tbl.size = ALIGN(sizeof(struct av1cdfs), 2048);
av1_dec->prob_tbl_out.cpu = dma_alloc_coherent(vpu->dev,
ALIGN(sizeof(struct av1cdfs), 2048),
&av1_dec->prob_tbl_out.dma,
GFP_KERNEL);
- if (!av1_dec->prob_tbl_out.cpu)
- return -ENOMEM;
+ if (!av1_dec->prob_tbl_out.cpu) {
+ ret = -ENOMEM;
+ goto prob_tbl_out_cpu_err;
+ }
+
av1_dec->prob_tbl_out.size = ALIGN(sizeof(struct av1cdfs), 2048);
av1_dec->cdfs = &av1_dec->default_cdfs;
av1_dec->cdfs_ndvc = &av1_dec->default_cdfs_ndvc;
@@ -420,11 +436,37 @@ int rockchip_vpu981_av1_dec_init(struct hantro_ctx *ctx)
AV1_TILE_SIZE,
&av1_dec->tile_buf.dma,
GFP_KERNEL);
- if (!av1_dec->tile_buf.cpu)
- return -ENOMEM;
+ if (!av1_dec->tile_buf.cpu) {
+ ret = -ENOMEM;
+ goto tile_buf_cpu_err;
+ }
+
av1_dec->tile_buf.size = AV1_TILE_SIZE;
- return 0;
+ return ret;
+
+tile_buf_cpu_err:
+ dma_free_coherent(vpu->dev, av1_dec->prob_tbl_out.size,
+ av1_dec->prob_tbl_out.cpu,
+ av1_dec->prob_tbl_out.dma);
+prob_tbl_out_cpu_err:
+ dma_free_coherent(vpu->dev, av1_dec->prob_tbl.size,
+ av1_dec->prob_tbl.cpu,
+ av1_dec->prob_tbl.dma);
+prob_tbl_cpu_err:
+ dma_free_coherent(vpu->dev, av1_dec->film_grain.size,
+ av1_dec->film_grain.cpu,
+ av1_dec->film_grain.dma);
+film_grain_cpu_err:
+ dma_free_coherent(vpu->dev, av1_dec->tile_info.size,
+ av1_dec->tile_info.cpu,
+ av1_dec->tile_info.dma);
+tile_info_cpu_err:
+ dma_free_coherent(vpu->dev, av1_dec->global_model.size,
+ av1_dec->global_model.cpu,
+ av1_dec->global_model.dma);
+global_model_cpu_err:
+ return ret;
}
static int rockchip_vpu981_av1_dec_prepare_run(struct hantro_ctx *ctx)
diff --git a/drivers/media/rc/ene_ir.c b/drivers/media/rc/ene_ir.c
index 6f7dccc965e7..f98d76277b62 100644
--- a/drivers/media/rc/ene_ir.c
+++ b/drivers/media/rc/ene_ir.c
@@ -1103,15 +1103,15 @@ static void ene_remove(struct pnp_dev *pnp_dev)
unsigned long flags;
rc_unregister_device(dev->rdev);
- timer_delete_sync(&dev->tx_sim_timer);
spin_lock_irqsave(&dev->hw_lock, flags);
ene_rx_disable(dev);
ene_rx_restore_hw_buffer(dev);
spin_unlock_irqrestore(&dev->hw_lock, flags);
- rc_free_device(dev->rdev);
free_irq(dev->irq, dev);
+ timer_delete_sync(&dev->tx_sim_timer);
release_region(dev->hw_io, ENE_IO_SIZE);
+ rc_free_device(dev->rdev);
kfree(dev);
}
diff --git a/drivers/media/rc/ir-hix5hd2.c b/drivers/media/rc/ir-hix5hd2.c
index 1b061e4a3dcf..aa3de4d57a58 100644
--- a/drivers/media/rc/ir-hix5hd2.c
+++ b/drivers/media/rc/ir-hix5hd2.c
@@ -316,6 +316,9 @@ static int hix5hd2_ir_probe(struct platform_device *pdev)
if (ret < 0)
goto clkerr;
+ priv->rdev = rdev;
+ priv->dev = dev;
+
if (devm_request_irq(dev, priv->irq, hix5hd2_ir_rx_interrupt,
0, pdev->name, priv) < 0) {
dev_err(dev, "IRQ %d register failed\n", priv->irq);
@@ -323,8 +326,6 @@ static int hix5hd2_ir_probe(struct platform_device *pdev)
goto regerr;
}
- priv->rdev = rdev;
- priv->dev = dev;
platform_set_drvdata(pdev, priv);
return ret;
diff --git a/drivers/media/rc/ir_toy.c b/drivers/media/rc/ir_toy.c
index ee645882dd5a..0819cd5cfa98 100644
--- a/drivers/media/rc/ir_toy.c
+++ b/drivers/media/rc/ir_toy.c
@@ -373,7 +373,7 @@ static int irtoy_tx_carrier(struct rc_dev *rc, uint32_t carrier)
u8 buf[3];
int err;
- if (carrier < 11800)
+ if (carrier < 11800 || carrier > 3000000)
return -EINVAL;
buf[0] = 0x06;
diff --git a/drivers/media/rc/meson-ir-tx.c b/drivers/media/rc/meson-ir-tx.c
index fded2c256f2a..a0d07da09f67 100644
--- a/drivers/media/rc/meson-ir-tx.c
+++ b/drivers/media/rc/meson-ir-tx.c
@@ -219,6 +219,9 @@ static int meson_irtx_set_carrier(struct rc_dev *rc, u32 carrier)
if (carrier == 0)
return -EINVAL;
+ if (!DIV_ROUND_CLOSEST(USEC_PER_SEC, carrier))
+ return -EINVAL;
+
ir->carrier = carrier;
meson_irtx_set_mod(ir);
@@ -288,11 +291,12 @@ static int meson_irtx_mod_clock_probe(struct meson_irtx *ir,
if (!np)
return -ENODEV;
- clock = devm_clk_get(ir->dev, "xtal");
- if (IS_ERR(clock) || clk_prepare_enable(clock))
- return -ENODEV;
-
*clk_nr = IRB_MOD_XTAL3_CLK;
+
+ clock = devm_clk_get_enabled(ir->dev, "xtal");
+ if (IS_ERR(clock))
+ return PTR_ERR(clock);
+
ir->clk_rate = clk_get_rate(clock) / 3;
if (ir->clk_rate < IRB_MOD_1US_CLK_RATE) {
@@ -324,7 +328,7 @@ static int meson_irtx_probe(struct platform_device *pdev)
irq = platform_get_irq(pdev, 0);
if (irq < 0)
- return -ENODEV;
+ return irq;
ir->dev = dev;
ir->carrier = MIRTX_DEFAULT_CARRIER;
@@ -345,7 +349,7 @@ static int meson_irtx_probe(struct platform_device *pdev)
if (ret)
return dev_err_probe(dev, ret, "irq request failed\n");
- rc = rc_allocate_device(RC_DRIVER_IR_RAW_TX);
+ rc = devm_rc_allocate_device(dev, RC_DRIVER_IR_RAW_TX);
if (!rc)
return -ENOMEM;
@@ -358,10 +362,8 @@ static int meson_irtx_probe(struct platform_device *pdev)
rc->s_tx_duty_cycle = meson_irtx_set_duty_cycle;
ret = devm_rc_register_device(dev, rc);
- if (ret < 0) {
- rc_free_device(rc);
+ if (ret < 0)
return dev_err_probe(dev, ret, "rc_dev registration failed\n");
- }
return 0;
}
diff --git a/drivers/media/rc/rc-main.c b/drivers/media/rc/rc-main.c
index 85c9be43b912..946521ace142 100644
--- a/drivers/media/rc/rc-main.c
+++ b/drivers/media/rc/rc-main.c
@@ -17,9 +17,8 @@
#include <linux/module.h>
#include "rc-core-priv.h"
-/* Sizes are in bytes, 256 bytes allows for 32 entries on x64 */
-#define IR_TAB_MIN_SIZE 256
-#define IR_TAB_MAX_SIZE 8192
+#define IR_TAB_MIN_ENTRIES 32
+#define IR_TAB_MAX_ENTRIES 1024
static const struct {
const char *name;
@@ -105,7 +104,6 @@ static struct rc_map_list *seek_rc_map(const char *name)
struct rc_map *rc_map_get(const char *name)
{
-
struct rc_map_list *map;
map = seek_rc_map(name);
@@ -202,7 +200,7 @@ static int scancode_to_u64(const struct input_keymap_entry *ke, u64 *scancode)
* ir_create_table() - initializes a scancode table
* @dev: the rc_dev device
* @rc_map: the rc_map to initialize
- * @name: name to assign to the table
+ * @map_name: name to assign to the table
* @rc_proto: ir type to assign to the new table
* @size: initial size of the table
*
@@ -212,23 +210,33 @@ static int scancode_to_u64(const struct input_keymap_entry *ke, u64 *scancode)
* return: zero on success or a negative error code
*/
static int ir_create_table(struct rc_dev *dev, struct rc_map *rc_map,
- const char *name, u64 rc_proto, size_t size)
+ const char *map_name, u64 rc_proto, size_t size)
{
- rc_map->name = kstrdup(name, GFP_KERNEL);
- if (!rc_map->name)
+ struct rc_map_table *scan;
+ unsigned int alloc;
+ char *name;
+
+ name = kstrdup(map_name, GFP_KERNEL);
+ if (!name)
return -ENOMEM;
- rc_map->rc_proto = rc_proto;
- rc_map->alloc = roundup_pow_of_two(size * sizeof(struct rc_map_table));
- rc_map->size = rc_map->alloc / sizeof(struct rc_map_table);
- rc_map->scan = kmalloc(rc_map->alloc, GFP_KERNEL);
- if (!rc_map->scan) {
- kfree(rc_map->name);
- rc_map->name = NULL;
+
+ alloc = roundup_pow_of_two(size);
+ scan = kmalloc_objs(struct rc_map_table, alloc, GFP_KERNEL);
+ if (!scan) {
+ kfree(name);
return -ENOMEM;
}
- dev_dbg(&dev->dev, "Allocated space for %u keycode entries (%u bytes)\n",
- rc_map->size, rc_map->alloc);
+ scoped_guard(spinlock_irqsave, &rc_map->lock) {
+ rc_map->name = name;
+ rc_map->scan = scan;
+ rc_map->rc_proto = rc_proto;
+ rc_map->len = 0;
+ rc_map->size = alloc;
+ }
+
+ dev_dbg(&dev->dev, "Allocated space for %u keycode entries (%zu bytes)\n",
+ alloc, alloc * sizeof(struct rc_map_table));
return 0;
}
@@ -236,16 +244,26 @@ static int ir_create_table(struct rc_dev *dev, struct rc_map *rc_map,
* ir_free_table() - frees memory allocated by a scancode table
* @rc_map: the table whose mappings need to be freed
*
- * This routine will free memory alloctaed for key mappings used by given
+ * This routine will free memory allocated for key mappings used by given
* scancode table.
*/
static void ir_free_table(struct rc_map *rc_map)
{
- rc_map->size = 0;
- kfree(rc_map->name);
- rc_map->name = NULL;
- kfree(rc_map->scan);
- rc_map->scan = NULL;
+ struct rc_map_table *scan;
+ const char *name;
+
+ scoped_guard(spinlock_irqsave, &rc_map->lock) {
+ name = rc_map->name;
+ scan = rc_map->scan;
+
+ rc_map->size = 0;
+ rc_map->len = 0;
+ rc_map->name = NULL;
+ rc_map->scan = NULL;
+ }
+
+ kfree(name);
+ kfree(scan);
}
/**
@@ -262,38 +280,38 @@ static void ir_free_table(struct rc_map *rc_map)
static int ir_resize_table(struct rc_dev *dev, struct rc_map *rc_map,
gfp_t gfp_flags)
{
- unsigned int oldalloc = rc_map->alloc;
- unsigned int newalloc = oldalloc;
- struct rc_map_table *oldscan = rc_map->scan;
+ unsigned int newsize = rc_map->size;
struct rc_map_table *newscan;
+ lockdep_assert_held(&rc_map->lock);
+
if (rc_map->size == rc_map->len) {
/* All entries in use -> grow keytable */
- if (rc_map->alloc >= IR_TAB_MAX_SIZE)
+ if (newsize >= IR_TAB_MAX_ENTRIES)
return -ENOMEM;
- newalloc *= 2;
- dev_dbg(&dev->dev, "Growing table to %u bytes\n", newalloc);
+ newsize *= 2;
+
+ dev_dbg(&dev->dev, "Growing table to %u entries\n", newsize);
}
- if ((rc_map->len * 3 < rc_map->size) && (oldalloc > IR_TAB_MIN_SIZE)) {
+ if (rc_map->len * 3 < rc_map->size && rc_map->size > IR_TAB_MIN_ENTRIES) {
/* Less than 1/3 of entries in use -> shrink keytable */
- newalloc /= 2;
- dev_dbg(&dev->dev, "Shrinking table to %u bytes\n", newalloc);
+ newsize /= 2;
+ dev_dbg(&dev->dev, "Shrinking table to %u entries\n", newsize);
}
- if (newalloc == oldalloc)
+ if (newsize == rc_map->size)
return 0;
- newscan = kmalloc(newalloc, gfp_flags);
+ newscan = krealloc_array(rc_map->scan, newsize,
+ sizeof(struct rc_map_table), gfp_flags);
if (!newscan)
return -ENOMEM;
- memcpy(newscan, rc_map->scan, rc_map->len * sizeof(struct rc_map_table));
rc_map->scan = newscan;
- rc_map->alloc = newalloc;
- rc_map->size = rc_map->alloc / sizeof(struct rc_map_table);
- kfree(oldscan);
+ rc_map->size = newsize;
+
return 0;
}
@@ -318,6 +336,8 @@ static unsigned int ir_update_mapping(struct rc_dev *dev,
int old_keycode = rc_map->scan[index].keycode;
int i;
+ lockdep_assert_held(&rc_map->lock);
+
/* Did the user wish to remove the mapping? */
if (new_keycode == KEY_RESERVED || new_keycode == KEY_UNKNOWN) {
dev_dbg(&dev->dev, "#%d: Deleting scan 0x%04llx\n",
@@ -373,6 +393,8 @@ static unsigned int ir_establish_scancode(struct rc_dev *dev,
{
unsigned int i;
+ lockdep_assert_held(&rc_map->lock);
+
/*
* Unfortunately, some hardware-based IR decoders don't provide
* all bits for the complete IR code. In general, they provide only
@@ -397,7 +419,7 @@ static unsigned int ir_establish_scancode(struct rc_dev *dev,
/* No previous mapping found, we might need to grow the table */
if (rc_map->size == rc_map->len) {
if (!resize || ir_resize_table(dev, rc_map, GFP_ATOMIC))
- return -1U;
+ return UINT_MAX;
}
/* i is the proper index to insert our new keycode */
@@ -479,16 +501,18 @@ static int ir_setkeytable(struct rc_dev *dev, const struct rc_map *from)
if (rc)
return rc;
- for (i = 0; i < from->size; i++) {
- index = ir_establish_scancode(dev, rc_map,
- from->scan[i].scancode, false);
- if (index >= rc_map->len) {
- rc = -ENOMEM;
- break;
- }
+ scoped_guard(spinlock_irqsave, &rc_map->lock) {
+ for (i = 0; i < from->size; i++) {
+ index = ir_establish_scancode(dev, rc_map,
+ from->scan[i].scancode, false);
+ if (index >= rc_map->len) {
+ rc = -ENOMEM;
+ break;
+ }
- ir_update_mapping(dev, rc_map, index,
- from->scan[i].keycode);
+ ir_update_mapping(dev, rc_map, index,
+ from->scan[i].keycode);
+ }
}
if (rc)
@@ -524,6 +548,8 @@ static unsigned int ir_lookup_by_scancode(const struct rc_map *rc_map,
{
struct rc_map_table *res;
+ lockdep_assert_held(&rc_map->lock);
+
res = bsearch(&scancode, rc_map->scan, rc_map->len,
sizeof(struct rc_map_table), rc_map_cmp);
if (!res)
@@ -1994,7 +2020,8 @@ int rc_register_device(struct rc_dev *dev)
scoped_guard(mutex, &dev->lock)
dev->registered = false;
out_free_table:
- ir_free_table(&dev->rc_map);
+ if (dev->driver_type != RC_DRIVER_IR_RAW_TX)
+ ir_free_table(&dev->rc_map);
out_raw:
ida_free(&rc_ida, minor);
return rc;
diff --git a/drivers/media/rc/redrat3.c b/drivers/media/rc/redrat3.c
index 565c490b7051..3e82b5e66477 100644
--- a/drivers/media/rc/redrat3.c
+++ b/drivers/media/rc/redrat3.c
@@ -358,8 +358,24 @@ static void redrat3_process_ir_data(struct redrat3_dev *rr3)
/* process each rr3 encoded byte into an int */
sig_size = be16_to_cpu(rr3->irdata.sig_size);
+
+ /*
+ * Note we are not checking if we are reading beyond the end of the
+ * packet which was sent, and reading stale data. If the device
+ * sends a packet which is short then we get garbage IR, but no
+ * out of bounds read.
+ */
+ if (sig_size > RR3_MAX_SIG_SIZE) {
+ dev_err(dev, "length %u is incorrect\n", sig_size);
+ return;
+ }
+
for (i = 0; i < sig_size; i++) {
offset = rr3->irdata.sigdata[i];
+ if (offset >= RR3_DRIVER_MAXLENS) {
+ dev_err(dev, "offset %u is incorrect\n", offset);
+ return;
+ }
val = get_unaligned_be16(&rr3->irdata.lens[offset]);
/* we should always get pulse/space/pulse/space samples */
@@ -757,17 +773,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
u8 curlencheck = 0;
unsigned i, sendbuf_len;
- if (rr3->transmitting) {
- dev_warn(dev, "%s: transmitter already in use\n", __func__);
- return -EAGAIN;
- }
-
if (count > RR3_MAX_SIG_SIZE - RR3_TX_TRAILER_LEN)
return -EINVAL;
- /* rr3 will disable rc detector on transmit */
- rr3->transmitting = true;
-
sample_lens = kzalloc_objs(*sample_lens, RR3_DRIVER_MAXLENS);
if (!sample_lens)
return -ENOMEM;
@@ -778,6 +786,9 @@ static int redrat3_transmit_ir(struct rc_dev *rcdev, unsigned *txbuf,
goto out;
}
+ /* rr3 will disable rc detector on transmit */
+ rr3->transmitting = true;
+
for (i = 0; i < count; i++) {
cur_sample_len = redrat3_us_to_len(txbuf[i]);
if (cur_sample_len > 0xffff) {
@@ -1152,6 +1163,7 @@ static int redrat3_dev_suspend(struct usb_interface *intf, pm_message_t message)
usb_kill_urb(rr3->narrow_urb);
usb_kill_urb(rr3->wide_urb);
usb_kill_urb(rr3->flash_urb);
+ usb_kill_urb(rr3->learn_urb);
return 0;
}
diff --git a/drivers/media/rc/serial_ir.c b/drivers/media/rc/serial_ir.c
index 992fff82b524..f6702b3df9a0 100644
--- a/drivers/media/rc/serial_ir.c
+++ b/drivers/media/rc/serial_ir.c
@@ -798,7 +798,7 @@ static int __init serial_ir_init_module(void)
static void __exit serial_ir_exit_module(void)
{
- timer_delete_sync(&serial_ir.timeout_timer);
+ timer_shutdown_sync(&serial_ir.timeout_timer);
serial_ir_exit();
}
diff --git a/drivers/media/rc/streamzap.c b/drivers/media/rc/streamzap.c
index 307985d74fe8..41195ad82734 100644
--- a/drivers/media/rc/streamzap.c
+++ b/drivers/media/rc/streamzap.c
@@ -365,6 +365,7 @@ static int streamzap_probe(struct usb_interface *intf,
return 0;
rc_submit_fail:
+ rc_unregister_device(sz->rdev);
rc_free_device(sz->rdev);
usb_set_intfdata(intf, NULL);
rc_dev_fail:
diff --git a/drivers/media/rc/sunxi-cir.c b/drivers/media/rc/sunxi-cir.c
index 28e840a7e5b8..af1ee08ffdbe 100644
--- a/drivers/media/rc/sunxi-cir.c
+++ b/drivers/media/rc/sunxi-cir.c
@@ -374,8 +374,8 @@ static void sunxi_ir_remove(struct platform_device *pdev)
struct sunxi_ir *ir = platform_get_drvdata(pdev);
rc_unregister_device(ir->rc);
- rc_free_device(ir->rc);
sunxi_ir_hw_exit(&pdev->dev);
+ rc_free_device(ir->rc);
}
static void sunxi_ir_shutdown(struct platform_device *pdev)
diff --git a/drivers/media/test-drivers/vicodec/codec-fwht.h b/drivers/media/test-drivers/vicodec/codec-fwht.h
index 0eab24020e9e..4b4d39031089 100644
--- a/drivers/media/test-drivers/vicodec/codec-fwht.h
+++ b/drivers/media/test-drivers/vicodec/codec-fwht.h
@@ -61,7 +61,7 @@
* both luma and chroma components resolutions are rounded up to
* a multiple of 8
*/
-#define vic_round_dim(dim, div) (round_up((dim) / (div), 8) * (div))
+#define vic_round_dim(dim, div) round_up(dim, 8 * (div))
struct fwht_cframe_hdr {
u32 magic1;
diff --git a/drivers/media/test-drivers/vicodec/vicodec-core.c b/drivers/media/test-drivers/vicodec/vicodec-core.c
index ff9d50fb05fd..a6d12ab66a0f 100644
--- a/drivers/media/test-drivers/vicodec/vicodec-core.c
+++ b/drivers/media/test-drivers/vicodec/vicodec-core.c
@@ -654,7 +654,7 @@ static int job_ready(void *priv)
* if the header is invalid the device_run will just drop the frame
* with an error
*/
- if (!is_header_valid(&ctx->state.header) && ctx->comp_has_frame)
+ if (!is_header_valid(&ctx->state.header))
return 1;
flags = ntohl(ctx->state.header.flags);
hdr_width_div = (flags & V4L2_FWHT_FL_CHROMA_FULL_WIDTH) ? 1 : 2;
@@ -890,6 +890,8 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
{
struct vicodec_ctx *ctx = file2ctx(file);
+ struct vicodec_q_data *q_data_out =
+ get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
struct v4l2_pix_format_mplane *pix_mp;
struct v4l2_pix_format *pix;
@@ -900,6 +902,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
pix = &f->fmt.pix;
pix->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
find_fmt(f->fmt.pix.pixelformat)->id;
+ if (ctx->is_enc) {
+ pix->width = q_data_out->coded_width;
+ pix->height = q_data_out->coded_height;
+ }
pix->colorspace = ctx->state.colorspace;
pix->xfer_func = ctx->state.xfer_func;
pix->ycbcr_enc = ctx->state.ycbcr_enc;
@@ -911,6 +917,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
pix_mp = &f->fmt.pix_mp;
pix_mp->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
find_fmt(pix_mp->pixelformat)->id;
+ if (ctx->is_enc) {
+ pix_mp->width = q_data_out->coded_width;
+ pix_mp->height = q_data_out->coded_height;
+ }
pix_mp->colorspace = ctx->state.colorspace;
pix_mp->xfer_func = ctx->state.xfer_func;
pix_mp->ycbcr_enc = ctx->state.ycbcr_enc;
@@ -1595,9 +1605,9 @@ static int vicodec_start_streaming(struct vb2_queue *q,
}
state->ref_stride = q_data->coded_width * info->luma_alpha_step;
- state->ref_frame.buf = kvmalloc(total_planes_size, GFP_KERNEL);
+ state->ref_frame.buf = kvzalloc(total_planes_size, GFP_KERNEL);
state->ref_frame.luma = state->ref_frame.buf;
- new_comp_frame = kvmalloc(ctx->comp_max_size, GFP_KERNEL);
+ new_comp_frame = kvzalloc(ctx->comp_max_size, GFP_KERNEL);
if (!state->ref_frame.luma || !new_comp_frame) {
kvfree(state->ref_frame.luma);
diff --git a/drivers/media/test-drivers/vidtv/vidtv_bridge.c b/drivers/media/test-drivers/vidtv/vidtv_bridge.c
index fd69b4ee16f4..f951b877ada6 100644
--- a/drivers/media/test-drivers/vidtv/vidtv_bridge.c
+++ b/drivers/media/test-drivers/vidtv/vidtv_bridge.c
@@ -474,6 +474,7 @@ static int vidtv_bridge_dvb_init(struct vidtv_dvb *dvb)
fail_demod_probe:
for (i = i - 1; i >= 0; --i) {
dvb_unregister_frontend(dvb->fe[i]);
+ dvb_frontend_detach(dvb->fe[i]);
fail_fe:
dvb_module_release(dvb->i2c_client_tuner[i]);
fail_tuner_probe:
@@ -550,8 +551,11 @@ static void vidtv_bridge_remove(struct platform_device *pdev)
mutex_destroy(&dvb->feed_lock);
+ vidtv_stop_streaming(dvb);
+
for (i = 0; i < NUM_FE; ++i) {
dvb_unregister_frontend(dvb->fe[i]);
+ dvb_frontend_detach(dvb->fe[i]);
dvb_module_release(dvb->i2c_client_tuner[i]);
dvb_module_release(dvb->i2c_client_demod[i]);
}
diff --git a/drivers/media/test-drivers/vidtv/vidtv_demod.c b/drivers/media/test-drivers/vidtv/vidtv_demod.c
index 6e5fe402976b..3aa586004638 100644
--- a/drivers/media/test-drivers/vidtv/vidtv_demod.c
+++ b/drivers/media/test-drivers/vidtv/vidtv_demod.c
@@ -343,13 +343,6 @@ static int vidtv_diseqc_send_burst(struct dvb_frontend *fe,
return 0;
}
-static void vidtv_demod_release(struct dvb_frontend *fe)
-{
- struct vidtv_demod_state *state = fe->demodulator_priv;
-
- kfree(state);
-}
-
static const struct dvb_frontend_ops vidtv_demod_ops = {
.delsys = {
SYS_DVBT,
@@ -390,8 +383,6 @@ static const struct dvb_frontend_ops vidtv_demod_ops = {
FE_CAN_HIERARCHY_AUTO,
},
- .release = vidtv_demod_release,
-
.set_frontend = vidtv_demod_set_frontend,
.get_frontend = vidtv_demod_get_frontend,
diff --git a/drivers/media/test-drivers/vim2m.c b/drivers/media/test-drivers/vim2m.c
index bb2dd11eef0e..f4a2c4083829 100644
--- a/drivers/media/test-drivers/vim2m.c
+++ b/drivers/media/test-drivers/vim2m.c
@@ -205,6 +205,7 @@ struct vim2m_ctx {
struct vim2m_dev *dev;
struct v4l2_ctrl_handler hdl;
+ struct v4l2_ctrl *trans_num_bufs_ctrl;
/* Processed buffers in this transaction */
u8 num_processed;
@@ -1258,9 +1259,27 @@ static int vim2m_start_streaming(struct vb2_queue *q, unsigned int count)
ctx->aborting = 0;
q_data->sequence = 0;
+ v4l2_ctrl_grab(ctx->trans_num_bufs_ctrl, true);
+
return 0;
}
+static bool vim2m_other_queue_is_streaming(struct vim2m_ctx *ctx,
+ struct vb2_queue *q)
+{
+ struct vb2_queue *other_vq;
+
+ if (!ctx->fh.m2m_ctx)
+ return false;
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type))
+ other_vq = v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx);
+ else
+ other_vq = v4l2_m2m_get_src_vq(ctx->fh.m2m_ctx);
+
+ return vb2_is_streaming(other_vq);
+}
+
static void vim2m_stop_streaming(struct vb2_queue *q)
{
struct vim2m_ctx *ctx = vb2_get_drv_priv(q);
@@ -1274,11 +1293,14 @@ static void vim2m_stop_streaming(struct vb2_queue *q)
else
vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
if (!vbuf)
- return;
+ break;
v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
&ctx->hdl);
v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
}
+
+ if (!vim2m_other_queue_is_streaming(ctx, q))
+ v4l2_ctrl_grab(ctx->trans_num_bufs_ctrl, false);
}
static void vim2m_buf_request_complete(struct vb2_buffer *vb)
@@ -1380,7 +1402,8 @@ static int vim2m_open(struct file *file)
vim2m_ctrl_trans_time_msec.def = default_transtime;
v4l2_ctrl_new_custom(hdl, &vim2m_ctrl_trans_time_msec, NULL);
- v4l2_ctrl_new_custom(hdl, &vim2m_ctrl_trans_num_bufs, NULL);
+ ctx->trans_num_bufs_ctrl =
+ v4l2_ctrl_new_custom(hdl, &vim2m_ctrl_trans_num_bufs, NULL);
if (hdl->error) {
rc = hdl->error;
v4l2_ctrl_handler_free(hdl);
@@ -1435,10 +1458,10 @@ static int vim2m_release(struct file *file)
v4l2_fh_del(&ctx->fh, file);
v4l2_fh_exit(&ctx->fh);
- v4l2_ctrl_handler_free(&ctx->hdl);
mutex_lock(&dev->dev_mutex);
v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
mutex_unlock(&dev->dev_mutex);
+ v4l2_ctrl_handler_free(&ctx->hdl);
kfree(ctx);
atomic_dec(&dev->num_inst);
diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
index 83dcc9d61ee0..7cb544fdfddb 100644
--- a/drivers/media/test-drivers/vimc/vimc-sensor.c
+++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
@@ -97,18 +97,14 @@ static void vimc_sensor_update_frame_timing(struct v4l2_subdev *sd,
{
struct vimc_sensor_device *vsensor =
container_of(sd, struct vimc_sensor_device, sd);
- u64 pixel_rate = vsensor->pixel_rate->val;
+ u32 pixel_rate = vsensor->pixel_rate->val;
u32 hts = width + vsensor->hblank->val;
u32 vts = height + vsensor->vblank->val;
u64 total_pixels = (u64)hts * vts;
u64 frame_interval_ns;
- /* Sanity check, pixel rate is fixed and fits in 32 bits. */
- if (WARN_ON(pixel_rate >= 0x100000000))
- return;
-
frame_interval_ns = total_pixels * NSEC_PER_SEC;
- do_div(frame_interval_ns, (u32)pixel_rate);
+ do_div(frame_interval_ns, pixel_rate);
vsensor->hw.fps_jiffies = nsecs_to_jiffies(frame_interval_ns);
if (vsensor->hw.fps_jiffies == 0)
vsensor->hw.fps_jiffies = 1;
diff --git a/drivers/media/test-drivers/vivid/vivid-core.c b/drivers/media/test-drivers/vivid/vivid-core.c
index 62cfb5feb2cf..c042d92db175 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.c
+++ b/drivers/media/test-drivers/vivid/vivid-core.c
@@ -854,6 +854,7 @@ static void vivid_dev_release(struct v4l2_device *v4l2_dev)
struct vivid_dev *dev = container_of(v4l2_dev, struct vivid_dev, v4l2_dev);
cancel_work_sync(&dev->update_hdmi_ctrl_work);
+ cancel_work_sync(&dev->update_svid_ctrl_work);
vivid_free_controls(dev);
v4l2_device_unregister(&dev->v4l2_dev);
#ifdef CONFIG_MEDIA_CONTROLLER
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.c b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
index e20449084709..147af0f9b077 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-cap.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
@@ -570,6 +570,7 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
const struct vivid_fmt *fmt;
unsigned bytesperline, max_bpl;
unsigned factor = 1;
+ unsigned int vdiv = 1;
unsigned w, h;
unsigned p;
bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
@@ -622,6 +623,18 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
mp->height = r.height / factor;
}
+ /*
+ * The chroma planes of vertically subsampled formats hold
+ * height / vdownsampling lines. If the height is not a multiple of
+ * the subsampling factor, then the buffer size calculations round
+ * that number down while the test pattern generator rounds it up,
+ * so the generator writes one line past the end of the buffer.
+ * Round the height down to keep both in sync.
+ */
+ for (p = 0; p < fmt->planes; p++)
+ vdiv = max(vdiv, fmt->vdownsampling[p]);
+ mp->height = rounddown(mp->height, vdiv);
+
/* This driver supports custom bytesperline values */
mp->num_planes = fmt->buffers;
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index 445cbeb7abae..c3826b3b8a2e 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -697,6 +697,10 @@ static int au0828_usb_probe(struct usb_interface *interface,
retval = au0828_v4l2_device_register(interface, dev);
if (retval) {
au0828_usb_v4l2_media_release(dev);
+#ifdef CONFIG_MEDIA_CONTROLLER
+ media_device_delete(dev->media_dev, KBUILD_MODNAME,
+ THIS_MODULE);
+#endif
mutex_unlock(&dev->lock);
kfree(dev);
return retval;
diff --git a/drivers/media/usb/au0828/au0828-dvb.c b/drivers/media/usb/au0828/au0828-dvb.c
index d3fff47f0edf..601a12e122f1 100644
--- a/drivers/media/usb/au0828/au0828-dvb.c
+++ b/drivers/media/usb/au0828/au0828-dvb.c
@@ -163,9 +163,6 @@ static int stop_urb_transfer(struct au0828_dev *dev)
dprintk(2, "%s()\n", __func__);
- if (!dev->urb_streaming)
- return 0;
-
if (dev->bulk_timeout_running == 1) {
dev->bulk_timeout_running = 0;
timer_delete(&dev->bulk_timeout);
@@ -179,6 +176,7 @@ static int stop_urb_transfer(struct au0828_dev *dev)
kfree(dev->urbs[i]->transfer_buffer);
usb_free_urb(dev->urbs[i]);
+ dev->urbs[i] = NULL;
}
}
@@ -200,8 +198,10 @@ static int start_urb_transfer(struct au0828_dev *dev)
for (i = 0; i < URB_COUNT; i++) {
dev->urbs[i] = usb_alloc_urb(0, GFP_KERNEL);
- if (!dev->urbs[i])
- return -ENOMEM;
+ if (!dev->urbs[i]) {
+ ret = -ENOMEM;
+ goto err;
+ }
purb = dev->urbs[i];
@@ -217,7 +217,7 @@ static int start_urb_transfer(struct au0828_dev *dev)
ret = -ENOMEM;
pr_err("%s: failed big buffer allocation, err = %d\n",
__func__, ret);
- return ret;
+ goto err;
}
purb->status = -EINPROGRESS;
@@ -235,10 +235,9 @@ static int start_urb_transfer(struct au0828_dev *dev)
for (i = 0; i < URB_COUNT; i++) {
ret = usb_submit_urb(dev->urbs[i], GFP_ATOMIC);
if (ret != 0) {
- stop_urb_transfer(dev);
pr_err("%s: failed urb submission, err = %d\n",
__func__, ret);
- return ret;
+ goto err;
}
}
@@ -249,6 +248,10 @@ static int start_urb_transfer(struct au0828_dev *dev)
dev->bulk_timeout_running = 1;
return 0;
+
+err:
+ stop_urb_transfer(dev);
+ return ret;
}
static void au0828_start_transport(struct au0828_dev *dev)
diff --git a/drivers/media/usb/cx231xx/cx231xx-audio.c b/drivers/media/usb/cx231xx/cx231xx-audio.c
index b24ceef497e4..af04fb104cc4 100644
--- a/drivers/media/usb/cx231xx/cx231xx-audio.c
+++ b/drivers/media/usb/cx231xx/cx231xx-audio.c
@@ -581,12 +581,19 @@ static int cx231xx_audio_init(struct cx231xx *dev)
dev_dbg(dev->dev,
"probing for cx231xx non standard usbaudio\n");
+ /*
+ * Extension init errors are ignored by the cx231xx core, so fini()
+ * must be safe even if initialization fails part way through.
+ */
+ spin_lock_init(&adev->slock);
+ INIT_WORK(&dev->wq_trigger, audio_trigger);
+ atomic_set(&dev->stream_started, 0);
+
err = snd_card_new(dev->dev, index[devnr], "Cx231xx Audio",
THIS_MODULE, 0, &card);
if (err < 0)
return err;
- spin_lock_init(&adev->slock);
err = snd_pcm_new(card, "Cx231xx Audio", 0, 0, 1, &pcm);
if (err < 0)
goto err_free_card;
@@ -601,8 +608,6 @@ static int cx231xx_audio_init(struct cx231xx *dev)
strscpy(card->shortname, "Cx231xx Audio", sizeof(card->shortname));
strscpy(card->longname, "Conexant cx231xx Audio", sizeof(card->longname));
- INIT_WORK(&dev->wq_trigger, audio_trigger);
-
err = snd_card_register(card);
if (err < 0)
goto err_free_card;
@@ -656,8 +661,10 @@ static int cx231xx_audio_init(struct cx231xx *dev)
err_free_pkt_size:
kfree(adev->alt_max_pkt_size);
+ adev->alt_max_pkt_size = NULL;
err_free_card:
snd_card_free(card);
+ adev->sndcard = NULL;
return err;
}
@@ -674,6 +681,8 @@ static int cx231xx_audio_fini(struct cx231xx *dev)
return 0;
}
+ disable_work_sync(&dev->wq_trigger);
+
if (dev->adev.sndcard) {
snd_card_free_when_closed(dev->adev.sndcard);
kfree(dev->adev.alt_max_pkt_size);
diff --git a/drivers/media/usb/cx231xx/cx231xx-cards.c b/drivers/media/usb/cx231xx/cx231xx-cards.c
index 69b24205bc56..b0941cd2a2d8 100644
--- a/drivers/media/usb/cx231xx/cx231xx-cards.c
+++ b/drivers/media/usb/cx231xx/cx231xx-cards.c
@@ -1826,7 +1826,7 @@ static int cx231xx_usb_probe(struct usb_interface *interface,
retval = cx231xx_init_v4l2(dev, udev, interface, isoc_pipe);
if (retval)
- goto err_init;
+ goto err_video_alt;
if (dev->current_pcb_config.ts1_source != 0xff) {
/* compute alternate max packet sizes for TS1 */
diff --git a/drivers/media/usb/cx231xx/cx231xx.h b/drivers/media/usb/cx231xx/cx231xx.h
index 19f5036a78d7..f85d054fbb5c 100644
--- a/drivers/media/usb/cx231xx/cx231xx.h
+++ b/drivers/media/usb/cx231xx/cx231xx.h
@@ -463,7 +463,7 @@ struct cx231xx_i2c_xfer_data {
u8 direction; /* 1 - IN, 0 - OUT */
u8 saddr_len; /* sub address len */
u16 saddr_dat; /* sub addr data */
- u8 buf_size; /* buffer size */
+ u16 buf_size; /* buffer size */
u8 *p_buffer; /* pointer to the buffer */
};
diff --git a/drivers/media/usb/dvb-usb-v2/rtl28xxu.c b/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
index 487c6ab784ab..d2957e4bf017 100644
--- a/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
+++ b/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
@@ -1391,8 +1391,16 @@ static int rtl2832u_tuner_attach(struct dvb_usb_adapter *adap)
"rtl2832_sdr",
PLATFORM_DEVID_AUTO,
&pdata, sizeof(pdata));
- if (IS_ERR(pdev) || pdev->dev.driver == NULL)
+ if (IS_ERR(pdev))
break;
+ if (!pdev->dev.driver) {
+ platform_device_unregister(pdev);
+ break;
+ }
+ if (!try_module_get(pdev->dev.driver->owner)) {
+ platform_device_unregister(pdev);
+ break;
+ }
dev->platform_device_sdr = pdev;
break;
default:
@@ -1426,8 +1434,10 @@ static int rtl28xxu_tuner_detach(struct dvb_usb_adapter *adap)
/* remove platform SDR */
pdev = dev->platform_device_sdr;
- if (pdev)
+ if (pdev) {
+ module_put(pdev->dev.driver->owner);
platform_device_unregister(pdev);
+ }
/* remove I2C tuner */
client = dev->i2c_client_tuner;
diff --git a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
index 0fb3fa6100e4..675d9d1d4f47 100644
--- a/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
+++ b/drivers/media/usb/dvb-usb/dvb-usb-firmware.c
@@ -141,6 +141,8 @@ int dvb_usb_get_hexline(const struct firmware *fw, struct hexline *hx,
if (hx->type == 0x04) {
/* b[4] and b[5] are the Extended linear address record data field */
+ if (hx->len != 2)
+ return -EINVAL;
hx->addr |= (b[4] << 24) | (b[5] << 16);
/* hx->len -= 2;
data_offs += 2; */
diff --git a/drivers/media/usb/em28xx/em28xx-core.c b/drivers/media/usb/em28xx/em28xx-core.c
index 0b998e7cafef..62a1c25a2de1 100644
--- a/drivers/media/usb/em28xx/em28xx-core.c
+++ b/drivers/media/usb/em28xx/em28xx-core.c
@@ -1080,9 +1080,6 @@ int em28xx_alloc_urbs(struct em28xx *dev, enum em28xx_mode mode, int xfer_bulk,
usb_bufs->buf[i] = kzalloc(sb_size, GFP_KERNEL);
if (!usb_bufs->buf[i]) {
- for (i--; i >= 0; i--)
- kfree(usb_bufs->buf[i]);
-
em28xx_uninit_usb_xfer(dev, mode);
return -ENOMEM;
}
diff --git a/drivers/media/usb/em28xx/em28xx-video.c b/drivers/media/usb/em28xx/em28xx-video.c
index 79af154767c1..e441f221dbb7 100644
--- a/drivers/media/usb/em28xx/em28xx-video.c
+++ b/drivers/media/usb/em28xx/em28xx-video.c
@@ -3001,7 +3001,7 @@ static int em28xx_v4l2_init(struct em28xx *dev)
v4l2->vbi_dev.queue->lock = &v4l2->vb_vbi_queue_lock;
v4l2->vbi_dev.device_caps = V4L2_CAP_STREAMING |
V4L2_CAP_READWRITE | V4L2_CAP_VBI_CAPTURE;
- if ((v4l2->vdev.device_caps & V4L2_CAP_TUNER) == 0)
+ if (v4l2->vdev.device_caps & V4L2_CAP_TUNER)
v4l2->vbi_dev.device_caps |= V4L2_CAP_TUNER;
/* disable inapplicable ioctls */
diff --git a/drivers/media/usb/go7007/go7007-usb.c b/drivers/media/usb/go7007/go7007-usb.c
index c0cb92fa6ab9..dc5b8750a110 100644
--- a/drivers/media/usb/go7007/go7007-usb.c
+++ b/drivers/media/usb/go7007/go7007-usb.c
@@ -1029,10 +1029,16 @@ static const struct i2c_algorithm go7007_usb_algo = {
.functionality = go7007_usb_functionality,
};
+static const struct i2c_adapter_quirks go7007_usb_quirks = {
+ .max_write_len = 12,
+ .max_read_len = 15,
+};
+
static struct i2c_adapter go7007_usb_adap_templ = {
.owner = THIS_MODULE,
.name = "WIS GO7007SB EZ-USB",
.algo = &go7007_usb_algo,
+ .quirks = &go7007_usb_quirks,
};
/********************* USB add/remove functions *********************/
@@ -1320,6 +1326,8 @@ static int go7007_usb_probe(struct usb_interface *intf,
return 0;
allocfail:
+ if (go->i2c_adapter_online)
+ i2c_del_adapter(&go->i2c_adapter);
go7007_usb_release(go);
kfree(go);
return -ENOMEM;
diff --git a/drivers/media/usb/pvrusb2/pvrusb2-sysfs.c b/drivers/media/usb/pvrusb2/pvrusb2-sysfs.c
index dba43d4c24f2..678e069b4c92 100644
--- a/drivers/media/usb/pvrusb2/pvrusb2-sysfs.c
+++ b/drivers/media/usb/pvrusb2/pvrusb2-sysfs.c
@@ -633,6 +633,7 @@ static void class_dev_create(struct pvr2_sysfs *sfp)
if (ret) {
pvr2_trace(PVR2_TRACE_ERROR_LEGS,
"device_register failed");
+ put_device(class_dev->parent);
put_device(class_dev);
return;
}
diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index 3ca108b83f1d..ea28c4e819bd 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -2209,7 +2209,7 @@ bool uvc_ctrl_status_event_async(struct urb *urb, struct uvc_video_chain *chain,
struct uvc_device *dev = chain->dev;
struct uvc_ctrl_work *w = &dev->async_ctrl;
- if (list_empty(&ctrl->info.mappings))
+ if (!ctrl->initialized || list_empty(&ctrl->info.mappings))
return false;
w->data = data;
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 65db7340ad38..4de8aa3ef7d2 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -323,7 +323,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_NV20, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_NV42, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
- { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 4, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
{ .format = V4L2_PIX_FMT_P012, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 4, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
{ .format = V4L2_PIX_FMT_YUV410, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 4, .vdiv = 4 },
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 648b88c868bc..6e751a043a21 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -115,6 +115,7 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl, u32 idx,
struct v4l2_ctrl_fwht_params *p_fwht_params;
struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix;
struct v4l2_ctrl_av1_sequence *p_av1_sequence;
+ struct v4l2_ctrl_hevc_sps *p_hevc_sps;
void *p = ptr.p + idx * ctrl->elem_size;
if (ctrl->p_def.p_const)
@@ -188,6 +189,12 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl, u32 idx,
*/
memset(p_h264_scaling_matrix, 16, sizeof(*p_h264_scaling_matrix));
break;
+ case V4L2_CTRL_TYPE_HEVC_SPS:
+ p_hevc_sps = p;
+
+ /* 4:2:0 */
+ p_hevc_sps->chroma_format_idc = 1;
+ break;
}
}
@@ -2554,7 +2561,10 @@ void v4l2_ctrl_cluster(unsigned ncontrols, struct v4l2_ctrl **controls)
int i;
/* The first control is the master control and it must not be NULL */
- if (WARN_ON(ncontrols == 0 || controls[0] == NULL))
+ if (WARN_ON(ncontrols == 0))
+ return;
+
+ if (!controls[0])
return;
for (i = 0; i < ncontrols; i++) {
@@ -2576,8 +2586,13 @@ void v4l2_ctrl_auto_cluster(unsigned ncontrols, struct v4l2_ctrl **controls,
u32 flag = 0;
int i;
+ if (WARN_ON(ncontrols <= 1))
+ return;
+
+ if (!master)
+ return;
+
v4l2_ctrl_cluster(ncontrols, controls);
- WARN_ON(ncontrols <= 1);
WARN_ON(manual_val < master->minimum || manual_val > master->maximum);
WARN_ON(set_volatile && !has_op(master, g_volatile_ctrl));
master->is_auto = true;
diff --git a/drivers/media/v4l2-core/v4l2-isp.c b/drivers/media/v4l2-core/v4l2-isp.c
index 1eb46e080afa..8e6c2ef326aa 100644
--- a/drivers/media/v4l2-core/v4l2-isp.c
+++ b/drivers/media/v4l2-core/v4l2-isp.c
@@ -99,12 +99,25 @@ int v4l2_isp_params_validate_buffer(struct device *dev, struct vb2_buffer *vb,
return -EINVAL;
}
+ /*
+ * An empty type info entry denotes a block type the driver
+ * does not support. Reject the buffer instead of ignoring the
+ * block: accepting it silently would let userspace fill it
+ * with data that a later kernel, once it implements the type,
+ * would validate and possibly reject.
+ */
+ info = &type_info[block->type];
+ if (!info->size) {
+ dev_dbg(dev, "Unsupported block type %u at offset %zu\n",
+ block->type, block_offset);
+ return -EINVAL;
+ }
+
/*
* Match the block reported size against the type info provided
* one, but allow the block to only contain the header in
* case it is going to be disabled.
*/
- info = &type_info[block->type];
if (block->size != info->size &&
(!(block->flags & V4L2_ISP_PARAMS_FL_BLOCK_DISABLE) ||
block->size != sizeof(*block))) {
diff --git a/drivers/memory/brcmstb_dpfe.c b/drivers/memory/brcmstb_dpfe.c
index 08d9e05b1b33..7b02c36c7fca 100644
--- a/drivers/memory/brcmstb_dpfe.c
+++ b/drivers/memory/brcmstb_dpfe.c
@@ -182,6 +182,8 @@ struct brcmstb_dpfe_priv {
void __iomem *regs;
void __iomem *dmem;
void __iomem *imem;
+ resource_size_t regs_size;
+ resource_size_t dmem_size;
struct device *dev;
const struct dpfe_api *dpfe_api;
struct mutex lock;
@@ -401,9 +403,14 @@ static void __iomem *get_msg_ptr(struct brcmstb_dpfe_priv *priv, u32 response,
*/
switch (msg_type) {
case 1:
+ if (DCPU_MSG_RAM_START + offset + DRAM_VENDOR_ERROR +
+ sizeof(u32) > priv->regs_size)
+ goto bad_offset;
ptr = priv->regs + DCPU_MSG_RAM_START + offset;
break;
case 0:
+ if (offset + DRAM_VENDOR_ERROR + sizeof(u32) > priv->dmem_size)
+ goto bad_offset;
ptr = priv->dmem + offset;
break;
default:
@@ -415,6 +422,12 @@ static void __iomem *get_msg_ptr(struct brcmstb_dpfe_priv *priv, u32 response,
}
return ptr;
+
+bad_offset:
+ dev_err(priv->dev, "DCPU returned out-of-range offset %#x\n", offset);
+ if (buf && size)
+ *size = sprintf(buf, "ERROR: DCPU offset out of range\n");
+ return NULL;
}
static void __finalize_command(struct brcmstb_dpfe_priv *priv)
@@ -490,6 +503,8 @@ static int __send_command(struct brcmstb_dpfe_priv *priv, unsigned int cmd,
for (i = 0; i < MSG_FIELD_MAX; i++)
result[i] = readl_relaxed(regs + DCPU_MSG_RAM(i));
chksum_idx = result[MSG_ARG_COUNT] + MSG_ARG_COUNT + 1;
+ if (chksum_idx >= MSG_FIELD_MAX)
+ ret = -EINVAL;
}
/* Tell DCPU we are done */
@@ -858,6 +873,7 @@ static int brcmstb_dpfe_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct brcmstb_dpfe_priv *priv;
+ struct resource *res;
int ret;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
@@ -869,17 +885,21 @@ static int brcmstb_dpfe_probe(struct platform_device *pdev)
mutex_init(&priv->lock);
platform_set_drvdata(pdev, priv);
- priv->regs = devm_platform_ioremap_resource_byname(pdev, "dpfe-cpu");
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dpfe-cpu");
+ priv->regs = devm_ioremap_resource(dev, res);
if (IS_ERR(priv->regs)) {
dev_err(dev, "couldn't map DCPU registers\n");
return -ENODEV;
}
+ priv->regs_size = resource_size(res);
- priv->dmem = devm_platform_ioremap_resource_byname(pdev, "dpfe-dmem");
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dpfe-dmem");
+ priv->dmem = devm_ioremap_resource(dev, res);
if (IS_ERR(priv->dmem)) {
dev_err(dev, "Couldn't map DCPU data memory\n");
return -ENOENT;
}
+ priv->dmem_size = resource_size(res);
priv->imem = devm_platform_ioremap_resource_byname(pdev, "dpfe-imem");
if (IS_ERR(priv->imem)) {
diff --git a/drivers/memory/renesas-rpc-if.c b/drivers/memory/renesas-rpc-if.c
index 3755956ae906..104c28532c20 100644
--- a/drivers/memory/renesas-rpc-if.c
+++ b/drivers/memory/renesas-rpc-if.c
@@ -34,6 +34,7 @@ static const struct regmap_access_table rpcif_volatile_table = {
static const struct regmap_range xspi_volatile_ranges[] = {
regmap_reg_range(XSPI_CDD0BUF0, XSPI_CDD0BUF0),
+ regmap_reg_range(XSPI_CDD1BUF0, XSPI_CDD1BUF0),
};
static const struct regmap_access_table xspi_volatile_table = {
diff --git a/drivers/memory/stm32_omm.c b/drivers/memory/stm32_omm.c
index 84eb23c824f5..3599683ab04a 100644
--- a/drivers/memory/stm32_omm.c
+++ b/drivers/memory/stm32_omm.c
@@ -274,6 +274,8 @@ static int stm32_omm_configure(struct device *dev)
writel_relaxed(omm->cr, omm->io_base + OMM_CR);
ret = stm32_omm_set_amcr(dev, true);
+ if (ret && (mux & CR_MUXEN))
+ stm32_omm_toggle_child_clock(dev, false);
error:
pm_runtime_put_sync_suspend(dev);
diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
index e4fd4572472f..857ca3bb0d5b 100644
--- a/drivers/mfd/Kconfig
+++ b/drivers/mfd/Kconfig
@@ -1006,6 +1006,8 @@ config MFD_MAX77705
depends on I2C
select MFD_CORE
select MFD_SIMPLE_MFD_I2C
+ select REGMAP_I2C
+ select REGMAP_IRQ
help
Say yes here to add support for Maxim Integrated MAX77705 PMIC.
This is a Power Management IC with Charger, safe LDOs, Flash, Haptic
diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c
index 86fa99022cb3..44fc2ce83cdd 100644
--- a/drivers/mfd/ab8500-core.c
+++ b/drivers/mfd/ab8500-core.c
@@ -1194,39 +1194,41 @@ static int ab8500_probe(struct platform_device *pdev)
return ret;
if (is_ab9540(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab9540_devs,
- ARRAY_SIZE(ab9540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab9540_devs,
+ ARRAY_SIZE(ab9540_devs), NULL,
+ 0, ab8500->domain);
else if (is_ab8540(ab8500)) {
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_devs,
- ARRAY_SIZE(ab8540_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8540_devs,
+ ARRAY_SIZE(ab8540_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
if (is_ab8540_1p2_or_earlier(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut1_devs,
- ARRAY_SIZE(ab8540_cut1_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut1_devs,
+ ARRAY_SIZE(ab8540_cut1_devs),
+ NULL, 0, ab8500->domain);
else /* ab8540 >= cut2 */
- ret = mfd_add_devices(ab8500->dev, 0, ab8540_cut2_devs,
- ARRAY_SIZE(ab8540_cut2_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0,
+ ab8540_cut2_devs,
+ ARRAY_SIZE(ab8540_cut2_devs),
+ NULL, 0, ab8500->domain);
} else if (is_ab8505(ab8500))
- ret = mfd_add_devices(ab8500->dev, 0, ab8505_devs,
- ARRAY_SIZE(ab8505_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8505_devs,
+ ARRAY_SIZE(ab8505_devs), NULL,
+ 0, ab8500->domain);
else
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_devs,
- ARRAY_SIZE(ab8500_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_devs,
+ ARRAY_SIZE(ab8500_devs), NULL,
+ 0, ab8500->domain);
if (ret)
return ret;
/* Add battery management devices */
- ret = mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
- ARRAY_SIZE(ab8500_bm_devs), NULL,
- 0, ab8500->domain);
+ ret = devm_mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
+ ARRAY_SIZE(ab8500_bm_devs), NULL,
+ 0, ab8500->domain);
if (ret)
dev_err(ab8500->dev, "error adding bm devices\n");
diff --git a/drivers/mfd/bq257xx.c b/drivers/mfd/bq257xx.c
index 054342c60b73..01f5fc97f538 100644
--- a/drivers/mfd/bq257xx.c
+++ b/drivers/mfd/bq257xx.c
@@ -52,8 +52,18 @@ static const struct regmap_access_table bq25792_writeable_regs = {
};
static const struct regmap_range bq25792_volatile_reg_ranges[] = {
+ regmap_reg_range(BQ25792_REG05_INPUT_VOLTAGE_LIMIT,
+ BQ25792_REG06_INPUT_CURRENT_LIMIT + 1),
+ regmap_reg_range(BQ25792_REG0F_CHARGER_CONTROL_0,
+ BQ25792_REG0F_CHARGER_CONTROL_0),
+ regmap_reg_range(BQ25792_REG11_CHARGER_CONTROL_2,
+ BQ25792_REG11_CHARGER_CONTROL_2),
+ regmap_reg_range(BQ25792_REG13_CHARGER_CONTROL_4,
+ BQ25792_REG13_CHARGER_CONTROL_4),
regmap_reg_range(BQ25792_REG19_ICO_CURRENT_LIMIT,
BQ25792_REG27_FAULT_FLAG_1),
+ regmap_reg_range(BQ25792_REG2E_ADC_CONTROL,
+ BQ25792_REG2E_ADC_CONTROL),
regmap_reg_range(BQ25792_REG31_IBUS_ADC,
BQ25792_REG47_DPDM_DRIVER),
};
diff --git a/drivers/mfd/da903x.c b/drivers/mfd/da903x.c
index e86b39de3303..e4ad48e1f139 100644
--- a/drivers/mfd/da903x.c
+++ b/drivers/mfd/da903x.c
@@ -421,6 +421,14 @@ static irqreturn_t da903x_irq_handler(int irq, void *data)
return IRQ_HANDLED;
}
+static void da903x_cancel_irq_work(void *data)
+{
+ struct da903x_chip *chip = data;
+
+ disable_irq(chip->client->irq);
+ cancel_work_sync(&chip->irq_work);
+}
+
static const struct da903x_chip_ops da903x_ops[] = {
[0] = {
.init_chip = da9030_init_chip,
@@ -456,6 +464,13 @@ static int da903x_remove_subdevs(struct da903x_chip *chip)
return device_for_each_child(chip->dev, NULL, __remove_subdev);
}
+static void da903x_remove_subdevs_action(void *data)
+{
+ struct da903x_chip *chip = data;
+
+ da903x_remove_subdevs(chip);
+}
+
static int da903x_add_subdevs(struct da903x_chip *chip,
struct da903x_platform_data *pdata)
{
@@ -484,7 +499,6 @@ static int da903x_add_subdevs(struct da903x_chip *chip,
return 0;
failed:
- da903x_remove_subdevs(chip);
return ret;
}
@@ -520,6 +534,11 @@ static int da903x_probe(struct i2c_client *client)
chip->ops->mask_events(chip, chip->events_mask);
chip->ops->read_events(chip, &tmp);
+ ret = devm_add_action_or_reset(&client->dev,
+ da903x_remove_subdevs_action, chip);
+ if (ret)
+ return ret;
+
ret = devm_request_irq(&client->dev, client->irq, da903x_irq_handler,
IRQF_TRIGGER_FALLING,
"da903x", chip);
@@ -529,14 +548,12 @@ static int da903x_probe(struct i2c_client *client)
return ret;
}
- return da903x_add_subdevs(chip, pdata);
-}
-
-static void da903x_remove(struct i2c_client *client)
-{
- struct da903x_chip *chip = i2c_get_clientdata(client);
+ ret = devm_add_action_or_reset(&client->dev, da903x_cancel_irq_work,
+ chip);
+ if (ret)
+ return ret;
- da903x_remove_subdevs(chip);
+ return da903x_add_subdevs(chip, pdata);
}
static struct i2c_driver da903x_driver = {
@@ -544,7 +561,6 @@ static struct i2c_driver da903x_driver = {
.name = "da903x",
},
.probe = da903x_probe,
- .remove = da903x_remove,
.id_table = da903x_id_table,
};
diff --git a/drivers/mfd/tps65217.c b/drivers/mfd/tps65217.c
index c240fac0ede7..569c28c8b20f 100644
--- a/drivers/mfd/tps65217.c
+++ b/drivers/mfd/tps65217.c
@@ -146,6 +146,31 @@ static const struct irq_domain_ops tps65217_irq_domain_ops = {
.map = tps65217_irq_map,
};
+static void tps65217_irq_cleanup(void *data)
+{
+ struct tps65217 *tps = data;
+ unsigned int virq;
+
+ if (!tps->irq_domain)
+ return;
+
+ for (int i = 0; i < TPS65217_NUM_IRQ; i++) {
+ virq = irq_find_mapping(tps->irq_domain, i);
+ if (virq)
+ irq_dispose_mapping(virq);
+ }
+
+ irq_domain_remove(tps->irq_domain);
+ tps->irq_domain = NULL;
+}
+
+static void tps65217_irq_wake_disable(void *data)
+{
+ unsigned long irq = (unsigned long)data;
+
+ disable_irq_wake(irq);
+}
+
static int tps65217_irq_init(struct tps65217 *tps, int irq)
{
int ret;
@@ -155,8 +180,12 @@ static int tps65217_irq_init(struct tps65217 *tps, int irq)
/* Mask all interrupt sources */
tps->irq_mask = TPS65217_INT_MASK;
- tps65217_set_bits(tps, TPS65217_REG_INT, TPS65217_INT_MASK,
- TPS65217_INT_MASK, TPS65217_PROTECT_NONE);
+ ret = tps65217_set_bits(tps, TPS65217_REG_INT, TPS65217_INT_MASK,
+ TPS65217_INT_MASK, TPS65217_PROTECT_NONE);
+ if (ret) {
+ dev_err(tps->dev, "Failed to mask interrupt sources: %d\n", ret);
+ return ret;
+ }
tps->irq_domain = irq_domain_create_linear(dev_fwnode(tps->dev), TPS65217_NUM_IRQ,
&tps65217_irq_domain_ops, tps);
@@ -165,6 +194,10 @@ static int tps65217_irq_init(struct tps65217 *tps, int irq)
return -ENOMEM;
}
+ ret = devm_add_action_or_reset(tps->dev, tps65217_irq_cleanup, tps);
+ if (ret)
+ return ret;
+
ret = devm_request_threaded_irq(tps->dev, irq, NULL,
tps65217_irq_thread, IRQF_ONESHOT,
"tps65217-irq", tps);
@@ -174,7 +207,16 @@ static int tps65217_irq_init(struct tps65217 *tps, int irq)
return ret;
}
- enable_irq_wake(irq);
+ ret = enable_irq_wake(irq);
+ if (ret) {
+ dev_warn(tps->dev, "failed to enable IRQ wake: %d\n", ret);
+ return 0;
+ }
+
+ ret = devm_add_action_or_reset(tps->dev, tps65217_irq_wake_disable,
+ (void *)(unsigned long)irq);
+ if (ret)
+ return ret;
return 0;
}
@@ -333,7 +375,9 @@ static int tps65217_probe(struct i2c_client *client)
}
if (client->irq) {
- tps65217_irq_init(tps, client->irq);
+ ret = tps65217_irq_init(tps, client->irq);
+ if (ret)
+ return ret;
} else {
int i;
@@ -373,22 +417,6 @@ static int tps65217_probe(struct i2c_client *client)
return 0;
}
-static void tps65217_remove(struct i2c_client *client)
-{
- struct tps65217 *tps = i2c_get_clientdata(client);
- unsigned int virq;
- int i;
-
- for (i = 0; i < TPS65217_NUM_IRQ; i++) {
- virq = irq_find_mapping(tps->irq_domain, i);
- if (virq)
- irq_dispose_mapping(virq);
- }
-
- irq_domain_remove(tps->irq_domain);
- tps->irq_domain = NULL;
-}
-
static const struct i2c_device_id tps65217_id_table[] = {
{"tps65217", TPS65217},
{ /* sentinel */ }
@@ -402,7 +430,6 @@ static struct i2c_driver tps65217_driver = {
},
.id_table = tps65217_id_table,
.probe = tps65217_probe,
- .remove = tps65217_remove,
};
static int __init tps65217_init(void)
diff --git a/drivers/mfd/twl-core.c b/drivers/mfd/twl-core.c
index c024a28b057e..128cd3f6c60c 100644
--- a/drivers/mfd/twl-core.c
+++ b/drivers/mfd/twl-core.c
@@ -719,6 +719,11 @@ static const struct mfd_cell twl6032_cells[] = {
{ .name = "twl6032-clk" },
};
+static void twl_unregister_device(void *data)
+{
+ platform_device_unregister(data);
+}
+
/* NOTE: This driver only handles a single twl4030/tps659x0 chip */
static int
twl_probe(struct i2c_client *client)
@@ -754,17 +759,22 @@ twl_probe(struct i2c_client *client)
return status;
}
+ status = devm_add_action_or_reset(&client->dev, twl_unregister_device,
+ pdev);
+ if (status)
+ return status;
+
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) {
dev_dbg(&client->dev, "can't talk I2C?\n");
status = -EIO;
- goto free;
+ goto out;
}
twl_priv = devm_kzalloc(&client->dev, sizeof(struct twl_private),
GFP_KERNEL);
if (!twl_priv) {
status = -ENOMEM;
- goto free;
+ goto out;
}
if ((id->driver_data) & TWL6030_CLASS) {
@@ -784,7 +794,7 @@ twl_probe(struct i2c_client *client)
GFP_KERNEL);
if (!twl_priv->twl_modules) {
status = -ENOMEM;
- goto free;
+ goto out;
}
for (i = 0; i < num_slaves; i++) {
@@ -880,7 +890,7 @@ twl_probe(struct i2c_client *client)
status = devm_mfd_add_devices(&client->dev, PLATFORM_DEVID_NONE,
cells, num_cells, NULL, 0, NULL);
if (status < 0)
- goto free;
+ goto out;
if (of_device_is_system_power_controller(node)) {
if (!pm_power_off)
@@ -896,10 +906,7 @@ twl_probe(struct i2c_client *client)
fail:
if (status < 0)
twl_remove(client);
-free:
- if (status < 0)
- platform_device_unregister(pdev);
-
+out:
return status;
}
diff --git a/drivers/mfd/wcd934x.c b/drivers/mfd/wcd934x.c
index 3c3080e8c8cf..d57796933d87 100644
--- a/drivers/mfd/wcd934x.c
+++ b/drivers/mfd/wcd934x.c
@@ -2,6 +2,7 @@
// Copyright (c) 2019, Linaro Limited
#include <linux/clk.h>
+#include <linux/dma-mapping.h>
#include <linux/gpio/consumer.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
@@ -217,13 +218,18 @@ static int wcd934x_slim_probe(struct slim_device *sdev)
struct gpio_desc *reset_gpio;
int ret;
+ ret = dma_coerce_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to set DMA mask\n");
+
ddata = devm_kzalloc(dev, sizeof(*ddata), GFP_KERNEL);
if (!ddata)
return -ENOMEM;
ddata->irq = of_irq_get(np, 0);
if (ddata->irq < 0)
- return dev_err_probe(ddata->dev, ddata->irq,
+ return dev_err_probe(dev, ddata->irq,
"Failed to get IRQ\n");
ddata->extclk = devm_clk_get(dev, "extclk");
diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c
index d4fac2caca86..adc6f850e753 100644
--- a/drivers/misc/fastrpc.c
+++ b/drivers/misc/fastrpc.c
@@ -70,8 +70,6 @@
#define ADSP_MMAP_HEAP_ADDR 4
/* MAP static DMA buffer on DSP User PD */
#define ADSP_MMAP_DMA_BUFFER 6
-/* Add memory to static PD pool protection thru hypervisor */
-#define ADSP_MMAP_REMOTE_HEAP_ADDR 8
/* Add memory to userPD pool, for user heap */
#define ADSP_MMAP_ADD_PAGES 0x1000
/* Add memory to userPD pool, for LLC heap */
@@ -314,10 +312,14 @@ struct fastrpc_channel_ctx {
struct kref refcount;
/* Flag if dsp attributes are cached */
bool valid_attributes;
+ /* Flag if audio PD init mem was allocated */
+ bool audio_init_mem;
+ /* Audio PD reserved remote heap region */
+ phys_addr_t remote_heap_addr;
+ u64 remote_heap_size;
u32 dsp_attributes[FASTRPC_MAX_DSP_ATTRIBUTES];
struct fastrpc_device *secure_fdevice;
struct fastrpc_device *fdevice;
- struct fastrpc_buf *remote_heap;
struct list_head invoke_interrupted_mmaps;
bool secure;
bool unsigned_support;
@@ -927,7 +929,7 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dev_dbg(sess->dev, "Bad size passed len 0x%llx map size 0x%llx\n",
len, map->size);
err = -EINVAL;
- goto map_err;
+ goto get_err;
}
map->va = sg_virt(map->table->sgl);
map->len = len;
@@ -950,7 +952,7 @@ static int fastrpc_map_attach(struct fastrpc_user *fl, int fd,
dev_err(sess->dev,
"Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
&map->dma_addr, map->len, err);
- goto map_err;
+ goto get_err;
}
}
spin_lock(&fl->lock);
@@ -1454,15 +1456,24 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
struct fastrpc_init_create_static init;
struct fastrpc_invoke_args *args;
struct fastrpc_phy_page pages[1];
+ struct fastrpc_channel_ctx *cctx = fl->cctx;
char *name;
int err;
- bool scm_done = false;
struct {
int client_id;
u32 namelen;
u32 pageslen;
} inbuf;
u32 sc;
+ unsigned long flags;
+ bool sent_heap = false;
+
+ if (!cctx->remote_heap_addr || !cctx->remote_heap_size) {
+ err = -ENOMEM;
+ dev_err(fl->sctx->dev,
+ "remote heap memory region is not added\n");
+ return err;
+ }
args = kzalloc_objs(*args, FASTRPC_CREATE_STATIC_PROCESS_NARGS);
if (!args)
@@ -1486,31 +1497,6 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
inbuf.client_id = fl->client_id;
inbuf.namelen = init.namelen;
inbuf.pageslen = 0;
- if (!fl->cctx->remote_heap) {
- err = fastrpc_remote_heap_alloc(fl, fl->sctx->dev, init.memlen,
- &fl->cctx->remote_heap);
- if (err)
- goto err_name;
-
- /* Map if we have any heap VMIDs associated with this ADSP Static Process. */
- if (fl->cctx->vmcount) {
- u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
-
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
- (u64)fl->cctx->remote_heap->size,
- &src_perms,
- fl->cctx->vmperms, fl->cctx->vmcount);
- if (err) {
- dev_err(fl->sctx->dev,
- "Failed to assign memory with dma_addr %pad size 0x%llx err %d\n",
- &fl->cctx->remote_heap->dma_addr,
- fl->cctx->remote_heap->size, err);
- goto err_map;
- }
- scm_done = true;
- inbuf.pageslen = 1;
- }
- }
fl->pd = USER_PD;
@@ -1522,8 +1508,25 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
args[1].length = inbuf.namelen;
args[1].fd = -1;
- pages[0].addr = fl->cctx->remote_heap->dma_addr;
- pages[0].size = fl->cctx->remote_heap->size;
+ /*
+ * Audio PD is a static PD and retains the remote heap
+ * information across daemon restarts. Therefore only
+ * the first attach should provide heap information to
+ * DSP. Subsequent attaches reuse the previously
+ * initialized memory pool.
+ */
+ spin_lock_irqsave(&cctx->lock, flags);
+ if (!cctx->audio_init_mem) {
+ pages[0].addr = cctx->remote_heap_addr;
+ pages[0].size = cctx->remote_heap_size;
+ cctx->audio_init_mem = true;
+ inbuf.pageslen = 1;
+ sent_heap = true;
+ } else {
+ pages[0].addr = 0;
+ pages[0].size = 0;
+ }
+ spin_unlock_irqrestore(&cctx->lock, flags);
args[2].ptr = (u64)(uintptr_t) pages;
args[2].length = sizeof(*pages);
@@ -1541,27 +1544,11 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl,
return 0;
err_invoke:
- if (fl->cctx->vmcount && scm_done) {
- u64 src_perms = 0;
- struct qcom_scm_vmperm dst_perms;
- u32 i;
-
- for (i = 0; i < fl->cctx->vmcount; i++)
- src_perms |= BIT(fl->cctx->vmperms[i].vmid);
-
- dst_perms.vmid = QCOM_SCM_VMID_HLOS;
- dst_perms.perm = QCOM_SCM_PERM_RWX;
- err = qcom_scm_assign_mem(fl->cctx->remote_heap->dma_addr,
- (u64)fl->cctx->remote_heap->size,
- &src_perms, &dst_perms, 1);
- if (err)
- dev_err(fl->sctx->dev, "Failed to assign memory dma_addr %pad size 0x%llx err %d\n",
- &fl->cctx->remote_heap->dma_addr, fl->cctx->remote_heap->size, err);
+ if (sent_heap) {
+ spin_lock_irqsave(&cctx->lock, flags);
+ cctx->audio_init_mem = false;
+ spin_unlock_irqrestore(&cctx->lock, flags);
}
-err_map:
- fastrpc_buf_free(fl->cctx->remote_heap);
- fl->cctx->remote_heap = NULL;
-err_name:
kfree(name);
err:
kfree(args);
@@ -2086,7 +2073,7 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
if (copy_from_user(&req, argp, sizeof(req)))
return -EFAULT;
- if (req.flags != ADSP_MMAP_ADD_PAGES && req.flags != ADSP_MMAP_REMOTE_HEAP_ADDR) {
+ if (req.flags != ADSP_MMAP_ADD_PAGES) {
dev_err(dev, "flag not supported 0x%x\n", req.flags);
return -EINVAL;
@@ -2097,10 +2084,7 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
return -EINVAL;
}
- if (req.flags == ADSP_MMAP_REMOTE_HEAP_ADDR)
- err = fastrpc_remote_heap_alloc(fl, dev, req.size, &buf);
- else
- err = fastrpc_buf_alloc(fl, dev, req.size, &buf);
+ err = fastrpc_buf_alloc(fl, dev, req.size, &buf);
if (err) {
dev_err(dev, "failed to allocate buffer\n");
@@ -2139,20 +2123,6 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp)
/* let the client know the address to use */
req.vaddrout = rsp_msg.vaddr;
- /* Add memory to static PD pool, protection thru hypervisor */
- if (req.flags == ADSP_MMAP_REMOTE_HEAP_ADDR && fl->cctx->vmcount) {
- u64 src_perms = BIT(QCOM_SCM_VMID_HLOS);
-
- err = qcom_scm_assign_mem(buf->dma_addr, (u64)buf->size,
- &src_perms, fl->cctx->vmperms, fl->cctx->vmcount);
- if (err) {
- dev_err(fl->sctx->dev,
- "Failed to assign memory dma_addr %pad size 0x%llx err %d",
- &buf->dma_addr, buf->size, err);
- goto err_assign;
- }
- }
-
spin_lock(&fl->lock);
list_add_tail(&buf->node, &fl->mmaps);
spin_unlock(&fl->lock);
@@ -2533,6 +2503,45 @@ static const struct of_device_id fastrpc_poll_supported_machines[] __maybe_unuse
{},
};
+static int fastrpc_init_reserved_mem(struct fastrpc_channel_ctx *cctx,
+ struct device *rdev, u32 domain_id)
+{
+ struct resource res;
+ u64 src_perms;
+ int err;
+
+ /* Only SDSP and ADSP domains use a reserved remote heap region */
+ if (domain_id != SDSP_DOMAIN_ID && domain_id != ADSP_DOMAIN_ID)
+ return 0;
+
+ err = of_reserved_mem_region_to_resource(rdev->of_node, 0, &res);
+ if (err) {
+ /*
+ * The reserved-memory region is optional at probe time. For
+ * the Audio PD (ADSP) domain its absence is validated later in
+ * fastrpc_init_create_static_process(), so keep probing here.
+ */
+ return 0;
+ }
+
+ /*
+ * Audio PD (ADSP) is a static PD: cache the region so its lifetime
+ * is tied to the rpmsg channel instead of being controlled by
+ * userspace.
+ */
+ if (domain_id == ADSP_DOMAIN_ID) {
+ cctx->remote_heap_addr = res.start;
+ cctx->remote_heap_size = resource_size(&res);
+ }
+
+ if (!cctx->vmcount)
+ return 0;
+
+ src_perms = BIT(QCOM_SCM_VMID_HLOS);
+ return qcom_scm_assign_mem(res.start, resource_size(&res), &src_perms,
+ cctx->vmperms, cctx->vmcount);
+}
+
static int fastrpc_rpmsg_probe(struct rpmsg_device *rpdev)
{
struct device *rdev = &rpdev->dev;
@@ -2580,21 +2589,9 @@ static int fastrpc_rpmsg_probe(struct rpmsg_device *rpdev)
}
}
- if (domain_id == SDSP_DOMAIN_ID) {
- struct resource res;
- u64 src_perms;
-
- err = of_reserved_mem_region_to_resource(rdev->of_node, 0, &res);
- if (!err) {
- src_perms = BIT(QCOM_SCM_VMID_HLOS);
-
- err = qcom_scm_assign_mem(res.start, resource_size(&res), &src_perms,
- data->vmperms, data->vmcount);
- if (err)
- goto err_free_data;
- }
-
- }
+ err = fastrpc_init_reserved_mem(data, rdev, domain_id);
+ if (err)
+ goto err_free_data;
secure_dsp = !(of_property_read_bool(rdev->of_node, "qcom,non-secure-domain"));
data->secure = secure_dsp;
@@ -2694,8 +2691,25 @@ static void fastrpc_rpmsg_remove(struct rpmsg_device *rpdev)
list_for_each_entry_safe(buf, b, &cctx->invoke_interrupted_mmaps, node)
list_del(&buf->node);
- if (cctx->remote_heap)
- fastrpc_buf_free(cctx->remote_heap);
+ if (cctx->remote_heap_size && cctx->vmcount) {
+ u64 src_perms = 0;
+ int err, i;
+ struct qcom_scm_vmperm dst_perms;
+
+ for (i = 0; i < cctx->vmcount; i++)
+ src_perms |= BIT(cctx->vmperms[i].vmid);
+
+ dst_perms.vmid = QCOM_SCM_VMID_HLOS;
+ dst_perms.perm = QCOM_SCM_PERM_RWX;
+
+ err = qcom_scm_assign_mem(cctx->remote_heap_addr,
+ cctx->remote_heap_size, &src_perms,
+ &dst_perms, 1);
+ if (err)
+ dev_err(&rpdev->dev,
+ "Failed to assign memory back to HLOS: addr %pa size %#llx err %d\n",
+ &cctx->remote_heap_addr, cctx->remote_heap_size, err);
+ }
of_platform_depopulate(&rpdev->dev);
diff --git a/drivers/misc/issei/Kconfig b/drivers/misc/issei/Kconfig
index 12a3fd809969..dfcfa2d30b9e 100644
--- a/drivers/misc/issei/Kconfig
+++ b/drivers/misc/issei/Kconfig
@@ -2,6 +2,7 @@
# Copyright (C) 2023-2026 Intel Corporation
config INTEL_SSEI
tristate "Intel Silicon Security Engine Interface"
+ depends on X86
help
The ISSEI (Intel Silicon Security Engine Interface)
subsystem provides a communication channel between the host and the
diff --git a/drivers/misc/mei/vsc-tp.c b/drivers/misc/mei/vsc-tp.c
index 432bc59f11f9..ce877cb95757 100644
--- a/drivers/misc/mei/vsc-tp.c
+++ b/drivers/misc/mei/vsc-tp.c
@@ -293,7 +293,7 @@ int vsc_tp_xfer(struct vsc_tp *tp, u8 cmd, const void *obuf, size_t olen,
pkt->hdr.seq = cpu_to_le32(++tp->seq);
memcpy(pkt->buf, obuf, olen);
- crc = ~crc32(~0, (u8 *)pkt, sizeof(pkt) + olen);
+ crc = ~crc32(~0, (u8 *)pkt, sizeof(pkt->hdr) + olen);
memcpy(pkt->buf + olen, &crc, sizeof(crc));
ret = vsc_tp_wakeup_request(tp);
diff --git a/drivers/misc/tifm_core.c b/drivers/misc/tifm_core.c
index aac8f0933e43..03d1324bc253 100644
--- a/drivers/misc/tifm_core.c
+++ b/drivers/misc/tifm_core.c
@@ -97,7 +97,6 @@ static void tifm_device_remove(struct device *dev)
sock->card_event = tifm_dummy_event;
sock->data_event = tifm_dummy_event;
drv->remove(sock);
- sock->dev.driver = NULL;
}
put_device(dev);
diff --git a/drivers/mtd/chips/cfi_cmdset_0002.c b/drivers/mtd/chips/cfi_cmdset_0002.c
index 517db2f2707f..100cab95a17b 100644
--- a/drivers/mtd/chips/cfi_cmdset_0002.c
+++ b/drivers/mtd/chips/cfi_cmdset_0002.c
@@ -283,6 +283,21 @@ static void fixup_use_write_buffers(struct mtd_info *mtd)
pr_debug("Using buffer write method\n");
mtd->_write = cfi_amdstd_write_buffers;
}
+
+ /*
+ * The word count of the Write to Buffer command is a single bus
+ * word per device, so an x8 device can be told to program at most
+ * 256 bytes however large a buffer it advertises - including when
+ * several of them are interleaved on a wider bus, where CMD()
+ * replicates the count into each device's lane and it is truncated
+ * there.
+ */
+ if (cfi->device_type == CFI_DEVICETYPE_X8 &&
+ cfi->cfiq->MaxBufWriteSize > 8) {
+ cfi->cfiq->MaxBufWriteSize = 8;
+ mtd->writebufsize = cfi_interleave(cfi) <<
+ cfi->cfiq->MaxBufWriteSize;
+ }
}
#endif /* !FORCE_WORD_WRITE */
diff --git a/drivers/mtd/maps/amd76xrom.c b/drivers/mtd/maps/amd76xrom.c
index 457e6ab7f3fb..0c3e0eb08b8d 100644
--- a/drivers/mtd/maps/amd76xrom.c
+++ b/drivers/mtd/maps/amd76xrom.c
@@ -152,6 +152,8 @@ static int amd76xrom_init_one(struct pci_dev *pdev,
printk(KERN_ERR MOD_NAME
" %s(): Unable to register resource %pR - kernel bug?\n",
__func__, &window->rsrc);
+ pci_dev_put(window->pdev);
+ window->pdev = NULL;
return -EBUSY;
}
diff --git a/drivers/mtd/maps/l440gx.c b/drivers/mtd/maps/l440gx.c
index 832b880d1aaf..1df5ea5fafca 100644
--- a/drivers/mtd/maps/l440gx.c
+++ b/drivers/mtd/maps/l440gx.c
@@ -110,8 +110,6 @@ static int __init init_l440gx(void)
/* Allocate the resource region */
if (pci_assign_resource(pm_dev, PIIXE_IOBASE_RESOURCE) != 0) {
- pci_dev_put(dev);
- pci_dev_put(pm_dev);
printk(KERN_WARNING "Could not allocate pm iobase resource\n");
iounmap(l440gx_map.virt);
return -ENXIO;
diff --git a/drivers/mtd/maps/pci.c b/drivers/mtd/maps/pci.c
index 34041cc8f173..f13a1f520d04 100644
--- a/drivers/mtd/maps/pci.c
+++ b/drivers/mtd/maps/pci.c
@@ -250,7 +250,7 @@ static int mtd_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
struct mtd_info *mtd = NULL;
int err;
- err = pci_enable_device(dev);
+ err = pcim_enable_device(dev);
if (err)
goto out;
diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c
index 491a27d0315f..b170c3144ca8 100644
--- a/drivers/mtd/mtdcore.c
+++ b/drivers/mtd/mtdcore.c
@@ -563,7 +563,7 @@ int mtd_pairing_info_to_wunit(struct mtd_info *mtd,
return -EINVAL;
if (master->pairing && master->pairing->get_wunit)
- return mtd->pairing->get_wunit(master, info);
+ return master->pairing->get_wunit(master, info);
return info->pair;
}
diff --git a/drivers/mtd/mtdsuper.c b/drivers/mtd/mtdsuper.c
index c709ff7aa6b5..02eede3fe582 100644
--- a/drivers/mtd/mtdsuper.c
+++ b/drivers/mtd/mtdsuper.c
@@ -31,8 +31,10 @@ static int mtd_get_sb(struct fs_context *fc,
int ret;
sb = sget_dev(fc, MKDEV(MTD_BLOCK_MAJOR, mtd->index));
- if (IS_ERR(sb))
+ if (IS_ERR(sb)) {
+ put_mtd_device(mtd);
return PTR_ERR(sb);
+ }
if (sb->s_root) {
/* new mountpoint for an already mounted superblock */
diff --git a/drivers/mtd/nand/raw/atmel/nand-controller.c b/drivers/mtd/nand/raw/atmel/nand-controller.c
index 8a19408abb63..0bdbcbb50d94 100644
--- a/drivers/mtd/nand/raw/atmel/nand-controller.c
+++ b/drivers/mtd/nand/raw/atmel/nand-controller.c
@@ -2085,7 +2085,7 @@ static int atmel_nand_controller_init(struct atmel_nand_controller *nc,
if (!np) {
dev_err(dev, "Missing or invalid atmel,smc property\n");
ret = -EINVAL;
- goto out_release_dma;
+ goto out_put_mck;
}
nc->smc = syscon_node_to_regmap(np);
@@ -2093,11 +2093,13 @@ static int atmel_nand_controller_init(struct atmel_nand_controller *nc,
if (IS_ERR(nc->smc)) {
ret = PTR_ERR(nc->smc);
dev_err(dev, "Could not get SMC regmap (err = %d)\n", ret);
- goto out_release_dma;
+ goto out_put_mck;
}
return 0;
+out_put_mck:
+ clk_put(nc->mck);
out_release_dma:
if (nc->dmac)
dma_release_channel(nc->dmac);
diff --git a/drivers/mtd/nand/raw/ndfc.c b/drivers/mtd/nand/raw/ndfc.c
index a48274297d3b..d6db04d48c45 100644
--- a/drivers/mtd/nand/raw/ndfc.c
+++ b/drivers/mtd/nand/raw/ndfc.c
@@ -185,10 +185,15 @@ static int ndfc_chip_init(struct ndfc_controller *ndfc,
static int ndfc_probe(struct platform_device *ofdev)
{
struct ndfc_controller *ndfc;
+ void __iomem *ndfcbase;
const __be32 *reg;
u32 ccr;
u32 cs;
- int err, len = 0;
+ int len = 0;
+
+ ndfcbase = devm_platform_ioremap_resource(ofdev, 0);
+ if (IS_ERR(ndfcbase))
+ return PTR_ERR(ndfcbase);
/* Read the reg property to get the chip select */
reg = of_get_property(ofdev->dev.of_node, "reg", &len);
@@ -210,11 +215,7 @@ static int ndfc_probe(struct platform_device *ofdev)
ndfc->ofdev = ofdev;
dev_set_drvdata(&ofdev->dev, ndfc);
- ndfc->ndfcbase = of_iomap(ofdev->dev.of_node, 0);
- if (!ndfc->ndfcbase) {
- dev_err(&ofdev->dev, "failed to get memory\n");
- return -EIO;
- }
+ ndfc->ndfcbase = ndfcbase;
ccr = NDFC_CCR_BS(ndfc->chip_select);
@@ -232,13 +233,7 @@ static int ndfc_probe(struct platform_device *ofdev)
iowrite32be(be32_to_cpup(reg), ndfc->ndfcbase + offset);
}
- err = ndfc_chip_init(ndfc, ofdev->dev.of_node);
- if (err) {
- iounmap(ndfc->ndfcbase);
- return err;
- }
-
- return 0;
+ return ndfc_chip_init(ndfc, ofdev->dev.of_node);
}
static void ndfc_remove(struct platform_device *ofdev)
diff --git a/drivers/mtd/nand/raw/nuvoton-ma35d1-nand-controller.c b/drivers/mtd/nand/raw/nuvoton-ma35d1-nand-controller.c
index 1a285cd8fad6..f659d3a6b9ef 100644
--- a/drivers/mtd/nand/raw/nuvoton-ma35d1-nand-controller.c
+++ b/drivers/mtd/nand/raw/nuvoton-ma35d1-nand-controller.c
@@ -993,7 +993,6 @@ static int ma35_nand_probe(struct platform_device *pdev)
ret = ma35_nand_chips_init(&pdev->dev, nand);
if (ret) {
dev_err(&pdev->dev, "failed to init NAND chips\n");
- clk_disable(nand->clk);
return ret;
}
diff --git a/drivers/mtd/nand/raw/qcom_nandc.c b/drivers/mtd/nand/raw/qcom_nandc.c
index d7642db2e2df..170af9f26d47 100644
--- a/drivers/mtd/nand/raw/qcom_nandc.c
+++ b/drivers/mtd/nand/raw/qcom_nandc.c
@@ -328,14 +328,16 @@ config_nand_cw_read(struct nand_chip *chip, bool use_ecc, int cw)
{
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
-
__le32 *reg = &nandc->regs->read_location0;
+ int reg_base = NAND_READ_LOCATION_0;
- if (nandc->props->qpic_version2 && qcom_nandc_is_last_cw(ecc, cw))
+ if (nandc->props->qpic_version2 && qcom_nandc_is_last_cw(ecc, cw)) {
reg = &nandc->regs->read_location_last0;
+ reg_base = NAND_READ_LOCATION_LAST_CW_0;
+ }
if (nandc->props->supports_bam)
- qcom_write_reg_dma(nandc, reg, NAND_READ_LOCATION_0, 4, NAND_BAM_NEXT_SGL);
+ qcom_write_reg_dma(nandc, reg, reg_base, 4, NAND_BAM_NEXT_SGL);
qcom_write_reg_dma(nandc, &nandc->regs->cmd, NAND_FLASH_CMD, 1, NAND_BAM_NEXT_SGL);
qcom_write_reg_dma(nandc, &nandc->regs->exec, NAND_EXEC_CMD, 1, NAND_BAM_NEXT_SGL);
diff --git a/drivers/mtd/nand/raw/sunxi_nand.c b/drivers/mtd/nand/raw/sunxi_nand.c
index 45ccbce91551..89fe3c0379f7 100644
--- a/drivers/mtd/nand/raw/sunxi_nand.c
+++ b/drivers/mtd/nand/raw/sunxi_nand.c
@@ -2012,7 +2012,7 @@ static int sunxi_nand_ooblayout_free(struct mtd_info *mtd, int section,
return 0;
}
- oobregion->offset = sunxi_get_ecc_offset(sunxi_nand, ecc, section);
+ oobregion->offset = sunxi_get_oob_offset(sunxi_nand, ecc, section);
oobregion->length = user_data_sz;
return 0;
@@ -2077,6 +2077,9 @@ static int sunxi_nand_hw_ecc_ctrl_init(struct nand_chip *nand,
if (nanddev->ecc.user_conf.flags & NAND_ECC_MAXIMIZE_STRENGTH) {
int bytes = mtd->oobsize;
+ if (mtd->writesize < 1024)
+ return -EINVAL;
+
ecc->size = 1024;
nsectors = mtd->writesize / ecc->size;
diff --git a/drivers/mtd/parsers/qcomsmempart.c b/drivers/mtd/parsers/qcomsmempart.c
index d4fdc46a0073..1591aea48b28 100644
--- a/drivers/mtd/parsers/qcomsmempart.c
+++ b/drivers/mtd/parsers/qcomsmempart.c
@@ -143,9 +143,9 @@ static int parse_qcomsmem_part(struct mtd_info *mtd,
*c = tolower(*c);
parts[j].name = name;
- parts[j].offset = le32_to_cpu(pentry->offset) * mtd->erasesize;
+ parts[j].offset = (u64)le32_to_cpu(pentry->offset) * mtd->erasesize;
parts[j].mask_flags = pentry->attr;
- parts[j].size = le32_to_cpu(pentry->length) * mtd->erasesize;
+ parts[j].size = (u64)le32_to_cpu(pentry->length) * mtd->erasesize;
pr_debug("%d: %s offs=0x%08x size=0x%08x attr:0x%08x\n",
i, pentry->name, le32_to_cpu(pentry->offset),
le32_to_cpu(pentry->length), pentry->attr);
diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c
index 8bc117b46e02..250e996284f9 100644
--- a/drivers/mtd/spi-nor/core.c
+++ b/drivers/mtd/spi-nor/core.c
@@ -1346,6 +1346,9 @@ int spi_nor_prep_and_lock(struct spi_nor *nor)
ret = wait_event_killable(nor->rww.wait,
spi_nor_rww_start_exclusive(nor));
+ if (ret)
+ spi_nor_unprep(nor);
+
return ret;
}
@@ -1417,6 +1420,9 @@ static int spi_nor_prep_and_lock_pe(struct spi_nor *nor, loff_t start, size_t le
ret = wait_event_killable(nor->rww.wait,
spi_nor_rww_start_pe(nor, start, len));
+ if (ret)
+ spi_nor_unprep(nor);
+
return ret;
}
@@ -1490,6 +1496,9 @@ static int spi_nor_prep_and_lock_rd(struct spi_nor *nor, loff_t start, size_t le
ret = wait_event_killable(nor->rww.wait,
spi_nor_rww_start_rd(nor, start, len));
+ if (ret)
+ spi_nor_unprep(nor);
+
return ret;
}
diff --git a/drivers/mtd/spi-nor/sfdp.c b/drivers/mtd/spi-nor/sfdp.c
index 4600983cb579..98559536d41b 100644
--- a/drivers/mtd/spi-nor/sfdp.c
+++ b/drivers/mtd/spi-nor/sfdp.c
@@ -1175,6 +1175,7 @@ static int spi_nor_parse_4bait(struct spi_nor *nor,
#define PROFILE1_DWORD5_DUMMY_166MHZ GENMASK(31, 27)
#define PROFILE1_DWORD5_DUMMY_133MHZ GENMASK(21, 17)
#define PROFILE1_DWORD5_DUMMY_100MHZ GENMASK(11, 7)
+#define SFDP_PROFILE1_DWORD_MIN 5
/**
* spi_nor_parse_profile1() - parse the xSPI Profile 1.0 table
@@ -1192,6 +1193,9 @@ static int spi_nor_parse_profile1(struct spi_nor *nor,
int ret;
u8 dummy, opcode;
+ if (profile1_header->length < SFDP_PROFILE1_DWORD_MIN)
+ return -EINVAL;
+
len = profile1_header->length * sizeof(*dwords);
dwords = kmalloc(len, GFP_KERNEL);
if (!dwords)
@@ -1262,6 +1266,7 @@ static int spi_nor_parse_profile1(struct spi_nor *nor,
}
#define SCCR_DWORD22_OCTAL_DTR_EN_VOLATILE BIT(31)
+#define SFDP_SCCR_DWORD_MIN 22
/**
* spi_nor_parse_sccr() - Parse the Status, Control and Configuration Register
@@ -1280,6 +1285,9 @@ static int spi_nor_parse_sccr(struct spi_nor *nor,
size_t len;
int ret;
+ if (sccr_header->length < SFDP_SCCR_DWORD_MIN)
+ return -EINVAL;
+
len = sccr_header->length * sizeof(*dwords);
dwords = kmalloc(len, GFP_KERNEL);
if (!dwords)
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 09aa45d71fab..7019e5d8fdc8 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -6273,7 +6273,7 @@ static int __init bond_check_params(struct bond_params *params)
}
for (arp_ip_count = 0, i = 0;
- (arp_ip_count < BOND_MAX_ARP_TARGETS) && arp_ip_target[i]; i++) {
+ (i < BOND_MAX_ARP_TARGETS) && arp_ip_target[i]; i++) {
__be32 ip;
/* not a complete check, but good enough to catch mistakes */
diff --git a/drivers/net/ethernet/broadcom/bgmac.c b/drivers/net/ethernet/broadcom/bgmac.c
index 3e9c57196a39..1835266fdd09 100644
--- a/drivers/net/ethernet/broadcom/bgmac.c
+++ b/drivers/net/ethernet/broadcom/bgmac.c
@@ -1183,9 +1183,6 @@ static int bgmac_open(struct net_device *net_dev)
if (err)
return err;
- /* Specs say about reclaiming rings here, but we do that in DMA init */
- bgmac_chip_init(bgmac);
-
err = request_irq(bgmac->irq, bgmac_interrupt, IRQF_SHARED,
net_dev->name, net_dev);
if (err < 0) {
@@ -1195,6 +1192,9 @@ static int bgmac_open(struct net_device *net_dev)
}
napi_enable(&bgmac->napi);
+ /* Specs say about reclaiming rings here, but we do that in DMA init */
+ bgmac_chip_init(bgmac);
+
phy_start(net_dev->phydev);
netif_start_queue(net_dev);
@@ -1608,10 +1608,10 @@ int bgmac_enet_resume(struct bgmac *bgmac)
if (rc)
return rc;
- bgmac_chip_init(bgmac);
-
napi_enable(&bgmac->napi);
+ bgmac_chip_init(bgmac);
+
netif_tx_lock(bgmac->net_dev);
netif_device_attach(bgmac->net_dev);
netif_tx_unlock(bgmac->net_dev);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 9e5009be8e98..96845c8646e0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -381,11 +381,31 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
return nxmit;
}
+static void bnxt_xdp_apply_cfg(struct bnxt *bp, int tx_xdp)
+{
+ struct net_device *dev = bp->dev;
+ int tc = bp->num_tc ? : 1;
+
+ if (bp->xdp_prog) {
+ bnxt_set_rx_skb_mode(bp, true);
+ xdp_features_set_redirect_target_locked(dev, true);
+ } else {
+ xdp_features_clear_redirect_target_locked(dev);
+ bnxt_set_rx_skb_mode(bp, false);
+ }
+ bp->tx_nr_rings_xdp = tx_xdp;
+ bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
+ bnxt_set_cp_rings(bp, true);
+ bnxt_set_tpa_flags(bp);
+ bnxt_set_ring_params(bp);
+}
+
static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
{
struct net_device *dev = bp->dev;
int tx_xdp = 0, rc, tc;
struct bpf_prog *old;
+ int old_tx_xdp;
netdev_assert_locked(dev);
@@ -418,25 +438,22 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
if (netif_running(dev))
bnxt_close_nic(bp, true, false);
+ old_tx_xdp = bp->tx_nr_rings_xdp;
old = xchg(&bp->xdp_prog, prog);
- if (old)
- bpf_prog_put(old);
-
- if (prog) {
- bnxt_set_rx_skb_mode(bp, true);
- xdp_features_set_redirect_target_locked(dev, true);
- } else {
- xdp_features_clear_redirect_target_locked(dev);
- bnxt_set_rx_skb_mode(bp, false);
+ bnxt_xdp_apply_cfg(bp, tx_xdp);
+
+ if (netif_running(dev)) {
+ rc = bnxt_open_nic(bp, true, false);
+ /* a detach cannot be undone, only an attach */
+ if (rc && prog) {
+ WRITE_ONCE(bp->xdp_prog, old);
+ bnxt_xdp_apply_cfg(bp, old_tx_xdp);
+ return rc;
+ }
}
- bp->tx_nr_rings_xdp = tx_xdp;
- bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc + tx_xdp;
- bnxt_set_cp_rings(bp, true);
- bnxt_set_tpa_flags(bp);
- bnxt_set_ring_params(bp);
- if (netif_running(dev))
- return bnxt_open_nic(bp, true, false);
+ if (old)
+ bpf_prog_put(old);
return 0;
}
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 8b6806a79edf..fd1c95f1111c 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -18188,41 +18188,17 @@ static int tg3_resume(struct device *device)
static SIMPLE_DEV_PM_OPS(tg3_pm_ops, tg3_suspend, tg3_resume);
-/* Systems where ACPI _PTS (Prepare To Sleep) S5 will result in a fatal
- * PCIe AER event on the tg3 device if the tg3 device is not, or cannot
- * be, powered down.
+/* Systems where powering down a tg3 device on restarts triggers
+ * NMIs and fatal hangs during reboot if the network adapters were
+ * initialized under SNP.
*/
-static const struct dmi_system_id tg3_restart_aer_quirk_table[] = {
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R440"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R540"),
- },
- },
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R640"),
- },
- },
+static const struct dmi_system_id tg3_restart_snp_quirk_table[] = {
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R650"),
},
},
- {
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_PRODUCT_NAME, "PowerEdge R740"),
- },
- },
{
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
@@ -18246,14 +18222,18 @@ static void tg3_shutdown(struct pci_dev *pdev)
if (netif_running(dev))
dev_close(dev);
- if (system_state == SYSTEM_POWER_OFF)
+ if (!dmi_first_match(tg3_restart_snp_quirk_table) ||
+ system_state == SYSTEM_POWER_OFF) {
tg3_power_down(tp);
- else if (system_state == SYSTEM_RESTART &&
- dmi_first_match(tg3_restart_aer_quirk_table) &&
+ } else if (system_state == SYSTEM_RESTART &&
pdev->current_state != PCI_D3cold &&
pdev->current_state != PCI_UNKNOWN) {
/* Disable PCIe AER on the tg3 to avoid a fatal
* error during this system restart.
+ * Needed since on restarts for systems in
+ * tg3_restart_snp_quirk_table skipping on powering down the
+ * tg3 device causes a fatal AER error during ACPI _PTS
+ * (Prepare To Sleep).
*/
pcie_capability_clear_word(pdev, PCI_EXP_DEVCTL,
PCI_EXP_DEVCTL_CERE |
diff --git a/drivers/net/ethernet/google/gve/gve_flow_rule.c b/drivers/net/ethernet/google/gve/gve_flow_rule.c
index 2c80cda28ef3..db8b9a39bfc8 100644
--- a/drivers/net/ethernet/google/gve/gve_flow_rule.c
+++ b/drivers/net/ethernet/google/gve/gve_flow_rule.c
@@ -206,6 +206,7 @@ int gve_get_flow_rule_entry(struct gve_priv *priv, struct ethtool_rxnfc *cmd)
return -EOPNOTSUPP;
if (!priv->flow_rules_cache.rules_cache_synced ||
+ !*cache_num ||
fsp->location < be32_to_cpu(rules_cache[0].location) ||
fsp->location > be32_to_cpu(rules_cache[*cache_num - 1].location)) {
err = gve_adminq_query_flow_rules(priv, GVE_FLOW_RULE_QUERY_RULES, fsp->location);
diff --git a/drivers/net/ethernet/hisilicon/hisi_femac.c b/drivers/net/ethernet/hisilicon/hisi_femac.c
index 016605e86f0b..a44ea0a2b1ee 100644
--- a/drivers/net/ethernet/hisilicon/hisi_femac.c
+++ b/drivers/net/ethernet/hisilicon/hisi_femac.c
@@ -491,6 +491,7 @@ static int hisi_femac_net_close(struct net_device *dev)
netif_stop_queue(dev);
napi_disable(&priv->napi);
+ hisi_femac_port_reset(priv);
hisi_femac_free_skb_rings(priv);
return 0;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index f91856498eb2..b9496f0bee13 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -9444,23 +9444,6 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
}
protocol = vlan_get_protocol(skb);
- if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
- adapter->ptp_clock) {
- if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
- !test_and_set_bit_lock(__IXGBE_PTP_TX_IN_PROGRESS,
- &adapter->state)) {
- skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
- tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
-
- /* schedule check for Tx timestamp */
- adapter->ptp_tx_skb = skb_get(skb);
- adapter->ptp_tx_start = jiffies;
- schedule_work(&adapter->ptp_tx_work);
- } else {
- adapter->tx_hwtstamp_skipped++;
- }
- }
-
#ifdef CONFIG_PCI_IOV
/*
* Use the l2switch_enable flag - would be false if the DMA
@@ -9525,6 +9508,29 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
#ifdef IXGBE_FCOE
xmit_fcoe:
#endif /* IXGBE_FCOE */
+ /* Reserve the Tx timestamp only after all skb preparation:
+ * skb_get() makes the skb shared, and pskb_expand_head()
+ * (reached from skb_cow_head() or skb_checksum_help())
+ * refuses shared skbs.
+ */
+ if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
+ adapter->ptp_clock) {
+ if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
+ !test_and_set_bit_lock(__IXGBE_PTP_TX_IN_PROGRESS,
+ &adapter->state)) {
+ skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+ tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
+ first->tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
+
+ /* schedule check for Tx timestamp */
+ adapter->ptp_tx_skb = skb_get(skb);
+ adapter->ptp_tx_start = jiffies;
+ schedule_work(&adapter->ptp_tx_work);
+ } else {
+ adapter->tx_hwtstamp_skipped++;
+ }
+ }
+
if (ixgbe_tx_map(tx_ring, first, hdr_len))
goto cleanup_tx_timestamp;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c
index 308d8a94865f..42e9689d7f35 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.c
@@ -2001,7 +2001,7 @@ mlxsw_core_health_fw_fatal_test(struct devlink_health_reporter *reporter,
struct netlink_ext_ack *extack)
{
struct mlxsw_core *mlxsw_core = devlink_health_reporter_priv(reporter);
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
@@ -2022,7 +2022,7 @@ mlxsw_core_health_fw_fatal_ops = {
static int mlxsw_core_health_fw_fatal_config(struct mlxsw_core *mlxsw_core,
bool enable)
{
- char mfgd_pl[MLXSW_REG_MFGD_LEN];
+ char mfgd_pl[MLXSW_REG_MFGD_LEN] = {0};
int err;
/* Read the register first to make sure no other bits are changed. */
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
index 30f172ce822c..a5e7aee730d6 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_hwmon.c
@@ -120,7 +120,7 @@ static ssize_t mlxsw_hwmon_temp_rst_store(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char mtmp_pl[MLXSW_REG_MTMP_LEN];
+ char mtmp_pl[MLXSW_REG_MTMP_LEN] = {0};
unsigned long val;
int index;
int err;
@@ -177,7 +177,7 @@ static ssize_t mlxsw_hwmon_fan_fault_show(struct device *dev,
container_of(attr, struct mlxsw_hwmon_attr, dev_attr);
struct mlxsw_hwmon_dev *mlxsw_hwmon_dev = mlxsw_hwmon_attr->mlxsw_hwmon_dev;
struct mlxsw_hwmon *mlxsw_hwmon = mlxsw_hwmon_dev->hwmon;
- char fore_pl[MLXSW_REG_FORE_LEN];
+ char fore_pl[MLXSW_REG_FORE_LEN] = {0};
bool fault;
int err;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
index ae63d549b542..d54fe5e61296 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c
@@ -1720,7 +1720,7 @@ static int mlxsw_sp2_span_init(struct mlxsw_sp *mlxsw_sp)
static int mlxsw_sp2_span_policer_id_base_set(struct mlxsw_sp *mlxsw_sp,
u16 policer_id_base)
{
- char mogcr_pl[MLXSW_REG_MOGCR_LEN];
+ char mogcr_pl[MLXSW_REG_MOGCR_LEN] = {0};
int err;
err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(mogcr), mogcr_pl);
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index afad90b9222a..a48c9e662fe9 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -1664,7 +1664,8 @@ static int macvlan_changelink(struct net_device *dev,
if (data && data[IFLA_MACVLAN_FLAGS]) {
__u16 flags = nla_get_u16(data[IFLA_MACVLAN_FLAGS]);
bool promisc = (flags ^ vlan->flags) & MACVLAN_FLAG_NOPROMISC;
- if (macvlan_passthru(vlan->port) && promisc) {
+ if (macvlan_passthru(vlan->port) && promisc &&
+ netif_running(dev)) {
int err;
if (flags & MACVLAN_FLAG_NOPROMISC)
diff --git a/drivers/net/netdevsim/bpf.c b/drivers/net/netdevsim/bpf.c
index 16aa88278398..f45c1c8f0584 100644
--- a/drivers/net/netdevsim/bpf.c
+++ b/drivers/net/netdevsim/bpf.c
@@ -144,6 +144,17 @@ int nsim_bpf_setup_tc_block_cb(enum tc_setup_type type,
return -EOPNOTSUPP;
}
+ /* A dev-bound program can be replayed here when a netdev joins
+ * a shared block, after the core attach-time check has run;
+ * enforce the device match as nfp does.
+ */
+ if (prog && bpf_prog_is_dev_bound(prog->aux) &&
+ !bpf_offload_dev_match(prog, ns->netdev)) {
+ NSIM_EA(cls_bpf->common.extack,
+ "program bound to a different device");
+ return -EINVAL;
+ }
+
if (cls_bpf->command != TC_CLSBPF_OFFLOAD)
return -EOPNOTSUPP;
diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c
index 1a610a18893b..3998ce4332f4 100644
--- a/drivers/net/ppp/ppp_generic.c
+++ b/drivers/net/ppp/ppp_generic.c
@@ -3157,6 +3157,7 @@ ppp_ccp_peek(struct ppp *ppp, struct sk_buff *skb, int inbound)
len = CCP_LENGTH(dp);
if (!pskb_may_pull(skb, len + 2))
return; /* too short */
+ dp = skb->data + 2;
dp += CCP_HDRLEN;
len -= CCP_HDRLEN;
if (len < CCP_OPT_MINLEN || len < CCP_OPT_LENGTH(dp))
diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c
index bf7414b46a26..ee0e318d6a2a 100644
--- a/drivers/net/ppp/pppoe.c
+++ b/drivers/net/ppp/pppoe.c
@@ -194,18 +194,16 @@ static int __set_item(struct pppoe_net *pn, struct pppox_sock *po)
return 0;
}
-static void __delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void __delete_item(struct pppoe_net *pn, const struct pppox_sock *po)
{
- int hash = hash_item(sid, addr);
+ int hash = hash_item(po->pppoe_pa.sid, po->pppoe_pa.remote);
struct pppox_sock *ret, __rcu **src;
ret = rcu_dereference_protected(pn->hash_table[hash], lockdep_is_held(&pn->hash_lock));
src = &pn->hash_table[hash];
while (ret) {
- if (cmp_addr(&ret->pppoe_pa, sid, addr) &&
- ret->pppoe_ifindex == ifindex) {
+ if (ret == po) {
struct pppox_sock *next;
next = rcu_dereference_protected(ret->next,
@@ -236,11 +234,10 @@ static inline struct pppox_sock *get_item(struct pppoe_net *pn, __be16 sid,
return po;
}
-static inline void delete_item(struct pppoe_net *pn, __be16 sid,
- char *addr, int ifindex)
+static void delete_item(struct pppoe_net *pn, const struct pppox_sock *po)
{
spin_lock(&pn->hash_lock);
- __delete_item(pn, sid, addr, ifindex);
+ __delete_item(pn, po);
spin_unlock(&pn->hash_lock);
}
@@ -570,8 +567,7 @@ static int pppoe_release(struct socket *sock)
* protect "po" from concurrent updates
* on pppoe_flush_dev
*/
- delete_item(pn, po->pppoe_pa.sid, po->pppoe_pa.remote,
- po->pppoe_ifindex);
+ delete_item(pn, po);
sock_orphan(sk);
sock->sk = NULL;
@@ -621,8 +617,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr_unsized *uservaddr
if (stage_session(po->pppoe_pa.sid)) {
pppox_unbind_sock(sk);
pn = pppoe_pernet(sock_net(sk));
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
if (po->pppoe_dev) {
dev_put(po->pppoe_dev);
po->pppoe_dev = NULL;
@@ -672,8 +667,7 @@ static int pppoe_connect(struct socket *sock, struct sockaddr_unsized *uservaddr
error = ppp_register_net_channel(dev_net(dev), &po->chan);
if (error) {
- delete_item(pn, po->pppoe_pa.sid,
- po->pppoe_pa.remote, po->pppoe_ifindex);
+ delete_item(pn, po);
goto err_put;
}
diff --git a/drivers/net/usb/pegasus.c b/drivers/net/usb/pegasus.c
index 7c7d6b69998f..68e4b6658dd2 100644
--- a/drivers/net/usb/pegasus.c
+++ b/drivers/net/usb/pegasus.c
@@ -520,11 +520,17 @@ static void read_bulk_callback(struct urb *urb)
}
/*
- * If the packet is unreasonably long, quietly drop it rather than
- * kernel panicing by calling skb_put.
+ * Drop frames shorter than an Ethernet header, because
+ * eth_type_trans() reads the header whatever the length, and frames
+ * longer than what was actually received. Passing either up would
+ * expose stale buffer contents, and skb_put() could panic on the
+ * latter. count is at most PEGASUS_MTU.
*/
- if (pkt_len > PEGASUS_MTU - rx_offset)
+ if (pkt_len < ETH_HLEN || pkt_len + rx_offset > count) {
+ net->stats.rx_errors++;
+ net->stats.rx_length_errors++;
goto goon;
+ }
/*
* at this point we are sure pegasus->rx_skb != NULL
diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index f61686433031..398d35f6fb7b 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -6979,11 +6979,8 @@ static void set_carrier(struct r8152 *tp)
{
struct net_device *netdev = tp->netdev;
struct napi_struct *napi = &tp->napi;
- u16 speed;
-
- speed = rtl8152_get_speed(tp);
- if (speed & LINK_STATUS) {
+ if (r8152_mdio_read(tp, MII_BMSR) & BMSR_LSTATUS) {
if (!netif_carrier_ok(netdev)) {
tp->rtl_ops.enable(tp);
netif_stop_queue(netdev);
@@ -7009,6 +7006,11 @@ static void set_carrier(struct r8152 *tp)
tasklet_enable(&tp->tx_tl);
netif_info(tp, link, netdev, "carrier off\n");
}
+
+ if (rtl8152_get_speed(tp) & LINK_STATUS) {
+ set_bit(RTL8152_LINK_CHG, &tp->flags);
+ schedule_delayed_work(&tp->schedule, 0);
+ }
}
}
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 643b97dc5245..7db389ae866d 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1622,10 +1622,9 @@ static int veth_xdp_set(struct net_device *dev, struct bpf_prog *prog,
}
}
- if (!old_prog) {
+ if (!old_prog)
peer->hw_features &= ~NETIF_F_GSO_SOFTWARE;
- peer->max_mtu = max_mtu;
- }
+ peer->max_mtu = max_mtu;
xdp_features_set_redirect_target(peer, true);
}
diff --git a/drivers/net/wireless/ath/ath11k/dp.c b/drivers/net/wireless/ath/ath11k/dp.c
index f389b97acbdd..be65d73b904b 100644
--- a/drivers/net/wireless/ath/ath11k/dp.c
+++ b/drivers/net/wireless/ath/ath11k/dp.c
@@ -313,6 +313,7 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring,
case HAL_RXDMA_DIR_BUF:
break;
default:
+ ath11k_dp_srng_cleanup(ab, ring);
ath11k_warn(ab, "Not a valid ring type in dp :%d\n", type);
return -EINVAL;
}
@@ -324,6 +325,7 @@ int ath11k_dp_srng_setup(struct ath11k_base *ab, struct dp_srng *ring,
ret = ath11k_hal_srng_setup(ab, type, ring_num, mac_id, ¶ms);
if (ret < 0) {
+ ath11k_dp_srng_cleanup(ab, ring);
ath11k_warn(ab, "failed to setup srng: %d ring_id %d\n",
ret, ring_num);
return ret;
diff --git a/drivers/net/wireless/ath/ath12k/wmi.c b/drivers/net/wireless/ath/ath12k/wmi.c
index d5160af60e00..3e812c79a649 100644
--- a/drivers/net/wireless/ath/ath12k/wmi.c
+++ b/drivers/net/wireless/ath/ath12k/wmi.c
@@ -5050,6 +5050,7 @@ static int ath12k_wmi_svc_rdy_ext_parse(struct ath12k_base *ab,
u16 tag, u16 len,
const void *ptr, void *data)
{
+ struct ath12k_svc_ext_info *svc_ext_info = &ab->wmi_ab.svc_ext_info;
struct ath12k_wmi_pdev *wmi_handle = &ab->wmi_ab.wmi[0];
struct ath12k_wmi_svc_rdy_ext_parse *svc_rdy_ext = data;
int ret;
@@ -5094,6 +5095,22 @@ static int ath12k_wmi_svc_rdy_ext_parse(struct ath12k_base *ab,
return ret;
}
+ if (svc_rdy_ext->tot_phy_id >
+ ARRAY_SIZE(svc_ext_info->mac_phy_info)) {
+ ath12k_warn(ab, "too many PHY entries %u (max %zu)\n",
+ svc_rdy_ext->tot_phy_id,
+ ARRAY_SIZE(svc_ext_info->mac_phy_info));
+ return -EINVAL;
+ }
+
+ if (svc_rdy_ext->n_mac_phy_caps < svc_rdy_ext->tot_phy_id) {
+ ath12k_warn(ab,
+ "not enough MAC/PHY caps %u, expected %u\n",
+ svc_rdy_ext->n_mac_phy_caps,
+ svc_rdy_ext->tot_phy_id);
+ return -EINVAL;
+ }
+
ath12k_wmi_save_all_mac_phy_info(ab, svc_rdy_ext);
svc_rdy_ext->mac_phy_done = true;
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
index 86cfa3e87a04..9ddd5d519893 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmsmac/phy/phy_n.c
@@ -221,7 +221,7 @@ static const struct nphy_ipa_txrxgain nphy_ipa_rxcal_gaintbl_5GHz[] = {
};
static const struct nphy_ipa_txrxgain nphy_ipa_rxcal_gaintbl_2GHz[] = {
- {0, 0, 0, 0, 0, 128},
+ {0, 0, 0, 0, 0, -128},
{0, 0, 0, 0, 0, 70},
{0, 0, 0, 0, 0, 20},
{0, 0, 0, 3, 0, 20},
diff --git a/drivers/net/wireless/mediatek/mt76/eeprom.c b/drivers/net/wireless/mediatek/mt76/eeprom.c
index afdb73661866..93552f6f7f34 100644
--- a/drivers/net/wireless/mediatek/mt76/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/eeprom.c
@@ -231,12 +231,14 @@ mt76_find_power_limits_node(struct mt76_dev *dev)
struct property *regd = of_find_property(cur, "regdomain", NULL);
if (!country && !regd) {
- fallback = cur;
+ of_node_put(fallback);
+ fallback = of_node_get(cur);
continue;
}
if (mt76_string_prop_find(country, dev->alpha2) ||
mt76_string_prop_find(regd, region_name)) {
+ of_node_put(fallback);
of_node_put(np);
return cur;
}
@@ -429,7 +431,7 @@ s8 mt76_get_rate_power_limits(struct mt76_phy *phy,
s8 target_power)
{
struct mt76_dev *dev = phy->dev;
- struct device_node *np;
+ struct device_node *np, *band_np;
const s8 *val;
char name[16];
char band;
@@ -459,15 +461,18 @@ s8 mt76_get_rate_power_limits(struct mt76_phy *phy,
band = '6';
break;
default:
+ of_node_put(np);
return target_power;
}
snprintf(name, sizeof(name), "txpower-%cg", band);
- np = of_get_child_by_name(np, name);
- if (!np)
+ band_np = of_get_child_by_name(np, name);
+ of_node_put(np);
+ if (!band_np)
return target_power;
- np = mt76_find_channel_node(np, chan);
+ np = mt76_find_channel_node(band_np, chan);
+ of_node_put(band_np);
if (!np)
return target_power;
@@ -514,6 +519,8 @@ s8 mt76_get_rate_power_limits(struct mt76_phy *phy,
ARRAY_SIZE(dest->path.ru_bf), val, len, target_power,
txs_delta, &max_power, n_chains, MT76_SKU_BACKOFF);
+ of_node_put(np);
+
return max_power;
}
EXPORT_SYMBOL_GPL(mt76_get_rate_power_limits);
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index abbe65cbcd89..3467eb6cea0b 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -1045,6 +1045,20 @@ void mt76_update_survey(struct mt76_phy *phy)
}
EXPORT_SYMBOL_GPL(mt76_update_survey);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef)
+{
+ if (!chandef->chan)
+ return;
+
+ mt76_update_survey(phy);
+
+ phy->chandef = *chandef;
+ phy->main_chandef = *chandef;
+ phy->chan_state = mt76_channel_state(phy, chandef->chan);
+}
+EXPORT_SYMBOL_GPL(mt76_phy_chandef_set);
+
int __mt76_set_channel(struct mt76_phy *phy, struct cfg80211_chan_def *chandef,
bool offchannel)
{
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 2ade848fe3bb..e9bbec1269fa 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -1555,6 +1555,8 @@ void mt76_sta_ps_transition(struct mt76_dev *dev, struct mt76_wcid *wcid,
bool mt76_has_tx_pending(struct mt76_phy *phy);
int mt76_update_channel(struct mt76_phy *phy);
void mt76_update_survey(struct mt76_phy *phy);
+void mt76_phy_chandef_set(struct mt76_phy *phy,
+ struct cfg80211_chan_def *chandef);
void mt76_update_survey_active_time(struct mt76_phy *phy, ktime_t time);
int mt76_get_survey(struct ieee80211_hw *hw, int idx,
struct survey_info *survey);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/init.c b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
index e437e088b2e9..665b0f1e939f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/init.c
@@ -541,8 +541,10 @@ int mt7615_register_ext_phy(struct mt7615_dev *dev)
mt7615_cap_dbdc_enable(dev);
mphy = mt76_alloc_phy(&dev->mt76, sizeof(*phy), &mt7615_ops, MT_BAND1);
- if (!mphy)
- return -ENOMEM;
+ if (!mphy) {
+ ret = -ENOMEM;
+ goto error;
+ }
phy = mphy->priv;
phy->dev = dev;
@@ -572,7 +574,7 @@ int mt7615_register_ext_phy(struct mt7615_dev *dev)
ret = mt76_eeprom_override(mphy);
if (ret)
- return ret;
+ goto error_free_hw;
/* second phy can only handle 5 GHz */
mphy->cap.has_5ghz = true;
@@ -590,7 +592,14 @@ int mt7615_register_ext_phy(struct mt7615_dev *dev)
ret = mt76_register_phy(mphy, true, mt76_rates,
ARRAY_SIZE(mt76_rates));
if (ret)
- ieee80211_free_hw(mphy->hw);
+ goto error_free_hw;
+
+ return 0;
+
+error_free_hw:
+ ieee80211_free_hw(mphy->hw);
+error:
+ mt7615_cap_dbdc_disable(dev);
return ret;
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/main.c b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
index 67f56e428d9a..175e847f49ef 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/main.c
@@ -254,8 +254,10 @@ static void mt7615_remove_interface(struct ieee80211_hw *hw,
mt7615_mutex_acquire(dev);
- mt7615_mcu_add_bss_info(phy, vif, NULL, false);
- mt7615_mcu_sta_add(phy, vif, NULL, false);
+ if (vif->type != NL80211_IFTYPE_MONITOR) {
+ mt7615_mcu_add_bss_info(phy, vif, NULL, false);
+ mt7615_mcu_sta_add(phy, vif, NULL, false);
+ }
mt76_testmode_reset(phy->mt76, true);
if (vif == phy->monitor_vif)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
index a9178e077fd6..7d8580720cd8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c
@@ -116,6 +116,7 @@ void mt7615_unregister_device(struct mt7615_dev *dev)
mcu_running = mt7615_wait_for_mcu_init(dev);
mt7615_unregister_ext_phy(dev);
+ cancel_work_sync(&dev->reset_work);
mt76_unregister_device(&dev->mt76);
if (mcu_running)
mt7615_mcu_exit(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
index 21f8b1e64101..231685bb62d2 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
@@ -302,10 +302,14 @@ mt76x02_mac_process_tx_rate(struct ieee80211_tx_rate *txrate, u16 rate,
txrate->flags |= IEEE80211_TX_RC_GREEN_FIELD;
fallthrough;
case MT_PHY_TYPE_HT:
+ if (idx > 31)
+ return -EINVAL;
txrate->flags |= IEEE80211_TX_RC_MCS;
txrate->idx = idx;
break;
case MT_PHY_TYPE_VHT:
+ if ((idx & 0xf) > 9)
+ return -EINVAL;
txrate->flags |= IEEE80211_TX_RC_VHT_MCS;
txrate->idx = idx;
break;
@@ -493,18 +497,21 @@ mt76x02_mac_fill_tx_status(struct mt76x02_dev *dev, struct mt76x02_sta *msta,
first_rate = st->rate & ~MT_PKTID_RATE;
first_rate |= st->pktid & MT_PKTID_RATE;
- mt76x02_mac_process_tx_rate(&rate[0], first_rate,
- dev->mphy.chandef.chan->band);
+ if (mt76x02_mac_process_tx_rate(&rate[0], first_rate,
+ dev->mphy.chandef.chan->band))
+ return;
} else if (rate[0].idx < 0) {
if (!msta)
return;
- mt76x02_mac_process_tx_rate(&rate[0], msta->wcid.tx_info,
- dev->mphy.chandef.chan->band);
+ if (mt76x02_mac_process_tx_rate(&rate[0], msta->wcid.tx_info,
+ dev->mphy.chandef.chan->band))
+ return;
}
- mt76x02_mac_process_tx_rate(&last_rate, st->rate,
- dev->mphy.chandef.chan->band);
+ if (mt76x02_mac_process_tx_rate(&last_rate, st->rate,
+ dev->mphy.chandef.chan->band))
+ return;
for (i = 0; i < ARRAY_SIZE(info->status.rates); i++) {
retry--;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
index 968c73e06a5f..6f931d483bd6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_usb_mcu.c
@@ -90,7 +90,7 @@ __mt76x02u_mcu_send_msg(struct mt76_dev *dev, struct sk_buff *skb,
MT_MCU_MSG_TYPE_CMD;
ret = mt76x02u_skb_dma_info(skb, CPU_TX_PORT, info);
if (ret)
- return ret;
+ goto out;
ret = mt76u_bulk_msg(dev, skb->data, skb->len, NULL, 500,
MT_EP_OUT_INBAND_CMD);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
index e69978184f68..ad1352ddf1fc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/mac.c
@@ -831,11 +831,14 @@ void mt7921_usb_sdio_tx_complete_skb(struct mt76_dev *mdev,
u16 idx;
idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX);
+
+ rcu_read_lock();
wcid = __mt76_wcid_ptr(mdev, idx);
sta = wcid_to_sta(wcid);
if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE)))
mt76_connac2_tx_check_aggr(sta, txwi);
+ rcu_read_unlock();
skb_pull(e->skb, headroom);
mt76_tx_complete_skb(mdev, e->wcid, e->skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7921/main.c b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
index 68a059504e83..3577c4cebb2e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7921/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7921/main.c
@@ -1423,6 +1423,7 @@ mt7921_change_chanctx(struct ieee80211_hw *hw,
} else {
mt76_connac_mcu_uni_set_chctx(mvif->phy->mt76, &mvif->bss_conf.mt76, ctx);
}
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
index 3b82c9de26c5..2e34e950e111 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c
@@ -1688,11 +1688,14 @@ void mt7925_usb_sdio_tx_complete_skb(struct mt76_dev *mdev,
u16 idx;
idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX);
+
+ rcu_read_lock();
wcid = __mt76_wcid_ptr(mdev, idx);
sta = wcid_to_sta(wcid);
if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE)))
mt7925_tx_check_aggr(sta, e->skb, wcid);
+ rcu_read_unlock();
skb_pull(e->skb, headroom);
mt76_tx_complete_skb(mdev, e->wcid, e->skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 549f9c06b363..e3e9e0be558e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -2063,6 +2063,7 @@ mt7925_change_chanctx(struct ieee80211_hw *hw,
link_conf, ctx);
}
}
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mt792x_mutex_release(phy->dev);
}
@@ -2411,6 +2412,7 @@ static int mt7925_assign_vif_chanctx(struct ieee80211_hw *hw,
mconf->mt76.ctx = ctx;
mctx->bss_conf = mconf;
+ mt76_phy_chandef_set(mvif->phy->mt76, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/usb.c b/drivers/net/wireless/mediatek/mt76/mt7925/usb.c
index 1757023ea70d..db6fd014447b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/usb.c
@@ -147,6 +147,15 @@ static int mt7925u_mac_reset(struct mt792x_dev *dev)
return err;
}
+static void mt7925u_stop(struct ieee80211_hw *hw, bool suspend)
+{
+ struct mt792x_dev *dev = mt792x_hw_dev(hw);
+
+ cancel_delayed_work_sync(&dev->mlo_pm_work);
+
+ mt792xu_stop(hw, suspend);
+}
+
static int mt7925u_probe(struct usb_interface *usb_intf,
const struct usb_device_id *id)
{
@@ -193,7 +202,7 @@ static int mt7925u_probe(struct usb_interface *usb_intf,
if (!ops)
return -ENOMEM;
- ops->stop = mt792xu_stop;
+ ops->stop = mt7925u_stop;
mdev = mt76_alloc_device(&usb_intf->dev, sizeof(*dev), ops, &drv_ops);
if (!mdev)
diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_core.c b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
index 0ad33f74c228..3ce6c6ceac81 100644
--- a/drivers/net/wireless/mediatek/mt76/mt792x_core.c
+++ b/drivers/net/wireless/mediatek/mt76/mt792x_core.c
@@ -472,6 +472,7 @@ int mt792x_assign_vif_chanctx(struct ieee80211_hw *hw,
mutex_lock(&dev->mt76.mutex);
mvif->bss_conf.mt76.ctx = ctx;
mctx->bss_conf = &mvif->bss_conf;
+ mt76_phy_chandef_set(&dev->mphy, &ctx->def);
mutex_unlock(&dev->mt76.mutex);
return 0;
@@ -857,7 +858,7 @@ int mt792x_init_wiphy(struct ieee80211_hw *hw)
}
EXPORT_SYMBOL_GPL(mt792x_init_wiphy);
-static u8
+static int
mt792x_get_offload_capability(struct device *dev, const char *fw_wm)
{
const struct mt76_connac2_fw_trailer *hdr;
@@ -926,13 +927,20 @@ mt792x_get_mac80211_ops(struct device *dev,
void *drv_data, u8 *fw_features)
{
struct ieee80211_ops *ops;
+ int ret;
ops = devm_kmemdup(dev, mac80211_ops, sizeof(struct ieee80211_ops),
GFP_KERNEL);
if (!ops)
return NULL;
- *fw_features = mt792x_get_offload_capability(dev, drv_data);
+ ret = mt792x_get_offload_capability(dev, drv_data);
+ if (ret < 0) {
+ dev_warn(dev, "failed to read firmware offload caps: %d\n", ret);
+ *fw_features = 0;
+ } else {
+ *fw_features = ret;
+ }
if (mt792x_needs_cnm_runtime(drv_data))
*fw_features |= MT792x_FW_CAP_CNM;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/pci.c b/drivers/net/wireless/mediatek/mt76/mt7996/pci.c
index b7d9193e042f..a7c23b33dde0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/pci.c
@@ -63,12 +63,18 @@ static void mt7996_put_hif2(struct mt7996_hif *hif)
static struct mt7996_hif *mt7996_pci_init_hif2(struct pci_dev *pdev)
{
+ struct pci_dev *tmp_pdev;
+
hif_idx++;
- if (!pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7996_DEVICE_ID_2, NULL) &&
- !pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7992_DEVICE_ID_2, NULL) &&
- !pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7990_DEVICE_ID_2, NULL))
+ tmp_pdev = pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7996_DEVICE_ID_2, NULL);
+ if (!tmp_pdev)
+ tmp_pdev = pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7992_DEVICE_ID_2, NULL);
+ if (!tmp_pdev)
+ tmp_pdev = pci_get_device(PCI_VENDOR_ID_MEDIATEK, MT7990_DEVICE_ID_2, NULL);
+ if (!tmp_pdev)
return NULL;
+ pci_dev_put(tmp_pdev);
writel(hif_idx | MT_PCIE_RECOG_ID_SEM,
pcim_iomap_table(pdev)[0] + MT_PCIE_RECOG_ID);
diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c
index 3c623df2e741..156d283243d3 100644
--- a/drivers/net/wireless/mediatek/mt76/tx.c
+++ b/drivers/net/wireless/mediatek/mt76/tx.c
@@ -885,7 +885,7 @@ int mt76_skb_adjust_pad(struct sk_buff *skb, int pad)
}
}
- if (skb_pad(last, pad))
+ if (__skb_pad(last, pad, false))
return -ENOMEM;
__skb_put(last, pad);
diff --git a/drivers/net/wireless/mediatek/mt76/usb.c b/drivers/net/wireless/mediatek/mt76/usb.c
index a9af3aa6b80a..dfc12f6944a7 100644
--- a/drivers/net/wireless/mediatek/mt76/usb.c
+++ b/drivers/net/wireless/mediatek/mt76/usb.c
@@ -893,6 +893,9 @@ mt76u_tx_queue_skb(struct mt76_phy *phy, struct mt76_queue *q,
enum mt76_txq_id qid, struct sk_buff *skb,
struct mt76_wcid *wcid, struct ieee80211_sta *sta)
{
+ struct ieee80211_tx_status status = {
+ .sta = sta,
+ };
struct mt76_tx_info tx_info = {
.skb = skb,
};
@@ -900,17 +903,27 @@ mt76u_tx_queue_skb(struct mt76_phy *phy, struct mt76_queue *q,
u16 idx = q->head;
int err;
- if (q->queued == q->ndesc)
- return -ENOSPC;
+ if (q->queued == q->ndesc) {
+ err = -ENOSPC;
+ goto err_free_skb;
+ }
skb->prev = skb->next = NULL;
err = dev->drv->tx_prepare_skb(dev, NULL, qid, wcid, sta, &tx_info);
if (err < 0)
- return err;
+ goto err_free_skb;
err = mt76u_tx_setup_buffers(dev, tx_info.skb, q->entry[idx].urb);
- if (err < 0)
- return err;
+ if (err < 0) {
+ /*
+ * mt76_tx_status_skb_get() walks the idr and dereferences
+ * a freed skb. This SKB is not counted in non-AQL counter
+ * due to error return, so using 0xffff as wcid to keep
+ * balanced.
+ */
+ mt76_tx_complete_skb(dev, 0xffff, tx_info.skb);
+ goto err_ret;
+ }
mt76u_fill_bulk_urb(dev, USB_DIR_OUT, q->ep, q->entry[idx].urb,
mt76u_complete_tx, &q->entry[idx]);
@@ -921,6 +934,14 @@ mt76u_tx_queue_skb(struct mt76_phy *phy, struct mt76_queue *q,
q->queued++;
return idx;
+
+err_free_skb:
+ status.skb = tx_info.skb;
+ spin_lock_bh(&dev->rx_lock);
+ ieee80211_tx_status_ext(dev->hw, &status);
+ spin_unlock_bh(&dev->rx_lock);
+err_ret:
+ return err;
}
static void mt76u_tx_kick(struct mt76_dev *dev, struct mt76_queue *q)
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c
index b01b98d24b0a..0b622d7663b6 100644
--- a/drivers/net/wireless/realtek/rtw88/mac80211.c
+++ b/drivers/net/wireless/realtek/rtw88/mac80211.c
@@ -603,7 +603,7 @@ static int rtw_ops_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
hw_key_idx = key->keyidx;
}
- if (hw_key_idx > sec->total_cam_num) {
+ if (hw_key_idx >= sec->total_cam_num) {
ret = -ENOSPC;
goto out;
}
diff --git a/drivers/net/wireless/realtek/rtw89/fw.c b/drivers/net/wireless/realtek/rtw89/fw.c
index 0099c5c03e7a..b2991bc93f85 100644
--- a/drivers/net/wireless/realtek/rtw89/fw.c
+++ b/drivers/net/wireless/realtek/rtw89/fw.c
@@ -156,8 +156,24 @@ static int rtw89_fw_hdr_parser_v0(struct rtw89_dev *rtwdev, const u8 *fw, u32 le
if (!info)
return -EINVAL;
+ if (len < sizeof(*fw_hdr)) {
+ rtw89_err(rtwdev, "[ERR]fw header truncated\n");
+ return -EINVAL;
+ }
+
info->section_num = le32_get_bits(fw_hdr->w6, FW_HDR_W6_SEC_NUM);
+ if (!info->section_num || info->section_num > FWDL_SECTION_MAX_NUM) {
+ rtw89_err(rtwdev, "[ERR]invalid fw section num %u\n",
+ info->section_num);
+ return -EINVAL;
+ }
+
base_hdr_len = struct_size(fw_hdr, sections, info->section_num);
+ if (base_hdr_len > len) {
+ rtw89_err(rtwdev, "[ERR]fw header truncated\n");
+ return -EINVAL;
+ }
+
info->dynamic_hdr_en = le32_get_bits(fw_hdr->w7, FW_HDR_W7_DYN_HDR);
info->idmem_share_mode = le32_get_bits(fw_hdr->w7, FW_HDR_W7_IDMEM_SHARE_MODE);
@@ -455,9 +471,25 @@ static int rtw89_fw_hdr_parser_v1(struct rtw89_dev *rtwdev, const u8 *fw, u32 le
int ret;
u32 i;
+ if (len < sizeof(*fw_hdr)) {
+ rtw89_err(rtwdev, "[ERR]fw header truncated\n");
+ return -EINVAL;
+ }
+
info->section_num = le32_get_bits(fw_hdr->w6, FW_HDR_V1_W6_SEC_NUM);
+ if (!info->section_num || info->section_num > FWDL_SECTION_MAX_NUM) {
+ rtw89_err(rtwdev, "[ERR]invalid fw section num %u\n",
+ info->section_num);
+ return -EINVAL;
+ }
+
info->dsp_checksum = le32_get_bits(fw_hdr->w6, FW_HDR_V1_W6_DSP_CHKSUM);
base_hdr_len = struct_size(fw_hdr, sections, info->section_num);
+ if (base_hdr_len > len) {
+ rtw89_err(rtwdev, "[ERR]fw header truncated\n");
+ return -EINVAL;
+ }
+
info->dynamic_hdr_en = le32_get_bits(fw_hdr->w7, FW_HDR_V1_W7_DYN_HDR);
info->idmem_share_mode = le32_get_bits(fw_hdr->w7, FW_HDR_V1_W7_IDMEM_SHARE_MODE);
diff --git a/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk.c b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk.c
index c63f0e6364da..71342461f2b3 100644
--- a/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk.c
+++ b/drivers/net/wireless/realtek/rtw89/rtw8851b_rfk.c
@@ -1895,8 +1895,8 @@ static void _dpk_information(struct rtw89_dev *rtwdev, enum rtw89_phy_idx phy,
u8 kidx = dpk->cur_idx[path];
dpk->bp[path][kidx].band = chan->band_type;
- dpk->bp[path][kidx].ch = chan->band_width;
- dpk->bp[path][kidx].bw = chan->channel;
+ dpk->bp[path][kidx].ch = chan->channel;
+ dpk->bp[path][kidx].bw = chan->band_width;
rtw89_debug(rtwdev, RTW89_DBG_RFK,
"[DPK] S%d[%d] (PHY%d): TSSI %s/ DBCC %s/ %s/ CH%d/ %s\n",
diff --git a/drivers/net/wireless/realtek/rtw89/usb.c b/drivers/net/wireless/realtek/rtw89/usb.c
index aafa8b9e93df..22a1a988f307 100644
--- a/drivers/net/wireless/realtek/rtw89/usb.c
+++ b/drivers/net/wireless/realtek/rtw89/usb.c
@@ -819,10 +819,15 @@ static void rtw89_usb_rx_agg_cfg_v2(struct rtw89_dev *rtwdev)
static void rtw89_usb_rx_agg_cfg_v3(struct rtw89_dev *rtwdev)
{
- const u32 rxagg_0 = FIELD_PREP_CONST(B_BE_RXAGG_0_EN, 1) |
- FIELD_PREP_CONST(B_BE_RXAGG_0_NUM_TH, 255) |
- FIELD_PREP_CONST(B_BE_RXAGG_0_TIME_32US_TH, 32) |
- FIELD_PREP_CONST(B_BE_RXAGG_0_BUF_SZ_1K, 20);
+ struct rtw89_usb *rtwusb = rtw89_usb_priv(rtwdev);
+ u32 rxagg_0 = FIELD_PREP_CONST(B_BE_RXAGG_0_EN, 1) |
+ FIELD_PREP_CONST(B_BE_RXAGG_0_NUM_TH, 255) |
+ FIELD_PREP_CONST(B_BE_RXAGG_0_BUF_SZ_1K, 20);
+
+ if (rtwusb->udev->speed == USB_SPEED_SUPER)
+ rxagg_0 |= FIELD_PREP_CONST(B_BE_RXAGG_0_TIME_32US_TH, 1);
+ else
+ rxagg_0 |= FIELD_PREP_CONST(B_BE_RXAGG_0_TIME_32US_TH, 32);
rtw89_write32(rtwdev, R_BE_RXAGG_0_V1, rxagg_0);
rtw89_write32(rtwdev, R_BE_RXAGG_1_V1, 0x1F);
diff --git a/drivers/net/wireless/realtek/rtw89/wow.c b/drivers/net/wireless/realtek/rtw89/wow.c
index 2dfa24c54e7d..178c59692b13 100644
--- a/drivers/net/wireless/realtek/rtw89/wow.c
+++ b/drivers/net/wireless/realtek/rtw89/wow.c
@@ -16,10 +16,14 @@ void __rtw89_wow_parse_akm(struct rtw89_dev *rtwdev, struct sk_buff *skb)
{
struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)skb->data;
struct rtw89_wow_param *rtw_wow = &rtwdev->wow;
+ size_t hdr_len = offsetof(struct ieee80211_mgmt, u.assoc_req.variable);
const u8 *rsn, *ies = mgmt->u.assoc_req.variable;
struct rtw89_rsn_ie *rsn_ie;
- rsn = cfg80211_find_ie(WLAN_EID_RSN, ies, skb->len);
+ if (skb->len < hdr_len)
+ return;
+
+ rsn = cfg80211_find_ie(WLAN_EID_RSN, ies, skb->len - hdr_len);
if (!rsn)
return;
diff --git a/drivers/net/wwan/iosm/iosm_ipc_protocol_ops.c b/drivers/net/wwan/iosm/iosm_ipc_protocol_ops.c
index f2f86f41fd10..004920068c45 100644
--- a/drivers/net/wwan/iosm/iosm_ipc_protocol_ops.c
+++ b/drivers/net/wwan/iosm/iosm_ipc_protocol_ops.c
@@ -14,6 +14,11 @@ ipc_protocol_free_msg_get(struct iosm_protocol *ipc_protocol, int *index)
u32 new_head = (head + 1) % IPC_MEM_MSG_ENTRIES;
union ipc_mem_msg_entry *msg;
+ if (head >= IPC_MEM_MSG_ENTRIES) {
+ dev_err(ipc_protocol->dev, "invalid message head %u", head);
+ return NULL;
+ }
+
if (new_head == le32_to_cpu(ipc_protocol->p_ap_shm->msg_tail)) {
dev_err(ipc_protocol->dev, "message ring is full");
return NULL;
diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 3cdb7fd659db..dc229756bdc2 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -1530,12 +1530,17 @@ static inline bool tx_dealloc_work_todo(struct xenvif_queue *queue)
void xenvif_unmap_frontend_data_rings(struct xenvif_queue *queue)
{
- if (queue->tx.sring)
+ if (queue->tx.sring) {
xenbus_unmap_ring_vfree(xenvif_to_xenbus_device(queue->vif),
queue->tx.sring);
- if (queue->rx.sring)
+ queue->tx.sring = NULL;
+ }
+
+ if (queue->rx.sring) {
xenbus_unmap_ring_vfree(xenvif_to_xenbus_device(queue->vif),
queue->rx.sring);
+ queue->rx.sring = NULL;
+ }
}
int xenvif_map_frontend_data_rings(struct xenvif_queue *queue,
diff --git a/drivers/net/xen-netback/xenbus.c b/drivers/net/xen-netback/xenbus.c
index a826fa24258b..74c2745086bb 100644
--- a/drivers/net/xen-netback/xenbus.c
+++ b/drivers/net/xen-netback/xenbus.c
@@ -731,6 +731,7 @@ static void connect(struct backend_info *be)
unsigned int queue_index;
unsigned int requested_num_queues;
struct xenvif_queue *queue;
+ unsigned int num_queues;
/* Check whether the frontend requested multiple queues
* and read the number requested.
@@ -837,14 +838,15 @@ static void connect(struct backend_info *be)
return;
err:
- if (be->vif->num_queues > 0)
+ num_queues = be->vif->num_queues;
+ if (num_queues > 0)
xenvif_disconnect_data(be->vif); /* Clean up existing queues */
+ be->vif->num_queues = 0;
synchronize_net();
- for (queue_index = 0; queue_index < be->vif->num_queues; ++queue_index)
+ for (queue_index = 0; queue_index < num_queues; ++queue_index)
xenvif_deinit_queue(&be->vif->queues[queue_index]);
vfree(be->vif->queues);
be->vif->queues = NULL;
- be->vif->num_queues = 0;
xenvif_disconnect_ctrl(be->vif);
return;
}
diff --git a/drivers/nvme/host/auth.c b/drivers/nvme/host/auth.c
index e55920642f2c..cbb38935fa22 100644
--- a/drivers/nvme/host/auth.c
+++ b/drivers/nvme/host/auth.c
@@ -358,8 +358,21 @@ static int nvme_auth_process_dhchap_success1(struct nvme_ctrl *ctrl,
nvme_auth_hmac_name(chap->hash_id),
nvme_auth_dhgroup_name(chap->dhgroup_id));
- if (!data->rvalid)
+ if (!data->rvalid) {
+ /*
+ * The controller did not return a response. If we asked it to
+ * authenticate itself the session must not be used, otherwise
+ * mutual authentication would silently degrade to one-way.
+ */
+ if (chap->bi_directional) {
+ dev_warn(ctrl->device,
+ "qid %d: controller did not authenticate\n",
+ chap->qid);
+ chap->status = NVME_AUTH_DHCHAP_FAILURE_FAILED;
+ return -ECONNREFUSED;
+ }
return 0;
+ }
/* Validate controller response */
if (crypto_memneq(chap->response, data->rval, data->hl)) {
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 7eb0f9fe6b18..af71d02b8476 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2273,13 +2273,18 @@ static int nvme_query_fdp_granularity(struct nvme_ctrl *ctrl,
}
size = le32_to_cpu(hdr.sze);
+ if (size < sizeof(*h) + sizeof(*desc)) {
+ dev_warn(ctrl->device, "FDP config size too small:%zu\n",
+ size);
+ return 0;
+ }
if (size > PAGE_SIZE * MAX_ORDER_NR_PAGES) {
dev_warn(ctrl->device, "FDP config size too large:%zu\n",
size);
return 0;
}
- h = kvmalloc(size, GFP_KERNEL);
+ h = kvzalloc(size, GFP_KERNEL);
if (!h)
return -ENOMEM;
@@ -2305,9 +2310,9 @@ static int nvme_query_fdp_granularity(struct nvme_ctrl *ctrl,
desc = log;
end = log + size - sizeof(*h);
for (i = 0; i < fdp_idx; i++) {
- u16 dsze = le16_to_cpu(desc->dsze);
+ u16 dsze = get_unaligned_le16(&desc->dsze);
- if (!dsze || log + dsze > end) {
+ if (dsze < sizeof(*desc) || log + dsze + sizeof(*desc) > end) {
dev_warn(ctrl->device,
"FDP invalid config descriptor at index %d\n", i);
ret = 0;
@@ -2317,13 +2322,13 @@ static int nvme_query_fdp_granularity(struct nvme_ctrl *ctrl,
desc = log;
}
- if (le32_to_cpu(desc->nrg) > 1) {
+ if (get_unaligned_le32(&desc->nrg) > 1) {
dev_warn(ctrl->device, "FDP NRG > 1 not supported\n");
ret = 0;
goto out;
}
- info->runs = le64_to_cpu(desc->runs);
+ info->runs = get_unaligned_le64(&desc->runs);
out:
kvfree(h);
return ret;
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 9cb811a2ce1f..5b9b540693b5 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1748,6 +1748,7 @@ static void nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
"Bogus remote invalidation for rkey %#x\n",
req->mr ? req->mr->rkey : 0);
nvme_rdma_error_recovery(queue->ctrl);
+ return;
}
} else if (req->mr) {
int ret;
diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c
index e1e3dcfd084b..c0ecb5df1a30 100644
--- a/drivers/nvme/host/sysfs.c
+++ b/drivers/nvme/host/sysfs.c
@@ -809,7 +809,7 @@ static DEVICE_ATTR(io_timeout, S_IRUGO | S_IWUSR,
nvme_io_timeout_show, nvme_io_timeout_store);
#ifdef CONFIG_NVME_HOST_AUTH
-static ssize_t nvme_ctrl_dhchap_secret_show(struct device *dev,
+static ssize_t dhchap_secret_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
@@ -820,7 +820,7 @@ static ssize_t nvme_ctrl_dhchap_secret_show(struct device *dev,
return sysfs_emit(buf, "%s\n", opts->dhchap_secret);
}
-static ssize_t nvme_ctrl_dhchap_secret_store(struct device *dev,
+static ssize_t dhchap_secret_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
@@ -864,10 +864,9 @@ static ssize_t nvme_ctrl_dhchap_secret_store(struct device *dev,
return count;
}
-static DEVICE_ATTR(dhchap_secret, S_IRUGO | S_IWUSR,
- nvme_ctrl_dhchap_secret_show, nvme_ctrl_dhchap_secret_store);
+static DEVICE_ATTR_ADMIN_RW(dhchap_secret);
-static ssize_t nvme_ctrl_dhchap_ctrl_secret_show(struct device *dev,
+static ssize_t dhchap_ctrl_secret_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
@@ -878,7 +877,7 @@ static ssize_t nvme_ctrl_dhchap_ctrl_secret_show(struct device *dev,
return sysfs_emit(buf, "%s\n", opts->dhchap_ctrl_secret);
}
-static ssize_t nvme_ctrl_dhchap_ctrl_secret_store(struct device *dev,
+static ssize_t dhchap_ctrl_secret_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
@@ -922,8 +921,7 @@ static ssize_t nvme_ctrl_dhchap_ctrl_secret_store(struct device *dev,
return count;
}
-static DEVICE_ATTR(dhchap_ctrl_secret, S_IRUGO | S_IWUSR,
- nvme_ctrl_dhchap_ctrl_secret_show, nvme_ctrl_dhchap_ctrl_secret_store);
+static DEVICE_ATTR_ADMIN_RW(dhchap_ctrl_secret);
#endif
static struct attribute *nvme_dev_attrs[] = {
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 7eec983c0ead..ce28c9b03817 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/err.h>
#include <linux/crc32.h>
@@ -47,9 +48,24 @@ MODULE_PARM_DESC(wq_unbound, "Use unbound workqueue for nvme-tcp IO context (def
/*
* TLS handshake timeout
*/
-static int tls_handshake_timeout = 10;
+static unsigned int tls_handshake_timeout = 10;
#ifdef CONFIG_NVME_TCP_TLS
-module_param(tls_handshake_timeout, int, 0644);
+static int tls_handshake_timeout_set(const char *val,
+ const struct kernel_param *kp)
+{
+ return param_set_uint_minmax(val, kp, 0,
+ min_t(unsigned long, UINT_MAX / MSEC_PER_SEC,
+ MAX_JIFFY_OFFSET / HZ));
+}
+
+static const struct kernel_param_ops tls_handshake_timeout_ops = {
+ .set = tls_handshake_timeout_set,
+ .get = param_get_uint,
+};
+
+module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops,
+ &tls_handshake_timeout, 0644);
+__MODULE_PARM_TYPE(tls_handshake_timeout, "uint");
MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
@@ -1421,6 +1437,19 @@ static int nvme_tcp_try_recv(struct nvme_tcp_queue *queue)
lock_sock(sk);
queue->nr_cqe = 0;
consumed = sock->ops->read_sock(sk, &rd_desc, nvme_tcp_recv_skb);
+ /*
+ * nvme_tcp_recv_skb() handles its own errors, so a queue that is
+ * still enabled failed inside read_sock(). Handle it before
+ * release_sock() processes the backlog.
+ */
+ if (unlikely(consumed < 0 && consumed != -EAGAIN &&
+ queue->rd_enabled)) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d: receive failed: %d\n",
+ nvme_tcp_queue_id(queue), consumed);
+ queue->rd_enabled = false;
+ nvme_tcp_error_recovery(&queue->ctrl->ctrl);
+ }
release_sock(sk);
return consumed == -EAGAIN ? 0 : consumed;
}
@@ -1433,9 +1462,11 @@ static void nvme_tcp_io_work(struct work_struct *w)
do {
bool pending = false;
+ bool got_send_mutex = false;
int result;
if (mutex_trylock(&queue->send_mutex)) {
+ got_send_mutex = true;
result = nvme_tcp_try_send(queue);
mutex_unlock(&queue->send_mutex);
if (result > 0)
@@ -1450,8 +1481,13 @@ static void nvme_tcp_io_work(struct work_struct *w)
else if (unlikely(result < 0))
return;
- /* did we get some space after spending time in recv? */
- if (nvme_tcp_queue_has_pending(queue) &&
+ /*
+ * did we get some space after spending time in recv?
+ * Only check this if we could take send_mutex. Otherwise
+ * the task holding it sends the requests and queues io_work
+ * again after it releases the mutex.
+ */
+ if (got_send_mutex && nvme_tcp_queue_has_pending(queue) &&
sk_stream_is_writeable(queue->sock->sk))
pending = true;
@@ -1784,8 +1820,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
{
int qid = nvme_tcp_queue_id(queue);
int ret;
+ long wait_ret;
struct tls_handshake_args args;
- unsigned long tmo = tls_handshake_timeout * HZ;
+ unsigned int timeout = READ_ONCE(tls_handshake_timeout);
+ unsigned long tmo = (unsigned long)timeout * HZ;
key_serial_t keyring = nvme_keyring_id();
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n",
@@ -1799,7 +1837,7 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
if (nctrl->opts->keyring)
keyring = key_serial(nctrl->opts->keyring);
args.ta_keyring = keyring;
- args.ta_timeout_ms = tls_handshake_timeout * 1000;
+ args.ta_timeout_ms = timeout * MSEC_PER_SEC;
queue->tls_err = -EOPNOTSUPP;
init_completion(&queue->tls_complete);
ret = tls_client_hello_psk(&args, GFP_KERNEL);
@@ -1808,10 +1846,10 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
qid, ret);
return ret;
}
- ret = wait_for_completion_interruptible_timeout(&queue->tls_complete, tmo);
- if (ret <= 0) {
- if (ret == 0)
- ret = -ETIMEDOUT;
+ wait_ret = wait_for_completion_interruptible_timeout(&queue->tls_complete,
+ tmo);
+ if (wait_ret <= 0) {
+ ret = wait_ret < 0 ? -ERESTARTSYS : -ETIMEDOUT;
dev_err(nctrl->device,
"queue %d: TLS handshake failed, error %d\n",
diff --git a/drivers/nvme/target/auth.c b/drivers/nvme/target/auth.c
index 1680e5e6839c..574025730834 100644
--- a/drivers/nvme/target/auth.c
+++ b/drivers/nvme/target/auth.c
@@ -237,6 +237,7 @@ void nvmet_auth_sq_free(struct nvmet_sq *sq)
sq->dhchap_c2 = NULL;
kfree(sq->dhchap_skey);
sq->dhchap_skey = NULL;
+ sq->dhchap_skey_len = 0;
}
void nvmet_auth_sq_destroy(struct nvmet_sq *sq)
diff --git a/drivers/nvme/target/configfs.c b/drivers/nvme/target/configfs.c
index b03b5d1c2dc3..83b86180c50f 100644
--- a/drivers/nvme/target/configfs.c
+++ b/drivers/nvme/target/configfs.c
@@ -1689,8 +1689,10 @@ static ssize_t nvmet_subsys_attr_qid_max_store(struct config_item *item,
subsys->max_qid = qid_max;
/* Force reconnect */
+ mutex_lock(&subsys->lock);
list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry)
ctrl->ops->delete_ctrl(ctrl);
+ mutex_unlock(&subsys->lock);
up_write(&nvmet_config_sem);
return cnt;
diff --git a/drivers/nvme/target/fabrics-cmd-auth.c b/drivers/nvme/target/fabrics-cmd-auth.c
index f30c2539a6fd..d9dab0cda3b0 100644
--- a/drivers/nvme/target/fabrics-cmd-auth.c
+++ b/drivers/nvme/target/fabrics-cmd-auth.c
@@ -324,6 +324,7 @@ void nvmet_execute_auth_send(struct nvmet_req *req)
/* Restart negotiation */
pr_debug("%s: ctrl %d qid %d reset negotiation\n",
__func__, ctrl->cntlid, req->sq->qid);
+ nvmet_auth_sq_free(req->sq);
if (!req->sq->qid) {
dhchap_status = nvmet_setup_auth(ctrl, req->sq,
true);
@@ -599,10 +600,11 @@ void nvmet_execute_auth_receive(struct nvmet_req *req)
switch (req->sq->dhchap_step) {
case NVME_AUTH_DHCHAP_MESSAGE_CHALLENGE:
if (nvmet_auth_challenge(req, d, al) < 0) {
- pr_warn("ctrl %d qid %d: challenge error (%d)\n",
- ctrl->cntlid, req->sq->qid, status);
+ pr_warn("ctrl %d qid %d: challenge error\n",
+ ctrl->cntlid, req->sq->qid);
status = NVME_SC_INTERNAL;
- break;
+ kfree(d);
+ goto done;
}
req->sq->dhchap_step = NVME_AUTH_DHCHAP_MESSAGE_REPLY;
break;
diff --git a/drivers/nvme/target/fabrics-cmd.c b/drivers/nvme/target/fabrics-cmd.c
index 42d1d1811671..b451029ab8f4 100644
--- a/drivers/nvme/target/fabrics-cmd.c
+++ b/drivers/nvme/target/fabrics-cmd.c
@@ -14,6 +14,10 @@ static void nvmet_execute_prop_set(struct nvmet_req *req)
if (!nvmet_check_transfer_len(req, 0))
return;
+ if (!nvmet_check_auth_status(req)) {
+ status = NVME_SC_AUTH_REQUIRED | NVME_STATUS_DNR;
+ goto out;
+ }
if (req->cmd->prop_set.attrib & 1) {
req->error_loc =
diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
index f3e80095d7d1..9b46e16641ed 100644
--- a/drivers/nvme/target/fc.c
+++ b/drivers/nvme/target/fc.c
@@ -1705,7 +1705,7 @@ nvmet_fc_ls_create_association(struct nvmet_fc_tgtport *tgtport,
be16_to_cpu(rqst->assoc_cmd.sqsize));
if (!queue) {
ret = VERR_QUEUE_ALLOC_FAIL;
- nvmet_fc_tgt_a_put(iod->assoc);
+ nvmet_fc_schedule_delete_assoc(iod->assoc);
}
}
}
@@ -2486,11 +2486,11 @@ __nvmet_fc_fcp_nvme_cmd_done(struct nvmet_fc_tgtport *tgtport,
} else {
/*
- * try to push the data even if the SQE status is non-zero.
- * There may be a status where data still was intended to
- * be moved
+ * A failed command may not have filled its read buffer, so
+ * only push data for success, as the tcp and rdma targets do.
*/
- if ((fod->io_dir == NVMET_FCP_READ) && (fod->data_sg_cnt)) {
+ if (fod->io_dir == NVMET_FCP_READ && fod->data_sg_cnt &&
+ !cqe->status) {
/* push the data over before sending rsp */
nvmet_fc_transfer_fcp_data(tgtport, fod,
NVMET_FCOP_READDATA);
diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c
index f2d9e8901df4..4b7483bc9cc3 100644
--- a/drivers/nvme/target/io-cmd-bdev.c
+++ b/drivers/nvme/target/io-cmd-bdev.c
@@ -316,7 +316,7 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
&prot_miter);
if (unlikely(rc)) {
bio_io_error(bio);
- return;
+ goto out_plug;
}
}
@@ -336,11 +336,12 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
rc = nvmet_bdev_alloc_bip(req, bio, &prot_miter);
if (unlikely(rc)) {
bio_io_error(bio);
- return;
+ goto out_plug;
}
}
submit_bio(bio);
+out_plug:
blk_finish_plug(&plug);
}
diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c
index 0b22d183f927..695963dd932e 100644
--- a/drivers/nvme/target/io-cmd-file.c
+++ b/drivers/nvme/target/io-cmd-file.c
@@ -133,8 +133,8 @@ static bool nvmet_file_execute_io(struct nvmet_req *req, int ki_flags)
pos = le64_to_cpu(req->cmd->rw.slba) << req->ns->blksize_shift;
if (unlikely(pos + req->transfer_len > req->ns->size)) {
- nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC));
- return true;
+ ret = -ENOSPC;
+ goto complete;
}
memset(&req->f.iocb, 0, sizeof(struct kiocb));
diff --git a/drivers/nvme/target/pci-epf.c b/drivers/nvme/target/pci-epf.c
index c683450df422..a8df176c1a57 100644
--- a/drivers/nvme/target/pci-epf.c
+++ b/drivers/nvme/target/pci-epf.c
@@ -1073,10 +1073,22 @@ static int nvmet_pci_epf_iod_parse_sgl_segments(struct nvmet_pci_epf_ctrl *ctrl,
return 0;
}
+static u64 nvmet_pci_epf_iod_data_segs_len(struct nvmet_pci_epf_iod *iod)
+{
+ u64 len = 0;
+ int i;
+
+ for (i = 0; i < iod->nr_data_segs; i++)
+ len += iod->data_segs[i].length;
+
+ return len;
+}
+
static int nvmet_pci_epf_iod_parse_sgls(struct nvmet_pci_epf_iod *iod)
{
struct nvmet_pci_epf_ctrl *ctrl = iod->ctrl;
struct nvme_sgl_desc *sgl = &iod->cmd.common.dptr.sgl;
+ int ret;
if (sgl->type == (NVME_SGL_FMT_DATA_DESC << 4)) {
/* Single data descriptor case. */
@@ -1084,10 +1096,23 @@ static int nvmet_pci_epf_iod_parse_sgls(struct nvmet_pci_epf_iod *iod)
iod->data_segs = &iod->data_seg;
iod->data_seg.pci_addr = le64_to_cpu(sgl->addr);
iod->data_seg.length = le32_to_cpu(sgl->length);
- return 0;
+ } else {
+ ret = nvmet_pci_epf_iod_parse_sgl_segments(ctrl, iod);
+ if (ret)
+ return ret;
}
- return nvmet_pci_epf_iod_parse_sgl_segments(ctrl, iod);
+ /*
+ * The command executes on data_len bytes, but each segment is
+ * allocated and transferred in full, so they must describe exactly
+ * data_len bytes.
+ */
+ if (nvmet_pci_epf_iod_data_segs_len(iod) != iod->data_len) {
+ iod->status = NVME_SC_SGL_INVALID_DATA | NVME_STATUS_DNR;
+ return -EINVAL;
+ }
+
+ return 0;
}
static int nvmet_pci_epf_alloc_iod_data_buf(struct nvmet_pci_epf_iod *iod)
@@ -1152,7 +1177,6 @@ static void nvmet_pci_epf_complete_iod(struct nvmet_pci_epf_iod *iod)
unsigned long flags;
/* Print an error message for failed commands, except AENs. */
- iod->status = le16_to_cpu(iod->cqe.status) >> 1;
if (iod->status && iod->cmd.common.opcode != nvme_admin_async_event)
dev_err(iod->ctrl->dev,
"CQ[%d]: Command %s (0x%x) status 0x%0x\n",
diff --git a/drivers/nvme/target/pr.c b/drivers/nvme/target/pr.c
index ad13a93afb78..14eaa99f98fa 100644
--- a/drivers/nvme/target/pr.c
+++ b/drivers/nvme/target/pr.c
@@ -589,7 +589,7 @@ static u16 nvmet_pr_preempt(struct nvmet_req *req,
&ctrl->hostid, abort);
}
- if (holder == reg) {
+ if (holder == reg && prkey == holder->rkey) {
status = nvmet_pr_update_reg_attr(pr, holder,
nvmet_pr_update_holder_rtype, &rtype);
if (!status && original_rtype != rtype)
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index ffa1e172c259..ebf4a7bde524 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -864,7 +864,8 @@ static u16 nvmet_rdma_map_sgl_inline(struct nvmet_rdma_rsp *rsp)
return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
}
- if (off + len > rsp->queue->dev->inline_data_size) {
+ if (off > (u64)rsp->queue->dev->inline_data_size ||
+ len > (u64)rsp->queue->dev->inline_data_size - off) {
pr_err("invalid inline data offset!\n");
return NVME_SC_SGL_INVALID_OFFSET | NVME_STATUS_DNR;
}
@@ -1412,7 +1413,7 @@ nvmet_rdma_parse_cm_connect_req(struct rdma_conn_param *conn,
struct nvme_rdma_cm_req *req;
req = (struct nvme_rdma_cm_req *)conn->private_data;
- if (!req || conn->private_data_len == 0)
+ if (!req || conn->private_data_len < sizeof(*req))
return NVME_RDMA_CM_INVALID_LEN;
if (le16_to_cpu(req->recfmt) != NVME_RDMA_CM_FMT_1_0)
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index e59810175262..7937377436d3 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -6,6 +6,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/crc32c.h>
#include <linux/err.h>
@@ -76,8 +77,24 @@ MODULE_PARM_DESC(idle_poll_period_usecs,
/*
* TLS handshake timeout
*/
-static int tls_handshake_timeout = 10;
-module_param(tls_handshake_timeout, int, 0644);
+static unsigned int tls_handshake_timeout = 10;
+
+static int tls_handshake_timeout_set(const char *val,
+ const struct kernel_param *kp)
+{
+ return param_set_uint_minmax(val, kp, 0,
+ min_t(unsigned long, UINT_MAX / MSEC_PER_SEC,
+ MAX_JIFFY_OFFSET / HZ));
+}
+
+static const struct kernel_param_ops tls_handshake_timeout_ops = {
+ .set = tls_handshake_timeout_set,
+ .get = param_get_uint,
+};
+
+module_param_cb(tls_handshake_timeout, &tls_handshake_timeout_ops,
+ &tls_handshake_timeout, 0644);
+__MODULE_PARM_TYPE(tls_handshake_timeout, "uint");
MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
@@ -1904,6 +1921,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
{
int ret = -EOPNOTSUPP;
struct tls_handshake_args args;
+ unsigned int timeout = READ_ONCE(tls_handshake_timeout);
if (queue->state != NVMET_TCP_Q_TLS_HANDSHAKE) {
pr_warn("cannot start TLS in state %d\n", queue->state);
@@ -1917,7 +1935,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
args.ta_done = nvmet_tcp_tls_handshake_done;
args.ta_data = queue;
args.ta_keyring = key_serial(queue->port->nport->keyring);
- args.ta_timeout_ms = tls_handshake_timeout * 1000;
+ args.ta_timeout_ms = timeout * MSEC_PER_SEC;
ret = tls_server_hello_psk(&args, GFP_KERNEL);
if (ret) {
@@ -1925,7 +1943,7 @@ static int nvmet_tcp_tls_handshake(struct nvmet_tcp_queue *queue)
pr_err("failed to start TLS, err=%d\n", ret);
} else {
queue_delayed_work(nvmet_wq, &queue->tls_handshake_tmo_work,
- tls_handshake_timeout * HZ);
+ (unsigned long)timeout * HZ);
}
return ret;
}
diff --git a/drivers/nvme/target/zns.c b/drivers/nvme/target/zns.c
index 23a17c02abee..78d82b6ff121 100644
--- a/drivers/nvme/target/zns.c
+++ b/drivers/nvme/target/zns.c
@@ -282,6 +282,11 @@ static void nvmet_bdev_zone_zmgmt_recv_work(struct work_struct *w)
if (status)
goto out;
+ /* Only the zone count and matching descriptors are written below. */
+ status = nvmet_zero_sgl(req, 0, out_bufsize);
+ if (status)
+ goto out;
+
if (!req_slba_nr_zones) {
status = NVME_SC_SUCCESS;
goto out;
@@ -317,6 +322,11 @@ static void nvmet_bdev_zone_zmgmt_recv_work(struct work_struct *w)
void nvmet_bdev_execute_zone_mgmt_recv(struct nvmet_req *req)
{
+ u32 out_bufsize = (le32_to_cpu(req->cmd->zmr.numd) + 1) << 2;
+
+ if (!nvmet_check_transfer_len(req, out_bufsize))
+ return;
+
INIT_WORK(&req->z.zmgmt_work, nvmet_bdev_zone_zmgmt_recv_work);
queue_work(zbd_wq, &req->z.zmgmt_work);
}
diff --git a/drivers/nvmem/rockchip-otp.c b/drivers/nvmem/rockchip-otp.c
index 2c0feb036f3f..f8a8c7cece98 100644
--- a/drivers/nvmem/rockchip-otp.c
+++ b/drivers/nvmem/rockchip-otp.c
@@ -12,6 +12,7 @@
#include <linux/io.h>
#include <linux/iopoll.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/nvmem-provider.h>
#include <linux/reset.h>
#include <linux/slab.h>
@@ -80,6 +81,8 @@ struct rockchip_otp {
void __iomem *base;
struct reset_control *rst;
const struct rockchip_data *data;
+ /* Serializes access to the OTP controller state machine */
+ struct mutex mutex;
struct clk_bulk_data clks[];
};
@@ -272,10 +275,12 @@ static int rockchip_otp_read(void *context, unsigned int offset,
if (!otp->data || !otp->data->reg_read)
return -EINVAL;
+ mutex_lock(&otp->mutex);
+
ret = clk_bulk_prepare_enable(otp->data->num_clks, otp->clks);
if (ret < 0) {
dev_err(otp->dev, "failed to prepare/enable clks\n");
- return ret;
+ goto unlock;
}
offset += otp->data->read_offset;
@@ -308,6 +313,9 @@ static int rockchip_otp_read(void *context, unsigned int offset,
err:
clk_bulk_disable_unprepare(otp->data->num_clks, otp->clks);
+unlock:
+ mutex_unlock(&otp->mutex);
+
return ret;
}
@@ -431,6 +439,11 @@ static int rockchip_otp_probe(struct platform_device *pdev)
otp->data = data;
otp->dev = dev;
+
+ ret = devm_mutex_init(dev, &otp->mutex);
+ if (ret)
+ return ret;
+
otp->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(otp->base))
return dev_err_probe(dev, PTR_ERR(otp->base),
diff --git a/drivers/parisc/eisa_enumerator.c b/drivers/parisc/eisa_enumerator.c
index e0a57086a8a8..011b9ac6f97b 100644
--- a/drivers/parisc/eisa_enumerator.c
+++ b/drivers/parisc/eisa_enumerator.c
@@ -311,6 +311,7 @@ static int configure_function(const unsigned char *buf, int *more)
static int parse_slot_config(int slot,
const unsigned char *buf,
+ unsigned int buflen,
struct eeprom_eisa_slot_info *es,
struct resource *io_parent,
struct resource *mem_parent)
@@ -333,8 +334,8 @@ static int parse_slot_config(int slot,
printk(KERN_INFO "EISA slot %d: %s %s ",
slot, board, es->flags&HPEE_FLAG_BOARD_IS_ISA ? "ISA" : "EISA");
- maxlen = es->config_data_length < HPEE_MAX_LENGTH ?
- es->config_data_length : HPEE_MAX_LENGTH;
+ /* never parse beyond the data that is actually in the buffer */
+ maxlen = min_t(unsigned int, es->config_data_length, buflen);
while ((pos < maxlen) && (num_func <= es->num_functions)) {
pos+=configure_function(buf+pos, &function_len);
@@ -491,6 +492,13 @@ int eisa_enumerator(unsigned long eeprom_addr,
printk(KERN_INFO "Enumerating EISA bus\n");
eh = (struct eeprom_header*)(eeprom_buf);
+ /*
+ * The EEPROM contents are not to be trusted: clamp the number of
+ * slots so that all slot records fit into eeprom_buf.
+ */
+ eh->num_slots = min_t(u8, eh->num_slots,
+ (HPEE_MAX_LENGTH - sizeof(*eh)) /
+ sizeof(struct eeprom_eisa_slot_info));
for (i=0;i<eh->num_slots;i++) {
struct eeprom_eisa_slot_info *es;
@@ -503,7 +511,8 @@ int eisa_enumerator(unsigned long eeprom_addr,
if (es->config_data_offset < HPEE_MAX_LENGTH) {
if (parse_slot_config(i+1, &eeprom_buf[es->config_data_offset],
- es, io_parent, mem_parent)) {
+ HPEE_MAX_LENGTH - es->config_data_offset,
+ es, io_parent, mem_parent)) {
return -1;
}
} else {
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index f7a2eb257c16..785f575e0134 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -542,7 +542,7 @@ static int imx_setup_phy_mpll(struct imx_pcie *imx_pcie)
return 0;
for (i = 0; i < imx_pcie->num_clks; i++)
- if (strncmp(clks[i].id, "pcie_phy", 8) == 0)
+ if (clks[i].id && strncmp(clks[i].id, "pcie_phy", 8) == 0)
phy_rate = clk_get_rate(clks[i].clk);
switch (phy_rate) {
@@ -1836,7 +1836,8 @@ static int imx_pcie_probe(struct platform_device *pdev)
return dev_err_probe(dev, imx_pcie->num_clks,
"failed to get clocks\n");
for (i = 0; i < imx_pcie->num_clks; i++)
- if (strncmp(imx_pcie->clks[i].id, "extref", 6) == 0)
+ if (imx_pcie->clks[i].id &&
+ strncmp(imx_pcie->clks[i].id, "extref", 6) == 0)
imx_pcie->enable_ext_refclk = true;
if (imx_check_flag(imx_pcie, IMX_PCIE_FLAG_HAS_PHYDRV)) {
diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c
index 56184e6ca6e6..28949386b0e8 100644
--- a/drivers/pci/controller/dwc/pcie-qcom-ep.c
+++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c
@@ -944,6 +944,8 @@ static void qcom_pcie_ep_remove(struct platform_device *pdev)
if (pcie_ep->link_status == QCOM_PCIE_EP_LINK_DISABLED)
return;
+ pci_epc_deinit_notify(pcie_ep->pci.ep.epc);
+ dw_pcie_ep_deinit(&pcie_ep->pci.ep);
qcom_pcie_disable_resources(pcie_ep);
}
diff --git a/drivers/pci/controller/dwc/pcie-qcom.c b/drivers/pci/controller/dwc/pcie-qcom.c
index b58a607b713f..89a476c37098 100644
--- a/drivers/pci/controller/dwc/pcie-qcom.c
+++ b/drivers/pci/controller/dwc/pcie-qcom.c
@@ -1953,7 +1953,7 @@ static bool qcom_pcie_is_child_node(struct device *dev,
return false;
}
-/* Parse PERST# from all nodes in depth first manner starting from @np */
+/* Collect PERST# GPIOs from PCI bridge nodes depth-first, starting at @np */
static int qcom_pcie_parse_perst(struct qcom_pcie *pcie,
struct qcom_pcie_port *port,
struct device_node *np)
@@ -2019,6 +2019,9 @@ static int qcom_pcie_parse_perst(struct qcom_pcie *pcie,
parse_child_node:
for_each_available_child_of_node_scoped(np, child) {
+ if (!of_node_is_type(child, "pci"))
+ continue;
+
ret = qcom_pcie_parse_perst(pcie, port, child);
if (ret)
return ret;
diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index fbe465a29068..241adcd72789 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -411,6 +411,57 @@ static int rcar_gen4_pcie_host_msi_init(struct dw_pcie_rp *pp)
return ret;
}
+static int rcar_gen4_pcie_enable_device(struct pci_host_bridge *bridge,
+ struct pci_dev *dev)
+{
+ /*
+ * R-Car Gen4 PCIe controller has a hardware limitation of 256 Bytes
+ * Max_Payload_Size (MPS). PCIe specification indicates that the MPS
+ * must not exceed minimum MPS of any element along the packet path.
+ * The controller reports Max_Payload_Size_Supported (MPSS) 256 Bytes
+ * for header type 0 and 128 Bytes for header type 1. The PCIe core
+ * will not allow MPS to be set higher than MPSS; warn here in case
+ * something went very wrong in the core.
+ *
+ * For details, refer to chapter "104.1.1 Features" in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ WARN_ON(pcie_get_mps(dev) > 256);
+
+ /*
+ * R-Car Gen4 Reference Manual, chapter 104.4.8 Usage notes for
+ * MRRS (Max Read Request Size) states:
+ *
+ * Please set "Max Read Request Size" to 128 bytes or 256 bytes.
+ * If "Max Read Request Size" is set to anything other than the
+ * above, the transferred data will not match the expected value.
+ *
+ * This limitation also seems to apply to devices issuing MRd TLPs.
+ * This limitation can be triggered by using non-HMB NVMe SSD with
+ * Max_Read_Request_Size 512 Bytes, for example Crucial P5 Plus.
+ * Any write into the SSD (MRd TLP issued by the SSD) longer than
+ * 256 Bytes wraps around at 256 Byte boundary, and the same data
+ * are written into the SSD starting at offset 0 and at 256 Bytes.
+ *
+ * Limit Max_Read_Request_Size to at most 256 Bytes for each
+ * device connected to this PCIe controller to avoid this behavior.
+ *
+ * For details, refer to aforementioned chapter in either of:
+ * R-Car S4 R19UH0161EJ0140 Rev.1.40 Jul. 31, 2026 or
+ * R-Car V4H R19UH0186EJ0140 Rev.1.40 Aug. 7, 2026 or
+ * R-Car V4M R19UH0217EJ0110 Rev.1.10 Jun. 30, 2026.
+ */
+ bridge->no_inc_mrrs = 1;
+ if (pcie_get_readrq(dev) > 256) {
+ pci_info(dev, "Limiting MRRS to 256 bytes\n");
+ pcie_set_readrq(dev, 256);
+ }
+
+ return 0;
+}
+
/* Host mode */
static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
{
@@ -418,6 +469,9 @@ static int rcar_gen4_pcie_host_init(struct dw_pcie_rp *pp)
struct rcar_gen4_pcie *rcar = to_rcar_gen4_pcie(dw);
int ret;
+ if (pp->bridge)
+ pp->bridge->enable_device = rcar_gen4_pcie_enable_device;
+
gpiod_set_value_cansleep(dw->pe_rst, 1);
ret = rcar_gen4_pcie_common_init(rcar);
diff --git a/drivers/pci/controller/pcie-aspeed.c b/drivers/pci/controller/pcie-aspeed.c
index 74b03a55643f..ecb7562f84c0 100644
--- a/drivers/pci/controller/pcie-aspeed.c
+++ b/drivers/pci/controller/pcie-aspeed.c
@@ -761,16 +761,21 @@ static int aspeed_pcie_port_init(struct aspeed_pcie_port *port)
port->slot);
ret = phy_init(port->phy);
- if (ret)
+ if (ret) {
+ clk_disable_unprepare(port->clk);
return dev_err_probe(dev, ret,
"failed to init phy pcie for slot (%d)\n",
port->slot);
+ }
ret = phy_set_mode_ext(port->phy, PHY_MODE_PCIE, PHY_MODE_PCIE_RC);
- if (ret)
+ if (ret) {
+ phy_exit(port->phy);
+ clk_disable_unprepare(port->clk);
return dev_err_probe(dev, ret,
"failed to set phy mode for slot (%d)\n",
port->slot);
+ }
reset_control_deassert(port->perst);
msleep(PCIE_RESET_CONFIG_WAIT_MS);
diff --git a/drivers/pci/controller/pcie-rockchip-ep.c b/drivers/pci/controller/pcie-rockchip-ep.c
index 799461335762..9ebc227a1ef8 100644
--- a/drivers/pci/controller/pcie-rockchip-ep.c
+++ b/drivers/pci/controller/pcie-rockchip-ep.c
@@ -553,19 +553,6 @@ static void rockchip_pcie_ep_link_training(struct work_struct *work)
if (ret)
goto again;
- /*
- * Check the current speed: if gen2 speed was requested and we are not
- * at gen2 speed yet, retrain again for gen2.
- */
- val = rockchip_pcie_read(rockchip, PCIE_CORE_CTRL);
- if (!PCIE_LINK_IS_GEN2(val) && rockchip->link_gen == 2) {
- /* Enable retrain for gen2 */
- rockchip_pcie_ep_retrain_link(rockchip);
- readl_poll_timeout(rockchip->apb_base + PCIE_CORE_CTRL,
- val, PCIE_LINK_IS_GEN2(val), 50,
- LINK_TRAIN_TIMEOUT);
- }
-
/* Check again that the link is up */
if (!rockchip_pcie_ep_link_up(rockchip))
goto again;
diff --git a/drivers/pci/controller/pcie-rockchip-host.c b/drivers/pci/controller/pcie-rockchip-host.c
index d203c4876d30..0b52f2e370f4 100644
--- a/drivers/pci/controller/pcie-rockchip-host.c
+++ b/drivers/pci/controller/pcie-rockchip-host.c
@@ -328,26 +328,6 @@ static int rockchip_pcie_host_init_port(struct rockchip_pcie *rockchip)
goto err_power_off_phy;
}
- if (rockchip->link_gen == 2) {
- /*
- * Enable retrain for gen2. This should be configured only after
- * gen1 finished.
- */
- status = rockchip_pcie_read(rockchip, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL2);
- status &= ~PCI_EXP_LNKCTL2_TLS;
- status |= PCI_EXP_LNKCTL2_TLS_5_0GT;
- rockchip_pcie_write(rockchip, status, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL2);
- status = rockchip_pcie_read(rockchip, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL);
- status |= PCI_EXP_LNKCTL_RL;
- rockchip_pcie_write(rockchip, status, PCIE_RC_CONFIG_CR + PCI_EXP_LNKCTL);
-
- err = readl_poll_timeout(rockchip->apb_base + PCIE_CORE_CTRL,
- status, PCIE_LINK_IS_GEN2(status), 20,
- 500 * USEC_PER_MSEC);
- if (err)
- dev_dbg(dev, "PCIe link training gen2 timeout, fall back to gen1!\n");
- }
-
/* Check the final link width from negotiated lane counter from MGMT */
status = rockchip_pcie_read(rockchip, PCIE_CORE_CTRL);
status = 0x1 << ((status & PCIE_CORE_PL_CONF_LANE_MASK) >>
diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c
index 8d78ab1b31f0..45cc888ac553 100644
--- a/drivers/pci/controller/pcie-rockchip.c
+++ b/drivers/pci/controller/pcie-rockchip.c
@@ -66,8 +66,10 @@ int rockchip_pcie_parse_dt(struct rockchip_pcie *rockchip)
}
rockchip->link_gen = of_pci_get_max_link_speed(node);
- if (rockchip->link_gen < 0 || rockchip->link_gen > 2)
- rockchip->link_gen = 2;
+ if (rockchip->link_gen < 0 || rockchip->link_gen >= 2) {
+ rockchip->link_gen = 1;
+ dev_warn(dev, "invalid max-link-speed, limited to 2.5 GT/s\n");
+ }
for (i = 0; i < ROCKCHIP_NUM_PM_RSTS; i++)
rockchip->pm_rsts[i].id = rockchip_pci_pm_rsts[i];
@@ -147,12 +149,8 @@ int rockchip_pcie_init_port(struct rockchip_pcie *rockchip)
goto err_exit_phy;
}
- if (rockchip->link_gen == 2)
- rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_2,
- PCIE_CLIENT_CONFIG);
- else
- rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1,
- PCIE_CLIENT_CONFIG);
+ rockchip_pcie_write(rockchip, PCIE_CLIENT_GEN_SEL_1,
+ PCIE_CLIENT_CONFIG);
regs = PCIE_CLIENT_ARI_ENABLE |
PCIE_CLIENT_CONF_LANE_NUM(rockchip->lanes);
diff --git a/drivers/pci/controller/pcie-rockchip.h b/drivers/pci/controller/pcie-rockchip.h
index 3e82a69b9c00..b5da15601b58 100644
--- a/drivers/pci/controller/pcie-rockchip.h
+++ b/drivers/pci/controller/pcie-rockchip.h
@@ -42,7 +42,6 @@
#define PCIE_CLIENT_MODE_RC HWORD_SET_BIT(0x0040)
#define PCIE_CLIENT_MODE_EP HWORD_CLR_BIT(0x0040)
#define PCIE_CLIENT_GEN_SEL_1 HWORD_CLR_BIT(0x0080)
-#define PCIE_CLIENT_GEN_SEL_2 HWORD_SET_BIT(0x0080)
#define PCIE_CLIENT_LEGACY_INT_CTRL (PCIE_CLIENT_BASE + 0x0c)
#define PCIE_CLIENT_INT_IN_ASSERT HWORD_SET_BIT(0x0002)
#define PCIE_CLIENT_INT_IN_DEASSERT HWORD_CLR_BIT(0x0002)
@@ -197,8 +196,6 @@
(((x) & PCIE_CORE_PL_CONF_LS_MASK) == PCIE_CORE_PL_CONF_LS_READY)
#define PCIE_LINK_UP(x) \
(((x) & PCIE_CLIENT_LINK_STATUS_MASK) == PCIE_CLIENT_LINK_STATUS_UP)
-#define PCIE_LINK_IS_GEN2(x) \
- (((x) & PCIE_CORE_PL_CONF_SPEED_MASK) == PCIE_CORE_PL_CONF_SPEED_5G)
#define RC_REGION_0_ADDR_TRANS_H 0x00000000
#define RC_REGION_0_ADDR_TRANS_L 0x00000000
diff --git a/drivers/pci/controller/pcie-rzg3s-host.c b/drivers/pci/controller/pcie-rzg3s-host.c
index a1a78bdb7af1..3f7393c49a08 100644
--- a/drivers/pci/controller/pcie-rzg3s-host.c
+++ b/drivers/pci/controller/pcie-rzg3s-host.c
@@ -1006,22 +1006,38 @@ static const struct irq_domain_ops rzg3s_pcie_intx_domain_ops = {
.xlate = irq_domain_xlate_onetwocell,
};
+static void rzg3s_pcie_teardown_intx(struct rzg3s_pcie_host *host,
+ int count)
+{
+ while (--count >= 0) {
+ irq_set_chained_handler_and_data(host->intx_irqs[count], NULL,
+ NULL);
+ }
+
+ if (host->intx_domain)
+ irq_domain_remove(host->intx_domain);
+}
+
static int rzg3s_pcie_init_irqdomain(struct rzg3s_pcie_host *host)
{
struct device *dev = host->dev;
struct platform_device *pdev = to_platform_device(dev);
+ int i, ret;
- for (int i = 0; i < PCI_NUM_INTX; i++) {
+ for (i = 0; i < PCI_NUM_INTX; i++) {
char irq_name[5] = {0};
int irq;
scnprintf(irq_name, ARRAY_SIZE(irq_name), "int%c", 'a' + i);
irq = platform_get_irq_byname(pdev, irq_name);
- if (irq < 0)
- return dev_err_probe(dev, -EINVAL,
- "Failed to parse and map INT%c IRQ\n",
- 'A' + i);
+ if (irq < 0) {
+ ret = irq;
+ dev_err_probe(dev, ret,
+ "Failed to parse and map INT%c IRQ\n",
+ 'A' + i);
+ goto teardown_intx;
+ }
host->intx_irqs[i] = irq;
irq_set_chained_handler_and_data(irq,
@@ -1033,21 +1049,27 @@ static int rzg3s_pcie_init_irqdomain(struct rzg3s_pcie_host *host)
PCI_NUM_INTX,
&rzg3s_pcie_intx_domain_ops,
host);
- if (!host->intx_domain)
- return dev_err_probe(dev, -EINVAL,
- "Failed to add irq domain for INTx IRQs\n");
+ if (!host->intx_domain) {
+ ret = -EINVAL;
+ dev_err_probe(dev, ret,
+ "Failed to add irq domain for INTx IRQs\n");
+ goto teardown_intx;
+ }
irq_domain_update_bus_token(host->intx_domain, DOMAIN_BUS_WIRED);
if (IS_ENABLED(CONFIG_PCI_MSI)) {
- int ret = rzg3s_pcie_init_msi(host);
+ ret = rzg3s_pcie_init_msi(host);
- if (ret) {
- irq_domain_remove(host->intx_domain);
- return ret;
- }
+ if (ret)
+ goto teardown_intx;
}
return 0;
+
+teardown_intx:
+ rzg3s_pcie_teardown_intx(host, i);
+
+ return ret;
}
static void rzg3s_pcie_teardown_irqdomain(struct rzg3s_pcie_host *host)
@@ -1055,7 +1077,7 @@ static void rzg3s_pcie_teardown_irqdomain(struct rzg3s_pcie_host *host)
if (IS_ENABLED(CONFIG_PCI_MSI))
rzg3s_pcie_teardown_msi(host);
- irq_domain_remove(host->intx_domain);
+ rzg3s_pcie_teardown_intx(host, PCI_NUM_INTX);
}
static int rzg3s_pcie_set_max_link_speed(struct rzg3s_pcie_host *host)
@@ -1897,6 +1919,7 @@ static int rzg3s_pcie_probe(struct platform_device *pdev)
return 0;
host_probe_teardown:
+ clk_disable_unprepare(host->port.refclk);
rzg3s_pcie_teardown_irqdomain(host);
host->data->config_deinit(host);
rpm_put:
diff --git a/drivers/pci/controller/pcie-xilinx-dma-pl.c b/drivers/pci/controller/pcie-xilinx-dma-pl.c
index b037c8f315e4..e18eeecdc8a9 100644
--- a/drivers/pci/controller/pcie-xilinx-dma-pl.c
+++ b/drivers/pci/controller/pcie-xilinx-dma-pl.c
@@ -581,7 +581,7 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port)
port->pldma_domain = irq_domain_create_linear(of_fwnode_handle(pcie_intc_node), 32,
&event_domain_ops, port);
if (!port->pldma_domain)
- return -ENOMEM;
+ goto err_put_node;
irq_domain_update_bus_token(port->pldma_domain, DOMAIN_BUS_NEXUS);
@@ -589,7 +589,7 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port)
&intx_domain_ops, port);
if (!port->intx_domain) {
dev_err(dev, "Failed to get a INTx IRQ domain\n");
- return -ENOMEM;
+ goto err_put_node;
}
irq_domain_update_bus_token(port->intx_domain, DOMAIN_BUS_WIRED);
@@ -597,13 +597,17 @@ static int xilinx_pl_dma_pcie_init_irq_domain(struct pl_dma_pcie *port)
ret = xilinx_pl_dma_pcie_init_msi_irq_domain(port);
if (ret != 0) {
irq_domain_remove(port->intx_domain);
- return -ENOMEM;
+ goto err_put_node;
}
of_node_put(pcie_intc_node);
raw_spin_lock_init(&port->lock);
return 0;
+
+err_put_node:
+ of_node_put(pcie_intc_node);
+ return -ENOMEM;
}
static int xilinx_pl_dma_pcie_setup_irq(struct pl_dma_pcie *port)
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 241023ecf677..9d97a832785e 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -92,6 +92,23 @@ enum vmd_features {
* referred to as MEMBAR2 or MSI-X BAR.
*/
VMD_FEAT_USE_BIOS_INFO = (1 << 6),
+
+ /*
+ * Some Intel VMD devices are affected by interrupt ordering errata,
+ * including MTL016 (0x7d0b) and ARL004 (0xad0b):
+ * the VMD may signal its MSI before the posted writes that carry the
+ * child device's DMA data have landed in memory. The demuxed handler
+ * then runs against a not-yet-coherent completion queue and misses the
+ * completion entirely, so the I/O is only recovered when the block
+ * layer timeout fires and polls the queue ("timeout, completion
+ * polled"). Intel's documented workaround is to issue a dummy read
+ * to the MSI initiator (the child device) before handling the
+ * interrupt: the read completion cannot pass the device's earlier
+ * posted writes, so it pulls them into memory per PCIe ordering
+ * rules. A read that terminates at the VMD itself does not order
+ * against the child's writes and is not sufficient (measured).
+ */
+ VMD_FEAT_INTERRUPT_QUIRK = (1 << 7),
};
#define VMD_BIOS_PM_QUIRK_LTR 0x1003 /* 3145728 ns */
@@ -114,6 +131,9 @@ static DEFINE_RAW_SPINLOCK(list_lock);
* @irq: back pointer to parent.
* @enabled: true if driver enabled IRQ
* @virq: the virtual IRQ value provided to the requesting driver.
+ * @flush_addr: config space address of the initiating device, read before
+ * demuxing to flush its posted writes; NULL if the
+ * VMD is not affected.
*
* Every MSI/MSI-X IRQ requested for a device in a VMD domain will be mapped to
* a VMD IRQ using this structure.
@@ -123,8 +143,14 @@ struct vmd_irq {
struct vmd_irq_list *irq;
bool enabled;
unsigned int virq;
+ void __iomem *flush_addr;
};
+struct vmd_dev;
+
+static void __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
+ unsigned int devfn, int reg, int len);
+
/**
* struct vmd_irq_list - list of driver requested IRQs mapping to a VMD vector
* @irq_list: the list of irq's the VMD one demuxes to.
@@ -132,12 +158,15 @@ struct vmd_irq {
* @count: number of child IRQs assigned to this vector; used to track
* sharing.
* @virq: The underlying VMD Linux interrupt number
+ * @vmd: back pointer to the owning VMD device; serializes the MTL016
+ * flush read against other config space access.
*/
struct vmd_irq_list {
struct list_head irq_list;
struct srcu_struct srcu;
unsigned int count;
unsigned int virq;
+ struct vmd_dev *vmd;
};
struct vmd_dev {
@@ -295,6 +324,13 @@ static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq,
INIT_LIST_HEAD(&vmdirq->node);
vmdirq->irq = vmd_next_irq(vmd, desc);
vmdirq->virq = virq + i;
+ if (vmd->features & VMD_FEAT_INTERRUPT_QUIRK) {
+ struct pci_dev *pdev = msi_desc_to_pci_dev(desc);
+
+ vmdirq->flush_addr = vmd_cfg_addr(vmd, pdev->bus,
+ pdev->devfn,
+ PCI_VENDOR_ID, 2);
+ }
irq_domain_set_info(domain, virq + i, vmdirq->irq->virq,
&vmd_msi_controller, vmdirq,
@@ -756,8 +792,20 @@ static irqreturn_t vmd_irq(int irq, void *data)
int idx;
idx = srcu_read_lock(&irqs->srcu);
- list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node)
+ list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node) {
+ /*
+ * MTL016: the MSI may have outrun the initiating device's
+ * posted writes (e.g. its NVMe completion entry). A read
+ * that completes at the initiator flushes them, so the
+ * demuxed handler observes a coherent completion queue.
+ * The value is discarded; only the ordering matters.
+ */
+ if (vmdirq->flush_addr) {
+ guard(raw_spinlock)(&irqs->vmd->cfg_lock);
+ readw(vmdirq->flush_addr);
+ }
generic_handle_irq(vmdirq->virq);
+ }
srcu_read_unlock(&irqs->srcu, idx);
return IRQ_HANDLED;
@@ -788,6 +836,7 @@ static int vmd_alloc_irqs(struct vmd_dev *vmd)
return err;
INIT_LIST_HEAD(&vmd->irqs[i].irq_list);
+ vmd->irqs[i].vmd = vmd;
vmd->irqs[i].virq = pci_irq_vector(dev, i);
err = devm_request_irq(&dev->dev, vmd->irqs[i].virq,
vmd_irq, IRQF_NO_THREAD,
@@ -1250,9 +1299,9 @@ static const struct pci_device_id vmd_ids[] = {
{PCI_VDEVICE(INTEL, 0xa77f),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0x7d0b),
- .driver_data = VMD_FEATS_CLIENT,},
+ .driver_data = VMD_FEATS_CLIENT | VMD_FEAT_INTERRUPT_QUIRK,},
{PCI_VDEVICE(INTEL, 0xad0b),
- .driver_data = VMD_FEATS_CLIENT,},
+ .driver_data = VMD_FEATS_CLIENT | VMD_FEAT_INTERRUPT_QUIRK,},
{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0xb60b),
diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c
index ac95b1d2d999..6b5d9ba8e969 100644
--- a/drivers/pci/doe.c
+++ b/drivers/pci/doe.c
@@ -208,8 +208,9 @@ static int pci_doe_sysfs_feature_populate(struct pci_dev *pdev,
pci_warn(pdev, "Failed adding %s to sysfs group\n",
attrs[i].attr.name);
goto fail;
- } else
- kfree(attrs[i].attr.name);
+ }
+ kfree(attrs[i].attr.name);
+ attrs[i].attr.name = NULL;
}
}
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index c3caec927d74..8ae50779452a 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -132,7 +132,7 @@ struct epf_ntb_ctrl {
} __packed;
struct epf_ntb {
- struct ntb_dev ntb;
+ struct ntb_dev *ntb;
struct pci_epf *epf;
struct config_group group;
@@ -166,10 +166,15 @@ struct epf_ntb {
void __iomem *vpci_mw_addr[MAX_MW];
struct delayed_work cmd_handler;
+ struct pci_bus *vpci_bus;
};
#define to_epf_ntb(epf_group) container_of((epf_group), struct epf_ntb, group)
-#define ntb_ndev(__ntb) container_of(__ntb, struct epf_ntb, ntb)
+
+static struct epf_ntb *ntb_ndev(struct ntb_dev *ntb)
+{
+ return ntb->pdev->sysdata;
+}
static struct pci_epf_header epf_ntb_header = {
.vendorid = PCI_ANY_ID,
@@ -195,7 +200,7 @@ static int epf_ntb_link_up(struct epf_ntb *ntb, bool link_up)
else
ntb->reg->link_status &= ~LINK_STATUS_UP;
- ntb_link_event(&ntb->ntb);
+ ntb_link_event(ntb->ntb);
return 0;
}
@@ -284,7 +289,7 @@ static void epf_ntb_cmd_handler(struct work_struct *work)
i++) {
if (ntb->epf_db[i]) {
atomic64_or(1 << (i - EPF_IRQ_DB_START), &ntb->db);
- ntb_db_event(&ntb->ntb, i - EPF_IRQ_DB_START);
+ ntb_db_event(ntb->ntb, i - EPF_IRQ_DB_START);
ntb->epf_db[i] = 0;
}
}
@@ -348,12 +353,18 @@ static void epf_ntb_cmd_handler(struct work_struct *work)
static irqreturn_t epf_ntb_doorbell_handler(int irq, void *data)
{
struct epf_ntb *ntb = data;
+ struct ntb_dev *ndev;
int i;
+ /* Pair with smp_store_release() in pci_vntb_probe() */
+ ndev = smp_load_acquire(&ntb->ntb);
+ if (!ndev)
+ return IRQ_HANDLED;
+
for (i = EPF_IRQ_DB_START; i < ntb->db_count; i++)
if (irq == ntb->epf->db_msg[i].virq) {
atomic64_or(1 << (i - EPF_IRQ_DB_START), &ntb->db);
- ntb_db_event(&ntb->ntb, i - EPF_IRQ_DB_START);
+ ntb_db_event(ndev, i - EPF_IRQ_DB_START);
}
return IRQ_HANDLED;
@@ -985,7 +996,6 @@ static int epf_ntb_epc_init(struct epf_ntb *ntb)
}
INIT_DELAYED_WORK(&ntb->cmd_handler, epf_ntb_cmd_handler);
- queue_work(kpcintb_workqueue, &ntb->cmd_handler.work);
atomic64_set(&ntb->peer_db_pending, 0);
enable_work(&ntb->peer_db_work);
@@ -1337,6 +1347,7 @@ static int vpci_scan_bus(void *sysdata)
pr_err("create pci bus failed\n");
return -EINVAL;
}
+ ndev->vpci_bus = vpci_bus;
pci_bus_add_devices(vpci_bus);
@@ -1420,7 +1431,7 @@ static int vntb_epf_mw_set_trans(struct ntb_dev *ndev, int pidx, int idx,
int ret;
struct device *dev;
- dev = &ntb->ntb.dev;
+ dev = &ndev->dev;
barno = ntb->epf_ntb_bar[BAR_MW1 + idx];
epf_bar = &ntb->epf->bar[barno];
epf_bar->phys_addr = addr;
@@ -1558,7 +1569,7 @@ static void vntb_epf_peer_db_work(struct work_struct *work)
ret = pci_epc_raise_irq(epf->epc, func_no, vfunc_no,
PCI_IRQ_MSI, interrupt_num);
if (ret)
- dev_err(&ntb->ntb.dev,
+ dev_err(&epf->dev,
"Failed to raise IRQ for interrupt_num %u: %d\n",
interrupt_num, ret);
}
@@ -1676,13 +1687,18 @@ static const struct ntb_dev_ops vntb_epf_ops = {
static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
- int ret;
struct epf_ntb *ndev = (struct epf_ntb *)pdev->sysdata;
struct device *dev = &pdev->dev;
+ struct ntb_dev *ntb;
+ int ret;
- ndev->ntb.pdev = pdev;
- ndev->ntb.topo = NTB_TOPO_NONE;
- ndev->ntb.ops = &vntb_epf_ops;
+ ntb = devm_kzalloc(dev, sizeof(*ntb), GFP_KERNEL);
+ if (!ntb)
+ return -ENOMEM;
+
+ ntb->pdev = pdev;
+ ntb->topo = NTB_TOPO_NONE;
+ ntb->ops = &vntb_epf_ops;
ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
if (ret) {
@@ -1690,16 +1706,41 @@ static int pci_vntb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return ret;
}
- ret = ntb_register_device(&ndev->ntb);
+ ret = ntb_register_device(ntb);
if (ret) {
dev_err(dev, "Failed to register NTB device\n");
return ret;
}
+ /* Publish after ntb_register_device() succeeds */
+ smp_store_release(&ndev->ntb, ntb);
+ queue_delayed_work(kpcintb_workqueue, &ndev->cmd_handler, 0);
+
dev_dbg(dev, "PCI Virtual NTB driver loaded\n");
return 0;
}
+static void pci_vntb_remove(struct pci_dev *pdev)
+{
+ struct epf_ntb *ndev = pdev->sysdata;
+ struct ntb_dev *ntb;
+ unsigned int i;
+
+ /* Stop the work reader, then close and drain the IRQ gate */
+ cancel_delayed_work_sync(&ndev->cmd_handler);
+
+ ntb = xchg(&ndev->ntb, NULL);
+
+ if (ndev->msi_doorbell)
+ for (i = 0; i < ndev->db_count; i++) {
+ if (epf_ntb_db_irq_is_duplicated(ndev->epf, i))
+ continue;
+ synchronize_irq(ndev->epf->db_msg[i].virq);
+ }
+
+ ntb_unregister_device(ntb);
+}
+
static struct pci_device_id pci_vntb_table[] = {
{
PCI_DEVICE(0xffff, 0xffff),
@@ -1711,6 +1752,7 @@ static struct pci_driver vntb_pci_driver = {
.name = "pci-vntb",
.id_table = pci_vntb_table,
.probe = pci_vntb_probe,
+ .remove = pci_vntb_remove,
};
/* ============ PCIe EPF Driver Bind ====================*/
@@ -1791,12 +1833,25 @@ static int epf_ntb_bind(struct pci_epf *epf)
*/
static void epf_ntb_unbind(struct pci_epf *epf)
{
+ struct pci_host_bridge *bridge;
struct epf_ntb *ntb = epf_get_drvdata(epf);
+ pci_unregister_driver(&vntb_pci_driver);
+
+ if (ntb->vpci_bus) {
+ bridge = to_pci_host_bridge(ntb->vpci_bus->bridge);
+
+ pci_lock_rescan_remove();
+ pci_stop_root_bus(ntb->vpci_bus);
+ pci_remove_root_bus(ntb->vpci_bus);
+ ntb->vpci_bus = NULL;
+ pci_unlock_rescan_remove();
+
+ pci_free_host_bridge(bridge);
+ }
+
epf_ntb_epc_cleanup(ntb);
epf_ntb_config_spad_bar_free(ntb);
-
- pci_unregister_driver(&vntb_pci_driver);
}
// EPF driver probe
diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c
index 2cafd3b26f34..5ce32dc7e178 100644
--- a/drivers/pci/hotplug/pciehp_core.c
+++ b/drivers/pci/hotplug/pciehp_core.c
@@ -38,7 +38,7 @@ int pciehp_poll_time;
* not really modular, but the easiest way to keep compat with existing
* bootargs behaviour is to continue using module_param here.
*/
-module_param(pciehp_poll_mode, bool, 0644);
+module_param(pciehp_poll_mode, bool, 0444);
module_param(pciehp_poll_time, int, 0644);
MODULE_PARM_DESC(pciehp_poll_mode, "Using polling mechanism for hot-plug events or not");
MODULE_PARM_DESC(pciehp_poll_time, "Polling mechanism frequency, in seconds");
diff --git a/drivers/pci/pci-label.c b/drivers/pci/pci-label.c
index 0c6446519640..c224619b9a26 100644
--- a/drivers/pci/pci-label.c
+++ b/drivers/pci/pci-label.c
@@ -144,7 +144,7 @@ static int dsm_label_utf16s_to_utf8s(union acpi_object *obj, char *buf)
int len;
len = utf16s_to_utf8s((const wchar_t *)obj->buffer.pointer,
- obj->buffer.length,
+ obj->buffer.length / sizeof(wchar_t),
UTF16_LITTLE_ENDIAN,
buf, PAGE_SIZE - 1);
buf[len++] = '\n';
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index d79d2ee4ac69..f85256413cf3 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3262,7 +3262,15 @@ void pci_pm_init(struct pci_dev *dev)
}
poweron:
- pci_pm_power_up_and_verify_state(dev);
+ pci_update_current_state(dev, PCI_UNKNOWN);
+ if (dev->current_state == PCI_D0) {
+ /* No need for full power-up, just sync-up platform state */
+ platform_pci_set_power_state(dev, PCI_D0);
+ } else {
+ /* Power up the device and restore BARs if necessary */
+ pci_set_full_power_state(dev, false);
+ }
+ pci_update_current_state(dev, PCI_D0);
pm_runtime_forbid(&dev->dev);
/*
@@ -6000,18 +6008,31 @@ static enum pci_bus_speed to_pcie_link_speed(u16 lnksta)
return pcie_link_speed[FIELD_GET(PCI_EXP_LNKSTA_CLS, lnksta)];
}
-int pcie_link_speed_mbps(struct pci_dev *pdev)
+/**
+ * pcie_link_bandwidth_mbps - get the current bandwidth of a single PCIe link
+ * @pdev: PCI device to query
+ *
+ * The bandwidth includes all negotiated lanes and does not account for
+ * encoding overhead. A negotiated width of zero gives zero bandwidth.
+ *
+ * Return: Bandwidth in Mb/s, or a negative errno on failure.
+ */
+int pcie_link_bandwidth_mbps(struct pci_dev *pdev)
{
u16 lnksta;
- int err;
+ int speed, err;
err = pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta);
if (err)
- return err;
+ return pcibios_err_to_errno(err);
+
+ speed = pcie_dev_speed_mbps(to_pcie_link_speed(lnksta));
+ if (speed < 0)
+ return speed;
- return pcie_dev_speed_mbps(to_pcie_link_speed(lnksta));
+ return speed * FIELD_GET(PCI_EXP_LNKSTA_NLW, lnksta);
}
-EXPORT_SYMBOL(pcie_link_speed_mbps);
+EXPORT_SYMBOL(pcie_link_bandwidth_mbps);
/**
* pcie_bandwidth_available - determine minimum link settings of a PCIe
diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
index 732f75a31760..be332d3f106c 100644
--- a/drivers/pci/pcie/aer.c
+++ b/drivers/pci/pcie/aer.c
@@ -1426,11 +1426,14 @@ void aer_recover_queue(int domain, unsigned int bus, unsigned int devfn,
};
if (kfifo_in_spinlocked(&aer_recover_ring, &entry, 1,
- &aer_recover_ring_lock))
+ &aer_recover_ring_lock)) {
schedule_work(&aer_recover_work);
- else
+ } else {
pr_err("buffer overflow in recovery for %04x:%02x:%02x.%x\n",
domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
+ ghes_estatus_pool_region_free((unsigned long)aer_regs,
+ sizeof(struct aer_capability_regs));
+ }
}
EXPORT_SYMBOL_GPL(aer_recover_queue);
#endif
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 27008e2ea5af..52a79b033b06 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1651,6 +1651,8 @@ void set_pcie_port_type(struct pci_dev *pdev)
pdev->aspm_l1_support = 1;
#endif
+ pdev->supported_speeds = pcie_get_supported_speeds(pdev);
+
parent = pci_upstream_bridge(pdev);
if (!parent)
return;
@@ -2074,9 +2076,6 @@ int pci_setup_device(struct pci_dev *dev)
set_pcie_untrusted(dev);
- if (pci_is_pcie(dev))
- dev->supported_speeds = pcie_get_supported_speeds(dev);
-
/* "Unknown power state" */
dev->current_state = PCI_UNKNOWN;
diff --git a/drivers/pci/pwrctrl/core.c b/drivers/pci/pwrctrl/core.c
index 10d817aefb09..c14df117ece3 100644
--- a/drivers/pci/pwrctrl/core.c
+++ b/drivers/pci/pwrctrl/core.c
@@ -388,6 +388,9 @@ static void pci_pwrctrl_destroy_device(struct device_node *np)
for_each_available_child_of_node_scoped(np, child)
pci_pwrctrl_destroy_device(child);
+ if (!pci_pwrctrl_is_required(np))
+ return;
+
pdev = of_find_device_by_node(np);
if (!pdev)
return;
diff --git a/drivers/pci/pwrctrl/generic.c b/drivers/pci/pwrctrl/generic.c
index d56171e0ee24..4b6cbbb52248 100644
--- a/drivers/pci/pwrctrl/generic.c
+++ b/drivers/pci/pwrctrl/generic.c
@@ -63,6 +63,7 @@ static void devm_slot_pwrctrl_release(void *data)
struct slot_pwrctrl *slot = data;
regulator_bulk_free(slot->num_supplies, slot->supplies);
+ kfree(slot->supplies);
}
static int slot_pwrctrl_probe(struct platform_device *pdev)
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 1aa87a3ef9ed..d4fcc336bc2a 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -108,7 +108,11 @@ int pcie_failed_link_retrain(struct pci_dev *dev)
pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta);
pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &oldlnkctl2);
- if (!(lnksta & PCI_EXP_LNKSTA_DLLLA) && pcie_lbms_seen(dev, lnksta)) {
+ if (lnksta & PCI_EXP_LNKSTA_DLLLA) {
+ ;
+ } else if (PCIE_LNKCTL2_TLS2SPEED(oldlnkctl2) == PCIE_SPEED_2_5GT) {
+ return ret;
+ } else if (pcie_lbms_seen(dev, lnksta)) {
pci_info(dev, "broken device, retraining non-functional downstream link at 2.5GT/s\n");
ret = pcie_set_target_speed(dev, PCIE_SPEED_2_5GT, false);
if (ret)
@@ -1941,6 +1945,8 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x4388, quirk_no_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x4389, quirk_no_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x438a, quirk_no_msi);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, 0x438b, quirk_no_msi);
+/* ULi M1575 EHCI (10b9:5239): MSI stalls the async schedule */
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AL, 0x5239, quirk_no_msi);
static void quirk_pcie_mch(struct pci_dev *pdev)
{
@@ -2381,6 +2387,24 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PLX, PCI_DEVICE_ID_PLX_9050,
DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2000, quirk_plx_pci9050);
DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2600, quirk_plx_pci9050);
+/*
+ * An idle downstream port of a PLX/Broadcom PEX 8532/8533/8548 switch
+ * autosuspends into D3hot, and that takes the whole switch down with
+ * it -- any device behind another port of the same switch then sees a
+ * fatal, non-recoverable bus fault on its next access. Keep these
+ * ports out of D3 entirely.
+ */
+static void quirk_plx_pex_no_d3(struct pci_dev *dev)
+{
+ if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
+ dev->dev_flags |= PCI_DEV_FLAGS_NO_D3;
+ pci_info(dev, "avoiding D3 on this downstream port\n");
+ }
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8532, quirk_plx_pex_no_d3);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8533, quirk_plx_pex_no_d3);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8548, quirk_plx_pex_no_d3);
+
static void quirk_netmos(struct pci_dev *dev)
{
unsigned int num_parallel = (dev->subsystem_device & 0xf0) >> 4;
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index cffc32d66032..5c6f7796d204 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -599,11 +599,16 @@ static pci_ers_result_t pcifront_common_process(int cmd,
case XEN_PCI_OP_aer_detected:
return pdrv->err_handler->error_detected(pcidev, state);
case XEN_PCI_OP_aer_mmio:
- return pdrv->err_handler->mmio_enabled(pcidev);
+ if (pdrv->err_handler->mmio_enabled)
+ return pdrv->err_handler->mmio_enabled(pcidev);
+ return PCI_ERS_RESULT_RECOVERED;
case XEN_PCI_OP_aer_slotreset:
- return pdrv->err_handler->slot_reset(pcidev);
+ if (pdrv->err_handler->slot_reset)
+ return pdrv->err_handler->slot_reset(pcidev);
+ return PCI_ERS_RESULT_RECOVERED;
case XEN_PCI_OP_aer_resume:
- pdrv->err_handler->resume(pcidev);
+ if (pdrv->err_handler->resume)
+ pdrv->err_handler->resume(pcidev);
return PCI_ERS_RESULT_NONE;
default:
dev_err(&pdev->xdev->dev,
diff --git a/drivers/peci/controller/peci-aspeed.c b/drivers/peci/controller/peci-aspeed.c
index a0c99ecf7f38..5d7f4f8fda73 100644
--- a/drivers/peci/controller/peci-aspeed.c
+++ b/drivers/peci/controller/peci-aspeed.c
@@ -529,14 +529,14 @@ static int aspeed_peci_probe(struct platform_device *pdev)
if (priv->irq < 0)
return priv->irq;
+ init_completion(&priv->xfer_complete);
+ spin_lock_init(&priv->lock);
+
ret = devm_request_irq(&pdev->dev, priv->irq, aspeed_peci_irq_handler,
0, "peci-aspeed", priv);
if (ret)
return ret;
- init_completion(&priv->xfer_complete);
- spin_lock_init(&priv->lock);
-
priv->rst = devm_reset_control_get(&pdev->dev, NULL);
if (IS_ERR(priv->rst))
return dev_err_probe(priv->dev, PTR_ERR(priv->rst),
diff --git a/drivers/peci/controller/peci-npcm.c b/drivers/peci/controller/peci-npcm.c
index 931868991241..3cc4e959aff3 100644
--- a/drivers/peci/controller/peci-npcm.c
+++ b/drivers/peci/controller/peci-npcm.c
@@ -253,14 +253,14 @@ static int npcm_peci_probe(struct platform_device *pdev)
if (priv->irq < 0)
return priv->irq;
+ init_completion(&priv->xfer_complete);
+ spin_lock_init(&priv->lock);
+
ret = devm_request_irq(&pdev->dev, priv->irq, npcm_peci_irq_handler,
0, "peci-npcm-irq", priv);
if (ret)
return ret;
- init_completion(&priv->xfer_complete);
- spin_lock_init(&priv->lock);
-
ret = npcm_peci_init_ctrl(priv);
if (ret)
return ret;
diff --git a/drivers/perf/xgene_pmu.c b/drivers/perf/xgene_pmu.c
index ab3623fab914..0aa2b605fd53 100644
--- a/drivers/perf/xgene_pmu.c
+++ b/drivers/perf/xgene_pmu.c
@@ -1873,6 +1873,8 @@ static int xgene_pmu_probe(struct platform_device *pdev)
if (irq < 0)
return -EINVAL;
+ raw_spin_lock_init(&xgene_pmu->lock);
+
rc = devm_request_irq(&pdev->dev, irq, xgene_pmu_isr,
IRQF_NOBALANCING | IRQF_NO_THREAD,
dev_name(&pdev->dev), xgene_pmu);
@@ -1881,8 +1883,6 @@ static int xgene_pmu_probe(struct platform_device *pdev)
xgene_pmu->irq = irq;
- raw_spin_lock_init(&xgene_pmu->lock);
-
/* Check for active MCBs and MCUs */
rc = xgene_pmu_probe_active_mcb_mcu_l3c(xgene_pmu, pdev);
if (rc) {
diff --git a/drivers/phy/apple/atc.c b/drivers/phy/apple/atc.c
index 4156fabad742..a3162f25e6cd 100644
--- a/drivers/phy/apple/atc.c
+++ b/drivers/phy/apple/atc.c
@@ -1121,7 +1121,7 @@ static int atcphy_configure_pipehandler_dummy(struct apple_atcphy *atcphy)
static int atcphy_configure_pipehandler(struct apple_atcphy *atcphy, bool host)
{
- int ret;
+ int ret = -EINVAL;
lockdep_assert_held(&atcphy->lock);
@@ -1136,8 +1136,10 @@ static int atcphy_configure_pipehandler(struct apple_atcphy *atcphy, bool host)
ret = atcphy_configure_pipehandler_dummy(atcphy);
atcphy->pipehandler_up = false;
break;
- default:
- ret = -EINVAL;
+ case ATCPHY_PIPEHANDLER_STATE_DUMMY:
+ ret = atcphy_configure_pipehandler_dummy(atcphy);
+ atcphy->pipehandler_up = false;
+ break;
}
return ret;
diff --git a/drivers/pinctrl/renesas/pinctrl-rzt2h.c b/drivers/pinctrl/renesas/pinctrl-rzt2h.c
index eb70aee39f1a..5cfce6d68cd9 100644
--- a/drivers/pinctrl/renesas/pinctrl-rzt2h.c
+++ b/drivers/pinctrl/renesas/pinctrl-rzt2h.c
@@ -96,6 +96,7 @@ struct rzt2h_pinctrl {
struct gpio_chip gpio_chip;
struct pinctrl_gpio_range gpio_range;
DECLARE_BITMAP(used_irqs, RZT2H_INTERRUPTS_NUM);
+ u8 saved_pm[RZT2H_INTERRUPTS_NUM];
raw_spinlock_t lock; /* lock read/write registers */
struct mutex mutex; /* serialize adding groups and functions */
bool safety_port_enabled;
@@ -170,6 +171,36 @@ static int rzt2h_validate_pin(struct rzt2h_pinctrl *pctrl, unsigned int offset)
return (pincfg & BIT(pin)) ? 0 : -EINVAL;
}
+static bool rzt2h_pin_mode_is_peripheral(struct rzt2h_pinctrl *pctrl, u8 port, u8 bit)
+{
+ return rzt2h_pinctrl_readb(pctrl, port, PMC(port)) & BIT(bit);
+}
+
+static bool rzt2h_pin_read_input(struct rzt2h_pinctrl *pctrl, u8 port, u8 bit)
+{
+ return rzt2h_pinctrl_readb(pctrl, port, PIN(port)) & BIT(bit);
+}
+
+static u8 rzt2h_pin_read_pm(struct rzt2h_pinctrl *pctrl, u8 port, u8 pin)
+{
+ u16 reg = rzt2h_pinctrl_readw(pctrl, port, PM(port));
+
+ return field_get(PM_PIN_MASK(pin), reg);
+}
+
+static void rzt2h_pin_write_pm(struct rzt2h_pinctrl *pctrl, u8 port, u8 pin,
+ unsigned int pm)
+{
+ u16 reg;
+
+ guard(raw_spinlock_irqsave)(&pctrl->lock);
+
+ reg = rzt2h_pinctrl_readw(pctrl, port, PM(port));
+ reg &= ~PM_PIN_MASK(pin);
+ reg |= pm << (pin * 2);
+ rzt2h_pinctrl_writew(pctrl, port, reg, PM(port));
+}
+
static void rzt2h_pinctrl_set_gpio_en(struct rzt2h_pinctrl *pctrl,
u8 port, u8 pin, bool en)
{
@@ -827,6 +858,9 @@ static int rzt2h_gpio_get(struct gpio_chip *chip, unsigned int offset)
u8 bit = RZT2H_PIN_ID_TO_PIN(offset);
u16 reg;
+ if (rzt2h_pin_mode_is_peripheral(pctrl, port, bit))
+ return rzt2h_pin_read_input(pctrl, port, bit);
+
reg = rzt2h_pinctrl_readw(pctrl, port, PM(port));
reg = (reg >> (bit * 2)) & PM_MASK;
if (reg & PM_INPUT)
@@ -1000,6 +1034,66 @@ static int rzt2h_gpio_irq_set_wake(struct irq_data *d, unsigned int on)
return 0;
}
+static int rzt2h_gpio_irq_request_resources(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct rzt2h_pinctrl *pctrl = gpiochip_get_data(gc);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
+ u8 port = RZT2H_PIN_ID_TO_PORT(hwirq);
+ u8 pin = RZT2H_PIN_ID_TO_PIN(hwirq);
+ u8 parent_irq, irq_idx;
+ int ret;
+
+ parent_irq = rzt2h_gpio_irq_map[hwirq];
+ if (parent_irq < RZT2H_INTERRUPTS_START)
+ return -EINVAL;
+
+ irq_idx = parent_irq - RZT2H_INTERRUPTS_START;
+ if (test_and_set_bit(irq_idx, pctrl->used_irqs))
+ return -EBUSY;
+
+ /*
+ * rzt2h_pinctrl_set_pfc_mode() sets PM to Hi-Z before switching to the
+ * interrupt function, losing the previous PM value.
+ * Save it so it can be restored when the IRQ is freed.
+ */
+ pctrl->saved_pm[irq_idx] = rzt2h_pin_read_pm(pctrl, port, pin);
+
+ rzt2h_pinctrl_set_pfc_mode(pctrl, port, pin, PFC_FUNC_INTERRUPT);
+
+ ret = gpiochip_lock_as_irq(gc, hwirq);
+ if (ret) {
+ clear_bit(irq_idx, pctrl->used_irqs);
+ rzt2h_pin_write_pm(pctrl, port, pin, pctrl->saved_pm[irq_idx]);
+ rzt2h_pinctrl_set_gpio_en(pctrl, port, pin, true);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void rzt2h_gpio_irq_release_resources(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct rzt2h_pinctrl *pctrl = gpiochip_get_data(gc);
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
+ u8 port = RZT2H_PIN_ID_TO_PORT(hwirq);
+ u8 pin = RZT2H_PIN_ID_TO_PIN(hwirq);
+ u8 parent_irq, irq_idx;
+
+ parent_irq = rzt2h_gpio_irq_map[hwirq];
+ if (parent_irq < RZT2H_INTERRUPTS_START)
+ return;
+
+ irq_idx = parent_irq - RZT2H_INTERRUPTS_START;
+ if (test_and_clear_bit(irq_idx, pctrl->used_irqs)) {
+ rzt2h_pin_write_pm(pctrl, port, pin, pctrl->saved_pm[irq_idx]);
+ rzt2h_pinctrl_set_gpio_en(pctrl, port, pin, true);
+ }
+
+ gpiochip_unlock_as_irq(gc, hwirq);
+}
+
static const struct irq_chip rzt2h_gpio_irqchip = {
.name = "rzt2h-gpio",
.irq_disable = rzt2h_gpio_irq_disable,
@@ -1009,9 +1103,10 @@ static const struct irq_chip rzt2h_gpio_irqchip = {
.irq_set_type = irq_chip_set_type_parent,
.irq_set_wake = rzt2h_gpio_irq_set_wake,
.irq_eoi = irq_chip_eoi_parent,
+ .irq_request_resources = rzt2h_gpio_irq_request_resources,
+ .irq_release_resources = rzt2h_gpio_irq_release_resources,
.irq_set_affinity = irq_chip_set_affinity_parent,
.flags = IRQCHIP_IMMUTABLE,
- GPIOCHIP_IRQ_RESOURCE_HELPERS,
};
static int rzt2h_gpio_child_to_parent_hwirq(struct gpio_chip *gc,
@@ -1020,43 +1115,17 @@ static int rzt2h_gpio_child_to_parent_hwirq(struct gpio_chip *gc,
unsigned int *parent,
unsigned int *parent_type)
{
- struct rzt2h_pinctrl *pctrl = gpiochip_get_data(gc);
- u8 port = RZT2H_PIN_ID_TO_PORT(child);
- u8 pin = RZT2H_PIN_ID_TO_PIN(child);
- u8 parent_irq;
+ u8 parent_irq = rzt2h_gpio_irq_map[child];
- parent_irq = rzt2h_gpio_irq_map[child];
if (parent_irq < RZT2H_INTERRUPTS_START)
return -EINVAL;
- if (test_and_set_bit(parent_irq - RZT2H_INTERRUPTS_START,
- pctrl->used_irqs))
- return -EBUSY;
-
- rzt2h_pinctrl_set_pfc_mode(pctrl, port, pin, PFC_FUNC_INTERRUPT);
-
*parent = parent_irq;
*parent_type = child_type;
return 0;
}
-static void rzt2h_gpio_irq_domain_free(struct irq_domain *domain, unsigned int virq,
- unsigned int nr_irqs)
-{
- struct irq_data *d = irq_domain_get_irq_data(domain, virq);
- struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
- struct rzt2h_pinctrl *pctrl = container_of(gc, struct rzt2h_pinctrl, gpio_chip);
- irq_hw_number_t hwirq = irqd_to_hwirq(d);
- u8 port = RZT2H_PIN_ID_TO_PORT(hwirq);
- u8 pin = RZT2H_PIN_ID_TO_PIN(hwirq);
-
- if (test_and_clear_bit(hwirq - RZT2H_INTERRUPTS_START, pctrl->used_irqs))
- rzt2h_pinctrl_set_gpio_en(pctrl, port, pin, false);
-
- irq_domain_free_irqs_common(domain, virq, nr_irqs);
-}
-
static void rzt2h_gpio_init_irq_valid_mask(struct gpio_chip *gc,
unsigned long *valid_mask,
unsigned int ngpios)
@@ -1122,7 +1191,6 @@ static int rzt2h_gpio_register(struct rzt2h_pinctrl *pctrl)
girq->parent_domain = parent_domain;
girq->child_to_parent_hwirq = rzt2h_gpio_child_to_parent_hwirq;
girq->populate_parent_alloc_arg = gpiochip_populate_parent_fwspec_twocell;
- girq->child_irq_domain_ops.free = rzt2h_gpio_irq_domain_free;
girq->init_valid_mask = rzt2h_gpio_init_irq_valid_mask;
}
diff --git a/drivers/pinctrl/stm32/pinctrl-stm32.c b/drivers/pinctrl/stm32/pinctrl-stm32.c
index 6a99708a5a23..93127da05d56 100644
--- a/drivers/pinctrl/stm32/pinctrl-stm32.c
+++ b/drivers/pinctrl/stm32/pinctrl-stm32.c
@@ -1666,8 +1666,12 @@ static int stm32_gpiolib_register_bank(struct stm32_pinctrl *pctl, struct fwnode
/* get the last defined gpio line (offset + nb of pins) */
npins = args.args[0] + args.args[2];
- while (!fwnode_property_get_reference_args(fwnode, "gpio-ranges", NULL, 3, ++i, &args))
+ fwnode_handle_put(args.fwnode);
+
+ while (!fwnode_property_get_reference_args(fwnode, "gpio-ranges", NULL, 3, ++i, &args)) {
npins = max(npins, (int)(args.args[0] + args.args[2]));
+ fwnode_handle_put(args.fwnode);
+ }
} else {
bank_nr = pctl->nbanks;
bank->gpio_chip.base = bank_nr * STM32_GPIO_PINS_PER_BANK;
diff --git a/drivers/platform/x86/amd/hfi/hfi.c b/drivers/platform/x86/amd/hfi/hfi.c
index e1c776b71f3e..6d1d3b6c28a9 100644
--- a/drivers/platform/x86/amd/hfi/hfi.c
+++ b/drivers/platform/x86/amd/hfi/hfi.c
@@ -20,6 +20,7 @@
#include <linux/module.h>
#include <linux/mailbox_client.h>
#include <linux/mutex.h>
+#include <linux/overflow.h>
#include <linux/percpu-defs.h>
#include <linux/platform_device.h>
#include <linux/smp.h>
@@ -168,6 +169,21 @@ static int amd_hfi_fill_metadata(struct amd_hfi_data *amd_hfi_data)
return -EINVAL;
}
+ /*
+ * The bitmaps enumerating the APIC IDs occupy the first n_bitmaps
+ * words of table_data. Reject a firmware-provided count that would
+ * push those reads past the end of the shared memory region.
+ */
+ if (struct_size(amd_hfi_data->shmem, table_data, amd_hfi_data->shmem->n_bitmaps) >
+ pcct_ext->length) {
+ dev_err(amd_hfi_data->dev, "invalid number of bitmaps: %u\n",
+ amd_hfi_data->shmem->n_bitmaps);
+ return -EINVAL;
+ }
+
+ /* The ranking data for each processor follows the bitmaps */
+ u32 *table = amd_hfi_data->shmem->table_data + amd_hfi_data->shmem->n_bitmaps;
+
for (unsigned int i = 0; i < amd_hfi_data->shmem->n_bitmaps; i++) {
u32 bitmap = amd_hfi_data->shmem->table_data[i];
@@ -192,10 +208,6 @@ static int amd_hfi_fill_metadata(struct amd_hfi_data *amd_hfi_data)
info = per_cpu_ptr(&amd_hfi_cpuinfo, cpu_index);
apic_index = apic_start * info->nr_class * 2;
for (unsigned int k = 0; k < info->nr_class; k++) {
- u32 *table = amd_hfi_data->shmem->table_data +
- amd_hfi_data->shmem->n_bitmaps +
- i * info->nr_class;
-
info->amd_hfi_classes[k].eff = table[apic_index + 2 * k];
info->amd_hfi_classes[k].perf = table[apic_index + 2 * k + 1];
}
diff --git a/drivers/platform/x86/amd/pmc/pmc.c b/drivers/platform/x86/amd/pmc/pmc.c
index 6792aa2c6187..f93c35efeaa0 100644
--- a/drivers/platform/x86/amd/pmc/pmc.c
+++ b/drivers/platform/x86/amd/pmc/pmc.c
@@ -14,6 +14,7 @@
#include <linux/array_size.h>
#include <linux/bitfield.h>
#include <linux/bits.h>
+#include <linux/cleanup.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/dmi.h>
@@ -648,7 +649,6 @@ static int amd_pmc_wa_irq1(struct amd_pmc_dev *pdev)
static int amd_pmc_verify_czn_rtc(struct amd_pmc_dev *pdev, u32 *arg)
{
- struct rtc_device *rtc_device;
time64_t then, now, duration;
struct rtc_wkalrm alarm;
struct rtc_time tm;
@@ -664,7 +664,8 @@ static int amd_pmc_verify_czn_rtc(struct amd_pmc_dev *pdev, u32 *arg)
if (pdev->major < 64 || (pdev->major == 64 && pdev->minor < 53))
return 0;
- rtc_device = rtc_class_open("rtc0");
+ struct rtc_device *rtc_device __free(rtc_class_close) =
+ rtc_class_open("rtc0");
if (!rtc_device)
return 0;
rc = rtc_read_next_alarm(rtc_device, &alarm);
@@ -701,19 +702,6 @@ static int amd_pmc_verify_czn_rtc(struct amd_pmc_dev *pdev, u32 *arg)
return rc;
}
-static bool amd_pmc_intermediate_wakeup_need_delay(struct amd_pmc_dev *pdev)
-{
- /*
- * Starting a new HW sleep cycle right after waking from one
- * can cause electrical problems triggering the over voltage protection.
- * That is avoided by delaying the next suspend a bit, see also
- * https://lore.kernel.org/all/20250414162446.3853194-1-superm1@kernel.org/
- */
- struct smu_metrics table;
-
- return get_metrics_table(pdev, &table) == 0 && table.s0i3_last_entry_status;
-}
-
static bool amd_pmc_want_suspend_delay(struct amd_pmc_dev *pdev)
{
/*
@@ -804,7 +792,7 @@ static void amd_pmc_s2idle_check(void)
struct amd_pmc_dev *pdev = &pmc;
int rc;
- if (amd_pmc_intermediate_wakeup_need_delay(pdev) ||
+ if (!pdev->is_first_check_after_suspend ||
amd_pmc_want_suspend_delay(pdev))
msleep(2500);
diff --git a/drivers/platform/x86/amd/pmf/core.c b/drivers/platform/x86/amd/pmf/core.c
index b4eae65e675b..5966190d1496 100644
--- a/drivers/platform/x86/amd/pmf/core.c
+++ b/drivers/platform/x86/amd/pmf/core.c
@@ -454,6 +454,7 @@ static const struct acpi_device_id amd_pmf_acpi_ids[] = {
{"AMDI0107", 0},
{"AMDI0108", 0},
{"AMDI0109", 0},
+ {"AMDI0112", 0},
{ }
};
MODULE_DEVICE_TABLE(acpi, amd_pmf_acpi_ids);
diff --git a/drivers/platform/x86/amd/pmf/spc.c b/drivers/platform/x86/amd/pmf/spc.c
index 592ba4de4c7f..efb09a7915f7 100644
--- a/drivers/platform/x86/amd/pmf/spc.c
+++ b/drivers/platform/x86/amd/pmf/spc.c
@@ -199,13 +199,11 @@ static int amd_pmf_get_battery_prop(enum power_supply_property prop)
continue;
ret = power_supply_get_property(psy, prop, &value);
- if (ret) {
- power_supply_put(psy);
- return ret;
- }
+ power_supply_put(psy);
+ return ret ? ret : value.intval;
}
- return value.intval;
+ return -ENODEV;
}
static int amd_pmf_get_battery_info(struct amd_pmf_dev *dev, struct ta_pmf_enact_table *in)
diff --git a/drivers/platform/x86/amd/pmf/sps.c b/drivers/platform/x86/amd/pmf/sps.c
index 0b70a5153f46..0918fb388fde 100644
--- a/drivers/platform/x86/amd/pmf/sps.c
+++ b/drivers/platform/x86/amd/pmf/sps.c
@@ -248,20 +248,24 @@ static int amd_pmf_update_sps_power_limits_v2(struct amd_pmf_dev *pdev, int pwr_
switch (pwr_mode) {
case POWER_MODE_PERFORMANCE:
index = config_store_v2.sps_idx.power_states[src][POWER_MODE_BEST_PERFORMANCE];
- amd_pmf_update_slider_v2(pdev, index);
break;
case POWER_MODE_BALANCED_POWER:
index = config_store_v2.sps_idx.power_states[src][POWER_MODE_BALANCED];
- amd_pmf_update_slider_v2(pdev, index);
break;
case POWER_MODE_POWER_SAVER:
index = config_store_v2.sps_idx.power_states[src][POWER_MODE_BEST_POWER_EFFICIENCY];
- amd_pmf_update_slider_v2(pdev, index);
break;
default:
return -EINVAL;
}
+ if (index >= APTS_MAX_STATES) {
+ dev_err(pdev->dev, "Invalid APTS index: %d\n", index);
+ return -EINVAL;
+ }
+
+ amd_pmf_update_slider_v2(pdev, index);
+
return 0;
}
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index daf8bd17e638..e4f8636f6462 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -16,6 +16,7 @@
#include <linux/acpi.h>
#include <linux/backlight.h>
#include <linux/bits.h>
+#include <linux/cleanup.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/dmi.h>
@@ -28,6 +29,7 @@
#include <linux/leds.h>
#include <linux/minmax.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
#include <linux/platform_data/x86/asus-wmi.h>
@@ -353,6 +355,20 @@ static void asus_wmi_show_deprecated(void)
/* WMI ************************************************************************/
+/*
+ * Concurrent evaluations of ASUS WMI methods can re-enter firmware
+ * SMI/EC mailbox handling and corrupt the mailbox.
+ */
+static DEFINE_MUTEX(asus_wmi_eval_lock);
+
+static acpi_status asus_wmi_evaluate_method_locked(u32 method_id,
+ struct acpi_buffer *input,
+ struct acpi_buffer *output)
+{
+ guard(mutex)(&asus_wmi_eval_lock);
+ return wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id, input, output);
+}
+
static int asus_wmi_evaluate_method3(u32 method_id,
u32 arg0, u32 arg1, u32 arg2, u32 *retval)
{
@@ -367,8 +383,7 @@ static int asus_wmi_evaluate_method3(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2);
@@ -419,8 +434,7 @@ static int asus_wmi_evaluate_method5(u32 method_id,
union acpi_object *obj;
u32 tmp = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x, 0x%08x, 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1, arg2, arg3, arg4);
@@ -467,8 +481,7 @@ static int asus_wmi_evaluate_method_buf(u32 method_id,
union acpi_object *obj;
int err = 0;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID, 0, method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(method_id, &input, &output);
pr_debug("%s called (0x%08x) with args: 0x%08x, 0x%08x\n",
__func__, method_id, arg0, arg1);
@@ -1119,10 +1132,10 @@ static ssize_t kbd_rgb_state_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t count)
{
- u32 flags, cmd, boot, awake, sleep, keyboard;
+ u32 flags, cmd, boot, awake, sleep, shutdown;
int err;
- if (sscanf(buf, "%d %d %d %d %d", &cmd, &boot, &awake, &sleep, &keyboard) != 5)
+ if (sscanf(buf, "%d %d %d %d %d", &cmd, &boot, &awake, &sleep, &shutdown) != 5)
return -EINVAL;
if (cmd)
@@ -1135,7 +1148,7 @@ static ssize_t kbd_rgb_state_store(struct device *dev,
flags |= BIT(3);
if (sleep)
flags |= BIT(5);
- if (keyboard)
+ if (shutdown)
flags |= BIT(7);
/* 0xbd is the required default arg0 for the method. Nothing happens otherwise */
@@ -1149,7 +1162,7 @@ static ssize_t kbd_rgb_state_store(struct device *dev,
static DEVICE_ATTR_WO(kbd_rgb_state);
static DEVICE_STRING_ATTR_RO(kbd_rgb_state_index, 0444,
- "cmd boot awake sleep keyboard");
+ "cmd boot awake sleep shutdown");
static struct attribute *kbd_rgb_state_attrs[] = {
&dev_attr_kbd_rgb_state.attr,
@@ -4498,13 +4511,20 @@ static int is_display_toggle(int code)
static int read_screenpad_backlight_power(struct asus_wmi *asus)
{
- int ret;
+ int ret, retval;
- ret = asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_SCREENPAD_POWER);
+ ret = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_SCREENPAD_POWER, &retval);
if (ret < 0)
return ret;
- /* 1 == powered */
- return ret ? BACKLIGHT_POWER_ON : BACKLIGHT_POWER_OFF;
+
+ /*
+ * The firmware reports the panel power in the low byte of the
+ * devstate as a raw EC status byte that is non-zero when the
+ * panel is powered; other models report it through
+ * ASUS_WMI_DSTS_STATUS_BIT, which lies inside the same mask.
+ */
+ return (retval & ASUS_WMI_DSTS_BRIGHTNESS_MASK) ?
+ BACKLIGHT_POWER_ON : BACKLIGHT_POWER_OFF;
}
static int read_screenpad_brightness(struct backlight_device *bd)
@@ -4532,7 +4552,8 @@ static int update_screenpad_bl_status(struct backlight_device *bd)
u32 ctrl_param = bd->props.brightness;
int err = 0;
- if (bd->props.power) {
+ switch (bd->props.power) {
+ case BACKLIGHT_POWER_ON:
err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_POWER, 1, NULL);
if (err < 0)
return err;
@@ -4540,12 +4561,17 @@ static int update_screenpad_bl_status(struct backlight_device *bd)
err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_LIGHT, ctrl_param, NULL);
if (err < 0)
return err;
- }
+ break;
- if (!bd->props.power) {
+ case BACKLIGHT_POWER_OFF:
err = asus_wmi_set_devstate(ASUS_WMI_DEVID_SCREENPAD_POWER, 0, NULL);
if (err < 0)
return err;
+ break;
+
+ default:
+ pr_warn("Invalid screenpad backlight power state: %d\n", bd->props.power);
+ return -EINVAL;
}
return err;
@@ -4562,16 +4588,17 @@ static int asus_screenpad_init(struct asus_wmi *asus)
struct backlight_device *bd;
struct backlight_properties props;
int err, power;
- int brightness = 0;
+ u32 brightness = 0;
- power = asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_SCREENPAD_POWER);
+ power = read_screenpad_backlight_power(asus);
if (power < 0)
return power;
- if (power) {
+ if (power == BACKLIGHT_POWER_ON) {
err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_SCREENPAD_LIGHT, &brightness);
if (err < 0)
return err;
+ brightness &= ASUS_WMI_DSTS_BRIGHTNESS_MASK;
}
memset(&props, 0, sizeof(struct backlight_properties));
@@ -5038,9 +5065,8 @@ static int show_call(struct seq_file *m, void *data)
union acpi_object *obj;
acpi_status status;
- status = wmi_evaluate_method(ASUS_WMI_MGMT_GUID,
- 0, asus->debug.method_id,
- &input, &output);
+ status = asus_wmi_evaluate_method_locked(asus->debug.method_id,
+ &input, &output);
if (ACPI_FAILURE(status))
return -EIO;
diff --git a/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c b/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c
index 309634c1cc20..8ffdc737af4d 100644
--- a/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c
+++ b/drivers/platform/x86/hp/hp-bioscfg/bioscfg.c
@@ -11,7 +11,7 @@
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/printk.h>
-#include <linux/string.h>
+#include <linux/string_helpers.h>
#include <linux/wmi.h>
#include "bioscfg.h"
#include "../../firmware_attributes_class.h"
@@ -53,74 +53,44 @@ int hp_get_integer_from_buffer(u8 **buffer, u32 *buffer_size, u32 *integer)
int hp_get_string_from_buffer(u8 **buffer, u32 *buffer_size, char *dst, u32 dst_size)
{
+ char utf8_buf[MAX_BUFF_SIZE];
u16 *src = (u16 *)*buffer;
+ int escaped_len;
+ u16 orig_size;
u16 src_size;
-
- u16 size;
- int i;
- int conv_dst_size;
+ int utf8_len;
if (*buffer_size < sizeof(u16))
return -EINVAL;
- src_size = *(src++);
- /* size value in u16 chars */
- size = src_size / sizeof(u16);
+ src_size = *src;
- /* Ensure there is enough space remaining to read and convert
- * the string
+ /*
+ * Ensure there is enough space remaining for the length prefix
+ * just read plus the string data it describes.
*/
- if (*buffer_size < src_size)
+ if (*buffer_size < sizeof(u16) + src_size)
return -EINVAL;
- for (i = 0; i < size; i++)
- if (src[i] == '\\' ||
- src[i] == '\r' ||
- src[i] == '\n' ||
- src[i] == '\t')
- size++;
-
- /*
- * Conversion is limited to destination string max number of
- * bytes.
- */
- conv_dst_size = size;
- if (size >= dst_size)
- conv_dst_size = dst_size - 1;
+ src++;
+ /* size value in u16 chars */
+ orig_size = src_size / sizeof(u16);
- /*
- * convert from UTF-16 unicode to ASCII
- */
- utf16s_to_utf8s(src, src_size, UTF16_HOST_ENDIAN, dst, conv_dst_size);
- dst[conv_dst_size] = 0;
-
- for (i = 0; i < conv_dst_size; i++) {
- if (*src == '\\' ||
- *src == '\r' ||
- *src == '\n' ||
- *src == '\t') {
- dst[i++] = '\\';
- if (i == conv_dst_size)
- break;
- }
+ utf8_len = utf16s_to_utf8s(src, orig_size, UTF16_HOST_ENDIAN,
+ utf8_buf, sizeof(utf8_buf));
- if (*src == '\r')
- dst[i] = 'r';
- else if (*src == '\n')
- dst[i] = 'n';
- else if (*src == '\t')
- dst[i] = 't';
- else if (*src == '"')
- dst[i] = '\'';
- else
- dst[i] = *src;
- src++;
- }
+ escaped_len = string_escape_mem(utf8_buf, utf8_len, dst, dst_size - 1,
+ ESCAPE_SPACE | ESCAPE_SPECIAL,
+ "\\\r\n\t");
+ if (escaped_len > dst_size - 1)
+ escaped_len = dst_size - 1;
+ dst[escaped_len] = '\0';
+ strreplace(dst, '"', '\'');
- *buffer = (u8 *)src;
- *buffer_size -= size * sizeof(u16);
+ *buffer += sizeof(u16) + src_size;
+ *buffer_size -= sizeof(u16) + src_size;
- return size;
+ return escaped_len;
}
int hp_get_common_data_from_buffer(u8 **buffer_ptr, u32 *buffer_size,
@@ -442,7 +412,7 @@ int hp_convert_hexstr_to_str(const char *input, u32 input_len, char **str, int *
*len = 0;
*str = NULL;
- new_str = kmalloc(input_len, GFP_KERNEL);
+ new_str = kmalloc(2 * DIV_ROUND_UP(input_len, 5) + 1, GFP_KERNEL);
if (!new_str)
return -ENOMEM;
diff --git a/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c b/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
index a9e178637416..f0551b455379 100644
--- a/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
+++ b/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
@@ -60,7 +60,8 @@ int hp_get_password_instance_for_type(const char *name)
return -EINVAL;
}
-static int validate_password_input(int instance_id, const char *buf)
+static int validate_password_input(int instance_id, const char *buf,
+ bool is_current)
{
int length;
struct password_data *password_data = &bioscfg_drv.password_data[instance_id];
@@ -69,13 +70,18 @@ static int validate_password_input(int instance_id, const char *buf)
if (length > 0 && buf[length - 1] == '\n')
length--;
+ /* An empty current_password clears the session password */
+ if (is_current && !length)
+ return 0;
+
if (length > MAX_PASSWD_SIZE)
- return INVALID_BIOS_AUTH;
+ return -E2BIG;
if (password_data->min_password_length > length ||
password_data->max_password_length < length)
- return INVALID_BIOS_AUTH;
- return SUCCESS;
+ return -E2BIG;
+
+ return 0;
}
ATTRIBUTE_N_PROPERTY_SHOW(is_enabled, password);
@@ -96,7 +102,7 @@ static int store_password_instance(struct kobject *kobj, const char *buf,
id = get_password_instance_id(kobj);
if (id >= 0)
- ret = validate_password_input(id, buf_cp);
+ ret = validate_password_input(id, buf_cp, is_current);
}
if (!ret) {
diff --git a/drivers/platform/x86/hp/hp-bioscfg/spmobj-attributes.c b/drivers/platform/x86/hp/hp-bioscfg/spmobj-attributes.c
index 4d94e48c1a4c..040964ed50ce 100644
--- a/drivers/platform/x86/hp/hp-bioscfg/spmobj-attributes.c
+++ b/drivers/platform/x86/hp/hp-bioscfg/spmobj-attributes.c
@@ -47,9 +47,6 @@ size_t hp_calculate_security_buffer(const char *authentication)
return sizeof(u16) * 2;
authlen = strlen(authentication);
- if (!authlen)
- return sizeof(u16) * 2;
-
size = sizeof(u16) + authlen * sizeof(u16);
if (!strstarts(authentication, BEAM_PREFIX))
size += strlen(UTF_PREFIX) * sizeof(u16);
@@ -310,7 +307,7 @@ static ssize_t auth_token_store(struct kobject *kobj,
length--;
/* allocate space and copy current auth token */
- bioscfg_drv.spm_data.auth_token = kmemdup(buf, length, GFP_KERNEL);
+ bioscfg_drv.spm_data.auth_token = kstrndup(buf, length, GFP_KERNEL);
if (!bioscfg_drv.spm_data.auth_token) {
ret = -ENOMEM;
goto exit_token;
diff --git a/drivers/platform/x86/hp/hp-wmi.c b/drivers/platform/x86/hp/hp-wmi.c
index 615b4cf6fc45..8a0fe50ef0c2 100644
--- a/drivers/platform/x86/hp/hp-wmi.c
+++ b/drivers/platform/x86/hp/hp-wmi.c
@@ -691,8 +691,9 @@ static int hp_wmi_perform_query(int query, enum hp_wmi_command command,
goto out_free;
}
- if (obj->type != ACPI_TYPE_BUFFER) {
- pr_warn("query 0x%x returned an invalid object 0x%x\n", query, ret);
+ if (obj->type != ACPI_TYPE_BUFFER ||
+ obj->buffer.length < sizeof(*bios_return)) {
+ pr_warn("query 0x%x returned wrong type or too small buffer\n", query);
ret = -EINVAL;
goto out_free;
}
@@ -711,7 +712,7 @@ static int hp_wmi_perform_query(int query, enum hp_wmi_command command,
if (!outsize)
goto out_free;
- actual_outsize = min(outsize, (int)(obj->buffer.length - sizeof(*bios_return)));
+ actual_outsize = min_t(u32, outsize, obj->buffer.length - sizeof(*bios_return));
memcpy(buffer, obj->buffer.pointer + sizeof(*bios_return), actual_outsize);
memset(buffer + actual_outsize, 0, outsize - actual_outsize);
diff --git a/drivers/platform/x86/intel/int3472/discrete.c b/drivers/platform/x86/intel/int3472/discrete.c
index 6c729fcfce5d..6bc0ab3e8d1f 100644
--- a/drivers/platform/x86/intel/int3472/discrete.c
+++ b/drivers/platform/x86/intel/int3472/discrete.c
@@ -15,6 +15,7 @@
#include <linux/platform_data/x86/int3472.h>
#include <linux/platform_device.h>
#include <linux/string_choices.h>
+#include <linux/time64.h>
#include <linux/uuid.h>
/*
@@ -178,13 +179,6 @@ static const struct int3472_gpio_map int3472_gpio_map[] = {
.type_to = INT3472_GPIO_TYPE_RESET,
.con_id = "enable",
},
- { /* ov08x40's handshake pin needs a 45 ms delay on some HP laptops */
- .hids = (const char * const[]) { "OVTI08F4", NULL },
- .type_from = INT3472_GPIO_TYPE_HANDSHAKE,
- .type_to = INT3472_GPIO_TYPE_HANDSHAKE,
- .con_id = "dvdd",
- .enable_time_us = 45 * USEC_PER_MSEC,
- },
{ /* Sensors which expect "vana" as con_id for power enable */
.hids = power_enable_hids_vana,
.type_from = INT3472_GPIO_TYPE_POWER_ENABLE,
@@ -273,8 +267,8 @@ static void int3472_get_con_id_and_polarity(struct int3472_discrete_device *int3
case INT3472_GPIO_TYPE_HANDSHAKE:
*con_id = "dvdd";
*gpio_flags = GPIO_ACTIVE_HIGH;
- /* Setups using a handshake pin need 25 ms enable delay */
- *enable_time_us = 25 * USEC_PER_MSEC;
+ /* Powering up the sensor through the handshake pin takes up to 200 ms */
+ *enable_time_us = 200 * USEC_PER_MSEC;
break;
default:
*con_id = "unknown";
@@ -329,7 +323,6 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
unsigned long gpio_flags;
union acpi_object *obj;
struct gpio_desc *gpio;
- const char *err_msg;
const char *con_id;
int ret;
@@ -375,7 +368,8 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
case INT3472_GPIO_TYPE_HOTPLUG_DETECT:
ret = skl_int3472_map_gpio_to_sensor(int3472, agpio, con_id, gpio_flags);
if (ret)
- err_msg = "Failed to map GPIO pin to sensor\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to map GPIO pin to sensor\n");
break;
case INT3472_GPIO_TYPE_CLK_ENABLE:
@@ -387,7 +381,7 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
gpio = skl_int3472_gpiod_get_from_temp_lookup(int3472, agpio, con_id, gpio_flags);
if (IS_ERR(gpio)) {
ret = PTR_ERR(gpio);
- err_msg = "Failed to get GPIO\n";
+ dev_err_probe(int3472->dev, ret, "Failed to get GPIO\n");
break;
}
@@ -395,14 +389,16 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
case INT3472_GPIO_TYPE_CLK_ENABLE:
ret = skl_int3472_register_gpio_clock(int3472, gpio);
if (ret)
- err_msg = "Failed to register clock\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to register clock\n");
break;
case INT3472_GPIO_TYPE_PRIVACY_LED:
case INT3472_GPIO_TYPE_STROBE:
ret = skl_int3472_register_led(int3472, gpio, con_id);
if (ret)
- err_msg = "Failed to register LED\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to register LED\n");
break;
case INT3472_GPIO_TYPE_POWER_ENABLE:
@@ -413,7 +409,8 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
ret = skl_int3472_register_regulator(int3472, gpio, enable_time_us,
con_id, second_sensor);
if (ret)
- err_msg = "Failed to register regulator\n";
+ dev_err_probe(int3472->dev, ret,
+ "Failed to register regulator\n");
break;
default: /* Never reached */
@@ -436,11 +433,11 @@ static int skl_int3472_handle_gpio_resources(struct acpi_resource *ares,
int3472->ngpios++;
ACPI_FREE(obj);
- if (ret < 0)
- return dev_err_probe(int3472->dev, ret, err_msg);
-
- /* Tell acpi_dev_get_resources() to not make a copy of the resource */
- return 1;
+ /*
+ * Either return an error or tell acpi_dev_get_resources() to not make a
+ * copy of the resource.
+ */
+ return ret < 0 ? ret : 1;
}
int int3472_discrete_parse_crs(struct int3472_discrete_device *int3472)
diff --git a/drivers/platform/x86/intel/pmt/class.c b/drivers/platform/x86/intel/pmt/class.c
index d0ab8e33c62a..c25892a51180 100644
--- a/drivers/platform/x86/intel/pmt/class.c
+++ b/drivers/platform/x86/intel/pmt/class.c
@@ -417,7 +417,14 @@ static int intel_pmt_dev_register(struct intel_pmt_entry *entry,
sysfs_bin_attr_init(&entry->pmt_bin_attr);
entry->pmt_bin_attr.attr.name = ns->name;
entry->pmt_bin_attr.attr.mode = 0440;
- entry->pmt_bin_attr.mmap = intel_pmt_mmap;
+
+ /*
+ * The read_telem callback controls access to the memory area. Block
+ * usage if the read_telem callback is present.
+ */
+ if (!(entry->cb && entry->cb->read_telem))
+ entry->pmt_bin_attr.mmap = intel_pmt_mmap;
+
entry->pmt_bin_attr.read = intel_pmt_read;
entry->pmt_bin_attr.size = entry->size;
diff --git a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency.c b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency.c
index 667f2c8b9594..c44e7e38cae2 100644
--- a/drivers/platform/x86/intel/uncore-frequency/uncore-frequency.c
+++ b/drivers/platform/x86/intel/uncore-frequency/uncore-frequency.c
@@ -211,7 +211,7 @@ static int uncore_pm_notify(struct notifier_block *nb, unsigned long mode,
struct uncore_data *data = &uncore_instances[i];
if (!data || !data->valid || !data->stored_uncore_data)
- return 0;
+ continue;
wrmsrq_on_cpu(data->control_cpu, MSR_UNCORE_RATIO_LIMIT,
data->stored_uncore_data);
diff --git a/drivers/platform/x86/intel/vsec.c b/drivers/platform/x86/intel/vsec.c
index 5ab2215fdd7f..7d51f27f09bc 100644
--- a/drivers/platform/x86/intel/vsec.c
+++ b/drivers/platform/x86/intel/vsec.c
@@ -641,7 +641,7 @@ static bool intel_vsec_get_features(struct pci_dev *pdev,
found = true;
if (info && (info->quirks & VSEC_QUIRK_NO_DVSEC) &&
- intel_vsec_walk_header(&pdev->dev, info))
+ !intel_vsec_walk_header(&pdev->dev, info))
found = true;
return found;
diff --git a/drivers/platform/x86/lenovo/wmi-capdata.c b/drivers/platform/x86/lenovo/wmi-capdata.c
index d5e961566136..58787da959b4 100644
--- a/drivers/platform/x86/lenovo/wmi-capdata.c
+++ b/drivers/platform/x86/lenovo/wmi-capdata.c
@@ -774,13 +774,11 @@ static int lwmi_cd_setup(struct lwmi_cd_priv *priv, enum lwmi_cd_type type)
/**
* lwmi_cd01_notifier_call() - Call method for cd01 notifier.
- * block call chain.
- * @nb: The notifier_block registered to lenovo-wmi-events driver.
+ * @nb: The cd01 ACPI notifier_block.
* @action: Unused.
* @data: The ACPI event.
*
- * For LWMI_EVENT_THERMAL_MODE, set current_mode and notify platform_profile
- * of a change.
+ * Refresh cached capdata on AC status change.
*
* Return: notifier_block status.
*/
@@ -799,8 +797,16 @@ static int lwmi_cd01_notifier_call(struct notifier_block *nb, unsigned long acti
switch (event->type) {
case ACPI_AC_NOTIFY_STATUS:
ret = lwmi_cd_cache(priv);
- if (ret)
- return NOTIFY_BAD;
+ if (ret) {
+ dev_warn(&priv->wdev->dev,
+ "failed to refresh capdata on AC change: %d\n", ret);
+
+ /*
+ * Our error is non-fatal for others, and we still want
+ * the other event consumers to process it.
+ */
+ return NOTIFY_DONE;
+ }
return NOTIFY_OK;
default:
diff --git a/drivers/platform/x86/oxpec.c b/drivers/platform/x86/oxpec.c
index 34bb17fca148..38c272995437 100644
--- a/drivers/platform/x86/oxpec.c
+++ b/drivers/platform/x86/oxpec.c
@@ -50,6 +50,7 @@ enum oxp_board {
oxp_x1,
oxp_g1_i,
oxp_g1_a,
+ oxp_apex,
};
static enum oxp_board board;
@@ -93,11 +94,14 @@ static struct device *oxp_dev;
#define OXP_X1_CHARGE_LIMIT_REG 0xA3 /* X1 charge limit (%) */
#define OXP_X1_CHARGE_INHIBIT_REG 0xA4 /* X1 bypass charging */
-#define OXP_X1_CHARGE_INHIBIT_MASK_AWAKE 0x01
+#define OXP_APEX_CHARGE_LIMIT_REG 0xE5 /* Apex charge limit (%) */
+#define OXP_APEX_CHARGE_INHIBIT_REG 0xE6 /* Apex bypass charging */
+
+#define OXP_CHARGE_INHIBIT_MASK_AWAKE 0x01
/* X1 Mask is 0x0A, F1Pro is 0x02 but the extra bit on the X1 does nothing. */
-#define OXP_X1_CHARGE_INHIBIT_MASK_OFF 0x02
-#define OXP_X1_CHARGE_INHIBIT_MASK_ALWAYS (OXP_X1_CHARGE_INHIBIT_MASK_AWAKE | \
- OXP_X1_CHARGE_INHIBIT_MASK_OFF)
+#define OXP_CHARGE_INHIBIT_MASK_OFF 0x02
+#define OXP_CHARGE_INHIBIT_MASK_ALWAYS (OXP_CHARGE_INHIBIT_MASK_AWAKE | \
+ OXP_CHARGE_INHIBIT_MASK_OFF)
static const struct dmi_system_id dmi_table[] = {
{
@@ -154,7 +158,7 @@ static const struct dmi_system_id dmi_table[] = {
DMI_MATCH(DMI_BOARD_VENDOR, "ONE-NETBOOK"),
DMI_EXACT_MATCH(DMI_BOARD_NAME, "ONEXPLAYER APEX"),
},
- .driver_data = (void *)oxp_fly,
+ .driver_data = (void *)oxp_apex,
},
{
.matches = {
@@ -352,6 +356,7 @@ static umode_t tt_toggle_is_visible(struct kobject *kobj,
case oxp_x1:
case oxp_g1_i:
case oxp_g1_a:
+ case oxp_apex:
return attr->mode;
default:
break;
@@ -381,6 +386,7 @@ static ssize_t tt_toggle_store(struct device *dev,
case oxp_fly:
case oxp_mini_amd_pro:
case oxp_g1_a:
+ case oxp_apex:
reg = OXP_TURBO_SWITCH_REG;
mask = OXP_TURBO_TAKE_VAL;
break;
@@ -427,6 +433,7 @@ static ssize_t tt_toggle_show(struct device *dev,
case oxp_fly:
case oxp_mini_amd_pro:
case oxp_g1_a:
+ case oxp_apex:
reg = OXP_TURBO_SWITCH_REG;
mask = OXP_TURBO_TAKE_VAL;
break;
@@ -524,6 +531,7 @@ static bool oxp_psy_ext_supported(void)
case oxp_g1_i:
case oxp_g1_a:
case oxp_fly:
+ case oxp_apex:
return true;
default:
break;
@@ -537,12 +545,24 @@ static int oxp_psy_ext_get_prop(struct power_supply *psy,
enum power_supply_property psp,
union power_supply_propval *val)
{
+ u8 limit_reg, inhibit_reg;
long raw_val;
int ret;
+ switch (board) {
+ case oxp_apex:
+ limit_reg = OXP_APEX_CHARGE_LIMIT_REG;
+ inhibit_reg = OXP_APEX_CHARGE_INHIBIT_REG;
+ break;
+ default:
+ limit_reg = OXP_X1_CHARGE_LIMIT_REG;
+ inhibit_reg = OXP_X1_CHARGE_INHIBIT_REG;
+ break;
+ }
+
switch (psp) {
case POWER_SUPPLY_PROP_CHARGE_CONTROL_END_THRESHOLD:
- ret = read_from_ec(OXP_X1_CHARGE_LIMIT_REG, 1, &raw_val);
+ ret = read_from_ec(limit_reg, 1, &raw_val);
if (ret)
return ret;
if (raw_val < 0 || raw_val > 100)
@@ -550,14 +570,14 @@ static int oxp_psy_ext_get_prop(struct power_supply *psy,
val->intval = raw_val;
return 0;
case POWER_SUPPLY_PROP_CHARGE_BEHAVIOUR:
- ret = read_from_ec(OXP_X1_CHARGE_INHIBIT_REG, 1, &raw_val);
+ ret = read_from_ec(inhibit_reg, 1, &raw_val);
if (ret)
return ret;
- if ((raw_val & OXP_X1_CHARGE_INHIBIT_MASK_ALWAYS) ==
- OXP_X1_CHARGE_INHIBIT_MASK_ALWAYS)
+ if ((raw_val & OXP_CHARGE_INHIBIT_MASK_ALWAYS) ==
+ OXP_CHARGE_INHIBIT_MASK_ALWAYS)
val->intval = POWER_SUPPLY_CHARGE_BEHAVIOUR_INHIBIT_CHARGE;
- else if ((raw_val & OXP_X1_CHARGE_INHIBIT_MASK_AWAKE) ==
- OXP_X1_CHARGE_INHIBIT_MASK_AWAKE)
+ else if ((raw_val & OXP_CHARGE_INHIBIT_MASK_AWAKE) ==
+ OXP_CHARGE_INHIBIT_MASK_AWAKE)
val->intval = POWER_SUPPLY_CHARGE_BEHAVIOUR_INHIBIT_CHARGE_AWAKE;
else
val->intval = POWER_SUPPLY_CHARGE_BEHAVIOUR_AUTO;
@@ -573,29 +593,41 @@ static int oxp_psy_ext_set_prop(struct power_supply *psy,
enum power_supply_property psp,
const union power_supply_propval *val)
{
+ u8 limit_reg, inhibit_reg;
long raw_val;
+ switch (board) {
+ case oxp_apex:
+ limit_reg = OXP_APEX_CHARGE_LIMIT_REG;
+ inhibit_reg = OXP_APEX_CHARGE_INHIBIT_REG;
+ break;
+ default:
+ limit_reg = OXP_X1_CHARGE_LIMIT_REG;
+ inhibit_reg = OXP_X1_CHARGE_INHIBIT_REG;
+ break;
+ }
+
switch (psp) {
case POWER_SUPPLY_PROP_CHARGE_CONTROL_END_THRESHOLD:
if (val->intval < 0 || val->intval > 100)
return -EINVAL;
- return write_to_ec(OXP_X1_CHARGE_LIMIT_REG, val->intval);
+ return write_to_ec(limit_reg, val->intval);
case POWER_SUPPLY_PROP_CHARGE_BEHAVIOUR:
switch (val->intval) {
case POWER_SUPPLY_CHARGE_BEHAVIOUR_AUTO:
raw_val = 0;
break;
case POWER_SUPPLY_CHARGE_BEHAVIOUR_INHIBIT_CHARGE_AWAKE:
- raw_val = OXP_X1_CHARGE_INHIBIT_MASK_AWAKE;
+ raw_val = OXP_CHARGE_INHIBIT_MASK_AWAKE;
break;
case POWER_SUPPLY_CHARGE_BEHAVIOUR_INHIBIT_CHARGE:
- raw_val = OXP_X1_CHARGE_INHIBIT_MASK_ALWAYS;
+ raw_val = OXP_CHARGE_INHIBIT_MASK_ALWAYS;
break;
default:
return -EINVAL;
}
- return write_to_ec(OXP_X1_CHARGE_INHIBIT_REG, raw_val);
+ return write_to_ec(inhibit_reg, raw_val);
default:
return -EINVAL;
}
@@ -656,6 +688,7 @@ static int oxp_pwm_enable(void)
case oxp_x1:
case oxp_g1_i:
case oxp_g1_a:
+ case oxp_apex:
return write_to_ec(OXP_SENSOR_PWM_ENABLE_REG, PWM_MODE_MANUAL);
default:
return -EINVAL;
@@ -676,6 +709,7 @@ static int oxp_pwm_disable(void)
case oxp_x1:
case oxp_g1_i:
case oxp_g1_a:
+ case oxp_apex:
return write_to_ec(OXP_SENSOR_PWM_ENABLE_REG, PWM_MODE_AUTO);
default:
return -EINVAL;
@@ -696,6 +730,7 @@ static int oxp_pwm_read(long *val)
case oxp_x1:
case oxp_g1_i:
case oxp_g1_a:
+ case oxp_apex:
return read_from_ec(OXP_SENSOR_PWM_ENABLE_REG, 1, val);
default:
return -EOPNOTSUPP;
@@ -732,6 +767,7 @@ static int oxp_pwm_fan_speed(long *val)
case oxp_mini_amd_a07:
case oxp_mini_amd_pro:
case oxp_g1_a:
+ case oxp_apex:
return read_from_ec(OXP_SENSOR_FAN_REG, 2, val);
default:
return -EOPNOTSUPP;
@@ -764,6 +800,7 @@ static int oxp_pwm_input_write(long val)
case oxp_fly:
case oxp_mini_amd_pro:
case oxp_g1_a:
+ case oxp_apex:
return write_to_ec(OXP_SENSOR_PWM_REG, val);
default:
return -EOPNOTSUPP;
@@ -803,6 +840,7 @@ static int oxp_pwm_input_read(long *val)
case oxp_fly:
case oxp_mini_amd_pro:
case oxp_g1_a:
+ case oxp_apex:
default:
ret = read_from_ec(OXP_SENSOR_PWM_REG, 1, val);
if (ret)
diff --git a/drivers/platform/x86/siemens/simatic-ipc.c b/drivers/platform/x86/siemens/simatic-ipc.c
index 7039874d8f11..4a4509fc9263 100644
--- a/drivers/platform/x86/siemens/simatic-ipc.c
+++ b/drivers/platform/x86/siemens/simatic-ipc.c
@@ -92,6 +92,7 @@ static int register_platform_devices(u32 station_id)
u8 wdtmode = SIMATIC_IPC_DEVICE_NONE;
u8 battmode = SIMATIC_IPC_DEVICE_NONE;
char *pdevname;
+ int ret;
int i;
for (i = 0; i < ARRAY_SIZE(device_modes); i++) {
@@ -139,8 +140,10 @@ static int register_platform_devices(u32 station_id)
pdevname, PLATFORM_DEVID_NONE,
&platform_data,
sizeof(struct simatic_ipc_platform));
- if (IS_ERR(ipc_led_platform_device))
- return PTR_ERR(ipc_led_platform_device);
+ if (IS_ERR(ipc_led_platform_device)) {
+ ret = PTR_ERR(ipc_led_platform_device);
+ goto err_unregister_batt;
+ }
pr_debug("device=%s created\n",
ipc_led_platform_device->name);
@@ -153,8 +156,10 @@ static int register_platform_devices(u32 station_id)
KBUILD_MODNAME "_wdt", PLATFORM_DEVID_NONE,
&platform_data,
sizeof(struct simatic_ipc_platform));
- if (IS_ERR(ipc_wdt_platform_device))
- return PTR_ERR(ipc_wdt_platform_device);
+ if (IS_ERR(ipc_wdt_platform_device)) {
+ ret = PTR_ERR(ipc_wdt_platform_device);
+ goto err_unregister_led;
+ }
pr_debug("device=%s created\n",
ipc_wdt_platform_device->name);
@@ -169,6 +174,14 @@ static int register_platform_devices(u32 station_id)
}
return 0;
+
+err_unregister_led:
+ if (ledmode != SIMATIC_IPC_DEVICE_NONE)
+ platform_device_unregister(ipc_led_platform_device);
+err_unregister_batt:
+ if (battmode != SIMATIC_IPC_DEVICE_NONE)
+ platform_device_unregister(ipc_batt_platform_device);
+ return ret;
}
static void request_additional_modules(u32 station_id)
diff --git a/drivers/platform/x86/tuxedo/nb04/wmi_ab.c b/drivers/platform/x86/tuxedo/nb04/wmi_ab.c
index 32d7756022c2..38a719381fdc 100644
--- a/drivers/platform/x86/tuxedo/nb04/wmi_ab.c
+++ b/drivers/platform/x86/tuxedo/nb04/wmi_ab.c
@@ -232,8 +232,8 @@ static const struct tux_kbl_map_entry_t sirius_16_ansii_kbl_map[] = {
{ 0x2e, { 246000, 67500, 5250 } },
{ 0x2a, { 269500, 67500, 5250 } },
{ 0x53, { 294500, 67500, 5250 } },
- { 0x55, { 311200, 67500, 5250 } },
- { 0x54, { 327900, 67500, 5250 } },
+ { 0x54, { 311200, 67500, 5250 } },
+ { 0x55, { 327900, 67500, 5250 } },
{ 0x56, { 344600, 67500, 5250 } },
{ 0x2b, { 31000, 85500, 5500 } },
{ 0x14, { 51500, 85500, 5500 } },
@@ -337,8 +337,8 @@ static const struct tux_kbl_map_entry_t sirius_16_iso_kbl_map[] = {
{ 0x2e, { 246000, 67500, 5250 } },
{ 0x2a, { 269500, 67500, 5250 } },
{ 0x53, { 294500, 67500, 5250 } },
- { 0x55, { 311200, 67500, 5250 } },
- { 0x54, { 327900, 67500, 5250 } },
+ { 0x54, { 311200, 67500, 5250 } },
+ { 0x55, { 327900, 67500, 5250 } },
{ 0x56, { 344600, 67500, 5250 } },
{ 0x2b, { 31000, 85500, 5500 } },
{ 0x14, { 51500, 85500, 5500 } },
diff --git a/drivers/power/supply/ab8500_fg.c b/drivers/power/supply/ab8500_fg.c
index e6b5efdf5dc9..c5eaa32c4a2d 100644
--- a/drivers/power/supply/ab8500_fg.c
+++ b/drivers/power/supply/ab8500_fg.c
@@ -1387,6 +1387,13 @@ static void ab8500_fg_check_capacity_limits(struct ab8500_fg *di, bool init)
}
}
+ /* Keep the charge properties current between percentage changes. */
+ if (di->bat_cap.prev_percent == percent &&
+ di->bat_cap.prev_mah != di->bat_cap.mah) {
+ di->bat_cap.prev_mah = di->bat_cap.mah;
+ changed = true;
+ }
+
if (changed) {
if (di->bm->capacity_scaling) {
di->bat_cap.cap_scale.scaled_cap =
@@ -2205,42 +2212,36 @@ static int ab8500_fg_get_ext_psy_data(struct power_supply *ext, void *data)
switch (prop) {
case POWER_SUPPLY_PROP_STATUS:
- switch (ext->desc->type) {
- case POWER_SUPPLY_TYPE_BATTERY:
- switch (ret.intval) {
- case POWER_SUPPLY_STATUS_UNKNOWN:
- case POWER_SUPPLY_STATUS_DISCHARGING:
- case POWER_SUPPLY_STATUS_NOT_CHARGING:
- if (!di->flags.charging)
- break;
- di->flags.charging = false;
- di->flags.fully_charged = false;
- if (di->bm->capacity_scaling)
- ab8500_fg_update_cap_scalers(di);
- queue_work(di->fg_wq, &di->fg_work);
+ switch (ret.intval) {
+ case POWER_SUPPLY_STATUS_UNKNOWN:
+ case POWER_SUPPLY_STATUS_DISCHARGING:
+ case POWER_SUPPLY_STATUS_NOT_CHARGING:
+ if (!di->flags.charging)
break;
- case POWER_SUPPLY_STATUS_FULL:
- if (di->flags.fully_charged)
- break;
- di->flags.fully_charged = true;
- di->flags.force_full = true;
- /* Save current capacity as maximum */
- di->bat_cap.max_mah = di->bat_cap.mah;
- queue_work(di->fg_wq, &di->fg_work);
- break;
- case POWER_SUPPLY_STATUS_CHARGING:
- if (di->flags.charging &&
- !di->flags.fully_charged)
- break;
- di->flags.charging = true;
- di->flags.fully_charged = false;
- if (di->bm->capacity_scaling)
- ab8500_fg_update_cap_scalers(di);
- queue_work(di->fg_wq, &di->fg_work);
+ di->flags.charging = false;
+ di->flags.fully_charged = false;
+ if (di->bm->capacity_scaling)
+ ab8500_fg_update_cap_scalers(di);
+ queue_work(di->fg_wq, &di->fg_work);
+ break;
+ case POWER_SUPPLY_STATUS_FULL:
+ if (di->flags.fully_charged)
break;
- }
+ di->flags.fully_charged = true;
+ di->flags.force_full = true;
+ /* Save current capacity as maximum */
+ di->bat_cap.max_mah = di->bat_cap.mah;
+ queue_work(di->fg_wq, &di->fg_work);
break;
- default:
+ case POWER_SUPPLY_STATUS_CHARGING:
+ if (di->flags.charging &&
+ !di->flags.fully_charged)
+ break;
+ di->flags.charging = true;
+ di->flags.fully_charged = false;
+ if (di->bm->capacity_scaling)
+ ab8500_fg_update_cap_scalers(di);
+ queue_work(di->fg_wq, &di->fg_work);
break;
}
break;
diff --git a/drivers/power/supply/bq27xxx_battery.c b/drivers/power/supply/bq27xxx_battery.c
index 01c9ec1116fa..cd30993b8a89 100644
--- a/drivers/power/supply/bq27xxx_battery.c
+++ b/drivers/power/supply/bq27xxx_battery.c
@@ -43,6 +43,7 @@
*/
#include <linux/device.h>
+#include <linux/devm-helpers.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/param.h>
@@ -2234,7 +2235,6 @@ int bq27xxx_battery_setup(struct bq27xxx_device_info *di)
};
int ret;
- INIT_DELAYED_WORK(&di->work, bq27xxx_battery_poll);
ret = devm_mutex_init(di->dev, &di->lock);
if (ret)
return ret;
@@ -2255,6 +2255,12 @@ int bq27xxx_battery_setup(struct bq27xxx_device_info *di)
psy_desc->get_property = bq27xxx_battery_get_property;
psy_desc->external_power_changed = bq27xxx_external_power_changed;
+ /* Cancel the poll work after the power_supply is unregistered. */
+ ret = devm_delayed_work_autocancel(di->dev, &di->work,
+ bq27xxx_battery_poll);
+ if (ret)
+ return ret;
+
di->bat = devm_power_supply_register(di->dev, psy_desc, &psy_cfg);
if (IS_ERR(di->bat))
return dev_err_probe(di->dev, PTR_ERR(di->bat),
@@ -2281,8 +2287,6 @@ void bq27xxx_battery_teardown(struct bq27xxx_device_info *di)
mutex_lock(&di->lock);
di->removed = true;
mutex_unlock(&di->lock);
-
- cancel_delayed_work_sync(&di->work);
}
EXPORT_SYMBOL_GPL(bq27xxx_battery_teardown);
@@ -2291,7 +2295,7 @@ static int bq27xxx_battery_suspend(struct device *dev)
{
struct bq27xxx_device_info *di = dev_get_drvdata(dev);
- cancel_delayed_work(&di->work);
+ cancel_delayed_work_sync(&di->work);
return 0;
}
diff --git a/drivers/power/supply/cros_charge-control.c b/drivers/power/supply/cros_charge-control.c
index e0f168624807..0b4aff1c2950 100644
--- a/drivers/power/supply/cros_charge-control.c
+++ b/drivers/power/supply/cros_charge-control.c
@@ -13,6 +13,7 @@
#include <linux/platform_data/cros_ec_commands.h>
#include <linux/platform_data/cros_ec_proto.h>
#include <linux/platform_device.h>
+#include <linux/pm.h>
#include <linux/types.h>
#define EC_CHARGE_CONTROL_BEHAVIOURS (BIT(POWER_SUPPLY_CHARGE_BEHAVIOUR_AUTO) | \
@@ -293,6 +294,7 @@ static int cros_chctl_probe(struct platform_device *pdev)
priv->dev = dev;
priv->cros_ec = cros_ec;
+ dev_set_drvdata(dev, priv);
if (priv->cmd_version == 1)
priv->psy_ext = &cros_chctl_psy_ext_v1;
@@ -318,6 +320,28 @@ static int cros_chctl_probe(struct platform_device *pdev)
return devm_battery_hook_register(dev, &priv->battery_hook);
}
+static int cros_chctl_resume(struct device *dev)
+{
+ struct cros_chctl_priv *priv = dev_get_drvdata(dev);
+ int ret;
+
+ /*
+ * The EC resets its charge control state when the AP powers off, so the
+ * configuration is lost across hibernation, while the values exposed to
+ * userspace still describe it. The cached state is the only remaining
+ * record of that configuration: reprogram the EC from it rather than
+ * re-reading the EC, which would adopt the reset defaults instead.
+ */
+ guard(mutex)(&priv->lock);
+ ret = cros_chctl_configure_ec(priv);
+ if (ret)
+ dev_err(dev, "failed to restore charge control state: %d\n", ret);
+
+ return ret;
+}
+
+static DEFINE_SIMPLE_DEV_PM_OPS(cros_chctl_pm_ops, NULL, cros_chctl_resume);
+
static const struct platform_device_id cros_chctl_id[] = {
{ .name = "cros-charge-control" },
{ }
@@ -325,7 +349,10 @@ static const struct platform_device_id cros_chctl_id[] = {
MODULE_DEVICE_TABLE(platform, cros_chctl_id);
static struct platform_driver cros_chctl_driver = {
- .driver.name = "cros-charge-control",
+ .driver = {
+ .name = "cros-charge-control",
+ .pm = pm_sleep_ptr(&cros_chctl_pm_ops),
+ },
.probe = cros_chctl_probe,
.id_table = cros_chctl_id,
};
diff --git a/drivers/power/supply/generic-adc-battery.c b/drivers/power/supply/generic-adc-battery.c
index f11a5c794e7e..937b84e04ba2 100644
--- a/drivers/power/supply/generic-adc-battery.c
+++ b/drivers/power/supply/generic-adc-battery.c
@@ -135,7 +135,7 @@ static void gab_work(struct work_struct *work)
adc_bat = container_of(delayed_work, struct gab, bat_work);
status = adc_bat->status;
- if (!power_supply_am_i_supplied(adc_bat->psy))
+ if (power_supply_am_i_supplied(adc_bat->psy) <= 0)
adc_bat->status = POWER_SUPPLY_STATUS_DISCHARGING;
else if (gab_charge_finished(adc_bat))
adc_bat->status = POWER_SUPPLY_STATUS_NOT_CHARGING;
diff --git a/drivers/power/supply/ltc2941-battery-gauge.c b/drivers/power/supply/ltc2941-battery-gauge.c
index d1be314f36a0..6ebb555f203d 100644
--- a/drivers/power/supply/ltc2941-battery-gauge.c
+++ b/drivers/power/supply/ltc2941-battery-gauge.c
@@ -464,9 +464,11 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
/* r_sense can be negative, when sense+ is connected to the battery
* instead of the sense-. This results in reversed measurements. */
ret = of_property_read_s32(np, "lltc,resistor-sense", &r_sense);
- if (ret < 0)
+ if (ret < 0) {
+ of_node_put(np);
return dev_err_probe(&client->dev, ret,
"Could not find lltc,resistor-sense in devicetree\n");
+ }
info->r_sense = r_sense;
ret = of_property_read_u32(np, "lltc,prescaler-exponent",
@@ -477,6 +479,8 @@ static int ltc294x_i2c_probe(struct i2c_client *client)
prescaler_exp = LTC2941_MAX_PRESCALER_EXP;
}
+ of_node_put(np);
+
if (info->id == LTC2943_ID) {
if (prescaler_exp > LTC2943_MAX_PRESCALER_EXP)
prescaler_exp = LTC2943_MAX_PRESCALER_EXP;
diff --git a/drivers/power/supply/power_supply_core.c b/drivers/power/supply/power_supply_core.c
index 1279785645fb..eba690f1dad3 100644
--- a/drivers/power/supply/power_supply_core.c
+++ b/drivers/power/supply/power_supply_core.c
@@ -190,7 +190,35 @@ static void power_supply_deferred_register_work(struct work_struct *work)
device_unlock(psy->dev.parent);
}
-#ifdef CONFIG_OF
+static int power_supply_check_supplies_by_name(struct power_supply *psy)
+{
+ struct device *parent = psy->dev.parent;
+ int nval, ret;
+
+ if (!parent)
+ return 0;
+
+ nval = device_property_string_array_count(parent, "supplied-from");
+ if (nval <= 0)
+ return 0;
+
+ psy->supplied_from = devm_kmalloc_array(&psy->dev, nval,
+ sizeof(*psy->supplied_from),
+ GFP_KERNEL);
+ if (!psy->supplied_from)
+ return -ENOMEM;
+
+ ret = device_property_read_string_array(parent, "supplied-from",
+ (const char **)psy->supplied_from,
+ nval);
+ if (ret < 0)
+ return ret;
+
+ psy->num_supplies = nval;
+
+ return 0;
+}
+
static int __power_supply_populate_supplied_from(struct power_supply *epsy,
void *data)
{
@@ -262,22 +290,21 @@ static int power_supply_find_supply_from_fwnode(struct fwnode_handle *supply_nod
static int power_supply_check_supplies(struct power_supply *psy)
{
struct fwnode_handle *np;
- int cnt = 0;
+ int cnt, ret;
/* If there is already a list honor it */
if (psy->supplied_from && psy->num_supplies > 0)
return 0;
- /* No device node found, nothing to do */
- if (!psy->dev.fwnode)
- return 0;
-
- do {
- int ret;
+ for (cnt = 0; psy->dev.fwnode; cnt++) {
+ np = fwnode_find_reference(psy->dev.fwnode, "power-supplies", cnt);
+ if (IS_ERR(np)) {
+ ret = PTR_ERR(np);
+ if (ret != -ENOENT)
+ return ret;
- np = fwnode_find_reference(psy->dev.fwnode, "power-supplies", cnt++);
- if (IS_ERR(np))
break;
+ }
ret = power_supply_find_supply_from_fwnode(np);
fwnode_handle_put(np);
@@ -286,48 +313,21 @@ static int power_supply_check_supplies(struct power_supply *psy)
dev_dbg(&psy->dev, "Failed to find supply!\n");
return ret;
}
- } while (!IS_ERR(np));
+ }
- /* Missing valid "power-supplies" entries */
- if (cnt == 1)
- return 0;
+ /* Fall back to the name-based property if no references were specified. */
+ if (!cnt)
+ return power_supply_check_supplies_by_name(psy);
/* All supplies found, allocate char * array for filling */
psy->supplied_from = devm_kcalloc(&psy->dev,
- cnt - 1, sizeof(*psy->supplied_from),
+ cnt, sizeof(*psy->supplied_from),
GFP_KERNEL);
if (!psy->supplied_from)
return -ENOMEM;
return power_supply_populate_supplied_from(psy);
}
-#else
-static int power_supply_check_supplies(struct power_supply *psy)
-{
- int nval, ret;
-
- if (!psy->dev.parent)
- return 0;
-
- nval = device_property_string_array_count(psy->dev.parent, "supplied-from");
- if (nval <= 0)
- return 0;
-
- psy->supplied_from = devm_kmalloc_array(&psy->dev, nval,
- sizeof(char *), GFP_KERNEL);
- if (!psy->supplied_from)
- return -ENOMEM;
-
- ret = device_property_read_string_array(psy->dev.parent,
- "supplied-from", (const char **)psy->supplied_from, nval);
- if (ret < 0)
- return ret;
-
- psy->num_supplies = nval;
-
- return 0;
-}
-#endif
struct psy_am_i_supplied_data {
struct power_supply *psy;
diff --git a/drivers/power/supply/qcom_battmgr.c b/drivers/power/supply/qcom_battmgr.c
index 7716eb9e1aff..97c8501c2996 100644
--- a/drivers/power/supply/qcom_battmgr.c
+++ b/drivers/power/supply/qcom_battmgr.c
@@ -1254,6 +1254,23 @@ static unsigned int qcom_battmgr_sc8280xp_convert_temp(unsigned int temperature)
return DIV_ROUND_CLOSEST(temperature, 10);
}
+/*
+ * The charger supplies' online state comes from the battery status response,
+ * and the firmware sends no notification of its own when it changes, so
+ * announce the change here. A supply that is not registered yet announces its
+ * state when it is.
+ */
+static void qcom_battmgr_sc8280xp_set_online(struct power_supply *psy,
+ bool *online, bool value)
+{
+ if (*online == value)
+ return;
+
+ *online = value;
+ if (psy)
+ power_supply_changed(psy);
+}
+
static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr,
const struct qcom_battmgr_message *resp,
size_t len)
@@ -1323,9 +1340,12 @@ static void qcom_battmgr_sc8280xp_callback(struct qcom_battmgr *battmgr,
battmgr->status.temperature = qcom_battmgr_sc8280xp_convert_temp(le32_to_cpu(resp->status.temperature));
source = le32_to_cpu(resp->status.charging_source);
- battmgr->ac.online = source == BATTMGR_CHARGING_SOURCE_AC;
- battmgr->usb.online = source == BATTMGR_CHARGING_SOURCE_USB;
- battmgr->wireless.online = source == BATTMGR_CHARGING_SOURCE_WIRELESS;
+ qcom_battmgr_sc8280xp_set_online(battmgr->ac_psy, &battmgr->ac.online,
+ source == BATTMGR_CHARGING_SOURCE_AC);
+ qcom_battmgr_sc8280xp_set_online(battmgr->usb_psy, &battmgr->usb.online,
+ source == BATTMGR_CHARGING_SOURCE_USB);
+ qcom_battmgr_sc8280xp_set_online(battmgr->wls_psy, &battmgr->wireless.online,
+ source == BATTMGR_CHARGING_SOURCE_WIRELESS);
if (battmgr->info.last_full_capacity != 0) {
/*
* 100 * battmgr->status.capacity can overflow a 32bit
diff --git a/drivers/power/supply/qcom_smbx.c b/drivers/power/supply/qcom_smbx.c
index 67fdb4335338..8c6b7c46b85f 100644
--- a/drivers/power/supply/qcom_smbx.c
+++ b/drivers/power/supply/qcom_smbx.c
@@ -352,6 +352,9 @@
/* pmi8998 registers represent current in increments of 1/40th of an amp */
#define CURRENT_SCALE_FACTOR 25000
+#define SMB2_FLOAT_VOLTAGE_MIN_UV 3487500
+#define SMB2_FLOAT_VOLTAGE_MAX_UV 4920000
+#define SMB2_FLOAT_VOLTAGE_STEP_UV 7500
/* clang-format on */
enum charger_status {
@@ -378,6 +381,7 @@ struct smb_init_register {
* @base: Base address for smb registers
* @regmap: Register map
* @batt_info: Battery data from DT
+ * @initial_charge_enable: Charging enable state before hardware setup
* @status_change_work: Worker to handle plug/unplug events
* @cable_irq: USB plugin IRQ
* @wakeup_enabled: If the cable IRQ will cause a wakeup
@@ -391,6 +395,7 @@ struct smb_chip {
unsigned int base;
struct regmap *regmap;
struct power_supply_battery_info *batt_info;
+ u8 initial_charge_enable;
struct delayed_work status_change_work;
int cable_irq;
@@ -632,26 +637,20 @@ static int smb_get_prop_health(struct smb_chip *chip, int *val)
return rc;
}
- switch (stat) {
- case CHARGER_ERROR_STATUS_BAT_OV_BIT:
+ if (stat & CHARGER_ERROR_STATUS_BAT_OV_BIT)
*val = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
- return 0;
- case BAT_TEMP_STATUS_TOO_COLD_BIT:
+ else if (stat & BAT_TEMP_STATUS_TOO_COLD_BIT)
*val = POWER_SUPPLY_HEALTH_COLD;
- return 0;
- case BAT_TEMP_STATUS_TOO_HOT_BIT:
+ else if (stat & BAT_TEMP_STATUS_TOO_HOT_BIT)
*val = POWER_SUPPLY_HEALTH_OVERHEAT;
- return 0;
- case BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT:
+ else if (stat & BAT_TEMP_STATUS_COLD_SOFT_LIMIT_BIT)
*val = POWER_SUPPLY_HEALTH_COOL;
- return 0;
- case BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT:
+ else if (stat & BAT_TEMP_STATUS_HOT_SOFT_LIMIT_BIT)
*val = POWER_SUPPLY_HEALTH_WARM;
- return 0;
- default:
+ else
*val = POWER_SUPPLY_HEALTH_GOOD;
- return 0;
- }
+
+ return 0;
}
static int smb_get_property(struct power_supply *psy,
@@ -723,16 +722,22 @@ static irqreturn_t smb_handle_batt_overvoltage(int irq, void *data)
{
struct smb_chip *chip = data;
unsigned int status;
+ int rc;
- regmap_read(chip->regmap, chip->base + BATTERY_CHARGER_STATUS_2,
- &status);
+ rc = regmap_read(chip->regmap,
+ chip->base + BATTERY_CHARGER_STATUS_2, &status);
+ if (rc < 0) {
+ dev_err(chip->dev, "Couldn't read charger status: %d\n", rc);
+ return IRQ_HANDLED;
+ }
if (status & CHARGER_ERROR_STATUS_BAT_OV_BIT) {
/* The hardware stops charging automatically */
dev_err(chip->dev, "battery overvoltage detected\n");
- power_supply_changed(chip->chg_psy);
}
+ power_supply_changed(chip->chg_psy);
+
return IRQ_HANDLED;
}
@@ -764,7 +769,7 @@ static irqreturn_t smb_handle_wdog_bark(int irq, void *data)
power_supply_changed(chip->chg_psy);
- rc = regmap_write(chip->regmap, BARK_BITE_WDOG_PET,
+ rc = regmap_write(chip->regmap, chip->base + BARK_BITE_WDOG_PET,
BARK_BITE_WDOG_PET_BIT);
if (rc < 0)
dev_err(chip->dev, "Couldn't pet the dog rc=%d\n", rc);
@@ -814,11 +819,7 @@ static const struct smb_init_register smb_init_seq[] = {
.mask = SOC_LT_CHG_RECHARGE_THRESH_SEL_BIT |
VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT,
.val = VBT_LT_CHG_RECHARGE_THRESH_SEL_BIT },
- /* Enable charging */
{ .addr = USBIN_OPTIONS_1_CFG, .mask = HVDCP_EN_BIT, .val = 0 },
- { .addr = CHARGING_ENABLE_CMD,
- .mask = CHARGING_ENABLE_CMD_BIT,
- .val = CHARGING_ENABLE_CMD_BIT },
/*
* Match downstream defaults
* CHG_EN_SRC_BIT - charger enable is controlled by software
@@ -839,6 +840,13 @@ static const struct smb_init_register smb_init_seq[] = {
AUTO_RECHG_BIT | EN_ANALOG_DROP_IN_VBATT_BIT |
CHARGER_INHIBIT_BIT,
.val = CHARGER_INHIBIT_BIT },
+ /*
+ * Set the enable command only once CHGR_CFG2 has selected software
+ * control, or a charger its watchdog has disabled stays disabled.
+ */
+ { .addr = CHARGING_ENABLE_CMD,
+ .mask = CHARGING_ENABLE_CMD_BIT,
+ .val = CHARGING_ENABLE_CMD_BIT },
/* STAT pin software override, match downstream. Parallel charging? */
{ .addr = STAT_CFG,
.mask = STAT_SW_OVERRIDE_CFG_BIT,
@@ -909,6 +917,19 @@ static int smb_init_hw(struct smb_chip *chip)
return 0;
}
+static void smb_restore_charge_enable(void *data)
+{
+ struct smb_chip *chip = data;
+ int rc;
+
+ rc = regmap_update_bits(chip->regmap,
+ chip->base + CHARGING_ENABLE_CMD,
+ CHARGING_ENABLE_CMD_BIT,
+ chip->initial_charge_enable);
+ if (rc < 0)
+ dev_err(chip->dev, "Couldn't restore charging state: %d\n", rc);
+}
+
static int smb_init_irq(struct smb_chip *chip, int *irq, const char *name,
irqreturn_t (*handler)(int irq, void *data))
{
@@ -935,6 +956,9 @@ static int smb_probe(struct platform_device *pdev)
struct power_supply_config supply_config = {};
struct power_supply_desc *desc;
struct smb_chip *chip;
+ unsigned int charge_enable;
+ unsigned int float_voltage_sel;
+ int float_voltage_uv;
int rc, irq;
chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL);
@@ -965,6 +989,18 @@ static int smb_probe(struct platform_device *pdev)
"Couldn't get usbin_i IIO channel\n");
}
+ rc = regmap_read(chip->regmap, chip->base + CHARGING_ENABLE_CMD,
+ &charge_enable);
+ if (rc < 0)
+ return dev_err_probe(chip->dev, rc,
+ "Couldn't read charging state\n");
+
+ chip->initial_charge_enable = charge_enable & CHARGING_ENABLE_CMD_BIT;
+ rc = devm_add_action_or_reset(chip->dev, smb_restore_charge_enable, chip);
+ if (rc)
+ return dev_err_probe(chip->dev, rc,
+ "Couldn't register charging state rollback\n");
+
rc = smb_init_hw(chip);
if (rc < 0)
return rc;
@@ -999,12 +1035,35 @@ static int smb_probe(struct platform_device *pdev)
return dev_err_probe(chip->dev, rc,
"Failed to init status change work\n");
- rc = (chip->batt_info->voltage_max_design_uv - 3487500) / 7500 + 1;
+ if (power_supply_battery_info_has_prop(chip->batt_info,
+ POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX)) {
+ float_voltage_uv =
+ chip->batt_info->constant_charge_voltage_max_uv;
+ } else if (power_supply_battery_info_has_prop(chip->batt_info,
+ POWER_SUPPLY_PROP_VOLTAGE_MAX_DESIGN)) {
+ float_voltage_uv = chip->batt_info->voltage_max_design_uv;
+ } else {
+ dev_warn(chip->dev, "No battery float voltage; preserving hardware setting\n");
+ goto skip_float_voltage;
+ }
+
+ if (float_voltage_uv < SMB2_FLOAT_VOLTAGE_MIN_UV ||
+ float_voltage_uv > SMB2_FLOAT_VOLTAGE_MAX_UV)
+ return dev_err_probe(chip->dev, -EINVAL,
+ "float voltage %d uV outside %d-%d uV\n",
+ float_voltage_uv,
+ SMB2_FLOAT_VOLTAGE_MIN_UV,
+ SMB2_FLOAT_VOLTAGE_MAX_UV);
+
+ float_voltage_sel =
+ (float_voltage_uv - SMB2_FLOAT_VOLTAGE_MIN_UV) /
+ SMB2_FLOAT_VOLTAGE_STEP_UV;
rc = regmap_update_bits(chip->regmap, chip->base + FLOAT_VOLTAGE_CFG,
- FLOAT_VOLTAGE_SETTING_MASK, rc);
+ FLOAT_VOLTAGE_SETTING_MASK, float_voltage_sel);
if (rc < 0)
return dev_err_probe(chip->dev, rc, "Couldn't set vbat max\n");
+skip_float_voltage:
rc = smb_init_irq(chip, &irq, "bat-ov", smb_handle_batt_overvoltage);
if (rc < 0)
return rc;
@@ -1028,6 +1087,8 @@ static int smb_probe(struct platform_device *pdev)
if (rc < 0)
return dev_err_probe(chip->dev, rc, "Couldn't set wake irq\n");
+ devm_remove_action(chip->dev, smb_restore_charge_enable, chip);
+
platform_set_drvdata(pdev, chip);
/* Initialise charger state */
diff --git a/drivers/pps/generators/pps_gen.c b/drivers/pps/generators/pps_gen.c
index d80e28dc31dc..452cc12a96f2 100644
--- a/drivers/pps/generators/pps_gen.c
+++ b/drivers/pps/generators/pps_gen.c
@@ -228,9 +228,18 @@ static void pps_gen_unregister_cdev(struct pps_gen_device *pps_gen)
* An open file keeps pps_gen around, but the driver may free info as
* soon as we return. The sysfs files are gone now, so wait for the
* ioctls using info and make later ones fail.
+ *
+ * The driver may have stopped the generator before calling us, but
+ * userspace could have enabled it again since. Nothing can enable it
+ * after this point, so stop it here for good.
*/
- scoped_guard(mutex, &pps_gen->info_lock)
+ scoped_guard(mutex, &pps_gen->info_lock) {
+ if (pps_gen->enabled) {
+ pps_gen->info->enable(pps_gen, false);
+ pps_gen->enabled = false;
+ }
pps_gen->info = NULL;
+ }
put_device(&pps_gen->dev);
}
diff --git a/drivers/pwm/pwm-tiehrpwm.c b/drivers/pwm/pwm-tiehrpwm.c
index 3a53489f0b98..e3407d0a1f7f 100644
--- a/drivers/pwm/pwm-tiehrpwm.c
+++ b/drivers/pwm/pwm-tiehrpwm.c
@@ -358,6 +358,9 @@ static void ehrpwm_pwm_disable(struct pwm_chip *chip, struct pwm_device *pwm)
/* Disable clock on PWM disable */
pm_runtime_put_sync(pwmchip_parent(chip));
+
+ /* A disabled channel must not constrain the sibling's period */
+ pc->period_cycles[pwm->hwpwm] = 0;
}
static void ehrpwm_pwm_free(struct pwm_chip *chip, struct pwm_device *pwm)
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 6a4008f387b5..e20f00c364ef 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -2239,6 +2239,7 @@ static int regulator_resolve_supply(struct regulator_dev *rdev)
if (r == rdev) {
dev_err(dev, "Supply for %s (%s) resolved to itself\n",
rdev->desc->name, rdev->supply_name);
+ put_device(&rdev->dev);
if (!have_full_constraints()) {
ret = -EINVAL;
goto out;
@@ -5861,6 +5862,7 @@ static void regulator_resolve_coupling(struct regulator_dev *rdev)
if (c_rdev->coupling_desc.coupler != coupler) {
rdev_err(rdev, "coupler mismatch with %s\n",
rdev_get_name(c_rdev));
+ put_device(&c_rdev->dev);
return;
}
@@ -5907,6 +5909,8 @@ static void regulator_remove_coupling(struct regulator_dev *rdev)
c_desc->coupled_rdevs[i] = NULL;
c_desc->n_resolved--;
+
+ put_device(&c_rdev->dev);
}
if (coupler && coupler->detach_regulator) {
diff --git a/drivers/regulator/rt6190-regulator.c b/drivers/regulator/rt6190-regulator.c
index f2cd9540038d..21ced348a2cf 100644
--- a/drivers/regulator/rt6190-regulator.c
+++ b/drivers/regulator/rt6190-regulator.c
@@ -108,19 +108,27 @@ static int rt6190_out_enable(struct regulator_dev *rdev)
ret = regmap_raw_read(regmap, RT6190_REG_OUTV, out_cfg,
sizeof(out_cfg));
if (ret)
- return ret;
+ goto err_pm_put;
ret = regulator_enable_regmap(rdev);
if (ret)
- return ret;
+ goto err_pm_put;
ret = regmap_raw_write(regmap, RT6190_REG_OUTV, out_cfg,
sizeof(out_cfg));
if (ret)
- return ret;
+ goto err_pm_put;
+
+ ret = regmap_update_bits(regmap, RT6190_REG_SET5, RT6190_ENGCP_MASK,
+ RT6190_ENGCP_MASK);
+ if (ret)
+ goto err_pm_put;
+
+ return 0;
- return regmap_update_bits(regmap, RT6190_REG_SET5, RT6190_ENGCP_MASK,
- RT6190_ENGCP_MASK);
+err_pm_put:
+ pm_runtime_put(data->dev);
+ return ret;
}
static int rt6190_out_disable(struct regulator_dev *rdev)
diff --git a/drivers/remoteproc/remoteproc_elf_loader.c b/drivers/remoteproc/remoteproc_elf_loader.c
index 94177e416047..da3cddbe7d4c 100644
--- a/drivers/remoteproc/remoteproc_elf_loader.c
+++ b/drivers/remoteproc/remoteproc_elf_loader.c
@@ -46,8 +46,9 @@ int rproc_elf_sanity_check(struct rproc *rproc, const struct firmware *fw)
struct elf32_hdr *ehdr;
u32 elf_shdr_get_size;
u64 phoff, shoff;
+ size_t shend;
char class;
- u16 phnum;
+ u16 phnum, shnum, shstrndx;
if (!fw) {
dev_err(dev, "failed to load %s\n", name);
@@ -90,9 +91,27 @@ int rproc_elf_sanity_check(struct rproc *rproc, const struct firmware *fw)
phoff = elf_hdr_get_e_phoff(class, fw->data);
shoff = elf_hdr_get_e_shoff(class, fw->data);
phnum = elf_hdr_get_e_phnum(class, fw->data);
+ shnum = elf_hdr_get_e_shnum(class, fw->data);
+ shstrndx = elf_hdr_get_e_shstrndx(class, fw->data);
elf_shdr_get_size = elf_size_of_shdr(class);
- if (fw->size < shoff + elf_shdr_get_size) {
+ /* keeps shoff in size_t range for the two bounds below */
+ if (shoff > fw->size) {
+ dev_err(dev, "Section header table is out of bounds\n");
+ return -EINVAL;
+ }
+
+ if (shnum) {
+ shend = size_add(size_mul(elf_shdr_get_size, shnum), shoff);
+ if (shend > fw->size) {
+ dev_err(dev, "Section headers are out of bounds\n");
+ return -EINVAL;
+ }
+ }
+
+ /* find_table() reads the header at shstrndx even with no sections */
+ shend = size_add(size_mul(elf_shdr_get_size, (size_t)shstrndx + 1), shoff);
+ if (shend > fw->size) {
dev_err(dev, "Image is too small\n");
return -EINVAL;
}
diff --git a/drivers/rtc/rtc-cmos.c b/drivers/rtc/rtc-cmos.c
index fa04ece151b8..209aa4ec732c 100644
--- a/drivers/rtc/rtc-cmos.c
+++ b/drivers/rtc/rtc-cmos.c
@@ -57,7 +57,28 @@
*/
static bool use_acpi_alarm;
-module_param(use_acpi_alarm, bool, 0444);
+static bool use_acpi_alarm_explicit;
+
+static int set_use_acpi_alarm(const char *val, const struct kernel_param *kp)
+{
+ int ret;
+
+ ret = param_set_bool(val, kp);
+ if (!ret)
+ use_acpi_alarm_explicit = true;
+
+ return ret;
+}
+
+static const struct kernel_param_ops use_acpi_alarm_ops = {
+ .flags = KERNEL_PARAM_OPS_FL_NOARG,
+ .set = set_use_acpi_alarm,
+ .get = param_get_bool,
+};
+
+param_check_bool(use_acpi_alarm, &use_acpi_alarm);
+module_param_cb(use_acpi_alarm, &use_acpi_alarm_ops, &use_acpi_alarm, 0444);
+__MODULE_PARM_TYPE(use_acpi_alarm, "bool");
static inline int cmos_use_acpi_alarm(void)
{
@@ -758,7 +779,9 @@ static u32 rtc_handler(void *context)
unsigned char rtc_intr;
unsigned long flags;
-
+ /* Quiesce this event before rtc->irqwork can arm the next one. */
+ acpi_clear_event(ACPI_EVENT_RTC);
+ acpi_disable_event(ACPI_EVENT_RTC, 0);
/*
* Always update rtc irq when ACPI is used as RTC Alarm.
* Or else, ACPI SCI is enabled during suspend/resume only,
@@ -781,8 +804,6 @@ static u32 rtc_handler(void *context)
}
pm_wakeup_hard_event(dev);
- acpi_clear_event(ACPI_EVENT_RTC);
- acpi_disable_event(ACPI_EVENT_RTC, 0);
return ACPI_INTERRUPT_HANDLED;
}
@@ -822,6 +843,9 @@ static void rtc_wake_off(struct device *dev)
#ifdef CONFIG_X86
static void use_acpi_alarm_quirks(void)
{
+ if (use_acpi_alarm_explicit)
+ return;
+
if (acpi_gbl_FADT.flags & ACPI_FADT_FIXED_RTC)
return;
diff --git a/drivers/rtc/rtc-pcf8563.c b/drivers/rtc/rtc-pcf8563.c
index 7eebdb058067..0767331a2cc7 100644
--- a/drivers/rtc/rtc-pcf8563.c
+++ b/drivers/rtc/rtc-pcf8563.c
@@ -531,7 +531,7 @@ static int pcf8563_probe(struct i2c_client *client)
err = devm_request_threaded_irq(&client->dev, client->irq,
NULL, pcf8563_irq,
IRQF_SHARED | IRQF_ONESHOT | irqflags,
- pcf8563_driver.driver.name, client);
+ pcf8563_driver.driver.name, pcf8563);
if (err) {
dev_err(&client->dev, "unable to request IRQ %d\n",
client->irq);
diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index d35f93451ee9..3c9e736d45e7 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -601,7 +601,8 @@ static int iscsi_tcp_r2t_rsp(struct iscsi_conn *conn, struct iscsi_hdr *hdr)
data_length, session->max_burst);
data_offset = be32_to_cpu(rhdr->data_offset);
- if (data_offset + data_length > task->sc->sdb.length) {
+ if (data_offset > task->sc->sdb.length ||
+ data_length > task->sc->sdb.length - data_offset) {
iscsi_conn_printk(KERN_ERR, conn,
"invalid R2T with data len %u at offset %u "
"and total length %d\n", data_length,
diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index c14f81403a09..81a0519a2984 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -9,6 +9,7 @@
#include <linux/blkdev.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
+#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/mm.h>
@@ -195,10 +196,14 @@ static int sg_set_timeout(struct scsi_device *sdev, int __user *p)
{
int timeout, err = get_user(timeout, p);
- if (!err)
- sdev->sg_timeout = clock_t_to_jiffies(timeout);
+ if (err)
+ return err;
+ if (timeout < 0)
+ return -EIO;
- return err;
+ sdev->sg_timeout = min(clock_t_to_jiffies(timeout),
+ min(UINT_MAX, MAX_JIFFY_OFFSET));
+ return 0;
}
static int sg_get_reserved_size(struct scsi_device *sdev, int __user *p)
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index a1b21ea14e54..208b915f25a2 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -4270,7 +4270,15 @@ static void sd_remove(struct scsi_device *sdp)
struct device *dev = &sdp->sdev_gendev;
struct scsi_disk *sdkp = dev_get_drvdata(dev);
- scsi_autopm_get_device(sdkp->device);
+ /*
+ * If runtime resume fails, e.g. because the device was unplugged
+ * while runtime suspended, the request queue stays in the pm-only
+ * state and any I/O submitted by del_gendisk(), e.g. from
+ * sync_filesystem(), would wait forever in __bio_queue_enter().
+ * Prevent this by marking the disk dead.
+ */
+ if (scsi_autopm_get_device(sdkp->device) < 0)
+ blk_mark_disk_dead(sdkp->disk);
device_del(&sdkp->disk_dev);
del_gendisk(sdkp->disk);
diff --git a/drivers/sh/intc/core.c b/drivers/sh/intc/core.c
index aa68fe190865..ffbe60234eef 100644
--- a/drivers/sh/intc/core.c
+++ b/drivers/sh/intc/core.c
@@ -275,9 +275,6 @@ int __init register_intc_controller(struct intc_desc *desc)
k += save_reg(d, k, hw->prio_regs[i].set_reg, smp);
k += save_reg(d, k, hw->prio_regs[i].clr_reg, smp);
}
-
- sort(d->prio, hw->nr_prio_regs, sizeof(*d->prio),
- intc_handle_int_cmp, NULL);
}
if (hw->sense_regs) {
@@ -287,9 +284,6 @@ int __init register_intc_controller(struct intc_desc *desc)
for (i = 0; i < hw->nr_sense_regs; i++)
k += save_reg(d, k, hw->sense_regs[i].reg, 0);
-
- sort(d->sense, hw->nr_sense_regs, sizeof(*d->sense),
- intc_handle_int_cmp, NULL);
}
if (hw->subgroups)
@@ -357,6 +351,11 @@ int __init register_intc_controller(struct intc_desc *desc)
}
}
+ sort(d->prio, d->nr_prio, sizeof(*d->prio),
+ intc_handle_int_cmp, NULL);
+ sort(d->sense, d->nr_sense, sizeof(*d->sense),
+ intc_handle_int_cmp, NULL);
+
intc_subgroup_init(desc, d);
/* enable bits matching force_enable after registering irqs */
diff --git a/drivers/sh/maple/maple.c b/drivers/sh/maple/maple.c
index 5585f220e495..7df88e00c20f 100644
--- a/drivers/sh/maple/maple.c
+++ b/drivers/sh/maple/maple.c
@@ -863,10 +863,10 @@ static int __init maple_bus_init(void)
kmem_cache_destroy(maple_queue_cache);
cleanup_bothirqs:
- free_irq(HW_EVENT_VSYNC, 0);
+ free_irq(HW_EVENT_VSYNC, &maple_unsupported_device);
cleanup_irq:
- free_irq(HW_EVENT_MAPLE_DMA, 0);
+ free_irq(HW_EVENT_MAPLE_DMA, &maple_unsupported_device);
cleanup_dma:
free_pages((unsigned long) maple_sendbuf, MAPLE_DMA_PAGES);
diff --git a/drivers/slimbus/messaging.c b/drivers/slimbus/messaging.c
index e2dbe4a66b70..ee127ef42046 100644
--- a/drivers/slimbus/messaging.c
+++ b/drivers/slimbus/messaging.c
@@ -139,7 +139,7 @@ int slim_do_transfer(struct slim_controller *ctrl, struct slim_msg_txn *txn)
if (need_tid) {
ret = slim_alloc_txn_tid(ctrl, txn);
if (ret)
- return ret;
+ goto slim_xfer_err;
if (!txn->msg->comp)
txn->comp = &done;
diff --git a/drivers/soc/fsl/dpio/dpio-driver.c b/drivers/soc/fsl/dpio/dpio-driver.c
index d5198ffc4455..d8f28fb7c37a 100644
--- a/drivers/soc/fsl/dpio/dpio-driver.c
+++ b/drivers/soc/fsl/dpio/dpio-driver.c
@@ -88,6 +88,7 @@ static void unregister_dpio_irq_handlers(struct fsl_mc_device *dpio_dev)
/* clear the affinity hint */
irq_set_affinity_hint(irq->virq, NULL);
+ devm_free_irq(&dpio_dev->dev, irq->virq, &dpio_dev->dev);
}
static int register_dpio_irq_handlers(struct fsl_mc_device *dpio_dev, int cpu)
@@ -244,9 +245,9 @@ static int dpaa2_dpio_probe(struct fsl_mc_device *dpio_dev)
return 0;
-err_dpaa2_io_create:
- unregister_dpio_irq_handlers(dpio_dev);
err_register_dpio_irq:
+ dpaa2_io_down(priv->io);
+err_dpaa2_io_create:
fsl_mc_free_irqs(dpio_dev);
err_allocate_irqs:
dpio_disable(dpio_dev->mc_io, 0, dpio_dev->mc_handle);
@@ -276,10 +277,10 @@ static void dpaa2_dpio_remove(struct fsl_mc_device *dpio_dev)
priv = dev_get_drvdata(dev);
cpu = dpaa2_io_get_cpu(priv->io);
- dpaa2_io_down(priv->io);
-
dpio_teardown_irqs(dpio_dev);
+ dpaa2_io_down(priv->io);
+
cpumask_set_cpu(cpu, cpus_unused_mask);
err = dpio_open(dpio_dev->mc_io, 0, dpio_dev->obj_desc.id,
diff --git a/drivers/soc/fsl/qe/tsa.c b/drivers/soc/fsl/qe/tsa.c
index 4a88e54d25b9..502e2ffe8689 100644
--- a/drivers/soc/fsl/qe/tsa.c
+++ b/drivers/soc/fsl/qe/tsa.c
@@ -865,12 +865,12 @@ static int tsa_of_parse_tdms(struct tsa *tsa, struct device_node *np)
clk_put(tsa->tdm[i].l1rclk_clk);
}
if (tsa->tdm[i].l1tsync_clk) {
- clk_disable_unprepare(tsa->tdm[i].l1rsync_clk);
- clk_put(tsa->tdm[i].l1rsync_clk);
+ clk_disable_unprepare(tsa->tdm[i].l1tsync_clk);
+ clk_put(tsa->tdm[i].l1tsync_clk);
}
if (tsa->tdm[i].l1tclk_clk) {
- clk_disable_unprepare(tsa->tdm[i].l1rclk_clk);
- clk_put(tsa->tdm[i].l1rclk_clk);
+ clk_disable_unprepare(tsa->tdm[i].l1tclk_clk);
+ clk_put(tsa->tdm[i].l1tclk_clk);
}
}
return ret;
@@ -1039,12 +1039,12 @@ static void tsa_remove(struct platform_device *pdev)
clk_put(tsa->tdm[i].l1rclk_clk);
}
if (tsa->tdm[i].l1tsync_clk) {
- clk_disable_unprepare(tsa->tdm[i].l1rsync_clk);
- clk_put(tsa->tdm[i].l1rsync_clk);
+ clk_disable_unprepare(tsa->tdm[i].l1tsync_clk);
+ clk_put(tsa->tdm[i].l1tsync_clk);
}
if (tsa->tdm[i].l1tclk_clk) {
- clk_disable_unprepare(tsa->tdm[i].l1rclk_clk);
- clk_put(tsa->tdm[i].l1rclk_clk);
+ clk_disable_unprepare(tsa->tdm[i].l1tclk_clk);
+ clk_put(tsa->tdm[i].l1tclk_clk);
}
}
}
diff --git a/drivers/soc/loongson/loongson2_pm.c b/drivers/soc/loongson/loongson2_pm.c
index b8e5e1e3528a..1e2c897d055b 100644
--- a/drivers/soc/loongson/loongson2_pm.c
+++ b/drivers/soc/loongson/loongson2_pm.c
@@ -175,7 +175,7 @@ static int loongson2_pm_probe(struct platform_device *pdev)
return irq;
if (!device_property_read_u64(dev, "loongson,suspend-address", &suspend_addr))
- loongson_sysconf.suspend_addr = (u64)phys_to_virt(suspend_addr);
+ loongson_sysconf.suspend_addr = (unsigned long)phys_to_virt(suspend_addr);
else
dev_err(dev, "No loongson,suspend-address, could not support S3!\n");
diff --git a/drivers/soc/qcom/pmic_glink_altmode.c b/drivers/soc/qcom/pmic_glink_altmode.c
index 13c434f8d03a..465e5d2316a0 100644
--- a/drivers/soc/qcom/pmic_glink_altmode.c
+++ b/drivers/soc/qcom/pmic_glink_altmode.c
@@ -451,7 +451,7 @@ static void pmic_glink_altmode_sc8180xp_notify(struct pmic_glink_altmode *altmod
alt_port->mode = mode;
alt_port->hpd_state = hpd_state;
alt_port->hpd_irq = hpd_irq;
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
#define SC8280XP_DPAM_MASK 0x3f
@@ -502,7 +502,7 @@ static void pmic_glink_altmode_sc8280xp_notify(struct pmic_glink_altmode *altmod
alt_port->tbt_data = *tbt;
}
- schedule_work(&alt_port->work);
+ queue_work(system_freezable_wq, &alt_port->work);
}
static void pmic_glink_altmode_callback(const void *data, size_t len, void *priv)
diff --git a/drivers/soc/qcom/qcom-geni-se.c b/drivers/soc/qcom/qcom-geni-se.c
index 873bfbd6b2b7..6ac2bdc16984 100644
--- a/drivers/soc/qcom/qcom-geni-se.c
+++ b/drivers/soc/qcom/qcom-geni-se.c
@@ -1230,25 +1230,28 @@ EXPORT_SYMBOL_GPL(geni_se_resources_init);
* @se: Pointer to the serial engine structure.
* @fw: Pointer to the firmware image.
* @protocol: Expected serial engine protocol type.
+ * @fw_size_out: Non-NULL output parameter; receives the rounded, validated
+ * firmware word count on success.
*
* Identifies the appropriate firmware image or configuration required for a
* specific communication protocol instance running on a Qualcomm GENI
* controller. Validates the firmware size against the hardware PROG_RAM_DEPTH
* read from SE_HW_PARAM_2.
*
- * Return: pointer to a valid 'struct se_fw_hdr' if found, or NULL otherwise.
+ * Return: pointer to a valid 'const struct se_fw_hdr' if found, or NULL otherwise.
*/
-static struct se_fw_hdr *geni_find_protocol_fw(struct geni_se *se, const struct firmware *fw,
- enum geni_se_protocol_type protocol)
+static const struct se_fw_hdr *geni_find_protocol_fw(struct geni_se *se, const struct firmware *fw,
+ enum geni_se_protocol_type protocol,
+ u32 *fw_size_out)
{
struct device *dev = se->dev;
const struct elf32_hdr *ehdr;
const struct elf32_phdr *phdrs;
const struct elf32_phdr *phdr;
- struct se_fw_hdr *sefw;
+ const struct se_fw_hdr *sefw;
u32 fw_end, cfg_idx_end, cfg_val_end;
u32 prog_ram_depth;
- u16 fw_size;
+ u32 fw_size;
int i;
if (!fw || fw->size < sizeof(struct elf32_hdr))
@@ -1287,24 +1290,23 @@ static struct se_fw_hdr *geni_find_protocol_fw(struct geni_se *se, const struct
if (phdr->p_filesz < sizeof(struct se_fw_hdr))
continue;
- sefw = (struct se_fw_hdr *)(fw->data + phdr->p_offset);
+ sefw = (const struct se_fw_hdr *)(fw->data + phdr->p_offset);
fw_size = le16_to_cpu(sefw->fw_size_in_items);
- fw_end = le16_to_cpu(sefw->fw_offset) + fw_size * sizeof(u32);
- cfg_idx_end = le16_to_cpu(sefw->cfg_idx_offset) +
- le16_to_cpu(sefw->cfg_size_in_items) * sizeof(u8);
- cfg_val_end = le16_to_cpu(sefw->cfg_val_offset) +
- le16_to_cpu(sefw->cfg_size_in_items) * sizeof(u32);
if (le32_to_cpu(sefw->magic) != SE_MAGIC_NUM || le32_to_cpu(sefw->version) != 1)
continue;
- if (le32_to_cpu(sefw->serial_protocol) != protocol)
+ if (le16_to_cpu(sefw->serial_protocol) != protocol)
continue;
- if (fw_size % 2 != 0) {
+ if (fw_size % 2 != 0)
fw_size++;
- sefw->fw_size_in_items = cpu_to_le16(fw_size);
- }
+
+ fw_end = le16_to_cpu(sefw->fw_offset) + fw_size * sizeof(u32);
+ cfg_idx_end = le16_to_cpu(sefw->cfg_idx_offset) +
+ le16_to_cpu(sefw->cfg_size_in_items) * sizeof(u8);
+ cfg_val_end = le16_to_cpu(sefw->cfg_val_offset) +
+ le16_to_cpu(sefw->cfg_size_in_items) * sizeof(u32);
prog_ram_depth = FIELD_GET(PROG_RAM_DEPTH_MSK,
readl_relaxed(se->base + SE_HW_PARAM_2));
@@ -1320,6 +1322,7 @@ static struct se_fw_hdr *geni_find_protocol_fw(struct geni_se *se, const struct
continue;
}
+ *fw_size_out = fw_size;
return sefw;
}
@@ -1430,17 +1433,17 @@ static int geni_load_se_fw(struct geni_se *se, const struct firmware *fw,
{
const u32 *fw_data, *cfg_val_arr;
const u8 *cfg_idx_arr;
- u32 i, reg_value;
+ u32 i, reg_value, fw_size_in_items;
int ret;
- struct se_fw_hdr *hdr;
+ const struct se_fw_hdr *hdr;
- hdr = geni_find_protocol_fw(se, fw, protocol);
+ hdr = geni_find_protocol_fw(se, fw, protocol, &fw_size_in_items);
if (!hdr)
return -EINVAL;
- fw_data = (const u32 *)((u8 *)hdr + le16_to_cpu(hdr->fw_offset));
+ fw_data = (const u32 *)((const u8 *)hdr + le16_to_cpu(hdr->fw_offset));
cfg_idx_arr = (const u8 *)hdr + le16_to_cpu(hdr->cfg_idx_offset);
- cfg_val_arr = (const u32 *)((u8 *)hdr + le16_to_cpu(hdr->cfg_val_offset));
+ cfg_val_arr = (const u32 *)((const u8 *)hdr + le16_to_cpu(hdr->cfg_val_offset));
ret = geni_icc_set_bw(se);
if (ret)
@@ -1511,7 +1514,7 @@ static int geni_load_se_fw(struct geni_se *se, const struct firmware *fw,
/* Program RAM address space. */
memcpy_toio(se->base + SE_GENI_CFG_RAMN, fw_data,
- le16_to_cpu(hdr->fw_size_in_items) * sizeof(u32));
+ fw_size_in_items * sizeof(u32));
/* Put default values on GENI's output pads. */
writel_relaxed(0x1, se->base + GENI_FORCE_DEFAULT_REG);
diff --git a/drivers/soc/samsung/exynos-pmu.c b/drivers/soc/samsung/exynos-pmu.c
index efccdd63e40e..c764f35fbfc2 100644
--- a/drivers/soc/samsung/exynos-pmu.c
+++ b/drivers/soc/samsung/exynos-pmu.c
@@ -38,6 +38,8 @@ struct exynos_pmu_context {
unsigned long *in_cpuhp;
bool sys_insuspend;
bool sys_inreboot;
+ int cpuhp_prepare_state;
+ int cpuhp_online_state;
};
void __iomem *pmu_base_addr;
@@ -458,16 +460,55 @@ static int setup_cpuhp_and_cpuidle(struct device *dev)
gs101_cpuhp_pmu_online(cpu);
/* register CPU hotplug callbacks */
- cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "soc/exynos-pmu:prepare",
- gs101_cpuhp_pmu_online, NULL);
+ pmu_context->cpuhp_prepare_state = CPUHP_INVALID;
+ pmu_context->cpuhp_online_state = CPUHP_INVALID;
- cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "soc/exynos-pmu:online",
- NULL, gs101_cpuhp_pmu_offline);
+ ret = cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "soc/exynos-pmu:prepare",
+ gs101_cpuhp_pmu_online, NULL);
+ if (ret < 0)
+ return ret;
+
+ pmu_context->cpuhp_prepare_state = ret;
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "soc/exynos-pmu:online",
+ NULL, gs101_cpuhp_pmu_offline);
+ if (ret < 0)
+ goto clean_cpuhp_state_prepare;
+
+ pmu_context->cpuhp_online_state = ret;
/* register CPU PM notifiers for cpuidle */
- cpu_pm_register_notifier(&gs101_cpu_pm_notifier);
- register_reboot_notifier(&exynos_cpupm_reboot_nb);
- return 0;
+ ret = cpu_pm_register_notifier(&gs101_cpu_pm_notifier);
+ if (ret)
+ goto clean_cpuhp_states;
+
+ ret = register_reboot_notifier(&exynos_cpupm_reboot_nb);
+ if (!ret)
+ /* Success */
+ return ret;
+
+ cpu_pm_unregister_notifier(&gs101_cpu_pm_notifier);
+
+clean_cpuhp_states:
+ cpuhp_remove_state(pmu_context->cpuhp_online_state);
+ pmu_context->cpuhp_online_state = CPUHP_INVALID;
+
+clean_cpuhp_state_prepare:
+ cpuhp_remove_state(pmu_context->cpuhp_prepare_state);
+ pmu_context->cpuhp_prepare_state = CPUHP_INVALID;
+
+ return ret;
+}
+
+static void destroy_cpuhp_and_cpuidle(void)
+{
+ cpu_pm_unregister_notifier(&gs101_cpu_pm_notifier);
+ unregister_reboot_notifier(&exynos_cpupm_reboot_nb);
+
+ if (pmu_context->cpuhp_prepare_state != CPUHP_INVALID)
+ cpuhp_remove_state(pmu_context->cpuhp_prepare_state);
+ if (pmu_context->cpuhp_online_state != CPUHP_INVALID)
+ cpuhp_remove_state(pmu_context->cpuhp_online_state);
}
static int exynos_pmu_probe(struct platform_device *pdev)
@@ -541,8 +582,12 @@ static int exynos_pmu_probe(struct platform_device *pdev)
ret = devm_mfd_add_devices(dev, PLATFORM_DEVID_NONE, exynos_pmu_devs,
ARRAY_SIZE(exynos_pmu_devs), NULL, 0, NULL);
- if (ret)
+ if (ret) {
+ if (pmu_context->pmu_data && pmu_context->pmu_data->pmu_cpuhp)
+ destroy_cpuhp_and_cpuidle();
+
return ret;
+ }
if (devm_of_platform_populate(dev))
dev_err(dev, "Error populating children, reboot and poweroff might not work properly\n");
diff --git a/drivers/spi/atmel-quadspi.c b/drivers/spi/atmel-quadspi.c
index ec19fa0180f0..96ed734b5621 100644
--- a/drivers/spi/atmel-quadspi.c
+++ b/drivers/spi/atmel-quadspi.c
@@ -1590,12 +1590,14 @@ static int atmel_qspi_suspend(struct device *dev)
if (aq->caps->has_gclk) {
ret = atmel_qspi_sama7g5_suspend(aq);
- clk_disable_unprepare(aq->pclk);
- return ret;
+ if (ret) {
+ pm_runtime_put_autosuspend(dev);
+ return ret;
+ }
+ } else {
+ atmel_qspi_write(QSPI_CR_QSPIDIS, aq, QSPI_CR);
}
- atmel_qspi_write(QSPI_CR_QSPIDIS, aq, QSPI_CR);
-
pm_runtime_mark_last_busy(dev);
pm_runtime_force_suspend(dev);
@@ -1621,20 +1623,20 @@ static int atmel_qspi_resume(struct device *dev)
return ret;
}
- if (aq->caps->has_gclk)
- return aq->caps->init(aq);
-
ret = pm_runtime_force_resume(dev);
if (ret < 0)
return ret;
- atmel_qspi_init(aq);
-
- atmel_qspi_write(aq->scr, aq, QSPI_SCR);
+ if (aq->caps->has_gclk) {
+ ret = aq->caps->init(aq);
+ } else {
+ atmel_qspi_init(aq);
+ atmel_qspi_write(aq->scr, aq, QSPI_SCR);
+ }
pm_runtime_put_autosuspend(dev);
- return 0;
+ return ret;
}
static int atmel_qspi_runtime_suspend(struct device *dev)
diff --git a/drivers/spi/spi-amd.c b/drivers/spi/spi-amd.c
index 71a6e5c475b0..d3953de255be 100644
--- a/drivers/spi/spi-amd.c
+++ b/drivers/spi/spi-amd.c
@@ -322,6 +322,10 @@ static inline int amd_spi_fifo_xfer(struct amd_spi *amd_spi,
list_for_each_entry(xfer, &message->transfers,
transfer_list) {
+ /* Skip zero-length transfers to avoid len underflow below */
+ if (!xfer->len)
+ continue;
+
if (xfer->speed_hz)
amd_set_spi_freq(amd_spi, xfer->speed_hz);
else
@@ -335,6 +339,10 @@ static inline int amd_spi_fifo_xfer(struct amd_spi *amd_spi,
xfer->len--;
}
tx_len += xfer->len;
+ if (tx_len + rx_len > AMD_SPI_FIFO_SIZE) {
+ message->status = -EMSGSIZE;
+ goto fin_msg;
+ }
/* Write data into the FIFO. */
for (i = 0; i < xfer->len; i++)
@@ -344,8 +352,13 @@ static inline int amd_spi_fifo_xfer(struct amd_spi *amd_spi,
}
/* Store no. of bytes to be received from FIFO */
- if (xfer->rx_buf)
+ if (xfer->rx_buf) {
rx_len += xfer->len;
+ if (tx_len + rx_len > AMD_SPI_FIFO_SIZE) {
+ message->status = -EMSGSIZE;
+ goto fin_msg;
+ }
+ }
}
if (!buf) {
diff --git a/drivers/spi/spi-bcm-qspi.c b/drivers/spi/spi-bcm-qspi.c
index 0f96ac59da11..5031b2142479 100644
--- a/drivers/spi/spi-bcm-qspi.c
+++ b/drivers/spi/spi-bcm-qspi.c
@@ -1197,8 +1197,9 @@ static int bcm_qspi_exec_mem_op(struct spi_mem *mem,
u32 addr = 0, len;
u_char *buf;
+ /* cmd[] holds 1 opcode + 4 address + 1 dummy byte at most */
if (!op->data.nbytes || !op->addr.nbytes || op->addr.nbytes > 4 ||
- op->data.dir != SPI_MEM_DATA_IN)
+ op->dummy.nbytes > 1 || op->data.dir != SPI_MEM_DATA_IN)
return -EOPNOTSUPP;
buf = op->data.buf.in;
diff --git a/drivers/spi/spi-bcm2835.c b/drivers/spi/spi-bcm2835.c
index 8f8715809c7c..23606967f7d1 100644
--- a/drivers/spi/spi-bcm2835.c
+++ b/drivers/spi/spi-bcm2835.c
@@ -1297,7 +1297,9 @@ static int bcm2835_spi_setup(struct spi_device *spi)
}
for (i = 0; i < ARRAY_SIZE(pinctrl_compats); i++) {
- if (of_find_compatible_node(NULL, NULL, pinctrl_compats[i]))
+ struct device_node *np __free(device_node) =
+ of_find_compatible_node(NULL, NULL, pinctrl_compats[i]);
+ if (np)
break;
}
diff --git a/drivers/spi/spi-cadence-xspi.c b/drivers/spi/spi-cadence-xspi.c
index 39c868a5b171..3687853e4ce4 100644
--- a/drivers/spi/spi-cadence-xspi.c
+++ b/drivers/spi/spi-cadence-xspi.c
@@ -268,6 +268,7 @@
#define MRVL_XFER_FUNC_START BIT(0)
#define MRVL_XFER_QWORD_COUNT 32
#define MRVL_XFER_QWORD_BYTECOUNT 8
+#define MRVL_XFER_MAX_LEN (MRVL_XFER_QWORD_COUNT * MRVL_XFER_QWORD_BYTECOUNT)
#define MRVL_XSPI_POLL_TIMEOUT_US 1000
#define MRVL_XSPI_POLL_DELAY_US 10
@@ -332,6 +333,7 @@ struct cdns_xspi_dev {
int irq;
int cur_cs;
unsigned int sdmasize;
+ unsigned int sdma_xfer_len;
struct completion cmd_complete;
struct completion auto_cmd_complete;
@@ -346,7 +348,7 @@ struct cdns_xspi_dev {
u8 hw_num_banks;
const struct cdns_xspi_driver_data *driver_data;
- void (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
+ int (*sdma_handler)(struct cdns_xspi_dev *cdns_xspi);
void (*set_interrupts_handler)(struct cdns_xspi_dev *cdns_xspi, bool enabled);
bool xfer_in_progress;
@@ -496,7 +498,7 @@ static inline void cdns_xspi_sdma_write(struct cdns_xspi_dev *cdns_xspi, size_t
iowrite8_rep(dst, (const u8 *)buf + offset, len);
}
-static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+static int cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
{
u32 sdma_size, sdma_trd_info;
u8 sdma_dir;
@@ -505,6 +507,13 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
switch (sdma_dir) {
case CDNS_XSPI_SDMA_DIR_READ:
cdns_xspi_sdma_read(cdns_xspi, sdma_size);
@@ -514,6 +523,8 @@ static void cdns_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
cdns_xspi_sdma_write(cdns_xspi, sdma_size);
break;
}
+
+ return 0;
}
static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
@@ -559,6 +570,7 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->in_buffer = op->data.buf.in;
cdns_xspi->out_buffer = op->data.buf.out;
+ cdns_xspi->sdma_xfer_len = op->data.nbytes;
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
@@ -567,7 +579,11 @@ static int cdns_xspi_send_stig_command(struct cdns_xspi_dev *cdns_xspi,
cdns_xspi->set_interrupts_handler(cdns_xspi, false);
return -EIO;
}
- cdns_xspi->sdma_handler(cdns_xspi);
+ ret = cdns_xspi->sdma_handler(cdns_xspi);
+ if (ret) {
+ cdns_xspi->set_interrupts_handler(cdns_xspi, false);
+ return ret;
+ }
}
wait_for_completion(&cdns_xspi->cmd_complete);
@@ -950,7 +966,7 @@ static void m_iowriteq(void __iomem *addr, const void *buf, int len)
}
}
-static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
+static int marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
{
u32 sdma_size, sdma_trd_info;
u8 sdma_dir;
@@ -959,6 +975,13 @@ static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
sdma_trd_info = readl(cdns_xspi->iobase + CDNS_XSPI_SDMA_TRD_INFO_REG);
sdma_dir = FIELD_GET(CDNS_XSPI_SDMA_DIR, sdma_trd_info);
+ if (sdma_size > cdns_xspi->sdma_xfer_len) {
+ dev_err(cdns_xspi->dev,
+ "SDMA size %u exceeds the requested length %u\n",
+ sdma_size, cdns_xspi->sdma_xfer_len);
+ return -EINVAL;
+ }
+
switch (sdma_dir) {
case CDNS_XSPI_SDMA_DIR_READ:
m_ioreadq(cdns_xspi->sdmabase,
@@ -970,6 +993,8 @@ static void marvell_xspi_sdma_handle(struct cdns_xspi_dev *cdns_xspi)
cdns_xspi->out_buffer, sdma_size);
break;
}
+
+ return 0;
}
static const struct spi_controller_mem_ops marvell_xspi_mem_ops = {
@@ -1083,7 +1108,7 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller,
struct spi_device *spi = m->spi;
struct spi_transfer *t = NULL;
- const unsigned int max_len = MRVL_XFER_QWORD_BYTECOUNT * MRVL_XFER_QWORD_COUNT;
+ const unsigned int max_len = MRVL_XFER_MAX_LEN;
int current_transfer_len;
int cs = spi_get_chipselect(spi, 0);
int cs_change = 0;
@@ -1106,13 +1131,16 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller,
list_for_each_entry(t, &m->transfers, transfer_list) {
u8 *txd = (u8 *) t->tx_buf;
u8 *rxd = (u8 *) t->rx_buf;
- u8 data[10];
+ u8 data[MRVL_XFER_MAX_LEN] = {0};
u32 cmd_regs[6];
if (!txd)
txd = data;
- cdns_xspi->in_buffer = txd + 1;
+ if (rxd)
+ cdns_xspi->in_buffer = rxd;
+ else
+ cdns_xspi->in_buffer = data;
cdns_xspi->out_buffer = txd + 1;
while (t->len) {
@@ -1131,14 +1159,18 @@ static int cdns_xspi_transfer_one_message_b0(struct spi_controller *controller,
cdns_xspi_prepare_transfer(cs, 1, current_transfer_len - 1,
cmd_regs);
cdns_xspi_trigger_command(cdns_xspi, cmd_regs);
+ cdns_xspi->sdma_xfer_len = current_transfer_len - 1;
if (!cdns_xspi_is_sdma_ready(cdns_xspi, true))
return -EIO;
- cdns_xspi->sdma_handler(cdns_xspi);
+ if (cdns_xspi->sdma_handler(cdns_xspi))
+ return -EIO;
if (!cdns_xspi_is_stig_ready(cdns_xspi, true))
return -EIO;
- cdns_xspi->in_buffer += current_transfer_len;
- cdns_xspi->out_buffer += current_transfer_len;
+ if (rxd)
+ cdns_xspi->in_buffer += current_transfer_len;
+ if (t->tx_buf)
+ cdns_xspi->out_buffer += current_transfer_len;
}
if (rxd) {
diff --git a/drivers/spi/spi-ch341.c b/drivers/spi/spi-ch341.c
index 6448a44a8b67..56a5caa5975a 100644
--- a/drivers/spi/spi-ch341.c
+++ b/drivers/spi/spi-ch341.c
@@ -189,6 +189,12 @@ static int ch341_probe(struct usb_interface *intf,
ctrl->transfer_one = ch341_transfer_one;
ctrl->set_cs = ch341_set_cs;
ctrl->auto_runtime_pm = false;
+ /*
+ * The CH341A SPI_STREAM command clocks data in both directions on
+ * every transfer, so the driver always needs a TX and an RX buffer;
+ * let the SPI core supply dummy buffers for half-duplex transfers.
+ */
+ ctrl->flags = SPI_CONTROLLER_MUST_TX | SPI_CONTROLLER_MUST_RX;
usb_set_intfdata(intf, ch341);
diff --git a/drivers/spi/spi-jcore.c b/drivers/spi/spi-jcore.c
index a75cd61ec7a3..fc87ad98e4e7 100644
--- a/drivers/spi/spi-jcore.c
+++ b/drivers/spi/spi-jcore.c
@@ -84,6 +84,8 @@ static void jcore_spi_baudrate(struct jcore_spi *hw, int speed)
{
if (speed == hw->speed_hz)
return;
+ if (speed <= 0)
+ speed = 1;
hw->speed_hz = speed;
if (speed >= hw->clock_freq / 2)
hw->speed_reg = 0;
diff --git a/drivers/spi/spi-ljca.c b/drivers/spi/spi-ljca.c
index 0c6e6248d8ba..b1babd43adda 100644
--- a/drivers/spi/spi-ljca.c
+++ b/drivers/spi/spi-ljca.c
@@ -105,7 +105,13 @@ static int ljca_spi_read_write(struct ljca_spi_dev *ljca_spi, const u8 *w_data,
(u8 *)r_packet, LJCA_SPI_BUF_SIZE);
if (ret < 0)
return ret;
- else if (ret < sizeof(*r_packet) || r_packet->len <= 0)
+
+ if (ret < sizeof(*r_packet))
+ return -EIO;
+
+ /* write acks carry no payload */
+ if (!r_packet->len || r_packet->len > len ||
+ (r_data && ret < sizeof(*r_packet) + r_packet->len))
return -EIO;
if (r_data)
diff --git a/drivers/spi/spi-npcm-fiu.c b/drivers/spi/spi-npcm-fiu.c
index 96dfef4d9067..9c0a655a54b3 100644
--- a/drivers/spi/spi-npcm-fiu.c
+++ b/drivers/spi/spi-npcm-fiu.c
@@ -343,6 +343,10 @@ static int npcm_fiu_uma_read(struct spi_mem *mem,
u32 val;
u32 i;
+ /* data_reg holds at most 16 bytes */
+ if (data_size > sizeof(data_reg))
+ return -EOPNOTSUPP;
+
regmap_update_bits(fiu->regmap, NPCM_FIU_UMA_CTS,
NPCM_FIU_UMA_CTS_DEV_NUM,
(spi_get_chipselect(mem->spi, 0) <<
@@ -398,6 +402,10 @@ static int npcm_fiu_uma_write(struct spi_mem *mem,
u32 val;
u32 i;
+ /* data_reg holds at most 16 bytes */
+ if (data_size > sizeof(data_reg))
+ return -EOPNOTSUPP;
+
regmap_update_bits(fiu->regmap, NPCM_FIU_UMA_CTS,
NPCM_FIU_UMA_CTS_DEV_NUM,
(spi_get_chipselect(mem->spi, 0) <<
diff --git a/drivers/spi/spi-qpic-snand.c b/drivers/spi/spi-qpic-snand.c
index 05efe6313b7f..ae4a3f2a35e1 100644
--- a/drivers/spi/spi-qpic-snand.c
+++ b/drivers/spi/spi-qpic-snand.c
@@ -411,6 +411,8 @@ static int qcom_spi_ecc_init_ctx_pipelined(struct nand_device *nand)
dev_dbg(snandc->dev, "ECC strength: %u bits per %u bytes\n",
ecc_cfg->strength, ecc_cfg->step_size);
+ snandc->qspi->ecc = ecc_cfg;
+
return 0;
err_free_ecc_cfg:
@@ -427,6 +429,7 @@ static void qcom_spi_ecc_cleanup_ctx_pipelined(struct nand_device *nand)
kfree(snandc->qspi->oob_buf);
snandc->qspi->oob_buf = NULL;
+ snandc->qspi->ecc = NULL;
kfree(ecc_cfg);
}
diff --git a/drivers/spi/spi-stm32.c b/drivers/spi/spi-stm32.c
index be88e62075af..605b847d8773 100644
--- a/drivers/spi/spi-stm32.c
+++ b/drivers/spi/spi-stm32.c
@@ -2511,6 +2511,7 @@ static int stm32_spi_probe(struct platform_device *pdev)
gen_pool_free(spi->sram_pool,
(unsigned long)spi->sram_rx_buf,
spi->sram_rx_buf_size);
+ spi->sram_rx_buf = NULL;
dev_warn(&pdev->dev,
"failed to request rx mdma channel, DMA only\n");
}
@@ -2548,7 +2549,7 @@ static int stm32_spi_probe(struct platform_device *pdev)
if (spi->mdma_rx)
dma_release_channel(spi->mdma_rx);
err_pool_free:
- if (spi->sram_pool)
+ if (spi->sram_rx_buf)
gen_pool_free(spi->sram_pool, (unsigned long)spi->sram_rx_buf,
spi->sram_rx_buf_size);
if (spi->dma_rx)
diff --git a/drivers/spi/spi-zynq-qspi.c b/drivers/spi/spi-zynq-qspi.c
index d762aaf452af..bfcbcfe85b91 100644
--- a/drivers/spi/spi-zynq-qspi.c
+++ b/drivers/spi/spi-zynq-qspi.c
@@ -229,11 +229,10 @@ static bool zynq_qspi_supports_op(struct spi_mem *mem,
return false;
/*
- * The number of address bytes should be equal to or less than 3 bytes.
+ * The number of address bytes should be equal to or less than 4 bytes.
*/
- if (op->addr.nbytes > 3)
+ if (op->addr.nbytes > 4)
return false;
-
return true;
}
@@ -526,6 +525,7 @@ static int zynq_qspi_exec_mem_op(struct spi_mem *mem,
const struct spi_mem_op *op)
{
struct zynq_qspi *xqspi = spi_controller_get_devdata(mem->spi->controller);
+ u8 opaddr[4] __aligned(4);
int err = 0, i;
u8 *tmpbuf;
@@ -547,12 +547,13 @@ static int zynq_qspi_exec_mem_op(struct spi_mem *mem,
}
if (op->addr.nbytes) {
- for (i = 0; i < op->addr.nbytes; i++) {
- xqspi->txbuf[i] = op->addr.val >>
+ for (i = 0; i < op->addr.nbytes; i++)
+ opaddr[i] = op->addr.val >>
(8 * (op->addr.nbytes - i - 1));
- }
reinit_completion(&xqspi->data_completion);
+ xqspi->txbuf = opaddr;
+
xqspi->rxbuf = NULL;
xqspi->tx_bytes = op->addr.nbytes;
xqspi->rx_bytes = op->addr.nbytes;
diff --git a/drivers/staging/greybus/uart.c b/drivers/staging/greybus/uart.c
index 172cf900eaf0..2c47418a11ef 100644
--- a/drivers/staging/greybus/uart.c
+++ b/drivers/staging/greybus/uart.c
@@ -53,7 +53,6 @@ struct gb_tty {
spinlock_t read_lock;
spinlock_t write_lock;
struct async_icount iocount;
- struct async_icount oldcount;
wait_queue_head_t wioctl;
struct mutex mutex;
u8 ctrlin; /* input control lines */
@@ -643,11 +642,16 @@ static int wait_serial_change(struct gb_tty *gb_tty, unsigned long arg)
if (!(arg & (TIOCM_DSR | TIOCM_RI | TIOCM_CD)))
return -EINVAL;
- do {
+ spin_lock_irq(&gb_tty->read_lock);
+ old = gb_tty->iocount;
+ spin_unlock_irq(&gb_tty->read_lock);
+
+ add_wait_queue(&gb_tty->wioctl, &wait);
+ for (;;) {
+ set_current_state(TASK_INTERRUPTIBLE);
+
spin_lock_irq(&gb_tty->read_lock);
- old = gb_tty->oldcount;
new = gb_tty->iocount;
- gb_tty->oldcount = new;
spin_unlock_irq(&gb_tty->read_lock);
if ((arg & TIOCM_DSR) && (old.dsr != new.dsr))
@@ -657,18 +661,20 @@ static int wait_serial_change(struct gb_tty *gb_tty, unsigned long arg)
if ((arg & TIOCM_RI) && (old.rng != new.rng))
break;
- add_wait_queue(&gb_tty->wioctl, &wait);
- set_current_state(TASK_INTERRUPTIBLE);
- schedule();
- remove_wait_queue(&gb_tty->wioctl, &wait);
if (gb_tty->disconnected) {
- if (arg & TIOCM_CD)
- break;
retval = -ENODEV;
- } else if (signal_pending(current)) {
+ break;
+ }
+
+ schedule();
+
+ if (signal_pending(current)) {
retval = -ERESTARTSYS;
+ break;
}
- } while (!retval);
+ }
+ __set_current_state(TASK_RUNNING);
+ remove_wait_queue(&gb_tty->wioctl, &wait);
return retval;
}
diff --git a/drivers/staging/media/av7110/sp8870.c b/drivers/staging/media/av7110/sp8870.c
index 29fb4934c039..be62933cef45 100644
--- a/drivers/staging/media/av7110/sp8870.c
+++ b/drivers/staging/media/av7110/sp8870.c
@@ -315,7 +315,6 @@ static int sp8870_init(struct dvb_frontend *fe)
sp8870_wake_up(state);
if (state->initialised)
return 0;
- state->initialised = 1;
dprintk("initialising frontend...\n");
@@ -353,6 +352,8 @@ static int sp8870_init(struct dvb_frontend *fe)
sp8870_writereg(state, 0x0D00, 0x010);
sp8870_writereg(state, 0x0D01, 0x000);
+ state->initialised = 1;
+
return 0;
}
@@ -401,7 +402,7 @@ static int sp8870_read_ber(struct dvb_frontend *fe, u32 *ber)
if (ret < 0)
return -EIO;
- tmp = ret << 6;
+ tmp |= ret << 6;
if (tmp >= 0x3FFF0)
tmp = ~0;
diff --git a/drivers/staging/media/imx/imx-media-csc-scaler.c b/drivers/staging/media/imx/imx-media-csc-scaler.c
index 00fcdd4d0487..359d973d297f 100644
--- a/drivers/staging/media/imx/imx-media-csc-scaler.c
+++ b/drivers/staging/media/imx/imx-media-csc-scaler.c
@@ -147,6 +147,7 @@ static void device_run(void *_ctx)
v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
v4l2_m2m_job_finish(priv->m2m_dev, ctx->fh.m2m_ctx);
+ kfree(run);
}
/*
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index c18bf352b199..be29d5a11c2e 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -34,7 +34,7 @@ struct dummy_buf {
static u32 get_output_size(u32 width, u32 height)
{
- return ALIGN(width * height, SZ_64K);
+ return ALIGN(ALIGN(width, 32) * ALIGN(height, 32), SZ_64K);
}
u32 amvdec_get_output_size(struct amvdec_session *sess)
diff --git a/drivers/staging/rtl8723bs/hal/sdio_ops.c b/drivers/staging/rtl8723bs/hal/sdio_ops.c
index da2d9088ab5a..b7a4ab0bedc9 100644
--- a/drivers/staging/rtl8723bs/hal/sdio_ops.c
+++ b/drivers/staging/rtl8723bs/hal/sdio_ops.c
@@ -644,6 +644,13 @@ static struct recv_buf *sd_recv_rxfifo(struct adapter *adapter, u32 size)
struct recv_priv *recv_priv;
struct recv_buf *recvbuf;
+ /* RX0_REQ_LEN is device-reported; the skb buffer is only
+ * MAX_RECVBUF_SZ bytes, reject bogus lengths instead of
+ * overflowing past it.
+ */
+ if (size > MAX_RECVBUF_SZ)
+ return NULL;
+
/* Patch for some SDIO Host 4 bytes issue */
/* ex. RK3188 */
readsize = round_up(size, 4);
diff --git a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
index f80dbe2ca7e4..b0284c2e2e74 100644
--- a/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
+++ b/drivers/thermal/intel/int340x_thermal/processor_thermal_device.c
@@ -179,17 +179,21 @@ static int proc_thermal_get_zone_temp(struct thermal_zone_device *zone,
{
int cpu;
int curr_temp, ret;
-
- *temp = 0;
+ bool temp_valid = false;
for_each_online_cpu(cpu) {
ret = intel_tcc_get_temp(cpu, &curr_temp, false);
if (ret < 0)
return ret;
- if (!*temp || curr_temp > *temp)
+ if (!temp_valid || curr_temp > *temp) {
*temp = curr_temp;
+ temp_valid = true;
+ }
}
+ if (!temp_valid)
+ return -ENODATA;
+
*temp *= 1000;
return 0;
diff --git a/drivers/thermal/intel/intel_powerclamp.c b/drivers/thermal/intel/intel_powerclamp.c
index bd7fd98dc310..8026651133d5 100644
--- a/drivers/thermal/intel/intel_powerclamp.c
+++ b/drivers/thermal/intel/intel_powerclamp.c
@@ -289,9 +289,10 @@ static int window_size_set(const char *arg, const struct kernel_param *kp)
pr_err("Out of recommended window size %lu, between 2-10\n",
new_window_size);
ret = -EINVAL;
+ goto exit_win;
}
- window_size = clamp(new_window_size, 2ul, 10ul);
+ window_size = new_window_size;
smp_mb();
exit_win:
diff --git a/drivers/tty/serial/8250/8250_exar.c b/drivers/tty/serial/8250/8250_exar.c
index 836792e861bf..2e7adf6a410c 100644
--- a/drivers/tty/serial/8250/8250_exar.c
+++ b/drivers/tty/serial/8250/8250_exar.c
@@ -231,8 +231,9 @@ struct exar8250_platform {
int (*rs485_config)(struct uart_port *port, struct ktermios *termios,
struct serial_rs485 *rs485);
const struct serial_rs485 *rs485_supported;
- int (*register_gpio)(struct pci_dev *pcidev, struct uart_8250_port *port);
- void (*unregister_gpio)(struct uart_8250_port *port);
+ int (*register_gpio)(struct exar8250 *priv, struct pci_dev *pcidev,
+ struct uart_8250_port *port);
+ void (*unregister_gpio)(struct exar8250 *priv);
};
/**
@@ -256,6 +257,7 @@ struct exar8250 {
struct exar8250_board *board;
struct eeprom_93cx6 eeprom;
void __iomem *virt;
+ struct platform_device *gpio;
int line[];
};
@@ -1167,22 +1169,21 @@ static const struct software_node exar_gpio_node = {
.properties = exar_gpio_properties,
};
-static int xr17v35x_register_gpio(struct pci_dev *pcidev, struct uart_8250_port *port)
+static int xr17v35x_register_gpio(struct exar8250 *priv, struct pci_dev *pcidev,
+ struct uart_8250_port *port)
{
if (pcidev->vendor == PCI_VENDOR_ID_EXAR)
- port->port.private_data =
- __xr17v35x_register_gpio(pcidev, &exar_gpio_node);
+ priv->gpio = __xr17v35x_register_gpio(pcidev, &exar_gpio_node);
return 0;
}
-static void xr17v35x_unregister_gpio(struct uart_8250_port *port)
+static void xr17v35x_unregister_gpio(struct exar8250 *priv)
{
- if (!port->port.private_data)
- return;
-
- __xr17v35x_unregister_gpio(port->port.private_data);
- port->port.private_data = NULL;
+ if (priv->gpio) {
+ __xr17v35x_unregister_gpio(priv->gpio);
+ priv->gpio = NULL;
+ }
}
static int sealevel_rs485_config(struct uart_port *port, struct ktermios *termios,
@@ -1279,8 +1280,9 @@ static const struct software_node iot2040_gpio_node = {
.properties = iot2040_gpio_properties,
};
-static int iot2040_register_gpio(struct pci_dev *pcidev,
- struct uart_8250_port *port)
+static int iot2040_register_gpio(struct exar8250 *priv,
+ struct pci_dev *pcidev,
+ struct uart_8250_port *port)
{
u8 __iomem *p = port->port.membase;
@@ -1289,8 +1291,7 @@ static int iot2040_register_gpio(struct pci_dev *pcidev,
writeb(IOT2040_UARTS_ENABLE, p + UART_EXAR_MPIOLVL_15_8);
writeb(IOT2040_UARTS_GPIO_HI_MODE, p + UART_EXAR_MPIOSEL_15_8);
- port->port.private_data =
- __xr17v35x_register_gpio(pcidev, &iot2040_gpio_node);
+ priv->gpio = __xr17v35x_register_gpio(pcidev, &iot2040_gpio_node);
return 0;
}
@@ -1367,7 +1368,7 @@ pci_xr17v35x_setup(struct exar8250 *priv, struct pci_dev *pcidev,
/* Setup Multipurpose Input/Output pins. */
setup_gpio(pcidev, p);
- ret = platform->register_gpio(pcidev, port);
+ ret = platform->register_gpio(priv, pcidev, port);
}
return ret;
@@ -1377,9 +1378,8 @@ static void pci_xr17v35x_exit(struct pci_dev *pcidev)
{
const struct exar8250_platform *platform = exar_get_platform();
struct exar8250 *priv = pci_get_drvdata(pcidev);
- struct uart_8250_port *port = serial8250_get_port(priv->line[0]);
- platform->unregister_gpio(port);
+ platform->unregister_gpio(priv);
}
static inline void exar_misc_clear(struct exar8250 *priv)
diff --git a/drivers/tty/serial/8250/8250_lpss.c b/drivers/tty/serial/8250/8250_lpss.c
index 776ec1ef29d6..325e587ae2c4 100644
--- a/drivers/tty/serial/8250/8250_lpss.c
+++ b/drivers/tty/serial/8250/8250_lpss.c
@@ -183,9 +183,8 @@ static int ehl_serial_setup(struct lpss8250 *lpss, struct uart_port *port)
static void ehl_serial_exit(struct lpss8250 *lpss)
{
- struct uart_8250_port *up = serial8250_get_port(lpss->data.line);
-
- up->dma = NULL;
+ if (lpss->data.line >= 0)
+ serial8250_get_port(lpss->data.line)->dma = NULL;
}
#ifdef CONFIG_SERIAL_8250_DMA
@@ -329,6 +328,8 @@ static int lpss8250_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return ret;
lpss->board = (struct lpss8250_board *)id->driver_data;
+ /* for tests in lpss->board->exit() */
+ lpss->data.line = -1;
memset(&uart, 0, sizeof(struct uart_8250_port));
diff --git a/drivers/tty/serial/8250/8250_omap.c b/drivers/tty/serial/8250/8250_omap.c
index ceecb39fb82d..70252258344c 100644
--- a/drivers/tty/serial/8250/8250_omap.c
+++ b/drivers/tty/serial/8250/8250_omap.c
@@ -711,6 +711,9 @@ static int omap_8250_startup(struct uart_port *port)
struct uart_8250_dma *dma = &priv->omap8250_dma;
int ret;
+ /* ISR and runtime PM need the line; probe sets it only after registration */
+ priv->line = port->line;
+
#ifdef CONFIG_PM
up->capabilities |= UART_CAP_RPM;
#endif
diff --git a/drivers/tty/serial/8250/8250_port.c b/drivers/tty/serial/8250/8250_port.c
index b01baff71f60..c961b969021e 100644
--- a/drivers/tty/serial/8250/8250_port.c
+++ b/drivers/tty/serial/8250/8250_port.c
@@ -2428,8 +2428,8 @@ void serial8250_do_shutdown(struct uart_port *port)
synchronize_irq(port->irq);
- serial8250_release_dma(up);
- up->dma = NULL;
+ if (up->dma)
+ serial8250_release_dma(up);
scoped_guard(uart_port_lock_irqsave, port) {
if (port->flags & UPF_FOURPORT) {
diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c
index e67f2ac454b6..c5e3b120060f 100644
--- a/drivers/tty/serial/amba-pl011.c
+++ b/drivers/tty/serial/amba-pl011.c
@@ -1420,10 +1420,6 @@ static void pl011_rs485_tx_start(struct uart_amba_port *uap)
struct uart_port *port = &uap->port;
u32 cr;
- if (uap->rs485_tx_state == WAIT_AFTER_RTS) {
- uap->rs485_tx_state = SEND;
- return;
- }
if (uap->rs485_tx_state == WAIT_AFTER_SEND ||
uap->rs485_tx_state == WAIT_AFTER_SEND_DELAY) {
hrtimer_try_to_cancel(&uap->trigger_stop_tx);
@@ -1462,6 +1458,10 @@ static void pl011_start_tx(struct uart_port *port)
if ((uap->port.rs485.flags & SER_RS485_ENABLED) &&
uap->rs485_tx_state != SEND) {
+ /* trigger_start_tx starts TX once delay_rts_before_send passed */
+ if (uap->rs485_tx_state == WAIT_AFTER_RTS)
+ return;
+
pl011_rs485_tx_start(uap);
if (uap->rs485_tx_state == WAIT_AFTER_RTS)
return;
@@ -1478,8 +1478,10 @@ static enum hrtimer_restart pl011_trigger_start_tx(struct hrtimer *t)
unsigned long flags;
uart_port_lock_irqsave(&uap->port, &flags);
- if (uap->rs485_tx_state == WAIT_AFTER_RTS)
+ if (uap->rs485_tx_state == WAIT_AFTER_RTS) {
+ uap->rs485_tx_state = SEND;
pl011_start_tx(&uap->port);
+ }
uart_port_unlock_irqrestore(&uap->port, flags);
return HRTIMER_NORESTART;
@@ -2397,8 +2399,31 @@ static int pl011_rs485_config(struct uart_port *port, struct ktermios *termios,
struct uart_amba_port *uap =
container_of(port, struct uart_amba_port, port);
- if (port->rs485.flags & SER_RS485_ENABLED)
+ if ((port->rs485.flags & SER_RS485_ENABLED) &&
+ !(rs485->flags & SER_RS485_ENABLED)) {
+ bool tx_pending = uap->rs485_tx_state == WAIT_AFTER_RTS;
+ u32 cr = pl011_read(uap, REG_CR);
+
+ /*
+ * Leaving RS485 mode, which only enables the transmitter while
+ * sending. Don't stop it, as that stalls a DMA transfer still
+ * in flight. Drop pending deferred starts and stops and keep
+ * it enabled.
+ */
+ hrtimer_try_to_cancel(&uap->trigger_start_tx);
+ hrtimer_try_to_cancel(&uap->trigger_stop_tx);
+ uap->rs485_tx_state = OFF;
+
+ cr |= UART011_CR_TXE | UART011_CR_RXE;
+ pl011_write(cr, uap, REG_CR);
+
+ /* Start TX that was still waiting for delay_rts_before_send */
+ if (tx_pending && !uart_tx_stopped(port) &&
+ !pl011_dma_tx_start(uap))
+ pl011_start_tx_pio(uap);
+ } else if (port->rs485.flags & SER_RS485_ENABLED) {
pl011_rs485_tx_stop(uap);
+ }
/* Make sure auto RTS is disabled */
if (rs485->flags & SER_RS485_ENABLED) {
diff --git a/drivers/tty/serial/pmac_zilog.c b/drivers/tty/serial/pmac_zilog.c
index e3a919328695..f49d6052d11b 100644
--- a/drivers/tty/serial/pmac_zilog.c
+++ b/drivers/tty/serial/pmac_zilog.c
@@ -444,7 +444,7 @@ static irqreturn_t pmz_interrupt(int irq, void *dev_id)
skip_a:
uart_port_unlock(&uap_a->port);
if (push)
- tty_flip_buffer_push(&uap->port.state->port);
+ tty_flip_buffer_push(&uap_a->port.state->port);
if (!uap_b)
goto out;
@@ -469,7 +469,7 @@ static irqreturn_t pmz_interrupt(int irq, void *dev_id)
skip_b:
uart_port_unlock(&uap_b->port);
if (push)
- tty_flip_buffer_push(&uap->port.state->port);
+ tty_flip_buffer_push(&uap_b->port.state->port);
out:
return rc;
diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c
index 296079c16fdb..fba588559595 100644
--- a/drivers/tty/serial/sc16is7xx.c
+++ b/drivers/tty/serial/sc16is7xx.c
@@ -501,7 +501,9 @@ static bool sc16is7xx_regmap_volatile(struct device *dev, unsigned int reg)
{
switch (reg) {
case SC16IS7XX_RHR_REG: /* Shared address space with THR & DLL */
+ case SC16IS7XX_IER_REG: /* Shared address space with DLH */
case SC16IS7XX_IIR_REG: /* Shared address space with FCR & EFR */
+ case SC16IS7XX_MCR_REG: /* Shared address space with XON1 */
case SC16IS7XX_LSR_REG: /* Shared address space with XON2 */
case SC16IS7XX_MSR_REG: /* Shared address space with TCR & XOFF1 */
case SC16IS7XX_SPR_REG: /* Shared address space with TLR & XOFF2 */
@@ -1516,7 +1518,7 @@ static int sc16is7xx_setup_channel(struct sc16is7xx_one *one, int i,
/* Initialize port data */
port->type = PORT_SC16IS7XX;
port->fifosize = SC16IS7XX_FIFO_SIZE;
- port->flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY;
+ port->flags = UPF_FIXED_TYPE | UPF_LOW_LATENCY | UPF_SOFT_FLOW;
port->iotype = UPIO_BUS;
port->rs485_config = sc16is7xx_config_rs485;
port->rs485_supported = sc16is7xx_rs485_supported;
diff --git a/drivers/ufs/core/ufs_bsg.c b/drivers/ufs/core/ufs_bsg.c
index 156637dd76c8..baf6ae917d36 100644
--- a/drivers/ufs/core/ufs_bsg.c
+++ b/drivers/ufs/core/ufs_bsg.c
@@ -73,6 +73,9 @@ static int ufs_bsg_exec_advanced_rpmb_req(struct ufs_hba *hba, struct bsg_job *j
if (hba->ufs_version < ufshci_version(4, 0) || !hba->dev_info.b_advanced_rpmb_en)
return -EINVAL;
+ if (job->request_len < sizeof(*rpmb_request))
+ return -EINVAL;
+
if (rpmb_request->ehs_req.length != 2 || rpmb_request->ehs_req.ehs_type != 1)
return -EINVAL;
@@ -140,7 +143,12 @@ static int ufs_bsg_request(struct bsg_job *job)
int ret;
bool rpmb = false;
- bsg_reply->reply_payload_rcv_len = 0;
+ memset(job->reply, 0, job->reply_len);
+
+ if (job->request_len < sizeof(*bsg_request)) {
+ ret = -EINVAL;
+ goto out_complete;
+ }
ufshcd_rpm_get_sync(hba);
@@ -171,6 +179,10 @@ static int ufs_bsg_request(struct bsg_job *job)
break;
case UPIU_TRANSACTION_UIC_CMD:
memcpy(&uc, &bsg_request->upiu_req.uc, UIC_CMD_SIZE);
+ if (uc.argument2 & MASK_UIC_COMMAND_RESULT) {
+ ret = -EINVAL;
+ break;
+ }
ret = ufshcd_send_bsg_uic_cmd(hba, &uc);
if (ret)
dev_err(hba->dev, "send uic cmd: error code %d\n", ret);
@@ -194,15 +206,14 @@ static int ufs_bsg_request(struct bsg_job *job)
out:
ufshcd_rpm_put_sync(hba);
kfree(buff);
+out_complete:
bsg_reply->result = ret;
- /* complete the job here only if no error */
- if (ret == 0) {
+ if (ret == 0)
job->reply_len = rpmb ? sizeof(struct ufs_rpmb_reply) :
sizeof(struct ufs_bsg_reply);
- bsg_job_done(job, ret, bsg_reply->reply_payload_rcv_len);
- }
+ bsg_job_done(job, ret, bsg_reply->reply_payload_rcv_len);
- return ret;
+ return 0;
}
/**
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index ca8329052cbc..dfbcf036d356 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -1360,13 +1360,13 @@ int ufshcd_pause_command_processing(struct ufs_hba *hba, u64 timeout_us)
int ret = 0;
mutex_lock(&hba->host->scan_mutex);
- blk_mq_quiesce_tagset(&hba->host->tag_set);
down_write(&hba->clk_scaling_lock);
+ blk_mq_quiesce_tagset(&hba->host->tag_set);
if (ufshcd_wait_for_pending_cmds(hba, timeout_us)) {
ret = -EBUSY;
- up_write(&hba->clk_scaling_lock);
blk_mq_unquiesce_tagset(&hba->host->tag_set);
+ up_write(&hba->clk_scaling_lock);
mutex_unlock(&hba->host->scan_mutex);
}
@@ -1383,8 +1383,8 @@ void ufshcd_resume_command_processing(struct ufs_hba *hba)
__releases(&hba->clk_scaling_lock)
__releases(&hba->host->scan_mutex)
{
- up_write(&hba->clk_scaling_lock);
blk_mq_unquiesce_tagset(&hba->host->tag_set);
+ up_write(&hba->clk_scaling_lock);
mutex_unlock(&hba->host->scan_mutex);
}
@@ -1461,16 +1461,16 @@ static int ufshcd_clock_scaling_prepare(struct ufs_hba *hba, u64 timeout_us)
* clock scaling is in progress
*/
mutex_lock(&hba->host->scan_mutex);
- blk_mq_quiesce_tagset(&hba->host->tag_set);
mutex_lock(&hba->wb_mutex);
down_write(&hba->clk_scaling_lock);
+ blk_mq_quiesce_tagset(&hba->host->tag_set);
if (!hba->clk_scaling.is_allowed ||
ufshcd_wait_for_pending_cmds(hba, timeout_us)) {
ret = -EBUSY;
+ blk_mq_unquiesce_tagset(&hba->host->tag_set);
up_write(&hba->clk_scaling_lock);
mutex_unlock(&hba->wb_mutex);
- blk_mq_unquiesce_tagset(&hba->host->tag_set);
mutex_unlock(&hba->host->scan_mutex);
goto out;
}
@@ -1487,9 +1487,9 @@ static void ufshcd_clock_scaling_unprepare(struct ufs_hba *hba, int err)
__releases(&hba->wb_mutex)
__releases(&hba->host->scan_mutex)
{
+ blk_mq_unquiesce_tagset(&hba->host->tag_set);
up_write(&hba->clk_scaling_lock);
mutex_unlock(&hba->wb_mutex);
- blk_mq_unquiesce_tagset(&hba->host->tag_set);
mutex_unlock(&hba->host->scan_mutex);
/* Enable Write Booster if current gear requires it else disable it */
@@ -4359,9 +4359,12 @@ int ufshcd_dme_set_attr(struct ufs_hba *hba, u32 attr_sel,
do {
/* for peer attributes we retry upon failure */
ret = ufshcd_send_uic_cmd(hba, &uic_cmd);
- if (ret)
+ if (ret) {
dev_dbg(hba->dev, "%s: attr-id 0x%x val 0x%x error code %d\n",
set, UIC_GET_ATTR_ID(attr_sel), mib_val, ret);
+ uic_cmd.argument2 &= ~MASK_UIC_COMMAND_RESULT;
+ uic_cmd.argument3 = mib_val;
+ }
} while (ret && peer && --retries);
if (ret)
@@ -4426,9 +4429,11 @@ int ufshcd_dme_get_attr(struct ufs_hba *hba, u32 attr_sel,
do {
/* for peer attributes we retry upon failure */
ret = ufshcd_send_uic_cmd(hba, &uic_cmd);
- if (ret)
+ if (ret) {
dev_dbg(hba->dev, "%s: attr-id 0x%x error code %d\n",
get, UIC_GET_ATTR_ID(attr_sel), ret);
+ uic_cmd.argument2 &= ~MASK_UIC_COMMAND_RESULT;
+ }
} while (ret && peer && --retries);
if (ret)
@@ -5997,6 +6002,37 @@ static void ufshcd_clear_polled(struct ufs_hba *hba,
}
}
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long __ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask,
+ bool from_irq)
+ __must_hold(&hba->outstanding_lock)
+{
+ unsigned long completed_reqs = mask & hba->outstanding_reqs;
+
+ if (from_irq) {
+ /* Do not complete polled requests from interrupt context. */
+ ufshcd_clear_polled(hba, &completed_reqs);
+ }
+ hba->outstanding_reqs &= ~completed_reqs;
+
+ return completed_reqs;
+}
+
+/*
+ * Return a bitmask of the transfer requests cleared from
+ * @hba->outstanding_reqs.
+ */
+static unsigned long ufshcd_clear_outstanding_reqs(struct ufs_hba *hba,
+ unsigned long mask)
+{
+ guard(spinlock_irqsave)(&hba->outstanding_lock);
+ return __ufshcd_clear_outstanding_reqs(hba, mask, false);
+}
+
/*
* Return: > 0 if one or more commands have been completed or 0 if no
* requests have been completed.
@@ -6007,6 +6043,7 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
unsigned long completed_reqs, flags;
u32 tr_doorbell;
struct ufs_hw_queue *hwq;
+ bool from_irq;
if (hba->mcq_enabled) {
hwq = &hba->uhq[queue_num];
@@ -6016,15 +6053,9 @@ static int ufshcd_poll(struct Scsi_Host *shost, unsigned int queue_num)
spin_lock_irqsave(&hba->outstanding_lock, flags);
tr_doorbell = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL);
- completed_reqs = ~tr_doorbell & hba->outstanding_reqs;
- WARN_ONCE(completed_reqs & ~hba->outstanding_reqs,
- "completed: %#lx; outstanding: %#lx\n", completed_reqs,
- hba->outstanding_reqs);
- if (queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT) {
- /* Do not complete polled requests from interrupt context. */
- ufshcd_clear_polled(hba, &completed_reqs);
- }
- hba->outstanding_reqs &= ~completed_reqs;
+ from_irq = queue_num == UFSHCD_POLL_FROM_INTERRUPT_CONTEXT;
+ completed_reqs = __ufshcd_clear_outstanding_reqs(hba, ~tr_doorbell,
+ from_irq);
spin_unlock_irqrestore(&hba->outstanding_lock, flags);
if (completed_reqs)
@@ -7905,7 +7936,8 @@ static bool ufshcd_clear_lu_cmds(struct request *req, void *priv)
return true;
}
- ufshcd_compl_one_cqe(hba, tag, NULL);
+ if (ufshcd_clear_outstanding_reqs(hba, BIT(tag)))
+ ufshcd_compl_one_cqe(hba, tag, NULL);
return true;
}
@@ -8058,7 +8090,6 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
struct ufshcd_lrb *lrbp = scsi_cmd_priv(cmd);
unsigned long flags;
int err = FAILED;
- bool outstanding;
u32 reg;
ufshcd_hold(hba);
@@ -8098,15 +8129,6 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
}
hba->req_abort_count++;
- if (!hba->mcq_enabled && !(reg & (1 << tag))) {
- /* only execute this code in single doorbell mode */
- dev_err(hba->dev,
- "%s: cmd was completed, but without a notifying intr, tag = %d",
- __func__, tag);
- __ufshcd_transfer_req_compl(hba, 1UL << tag);
- goto release;
- }
-
/*
* Task abort to the device W-LUN is illegal. When this command
* will fail, due to spec violation, scsi err handling next step
@@ -8153,11 +8175,8 @@ static int ufshcd_abort(struct scsi_cmnd *cmd)
* Clear the corresponding bit from outstanding_reqs since the command
* has been aborted successfully.
*/
- spin_lock_irqsave(&hba->outstanding_lock, flags);
- outstanding = __test_and_clear_bit(tag, &hba->outstanding_reqs);
- spin_unlock_irqrestore(&hba->outstanding_lock, flags);
-
- if (outstanding)
+ if (ufshcd_clear_outstanding_reqs(hba, BIT(tag)) &&
+ ufshcd_is_scsi_cmd(cmd))
ufshcd_release_scsi_cmd(hba, cmd);
err = SUCCESS;
@@ -9579,12 +9598,10 @@ static enum scsi_timeout_action ufshcd_eh_timed_out(struct scsi_cmnd *scmd)
*/
if (!test_bit(SCMD_STATE_COMPLETE, &scmd->state)) {
if (!hba->mcq_enabled) {
- unsigned long flags;
struct request *rq = scsi_cmd_to_rq(scmd);
- spin_lock_irqsave(&hba->outstanding_lock, flags);
- __clear_bit(rq->tag, &hba->outstanding_reqs);
- spin_unlock_irqrestore(&hba->outstanding_lock, flags);
+ if (!ufshcd_clear_outstanding_reqs(hba, BIT(rq->tag)))
+ return SCSI_EH_DONE;
}
if (ufshcd_is_scsi_cmd(scmd)) {
diff --git a/drivers/usb/class/cdc-acm.c b/drivers/usb/class/cdc-acm.c
index 0d3f815b2f1e..489575d40bd1 100644
--- a/drivers/usb/class/cdc-acm.c
+++ b/drivers/usb/class/cdc-acm.c
@@ -782,6 +782,7 @@ static void acm_port_shutdown(struct tty_port *port)
urb = usb_get_from_anchor(&acm->delayed);
if (!urb)
break;
+ usb_put_urb(urb);
wb = urb->context;
wb->use = false;
usb_autopm_put_interface_async(acm->control);
@@ -1710,7 +1711,7 @@ static int acm_resume(struct usb_interface *intf)
urb = usb_get_from_anchor(&acm->delayed);
if (!urb)
break;
-
+ usb_put_urb(urb);
acm_start_wb(acm, urb->context);
}
@@ -1818,6 +1819,9 @@ static const struct usb_device_id acm_ids[] = {
{ USB_DEVICE(0x11ca, 0x0201), /* VeriFone Mx870 Gadget Serial */
.driver_info = SINGLE_RX_URB,
},
+ { USB_DEVICE(0x11ca, 0x0219), /* VeriFone Vx820 Gadget Serial */
+ .driver_info = SINGLE_RX_URB,
+ },
{ USB_DEVICE(0x1901, 0x0006), /* GE Healthcare Patient Monitor UI Controller */
.driver_info = DISABLE_ECHO, /* DISABLE ECHO in termios flag */
},
diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index bf660b15a0f6..9af0184276c1 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -410,7 +410,6 @@ static int usb_probe_interface(struct device *dev)
return error;
err:
- usb_set_intfdata(intf, NULL);
intf->needs_remote_wakeup = 0;
intf->condition = USB_INTERFACE_UNBOUND;
@@ -497,7 +496,6 @@ static int usb_unbind_interface(struct device *dev)
} else {
intf->needs_altsetting0 = 1;
}
- usb_set_intfdata(intf, NULL);
intf->condition = USB_INTERFACE_UNBOUND;
intf->needs_remote_wakeup = 0;
@@ -644,6 +642,7 @@ void usb_driver_release_interface(struct usb_driver *driver,
} else {
device_lock(dev);
usb_unbind_interface(dev);
+ dev_set_drvdata(dev, NULL);
dev->driver = NULL;
device_unlock(dev);
}
diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c
index b5b577f0b931..a4bcbe9c913c 100644
--- a/drivers/usb/core/quirks.c
+++ b/drivers/usb/core/quirks.c
@@ -612,6 +612,9 @@ static const struct usb_device_id usb_quirk_list[] = {
/* ezcap401 - BOS descriptor fetch hangs at SuperSpeed Plus */
{ USB_DEVICE(0x32ed, 0x0401), .driver_info = USB_QUIRK_NO_BOS },
+ /* GameSir G7 SE */
+ { USB_DEVICE(0x3537, 0x10a0), .driver_info = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE },
+
/* DELL USB GEN2 */
{ USB_DEVICE(0x413c, 0xb062), .driver_info = USB_QUIRK_NO_LPM | USB_QUIRK_RESET_RESUME },
diff --git a/drivers/usb/fotg210/fotg210-udc.c b/drivers/usb/fotg210/fotg210-udc.c
index d9e024873a42..127a259c8d1e 100644
--- a/drivers/usb/fotg210/fotg210-udc.c
+++ b/drivers/usb/fotg210/fotg210-udc.c
@@ -661,7 +661,7 @@ static void fotg210_set_feature(struct fotg210_udc *fotg210,
case USB_RECIP_ENDPOINT: {
u8 epnum;
epnum = le16_to_cpu(ctrl->wIndex) & USB_ENDPOINT_NUMBER_MASK;
- if (epnum)
+ if (epnum && epnum < FOTG210_MAX_NUM_EP)
fotg210_set_epnstall(fotg210->ep[epnum]);
else
fotg210_set_cxstall(fotg210);
@@ -677,8 +677,8 @@ static void fotg210_set_feature(struct fotg210_udc *fotg210,
static void fotg210_clear_feature(struct fotg210_udc *fotg210,
struct usb_ctrlrequest *ctrl)
{
- struct fotg210_ep *ep =
- fotg210->ep[ctrl->wIndex & USB_ENDPOINT_NUMBER_MASK];
+ u8 epnum = le16_to_cpu(ctrl->wIndex) & USB_ENDPOINT_NUMBER_MASK;
+ struct fotg210_ep *ep;
switch (ctrl->bRequestType & USB_RECIP_MASK) {
case USB_RECIP_DEVICE:
@@ -688,7 +688,12 @@ static void fotg210_clear_feature(struct fotg210_udc *fotg210,
fotg210_set_cxdone(fotg210);
break;
case USB_RECIP_ENDPOINT:
- if (ctrl->wIndex & USB_ENDPOINT_NUMBER_MASK) {
+ if (epnum >= FOTG210_MAX_NUM_EP) {
+ fotg210_request_error(fotg210);
+ break;
+ }
+ if (epnum) {
+ ep = fotg210->ep[epnum];
if (ep->wedged) {
fotg210_set_cxdone(fotg210);
break;
@@ -744,8 +749,8 @@ static void fotg210_get_status(struct fotg210_udc *fotg210,
fotg210->ep0_data = cpu_to_le16(0);
break;
case USB_RECIP_ENDPOINT:
- epnum = ctrl->wIndex & USB_ENDPOINT_NUMBER_MASK;
- if (epnum)
+ epnum = le16_to_cpu(ctrl->wIndex) & USB_ENDPOINT_NUMBER_MASK;
+ if (epnum && epnum < FOTG210_MAX_NUM_EP)
fotg210->ep0_data =
cpu_to_le16(fotg210_is_epnstall(fotg210->ep[epnum])
<< USB_ENDPOINT_HALT);
diff --git a/drivers/usb/gadget/legacy/inode.c b/drivers/usb/gadget/legacy/inode.c
index ce61348d9f2b..d15a48bdd759 100644
--- a/drivers/usb/gadget/legacy/inode.c
+++ b/drivers/usb/gadget/legacy/inode.c
@@ -613,7 +613,7 @@ ep_read_iter(struct kiocb *iocb, struct iov_iter *to)
return -EBADMSG;
}
- buf = kmalloc(len, GFP_KERNEL);
+ buf = kmalloc(len, GFP_KERNEL | __GFP_NOWARN);
if (unlikely(!buf)) {
mutex_unlock(&epdata->lock);
return -ENOMEM;
@@ -675,7 +675,7 @@ ep_write_iter(struct kiocb *iocb, struct iov_iter *from)
return -EBADMSG;
}
- buf = kmalloc(len, GFP_KERNEL);
+ buf = kmalloc(len, GFP_KERNEL | __GFP_NOWARN);
if (unlikely(!buf)) {
mutex_unlock(&epdata->lock);
return -ENOMEM;
diff --git a/drivers/usb/host/octeon-hcd.c b/drivers/usb/host/octeon-hcd.c
index 256e459ff6d8..4929c6a8e733 100644
--- a/drivers/usb/host/octeon-hcd.c
+++ b/drivers/usb/host/octeon-hcd.c
@@ -708,6 +708,7 @@ static int cvmx_usb_initialize(struct device *dev,
int channel;
int divisor;
int retries = 0;
+ int status;
union cvmx_usbcx_hcfg usbcx_hcfg;
union cvmx_usbnx_clk_ctl usbn_clk_ctl;
union cvmx_usbcx_gintsts usbc_gintsts;
@@ -872,7 +873,8 @@ static int cvmx_usb_initialize(struct device *dev,
if (usbcx_gahbcfg.u32 == 0xffffffff || usbcx_gahbcfg.u32 == 0) {
dev_err(dev, "USB core is not responding (GHWCFG3=0x%08x)\n",
usbcx_gahbcfg.u32);
- return -ENODEV;
+ status = -ENODEV;
+ goto err_shutdown;
}
usbcx_gahbcfg.u32 = 0;
@@ -961,14 +963,27 @@ static int cvmx_usb_initialize(struct device *dev,
dev_dbg(dev, "gintsts after reset: 0x%x\n", (int)usbc_gintsts.u32);
if (!usbc_gintsts.s.disconnint && !usbc_gintsts.s.prtint)
return 0;
- if (retries++ >= 5)
- return -EAGAIN;
+ if (retries++ >= 5) {
+ status = -EAGAIN;
+ goto err_shutdown;
+ }
dev_info(dev, "controller reset failed (gintsts=0x%x) - retrying\n",
(int)usbc_gintsts.u32);
msleep(50);
cvmx_usb_shutdown(usb);
msleep(50);
goto retry;
+
+err_shutdown:
+ /*
+ * Steps 2b-2e above selected the reference clock, enabled
+ * USBN_CLK_CTL and took the PHY out of reset, and the caller only
+ * frees the HCD, so hand the block back before giving up. The pipe
+ * lists cvmx_usb_shutdown() checks are initialised by the caller
+ * before it gets here, so it cannot refuse with -EBUSY.
+ */
+ cvmx_usb_shutdown(usb);
+ return status;
}
/**
@@ -1923,6 +1938,7 @@ static void cvmx_usb_schedule(struct octeon_hcd *usb, int is_sof)
struct cvmx_usb_pipe *pipe;
int need_sof;
u64 min_due;
+ bool due_now;
enum cvmx_usb_transfer ttype;
if (usb->init_flags & CVMX_USB_INITIALIZE_FLAGS_NO_DMA) {
@@ -1964,15 +1980,32 @@ static void cvmx_usb_schedule(struct octeon_hcd *usb, int is_sof)
*/
need_sof = 0;
min_due = ~0ull;
+ due_now = false;
for (ttype = CVMX_USB_TRANSFER_CONTROL;
ttype <= CVMX_USB_TRANSFER_INTERRUPT; ttype++) {
list_for_each_entry(pipe, &usb->active_pipes[ttype], node) {
+ /*
+ * Do not sleep past a pipe that is only retried from a
+ * SOF: a periodic one that is due but was not started
+ * (no idle channel, closed split window, another pipe's
+ * split active), or any pipe waiting for the frame its
+ * split complete may go out in. Other control and bulk
+ * pipes are left out: their next_tx_frame is not updated
+ * on completion, so they always look due.
+ */
+ if (pipe->split_sc_frame != -1 ||
+ (pipe->next_tx_frame <= usb->frame_number &&
+ (ttype == CVMX_USB_TRANSFER_ISOCHRONOUS ||
+ ttype == CVMX_USB_TRANSFER_INTERRUPT)))
+ due_now = true;
if (pipe->next_tx_frame > usb->frame_number &&
pipe->next_tx_frame < min_due)
min_due = pipe->next_tx_frame;
}
}
- if (min_due != ~0ull) {
+ if (due_now) {
+ need_sof = 1;
+ } else if (min_due != ~0ull) {
u64 delta = min_due - usb->frame_number;
/*
diff --git a/drivers/usb/host/xhci-pci-prom21.c b/drivers/usb/host/xhci-pci-prom21.c
index 6486f4a09345..5f088a47d79f 100644
--- a/drivers/usb/host/xhci-pci-prom21.c
+++ b/drivers/usb/host/xhci-pci-prom21.c
@@ -111,6 +111,9 @@ static void prom21_xhci_remove(struct pci_dev *dev)
}
static const struct pci_device_id pci_ids[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43F7) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43F9) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FA) },
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FC) },
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FD) },
{ /* end: all zeroes */ }
diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c
index a8889081ae82..b407676808fc 100644
--- a/drivers/usb/host/xhci-pci.c
+++ b/drivers/usb/host/xhci-pci.c
@@ -701,6 +701,9 @@ static const struct pci_device_id pci_ids_renesas[] = {
/* handled by xhci-pci-prom21 if enabled */
static const struct pci_device_id pci_ids_prom21[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43F7) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43F9) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FA) },
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FC) },
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_PROM21_XHCI_43FD) },
{ /* end: all zeroes */ }
diff --git a/drivers/usb/host/xhci-pci.h b/drivers/usb/host/xhci-pci.h
index 11f435f94322..533efc7db801 100644
--- a/drivers/usb/host/xhci-pci.h
+++ b/drivers/usb/host/xhci-pci.h
@@ -4,6 +4,9 @@
#ifndef XHCI_PCI_H
#define XHCI_PCI_H
+#define PCI_DEVICE_ID_AMD_PROM21_XHCI_43F7 0x43f7
+#define PCI_DEVICE_ID_AMD_PROM21_XHCI_43F9 0x43f9
+#define PCI_DEVICE_ID_AMD_PROM21_XHCI_43FA 0x43fa
#define PCI_DEVICE_ID_AMD_PROM21_XHCI_43FC 0x43fc
#define PCI_DEVICE_ID_AMD_PROM21_XHCI_43FD 0x43fd
diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index ec278a9f9540..fffd901b59e7 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -3544,15 +3544,13 @@ static u32 xhci_td_remainder(struct xhci_hcd *xhci, int transferred,
static int xhci_align_td(struct xhci_hcd *xhci, struct urb *urb, u32 enqd_len,
- u32 *trb_buff_len, struct xhci_segment *seg)
+ u32 *trb_buff_len, struct xhci_segment *seg, u32 max_pkt)
{
struct device *dev = xhci_to_hcd(xhci)->self.sysdev;
unsigned int unalign;
- unsigned int max_pkt;
u32 new_buff_len;
size_t len;
- max_pkt = xhci_usb_endpoint_maxp(urb->dev, urb->ep);
unalign = (enqd_len + *trb_buff_len) % max_pkt;
/* we got lucky, last normal TRB data on segment is packet aligned */
@@ -3697,9 +3695,8 @@ int xhci_queue_bulk_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
if (enqd_len + trb_buff_len < full_len) {
field |= TRB_CHAIN;
if (trb_is_link(ring->enqueue + 1)) {
- if (xhci_align_td(xhci, urb, enqd_len,
- &trb_buff_len,
- ring->enq_seg)) {
+ if (xhci_align_td(xhci, urb, enqd_len, &trb_buff_len,
+ ring->enq_seg, ring->bounce_buf_len)) {
send_addr = ring->enq_seg->bounce_dma;
/* TD bounced at least, and last on this seg */
td->bounce_seg = ring->enq_seg;
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index a9e47e178c28..28bfe5fd0b6f 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -4435,10 +4435,13 @@ static int xhci_setup_device(struct usb_hcd *hcd, struct usb_device *udev,
dev_warn(&udev->dev, "Device not responding to setup %s.\n", act);
mutex_unlock(&xhci->mutex);
- ret = xhci_disable_and_free_slot(xhci, udev->slot_id);
- if (!ret) {
- if (xhci_alloc_dev(hcd, udev) == 1)
- xhci_setup_addressable_virt_dev(xhci, udev);
+
+ if (!virt_dev->sideband) {
+ ret = xhci_disable_and_free_slot(xhci, udev->slot_id);
+ if (!ret) {
+ if (xhci_alloc_dev(hcd, udev) == 1)
+ xhci_setup_addressable_virt_dev(xhci, udev);
+ }
}
kfree(command->completion);
kfree(command);
diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c
index 828b1be3fee2..65506f6e1502 100644
--- a/drivers/usb/serial/option.c
+++ b/drivers/usb/serial/option.c
@@ -2443,6 +2443,7 @@ static const struct usb_device_id option_ids[] = {
{ USB_DEVICE_AND_INTERFACE_INFO(0x04b7, 0x8217, 0xff, 0xff, 0x30) }, /* Compal EXM-G1x */
{ USB_DEVICE_AND_INTERFACE_INFO(0x04b7, 0x8217, 0xff, 0xff, 0x40) },
{ USB_DEVICE_AND_INTERFACE_INFO(0x04b7, 0x8217, 0xff, 0xff, 0x60) },
+ { USB_DEVICE_INTERFACE_CLASS(0x1286, 0x4e3c, 0xff) }, /* ASR modem */
{ USB_DEVICE(0x1508, 0x1001), /* Fibocom NL668 (IOT version) */
.driver_info = RSVD(4) | RSVD(5) | RSVD(6) },
{ USB_DEVICE(0x1782, 0x4d10) }, /* Fibocom L610 (AT mode) */
@@ -2510,6 +2511,10 @@ static const struct usb_device_id option_ids[] = {
.driver_info = RSVD(5) },
{ USB_DEVICE_INTERFACE_CLASS(0x33f8, 0x1003, 0xff), /* Rolling RW135R-GL (laptop MBIM) */
.driver_info = RSVD(5) },
+ { USB_DEVICE_AND_INTERFACE_INFO(0x33f8, 0x9300, 0xff, 0xff, 0x30) }, /* Rolling RN947R */
+ { USB_DEVICE_AND_INTERFACE_INFO(0x33f8, 0x9300, 0xff, 0xff, 0x40) },
+ { USB_DEVICE_AND_INTERFACE_INFO(0x33f8, 0x9300, 0xff, 0xff, 0x10),
+ .driver_info = ZLP },
{ USB_DEVICE_INTERFACE_CLASS(0x3466, 0x3301, 0xff) }, /* TDTECH MT5710-CN */
{ USB_DEVICE_AND_INTERFACE_INFO(0x3731, 0x0100, 0xff, 0xff, 0x30) }, /* NetPrisma LCUK54-WWD for Global */
{ USB_DEVICE_AND_INTERFACE_INFO(0x3731, 0x0100, 0xff, 0x00, 0x40) },
diff --git a/drivers/usb/serial/sierra.c b/drivers/usb/serial/sierra.c
index 6e443aacae07..f2cc7f364d20 100644
--- a/drivers/usb/serial/sierra.c
+++ b/drivers/usb/serial/sierra.c
@@ -972,12 +972,13 @@ static int sierra_submit_delayed_urbs(struct usb_serial_port *port)
intfdata->in_flight--;
usb_unanchor_urb(urb);
kfree(urb->transfer_buffer);
- usb_free_urb(urb);
spin_lock(&portdata->lock);
portdata->outstanding_urbs--;
spin_unlock(&portdata->lock);
}
+
+ usb_put_urb(urb);
}
if (ec)
diff --git a/drivers/usb/serial/usb_wwan.c b/drivers/usb/serial/usb_wwan.c
index a183cc4515ed..6beb4062bda3 100644
--- a/drivers/usb/serial/usb_wwan.c
+++ b/drivers/usb/serial/usb_wwan.c
@@ -401,6 +401,7 @@ void usb_wwan_close(struct usb_serial_port *port)
urb = usb_get_from_anchor(&portdata->delayed);
if (!urb)
break;
+ usb_put_urb(urb);
unbusy_queued_urb(urb, portdata);
usb_autopm_put_interface_async(serial->interface);
}
@@ -578,6 +579,8 @@ static int usb_wwan_submit_delayed_urbs(struct usb_serial_port *port)
if (!urb)
break;
+ usb_put_urb(urb);
+
err = usb_submit_urb(urb, GFP_ATOMIC);
if (err) {
dev_err(&port->dev, "%s: submit urb failed: %d\n",
diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c
index c1450639c795..264338fd594a 100644
--- a/drivers/usb/typec/ucsi/ucsi.c
+++ b/drivers/usb/typec/ucsi/ucsi.c
@@ -1519,7 +1519,8 @@ static void ucsi_handle_connector_change(struct work_struct *work)
if (con->partner && (change & UCSI_CONSTAT_PARTNER_CHANGE)) {
ucsi_partner_change(con);
- ucsi_altmode_update_active(con);
+ if (con->partner_altmode[0])
+ ucsi_altmode_update_active(con);
/* Complete pending data role swap */
if (!completion_done(&con->complete))
diff --git a/drivers/usb/typec/ucsi/ucsi_ccg.c b/drivers/usb/typec/ucsi/ucsi_ccg.c
index 91c2958a708c..3f12db95c7f4 100644
--- a/drivers/usb/typec/ucsi/ucsi_ccg.c
+++ b/drivers/usb/typec/ucsi/ucsi_ccg.c
@@ -135,6 +135,8 @@ struct version_format {
#define NVIDIA_FTB_DP_OFFSET (2)
#define NVIDIA_FTB_DBG_OFFSET (3)
+#define CCG_READ_MAX_RETRIES 4
+
struct version_info {
struct version_format base;
struct version_format app;
@@ -255,6 +257,7 @@ static int ccg_read(struct ucsi_ccg *uc, u16 rab, u8 *data, u32 len)
};
u32 rlen, rem_len = len, max_read_len = len;
int status;
+ int retry_count;
/* check any max_read_len limitation on i2c adapter */
if (quirks && quirks->max_read_len)
@@ -266,7 +269,17 @@ static int ccg_read(struct ucsi_ccg *uc, u16 rab, u8 *data, u32 len)
rlen = min_t(u16, rem_len, max_read_len);
msgs[1].len = rlen;
put_unaligned_le16(rab, buf);
- status = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
+
+ for (retry_count = 0; retry_count < CCG_READ_MAX_RETRIES; retry_count++) {
+ status = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
+
+ /* Only retry on -EREMOTEIO, and not after the last attempt */
+ if (status != -EREMOTEIO || retry_count == CCG_READ_MAX_RETRIES - 1)
+ break;
+
+ fsleep(1000);
+ }
+
if (status < 0) {
dev_err(uc->dev, "i2c_transfer failed %d\n", status);
pm_runtime_put_sync(uc->dev);
@@ -384,14 +397,28 @@ static int ucsi_ccg_init(struct ucsi_ccg *uc)
return -ETIMEDOUT;
}
-static void ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data)
+static int ucsi_ccg_update_get_current_cam_cmd(struct ucsi_ccg *uc, u8 *data)
{
u8 cam, new_cam;
cam = data[0];
+ if (cam >= UCSI_MAX_ALTMODES) {
+ dev_err(uc->dev, "PPM returned invalid alternate mode index %u\n",
+ cam);
+ return -EPROTO;
+ }
+
new_cam = uc->orig[cam].linked_idx;
+ if (new_cam >= UCSI_MAX_ALTMODES) {
+ dev_err(uc->dev, "invalid linked alternate mode index %u for %u\n",
+ new_cam, cam);
+ return -EPROTO;
+ }
+
uc->updated[new_cam].active_idx = cam;
data[0] = new_cam;
+
+ return 0;
}
static bool ucsi_ccg_update_altmodes(struct ucsi *ucsi,
@@ -636,8 +663,10 @@ static int ucsi_ccg_sync_control(struct ucsi *ucsi, u64 command, u32 *cci,
switch (UCSI_COMMAND(command)) {
case UCSI_GET_CURRENT_CAM:
- if (uc->has_multiple_dp)
- ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)data);
+ if (!ret && uc->has_multiple_dp &&
+ (*cci & UCSI_CCI_COMMAND_COMPLETE) &&
+ !(*cci & UCSI_CCI_ERROR))
+ ret = ucsi_ccg_update_get_current_cam_cmd(uc, (u8 *)data);
break;
case UCSI_GET_ALTERNATE_MODES:
if (UCSI_ALTMODE_RECIPIENT(command) == UCSI_RECIPIENT_SOP) {
diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index 5fe0621b5fbd..04180035a7fe 100644
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -1644,7 +1644,7 @@ set_report_output(u32 size, int index, struct mlx5_vhca_qp *qp,
int i;
buf_offset = index * qp->max_msg_size;
- if (WARN_ON(buf_offset + size >= qp->recv_buf.npages * PAGE_SIZE ||
+ if (WARN_ON(buf_offset + size > qp->recv_buf.npages * PAGE_SIZE ||
(nent > qp->max_msg_size / entry_size)))
return;
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index 7f14dd46de17..df8848087681 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -423,9 +423,6 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset,
pos >= vdev->msix_offset + vdev->msix_size))
return -EINVAL;
- if (count == 8)
- return -EINVAL;
-
io = vfio_pci_core_get_iomap(vdev, bar);
if (IS_ERR(io))
return PTR_ERR(io);
diff --git a/drivers/video/fbdev/atmel_lcdfb.c b/drivers/video/fbdev/atmel_lcdfb.c
index 9a0cc09004b7..bed11a652d14 100644
--- a/drivers/video/fbdev/atmel_lcdfb.c
+++ b/drivers/video/fbdev/atmel_lcdfb.c
@@ -1007,7 +1007,9 @@ static int atmel_lcdfb_of_init(struct atmel_lcdfb_info *sinfo)
if (ret < 0)
goto put_display_node;
- fb_add_videomode(&fb_vm, &info->modelist);
+ ret = fb_add_videomode(&fb_vm, &info->modelist);
+ if (ret < 0)
+ goto put_display_node;
put_display_node:
of_node_put(display_np);
@@ -1075,8 +1077,13 @@ static int atmel_lcdfb_probe(struct platform_device *pdev)
}
atmel_lcdfb_start_clock(sinfo);
- modelist = list_first_entry(&info->modelist,
- struct fb_modelist, list);
+ modelist = list_first_entry_or_null(&info->modelist,
+ struct fb_modelist, list);
+ if (!modelist) {
+ dev_err(dev, "no video mode available\n");
+ ret = -EINVAL;
+ goto stop_clk;
+ }
fb_videomode_to_var(&info->var, &modelist->mode);
atmel_lcdfb_check_var(&info->var, info);
diff --git a/drivers/video/fbdev/core/bitblit.c b/drivers/video/fbdev/core/bitblit.c
index 39f44258d793..323a9e79a684 100644
--- a/drivers/video/fbdev/core/bitblit.c
+++ b/drivers/video/fbdev/core/bitblit.c
@@ -18,6 +18,7 @@
#include <linux/console.h>
#include <asm/types.h>
#include "fbcon.h"
+#include "fbcon_rotate.h"
/*
* Accelerated handlers.
@@ -150,7 +151,9 @@ static void bit_putcs(struct vc_data *vc, struct fb_info *info,
const unsigned short *s, int count, int yy, int xx,
int fg, int bg)
{
+ struct fbcon_par *par = info->fbcon_par;
struct fb_image image;
+ u32 vyres = GETVYRES(par->p, info);
u32 width = DIV_ROUND_UP(vc->vc_font.width, 8);
u32 cellsize = width * vc->vc_font.height;
u32 maxcnt = info->pixmap.size/cellsize;
@@ -167,10 +170,10 @@ static void bit_putcs(struct vc_data *vc, struct fb_info *info,
image.height = vc->vc_font.height;
image.depth = 1;
- if (image.dy >= info->var.yres)
+ if (image.dy >= vyres)
return;
- image.height = min(image.height, info->var.yres - image.dy);
+ image.height = min(image.height, vyres - image.dy);
if (attribute) {
buf = kmalloc(cellsize, GFP_ATOMIC);
diff --git a/drivers/video/fbdev/core/fb_info.c b/drivers/video/fbdev/core/fb_info.c
index 52f9bd2c5417..05988a936beb 100644
--- a/drivers/video/fbdev/core/fb_info.c
+++ b/drivers/video/fbdev/core/fb_info.c
@@ -43,6 +43,7 @@ struct fb_info *framebuffer_alloc(size_t size, struct device *dev)
info->device = dev;
info->fbcon_rotate_hint = -1;
info->blank = FB_BLANK_UNBLANK;
+ INIT_LIST_HEAD(&info->modelist);
#if IS_ENABLED(CONFIG_FB_BACKLIGHT)
mutex_init(&info->bl_curve_mutex);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 01715873ea49..9a5d50d69d00 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -2414,6 +2414,10 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
font_data_t *old_fontdata = p->fontdata;
const u8 *old_data = vc->vc_font.data;
unsigned short old_hi_font_mask = vc->vc_hi_font_mask;
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+ struct fbcon_rotated saved_rotated = { };
+ bool rotated_saved = false;
+#endif
font_data_get(data);
@@ -2432,6 +2436,27 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
else if (!vc->vc_hi_font_mask && charcount == 512)
set_vc_hi_font(vc, true);
+ /*
+ * Select the console's blitting state and update par->rotated to
+ * match the new font, so that the rendering below does not use a
+ * cache that was built for the old font. Clear the cached state so
+ * the rebuild allocates a fresh buffer; the old state is kept until
+ * the font change can no longer fail.
+ */
+ if (con_is_visible(vc)) {
+ set_blitting_type(vc, info);
+ if (par->bitops->rotate_font) {
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+ saved_rotated = par->rotated;
+ memset(&par->rotated, 0, sizeof(par->rotated));
+ rotated_saved = true;
+#endif
+ ret = par->bitops->rotate_font(info, vc);
+ if (ret)
+ goto err_out;
+ }
+ }
+
if (resize) {
int cols, rows;
@@ -2448,12 +2473,26 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
update_screen(vc);
}
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+ if (rotated_saved)
+ kfree(saved_rotated.buf);
+#endif
if (old_fontdata)
font_data_put(old_fontdata);
return 0;
err_out:
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+ /*
+ * Drop the buffer built for the new font and restore the saved
+ * state.
+ */
+ if (rotated_saved) {
+ kfree(par->rotated.buf);
+ par->rotated = saved_rotated;
+ }
+#endif
p->fontdata = old_fontdata;
vc->vc_font.data = old_data;
vc->vc_font.width = old_width;
@@ -2466,6 +2505,13 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
else if (!old_hi_font_mask && vc->vc_hi_font_mask)
set_vc_hi_font(vc, false);
+ /*
+ * Re-select the blitting type, which re-uploads the font map on
+ * tile-blitting hardware.
+ */
+ if (con_is_visible(vc))
+ set_blitting_type(vc, info);
+
font_data_put(data);
return ret;
diff --git a/drivers/video/fbdev/core/fbcon.h b/drivers/video/fbdev/core/fbcon.h
index 407d207b14f1..a618adf3c871 100644
--- a/drivers/video/fbdev/core/fbcon.h
+++ b/drivers/video/fbdev/core/fbcon.h
@@ -72,6 +72,18 @@ struct fbcon_bitops {
int (*rotate_font)(struct fb_info *info, struct vc_data *vc);
};
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+struct fbcon_rotated {
+ font_data_t *fontdata; /* source font */
+ u8 *buf; /* rotated glyphs */
+ size_t bufsize;
+ int buf_rotate; /* rotation of buf */
+ unsigned int buf_width; /* geometry of source font */
+ unsigned int buf_height;
+ unsigned int buf_charcount;
+};
+#endif
+
struct fbcon_par {
struct fb_var_screeninfo var; /* copy of the current fb_var_screeninfo */
struct delayed_work cursor_work; /* Cursor timer */
@@ -88,12 +100,7 @@ struct fbcon_par {
int rotate;
char *cursor_data;
#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
- struct {
- font_data_t *fontdata; /* source font */
- u8 *buf; /* rotated glyphs */
- size_t bufsize;
- int buf_rotate; /* rotation of buf */
- } rotated;
+ struct fbcon_rotated rotated;
#endif
u8 *cursor_src;
u32 cursor_size;
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 6cdbc96eeca6..7afd63dcee7b 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -21,11 +21,19 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
unsigned char *buf;
int ret;
- if (par->p->fontdata == par->rotated.fontdata && par->rotate == par->rotated.buf_rotate)
+ if (par->p->fontdata == par->rotated.fontdata &&
+ par->rotate == par->rotated.buf_rotate &&
+ vc->vc_font.width == par->rotated.buf_width &&
+ vc->vc_font.height == par->rotated.buf_height &&
+ vc->vc_font.charcount == par->rotated.buf_charcount &&
+ par->rotated.buf)
return 0;
par->rotated.fontdata = par->p->fontdata;
par->rotated.buf_rotate = par->rotate;
+ par->rotated.buf_width = vc->vc_font.width;
+ par->rotated.buf_height = vc->vc_font.height;
+ par->rotated.buf_charcount = vc->vc_font.charcount;
if (info->fbops->fb_sync)
info->fbops->fb_sync(info);
diff --git a/drivers/video/fbdev/kyro/fbdev.c b/drivers/video/fbdev/kyro/fbdev.c
index c23738988822..94d6af54e6c4 100644
--- a/drivers/video/fbdev/kyro/fbdev.c
+++ b/drivers/video/fbdev/kyro/fbdev.c
@@ -765,7 +765,7 @@ static int kyrofb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
fb_memset_io(info->screen_base, 0, size);
if (register_framebuffer(info) < 0)
- goto out_free_fb;
+ goto out_free_wc;
fb_info(info, "%s frame buffer device, at %dx%d@%d using %ldk/%ldk of VRAM\n",
info->fix.id,
@@ -776,6 +776,8 @@ static int kyrofb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
return 0;
+out_free_wc:
+ arch_phys_wc_del(currentpar->wc_cookie);
out_free_fb:
framebuffer_release(info);
diff --git a/drivers/video/fbdev/nvidia/nvidia.c b/drivers/video/fbdev/nvidia/nvidia.c
index 4ef7a5ba43cc..395f5d0455fa 100644
--- a/drivers/video/fbdev/nvidia/nvidia.c
+++ b/drivers/video/fbdev/nvidia/nvidia.c
@@ -1423,6 +1423,7 @@ static int nvidiafb_probe(struct pci_dev *pd, const struct pci_device_id *ent)
return 0;
err_out_iounmap_fb:
+ arch_phys_wc_del(par->wc_cookie);
fb_destroy_modelist(&info->modelist);
err_out_free_base1:
fb_destroy_modedb(info->monspecs.modedb);
diff --git a/drivers/video/fbdev/sh_mobile_lcdcfb.c b/drivers/video/fbdev/sh_mobile_lcdcfb.c
index e8324b01700f..a8ef7c599391 100644
--- a/drivers/video/fbdev/sh_mobile_lcdcfb.c
+++ b/drivers/video/fbdev/sh_mobile_lcdcfb.c
@@ -1042,7 +1042,11 @@ static int sh_mobile_lcdc_start(struct sh_mobile_lcdc_priv *priv)
ch->defio.deferred_io = sh_mobile_lcdc_deferred_io;
ch->defio.delay = msecs_to_jiffies(tmp);
ch->info->fbdefio = &ch->defio;
- fb_deferred_io_init(ch->info);
+ ret = fb_deferred_io_init(ch->info);
+ if (ret) {
+ ch->info->fbdefio = NULL;
+ return ret;
+ }
}
sh_mobile_lcdc_display_on(ch);
@@ -1337,22 +1341,24 @@ overlay_rop3_store(struct device *dev, struct device_attribute *attr,
return count;
}
-static DEVICE_ATTR_RW(overlay_alpha);
-static DEVICE_ATTR_RW(overlay_mode);
-static DEVICE_ATTR_RW(overlay_position);
-static DEVICE_ATTR_RW(overlay_rop3);
-
-static struct attribute *overlay_sysfs_attrs[] __maybe_unused = {
- &dev_attr_overlay_alpha.attr,
- &dev_attr_overlay_mode.attr,
- &dev_attr_overlay_position.attr,
- &dev_attr_overlay_rop3.attr,
+static struct device_attribute dev_attr_ovl_alpha =
+ __ATTR(ovl_alpha, 0644, overlay_alpha_show, overlay_alpha_store);
+static struct device_attribute dev_attr_ovl_mode =
+ __ATTR(ovl_mode, 0644, overlay_mode_show, overlay_mode_store);
+static struct device_attribute dev_attr_ovl_position =
+ __ATTR(ovl_position, 0644, overlay_position_show, overlay_position_store);
+static struct device_attribute dev_attr_ovl_rop3 =
+ __ATTR(ovl_rop3, 0644, overlay_rop3_show, overlay_rop3_store);
+
+static struct attribute *overlay_sysfs_attrs[] = {
+ &dev_attr_ovl_alpha.attr,
+ &dev_attr_ovl_mode.attr,
+ &dev_attr_ovl_position.attr,
+ &dev_attr_ovl_rop3.attr,
NULL,
};
-#ifdef CONFIG_FB_DEVICE
ATTRIBUTE_GROUPS(overlay_sysfs);
-#endif
static const struct fb_fix_screeninfo sh_mobile_lcdc_overlay_fix = {
.id = "SH Mobile LCDC",
@@ -1510,9 +1516,10 @@ sh_mobile_lcdc_overlay_fb_unregister(struct sh_mobile_lcdc_overlay *ovl)
{
struct fb_info *info = ovl->info;
- if (info == NULL || info->dev == NULL)
+ if (!info || !dev_of_fbinfo(info))
return;
+ device_remove_groups(dev_of_fbinfo(info), overlay_sysfs_groups);
unregister_framebuffer(ovl->info);
}
@@ -1530,6 +1537,15 @@ sh_mobile_lcdc_overlay_fb_register(struct sh_mobile_lcdc_overlay *ovl)
if (ret < 0)
return ret;
+ /* The framebuffer device is optional, see fb_device_create(). */
+ if (dev_of_fbinfo(info)) {
+ ret = device_add_groups(dev_of_fbinfo(info), overlay_sysfs_groups);
+ if (ret < 0) {
+ unregister_framebuffer(info);
+ return ret;
+ }
+ }
+
dev_info(lcdc->dev, "registered %s/overlay %u as %dx%d %dbpp.\n",
dev_name(lcdc->dev), ovl->index, info->var.xres,
info->var.yres, info->var.bits_per_pixel);
@@ -2637,7 +2653,6 @@ static int sh_mobile_lcdc_probe(struct platform_device *pdev)
static struct platform_driver sh_mobile_lcdc_driver = {
.driver = {
.name = "sh_mobile_lcdc_fb",
- .dev_groups = overlay_sysfs_groups,
.pm = &sh_mobile_lcdc_dev_pm_ops,
},
.probe = sh_mobile_lcdc_probe,
diff --git a/drivers/video/fbdev/smscufx.c b/drivers/video/fbdev/smscufx.c
index 5704f60e1741..34d8abd5b354 100644
--- a/drivers/video/fbdev/smscufx.c
+++ b/drivers/video/fbdev/smscufx.c
@@ -1041,10 +1041,13 @@ static int ufx_ops_open(struct fb_info *info, int user)
if (fbdefio) {
fbdefio->delay = UFX_DEFIO_WRITE_DELAY;
fbdefio->deferred_io = ufx_dpy_deferred_io;
- }
- info->fbdefio = fbdefio;
- fb_deferred_io_init(info);
+ info->fbdefio = fbdefio;
+ if (fb_deferred_io_init(info)) {
+ kfree(fbdefio);
+ info->fbdefio = NULL;
+ }
+ }
}
pr_debug("open /dev/fb%d user=%d fb_info=%p count=%d",
diff --git a/drivers/video/fbdev/ssd1307fb.c b/drivers/video/fbdev/ssd1307fb.c
index 4d185c754284..8fd96c37ee1a 100644
--- a/drivers/video/fbdev/ssd1307fb.c
+++ b/drivers/video/fbdev/ssd1307fb.c
@@ -767,7 +767,11 @@ static int ssd1307fb_probe(struct i2c_client *client)
info->fix.smem_start = __pa(vmem);
info->fix.smem_len = vmem_size;
- fb_deferred_io_init(info);
+ ret = fb_deferred_io_init(info);
+ if (ret) {
+ dev_err(dev, "failed to init fb deferred io: %d\n", ret);
+ goto fb_defio_error;
+ }
i2c_set_clientdata(client, info);
diff --git a/drivers/video/fbdev/udlfb.c b/drivers/video/fbdev/udlfb.c
index 92b352bc7e96..8b7b4bdac4fb 100644
--- a/drivers/video/fbdev/udlfb.c
+++ b/drivers/video/fbdev/udlfb.c
@@ -947,10 +947,13 @@ static int dlfb_ops_open(struct fb_info *info, int user)
fbdefio->delay = DL_DEFIO_WRITE_DELAY;
fbdefio->sort_pagereflist = true;
fbdefio->deferred_io = dlfb_dpy_deferred_io;
- }
- info->fbdefio = fbdefio;
- fb_deferred_io_init(info);
+ info->fbdefio = fbdefio;
+ if (fb_deferred_io_init(info)) {
+ kfree(fbdefio);
+ info->fbdefio = NULL;
+ }
+ }
}
dev_dbg(info->dev, "open, user=%d fb_info=%p count=%d\n",
diff --git a/drivers/video/fbdev/xen-fbfront.c b/drivers/video/fbdev/xen-fbfront.c
index 4385976277ac..7ef909d4242d 100644
--- a/drivers/video/fbdev/xen-fbfront.c
+++ b/drivers/video/fbdev/xen-fbfront.c
@@ -443,7 +443,11 @@ static int xenfb_probe(struct xenbus_device *dev,
}
fb_info->fbdefio = &xenfb_defio;
- fb_deferred_io_init(fb_info);
+ ret = fb_deferred_io_init(fb_info);
+ if (ret < 0) {
+ xenbus_dev_fatal(dev, ret, "fb_deferred_io_init");
+ goto error_cmap;
+ }
xenfb_init_shared_page(info, fb_info);
@@ -465,6 +469,7 @@ static int xenfb_probe(struct xenbus_device *dev,
error_fb:
fb_deferred_io_cleanup(fb_info);
+error_cmap:
fb_dealloc_cmap(&fb_info->cmap);
framebuffer_release(fb_info);
error_nomem:
diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c
index 8af1fad2de0b..78227d200afe 100644
--- a/drivers/watchdog/hpwdt.c
+++ b/drivers/watchdog/hpwdt.c
@@ -199,7 +199,7 @@ static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
}
hex_byte_pack(panic_msg, nmistat);
- nmi_panic(regs, panic_msg);
+ nmi_panic(regs, "%s", panic_msg);
return NMI_HANDLED;
}
diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index 83b0df460894..df1883aca850 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -96,8 +96,10 @@ dmabuf_exp_wait_obj_new(struct gntdev_dmabuf_priv *priv,
struct gntdev_dmabuf_wait_obj *obj;
obj = kzalloc_obj(*obj);
- if (!obj)
+ if (!obj) {
+ kref_put(&gntdev_dmabuf->u.exp.refcount, dmabuf_exp_release);
return ERR_PTR(-ENOMEM);
+ }
init_completion(&obj->completion);
obj->gntdev_dmabuf = gntdev_dmabuf;
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 1dcc4675580e..b71f5bae25b6 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -1066,6 +1066,13 @@ static int gntdev_mmap(struct file *flip, struct vm_area_struct *vma)
if ((vma->vm_flags & VM_WRITE) && !(vma->vm_flags & VM_SHARED))
return -EINVAL;
+ /*
+ * Private mappings of foreign grant pages must never become
+ * writable: they would take the COW path on granted pages.
+ */
+ if (!(vma->vm_flags & VM_SHARED))
+ vm_flags_clear(vma, VM_MAYWRITE);
+
pr_debug("map %d+%d at %lx (pgoff %lx)\n",
index, count, vma->vm_start, vma->vm_pgoff);
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 7cfc28f1bb86..c67ec6d282cf 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -1169,51 +1169,40 @@ static irqreturn_t ioeventfd_interrupt(int irq, void *dev_id)
struct privcmd_kernel_ioreq *kioreq = port->kioreq;
struct ioreq *ioreq = &kioreq->ioreq[port->vcpu];
struct privcmd_kernel_ioeventfd *kioeventfd;
- unsigned int state = STATE_IOREQ_READY;
+ bool matched = false;
- if (ioreq->state != STATE_IOREQ_READY ||
- ioreq->type != IOREQ_TYPE_COPY || ioreq->dir != IOREQ_WRITE)
+ if (ioreq->state != STATE_IOREQ_READY)
return IRQ_NONE;
- /*
- * We need a barrier, smp_mb(), here to ensure reads are finished before
- * `state` is updated. Since the lock implementation ensures that
- * appropriate barrier will be added anyway, we can avoid adding
- * explicit barrier here.
- *
- * Ideally we don't need to update `state` within the locks, but we do
- * that here to avoid adding explicit barrier.
- */
+ /* Xen publishes the request before changing its state to READY. */
+ virt_rmb();
- spin_lock(&kioreq->lock);
- ioreq->state = STATE_IOREQ_INPROCESS;
+ if (ioreq->type != IOREQ_TYPE_COPY || ioreq->dir != IOREQ_WRITE)
+ return IRQ_NONE;
+ spin_lock(&kioreq->lock);
list_for_each_entry(kioeventfd, &kioreq->ioeventfds, list) {
if (ioreq->addr == kioeventfd->addr + VIRTIO_MMIO_QUEUE_NOTIFY &&
ioreq->size == kioeventfd->addr_len &&
(ioreq->data & QUEUE_NOTIFY_VQ_MASK) == kioeventfd->vq) {
+ /* Finish reading the request before claiming it. */
+ virt_mb();
+ ioreq->state = STATE_IOREQ_INPROCESS;
eventfd_signal(kioeventfd->eventfd);
- state = STATE_IORESP_READY;
+ matched = true;
break;
}
}
spin_unlock(&kioreq->lock);
- /*
- * We need a barrier, smp_mb(), here to ensure writes are finished
- * before `state` is updated. Since the lock implementation ensures that
- * appropriate barrier will be added anyway, we can avoid adding
- * explicit barrier here.
- */
-
- ioreq->state = state;
-
- if (state == STATE_IORESP_READY) {
- notify_remote_via_evtchn(port->port);
- return IRQ_HANDLED;
- }
+ if (!matched)
+ return IRQ_NONE;
- return IRQ_NONE;
+ /* Publish the response only after signaling the eventfd. */
+ virt_wmb();
+ ioreq->state = STATE_IORESP_READY;
+ notify_remote_via_evtchn(port->port);
+ return IRQ_HANDLED;
}
static void ioreq_free(struct privcmd_kernel_ioreq *kioreq)
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 79a2b5dfd694..d1a335233aab 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -234,13 +234,16 @@ static int pcistub_get_gsi_from_sbdf(unsigned int sbdf)
int bus = PCI_BUS_NUM(sbdf);
int slot = PCI_SLOT(sbdf);
int func = PCI_FUNC(sbdf);
+ int gsi;
psdev = pcistub_device_find(domain, bus, slot, func);
-
if (!psdev)
return -ENODEV;
- return psdev->gsi;
+ gsi = psdev->gsi;
+ pcistub_device_put(psdev);
+
+ return gsi;
}
#endif
diff --git a/fs/affs/bitmap.c b/fs/affs/bitmap.c
index 40bc4ce6af4a..f013a2a94d4d 100644
--- a/fs/affs/bitmap.c
+++ b/fs/affs/bitmap.c
@@ -46,7 +46,7 @@ affs_free_block(struct super_block *sb, u32 block)
pr_debug("%s(%u)\n", __func__, block);
- if (block > sbi->s_partition_size)
+ if (!affs_validblock(sb, block))
goto err_range;
blk = block - sbi->s_reserved;
diff --git a/fs/aio.c b/fs/aio.c
index d78acc69f487..f77cbc046e91 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -528,7 +528,18 @@ static int aio_migrate_folio(struct address_space *mapping, struct folio *dst,
#define aio_migrate_folio NULL
#endif
+/*
+ * filemap_fault() reads a folio that it did not find in the page cache. All
+ * folios of a live ring are there, so the ring was truncated under the fault.
+ */
+static int aio_read_folio(struct file *file, struct folio *folio)
+{
+ folio_unlock(folio);
+ return -EIO;
+}
+
static const struct address_space_operations aio_ctx_aops = {
+ .read_folio = aio_read_folio,
.dirty_folio = noop_dirty_folio,
.migrate_folio = aio_migrate_folio,
};
@@ -1869,7 +1880,12 @@ static int aio_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync,
list_del_init(&req->wait.entry);
list_del(&iocb->ki_list);
iocb->ki_res.res = mangle_poll(mask);
- if (iocb->ki_eventfd && !eventfd_signal_allowed()) {
+ /*
+ * We hold an arbitrary provider waitqueue lock here. Signaling a
+ * result eventfd can feed back through epoll and try to take the same
+ * lock again. Defer all eventfd-backed poll completions.
+ */
+ if (iocb->ki_eventfd) {
iocb = NULL;
INIT_WORK(&req->work, aio_poll_put_work);
schedule_work(&req->work);
diff --git a/fs/backing-file.c b/fs/backing-file.c
index cc101143f921..4f6b3b588f74 100644
--- a/fs/backing-file.c
+++ b/fs/backing-file.c
@@ -36,13 +36,36 @@ struct file *backing_file_open(const struct file *user_file, int flags,
const struct cred *cred)
{
const struct path *user_path = file_user_path(user_file);
+ struct inode *real_inode = d_inode(real_path->dentry);
struct file *f;
int error;
+ /*
+ * may_open() checked the inode of the user file, which need not carry
+ * S_APPEND and S_IMMUTABLE of the backing inode.
+ */
+ if ((flags & O_ACCMODE) != O_RDONLY) {
+ if (IS_IMMUTABLE(real_inode))
+ return ERR_PTR(-EPERM);
+ if (IS_APPEND(real_inode) && !(flags & O_APPEND))
+ return ERR_PTR(-EPERM);
+ }
+
f = alloc_empty_backing_file(flags, cred, user_file);
if (IS_ERR(f))
return f;
+ /*
+ * A user file that must not generate fsnotify events (e.g. the fd
+ * that fanotify creates for a permission event with
+ * dentry_open_nonotify()) must not generate them through its backing
+ * file either. Otherwise a listener watching the backing filesystem
+ * gets a nested permission event from its own read(2), which only
+ * another reader of the same group can answer.
+ */
+ if (FMODE_FSNOTIFY_NONE(user_file->f_mode))
+ file_set_fsnotify_mode(f, FMODE_NONOTIFY);
+
path_get(user_path);
backing_file_set_user_path(f, user_path);
error = vfs_open(real_path, f);
@@ -84,6 +107,7 @@ struct backing_aio {
refcount_t ref;
struct kiocb *orig_iocb;
/* used for aio completion */
+ bool is_write;
void (*end_write)(struct kiocb *iocb, ssize_t);
struct work_struct work;
long res;
@@ -124,7 +148,8 @@ static void backing_aio_rw_complete(struct kiocb *iocb, long res)
struct backing_aio *aio = container_of(iocb, struct backing_aio, iocb);
struct kiocb *orig_iocb = aio->orig_iocb;
- if (iocb->ki_flags & IOCB_WRITE)
+ /* Pairs with kiocb_start_write() in vfs_iocb_iter_write() */
+ if (aio->is_write)
kiocb_end_write(iocb);
backing_aio_cleanup(aio, res);
@@ -239,6 +264,7 @@ static int do_backing_file_write_iter(struct file *file, struct iov_iter *iter,
return -ENOMEM;
aio->orig_iocb = iocb;
+ aio->is_write = true;
aio->end_write = end_write;
kiocb_clone(&aio->iocb, iocb, get_file(file));
aio->iocb.ki_flags = flags;
@@ -263,6 +289,14 @@ ssize_t backing_file_write_iter(struct file *file, struct iov_iter *iter,
if (!iov_iter_count(iter))
return 0;
+ /*
+ * F_SETFL and RWF_NOAPPEND can take append mode away from the user
+ * file. They checked its inode, not the backing one.
+ */
+ if ((file->f_flags & O_APPEND) && !(flags & IOCB_APPEND) &&
+ IS_APPEND(file_inode(file)))
+ return -EPERM;
+
ret = file_remove_privs(iocb->ki_filp);
if (ret)
return ret;
@@ -309,6 +343,10 @@ ssize_t backing_file_splice_write(struct pipe_inode_info *pipe,
if (!out->f_op->splice_write)
return -EINVAL;
+ /* splice refuses a user file in append mode, so F_SETFL cleared it */
+ if ((out->f_flags & O_APPEND) && IS_APPEND(file_inode(out)))
+ return -EPERM;
+
ret = file_remove_privs(iocb->ki_filp);
if (ret)
return ret;
@@ -338,6 +376,10 @@ int backing_file_mmap(struct file *file, struct vm_area_struct *vma,
if (!can_mmap_file(file))
return -ENODEV;
+ /* do_mmap() checked the inode of the user file */
+ if ((vma->vm_flags & VM_SHARED) && IS_APPEND(file_inode(file)))
+ return -EACCES;
+
vma_set_file(vma, file);
scoped_with_creds(ctx->cred) {
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index faf6ed7deb18..aa1201fe4d7f 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -8252,7 +8252,6 @@ static int btrfs_getattr(struct mnt_idmap *idmap,
struct inode *inode = d_inode(path->dentry);
u32 blocksize = btrfs_sb(inode->i_sb)->sectorsize;
u32 bi_flags = BTRFS_I(inode)->flags;
- u32 bi_ro_flags = BTRFS_I(inode)->ro_flags;
stat->result_mask |= STATX_BTIME;
stat->btime.tv_sec = BTRFS_I(inode)->i_otime_sec;
@@ -8265,8 +8264,6 @@ static int btrfs_getattr(struct mnt_idmap *idmap,
stat->attributes |= STATX_ATTR_IMMUTABLE;
if (bi_flags & BTRFS_INODE_NODUMP)
stat->attributes |= STATX_ATTR_NODUMP;
- if (bi_ro_flags & BTRFS_INODE_RO_VERITY)
- stat->attributes |= STATX_ATTR_VERITY;
stat->attributes_mask |= (STATX_ATTR_APPEND |
STATX_ATTR_COMPRESSED |
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index ddb620ac241b..2c9b29aa59a2 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -2305,6 +2305,9 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
case BTRFS_IOC_GET_SUPPORTED_FEATURES:
ret = btrfs_ioctl_get_supported_features((void __user*)arg);
break;
+ default:
+ ret = -ENOTTY;
+ break;
}
out:
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 85ea9c5d4536..e7d3cd66a98d 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -8971,6 +8971,7 @@ static int relocating_repair_kthread(void *data)
bool btrfs_repair_one_zone(struct btrfs_fs_info *fs_info, u64 logical)
{
struct btrfs_block_group *cache;
+ struct task_struct *task;
if (!btrfs_is_zoned(fs_info))
return false;
@@ -8988,8 +8989,9 @@ bool btrfs_repair_one_zone(struct btrfs_fs_info *fs_info, u64 logical)
return true;
}
- kthread_run(relocating_repair_kthread, cache,
- "btrfs-relocating-repair");
+ task = kthread_run(relocating_repair_kthread, cache, "btrfs-relocating-repair");
+ if (IS_ERR(task))
+ btrfs_put_block_group(cache);
return true;
}
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index e598b2d424ec..e07a7b1d3663 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -322,6 +322,12 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
return false;
}
+ if (subreq->start >= iinfo->inline_len) {
+ ceph_mdsc_put_request(req);
+ err = -ENODATA;
+ goto out;
+ }
+
len = min_t(size_t, iinfo->inline_len - subreq->start, subreq->len);
err = copy_to_iter(iinfo->inline_data + subreq->start, len, &subreq->io_iter);
if (err == 0) {
@@ -364,7 +370,7 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
struct ceph_client *cl = fsc->client;
struct ceph_osd_request *req = NULL;
struct ceph_vino vino = ceph_vino(inode);
- int err;
+ int err = 0;
u64 len;
bool sparse = IS_ENCRYPTED(inode) || ceph_test_mount_opt(fsc, SPARSEREAD);
u64 off = subreq->start;
@@ -1889,7 +1895,12 @@ static int ceph_netfs_check_write_begin(struct file *file, loff_t pos, unsigned
r = wait_event_killable(ci->i_cap_wq,
context_is_writeable_or_written(inode, snapc));
ceph_put_snap_context(snapc);
- return r == 0 ? -EAGAIN : r;
+ /*
+ * *foliop is cleared, so a return of 0 makes
+ * netfs_write_begin() grab the folio again and call us again
+ * to recheck it. Any error (fatal signal) fails the write.
+ */
+ return r;
}
return 0;
}
diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c
index bcb04c6cb92c..4cedbdd1d403 100644
--- a/fs/ceph/caps.c
+++ b/fs/ceph/caps.c
@@ -1569,6 +1569,8 @@ static void __send_cap(struct cap_msg_args *arg, struct ceph_inode_info *ci)
spin_lock(&ci->i_ceph_lock);
__cap_delay_requeue(arg->session->s_mdsc, ci);
spin_unlock(&ci->i_ceph_lock);
+ ceph_buffer_put(arg->old_xattr_buf);
+ ceph_buffer_put(arg->xattr_buf);
return;
}
@@ -2889,6 +2891,10 @@ static int try_get_cap_refs(struct inode *inode, int need, int want,
}
/* finish pending truncate */
+ if (ci->i_truncate_pending && (flags & NON_BLOCKING)) {
+ ret = -EAGAIN;
+ goto out_unlock;
+ }
while (ci->i_truncate_pending) {
spin_unlock(&ci->i_ceph_lock);
if (snap_rwsem_locked) {
diff --git a/fs/ceph/crypto.c b/fs/ceph/crypto.c
index bc0a097a4cea..46acdaf9a822 100644
--- a/fs/ceph/crypto.c
+++ b/fs/ceph/crypto.c
@@ -30,6 +30,9 @@ static int ceph_crypt_get_context(struct inode *inode, void *ctx, size_t len)
return -ENOBUFS;
ctxlen = le32_to_cpu(cfa->cfa_blob_len);
+ if (ctxlen > ci->fscrypt_auth_len -
+ offsetof(struct ceph_fscrypt_auth, cfa_blob))
+ return -ENOBUFS;
if (len < ctxlen)
return -ERANGE;
@@ -313,6 +316,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, unsigned char *tname,
struct fscrypt_str iname;
char *name = fname->name;
int name_len = fname->name_len;
+ int dst_len;
int ret;
if (WARN_ON_ONCE(tname && is_vmalloc_addr(tname)))
@@ -387,6 +391,9 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, unsigned char *tname,
iname.len = fname->ctext_len;
}
+ /* oname->len is overwritten by the decryption below */
+ dst_len = oname->len;
+
_oname.name = unlikely(is_vmalloc_addr(oname->name)) ? tname : oname->name;
_oname.len = oname->len;
@@ -403,6 +410,10 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, unsigned char *tname,
name_len = snprintf(tmp_buf, sizeof(tmp_buf), "_%.*s_%llu",
oname->len, oname->name, dir->i_ino);
+ if (name_len > dst_len) {
+ ret = -ENAMETOOLONG;
+ goto out;
+ }
memcpy(oname->name, tmp_buf, name_len);
oname->len = name_len;
}
diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c
index 085ae0cfb5f7..1806cdc20271 100644
--- a/fs/ceph/mds_client.c
+++ b/fs/ceph/mds_client.c
@@ -15,6 +15,7 @@
#include <linux/ktime.h>
#include <linux/bitmap.h>
#include <linux/mnt_idmapping.h>
+#include <linux/overflow.h>
#include "super.h"
#include "mds_client.h"
@@ -120,6 +121,7 @@ static int parse_reply_info_in(void **p, void *end,
struct ceph_mds_client *mdsc)
{
int err = 0;
+ size_t splits_len;
u8 struct_v = 0;
u8 struct_compat = 0;
u32 struct_len = 0;
@@ -140,9 +142,11 @@ static int parse_reply_info_in(void **p, void *end,
ceph_decode_need(p, end, sizeof(struct ceph_mds_reply_inode), bad);
info->in = *p;
- *p += sizeof(struct ceph_mds_reply_inode) +
- sizeof(*info->in->fragtree.splits) *
- le32_to_cpu(info->in->fragtree.nsplits);
+ splits_len = size_mul(sizeof(*info->in->fragtree.splits),
+ le32_to_cpu(info->in->fragtree.nsplits));
+ *p += sizeof(struct ceph_mds_reply_inode);
+ ceph_decode_need(p, end, splits_len, bad);
+ *p += splits_len;
ceph_decode_32_safe(p, end, info->symlink_len, bad);
ceph_decode_need(p, end, info->symlink_len, bad);
@@ -380,6 +384,8 @@ static int parse_reply_info_lease(void **p, void *end,
lend = *p + struct_len;
ceph_decode_need(p, end, struct_len, bad);
+ if (struct_len < sizeof(**lease))
+ goto bad;
*lease = *p;
*p += sizeof(**lease);
@@ -479,10 +485,8 @@ static int parse_reply_info_readdir(void **p, void *end,
goto done;
BUG_ON(!info->dir_entries);
- if ((unsigned long)(info->dir_entries + num) >
- (unsigned long)info->dir_entries + info->dir_buf_size) {
+ if (num > info->dir_buf_size / sizeof(*info->dir_entries)) {
pr_err_client(cl, "dir contents are larger than expected\n");
- WARN_ON(1);
goto bad;
}
@@ -1803,8 +1807,10 @@ __open_export_target_session(struct ceph_mds_client *mdsc, int target)
if (session->s_state == CEPH_MDS_SESSION_NEW ||
session->s_state == CEPH_MDS_SESSION_CLOSING) {
ret = __open_session(mdsc, session);
- if (ret)
+ if (ret) {
+ ceph_put_mds_session(session);
return ERR_PTR(ret);
+ }
}
return session;
@@ -3985,6 +3991,14 @@ int ceph_mdsc_wait_request(struct ceph_mds_client *mdsc,
/* only abort if we didn't race with a real reply */
if (test_bit(CEPH_MDS_R_GOT_RESULT, &req->r_req_flags)) {
err = le32_to_cpu(req->r_reply_info.head->result);
+ if (err > 0 || err < -MAX_ERRNO) {
+ /*
+ * The result is server-controlled. Values outside
+ * the errno range are turned into bogus pointers by
+ * the ERR_PTR() users among the callers.
+ */
+ err = -EIO;
+ }
} else if (err < 0) {
doutc(cl, "aborted request %lld with %d\n", req->r_tid, err);
diff --git a/fs/ceph/mdsmap.c b/fs/ceph/mdsmap.c
index 53079ef34c3a..8e2937e4e562 100644
--- a/fs/ceph/mdsmap.c
+++ b/fs/ceph/mdsmap.c
@@ -170,6 +170,10 @@ struct ceph_mdsmap *ceph_mdsmap_decode(struct ceph_mds_client *mdsc, void **p,
* and the mds rank >= m_num_active_mds.
*/
m->possible_max_rank = max(m->m_num_active_mds, m->m_max_mds);
+ if (m->possible_max_rank > CEPH_MAX_MDS) {
+ err = -EIO;
+ goto corrupt;
+ }
m->m_info = kzalloc_objs(*m->m_info, m->possible_max_rank, GFP_NOFS);
if (!m->m_info)
@@ -205,7 +209,7 @@ struct ceph_mdsmap *ceph_mdsmap_decode(struct ceph_mds_client *mdsc, void **p,
ceph_decode_need(p, end, sizeof(u64) + sizeof(u32), bad);
*p += sizeof(u64);
namelen = ceph_decode_32(p); /* skip mds name */
- *p += namelen;
+ ceph_decode_skip_n(p, end, namelen, bad);
ceph_decode_32_safe(p, end, mds, bad);
ceph_decode_32_safe(p, end, inc, bad);
@@ -223,7 +227,7 @@ struct ceph_mdsmap *ceph_mdsmap_decode(struct ceph_mds_client *mdsc, void **p,
laggy = laggy_since.tv_sec != 0 || laggy_since.tv_nsec != 0;
*p += sizeof(u32);
ceph_decode_32_safe(p, end, namelen, bad);
- *p += namelen;
+ ceph_decode_skip_n(p, end, namelen, bad);
if (info_v >= 2) {
ceph_decode_32_safe(p, end, num_export_targets, bad);
export_targets_len = size_mul(num_export_targets,
@@ -316,6 +320,10 @@ struct ceph_mdsmap *ceph_mdsmap_decode(struct ceph_mds_client *mdsc, void **p,
{
int num_laggy = 0;
ceph_decode_32_safe(p, end, n, bad_ext);
+ if (n > CEPH_MAX_MDS) {
+ err = -EIO;
+ goto corrupt;
+ }
ceph_decode_need(p, end, sizeof(u32) * n, bad_ext);
for (i = 0; i < n; i++) {
diff --git a/fs/ceph/snap.c b/fs/ceph/snap.c
index 9b79a5eaca93..5ba325aa147c 100644
--- a/fs/ceph/snap.c
+++ b/fs/ceph/snap.c
@@ -788,6 +788,7 @@ int ceph_update_snap_trace(struct ceph_mds_client *mdsc,
struct ceph_snap_realm *first_realm = NULL;
struct ceph_snap_realm *realm_to_rebuild = NULL;
struct ceph_client *client = mdsc->fsc->client;
+ size_t snaps_len;
int rebuild_snapcs;
int err = -ENOMEM;
int ret;
@@ -802,8 +803,10 @@ int ceph_update_snap_trace(struct ceph_mds_client *mdsc,
ceph_decode_need(&p, e, sizeof(*ri), bad);
ri = p;
p += sizeof(*ri);
- ceph_decode_need(&p, e, sizeof(u64)*(le32_to_cpu(ri->num_snaps) +
- le32_to_cpu(ri->num_prior_parent_snaps)), bad);
+ snaps_len = size_mul(size_add(le32_to_cpu(ri->num_snaps),
+ le32_to_cpu(ri->num_prior_parent_snaps)),
+ sizeof(u64));
+ ceph_decode_need(&p, e, snaps_len, bad);
snaps = p;
p += sizeof(u64) * le32_to_cpu(ri->num_snaps);
prior_parent_snaps = p;
diff --git a/fs/ceph/xattr.c b/fs/ceph/xattr.c
index cc4ffbbcb719..ff70e4df9331 100644
--- a/fs/ceph/xattr.c
+++ b/fs/ceph/xattr.c
@@ -11,6 +11,7 @@
#include <linux/security.h>
#include <linux/posix_acl_xattr.h>
#include <linux/slab.h>
+#include <linux/overflow.h>
#define XATTR_CEPH_PREFIX "ceph."
#define XATTR_CEPH_PREFIX_LEN (sizeof (XATTR_CEPH_PREFIX) - 1)
@@ -843,14 +844,23 @@ static int __build_xattrs(struct inode *inode)
}
err = -EIO;
while (numattr--) {
+ int names_size, vals_size;
+
ceph_decode_32_safe(&p, end, len, bad);
namelen = len;
name = p;
+ ceph_decode_need(&p, end, namelen, bad);
p += len;
ceph_decode_32_safe(&p, end, len, bad);
ceph_decode_need(&p, end, len, bad);
val = p;
p += len;
+ if (namelen > INT_MAX || len > INT_MAX ||
+ check_add_overflow(ci->i_xattrs.names_size,
+ (int)namelen, &names_size) ||
+ check_add_overflow(ci->i_xattrs.vals_size,
+ (int)len, &vals_size))
+ goto bad;
err = __set_xattr(ci, name, namelen, val, len,
0, 0, &xattrs[numattr]);
diff --git a/fs/crypto/keysetup.c b/fs/crypto/keysetup.c
index 892044ebcaca..a77b889d7578 100644
--- a/fs/crypto/keysetup.c
+++ b/fs/crypto/keysetup.c
@@ -205,8 +205,14 @@ fscrypt_find_mode_key(struct fscrypt_master_key *mk, u8 hkdf_context,
if (node->hkdf_context == hkdf_context &&
node->mode_num == mode_num &&
node->data_unit_bits == ci->ci_data_unit_bits &&
- fscrypt_is_key_prepared(&node->key, ci))
+ fscrypt_is_key_prepared(&node->key, ci)) {
+ /*
+ * Upgrade the RCU dereference to a full ACQUIRE, so
+ * that blk-crypto global state is seen as initialized.
+ */
+ smp_acquire__after_ctrl_dep();
return &node->key;
+ }
}
return NULL;
}
diff --git a/fs/dax.c b/fs/dax.c
index 1fbba0d21c13..61dce4601aaa 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -2200,6 +2200,7 @@ static int dax_range_compare_iter(struct iomap_iter *it_src,
const struct iomap *dmap = &it_dest->iomap;
loff_t pos1 = it_src->pos, pos2 = it_dest->pos;
void *saddr, *daddr;
+ size_t size1, size2;
int id, ret;
len = min(len, min(smap->length, dmap->length));
@@ -2215,17 +2216,18 @@ static int dax_range_compare_iter(struct iomap_iter *it_src,
}
id = dax_read_lock();
- ret = dax_iomap_direct_access(smap, pos1, ALIGN(pos1 + len, PAGE_SIZE),
- &saddr, NULL);
+ size1 = ALIGN(len + offset_in_page(pos1), PAGE_SIZE);
+ ret = dax_iomap_direct_access(smap, pos1, size1, &saddr, NULL);
if (ret < 0)
goto out_unlock;
- ret = dax_iomap_direct_access(dmap, pos2, ALIGN(pos2 + len, PAGE_SIZE),
- &daddr, NULL);
+ size2 = ALIGN(len + offset_in_page(pos2), PAGE_SIZE);
+ ret = dax_iomap_direct_access(dmap, pos2, size2, &daddr, NULL);
if (ret < 0)
goto out_unlock;
- *same = !memcmp(saddr, daddr, len);
+ *same = !memcmp(saddr + offset_in_page(pos1),
+ daddr + offset_in_page(pos2), len);
if (!*same)
len = 0;
dax_read_unlock(id);
@@ -2264,7 +2266,7 @@ int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff,
status = dax_range_compare_iter(&src_iter, &dst_iter,
min(src_iter.len, dst_iter.len), same);
if (status < 0)
- return ret;
+ return status;
src_iter.status = dst_iter.status = status;
}
return ret;
diff --git a/fs/dcache.c b/fs/dcache.c
index f44c7b39c3b3..f9c9f3b1ba80 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -450,6 +450,16 @@ static void dentry_free(struct dentry *dentry)
call_rcu(&dentry->d_rcu, __d_free);
}
+/*
+ * When inode is unlinked and doesn't have any aliases (i.e., all fds pointing
+ * to it are closed) we report FS_DELETE_SELF to fsnotify and cleanup all inode
+ * marks.
+ */
+static bool inode_notify_dead(struct inode *inode)
+{
+ return !inode->i_nlink && hlist_empty(&inode->i_dentry);
+}
+
/*
* Release the dentry's inode, using the filesystem
* d_iput() operation if defined.
@@ -459,6 +469,7 @@ static void dentry_unlink_inode(struct dentry * dentry)
__releases(dentry->d_inode->i_lock)
{
struct inode *inode = dentry->d_inode;
+ bool notify_dead;
raw_write_seqcount_begin(&dentry->d_seq);
__d_clear_type_and_inode(dentry);
@@ -469,9 +480,10 @@ static void dentry_unlink_inode(struct dentry * dentry)
*/
dentry->waiters = NULL;
raw_write_seqcount_end(&dentry->d_seq);
+ notify_dead = inode_notify_dead(inode);
spin_unlock(&dentry->d_lock);
spin_unlock(&inode->i_lock);
- if (!inode->i_nlink)
+ if (notify_dead)
fsnotify_inoderemove(inode);
if (dentry->d_op && dentry->d_op->d_iput)
dentry->d_op->d_iput(dentry, inode);
@@ -830,7 +842,7 @@ static struct dentry *dentry_kill(struct dentry *dentry)
if (dentry->d_op && dentry->d_op->d_release)
dentry->d_op->d_release(dentry);
- cond_resched();
+ cond_resched_tasks_rcu_qs();
/* now that it's negative, ->d_parent is stable */
if (!IS_ROOT(dentry)) {
parent = dentry->d_parent;
diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c
index 2aff1c7c17de..abe9ae4c643f 100644
--- a/fs/dlm/lowcomms.c
+++ b/fs/dlm/lowcomms.c
@@ -522,10 +522,8 @@ static void lowcomms_write_space(struct sock *sk)
clear_bit(SOCK_NOSPACE, &con->sock->flags);
spin_lock_bh(&con->writequeue_lock);
- if (test_and_clear_bit(CF_APP_LIMITED, &con->flags)) {
+ if (test_and_clear_bit(CF_APP_LIMITED, &con->flags))
con->sock->sk->sk_write_pending--;
- clear_bit(SOCKWQ_ASYNC_NOSPACE, &con->sock->flags);
- }
lowcomms_queue_swork(con);
spin_unlock_bh(&con->writequeue_lock);
@@ -1391,7 +1389,7 @@ static int send_to_sock(struct connection *con)
if (ret == -EAGAIN || ret == 0) {
lock_sock(con->sock->sk);
spin_lock_bh(&con->writequeue_lock);
- if (test_bit(SOCKWQ_ASYNC_NOSPACE, &con->sock->flags) &&
+ if (test_bit(SOCK_NOSPACE, &con->sock->flags) &&
!test_and_set_bit(CF_APP_LIMITED, &con->flags)) {
/* Notify TCP that we're limited by the
* application window size.
diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index d387b27c4ee2..771d862afc2e 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -23,7 +23,7 @@ static int z_erofs_load_lz4_config(struct super_block *sb,
if (!sbi->lz4.max_pclusterblks) {
sbi->lz4.max_pclusterblks = 1; /* reserved case */
} else if (sbi->lz4.max_pclusterblks >
- erofs_blknr(sb, Z_EROFS_PCLUSTER_MAX_SIZE)) {
+ Z_EROFS_PCLUSTER_MAX_SIZE >> sbi->blkszbits) {
erofs_err(sb, "too large lz4 pclusterblks %u",
sbi->lz4.max_pclusterblks);
return -EINVAL;
diff --git a/fs/erofs/ishare.c b/fs/erofs/ishare.c
index a1cf2aff8d32..61f48ceb6cb6 100644
--- a/fs/erofs/ishare.c
+++ b/fs/erofs/ishare.c
@@ -30,9 +30,18 @@ static int erofs_ishare_iget5_set(struct inode *inode, void *data)
return 0;
}
+static int erofs_ishare_backing_release(struct inode *inode, struct file *file)
+{
+ iput(file->private_data);
+ file->private_data = NULL;
+ return 0;
+}
+
bool erofs_ishare_fill_inode(struct inode *inode)
{
- static const struct file_operations empty_fops = {};
+ static const struct file_operations backing_fops = {
+ .release = erofs_ishare_backing_release,
+ };
struct erofs_sb_info *sbi = EROFS_SB(inode->i_sb);
const struct address_space_operations *aops;
struct erofs_inode *vi = EROFS_I(inode);
@@ -50,7 +59,7 @@ bool erofs_ishare_fill_inode(struct inode *inode)
xxh32(fp.opaque, fp.size, 0),
erofs_ishare_iget5_eq, erofs_ishare_iget5_set, &fp);
if (si && (inode_state_read_once(si) & I_NEW)) {
- si->i_fop = &empty_fops;
+ si->i_fop = &backing_fops;
si->i_mapping->a_ops = aops;
si->i_mode = 0444 | S_IFREG;
si->i_size = inode->i_size;
@@ -100,16 +109,24 @@ static int erofs_ishare_file_open(struct inode *inode, struct file *file)
.mnt = erofs_ishare_mnt,
.dentry = EROFS_I(inode)->sharedentry,
};
- struct file *rf;
+ struct file *sf;
if (file->f_flags & O_DIRECT)
return -EINVAL;
- rf = backing_file_open(file, file->f_flags | O_NOATIME,
+ sf = backing_file_open(file, file->f_flags | O_NOATIME,
&sharedpath, current_cred());
- if (IS_ERR(rf))
- return PTR_ERR(rf);
- file->private_data = rf;
+ if (IS_ERR(sf))
+ return PTR_ERR(sf);
+ /*
+ * A VMA only holds a reference to the shared file so that the original
+ * file may be released: the shared file must therefore hold a reference
+ * to one of the real inodes ("inode" passed here), and that reference
+ * will be released by erofs_ishare_backing_release().
+ */
+ ihold(inode);
+ sf->private_data = inode;
+ file->private_data = sf;
return 0;
}
@@ -123,13 +140,13 @@ static int erofs_ishare_file_release(struct inode *inode, struct file *file)
static ssize_t erofs_ishare_file_read_iter(struct kiocb *iocb,
struct iov_iter *to)
{
- struct file *realfile = iocb->ki_filp->private_data;
+ struct file *sf = iocb->ki_filp->private_data;
struct kiocb dedup_iocb;
ssize_t nread;
if (!iov_iter_count(to))
return 0;
- kiocb_clone(&dedup_iocb, iocb, realfile);
+ kiocb_clone(&dedup_iocb, iocb, sf);
nread = filemap_read(&dedup_iocb, to, 0);
iocb->ki_pos = dedup_iocb.ki_pos;
return nread;
@@ -137,12 +154,12 @@ static ssize_t erofs_ishare_file_read_iter(struct kiocb *iocb,
static int erofs_ishare_mmap(struct file *file, struct vm_area_struct *vma)
{
- struct file *realfile = file->private_data;
+ struct file *sf = file->private_data;
int err;
- vma_set_file(vma, realfile);
+ vma_set_file(vma, sf);
- err = security_mmap_backing_file(vma, realfile, file);
+ err = security_mmap_backing_file(vma, sf, file);
if (err)
return err;
@@ -176,7 +193,7 @@ const struct file_operations erofs_ishare_fops = {
struct inode *erofs_real_inode(struct inode *inode, bool *need_iput)
{
struct erofs_inode *vi, *vi_share;
- struct inode *realinode;
+ struct inode *realinode = NULL;
*need_iput = false;
if (inode->i_sb != erofs_ishare_mnt->mnt_sb)
diff --git a/fs/exfat/exfat_fs.h b/fs/exfat/exfat_fs.h
index a9131fe03302..dd6734632bb2 100644
--- a/fs/exfat/exfat_fs.h
+++ b/fs/exfat/exfat_fs.h
@@ -7,6 +7,7 @@
#define _EXFAT_FS_H
#include <linux/fs.h>
+#include <linux/atomic.h>
#include <linux/ratelimit.h>
#include <linux/nls.h>
#include <linux/blkdev.h>
@@ -293,9 +294,16 @@ struct exfat_inode_info {
/* on-disk position of directory entry or 0 */
loff_t i_pos;
- loff_t valid_size;
+ /*
+ * valid_size and zeroed_size are updated from multiple contexts that
+ * are not serialised against each other (page_mkwrite runs without
+ * i_rwsem, while buffered/DIO writes advance them under i_rwsem). Keep
+ * them atomic and only ever advance them with a cmpxchg loop so a
+ * concurrent update can never regress the value or tear on 32-bit.
+ */
+ atomic64_t valid_size;
/* block-aligned size zeroed in the page cache (>= valid_size) */
- loff_t zeroed_size;
+ atomic64_t zeroed_size;
/* hash by i_location */
struct hlist_node i_hash_fat;
struct inode vfs_inode;
@@ -313,6 +321,54 @@ static inline struct exfat_inode_info *EXFAT_I(struct inode *inode)
return container_of(inode, struct exfat_inode_info, vfs_inode);
}
+static inline loff_t exfat_get_valid_size(struct exfat_inode_info *ei)
+{
+ return atomic64_read(&ei->valid_size);
+}
+
+static inline loff_t exfat_get_zeroed_size(struct exfat_inode_info *ei)
+{
+ return atomic64_read(&ei->zeroed_size);
+}
+
+static inline void exfat_set_valid_size(struct exfat_inode_info *ei, loff_t v)
+{
+ atomic64_set(&ei->valid_size, v);
+}
+
+static inline void exfat_set_zeroed_size(struct exfat_inode_info *ei, loff_t v)
+{
+ atomic64_set(&ei->zeroed_size, v);
+}
+
+/*
+ * Monotonically advance @v to at least @new. Returns true if the value was
+ * actually raised, so callers can decide whether to mark the inode dirty.
+ */
+static inline bool exfat_size_advance(atomic64_t *v, loff_t new)
+{
+ loff_t old = atomic64_read(v);
+
+ do {
+ if (old >= new)
+ return false;
+ } while (!atomic64_try_cmpxchg(v, &old, new));
+
+ return true;
+}
+
+static inline bool exfat_advance_valid_size(struct exfat_inode_info *ei,
+ loff_t new)
+{
+ return exfat_size_advance(&ei->valid_size, new);
+}
+
+static inline bool exfat_advance_zeroed_size(struct exfat_inode_info *ei,
+ loff_t new)
+{
+ return exfat_size_advance(&ei->zeroed_size, new);
+}
+
static inline int exfat_forced_shutdown(struct super_block *sb)
{
return test_bit(EXFAT_FLAGS_SHUTDOWN, &EXFAT_SB(sb)->s_exfat_flags);
@@ -623,19 +679,29 @@ static inline int exfat_chain_advance(struct super_block *sb,
struct exfat_chain *chain, unsigned int step)
{
unsigned int clu = chain->dir;
+ unsigned int i;
- if (unlikely(chain->size < step))
- return -EIO;
+ if (chain->flags == ALLOC_NO_FAT_CHAIN) {
+ if (unlikely(chain->size < step))
+ return -EIO;
- if (exfat_cluster_walk(sb, &clu, step, chain->flags))
- return -EIO;
+ chain->size -= step;
+ chain->dir = chain->size ? clu + step : EXFAT_EOF_CLUSTER;
+ return 0;
+ }
- chain->size -= step;
+ /*
+ * Only the FAT tells where a chain of this kind ends. The size can be
+ * smaller than the chain, see commit 1e5654de0f51 ("exfat: handle
+ * wrong stream entry size in exfat_readdir()").
+ */
+ for (i = 0; i < step && clu != EXFAT_EOF_CLUSTER; i++) {
+ if (exfat_get_next_cluster(sb, &clu))
+ return -EIO;
+ }
- if (chain->size == 0 && chain->flags == ALLOC_NO_FAT_CHAIN)
- chain->dir = EXFAT_EOF_CLUSTER;
- else
- chain->dir = clu;
+ chain->size -= min(chain->size, step);
+ chain->dir = clu;
return 0;
}
diff --git a/fs/exfat/file.c b/fs/exfat/file.c
index a2a9ee1a2004..dc57d2c2f0f0 100644
--- a/fs/exfat/file.c
+++ b/fs/exfat/file.c
@@ -17,11 +17,70 @@
#include <linux/fileattr.h>
#include <linux/iomap.h>
#include <linux/pagemap.h>
+#include <linux/rmap.h>
#include "exfat_raw.h"
#include "exfat_fs.h"
#include "iomap.h"
+/*
+ * Unlike pagecache_isize_extended(), this function updates i_size while
+ * holding the straddling folio lock, and does not skip the straddling
+ * folio when the filesystem block size is >= PAGE_SIZE or when 'from'
+ * and 'to' fall within the same filesystem block.
+ *
+ * This ensures that the folio is marked RO and the post-EOF range is
+ * zeroed before readers or writeback can observe the updated i_size.
+ */
+static void exfat_pagecache_isize_extended(struct inode *inode, loff_t from,
+ loff_t to)
+{
+ struct folio *folio;
+
+ if (!(from & (PAGE_SIZE - 1))) {
+ i_size_write(inode, to);
+ return;
+ }
+
+ folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
+
+ /* Folio not cached? Nothing to do */
+ if (IS_ERR(folio)) {
+ i_size_write(inode, to);
+ return;
+ }
+
+ /*
+ * See folio_clear_dirty_for_io() for details why folio_mark_dirty()
+ * is needed.
+ */
+ if (folio_mkclean(folio))
+ folio_mark_dirty(folio);
+
+ /*
+ * Zero the post-EOF range before publishing the new size. Writeback
+ * normally does this, but the extension would expose the tail to
+ * readers before writeback gets a chance to zero it.
+ */
+ if (folio_test_dirty(folio)) {
+ loff_t offset, end;
+
+ offset = from - folio_pos(folio);
+ end = min_t(loff_t, to - folio_pos(folio),
+ folio_size(folio));
+ folio_zero_segment(folio, offset, end);
+ }
+
+ /*
+ * Publish the new size after zeroing the tail. Uptodate folios can
+ * be read without taking the folio lock.
+ */
+ i_size_write(inode, to);
+
+ folio_unlock(folio);
+ folio_put(folio);
+}
+
static int exfat_cont_expand(struct inode *inode, loff_t size)
{
int ret;
@@ -32,8 +91,6 @@ static int exfat_cont_expand(struct inode *inode, loff_t size)
struct exfat_chain clu;
loff_t oldsize = i_size_read(inode);
- truncate_pagecache(inode, oldsize);
-
ret = inode_newsize_ok(inode, size);
if (ret)
return ret;
@@ -81,15 +138,12 @@ static int exfat_cont_expand(struct inode *inode, loff_t size)
out:
inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode));
- /* Expanded range not zeroed, do not update valid_size */
- i_size_write(inode, size);
/*
- * When extending file size, call truncate_pagecache() first,
- * then update i_size, and call pagecache_isize_extended()
- * to ensures the straddling folio is properly marked RO so
- * page_mkwrite() is called and post-EOF area is zeroed.
+ * When extending file size, call exfat_pagecache_isize_extended()
+ * to update i_size and ensure the straddling folio is properly
+ * marked RO so page_mkwrite() is called and post-EOF area is zeroed.
*/
- pagecache_isize_extended(inode, oldsize, inode->i_size);
+ exfat_pagecache_isize_extended(inode, oldsize, size);
inode->i_blocks = round_up(size, sbi->cluster_size) >> 9;
mark_inode_dirty(inode);
@@ -247,8 +301,10 @@ int __exfat_truncate(struct inode *inode)
ei->start_clu = EXFAT_EOF_CLUSTER;
}
- if (i_size_read(inode) < ei->valid_size)
- ei->valid_size = ei->zeroed_size = i_size_read(inode);
+ if (i_size_read(inode) < exfat_get_valid_size(ei)) {
+ exfat_set_valid_size(ei, i_size_read(inode));
+ exfat_set_zeroed_size(ei, i_size_read(inode));
+ }
if (ei->type == TYPE_FILE)
ei->attr |= EXFAT_ATTR_ARCHIVE;
@@ -413,6 +469,7 @@ int exfat_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
* about to be freed.
*/
inode_dio_wait(inode);
+ filemap_invalidate_lock(inode->i_mapping);
truncate_setsize(inode, attr->ia_size);
/*
@@ -420,6 +477,7 @@ int exfat_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
* is already written by it, so mark_inode_dirty() is unneeded.
*/
exfat_truncate(inode);
+ filemap_invalidate_unlock(inode->i_mapping);
} else
mark_inode_dirty(inode);
@@ -654,98 +712,42 @@ int exfat_file_fsync(struct file *filp, loff_t start, loff_t end, int datasync)
}
/*
- * exfat_zero_new_range - zero [start, end) without overwriting uptodate blocks
+ * exfat_prepare_valid_range - populate and dirty folios covering [start, end)
+ *
+ * Populate missing blocks via the read path, which zero-fills data
+ * beyond the current valid_size and skips uptodate blocks.
+ * Mark entire folios dirty so the newly valid range is written back.
*
- * Uptodate blocks may contain data written through a shared mapping beyond
- * valid_size.
+ * Call before advancing valid_size.
*/
-static int exfat_zero_new_range(struct inode *inode, loff_t start, loff_t end)
+static int exfat_prepare_valid_range(struct inode *inode, loff_t start,
+ loff_t end)
{
struct address_space *mapping = inode->i_mapping;
- unsigned int blocksize = i_blocksize(inode);
- loff_t pos = start;
- int err;
+ loff_t next, pos = start;
+ struct folio *folio;
+ pgoff_t index;
- while (pos < end) {
- loff_t next = min_t(loff_t,
- round_down(pos, PAGE_SIZE) + PAGE_SIZE, end);
- struct folio *folio;
- loff_t bpos;
-
- folio = filemap_get_folio(mapping, pos >> PAGE_SHIFT);
- if (IS_ERR(folio)) {
- err = iomap_zero_range(inode, pos, next - pos, NULL,
- &exfat_iomap_ops, NULL, NULL);
- if (err < 0)
- return err;
- pos = next;
- continue;
- }
-
- if (folio_test_uptodate(folio)) {
- folio_lock(folio);
- if (folio->mapping == mapping)
- folio_mark_dirty(folio);
- folio_unlock(folio);
- folio_put(folio);
- pos = next;
- continue;
- }
+ if (pos >= end)
+ return 0;
- /*
- * Zero not-uptodate block runs. iomap_zero_range() requires an
- * unlocked folio, so recheck ->mapping after each call.
- */
- folio_lock(folio);
- bpos = pos;
- while (bpos < next) {
- loff_t rstart, rend;
-
- if (folio->mapping != mapping) {
- folio_unlock(folio);
- err = iomap_zero_range(inode, bpos, next - bpos,
- NULL, &exfat_iomap_ops, NULL, NULL);
- if (err < 0) {
- folio_put(folio);
- return err;
- }
- folio_lock(folio);
- break;
- }
-
- if (iomap_is_partially_uptodate(folio,
- offset_in_folio(folio, bpos), blocksize)) {
- bpos += blocksize;
- continue;
- }
+ while (pos < end) {
+ index = pos >> PAGE_SHIFT;
+ next = min(((loff_t)index + 1) << PAGE_SHIFT, end);
- rstart = bpos;
- rend = min_t(loff_t, bpos + blocksize, next);
- while (rend < next &&
- !iomap_is_partially_uptodate(folio,
- offset_in_folio(folio, rend), blocksize))
- rend = min_t(loff_t, rend + blocksize, next);
+ balance_dirty_pages_ratelimited(mapping);
- folio_unlock(folio);
- err = iomap_zero_range(inode, rstart, rend - rstart,
- NULL, &exfat_iomap_ops, NULL, NULL);
- if (err < 0) {
- folio_put(folio);
- return err;
- }
- folio_lock(folio);
- bpos = rend;
- }
+ folio = read_mapping_folio(mapping, index, NULL);
+ if (IS_ERR(folio))
+ return PTR_ERR(folio);
- /*
- * Dirty only a fully uptodate folio. Dirtying a partial folio could
- * write uninitialised cache contents over valid on-disk blocks.
- */
- if (folio->mapping == mapping && folio_test_uptodate(folio))
+ folio_lock(folio);
+ if (folio->mapping == mapping) {
folio_mark_dirty(folio);
+ pos = next;
+ }
folio_unlock(folio);
folio_put(folio);
- pos = next;
}
return 0;
@@ -754,12 +756,14 @@ static int exfat_zero_new_range(struct inode *inode, loff_t start, loff_t end)
static int exfat_extend_valid_size(struct inode *inode, loff_t new_valid_size)
{
struct exfat_inode_info *ei = EXFAT_I(inode);
- loff_t old_valid_size = ei->valid_size;
+ loff_t old_valid_size = exfat_get_valid_size(ei);
int ret = 0;
if (old_valid_size < new_valid_size) {
+ inode_dio_wait(inode);
+
/* Do not re-zero blocks already covered by zeroed_size. */
- loff_t gap_start = max(old_valid_size, ei->zeroed_size);
+ loff_t gap_start = max(old_valid_size, exfat_get_zeroed_size(ei));
if (i_size_read(inode) < new_valid_size) {
/*
@@ -781,23 +785,38 @@ static int exfat_extend_valid_size(struct inode *inode, loff_t new_valid_size)
if (gap_start < new_valid_size)
unmap_mapping_range(inode->i_mapping, gap_start,
new_valid_size - gap_start, 0);
- ret = exfat_zero_new_range(inode, gap_start, new_valid_size);
+ ret = exfat_prepare_valid_range(inode, gap_start, new_valid_size);
filemap_invalidate_unlock(inode->i_mapping);
if (ret) {
+ filemap_invalidate_lock(inode->i_mapping);
truncate_setsize(inode, old_valid_size);
exfat_truncate(inode);
+ filemap_invalidate_unlock(inode->i_mapping);
return ret;
}
- ei->valid_size = new_valid_size;
- if (ei->zeroed_size < round_up(new_valid_size, i_blocksize(inode)))
- ei->zeroed_size = round_up(new_valid_size, i_blocksize(inode));
+ exfat_set_valid_size(ei, new_valid_size);
+ exfat_advance_zeroed_size(ei,
+ round_up(new_valid_size, i_blocksize(inode)));
mark_inode_dirty(inode);
}
return ret;
}
+/*
+ * The clusters for a write are allocated before the data is copied. Free the
+ * ones that a failed or short write has left beyond the end of the file.
+ */
+static void exfat_write_failed(struct inode *inode)
+{
+ inode_dio_wait(inode);
+ filemap_invalidate_lock(inode->i_mapping);
+ truncate_pagecache(inode, i_size_read(inode));
+ exfat_truncate(inode);
+ filemap_invalidate_unlock(inode->i_mapping);
+}
+
static ssize_t exfat_fallback_buffered_write(struct kiocb *iocb,
struct iov_iter *from)
{
@@ -807,6 +826,8 @@ static ssize_t exfat_fallback_buffered_write(struct kiocb *iocb,
iocb->ki_flags &= ~IOCB_DIRECT;
+ inode_dio_wait(file_inode(iocb->ki_filp));
+
written = iomap_file_buffered_write(iocb, from, &exfat_write_iomap_ops,
NULL, NULL);
if (written < 0)
@@ -854,8 +875,7 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
struct file *file = iocb->ki_filp;
struct inode *inode = file_inode(file);
struct exfat_inode_info *ei = EXFAT_I(inode);
- loff_t pos = iocb->ki_pos;
- loff_t valid_size;
+ loff_t pos, valid_size, end;
int err;
if (unlikely(exfat_forced_shutdown(inode->i_sb)))
@@ -863,11 +883,6 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
inode_lock(inode);
- if (pos > i_size_read(inode))
- truncate_pagecache(inode, i_size_read(inode));
-
- valid_size = ei->valid_size;
-
ret = generic_write_checks(iocb, iter);
if (ret <= 0)
goto unlock;
@@ -878,6 +893,12 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
goto unlock;
}
+ pos = iocb->ki_pos;
+ end = pos + iov_iter_count(iter);
+ if (pos > i_size_read(inode))
+ truncate_pagecache(inode, i_size_read(inode));
+
+ valid_size = exfat_get_valid_size(ei);
if (pos > valid_size) {
ret = exfat_extend_valid_size(inode, pos);
if (ret < 0 && ret != -ENOSPC) {
@@ -887,21 +908,28 @@ static ssize_t exfat_file_write_iter(struct kiocb *iocb, struct iov_iter *iter)
}
if (ret < 0)
goto unlock;
+
+ pos = valid_size;
}
- if (iocb->ki_flags & IOCB_DIRECT)
+ if (iocb->ki_flags & IOCB_DIRECT) {
ret = exfat_dio_write_iter(iocb, iter);
- else
+ } else {
+ /*
+ * Prevent concurrent direct I/O and buffered I/O to the same file
+ * range. Wait for in-flight DIO to finish before dirtying pages.
+ */
+ inode_dio_wait(inode);
ret = iomap_file_buffered_write(iocb, iter,
&exfat_write_iomap_ops, NULL, NULL);
+ }
+ if (ret != -EIOCBQUEUED && end > i_size_read(inode))
+ exfat_write_failed(inode);
if (ret < 0)
goto unlock;
inode_unlock(inode);
- if (pos > valid_size)
- pos = valid_size;
-
if (iocb->ki_pos > pos) {
ssize_t err = generic_write_sync(iocb, iocb->ki_pos - pos);
@@ -954,8 +982,10 @@ static vm_fault_t exfat_page_mkwrite(struct vm_fault *vmf)
fault_page_start = ((loff_t)vmf->pgoff) << PAGE_SHIFT;
new_valid_size = min(mmap_valid_size, i_size_read(inode));
- if (ei->valid_size < new_valid_size) {
- if (ei->zeroed_size < fault_page_start) {
+ if (exfat_get_valid_size(ei) < new_valid_size) {
+ loff_t zeroed_size = exfat_get_zeroed_size(ei);
+
+ if (zeroed_size < fault_page_start) {
int err;
/*
@@ -963,7 +993,7 @@ static vm_fault_t exfat_page_mkwrite(struct vm_fault *vmf)
* fault populated its folio and iomap_page_mkwrite()
* will dirty it.
*/
- err = exfat_zero_new_range(inode, ei->zeroed_size,
+ err = exfat_prepare_valid_range(inode, zeroed_size,
fault_page_start);
if (err < 0) {
inode_unlock(inode);
@@ -976,9 +1006,9 @@ static vm_fault_t exfat_page_mkwrite(struct vm_fault *vmf)
* at i_size recording blocks wholly beyond it could skip a
* later required zeroing.
*/
- if (ei->zeroed_size < round_up(new_valid_size, i_blocksize(inode)))
- ei->zeroed_size = round_up(new_valid_size, i_blocksize(inode));
- ei->valid_size = new_valid_size;
+ exfat_advance_zeroed_size(ei,
+ round_up(new_valid_size, i_blocksize(inode)));
+ exfat_set_valid_size(ei, new_valid_size);
mark_inode_dirty(inode);
}
diff --git a/fs/exfat/inode.c b/fs/exfat/inode.c
index ccd13630187e..5129df1e79b4 100644
--- a/fs/exfat/inode.c
+++ b/fs/exfat/inode.c
@@ -29,6 +29,7 @@ int __exfat_write_inode(struct inode *inode, int sync)
struct exfat_sb_info *sbi = EXFAT_SB(sb);
struct exfat_inode_info *ei = EXFAT_I(inode);
bool is_dir = (ei->type == TYPE_DIR);
+ loff_t valid_size = exfat_get_valid_size(ei);
struct timespec64 ts;
if (inode->i_ino == EXFAT_ROOT_INO)
@@ -81,7 +82,7 @@ int __exfat_write_inode(struct inode *inode, int sync)
* preventing fsck from reporting "more clusters are allocated".
*/
on_disk_size = max_t(unsigned long long, i_size_read(inode),
- ei->valid_size);
+ valid_size);
if (ei->start_clu == EXFAT_EOF_CLUSTER)
on_disk_size = 0;
@@ -89,7 +90,7 @@ int __exfat_write_inode(struct inode *inode, int sync)
* valid_size on disk must reflect only confirmed data (up to i_size)
* and must not exceed on_disk_size.
*/
- on_disk_valid_size = min_t(unsigned long long, ei->valid_size,
+ on_disk_valid_size = min_t(unsigned long long, valid_size,
i_size_read(inode));
if (ei->start_clu == EXFAT_EOF_CLUSTER)
on_disk_valid_size = 0;
@@ -258,15 +259,16 @@ static void exfat_readahead(struct readahead_control *rac)
struct inode *inode = mapping->host;
struct exfat_inode_info *ei = EXFAT_I(inode);
loff_t pos = readahead_pos(rac);
+ loff_t valid_size = exfat_get_valid_size(ei);
struct iomap_read_folio_ctx ctx = {
.ops = &exfat_iomap_bio_read_ops,
.rac = rac,
};
/* Range cross valid_size, read it page by page. */
- if (ei->valid_size < i_size_read(inode) &&
- pos <= ei->valid_size &&
- ei->valid_size < pos + readahead_length(rac))
+ if (valid_size < i_size_read(inode) &&
+ pos <= valid_size &&
+ valid_size < pos + readahead_length(rac))
return;
iomap_readahead(&exfat_iomap_ops, &ctx, NULL);
@@ -371,8 +373,8 @@ static int exfat_fill_inode(struct inode *inode, struct exfat_dir_entry *info)
ei->start_clu = info->start_clu;
ei->flags = info->flags;
ei->type = info->type;
- ei->valid_size = info->valid_size;
- ei->zeroed_size = info->valid_size;
+ exfat_set_valid_size(ei, info->valid_size);
+ exfat_set_zeroed_size(ei, info->valid_size);
ei->version = 0;
ei->hint_stat.eidx = 0;
diff --git a/fs/exfat/iomap.c b/fs/exfat/iomap.c
index 8911aa84a730..f734d13d3fce 100644
--- a/fs/exfat/iomap.c
+++ b/fs/exfat/iomap.c
@@ -46,6 +46,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
struct exfat_inode_info *ei = EXFAT_I(inode);
unsigned int cluster, num_clusters;
loff_t cluster_offset, cluster_length;
+ loff_t valid_size = exfat_get_valid_size(ei);
int err;
bool balloc = false;
@@ -74,6 +75,17 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
err = exfat_map_cluster(inode, exfat_bytes_to_cluster(sbi, offset),
&cluster, &num_clusters, may_alloc, &balloc);
+ if (err == -ENOSPC && !(flags & IOMAP_DIRECT)) {
+ /*
+ * There are not enough free clusters for the whole range. A
+ * buffered write can be short, so map what still fits.
+ */
+ num_clusters = 1;
+ err = exfat_map_cluster(inode,
+ exfat_bytes_to_cluster(sbi, offset),
+ &cluster, &num_clusters, may_alloc,
+ &balloc);
+ }
if (err)
goto out;
@@ -88,7 +100,7 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
if (may_alloc || flags & IOMAP_ZERO) {
if (balloc)
iomap->flags |= IOMAP_F_NEW;
- else if (iomap->offset + iomap->length >= ei->valid_size) {
+ else if (iomap->offset + iomap->length >= valid_size) {
/*
* This is a write that starts at or extends beyond
* the current valid_size. The region between the old
@@ -114,10 +126,10 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
* return IOMAP_UNWRITTEN so the write path can
* distinguish it from a real hole.
*/
- if (offset >= ei->valid_size) {
+ if (offset >= valid_size) {
iomap->type = flags & IOMAP_REPORT ?
IOMAP_HOLE : IOMAP_UNWRITTEN;
- } else if (offset + iomap->length > ei->valid_size) {
+ } else if (offset + iomap->length > valid_size) {
if (flags & IOMAP_REPORT) {
/*
* For SEEK_HOLE/SEEK_DATA, clip the length
@@ -125,9 +137,9 @@ static int __exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length
* This ensures the caller gets the precise
* hole position in byte units.
*/
- iomap->length = ei->valid_size - iomap->offset;
+ iomap->length = valid_size - iomap->offset;
} else
- iomap->length = round_up(ei->valid_size,
+ iomap->length = round_up(valid_size,
i_blocksize(inode)) -
iomap->offset;
}
@@ -148,7 +160,27 @@ static int exfat_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
static int exfat_write_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
unsigned int flags, struct iomap *iomap, struct iomap *srcmap)
{
- return __exfat_iomap_begin(inode, offset, length, flags, iomap, true);
+ struct exfat_inode_info *ei = EXFAT_I(inode);
+ loff_t valid_size = exfat_get_valid_size(ei);
+ loff_t end;
+ int err;
+
+ err = __exfat_iomap_begin(inode, offset, length, flags, iomap, true);
+ if (err || flags & (IOMAP_DIRECT | IOMAP_FAULT) ||
+ iomap->flags & IOMAP_F_NEW)
+ return err;
+
+ end = iomap->offset + iomap->length;
+ if (end >= valid_size ||
+ round_up(end, i_blocksize(inode)) <= valid_size)
+ return 0;
+
+ /*
+ * Zero the invalid bytes from valid_size to the end of the block
+ * to prevent stale data from being exposed through the page cache.
+ */
+ return iomap_truncate_page(inode, valid_size, NULL, &exfat_iomap_ops,
+ NULL, NULL);
}
static DEFINE_IOMAP_ITER_NEXT(exfat_iomap_next, exfat_iomap_begin);
@@ -175,10 +207,8 @@ static int exfat_write_iomap_end(struct inode *inode, loff_t pos, loff_t length,
end = pos + written;
- if (ei->valid_size < end) {
- ei->valid_size = end;
+ if (exfat_advance_valid_size(ei, end))
dirtied = true;
- }
/*
* IOMAP_F_ZERO_TAIL zeroes the remainder of the last block. Track that
@@ -186,8 +216,7 @@ static int exfat_write_iomap_end(struct inode *inode, loff_t pos, loff_t length,
*/
if (iomap->flags & IOMAP_F_ZERO_TAIL)
end = round_up(end, i_blocksize(inode));
- if (ei->zeroed_size < end)
- ei->zeroed_size = end;
+ exfat_advance_zeroed_size(ei, end);
if (dirtied || iomap->flags & IOMAP_F_SIZE_CHANGED)
mark_inode_dirty(inode);
@@ -250,7 +279,7 @@ static void exfat_iomap_read_end_io(struct bio *bio)
s64 valid_size;
loff_t pos = folio_pos(folio);
- valid_size = ei->valid_size;
+ valid_size = exfat_get_valid_size(ei);
if (pos + iter.offset < valid_size &&
pos + iter.offset + iter.length > valid_size)
folio_zero_segment(folio, offset_in_folio(folio, valid_size),
diff --git a/fs/exfat/namei.c b/fs/exfat/namei.c
index a4dc83b5949c..1ab26833ba91 100644
--- a/fs/exfat/namei.c
+++ b/fs/exfat/namei.c
@@ -380,7 +380,7 @@ int exfat_find_empty_entry(struct inode *inode,
/* directory inode should be updated in here */
i_size_write(inode, size);
- ei->valid_size += sbi->cluster_size;
+ atomic64_add(sbi->cluster_size, &ei->valid_size);
ei->flags = p_dir->flags;
inode->i_blocks += sbi->cluster_size >> 9;
}
@@ -1248,7 +1248,7 @@ static int __exfat_rename(struct inode *old_parent_inode,
}
i_size_write(new_inode, 0);
- new_ei->valid_size = 0;
+ exfat_set_valid_size(new_ei, 0);
new_ei->start_clu = EXFAT_EOF_CLUSTER;
new_ei->flags = ALLOC_NO_FAT_CHAIN;
}
diff --git a/fs/exportfs/expfs.c b/fs/exportfs/expfs.c
index eafd99507afe..b5768f94fbb5 100644
--- a/fs/exportfs/expfs.c
+++ b/fs/exportfs/expfs.c
@@ -336,7 +336,11 @@ static int get_name(const struct path *path, char *name, struct dentry *child)
}
out_close:
- fput(file);
+ /*
+ * @file is the read-only directory opened above, so __fput()
+ * cannot block on anything the caller holds.
+ */
+ __fput_sync(file);
out:
return error;
}
diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c
index 52f4c5169f91..778fe8788f06 100644
--- a/fs/ext4/balloc.c
+++ b/fs/ext4/balloc.c
@@ -329,7 +329,7 @@ struct ext4_group_info *ext4_get_group_info(struct super_block *sb,
struct ext4_group_info **grp_info;
long indexv, indexh;
- if (unlikely(group >= EXT4_SB(sb)->s_groups_count))
+ if (unlikely(group >= ext4_get_groups_count(sb)))
return NULL;
if (unlikely(!EXT4_SB(sb)->s_group_info))
return NULL;
diff --git a/fs/ext4/crypto.c b/fs/ext4/crypto.c
index 1a0fccb084ef..318b2eaea741 100644
--- a/fs/ext4/crypto.c
+++ b/fs/ext4/crypto.c
@@ -156,6 +156,15 @@ static int ext4_set_context(struct inode *inode, const void *ctx, size_t len,
if (ext4_test_inode_flag(inode, EXT4_INODE_DAX))
return -EOPNOTSUPP;
+ /*
+ * Encryption is not supported when the block size is larger
+ * than the page size. This is rejected at mount time by
+ * ext4_check_large_folio(), and this check here is for the case
+ * when the 'encrypt' feature is enabled on a mounted filesystem.
+ */
+ if (inode->i_sb->s_blocksize > PAGE_SIZE)
+ return -EOPNOTSUPP;
+
res = ext4_convert_inline_data(inode);
if (res)
return res;
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index af70bb26bb54..859227f36162 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3907,6 +3907,9 @@ extern void ext4_ext_tree_init(handle_t *handle, struct inode *inode);
extern int ext4_ext_index_trans_blocks(struct inode *inode, int extents);
extern int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
struct ext4_map_blocks *map, int flags);
+ext4_lblk_t ext4_determine_insert_hole(struct inode *inode, ext4_lblk_t lblk,
+ ext4_lblk_t hole_start,
+ ext4_lblk_t hole_len);
extern int ext4_ext_truncate(handle_t *, struct inode *);
extern int ext4_ext_remove_space(struct inode *inode, ext4_lblk_t start,
ext4_lblk_t end);
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 3bf0a6cdbec2..c36beaf7a425 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -4190,21 +4190,19 @@ static int get_implied_cluster_alloc(struct super_block *sb,
}
/*
- * Determine hole length around the given logical block, first try to
- * locate and expand the hole from the given @path, and then adjust it
- * if it's partially or completely converted to delayed extents, insert
- * it into the extent cache tree if it's indeed a hole, finally return
- * the length of the determined extent.
+ * Adjust the hole [@hole_start, @hole_start + @hole_len) the caller found
+ * around the queried block @lblk if it's partially or completely converted
+ * to delayed extents, insert it into the extent cache tree if it's indeed a
+ * hole, finally return the length of the determined extent starting at
+ * @lblk. @hole_start must not be behind @lblk.
*/
-static ext4_lblk_t ext4_ext_determine_insert_hole(struct inode *inode,
- struct ext4_ext_path *path,
- ext4_lblk_t lblk)
+ext4_lblk_t ext4_determine_insert_hole(struct inode *inode, ext4_lblk_t lblk,
+ ext4_lblk_t hole_start,
+ ext4_lblk_t hole_len)
{
- ext4_lblk_t hole_start, len;
+ ext4_lblk_t len = hole_len;
struct extent_status es;
- hole_start = lblk;
- len = ext4_ext_find_hole(inode, path, &hole_start);
again:
ext4_es_find_extent_range(inode, &ext4_es_is_delayed, hole_start,
hole_start + len - 1, &es);
@@ -4371,9 +4369,11 @@ int ext4_ext_map_blocks(handle_t *handle, struct inode *inode,
* we couldn't try to create block if flags doesn't contain EXT4_GET_BLOCKS_CREATE
*/
if ((flags & EXT4_GET_BLOCKS_CREATE) == 0) {
- ext4_lblk_t len;
+ ext4_lblk_t hole_start = map->m_lblk, len;
- len = ext4_ext_determine_insert_hole(inode, path, map->m_lblk);
+ len = ext4_ext_find_hole(inode, path, &hole_start);
+ len = ext4_determine_insert_hole(inode, map->m_lblk,
+ hole_start, len);
map->m_pblk = 0;
map->m_len = min_t(unsigned int, map->m_len, len);
diff --git a/fs/ext4/indirect.c b/fs/ext4/indirect.c
index 5aec759eed70..a495080df710 100644
--- a/fs/ext4/indirect.c
+++ b/fs/ext4/indirect.c
@@ -586,6 +586,23 @@ int ext4_ind_map_blocks(handle_t *handle, struct inode *inode,
for (i = partial - chain + 1; i < depth; i++)
count = count * epb + (epb - offsets[i] - 1);
count++;
+
+ /*
+ * The count knows nothing about delayed allocation, so let
+ * the common helper reconcile it with the extent status
+ * tree. Only when the lookup really did find a hole:
+ * ext4_get_branch() also returns a non-NULL partial when it
+ * failed, and caching a hole for a range it could not read
+ * would answer every later read from that cache instead of
+ * retrying. Clamp the count first, a subtree can be bigger
+ * than the logical block space when the block size is large.
+ */
+ if (!err) {
+ count = umin(count, EXT_MAX_BLOCKS - map->m_lblk);
+ count = ext4_determine_insert_hole(inode, map->m_lblk,
+ map->m_lblk, count);
+ }
+
/* Fill in size of a hole we found */
map->m_pblk = 0;
map->m_len = umin(map->m_len, count);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 26f0f9714f03..6697c33d3aa3 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -5671,6 +5671,12 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
ret = -EFSCORRUPTED;
goto bad_inode;
}
+ if (IS_ENCRYPTED(inode) && sb->s_blocksize > PAGE_SIZE) {
+ ext4_error_inode(inode, function, line, 0,
+ "encrypted inode with block size larger than page size");
+ ret = -EFSCORRUPTED;
+ goto bad_inode;
+ }
ext4_set_inode_mapping_order(inode);
@@ -6318,15 +6324,12 @@ int ext4_getattr(struct mnt_idmap *idmap, const struct path *path,
stat->attributes |= STATX_ATTR_IMMUTABLE;
if (flags & EXT4_NODUMP_FL)
stat->attributes |= STATX_ATTR_NODUMP;
- if (flags & EXT4_VERITY_FL)
- stat->attributes |= STATX_ATTR_VERITY;
stat->attributes_mask |= (STATX_ATTR_APPEND |
STATX_ATTR_COMPRESSED |
STATX_ATTR_ENCRYPTED |
STATX_ATTR_IMMUTABLE |
- STATX_ATTR_NODUMP |
- STATX_ATTR_VERITY);
+ STATX_ATTR_NODUMP);
generic_fillattr(idmap, request_mask, inode, stat);
return 0;
@@ -6603,8 +6606,14 @@ static int ext4_try_to_expand_extra_isize(struct inode *inode,
* When !SB_ACTIVE, iput triggers write_inode_now() which acquires
* s_writepages_rwsem, causing a deadlock with the caller's active
* jbd2 handle (lock order: s_writepages_rwsem -> jbd2_handle).
+ *
+ * Skip it while unmounting as well. shrink_dcache_for_umount()
+ * clears s_root before generic_shutdown_super() clears SB_ACTIVE, and
+ * the last iput() of a lazytime inode in that window redirties it and
+ * lands here. Moving xattrs out to a block then needs a new EA inode,
+ * which ext4_xattr_inode_create() refuses without s_root.
*/
- if (unlikely(!(inode->i_sb->s_flags & SB_ACTIVE)))
+ if (unlikely(!(inode->i_sb->s_flags & SB_ACTIVE) || !inode->i_sb->s_root))
return -EBUSY;
/*
diff --git a/fs/ext4/ioctl.c b/fs/ext4/ioctl.c
index c8387e6a2c6e..1f6d7afb62ec 100644
--- a/fs/ext4/ioctl.c
+++ b/fs/ext4/ioctl.c
@@ -1362,7 +1362,8 @@ static void ext4_sb_setparams(struct ext4_sb_info *sbi,
es->s_encoding = cpu_to_le16(params->encoding);
if (params->set_flags & EXT4_TUNE_FL_ENCODING_FLAGS)
es->s_encoding_flags = cpu_to_le16(params->encoding_flags);
- strscpy_pad(es->s_mount_opts, params->mount_opts);
+ if (params->set_flags & EXT4_TUNE_FL_MOUNT_OPTS)
+ strscpy_pad(es->s_mount_opts, params->mount_opts);
if (params->set_flags & EXT4_TUNE_FL_EDIT_FEATURES) {
es->s_feature_compat |=
cpu_to_le32(params->set_feature_compat_mask);
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index a6386c1d237f..231a274b814e 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -3209,19 +3209,20 @@ static int ext4_rmdir(struct inode *dir, struct dentry *dentry)
ext4_fc_track_unlink(handle, dentry);
retval = ext4_mark_inode_dirty(handle, dir);
+end_rmdir:
+ brelse(bh);
+ if (handle)
+ ext4_journal_stop(handle);
+
/* VFS negative dentries are incompatible with Encoding and
* Case-insensitiveness. Eventually we'll want avoid
* invalidating the dentries here, alongside with returning the
* negative dentries at ext4_lookup(), when it is better
* supported by the VFS for the CI case.
*/
- if (IS_ENABLED(CONFIG_UNICODE) && IS_CASEFOLDED(dir))
+ if (!retval && IS_ENABLED(CONFIG_UNICODE) && IS_CASEFOLDED(dir))
d_invalidate(dentry);
-end_rmdir:
- brelse(bh);
- if (handle)
- ext4_journal_stop(handle);
return retval;
}
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 3ace06b4f9e3..a8c4a227edf1 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -2840,6 +2840,14 @@ static int ext4_check_opt_consistency(struct fs_context *fc,
}
if (is_remount) {
+ if (test_opt(sb, DELALLOC) &&
+ (ctx->mask_s_mount_opt & EXT4_MOUNT_DELALLOC) &&
+ !ctx_test_mount_opt(ctx, EXT4_MOUNT_DELALLOC)) {
+ ext4_msg(sb, KERN_ERR,
+ "can't disable delalloc during remount");
+ return -EINVAL;
+ }
+
if (!sbi->s_journal &&
ctx_test_mount_opt(ctx, EXT4_MOUNT_DATA_ERR_ABORT)) {
ext4_msg(NULL, KERN_WARNING,
@@ -6133,6 +6141,7 @@ static int ext4_load_journal(struct super_block *sb,
int err = 0;
int really_read_only;
int journal_dev_ro;
+ bool enable_write = false;
if (WARN_ON_ONCE(!ext4_has_feature_journal(sb)))
return -EFSCORRUPTED;
@@ -6189,6 +6198,7 @@ static int ext4_load_journal(struct super_block *sb,
}
ext4_msg(sb, KERN_INFO, "write access will "
"be enabled during recovery");
+ enable_write = true;
}
}
@@ -6205,7 +6215,19 @@ static int ext4_load_journal(struct super_block *sb,
if (save)
memcpy(save, ((char *) es) +
EXT4_S_ERR_START, EXT4_S_ERR_LEN);
+ /*
+ * Fast commit replay performs regular ext4 metadata updates
+ * (see ext4_fc_replay()) which refuse to run on a read-only
+ * superblock. We promised write access above, so make that
+ * true for the duration of the recovery, the same way
+ * ext4_orphan_cleanup() does. The superblock is not published
+ * yet, so nothing can observe the transient state.
+ */
+ if (enable_write)
+ sb->s_flags &= ~SB_RDONLY;
err = jbd2_journal_load(journal);
+ if (enable_write)
+ sb->s_flags |= SB_RDONLY;
if (save && memcmp(((char *) es) + EXT4_S_ERR_START,
save, EXT4_S_ERR_LEN)) {
memcpy(((char *) es) + EXT4_S_ERR_START,
@@ -6698,13 +6720,6 @@ static int __ext4_remount(struct fs_context *fc, struct super_block *sb)
goto restore_opts;
}
- if ((old_opts.s_mount_opt & EXT4_MOUNT_DELALLOC) &&
- !test_opt(sb, DELALLOC)) {
- ext4_msg(sb, KERN_ERR, "can't disable delalloc during remount");
- err = -EINVAL;
- goto restore_opts;
- }
-
sb->s_flags = (sb->s_flags & ~SB_POSIXACL) |
(test_opt(sb, POSIX_ACL) ? SB_POSIXACL : 0);
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index ce88092d9ce2..b71107604012 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -1222,6 +1222,7 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
int i;
int err;
+repeat:
err = f2fs_is_compressed_cluster(inode, start_idx);
if (err < 0)
return err;
@@ -1233,12 +1234,11 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
/* truncate compressed cluster */
err = f2fs_prepare_compress_overwrite(inode, &pagep,
start_idx, &fsdata);
-
- /* should not be a normal cluster */
- f2fs_bug_on(F2FS_I_SB(inode), err == 0);
-
- if (err <= 0)
+ if (err < 0)
return err;
+ else if (err == 0)
+ /* the cluster became non-compressed one due to race case */
+ goto repeat;
rpages = fsdata;
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 21f396ebe22c..cfd81bfc28a2 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -303,7 +303,7 @@ static void f2fs_read_end_io(struct bio *bio)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(bio_first_folio_all(bio));
struct bio_post_read_ctx *ctx;
- bool intask = in_task() && !irqs_disabled();
+ bool intask = !bio_in_atomic();
iostat_update_and_unbind_ctx(bio);
ctx = bio->bi_private;
@@ -2260,7 +2260,6 @@ static int f2fs_read_single_page(struct inode *inode, struct fsverity_info *vi,
got_it:
if ((map->m_flags & F2FS_MAP_MAPPED)) {
block_nr = map->m_pblk + block_in_file - map->m_lblk;
- folio_set_mappedtodisk(folio);
if (!f2fs_is_valid_blkaddr(F2FS_I_SB(inode), block_nr,
DATA_GENERIC_ENHANCE_READ)) {
@@ -2909,10 +2908,12 @@ bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio)
return true;
if (f2fs_used_in_atomic_write(inode))
return true;
- /* rewrite low ratio compress data w/ OPU mode to avoid fragmentation */
- if (f2fs_compressed_file(inode) &&
- F2FS_OPTION(sbi).compress_mode == COMPR_MODE_USER &&
- is_inode_flag_set(inode, FI_ENABLE_COMPRESS))
+ /*
+ * rewrite low ratio compress data w/ OPU mode to avoid fragmentation.
+ * If IO comes from compressed write path and fallback to raw write,
+ * force out‑place to prevent metadata‑data inconsistency.
+ */
+ if (fio && f2fs_compressed_file(inode))
return true;
/* swap file is migrating in aligned write mode */
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 9940a6cecf1a..0c85d90307f7 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -466,6 +466,7 @@ struct discard_entry {
#define MAX_PLIST_NUM 512
#define plist_idx(blk_num) ((blk_num) >= MAX_PLIST_NUM ? \
(MAX_PLIST_NUM - 1) : ((blk_num) - 1))
+#define MAX_DISCARD_DROP_COUNT 512
enum {
D_PREP, /* initial */
@@ -4064,6 +4065,8 @@ void f2fs_reserve_device_alias(struct f2fs_sb_info *sbi, block_t addr,
bool f2fs_is_checkpointed_data(struct f2fs_sb_info *sbi, block_t blkaddr);
int f2fs_start_discard_thread(struct f2fs_sb_info *sbi);
void f2fs_drop_discard_cmd(struct f2fs_sb_info *sbi);
+void f2fs_drop_discard_cmd_range(struct f2fs_sb_info *sbi,
+ block_t start, block_t len);
void f2fs_stop_discard_thread(struct f2fs_sb_info *sbi);
bool f2fs_issue_discard_timeout(struct f2fs_sb_info *sbi, bool need_check);
void f2fs_clear_prefree_segments(struct f2fs_sb_info *sbi,
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index edc352569e87..c477e6d15314 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -124,6 +124,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct dnode_of_data dn;
bool need_alloc = !f2fs_is_pinned_file(inode);
+ loff_t isize;
int err = 0;
vm_fault_t ret;
@@ -186,9 +187,26 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
file_update_time(vmf->vma->vm_file);
filemap_invalidate_lock_shared(inode->i_mapping);
+ /*
+ * mmap stores bypass the COW inode and would break atomicity. Check
+ * under invalidate_lock: f2fs_ioc_start_atomic_write() holds it while
+ * flushing dirty folios and setting FI_ATOMIC_FILE, so either it sees
+ * our dirty folio and writes it back, or we see the flag.
+ */
+ if (unlikely(f2fs_is_atomic_file(inode))) {
+ err = -EOPNOTSUPP;
+ goto out_sem;
+ }
+
folio_lock(folio);
+ /*
+ * A concurrent buffered write can extend isize, so we store isize in a
+ * local variable to let the zeroing offset match the EOF check below.
+ */
+ isize = i_size_read(inode);
+
if (unlikely(folio->mapping != inode->i_mapping ||
- folio_pos(folio) > i_size_read(inode) ||
+ folio_pos(folio) > isize ||
!folio_test_uptodate(folio))) {
folio_unlock(folio);
err = -EFAULT;
@@ -217,18 +235,11 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
/* wait for GCed page writeback via META_MAPPING */
f2fs_wait_on_block_writeback(inode, dn.data_blkaddr);
- /*
- * check to see if the page is mapped already (no holes)
- */
- if (folio_test_mappedtodisk(folio))
- goto out_sem;
-
/* page is wholly or partially inside EOF */
- if (((loff_t)(folio->index + 1) << PAGE_SHIFT) >
- i_size_read(inode)) {
+ if (((loff_t)(folio->index + 1) << PAGE_SHIFT) > isize) {
loff_t offset;
- offset = i_size_read(inode) & ~PAGE_MASK;
+ offset = isize & ~PAGE_MASK;
folio_zero_segment(folio, offset, folio_size(folio));
}
folio_mark_dirty(folio);
@@ -1075,15 +1086,12 @@ int f2fs_getattr(struct mnt_idmap *idmap, const struct path *path,
stat->attributes |= STATX_ATTR_IMMUTABLE;
if (flags & F2FS_NODUMP_FL)
stat->attributes |= STATX_ATTR_NODUMP;
- if (IS_VERITY(inode))
- stat->attributes |= STATX_ATTR_VERITY;
stat->attributes_mask |= (STATX_ATTR_COMPRESSED |
STATX_ATTR_APPEND |
STATX_ATTR_ENCRYPTED |
STATX_ATTR_IMMUTABLE |
- STATX_ATTR_NODUMP |
- STATX_ATTR_VERITY);
+ STATX_ATTR_NODUMP);
generic_fillattr(idmap, request_mask, inode, stat);
@@ -2110,9 +2118,11 @@ static long f2fs_fallocate(struct file *file, int mode,
/*
* Pinned file should not support partial truncation since the block
- * can be used by applications.
+ * can be used by applications. Atomic files should not either, since
+ * these modify the original inode directly and break atomicity.
*/
- if ((f2fs_compressed_file(inode) || f2fs_is_pinned_file(inode)) &&
+ if ((f2fs_compressed_file(inode) || f2fs_is_pinned_file(inode) ||
+ f2fs_is_atomic_file(inode)) &&
(mode & (FALLOC_FL_PUNCH_HOLE | FALLOC_FL_COLLAPSE_RANGE |
FALLOC_FL_ZERO_RANGE | FALLOC_FL_INSERT_RANGE))) {
ret = -EOPNOTSUPP;
@@ -2400,6 +2410,8 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
f2fs_down_write(&fi->i_gc_rwsem[READ]);
+ /* serialize with f2fs_vm_page_mkwrite(), see the comment there */
+ filemap_invalidate_lock(inode->i_mapping);
/*
* Should wait end_io to count F2FS_WB_CP_DATA correctly by
@@ -2448,13 +2460,14 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
fi->original_i_size = isize;
if (truncate) {
set_inode_flag(inode, FI_ATOMIC_REPLACE);
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
f2fs_i_size_write(inode, 0);
isize = 0;
}
f2fs_i_size_write(fi->cow_inode, isize);
out_unlock:
+ filemap_invalidate_unlock(inode->i_mapping);
f2fs_up_write(&fi->i_gc_rwsem[READ]);
f2fs_up_write(&fi->i_gc_rwsem[WRITE]);
if (ret)
@@ -3032,8 +3045,8 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
/* writeback all dirty pages in the range */
err = filemap_write_and_wait_range(inode->i_mapping,
- pg_start << PAGE_SHIFT,
- (pg_end << PAGE_SHIFT) - 1);
+ (loff_t)pg_start << PAGE_SHIFT,
+ ((loff_t)pg_end << PAGE_SHIFT) - 1);
if (err)
goto out;
@@ -3843,6 +3856,7 @@ static int f2fs_ioc_reserve_dev_alias(struct file *filp)
write_unlock(&et->lock);
clear_inode_flag(inode, FI_NO_EXTENT);
+ f2fs_drop_discard_cmd_range(sbi, ei.blk, ei.len);
f2fs_reserve_device_alias(sbi, ei.blk, ei.len);
i_size_write(inode, (loff_t)ei.len << sbi->log_blocksize);
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 63b712d3d599..f3a73b6c24c3 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -194,7 +194,7 @@ void f2fs_abort_atomic_write(struct inode *inode, bool clean)
return;
if (clean)
- truncate_inode_pages_final(inode->i_mapping);
+ truncate_inode_pages(inode->i_mapping, 0);
release_atomic_write_cnt(inode);
clear_inode_flag(inode, FI_ATOMIC_COMMITTED);
@@ -1871,6 +1871,60 @@ static unsigned int __wait_all_discard_cmd(struct f2fs_sb_info *sbi,
return discard_blks;
}
+void f2fs_drop_discard_cmd_range(struct f2fs_sb_info *sbi,
+ block_t start, block_t len)
+{
+ struct discard_cmd_control *dcc = SM_I(sbi)->dcc_info;
+ struct discard_cmd *prev_dc = NULL, *next_dc = NULL;
+ struct rb_node **insert_p = NULL, *insert_parent = NULL;
+ struct discard_cmd *dc, *wait_dc;
+ u64 cur = start;
+ u64 end = (u64)start + len;
+ int count;
+
+ if (!f2fs_realtime_discard_enable(sbi))
+ return;
+
+next:
+ count = 0;
+ wait_dc = NULL;
+
+ mutex_lock(&dcc->cmd_lock);
+ while (cur < end) {
+ dc = __lookup_discard_cmd_ret(&dcc->root, cur,
+ &prev_dc, &next_dc, &insert_p, &insert_parent);
+ if (!dc)
+ dc = next_dc;
+
+ if (!dc || (u64)dc->di.lstart >= end)
+ break;
+
+ if (dc->state == D_PREP) {
+ cur = (u64)dc->di.lstart + dc->di.len;
+ __remove_discard_cmd(sbi, dc);
+ if (++count >= MAX_DISCARD_DROP_COUNT)
+ break;
+ continue;
+ }
+
+ dc->ref++;
+ cur = (u64)dc->di.lstart + dc->di.len;
+ wait_dc = dc;
+ break;
+ }
+ mutex_unlock(&dcc->cmd_lock);
+
+ if (wait_dc) {
+ __wait_one_discard_bio(sbi, wait_dc);
+ goto next;
+ }
+
+ if (count >= MAX_DISCARD_DROP_COUNT) {
+ cond_resched();
+ goto next;
+ }
+}
+
/* This should be covered by global mutex, &sit_i->sentry_lock */
static void f2fs_wait_discard_bio(struct f2fs_sb_info *sbi, block_t blkaddr)
{
@@ -4214,6 +4268,7 @@ int f2fs_inplace_write_data(struct f2fs_io_info *fio)
else
err = f2fs_submit_page_bio(fio);
if (!err) {
+ fio->submitted = 1;
f2fs_update_device_state(fio->sbi, fio->ino,
fio->new_blkaddr, 1);
f2fs_update_iostat(fio->sbi, fio_inode(fio),
@@ -4814,7 +4869,7 @@ void f2fs_flush_sit_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc)
down_write(&sit_i->sentry_lock);
- if (!sit_i->dirty_sentries)
+ if (!sit_i->dirty_sentries && to_journal)
goto out;
/*
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 1314b6ccced9..b9adad85e18c 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -338,11 +338,11 @@ static const struct fs_parameter_spec f2fs_param_specs[] = {
fsparam_flag("usrquota", Opt_usrquota),
fsparam_flag("grpquota", Opt_grpquota),
fsparam_flag("prjquota", Opt_prjquota),
- fsparam_string("usrjquota", Opt_usrjquota),
+ fsparam_string_empty("usrjquota", Opt_usrjquota),
fsparam_flag("usrjquota", Opt_usrjquota),
- fsparam_string("grpjquota", Opt_grpjquota),
+ fsparam_string_empty("grpjquota", Opt_grpjquota),
fsparam_flag("grpjquota", Opt_grpjquota),
- fsparam_string("prjjquota", Opt_prjjquota),
+ fsparam_string_empty("prjjquota", Opt_prjjquota),
fsparam_flag("prjjquota", Opt_prjjquota),
fsparam_flag("nat_bits", Opt_nat_bits),
fsparam_enum("jqfmt", Opt_jqfmt, f2fs_param_jqfmt),
@@ -3305,15 +3305,42 @@ static qsize_t *f2fs_get_reserved_space(struct inode *inode)
return &F2FS_I(inode)->i_reserved_quota;
}
+/*
+ * Quota files only grow and there are at most MAXQUOTAS of them, so there is
+ * no point in keeping them inline. Writeback of quota files does not handle
+ * inline data and would drop what was written to them.
+ */
+static int f2fs_quota_convert_inline(struct inode *inode)
+{
+ int err;
+
+ if (!f2fs_has_inline_data(inode))
+ return 0;
+
+ inode_lock(inode);
+ err = f2fs_convert_inline_inode(inode);
+ inode_unlock(inode);
+ return err;
+}
+
static int f2fs_quota_on_mount(struct f2fs_sb_info *sbi, int type)
{
+ int err;
+
if (is_set_ckpt_flags(sbi, CP_QUOTA_NEED_FSCK_FLAG)) {
f2fs_err(sbi, "quota sysfile may be corrupted, skip loading it");
return 0;
}
- return dquot_quota_on_mount(sbi->sb, F2FS_OPTION(sbi).s_qf_names[type],
+ err = dquot_quota_on_mount(sbi->sb, F2FS_OPTION(sbi).s_qf_names[type],
F2FS_OPTION(sbi).s_jquota_fmt, type);
+ if (err)
+ return err;
+
+ err = f2fs_quota_convert_inline(sb_dqopt(sbi->sb)->files[type]);
+ if (err)
+ dquot_quota_off(sbi->sb, type);
+ return err;
}
int f2fs_enable_quota_files(struct f2fs_sb_info *sbi, bool rdonly)
@@ -3518,6 +3545,10 @@ static int f2fs_quota_on(struct super_block *sb, int type, int format_id,
inode = d_inode(path->dentry);
+ err = f2fs_quota_convert_inline(inode);
+ if (err)
+ goto out;
+
err = filemap_fdatawrite(inode->i_mapping);
if (err)
goto out;
@@ -4016,6 +4047,12 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
/* fix in-memory information all the time */
raw_super->segment_count = cpu_to_le32((main_end_blkaddr -
segment0_blkaddr) >> log_blocks_per_seg);
+ if (__F2FS_HAS_FEATURE(raw_super, F2FS_FEATURE_SB_CHKSUM)) {
+ __u32 crc = f2fs_crc32(raw_super,
+ offsetof(struct f2fs_super_block, crc));
+
+ raw_super->crc = cpu_to_le32(crc);
+ }
if (f2fs_readonly(sb) || f2fs_hw_is_readonly(sbi)) {
set_sbi_flag(sbi, SBI_NEED_SB_WRITE);
@@ -4276,7 +4313,7 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
return 1;
if (f2fs_sb_has_readonly(sbi))
- goto check_data;
+ continue;
for (j = i + 1; j < NR_CURSEG_NODE_TYPE; j++) {
if (le32_to_cpu(ckpt->cur_node_segno[i]) ==
@@ -4288,14 +4325,14 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
}
}
}
-check_data:
+
for (i = 0; i < NR_CURSEG_DATA_TYPE; i++) {
if (le32_to_cpu(ckpt->cur_data_segno[i]) >= main_segs ||
le16_to_cpu(ckpt->cur_data_blkoff[i]) >= blocks_per_seg)
return 1;
if (f2fs_sb_has_readonly(sbi))
- goto skip_cross;
+ continue;
for (j = i + 1; j < NR_CURSEG_DATA_TYPE; j++) {
if (le32_to_cpu(ckpt->cur_data_segno[i]) ==
@@ -4307,6 +4344,9 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
}
}
}
+ if (f2fs_sb_has_readonly(sbi))
+ goto skip_cross;
+
for (i = 0; i < NR_CURSEG_NODE_TYPE; i++) {
for (j = 0; j < NR_CURSEG_DATA_TYPE; j++) {
if (le32_to_cpu(ckpt->cur_node_segno[i]) ==
diff --git a/fs/fat/fat.h b/fs/fat/fat.h
index 61338413d9f3..fbd207c55859 100644
--- a/fs/fat/fat.h
+++ b/fs/fat/fat.h
@@ -392,7 +392,7 @@ extern void fat_ent_access_init(struct super_block *sb);
extern int fat_ent_read(struct inode *inode, struct fat_entry *fatent,
int entry);
extern int fat_ent_write(struct inode *inode, struct fat_entry *fatent,
- int new, int wait);
+ int new, int old, int wait);
extern int fat_alloc_clusters(struct inode *inode, int *cluster,
int nr_cluster);
extern int fat_free_clusters(struct inode *inode, int cluster);
diff --git a/fs/fat/fatent.c b/fs/fat/fatent.c
index f0801d99dd62..df23fc85f313 100644
--- a/fs/fat/fatent.c
+++ b/fs/fat/fatent.c
@@ -413,7 +413,7 @@ static int fat_mirror_bhs(struct super_block *sb, struct buffer_head **bhs,
}
int fat_ent_write(struct inode *inode, struct fat_entry *fatent,
- int new, int wait)
+ int new, int old, int wait)
{
struct super_block *sb = inode->i_sb;
const struct fatent_operations *ops = MSDOS_SB(sb)->fatent_ops;
@@ -422,10 +422,25 @@ int fat_ent_write(struct inode *inode, struct fat_entry *fatent,
ops->ent_put(fatent, new);
if (wait) {
err = fat_sync_bhs(fatent->bhs, fatent->nr_bhs);
- if (err)
+ if (err) {
+ /*
+ * bhs are not uptodate after I/O error. So we
+ * can't simply re-dirty to revert. And it
+ * would not have value to write again on I/O
+ * error.
+ */
return err;
+ }
}
- return fat_mirror_bhs(sb, fatent->bhs, fatent->nr_bhs);
+
+ err = fat_mirror_bhs(sb, fatent->bhs, fatent->nr_bhs);
+ if (err) {
+ /* Try to revert if got the error on mirror FAT */
+ ops->ent_put(fatent, old);
+ if (wait)
+ fat_sync_bhs(fatent->bhs, fatent->nr_bhs);
+ }
+ return err;
}
static inline int fat_ent_next(struct msdos_sb_info *sbi,
diff --git a/fs/fat/file.c b/fs/fat/file.c
index 1c835ca5f21a..6c475c53334c 100644
--- a/fs/fat/file.c
+++ b/fs/fat/file.c
@@ -363,7 +363,8 @@ static int fat_free(struct inode *inode, int skip)
__func__, MSDOS_I(inode)->i_pos);
ret = -EIO;
} else if (ret > 0) {
- err = fat_ent_write(inode, &fatent, FAT_ENT_EOF, wait);
+ err = fat_ent_write(inode, &fatent, FAT_ENT_EOF, ret,
+ wait);
if (err)
ret = err;
}
diff --git a/fs/fat/misc.c b/fs/fat/misc.c
index e79762cf1975..c44296756eae 100644
--- a/fs/fat/misc.c
+++ b/fs/fat/misc.c
@@ -133,11 +133,9 @@ int fat_chain_add(struct inode *inode, int new_dclus, int nr_cluster)
ret = fat_ent_read(inode, &fatent, last);
if (ret >= 0) {
int wait = inode_needs_sync(inode);
- int old = ret;
- ret = fat_ent_write(inode, &fatent, new_dclus, wait);
- if (ret < 0)
- fat_ent_write(inode, &fatent, old, wait);
+ ret = fat_ent_write(inode, &fatent, new_dclus, ret,
+ wait);
fatent_brelse(&fatent);
}
if (ret < 0)
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index ea3eb40bf828..e2ee11119518 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -468,12 +468,12 @@ static bool inode_do_switch_wbs(struct inode *inode,
if (inode_state_read(inode) & I_DIRTY_ALL) {
/*
* We need to keep b_dirty list sorted by
- * dirtied_time_when. However properly sorting the
+ * dirtied_when. However properly sorting the
* inode in the list gets too expensive when switching
* many inodes. So just attach inode at the end of the
- * dirty list and clobber the dirtied_time_when.
+ * dirty list and clobber the dirtied_when.
*/
- inode->dirtied_time_when = jiffies;
+ inode->dirtied_when = jiffies;
inode_io_list_move_locked(inode, new_wb,
&new_wb->b_dirty);
} else {
@@ -566,6 +566,7 @@ static void process_inode_switch_wbs(struct bdi_writeback *new_wb,
}
}
+ wb_io_lists_depopulated(old_wb);
spin_unlock(&new_wb->list_lock);
spin_unlock(&old_wb->list_lock);
@@ -789,6 +790,13 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
new_wb = &wb->bdi->wb; /* wb_get() is noop for bdi's wb */
nr = 0;
+ /*
+ * The scan moves walked inodes to the newest end of b_dirty_time,
+ * past ones it didn't reach. If one of those was dirtied after a
+ * sync's cutoff, the sync's queue_io() stops there and misses the
+ * moved ones. Exclude syncs as process_inode_switch_wbs() does.
+ */
+ down_read(&wb->bdi->wb_switch_rwsem);
spin_lock(&wb->list_lock);
/*
* In addition to the inodes that have completed writeback, also switch
@@ -803,6 +811,7 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb)
restart = isw_prepare_wbs_switch(new_wb, isw, &wb->b_dirty_time,
&nr);
spin_unlock(&wb->list_lock);
+ up_read(&wb->bdi->wb_switch_rwsem);
/* no attached inodes? bail out */
if (nr == 0) {
diff --git a/fs/fs_pin.c b/fs/fs_pin.c
index 47ef3c71ce90..1a508f2167e0 100644
--- a/fs/fs_pin.c
+++ b/fs/fs_pin.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/fs.h>
+#include <linux/rculist.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include "internal.h"
@@ -22,8 +23,8 @@ void pin_remove(struct fs_pin *pin)
void pin_insert(struct fs_pin *pin, struct vfsmount *m)
{
spin_lock(&pin_lock);
- hlist_add_head(&pin->s_list, &m->mnt_sb->s_pins);
- hlist_add_head(&pin->m_list, &real_mount(m)->mnt_pins);
+ hlist_add_head_rcu(&pin->s_list, &m->mnt_sb->s_pins);
+ hlist_add_head_rcu(&pin->m_list, &real_mount(m)->mnt_pins);
spin_unlock(&pin_lock);
}
@@ -73,7 +74,7 @@ void mnt_pin_kill(struct mount *m)
while (1) {
struct hlist_node *p;
rcu_read_lock();
- p = READ_ONCE(m->mnt_pins.first);
+ p = rcu_dereference(hlist_first_rcu(&m->mnt_pins));
if (!p) {
rcu_read_unlock();
break;
@@ -87,7 +88,7 @@ void group_pin_kill(struct hlist_head *p)
while (1) {
struct hlist_node *q;
rcu_read_lock();
- q = READ_ONCE(p->first);
+ q = rcu_dereference(hlist_first_rcu(p));
if (!q) {
rcu_read_unlock();
break;
diff --git a/fs/fuse/file.c b/fs/fuse/file.c
index 9a36d0329e22..bf79c6eb3062 100644
--- a/fs/fuse/file.c
+++ b/fs/fuse/file.c
@@ -864,18 +864,29 @@ static int fuse_do_readfolio(struct file *file, struct folio *folio,
attr_ver = fuse_get_attr_version(fm->fc);
- /* Don't overflow end offset */
- if (pos + (desc.length - 1) == LLONG_MAX)
- desc.length--;
+ /*
+ * Don't overflow end offset.
+ *
+ * Ask the server for len - 1 bytes. desc.length still holds the full
+ * length. When the reply comes back, it will be one byte shorter than
+ * desc.length and fuse_copy_folios() will zero that last byte.
+ *
+ * For this reason, desc.length must not be decremented too. The caller
+ * reports the full length to iomap_finish_folio_read(), which marks
+ * every block it covers uptodate. Shortening the descriptor would
+ * suppress zeroing and leave the last byte holding stale data.
+ */
+ if (pos + (len - 1) == LLONG_MAX)
+ len--;
- fuse_read_args_fill(&ia, file, pos, desc.length, FUSE_READ);
+ fuse_read_args_fill(&ia, file, pos, len, FUSE_READ);
res = fuse_simple_request(fm, &ia.ap.args);
if (res < 0)
return res;
/*
* Short read means EOF. If file size is larger, truncate it
*/
- if (res < desc.length)
+ if (res < len)
fuse_short_read(inode, attr_ver, res, &ia.ap);
return 0;
@@ -1068,11 +1079,20 @@ static void fuse_send_readpages(struct fuse_io_args *ia, struct file *file,
ap->args.page_zeroing = true;
ap->args.page_replace = true;
- /* Don't overflow end offset */
- if (pos + (count - 1) == LLONG_MAX) {
+ /*
+ * Don't overflow end offset.
+ *
+ * Ask the server for count - 1 bytes. The reply is then one byte
+ * shorter than what the descriptor lengths add up to, so
+ * fuse_copy_folios() zeroes the last byte when it walks the folios.
+ *
+ * ap->descs[] must not be decremented here. It is what
+ * fuse_readpages_end() reports back to iomap_finish_folio_read(), and
+ * iomap has already accounted the full descriptor length, so shortening
+ * it would leave ifs->read_bytes_pending nonzero and the folio locked.
+ */
+ if (pos + (count - 1) == LLONG_MAX)
count--;
- ap->descs[ap->num_folios - 1].length--;
- }
WARN_ON((loff_t) (pos + count) < 0);
fuse_read_args_fill(ia, file, pos, count, FUSE_READ);
@@ -2327,6 +2347,9 @@ static int fuse_launder_folio(struct folio *folio)
.wb_ctx = &data,
};
+ if (fuse_is_bad(wpc.inode))
+ return -EIO;
+
if (folio_clear_dirty_for_io(folio)) {
err = iomap_writeback_folio(&wpc, folio);
err = fuse_iomap_writeback_submit(&wpc, err);
@@ -3019,7 +3042,7 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in,
}
err = fuse_simple_request(fm, &args);
if (err == -ENOSYS) {
- if (fc->no_copy_file_range_64) {
+ if (args.opcode == FUSE_COPY_FILE_RANGE) {
fc->no_copy_file_range = 1;
err = -EOPNOTSUPP;
} else {
@@ -3030,7 +3053,7 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in,
if (err)
goto out;
- bytes_copied = fc->no_copy_file_range_64 ?
+ bytes_copied = args.opcode == FUSE_COPY_FILE_RANGE ?
outarg.size : outarg_64.bytes_copied;
if (bytes_copied > len) {
@@ -3038,9 +3061,11 @@ static ssize_t __fuse_copy_file_range(struct file *file_in, loff_t pos_in,
goto out;
}
- truncate_inode_pages_range(inode_out->i_mapping,
- ALIGN_DOWN(pos_out, PAGE_SIZE),
- ALIGN(pos_out + bytes_copied, PAGE_SIZE) - 1);
+ if (bytes_copied)
+ truncate_inode_pages_range(inode_out->i_mapping,
+ ALIGN_DOWN(pos_out, PAGE_SIZE),
+ ALIGN(pos_out + bytes_copied,
+ PAGE_SIZE) - 1);
file_update_time(file_out);
fuse_write_update_attr(inode_out, pos_out + bytes_copied, bytes_copied);
diff --git a/fs/fuse/notify.c b/fs/fuse/notify.c
index 1ba763705d91..fff059f8ad85 100644
--- a/fs/fuse/notify.c
+++ b/fs/fuse/notify.c
@@ -155,7 +155,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
nodeid = outarg.nodeid;
pos = outarg.offset;
- num = min(outarg.size, MAX_LFS_FILESIZE - pos);
+ num = umin(outarg.size, MAX_LFS_FILESIZE - pos);
down_read(&fc->killsb);
diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c
index f15e516ebcb5..0150c5cfc311 100644
--- a/fs/fuse/virtio_fs.c
+++ b/fs/fuse/virtio_fs.c
@@ -455,26 +455,23 @@ static int virtio_fs_add_instance(struct virtio_device *vdev,
return ret;
}
-/* Return the virtio_fs with a given tag, or NULL */
+/* Return the virtio_fs with a given tag, or NULL.
+ * Callers hold virtio_fs_mutex, which also keeps the virtqueues alive.
+ */
static struct virtio_fs *virtio_fs_find_instance(const char *tag)
{
struct virtio_fs *fs;
- mutex_lock(&virtio_fs_mutex);
+ lockdep_assert_held(&virtio_fs_mutex);
list_for_each_entry(fs, &virtio_fs_instances, list) {
if (strcmp(fs->tag, tag) == 0) {
kobject_get(&fs->kobj);
- goto found;
+ return fs;
}
}
- fs = NULL; /* not found */
-
-found:
- mutex_unlock(&virtio_fs_mutex);
-
- return fs;
+ return NULL; /* not found */
}
static void virtio_fs_free_devs(struct virtio_fs *fs)
@@ -730,6 +727,10 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
unsigned int num_out;
unsigned int i;
+ /* fuse_request_end() returns this error; there are no args to copy. */
+ if (req->out.h.error)
+ goto out;
+
remaining = req->out.h.len - sizeof(req->out.h);
num_in = args->in_numargs - args->in_pages;
num_out = args->out_numargs - args->out_pages;
@@ -755,6 +756,7 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
if (args->out_argvar)
args->out_args[args->out_numargs - 1].size = remaining;
+out:
kfree(req->argbuf);
req->argbuf = NULL;
}
@@ -762,7 +764,9 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req)
/* Verify that the server properly follows the FUSE protocol */
static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len)
{
+ struct fuse_args *args = req->args;
struct fuse_out_header *oh = &req->out.h;
+ unsigned int expected;
if (len < sizeof(*oh)) {
pr_warn("virtio-fs: response too short (%u)\n", len);
@@ -777,6 +781,29 @@ static bool virtio_fs_verify_response(struct fuse_req *req, unsigned int len)
oh->unique, req->in.h.unique);
return false;
}
+ if (oh->error <= -ERESTARTSYS || oh->error > 0) {
+ pr_warn("virtio-fs: invalid error value (%d)\n", oh->error);
+ return false;
+ }
+
+ if (oh->error) {
+ if (len != sizeof(*oh)) {
+ pr_warn("virtio-fs: error response too long (%u)\n", len);
+ return false;
+ }
+ return true;
+ }
+
+ expected = sizeof(*oh) +
+ fuse_len_args(args->out_numargs, args->out_args);
+ if (len > expected ||
+ (len < expected &&
+ (!args->out_argvar ||
+ expected - len > args->out_args[args->out_numargs - 1].size))) {
+ pr_warn("virtio-fs: invalid response length (%u, expected %u)\n",
+ len, expected);
+ return false;
+ }
return true;
}
@@ -1698,13 +1725,17 @@ static int virtio_fs_get_tree(struct fs_context *fsc)
* in chan->iq->priv. Once fuse_conn is going away, it calls ->put()
* to drop the reference to this object.
*/
+ mutex_lock(&virtio_fs_mutex);
fs = virtio_fs_find_instance(fsc->source);
+ if (fs)
+ virtqueue_size = virtqueue_get_vring_size(fs->vqs[VQ_REQUEST].vq);
+ mutex_unlock(&virtio_fs_mutex);
+
if (!fs) {
pr_info("virtio-fs: tag <%s> not found\n", fsc->source);
return -EINVAL;
}
- virtqueue_size = virtqueue_get_vring_size(fs->vqs[VQ_REQUEST].vq);
if (WARN_ON(virtqueue_size <= FUSE_HEADER_OVERHEAD))
goto out_err;
diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c
index 28f32424ee64..2e53ba3709eb 100644
--- a/fs/gfs2/glops.c
+++ b/fs/gfs2/glops.c
@@ -393,6 +393,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
umode_t mode = be32_to_cpu(str->di_mode);
struct inode *inode = &ip->i_inode;
bool is_new = inode_state_read_once(inode) & I_NEW;
+ u64 size;
if (unlikely(ip->i_no_addr != be64_to_cpu(str->di_num.no_addr))) {
gfs2_consist_inode(ip);
@@ -418,7 +419,12 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
i_uid_write(inode, be32_to_cpu(str->di_uid));
i_gid_write(inode, be32_to_cpu(str->di_gid));
set_nlink(inode, be32_to_cpu(str->di_nlink));
- i_size_write(inode, be64_to_cpu(str->di_size));
+ size = be64_to_cpu(str->di_size);
+ if (unlikely(size > inode->i_sb->s_maxbytes)) {
+ gfs2_consist_inode(ip);
+ return -EIO;
+ }
+ i_size_write(inode, size);
gfs2_set_inode_blocks(inode, be64_to_cpu(str->di_blocks));
atime.tv_sec = be64_to_cpu(str->di_atime);
atime.tv_nsec = be32_to_cpu(str->di_atime_nsec);
@@ -462,7 +468,7 @@ static int gfs2_dinode_in(struct gfs2_inode *ip, const void *buf)
return -EIO;
}
- if (gfs2_is_stuffed(ip) && inode->i_size > gfs2_max_stuffed_size(ip)) {
+ if (gfs2_is_stuffed(ip) && size > gfs2_max_stuffed_size(ip)) {
gfs2_consist_inode(ip);
return -EIO;
}
diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c
index 6dabe73ad790..b8c9024c9fed 100644
--- a/fs/gfs2/lops.c
+++ b/fs/gfs2/lops.c
@@ -689,7 +689,9 @@ static void gfs2_before_commit(struct gfs2_sbd *sdp, unsigned int limit,
page = mempool_alloc(gfs2_page_pool, GFP_NOIO);
p = page_address(page);
- memcpy_from_page(p, page, bh_offset(bd2->bd_bh), bd2->bd_bh->b_size);
+ memcpy_from_folio(p, bd2->bd_bh->b_folio,
+ bh_offset(bd2->bd_bh),
+ bd2->bd_bh->b_size);
*(__be32 *)p = 0;
clear_buffer_escaped(bd2->bd_bh);
unlock_buffer(bd2->bd_bh);
diff --git a/fs/inode.c b/fs/inode.c
index a9d37be390a1..bc93c1757a38 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -593,12 +593,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
static void inode_unpin_lru_isolating(struct inode *inode)
{
- spin_lock(&inode->i_lock);
+ lockdep_assert_held(&inode->i_lock);
WARN_ON(!(inode_state_read(inode) & I_LRU_ISOLATING));
inode_state_clear(inode, I_LRU_ISOLATING);
/* Called with inode->i_lock which ensures memory ordering. */
inode_wake_up_bit(inode, __I_LRU_ISOLATING);
- spin_unlock(&inode->i_lock);
}
static void inode_wait_for_lru_isolating(struct inode *inode)
@@ -942,6 +941,12 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
if (!spin_trylock(&inode->i_lock))
return LRU_SKIP;
+ /* Another walker is dropping its page cache, leave it to that one */
+ if (inode_state_read(inode) & I_LRU_ISOLATING) {
+ spin_unlock(&inode->i_lock);
+ return LRU_SKIP;
+ }
+
/*
* Inodes can get referenced, redirtied, or repopulated while
* they're already on the LRU, and this can make them
@@ -981,7 +986,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
else
__count_vm_events(PGINODESTEAL, reap);
mm_account_reclaimed_pages(reap);
+
+ spin_lock(&inode->i_lock);
+ /* The walk restarts here: don't retry leftovers right away */
+ if (!mapping_empty(&inode->i_data))
+ inode_state_set(inode, I_REFERENCED);
inode_unpin_lru_isolating(inode);
+ spin_unlock(&inode->i_lock);
return LRU_RETRY;
}
diff --git a/fs/iomap/bio.c b/fs/iomap/bio.c
index 48100c614431..ee85b009c5c1 100644
--- a/fs/iomap/bio.c
+++ b/fs/iomap/bio.c
@@ -151,7 +151,7 @@ int iomap_bio_read_folio_range(const struct iomap_iter *iter,
if (!bio ||
bio_end_sector(bio) != iomap_sector(&iter->iomap, iter->pos) ||
- bio->bi_iter.bi_size > iomap_max_bio_size(&iter->iomap) - plen ||
+ !iomap_bio_has_room(&iter->iomap, bio, plen) ||
!bio_add_folio(bio, folio, plen, offset_in_folio(folio, iter->pos)))
iomap_read_alloc_bio(iter, ctx, plen);
return 0;
diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index 0a5ebfda90f1..44f9a0e6f8fc 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -920,6 +920,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter,
if (iomap_block_needs_zeroing(iter, block_start)) {
if (WARN_ON_ONCE(iter->flags & IOMAP_UNSHARE))
return -EIO;
+ /* uptodate once copied, see __iomap_write_end() */
folio_zero_segments(folio, poff, from, to, poff + plen);
} else {
const struct iomap *iomap = iomap_iter_srcmap(iter);
@@ -943,8 +944,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter,
if (iomap->flags & IOMAP_F_ZERO_TAIL)
folio_zero_segment(folio, to, poff + plen);
+ iomap_set_range_uptodate(folio, poff, plen);
}
- iomap_set_range_uptodate(folio, poff, plen);
} while ((block_start += plen) < block_end);
return 0;
diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c
index 8b4039d16ce8..7f1e85f95a46 100644
--- a/fs/iomap/direct-io.c
+++ b/fs/iomap/direct-io.c
@@ -296,7 +296,7 @@ u32 iomap_finish_ioend_direct(struct iomap_ioend *ioend)
return vec_count;
}
-static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
+static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
loff_t pos, unsigned len)
{
struct inode *inode = file_inode(dio->iocb->ki_filp);
@@ -305,14 +305,14 @@ static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
int nr_vecs = max(1, i_blocksize(inode) / folio_size(zero_folio));
if (!len)
- return 0;
+ return;
/*
* This limit shall never be reached as most filesystems have a
* maximum blocksize of 64k.
*/
if (WARN_ON_ONCE(nr_vecs > BIO_MAX_VECS))
- return -EINVAL;
+ return;
bio = iomap_dio_alloc_bio(iter, dio, nr_vecs,
REQ_OP_WRITE | REQ_SYNC | REQ_IDLE);
@@ -328,8 +328,6 @@ static int iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio,
len -= io_len;
}
iomap_dio_submit_bio(iter, dio, bio, pos);
-
- return 0;
}
static ssize_t iomap_dio_bio_iter_one(struct iomap_iter *iter,
@@ -541,10 +539,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio)
if (need_zeroout) {
/* zero out from the start of the block to the write offset */
pad = pos & (fs_block_size - 1);
-
- ret = iomap_dio_zero(iter, dio, pos - pad, pad);
- if (ret)
- goto out;
+ iomap_dio_zero(iter, dio, pos - pad, pad);
}
do {
@@ -582,8 +577,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio)
/* zero out from the end of the write to the end of the block */
pad = pos & (fs_block_size - 1);
if (pad)
- ret = iomap_dio_zero(iter, dio, pos,
- fs_block_size - pad);
+ iomap_dio_zero(iter, dio, pos, fs_block_size - pad);
}
out:
/* Undo iter limitation to current extent */
diff --git a/fs/iomap/fiemap.c b/fs/iomap/fiemap.c
index d11dadff8286..54b824b7edb5 100644
--- a/fs/iomap/fiemap.c
+++ b/fs/iomap/fiemap.c
@@ -76,15 +76,15 @@ int iomap_fiemap(struct inode *inode, struct fiemap_extent_info *fi,
while ((ret = iomap_iter(&iter, ops)) > 0)
iter.status = iomap_fiemap_iter(&iter, fi, &prev);
+ /* inode with no (attribute) mapping will give ENOENT */
+ if (ret < 0 && ret != -ENOENT)
+ return ret;
+
if (prev.type != IOMAP_HOLE) {
ret = iomap_to_fiemap(fi, &prev, FIEMAP_EXTENT_LAST);
if (ret < 0)
return ret;
}
-
- /* inode with no (attribute) mapping will give ENOENT */
- if (ret < 0 && ret != -ENOENT)
- return ret;
return 0;
}
EXPORT_SYMBOL_GPL(iomap_fiemap);
diff --git a/fs/iomap/internal.h b/fs/iomap/internal.h
index 74e898b196dc..e69b3ac02cc2 100644
--- a/fs/iomap/internal.h
+++ b/fs/iomap/internal.h
@@ -17,6 +17,15 @@ static inline size_t iomap_max_bio_size(const struct iomap *iomap)
return BIO_MAX_SIZE;
}
+/* Can @len more bytes go into @bio? */
+static inline bool iomap_bio_has_room(const struct iomap *iomap,
+ const struct bio *bio, size_t len)
+{
+ size_t max_size = iomap_max_bio_size(iomap);
+
+ return len <= max_size && bio->bi_iter.bi_size <= max_size - len;
+}
+
u32 iomap_finish_ioend_buffered_read(struct iomap_ioend *ioend);
u32 iomap_finish_ioend_direct(struct iomap_ioend *ioend);
diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index 32ae292a84cb..c2891d640e82 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -176,8 +176,7 @@ static bool iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t pos,
{
struct iomap_ioend *ioend = wpc->wb_ctx;
- if (ioend->io_bio.bi_iter.bi_size >
- iomap_max_bio_size(&wpc->iomap) - map_len)
+ if (!iomap_bio_has_room(&wpc->iomap, &ioend->io_bio, map_len))
return false;
if (ioend_flags & IOMAP_IOEND_BOUNDARY)
return false;
diff --git a/fs/isofs/inode.c b/fs/isofs/inode.c
index 337836a0a170..b766b5c9c593 100644
--- a/fs/isofs/inode.c
+++ b/fs/isofs/inode.c
@@ -821,6 +821,8 @@ static int isofs_fill_super(struct super_block *s, struct fs_context *fc)
if (!sb_set_blocksize(s, orig_zonesize))
goto out_freesbi;
+ sbi->s_session_start = (sector_t)vol_desc_start <<
+ (ISOFS_BLOCK_BITS - s->s_blocksize_bits);
sbi->s_nls_iocharset = NULL;
#ifdef CONFIG_JOLIET
diff --git a/fs/isofs/isofs.h b/fs/isofs/isofs.h
index dacb9cdae4fd..79ca0256843a 100644
--- a/fs/isofs/isofs.h
+++ b/fs/isofs/isofs.h
@@ -35,6 +35,8 @@ struct isofs_sb_info {
unsigned long s_firstdatazone;
unsigned long s_log_zone_size;
unsigned long s_max_size;
+ /* Session start in filesystem block units. */
+ sector_t s_session_start;
int s_rock_offset; /* offset of SUSP fields within SU area */
s32 s_sbsector;
diff --git a/fs/isofs/rock.c b/fs/isofs/rock.c
index 2628f31bd3a5..84e0d764c210 100644
--- a/fs/isofs/rock.c
+++ b/fs/isofs/rock.c
@@ -9,6 +9,7 @@
#include <linux/slab.h>
#include <linux/pagemap.h>
+#include <linux/blkdev.h>
#include "isofs.h"
#include "rock.h"
@@ -84,6 +85,10 @@ static void init_rock_state(struct rock_state *rs, struct inode *inode)
*/
static int rock_continue(struct rock_state *rs)
{
+ struct super_block *sb = rs->inode->i_sb;
+ struct isofs_sb_info *sbi = ISOFS_SB(sb);
+ sector_t session_end = sbi->s_session_start +
+ ((sector_t)sbi->s_nzones << (ISOFS_BLOCK_BITS - sb->s_blocksize_bits));
int ret = 1;
int blocksize = 1 << rs->inode->i_blkbits;
const int min_de_size = offsetof(struct rock_ridge, u);
@@ -101,11 +106,13 @@ static int rock_continue(struct rock_state *rs)
goto out;
}
- if ((unsigned)rs->cont_extent >= ISOFS_SB(rs->inode->i_sb)->s_nzones) {
+ if (rs->cont_extent &&
+ (rs->cont_extent < sbi->s_session_start ||
+ rs->cont_extent >= session_end)) {
printk(KERN_NOTICE "rock: corrupted directory entry. "
"extent=%u out of volume (nzones=%lu)\n",
(unsigned)rs->cont_extent,
- ISOFS_SB(rs->inode->i_sb)->s_nzones);
+ sbi->s_nzones);
ret = -EIO;
goto out;
}
diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
index 00f5a98f3d4f..55b4ce010228 100644
--- a/fs/jbd2/journal.c
+++ b/fs/jbd2/journal.c
@@ -843,7 +843,6 @@ int jbd2_fc_get_buf(journal_t *journal, struct buffer_head **bh_out)
fc_off = journal->j_fc_off;
blocknr = journal->j_fc_first + fc_off;
- journal->j_fc_off++;
ret = jbd2_journal_bmap(journal, blocknr, &pblock);
if (ret)
return ret;
@@ -853,6 +852,7 @@ int jbd2_fc_get_buf(journal_t *journal, struct buffer_head **bh_out)
return -ENOMEM;
journal->j_fc_wbuf[fc_off] = bh;
+ journal->j_fc_off++;
*bh_out = bh;
diff --git a/fs/namespace.c b/fs/namespace.c
index c30f063742f4..cf627543572b 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -6058,7 +6058,8 @@ static int prepare_kstatmount(struct kstatmount *ks, struct mnt_id_req *kreq,
ks->bufsize = bufsize;
if (ks->mask & STATMOUNT_STRING_REQ) {
- if (bufsize == sizeof(ks->sm))
+ /* Even the reserved empty string needs room past the fixed part. */
+ if (bufsize <= sizeof(ks->sm))
return -EOVERFLOW;
ks->seq.buf = kvmalloc(seq_size, GFP_KERNEL_ACCOUNT);
@@ -6252,6 +6253,8 @@ static ssize_t do_listmount(struct klistmount *kls, bool reverse)
} else {
if (reverse)
first = mnt_find_id_at_reverse(ns, last_mnt_id - 1);
+ else if (last_mnt_id == U64_MAX)
+ first = NULL; /* nothing comes after the largest id */
else
first = mnt_find_id_at(ns, last_mnt_id + 1);
}
@@ -6717,7 +6720,7 @@ static int mntns_install(struct nsset *nsset, struct ns_common *ns)
if (is_anon_ns(mnt_ns))
return -EINVAL;
- if (fs->users != 1)
+ if (current->fs->users != 1)
return -EINVAL;
get_mnt_ns(mnt_ns);
diff --git a/fs/nilfs2/dir.c b/fs/nilfs2/dir.c
index 8b53802b6ebd..7686ca9228fc 100644
--- a/fs/nilfs2/dir.c
+++ b/fs/nilfs2/dir.c
@@ -231,11 +231,27 @@ static struct nilfs_dir_entry *nilfs_next_entry(struct nilfs_dir_entry *p)
nilfs_rec_len_from_disk(p->rec_len));
}
+static inline unsigned int
+nilfs_validate_entry(char *base, unsigned int offset, unsigned int mask)
+{
+ struct nilfs_dir_entry *de = (struct nilfs_dir_entry *)(base + offset);
+ struct nilfs_dir_entry *p =
+ (struct nilfs_dir_entry *)(base + (offset & mask));
+
+ while ((char *)p < (char *)de) {
+ if (p->rec_len == 0)
+ break;
+ p = nilfs_next_entry(p);
+ }
+ return (char *)p - base;
+}
+
static int nilfs_readdir(struct file *file, struct dir_context *ctx)
{
loff_t pos = ctx->pos;
struct inode *inode = file_inode(file);
struct super_block *sb = inode->i_sb;
+ unsigned int chunk_size = nilfs_chunk_size(inode);
unsigned int offset = pos & ~PAGE_MASK;
unsigned long n = pos >> PAGE_SHIFT;
unsigned long npages = dir_pages(inode);
@@ -254,6 +270,24 @@ static int nilfs_readdir(struct file *file, struct dir_context *ctx)
ctx->pos += PAGE_SIZE - offset;
return -EIO;
}
+ /*
+ * ctx->pos comes from userspace via llseek and may point
+ * into the middle of an entry -- or at unvalidated bytes
+ * of a crafted image. offset is only nonzero here on the
+ * very first iteration (subsequent pages always start at
+ * offset 0, which -- like any chunk boundary -- is always
+ * a legitimate entry start, since no rec_len chain crosses
+ * a chunk boundary). Snap a non-chunk-aligned offset onto
+ * a real entry boundary by walking the chain from the
+ * enclosing chunk's start, as ext2 does; otherwise
+ * dir_emit() copies name_len bytes from a fake entry,
+ * reading past the end of the folio.
+ */
+ if (offset & (chunk_size - 1)) {
+ offset = nilfs_validate_entry(kaddr, offset,
+ ~(chunk_size - 1));
+ ctx->pos = ((loff_t)n << PAGE_SHIFT) + offset;
+ }
de = (struct nilfs_dir_entry *)(kaddr + offset);
limit = kaddr + nilfs_last_byte(inode, n) -
NILFS_DIR_REC_LEN(1);
diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c
index 34e6096069ad..33490e8063eb 100644
--- a/fs/nilfs2/inode.c
+++ b/fs/nilfs2/inode.c
@@ -163,7 +163,7 @@ static int nilfs_writepages(struct address_space *mapping,
int err = 0;
if (sb_rdonly(inode->i_sb)) {
- nilfs_clear_dirty_pages(mapping);
+ nilfs_clear_dirty_pages(mapping, false);
return -EROFS;
}
@@ -895,13 +895,7 @@ void nilfs_evict_inode(struct inode *inode)
nilfs_clear_inode(inode);
- if (IS_SYNC(inode))
- nilfs_set_transaction_flag(NILFS_TI_SYNC);
nilfs_transaction_commit(sb);
- /*
- * May construct a logical segment and may fail in sync mode.
- * But delete_inode has no return value.
- */
}
int nilfs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index 01a04080ef70..b2b67d6a1f09 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
@@ -965,10 +965,10 @@ static int nilfs_ioctl_clean_segments(struct inode *inode, struct file *filp,
* Return: 0 on success, or one of the following negative error codes on
* failure:
* * %-EFAULT - Failure during execution of requested operation.
+ * * %-EINTR - Interrupted.
* * %-EIO - I/O error.
* * %-ENOMEM - Insufficient memory available.
* * %-ENOSPC - No space left on device (only in a panic state).
- * * %-ERESTARTSYS - Interrupted.
* * %-EROFS - Read only filesystem.
*/
static int nilfs_ioctl_sync(struct inode *inode, struct file *filp,
diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c
index 2a435349fd21..b50c88b65183 100644
--- a/fs/nilfs2/mdt.c
+++ b/fs/nilfs2/mdt.c
@@ -405,7 +405,7 @@ static int nilfs_mdt_write_folio(struct folio *folio,
* have dirty folios that try to be flushed in background.
* So, here we simply discard this dirty folio.
*/
- nilfs_clear_folio_dirty(folio);
+ nilfs_clear_folio_dirty(folio, false);
folio_unlock(folio);
return -EROFS;
}
@@ -648,10 +648,10 @@ void nilfs_mdt_restore_from_shadow_map(struct inode *inode)
if (mi->mi_palloc_cache)
nilfs_palloc_clear_cache(inode);
- nilfs_clear_dirty_pages(inode->i_mapping);
+ nilfs_clear_dirty_pages(inode->i_mapping, true);
nilfs_copy_back_pages(inode->i_mapping, shadow->inode->i_mapping);
- nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping);
+ nilfs_clear_dirty_pages(ii->i_assoc_inode->i_mapping, true);
nilfs_copy_back_pages(ii->i_assoc_inode->i_mapping,
NILFS_I(shadow->inode)->i_assoc_inode->i_mapping);
diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c
index cf4f1c6798f5..857926da9f21 100644
--- a/fs/nilfs2/page.c
+++ b/fs/nilfs2/page.c
@@ -369,8 +369,9 @@ void nilfs_copy_back_pages(struct address_space *dmap,
/**
* nilfs_clear_dirty_pages - discard dirty pages in address space
* @mapping: address space with dirty pages for discarding
+ * @force: whether to clear the dirty state regardless of busy buffer heads
*/
-void nilfs_clear_dirty_pages(struct address_space *mapping)
+void nilfs_clear_dirty_pages(struct address_space *mapping, bool force)
{
struct folio_batch fbatch;
unsigned int i;
@@ -391,7 +392,7 @@ void nilfs_clear_dirty_pages(struct address_space *mapping)
* was acquired. Skip processing in that case.
*/
if (likely(folio->mapping == mapping))
- nilfs_clear_folio_dirty(folio);
+ nilfs_clear_folio_dirty(folio, force);
folio_unlock(folio);
}
@@ -403,13 +404,16 @@ void nilfs_clear_dirty_pages(struct address_space *mapping)
/**
* nilfs_clear_folio_dirty - discard dirty folio
* @folio: dirty folio that will be discarded
+ * @force: whether to clear the states regardless of busy buffer heads
*
* nilfs_clear_folio_dirty() clears working states including dirty state for
- * the folio and its buffers. If the folio has buffers, clear only if it is
- * confirmed that none of the buffer heads are busy (none have valid
- * references and none are locked).
+ * the folio and its buffers. If the folio has buffers and force is false,
+ * clear only if it is confirmed that none of the buffer heads are busy (none
+ * have valid references and none are locked). If force is true, the states
+ * are cleared unconditionally, the caller should guarantee that the folio is
+ * not being modified concurrently.
*/
-void nilfs_clear_folio_dirty(struct folio *folio)
+void nilfs_clear_folio_dirty(struct folio *folio, bool force)
{
struct buffer_head *bh, *head;
@@ -422,24 +426,28 @@ void nilfs_clear_folio_dirty(struct folio *folio)
BIT(BH_Async_Write) | BIT(BH_NILFS_Volatile) |
BIT(BH_NILFS_Checked) | BIT(BH_NILFS_Redirected) |
BIT(BH_Delay));
- bool busy, invalidated = false;
+
+ if (!force) {
+ bool busy, invalidated = false;
recheck_buffers:
- busy = false;
- bh = head;
- do {
- if (atomic_read(&bh->b_count) | buffer_locked(bh)) {
- busy = true;
- break;
+ busy = false;
+ bh = head;
+ do {
+ if (atomic_read(&bh->b_count) |
+ buffer_locked(bh)) {
+ busy = true;
+ break;
+ }
+ } while (bh = bh->b_this_page, bh != head);
+
+ if (busy) {
+ if (invalidated)
+ return;
+ invalidate_bh_lrus();
+ invalidated = true;
+ goto recheck_buffers;
}
- } while (bh = bh->b_this_page, bh != head);
-
- if (busy) {
- if (invalidated)
- return;
- invalidate_bh_lrus();
- invalidated = true;
- goto recheck_buffers;
}
bh = head;
diff --git a/fs/nilfs2/page.h b/fs/nilfs2/page.h
index 136cd1c143c9..c3ba3468af5c 100644
--- a/fs/nilfs2/page.h
+++ b/fs/nilfs2/page.h
@@ -41,8 +41,8 @@ void nilfs_folio_bug(struct folio *);
int nilfs_copy_dirty_pages(struct address_space *, struct address_space *);
void nilfs_copy_back_pages(struct address_space *, struct address_space *);
-void nilfs_clear_folio_dirty(struct folio *folio);
-void nilfs_clear_dirty_pages(struct address_space *mapping);
+void nilfs_clear_folio_dirty(struct folio *folio, bool force);
+void nilfs_clear_dirty_pages(struct address_space *mapping, bool force);
unsigned int nilfs_page_count_clean_buffers(struct folio *folio,
unsigned int from, unsigned int to);
unsigned long nilfs_find_uncommitted_extent(struct inode *inode,
diff --git a/fs/nilfs2/recovery.c b/fs/nilfs2/recovery.c
index 45fb37215669..a7086a93480b 100644
--- a/fs/nilfs2/recovery.c
+++ b/fs/nilfs2/recovery.c
@@ -785,11 +785,11 @@ static void nilfs_abort_roll_forward(struct the_nilfs *nilfs)
*
* Return: 0 on success, or one of the following negative error codes on
* failure:
+ * * %-EINTR - Interrupted.
* * %-EINVAL - Inconsistent filesystem state.
* * %-EIO - I/O error.
* * %-ENOMEM - Insufficient memory available.
* * %-ENOSPC - No space left on device (only in a panic state).
- * * %-ERESTARTSYS - Interrupted.
*/
int nilfs_salvage_orphan_logs(struct the_nilfs *nilfs,
struct super_block *sb,
diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
index 829573cb6131..af0c3560444f 100644
--- a/fs/nilfs2/segment.c
+++ b/fs/nilfs2/segment.c
@@ -1829,6 +1829,7 @@ static void nilfs_abort_logs(struct list_head *logs, int err)
list_for_each_entry(segbuf, logs, sb_list) {
list_for_each_entry(bh, &segbuf->sb_segsum_buffers,
b_assoc_buffers) {
+ clear_buffer_dirty(bh);
clear_buffer_uptodate(bh);
if (bh->b_folio != bd_folio) {
if (bd_folio)
@@ -1840,6 +1841,7 @@ static void nilfs_abort_logs(struct list_head *logs, int err)
list_for_each_entry(bh, &segbuf->sb_payload_buffers,
b_assoc_buffers) {
if (bh == segbuf->sb_super_root) {
+ clear_buffer_dirty(bh);
clear_buffer_uptodate(bh);
if (bh->b_folio != bd_folio) {
folio_end_writeback(bd_folio);
@@ -2257,7 +2259,7 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci)
wake_up(&sci->sc_wait_daemon);
for (;;) {
- set_current_state(TASK_INTERRUPTIBLE);
+ set_current_state(TASK_KILLABLE);
/*
* Synchronize only while the log writer thread is alive.
@@ -2271,11 +2273,11 @@ static int nilfs_segctor_sync(struct nilfs_sc_info *sci)
err = wait_req.err;
break;
}
- if (!signal_pending(current)) {
+ if (!fatal_signal_pending(current)) {
schedule();
continue;
}
- err = -ERESTARTSYS;
+ err = -EINTR;
break;
}
finish_wait(&sci->sc_wait_request, &wait_req.wq);
@@ -2309,10 +2311,10 @@ static void nilfs_segctor_wakeup(struct nilfs_sc_info *sci, int err, bool force)
*
* Return: 0 on success, or one of the following negative error codes on
* failure:
+ * * %-EINTR - Interrupted.
* * %-EIO - I/O error (including metadata corruption).
* * %-ENOMEM - Insufficient memory available.
* * %-ENOSPC - No space left on device (only in a panic state).
- * * %-ERESTARTSYS - Interrupted.
* * %-EROFS - Read only filesystem.
*/
int nilfs_construct_segment(struct super_block *sb)
@@ -2339,10 +2341,10 @@ int nilfs_construct_segment(struct super_block *sb)
*
* Return: 0 on success, or one of the following negative error codes on
* failure:
+ * * %-EINTR - Interrupted.
* * %-EIO - I/O error (including metadata corruption).
* * %-ENOMEM - Insufficient memory available.
* * %-ENOSPC - No space left on device (only in a panic state).
- * * %-ERESTARTSYS - Interrupted.
* * %-EROFS - Read only filesystem.
*/
int nilfs_construct_dsync_segment(struct super_block *sb, struct inode *inode,
diff --git a/fs/nilfs2/sysfs.c b/fs/nilfs2/sysfs.c
index bc52afbfc5c7..5bb74a45ad1e 100644
--- a/fs/nilfs2/sysfs.c
+++ b/fs/nilfs2/sysfs.c
@@ -195,8 +195,10 @@ int nilfs_sysfs_create_snapshot_group(struct nilfs_root *root)
"%llu", root->cno);
}
- if (err)
+ if (err) {
kobject_put(&root->snapshot_kobj);
+ wait_for_completion(&root->snapshot_kobj_unregister);
+ }
return err;
}
@@ -204,6 +206,7 @@ int nilfs_sysfs_create_snapshot_group(struct nilfs_root *root)
void nilfs_sysfs_delete_snapshot_group(struct nilfs_root *root)
{
kobject_put(&root->snapshot_kobj);
+ wait_for_completion(&root->snapshot_kobj_unregister);
}
/************************************************************************
diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c
index f3805e7aabeb..38c6d4afefba 100644
--- a/fs/nilfs2/the_nilfs.c
+++ b/fs/nilfs2/the_nilfs.c
@@ -887,6 +887,26 @@ nilfs_find_or_create_root(struct the_nilfs *nilfs, __u64 cno)
if (!new)
return NULL;
+ new->cno = cno;
+ new->ifile = NULL;
+ new->nilfs = nilfs;
+ refcount_set(&new->count, 1);
+ atomic64_set(&new->inodes_count, 0);
+ atomic64_set(&new->blocks_count, 0);
+
+ /*
+ * Register the sysfs group before publishing the root in the
+ * checkpoint tree. nilfs_sysfs_create_snapshot_group() can sleep,
+ * so it must run outside ns_cptree_lock; creating it first also
+ * ensures a concurrent nilfs_lookup_root() can never observe a root
+ * whose sysfs registration later fails and gets freed.
+ */
+ err = nilfs_sysfs_create_snapshot_group(new);
+ if (err) {
+ kfree(new);
+ return NULL;
+ }
+
spin_lock(&nilfs->ns_cptree_lock);
p = &nilfs->ns_cptree.rb_node;
@@ -903,29 +923,17 @@ nilfs_find_or_create_root(struct the_nilfs *nilfs, __u64 cno)
} else {
refcount_inc(&root->count);
spin_unlock(&nilfs->ns_cptree_lock);
+ nilfs_sysfs_delete_snapshot_group(new);
kfree(new);
return root;
}
}
- new->cno = cno;
- new->ifile = NULL;
- new->nilfs = nilfs;
- refcount_set(&new->count, 1);
- atomic64_set(&new->inodes_count, 0);
- atomic64_set(&new->blocks_count, 0);
-
rb_link_node(&new->rb_node, parent, p);
rb_insert_color(&new->rb_node, &nilfs->ns_cptree);
spin_unlock(&nilfs->ns_cptree_lock);
- err = nilfs_sysfs_create_snapshot_group(new);
- if (err) {
- kfree(new);
- new = NULL;
- }
-
return new;
}
diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index a208a7ec1692..71b3a2361929 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -648,6 +648,9 @@ static struct fanotify_event *fanotify_alloc_name_event(struct inode *dir,
unsigned long name2_len = name2 ? name2->len : 0;
unsigned int len, size;
+ if (WARN_ON_ONCE(name_len >= PATH_MAX || name2_len >= PATH_MAX))
+ return NULL;
+
/* Reserve terminating null byte even for empty name */
size = sizeof(*fne) + name_len + name2_len + 2;
if (dir_fh_len)
@@ -656,7 +659,7 @@ static struct fanotify_event *fanotify_alloc_name_event(struct inode *dir,
size += FANOTIFY_FH_HDR_LEN + dir2_fh_len;
if (child_fh_len)
size += FANOTIFY_FH_HDR_LEN + child_fh_len;
- fne = kmalloc(size, gfp);
+ fne = kvmalloc(size, gfp);
if (!fne)
return NULL;
@@ -1050,7 +1053,7 @@ static void fanotify_free_fid_event(struct fanotify_event *event)
static void fanotify_free_name_event(struct fanotify_event *event)
{
- kfree(FANOTIFY_NE(event));
+ kvfree(FANOTIFY_NE(event));
}
static void fanotify_free_error_event(struct fsnotify_group *group,
diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index 3710543dbf82..87ba788574bf 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -43,9 +43,16 @@ struct fanotify_info {
u8 dir_fh_totlen;
u8 dir2_fh_totlen;
u8 file_fh_totlen;
- u8 name_len;
- u8 name2_len;
- u8 pad[3];
+ u8 pad;
+ /*
+ * Deliberately not limited to NAME_MAX: some filesystems return longer
+ * names (vfat up to 255 UTF-16 characters, as Windows allows, which is
+ * up to 765 bytes of UTF-8; FUSE up to PATH_MAX - 1), and readdir() and
+ * inotify report them in full. Only names of PATH_MAX bytes or more,
+ * which no path can refer to, are dropped.
+ */
+ u16 name_len;
+ u16 name2_len;
unsigned char buf[];
/*
* (struct fanotify_fh) dir_fh starts at buf[0]
@@ -168,7 +175,7 @@ static inline char *fanotify_info_name2(struct fanotify_info *info)
static inline void fanotify_info_init(struct fanotify_info *info)
{
BUILD_BUG_ON(FANOTIFY_FH_HDR_LEN + MAX_HANDLE_SZ > U8_MAX);
- BUILD_BUG_ON(NAME_MAX > U8_MAX);
+ BUILD_BUG_ON(PATH_MAX > U16_MAX);
info->dir_fh_totlen = 0;
info->dir2_fh_totlen = 0;
@@ -214,8 +221,7 @@ static inline void fanotify_info_set_file_fh(struct fanotify_info *info,
static inline void fanotify_info_copy_name(struct fanotify_info *info,
const struct qstr *name)
{
- if (WARN_ON_ONCE(name->len > NAME_MAX) ||
- WARN_ON_ONCE(info->name2_len > 0))
+ if (WARN_ON_ONCE(info->name2_len > 0))
return;
info->name_len = name->len;
@@ -225,9 +231,6 @@ static inline void fanotify_info_copy_name(struct fanotify_info *info,
static inline void fanotify_info_copy_name2(struct fanotify_info *info,
const struct qstr *name)
{
- if (WARN_ON_ONCE(name->len > NAME_MAX))
- return;
-
info->name2_len = name->len;
strscpy(fanotify_info_name2(info), name->name, name->len + 1);
}
diff --git a/fs/ntfs/attrib.c b/fs/ntfs/attrib.c
index c949ff765075..1f64680d5c8f 100644
--- a/fs/ntfs/attrib.c
+++ b/fs/ntfs/attrib.c
@@ -103,6 +103,17 @@ int ntfs_map_runlist_nolock(struct ntfs_inode *ni, s64 vcn, struct ntfs_attr_sea
base_ni = ni;
else
base_ni = ni->ext.base_ntfs_ino;
+ /*
+ * ntfs_read_inode_mount() builds $MFT's runlist itself, so nothing
+ * should reach here for $MFT. A crafted image can: the read that
+ * gets here already holds the $MFT folio lock it would wait on.
+ */
+ if (unlikely(NVolMftBootstrap(ni->vol) &&
+ base_ni == NTFS_I(ni->vol->mft_ino))) {
+ ntfs_error(ni->vol->sb,
+ "$MFT needs its own extent records to describe itself; cannot mount.");
+ return -EIO;
+ }
if (!ctx) {
ctx_is_temporary = ctx_needs_reset = true;
m = map_mft_record(base_ni);
@@ -317,7 +328,7 @@ int ntfs_map_runlist(struct ntfs_inode *ni, s64 vcn)
struct runlist_element *ntfs_attr_vcn_to_rl(struct ntfs_inode *ni, s64 vcn, s64 *lcn)
{
struct runlist_element *rl = ni->runlist.rl;
- int err;
+ int err = 0;
bool is_retry = false;
if (!rl) {
@@ -335,12 +346,34 @@ struct runlist_element *ntfs_attr_vcn_to_rl(struct ntfs_inode *ni, s64 vcn, s64
if (*lcn <= LCN_RL_NOT_MAPPED && is_retry == false) {
is_retry = true;
- if (!ntfs_map_runlist_nolock(ni, vcn, NULL)) {
+ err = ntfs_map_runlist_nolock(ni, vcn, NULL);
+ if (!err) {
rl = ni->runlist.rl;
goto remap_rl;
}
}
+ /*
+ * Neither the runlist nor the retry mapped @vcn, e.g. because the
+ * extent mft record holding it is corrupt or because the mapping
+ * pairs end too soon. ntfs_map_runlist_nolock() reports the latter
+ * as -ENOENT, as @vcn lies past the extent it found. Below the
+ * allocated size, callers would treat LCN_RL_NOT_MAPPED or LCN_ENOENT
+ * as a hole, so fail instead. At or beyond it nothing is mapped: the
+ * runlist ends there with LCN_ENOENT, or with LCN_RL_NOT_MAPPED if
+ * only a later extent has been mapped, so return that end as it is.
+ */
+ if (*lcn <= LCN_RL_NOT_MAPPED) {
+ unsigned long flags;
+ s64 allocated_size;
+
+ read_lock_irqsave(&ni->size_lock, flags);
+ allocated_size = ni->allocated_size;
+ read_unlock_irqrestore(&ni->size_lock, flags);
+ if ((s64)ntfs_cluster_to_bytes(ni->vol, vcn) < allocated_size)
+ return ERR_PTR(err == -ENOMEM ? -ENOMEM : -EIO);
+ }
+
return rl;
}
@@ -1775,7 +1808,7 @@ int ntfs_attr_size_bounds_check(const struct ntfs_volume *vol, const __le32 type
* $ATTRIBUTE_LIST has a maximum size of 256kiB, but this is not
* listed in $AttrDef.
*/
- if (unlikely(type == AT_ATTRIBUTE_LIST && size > 256 * 1024))
+ if (unlikely(type == AT_ATTRIBUTE_LIST && size > NTFS_MAX_ATTR_LIST_SIZE))
return -ERANGE;
/* Get the $AttrDef entry for the attribute @type. */
ad = ntfs_attr_find_in_attrdef(vol, type);
@@ -4234,12 +4267,14 @@ static int ntfs_attr_make_resident(struct ntfs_inode *ni, struct ntfs_attr_searc
* ntfs_non_resident_attr_shrink - shrink a non-resident, open ntfs attribute
* @ni: non-resident ntfs attribute to shrink
* @newsize: new size (in bytes) to which to shrink the attribute
+ * @pagecache_truncated: page cache was already truncated to @newsize
*
* Reduce the size of a non-resident, open ntfs attribute @na to @newsize bytes.
*/
static int ntfs_non_resident_attr_shrink(struct ntfs_inode *ni,
const s64 newsize,
- struct ntfs_inode *locked_ni)
+ struct ntfs_inode *locked_ni,
+ bool pagecache_truncated)
{
struct ntfs_volume *vol;
struct ntfs_attr_search_ctx *ctx;
@@ -4389,7 +4424,8 @@ static int ntfs_non_resident_attr_shrink(struct ntfs_inode *ni,
* later writeback map a vcn past the new allocation, which fails with
* -ENOENT and loses the write.
*/
- truncate_inode_pages(VFS_I(ni)->i_mapping, newsize);
+ if (!pagecache_truncated)
+ truncate_inode_pages(VFS_I(ni)->i_mapping, newsize);
/* Update data size in the index. */
if (ni->type == AT_DATA && ni->name == AT_UNNAMED)
@@ -4450,6 +4486,8 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
struct ntfs_inode *base_ni;
struct super_block *sb = ni->vol->sb;
size_t new_rl_count;
+ unsigned long flags;
+ bool runlist_locked = locked_ni == ni;
ntfs_debug("Inode 0x%llx, attr 0x%x, new size %lld old size %lld\n",
(unsigned long long)ni->mft_no, ni->type,
@@ -4491,10 +4529,19 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
* clusters if there is a change.
*/
if (ntfs_bytes_to_cluster(vol, ni->allocated_size) < first_free_vcn) {
+ /*
+ * The runlist array is replaced below. Readers such as the
+ * iomap read path hold only the runlist lock, not mrec_lock.
+ */
+ if (runlist_locked)
+ lockdep_assert_held_write(&ni->runlist.lock);
+ else
+ down_write(&ni->runlist.lock);
+
err = ntfs_attr_map_whole_runlist(ni);
if (err) {
ntfs_error(sb, "ntfs_attr_map_whole_runlist failed");
- return err;
+ goto unlock_runlist;
}
/*
@@ -4519,7 +4566,7 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
last + more_entries + 1);
if (IS_ERR(rl)) {
err = -ENOMEM;
- goto put_err_out;
+ goto unlock_runlist;
}
alloc_size = ni->allocated_size;
@@ -4544,7 +4591,7 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
rl = kmalloc(sizeof(struct runlist_element) * 2, GFP_NOFS);
if (!rl) {
err = -ENOMEM;
- goto put_err_out;
+ goto unlock_runlist;
}
rl[0].vcn = ntfs_bytes_to_cluster(vol, ni->allocated_size);
@@ -4591,7 +4638,8 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
ntfs_debug("Cluster allocation failed (%lld)",
(long long)first_free_vcn -
ntfs_bytes_to_cluster(vol, ni->allocated_size));
- return PTR_ERR(rl);
+ err = PTR_ERR(rl);
+ goto unlock_runlist;
}
/*
* A contiguous ATTRIBUTE_LIST allocation keeps its mapping
@@ -4615,8 +4663,10 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
ni->allocated_size),
lcn_seek_from, DATA_ZONE, false,
false, false);
- if (IS_ERR(rl))
- return PTR_ERR(rl);
+ if (IS_ERR(rl)) {
+ err = PTR_ERR(rl);
+ goto unlock_runlist;
+ }
}
}
@@ -4628,7 +4678,8 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
ntfs_error(sb, "Run list merge failed");
ntfs_cluster_free_from_rl(vol, rl);
kvfree(rl);
- return -EIO;
+ err = -EIO;
+ goto unlock_runlist;
}
ni->runlist.rl = rln;
ni->runlist.count = new_rl_count;
@@ -4636,11 +4687,28 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
/* Prepare to mapping pairs update. */
ni->allocated_size = ntfs_cluster_to_bytes(vol, first_free_vcn);
- err = ntfs_attr_update_mapping_pairs_locked(
- ni, 0, locked_ni);
- if (err) {
- ntfs_debug("Mapping pairs update failed");
- goto rollback;
+ if (ni->type == AT_ATTRIBUTE_LIST && !runlist_locked) {
+ /*
+ * Making room for the list's mapping pairs can resize
+ * this attribute list again through
+ * ntfs_attrlist_update_locked(), which takes its
+ * runlist lock.
+ */
+ up_write(&ni->runlist.lock);
+ err = ntfs_attr_update_mapping_pairs_locked(ni, 0,
+ locked_ni);
+ if (err) {
+ ntfs_debug("Mapping pairs update failed");
+ goto rollback;
+ }
+ } else {
+ err = ntfs_attr_update_mapping_pairs_locked(ni, 0, ni);
+ if (err) {
+ ntfs_debug("Mapping pairs update failed");
+ goto rollback_locked;
+ }
+ if (!runlist_locked)
+ up_write(&ni->runlist.lock);
}
}
@@ -4674,19 +4742,27 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
ntfs_attr_put_search_ctx(ctx);
return 0;
rollback:
+ if (!runlist_locked)
+ down_write(&ni->runlist.lock);
+rollback_locked:
/* Free allocated clusters. */
err2 = ntfs_cluster_free(ni, ntfs_bytes_to_cluster(vol, org_alloc_size),
-1, ctx);
if (err2)
ntfs_debug("Leaking clusters");
- /* Now, truncate the runlist itself. */
- if (ni != locked_ni)
- down_write(&ni->runlist.lock);
+ /*
+ * Now, truncate the runlist itself. Restore allocated_size before
+ * dropping the lock: ntfs_attr_vcn_to_rl() fails a lookup below the
+ * allocated size that falls past the end of the runlist.
+ */
err2 = ntfs_rl_truncate_nolock(vol, &ni->runlist,
ntfs_bytes_to_cluster(vol, org_alloc_size));
- if (ni != locked_ni)
- up_write(&ni->runlist.lock);
+ if (!err2) {
+ write_lock_irqsave(&ni->size_lock, flags);
+ ni->allocated_size = org_alloc_size;
+ write_unlock_irqrestore(&ni->size_lock, flags);
+ }
if (err2) {
/*
* Failed to truncate the runlist, so just throw it away, it
@@ -4696,15 +4772,9 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
ni->runlist.rl = NULL;
ntfs_error(sb, "Couldn't truncate runlist. Rollback failed");
} else {
- /* Prepare to mapping pairs update. */
- ni->allocated_size = org_alloc_size;
/* Restore mapping pairs. */
- if (ni != locked_ni)
- down_read(&ni->runlist.lock);
- if (__ntfs_attr_update_mapping_pairs(ni, 0, locked_ni, true))
+ if (__ntfs_attr_update_mapping_pairs(ni, 0, ni, true))
ntfs_error(sb, "Failed to restore old mapping pairs");
- if (ni != locked_ni)
- up_read(&ni->runlist.lock);
if (NInoSparse(ni) || NInoCompressed(ni)) {
ni->itype.compressed.size = org_compressed_size;
@@ -4712,6 +4782,8 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
} else
VFS_I(base_ni)->i_blocks = ni->allocated_size >> 9;
}
+ if (!runlist_locked)
+ up_write(&ni->runlist.lock);
if (ctx)
ntfs_attr_put_search_ctx(ctx);
return err;
@@ -4719,6 +4791,10 @@ static int ntfs_non_resident_attr_expand(struct ntfs_inode *ni, const s64 newsiz
if (ctx)
ntfs_attr_put_search_ctx(ctx);
return err;
+unlock_runlist:
+ if (!runlist_locked)
+ up_write(&ni->runlist.lock);
+ return err;
}
/*
@@ -4991,7 +5067,7 @@ int __ntfs_attr_truncate_vfs(struct ntfs_inode *ni, const s64 newsize,
up_write(&ni->runlist.lock);
} else
err = ntfs_non_resident_attr_shrink(
- ni, newsize, NULL);
+ ni, newsize, NULL, true);
} else
err = ntfs_resident_attr_resize(ni, newsize, 0,
NVolDisableSparse(ni->vol) ?
@@ -5104,7 +5180,7 @@ int ntfs_attr_truncate_i_locked(struct ntfs_inode *ni, const s64 newsize,
ni, newsize, 0, holes, locked_ni);
else
err = ntfs_non_resident_attr_shrink(
- ni, newsize, locked_ni);
+ ni, newsize, locked_ni, false);
} else
err = ntfs_resident_attr_resize(ni, newsize, 0, holes);
ntfs_debug("Return status %d\n", err);
@@ -5855,7 +5931,7 @@ int ntfs_attr_fallocate(struct ntfs_inode *ni, loff_t start, loff_t byte_len, bo
goto out;
}
- if (signal_pending(current))
+ if (fatal_signal_pending(current))
goto signal_out;
vcn += alloc_cnt;
@@ -5876,7 +5952,7 @@ int ntfs_attr_fallocate(struct ntfs_inode *ni, loff_t start, loff_t byte_len, bo
try_alloc_cnt, &balloc, false, false);
up_write(&ni->runlist.lock);
mutex_unlock(&ni->mrec_lock);
- if (err || signal_pending(current))
+ if (err || fatal_signal_pending(current))
goto signal_out;
vcn += alloc_cnt;
diff --git a/fs/ntfs/attrlist.c b/fs/ntfs/attrlist.c
index bb191953dcb1..78e9a1dc7925 100644
--- a/fs/ntfs/attrlist.c
+++ b/fs/ntfs/attrlist.c
@@ -14,8 +14,6 @@
#include "attrlist.h"
#include "lcnalloc.h"
-#define NTFS_MAX_ATTR_LIST_SIZE (256 * 1024)
-
/*
* ntfs_attrlist_need - check whether inode need attribute list
* @ni: opened ntfs inode for which perform check
@@ -170,14 +168,18 @@ static int ntfs_attrlist_repack(struct inode *attr_vi,
return 0;
restore_old_runlist:
+ /*
+ * Restore allocated_size before dropping the runlist lock:
+ * ntfs_attr_vcn_to_rl() fails a lookup below the allocated size that
+ * falls past the end of the runlist.
+ */
down_write(&attr_ni->runlist.lock);
attr_ni->runlist.rl = old_rl;
attr_ni->runlist.count = old_rl_count;
- up_write(&attr_ni->runlist.lock);
-
write_lock_irqsave(&attr_ni->size_lock, flags);
attr_ni->allocated_size = old_alloc_size;
write_unlock_irqrestore(&attr_ni->size_lock, flags);
+ up_write(&attr_ni->runlist.lock);
restore_err = ntfs_attr_update_mapping_pairs_locked(
attr_ni, 0, locked_ni);
diff --git a/fs/ntfs/dir.c b/fs/ntfs/dir.c
index df60138f9b2d..09b890b41c4a 100644
--- a/fs/ntfs/dir.c
+++ b/fs/ntfs/dir.c
@@ -1182,6 +1182,8 @@ static int ntfs_dir_fsync(struct file *filp, loff_t start, loff_t end,
err = sync_blockdev(vi->i_sb->s_bdev);
if (unlikely(err && !ret))
ret = err;
+ if (!ret)
+ ret = blkdev_issue_flush(vi->i_sb->s_bdev);
if (likely(!ret))
ntfs_debug("Done.");
else
diff --git a/fs/ntfs/ea.c b/fs/ntfs/ea.c
index b4fcfbe2da4c..6ac551fe8ea6 100644
--- a/fs/ntfs/ea.c
+++ b/fs/ntfs/ea.c
@@ -1027,8 +1027,9 @@ static noinline int ntfs_set_acl_ex(struct mnt_idmap *idmap,
{
const char *name;
size_t size, name_len;
- void *value;
- int err;
+ void *value, *old_value = NULL;
+ int err, old_len = -ENODATA;
+ bool acl_committed = false;
int flags;
umode_t mode;
@@ -1072,7 +1073,34 @@ static noinline int ntfs_set_acl_ex(struct mnt_idmap *idmap,
}
mutex_lock(&NTFS_I(inode)->mrec_lock);
+ if (!init_acl) {
+ /* Save the old xattr for rollback. */
+ old_len = ntfs_get_ea(inode, name, name_len, NULL, 0);
+ if (old_len >= 0) {
+ /* kmalloc(0) returns the non-NULL ZERO_SIZE_PTR. */
+ old_value = kmalloc(old_len, GFP_NOFS);
+ if (!old_value) {
+ err = -ENOMEM;
+ goto out_unlock;
+ }
+ err = ntfs_get_ea(inode, name, name_len,
+ old_value, old_len);
+ if (err != old_len) {
+ kfree(old_value);
+ old_value = NULL;
+ if (err >= 0)
+ err = -EIO;
+ goto out_unlock;
+ }
+ } else if (old_len != -ENODATA) {
+ err = old_len;
+ goto out_unlock;
+ }
+ }
err = ntfs_set_ea(inode, name, name_len, value, size, flags, NULL);
+ if (!err)
+ acl_committed = true;
+out_unlock:
mutex_unlock(&NTFS_I(inode)->mrec_lock);
if (err == -ENODATA && !size)
err = 0; /* Removing non existed xattr. */
@@ -1084,8 +1112,38 @@ static noinline int ntfs_set_acl_ex(struct mnt_idmap *idmap,
mutex_lock(&NTFS_I(inode)->mrec_lock);
err = ntfs_ea_set_wsl_inode(inode, 0, &ea_size, NTFS_EA_MODE);
if (err) {
- ntfs_set_ea(inode, name, name_len, NULL, 0,
- XATTR_REPLACE, NULL);
+ if (acl_committed) {
+ int undo_err;
+
+ if (old_len >= 0) {
+ /*
+ * A removed or zero-length xattr must
+ * be recreated without XATTR_REPLACE.
+ */
+ int undo_flags = acl && old_len ?
+ XATTR_REPLACE : 0;
+
+ undo_err = ntfs_set_ea(inode, name,
+ name_len, old_value,
+ old_len, undo_flags,
+ NULL);
+ } else {
+ undo_err = ntfs_set_ea(inode, name,
+ name_len, NULL, 0,
+ XATTR_REPLACE, NULL);
+ }
+ if (undo_err) {
+ /*
+ * Record a failed rollback so the next
+ * persistence point marks the volume
+ * dirty. Run chkdsk.
+ */
+ NVolSetErrors(NTFS_I(inode)->vol);
+ ntfs_error(inode->i_sb,
+ "Failed to undo the ACL xattr: %d",
+ undo_err);
+ }
+ }
mutex_unlock(&NTFS_I(inode)->mrec_lock);
inode->i_mode = old_mode;
goto out;
@@ -1098,6 +1156,7 @@ static noinline int ntfs_set_acl_ex(struct mnt_idmap *idmap,
}
out:
+ kfree(old_value);
kfree(value);
return err;
diff --git a/fs/ntfs/file.c b/fs/ntfs/file.c
index 007d1614b9ac..f8f4418cbcda 100644
--- a/fs/ntfs/file.c
+++ b/fs/ntfs/file.c
@@ -241,17 +241,33 @@ static int ntfs_file_fsync(struct file *filp, loff_t start, loff_t end,
err = sync_blockdev(vi->i_sb->s_bdev);
if (unlikely(err && !ret))
ret = err;
+ if (!ret)
+ ret = blkdev_issue_flush(vi->i_sb->s_bdev);
if (likely(!ret))
ntfs_debug("Done.");
else
ntfs_warning(vi->i_sb,
"Failed to f%ssync inode 0x%llx. Error %u.",
datasync ? "data" : "", ni->mft_no, -ret);
- if (!ret)
- blkdev_issue_flush(vi->i_sb->s_bdev);
return ret;
}
+static void ntfs_pagecache_extend(struct inode *vi, loff_t from, loff_t to)
+{
+ pagecache_isize_extended(vi, from, to);
+
+ /*
+ * NTFS initialized_size is byte-granular, so a partial old-EOF page
+ * must stay write-protected until page_mkwrite() updates it. The
+ * generic helper skips this when the block size is at least a page,
+ * the extension ends before the rounded block boundary, or that
+ * boundary is page-aligned.
+ */
+ if (from < to && from & (PAGE_SIZE - 1))
+ unmap_mapping_range(vi->i_mapping, round_down(from, PAGE_SIZE),
+ PAGE_SIZE, 0);
+}
+
static int ntfs_setattr_size(struct inode *vi, struct iattr *attr)
{
struct ntfs_inode *ni = NTFS_I(vi);
@@ -281,7 +297,7 @@ static int ntfs_setattr_size(struct inode *vi, struct iattr *attr)
if (attr->ia_size > old_size) {
truncate_pagecache(vi, old_size);
i_size_write(vi, attr->ia_size);
- pagecache_isize_extended(vi, old_size, attr->ia_size);
+ ntfs_pagecache_extend(vi, old_size, attr->ia_size);
} else {
truncate_setsize(vi, attr->ia_size);
}
@@ -325,8 +341,7 @@ int ntfs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
goto out;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
if (ia_valid & ATTR_SIZE) {
err = ntfs_setattr_size(vi, attr);
@@ -620,8 +635,14 @@ static ssize_t ntfs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
goto out_lock;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ /*
+ * The volume must be marked dirty before the modification is made,
+ * without an unlocked check of the in-memory flag: the dirty bit
+ * is only cleared at the quiescent transitions, under the same
+ * $Volume mrec_lock this call takes, so an unlocked skip could
+ * lose the set to one of them.
+ */
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
pos = iocb->ki_pos;
count = ret;
@@ -1131,7 +1152,7 @@ static long ntfs_fallocate(struct file *file, int mode, loff_t offset, loff_t le
struct ntfs_inode *ni = NTFS_I(vi);
struct ntfs_volume *vol = ni->vol;
int err = 0;
- loff_t old_size;
+ loff_t old_size, new_size;
if (mode & ~(NTFS_FALLOC_FL_SUPPORTED))
return -EOPNOTSUPP;
@@ -1153,19 +1174,16 @@ static long ntfs_fallocate(struct file *file, int mode, loff_t offset, loff_t le
return err;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY)) {
- err = ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
- if (err)
- return err;
- }
-
- old_size = i_size_read(vi);
+ err = ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ if (err)
+ return err;
inode_lock(vi);
if (NInoCompressed(ni) || NInoEncrypted(ni) || NInoWofCompressed(ni)) {
inode_unlock(vi);
return -EOPNOTSUPP;
}
+ old_size = i_size_read(vi);
inode_dio_wait(vi);
/* Take invalidate_lock for all fallocate operations to prevent races */
@@ -1194,10 +1212,12 @@ static long ntfs_fallocate(struct file *file, int mode, loff_t offset, loff_t le
err = file_modified(file);
out:
- if (!err && mode == 0 && NInoNonResident(ni) &&
- offset > old_size) {
- truncate_pagecache(vi, old_size);
- pagecache_isize_extended(vi, old_size, offset);
+ if (!err && mode == 0 && NInoNonResident(ni)) {
+ new_size = i_size_read(vi);
+ if (new_size > old_size) {
+ truncate_pagecache(vi, old_size);
+ ntfs_pagecache_extend(vi, old_size, new_size);
+ }
}
filemap_invalidate_unlock(vi->i_mapping);
diff --git a/fs/ntfs/inode.c b/fs/ntfs/inode.c
index a777de8a80c7..213c4a12b33a 100644
--- a/fs/ntfs/inode.c
+++ b/fs/ntfs/inode.c
@@ -805,6 +805,8 @@ static int ntfs_read_locked_inode(struct inode *vi)
goto unm_err_out;
}
} else {
+ s64 attr_list_size;
+
if (vi->i_ino == FILE_MFT)
goto skip_attr_list_load;
ntfs_debug("Attribute list found in inode 0x%llx.", ni->mft_no);
@@ -827,11 +829,14 @@ static int ntfs_read_locked_inode(struct inode *vi)
ni->mft_no);
}
/* Now allocate memory for the attribute list. */
- ni->attr_list_size = (u32)ntfs_attr_size(a);
- if (!ni->attr_list_size) {
- ntfs_error(vi->i_sb, "Attr_list_size is zero");
+ attr_list_size = ntfs_attr_size(a);
+ if (attr_list_size <= 0 ||
+ attr_list_size > NTFS_MAX_ATTR_LIST_SIZE) {
+ ntfs_error(vi->i_sb, "Invalid attribute list size %lld (mft_no 0x%llx).",
+ (long long)attr_list_size, ni->mft_no);
goto unm_err_out;
}
+ ni->attr_list_size = (u32)attr_list_size;
ni->attr_list = kvzalloc(ni->attr_list_size, GFP_NOFS);
if (!ni->attr_list) {
ntfs_error(vi->i_sb,
@@ -1955,6 +1960,7 @@ int ntfs_read_inode_mount(struct inode *vi)
} else /* if (!err) */ {
struct attr_list_entry *al_entry, *next_al_entry;
u8 *al_end;
+ s64 attr_list_size;
static const char *es = " Not allowed. $MFT is corrupt. You should run chkdsk.";
ntfs_debug("Attribute list attribute found in $MFT.");
@@ -1978,11 +1984,14 @@ int ntfs_read_inode_mount(struct inode *vi)
"Resident attribute list attribute in $MFT system file is marked encrypted/sparse which is not true. However, Windows allows this and chkdsk does not detect or correct it so we will just ignore the invalid flags and pretend they are not set.");
}
/* Now allocate memory for the attribute list. */
- ni->attr_list_size = (u32)ntfs_attr_size(a);
- if (!ni->attr_list_size) {
- ntfs_error(sb, "Attr_list_size is zero");
+ attr_list_size = ntfs_attr_size(a);
+ if (attr_list_size <= 0 ||
+ attr_list_size > NTFS_MAX_ATTR_LIST_SIZE) {
+ ntfs_error(sb, "Invalid attribute list size %lld (mft_no 0x%llx).%s",
+ (long long)attr_list_size, ni->mft_no, es);
goto put_err_out;
}
+ ni->attr_list_size = (u32)attr_list_size;
ni->attr_list = kvzalloc(round_up(ni->attr_list_size, SECTOR_SIZE),
GFP_NOFS);
if (!ni->attr_list) {
@@ -2071,6 +2080,11 @@ int ntfs_read_inode_mount(struct inode *vi)
/* Now load all attribute extents. */
a = NULL;
next_vcn = last_vcn = highest_vcn = 0;
+ /*
+ * Reading one of $MFT's own extent records in this loop can re-enter
+ * ntfs_map_runlist_nolock() for $MFT; see the check there.
+ */
+ NVolSetMftBootstrap(vol);
while (!(err = ntfs_attr_lookup(AT_DATA, NULL, 0, 0, next_vcn, NULL, 0,
ctx))) {
struct runlist_element *nrl;
@@ -2122,6 +2136,16 @@ int ntfs_read_inode_mount(struct inode *vi)
vi->i_size = le64_to_cpu(a->data.non_resident.data_size);
ni->initialized_size = le64_to_cpu(a->data.non_resident.initialized_size);
ni->allocated_size = le64_to_cpu(a->data.non_resident.allocated_size);
+ /*
+ * Records between allocated_size and data_size are not
+ * on disk, and would be read as zeros.
+ */
+ if (vi->i_size > ni->allocated_size) {
+ ntfs_error(sb,
+ "$MFT data size %lld exceeds its allocated size %lld. $MFT is corrupt. Run chkdsk.",
+ vi->i_size, ni->allocated_size);
+ goto put_err_out;
+ }
/*
* Verify the number of mft records does not exceed
* 2^32 - 1.
@@ -2153,6 +2177,7 @@ int ntfs_read_inode_mount(struct inode *vi)
err = ntfs_read_locked_inode(vi);
if (err) {
ntfs_error(sb, "ntfs_read_inode() of $MFT failed.\n");
+ NVolClearMftBootstrap(vol);
ntfs_attr_put_search_ctx(ctx);
/* Revert to the safe super operations. */
kfree(m);
@@ -2186,6 +2211,7 @@ int ntfs_read_inode_mount(struct inode *vi)
goto put_err_out;
}
}
+ NVolClearMftBootstrap(vol);
if (err != -ENOENT) {
ntfs_error(sb, "Failed to lookup $MFT/$DATA attribute extent. Run chkdsk.\n");
goto put_err_out;
@@ -2220,6 +2246,8 @@ int ntfs_read_inode_mount(struct inode *vi)
put_err_out:
ntfs_attr_put_search_ctx(ctx);
err_out:
+ /* Also reached from inside the $DATA loop. */
+ NVolClearMftBootstrap(vol);
ntfs_error(sb, "Failed. Marking inode as bad.");
kfree(m);
return -1;
@@ -3739,6 +3767,8 @@ static s64 __ntfs_inode_non_resident_attr_pwrite(struct inode *vi,
folio_lock(folio);
}
+ folio_wait_writeback(folio);
+
if (count == PAGE_SIZE) {
offset = 0;
attr_len = count;
@@ -3758,7 +3788,9 @@ static s64 __ntfs_inode_non_resident_attr_pwrite(struct inode *vi,
struct runlist_element *rl;
int bio_err;
- lcn_count = max_t(s64, 1, ntfs_bytes_to_cluster(vol, attr_len));
+ lcn_count = max_t(s64, 1,
+ ntfs_bytes_to_cluster(vol, attr_len +
+ vol->cluster_size - 1));
vcn = ntfs_pidx_to_cluster(vol, folio->index);
do {
diff --git a/fs/ntfs/iomap.c b/fs/ntfs/iomap.c
index c812d7f19b36..b4475963e57a 100644
--- a/fs/ntfs/iomap.c
+++ b/fs/ntfs/iomap.c
@@ -394,7 +394,7 @@ static int ntfs_write_simple_iomap_begin_non_resident(struct inode *inode, loff_
loff_t vcn_ofs, rl_length;
struct runlist_element *rl, *rlc;
bool is_retry = false;
- int err = 0;
+ int err = 0, map_err = 0;
s64 vcn, lcn;
s64 max_clu_count =
ntfs_bytes_to_cluster(vol, round_up(length, vol->cluster_size));
@@ -429,12 +429,24 @@ static int ntfs_write_simple_iomap_begin_non_resident(struct inode *inode, loff_
if (lcn <= LCN_RL_NOT_MAPPED && is_retry == false) {
is_retry = true;
- if (!ntfs_map_runlist_nolock(ni, vcn, NULL)) {
+ map_err = ntfs_map_runlist_nolock(ni, vcn, NULL);
+ if (!map_err) {
rl = ni->runlist.rl;
goto remap_rl;
}
}
+ /*
+ * As in ntfs_attr_vcn_to_rl(): a runlist fragment that could not be
+ * mapped is not a hole. Treating it as one would put a delalloc
+ * extent over clusters that are allocated on disk but unknown to us.
+ */
+ if (lcn == LCN_RL_NOT_MAPPED) {
+ up_write(&ni->runlist.lock);
+ mutex_unlock(&ni->mrec_lock);
+ return map_err == -ENOMEM ? -ENOMEM : -EIO;
+ }
+
max_clu_count = min(max_clu_count, rl->length - (vcn - rl->vcn));
if (max_clu_count == 0) {
ntfs_error(inode->i_sb,
diff --git a/fs/ntfs/logfile.c b/fs/ntfs/logfile.c
index 1404664dacc0..ce38ce87f8fc 100644
--- a/fs/ntfs/logfile.c
+++ b/fs/ntfs/logfile.c
@@ -457,7 +457,7 @@ bool ntfs_check_logfile(struct inode *log_vi, struct restart_page_header **rp)
struct ntfs_volume *vol = NTFS_SB(log_vi->i_sb);
struct address_space *mapping = log_vi->i_mapping;
struct folio *folio = NULL;
- u8 *kaddr = NULL;
+ u8 *kaddr = NULL, *kbase = NULL;
struct restart_page_header *rstr1_ph = NULL;
struct restart_page_header *rstr2_ph = NULL;
int log_page_size, err;
@@ -511,7 +511,7 @@ bool ntfs_check_logfile(struct inode *log_vi, struct restart_page_header **rp)
if (!folio || folio->index != idx) {
if (folio) {
- kunmap_local(kaddr);
+ kunmap_local(kbase);
folio_put(folio);
}
folio = read_mapping_folio(mapping, idx, NULL);
@@ -520,8 +520,9 @@ bool ntfs_check_logfile(struct inode *log_vi, struct restart_page_header **rp)
idx);
goto err_out;
}
+ kbase = kmap_local_folio(folio, 0);
}
- kaddr = (u8 *)kmap_local_folio(folio, 0) + (pos & ~PAGE_MASK);
+ kaddr = kbase + (pos & ~PAGE_MASK);
/*
* A non-empty block means the logfile is not empty while an
* empty block after a non-empty block has been encountered
@@ -574,7 +575,7 @@ bool ntfs_check_logfile(struct inode *log_vi, struct restart_page_header **rp)
* find a valid one further in the file.
*/
if (err != -EINVAL) {
- kunmap_local(kaddr);
+ kunmap_local(kbase);
folio_put(folio);
goto err_out;
}
@@ -583,7 +584,7 @@ bool ntfs_check_logfile(struct inode *log_vi, struct restart_page_header **rp)
pos = NTFS_BLOCK_SIZE >> 1;
}
if (folio) {
- kunmap_local(kaddr);
+ kunmap_local(kbase);
folio_put(folio);
}
if (logfile_is_empty) {
diff --git a/fs/ntfs/mft.c b/fs/ntfs/mft.c
index 4b7449495375..482a216ade2c 100644
--- a/fs/ntfs/mft.c
+++ b/fs/ntfs/mft.c
@@ -10,7 +10,10 @@
#include <linux/writeback.h>
#include <linux/bio.h>
+#include <linux/blkdev.h>
+#include <linux/completion.h>
#include <linux/iomap.h>
+#include <linux/math64.h>
#include "bitmap.h"
#include "lcnalloc.h"
@@ -429,81 +432,152 @@ void __mark_mft_record_dirty(struct ntfs_inode *ni)
__mark_inode_dirty(VFS_I(base_ni), I_DIRTY_DATASYNC);
}
-/*
- * ntfs_bio_end_io - bio completion callback for MFT record writes
- *
- * Decrements the folio reference count that was incremented before
- * submit_bio(). This prevents a race condition where umount could
- * evict the inode and release the folio while I/O is still in flight,
- * potentially causing data corruption or use-after-free.
- */
-static void ntfs_bio_end_io(struct bio *bio)
+struct ntfs_mft_io_unit {
+ sector_t sector;
+ unsigned int folio_ofs;
+ unsigned int len;
+};
+
+struct ntfs_mft_write_ctx {
+ struct folio *folio;
+ struct address_space *mapping;
+ struct ntfs_volume *vol;
+ struct completion *done;
+ int error;
+ bool writeback_started;
+ struct bio bio;
+};
+
+static struct bio_set ntfs_mft_bioset;
+
+int ntfs_mft_bioset_init(void)
+{
+ return bioset_init(&ntfs_mft_bioset, BIO_POOL_SIZE,
+ offsetof(struct ntfs_mft_write_ctx, bio),
+ BIOSET_NEED_BVECS);
+}
+
+void ntfs_mft_bioset_exit(void)
+{
+ bioset_exit(&ntfs_mft_bioset);
+}
+
+static void ntfs_mft_end_io(struct bio *bio)
+{
+ struct ntfs_mft_write_ctx *ctx =
+ container_of(bio, struct ntfs_mft_write_ctx, bio);
+ struct folio *folio = ctx->folio;
+ struct completion *done = ctx->done;
+ int err;
+
+ if (bio->bi_status)
+ err = blk_status_to_errno(bio->bi_status);
+ else
+ err = ctx->error;
+ if (err) {
+ mapping_set_error(ctx->mapping, err);
+ NVolSetErrors(ctx->vol);
+ ntfs_error(ctx->vol->sb, "I/O error while writing MFT: %d",
+ err);
+ }
+
+ if (!done)
+ bio_put(bio);
+ folio_end_writeback(folio);
+ folio_put(folio);
+ if (done)
+ complete(done);
+}
+
+static void ntfs_start_mft_writeback(struct ntfs_mft_write_ctx *ctx)
+{
+ if (ctx->writeback_started)
+ return;
+
+ inode_attach_wb(ctx->mapping->host, NULL);
+ folio_get(ctx->folio);
+ folio_start_writeback(ctx->folio);
+ ctx->writeback_started = true;
+}
+
+static struct bio *ntfs_alloc_mft_parent_bio(struct ntfs_volume *vol,
+ struct folio *folio,
+ struct completion *done)
{
- if (bio->bi_private)
- folio_put((struct folio *)bio->bi_private);
- bio_put(bio);
+ struct ntfs_mft_write_ctx *ctx;
+ struct bio *parent;
+
+ parent = bio_alloc_bioset(vol->sb->s_bdev, 1, REQ_OP_WRITE, GFP_NOIO,
+ &ntfs_mft_bioset);
+ if (!parent)
+ return NULL;
+
+ ctx = container_of(parent, struct ntfs_mft_write_ctx, bio);
+ ctx->folio = folio;
+ ctx->mapping = folio->mapping;
+ ctx->vol = vol;
+ ctx->done = done;
+ ctx->error = 0;
+ ctx->writeback_started = false;
+ parent->bi_end_io = ntfs_mft_end_io;
+ return parent;
}
/*
- * ntfs_sync_mft_mirror - synchronize an mft record to the mft mirror
- * @vol: ntfs volume on which the mft record to synchronize resides
- * @mft_no: mft record number of mft record to synchronize
- * @m: mapped, mst protected (extent) mft record to synchronize
- *
- * Write the mapped, mst protected (extent) mft record @m with mft record
- * number @mft_no to the mft mirror ($MFTMirr) of the ntfs volume @vol.
- *
- * On success return 0. On error return -errno and set the volume errors flag
- * in the ntfs volume @vol.
- *
- * NOTE: We always perform synchronous i/o.
+ * Write one MFT I/O unit to $MFTMirr. The source folio contains the MST
+ * protected image that will be written to $MFT.
*/
-int ntfs_sync_mft_mirror(struct ntfs_volume *vol, const u64 mft_no,
- struct mft_record *m)
+static int ntfs_sync_mft_mirror_unit(struct ntfs_volume *vol,
+ struct folio *source, u64 mirror_file_ofs,
+ const struct ntfs_mft_io_unit *unit)
{
- u8 *kmirr;
- struct folio *folio;
- unsigned int folio_ofs;
- int err = 0;
- struct bio *bio;
+ struct folio *mirror;
+ struct bio_vec bvec;
+ struct bio bio;
+ u64 mirror_size;
+ unsigned int mirror_ofs;
+ u8 *src, *dst;
+ int err;
- ntfs_debug("Entering for inode 0x%llx.", mft_no);
+ if (unlikely(!vol->mftmirr_ino))
+ return -EIO;
+
+ mirror_size = (u64)vol->mftmirr_size * vol->mft_record_size;
+ if (mirror_file_ofs >= mirror_size ||
+ unit->len > mirror_size - mirror_file_ofs)
+ return -EIO;
+ if (unit->folio_ofs + unit->len > folio_size(source))
+ return -EIO;
- if (unlikely(!vol->mftmirr_ino)) {
- /* This could happen during umount... */
+ mirror = read_mapping_folio(vol->mftmirr_ino->i_mapping,
+ mirror_file_ofs >> PAGE_SHIFT, NULL);
+ if (IS_ERR(mirror))
+ return PTR_ERR(mirror);
+
+ folio_lock(mirror);
+ if (folio_test_writeback(mirror))
+ folio_wait_writeback(mirror);
+ mirror_ofs = mirror_file_ofs - folio_pos(mirror);
+ if (mirror_ofs + unit->len > folio_size(mirror)) {
err = -EIO;
- goto err_out;
- }
- /* Get the page containing the mirror copy of the mft record @m. */
- folio = read_mapping_folio(vol->mftmirr_ino->i_mapping,
- NTFS_MFT_NR_TO_PIDX(vol, mft_no), NULL);
- if (IS_ERR(folio)) {
- ntfs_error(vol->sb, "Failed to map mft mirror page.");
- err = PTR_ERR(folio);
- goto err_out;
+ goto out_unlock;
}
- folio_lock(folio);
- folio_clear_uptodate(folio);
- /* Offset of the mft mirror record inside the page. */
- folio_ofs = NTFS_MFT_NR_TO_POFS(vol, mft_no);
- /* The address in the page of the mirror copy of the mft record @m. */
- kmirr = kmap_local_folio(folio, 0) + folio_ofs;
- /* Copy the mst protected mft record to the mirror. */
- memcpy(kmirr, m, vol->mft_record_size);
- kunmap_local(kmirr);
-
- bio = bio_alloc(vol->sb->s_bdev, 1, REQ_OP_WRITE, GFP_NOIO);
- bio->bi_iter.bi_sector =
- ntfs_bytes_to_bio_sector(NTFS_CLU_TO_B(vol, vol->mftmirr_lcn) +
- ((u64)folio->index << PAGE_SHIFT) +
- folio_ofs);
-
- if (bio_add_folio(bio, folio, vol->mft_record_size, folio_ofs))
- err = submit_bio_wait(bio);
- else
+ folio_clear_uptodate(mirror);
+ src = kmap_local_folio(source, unit->folio_ofs);
+ dst = kmap_local_folio(mirror, mirror_ofs);
+ memcpy(dst, src, unit->len);
+ kunmap_local(dst);
+ kunmap_local(src);
+
+ bio_init(&bio, vol->sb->s_bdev, &bvec, 1, REQ_OP_WRITE);
+ bio.bi_iter.bi_sector = ntfs_bytes_to_bio_sector(
+ NTFS_CLU_TO_B(vol, vol->mftmirr_lcn) + mirror_file_ofs);
+ if (!bio_add_folio(&bio, mirror, unit->len, mirror_ofs))
err = -EIO;
- bio_put(bio);
+ else
+ err = submit_bio_wait(&bio);
+ bio_uninit(&bio);
/*
* The in-memory mirror is now valid because we just memcpy()'d the
@@ -512,23 +586,79 @@ int ntfs_sync_mft_mirror(struct ntfs_volume *vol, const u64 mft_no,
* the stale on-disk mirror and overwrite this copy. The error is
* propagated to the caller via @err.
*/
- folio_mark_uptodate(folio);
+ folio_mark_uptodate(mirror);
- folio_unlock(folio);
- folio_put(folio);
- if (likely(!err)) {
- ntfs_debug("Done.");
- } else {
- ntfs_error(vol->sb, "I/O error while writing mft mirror record 0x%llx!", mft_no);
-err_out:
- ntfs_error(vol->sb,
- "Failed to synchronize $MFTMirr (error code %i). Volume will be left marked dirty on umount. Run chkdsk on the partition after umounting to correct this.",
- err);
- NVolSetErrors(vol);
- }
+out_unlock:
+ folio_unlock(mirror);
+ folio_put(mirror);
return err;
}
+static int ntfs_sync_mft_mirror_record(struct ntfs_volume *vol,
+ struct folio *source, const u64 mft_no)
+{
+ u64 mirror_file_ofs = (u64)mft_no * vol->mft_record_size;
+ struct ntfs_mft_io_unit unit = {
+ .folio_ofs = NTFS_MFT_NR_TO_POFS(vol, mft_no),
+ .len = vol->mft_record_size,
+ };
+
+ mirror_file_ofs =
+ round_down(mirror_file_ofs, (u64)vol->mft_io_unit_size);
+ unit.folio_ofs = round_down(unit.folio_ofs, vol->mft_io_unit_size);
+ unit.len = vol->mft_io_unit_size;
+
+ return ntfs_sync_mft_mirror_unit(vol, source, mirror_file_ofs, &unit);
+}
+
+static int ntfs_prepare_mft_record_io_units(struct ntfs_inode *ni,
+ struct ntfs_mft_io_unit units[2])
+{
+ struct ntfs_volume *vol = ni->vol;
+ u64 record_byte = (u64)ni->mft_no * vol->mft_record_size;
+ u64 disk_byte;
+ unsigned int nr_units = ni->mft_lcn_count;
+ unsigned int cluster_ofs;
+ unsigned int i;
+
+ if (!nr_units || nr_units > ARRAY_SIZE(ni->mft_lcn))
+ return -EIO;
+ for (i = 0; i < nr_units; i++)
+ if (ni->mft_lcn[i] < 0)
+ return -EIO;
+
+ cluster_ofs = ntfs_bytes_to_cluster_off(vol, record_byte);
+ if (vol->mft_io_unit_size > vol->mft_record_size) {
+ cluster_ofs = round_down(cluster_ofs, vol->mft_io_unit_size);
+ disk_byte = NTFS_CLU_TO_B(vol, ni->mft_lcn[0]) + cluster_ofs;
+ units[0] = (struct ntfs_mft_io_unit){
+ .sector = ntfs_bytes_to_bio_sector(disk_byte),
+ .folio_ofs = round_down(ni->folio_ofs,
+ vol->mft_io_unit_size),
+ .len = vol->mft_io_unit_size,
+ };
+ return 1;
+ }
+
+ disk_byte = NTFS_CLU_TO_B(vol, ni->mft_lcn[0]) + cluster_ofs;
+ units[0] = (struct ntfs_mft_io_unit){
+ .sector = ntfs_bytes_to_bio_sector(disk_byte),
+ .folio_ofs = ni->folio_ofs,
+ .len = vol->mft_record_size,
+ };
+ if (nr_units == 1)
+ return 1;
+
+ units[0].len = vol->cluster_size;
+ disk_byte = NTFS_CLU_TO_B(vol, ni->mft_lcn[1]);
+ units[1] = (struct ntfs_mft_io_unit){
+ .sector = ntfs_bytes_to_bio_sector(disk_byte),
+ .folio_ofs = ni->folio_ofs + vol->cluster_size,
+ .len = vol->mft_record_size - vol->cluster_size,
+ };
+ return 2;
+}
+
/*
* write_mft_record_nolock - write out a mapped (extent) mft record
* @ni: ntfs inode describing the mapped (extent) mft record
@@ -541,25 +671,35 @@ int ntfs_sync_mft_mirror(struct ntfs_volume *vol, const u64 mft_no,
*
* We only write the mft record if the ntfs inode @ni is dirty.
*
- * On success, clean the mft record and return 0.
- * On error (specifically ENOMEM), we redirty the record so it can be retried.
- * For other errors, we mark the volume with errors.
+ * On success, clean the mft record and return 0. On ENOMEM, redirty the
+ * record so it can be retried. Asynchronous callers return success after
+ * redirtying while synchronous callers receive the error. For other errors,
+ * mark the volume with errors.
+ *
+ * If @sync is false, PG_writeback keeps the folio stable and serializes later
+ * writers until the I/O completes.
*/
int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int sync)
{
struct ntfs_volume *vol = ni->vol;
struct folio *folio = ni->folio;
- int err = 0, i = 0;
+ DECLARE_COMPLETION_ONSTACK(done);
+ struct ntfs_mft_io_unit units[2];
+ struct ntfs_mft_write_ctx *ctx;
+ struct bio *parent, *child = NULL;
+ unsigned int nr_units;
+ int err = 0;
u8 *kaddr;
struct mft_record *fixup_m;
- struct bio *bio;
- unsigned int offset = 0, folio_size;
ntfs_debug("Entering for inode 0x%llx.", ni->mft_no);
WARN_ON(NInoAttr(ni));
WARN_ON(!folio_test_locked(folio));
+ if (folio_test_writeback(folio))
+ folio_wait_writeback(folio);
+
/*
* If the struct ntfs_inode is clean no need to do anything. If it is dirty,
* mark it as clean now so that it can be redirtied later on if needed.
@@ -569,6 +709,11 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
if (!NInoTestClearDirty(ni))
goto done;
+ err = ntfs_prepare_mft_record_io_units(ni, units);
+ if (err < 0)
+ goto err_out;
+ nr_units = err;
+
kaddr = kmap_local_folio(folio, 0);
fixup_m = (struct mft_record *)(kaddr + ni->folio_ofs);
memcpy(fixup_m, m, vol->mft_record_size);
@@ -580,68 +725,74 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
goto unmap_err_out;
}
- folio_size = vol->mft_record_size / ni->mft_lcn_count;
- while (i < ni->mft_lcn_count) {
- unsigned int clu_off;
-
- clu_off = (unsigned int)((s64)ni->mft_no * vol->mft_record_size + offset) &
- vol->cluster_size_mask;
-
- bio = bio_alloc(vol->sb->s_bdev, 1, REQ_OP_WRITE, GFP_NOIO);
- bio->bi_iter.bi_sector =
- ntfs_bytes_to_bio_sector(NTFS_CLU_TO_B(vol, ni->mft_lcn[i]) +
- clu_off);
+ parent = ntfs_alloc_mft_parent_bio(vol, folio, sync ? &done : NULL);
+ if (!parent) {
+ err = -ENOMEM;
+ goto unmap_err_out;
+ }
+ ctx = container_of(parent, struct ntfs_mft_write_ctx, bio);
- if (!bio_add_folio(bio, folio, folio_size,
- ni->folio_ofs + offset)) {
- err = -EIO;
- goto put_bio_out;
+ parent->bi_iter.bi_sector = units[0].sector;
+ if (!bio_add_folio(parent, folio, units[0].len, units[0].folio_ofs)) {
+ err = -EIO;
+ goto put_bios_out;
+ }
+
+ if (nr_units == 2) {
+ if (bio_end_sector(parent) != units[1].sector ||
+ !bio_add_folio(parent, folio, units[1].len,
+ units[1].folio_ofs)) {
+ child = bio_alloc(vol->sb->s_bdev, 1, REQ_OP_WRITE,
+ GFP_NOIO);
+ if (!child) {
+ err = -ENOMEM;
+ goto put_bios_out;
+ }
+ child->bi_iter.bi_sector = units[1].sector;
+ if (!bio_add_folio(child, folio, units[1].len,
+ units[1].folio_ofs)) {
+ err = -EIO;
+ goto put_bios_out;
+ }
}
+ }
- /* Synchronize the mft mirror now if not @sync. */
- if (!sync && ni->mft_no < vol->mftmirr_size) {
- int sub_err = ntfs_sync_mft_mirror(vol, ni->mft_no,
- fixup_m);
- if (unlikely(sub_err) && !err)
- err = sub_err;
- }
+ if (ni->mft_no < vol->mftmirr_size) {
+ err = ntfs_sync_mft_mirror_record(vol, folio, ni->mft_no);
+ if (err)
+ ctx->error = err;
+ }
- if (sync) {
- int sub_err = submit_bio_wait(bio);
+ kunmap_local(kaddr);
- bio_put(bio);
- if (unlikely(sub_err) && !err)
- err = sub_err;
- } else {
- folio_get(folio);
- bio->bi_private = folio;
- bio->bi_end_io = ntfs_bio_end_io;
- submit_bio(bio);
- }
- offset += vol->cluster_size;
- i++;
+ ntfs_start_mft_writeback(ctx);
+ if (child) {
+ bio_chain(child, parent);
+ submit_bio(child);
}
+ submit_bio(parent);
- /* If @sync, now synchronize the mft mirror. */
- if (sync && ni->mft_no < vol->mftmirr_size) {
- int sub_err = ntfs_sync_mft_mirror(vol, ni->mft_no, fixup_m);
-
- if (unlikely(sub_err) && !err)
- err = sub_err;
+ if (sync) {
+ wait_for_completion(&done);
+ if (!err && parent->bi_status)
+ err = blk_status_to_errno(parent->bi_status);
+ bio_put(parent);
}
- kunmap_local(kaddr);
+
if (unlikely(err)) {
/* I/O error during writing. This is really bad! */
ntfs_error(vol->sb,
"I/O error while writing mft record 0x%llx! Marking base inode as bad. You should unmount the volume and run chkdsk.",
ni->mft_no);
- goto err_out;
+ return err;
}
done:
ntfs_debug("Done.");
return 0;
-put_bio_out:
- bio_put(bio);
+put_bios_out:
+ if (child)
+ bio_put(child);
+ bio_put(parent);
unmap_err_out:
kunmap_local(kaddr);
err_out:
@@ -655,7 +806,8 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
ntfs_error(vol->sb,
"Not enough memory to write mft record. Redirtying so the write is retried later.");
mark_mft_record_dirty(ni);
- err = 0;
+ if (!sync)
+ err = 0;
} else
NVolSetErrors(vol);
return err;
@@ -714,7 +866,8 @@ static int ntfs_test_inode_wb(struct inode *vi, u64 ino, void *data)
* it in @ref_vi instead of calling iput() directly. The caller must call
* iput() on @ref_vi after releasing the folio lock.
*
- * Return 'true' if the mft record may be written out and 'false' if not.
+ * Return 'true' if the mft record may be written out by this path and 'false'
+ * if direct inode writeback owns it.
*
* The caller has locked the page and cleared the uptodate flag on it which
* means that we can safely write out any dirty mft records that do not have
@@ -811,9 +964,10 @@ static bool ntfs_may_write_mft_record(struct ntfs_volume *vol, const u64 mft_no,
mft_no);
/*
* The write has to occur while we hold the mft record lock so
- * return the locked ntfs inode.
+ * return the locked ntfs inode and its vfs inode reference.
*/
*locked_ni = ni;
+ *ref_vi = vi;
return true;
}
ntfs_debug("Inode 0x%llx is not in icache.", mft_no);
@@ -1617,7 +1771,7 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
s64 min_nr, nr, ll;
unsigned long flags;
struct ntfs_inode *mft_ni;
- struct runlist_element *rl, *rl2;
+ struct runlist_element *rl, *rl2, *alloc_rl;
struct ntfs_attr_search_ctx *ctx = NULL;
struct mft_record *mrec;
struct attr_record *a = NULL;
@@ -1685,15 +1839,15 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
up_write(&mft_ni->runlist.lock);
do {
- rl2 = ntfs_cluster_alloc(vol, old_last_vcn, nr, lcn, MFT_ZONE,
- true, false, false);
- if (!IS_ERR(rl2))
+ alloc_rl = ntfs_cluster_alloc(vol, old_last_vcn, nr, lcn,
+ MFT_ZONE, true, false, false);
+ if (!IS_ERR(alloc_rl))
break;
- if (PTR_ERR(rl2) != -ENOSPC || nr == min_nr) {
+ if (PTR_ERR(alloc_rl) != -ENOSPC || nr == min_nr) {
ntfs_error(vol->sb,
"Failed to allocate the minimal number of clusters (%lli) for the mft data attribute.",
nr);
- return PTR_ERR(rl2);
+ return PTR_ERR(alloc_rl);
}
/*
* There is not enough space to do the allocation, but there
@@ -1704,17 +1858,22 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
ntfs_debug("Retrying mft data allocation with minimal cluster count %lli.", nr);
} while (1);
+ /*
+ * Keep @alloc_rl until the extension is complete: it describes exactly
+ * the new clusters, which undo_alloc frees if a later step fails.
+ */
down_write(&mft_ni->runlist.lock);
- rl = ntfs_runlists_merge(&mft_ni->runlist, rl2, 0, &new_rl_count);
+ rl = ntfs_runlists_merge_keep_src(&mft_ni->runlist, alloc_rl, 0,
+ &new_rl_count);
if (IS_ERR(rl)) {
up_write(&mft_ni->runlist.lock);
ntfs_error(vol->sb, "Failed to merge runlists for mft data attribute.");
- if (ntfs_cluster_free_from_rl(vol, rl2)) {
+ if (ntfs_cluster_free_from_rl(vol, alloc_rl)) {
ntfs_error(vol->sb,
"Failed to deallocate clusters from the mft data attribute.%s", es);
NVolSetErrors(vol);
}
- kvfree(rl2);
+ kvfree(alloc_rl);
return PTR_ERR(rl);
}
mft_ni->runlist.rl = rl;
@@ -1730,7 +1889,6 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
if (IS_ERR(mrec)) {
ntfs_error(vol->sb, "Failed to map mft record.");
ret = PTR_ERR(mrec);
- down_write(&mft_ni->runlist.lock);
goto undo_alloc;
}
ctx = ntfs_attr_get_search_ctx(mft_ni, mrec);
@@ -1828,6 +1986,7 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
mark_mft_record_dirty(ctx->ntfs_ino);
ntfs_attr_put_search_ctx(ctx);
unmap_mft_record(mft_ni);
+ kvfree(alloc_rl);
ntfs_debug("Done.");
return 0;
restore_undo_alloc:
@@ -1841,7 +2000,7 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
write_unlock_irqrestore(&mft_ni->size_lock, flags);
ntfs_attr_put_search_ctx(ctx);
unmap_mft_record(mft_ni);
- up_write(&mft_ni->runlist.lock);
+ kvfree(alloc_rl);
/*
* The only thing that is now wrong is ->allocated_size of the
* base attribute extent which chkdsk should be able to fix.
@@ -1852,15 +2011,28 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
ctx->attr->data.non_resident.highest_vcn =
cpu_to_le64(old_last_vcn - 1);
undo_alloc:
- if (ntfs_cluster_free(mft_ni, old_last_vcn, -1, ctx) < 0) {
- ntfs_error(vol->sb, "Failed to free clusters from mft data attribute.%s", es);
- NVolSetErrors(vol);
- }
-
+ /*
+ * Entered without the runlist lock. Take the new runs off the
+ * runlist under it, and free their clusters from @alloc_rl once it
+ * is dropped, as lcnbmp_lock nests outside it (see above).
+ */
+ down_write(&mft_ni->runlist.lock);
if (ntfs_rl_truncate_nolock(vol, &mft_ni->runlist, old_last_vcn)) {
ntfs_error(vol->sb, "Failed to truncate mft data attribute runlist.%s", es);
NVolSetErrors(vol);
+ /*
+ * The runlist may still reference the clusters, so leave them
+ * allocated until chkdsk.
+ */
+ kvfree(alloc_rl);
+ alloc_rl = NULL;
+ }
+ up_write(&mft_ni->runlist.lock);
+ if (ntfs_cluster_free_from_rl(vol, alloc_rl)) {
+ ntfs_error(vol->sb, "Failed to free clusters from mft data attribute.%s", es);
+ NVolSetErrors(vol);
}
+ kvfree(alloc_rl);
if (mp_extended && ntfs_attr_update_mapping_pairs(mft_ni, 0)) {
ntfs_error(vol->sb, "Failed to restore mapping pairs.%s",
es);
@@ -1869,11 +2041,16 @@ static int ntfs_mft_data_extend_allocation_nolock(struct ntfs_volume *vol)
if (ctx) {
a = ctx->attr;
if (mp_rebuilt && !IS_ERR(ctx->mrec)) {
- if (ntfs_mapping_pairs_build(vol, (u8 *)a + le16_to_cpu(
+ int err;
+
+ down_read(&mft_ni->runlist.lock);
+ err = ntfs_mapping_pairs_build(vol, (u8 *)a + le16_to_cpu(
a->data.non_resident.mapping_pairs_offset),
old_alen - le16_to_cpu(
a->data.non_resident.mapping_pairs_offset),
- rl2, ll, -1, NULL, NULL, NULL)) {
+ mft_ni->runlist.rl, ll, -1, NULL, NULL, NULL);
+ up_read(&mft_ni->runlist.lock);
+ if (err) {
ntfs_error(vol->sb, "Failed to restore mapping pairs array.%s", es);
NVolSetErrors(vol);
}
@@ -2017,6 +2194,8 @@ static int ntfs_mft_record_format(const struct ntfs_volume *vol, const s64 mft_n
return PTR_ERR(folio);
}
folio_lock(folio);
+ if (folio_test_writeback(folio))
+ folio_wait_writeback(folio);
folio_clear_uptodate(folio);
m = (struct mft_record *)((u8 *)kmap_local_folio(folio, 0) + ofs);
err = ntfs_mft_record_layout(vol, mft_no, m);
@@ -2534,6 +2713,8 @@ int ntfs_mft_record_alloc(struct ntfs_volume *vol, const int mode,
goto undo_mftbmp_alloc;
}
folio_lock(folio);
+ if (folio_test_writeback(folio))
+ folio_wait_writeback(folio);
folio_clear_uptodate(folio);
m = (struct mft_record *)((u8 *)kmap_local_folio(folio, 0) + ofs);
/* If we just formatted the mft record no need to do it again. */
@@ -2804,7 +2985,7 @@ int ntfs_mft_record_free(struct ntfs_volume *vol, struct ntfs_inode *ni)
* record to be freed is guaranteed to do it already.
*/
NInoSetDirty(ni);
- err = write_mft_record(ni, ni_mrec, 0);
+ err = write_mft_record(ni, ni_mrec, 1);
if (err)
goto sync_rollback;
@@ -2854,19 +3035,150 @@ int ntfs_mft_record_free(struct ntfs_volume *vol, struct ntfs_inode *ni)
return err;
}
-static s64 lcn_from_index(struct ntfs_volume *vol, struct ntfs_inode *ni,
- unsigned long index)
+static bool ntfs_add_mft_io_unit(struct bio *bio, struct folio *folio,
+ const struct ntfs_mft_io_unit *unit)
{
- s64 vcn;
- s64 lcn;
+ if (!bio->bi_iter.bi_size)
+ bio->bi_iter.bi_sector = unit->sector;
+ else if (bio_end_sector(bio) != unit->sector)
+ return false;
+
+ return bio_add_folio(bio, folio, unit->len, unit->folio_ofs);
+}
+
+static void ntfs_release_mft_write_refs(struct ntfs_inode **locked_nis,
+ unsigned int nr_locked_nis,
+ struct inode **ref_inos,
+ unsigned int nr_ref_inos)
+{
+ while (nr_locked_nis-- > 0) {
+ struct ntfs_inode *tni = locked_nis[nr_locked_nis];
+
+ mutex_unlock(&tni->mrec_lock);
+ atomic_dec(&tni->count);
+ }
+
+ while (nr_ref_inos-- > 0)
+ iput(ref_inos[nr_ref_inos]);
+}
- vcn = ntfs_pidx_to_cluster(vol, index);
+static int ntfs_map_mft_io_locked(struct ntfs_inode *ni, u64 folio_byte,
+ struct ntfs_mft_io_unit *unit)
+{
+ struct ntfs_volume *vol = ni->vol;
+ u64 file_ofs = folio_byte + unit->folio_ofs;
+ s64 vcn, lcn;
+
+ lockdep_assert_held(&ni->runlist.lock);
+ if (!ni->runlist.rl)
+ return -EIO;
+
+ vcn = ntfs_bytes_to_cluster(vol, file_ofs);
+ /*
+ * $MFT is fully mapped at mount time and extensions merge already
+ * mapped runs under the write lock. Do not remap here: the generic
+ * helper may drop this read lock while upgrading it.
+ */
+ lcn = ntfs_rl_vcn_to_lcn(ni->runlist.rl, vcn);
+ if (lcn == LCN_RL_NOT_MAPPED)
+ return -EAGAIN;
+ if (lcn < 0)
+ return -EIO;
+ if (ntfs_bytes_to_cluster_off(vol, file_ofs) + unit->len >
+ vol->cluster_size)
+ return -EIO;
+
+ unit->sector = ntfs_bytes_to_bio_sector(
+ NTFS_CLU_TO_B(vol, lcn) +
+ ntfs_bytes_to_cluster_off(vol, file_ofs));
+ return 0;
+}
+
+static int ntfs_map_mft_io_for_folio(struct ntfs_inode *ni, u64 folio_byte,
+ struct ntfs_mft_io_unit *unit)
+{
+ int err;
- down_read(&ni->runlist.lock);
- lcn = ntfs_attr_vcn_to_lcn_nolock(ni, vcn, false);
+ if (!down_read_trylock(&ni->runlist.lock))
+ return -EAGAIN;
+ err = ntfs_map_mft_io_locked(ni, folio_byte, unit);
up_read(&ni->runlist.lock);
+ return err;
+}
- return lcn;
+static int ntfs_prepare_mft_folio_units(struct ntfs_inode *ni, u64 folio_byte,
+ u64 file_limit,
+ const unsigned long *record_selected,
+ struct ntfs_mft_io_unit *units,
+ unsigned int *nr_units,
+ unsigned int max_units, bool *defer)
+{
+ struct ntfs_volume *vol = ni->vol;
+ u64 folio_end = folio_byte + PAGE_SIZE;
+ u64 unit_byte = folio_byte;
+
+ while (unit_byte < folio_end && unit_byte < file_limit) {
+ struct ntfs_mft_io_unit unit;
+ u64 record_byte;
+ u64 unit_end;
+ u64 io_unit_end;
+ u64 cluster_end;
+ bool selected = false;
+ int err;
+
+ io_unit_end =
+ round_down(unit_byte, (u64)vol->mft_io_unit_size) +
+ vol->mft_io_unit_size;
+ cluster_end = ntfs_cluster_to_bytes(
+ vol, ntfs_bytes_to_cluster(vol, unit_byte) + 1);
+ unit_end = min3(io_unit_end, cluster_end, folio_end);
+
+ record_byte = round_down(unit_byte, (u64)vol->mft_record_size);
+ while (record_byte < min(unit_end, file_limit)) {
+ unsigned int record;
+
+ record = div_u64(record_byte - folio_byte,
+ vol->mft_record_size);
+ if (test_bit(record, record_selected)) {
+ selected = true;
+ break;
+ }
+ record_byte += vol->mft_record_size;
+ }
+ /*
+ * Direct inode writeback owns unselected records. Their folio
+ * images are stable while this folio is locked, so a containing
+ * unit can preserve them when writing a selected record.
+ */
+ if (!selected)
+ goto next;
+
+ unit = (struct ntfs_mft_io_unit){
+ .folio_ofs = unit_byte - folio_byte,
+ .len = unit_end - unit_byte,
+ };
+ err = ntfs_map_mft_io_for_folio(ni, folio_byte, &unit);
+ if (err == -EAGAIN) {
+ *defer = true;
+ goto next;
+ }
+ if (err)
+ return err;
+ if (*nr_units >= max_units)
+ return -EOVERFLOW;
+ units[(*nr_units)++] = unit;
+next:
+ unit_byte = unit_end;
+ }
+ return 0;
+}
+
+static void ntfs_mft_write_error(struct ntfs_volume *vol,
+ struct address_space *mapping, int err)
+{
+ mapping_set_error(mapping, err);
+ NVolSetErrors(vol);
+ ntfs_error(vol->sb, "Error while writing MFT folio: %d", err);
}
/*
@@ -2875,9 +3187,8 @@ static s64 lcn_from_index(struct ntfs_volume *vol, struct ntfs_inode *ni,
* @wbc: Writeback control structure
*
* This function is called as part of the address_space_operations
- * .writepages implementation for the $MFT inode (or $MFTMirr).
- * It handles writing one folio (normally 4KiB page) worth of MFT records
- * to the underlying block device.
+ * .writepages implementation for the $MFT inode. Despite its historical
+ * name, it handles one folio containing MFT records.
*
* Return: 0 on success, or -errno on error.
*/
@@ -2887,195 +3198,190 @@ static int ntfs_write_mft_block(struct folio *folio, struct writeback_control *w
struct inode *vi = mapping->host;
struct ntfs_inode *ni = NTFS_I(vi);
struct ntfs_volume *vol = ni->vol;
- u8 *kaddr;
- struct ntfs_inode **locked_nis __free(kfree) = kmalloc_objs(struct ntfs_inode *,
- PAGE_SIZE / NTFS_BLOCK_SIZE,
- GFP_NOFS);
- int nr_locked_nis = 0, err = 0, mft_ofs, prev_mft_ofs;
- struct inode **ref_inos __free(kfree) = kmalloc_objs(struct inode *,
- PAGE_SIZE / NTFS_BLOCK_SIZE,
- GFP_NOFS);
- int nr_ref_inos = 0;
- struct bio *bio = NULL;
- u64 mft_no;
- struct ntfs_inode *tni;
- s64 lcn;
- s64 vcn = ntfs_pidx_to_cluster(vol, folio->index);
- s64 end_vcn = ntfs_bytes_to_cluster(vol, ni->allocated_size);
- unsigned int folio_sz;
- loff_t i_size = i_size_read(vi);
+ struct ntfs_inode **locked_nis __free(kfree) =
+ kmalloc_objs(struct ntfs_inode *, PAGE_SIZE / NTFS_BLOCK_SIZE,
+ GFP_NOFS);
+ struct inode **ref_inos __free(kfree) =
+ kmalloc_objs(struct inode *, PAGE_SIZE / NTFS_BLOCK_SIZE, GFP_NOFS);
+ struct ntfs_mft_io_unit *units __free(kfree) = NULL;
+ struct bio *parent = NULL, *child = NULL;
+ struct ntfs_mft_write_ctx *ctx = NULL;
+ u8 *kaddr = NULL;
+ DECLARE_BITMAP(record_selected, PAGE_SIZE / NTFS_BLOCK_SIZE) = {};
+ u64 folio_byte, file_limit, folio_end, mirror_size;
+ unsigned int nr_records, nr_units = 0, max_units;
+ unsigned int nr_locked_nis = 0, nr_ref_inos = 0;
+ unsigned int record, unit_idx;
+ unsigned long flags;
+ loff_t i_size;
+ s64 allocated_size;
+ bool defer = false, redirty = false;
+ int err = 0;
- ntfs_debug("Entering for inode 0x%llx, attribute type 0x%x, folio index 0x%lx.",
- ni->mft_no, ni->type, folio->index);
+ read_lock_irqsave(&ni->size_lock, flags);
+ i_size = i_size_read(vi);
+ allocated_size = ni->allocated_size;
+ read_unlock_irqrestore(&ni->size_lock, flags);
- if (!locked_nis || !ref_inos) {
- folio_redirty_for_writepage(wbc, folio);
- folio_unlock(folio);
- return -ENOMEM;
+ ntfs_debug("Entering for inode 0x%llx, folio index 0x%lx.", ni->mft_no,
+ folio->index);
+ WARN_ON(!folio_test_locked(folio));
+
+ folio_byte = folio_pos(folio);
+ folio_end = folio_byte + PAGE_SIZE;
+ mirror_size = (u64)vol->mftmirr_size * vol->mft_record_size;
+
+ if (i_size <= folio_byte)
+ folio_zero_segment(folio, 0, PAGE_SIZE);
+ else if ((u64)i_size < folio_end)
+ folio_zero_segment(folio, i_size - folio_byte, PAGE_SIZE);
+
+ file_limit = min_t(u64, i_size, allocated_size);
+ nr_records = PAGE_SIZE / vol->mft_record_size;
+ max_units = PAGE_SIZE / min(vol->cluster_size, vol->mft_record_size);
+ units = kmalloc_array(max_units, sizeof(*units), GFP_NOFS);
+ if (!locked_nis || !ref_inos || !units) {
+ err = -ENOMEM;
+ goto out_noio;
}
- /* We have to zero every time due to mmap-at-end-of-file. */
- if (folio->index >= (i_size >> folio_shift(folio)))
- /* The page straddles i_size. */
- folio_zero_segment(folio,
- offset_in_folio(folio, i_size),
- folio_size(folio));
+ kaddr = kmap_local_folio(folio, 0);
+ folio_clear_uptodate(folio);
- lcn = lcn_from_index(vol, ni, folio->index);
- if (lcn <= LCN_HOLE) {
+ for (record = 0; record < nr_records; record++) {
+ struct ntfs_inode *tni = NULL;
+ struct inode *ref_vi = NULL;
+ u64 record_byte =
+ folio_byte + (u64)record * vol->mft_record_size;
+ u64 mft_no;
+
+ if (record_byte >= file_limit)
+ continue;
+ mft_no = record_byte >> vol->mft_record_size_bits;
+ if (!ntfs_may_write_mft_record(
+ vol, mft_no,
+ (struct mft_record *)(kaddr +
+ record * vol->mft_record_size),
+ &tni, &ref_vi)) {
+ if (ref_vi)
+ ref_inos[nr_ref_inos++] = ref_vi;
+ continue;
+ }
+ if (ref_vi)
+ ref_inos[nr_ref_inos++] = ref_vi;
+ if (tni) {
+ locked_nis[nr_locked_nis++] = tni;
+ if (tni->nr_extents < 0 &&
+ tni->ext.base_ntfs_ino == NTFS_I(vol->mft_ino))
+ continue;
+ }
+ __set_bit(record, record_selected);
+ }
+
+ err = ntfs_prepare_mft_folio_units(ni, folio_byte, file_limit,
+ record_selected, units, &nr_units,
+ max_units, &defer);
+ if (err)
+ goto out_noio;
+ if (!nr_units) {
+ if (defer)
+ goto out_noio;
+ folio_mark_uptodate(folio);
+ kunmap_local(kaddr);
folio_start_writeback(folio);
folio_unlock(folio);
folio_end_writeback(folio);
- return -EIO;
+ ntfs_release_mft_write_refs(locked_nis, nr_locked_nis, ref_inos,
+ nr_ref_inos);
+ return 0;
}
- /* Map folio so we can access its contents. */
- kaddr = kmap_local_folio(folio, 0);
- /* Clear the page uptodate flag whilst the mst fixups are applied. */
- folio_clear_uptodate(folio);
+ parent = ntfs_alloc_mft_parent_bio(vol, folio, NULL);
+ if (!parent) {
+ err = -ENOMEM;
+ goto out_noio;
+ }
+ ctx = container_of(parent, struct ntfs_mft_write_ctx, bio);
- for (mft_ofs = 0; mft_ofs < PAGE_SIZE && vcn < end_vcn;
- mft_ofs += vol->mft_record_size) {
- /* Get the mft record number. */
- mft_no = (((s64)folio->index << PAGE_SHIFT) + mft_ofs) >>
- vol->mft_record_size_bits;
- vcn = ntfs_mft_no_to_cluster(vol, mft_no);
- /* Check whether to write this mft record. */
- tni = NULL;
- if (ntfs_may_write_mft_record(vol, mft_no,
- (struct mft_record *)(kaddr + mft_ofs),
- &tni, &ref_inos[nr_ref_inos])) {
- unsigned int mft_record_off = 0;
- s64 vcn_off = vcn;
- s64 rl_len = 0;
+ if (!ntfs_add_mft_io_unit(parent, folio, &units[0])) {
+ err = -EIO;
+ goto out_noio;
+ }
- /*
- * The record should be written. If a locked ntfs
- * inode was returned, add it to the array of locked
- * ntfs inodes.
- */
- if (tni)
- locked_nis[nr_locked_nis++] = tni;
- else if (ref_inos[nr_ref_inos])
- nr_ref_inos++;
-
- if (bio && (mft_ofs != prev_mft_ofs + vol->mft_record_size)) {
-flush_bio:
- bio->bi_end_io = ntfs_bio_end_io;
- submit_bio(bio);
- bio = NULL;
- }
+ for (unit_idx = 0; unit_idx < nr_units; unit_idx++) {
+ struct ntfs_mft_io_unit *unit = &units[unit_idx];
+ u64 file_ofs = folio_byte + unit->folio_ofs;
- if (vol->cluster_size < folio_size(folio)) {
- struct runlist_element *rl;
+ if (unit_idx) {
+ struct bio *target = child ? child : parent;
- down_write(&ni->runlist.lock);
- rl = ntfs_attr_vcn_to_rl(ni, vcn_off, &lcn);
- if (!IS_ERR(rl))
- rl_len = rl->length - (vcn_off - rl->vcn);
- up_write(&ni->runlist.lock);
- if (IS_ERR(rl) || lcn < 0) {
- err = -EIO;
- goto unm_done;
+ if (!ntfs_add_mft_io_unit(target, folio, unit)) {
+ if (child) {
+ ntfs_start_mft_writeback(ctx);
+ bio_chain(child, parent);
+ submit_bio(child);
+ child = NULL;
}
-
- if (bio &&
- (bio_end_sector(bio) >> (vol->cluster_size_bits - 9)) !=
- lcn) {
- bio->bi_end_io = ntfs_bio_end_io;
- submit_bio(bio);
- bio = NULL;
+ child = bio_alloc(vol->sb->s_bdev, 1,
+ REQ_OP_WRITE, GFP_NOIO);
+ if (!child) {
+ defer = true;
+ break;
+ }
+ if (!ntfs_add_mft_io_unit(child, folio, unit)) {
+ bio_put(child);
+ child = NULL;
+ if (!ctx->error)
+ ctx->error = -EIO;
+ redirty = true;
+ err = -EIO;
+ break;
}
}
+ }
+ if (file_ofs < mirror_size) {
+ int mirror_err = ntfs_sync_mft_mirror_unit(
+ vol, folio, file_ofs, unit);
- if (!bio) {
- unsigned int off;
-
- off = ((mft_no << vol->mft_record_size_bits) +
- mft_record_off) & vol->cluster_size_mask;
-
- bio = bio_alloc(vol->sb->s_bdev, 1, REQ_OP_WRITE,
- GFP_NOIO);
- bio->bi_iter.bi_sector =
- ntfs_bytes_to_bio_sector(
- ntfs_cluster_to_bytes(vol, lcn) + off);
- }
-
- if (vol->cluster_size == NTFS_BLOCK_SIZE &&
- (mft_record_off ||
- rl_len == 1 ||
- mft_ofs + NTFS_BLOCK_SIZE >= PAGE_SIZE))
- folio_sz = NTFS_BLOCK_SIZE;
- else
- folio_sz = vol->mft_record_size;
- if (!bio_add_folio(bio, folio, folio_sz,
- mft_ofs + mft_record_off)) {
- err = -EIO;
- bio_put(bio);
- goto unm_done;
- }
- mft_record_off += folio_sz;
-
- if (mft_record_off != vol->mft_record_size) {
- vcn_off++;
- goto flush_bio;
- }
- prev_mft_ofs = mft_ofs;
-
- if (mft_no < vol->mftmirr_size) {
- int sub_err = ntfs_sync_mft_mirror(vol, mft_no,
- (struct mft_record *)(kaddr + mft_ofs));
-
- if (unlikely(sub_err) && !err)
- err = sub_err;
- }
- } else if (ref_inos[nr_ref_inos])
- nr_ref_inos++;
+ if (mirror_err && !ctx->error)
+ ctx->error = mirror_err;
+ }
}
- if (bio) {
- bio->bi_end_io = ntfs_bio_end_io;
- submit_bio(bio);
+ if (child) {
+ ntfs_start_mft_writeback(ctx);
+ bio_chain(child, parent);
+ submit_bio(child);
}
-unm_done:
+ ntfs_start_mft_writeback(ctx);
folio_mark_uptodate(folio);
+ if (defer || redirty)
+ folio_redirty_for_writepage(wbc, folio);
kunmap_local(kaddr);
-
- folio_start_writeback(folio);
+ kaddr = NULL;
folio_unlock(folio);
- folio_end_writeback(folio);
+ submit_bio(parent);
+ ntfs_release_mft_write_refs(locked_nis, nr_locked_nis, ref_inos,
+ nr_ref_inos);
- /* Unlock any locked inodes. */
- while (nr_locked_nis-- > 0) {
- struct ntfs_inode *base_tni;
-
- tni = locked_nis[nr_locked_nis];
- mutex_unlock(&tni->mrec_lock);
-
- /* Get the base inode. */
- mutex_lock(&tni->extent_lock);
- if (tni->nr_extents >= 0)
- base_tni = tni;
- else
- base_tni = tni->ext.base_ntfs_ino;
- mutex_unlock(&tni->extent_lock);
- ntfs_debug("Unlocking %s inode 0x%llx.",
- tni == base_tni ? "base" : "extent",
- tni->mft_no);
- atomic_dec(&tni->count);
- iput(VFS_I(base_tni));
- }
+ return 0;
- /* Dropping deferred references */
- while (nr_ref_inos-- > 0) {
- if (ref_inos[nr_ref_inos])
- iput(ref_inos[nr_ref_inos]);
+out_noio:
+ if (kaddr) {
+ folio_mark_uptodate(folio);
+ kunmap_local(kaddr);
}
-
- if (unlikely(err && err != -ENOMEM))
- NVolSetErrors(vol);
- if (likely(!err))
- ntfs_debug("Done.");
+ if (err && err != -ENOMEM)
+ ntfs_mft_write_error(vol, mapping, err);
+ if (parent)
+ bio_put(parent);
+ if (redirty || defer || err == -ENOMEM)
+ folio_redirty_for_writepage(wbc, folio);
+ folio_unlock(folio);
+ ntfs_release_mft_write_refs(locked_nis, nr_locked_nis, ref_inos,
+ nr_ref_inos);
+ if (err == -ENOMEM || (defer && !err))
+ return 0;
return err;
}
diff --git a/fs/ntfs/mft.h b/fs/ntfs/mft.h
index ed5c1d595c0d..d2a31205e08c 100644
--- a/fs/ntfs/mft.h
+++ b/fs/ntfs/mft.h
@@ -42,8 +42,8 @@ static inline void mark_mft_record_dirty(struct ntfs_inode *ni)
__mark_mft_record_dirty(ni);
}
-int ntfs_sync_mft_mirror(struct ntfs_volume *vol, const u64 mft_no,
- struct mft_record *m);
+int ntfs_mft_bioset_init(void);
+void ntfs_mft_bioset_exit(void);
int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int sync);
/*
@@ -53,16 +53,19 @@ int write_mft_record_nolock(struct ntfs_inode *ni, struct mft_record *m, int syn
* @sync: if true, wait for i/o completion
*
* This is just a wrapper for write_mft_record_nolock() (see mft.c), which
- * locks the page for the duration of the write. This ensures that there are
- * no race conditions between writing the mft record via the dirty inode code
- * paths and via the page cache write back code paths or between writing
- * neighbouring mft records residing in the same page.
+ * locks the folio while preparing the write. write_mft_record_nolock() waits
+ * for prior folio writeback before modifying the folio and keeps PG_writeback
+ * set until the submitted I/O completes. Together these serialize dirty
+ * inode writes, page cache writeback, and neighbouring mft record writes in
+ * the same folio.
*
* Locking the page also serializes us against ->read_folio() if the page is not
* uptodate.
*
- * On success, clean the mft record and return 0. On error, leave the mft
- * record dirty and return -errno.
+ * On success, clean the mft record and return 0. On allocation failure,
+ * redirty the record for retry. Asynchronous callers return 0 after
+ * redirtying, while synchronous callers receive -ENOMEM. On other errors,
+ * return -errno and mark the volume with errors.
*/
static inline int write_mft_record(struct ntfs_inode *ni, struct mft_record *m, int sync)
{
diff --git a/fs/ntfs/namei.c b/fs/ntfs/namei.c
index fdf52fac4329..3e0adb9a0ea4 100644
--- a/fs/ntfs/namei.c
+++ b/fs/ntfs/namei.c
@@ -757,8 +757,7 @@ static int ntfs_create(struct mnt_idmap *idmap, struct inode *dir,
return err;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
ni = __ntfs_create(idmap, dir, uname, uname_len, S_IFREG | mode, 0, NULL, 0);
kmem_cache_free(ntfs_name_cache, uname);
@@ -1032,8 +1031,7 @@ static int ntfs_unlink(struct inode *dir, struct dentry *dentry)
return err;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
err = ntfs_delete(ni, NTFS_I(dir), uname, uname_len, true);
if (err)
@@ -1076,8 +1074,7 @@ static struct dentry *ntfs_mkdir(struct mnt_idmap *idmap, struct inode *dir,
return ERR_PTR(err);
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
ni = __ntfs_create(idmap, dir, uname, uname_len, mode, 0, NULL, 0);
kmem_cache_free(ntfs_name_cache, uname);
@@ -1118,8 +1115,7 @@ static int ntfs_rmdir(struct inode *dir, struct dentry *dentry)
return err;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
err = ntfs_delete(ni, NTFS_I(dir), uname, uname_len, true);
if (err)
@@ -1305,8 +1301,7 @@ static int ntfs_rename(struct mnt_idmap *idmap, struct inode *old_dir,
new_dir_first = is_subdir(new_dentry->d_parent,
old_dentry->d_parent);
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
mutex_lock_nested(&old_ni->mrec_lock, NTFS_INODE_MUTEX_NORMAL);
if (new_ni)
@@ -1429,8 +1424,7 @@ static int ntfs_symlink(struct mnt_idmap *idmap, struct inode *dir,
goto out;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
ni = __ntfs_create(idmap, dir, usrc, usrc_len, S_IFLNK | 0777, 0,
symname, symlen);
@@ -1474,8 +1468,7 @@ static int ntfs_mknod(struct mnt_idmap *idmap, struct inode *dir,
return err;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
switch (mode & S_IFMT) {
case S_IFCHR:
@@ -1521,8 +1514,7 @@ static int ntfs_link(struct dentry *old_dentry, struct inode *dir,
return -ENOMEM;
}
- if (!(vol->vol_flags & VOLUME_IS_DIRTY))
- ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
+ ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY);
ihold(vi);
mutex_lock_nested(&ni->mrec_lock, NTFS_INODE_MUTEX_NORMAL);
diff --git a/fs/ntfs/ntfs.h b/fs/ntfs/ntfs.h
index 45f77848a9cf..cb69707c3f3d 100644
--- a/fs/ntfs/ntfs.h
+++ b/fs/ntfs/ntfs.h
@@ -80,6 +80,8 @@ enum {
NTFS_MAX_LABEL_LEN = 128,
};
+#define NTFS_MAX_ATTR_LIST_SIZE (256 * 1024)
+
enum {
CASE_SENSITIVE = 0,
IGNORE_CASE = 1,
@@ -219,7 +221,6 @@ struct option_t {
};
extern const struct option_t on_errors_arr[];
int ntfs_set_volume_flags(struct ntfs_volume *vol, __le16 flags);
-int ntfs_clear_volume_flags(struct ntfs_volume *vol, __le16 flags);
int ntfs_write_volume_label(struct ntfs_volume *vol, char *label);
/* From fs/ntfs/mst.c */
diff --git a/fs/ntfs/runlist.c b/fs/ntfs/runlist.c
index 3a61f19bcbee..89e13a452a84 100644
--- a/fs/ntfs/runlist.c
+++ b/fs/ntfs/runlist.c
@@ -47,7 +47,7 @@ static inline void ntfs_rl_mm(struct runlist_element *base, int dst, int src, in
* @srcbase.
*/
static inline void ntfs_rl_mc(struct runlist_element *dstbase, int dst,
- struct runlist_element *srcbase, int src, int size)
+ const struct runlist_element *srcbase, int src, int size)
{
if (likely(size > 0))
memcpy(dstbase + dst, srcbase + src, size * sizeof(*dstbase));
@@ -152,8 +152,8 @@ static inline struct runlist_element *ntfs_rl_realloc_nofail(struct runlist_elem
* Return: true Success, the runlists can be merged.
* false Failure, the runlists cannot be merged.
*/
-static inline bool ntfs_are_rl_mergeable(struct runlist_element *dst,
- struct runlist_element *src)
+static inline bool ntfs_are_rl_mergeable(const struct runlist_element *dst,
+ const struct runlist_element *src)
{
/* We can merge unmapped regions even if they are misaligned. */
if ((dst->lcn == LCN_RL_NOT_MAPPED) && (src->lcn == LCN_RL_NOT_MAPPED))
@@ -186,7 +186,8 @@ static inline bool ntfs_are_rl_mergeable(struct runlist_element *dst,
*
* It is up to the caller to serialize access to the runlists @dst and @src.
*/
-static inline void __ntfs_rl_merge(struct runlist_element *dst, struct runlist_element *src)
+static inline void __ntfs_rl_merge(struct runlist_element *dst,
+ const struct runlist_element *src)
{
dst->length += src->length;
}
@@ -214,10 +215,11 @@ static inline void __ntfs_rl_merge(struct runlist_element *dst, struct runlist_e
* On error, return -errno. Both runlists are left unmodified.
*/
static inline struct runlist_element *ntfs_rl_append(struct runlist_element *dst,
- int dsize, struct runlist_element *src, int ssize, int loc,
+ int dsize, const struct runlist_element *src, int ssize, int loc,
size_t *new_size)
{
bool right = false; /* Right end of @src needs merging. */
+ s64 right_length = 0; /* Length of the run merged at the right. */
int marker; /* End of the inserted runs. */
/* First, check if the right hand end needs merging. */
@@ -235,9 +237,9 @@ static inline struct runlist_element *ntfs_rl_append(struct runlist_element *dst
* original runlists.
*/
- /* First, merge the right hand end, if necessary. */
+ /* Remember the run that the right hand end absorbs, if any. */
if (right)
- __ntfs_rl_merge(src + ssize - 1, dst + loc + 1);
+ right_length = dst[loc + 1].length;
/* First run after the @src runs that have been inserted. */
marker = loc + ssize + 1;
@@ -246,6 +248,9 @@ static inline struct runlist_element *ntfs_rl_append(struct runlist_element *dst
ntfs_rl_mm(dst, marker, loc + 1 + right, dsize - (loc + 1 + right));
ntfs_rl_mc(dst, loc + 1, src, 0, ssize);
+ /* Merge the right hand end into the copy, leaving @src intact. */
+ dst[marker - 1].length += right_length;
+
/* Adjust the size of the preceding hole. */
dst[loc].length = dst[loc + 1].vcn - dst[loc].vcn;
@@ -279,7 +284,7 @@ static inline struct runlist_element *ntfs_rl_append(struct runlist_element *dst
* On error, return -errno. Both runlists are left unmodified.
*/
static inline struct runlist_element *ntfs_rl_insert(struct runlist_element *dst,
- int dsize, struct runlist_element *src, int ssize, int loc,
+ int dsize, const struct runlist_element *src, int ssize, int loc,
size_t *new_size)
{
bool left = false; /* Left end of @src needs merging. */
@@ -374,12 +379,13 @@ static inline struct runlist_element *ntfs_rl_insert(struct runlist_element *dst
* On error, return -errno. Both runlists are left unmodified.
*/
static inline struct runlist_element *ntfs_rl_replace(struct runlist_element *dst,
- int dsize, struct runlist_element *src, int ssize, int loc,
+ int dsize, const struct runlist_element *src, int ssize, int loc,
size_t *new_size)
{
int delta;
bool left = false; /* Left end of @src needs merging. */
bool right = false; /* Right end of @src needs merging. */
+ s64 right_length = 0; /* Length of the run merged at the right. */
int tail; /* Start of tail of @dst. */
int marker; /* End of the inserted runs. */
@@ -405,9 +411,12 @@ static inline struct runlist_element *ntfs_rl_replace(struct runlist_element *ds
* original runlists.
*/
- /* First, merge the left and right ends, if necessary. */
+ /*
+ * Remember the run that the right hand end absorbs, if any, then
+ * merge the left hand end.
+ */
if (right)
- __ntfs_rl_merge(src + ssize - 1, dst + loc + 1);
+ right_length = dst[loc + 1].length;
if (left)
__ntfs_rl_merge(dst + loc - 1, src);
/*
@@ -432,6 +441,13 @@ static inline struct runlist_element *ntfs_rl_replace(struct runlist_element *ds
ntfs_rl_mm(dst, marker, tail, dsize - tail);
ntfs_rl_mc(dst, loc, src, left, ssize - left);
+ /*
+ * Merge the right hand end into the last run of @src, now in @dst at
+ * @marker - 1 (or into @dst[@loc - 1] if that absorbed all of @src),
+ * leaving @src intact.
+ */
+ dst[marker - 1].length += right_length;
+
/* We may have changed the length of the file, so fix the end marker. */
if (dsize - tail > 0 && dst[marker].lcn == LCN_ENOENT)
dst[marker].vcn = dst[marker - 1].vcn + dst[marker - 1].length;
@@ -461,7 +477,7 @@ static inline struct runlist_element *ntfs_rl_replace(struct runlist_element *ds
* On error, return -errno. Both runlists are left unmodified.
*/
static inline struct runlist_element *ntfs_rl_split(struct runlist_element *dst, int dsize,
- struct runlist_element *src, int ssize, int loc,
+ const struct runlist_element *src, int ssize, int loc,
size_t *new_size)
{
/* Space required: @dst size + @src size + one new hole. */
@@ -488,11 +504,12 @@ static inline struct runlist_element *ntfs_rl_split(struct runlist_element *dst,
}
/*
- * ntfs_runlists_merge - merge two runlists into one
+ * __ntfs_runlists_merge - merge two runlists into one
* @d_runlist: destination runlist structure to merge into
* @srl: source runlist to merge from
* @s_rl_count: number of elements in @srl (0 to auto-detect)
* @new_rl_count: on success, set to the new combined runlist size
+ * @keep_srl: leave @srl to the caller instead of deallocating it
*
* First we sanity check the two runlists @srl and @drl to make sure that they
* are sensible and can be merged. The runlist @srl must be either after the
@@ -515,13 +532,14 @@ static inline struct runlist_element *ntfs_rl_split(struct runlist_element *dst,
* On success, return a pointer to the new, combined, runlist. Note, both
* runlists @drl and @srl are deallocated before returning so you cannot use
* the pointers for anything any more. (Strictly speaking the returned runlist
- * may be the same as @dst but this is irrelevant.)
+ * may be the same as @dst but this is irrelevant.) If @keep_srl is true,
+ * @srl is neither modified nor deallocated, and @drl must not be NULL.
*
* On error, return -errno. Both runlists are left unmodified.
*/
-struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
- struct runlist_element *srl, size_t s_rl_count,
- size_t *new_rl_count)
+static struct runlist_element *__ntfs_runlists_merge(struct runlist *d_runlist,
+ struct runlist_element *srl, size_t s_rl_count,
+ size_t *new_rl_count, bool keep_srl)
{
int di, si; /* Current index into @[ds]rl. */
int sstart; /* First index with lcn > LCN_RL_NOT_MAPPED. */
@@ -553,6 +571,8 @@ struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
/* Check for the case where the first mapping is being done now. */
if (unlikely(!drl)) {
+ if (WARN_ON(keep_srl))
+ return ERR_PTR(-EINVAL);
drl = srl;
/* Complete the source runlist if necessary. */
if (unlikely(drl[0].vcn)) {
@@ -582,7 +602,8 @@ struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
si++;
/* Can't have an entirely unmapped source runlist. */
- WARN_ON(!srl[si].length);
+ if (WARN_ON(!srl[si].length))
+ return ERR_PTR(-EINVAL);
/* Record the starting points. */
sstart = si;
@@ -652,7 +673,8 @@ struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
ntfs_error(NULL, "Merge failed.");
return drl;
}
- kvfree(srl);
+ if (!keep_srl)
+ kvfree(srl);
if (marker) {
ntfs_debug("Triggering marker code.");
for (ds = dend; drl[ds].length; ds++)
@@ -714,6 +736,44 @@ struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
return drl;
}
+/*
+ * ntfs_runlists_merge - merge two runlists into one
+ * @d_runlist: destination runlist structure to merge into
+ * @srl: source runlist to merge from, deallocated on success
+ * @s_rl_count: number of elements in @srl (0 to auto-detect)
+ * @new_rl_count: on success, set to the new combined runlist size
+ *
+ * See __ntfs_runlists_merge().
+ */
+struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
+ struct runlist_element *srl, size_t s_rl_count,
+ size_t *new_rl_count)
+{
+ return __ntfs_runlists_merge(d_runlist, srl, s_rl_count, new_rl_count,
+ false);
+}
+
+/*
+ * ntfs_runlists_merge_keep_src - merge two runlists, keeping the source
+ * @d_runlist: destination runlist structure to merge into, not empty
+ * @srl: source runlist to merge from
+ * @s_rl_count: number of elements in @srl (0 to auto-detect)
+ * @new_rl_count: on success, set to the new combined runlist size
+ *
+ * Like ntfs_runlists_merge(), but @srl is neither modified nor deallocated,
+ * on success or on error, and remains the caller's to kvfree(). Even when
+ * runs of @srl are coalesced with runs of @d_runlist, @srl keeps describing
+ * exactly its own runs, e.g. the clusters a caller has to free to undo the
+ * merge.
+ */
+struct runlist_element *ntfs_runlists_merge_keep_src(struct runlist *d_runlist,
+ struct runlist_element *srl, size_t s_rl_count,
+ size_t *new_rl_count)
+{
+ return __ntfs_runlists_merge(d_runlist, srl, s_rl_count, new_rl_count,
+ true);
+}
+
/*
* ntfs_mapping_pairs_decompress - convert mapping pairs array to runlist
* @vol: ntfs volume
diff --git a/fs/ntfs/runlist.h b/fs/ntfs/runlist.h
index cc92e09fbf68..0d4c32b1a338 100644
--- a/fs/ntfs/runlist.h
+++ b/fs/ntfs/runlist.h
@@ -68,6 +68,9 @@ enum {
struct runlist_element *ntfs_runlists_merge(struct runlist *d_runlist,
struct runlist_element *srl, size_t s_rl_count,
size_t *new_rl_count);
+struct runlist_element *ntfs_runlists_merge_keep_src(struct runlist *d_runlist,
+ struct runlist_element *srl, size_t s_rl_count,
+ size_t *new_rl_count);
struct runlist_element *ntfs_mapping_pairs_decompress(const struct ntfs_volume *vol,
const struct attr_record *attr, struct runlist *old_runlist,
size_t *new_rl_count);
diff --git a/fs/ntfs/super.c b/fs/ntfs/super.c
index 4066bacabe37..0a1faa262c41 100644
--- a/fs/ntfs/super.c
+++ b/fs/ntfs/super.c
@@ -18,6 +18,7 @@
#include "sysctl.h"
#include "logfile.h"
#include "index.h"
+#include "mft.h"
#include "ntfs.h"
#include "ea.h"
#include "volume.h"
@@ -262,14 +263,23 @@ static int ntfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
return 0;
}
+static int ntfs_sync_volume_dirty_state(struct ntfs_volume *vol);
+
static int ntfs_reconfigure(struct fs_context *fc)
{
struct super_block *sb = fc->root->d_sb;
struct ntfs_volume *vol = NTFS_SB(sb);
+ int err;
ntfs_debug("Entering with remount");
- sync_filesystem(sb);
+ err = sync_filesystem(sb);
+ if (err) {
+ ntfs_warning(sb, "Failed to sync the filesystem.");
+ /* A forced remount must still turn the superblock read-only. */
+ if (!(fc->sb_flags & SB_FORCE))
+ return err;
+ }
/*
* For the read-write compiled driver, if we are remounting read-write,
@@ -286,6 +296,12 @@ static int ntfs_reconfigure(struct fs_context *fc)
static const char *es = ". Cannot remount read-write.";
/* Remounting read-write. */
+ if (!vol->mft_write_supported) {
+ ntfs_error(sb,
+ "MFT writeback is unsupported for this geometry%s",
+ es);
+ return -EROFS;
+ }
if (NVolErrors(vol)) {
ntfs_error(sb, "Volume has errors and is read-only%s",
es);
@@ -312,10 +328,58 @@ static int ntfs_reconfigure(struct fs_context *fc)
}
} else if (!sb_rdonly(sb) && (fc->sb_flags & SB_RDONLY)) {
/* Remounting read-only. */
- if (!NVolErrors(vol)) {
- if (ntfs_clear_volume_flags(vol, VOLUME_IS_DIRTY))
+ /*
+ * With errors recorded the dirty bit is set rather than
+ * cleared, and it is committed right away: the VFS does
+ * not sync the filesystem during a remount, and once the
+ * remount succeeds no further persistence point exists -
+ * ntfs_sync_fs() is only ever invoked for read-write
+ * superblocks (all its VFS callers skip read-only ones)
+ * and ntfs_put_super() skips them, so the only remaining
+ * write would be the evict-time commit at unmount, which
+ * a crash never reaches. An error recorded only after
+ * the remount is still never persisted; a failed commit
+ * or flush fails the remount, leaving the superblock
+ * read-write so ntfs_put_super() retries at unmount.
+ */
+ /*
+ * A forced remount does not drain writers in progress,
+ * so one may still be modifying metadata when the flags
+ * are committed: never clear the dirty bit then; if
+ * errors have been recorded, the update preserves or
+ * sets it; otherwise, skip the update entirely.
+ */
+ if (!(fc->sb_flags & SB_FORCE) || NVolErrors(vol)) {
+ err = ntfs_sync_volume_dirty_state(vol);
+ if (err) {
+ ntfs_warning(sb,
+ "Failed to update dirty bit in volume information flags. Run chkdsk.");
+ return err;
+ }
+ }
+ if (NInoDirty(NTFS_I(vol->vol_ino))) {
+ /* ntfs_commit_inode() would discard the error. */
+ err = __ntfs_write_inode(vol->vol_ino, 1);
+ if (err) {
+ ntfs_warning(sb,
+ "Failed to commit volume information flags. Run chkdsk.");
+ return err;
+ }
+ /*
+ * write_mft_record() redirties the record on
+ * -ENOMEM and still reports success.
+ */
+ if (NInoDirty(NTFS_I(vol->vol_ino))) {
ntfs_warning(sb,
- "Failed to clear dirty bit in volume information flags. Run chkdsk.");
+ "Volume information flags remain dirty after commit. Run chkdsk.");
+ return -EIO;
+ }
+ err = blkdev_issue_flush(sb->s_bdev);
+ if (err) {
+ ntfs_warning(sb,
+ "Failed to flush volume information flags. Run chkdsk.");
+ return err;
+ }
}
}
@@ -353,31 +417,52 @@ void ntfs_handle_error(struct super_block *sb)
}
/*
- * ntfs_write_volume_flags - write new flags to the volume information flags
+ * ntfs_write_volume_flags - apply flag changes to the volume information flags
* @vol: ntfs volume on which to modify the flags
- * @flags: new flags value for the volume information flags
+ * @set_bits: bits to set in the volume information flags
+ * @clear_bits: bits to clear in the volume information flags
+ * @dirty_if_errors: force VOLUME_IS_DIRTY on when NVolErrors() is set
*
* Internal function. You probably want to use ntfs_{set,clear}_volume_flags()
- * instead (see below).
+ * or ntfs_sync_volume_dirty_state() instead (see below).
*
- * Replace the volume information flags on the volume @vol with the value
- * supplied in @flags. Note, this overwrites the volume information flags, so
- * make sure to combine the flags you want to modify with the old flags and use
- * the result when calling ntfs_write_volume_flags().
+ * Combine @set_bits and @clear_bits with the current in-memory flag state and
+ * write the result back. The set/clear helpers pass only the bits to modify,
+ * not the complete flag state. The read-modify-write happens under
+ * ni->mrec_lock so that concurrent set/clear operations cannot lose updates.
+ * All bit manipulation is done on CPU-endian values, and the result is
+ * converted back to little-endian before storing it.
+ *
+ * When @dirty_if_errors is true and errors have been recorded on @vol,
+ * VOLUME_IS_DIRTY is forced on after the requested changes. NVolErrors() is
+ * evaluated under the same mrec_lock, which orders this against other
+ * locked flag updates; the runtime error paths themselves record the flag
+ * lock-free, so see ntfs_sync_volume_dirty_state() for the guarantee this
+ * provides against them.
*
* Return 0 on success and -errno on error.
*/
-static int ntfs_write_volume_flags(struct ntfs_volume *vol, const __le16 flags)
+static int ntfs_write_volume_flags(struct ntfs_volume *vol,
+ const __le16 set_bits, const __le16 clear_bits,
+ const bool dirty_if_errors)
{
struct ntfs_inode *ni = NTFS_I(vol->vol_ino);
struct volume_information *vi;
struct ntfs_attr_search_ctx *ctx;
+ u16 flags;
int err;
- ntfs_debug("Entering, old flags = 0x%x, new flags = 0x%x.",
- le16_to_cpu(vol->vol_flags), le16_to_cpu(flags));
mutex_lock(&ni->mrec_lock);
- if (vol->vol_flags == flags)
+
+ flags = le16_to_cpu(vol->vol_flags);
+ flags |= le16_to_cpu(set_bits) & le16_to_cpu(VOLUME_FLAGS_MASK);
+ flags &= ~(le16_to_cpu(clear_bits) & le16_to_cpu(VOLUME_FLAGS_MASK));
+ if (dirty_if_errors && NVolErrors(vol))
+ flags |= le16_to_cpu(VOLUME_IS_DIRTY);
+ ntfs_debug("Entering, old flags = 0x%x, new flags = 0x%x.",
+ le16_to_cpu(vol->vol_flags), flags);
+
+ if (le16_to_cpu(vol->vol_flags) == flags)
goto done;
ctx = ntfs_attr_get_search_ctx(ni, NULL);
@@ -393,7 +478,7 @@ static int ntfs_write_volume_flags(struct ntfs_volume *vol, const __le16 flags)
vi = (struct volume_information *)((u8 *)ctx->attr +
le16_to_cpu(ctx->attr->data.resident.value_offset));
- vol->vol_flags = vi->flags = flags;
+ vol->vol_flags = vi->flags = cpu_to_le16(flags);
mark_mft_record_dirty(ctx->ntfs_ino);
ntfs_attr_put_search_ctx(ctx);
done:
@@ -414,29 +499,56 @@ static int ntfs_write_volume_flags(struct ntfs_volume *vol, const __le16 flags)
* @flags: flags to set on the volume
*
* Set the bits in @flags in the volume information flags on the volume @vol.
+ * The bits are combined with the current flag state under the lock in
+ * ntfs_write_volume_flags(), so concurrent updates are not lost.
*
* Return 0 on success and -errno on error.
*/
int ntfs_set_volume_flags(struct ntfs_volume *vol, __le16 flags)
{
- flags &= VOLUME_FLAGS_MASK;
- return ntfs_write_volume_flags(vol, vol->vol_flags | flags);
+ return ntfs_write_volume_flags(vol, flags, 0, false);
}
/*
- * ntfs_clear_volume_flags - clear bits in the volume information flags
- * @vol: ntfs volume on which to modify the flags
- * @flags: flags to clear on the volume
+ * ntfs_sync_volume_dirty_state - persist the dirty bit per the error state
+ * @vol: ntfs volume whose dirty bit to persist
*
- * Clear the bits in @flags in the volume information flags on the volume @vol.
+ * Set VOLUME_IS_DIRTY if errors have been recorded on @vol and clear it
+ * otherwise, under the $Volume mrec_lock.
+ *
+ * The guarantee this provides is eventual, not instantaneous: the runtime
+ * error paths record NVolErrors() with a lock-free set_bit(), so a
+ * persistence point that evaluates the flag just before an error is
+ * recorded can still leave the on-disk bit clean. This is sound because
+ * NVolErrors() is sticky (nothing clears it for the lifetime of the mount)
+ * and every persistence point re-derives the on-disk bit from it; the
+ * last one, ntfs_put_super(), runs after evict_inodes() on a quiesced
+ * filesystem, so a volume that is read-write at unmount time cannot
+ * unmount clean. A volume that is already read-only when the error is
+ * recorded (errors=remount-ro flips the superblock on the first error,
+ * as does an earlier remount-ro) has no persistence point left and
+ * keeps whatever on-disk bit it had; that behaviour is unchanged. The
+ * residual window is a crash between the error and the next
+ * persistence point.
+ *
+ * This is the single point that persists the in-memory error state to disk.
+ * The runtime error paths only record NVolErrors() because they run under a
+ * variety of ntfs locks the dirty-bit write cannot be taken under (runlist
+ * locks, vol->lcnbmp_lock, vol->mftbmp_lock, mrec_locks); a sync of a
+ * volume with recorded errors, a remount to read-only, or the unmount,
+ * then persists the flag here.
+ *
+ * A hibernated volume is not written from these persistence paths:
+ * resuming Windows from a modified image corrupts it, so the dirty bit
+ * is left as it is on disk and only the in-memory error state is kept.
*
* Return 0 on success and -errno on error.
*/
-int ntfs_clear_volume_flags(struct ntfs_volume *vol, __le16 flags)
+static int ntfs_sync_volume_dirty_state(struct ntfs_volume *vol)
{
- flags &= VOLUME_FLAGS_MASK;
- flags = vol->vol_flags & cpu_to_le16(~le16_to_cpu(flags));
- return ntfs_write_volume_flags(vol, flags);
+ if (NVolHibernated(vol))
+ return 0;
+ return ntfs_write_volume_flags(vol, 0, VOLUME_IS_DIRTY, true);
}
int ntfs_write_volume_label(struct ntfs_volume *vol, char *label)
@@ -530,6 +642,8 @@ int ntfs_write_volume_label(struct ntfs_volume *vol, char *label)
static bool is_boot_sector_ntfs(const struct super_block *sb,
const struct ntfs_boot_sector *b, const bool silent)
{
+ u16 sector_size = le16_to_cpu(b->bpb.bytes_per_sector);
+
/*
* Check that checksum == sum of u32 values from b to the checksum
* field. If checksum is zero, no checking is done. We will work when
@@ -550,8 +664,8 @@ static bool is_boot_sector_ntfs(const struct super_block *sb,
if (b->oem_id != magicNTFS)
goto not_ntfs;
/* Check bytes per sector value is between 256 and 4096. */
- if (le16_to_cpu(b->bpb.bytes_per_sector) < 0x100 ||
- le16_to_cpu(b->bpb.bytes_per_sector) > 0x1000)
+ if (sector_size < 0x100 || sector_size > 0x1000 ||
+ !is_power_of_2(sector_size))
goto not_ntfs;
/*
* Check sectors per cluster value is valid and the cluster size
@@ -632,6 +746,60 @@ static char *read_ntfs_boot_sector(struct super_block *sb,
return boot_sector;
}
+static bool ntfs_validate_mft_io_geometry(struct ntfs_volume *vol,
+ const unsigned int logical_block_size)
+{
+ struct super_block *sb = vol->sb;
+ u32 mft_io_unit_size = 0;
+ bool mft_write_supported = true;
+
+ if (!is_power_of_2(logical_block_size) ||
+ logical_block_size > PAGE_SIZE ||
+ !is_power_of_2(vol->sector_size) ||
+ vol->sector_size < logical_block_size ||
+ vol->sector_size % logical_block_size ||
+ !is_power_of_2(vol->cluster_size) ||
+ !is_power_of_2(vol->mft_record_size) ||
+ vol->mft_record_size > PAGE_SIZE)
+ goto err;
+
+ mft_io_unit_size = max(logical_block_size, vol->mft_record_size);
+ if (mft_io_unit_size > PAGE_SIZE ||
+ mft_io_unit_size % logical_block_size ||
+ mft_io_unit_size % vol->mft_record_size)
+ goto err;
+
+ /*
+ * The current direct-write path stores at most two MFT runlist
+ * segments. Keep valid but larger records read-only until that path
+ * can map an arbitrary number of segments.
+ */
+ if (vol->mft_record_size > 2 * (u64)vol->cluster_size)
+ mft_write_supported = false;
+
+ /*
+ * A containing device block is currently mapped through one MFT
+ * runlist element. Keep valid geometries that require crossing a
+ * cluster read-only until the mapping is generalized.
+ */
+ if (mft_io_unit_size > vol->mft_record_size &&
+ (vol->cluster_size < mft_io_unit_size ||
+ vol->cluster_size % mft_io_unit_size))
+ mft_write_supported = false;
+
+ vol->mft_io_unit_size = mft_io_unit_size;
+ vol->mft_write_supported = mft_write_supported;
+ return true;
+
+err:
+ ntfs_error(sb,
+ "Unsupported MFT I/O geometry (logical %u, block %lu, sector %u, cluster %u, MFT record %u, I/O unit %u).",
+ logical_block_size, sb->s_blocksize,
+ (unsigned int)vol->sector_size, vol->cluster_size,
+ vol->mft_record_size, mft_io_unit_size);
+ return false;
+}
+
/*
* parse_ntfs_boot_sector - parse the boot sector and store the data in @vol
* @vol: volume structure to initialise with data from boot sector
@@ -644,9 +812,11 @@ static bool parse_ntfs_boot_sector(struct ntfs_volume *vol,
const struct ntfs_boot_sector *b)
{
unsigned int sectors_per_cluster, sectors_per_cluster_bits, nr_hidden_sects;
+ unsigned int logical_block_size;
int clusters_per_mft_record, clusters_per_index_record;
u64 ll;
+ logical_block_size = bdev_logical_block_size(vol->sb->s_bdev);
vol->sector_size = le16_to_cpu(b->bpb.bytes_per_sector);
vol->sector_size_bits = ffs(vol->sector_size) - 1;
ntfs_debug("vol->sector_size = %i (0x%x)", vol->sector_size,
@@ -719,6 +889,9 @@ static bool parse_ntfs_boot_sector(struct ntfs_volume *vol,
ntfs_warning(vol->sb, "Mft record size (%i) is smaller than the sector size (%i).",
vol->mft_record_size, vol->sector_size);
}
+ if (!ntfs_validate_mft_io_geometry(vol, logical_block_size))
+ return false;
+
clusters_per_index_record = b->clusters_per_index_record;
ntfs_debug("clusters_per_index_record = %i (0x%x)",
clusters_per_index_record, clusters_per_index_record);
@@ -959,7 +1132,7 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
struct super_block *sb = vol->sb;
struct ntfs_inode *mirr_ni;
struct folio *mft_folio = NULL, *mirr_folio = NULL;
- u8 *kmft = NULL, *kmirr = NULL;
+ u8 *kmft = NULL, *kmirr = NULL, *kmft_base = NULL, *kmirr_base = NULL;
struct runlist_element *rl, rl2[2];
pgoff_t index;
int mrecs_per_page, i;
@@ -974,9 +1147,9 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
/* Switch pages if necessary. */
if (!(i % mrecs_per_page)) {
if (index) {
- kunmap_local(kmirr);
+ kunmap_local(kmirr_base);
folio_put(mirr_folio);
- kunmap_local(kmft);
+ kunmap_local(kmft_base);
folio_put(mft_folio);
}
/* Get the $MFT page. */
@@ -986,7 +1159,8 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
ntfs_error(sb, "Failed to read $MFT.");
return false;
}
- kmft = kmap_local_folio(mft_folio, 0);
+ kmft_base = kmap_local_folio(mft_folio, 0);
+ kmft = kmft_base;
/* Get the $MFTMirr page. */
mirr_folio = read_mapping_folio(vol->mftmirr_ino->i_mapping,
index, NULL);
@@ -994,7 +1168,8 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
ntfs_error(sb, "Failed to read $MFTMirr.");
goto mft_unmap_out;
}
- kmirr = kmap_local_folio(mirr_folio, 0);
+ kmirr_base = kmap_local_folio(mirr_folio, 0);
+ kmirr = kmirr_base;
++index;
}
@@ -1006,10 +1181,10 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
"Incomplete multi sector transfer detected in mft record %i.",
i);
mm_unmap_out:
- kunmap_local(kmirr);
+ kunmap_local(kmirr_base);
folio_put(mirr_folio);
mft_unmap_out:
- kunmap_local(kmft);
+ kunmap_local(kmft_base);
folio_put(mft_folio);
return false;
}
@@ -1045,9 +1220,9 @@ static bool check_mft_mirror(struct ntfs_volume *vol)
kmirr += vol->mft_record_size;
} while (++i < vol->mftmirr_size);
/* Release the last folios. */
- kunmap_local(kmirr);
+ kunmap_local(kmirr_base);
folio_put(mirr_folio);
- kunmap_local(kmft);
+ kunmap_local(kmft_base);
folio_put(mft_folio);
/* Construct the mft mirror runlist by hand. */
@@ -1404,6 +1579,7 @@ static bool load_system_files(struct ntfs_volume *vol)
struct ntfs_attr_search_ctx *ctx;
struct restart_page_header *rp;
int err;
+ u8 saved_on_errors;
ntfs_debug("Entering.");
/* Get mft mirror inode compare the contents of $MFT and $MFTMirr. */
@@ -1468,8 +1644,7 @@ static bool load_system_files(struct ntfs_volume *vol)
*/
vol->vol_ino = ntfs_iget(sb, FILE_Volume);
if (IS_ERR(vol->vol_ino)) {
- if (!IS_ERR(vol->vol_ino))
- iput(vol->vol_ino);
+ vol->vol_ino = NULL;
volume_failed:
ntfs_error(sb, "Failed to load $Volume.");
goto iput_lcnbmp_err_out;
@@ -1478,6 +1653,7 @@ static bool load_system_files(struct ntfs_volume *vol)
if (IS_ERR(m)) {
iput_volume_failed:
iput(vol->vol_ino);
+ vol->vol_ino = NULL;
goto volume_failed;
}
@@ -1578,8 +1754,19 @@ static bool load_system_files(struct ntfs_volume *vol)
* NVolErrors() without setting the dirty volume flag and mount
* read-only. This will prevent read-write remounting and it will also
* prevent all writes.
+ *
+ * Nested lookup and inode-loading errors must not panic before the
+ * read-only fallback has run. Temporarily use errors=remount-ro
+ * instead of errors=panic, preserving any read-only transition even
+ * if an error is not propagated to the check's return value or
+ * recorded in NVolErrors(). This is safe during initial mount,
+ * before the super block is published.
*/
+ saved_on_errors = vol->on_errors;
+ if (saved_on_errors == ON_ERRORS_PANIC)
+ vol->on_errors = ON_ERRORS_REMOUNT_RO;
err = check_windows_hibernation_status(vol);
+ vol->on_errors = saved_on_errors;
if (unlikely(err)) {
static const char *es1a = "Failed to determine if Windows is hibernated";
static const char *es1b = "Windows is hibernated";
@@ -1587,12 +1774,21 @@ static bool load_system_files(struct ntfs_volume *vol)
const char *es1;
es1 = err < 0 ? es1a : es1b;
- /* If a read-write mount, convert it to a read-only mount. */
- if (!sb_rdonly(sb) && vol->on_errors == ON_ERRORS_REMOUNT_RO) {
- sb->s_flags |= SB_RDONLY;
- ntfs_error(sb, "%s. Mounting read-only%s", es1, es2);
- }
+ /* Hibernation safety takes precedence over the errors= policy. */
+ sb->s_flags |= SB_RDONLY;
NVolSetErrors(vol);
+ ntfs_error(sb, "%s. Mounting read-only%s", es1, es2);
+
+ /*
+ * Remember it for the lifetime of the mount: see
+ * ntfs_sync_volume_dirty_state().
+ */
+ NVolSetHibernated(vol);
+ } else if (!sb_rdonly(sb) && NVolErrors(vol)) {
+ /* Match the read-write remount restriction for recorded errors. */
+ sb->s_flags |= SB_RDONLY;
+ ntfs_error(sb,
+ "Errors were recorded during mount. Mounting read-only. Run chkdsk.");
}
/* If (still) a read-write mount, empty the logfile. */
@@ -1638,6 +1834,8 @@ static bool load_system_files(struct ntfs_volume *vol)
if (vol->logfile_ino)
iput(vol->logfile_ino);
iput(vol->vol_ino);
+ /* Do not leave a stale pointer behind for the rest of the teardown. */
+ vol->vol_ino = NULL;
iput_lcnbmp_err_out:
iput(vol->lcnbmp_ino);
iput_attrdef_err_out:
@@ -1748,28 +1946,21 @@ static void ntfs_put_super(struct super_block *sb)
ntfs_commit_inode(vol->mft_ino);
/*
- * If a read-write mount and no volume errors have occurred, mark the
- * volume clean. Also, re-commit all affected inodes.
+ * If a read-write mount, re-commit all affected inodes once more.
+ * The dirty state itself is persisted at the end of ntfs_put_super(),
+ * after the last commits and the final write_inode_now(): those can
+ * still record errors via __ntfs_write_inode(), and the sync must
+ * evaluate NVolErrors() with the last setter already run.
*/
if (!sb_rdonly(sb)) {
if (!NVolErrors(vol)) {
- if (ntfs_clear_volume_flags(vol, VOLUME_IS_DIRTY))
- ntfs_warning(sb,
- "Failed to clear dirty bit in volume information flags. Run chkdsk.");
- ntfs_commit_inode(vol->vol_ino);
ntfs_commit_inode(vol->root_ino);
if (vol->mftmirr_ino)
ntfs_commit_inode(vol->mftmirr_ino);
ntfs_commit_inode(vol->mft_ino);
- } else {
- ntfs_warning(sb,
- "Volume has errors. Leaving volume marked dirty. Run chkdsk.");
}
}
- iput(vol->vol_ino);
- vol->vol_ino = NULL;
-
/* NTFS 3.0+ specific clean up. */
if (vol->major_ver >= 3) {
if (vol->extend_ino) {
@@ -1799,8 +1990,6 @@ static void ntfs_put_super(struct super_block *sb)
/* Re-commit the mft mirror and mft just in case. */
ntfs_commit_inode(vol->mftmirr_ino);
ntfs_commit_inode(vol->mft_ino);
- iput(vol->mftmirr_ino);
- vol->mftmirr_ino = NULL;
}
/*
* We should have no dirty inodes left, due to
@@ -1810,6 +1999,59 @@ static void ntfs_put_super(struct super_block *sb)
ntfs_commit_inode(vol->mft_ino);
write_inode_now(vol->mft_ino, 1);
+ /*
+ * If a read-write mount, persist the error state in the volume flags:
+ * mark the volume clean if no volume errors have occurred, and make
+ * sure VOLUME_IS_DIRTY is on disk if any have, so chkdsk runs on the
+ * next mount.
+ */
+ if (!sb_rdonly(sb)) {
+ if (ntfs_sync_volume_dirty_state(vol)) {
+ ntfs_warning(sb,
+ "Failed to sync dirty bit in volume information flags. Run chkdsk.");
+ } else {
+ /*
+ * __ntfs_write_inode(), not the void
+ * ntfs_commit_inode() wrapper: the error can only
+ * be warned about here. The mirror inode is only
+ * released below: writing the $Volume record (mft
+ * record number 3, below vol->mftmirr_size) mirrors
+ * it through ntfs_sync_mft_mirror(), which fails
+ * with -EIO once vol->mftmirr_ino is gone.
+ */
+ if (__ntfs_write_inode(vol->vol_ino, 1)) {
+ ntfs_warning(sb,
+ "Failed to commit volume information flags. Run chkdsk.");
+ } else if (NInoDirty(NTFS_I(vol->vol_ino))) {
+ ntfs_warning(sb,
+ "Volume information flags remain dirty after commit. Run chkdsk.");
+ } else if (NVolErrors(vol)) {
+ /*
+ * Only warn once the commit has succeeded,
+ * or this could contradict a failure
+ * reported above.
+ */
+ ntfs_warning(sb,
+ "Volume has errors. Leaving volume marked dirty. Run chkdsk.");
+ }
+ }
+ }
+
+ /*
+ * Release $Volume while the mft inode is still available: if the
+ * commit above failed before it could clear the dirty flag,
+ * ntfs_evict_big_inode() commits the inode again on its way out,
+ * and __ntfs_write_inode() needs vol->mft_ino to look up the
+ * runlist of the record to write.
+ */
+ iput(vol->vol_ino);
+ vol->vol_ino = NULL;
+
+ if (vol->mftmirr_ino) {
+ iput(vol->mftmirr_ino);
+ vol->mftmirr_ino = NULL;
+ }
+
iput(vol->mft_ino);
vol->mft_ino = NULL;
blkdev_issue_flush(sb->s_bdev);
@@ -1853,7 +2095,7 @@ static void ntfs_shutdown(struct super_block *sb)
static int ntfs_sync_fs(struct super_block *sb, int wait)
{
struct ntfs_volume *vol = NTFS_SB(sb);
- int err = 0;
+ int ret, err = 0;
if (NVolShutdown(vol))
return -EIO;
@@ -1861,15 +2103,30 @@ static int ntfs_sync_fs(struct super_block *sb, int wait)
if (!wait)
return 0;
- /* If there are some dirty buffers in the bdev inode */
- if (!NVolErrors(vol) &&
- ntfs_clear_volume_flags(vol, VOLUME_IS_DIRTY)) {
- ntfs_warning(sb, "Failed to clear dirty bit in volume information flags. Run chkdsk.");
+ /*
+ * The volume dirty bit is deliberately not cleared here: a sync
+ * running concurrently with an in-flight modification could clear
+ * and persist a bit that was just set, leaving the modification
+ * on a volume that is clean on disk. The bit is only cleared at
+ * the quiescent state transitions, remounting read-only and clean
+ * unmount. A recorded error state, however, is persisted right
+ * away so that it is not lost to a crash on a volume that has
+ * seen no modification; with NVolErrors() set this can only set
+ * the bit, never clear it.
+ */
+ if (NVolErrors(vol) &&
+ ntfs_sync_volume_dirty_state(vol)) {
+ ntfs_warning(sb, "Failed to sync dirty bit in volume information flags. Run chkdsk.");
err = -EIO;
}
sync_inodes_sb(sb);
- sync_blockdev(sb->s_bdev);
- blkdev_issue_flush(sb->s_bdev);
+ ret = sync_blockdev(sb->s_bdev);
+ if (ret && !err)
+ err = ret;
+
+ ret = blkdev_issue_flush(sb->s_bdev);
+ if (ret && !err)
+ err = ret;
return err;
}
@@ -2290,6 +2547,11 @@ static int ntfs_fill_super(struct super_block *sb, struct fs_context *fc)
ntfs_error(sb, "Unsupported NTFS filesystem.");
goto err_out_now;
}
+ if (!vol->mft_write_supported && !sb_rdonly(sb)) {
+ sb->s_flags |= SB_RDONLY;
+ ntfs_warning(sb,
+ "MFT writeback is unsupported for this geometry. Mounting read-only.");
+ }
if (vol->sector_size > blocksize) {
blocksize = sb_set_blocksize(sb, vol->sector_size);
@@ -2613,6 +2875,13 @@ static int __init init_ntfs_fs(void)
return err;
}
+ err = ntfs_mft_bioset_init();
+ if (err) {
+ pr_crit("Failed to initialize NTFS MFT bioset!\n");
+ ntfs_workqueue_destroy();
+ return err;
+ }
+
ntfs_index_ctx_cache = kmem_cache_create(ntfs_index_ctx_cache_name,
sizeof(struct ntfs_index_context), 0 /* offset */,
SLAB_HWCACHE_ALIGN, NULL /* ctor */);
@@ -2680,6 +2949,7 @@ static int __init init_ntfs_fs(void)
actx_err_out:
kmem_cache_destroy(ntfs_index_ctx_cache);
ictx_err_out:
+ ntfs_mft_bioset_exit();
if (!err) {
pr_crit("Aborting NTFS filesystem driver registration...\n");
err = -ENOMEM;
@@ -2698,6 +2968,7 @@ static void __exit exit_ntfs_fs(void)
* destroy cache.
*/
rcu_barrier();
+ ntfs_mft_bioset_exit();
#ifdef CONFIG_NTFS_FS_WOF_COMPRESSION
ntfs_wof_free_workspaces();
#endif
diff --git a/fs/ntfs/volume.h b/fs/ntfs/volume.h
index bc85a9592245..0473b602084c 100644
--- a/fs/ntfs/volume.h
+++ b/fs/ntfs/volume.h
@@ -43,6 +43,7 @@
* @mft_record_size: in bytes
* @mft_record_size_mask: mft_record_size - 1
* @mft_record_size_bits: log2(mft_record_size)
+ * @mft_write_supported: Whether the MFT write paths support the geometry.
* @index_record_size: in bytes
* @index_record_size_mask: index_record_size - 1
* @index_record_size_bits: log2(index_record_size)
@@ -111,6 +112,13 @@ struct ntfs_volume {
u32 mft_record_size;
u32 mft_record_size_mask;
u8 mft_record_size_bits;
+ bool mft_write_supported;
+ /*
+ * Unit size used for MFT I/O. This is the MFT record size when
+ * it is at least as large as the device logical block, or the
+ * containing device logical block when the record is smaller.
+ */
+ u32 mft_io_unit_size;
u32 index_record_size;
u32 index_record_size_mask;
u8 index_record_size_bits;
@@ -181,9 +189,12 @@ struct ntfs_volume {
* Windows-reserved names (CON, AUX, NUL, COM1,
* LPT1, etc.) or invalid characters.
*
+ * NV_Hibernated Windows is hibernated on the volume; the sync
+ * paths must not write the volume flags.
* NV_Discard Issue discard/TRIM commands for freed clusters.
* NV_DisableSparse Disable creation of sparse regions.
* NV_NativeSymlinkRel Translate absolute Windows reparse targets (native_symlink=rel).
+ * NV_MftBootstrap Mount is still assembling $MFT's own runlist.
*/
enum {
NV_Errors,
@@ -199,10 +210,12 @@ enum {
NV_ShowHiddenFiles,
NV_HideDotFiles,
NV_CheckWindowsNames,
+ NV_Hibernated,
NV_Discard,
NV_DisableSparse,
NV_NativeSymlinkRel,
NV_SymlinkNative,
+ NV_MftBootstrap,
};
/*
@@ -237,10 +250,12 @@ DEFINE_NVOL_BIT_OPS(SysImmutable)
DEFINE_NVOL_BIT_OPS(ShowHiddenFiles)
DEFINE_NVOL_BIT_OPS(HideDotFiles)
DEFINE_NVOL_BIT_OPS(CheckWindowsNames)
+DEFINE_NVOL_BIT_OPS(Hibernated)
DEFINE_NVOL_BIT_OPS(Discard)
DEFINE_NVOL_BIT_OPS(DisableSparse)
DEFINE_NVOL_BIT_OPS(NativeSymlinkRel)
DEFINE_NVOL_BIT_OPS(SymlinkNative)
+DEFINE_NVOL_BIT_OPS(MftBootstrap)
static inline void ntfs_inc_free_clusters(struct ntfs_volume *vol, s64 nr)
{
diff --git a/fs/ntfs3/attrib.c b/fs/ntfs3/attrib.c
index b1c315206ffa..fd2640878e6a 100644
--- a/fs/ntfs3/attrib.c
+++ b/fs/ntfs3/attrib.c
@@ -1034,6 +1034,21 @@ int attr_data_get_block_locked(struct ntfs_inode *ni, CLST vcn, CLST clen,
if (!attr_b->non_res) {
u32 data_size = le32_to_cpu(attr_b->res.data_size);
+
+ /*
+ * A resident attribute is copied into a single page below
+ * (alloc_page() + memcpy()) and mapped as a one-page
+ * IOMAP_INLINE extent by ntfs_iomap_begin(). mi_enum_attr()
+ * only bounds the resident value length against the MFT record
+ * size, so a corrupted volume whose records are larger than a
+ * page can report data_size > PAGE_SIZE; copying that many bytes
+ * would overflow the single destination page. Reject it.
+ */
+ if (data_size > PAGE_SIZE) {
+ err = -EINVAL;
+ goto out;
+ }
+
*lcn = RESIDENT_LCN;
*len = data_size;
if (res) {
diff --git a/fs/ntfs3/fslog.c b/fs/ntfs3/fslog.c
index ed50c1d0c23e..2dc12e72cc9b 100644
--- a/fs/ntfs3/fslog.c
+++ b/fs/ntfs3/fslog.c
@@ -7,6 +7,7 @@
#include <linux/blkdev.h>
#include <linux/fs.h>
+#include <linux/overflow.h>
#include <linux/random.h>
#include <linux/slab.h>
@@ -2457,7 +2458,8 @@ static int find_log_rec(struct ntfs_log *log, u64 lsn, struct lcb *lcb)
* Check that the length field isn't greater than the total
* available space the log file.
*/
- rec_len = len + log->record_header_len;
+ if (check_add_overflow(len, log->record_header_len, &rec_len))
+ return -EINVAL;
if (rec_len >= log->total_avail)
return -EINVAL;
@@ -3318,10 +3320,15 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
asize = le32_to_cpu(attr2->size);
used = le32_to_cpu(rec->used);
+ /*
+ * attr2->size is taken from the log record and is bounded only
+ * by the log record length, not by the MFT record. Refuse an
+ * attribute that does not fit, as mi_insert_attr() does.
+ */
if (!check_if_attr(rec, lrh) || dlen < SIZEOF_RESIDENT ||
!IS_ALIGNED(asize, 8) ||
Add2Ptr(attr2, asize) > Add2Ptr(lrh, rec_len) ||
- dlen > record_size - used) {
+ dlen > record_size - used || asize > record_size - used) {
goto dirty_vol;
}
@@ -3344,10 +3351,21 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
break;
case DeleteAttribute:
- asize = le32_to_cpu(attr->size);
used = le32_to_cpu(rec->used);
- if (!check_if_attr(rec, lrh))
+ /*
+ * check_if_attr() accepts a record_off that points at the
+ * ATTR_END marker, which is how CreateAttribute appends. The
+ * bytes there are not an attribute and check_attr() never
+ * validated them, so refuse an offset that cannot hold a
+ * resident header before reading attr->size, and then bound it
+ * as mi_remove_attr() does.
+ */
+ if (!check_if_attr(rec, lrh) || roff + SIZEOF_RESIDENT > used)
+ goto dirty_vol;
+
+ asize = le32_to_cpu(attr->size);
+ if (asize > used - roff)
goto dirty_vol;
rec->used = cpu_to_le32(used - asize);
@@ -3421,7 +3439,8 @@ static int do_action(struct ntfs_log *log, struct OPEN_ATTR_ENRTY *oe,
if (!check_if_attr(rec, lrh) || !attr->non_res ||
aoff < le16_to_cpu(attr->nres.run_off) || aoff > asize ||
- (nsize > asize && nsize - asize > record_size - used)) {
+ (nsize > asize &&
+ ALIGN(nsize, 8) - asize > record_size - used)) {
goto dirty_vol;
}
diff --git a/fs/ntfs3/index.c b/fs/ntfs3/index.c
index 689712d3463d..37e3e16b1f84 100644
--- a/fs/ntfs3/index.c
+++ b/fs/ntfs3/index.c
@@ -594,6 +594,8 @@ static const struct NTFS_DE *hdr_insert_head(struct INDEX_HDR *hdr,
if (!e)
return NULL;
+ if (size_add(used, ins_bytes) > le32_to_cpu(hdr->total))
+ return NULL;
/* Now we just make room for the inserted entries and jam it in. */
to_move = used - le32_to_cpu(hdr->de_off);
@@ -1801,7 +1803,10 @@ static int indx_insert_into_root(struct ntfs_index *indx, struct ntfs_inode *ni,
}
/* Copy root entries into new buffer. */
- hdr_insert_head(hdr, re, to_move);
+ if (!hdr_insert_head(hdr, re, to_move)) {
+ err = -EINVAL;
+ goto out_put_n;
+ }
/* Update bitmap attribute. */
indx_mark_used(indx, ni, new_vbn >> indx->idx2vbn_bits);
@@ -1955,7 +1960,11 @@ static int indx_insert_into_buffer(struct ntfs_index *indx,
/* Copy all the entries <= sp into the new buffer. */
de_t = hdr_first_de(hdr1);
to_copy = PtrOffset(de_t, sp);
- hdr_insert_head(hdr2, de_t, to_copy);
+ if (!hdr_insert_head(hdr2, de_t, to_copy)) {
+ err = -EINVAL;
+ put_indx_node(n2);
+ goto out;
+ }
/* Remove all entries (sp including) from hdr1. */
used = used1 - to_copy - sp_size;
diff --git a/fs/ntfs3/inode.c b/fs/ntfs3/inode.c
index 4ac26c80bd34..fd844f9c556a 100644
--- a/fs/ntfs3/inode.c
+++ b/fs/ntfs3/inode.c
@@ -981,6 +981,8 @@ static int ntfs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
if (err) {
return err;
}
+ if (!clen)
+ return -EINVAL;
if (lcn == EOF_LCN) {
/* request out of file. */
@@ -1016,11 +1018,6 @@ static int ntfs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
return 0;
}
- if (!clen) {
- /* broken file? */
- return -EINVAL;
- }
-
iomap->bdev = inode->i_sb->s_bdev;
iomap->offset = offset;
iomap->length = ((loff_t)clen << cluster_bits) - off;
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index be09e766ac1f..c85a472965d5 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -1817,6 +1817,7 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
int i, ret = 0;
u32 range;
u64 blkno;
+ u32 prev_depth = OCFS2_MAX_PATH_DEPTH;
struct buffer_head *bh = NULL;
struct ocfs2_extent_block *eb;
struct ocfs2_extent_list *el;
@@ -1824,14 +1825,16 @@ static int __ocfs2_find_path(struct ocfs2_caching_info *ci,
el = root_el;
while (el->l_tree_depth) {
- if (unlikely(le16_to_cpu(el->l_tree_depth) >= OCFS2_MAX_PATH_DEPTH)) {
+ if (unlikely(le16_to_cpu(el->l_tree_depth) >= prev_depth)) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
- "Owner %llu has invalid tree depth %u in extent list\n",
+ "Owner %llu has invalid tree depth %u in extent list (max %u)\n",
(unsigned long long)ocfs2_metadata_cache_owner(ci),
- le16_to_cpu(el->l_tree_depth));
+ le16_to_cpu(el->l_tree_depth), prev_depth - 1);
ret = -EROFS;
goto out;
}
+ prev_depth = le16_to_cpu(el->l_tree_depth);
+
if (!el->l_next_free_rec || !el->l_count) {
ocfs2_error(ocfs2_metadata_cache_get_super(ci),
"Owner %llu has empty extent list at depth %u\n"
diff --git a/fs/ocfs2/cluster/heartbeat.c b/fs/ocfs2/cluster/heartbeat.c
index 602f8b913de7..dbc146717e8f 100644
--- a/fs/ocfs2/cluster/heartbeat.c
+++ b/fs/ocfs2/cluster/heartbeat.c
@@ -2137,6 +2137,18 @@ static struct config_item *o2hb_heartbeat_group_make_item(struct config_group *g
}
reg->hr_node_num = O2NM_MAX_NODES;
+ INIT_LIST_HEAD(®->hr_all_item);
+ mutex_init(®->hr_arming_mutex);
+ reg->hr_stopping = true;
+ INIT_DELAYED_WORK(®->hr_write_timeout_work, o2hb_write_timeout);
+ INIT_DELAYED_WORK(®->hr_nego_timeout_work, o2hb_nego_timeout);
+ INIT_LIST_HEAD(®->hr_handler_list);
+ config_item_init_type_name(®->hr_item, name, &o2hb_region_type);
+ if (!config_item_name(®->hr_item)) {
+ ret = -ENOMEM;
+ goto put_item;
+ }
+
spin_lock(&o2hb_live_lock);
reg->hr_region_num = 0;
if (o2hb_global_heartbeat_active()) {
@@ -2145,26 +2157,19 @@ static struct config_item *o2hb_heartbeat_group_make_item(struct config_group *g
if (reg->hr_region_num >= O2NM_MAX_REGIONS) {
spin_unlock(&o2hb_live_lock);
ret = -EFBIG;
- goto free;
+ goto put_item;
}
set_bit(reg->hr_region_num, o2hb_region_bitmap);
}
list_add_tail(®->hr_all_item, &o2hb_all_regions);
spin_unlock(&o2hb_live_lock);
- config_item_init_type_name(®->hr_item, name, &o2hb_region_type);
-
/* this is the same way to generate msg key as dlm, for local heartbeat,
* name is also the same, so make initial crc value different to avoid
* message key conflict.
*/
reg->hr_key = crc32_le(reg->hr_region_num + O2NM_MAX_REGIONS,
name, strlen(name));
- mutex_init(®->hr_arming_mutex);
- reg->hr_stopping = true;
- INIT_DELAYED_WORK(®->hr_write_timeout_work, o2hb_write_timeout);
- INIT_DELAYED_WORK(®->hr_nego_timeout_work, o2hb_nego_timeout);
- INIT_LIST_HEAD(®->hr_handler_list);
ret = o2net_register_handler(O2HB_NEGO_TIMEOUT_MSG, reg->hr_key,
sizeof(struct o2hb_nego_msg),
o2hb_nego_timeout_handler,
@@ -2187,10 +2192,13 @@ static struct config_item *o2hb_heartbeat_group_make_item(struct config_group *g
o2net_unregister_and_flush_handler_list(®->hr_handler_list);
remove_item:
spin_lock(&o2hb_live_lock);
- list_del(®->hr_all_item);
+ list_del_init(®->hr_all_item);
if (o2hb_global_heartbeat_active())
clear_bit(reg->hr_region_num, o2hb_region_bitmap);
spin_unlock(&o2hb_live_lock);
+put_item:
+ config_item_put(®->hr_item);
+ return ERR_PTR(ret);
free:
kfree(reg);
return ERR_PTR(ret);
diff --git a/fs/ocfs2/cluster/nodemanager.c b/fs/ocfs2/cluster/nodemanager.c
index ced4fe6d3b59..91f8fcc6193f 100644
--- a/fs/ocfs2/cluster/nodemanager.c
+++ b/fs/ocfs2/cluster/nodemanager.c
@@ -627,8 +627,11 @@ static void o2nm_node_group_drop_item(struct config_group *group,
write_lock(&cluster->cl_nodes_lock);
- /* XXX sloppy */
- if (node->nd_ipv4_address)
+ /*
+ * 0.0.0.0 is a valid configured address, so nd_ipv4_address does
+ * not tell us whether nd_ip_node was published in the IP tree.
+ */
+ if (test_bit(O2NM_NODE_ATTR_ADDRESS, &node->nd_set_attributes))
rb_erase(&node->nd_ip_node, &cluster->cl_node_ip_tree);
/* nd_num might be 0 if the node number hasn't been set.. */
diff --git a/fs/ocfs2/dir.c b/fs/ocfs2/dir.c
index 0075e1624310..b358b58a121d 100644
--- a/fs/ocfs2/dir.c
+++ b/fs/ocfs2/dir.c
@@ -221,7 +221,8 @@ static void TEA_transform(__u32 buf[4], __u32 const in[])
buf[1] += b1;
}
-static void str2hashbuf(const char *msg, int len, __u32 *buf, int num)
+static void str2hashbuf(const char *msg, int len, __u32 *buf, int num,
+ bool legacy_signed)
{
__u32 pad, val;
int i;
@@ -235,7 +236,10 @@ static void str2hashbuf(const char *msg, int len, __u32 *buf, int num)
for (i = 0; i < len; i++) {
if ((i % 4) == 0)
val = pad;
- val = msg[i] + (val << 8);
+ if (legacy_signed)
+ val = (signed char)msg[i] + (val << 8);
+ else
+ val = (unsigned char)msg[i] + (val << 8);
if ((i % 4) == 3) {
*buf++ = val;
val = pad;
@@ -248,8 +252,9 @@ static void str2hashbuf(const char *msg, int len, __u32 *buf, int num)
*buf++ = pad;
}
-static void ocfs2_dx_dir_name_hash(struct inode *dir, const char *name, int len,
- struct ocfs2_dx_hinfo *hinfo)
+static void __ocfs2_dx_dir_name_hash(struct inode *dir, const char *name,
+ int len, struct ocfs2_dx_hinfo *hinfo,
+ bool legacy_signed)
{
struct ocfs2_super *osb = OCFS2_SB(dir->i_sb);
const char *p;
@@ -279,7 +284,7 @@ static void ocfs2_dx_dir_name_hash(struct inode *dir, const char *name, int len,
p = name;
while (len > 0) {
- str2hashbuf(p, len, in, 4);
+ str2hashbuf(p, len, in, 4, legacy_signed);
TEA_transform(buf, in);
len -= 16;
p += 16;
@@ -290,6 +295,19 @@ static void ocfs2_dx_dir_name_hash(struct inode *dir, const char *name, int len,
hinfo->minor_hash = buf[1];
}
+static void ocfs2_dx_dir_name_hash(struct inode *dir, const char *name, int len,
+ struct ocfs2_dx_hinfo *hinfo)
+{
+ __ocfs2_dx_dir_name_hash(dir, name, len, hinfo, false);
+}
+
+static void ocfs2_dx_dir_name_hash_signed(struct inode *dir, const char *name,
+ int len,
+ struct ocfs2_dx_hinfo *hinfo)
+{
+ __ocfs2_dx_dir_name_hash(dir, name, len, hinfo, true);
+}
+
/*
* bh passed here can be an inode block or a dir data block, depending
* on the inode inline data flag.
@@ -966,10 +984,10 @@ static int ocfs2_dx_dir_lookup(struct inode *inode,
return ret;
}
-static int ocfs2_dx_dir_search(const char *name, int namelen,
- struct inode *dir,
- struct ocfs2_dx_root_block *dx_root,
- struct ocfs2_dir_lookup_result *res)
+static int __ocfs2_dx_dir_search(const char *name, int namelen,
+ struct inode *dir,
+ struct ocfs2_dx_root_block *dx_root,
+ struct ocfs2_dir_lookup_result *res)
{
int ret, i, found;
u64 phys;
@@ -982,8 +1000,6 @@ static int ocfs2_dx_dir_search(const char *name, int namelen,
struct ocfs2_extent_list *dr_el;
struct ocfs2_dx_entry_list *entry_list;
- ocfs2_dx_dir_name_hash(dir, name, namelen, &res->dl_hinfo);
-
if (ocfs2_dx_root_inline(dx_root)) {
entry_list = &dx_root->dr_entries;
goto search;
@@ -1080,6 +1096,44 @@ static int ocfs2_dx_dir_search(const char *name, int namelen,
return ret;
}
+static int ocfs2_dx_dir_search(const char *name, int namelen,
+ struct inode *dir,
+ struct ocfs2_dx_root_block *dx_root,
+ struct ocfs2_dir_lookup_result *res)
+{
+ struct ocfs2_dx_hinfo legacy;
+ int ret;
+
+ ocfs2_dx_dir_name_hash(dir, name, namelen, &res->dl_hinfo);
+
+ ret = __ocfs2_dx_dir_search(name, namelen, dir, dx_root, res);
+ if (ret != -ENOENT)
+ return ret;
+
+ /*
+ * Nothing under the current hash. The entry may have been indexed by
+ * an older kernel, which sign-extended the name bytes when hashing.
+ * New entries are always indexed under the unsigned hash, so only fall
+ * back to the legacy signed one when it can actually differ: an ASCII
+ * name hashes the same either way, and a genuine miss on one should
+ * not have to walk the index twice.
+ */
+ ocfs2_dx_dir_name_hash_signed(dir, name, namelen, &legacy);
+ if (legacy.major_hash == res->dl_hinfo.major_hash &&
+ legacy.minor_hash == res->dl_hinfo.minor_hash)
+ return ret;
+
+ res->dl_hinfo = legacy;
+
+ ret = __ocfs2_dx_dir_search(name, namelen, dir, dx_root, res);
+ if (ret)
+ return ret;
+
+ pr_warn_once("ocfs2: directory index with signed name hash\n");
+
+ return 0;
+}
+
static int ocfs2_find_entry_dx(const char *name, int namelen,
struct inode *dir,
struct ocfs2_dir_lookup_result *lookup)
diff --git a/fs/ocfs2/dlmglue.c b/fs/ocfs2/dlmglue.c
index 54e8f81119fb..1a83241b6d41 100644
--- a/fs/ocfs2/dlmglue.c
+++ b/fs/ocfs2/dlmglue.c
@@ -3527,6 +3527,10 @@ void ocfs2_mark_lockres_freeing(struct ocfs2_super *osb,
struct ocfs2_mask_waiter mw;
unsigned long flags, flags2;
+ /* We didn't get anywhere near actually using this lockres. */
+ if (!(lockres->l_flags & OCFS2_LOCK_INITIALIZED))
+ return;
+
ocfs2_init_mask_waiter(&mw);
spin_lock_irqsave(&lockres->l_lock, flags);
diff --git a/fs/ocfs2/file.c b/fs/ocfs2/file.c
index c6e7606ff9e1..b3a37b9d259a 100644
--- a/fs/ocfs2/file.c
+++ b/fs/ocfs2/file.c
@@ -141,18 +141,26 @@ static int ocfs2_file_open(struct inode *inode, struct file *file)
static int ocfs2_file_release(struct inode *inode, struct file *file)
{
struct ocfs2_inode_info *oi = OCFS2_I(inode);
+ struct dentry *dentry = file->f_path.dentry;
+ struct name_snapshot name;
+ bool trace_enabled = trace_ocfs2_file_release_enabled();
+
+ if (trace_enabled)
+ take_dentry_name_snapshot(&name, dentry);
spin_lock(&oi->ip_lock);
if (!--oi->ip_open_count)
oi->ip_flags &= ~OCFS2_INODE_OPEN_DIRECT;
- trace_ocfs2_file_release(inode, file, file->f_path.dentry,
- oi->ip_blkno,
- file->f_path.dentry->d_name.len,
- file->f_path.dentry->d_name.name,
- oi->ip_open_count);
+ if (trace_enabled)
+ trace_ocfs2_file_release(inode, file, dentry,
+ oi->ip_blkno, name.name.len,
+ name.name.name, oi->ip_open_count);
spin_unlock(&oi->ip_lock);
+ if (trace_enabled)
+ release_dentry_name_snapshot(&name);
+
ocfs2_free_file_private(inode, file);
return 0;
diff --git a/fs/ocfs2/inode.c b/fs/ocfs2/inode.c
index 180107a11046..9228d6ef23c2 100644
--- a/fs/ocfs2/inode.c
+++ b/fs/ocfs2/inode.c
@@ -638,14 +638,18 @@ static int ocfs2_read_locked_inode(struct inode *inode,
fe = (struct ocfs2_dinode *) bh->b_data;
/*
- * This is a code bug. Right now the caller needs to
- * understand whether it is asking for a system file inode or
- * not so the proper lock names can be built.
+ * The caller must know whether it is asking for a system file inode
+ * or not so the proper lock names can be built. Since i_flags comes
+ * from disk, a mismatch is filesystem corruption instead of a code
+ * bug, so handle it with ocfs2_error() rather than BUG_ON().
*/
- mlog_bug_on_msg(!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) !=
- !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE),
- "Inode %llu: system file state is ambiguous\n",
- (unsigned long long)args->fi_blkno);
+ if (!!(fe->i_flags & cpu_to_le32(OCFS2_SYSTEM_FL)) !=
+ !!(args->fi_flags & OCFS2_FI_FLAG_SYSFILE)) {
+ status = ocfs2_error(osb->sb,
+ "Inode %llu: system file state is ambiguous\n",
+ (unsigned long long)args->fi_blkno);
+ goto bail;
+ }
if (S_ISCHR(le16_to_cpu(fe->i_mode)) ||
S_ISBLK(le16_to_cpu(fe->i_mode)))
@@ -1520,8 +1524,23 @@ int ocfs2_validate_inode_block(struct super_block *sb,
goto bail;
}
- if (le16_to_cpu(di->i_suballoc_slot) != (u16)OCFS2_INVALID_SLOT &&
- (u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) {
+ /*
+ * Only system inodes created by mkfs.ocfs2 are allocated from the
+ * global allocator and thus legitimately carry OCFS2_INVALID_SLOT.
+ * Regular inodes are always allocated from a per-slot suballocator.
+ * If a regular inode with OCFS2_INVALID_SLOT was accepted here,
+ * deleting it would pass the slot to get_local_system_inode() via
+ * ocfs2_remove_inode() and trigger BUG_ON(slot == OCFS2_INVALID_SLOT).
+ */
+ if (le16_to_cpu(di->i_suballoc_slot) == (u16)OCFS2_INVALID_SLOT) {
+ if (!(le32_to_cpu(di->i_flags) & OCFS2_SYSTEM_FL)) {
+ rc = ocfs2_error(sb,
+ "Invalid dinode %llu: suballoc slot %u for non-system inode\n",
+ (unsigned long long)bh->b_blocknr,
+ le16_to_cpu(di->i_suballoc_slot));
+ goto bail;
+ }
+ } else if ((u32)le16_to_cpu(di->i_suballoc_slot) > OCFS2_SB(sb)->max_slots - 1) {
rc = ocfs2_error(sb, "Invalid dinode %llu: suballoc slot %u\n",
(unsigned long long)bh->b_blocknr,
le16_to_cpu(di->i_suballoc_slot));
diff --git a/fs/ocfs2/journal.c b/fs/ocfs2/journal.c
index d8afbc1a76bb..a3938a03e93b 100644
--- a/fs/ocfs2/journal.c
+++ b/fs/ocfs2/journal.c
@@ -156,7 +156,7 @@ static void ocfs2_queue_replay_slots(struct ocfs2_super *osb,
replay_map->rm_state = REPLAY_DONE;
}
-void ocfs2_free_replay_slots(struct ocfs2_super *osb)
+static void ocfs2_free_replay_slots(struct ocfs2_super *osb)
{
struct ocfs2_replay_map *replay_map = osb->replay_map;
@@ -243,6 +243,7 @@ void ocfs2_recovery_exit(struct ocfs2_super *osb)
/* XXX: Should we bug if there are dirty entries? */
kfree(rm);
+ ocfs2_free_replay_slots(osb);
}
static int __ocfs2_recovery_map_test(struct ocfs2_super *osb,
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index f8b3b2a3d630..19fc920d26b1 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -151,7 +151,6 @@ void ocfs2_recovery_exit(struct ocfs2_super *osb);
void ocfs2_recovery_disable_quota(struct ocfs2_super *osb);
int ocfs2_compute_replay_slots(struct ocfs2_super *osb);
-void ocfs2_free_replay_slots(struct ocfs2_super *osb);
/*
* Journal Control:
* Initialize, Load, Shutdown, Wipe a journal.
diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c
index acce465f45b0..169c576f06bf 100644
--- a/fs/ocfs2/super.c
+++ b/fs/ocfs2/super.c
@@ -1163,15 +1163,23 @@ static int ocfs2_fill_super(struct super_block *sb, struct fs_context *fc)
out_dismount:
atomic_set(&osb->vol_state, VOLUME_DISABLED);
wake_up(&osb->osb_mount_event);
- ocfs2_free_replay_slots(osb);
ocfs2_dismount_volume(sb, 1);
goto out;
out_debugfs:
debugfs_remove_recursive(osb->osb_debug_root);
out_super:
+ /*
+ * A recovery thread launched by a node failure event may still be
+ * waiting for the volume to be mounted. Set VOLUME_DISABLED and
+ * wake it up, then wait for it to exit before osb is freed,
+ * otherwise the kthread would leak and stay blocked on the wait
+ * queue embedded in the freed osb.
+ */
+ atomic_set(&osb->vol_state, VOLUME_DISABLED);
+ wake_up(&osb->osb_mount_event);
ocfs2_release_system_inodes(osb);
- kfree(osb->recovery_map);
+ ocfs2_recovery_exit(osb);
ocfs2_delete_osb(osb);
kfree(osb);
out:
@@ -1768,14 +1776,12 @@ static int ocfs2_mount_volume(struct super_block *sb)
status = ocfs2_truncate_log_init(osb);
if (status < 0) {
mlog_errno(status);
- goto out_check_volume;
+ goto out_system_inodes;
}
ocfs2_super_unlock(osb, 1);
return 0;
-out_check_volume:
- ocfs2_free_replay_slots(osb);
out_system_inodes:
if (osb->local_alloc_state == OCFS2_LA_ENABLED)
ocfs2_shutdown_local_alloc(osb);
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index bfafe059bedf..18ebe3cc64ec 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -561,9 +561,10 @@ static inline const char *ocfs2_xattr_prefix(int name_index)
return handler ? xattr_prefix(handler) : NULL;
}
-static u32 ocfs2_xattr_name_hash(struct inode *inode,
- const char *name,
- int name_len)
+static u32 __ocfs2_xattr_name_hash(struct inode *inode,
+ const char *name,
+ int name_len,
+ bool legacy_signed)
{
/* Get hash value of uuid from super block */
u32 hash = OCFS2_SB(inode->i_sb)->uuid_hash;
@@ -572,13 +573,30 @@ static u32 ocfs2_xattr_name_hash(struct inode *inode,
/* hash extended attribute name */
for (i = 0; i < name_len; i++) {
hash = (hash << OCFS2_HASH_SHIFT) ^
- (hash >> (8*sizeof(hash) - OCFS2_HASH_SHIFT)) ^
- *name++;
+ (hash >> (8*sizeof(hash) - OCFS2_HASH_SHIFT));
+ if (legacy_signed)
+ hash ^= (signed char)name[i];
+ else
+ hash ^= (unsigned char)name[i];
}
return hash;
}
+static u32 ocfs2_xattr_name_hash(struct inode *inode,
+ const char *name,
+ int name_len)
+{
+ return __ocfs2_xattr_name_hash(inode, name, name_len, false);
+}
+
+static u32 ocfs2_xattr_name_hash_signed(struct inode *inode,
+ const char *name,
+ int name_len)
+{
+ return __ocfs2_xattr_name_hash(inode, name, name_len, true);
+}
+
static int ocfs2_xattr_entry_real_size(int name_len, size_t value_len)
{
return namevalue_size(name_len, value_len) +
@@ -2073,12 +2091,17 @@ static void ocfs2_xa_bucket_add_entry(struct ocfs2_xa_loc *loc, u32 name_hash)
}
}
+ /*
+ * Increment xh_count before memmove() so __counted_by_le(xh_count)
+ * includes the new entry in the destination bounds.
+ */
+ le16_add_cpu(&xh->xh_count, 1);
+
if (low != count)
memmove(&xh->xh_entries[low + 1],
&xh->xh_entries[low],
((count - low) * sizeof(struct ocfs2_xattr_entry)));
- le16_add_cpu(&xh->xh_count, 1);
loc->xl_entry = &xh->xh_entries[low];
memset(loc->xl_entry, 0, sizeof(struct ocfs2_xattr_entry));
}
@@ -3376,6 +3399,14 @@ static int ocfs2_calc_xattr_set_need(struct inode *inode,
credits += ocfs2_calc_extend_credits(
inode->i_sb,
&def_xv.xv.xr_list);
+ /*
+ * A value too big to stay inline is only charged as
+ * OCFS2_XATTR_ROOT_SIZE above, so it can still need a
+ * value tree of its own. No xattr block is allocated
+ * on this path, so the tree is all we reserve for.
+ */
+ if (xi->xi_value_len > OCFS2_XATTR_INLINE_SIZE)
+ meta_add += ocfs2_extend_meta_needed(&def_xv.xv.xr_list);
goto out;
}
}
@@ -3476,7 +3507,13 @@ static int ocfs2_calc_xattr_set_need(struct inode *inode,
credits += OCFS2_XATTR_BLOCK_CREATE_CREDITS;
if (xi->xi_value_len > OCFS2_XATTR_INLINE_SIZE) {
struct ocfs2_extent_list *el = &def_xv.xv.xr_list;
- meta_add += ocfs2_extend_meta_needed(el);
+ /*
+ * One block for the xattr block we may have to
+ * allocate, the rest for growing the value tree it
+ * would hold. If ocfs2_xattr_ibody_set() succeeds
+ * instead, that first block goes unused.
+ */
+ meta_add += 1 + ocfs2_extend_meta_needed(el);
credits += ocfs2_calc_extend_credits(inode->i_sb,
el);
} else {
@@ -4168,11 +4205,12 @@ static int ocfs2_xattr_bucket_find(struct inode *inode,
return ret;
}
-static int ocfs2_xattr_index_block_find(struct inode *inode,
- struct buffer_head *root_bh,
- int name_index,
- const char *name,
- struct ocfs2_xattr_search *xs)
+static int __ocfs2_xattr_index_block_find(struct inode *inode,
+ struct buffer_head *root_bh,
+ int name_index,
+ const char *name,
+ u32 name_hash,
+ struct ocfs2_xattr_search *xs)
{
int ret;
struct ocfs2_xattr_block *xb =
@@ -4181,7 +4219,6 @@ static int ocfs2_xattr_index_block_find(struct inode *inode,
struct ocfs2_extent_list *el = &xb_root->xt_list;
u64 p_blkno = 0;
u32 first_hash, num_clusters = 0;
- u32 name_hash = ocfs2_xattr_name_hash(inode, name, strlen(name));
if (le16_to_cpu(el->l_next_free_rec) == 0)
return -ENODATA;
@@ -4212,6 +4249,59 @@ static int ocfs2_xattr_index_block_find(struct inode *inode,
return ret;
}
+static int ocfs2_xattr_index_block_find(struct inode *inode,
+ struct buffer_head *root_bh,
+ int name_index,
+ const char *name,
+ struct ocfs2_xattr_search *xs)
+{
+ u32 name_hash, legacy_hash;
+ int name_len = strlen(name);
+ int ret;
+
+ name_hash = ocfs2_xattr_name_hash(inode, name, name_len);
+
+ ret = __ocfs2_xattr_index_block_find(inode, root_bh, name_index, name,
+ name_hash, xs);
+ if (ret != -ENODATA)
+ return ret;
+
+ /*
+ * Nothing under the current hash. The entry may have been stored by
+ * an older kernel, which sign-extended the name bytes when hashing.
+ * Skip the retry when the two hashes are equal, so that a name made
+ * only of ASCII does not have to walk the tree twice.
+ */
+ legacy_hash = ocfs2_xattr_name_hash_signed(inode, name, name_len);
+ if (legacy_hash == name_hash)
+ return ret;
+
+ /*
+ * A miss still leaves xs->bucket holding the bucket a new entry would
+ * be inserted into, so drop it before searching again.
+ */
+ ocfs2_xattr_bucket_relse(xs->bucket);
+
+ ret = __ocfs2_xattr_index_block_find(inode, root_bh, name_index, name,
+ legacy_hash, xs);
+ if (!ret) {
+ pr_warn_once("ocfs2: xattr tree with signed name hash\n");
+ return ret;
+ }
+ if (ret != -ENODATA)
+ return ret;
+
+ /*
+ * Not under either hash. Restore the unsigned placement, since that
+ * is where a new entry is stored: leaving the bucket where the legacy
+ * hash put it would break the ordering the search relies on.
+ */
+ ocfs2_xattr_bucket_relse(xs->bucket);
+
+ return __ocfs2_xattr_index_block_find(inode, root_bh, name_index, name,
+ name_hash, xs);
+}
+
static int ocfs2_iterate_xattr_buckets(struct inode *inode,
u64 blkno,
u32 clusters,
@@ -5750,16 +5840,14 @@ static int ocfs2_rm_xattr_cluster(struct inode *inode,
/*
* check whether the xattr bucket is filled up with the same hash value.
- * If we want to insert the xattr with the same hash, return -ENOSPC.
- * If we want to insert a xattr with different hash value, go ahead
- * and ocfs2_divide_xattr_bucket will handle this.
+ * If the entry being set carries that same hash, return -ENOSPC, since
+ * ocfs2_divide_xattr_bucket() has no divide position to work with.
+ * Otherwise go ahead and ocfs2_divide_xattr_bucket() will handle this.
*/
-static int ocfs2_check_xattr_bucket_collision(struct inode *inode,
- struct ocfs2_xattr_bucket *bucket,
- const char *name)
+static int ocfs2_check_xattr_bucket_collision(struct ocfs2_xattr_bucket *bucket,
+ u32 name_hash)
{
struct ocfs2_xattr_header *xh = bucket_xh(bucket);
- u32 name_hash = ocfs2_xattr_name_hash(inode, name, strlen(name));
if (name_hash != le32_to_cpu(xh->xh_entries[0].xe_name_hash))
return 0;
@@ -5828,10 +5916,27 @@ static int ocfs2_xattr_set_entry_index_block(struct inode *inode,
struct ocfs2_xattr_search *xs,
struct ocfs2_xattr_set_ctxt *ctxt)
{
+ u32 name_hash;
int ret;
trace_ocfs2_xattr_set_entry_index_block(xi->xi_name);
+ /*
+ * An existing entry keeps the hash it was stored under, and that is
+ * the hash a split has to work with. A new entry is stored under the
+ * unsigned one, which is what ocfs2_xa_add_entry() will write.
+ *
+ * Read it before the set below. Filling the bucket may defragment it,
+ * and ocfs2_defrag_xattr_bucket() re-sorts the entry array, so
+ * afterwards xs->here only denotes whichever entry the sort left in
+ * that slot.
+ */
+ if (xs->not_found)
+ name_hash = ocfs2_xattr_name_hash(inode, xi->xi_name,
+ xi->xi_name_len);
+ else
+ name_hash = le32_to_cpu(xs->here->xe_name_hash);
+
ret = ocfs2_xattr_set_entry_bucket(inode, xi, xs, ctxt);
if (!ret)
goto out;
@@ -5848,9 +5953,7 @@ static int ocfs2_xattr_set_entry_index_block(struct inode *inode,
* one bucket's worth, so check it here whether we need to
* add a new bucket for the insert.
*/
- ret = ocfs2_check_xattr_bucket_collision(inode,
- xs->bucket,
- xi->xi_name);
+ ret = ocfs2_check_xattr_bucket_collision(xs->bucket, name_hash);
if (ret) {
mlog_errno(ret);
goto out;
diff --git a/fs/pidfs.c b/fs/pidfs.c
index a6a643f15d08..1bd130eb909a 100644
--- a/fs/pidfs.c
+++ b/fs/pidfs.c
@@ -362,6 +362,12 @@ static __u32 pidfs_coredump_mask(enum task_dumpable dumpable)
PIDFD_INFO_COREDUMP_SIGNAL | \
PIDFD_INFO_COREDUMP_CODE)
+static bool pidfs_task_owns_pid(struct task_struct *task, struct pid *pid)
+{
+ guard(rcu)();
+ return pid_task(pid, PIDTYPE_PID) == task;
+}
+
static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)
{
struct pidfd_info __user *uinfo = (struct pidfd_info __user *)arg;
@@ -392,6 +398,16 @@ static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)
return -EREMOTE;
attr = READ_ONCE(pid->attr);
+again:
+ /*
+ * Look for the task first. pidfs_exit() records the exit information
+ * before the task is unhashed, so if the task is gone the information
+ * is there.
+ */
+ task = get_pid_task(pid, PIDTYPE_PID);
+ if (!task)
+ smp_rmb(); /* pairs with smp_mb__after_atomic() in pidfs_exit() */
+
if (mask & PIDFD_INFO_EXIT) {
if (test_bit(PIDFS_ATTR_BIT_EXIT, &attr->attr_mask)) {
smp_rmb();
@@ -414,7 +430,6 @@ static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)
}
}
- task = get_pid_task(pid, PIDTYPE_PID);
if (!task) {
/*
* If the task has already been reaped, only exit
@@ -468,16 +483,27 @@ static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)
* task might return as soon as the copy_to_user finishes, but that's ok
* and userspace expects that might happen and can act accordingly, so
* this is just best-effort. What we can do however is checking that all
- * the fields are set correctly, or return ESRCH to avoid providing
- * incomplete information. */
+ * the fields are set correctly, to avoid providing incomplete
+ * information.
+ */
kinfo.ppid = task_ppid_vnr(task);
kinfo.tgid = task_tgid_vnr(task);
- kinfo.pid = task_pid_vnr(task);
+ kinfo.pid = pid_vnr(pid);
kinfo.mask |= PIDFD_INFO_PID;
- if (kinfo.pid == 0 || kinfo.tgid == 0)
- return -ESRCH;
+ /*
+ * The task was reaped, or lost @pid to a sub-thread that called exec,
+ * while we were looking at it. Start over: there is exit information
+ * now, or a new task. Don't go by ->thread_pid, exchange_tids()
+ * updates it only after it has moved @pid.
+ */
+ if (!pidfs_task_owns_pid(task, pid) || kinfo.tgid == 0) {
+ put_task_struct(task);
+ task = NULL;
+ memset(&kinfo, 0, sizeof(kinfo));
+ goto again;
+ }
copy_out:
if (mask & PIDFD_INFO_SUPPORTED_MASK) {
@@ -790,6 +816,11 @@ void pidfs_exit(struct task_struct *tsk)
/* Ensure that PIDFD_GET_INFO sees either all or nothing. */
smp_wmb();
set_bit(PIDFS_ATTR_BIT_EXIT, &attr->attr_mask);
+ /*
+ * Whoever sees that the task is unhashed must see the bit, see
+ * pidfd_info(). Taking tasklist_lock does not order the two stores.
+ */
+ smp_mb__after_atomic();
}
#ifdef CONFIG_COREDUMP
diff --git a/fs/proc/generic.c b/fs/proc/generic.c
index 26086a283672..65daaf5f4c68 100644
--- a/fs/proc/generic.c
+++ b/fs/proc/generic.c
@@ -256,9 +256,10 @@ struct dentry *proc_lookup_de(struct inode *dir, struct dentry *dentry,
inode = proc_get_inode(dir->i_sb, de);
if (!inode)
return ERR_PTR(-ENOMEM);
+ /* never reused and so never a mountpoint for long */
if (de->flags & PROC_ENTRY_FORCE_LOOKUP)
- return d_splice_alias_ops(inode, dentry,
- &proc_net_dentry_ops);
+ return proc_splice_unmountable(inode, dentry,
+ &proc_net_dentry_ops);
return d_splice_alias_ops(inode, dentry,
&proc_misc_dentry_ops);
}
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index dd025474b0fc..a2db957aa799 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1786,7 +1786,9 @@ static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
if (!pmd_present(*pmd))
goto out;
- folio = pmd_folio(*pmd);
+ folio = vm_normal_folio_pmd(vma, addr, *pmd);
+ if (!folio)
+ goto out;
/* Clear accessed and referenced bits. */
pmdp_test_and_clear_young(vma, addr, pmd);
@@ -2106,7 +2108,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigned long addr,
goto populate_pagemap;
if (pmd_present(pmd)) {
- page = pmd_page(pmd);
+ page = vm_normal_page_pmd(vma, addr, pmd);
flags |= PM_PRESENT;
if (pmd_soft_dirty(pmd))
diff --git a/fs/read_write.c b/fs/read_write.c
index e8c14e2760b2..ccaaa454dce0 100644
--- a/fs/read_write.c
+++ b/fs/read_write.c
@@ -1531,9 +1531,11 @@ static int generic_copy_file_checks(struct file *file_in, loff_t pos_in,
else
count = min(count, size_in - (uint64_t)pos_in);
- ret = generic_write_check_limits(file_out, pos_out, &count);
- if (ret)
- return ret;
+ if (count) {
+ ret = generic_write_check_limits(file_out, pos_out, &count);
+ if (ret)
+ return ret;
+ }
/* Don't allow overlapped copying within the same file. */
if (inode_in == inode_out &&
diff --git a/fs/signalfd.c b/fs/signalfd.c
index dff53745e352..22bc0870a824 100644
--- a/fs/signalfd.c
+++ b/fs/signalfd.c
@@ -48,17 +48,30 @@ static int signalfd_release(struct inode *inode, struct file *file)
return 0;
}
+static void refine_sigmask(struct signalfd_ctx *ctx, sigset_t *sigmask)
+{
+ struct k_sigaction *k = current->sighand->action;
+ int n;
+
+ *sigmask = ctx->sigmask;
+ for (n = 1; n <= _NSIG; ++n, ++k) {
+ if (k->sa.sa_flags & SA_IMMUTABLE)
+ sigaddset(sigmask, n);
+ }
+}
+
static __poll_t signalfd_poll(struct file *file, poll_table *wait)
{
struct signalfd_ctx *ctx = file->private_data;
__poll_t events = 0;
+ sigset_t sigmask;
poll_wait(file, ¤t->sighand->signalfd_wqh, wait);
spin_lock_irq(¤t->sighand->siglock);
- if (next_signal(¤t->pending, &ctx->sigmask) ||
- next_signal(¤t->signal->shared_pending,
- &ctx->sigmask))
+ refine_sigmask(ctx, &sigmask);
+ if (next_signal(¤t->pending, &sigmask) ||
+ next_signal(¤t->signal->shared_pending, &sigmask))
events |= EPOLLIN;
spin_unlock_irq(¤t->sighand->siglock);
@@ -155,11 +168,13 @@ static ssize_t signalfd_dequeue(struct signalfd_ctx *ctx, kernel_siginfo_t *info
int nonblock)
{
enum pid_type type;
- ssize_t ret;
DECLARE_WAITQUEUE(wait, current);
+ sigset_t sigmask;
+ ssize_t ret;
spin_lock_irq(¤t->sighand->siglock);
- ret = dequeue_signal(&ctx->sigmask, info, &type);
+ refine_sigmask(ctx, &sigmask);
+ ret = dequeue_signal(&sigmask, info, &type);
switch (ret) {
case 0:
if (!nonblock)
@@ -174,7 +189,7 @@ static ssize_t signalfd_dequeue(struct signalfd_ctx *ctx, kernel_siginfo_t *info
add_wait_queue(¤t->sighand->signalfd_wqh, &wait);
for (;;) {
set_current_state(TASK_INTERRUPTIBLE);
- ret = dequeue_signal(&ctx->sigmask, info, &type);
+ ret = dequeue_signal(&sigmask, info, &type);
if (ret != 0)
break;
if (signal_pending(current)) {
@@ -184,6 +199,7 @@ static ssize_t signalfd_dequeue(struct signalfd_ctx *ctx, kernel_siginfo_t *info
spin_unlock_irq(¤t->sighand->siglock);
schedule();
spin_lock_irq(¤t->sighand->siglock);
+ refine_sigmask(ctx, &sigmask);
}
spin_unlock_irq(¤t->sighand->siglock);
diff --git a/fs/splice.c b/fs/splice.c
index 9d8f63e2fd1a..96378d3e6558 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -1349,10 +1349,10 @@ ssize_t do_splice(struct file *in, loff_t *off_in, struct file *out,
ret = do_splice_from(ipipe, out, &offset, len, flags);
file_end_write(out);
- if (!off_out)
- out->f_pos = offset;
- else
+ if (off_out)
*off_out = offset;
+ else if (ret >= 0)
+ out->f_pos = offset;
} else if (opipe) {
if (off_out)
return -ESPIPE;
@@ -1373,10 +1373,10 @@ ssize_t do_splice(struct file *in, loff_t *off_in, struct file *out,
ret = splice_file_to_pipe(in, opipe, &offset, len, flags);
- if (!off_in)
- in->f_pos = offset;
- else
+ if (off_in)
*off_in = offset;
+ else if (ret >= 0)
+ in->f_pos = offset;
} else {
ret = -EINVAL;
}
diff --git a/fs/stat.c b/fs/stat.c
index c461c3054234..e03954c36b33 100644
--- a/fs/stat.c
+++ b/fs/stat.c
@@ -203,8 +203,12 @@ int vfs_getattr_nosec(const struct path *path, struct kstat *stat,
if (IS_DAX(inode))
stat->attributes |= STATX_ATTR_DAX;
+ if (IS_VERITY(inode))
+ stat->attributes |= STATX_ATTR_VERITY;
+
stat->attributes_mask |= (STATX_ATTR_AUTOMOUNT |
- STATX_ATTR_DAX);
+ STATX_ATTR_DAX |
+ STATX_ATTR_VERITY);
idmap = mnt_idmap(path->mnt);
if (inode->i_op->getattr) {
diff --git a/fs/udf/inode.c b/fs/udf/inode.c
index 2f04c4f0e1b8..49673b555c62 100644
--- a/fs/udf/inode.c
+++ b/fs/udf/inode.c
@@ -1336,6 +1336,26 @@ int udf_setsize(struct inode *inode, loff_t newsize)
return err;
}
+/*
+ * Verify validity of struct deviceSpec on disk. udf_get_extendedattr() has
+ * already verified the generic header and made sure attribute fits in the
+ * inode so we just have to make sure attribute space is large enough for
+ * deviceSpec struct and required impUse information.
+ */
+static bool udf_device_spec_valid(struct deviceSpec *dsea)
+{
+ u32 attr_length, imp_use_length;
+
+ attr_length = le32_to_cpu(dsea->attrLength);
+ imp_use_length = le32_to_cpu(dsea->impUseLength);
+ if (attr_length < sizeof(struct deviceSpec) ||
+ imp_use_length < sizeof(struct regid) ||
+ imp_use_length > attr_length - sizeof(struct deviceSpec))
+ return false;
+
+ return true;
+}
+
/*
* Maximum length of linked list formed by ICB hierarchy. The chosen number is
* arbitrary - just that we hopefully don't limit any real use of rewritten
@@ -1654,13 +1674,12 @@ static int udf_read_inode(struct inode *inode, bool hidden_inode)
if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode)) {
struct deviceSpec *dsea =
(struct deviceSpec *)udf_get_extendedattr(inode, 12, 1);
- if (dsea) {
- init_special_inode(inode, inode->i_mode,
+
+ if (!dsea || !udf_device_spec_valid(dsea))
+ goto out;
+ init_special_inode(inode, inode->i_mode,
MKDEV(le32_to_cpu(dsea->majorDeviceIdent),
le32_to_cpu(dsea->minorDeviceIdent)));
- /* Developer ID ??? */
- } else
- goto out;
}
ret = 0;
out:
@@ -1757,6 +1776,7 @@ int udf_write_inode(struct inode *inode, struct writeback_control *wbc)
struct udf_sb_info *sbi = UDF_SB(inode->i_sb);
unsigned char blocksize_bits = inode->i_sb->s_blocksize_bits;
struct udf_inode_info *iinfo = UDF_I(inode);
+ int err;
bh = sb_getblk(inode->i_sb,
udf_get_lb_pblock(inode->i_sb, &iinfo->i_location, 0));
@@ -1821,6 +1841,10 @@ int udf_write_inode(struct inode *inode, struct writeback_control *wbc)
udf_add_extendedattr(inode,
sizeof(struct deviceSpec) +
sizeof(struct regid), 12, 0x3);
+ if (!dsea) {
+ err = -ENOSPC;
+ goto out_unlock;
+ }
dsea->attrType = cpu_to_le32(12);
dsea->attrSubtype = 1;
dsea->attrLength = cpu_to_le32(
@@ -1828,6 +1852,10 @@ int udf_write_inode(struct inode *inode, struct writeback_control *wbc)
sizeof(struct regid));
dsea->impUseLength = cpu_to_le32(sizeof(struct regid));
}
+ if (!udf_device_spec_valid(dsea)) {
+ err = -EFSCORRUPTED;
+ goto out_unlock;
+ }
eid = (struct regid *)dsea->impUse;
memset(eid, 0, sizeof(*eid));
strcpy(eid->ident, UDF_ID_DEVELOPER);
@@ -1962,6 +1990,11 @@ int udf_write_inode(struct inode *inode, struct writeback_control *wbc)
set_inode_metadata_writeback(inode);
return 0;
+
+out_unlock:
+ unlock_buffer(bh);
+ brelse(bh);
+ return err;
}
struct inode *__udf_iget(struct super_block *sb, struct kernel_lb_addr *ino,
diff --git a/fs/udf/super.c b/fs/udf/super.c
index 2ba5973ef4dd..5351755aca3e 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1617,7 +1617,10 @@ static bool udf_lvid_valid(struct super_block *sb,
parts = le32_to_cpu(lvid->numOfPartitions);
impuselen = le32_to_cpu(lvid->lengthOfImpUse);
- if (parts >= sb->s_blocksize || impuselen >= sb->s_blocksize ||
+ if (sizeof(struct logicalVolIntegrityDescImpUse) > impuselen ||
+ impuselen >= sb->s_blocksize)
+ return false;
+ if (parts >= sb->s_blocksize ||
sizeof(struct logicalVolIntegrityDesc) + impuselen +
2 * parts * sizeof(u32) > sb->s_blocksize)
return false;
diff --git a/fs/verity/enable.c b/fs/verity/enable.c
index 64fccb526ccd..82032c87e5a3 100644
--- a/fs/verity/enable.c
+++ b/fs/verity/enable.c
@@ -298,6 +298,14 @@ static int enable_verity(struct file *filp,
} else if (WARN_ON_ONCE(!IS_VERITY(inode))) {
fsverity_remove_info(vi);
err = -EINVAL;
+ } else {
+ /*
+ * Now that verity is enabled, report the signature (if any) to
+ * LSMs. We tolerate that a process racing with this ioctl can
+ * briefly see verity enabled with no signature reported yet.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
}
out:
kfree(params.hashstate);
diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h
index 6f85f63861aa..2c41428d484b 100644
--- a/fs/verity/fsverity_private.h
+++ b/fs/verity/fsverity_private.h
@@ -160,6 +160,8 @@ void __init fsverity_init_info_cache(void);
extern int fsverity_require_signatures;
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size);
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size);
void __init fsverity_init_signature(void);
#else /* !CONFIG_FS_VERITY_BUILTIN_SIGNATURES */
@@ -170,6 +172,12 @@ fsverity_verify_signature(const struct fsverity_info *vi,
return 0;
}
+static inline void fsverity_report_valid_sig(const struct inode *inode,
+ const u8 *signature,
+ size_t sig_size)
+{
+}
+
static inline void fsverity_init_signature(void)
{
}
diff --git a/fs/verity/measure.c b/fs/verity/measure.c
index 68dfccb69772..0c75616dca01 100644
--- a/fs/verity/measure.c
+++ b/fs/verity/measure.c
@@ -31,6 +31,16 @@ int fsverity_ioctl_measure(struct file *filp, void __user *_uarg)
const struct fsverity_hash_alg *hash_alg;
struct fsverity_digest arg;
+ /*
+ * This ioctl returns a hash of the file contents, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
diff --git a/fs/verity/open.c b/fs/verity/open.c
index 52c8cd014921..69d7512e4842 100644
--- a/fs/verity/open.c
+++ b/fs/verity/open.c
@@ -270,12 +270,6 @@ static bool validate_fsverity_descriptor(struct inode *inode,
const struct fsverity_descriptor *desc,
size_t desc_size)
{
- if (desc_size < sizeof(*desc)) {
- fsverity_err(inode, "Unrecognized descriptor size: %zu bytes",
- desc_size);
- return false;
- }
-
if (desc->version != 1) {
fsverity_err(inode, "Unrecognized descriptor version: %u",
desc->version);
@@ -314,31 +308,37 @@ static bool validate_fsverity_descriptor(struct inode *inode,
int fsverity_get_descriptor(struct inode *inode,
struct fsverity_descriptor **desc_ret)
{
- int res;
+ int size, res;
struct fsverity_descriptor *desc;
- res = inode->i_sb->s_vop->get_verity_descriptor(inode, NULL, 0);
- if (res < 0) {
- fsverity_err(inode,
- "Error %d getting verity descriptor size", res);
- return res;
+ size = inode->i_sb->s_vop->get_verity_descriptor(inode, NULL, 0);
+ if (size < 0) {
+ fsverity_err(inode, "Error %d getting verity descriptor size",
+ size);
+ return size;
}
- if (res > FS_VERITY_MAX_DESCRIPTOR_SIZE) {
- fsverity_err(inode, "Verity descriptor is too large (%d bytes)",
- res);
+ if (size < sizeof(*desc) || size > FS_VERITY_MAX_DESCRIPTOR_SIZE) {
+ fsverity_err(inode,
+ "Verity descriptor has invalid size (%d bytes)",
+ size);
return -EMSGSIZE;
}
- desc = kmalloc(res, GFP_KERNEL);
+ desc = kmalloc(size, GFP_KERNEL);
if (!desc)
return -ENOMEM;
- res = inode->i_sb->s_vop->get_verity_descriptor(inode, desc, res);
+ res = inode->i_sb->s_vop->get_verity_descriptor(inode, desc, size);
if (res < 0) {
fsverity_err(inode, "Error %d reading verity descriptor", res);
kfree(desc);
return res;
}
-
- if (!validate_fsverity_descriptor(inode, desc, res)) {
+ if (res != size) {
+ fsverity_err(inode, "Verity descriptor size changed (%d => %d)",
+ size, res);
+ kfree(desc);
+ return -EINVAL;
+ }
+ if (!validate_fsverity_descriptor(inode, desc, size)) {
kfree(desc);
return -EINVAL;
}
@@ -366,6 +366,15 @@ static int ensure_verity_info(struct inode *inode)
goto out_free_desc;
}
+ /*
+ * Now that the signature (if any) has been verified, report it to LSMs.
+ * Do this before inserting the fsverity_info into the hashtable, so
+ * that anyone who concurrently retrieves the fsverity_info for the same
+ * inode is guaranteed to see the LSM's signature status populated.
+ */
+ fsverity_report_valid_sig(inode, desc->signature,
+ le32_to_cpu(desc->sig_size));
+
/*
* Multiple tasks may race to set the inode's verity info, in which case
* we might find an existing fsverity_info in the hash table.
diff --git a/fs/verity/read_metadata.c b/fs/verity/read_metadata.c
index b4c0892430cd..6694149f7b46 100644
--- a/fs/verity/read_metadata.c
+++ b/fs/verity/read_metadata.c
@@ -166,13 +166,19 @@ int fsverity_ioctl_read_metadata(struct file *filp, const void __user *uarg)
int length;
void __user *buf;
+ /*
+ * This ioctl returns file contents block hashes, so it should require
+ * read permission. Disallow it on ioctl-only (i.e., nonstandard access
+ * mode 3) fds, which can in some cases be acquired without passing all
+ * LSM read-permission checks. Also disallow write-only fds, which can
+ * refer only to non-verity files which would fail with ENODATA anyway.
+ */
+ if (!(filp->f_mode & FMODE_READ))
+ return -EBADF;
+
vi = fsverity_get_info(inode);
if (!vi)
return -ENODATA; /* not a verity file */
- /*
- * Note that we don't have to explicitly check that the file is open for
- * reading, since verity files can only be opened for reading.
- */
if (copy_from_user(&arg, uarg, sizeof(arg)))
return -EFAULT;
diff --git a/fs/verity/signature.c b/fs/verity/signature.c
index 0302a4e506ec..212e873c8499 100644
--- a/fs/verity/signature.c
+++ b/fs/verity/signature.c
@@ -42,13 +42,16 @@ static struct key *fsverity_keyring;
* @sig_size: size of signature in bytes, or 0 if no signature
*
* If the file includes a signature of its fs-verity file digest, verify it
- * against the certificates in the fs-verity keyring. Note that signatures
- * are verified regardless of the state of the 'fsverity_require_signatures'
- * variable and the LSM subsystem relies on this behavior to help enforce
- * file integrity policies. Please discuss changes with the LSM list
- * (thank you!).
+ * against the certificates in the fs-verity keyring.
*
- * Return: 0 on success (signature valid or not required); -errno on failure
+ * Note that signatures are verified regardless of the state of the
+ * 'fsverity_require_signatures' variable, and the LSM subsystem relies on this
+ * behavior to help enforce file integrity policies. (See
+ * fsverity_report_valid_sig(), which assumes that the signature, if present,
+ * was verified here.) Please discuss changes with the LSM list (thank you!).
+ *
+ * Return: 0 if the signature is present and valid, or absent and not required;
+ * -errno on failure
*/
int fsverity_verify_signature(const struct fsverity_info *vi,
const u8 *signature, size_t sig_size)
@@ -111,18 +114,40 @@ int fsverity_verify_signature(const struct fsverity_info *vi,
return err;
}
- err = security_inode_setintegrity(inode,
- LSM_INT_FSVERITY_BUILTINSIG_VALID,
- signature,
- sig_size);
+ return 0;
+}
- if (err) {
+/*
+ * If the signature exists, this tells the LSM that it was verified.
+ *
+ * fsverity_verify_signature() must have already been called with the same
+ * signature, *and* the inode must have verity enabled with the signature
+ * matching it. If called from FS_IOC_ENABLE_VERITY, the latter means that we
+ * must be past the point of no return, i.e. verity won't fail to be enabled at
+ * a later point. This is critical, since LSMs must be told about the signature
+ * only when the verity status is permanent in the in-core inode.
+ */
+void fsverity_report_valid_sig(const struct inode *inode, const u8 *signature,
+ size_t sig_size)
+{
+ int err;
+
+ if (WARN_ON_ONCE(!IS_VERITY(inode)))
+ return;
+
+ if (sig_size == 0)
+ return;
+
+ err = security_inode_setintegrity(
+ inode, LSM_INT_FSVERITY_BUILTINSIG_VALID, signature, sig_size);
+ if (err)
fsverity_err(inode, "Error %d exposing file signature to LSMs",
err);
- return err;
- }
-
- return 0;
+ /*
+ * Besides the logging above, ignore errors here. The LSM hook just
+ * records the signature and doesn't perform access control. And in the
+ * case of FS_IOC_ENABLE_VERITY verity was already enabled.
+ */
}
void __init fsverity_init_signature(void)
diff --git a/fs/xfs/scrub/xfarray.c b/fs/xfs/scrub/xfarray.c
index dc18c6310cbd..24e9837211f3 100644
--- a/fs/xfs/scrub/xfarray.c
+++ b/fs/xfs/scrub/xfarray.c
@@ -794,6 +794,46 @@ xfarray_sort_scan_done(
si->folio = NULL;
}
+static int
+xfarray_sort_load_folio(
+ struct xfarray_sortinfo *si,
+ xfarray_idx_t idx,
+ loff_t idx_pos)
+{
+ struct folio *folio;
+ loff_t next_pos;
+
+ folio = xfile_get_folio(si->array->xfile, idx_pos, si->array->obj_size,
+ XFILE_ALLOC);
+ if (IS_ERR(folio))
+ return PTR_ERR(folio);
+ si->folio = folio;
+
+ /* No folio? Get the caller to read into the scratchpad. */
+ if (!si->folio)
+ return 0;
+
+ si->first_folio_idx = xfarray_idx(si->array,
+ folio_pos(si->folio) + si->array->obj_size - 1);
+
+ next_pos = folio_next_pos(si->folio);
+ si->last_folio_idx = xfarray_idx(si->array, next_pos - 1);
+ if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos)
+ si->last_folio_idx--;
+
+ /*
+ * If this folio still doesn't cover the desired element, it must cross
+ * a folio boundary. Get the caller to read into the scratchpad.
+ */
+ if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ xfarray_sort_scan_done(si);
+ return 0;
+ }
+
+ trace_xfarray_sort_scan(si, idx);
+ return 0;
+}
+
/*
* Cache the folio backing the start of the given array element. If the array
* element is contained entirely within the folio, return a pointer to the
@@ -819,33 +859,18 @@ xfarray_sort_scan(
(idx < si->first_folio_idx || idx > si->last_folio_idx))
xfarray_sort_scan_done(si);
- /* Grab the first folio that backs this array element. */
+ /* Grab the folio that backs this array element. */
if (!si->folio) {
- struct folio *folio;
- loff_t next_pos;
-
- folio = xfile_get_folio(si->array->xfile, idx_pos,
- si->array->obj_size, XFILE_ALLOC);
- if (IS_ERR(folio))
- return PTR_ERR(folio);
- si->folio = folio;
-
- si->first_folio_idx = xfarray_idx(si->array,
- folio_pos(si->folio) + si->array->obj_size - 1);
-
- next_pos = folio_next_pos(si->folio);
- si->last_folio_idx = xfarray_idx(si->array, next_pos - 1);
- if (xfarray_pos(si->array, si->last_folio_idx + 1) > next_pos)
- si->last_folio_idx--;
-
- trace_xfarray_sort_scan(si, idx);
+ error = xfarray_sort_load_folio(si, idx, idx_pos);
+ if (error)
+ return error;
}
/*
- * If this folio still doesn't cover the desired element, it must cross
- * a folio boundary. Read into the scratchpad and we're done.
+ * If we don't have a folio mapping the entire array element, read into
+ * the scratchpad and we're done.
*/
- if (idx < si->first_folio_idx || idx > si->last_folio_idx) {
+ if (!si->folio) {
void *temp = xfarray_scratch(si->array);
error = xfile_load(si->array->xfile, temp, si->array->obj_size,
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index bc6f2fdd7ad3..80f96ea59f1a 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -195,4 +195,19 @@ static inline void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)
WRITE_ONCE(*ctxt->offset.vm_offset, offset);
}
+/*
+ * CVAL to program when the hardware won't apply the timer's offset, so that
+ * its compare against the raw counter matches the guest's at 'now'. An
+ * expired timer stays expired; one past the wrap never fires.
+ */
+static inline u64 timer_apply_offset(u64 cval, u64 offset, u64 now)
+{
+ u64 hw = cval + offset;
+
+ if (now - offset >= cval)
+ return 0;
+
+ return hw > now ? hw : U64_MAX;
+}
+
#endif
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 086b7578e5f3..5c3a0154becc 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -256,6 +256,7 @@ struct vgic_irq {
bool active:1;
bool hw:1; /* Tied to HW IRQ */
bool on_lr:1; /* Present in a CPU LR */
+ bool mapped:1; /* LPIs: mapped by an ITE */
refcount_t refcount; /* Used for LPIs */
u32 hwintid; /* HW INTID number */
unsigned int host_irq; /* linux irq corresponding to hwintid */
diff --git a/include/linux/fault-inject.h b/include/linux/fault-inject.h
index 58fd14c82270..5c74748a53f3 100644
--- a/include/linux/fault-inject.h
+++ b/include/linux/fault-inject.h
@@ -18,6 +18,13 @@ enum fault_flags {
#include <linux/configfs.h>
#include <linux/ratelimit.h>
+/*
+ * Length of the debugfs directory name embedded in struct fault_attr.
+ * Chosen to accommodate every in-tree caller of fault_create_debugfs_attr()
+ * (the longest is "fail_dma_array_full", 19 chars) with generous headroom.
+ */
+#define FAULT_ATTR_DNAME_LEN 64
+
/*
* For explanation of the elements of this struct, see
* Documentation/fault-injection/fault-injection.rst
@@ -37,7 +44,7 @@ struct fault_attr {
unsigned long count;
struct ratelimit_state ratelimit_state;
- struct dentry *dname;
+ char dname[FAULT_ATTR_DNAME_LEN];
};
#define FAULT_ATTR_INITIALIZER { \
@@ -47,7 +54,6 @@ struct fault_attr {
.stacktrace_depth = 32, \
.ratelimit_state = RATELIMIT_STATE_INIT_DISABLED, \
.verbose = 2, \
- .dname = NULL, \
}
#define DECLARE_FAULT_ATTR(name) struct fault_attr name = FAULT_ATTR_INITIALIZER
diff --git a/include/linux/fs.h b/include/linux/fs.h
index deb411e86661..f155984f5bd0 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -815,7 +815,9 @@ struct inode {
/* 32-bit hole */
struct rw_semaphore i_rwsem;
- unsigned long dirtied_when; /* jiffies of first dirtying */
+ /* Timestamp for writeback list ordering and age checks (jiffies). */
+ unsigned long dirtied_when;
+ /* Start time for deciding when lazytime updates need writeback (jiffies). */
unsigned long dirtied_time_when;
struct hlist_node i_hash;
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 16c4c4caa126..45ada75dc04e 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -699,7 +699,8 @@ enum hugetlb_alloc_flag {
#define HUGETLB_ALLOC_USE_GLOBAL_RESERVATIONS BIT(HUGETLB_ALLOC_USE_GLOBAL_RESERVATIONS_BIT)
struct folio *hugetlb_alloc_folio(struct hstate *h,
- struct mempolicy_interpreted *mpoli, u8 alloc_flags);
+ struct mempolicy_interpreted *mpoli, struct mm_struct *mm,
+ u8 alloc_flags);
struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
unsigned long addr, bool cow_from_owner);
struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid,
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 48bde275968d..3bd74d79f812 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -1147,6 +1147,7 @@ extern struct ata_host *ata_host_alloc_pinfo(struct device *dev,
extern void ata_host_get(struct ata_host *host);
extern void ata_host_put(struct ata_host *host);
extern int ata_host_start(struct ata_host *host);
+void ata_host_undo_start(struct ata_host *host);
extern int ata_host_register(struct ata_host *host,
const struct scsi_host_template *sht);
extern int ata_host_activate(struct ata_host *host, int irq,
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 7d1c0ce189a8..6d1c9e9a28d9 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -662,7 +662,8 @@ static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm,
return __mem_cgroup_charge(folio, mm, gfp);
}
-int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp);
+int mem_cgroup_charge_hugetlb(struct folio *folio, struct mm_struct *mm,
+ gfp_t gfp);
int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id,
struct mm_struct *mm, gfp_t gfp);
@@ -957,6 +958,9 @@ unsigned long lruvec_page_state_monotonic(struct lruvec *lruvec,
unsigned long lruvec_page_state_local(struct lruvec *lruvec,
enum node_stat_item idx);
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val);
+
void mem_cgroup_flush_stats(struct mem_cgroup *memcg);
void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg);
@@ -1156,9 +1160,10 @@ static inline int mem_cgroup_charge(struct folio *folio,
return 0;
}
-static inline int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp)
+static inline int mem_cgroup_charge_hugetlb(struct folio *folio,
+ struct mm_struct *mm, gfp_t gfp)
{
- return 0;
+ return 0;
}
static inline int mem_cgroup_swapin_charge_folio(struct folio *folio,
@@ -1418,6 +1423,11 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,
return node_page_state(lruvec_pgdat(lruvec), idx);
}
+static inline void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val)
+{
+}
+
static inline void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
{
}
@@ -1890,6 +1900,16 @@ static inline bool memcg_is_dying(struct mem_cgroup *memcg)
}
#endif /* CONFIG_MEMCG */
+#if defined(CONFIG_MEMCG) && defined(CONFIG_LRU_GEN)
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg);
+#else
+static inline void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+}
+#endif
+
#if defined(CONFIG_MEMCG) && defined(CONFIG_ZSWAP)
bool obj_cgroup_may_zswap(struct obj_cgroup *objcg);
void obj_cgroup_charge_zswap(struct obj_cgroup *objcg, size_t size);
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 3cff2174dc03..3e64af0b932d 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2841,6 +2841,13 @@ static inline void *netdev_priv(const struct net_device *dev)
*/
static inline struct net_device *netdev_from_priv(const void *priv)
{
+ /*
+ * When GCC can see that @priv came from netdev_priv(), it can
+ * compute an object size of 0 for the net_device and its members,
+ * which makes FORTIFY_SOURCE fail valid accesses through the result:
+ * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=127875
+ */
+ OPTIMIZER_HIDE_VAR(priv);
return container_of(priv, struct net_device, priv);
}
diff --git a/include/linux/panic.h b/include/linux/panic.h
index f1dd417e54b2..a9128bf3c168 100644
--- a/include/linux/panic.h
+++ b/include/linux/panic.h
@@ -13,7 +13,8 @@ __printf(1, 2)
void panic(const char *fmt, ...) __noreturn __cold;
__printf(1, 0)
void vpanic(const char *fmt, va_list args) __noreturn __cold;
-void nmi_panic(struct pt_regs *regs, const char *msg);
+__printf(2, 3)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...);
void check_panic_on_warn(const char *origin);
extern void oops_enter(void);
extern void oops_exit(void);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 5ffcf5ce8c96..272378dd8973 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1478,7 +1478,7 @@ int pcie_set_mps(struct pci_dev *dev, int mps);
u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev,
enum pci_bus_speed *speed,
enum pcie_link_width *width);
-int pcie_link_speed_mbps(struct pci_dev *pdev);
+int pcie_link_bandwidth_mbps(struct pci_dev *pdev);
void pcie_print_link_status(struct pci_dev *dev);
int pcie_reset_flr(struct pci_dev *dev, bool probe);
int pcie_flr(struct pci_dev *dev);
diff --git a/include/linux/rtc.h b/include/linux/rtc.h
index c09fc22819d0..0f43b5027235 100644
--- a/include/linux/rtc.h
+++ b/include/linux/rtc.h
@@ -13,6 +13,7 @@
#define _LINUX_RTC_H_
+#include <linux/cleanup.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/nvmem-provider.h>
@@ -201,6 +202,7 @@ extern void rtc_update_irq(struct rtc_device *rtc,
extern struct rtc_device *rtc_class_open(const char *name);
extern void rtc_class_close(struct rtc_device *rtc);
+DEFINE_FREE(rtc_class_close, struct rtc_device *, if (_T) rtc_class_close(_T));
extern int rtc_irq_set_state(struct rtc_device *rtc, int enabled);
extern int rtc_irq_set_freq(struct rtc_device *rtc, int freq);
diff --git a/include/math-emu/op-common.h b/include/math-emu/op-common.h
index 8ce066c035cf..1d1ce5c08efc 100644
--- a/include/math-emu/op-common.h
+++ b/include/math-emu/op-common.h
@@ -135,6 +135,24 @@ do { \
else \
{ \
/* we've got a denormalized number */ \
+ int _FP_PACK_CANONICAL_is_tiny = 1; \
+ if (_FP_TININESS_AFTER_ROUNDING && X##_e == 0) \
+ { \
+ /* Architectures that detect tininess after rounding \
+ only signal underflow if the result is still \
+ subnormal once rounded as if the exponent range \
+ were unbounded. Round a copy to find out. */ \
+ FP_DECL_##fs(_FP_PACK_CANONICAL_T); \
+ /* The class field is not used by the rounding below, \
+ and is unused entirely where this block is dead. */ \
+ (void)_FP_PACK_CANONICAL_T##_c; \
+ _FP_FRAC_COPY_##wc(_FP_PACK_CANONICAL_T, X); \
+ _FP_PACK_CANONICAL_T##_s = X##_s; \
+ _FP_PACK_CANONICAL_T##_e = X##_e; \
+ _FP_ROUND(wc, _FP_PACK_CANONICAL_T); \
+ if (_FP_FRAC_OVERP_##wc(fs, _FP_PACK_CANONICAL_T)) \
+ _FP_PACK_CANONICAL_is_tiny = 0; \
+ } \
X##_e = -X##_e + 1; \
if (X##_e <= _FP_WFRACBITS_##fs) \
{ \
@@ -161,8 +179,9 @@ do { \
_FP_FRAC_SRL_##wc(X, _FP_WORKBITS); \
} \
} \
- if ((FP_CUR_EXCEPTIONS & FP_EX_INEXACT) || \
- (FP_TRAPPING_EXCEPTIONS & FP_EX_UNDERFLOW)) \
+ if (_FP_PACK_CANONICAL_is_tiny \
+ && ((FP_CUR_EXCEPTIONS & FP_EX_INEXACT) || \
+ (FP_TRAPPING_EXCEPTIONS & FP_EX_UNDERFLOW))) \
FP_SET_EXCEPTION(FP_EX_UNDERFLOW); \
} \
else \
diff --git a/include/math-emu/soft-fp.h b/include/math-emu/soft-fp.h
index 5650c1628383..02ada0a9fca1 100644
--- a/include/math-emu/soft-fp.h
+++ b/include/math-emu/soft-fp.h
@@ -31,6 +31,14 @@
#include <endian.h>
#endif
+/* Whether the architecture determines tininess of a floating-point
+ result after rounding rather than before it. IEEE 754 permits either
+ but requires the same choice for every operation, so this has to agree
+ with what the hardware does for the results it produces itself. */
+#ifndef _FP_TININESS_AFTER_ROUNDING
+#define _FP_TININESS_AFTER_ROUNDING 0
+#endif
+
#define _FP_WORKBITS 3
#define _FP_WORK_LSB ((_FP_W_TYPE)1 << 3)
#define _FP_WORK_ROUND ((_FP_W_TYPE)1 << 2)
diff --git a/include/media/rc-map.h b/include/media/rc-map.h
index d95ed3e96de2..f167c37179c8 100644
--- a/include/media/rc-map.h
+++ b/include/media/rc-map.h
@@ -148,7 +148,6 @@ struct rc_map_table {
* @scan: pointer to struct &rc_map_table
* @size: Max number of entries
* @len: Number of entries that are in use
- * @alloc: size of \*scan, in bytes
* @rc_proto: type of the remote controller protocol, as defined at
* enum &rc_proto
* @name: name of the key map table
@@ -158,7 +157,6 @@ struct rc_map {
struct rc_map_table *scan;
unsigned int size;
unsigned int len;
- unsigned int alloc;
enum rc_proto rc_proto;
const char *name;
spinlock_t lock;
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 327976b14d50..cec9217d97ac 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -834,6 +834,9 @@ bool v4l2_ctrl_radio_filter(const struct v4l2_ctrl *ctrl);
*
* @ncontrols: The number of controls in this cluster.
* @controls: The cluster control array of size @ncontrols.
+ *
+ * If controls[0] is NULL, then this function does nothing and just
+ * returns.
*/
void v4l2_ctrl_cluster(unsigned int ncontrols, struct v4l2_ctrl **controls);
@@ -845,6 +848,9 @@ void v4l2_ctrl_cluster(unsigned int ncontrols, struct v4l2_ctrl **controls);
* @ncontrols: The number of controls in this cluster.
* @controls: The cluster control array of size @ncontrols. The first control
* must be the 'auto' control (e.g. autogain, autoexposure, etc.)
+ *
+ * If controls[0] is NULL, then this function does nothing and just
+ * returns.
* @manual_val: The value for the first control in the cluster that equals the
* manual setting.
* @set_volatile: If true, then all controls except the first auto control will
diff --git a/include/net/addrconf.h b/include/net/addrconf.h
index e6764245995f..6818057cf727 100644
--- a/include/net/addrconf.h
+++ b/include/net/addrconf.h
@@ -81,6 +81,9 @@ struct ifa6_config {
u8 ifa_proto;
const struct in6_addr *peer_pfx;
+ struct inet6_ifaddr *ifpub;
+ unsigned long cstamp;
+ unsigned long tstamp;
u32 rt_priority;
u32 ifa_flags;
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index 365925c9d262..b02d2472b003 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -171,6 +171,7 @@ extern const u8 ip_frag_ecn_table[16];
#define IPFRAG_OK 0
#define IPFRAG_DUP 1
#define IPFRAG_OVERLAP 2
+#define IPFRAG_MIXED_SECPATH 3
int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
int offset, int end);
void *inet_frag_reasm_prepare(struct inet_frag_queue *q, struct sk_buff *skb,
diff --git a/include/net/ip.h b/include/net/ip.h
index d07c2c573024..f0aeab1b1a3b 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -513,31 +513,6 @@ static inline unsigned int ip_dst_mtu_maybe_forward(const struct dst_entry *dst,
return res;
}
-/* Configured/administrative MTU of a route, for advertising the TCP MSS.
- *
- * Unlike ip_dst_mtu_maybe_forward(), this deliberately ignores the
- * ICMP-learned path MTU (rt->rt_pmtu). The advertised MSS bounds what the
- * peer may send to us and must reflect our receive capability (the device or
- * route-configured MTU), not a path MTU learned on the reverse (send)
- * direction, which may not apply to the peer->us path and outlives the fnhe
- * for the whole connection. See RFC 2923 section 2.3 and the comment above
- * tcp_advertise_mss().
- */
-static inline unsigned int ip_dst_mtu_configured(const struct dst_entry *dst)
-{
- unsigned int mtu, res;
-
- rcu_read_lock();
- mtu = dst_metric_raw(dst, RTAX_MTU);
- if (!mtu)
- mtu = READ_ONCE(dst_dev_rcu(dst)->mtu);
- mtu = min_t(unsigned int, mtu, IP_MAX_MTU);
- res = mtu - lwtunnel_headroom(dst->lwtstate, mtu);
- rcu_read_unlock();
-
- return res;
-}
-
static inline unsigned int ip_skb_dst_mtu(struct sock *sk,
const struct sk_buff *skb)
{
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 0f9b7a260d25..5fd0c0e4beac 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -389,43 +389,6 @@ static inline unsigned int ip6_dst_mtu_maybe_forward(const struct dst_entry *dst
return mtu - lwtunnel_headroom(dst->lwtstate, mtu);
}
-/* Configured/administrative MTU of a route, for advertising the TCP MSS.
- *
- * Unlike ip6_dst_mtu_maybe_forward(), this ignores any ICMPv6-learned path
- * MTU (which is kept on the RTF_CACHE exception route) and returns the MTU of
- * the underlying route (fib6_pmtu) or the egress device. The advertised MSS
- * bounds what the peer may send to us and must reflect our receive
- * capability, not a path MTU learned on the reverse (send) direction. See
- * RFC 2923 section 2.3 and the comment above tcp_advertise_mss().
- */
-static inline unsigned int ip6_dst_mtu_configured(const struct dst_entry *dst)
-{
- const struct rt6_info *rt = dst_rt6_info(dst);
- const struct fib6_info *from;
- struct inet6_dev *idev;
- unsigned int mtu = 0;
-
- rcu_read_lock();
- /* IPv6 keeps the learned PMTU and the configured MTU in the same
- * RTAX_MTU slot: the learned value sits on this (possibly RTF_CACHE)
- * dst, the configured one on the underlying route. Reach the latter
- * via ->from (fib6_pmtu), populated by ip6_route_info_create().
- */
- from = rcu_dereference(rt->from);
- if (from)
- mtu = from->fib6_pmtu;
- if (!mtu) {
- mtu = IPV6_MIN_MTU;
- idev = __in6_dev_get(dst_dev_rcu(dst));
- if (idev)
- mtu = max_t(unsigned int, mtu, READ_ONCE(idev->cnf.mtu6));
- }
- rcu_read_unlock();
-
- mtu = min_t(unsigned int, mtu, IP6_MAX_MTU);
- return mtu - lwtunnel_headroom(dst->lwtstate, mtu);
-}
-
u32 ip6_mtu_from_fib6(const struct fib6_result *res,
const struct in6_addr *daddr,
const struct in6_addr *saddr);
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index a6d69aaa6cd2..298ca9bb0083 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -1201,6 +1201,23 @@ secpath_reset(struct sk_buff *skb)
#endif
}
+static inline bool
+secpath_equal(const struct sk_buff *skb1, const struct sk_buff *skb2)
+{
+ const struct sec_path *sp1 = skb_sec_path(skb1);
+ const struct sec_path *sp2 = skb_sec_path(skb2);
+ int i;
+
+ if (!sp1 || !sp2)
+ return sp1 == sp2;
+ if (sp1->len != sp2->len)
+ return false;
+ for (i = 0; i < sp1->len; i++)
+ if (sp1->xvec[i] != sp2->xvec[i])
+ return false;
+ return true;
+}
+
static inline int
xfrm_addr_any(const xfrm_address_t *addr, unsigned short family)
{
diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
index 206fa667c782..5d47d1421c1f 100644
--- a/include/uapi/linux/iommufd.h
+++ b/include/uapi/linux/iommufd.h
@@ -948,7 +948,9 @@ struct iommu_hwpt_vtd_s1_invalidate {
* CMDQ_OP_CFGI_CD
* CMDQ_OP_CFGI_CD_ALL
*
- * -EIO will be returned if the command is not supported.
+ * User space must forward only valid commands: the kernel rejects, with
+ * -EIO, any command carrying an unsupported opcode, an unsupported field,
+ * or a field value that the underlying SMMU hardware does not implement.
*/
struct iommu_viommu_arm_smmuv3_invalidate {
__aligned_le64 cmd[2];
diff --git a/include/uapi/linux/media/arm/mali-c55-config.h b/include/uapi/linux/media/arm/mali-c55-config.h
index ef0fe17df96e..3ba416d81cd7 100644
--- a/include/uapi/linux/media/arm/mali-c55-config.h
+++ b/include/uapi/linux/media/arm/mali-c55-config.h
@@ -794,12 +794,14 @@ struct mali_c55_params_mesh_shading_selection {
* Q4.8 format
* @gains: Gains for red, green and blue channels in unsigned Q4.8 format
* @offs: Offsets for red, green and blue channels
+ * @__pad: Reserved bytes
*/
struct mali_c55_params_ccm {
struct v4l2_isp_params_block_header header;
__u16 coeffs[3][3];
__u16 gains[3];
__u16 offs[3];
+ __u16 __pad;
};
/**
diff --git a/init/main.c b/init/main.c
index 31f2bf54976a..4efcb9956c39 100644
--- a/init/main.c
+++ b/init/main.c
@@ -1028,12 +1028,12 @@ void start_kernel(void)
__stop___param - __start___param,
-1, -1, NULL, &unknown_bootoption);
print_unknown_bootoptions();
- if (!IS_ERR_OR_NULL(after_dashes))
- parse_args("Setting init args", after_dashes, NULL, 0, -1, -1,
- NULL, set_init_arg);
if (extra_init_args)
parse_args("Setting extra init args", extra_init_args,
NULL, 0, -1, -1, NULL, set_init_arg);
+ if (!IS_ERR_OR_NULL(after_dashes))
+ parse_args("Setting init args", after_dashes, NULL, 0, -1, -1,
+ NULL, set_init_arg);
/* Architectural and non-timekeeping rng init, before allocator init */
random_init_early(command_line);
diff --git a/init/version.c b/init/version.c
index 94c96f6fbfe6..0bd5c45aabc4 100644
--- a/init/version.c
+++ b/init/version.c
@@ -23,6 +23,9 @@ static int __init early_hostname(char *arg)
size_t maxlen = bufsize - 1;
ssize_t arglen;
+ if (!arg)
+ return -EINVAL;
+
arglen = strscpy(init_uts_ns.name.nodename, arg, bufsize);
if (arglen < 0) {
pr_warn("hostname parameter exceeds %zd characters and will be truncated",
diff --git a/ipc/mqueue.c b/ipc/mqueue.c
index d1dd36a651b0..d1a1965c9811 100644
--- a/ipc/mqueue.c
+++ b/ipc/mqueue.c
@@ -528,6 +528,13 @@ static void mqueue_evict_inode(struct inode *inode)
list_add_tail(&msg->m_list, &tmp_msg);
kfree(info->node_cache);
spin_unlock(&info->lock);
+ /*
+ * A shared file table can let the notification owner exit without
+ * running ->flush(). No users of the inode remain during eviction, so
+ * tear down any stale notification after dropping info->lock because
+ * netlink_sendskb() may release the final socket reference.
+ */
+ remove_notification(info);
list_for_each_entry_safe(msg, nmsg, &tmp_msg, m_list) {
list_del(&msg->m_list);
diff --git a/ipc/msgutil.c b/ipc/msgutil.c
index e28f0cecb2ec..12663a40e9ae 100644
--- a/ipc/msgutil.c
+++ b/ipc/msgutil.c
@@ -58,7 +58,8 @@ static struct msg_msg *alloc_msg(size_t len)
size_t alen;
alen = min(len, DATALEN_MSG);
- msg = kmem_buckets_alloc(msg_buckets, sizeof(*msg) + alen, GFP_KERNEL);
+ msg = kmem_buckets_alloc(msg_buckets, sizeof(*msg) + alen,
+ GFP_KERNEL_ACCOUNT);
if (msg == NULL)
return NULL;
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 75aa7b469348..b54d23fd83d7 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -255,3 +255,10 @@ const struct proc_ns_operations ipcns_operations = {
.install = ipcns_install,
.owner = ipcns_owner,
};
+
+static int __init ipc_ns_tree_init(void)
+{
+ ns_tree_add(&init_ipc_ns);
+ return 0;
+}
+pure_initcall(ipc_ns_tree_init);
diff --git a/ipc/shm.c b/ipc/shm.c
index bb1a721a3e74..83ac223352ba 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -45,7 +45,6 @@
#include <linux/mount.h>
#include <linux/ipc_namespace.h>
#include <linux/rhashtable.h>
-#include <linux/nstree.h>
#include <linux/uaccess.h>
@@ -149,7 +148,6 @@ void shm_exit_ns(struct ipc_namespace *ns)
static int __init ipc_ns_init(void)
{
shm_init_ns(&init_ipc_ns);
- ns_tree_add(&init_ipc_ns);
return 0;
}
@@ -936,26 +934,11 @@ static inline unsigned long copy_shminfo_to_user(void __user *buf, struct shminf
static void shm_add_rss_swap(struct shmid_kernel *shp,
unsigned long *rss_add, unsigned long *swp_add)
{
- struct inode *inode;
+ struct inode *inode = file_inode(shp->shm_file);
- inode = file_inode(shp->shm_file);
-
- if (is_file_hugepages(shp->shm_file)) {
- struct address_space *mapping = inode->i_mapping;
- struct hstate *h = hstate_file(shp->shm_file);
- *rss_add += pages_per_huge_page(h) * mapping->nrpages;
- } else {
-#ifdef CONFIG_SHMEM
- struct shmem_inode_info *info = SHMEM_I(inode);
-
- spin_lock_irq(&info->lock);
- *rss_add += inode->i_mapping->nrpages;
- *swp_add += info->swapped;
- spin_unlock_irq(&info->lock);
-#else
- *rss_add += inode->i_mapping->nrpages;
-#endif
- }
+ *rss_add += READ_ONCE(inode->i_mapping->nrpages);
+ if (IS_ENABLED(CONFIG_SHMEM) && !is_file_hugepages(shp->shm_file))
+ *swp_add += READ_ONCE(SHMEM_I(inode)->swapped);
}
/*
diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c
index 6e8b18c32a10..1cee2ba6e332 100644
--- a/kernel/bpf/queue_stack_maps.c
+++ b/kernel/bpf/queue_stack_maps.c
@@ -59,6 +59,9 @@ static int queue_stack_map_alloc_check(union bpf_attr *attr)
*/
return -E2BIG;
+ if (attr->max_entries >= U32_MAX / attr->value_size)
+ return -E2BIG;
+
return 0;
}
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 42cc77dfa1c8..2d582603a711 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -4289,7 +4289,7 @@ static void __cgroup_kill(struct cgroup *cgrp)
cgrp->kill_seq++;
spin_unlock_irq(&css_set_lock);
- css_task_iter_start(&cgrp->self, CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, &it);
+ css_task_iter_start(&cgrp->self, CSS_TASK_ITER_THREADED, &it);
while ((task = css_task_iter_next(&it))) {
/* Ignore kernel threads here. */
if (task->flags & PF_KTHREAD)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 8b220e9df91e..86b5921998af 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -478,10 +478,10 @@ static inline bool partition_is_populated(struct cpuset *cs,
rcu_read_lock();
cpuset_for_each_descendant_pre(cp, pos_css, cs) {
- if (cp == cs || cp == excluded_child)
+ if (cp == cs)
continue;
- if (is_partition_valid(cp)) {
+ if (cp == excluded_child || is_partition_valid(cp)) {
pos_css = css_rightmost_descendant(pos_css);
continue;
}
diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c
index 7481fbb947d3..a2a598ad6e6d 100644
--- a/kernel/cpu_pm.c
+++ b/kernel/cpu_pm.c
@@ -10,6 +10,7 @@
#include <linux/cpu_pm.h>
#include <linux/module.h>
#include <linux/notifier.h>
+#include <linux/rcupdate.h>
#include <linux/spinlock.h>
#include <linux/syscore_ops.h>
@@ -76,7 +77,8 @@ EXPORT_SYMBOL_GPL(cpu_pm_register_notifier);
*
* Remove a driver from the CPU PM notifier list.
*
- * This function has the same return conditions as raw_notifier_chain_unregister.
+ * This function may sleep, and has the same return conditions as
+ * raw_notifier_chain_unregister.
*/
int cpu_pm_unregister_notifier(struct notifier_block *nb)
{
@@ -86,6 +88,11 @@ int cpu_pm_unregister_notifier(struct notifier_block *nb)
raw_spin_lock_irqsave(&cpu_pm_notifier.lock, flags);
ret = raw_notifier_chain_unregister(&cpu_pm_notifier.chain, nb);
raw_spin_unlock_irqrestore(&cpu_pm_notifier.lock, flags);
+
+ /* Wait for the rcu_read_lock() walkers in cpu_pm_notify(). */
+ if (!ret)
+ synchronize_rcu();
+
return ret;
}
EXPORT_SYMBOL_GPL(cpu_pm_unregister_notifier);
diff --git a/kernel/debug/gdbstub.c b/kernel/debug/gdbstub.c
index e271a436d60e..fc6a7a8c1f6e 100644
--- a/kernel/debug/gdbstub.c
+++ b/kernel/debug/gdbstub.c
@@ -43,6 +43,8 @@
/* Our I/O buffers. */
static char remcom_in_buffer[BUFMAX];
static char remcom_out_buffer[BUFMAX];
+/* Payload bytes get_packet() stored in remcom_in_buffer, excluding the NUL. */
+static int remcom_in_len;
static int gdbstub_use_prev_in_buf;
static int gdbstub_prev_in_buf_pos;
@@ -87,7 +89,7 @@ static int gdbstub_read_wait(void)
}
#endif
/* scan for the sequence $<data>#<checksum> */
-static void get_packet(char *buffer)
+static int get_packet(char *buffer)
{
unsigned char checksum;
unsigned char xmitcsum;
@@ -135,6 +137,8 @@ static void get_packet(char *buffer)
}
buffer[count] = 0;
} while (checksum != xmitcsum);
+
+ return count;
}
/*
@@ -321,16 +325,26 @@ int kgdb_hex2long(char **ptr, unsigned long *long_val)
* Copy the binary array pointed to by buf into mem. Fix $, #, and
* 0x7d escaped with 0x7d. Return -EFAULT on failure or 0 on success.
* The input buf is overwritten with the result to write to mem.
+ *
+ * buf_end points one past the last byte received for this packet. Each
+ * 0x7d escape consumes a second input byte, so decoding count bytes can
+ * read up to 2 * count input bytes; stop at buf_end rather than running
+ * off the end of remcom_in_buffer.
*/
-static int kgdb_ebin2mem(char *buf, char *mem, int count)
+static int kgdb_ebin2mem(char *buf, char *mem, int count, char *buf_end)
{
int size = 0;
char *c = buf;
while (count-- > 0) {
+ if (buf >= buf_end)
+ return -EINVAL;
c[size] = *buf++;
- if (c[size] == 0x7d)
+ if (c[size] == 0x7d) {
+ if (buf >= buf_end)
+ return -EINVAL;
c[size] = *buf++ ^ 0x20;
+ }
size++;
}
@@ -367,16 +381,33 @@ void gdb_regs_to_pt_regs(unsigned long *gdb_regs, struct pt_regs *regs)
static int write_mem_msg(int binary)
{
char *ptr = &remcom_in_buffer[1];
+ char *buf_end = &remcom_in_buffer[remcom_in_len];
unsigned long addr;
unsigned long length;
int err;
if (kgdb_hex2long(&ptr, &addr) > 0 && *(ptr++) == ',' &&
kgdb_hex2long(&ptr, &length) > 0 && *(ptr++) == ':') {
- if (binary)
- err = kgdb_ebin2mem(ptr, (char *)addr, length);
- else
+ /*
+ * Trust only the bytes actually received, not length: the
+ * client can claim more payload than it sent. 'M' decodes two
+ * hex chars per byte in place, so it needs 2 * length bytes at
+ * ptr; 'X' decodes one byte per byte, and kgdb_ebin2mem() bounds
+ * its extra 0x7d-escape reads against buf_end itself.
+ */
+ if (ptr >= buf_end)
+ return -EINVAL;
+
+ if (binary) {
+ if (length > (unsigned long)(buf_end - ptr))
+ return -EINVAL;
+ err = kgdb_ebin2mem(ptr, (char *)addr, length, buf_end);
+ } else {
+ if (length > (unsigned long)(buf_end - ptr) / 2)
+ return -EINVAL;
err = kgdb_hex2mem(ptr, (char *)addr, length);
+ }
+
if (err)
return err;
if (CACHE_FLUSH_IS_SAFE)
@@ -563,6 +594,14 @@ static void gdb_cmd_memread(struct kgdb_state *ks)
if (kgdb_hex2long(&ptr, &addr) > 0 && *ptr++ == ',' &&
kgdb_hex2long(&ptr, &length) > 0) {
+ /*
+ * The hex-encoded reply needs 2 * length + 1 bytes
+ * in remcom_out_buffer[BUFMAX].
+ */
+ if (length > (BUFMAX - 1) / 2) {
+ error_packet(remcom_out_buffer, -EINVAL);
+ return;
+ }
err = kgdb_mem2hex((char *)addr, remcom_out_buffer, length);
if (!err)
error_packet(remcom_out_buffer, -EINVAL);
@@ -778,9 +817,9 @@ static void gdb_cmd_query(struct kgdb_state *ks)
strscpy(remcom_out_buffer, "E01");
break;
}
+ len = len / 2;
kgdb_hex2mem(remcom_in_buffer + 6,
remcom_out_buffer, len);
- len = len / 2;
remcom_out_buffer[len++] = 0;
kdb_common_init_state(ks);
@@ -985,7 +1024,7 @@ int gdb_serial_stub(struct kgdb_state *ks)
/* Clear the out buffer. */
memset(remcom_out_buffer, 0, sizeof(remcom_out_buffer));
- get_packet(remcom_in_buffer);
+ remcom_in_len = get_packet(remcom_in_buffer);
switch (remcom_in_buffer[0]) {
case '?': /* gdbserial status */
diff --git a/kernel/fork.c b/kernel/fork.c
index 26b666e98bb8..8ad34646962d 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -858,7 +858,8 @@ void __init fork_init(void)
#ifndef ARCH_MIN_TASKALIGN
#define ARCH_MIN_TASKALIGN 0
#endif
- int align = max_t(int, L1_CACHE_BYTES, ARCH_MIN_TASKALIGN);
+ int align = max3(L1_CACHE_BYTES, ARCH_MIN_TASKALIGN,
+ __alignof__(struct task_struct));
unsigned long useroffset, usersize;
/* create a slab on which task_structs can be allocated */
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index 2c30e7537987..703a062be878 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -83,6 +83,7 @@ enum {
#include "settings.h"
extern int __irq_set_trigger(struct irq_desc *desc, unsigned long flags);
+extern int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val);
extern void __disable_irq(struct irq_desc *desc);
extern void __enable_irq(struct irq_desc *desc);
diff --git a/kernel/irq/kexec.c b/kernel/irq/kexec.c
index 1a3deffe6b5b..f245b9876422 100644
--- a/kernel/irq/kexec.c
+++ b/kernel/irq/kexec.c
@@ -25,7 +25,7 @@ void machine_kexec_mask_interrupts(void)
* First try to remove the active state from an interrupt which is forwarded
* to a VM. If the interrupt is not forwarded, try to EOI the interrupt.
*/
- check_eoi = irq_set_irqchip_state(i, IRQCHIP_STATE_ACTIVE, false);
+ check_eoi = __irq_set_irqchip_state(desc, IRQCHIP_STATE_ACTIVE, false);
}
if (check_eoi && chip->irq_eoi && irqd_irq_inprogress(&desc->irq_data))
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 3b647d9b458a..ec3ecb0e48ce 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -2748,6 +2748,28 @@ int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which, bool *
}
EXPORT_SYMBOL_GPL(irq_get_irqchip_state);
+int __irq_set_irqchip_state(struct irq_desc *desc, enum irqchip_irq_state which, bool val)
+{
+ struct irq_data *data = irq_desc_get_irq_data(desc);
+ struct irq_chip *chip;
+
+ do {
+ chip = irq_data_get_irq_chip(data);
+
+ if (WARN_ON_ONCE(!chip))
+ return -ENODEV;
+
+ if (chip->irq_set_irqchip_state)
+ break;
+
+ data = irqd_get_parent_data(data);
+ } while (data);
+
+ if (data)
+ return chip->irq_set_irqchip_state(data, which, val);
+ return -EINVAL;
+}
+
/**
* irq_set_irqchip_state - set the state of a forwarded interrupt.
* @irq: Interrupt line that is forwarded to a VM
@@ -2762,25 +2784,8 @@ EXPORT_SYMBOL_GPL(irq_get_irqchip_state);
*/
int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which, bool val)
{
- scoped_irqdesc_get_and_buslock(irq, 0) {
- struct irq_data *data = irq_desc_get_irq_data(scoped_irqdesc);
- struct irq_chip *chip;
-
- do {
- chip = irq_data_get_irq_chip(data);
-
- if (WARN_ON_ONCE(!chip))
- return -ENODEV;
-
- if (chip->irq_set_irqchip_state)
- break;
-
- data = irqd_get_parent_data(data);
- } while (data);
-
- if (data)
- return chip->irq_set_irqchip_state(data, which, val);
- }
+ scoped_irqdesc_get_and_buslock(irq, 0)
+ return __irq_set_irqchip_state(scoped_irqdesc, which, val);
return -EINVAL;
}
EXPORT_SYMBOL_GPL(irq_set_irqchip_state);
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 7c4d86daf86d..3c87c2a3d6f0 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -12,6 +12,7 @@
#include <linux/cleanup.h>
#include <linux/cma.h>
+#include <linux/crash_dump.h>
#include <linux/kmemleak.h>
#include <linux/count_zeros.h>
#include <linux/kasan.h>
@@ -1939,6 +1940,12 @@ void __init kho_memory_init_early(void)
void __init kho_memory_init(void)
{
+ if (is_kdump_kernel()) {
+ kho_enable = false;
+ pr_info("disabled in the kdump kernel\n");
+ return;
+ }
+
if (kho_in.scratch_phys)
kho_mem_retrieve();
else
diff --git a/kernel/nstree.c b/kernel/nstree.c
index 831f279d174a..a756ae2e7881 100644
--- a/kernel/nstree.c
+++ b/kernel/nstree.c
@@ -408,15 +408,12 @@ struct klistns {
u32 ns_type;
struct user_namespace *user_ns;
bool userns_capable;
- struct ns_common *first_ns;
};
static void __free_klistns_free(const struct klistns *kls)
{
if (kls->user_ns_id != LISTNS_CURRENT_USER)
put_user_ns(kls->user_ns);
- if (kls->first_ns && kls->first_ns->ops)
- kls->first_ns->ops->put(kls->first_ns);
}
#define NS_ALL (PID_NS | USER_NS | MNT_NS | UTS_NS | IPC_NS | NET_NS | CGROUP_NS | TIME_NS)
@@ -459,10 +456,13 @@ static inline int prepare_klistns(struct klistns *kls, struct ns_id_req *kreq,
}
/*
- * Lookup a namespace owned by owner with id >= ns_id.
- * Returns the namespace with the smallest id that is >= ns_id.
+ * Lookup a namespace owned by owner with id > ns_id.
+ * Returns the namespace with the smallest id that is > ns_id.
+ *
+ * No reference is taken. The caller must hold rcu_read_lock() and may only
+ * use the result as the starting point of an RCU list walk.
*/
-static struct ns_common *lookup_ns_owner_at(u64 ns_id, struct ns_common *owner)
+static struct ns_common *lookup_ns_owner_after(u64 ns_id, struct ns_common *owner)
{
struct ns_common *ret = NULL;
struct rb_node *node;
@@ -476,18 +476,14 @@ static struct ns_common *lookup_ns_owner_at(u64 ns_id, struct ns_common *owner)
struct ns_common *ns;
ns = node_to_ns_owner(node);
- if (ns_id <= ns->ns_id) {
+ if (ns_id < ns->ns_id) {
ret = ns;
- if (ns_id == ns->ns_id)
- break;
node = node->rb_left;
} else {
node = node->rb_right;
}
}
- if (ret)
- ret = ns_get_unless_inactive(ret);
return ret;
}
@@ -560,25 +556,27 @@ static ssize_t do_listns_userns(struct klistns *kls)
return -EINVAL;
kls->user_ns = to_user_ns(ns);
- /*
- * Use the rbtree to find the first namespace we care about and
- * then use it's list entry to iterate from there.
- */
- if (kls->last_ns_id) {
- kls->first_ns = lookup_ns_owner_at(kls->last_ns_id + 1, ns);
- if (!kls->first_ns)
- return -ENOENT;
- first_ns = kls->first_ns;
- }
-
ret = 0;
head = &to_ns_common(kls->user_ns)->ns_owner_root.ns_list_head;
kls->userns_capable = may_see_all_namespaces();
rcu_read_lock();
- if (!first_ns)
+ /*
+ * Use the rbtree to find the first namespace we care about and
+ * then use it's list entry to iterate from there. That namespace
+ * is filtered like any other, so the walk doesn't end early just
+ * because it is inactive or already dying.
+ */
+ if (kls->last_ns_id) {
+ first_ns = lookup_ns_owner_after(kls->last_ns_id, ns);
+ if (!first_ns) {
+ rcu_read_unlock();
+ return -ENOENT;
+ }
+ } else {
first_ns = list_entry_rcu(head->next, typeof(*first_ns), ns_owner_node.ns_list_entry);
+ }
ns = first_ns;
list_for_each_entry_from_rcu(ns, head, ns_owner_node.ns_list_entry) {
@@ -613,10 +611,13 @@ static ssize_t do_listns_userns(struct klistns *kls)
}
/*
- * Lookup a namespace with id >= ns_id in either the unified tree or a type-specific tree.
- * Returns the namespace with the smallest id that is >= ns_id.
+ * Lookup a namespace with id > ns_id in either the unified tree or a type-specific tree.
+ * Returns the namespace with the smallest id that is > ns_id.
+ *
+ * No reference is taken. The caller must hold rcu_read_lock() and may only
+ * use the result as the starting point of an RCU list walk.
*/
-static struct ns_common *lookup_ns_id_at(u64 ns_id, int ns_type)
+static struct ns_common *lookup_ns_id_after(u64 ns_id, int ns_type)
{
struct ns_common *ret = NULL;
struct ns_tree_root *ns_tree = NULL;
@@ -643,21 +644,14 @@ static struct ns_common *lookup_ns_id_at(u64 ns_id, int ns_type)
else
ns = node_to_ns_unified(node);
- if (ns_id <= ns->ns_id) {
- if (ns_type)
- ret = node_to_ns(node);
- else
- ret = node_to_ns_unified(node);
- if (ns_id == ns->ns_id)
- break;
+ if (ns_id < ns->ns_id) {
+ ret = ns;
node = node->rb_left;
} else {
node = node->rb_right;
}
}
- if (ret)
- ret = ns_get_unless_inactive(ret);
return ret;
}
@@ -707,13 +701,6 @@ static ssize_t do_listns(struct klistns *kls)
return -EINVAL;
}
- if (kls->last_ns_id) {
- kls->first_ns = lookup_ns_id_at(kls->last_ns_id + 1, ns_type);
- if (!kls->first_ns)
- return -ENOENT;
- first_ns = kls->first_ns;
- }
-
ret = 0;
if (ns_tree)
head = &ns_tree->ns_list_head;
@@ -722,8 +709,15 @@ static ssize_t do_listns(struct klistns *kls)
rcu_read_lock();
- if (!first_ns)
+ if (kls->last_ns_id) {
+ first_ns = lookup_ns_id_after(kls->last_ns_id, ns_type);
+ if (!first_ns) {
+ rcu_read_unlock();
+ return -ENOENT;
+ }
+ } else {
first_ns = first_ns_common(head, ns_tree);
+ }
for (ns = first_ns; !ns_common_is_head(ns, head, ns_tree) && nr_ns_ids;
ns = next_ns_common(ns, ns_tree)) {
diff --git a/kernel/panic.c b/kernel/panic.c
index 213725b612aa..cec271601935 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -307,7 +307,7 @@ atomic_t panic_cpu = ATOMIC_INIT(PANIC_CPU_INVALID);
atomic_t panic_redirect_cpu = ATOMIC_INIT(PANIC_CPU_INVALID);
#if defined(CONFIG_SMP) && defined(CONFIG_CRASH_DUMP)
-static char *panic_force_buf;
+static char panic_force_buf[PANIC_MSG_BUFSZ];
static int __init panic_force_cpu_setup(char *str)
{
@@ -326,17 +326,6 @@ static int __init panic_force_cpu_setup(char *str)
}
early_param("panic_force_cpu", panic_force_cpu_setup);
-static int __init panic_force_cpu_late_init(void)
-{
- if (panic_force_cpu < 0)
- return 0;
-
- panic_force_buf = kmalloc(PANIC_MSG_BUFSZ, GFP_KERNEL);
-
- return 0;
-}
-late_initcall(panic_force_cpu_late_init);
-
static void do_panic_on_target_cpu(void *info)
{
panic("%s", (char *)info);
@@ -371,15 +360,16 @@ int __weak panic_smp_redirect_cpu(int target_cpu, void *msg)
* for the crash kernel to function correctly. This function redirects
* panic handling to the CPU specified via the panic_force_cpu= boot parameter.
*
- * Returns false if panic should proceed on current CPU.
- * Returns true if panic was redirected.
+ * Returns true when this CPU must stop: the panic was redirected or is
+ * already running on another CPU.
+ * Returns false when panic() should proceed on this CPU.
*/
__printf(1, 0)
static bool panic_try_force_cpu(const char *fmt, va_list args)
{
int this_cpu = raw_smp_processor_id();
int old_cpu = PANIC_CPU_INVALID;
- const char *msg;
+ va_list ap;
/* Feature not enabled via boot parameter */
if (panic_force_cpu < 0)
@@ -396,27 +386,27 @@ static bool panic_try_force_cpu(const char *fmt, va_list args)
return false;
}
- /* Another panic already in progress */
+ /*
+ * Don't redirect when a panic is already in progress. Stop this
+ * CPU when it's another one, proceed when it's this one.
+ */
if (panic_in_progress())
- return false;
+ return panic_on_other_cpu();
/*
- * Only one CPU can do the redirect. Use atomic cmpxchg to ensure
- * we don't race with another CPU also trying to redirect.
+ * Only one CPU can do the redirection. Others should go offline.
+ * Continue with panic() when we already tried the redirection
+ * from this CPU before, for example via nmi_panic().
*/
if (!atomic_try_cmpxchg(&panic_redirect_cpu, &old_cpu, this_cpu))
- return false;
+ return old_cpu != this_cpu;
/*
- * Use dynamically allocated buffer if available, otherwise
- * fall back to static message for early boot panics or allocation failure.
+ * Do not consume args, the caller reuses them if we fail.
*/
- if (panic_force_buf) {
- vsnprintf(panic_force_buf, PANIC_MSG_BUFSZ, fmt, args);
- msg = panic_force_buf;
- } else {
- msg = "Redirected panic (buffer unavailable)";
- }
+ va_copy(ap, args);
+ vsnprintf(panic_force_buf, PANIC_MSG_BUFSZ, fmt, ap);
+ va_end(ap);
console_verbose();
bust_spinlocks(1);
@@ -431,7 +421,7 @@ static bool panic_try_force_cpu(const char *fmt, va_list args)
dump_stack();
}
- if (panic_smp_redirect_cpu(panic_force_cpu, (void *)msg) != 0) {
+ if (panic_smp_redirect_cpu(panic_force_cpu, panic_force_buf) != 0) {
atomic_set(&panic_redirect_cpu, PANIC_CPU_INVALID);
pr_warn("panic: failed to redirect to CPU %d, continuing on CPU %d\n",
panic_force_cpu, this_cpu);
@@ -503,17 +493,35 @@ bool panic_on_other_cpu(void)
EXPORT_SYMBOL(panic_on_other_cpu);
/*
- * A variant of panic() called from NMI context. We return if we've already
- * panicked on this CPU. If another CPU already panicked, loop in
- * nmi_panic_self_stop() which can provide architecture dependent code such
- * as saving register state for crash dump.
+ * A variant of panic() called from NMI context. The panic is first
+ * redirected to the CPU requested via panic_force_cpu=, when configured.
+ * We return if we've already panicked on this CPU. If another CPU already
+ * panicked, loop in nmi_panic_self_stop() which can provide architecture
+ * dependent code for saving register state for crash dump.
*/
-void nmi_panic(struct pt_regs *regs, const char *msg)
+void nmi_panic(struct pt_regs *regs, const char *fmt, ...)
{
+ va_list args;
+
+ va_start(args, fmt);
+
+ /* Try to redirect to the requested CPU before claiming panic_cpu. */
+ if (panic_try_force_cpu(fmt, args)) {
+ /*
+ * Mark ourselves offline so panic_other_cpus_shutdown() won't
+ * wait for us on architectures that check num_online_cpus().
+ */
+ set_cpu_online(raw_smp_processor_id(), false);
+ nmi_panic_self_stop(regs);
+ }
+
if (panic_try_start())
- panic("%s", msg);
- else if (panic_on_other_cpu())
+ vpanic(fmt, args);
+
+ if (panic_on_other_cpu())
nmi_panic_self_stop(regs);
+
+ va_end(args);
}
EXPORT_SYMBOL(nmi_panic);
diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c
index a76089e2ce8c..d19a65461ae8 100644
--- a/kernel/power/energy_model.c
+++ b/kernel/power/energy_model.c
@@ -439,7 +439,9 @@ static int em_create_pd(struct device *dev, int nr_states,
cpumask_copy(em_span_cpus(pd), cpus);
} else {
- pd = kzalloc_obj(*pd);
+ /* Readers expect pd->cpus to exist and remain empty. */
+ pd = kzalloc_flex(*pd, cpus,
+ BITS_TO_LONGS(large_cpumask_bits));
if (!pd)
return -ENOMEM;
}
diff --git a/kernel/resource.c b/kernel/resource.c
index 08bd7579cf40..a98fd2b29cea 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1350,8 +1350,8 @@ static int __request_region_locked(struct resource *res, struct resource *parent
}
if (conflict->flags & flags & IORESOURCE_MUXED) {
add_wait_queue(&muxed_resource_wait, &wait);
- write_unlock(&resource_lock);
set_current_state(TASK_UNINTERRUPTIBLE);
+ write_unlock(&resource_lock);
schedule();
remove_wait_queue(&muxed_resource_wait, &wait);
write_lock(&resource_lock);
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index 49ccd6f1c185..14405a614991 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -73,6 +73,18 @@ static void sugov_update_rate_limit_us(struct sugov_policy *sg_policy)
(s64)sg_policy->tunables->rate_limit_us * NSEC_PER_USEC;
}
+static void sugov_update_rate_limit_us_sync(struct sugov_policy *sg_policy)
+{
+ /*
+ * Serialize with rate_limit_us_store() to prevent this path from using
+ * a stale rate limit tunable value to set the limit for the given
+ * policy.
+ */
+ guard(mutex)(&sg_policy->tunables->attr_set.update_lock);
+
+ sugov_update_rate_limit_us(sg_policy);
+}
+
static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time)
{
s64 delta_ns;
@@ -861,7 +873,7 @@ static int sugov_start(struct cpufreq_policy *policy)
void (*uu)(struct update_util_data *data, u64 time, unsigned int flags);
unsigned int cpu;
- sugov_update_rate_limit_us(sg_policy);
+ sugov_update_rate_limit_us_sync(sg_policy);
sg_policy->last_freq_update_time = 0;
sg_policy->next_freq = 0;
sg_policy->work_in_progress = false;
diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 04167ed9b580..84774147a1d5 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -2232,7 +2232,11 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
struct scx_sched *sch = scx_task_sched(p);
unsigned long opss;
- /* dequeue is always temporary, don't reset runnable_at */
+ /*
+ * Don't reset runnable_at. If @p is waiting, the wait continues across
+ * temporary dequeues, e.g. for a migration or an attribute change.
+ * scx_disable_task() ends it if @p leaves its scheduler.
+ */
clr_task_runnable(p, false);
retry:
@@ -3335,7 +3339,8 @@ static struct task_struct *first_local_task(struct rq *rq)
}
/*
- * Run dispatch and queue the follow-up work for a pick.
+ * Run dispatch for a pick. The follow-up work is queued by do_pick_task_scx()
+ * once the pick is final.
*/
static enum scx_dsp_verdict dispatch_pick(struct rq *rq, struct rq_flags *rf,
struct task_struct *prev)
@@ -3345,18 +3350,6 @@ static enum scx_dsp_verdict dispatch_pick(struct rq *rq, struct rq_flags *rf,
rq_unpin_lock(rq, rf);
verdict = dispatch_one(rq, prev);
rq_repin_lock(rq, rf);
- maybe_queue_balance_callback(rq);
-
- /*
- * Defer to a balance callback which can drop rq lock and enable IRQs.
- * Waiting directly in the pick path would deadlock against CPUs sending
- * us IPIs (e.g. TLB flushes) while we wait for them.
- */
- if (unlikely(rq->scx.kick_sync_pending)) {
- rq->scx.kick_sync_pending = false;
- queue_balance_callback(rq, &rq->scx.kick_sync_bal_cb,
- kick_sync_wait_bal_cb);
- }
return verdict;
}
@@ -3383,23 +3376,20 @@ static enum scx_dsp_verdict dispatch_core_pick(struct rq *rq, struct rq_flags *r
verdict = dispatch_one(rq, prev);
- if (cpu_of(rq) == smp_processor_id()) {
- maybe_queue_balance_callback(rq);
-
- /* see dispatch_pick() */
- if (unlikely(rq->scx.kick_sync_pending)) {
- rq->scx.kick_sync_pending = false;
- queue_balance_callback(rq, &rq->scx.kick_sync_bal_cb,
- kick_sync_wait_bal_cb);
- }
- } else if (unlikely(rq->scx.flags & SCX_RQ_BAL_CB_PENDING)) {
- /*
- * Balance callbacks must run in the context that queued them,
- * so they can't be queued on another CPU's rq. Run the deferred
- * work directly instead.
- */
+ /*
+ * A pick that returns a task isn't final under core scheduling: the
+ * selection goes on to pick the siblings, and their dispatch can
+ * release the rq lock, which can't happen with balance callbacks
+ * queued. Don't queue the deferred work as a balance callback, for this
+ * CPU or a sibling, and run it from irq_work instead.
+ *
+ * do_pick_task_scx() leaves %SCX_RQ_BAL_CB_PENDING alone under core
+ * scheduling, so that the early return above leaves it to the dispatch
+ * in flight, which handles it here.
+ */
+ if (unlikely(rq->scx.flags & SCX_RQ_BAL_CB_PENDING)) {
rq->scx.flags &= ~SCX_RQ_BAL_CB_PENDING;
- run_deferred(rq);
+ schedule_deferred(rq);
}
rq_repin_lock(rq, rf);
@@ -3422,6 +3412,7 @@ static struct task_struct *
do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
{
struct task_struct *prev = rq->curr;
+ bool core_sched = sched_core_enabled(rq);
enum scx_dsp_verdict verdict;
struct task_struct *p;
@@ -3430,13 +3421,20 @@ do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
rq_modified_begin(rq, &ext_sched_class);
- if (sched_core_enabled(rq))
+ /*
+ * Core scheduling can get enabled while dispatch_pick() has the rq lock
+ * released. Stick to @core_sched, so that the follow-up work below
+ * matches the dispatch that ran.
+ */
+ if (core_sched)
verdict = dispatch_core_pick(rq, rf, prev);
else
verdict = dispatch_pick(rq, rf, prev);
- if (verdict == SCX_DSP_RETRY)
- return RETRY_TASK;
+ if (verdict == SCX_DSP_RETRY) {
+ p = RETRY_TASK;
+ goto out;
+ }
/*
* If any higher-priority sched class enqueued a runnable task on this
@@ -3446,8 +3444,10 @@ do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
* If @force_scx is true, always try to pick a SCHED_EXT task,
* regardless of any higher-priority sched classes activity.
*/
- if (!force_scx && rq_modified_above(rq, &ext_sched_class))
- return RETRY_TASK;
+ if (!force_scx && rq_modified_above(rq, &ext_sched_class)) {
+ p = RETRY_TASK;
+ goto out;
+ }
/*
* If we're keeping @prev, replenish slice if necessary and keep running
@@ -3463,7 +3463,7 @@ do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
} else {
p = first_local_task(rq);
if (!p)
- return NULL;
+ goto out;
if (unlikely(!p->scx.slice) && scx_task_can_stay_on_cpu(rq, p)) {
struct scx_sched *sch = scx_task_sched(p);
@@ -3478,6 +3478,40 @@ do_pick_task_scx(struct rq *rq, struct rq_flags *rf, bool force_scx)
}
}
+out:
+ if (cpu_of(rq) != smp_processor_id())
+ return p;
+
+ /*
+ * Balance callbacks must be consumed before the rq lock is released, so
+ * queue them only when the pick is final. On RETRY_TASK, or after a
+ * NULL pick from the dl_server, the core continues picking and can
+ * release the rq lock (e.g., prev_balance() or another dispatch). In
+ * that case, run the deferred work from irq_work, which doesn't depend
+ * on the current rq lock section, and leave the kick sync wait pending
+ * for the next pick.
+ */
+ if (p != RETRY_TASK && (p || !force_scx)) {
+ /* under core scheduling, see dispatch_core_pick() */
+ if (!core_sched)
+ maybe_queue_balance_callback(rq);
+
+ /*
+ * Defer to a balance callback which can drop rq lock and enable
+ * IRQs. Waiting directly in the pick path would deadlock against
+ * CPUs sending us IPIs (e.g., TLB flushes) while we wait for
+ * them.
+ */
+ if (unlikely(rq->scx.kick_sync_pending)) {
+ rq->scx.kick_sync_pending = false;
+ queue_balance_callback(rq, &rq->scx.kick_sync_bal_cb,
+ kick_sync_wait_bal_cb);
+ }
+ } else if (!core_sched && (rq->scx.flags & SCX_RQ_BAL_CB_PENDING)) {
+ rq->scx.flags &= ~SCX_RQ_BAL_CB_PENDING;
+ schedule_deferred(rq);
+ }
+
return p;
}
@@ -4072,6 +4106,14 @@ static void scx_disable_task(struct scx_sched *sch, struct task_struct *p)
scx_set_task_slice(p, 0);
p->scx.reenq_cnt = 0;
+ /*
+ * If @p is runnable but not running, its wait started at runnable_at,
+ * which ops_dequeue() didn't reset. Leaving @sch ends the wait. Reset
+ * runnable_at the next time @p is enqueued on SCX, so that the watchdog
+ * doesn't count the time @p spends elsewhere.
+ */
+ p->scx.flags |= SCX_TASK_RESET_RUNNABLE_AT;
+
/*
* Verify the task is not in BPF scheduler's custody. If flag
* transitions are consistent, the flag should always be clear
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0f2364479f9d..b35e23f93bcd 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2194,7 +2194,7 @@ void init_sched_mm(struct task_struct *p) { }
static void task_tick_cache(struct rq *rq, struct task_struct *p) { }
static inline int get_pref_llc(struct task_struct *p,
- struct mm_struct *mm)
+ struct sched_cache_group *grp)
{
return -1;
}
@@ -3713,7 +3713,7 @@ static void update_task_scan_period(struct task_struct *p,
p->mm->numa_next_scan = jiffies +
msecs_to_jiffies(p->numa_scan_period);
- return;
+ goto out;
}
/*
@@ -3757,7 +3757,10 @@ static void update_task_scan_period(struct task_struct *p,
p->numa_scan_period = clamp(p->numa_scan_period + diff,
task_scan_min(p), task_scan_max(p));
- memset(p->numa_faults_locality, 0, sizeof(p->numa_faults_locality));
+
+out:
+ memset(p->numa_faults_locality, 0,
+ sizeof(p->numa_faults_locality));
}
/*
@@ -10942,6 +10945,8 @@ static inline bool task_misfits_asym_cpu(struct lb_env *env, struct task_struct
return false;
}
+static inline bool sched_asym(struct sched_domain *sd, int dst_cpu, int src_cpu);
+
/*
* Check if task p can migrate from source LLC to
* destination LLC in terms of cache aware load balance.
@@ -10966,6 +10971,10 @@ static enum llc_mig can_migrate_llc_task(struct lb_env *env,
if (cpu < 0 || cpus_share_cache(src_cpu, dst_cpu))
return mig_unrestricted;
+ /* Prioritize asym packing to idle core over cache awareness */
+ if (env->idle && sched_asym(env->sd, dst_cpu, src_cpu))
+ return mig_unrestricted;
+
/* skip cache aware load balance for too many threads */
if (invalid_llc_nr(grp, p, dst_cpu) ||
exceed_llc_capacity(grp, dst_cpu)) {
@@ -12162,6 +12171,14 @@ static inline bool llc_balance(struct lb_env *env, struct sg_lb_stats *sgs,
sgs->group_misfit_task_load)
return false;
+ /*
+ * On asym packing domains, if the destination CPU
+ * has higher priority than all CPUs in the source group,
+ * prioritize asym packing.
+ */
+ if (sgs->group_asym_packing)
+ return false;
+
/*
* Skip cache aware tagging if nr_balanced_failed is sufficiently high.
* Threshold of cache_nice_tries is set to 1 higher than nr_balance_failed
@@ -13577,12 +13594,12 @@ static int need_active_balance(struct lb_env *env)
{
struct sched_domain *sd = env->sd;
- if (alb_break_llc(env))
- return 0;
-
if (asym_active_balance(env))
return 1;
+ if (alb_break_llc(env))
+ return 0;
+
if (imbalanced_active_balance(env))
return 1;
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 86cf4460d69e..94c7ba80a8f7 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,
* We need to check again if the addfd request has been handled,
* and if not, we will remove it from the queue.
*/
- if (list_empty(&kaddfd.list))
+ if (list_empty(&kaddfd.list)) {
ret = kaddfd.ret;
- else
+ } else {
list_del(&kaddfd.list);
+ if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND)
+ knotif->state = SECCOMP_NOTIFY_SENT;
+ }
out_unlock:
mutex_unlock(&filter->notify_lock);
diff --git a/kernel/sys.c b/kernel/sys.c
index 35b538ba843c..755bfa5ccd92 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -1077,6 +1077,9 @@ SYSCALL_DEFINE1(times, struct tms __user *, tbuf)
#ifdef CONFIG_COMPAT
static compat_clock_t clock_t_to_compat_clock_t(clock_t x)
{
+ if (USER_HZ == COMPAT_USER_HZ)
+ return x;
+
return compat_jiffies_to_clock_t(clock_t_to_jiffies(x));
}
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index f1253f5795c6..58c0ca11cdc5 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -673,7 +673,7 @@ static void clocksource_watchdog(struct timer_list *unused)
/* Could have been rearmed by a stop/start cycle */
if (!timer_pending(&watchdog_timer)) {
- watchdog_timer.expires += WATCHDOG_INTERVAL;
+ watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
add_timer_local(&watchdog_timer);
}
}
diff --git a/kernel/time/itimer.c b/kernel/time/itimer.c
index 03a32dffc56c..f7de39769095 100644
--- a/kernel/time/itimer.c
+++ b/kernel/time/itimer.c
@@ -60,7 +60,7 @@ static void get_cpu_itimer(struct task_struct *tsk, unsigned int clock_id,
thread_group_sample_cputime(tsk, samples);
t = samples[clock_id];
- if (val < t)
+ if (val <= t)
/* about to fire */
val = TICK_NSEC;
else
diff --git a/kernel/time/namespace.c b/kernel/time/namespace.c
index 3aff27bb0a15..635f7e544bcc 100644
--- a/kernel/time/namespace.c
+++ b/kernel/time/namespace.c
@@ -51,7 +51,7 @@ ktime_t do_timens_ktime_to_host(clockid_t clockid, ktime_t tim,
tim = 0;
} else {
tim = ktime_sub(tim, offset);
- if (unlikely(tim > KTIME_MAX))
+ if (unlikely(tim < 0))
tim = KTIME_MAX;
}
diff --git a/kernel/time/posix-cpu-timers.c b/kernel/time/posix-cpu-timers.c
index 0bf4fcd969c8..65cc612046d1 100644
--- a/kernel/time/posix-cpu-timers.c
+++ b/kernel/time/posix-cpu-timers.c
@@ -1600,8 +1600,8 @@ static int do_cpu_nanosleep(const clockid_t which_clock, int flags,
*/
expires = cpu_timer_getexpires(&timer.it.cpu);
error = posix_cpu_timer_set(&timer, 0, &zero_it, &it);
- if (!error) {
- /* Timer is now unarmed, deletion can not fail. */
+ if (error != TIMER_RETRY) {
+ /* Unarmed or target gone (-ESRCH), deletion can not fail. */
posix_cpu_timer_del(&timer);
} else {
while (error == TIMER_RETRY) {
diff --git a/kernel/time/posix-stubs.c b/kernel/time/posix-stubs.c
index 9b6fcb8d85e7..5a27335a917e 100644
--- a/kernel/time/posix-stubs.c
+++ b/kernel/time/posix-stubs.c
@@ -83,7 +83,7 @@ SYSCALL_DEFINE2(clock_getres, const clockid_t, which_clock, struct __kernel_time
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_timespec64(&rtn_tp, tp))
+ if (tp && put_timespec64(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
@@ -166,7 +166,7 @@ SYSCALL_DEFINE2(clock_getres_time32, clockid_t, which_clock,
case CLOCK_REALTIME:
case CLOCK_MONOTONIC:
case CLOCK_BOOTTIME:
- if (put_old_timespec32(&rtn_tp, tp))
+ if (tp && put_old_timespec32(&rtn_tp, tp))
return -EFAULT;
return 0;
default:
diff --git a/kernel/time/posix-timers.c b/kernel/time/posix-timers.c
index 436ba794cc0b..4c80c2dada9d 100644
--- a/kernel/time/posix-timers.c
+++ b/kernel/time/posix-timers.c
@@ -694,16 +694,8 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting)
if (iv) {
cur_setting->it_interval = ktime_to_timespec64(iv);
} else if (timr->it_status == POSIX_TIMER_DISARMED) {
- /*
- * SIGEV_NONE oneshot timers are never queued and therefore
- * timr->it_status is always DISARMED. The check below
- * vs. remaining time will handle this case.
- *
- * For all other timers there is nothing to update here, so
- * return.
- */
- if (!sig_none)
- return;
+ /* Nothing to update for a disarmed timer */
+ return;
}
now = kc->clock_get_ktime(timr->it_clock);
@@ -713,7 +705,7 @@ void common_timer_get(struct k_itimer *timr, struct itimerspec64 *cur_setting)
* is a SIGEV_NONE timer move the expiry time forward by intervals,
* so expiry is > now.
*/
- if (iv && timr->it_status != POSIX_TIMER_ARMED)
+ if (iv && (sig_none || timr->it_status != POSIX_TIMER_ARMED))
timr->it_overrun += kc->timer_forward(timr, now);
remaining = kc->timer_remaining(timr, now);
@@ -915,8 +907,15 @@ int common_timer_set(struct k_itimer *timr, int flags,
sigev_none = timr->it_sigev_notify == SIGEV_NONE;
if (kc->timer_arm(timr, expires, flags & TIMER_ABSTIME, sigev_none)) {
- if (!sigev_none)
- timr->it_status = POSIX_TIMER_ARMED;
+ /*
+ * SIGEV_NONE timers are not queued, but mark them armed
+ * so that common_timer_get() can tell them apart from
+ * disarmed timers. An expired SIGEV_NONE timer stays
+ * armed, which is harmless: common_timer_get() reports
+ * zero remaining time for it and it never reaches the
+ * signal delivery code.
+ */
+ timr->it_status = POSIX_TIMER_ARMED;
} else {
/* Timer was already expired, queue the signal */
posix_timer_queue_signal(timr);
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 6c3fea386713..baa3ac9b48f8 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -400,9 +400,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)
return true;
}
+static void tick_nohz_full_update_tick(struct tick_sched *ts);
+
static void nohz_full_kick_func(struct irq_work *work)
{
- /* Empty, the tick restart happens on tick_nohz_irq_exit() */
+ struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched);
+
+ if (WARN_ON_ONCE(!tick_nohz_full_cpu(smp_processor_id())))
+ return;
+
+ /* Task-context flushes may not be followed by an IRQ exit. */
+ if (!in_hardirq()) {
+ if (WARN_ON_ONCE(tick_sched_flag_test(ts, TS_FLAG_INIDLE)))
+ return;
+ tick_nohz_full_update_tick(ts);
+ }
}
static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index ea2e6e55f37b..cae3e51eaf55 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -14,6 +14,7 @@
#include <linux/random.h>
#include <linux/sched/clock.h>
#include <linux/sched/loadavg.h>
+#include <linux/security.h>
#include <linux/static_key.h>
#include <linux/stop_machine.h>
#include <linux/syscore_ops.h>
@@ -2848,6 +2849,7 @@ ktime_t ktime_get_update_offsets_now(u32 *cwsseq, ktime_t *offs_real,
ktime_t *offs_boot, ktime_t *offs_tai)
{
struct timekeeper *tk = &tk_core.timekeeper;
+ u32 cwsseq_orig = *cwsseq;
unsigned int seq;
ktime_t base;
u64 nsecs;
@@ -2859,7 +2861,7 @@ ktime_t ktime_get_update_offsets_now(u32 *cwsseq, ktime_t *offs_real,
nsecs = timekeeping_get_ns(&tk->tkr_mono);
base = ktime_add_ns(base, nsecs);
- if (*cwsseq != tk->clock_was_set_seq) {
+ if (cwsseq_orig != tk->clock_was_set_seq) {
*cwsseq = tk->clock_was_set_seq;
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
@@ -3215,11 +3217,15 @@ static int aux_clock_set(const clockid_t id, const struct timespec64 *tnew)
struct tk_data *aux_tkd = aux_get_tk_data(id);
struct timekeeper *aux_tks;
ktime_t tnow, nsecs;
+ int ret;
if (!timespec64_valid_settod(tnew))
return -EINVAL;
if (!aux_tkd)
return -ENODEV;
+ ret = security_settime64(tnew, NULL);
+ if (ret)
+ return ret;
aux_tks = &aux_tkd->shadow_timekeeper;
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 0ab5916575a9..6587450cfcd1 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -1329,6 +1329,7 @@ source "kernel/trace/rv/Kconfig"
config TRACE_REMOTE
bool
+ select TRACING
config SIMPLE_RING_BUFFER
bool
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 9c4e40847e43..1a1219b6d0fc 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -7466,6 +7466,8 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
if (psize > RB_WRITE_MASK + 1)
return -EINVAL;
+ guard(cpus_read_lock)();
+
/* prevent another thread from changing buffer sizes */
guard(mutex)(&buffer->mutex);
diff --git a/kernel/ucount.c b/kernel/ucount.c
index ec8b1445e287..c3de5b94d25f 100644
--- a/kernel/ucount.c
+++ b/kernel/ucount.c
@@ -308,23 +308,26 @@ long inc_rlimit_get_ucounts(struct ucounts *ucounts, enum rlimit_type type,
for (iter = ucounts; iter; iter = iter->ns->ucounts) {
long new = atomic_long_add_return(1, &iter->rlimit[type]);
- if (new < 0 || new > max)
+ /*
+ * Take the extra reference on the 0 -> 1 transition before the
+ * limit check: a rejected walk must not leave a concurrent one
+ * holding the surviving increment with no reference.
+ */
+ if (new == 1 && !get_ucounts(iter))
goto dec_unwind;
+ if (new < 0 || new > max) {
+ /* undo iter too, releasing the reference taken above */
+ do_dec_rlimit_put_ucounts(ucounts, iter->ns->ucounts, type);
+ return 0;
+ }
if (iter == ucounts)
ret = new;
if (!override_rlimit)
max = get_userns_rlimit_max(iter->ns, type);
- /*
- * Grab an extra ucount reference for the caller when
- * the rlimit count was previously 0.
- */
- if (new != 1)
- continue;
- if (!get_ucounts(iter))
- goto dec_unwind;
}
return ret;
dec_unwind:
+ /* get_ucounts() failed for iter: undo iter without a put */
dec = atomic_long_sub_return(1, &iter->rlimit[type]);
WARN_ON_ONCE(dec < 0);
do_dec_rlimit_put_ucounts(ucounts, iter, type);
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index 0bed462e9b2a..af06883dff0b 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -45,8 +45,11 @@ static void set_cred_user_ns(struct cred *cred, struct user_namespace *user_ns)
{
/* Start with the same capabilities as init but useless for doing
* anything as the capabilities are bound to the new user namespace.
+ * The exec securebits grant nothing, they are restrictions the task
+ * placed on itself. Keep them and their locks.
*/
- cred->securebits = SECUREBITS_DEFAULT;
+ cred->securebits = SECUREBITS_DEFAULT |
+ (cred->securebits & (SECURE_ALL_UNPRIVILEGED | SECURE_ALL_UNPRIVILEGED << 1));
cred->cap_inheritable = CAP_EMPTY_SET;
cred->cap_permitted = CAP_FULL_SET;
cred->cap_effective = CAP_FULL_SET;
diff --git a/lib/842/842_decompress.c b/lib/842/842_decompress.c
index 582085ef8b49..45a9815abd63 100644
--- a/lib/842/842_decompress.c
+++ b/lib/842/842_decompress.c
@@ -165,6 +165,9 @@ static int __do_index(struct sw842_param *p, u8 size, u8 bits, u64 fsize)
u64 index, offset, total = round_down(p->out - p->ostart, 8);
int ret;
+ if (size > p->olen)
+ return -ENOSPC;
+
ret = next_bits(p, &index, bits);
if (ret)
return ret;
@@ -306,7 +309,7 @@ int sw842_decompress(const u8 *in, unsigned int ilen,
if (ret)
return ret;
- if (p.out == out) /* no previous bytes */
+ if (p.out - p.ostart < 8) /* no complete previous block */
return -EINVAL;
/* copy rep + 1 */
@@ -344,6 +347,8 @@ int sw842_decompress(const u8 *in, unsigned int ilen,
if (!bytes || bytes > SHORT_DATA_BITS_MAX)
return -EINVAL;
+ if (bytes > p.olen)
+ return -ENOSPC;
while (bytes-- > 0) {
ret = next_bits(&p, &tmp, 8);
diff --git a/lib/buildid.c b/lib/buildid.c
index c4b737640621..995a327f46ae 100644
--- a/lib/buildid.c
+++ b/lib/buildid.c
@@ -47,6 +47,10 @@ static int freader_get_folio(struct freader *r, loff_t file_off)
freader_put_folio(r);
+ /* the page index must fit in pgoff_t */
+ if (file_off < 0 || file_off > MAX_LFS_FILESIZE)
+ return -EFAULT;
+
/* only use page cache lookup - fail if not already cached */
r->folio = filemap_get_folio(r->file->f_mapping, file_off >> PAGE_SHIFT);
diff --git a/lib/cmdline.c b/lib/cmdline.c
index 16cce6621cec..40b98d943a9a 100644
--- a/lib/cmdline.c
+++ b/lib/cmdline.c
@@ -11,6 +11,7 @@
#include <linux/export.h>
#include <linux/kernel.h>
+#include <linux/overflow.h>
#include <linux/string.h>
#include <linux/ctype.h>
@@ -26,7 +27,9 @@ static int get_range(char **str, int *pint, int n)
(*str)++;
upper_range = simple_strtol((*str), NULL, 0);
- inc_counter = upper_range - *pint;
+ /* Keep the sign of the result when the difference doesn't fit */
+ if (check_sub_overflow(upper_range, *pint, &inc_counter))
+ inc_counter = upper_range < *pint ? -1 : INT_MAX;
for (x = *pint; n && x < upper_range; x++, n--)
*pint++ = x;
return inc_counter;
@@ -122,6 +125,14 @@ char *get_options(const char *str, int nints, int *ints)
range_nums = get_range((char **)&str, pint, n);
if (range_nums < 0)
break;
+ /* The range didn't fit, so the array is full */
+ if (!validate && range_nums > n) {
+ i = nints;
+ break;
+ }
+ /* Leave room for the upper number of the range */
+ if (range_nums >= INT_MAX - i)
+ break;
/*
* Decrement the result by one to leave out the
* last number in the range. The next iteration
diff --git a/lib/decompress_bunzip2.c b/lib/decompress_bunzip2.c
index 1288f146661f..a9236d3e9678 100644
--- a/lib/decompress_bunzip2.c
+++ b/lib/decompress_bunzip2.c
@@ -428,6 +428,10 @@ static int INIT get_next_block(struct bunzip_data *bd)
t += (runPos << nextSym);
/* +runPos if RUNA; +2*runPos if RUNB */
+ /* Bound the run so t and runPos cannot overflow. */
+ if (t >= dbufSize)
+ return RETVAL_DATA_ERROR;
+
runPos <<= 1;
continue;
}
@@ -439,7 +443,7 @@ static int INIT get_next_block(struct bunzip_data *bd)
array.) */
if (runPos) {
runPos = 0;
- if (dbufCount+t >= dbufSize)
+ if (dbufCount+t > dbufSize)
return RETVAL_DATA_ERROR;
uc = symToByte[mtfSymbol[0]];
diff --git a/lib/fault-inject.c b/lib/fault-inject.c
index 999053fa133e..02916ef2761c 100644
--- a/lib/fault-inject.c
+++ b/lib/fault-inject.c
@@ -5,6 +5,7 @@
#include <linux/debugfs.h>
#include <linux/sched.h>
#include <linux/stat.h>
+#include <linux/string.h>
#include <linux/types.h>
#include <linux/fs.h>
#include <linux/export.h>
@@ -64,7 +65,7 @@ static void fail_dump(struct fault_attr *attr)
{
if (attr->verbose > 0 && __ratelimit(&attr->ratelimit_state)) {
printk(KERN_NOTICE "FAULT_INJECTION: forcing a failure.\n"
- "name %pd, interval %lu, probability %lu, "
+ "name %s, interval %lu, probability %lu, "
"space %d, times %d\n", attr->dname,
attr->interval, attr->probability,
atomic_read(&attr->space),
@@ -261,7 +262,9 @@ struct dentry *fault_create_debugfs_attr(const char *name,
debugfs_create_xul("reject-end", mode, dir, &attr->reject_end);
#endif /* CONFIG_FAULT_INJECTION_STACKTRACE_FILTER */
- attr->dname = dget(dir);
+ if (strscpy(attr->dname, name, sizeof(attr->dname)) == -E2BIG)
+ pr_warn("FAULT_INJECTION: name '%s' truncated to '%s'\n",
+ name, attr->dname);
return dir;
}
EXPORT_SYMBOL_GPL(fault_create_debugfs_attr);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 2072c04e99d0..56c59a2b589d 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -848,6 +848,10 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
return 0;
}
+ /* An empty iterator may have no segment array at all. */
+ if (!i->count)
+ return 0;
+
/* iovec and kvec have identical layouts */
if (likely(iter_is_iovec(i) || iov_iter_is_kvec(i)))
return iov_iter_alignment_iovec(i);
diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index 6ecd39e4b704..84853e096811 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -83,7 +83,7 @@ static int kobject_action_type(const char *buf, size_t count,
for (action = 0; action < ARRAY_SIZE(kobject_actions); action++) {
if (strncmp(kobject_actions[action], buf, count_first) != 0)
continue;
- if (kobject_actions[action][count_first] != '\0')
+ if (strlen(kobject_actions[action]) != count_first)
continue;
if (args)
*args = args_start;
diff --git a/lib/plist.c b/lib/plist.c
index a5bef38add43..b0273533c04d 100644
--- a/lib/plist.c
+++ b/lib/plist.c
@@ -174,8 +174,15 @@ void plist_requeue(struct plist_node *node, struct plist_head *head)
/*
* After plist_del(), iter is the replacement of the node. If the node
* was on prio_list, take shortcut to find node_next instead of looping.
+ *
+ * prio_list is a headless ring, so from the LAST bucket ->next wraps
+ * round to the first one; in that case node_next is the list head.
*/
if (!list_empty(&iter->prio_list)) {
+ struct plist_node *first = plist_first(head);
+
+ if (iter->prio_list.next == &first->prio_list)
+ goto queue;
iter = list_entry(iter->prio_list.next, struct plist_node,
prio_list);
node_next = &iter->node_list;
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 7d829c669ea7..3cc2fe7ac42c 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -67,6 +67,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -115,6 +116,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -175,6 +177,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -243,6 +246,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -334,6 +338,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -444,6 +449,7 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
asm volatile("vmovntdq %%ymm14,%0" : "=m" (q[d+96]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index e671eb5bde63..772bfc4af6df 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -78,6 +78,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -137,6 +138,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -208,6 +210,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -292,6 +295,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -396,6 +400,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -529,6 +534,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
"m" (q[d+128]), "m" (q[d+192]));
}
asm volatile("sfence" : : : "memory");
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
const struct raid6_calls raid6_avx512x4 = {
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index a714a780a2d8..820871046e30 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -176,6 +176,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -293,6 +294,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index ec72d5a30c01..299a3f044d61 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -211,6 +211,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
@@ -353,6 +354,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
#endif
}
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/raid/xor/x86/xor-avx.c b/lib/raid/xor/x86/xor-avx.c
index f7777d7aa269..95b21e7225e8 100644
--- a/lib/raid/xor/x86/xor-avx.c
+++ b/lib/raid/xor/x86/xor-avx.c
@@ -147,6 +147,7 @@ static void xor_gen_avx(void *dest, void **srcs, unsigned int src_cnt,
{
kernel_fpu_begin();
xor_gen_avx_inner(dest, srcs, src_cnt, bytes);
+ asm volatile("vzeroupper");
kernel_fpu_end();
}
diff --git a/lib/stackdepot.c b/lib/stackdepot.c
index dd2717ff94bf..90c52f2e0d3f 100644
--- a/lib/stackdepot.c
+++ b/lib/stackdepot.c
@@ -323,7 +323,7 @@ static bool depot_init_pool(void **prealloc)
* NULL; do not reset to NULL if we have reached the maximum number of
* pools.
*/
- if (pools_num < stack_max_pools)
+ if (pools_num + 1 < stack_max_pools)
WRITE_ONCE(new_pool, NULL);
else
WRITE_ONCE(new_pool, STACK_DEPOT_POISON);
diff --git a/lib/tests/cmdline_kunit.c b/lib/tests/cmdline_kunit.c
index 3f61ff8d3178..12b85f255b65 100644
--- a/lib/tests/cmdline_kunit.c
+++ b/lib/tests/cmdline_kunit.c
@@ -140,6 +140,44 @@ static void cmdline_test_range(struct kunit *test)
} while (++i < ARRAY_SIZE(cmdline_test_range_strings));
}
+static const struct {
+ const char *in;
+ int parsed[4];
+ int validated;
+} cmdline_test_range_overflow_cases[] = {
+ { "1-100", { 3, 1, 2, 3, }, 100, },
+ { "1,5-100,7", { 3, 1, 5, 6, }, 98, },
+ { "1-2147483646", { 3, 1, 2, 3, }, 2147483646, },
+ { "0-2147483647", { 3, 0, 1, 2, }, 0, },
+ { "-5-2147483647", { 3, -5, -4, -3, }, 0, },
+ { "2147483647--5", { 0, 2147483647, }, 0, },
+ { "1,0-2147483647", { 3, 1, 0, 1, }, 1, },
+};
+
+static void cmdline_test_range_overflow(struct kunit *test)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(cmdline_test_range_overflow_cases); i++) {
+ const char *in = cmdline_test_range_overflow_cases[i].in;
+ const int *e = cmdline_test_range_overflow_cases[i].parsed;
+ /* Two guard elements past the array handed to get_options() */
+ int r[ARRAY_SIZE(cmdline_test_range_overflow_cases[0].parsed) + 2];
+ int n;
+
+ memset(r, 0, sizeof(r));
+ get_options(in, ARRAY_SIZE(r) - 2, r);
+ for (j = 0; j < ARRAY_SIZE(r) - 2; j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], e[j], "Pattern: %s at %u", in, j);
+ for (; j < ARRAY_SIZE(r); j++)
+ KUNIT_EXPECT_EQ_MSG(test, r[j], 0, "Pattern: %s out of bound at %u", in, j);
+
+ get_options(in, 0, &n);
+ KUNIT_EXPECT_EQ_MSG(test, n, cmdline_test_range_overflow_cases[i].validated,
+ "Pattern: %s (validated)", in);
+ }
+}
+
static void cmdline_test_next_arg_quoted_value(struct kunit *test)
{
char in[] = "foo=\"bar baz\" qux=1";
@@ -258,6 +296,7 @@ static struct kunit_case cmdline_test_cases[] = {
KUNIT_CASE(cmdline_test_lead_int),
KUNIT_CASE(cmdline_test_tail_int),
KUNIT_CASE(cmdline_test_range),
+ KUNIT_CASE(cmdline_test_range_overflow),
KUNIT_CASE(cmdline_test_next_arg_quoted_value),
KUNIT_CASE(cmdline_test_next_arg_bare_quote_regression),
KUNIT_CASE(cmdline_test_next_arg_mixed_tokens),
diff --git a/mm/alloc_tag.c b/mm/alloc_tag.c
index b33410310477..82e2c3448dcf 100644
--- a/mm/alloc_tag.c
+++ b/mm/alloc_tag.c
@@ -621,7 +621,7 @@ void __init alloc_tag_sec_init(void)
kernel_tags.count = last_codetag - kernel_tags.first_tag;
/* Check if kernel tags fit into page flags */
- if (kernel_tags.count > (1UL << NR_UNUSED_PAGEFLAG_BITS)) {
+ if (CODETAG_ID_FIRST + kernel_tags.count > (1UL << NR_UNUSED_PAGEFLAG_BITS)) {
shutdown_mem_profiling(false); /* allocinfo file does not exist yet */
pr_err("%lu allocation tags cannot be references using %d available page flag bits. Memory allocation profiling is disabled!\n",
kernel_tags.count, NR_UNUSED_PAGEFLAG_BITS);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 74d22a596fc1..e018635774be 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -635,6 +635,8 @@ struct damos_quota_goal *damos_new_quota_goal(
return NULL;
goal->metric = metric;
goal->target_value = target_value;
+ if (metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
+ goal->last_psi_total = U64_MAX;
INIT_LIST_HEAD(&goal->list);
return goal;
}
@@ -666,7 +668,7 @@ static bool damos_quota_goals_empty(struct damos_quota *q)
}
/* initialize fields of @quota that normally API users wouldn't set */
-static struct damos_quota *damos_quota_init(struct damos_quota *quota)
+static void damos_quota_init(struct damos_quota *quota)
{
quota->esz = 0;
quota->total_charged_sz = 0;
@@ -676,7 +678,6 @@ static struct damos_quota *damos_quota_init(struct damos_quota *quota)
quota->charge_target_from = NULL;
quota->charge_addr_from = 0;
quota->esz_bp = 0;
- return quota;
}
struct damos *damon_new_scheme(struct damos_access_pattern *pattern,
@@ -708,7 +709,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern,
scheme->last_applied = NULL;
INIT_LIST_HEAD(&scheme->list);
- scheme->quota = *(damos_quota_init(quota));
+ scheme->quota = *quota;
+ damos_quota_init(&scheme->quota);
/* quota.goals should be separately set by caller */
INIT_LIST_HEAD(&scheme->quota.goals);
@@ -1122,6 +1124,9 @@ static void damos_commit_quota_goal_union(
struct damos_quota_goal *dst, struct damos_quota_goal *src)
{
switch (dst->metric) {
+ case DAMOS_QUOTA_SOME_MEM_PSI_US:
+ dst->last_psi_total = U64_MAX;
+ break;
case DAMOS_QUOTA_NODE_MEM_USED_BP:
case DAMOS_QUOTA_NODE_MEM_FREE_BP:
dst->nid = src->nid;
@@ -1131,6 +1136,9 @@ static void damos_commit_quota_goal_union(
dst->nid = src->nid;
dst->memcg_id = src->memcg_id;
break;
+ case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+ dst->nid = src->nid;
+ break;
default:
break;
}
@@ -1145,7 +1153,6 @@ static int damos_commit_quota_goal(
dst->target_value = src->target_value;
if (dst->metric == DAMOS_QUOTA_USER_INPUT)
dst->current_value = src->current_value;
- /* keep last_psi_total as is, since it will be updated in next cycle */
damos_commit_quota_goal_union(dst, src);
return 0;
}
@@ -2401,13 +2408,14 @@ static void damos_update_stat(struct damos *s,
static bool damos_filter_match(struct damon_ctx *ctx, struct damon_target *t,
struct damon_region *r, struct damos_filter *filter,
- unsigned long min_region_sz)
+ unsigned long min_region_sz, int *err)
{
bool matched = false;
struct damon_target *ti;
int target_idx = 0;
unsigned long start, end;
+ *err = 0;
switch (filter->type) {
case DAMOS_FILTER_TYPE_TARGET:
damon_for_each_target(ti, ctx) {
@@ -2433,12 +2441,13 @@ static bool damos_filter_match(struct damon_ctx *ctx, struct damon_target *t,
}
/* start before the range and overlap */
if (r->ar.start < start) {
- damon_split_region_at(t, r, start - r->ar.start);
+ *err = damon_split_region_at(t, r,
+ start - r->ar.start);
matched = false;
break;
}
/* start inside the range */
- damon_split_region_at(t, r, end - r->ar.start);
+ *err = damon_split_region_at(t, r, end - r->ar.start);
matched = true;
break;
default:
@@ -2452,10 +2461,16 @@ static bool damos_core_filter_out(struct damon_ctx *ctx, struct damon_target *t,
struct damon_region *r, struct damos *s)
{
struct damos_filter *filter;
+ bool matching;
+ int err;
s->core_filters_allowed = false;
damos_for_each_core_filter(filter, s) {
- if (damos_filter_match(ctx, t, r, filter, ctx->min_region_sz)) {
+ matching = damos_filter_match(ctx, t, r, filter,
+ ctx->min_region_sz, &err);
+ if (err)
+ return true;
+ if (matching) {
if (filter->allow)
s->core_filters_allowed = true;
return !filter->allow;
@@ -2626,10 +2641,12 @@ static void damos_apply_scheme(struct damon_ctx *c, struct damon_target *t,
c->min_region_sz);
if (!sz)
goto update_stat;
- damon_split_region_at(t, r, sz);
+ if (damon_split_region_at(t, r, sz))
+ goto update_stat;
}
if (damos_core_filter_out(c, t, r, s))
return;
+ sz = damon_sz_region(r);
ktime_get_coarse_ts64(&begin);
trace_damos_before_apply(cidx, sidx, tidx, r, nr_accesses,
damon_nr_regions(t), do_trace);
@@ -2808,6 +2825,28 @@ static inline u64 damos_get_some_mem_psi_total(void)
#endif /* CONFIG_PSI */
+static void damos_set_psi_current_val(u64 now_psi_total,
+ struct damos_quota_goal *goal, struct damos *s)
+{
+ u64 last_psi_total = goal->last_psi_total;
+
+ goal->last_psi_total = now_psi_total;
+ if (last_psi_total != U64_MAX) {
+ goal->current_value = now_psi_total - last_psi_total;
+ return;
+ }
+ /* uninitialized last_psi_total; make no effect this round */
+ if (s->quota.goal_tuner == DAMOS_QUOTA_GOAL_TUNER_CONSIST) {
+ goal->current_value = goal->target_value;
+ return;
+ }
+ /* let temporal tuner show the same achievement as in the last round */
+ if (!s->quota.esz)
+ goal->current_value = goal->target_value;
+ else
+ goal->current_value = 0;
+}
+
#ifdef CONFIG_NUMA
static bool invalid_mem_node(int nid)
{
@@ -2828,6 +2867,13 @@ static __kernel_ulong_t damos_get_node_mem_bp(
}
si_meminfo_node(&i, goal->nid);
+ if (!i.totalram || i.totalram < i.freeram) {
+ if (goal->metric == DAMOS_QUOTA_NODE_MEM_USED_BP)
+ return 10000;
+ else /* DAMOS_QUOTA_NODE_MEM_FREE_BP */
+ return 0;
+ }
+
if (goal->metric == DAMOS_QUOTA_NODE_MEM_USED_BP)
numerator = i.totalram - i.freeram;
else /* DAMOS_QUOTA_NODE_MEM_FREE_BP */
@@ -2868,6 +2914,12 @@ static unsigned long damos_get_node_memcg_used_bp(
mem_cgroup_put(memcg);
si_meminfo_node(&i, goal->nid);
+ if (!i.totalram || i.totalram < used_pages) {
+ if (goal->metric == DAMOS_QUOTA_NODE_MEMCG_USED_BP)
+ return 10000;
+ else /* DAMOS_QUOTA_NODE_MEMCG_FREE_BP */
+ return 0;
+ }
if (goal->metric == DAMOS_QUOTA_NODE_MEMCG_USED_BP)
numerator = used_pages;
else /* DAMOS_QUOTA_NODE_MEMCG_FREE_BP */
@@ -3014,7 +3066,7 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
global_node_page_state(NR_LRU_BASE + LRU_ACTIVE_FILE);
inactive = global_node_page_state(NR_LRU_BASE + LRU_INACTIVE_ANON) +
global_node_page_state(NR_LRU_BASE + LRU_INACTIVE_FILE);
- total = active + inactive;
+ total = max(active + inactive, 1);
if (active_ratio)
return mult_frac(active, 10000, total);
return mult_frac(inactive, 10000, total);
@@ -3031,8 +3083,7 @@ static void damos_set_quota_goal_current_value(struct damon_ctx *c,
break;
case DAMOS_QUOTA_SOME_MEM_PSI_US:
now_psi_total = damos_get_some_mem_psi_total();
- goal->current_value = now_psi_total - goal->last_psi_total;
- goal->last_psi_total = now_psi_total;
+ damos_set_psi_current_val(now_psi_total, goal, s);
break;
case DAMOS_QUOTA_NODE_MEM_USED_BP:
case DAMOS_QUOTA_NODE_MEM_FREE_BP:
@@ -3090,7 +3141,7 @@ static void damos_goal_tune_esz_bp_temporal(struct damon_ctx *c,
if (score >= 10000)
quota->esz_bp = 0;
- else if (quota->sz)
+ else if (quota->sz && quota->sz <= ULONG_MAX / 10000)
quota->esz_bp = quota->sz * 10000;
else
quota->esz_bp = ULONG_MAX;
diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
index 5c6c3a597fd0..9ddd1ec8202b 100644
--- a/mm/damon/paddr.c
+++ b/mm/damon/paddr.c
@@ -264,7 +264,7 @@ static unsigned long damon_pa_pageout(struct damon_region *r,
else
list_add(&folio->lru, &folio_list);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
if (install_young_filter)
@@ -302,7 +302,7 @@ static inline unsigned long damon_pa_de_activate(
folio_deactivate(folio);
applied += folio_nr_pages(folio);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
s->last_applied = folio;
@@ -350,7 +350,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r,
folio_is_file_lru(folio));
list_add(&folio->lru, &folio_list);
put_folio:
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
applied = damon_migrate_pages(&folio_list, s->target_nid);
@@ -379,7 +379,7 @@ static unsigned long damon_pa_stat(struct damon_region *r,
if (!damos_pa_filter_out(s, folio))
*sz_filter_passed += folio_size(folio) / addr_unit;
- addr += folio_size(folio);
+ addr = PFN_PHYS(folio_pfn(folio)) + folio_size(folio);
folio_put(folio);
}
s->last_applied = folio;
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 32f495a96b17..974c456a9e09 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -2822,12 +2822,12 @@ static int damon_sysfs_add_scheme_filters(struct damos *scheme,
return err;
}
} else if (filter->type == DAMOS_FILTER_TYPE_ADDR) {
- if (sysfs_filter->addr_range.end <
- sysfs_filter->addr_range.start) {
+ filter->addr_range = sysfs_filter->addr_range;
+ if (filter->addr_range.end <
+ filter->addr_range.start) {
damos_destroy_filter(filter);
return -EINVAL;
}
- filter->addr_range = sysfs_filter->addr_range;
} else if (filter->type == DAMOS_FILTER_TYPE_TARGET) {
filter->target_idx = sysfs_filter->target_idx;
} else if (filter->type == DAMOS_FILTER_TYPE_HUGEPAGE_SIZE) {
diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c
index e3858ffab4b2..f05b256c90ee 100644
--- a/mm/damon/sysfs.c
+++ b/mm/damon/sysfs.c
@@ -1789,6 +1789,7 @@ struct damon_sysfs_kdamond {
struct damon_sysfs_contexts *contexts;
struct damon_ctx *damon_ctx;
unsigned int refresh_ms;
+ unsigned long next_refresh_jiffies;
};
static struct damon_sysfs_kdamond *damon_sysfs_kdamond_alloc(void)
@@ -2215,17 +2216,15 @@ static struct damon_ctx *damon_sysfs_build_ctx(
return ctx;
}
-static unsigned long damon_sysfs_next_update_jiffies;
-
static int damon_sysfs_repeat_call_fn(void *data)
{
struct damon_sysfs_kdamond *sysfs_kdamond = data;
if (!sysfs_kdamond->refresh_ms)
return 0;
- if (time_before(jiffies, damon_sysfs_next_update_jiffies))
+ if (time_before(jiffies, sysfs_kdamond->next_refresh_jiffies))
return 0;
- damon_sysfs_next_update_jiffies = jiffies +
+ sysfs_kdamond->next_refresh_jiffies = jiffies +
msecs_to_jiffies(sysfs_kdamond->refresh_ms);
if (!mutex_trylock(&damon_sysfs_lock))
@@ -2273,8 +2272,8 @@ static int damon_sysfs_turn_damon_on(struct damon_sysfs_kdamond *kdamond)
}
kdamond->damon_ctx = ctx;
- damon_sysfs_next_update_jiffies =
- jiffies + msecs_to_jiffies(kdamond->refresh_ms);
+ kdamond->next_refresh_jiffies = jiffies +
+ msecs_to_jiffies(kdamond->refresh_ms);
repeat_call_control->fn = damon_sysfs_repeat_call_fn;
repeat_call_control->data = kdamond;
diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h
index 4a536d41cdb2..65a09d29175d 100644
--- a/mm/damon/tests/core-kunit.h
+++ b/mm/damon/tests/core-kunit.h
@@ -751,19 +751,16 @@ static void damos_test_commit_quota_goal_for(struct kunit *test,
struct damos_quota_goal *dst,
struct damos_quota_goal *src)
{
- u64 dst_last_psi_total = 0;
-
- if (dst->metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
- dst_last_psi_total = dst->last_psi_total;
damos_commit_quota_goal(dst, src);
KUNIT_EXPECT_EQ(test, dst->metric, src->metric);
KUNIT_EXPECT_EQ(test, dst->target_value, src->target_value);
if (src->metric == DAMOS_QUOTA_USER_INPUT)
KUNIT_EXPECT_EQ(test, dst->current_value, src->current_value);
- if (dst_last_psi_total && src->metric == DAMOS_QUOTA_SOME_MEM_PSI_US)
- KUNIT_EXPECT_EQ(test, dst->last_psi_total, dst_last_psi_total);
switch (dst->metric) {
+ case DAMOS_QUOTA_SOME_MEM_PSI_US:
+ KUNIT_EXPECT_EQ(test, dst->last_psi_total, U64_MAX);
+ break;
case DAMOS_QUOTA_NODE_MEM_USED_BP:
case DAMOS_QUOTA_NODE_MEM_FREE_BP:
KUNIT_EXPECT_EQ(test, dst->nid, src->nid);
@@ -787,6 +784,13 @@ static void damos_test_commit_quota_goal(struct kunit *test)
.last_psi_total = 456,
};
+ damos_test_commit_quota_goal_for(test, &dst,
+ &(struct damos_quota_goal) {
+ .metric = DAMOS_QUOTA_SOME_MEM_PSI_US,
+ .target_value = 234,
+ .current_value = 345,
+ .last_psi_total = 567,
+ });
damos_test_commit_quota_goal_for(test, &dst,
&(struct damos_quota_goal){
.metric = DAMOS_QUOTA_USER_INPUT,
@@ -1336,6 +1340,7 @@ static void damos_test_filter_out(struct kunit *test)
struct damon_target *t;
struct damon_region *r, *r2;
struct damos_filter *f;
+ int err;
f = damos_new_filter(DAMOS_FILTER_TYPE_ADDR, true, false);
if (!f)
@@ -1356,27 +1361,27 @@ static void damos_test_filter_out(struct kunit *test)
damon_add_region(r, t);
/* region in the range */
- KUNIT_EXPECT_TRUE(test, damos_filter_match(NULL, t, r, f, 1));
+ KUNIT_EXPECT_TRUE(test, damos_filter_match(NULL, t, r, f, 1, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region before the range */
r->ar.start = 1;
r->ar.end = 2;
KUNIT_EXPECT_FALSE(test,
- damos_filter_match(NULL, t, r, f, 1));
+ damos_filter_match(NULL, t, r, f, 1, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region after the range */
r->ar.start = 6;
r->ar.end = 8;
KUNIT_EXPECT_FALSE(test,
- damos_filter_match(NULL, t, r, f, 1));
+ damos_filter_match(NULL, t, r, f, 1, &err));
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1);
/* region started before the range */
r->ar.start = 1;
r->ar.end = 4;
- KUNIT_EXPECT_FALSE(test, damos_filter_match(NULL, t, r, f, 1));
+ KUNIT_EXPECT_FALSE(test, damos_filter_match(NULL, t, r, f, 1, &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, 1);
KUNIT_EXPECT_EQ(test, r->ar.end, 2);
@@ -1392,7 +1397,7 @@ static void damos_test_filter_out(struct kunit *test)
r->ar.start = 2;
r->ar.end = 8;
KUNIT_EXPECT_TRUE(test,
- damos_filter_match(NULL, t, r, f, 1));
+ damos_filter_match(NULL, t, r, f, 1, &err));
/* filter should have split the region */
KUNIT_EXPECT_EQ(test, r->ar.start, 2);
KUNIT_EXPECT_EQ(test, r->ar.end, 6);
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 04ee2a2c6a4d..5b4d16c8db62 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -676,6 +676,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
for (; addr < next; pte += nr, addr += nr * PAGE_SIZE) {
+ unsigned long page_idx;
+
nr = 1;
ptent = ptep_get(pte);
@@ -688,7 +690,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr,
continue;
damos_va_migrate_dests_add(folio, walk->vma, addr, dests,
migration_lists);
- nr = folio_nr_pages(folio);
+ page_idx = folio_page_idx(folio, pte_page(ptent));
+ nr = folio_nr_pages(folio) - page_idx;
}
pte_unmap_unlock(start_pte, ptl);
return 0;
@@ -838,6 +841,8 @@ static int damos_va_stat_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
for (; addr < next; pte += nr, addr += nr * PAGE_SIZE) {
+ unsigned long page_idx;
+
nr = 1;
ptent = ptep_get(pte);
@@ -851,7 +856,8 @@ static int damos_va_stat_pmd_entry(pmd_t *pmd, unsigned long addr,
if (!damos_va_filter_out(s, folio, vma, addr, pte, NULL))
*sz_filter_passed += folio_size(folio);
- nr = folio_nr_pages(folio);
+ page_idx = folio_page_idx(folio, pte_page(ptent));
+ nr = folio_nr_pages(folio) - page_idx;
s->last_applied = folio;
}
pte_unmap_unlock(start_pte, ptl);
diff --git a/mm/execmem.c b/mm/execmem.c
index 74a178a87e75..33e4e68f655f 100644
--- a/mm/execmem.c
+++ b/mm/execmem.c
@@ -165,9 +165,11 @@ static void execmem_cache_clean(struct work_struct *work)
mutex_lock(mutex);
mas_for_each(&mas, area, ULONG_MAX) {
+ struct vm_struct *vm = find_vm_area(area);
size_t size = mas_range_len(&mas);
- if (IS_ALIGNED(size, PMD_SIZE) &&
+ if (vm && get_vm_area_size(vm) == size &&
+ IS_ALIGNED(size, PMD_SIZE) &&
IS_ALIGNED(mas.index, PMD_SIZE)) {
struct vm_struct *vm = find_vm_area(area);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 1e5d68acf62a..779c02e0bf6c 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2423,6 +2423,10 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,
}
folio = pmd_folio(orig_pmd);
+
+ if (folio_is_zone_device(folio))
+ goto out;
+
/*
* If other processes are mapping this folio, we couldn't discard
* the folio unless they all do MADV_FREE so let's skip the folio.
@@ -4634,22 +4638,11 @@ static enum lru_status deferred_split_isolate(struct list_head *item,
struct folio *folio = container_of(item, struct folio, _deferred_list);
struct list_head *freeable = cb_arg;
- if (folio_try_get(folio)) {
- list_lru_isolate_move(lru, item, freeable);
- return LRU_REMOVED;
- }
+ /* Lost race to folio_put() or the folio is under folio_ref_freeze() */
+ if (!folio_try_get(folio))
+ return LRU_SKIP;
- /*
- * We lost race with folio_put(). Read folio state before the
- * isolate: folio_unqueue_deferred_split() checks list_empty()
- * locklessly, so once removed the folio can be freed any time.
- */
- if (folio_test_partially_mapped(folio)) {
- folio_clear_partially_mapped(folio);
- mod_mthp_stat(folio_order(folio),
- MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1);
- }
- list_lru_isolate(lru, item);
+ list_lru_isolate_move(lru, item, freeable);
return LRU_REMOVED;
}
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 86928d76ef12..e71e0ab116bf 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -23,6 +23,7 @@
#include <linux/mmdebug.h>
#include <linux/sched/signal.h>
#include <linux/rmap.h>
+#include <linux/rcupdate.h>
#include <linux/string_choices.h>
#include <linux/string_helpers.h>
#include <linux/swap.h>
@@ -2191,6 +2192,11 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
h->resv_huge_pages--;
spin_unlock_irq(&hugetlb_lock);
+
+ if (folio)
+ lruvec_stat_mod_folio(folio, NR_HUGETLB,
+ folio_nr_pages(folio));
+
return folio;
}
@@ -2198,24 +2204,30 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid,
struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid,
nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback)
{
- spin_lock_irq(&hugetlb_lock);
- if (available_huge_pages(h)) {
- struct folio *folio;
+ struct folio *folio = NULL;
+ spin_lock_irq(&hugetlb_lock);
+ if (available_huge_pages(h))
folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask,
preferred_nid, nmask);
- if (folio) {
- spin_unlock_irq(&hugetlb_lock);
- return folio;
- }
- }
spin_unlock_irq(&hugetlb_lock);
- /* We cannot fallback to other nodes, as we could break the per-node pool. */
- if (!allow_alloc_fallback)
- gfp_mask |= __GFP_THISNODE;
+ if (!folio) {
+ /*
+ * We cannot fallback to other nodes, as we could break the
+ * per-node pool.
+ */
+ if (!allow_alloc_fallback)
+ gfp_mask |= __GFP_THISNODE;
+
+ folio = alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid,
+ nmask);
+ }
- return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask);
+ if (folio)
+ lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio));
+
+ return folio;
}
static nodemask_t *policy_mbind_nodemask(gfp_t gfp)
@@ -2827,6 +2839,7 @@ void wait_for_freed_hugetlb_folios(void)
* hugetlb_alloc_folio - Allocate a hugetlb folio.
* @h: Hugetlb state control block.
* @mpoli: Interpreted memory policy to use for allocation.
+ * @mm: Memory descriptor of the allocation target.
* @alloc_flags: Flags controlling the allocation behavior.
*
* Allocates a hugetlb folio and handles cgroup charging and global hstate
@@ -2834,10 +2847,10 @@ void wait_for_freed_hugetlb_folios(void)
*
* Return: A pointer to the allocated folio, or an ERR_PTR on failure.
* -ENOSPC if cgroup charging fails or no folio is available.
- * -ENOMEM if mem cgroup charging fails.
*/
struct folio *hugetlb_alloc_folio(struct hstate *h,
- struct mempolicy_interpreted *mpoli, u8 alloc_flags)
+ struct mempolicy_interpreted *mpoli, struct mm_struct *mm,
+ u8 alloc_flags)
{
bool charge_hugetlb_cgroup_rsvd = alloc_flags &
HUGETLB_ALLOC_CHARG_CGROUP_RSVD;
@@ -2892,7 +2905,8 @@ struct folio *hugetlb_alloc_folio(struct hstate *h,
spin_unlock_irq(&hugetlb_lock);
- ret = mem_cgroup_charge_hugetlb(folio, gfp | __GFP_RETRY_MAYFAIL);
+ ret = mem_cgroup_charge_hugetlb(folio, mm,
+ gfp | __GFP_RETRY_MAYFAIL);
/*
* Unconditionally increment NR_HUGETLB here because if
* mem_cgroup_charge_hugetlb failed, freeing the page will
@@ -2901,13 +2915,17 @@ struct folio *hugetlb_alloc_folio(struct hstate *h,
lruvec_stat_mod_folio(folio, NR_HUGETLB, nr_pages);
if (ret == -ENOMEM) {
- free_huge_folio(folio);
+ folio_put(folio);
/*
* Skip uncharging hugetlb_cgroup since the charges
* were committed to the folio and freeing the folio
* would have cleared those up.
*/
- return ERR_PTR(ret);
+ /*
+ * Return -ENOSPC, since retrying the fault is futile:
+ * the OOM killer is not triggered for HugeTLB.
+ */
+ return ERR_PTR(-ENOSPC);
}
return folio;
@@ -3031,7 +3049,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
.nodemask = nodemask,
};
- folio = hugetlb_alloc_folio(h, &mpoli, alloc_flags);
+ folio = hugetlb_alloc_folio(h, &mpoli, vma->vm_mm, alloc_flags);
mpol_cond_put(mpol);
@@ -5267,7 +5285,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
* drop the uffd-wp bit in this zap, then replace the
* pte with a marker.
*/
- if (pte_swp_uffd_any(pte) &&
+ if (userfaultfd_wp(vma) && pte_swp_uffd_any(pte) &&
!(zap_flags & ZAP_FLAG_DROP_MARKER))
set_huge_pte_at(mm, address, ptep,
make_pte_marker(PTE_MARKER_UFFD_WP),
@@ -5303,7 +5321,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (huge_pte_dirty(pte))
folio_mark_dirty(folio);
/* Leave a uffd-wp pte marker if needed */
- if (huge_pte_uffd(pte) &&
+ if (userfaultfd_wp(vma) && huge_pte_uffd(pte) &&
!(zap_flags & ZAP_FLAG_DROP_MARKER))
set_huge_pte_at(mm, address, ptep,
make_pte_marker(PTE_MARKER_UFFD_WP),
@@ -5440,6 +5458,8 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
struct address_space *mapping;
pgoff_t pgoff;
+ VM_WARN_ON_ONCE(!folio_test_anon(folio));
+
/*
* vm_pgoff is in PAGE_SIZE units, hence the different calculation
* from page cache lookup which is in HPAGE_SIZE units.
@@ -5474,10 +5494,17 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
* could insert a zeroed page instead of the data existing
* from the time of fork. This would look like data corruption
*/
- if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
- unmap_hugepage_range(iter_vma, address,
- address + huge_page_size(h),
- folio, 0);
+ if (is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
+ continue;
+
+ /*
+ * The address in other VMAs can differ. Even though our old
+ * folio is anon, we always operate on the underlying hugetlb
+ * file.
+ */
+ address = vma_filebacked_address(iter_vma, pgoff, 1);
+ unmap_hugepage_range(iter_vma, address,
+ address + huge_page_size(h), folio, 0);
}
i_mmap_unlock_write(mapping);
}
@@ -5930,8 +5957,11 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,
/*
* If this pte was previously wr-protected, keep it wr-protected even
* if populated.
+ *
+ * TODO: fix all cases where uffd-wp markers remain in VMAs that are no
+ * longer registered with uffd.
*/
- if (unlikely(pte_is_uffd_wp_marker(vmf->orig_pte)))
+ if (unlikely(userfaultfd_wp(vma) && pte_is_uffd_wp_marker(vmf->orig_pte)))
new_pte = huge_pte_mkuffd(new_pte);
set_huge_pte_at(mm, vmf->address, vmf->pte, new_pte, huge_page_size(h));
@@ -6118,16 +6148,20 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,
* NUMA hinting), so let normal hugetlb fault handling proceed.
*/
if (pte_protnone(vmf.orig_pte) && vma_is_accessible(vma) &&
- userfaultfd_rwp(vma) && huge_pte_uffd(vmf.orig_pte)) {
+ huge_pte_uffd(vmf.orig_pte)) {
spinlock_t *ptl;
pte_t pte;
/* Sync: drop hugetlb locks before blocking in handle_userfault() */
- if (!userfaultfd_rwp_async(vma))
+ if (userfaultfd_rwp(vma) && !userfaultfd_rwp_async(vma))
return hugetlb_handle_userfault(&vmf, mapping, VM_UFFD_RWP);
ptl = huge_pte_lock(h, mm, vmf.pte);
pte = huge_ptep_get(mm, vmf.address, vmf.pte);
+ /*
+ * TODO: fix all cases where uffd PTE bits remain in VMAs that
+ * are no longer registered with uffd.
+ */
if (pte_protnone(pte) && huge_pte_uffd(pte)) {
unsigned int shift = huge_page_shift(h);
@@ -6175,15 +6209,19 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,
goto out_ptl;
/* Handle userfault-wp first, before trying to lock more pages */
- if (userfaultfd_wp(vma) && huge_pte_uffd(huge_ptep_get(mm, vmf.address, vmf.pte)) &&
+ if (huge_pte_uffd(huge_ptep_get(mm, vmf.address, vmf.pte)) &&
(flags & FAULT_FLAG_WRITE) && !huge_pte_write(vmf.orig_pte)) {
- if (!userfaultfd_wp_async(vma)) {
+ if (userfaultfd_wp(vma) && !userfaultfd_wp_async(vma)) {
spin_unlock(vmf.ptl);
hugetlb_vma_unlock_read(vma);
mutex_unlock(&hugetlb_fault_mutex_table[hash]);
return handle_userfault(&vmf, VM_UFFD_WP);
}
+ /*
+ * TODO: fix all cases where uffd PTE bits remain in VMAs that
+ * are no longer registered with uffd.
+ */
vmf.orig_pte = huge_pte_clear_uffd(vmf.orig_pte);
set_huge_pte_at(mm, vmf.address, vmf.pte, vmf.orig_pte,
huge_page_size(hstate_vma(vma)));
@@ -6831,15 +6869,6 @@ long hugetlb_reserve_pages(struct inode *inode,
out_put_pages:
spool_resv = chg - gbl_reserve;
- if (spool_resv) {
- /* put sub pool's reservation back, chg - gbl_reserve */
- gbl_resv = hugepage_subpool_put_pages(spool, spool_resv);
- /*
- * subpool's reserved pages can not be put back due to race,
- * return to hstate.
- */
- hugetlb_acct_memory(h, -gbl_resv);
- }
/* Restore used_hpages for pages that failed global reservation */
if (gbl_reserve && spool) {
unsigned long flags;
@@ -6849,6 +6878,15 @@ long hugetlb_reserve_pages(struct inode *inode,
spool->used_hpages -= gbl_reserve;
unlock_or_release_subpool(spool, flags);
}
+ if (spool_resv) {
+ /* put sub pool's reservation back, chg - gbl_reserve */
+ gbl_resv = hugepage_subpool_put_pages(spool, spool_resv);
+ /*
+ * subpool's reserved pages can not be put back due to race,
+ * return to hstate.
+ */
+ hugetlb_acct_memory(h, -gbl_resv);
+ }
out_uncharge_cgroup:
hugetlb_cgroup_uncharge_cgroup_rsvd(hstate_index(h),
chg * pages_per_huge_page(h), h_cg);
@@ -7323,12 +7361,36 @@ void folio_putback_hugetlb(struct folio *folio)
folio_put(folio);
}
+static void move_hugetlb_lruvec_stat(struct folio *old_folio,
+ struct folio *new_folio)
+{
+ struct mem_cgroup *memcg;
+ long nr_pages = folio_nr_pages(old_folio);
+ int old_nid = folio_nid(old_folio);
+ int new_nid = folio_nid(new_folio);
+
+ if (old_nid == new_nid)
+ return;
+
+ guard(rcu)();
+
+ memcg = folio_memcg(new_folio);
+ if (!memcg)
+ return;
+
+ mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)),
+ NR_HUGETLB, -nr_pages);
+ mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)),
+ NR_HUGETLB, nr_pages);
+}
+
void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio,
enum migrate_reason reason)
{
struct hstate *h = folio_hstate(old_folio);
hugetlb_cgroup_migrate(old_folio, new_folio);
+ move_hugetlb_lruvec_stat(old_folio, new_folio);
folio_set_owner_migrate_reason(new_folio, reason);
/*
@@ -7458,6 +7520,15 @@ void hugetlb_unshare_all_pmds(struct vm_area_struct *vma)
*/
void fixup_hugetlb_reservations(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
- clear_vma_resv_huge_pages(vma);
+ if (!is_vm_hugetlb_page(vma))
+ return;
+
+ /*
+ * Rmap walks might still find this VMA and take the lock
+ * inside the reservation map for __vma_private_lock(). Don't clear
+ * the reservation map while that is possible.
+ */
+ i_mmap_lock_write(vma->vm_file->f_mapping);
+ clear_vma_resv_huge_pages(vma);
+ i_mmap_unlock_write(vma->vm_file->f_mapping);
}
diff --git a/mm/hugetlb_cgroup.c b/mm/hugetlb_cgroup.c
index ecb6e0b7819a..3fb41311e4c7 100644
--- a/mm/hugetlb_cgroup.c
+++ b/mm/hugetlb_cgroup.c
@@ -179,6 +179,34 @@ static void hugetlb_cgroup_css_free(struct cgroup_subsys_state *css)
hugetlb_cgroup_free(hugetlb_cgroup_from_css(css));
}
+static void hugetlb_cgroup_move_usage(struct hugetlb_cgroup *from,
+ struct hugetlb_cgroup *to,
+ struct folio *from_folio,
+ struct folio *to_folio)
+{
+ int idx = hstate_index(folio_hstate(from_folio));
+ unsigned long nr_pages = folio_nr_pages(from_folio);
+ int from_nid = folio_nid(from_folio);
+ int to_nid = folio_nid(to_folio);
+ unsigned long usage;
+
+ lockdep_assert_held(&hugetlb_lock);
+
+ if (!from || !to)
+ return;
+
+ if (from == to && from_nid == to_nid)
+ return;
+
+ usage = from->nodeinfo[from_nid]->usage[idx];
+ if (WARN_ON_ONCE(usage < nr_pages))
+ return;
+ WRITE_ONCE(from->nodeinfo[from_nid]->usage[idx], usage - nr_pages);
+
+ usage = to->nodeinfo[to_nid]->usage[idx];
+ WRITE_ONCE(to->nodeinfo[to_nid]->usage[idx], usage + nr_pages);
+}
+
/*
* Should be called with hugetlb_lock held.
* Since we are holding hugetlb_lock, pages cannot get moved from
@@ -213,6 +241,8 @@ static void hugetlb_cgroup_move_parent(int idx, struct hugetlb_cgroup *h_cg,
/* Take the pages off the local counter */
page_counter_cancel(counter, nr_pages);
+ hugetlb_cgroup_move_usage(h_cg, parent, folio, folio);
+
set_hugetlb_cgroup(folio, parent);
out:
return;
@@ -906,6 +936,9 @@ void hugetlb_cgroup_migrate(struct folio *old_folio, struct folio *new_folio)
/* move the h_cg details to new cgroup */
set_hugetlb_cgroup(new_folio, h_cg);
set_hugetlb_cgroup_rsvd(new_folio, h_cg_rsvd);
+
+ hugetlb_cgroup_move_usage(h_cg, h_cg, old_folio, new_folio);
+
list_move(&new_folio->lru, &h->hugepage_activelist);
spin_unlock_irq(&hugetlb_lock);
}
diff --git a/mm/ksm.c b/mm/ksm.c
index 49d48d1e0998..fe42c17490b8 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -1115,7 +1115,8 @@ static inline void folio_set_stable_node(struct folio *folio,
struct ksm_stable_node *stable_node)
{
VM_WARN_ON_FOLIO(folio_test_anon(folio) && PageAnonExclusive(&folio->page), folio);
- folio->mapping = (void *)((unsigned long)stable_node | FOLIO_MAPPING_KSM);
+ WRITE_ONCE(folio->mapping,
+ (void *)((unsigned long)stable_node | FOLIO_MAPPING_KSM));
}
#ifdef CONFIG_SYSFS
@@ -3316,7 +3317,7 @@ void folio_migrate_ksm(struct folio *newfolio, struct folio *folio)
stable_node = folio_stable_node(folio);
if (stable_node) {
VM_BUG_ON_FOLIO(stable_node->kpfn != folio_pfn(folio), folio);
- stable_node->kpfn = folio_pfn(newfolio);
+ WRITE_ONCE(stable_node->kpfn, folio_pfn(newfolio));
/*
* newfolio->mapping was set in advance; now we need smp_wmb()
* to make sure that the new stable_node->kpfn is visible
diff --git a/mm/list_lru.c b/mm/list_lru.c
index 36662d02ff96..96d02ea206a8 100644
--- a/mm/list_lru.c
+++ b/mm/list_lru.c
@@ -666,7 +666,12 @@ int __list_lru_init(struct list_lru *lru, bool memcg_aware, struct shrinker *shr
int i;
#ifdef CONFIG_MEMCG
- if (shrinker)
+ /*
+ * If the shrinker fell back to being non-memcg-aware (e.g. with
+ * cgroup.memory=nokmem), its id was never assigned and holds a
+ * stale 0. Don't let set_shrinker_bit() act on it.
+ */
+ if (shrinker && (shrinker->flags & SHRINKER_MEMCG_AWARE))
lru->shrinker_id = shrinker->id;
else
lru->shrinker_id = -1;
diff --git a/mm/madvise.c b/mm/madvise.c
index eeee82cf2b3f..54d8397059ba 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -393,16 +393,18 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
return 0;
orig_pmd = *pmd;
- if (is_huge_zero_pmd(orig_pmd))
- goto huge_unlock;
-
if (unlikely(!pmd_present(orig_pmd))) {
VM_WARN_ON_ONCE(!pmd_is_migration_entry(orig_pmd) &&
!pmd_is_device_private_entry(orig_pmd));
goto huge_unlock;
}
- folio = pmd_folio(orig_pmd);
+ folio = vm_normal_folio_pmd(vma, addr, orig_pmd);
+ if (!folio)
+ goto huge_unlock;
+
+ if (folio_is_zone_device(folio))
+ goto huge_unlock;
/* Do not interfere with other mappings of this folio */
if (folio_maybe_mapped_shared(folio))
@@ -414,9 +416,10 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
if (next - addr != HPAGE_PMD_SIZE) {
int err;
+ if (!folio_trylock(folio))
+ goto huge_unlock;
folio_get(folio);
spin_unlock(ptl);
- folio_lock(folio);
err = split_folio(folio);
folio_unlock(folio);
folio_put(folio);
@@ -426,10 +429,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
}
if (!pageout && pmd_young(orig_pmd)) {
- pmdp_invalidate(vma, addr, pmd);
- orig_pmd = pmd_mkold(orig_pmd);
-
- set_pmd_at(mm, addr, pmd, orig_pmd);
+ pmdp_test_and_clear_young(vma, addr, pmd);
tlb_remove_pmd_tlb_entry(tlb, pmd, addr);
}
@@ -450,6 +450,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
spin_unlock(ptl);
if (pageout)
reclaim_pages(&folio_list);
+ cond_resched();
return 0;
}
@@ -459,7 +460,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
restart:
start_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
if (!start_pte)
- return 0;
+ goto out;
flush_tlb_batched_pending(mm);
lazy_mmu_mode_enable();
for (; addr < end; pte += nr, addr += nr * PAGE_SIZE) {
@@ -563,6 +564,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
folio_deactivate(folio);
}
+out:
if (start_pte) {
lazy_mmu_mode_disable();
pte_unmap_unlock(start_pte, ptl);
diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h
index 1e394269c613..0578b7076764 100644
--- a/mm/memcontrol-v1.h
+++ b/mm/memcontrol-v1.h
@@ -25,6 +25,9 @@ int memory_stat_show(struct seq_file *m, void *v);
struct mem_cgroup *mem_cgroup_private_id_get_online(struct mem_cgroup *memcg,
unsigned int n);
+void reparent_memcg_lruvec_state_local(struct mem_cgroup *memcg,
+ struct mem_cgroup *parent, int idx);
+
/* Cgroup v1-specific declarations */
#ifdef CONFIG_MEMCG_V1
@@ -73,8 +76,6 @@ void reparent_memcg1_lruvec_state_local(struct mem_cgroup *memcg, struct mem_cgr
void reparent_memcg_state_local(struct mem_cgroup *memcg,
struct mem_cgroup *parent, int idx);
-void reparent_memcg_lruvec_state_local(struct mem_cgroup *memcg,
- struct mem_cgroup *parent, int idx);
void memcg1_account_kmem(struct mem_cgroup *memcg, int nr_pages);
static inline bool memcg1_tcpmem_active(struct mem_cgroup *memcg)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 856a7d07586c..b5137e86bed7 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -232,14 +232,29 @@ static inline struct obj_cgroup *__memcg_reparent_objcgs(struct mem_cgroup *memc
return objcg;
}
-#ifdef CONFIG_MEMCG_V1
static void __mem_cgroup_flush_stats(struct mem_cgroup *memcg, bool force);
-static inline void reparent_state_local(struct mem_cgroup *memcg, struct mem_cgroup *parent)
+/*
+ * Reparent the non-hierarchical lruvec stats that count_shadow_nodes() reads
+ * to approximate the shadow node budget. They are not exposed to userspace
+ * on cgroup v2, but they must follow the reparented folios; otherwise the
+ * ancestor would only receive the negative deltas when the folios are freed
+ * without ever having received the positive base, and its local stats would
+ * permanently underflow.
+ */
+static void reparent_v2_lruvec_state_local(struct mem_cgroup *memcg, struct mem_cgroup *parent)
{
- if (cgroup_subsys_on_dfl(memory_cgrp_subsys))
- return;
+ int i;
+
+ for (i = 0; i < NR_LRU_LISTS; i++)
+ reparent_memcg_lruvec_state_local(memcg, parent, NR_LRU_BASE + i);
+ reparent_memcg_lruvec_state_local(memcg, parent, NR_SLAB_RECLAIMABLE_B);
+ reparent_memcg_lruvec_state_local(memcg, parent, NR_SLAB_UNRECLAIMABLE_B);
+}
+
+static inline void reparent_state_local(struct mem_cgroup *memcg, struct mem_cgroup *parent)
+{
/*
* Reparent stats exposed non-hierarchically. Flush @memcg's stats first
* to read its stats accurately , and conservatively flush @parent's
@@ -248,17 +263,18 @@ static inline void reparent_state_local(struct mem_cgroup *memcg, struct mem_cgr
*/
__mem_cgroup_flush_stats(memcg, true);
- /* The following counts are all non-hierarchical and need to be reparented. */
- reparent_memcg1_state_local(memcg, parent);
- reparent_memcg1_lruvec_state_local(memcg, parent);
+ if (cgroup_subsys_on_dfl(memory_cgrp_subsys)) {
+ reparent_v2_lruvec_state_local(memcg, parent);
+ } else {
+#ifdef CONFIG_MEMCG_V1
+ /* The following counts are all non-hierarchical and need to be reparented. */
+ reparent_memcg1_state_local(memcg, parent);
+ reparent_memcg1_lruvec_state_local(memcg, parent);
+#endif
+ }
__mem_cgroup_flush_stats(parent, true);
}
-#else
-static inline void reparent_state_local(struct mem_cgroup *memcg, struct mem_cgroup *parent)
-{
-}
-#endif
static inline void reparent_locks(struct mem_cgroup *memcg, struct mem_cgroup *parent, int nid)
{
@@ -570,7 +586,6 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec,
return x;
}
-#ifdef CONFIG_MEMCG_V1
static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
enum node_stat_item idx, long val);
@@ -592,7 +607,6 @@ void reparent_memcg_lruvec_state_local(struct mem_cgroup *memcg,
__mod_memcg_lruvec_state(parent_pn, idx, value);
}
}
-#endif
/* Subset of vm_event_item to report for memcg event stats */
static const unsigned int memcg_vm_event_stat[] = {
@@ -757,7 +771,7 @@ static void __mem_cgroup_flush_stats(struct mem_cgroup *memcg, bool force)
return;
if (mem_cgroup_is_root(memcg))
- WRITE_ONCE(flush_last_time, jiffies_64);
+ WRITE_ONCE(flush_last_time, get_jiffies_64());
css_rstat_flush(&memcg->css);
}
@@ -785,7 +799,7 @@ void mem_cgroup_flush_stats(struct mem_cgroup *memcg)
void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg)
{
/* Only flush if the periodic flusher is one full cycle late */
- if (time_after64(jiffies_64, READ_ONCE(flush_last_time) + 2*FLUSH_TIME))
+ if (time_after64(get_jiffies_64(), READ_ONCE(flush_last_time) + 2 * FLUSH_TIME))
mem_cgroup_flush_stats(memcg);
}
@@ -845,16 +859,21 @@ static long memcg_state_val_in_pages(int idx, long val)
return val < 0 ? -res : res;
}
-#ifdef CONFIG_MEMCG_V1
/*
- * Used in mod_memcg_state() and mod_memcg_lruvec_state() to avoid race with
- * reparenting of non-hierarchical state_locals.
+ * Used in mod_memcg_state() and mod_memcg_lruvec_state() to avoid race
+ * with reparenting of non-hierarchical state_locals. Offlining a
+ * memcg is rare, so do the redirection for all cgroup hierarchies.
*/
-static inline struct mem_cgroup *get_non_dying_memcg_start(struct mem_cgroup *memcg,
- bool *rcu_locked)
+static inline struct mem_cgroup *
+get_non_dying_memcg_start(struct mem_cgroup *memcg, bool *rcu_locked)
{
- /* Rebinding can cause this value to be changed at runtime */
- if (cgroup_subsys_on_dfl(memory_cgrp_subsys)) {
+ /*
+ * Fast path: the caller holds a reference to @memcg, so reading
+ * its CSS_DYING flag without the RCU lock is safe. The RCU lock
+ * is only needed to walk up to a non-dying ancestor, which
+ * happens only while a memcg is actually being offlined.
+ */
+ if (!memcg_is_dying(memcg)) {
*rcu_locked = false;
return memcg;
}
@@ -875,17 +894,6 @@ static inline void get_non_dying_memcg_end(bool rcu_locked)
rcu_read_unlock();
}
-#else
-static inline struct mem_cgroup *get_non_dying_memcg_start(struct mem_cgroup *memcg,
- bool *rcu_locked)
-{
- return memcg;
-}
-
-static inline void get_non_dying_memcg_end(bool rcu_locked)
-{
-}
-#endif
static void __mod_memcg_state(struct mem_cgroup *memcg,
enum memcg_stat_item idx, long val)
@@ -979,9 +987,8 @@ static void __mod_memcg_lruvec_state(struct mem_cgroup_per_node *pn,
put_cpu();
}
-static void mod_memcg_lruvec_state(struct lruvec *lruvec,
- enum node_stat_item idx,
- int val)
+void mod_memcg_lruvec_state(struct lruvec *lruvec,
+ enum node_stat_item idx, int val)
{
struct pglist_data *pgdat = lruvec_pgdat(lruvec);
struct mem_cgroup_per_node *pn;
@@ -2306,7 +2313,7 @@ static bool is_memcg_drain_needed(struct memcg_stock_pcp *stock,
return flush;
}
-static void schedule_drain_work(int cpu, struct work_struct *work)
+static bool schedule_drain_work(int cpu, struct work_struct *work)
{
/*
* Protect housekeeping cpumask read and work enqueue together
@@ -2315,8 +2322,11 @@ static void schedule_drain_work(int cpu, struct work_struct *work)
* pending work on newly isolated CPUs.
*/
guard(rcu)();
- if (!cpu_is_isolated(cpu))
- queue_work_on(cpu, memcg_wq, work);
+ if (cpu_is_isolated(cpu))
+ return false;
+
+ queue_work_on(cpu, memcg_wq, work);
+ return true;
}
/*
@@ -2348,8 +2358,9 @@ void drain_all_stock(struct mem_cgroup *root_memcg)
&memcg_st->flags)) {
if (cpu == curcpu)
drain_local_memcg_stock(&memcg_st->work);
- else
- schedule_drain_work(cpu, &memcg_st->work);
+ else if (!schedule_drain_work(cpu, &memcg_st->work))
+ clear_bit(FLUSHING_CACHED_CHARGE,
+ &memcg_st->flags);
}
if (!test_bit(FLUSHING_CACHED_CHARGE, &obj_st->flags) &&
@@ -2358,8 +2369,9 @@ void drain_all_stock(struct mem_cgroup *root_memcg)
&obj_st->flags)) {
if (cpu == curcpu)
drain_local_obj_stock(&obj_st->work);
- else
- schedule_drain_work(cpu, &obj_st->work);
+ else if (!schedule_drain_work(cpu, &obj_st->work))
+ clear_bit(FLUSHING_CACHED_CHARGE,
+ &obj_st->flags);
}
}
migrate_enable();
@@ -2656,10 +2668,11 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
bool raised_max_event = false;
unsigned long pflags;
bool allow_spinning = gfpflags_allow_spinning(gfp_mask);
+ int ret = 0;
retry:
if (consume_stock(memcg, nr_pages))
- return 0;
+ return ret;
if (!allow_spinning)
/* Avoid the refill and flush of the older stock */
@@ -2770,16 +2783,11 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
* put the burden of reclaim on regular allocation requests
* and let these go through as privileged allocations.
*/
- if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH)))
- return -ENOMEM;
+ if (!(gfp_mask & (__GFP_NOFAIL | __GFP_HIGH))) {
+ ret = -ENOMEM;
+ goto out;
+ }
force:
- /*
- * If the allocation has to be enforced, don't forget to raise
- * a MEMCG_MAX event.
- */
- if (!raised_max_event)
- __memcg_memory_event(mem_over_limit, MEMCG_MAX, allow_spinning);
-
/*
* The allocation either can't fail or will lead to more memory
* being freed very soon. Allow memory usage go over the limit
@@ -2789,7 +2797,15 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
if (do_memsw_account())
page_counter_charge(&memcg->memsw, nr_pages);
- return 0;
+out:
+ /*
+ * Don't forget to raise a MEMCG_MAX event for forced or rejected
+ * requests.
+ */
+ if (!raised_max_event)
+ __memcg_memory_event(mem_over_limit, MEMCG_MAX, allow_spinning);
+
+ return ret;
done_restock:
if (batch > nr_pages)
@@ -2848,7 +2864,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
!(current->flags & PF_MEMALLOC) &&
gfpflags_allow_blocking(gfp_mask))
__mem_cgroup_handle_over_high(gfp_mask);
- return 0;
+ return ret;
}
static inline int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask,
@@ -3946,7 +3962,7 @@ void mem_cgroup_flush_foreign(struct bdi_writeback *wb)
{
struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css);
unsigned long intv = msecs_to_jiffies(dirty_expire_interval * 10);
- u64 now = jiffies_64;
+ u64 now = get_jiffies_64();
int i;
for (i = 0; i < MEMCG_CGWB_FRN_CNT; i++) {
@@ -5198,6 +5214,51 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection);
}
+#ifdef CONFIG_LRU_GEN
+/**
+ * mem_cgroup_calculate_protection_path - compute protection along a path
+ * @root: the top ancestor of the sub-tree being checked (NULL for root_mem_cgroup)
+ * @memcg: the target memory cgroup
+ *
+ * Walk the ancestor path from @root down to @memcg and compute the effective
+ * protection at each level. This is safe for isolated queries because it
+ * ensures parents are computed before children.
+ */
+void mem_cgroup_calculate_protection_path(struct mem_cgroup *root,
+ struct mem_cgroup *memcg)
+{
+ bool recursive_protection =
+ cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT;
+ struct cgroup *cg;
+ int root_level, i;
+
+ if (mem_cgroup_disabled())
+ return;
+
+ if (!root)
+ root = root_mem_cgroup;
+
+ if (memcg == root)
+ return;
+
+ root_level = root->css.cgroup->level;
+ cg = memcg->css.cgroup;
+
+ rcu_read_lock();
+ for (i = root_level + 1; i <= cg->level; i++) {
+ struct mem_cgroup *cur;
+
+ cur = mem_cgroup_from_css(cgroup_css(cg->ancestors[i],
+ &memory_cgrp_subsys));
+ if (cur)
+ page_counter_calculate_protection(&root->memory,
+ &cur->memory,
+ recursive_protection);
+ }
+ rcu_read_unlock();
+}
+#endif /* CONFIG_LRU_GEN */
+
static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg,
gfp_t gfp)
{
@@ -5233,6 +5294,7 @@ int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
/**
* mem_cgroup_charge_hugetlb - charge the memcg for a hugetlb folio
* @folio: folio being charged
+ * @mm: mm context of the allocation target
* @gfp: reclaim mode
*
* This function is called when allocating a huge page folio, after the page has
@@ -5242,9 +5304,10 @@ int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
* Returns ENOMEM if the memcg is already full.
* Returns 0 if either the charge was successful, or if we skip the charging.
*/
-int mem_cgroup_charge_hugetlb(struct folio *folio, gfp_t gfp)
+int mem_cgroup_charge_hugetlb(struct folio *folio, struct mm_struct *mm,
+ gfp_t gfp)
{
- struct mem_cgroup *memcg = get_mem_cgroup_from_current();
+ struct mem_cgroup *memcg = get_mem_cgroup_from_mm(mm);
int ret = 0;
/*
diff --git a/mm/memfd.c b/mm/memfd.c
index c708d92533f4..0f6fff004f5e 100644
--- a/mm/memfd.c
+++ b/mm/memfd.c
@@ -82,22 +82,31 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
struct hstate *h = hstate_file(memfd);
int err = -ENOMEM;
long nr_resv;
+ u32 hash;
gfp_mask = htlb_alloc_mask(h);
gfp_mask &= ~(__GFP_HIGHMEM | __GFP_MOVABLE);
idx >>= huge_page_order(h);
+ /*
+ * Serialize hugepage allocation and instantiation to prevent
+ * races with concurrent allocations, as required by all other
+ * callers of hugetlb_add_to_page_cache().
+ */
+ hash = hugetlb_fault_mutex_hash(memfd->f_mapping, idx);
+ mutex_lock(&hugetlb_fault_mutex_table[hash]);
+
nr_resv = hugetlb_reserve_pages(inode, idx, idx + 1, NULL, EMPTY_VMA_FLAGS);
- if (nr_resv < 0)
- return ERR_PTR(nr_resv);
+ if (nr_resv < 0) {
+ err = nr_resv;
+ goto out_unlock;
+ }
folio = alloc_hugetlb_folio_reserve(h,
numa_node_id(),
NULL,
gfp_mask);
if (folio) {
- u32 hash;
-
/*
* Zero the folio to prevent information leaks to userspace.
* Use folio_zero_user() which is optimized for huge/gigantic
@@ -112,20 +121,9 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
*/
__folio_mark_uptodate(folio);
- /*
- * Serialize hugepage allocation and instantiation to prevent
- * races with concurrent allocations, as required by all other
- * callers of hugetlb_add_to_page_cache().
- */
- hash = hugetlb_fault_mutex_hash(memfd->f_mapping, idx);
- mutex_lock(&hugetlb_fault_mutex_table[hash]);
-
err = hugetlb_add_to_page_cache(folio,
memfd->f_mapping,
idx);
-
- mutex_unlock(&hugetlb_fault_mutex_table[hash]);
-
if (err) {
folio_put(folio);
goto err_unresv;
@@ -133,11 +131,14 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx)
hugetlb_set_folio_subpool(folio, subpool_inode(inode));
folio_unlock(folio);
+ mutex_unlock(&hugetlb_fault_mutex_table[hash]);
return folio;
}
err_unresv:
if (nr_resv > 0)
hugetlb_unreserve_pages(inode, idx, idx + 1, 0);
+out_unlock:
+ mutex_unlock(&hugetlb_fault_mutex_table[hash]);
return ERR_PTR(err);
}
#endif
diff --git a/mm/memory.c b/mm/memory.c
index 8b0c2c735d3d..9e395ad33ffd 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -7159,14 +7159,34 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
resource_size_t phys_addr;
pgprot_t prot = __pgprot(0);
void __iomem *maddr;
+ struct page *page;
int offset = offset_in_page(addr);
int ret = -EINVAL;
bool writable;
struct follow_pfnmap_args args = { .vma = vma, .address = addr };
+ /*
+ * Limit access to one page at a time, as that's what follow_pfnmap_start()
+ * guarantees; expect the caller to retry to read larger ranges.
+ */
+ len = min_t(int, len, PAGE_SIZE - offset);
+
retry:
if (follow_pfnmap_start(&args))
return -EINVAL;
+
+ /*
+ * Reject anything that looks like ordinary RAM, including anonymous
+ * folios in CoW mappings or allocated kernel memory. Note that we
+ * cannot distinguish memblock allocations from non-RAM reliably, so
+ * we cannot easily reject them here.
+ */
+ page = pfn_to_online_page(args.pfn);
+ if (page && !PageReserved(page)) {
+ follow_pfnmap_end(&args);
+ return -EINVAL;
+ }
+
prot = args.pgprot;
phys_addr = (resource_size_t)args.pfn << PAGE_SHIFT;
writable = args.writable;
@@ -7175,7 +7195,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
if ((write & FOLL_WRITE) && !writable)
return -EINVAL;
- maddr = ioremap_prot(phys_addr, PAGE_ALIGN(len + offset), prot);
+ maddr = ioremap_prot(phys_addr, PAGE_SIZE, prot);
if (!maddr)
return -ENOMEM;
@@ -7183,7 +7203,7 @@ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
goto out_unmap;
if ((pgprot_val(prot) != pgprot_val(args.pgprot)) ||
- (phys_addr != (args.pfn << PAGE_SHIFT)) ||
+ (phys_addr != ((resource_size_t)args.pfn << PAGE_SHIFT)) ||
(writable != args.writable)) {
follow_pfnmap_end(&args);
iounmap(maddr);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 79053ece02cd..839c3fe985a0 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -650,7 +650,8 @@ static inline bool queue_folio_required(struct folio *folio,
return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
}
-static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
+static void queue_folios_pmd(pmd_t *pmd, unsigned long addr,
+ struct mm_walk *walk)
{
struct folio *folio;
struct queue_pages *qp = walk->private;
@@ -661,11 +662,9 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk *walk)
qp->nr_failed++;
return;
}
- folio = pmd_folio(pmdval);
- if (is_huge_zero_folio(folio)) {
- walk->action = ACTION_CONTINUE;
+ folio = vm_normal_folio_pmd(walk->vma, addr, pmdval);
+ if (!folio || folio_is_zone_device(folio))
return;
- }
if (!queue_folio_required(folio, qp))
return;
if (!(qp->flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
@@ -698,7 +697,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
ptl = pmd_trans_huge_lock(pmd, vma);
if (ptl) {
- queue_folios_pmd(pmd, walk);
+ queue_folios_pmd(pmd, addr, walk);
spin_unlock(ptl);
goto out;
}
@@ -2277,7 +2276,7 @@ static nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *pol,
break;
case MPOL_BIND:
/* Restrict to nodemask (but not on lower zones) */
- if (apply_policy_zone(pol, gfp_zone(gfp)) &&
+ if (apply_policy_zone(pol, gfp_zone(current_gfp_context(gfp))) &&
cpuset_nodemask_valid_mems_allowed(&pol->nodes))
nodemask = &pol->nodes;
if (pol->home_node != NUMA_NO_NODE)
@@ -2592,6 +2591,7 @@ static unsigned long alloc_pages_bulk_interleave(gfp_t gfp,
struct mempolicy *pol, unsigned long nr_pages,
struct page **page_array)
{
+ unsigned int cpuset_mems_cookie;
int nodes;
unsigned long nr_pages_per_node;
int delta;
@@ -2599,7 +2599,16 @@ static unsigned long alloc_pages_bulk_interleave(gfp_t gfp,
unsigned long nr_allocated;
unsigned long total_allocated = 0;
- nodes = nodes_weight(pol->nodes);
+ /* count the nodes, retry if a rebind happened during the read */
+ do {
+ cpuset_mems_cookie = read_mems_allowed_begin();
+ nodes = nodes_weight(pol->nodes);
+ } while (read_mems_allowed_retry(cpuset_mems_cookie));
+
+ /* if the nodemask has become invalid, we cannot do anything */
+ if (!nodes)
+ return 0;
+
nr_pages_per_node = nr_pages / nodes;
delta = nr_pages - nodes * nr_pages_per_node;
diff --git a/mm/mlock.c b/mm/mlock.c
index 39215a3eab1f..4fb43cddafa8 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -470,7 +470,7 @@ static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,
{
const vma_flags_t old_vma_flags = vma->flags;
struct mm_struct *mm = vma->vm_mm;
- int nr_pages;
+ long nr_pages;
int ret = 0;
if (vma_flags_same_pair(&old_vma_flags, new_vma_flags) ||
diff --git a/mm/mremap.c b/mm/mremap.c
index 444f37d7cd28..d404e4b1b6cb 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -887,12 +887,6 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
return pmc_progress(pmc);
}
-/* Set vrm->delta to the difference in VMA size specified by user. */
-static void vrm_set_delta(struct vma_remap_struct *vrm)
-{
- vrm->delta = abs_diff(vrm->old_len, vrm->new_len);
-}
-
/* Determine what kind of remap this is - shrink, expand or no resize at all. */
static enum mremap_type vrm_remap_type(struct vma_remap_struct *vrm)
{
@@ -905,6 +899,13 @@ static enum mremap_type vrm_remap_type(struct vma_remap_struct *vrm)
return MREMAP_EXPAND;
}
+/* Populate calculated fields. */
+static void vrm_populate_calculated(struct vma_remap_struct *vrm)
+{
+ vrm->delta = abs_diff(vrm->old_len, vrm->new_len);
+ vrm->remap_type = vrm_remap_type(vrm);
+}
+
/*
* When moving a VMA to vrm->new_adr, does this result in the new and old VMAs
* overlapping?
@@ -1338,6 +1339,20 @@ static int copy_vma_and_data(struct vma_remap_struct *vrm,
pmc_revert.need_rmap_locks = true;
move_page_tables(&pmc_revert);
+ /*
+ * When copy_vma_and_data() returns, the new VMA will be
+ * unmapped rather than the old one.
+ *
+ * When an accountable move is being performed, this unmapping
+ * will not uncharge any expansion, so do that now.
+ *
+ * A move is accountable if the VMA is accountable and
+ * MREMAP_DONTUNMAP is not specified, so only do so if the
+ * latter is true.
+ */
+ if (!(vrm->flags & MREMAP_DONTUNMAP))
+ vrm_uncharge(vrm);
+
vrm->vma = new_vma;
vrm->old_len = vrm->new_len;
vrm->addr = vrm->new_addr;
@@ -1345,7 +1360,7 @@ static int copy_vma_and_data(struct vma_remap_struct *vrm,
mremap_userfaultfd_prep(new_vma, vrm->uf);
}
- fixup_hugetlb_reservations(vma);
+ fixup_hugetlb_reservations(vrm->vma);
*new_vma_ptr = new_vma;
return err;
@@ -1523,6 +1538,7 @@ static unsigned long mremap_to(struct vma_remap_struct *vrm)
/* Set up for the move now shrink has been executed. */
vrm->old_len = vrm->new_len;
+ vrm_populate_calculated(vrm);
}
/* MREMAP_DONTUNMAP expands by old_len since old_len == new_len */
@@ -1795,8 +1811,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
if (is_vm_hugetlb_page(vma) && !align_hugetlb(vrm))
return -EINVAL;
- vrm_set_delta(vrm);
- vrm->remap_type = vrm_remap_type(vrm);
+ vrm_populate_calculated(vrm);
/* For convenience, we set new_addr even if VMA won't move. */
if (!vrm_implies_new_addr(vrm))
vrm->new_addr = addr;
diff --git a/mm/nommu.c b/mm/nommu.c
index 498e01ee40b0..ed44510e3770 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1674,6 +1674,9 @@ static int __access_remote_vm(struct mm_struct *mm, unsigned long addr,
struct vm_area_struct *vma;
int write = gup_flags & FOLL_WRITE;
+ if (addr + len < addr)
+ return 0;
+
if (mmap_read_lock_killable(mm))
return 0;
@@ -1727,9 +1730,6 @@ int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, in
{
struct mm_struct *mm;
- if (addr + len < addr)
- return 0;
-
mm = get_task_mm(tsk);
if (!mm)
return 0;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..8ca2289d0491 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -37,6 +37,7 @@
#include <linux/vmstat.h>
#include <linux/fault-inject.h>
#include <linux/compaction.h>
+#include <linux/crash_dump.h>
#include <trace/events/kmem.h>
#include <trace/events/oom.h>
#include <linux/prefetch.h>
@@ -988,10 +989,12 @@ static inline void __free_one_page(struct page *page,
* Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page,
* merge with it and move up one order.
*/
- if (page_is_guard(buddy))
+ if (page_is_guard(buddy)) {
clear_page_guard(zone, buddy, order);
- else
+ account_freepages(zone, 1 << order, migratetype);
+ } else {
__del_page_from_free_list(buddy, zone, order, buddy_mt);
+ }
if (unlikely(buddy_mt != migratetype)) {
/*
@@ -2178,10 +2181,17 @@ static inline bool boost_watermark(struct zone *zone)
if (!watermark_boost_factor)
return false;
+
+ /*
+ * A kdump capture kernel exits before a boost can pay off, while
+ * the raised watermark can exceed the memory left for the dump.
+ */
+ if (is_kdump_kernel())
+ return false;
+
/*
* Don't bother in zones that are unlikely to produce results.
- * On small machines, including kdump capture kernels running
- * in a small area, boosting the watermark can cause an out of
+ * On small machines, boosting the watermark can cause an out of
* memory situation immediately.
*/
if ((pageblock_nr_pages * 4) > zone_managed_pages(zone))
@@ -4127,6 +4137,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
return page;
}
+static inline bool order_defrag_promoted(unsigned int order,
+ const struct alloc_context *ac)
+{
+ return ac->reclaim_compact_order != order;
+}
+
/*
* Maximum number of compaction retries with a progress before OOM
* killer is consider as the only way to move forward.
@@ -4149,22 +4165,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
.order = order,
.page = NULL,
};
- int compact_order = order;
-
- /*
- * If fallbacks are not permitted (defrag_mode), we either
- * need to reclaim space in a block of matching type, or clear
- * out an entire block to allow __rmqueue_claim() to convert.
- *
- * Reclaim by itself is primarily freeing space in movable
- * blocks, since that's where the LRU pages live. So this
- * works for movable requests, but not for others.
- *
- * For those, promote the order to help make blocks, instead
- * of spinning in reclaim alone unproductively.
- */
- if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
- compact_order = max(order, pageblock_order);
+ int compact_order = ac->reclaim_compact_order;
if (!compact_order)
return NULL;
@@ -4257,7 +4258,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
int retries = *compaction_retries;
enum compact_priority priority = *compact_priority;
- if (!order)
+ if (!ac->reclaim_compact_order)
return false;
if (fatal_signal_pending(current))
@@ -4266,10 +4267,12 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
/*
* Compaction was skipped due to a lack of free order-0
* migration targets. Continue if reclaim can help.
+ * Promoted requests fall back instead.
*/
if (compact_result == COMPACT_SKIPPED) {
- ret = compaction_zonelist_suitable(ac, order, alloc_flags,
- gfp_mask);
+ if (!order_defrag_promoted(order, ac))
+ ret = compaction_zonelist_suitable(ac, order,
+ alloc_flags, gfp_mask);
goto out;
}
@@ -4468,11 +4471,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
struct page *page = NULL;
unsigned long pflags;
bool drained = false;
- int reclaim_order = order;
-
- /* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
- if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
- reclaim_order = max(order, pageblock_order);
+ int reclaim_order = ac->reclaim_compact_order;
psi_memstall_enter(&pflags);
*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
@@ -4648,9 +4647,12 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
/*
* Costly allocations might have made a progress but this doesn't mean
* their order will become available due to high fragmentation so
- * always increment the no progress counter for them
+ * always increment the no progress counter for them. The same goes for
+ * promoted requests: their reclaim reports progress when it backs off
+ * for compaction without freeing anything.
*/
- if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
+ if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
+ !order_defrag_promoted(order, ac))
*no_progress_loops = 0;
else
(*no_progress_loops)++;
@@ -4934,6 +4936,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
/* If allocation has taken excessively long, warn about it */
check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
+ /*
+ * If fallbacks are not permitted (defrag_mode), we either
+ * need to reclaim space in a block of matching type, or clear
+ * out an entire block to allow __rmqueue_claim() to convert.
+ *
+ * Reclaim by itself is primarily freeing space in movable
+ * blocks, since that's where the LRU pages live. So this
+ * works for movable requests, but not for others.
+ *
+ * For those, promote the order to help make blocks, instead
+ * of spinning in reclaim alone unproductively.
+ */
+ ac->reclaim_compact_order = order;
+ if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
+ ac->reclaim_compact_order = max(order, pageblock_order);
+
/* Try direct reclaim and then allocating */
if (!compact_first) {
page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
@@ -5015,6 +5033,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
/* Reclaim/compaction failed to prevent the fallback */
if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
alloc_flags &= ~ALLOC_NOFRAGMENT;
+ /* Retries went into making blocks, start the fallback afresh */
+ if (order_defrag_promoted(order, ac)) {
+ no_progress_loops = 0;
+ compaction_retries = 0;
+ }
goto retry;
}
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index b9259deddb59..2b1017a91928 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -93,6 +93,8 @@ struct alloc_context {
bool spread_dirty_pages;
/* Only flags that are global to the whole allocation go here. */
unsigned int alloc_flags;
+ /* Set by __alloc_pages_slowpath() before reclaim and compaction */
+ unsigned int reclaim_compact_order;
};
/*
diff --git a/mm/page_counter.c b/mm/page_counter.c
index 661e0f2a5127..ea0d1646cff8 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -8,6 +8,7 @@
#include <linux/page_counter.h>
#include <linux/atomic.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/string.h>
#include <linux/sched.h>
#include <linux/bug.h>
@@ -356,7 +357,8 @@ static unsigned long effective_protection(unsigned long usage,
* otherwise get a smaller chunk than what they claimed.
*/
if (siblings_protected > parent_effective)
- return protected * parent_effective / siblings_protected;
+ return mul_u64_u64_div_u64(protected, parent_effective,
+ siblings_protected);
/*
* Ok, utilized protection of all children is within what the
@@ -397,13 +399,20 @@ static unsigned long effective_protection(unsigned long usage,
if (parent_effective > siblings_protected &&
parent_usage > siblings_protected &&
usage > protected) {
- unsigned long unclaimed;
+ unsigned long parent_unclaimed, parent_unprotected, unprotected;
- unclaimed = parent_effective - siblings_protected;
- unclaimed *= usage - protected;
- unclaimed /= parent_usage - siblings_protected;
+ parent_unclaimed = parent_effective - siblings_protected;
+ parent_unprotected = parent_usage - siblings_protected;
- ep += unclaimed;
+ /*
+ * The usages aren't read atomically, so a child can transiently
+ * appear to use more than its parent, making the ratio exceed 1
+ * and the quotient overflow 64 bits (#DE on x86). Cap it.
+ */
+ unprotected = min(usage - protected, parent_unprotected);
+
+ ep += mul_u64_u64_div_u64(parent_unclaimed, unprotected,
+ parent_unprotected);
}
return ep;
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index e5dfc7bf4944..5aaad037384e 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -388,13 +388,19 @@ static int isolate_single_pageblock(unsigned long boundary_pfn,
}
if (PageBuddy(page)) {
- int order = buddy_order(page);
+ unsigned int order = buddy_order_unsafe(page);
- /* pageblock_isolate_and_move_free_pages() handled this */
- VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn);
-
- pfn += 1UL << order;
- continue;
+ /* buddy_order_unsafe() is racy. Validate the order before shifting. */
+ if (order <= MAX_PAGE_ORDER &&
+ /*
+ * pageblock_isolate_and_move_free_pages() splits
+ * cross-boundary PageBuddy, verify it.
+ */
+ pfn + (1UL << order) <= boundary_pfn) {
+ pfn += 1UL << order;
+ continue;
+ }
+ goto failed;
}
/*
@@ -413,10 +419,28 @@ static int isolate_single_pageblock(unsigned long boundary_pfn,
if (PageCompound(page)) {
struct page *head = compound_head(page);
unsigned long head_pfn = page_to_pfn(head);
- unsigned long nr_pages = compound_nr(head);
+ unsigned int order = compound_order(head);
+ unsigned long nr_pages;
+
+ /* compound_order() is racy. Cap it at MAX_FOLIO_ORDER. */
+ if (order > MAX_FOLIO_ORDER)
+ goto failed;
+
+ nr_pages = 1UL << order;
+
+ /*
+ * compound_head() is also racy, so the derived head_pfn
+ * needs additional checks to make sure it is valid.
+ * Otherwise, just fail the check. pfn comes from
+ * __first_valid_page() as a legitimate PFN, so use it to
+ * check head_pfn.
+ */
+ if (head_pfn > pfn || !IS_ALIGNED(head_pfn, nr_pages) ||
+ pfn - head_pfn >= nr_pages)
+ goto failed;
if (head_pfn + nr_pages <= boundary_pfn ||
- PageHuge(page)) {
+ PageHuge(head)) {
pfn = head_pfn + nr_pages;
continue;
}
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 6ffc536359cd..75818f1bc127 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -67,7 +67,7 @@ static void page_table_check_clear(unsigned long pfn, unsigned long pgcnt)
struct page *page;
bool anon;
- if (!pfn_valid(pfn))
+ if (!pfn_valid(pfn) || is_zero_pfn(pfn) || is_huge_zero_pfn(pfn))
return;
page = pfn_to_page(pfn);
@@ -102,7 +102,7 @@ static void page_table_check_set(unsigned long pfn, unsigned long pgcnt,
struct page *page;
bool anon;
- if (!pfn_valid(pfn))
+ if (!pfn_valid(pfn) || is_zero_pfn(pfn) || is_huge_zero_pfn(pfn))
return;
page = pfn_to_page(pfn);
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 4e964545e5e8..67c8039e9d28 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -258,28 +258,28 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
softleaf_t entry;
if (!(pvmw->flags & PVMW_MIGRATION))
- return not_found(pvmw);
+ goto next_pmd;
entry = softleaf_from_pmd(pmde);
if (!check_pmd(softleaf_to_pfn(entry), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
} else if (pmd_is_device_private_entry(pmde)) {
softleaf_t entry;
if (pvmw->flags & PVMW_MIGRATION)
- return not_found(pvmw);
+ goto next_pmd;
entry = softleaf_from_pmd(pmde);
if (!check_pmd(softleaf_to_pfn(entry), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
} else if (!pmd_present(pmde)) {
- return not_found(pvmw);
+ goto next_pmd;
}
if (likely(pmd_trans_huge(pmde))) {
if (pvmw->flags & PVMW_MIGRATION)
- return not_found(pvmw);
+ goto next_pmd;
if (!check_pmd(pmd_pfn(pmde), pvmw))
- return not_found(pvmw);
+ goto next_pmd;
return true;
}
/* THP/device-private pmd was split under us: handle on pte level */
@@ -292,8 +292,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
(pvmw->nr_pages >= HPAGE_PMD_NR))
sync_with_folio_pmd_zap(mm, pvmw->pmd);
- step_forward(pvmw, PMD_SIZE);
- continue;
+ goto next_pmd;
}
if (!map_pte(pvmw, &pmde, &ptl)) {
if (!pvmw->pte)
@@ -336,6 +335,17 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
pvmw->ptl = ptl;
}
goto this_pte;
+next_pmd:
+ /*
+ * A large folio can be PTE-mapped across PMDs, so a
+ * nonmatching PMD does not end the walk.
+ */
+ if (pvmw->ptl) {
+ spin_unlock(pvmw->ptl);
+ pvmw->ptl = NULL;
+ }
+ pvmw->pmd = NULL;
+ step_forward(pvmw, PMD_SIZE);
} while (pvmw->address < end);
return false;
diff --git a/mm/percpu.c b/mm/percpu.c
index a802d72c116f..b0a4aa4e852b 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -1698,14 +1698,15 @@ static void pcpu_alloc_tag_alloc_hook(struct pcpu_chunk *chunk, int off,
{
if (mem_alloc_profiling_enabled() && likely(chunk->obj_exts)) {
alloc_tag_add(&chunk->obj_exts[off >> PCPU_MIN_ALLOC_SHIFT].tag,
- current->alloc_tag, size);
+ current->alloc_tag, pcpu_obj_full_size(size));
}
}
static void pcpu_alloc_tag_free_hook(struct pcpu_chunk *chunk, int off, size_t size)
{
if (mem_alloc_profiling_enabled() && likely(chunk->obj_exts))
- alloc_tag_sub(&chunk->obj_exts[off >> PCPU_MIN_ALLOC_SHIFT].tag, size);
+ alloc_tag_sub(&chunk->obj_exts[off >> PCPU_MIN_ALLOC_SHIFT].tag,
+ pcpu_obj_full_size(size));
}
#else
static void pcpu_alloc_tag_alloc_hook(struct pcpu_chunk *chunk, int off,
diff --git a/mm/shmem.c b/mm/shmem.c
index 848316eaa7f4..de2362209cc3 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -976,6 +976,7 @@ static long shmem_free_swap(struct address_space *mapping,
pgoff_t index, pgoff_t end, void *radswap)
{
XA_STATE(xas, &mapping->i_pages, index);
+ const softleaf_t swp = radix_to_swp_entry(radswap);
unsigned int nr_pages = 0;
pgoff_t base;
void *entry;
@@ -992,8 +993,9 @@ static long shmem_free_swap(struct address_space *mapping,
}
xas_unlock_irq(&xas);
- if (nr_pages)
- swap_put_entries_direct(radix_to_swp_entry(radswap), nr_pages);
+ /* A swapin-error marker holds no swap slot, so just drop it. */
+ if (nr_pages && softleaf_is_swap(swp))
+ swap_put_entries_direct(swp, nr_pages);
return nr_pages;
}
@@ -1159,7 +1161,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
}
folio_batch_remove_exceptionals(&fbatch);
folio_batch_release(&fbatch);
- cond_resched();
+ cond_resched_tasks_rcu_qs();
}
/*
@@ -1200,7 +1202,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend,
index = start;
while (index < end) {
- cond_resched();
+ cond_resched_tasks_rcu_qs();
if (!find_get_entries(mapping, &index, end - 1, &fbatch,
indices)) {
@@ -4509,6 +4511,9 @@ static int shmem_parse_opt_casefold(struct fs_context *fc, struct fs_parameter *
struct unicode_map *encoding;
char *version_str = param->string + 5;
+ if (ctx->encoding)
+ return invalfc(fc, "casefold parameter cannot be specified twice\n");
+
if (!latest_version) {
if (strncmp(param->string, "utf8-", 5))
return invalfc(fc, "Only UTF-8 encodings are supported "
@@ -4998,6 +5003,7 @@ static int shmem_fill_super(struct super_block *sb, struct fs_context *fc)
if (ctx->encoding) {
sb->s_encoding = ctx->encoding;
+ ctx->encoding = NULL;
set_default_d_op(sb, &shmem_ci_dentry_ops);
if (ctx->strict_encoding)
sb->s_encoding_flags = SB_ENC_STRICT_MODE_FL;
@@ -5095,6 +5101,9 @@ static void shmem_free_fc(struct fs_context *fc)
struct shmem_options *ctx = fc->fs_private;
if (ctx) {
+#if IS_ENABLED(CONFIG_UNICODE)
+ utf8_unload(ctx->encoding);
+#endif
mpol_put(ctx->mpol);
kfree(ctx);
}
diff --git a/mm/swap.h b/mm/swap.h
index 90a551a88df6..c9202516b413 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -319,7 +319,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp_mask,
void __swap_cache_add_folio(struct swap_cluster_info *ci,
struct folio *folio, swp_entry_t entry);
void __swap_cache_del_folio(struct swap_cluster_info *ci,
- struct folio *folio, swp_entry_t entry, void *shadow);
+ struct folio *folio, swp_entry_t entry, void *shadow,
+ bool swapout);
void __swap_cache_replace_folio(struct swap_cluster_info *ci,
struct folio *old, struct folio *new);
@@ -453,7 +454,8 @@ static inline void swap_cache_del_folio(struct folio *folio)
}
static inline void __swap_cache_del_folio(struct swap_cluster_info *ci,
- struct folio *folio, swp_entry_t entry, void *shadow)
+ struct folio *folio, swp_entry_t entry, void *shadow,
+ bool swapout)
{
}
diff --git a/mm/swap_state.c b/mm/swap_state.c
index f3961fdd857d..0d9b0d876b56 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -92,17 +92,27 @@ void show_swap_cache_info(void)
*/
struct folio *swap_cache_get_folio(swp_entry_t entry)
{
+ struct swap_cluster_info *ci = __swap_entry_to_cluster(entry);
+ unsigned int ci_off = swp_cluster_offset(entry);
unsigned long swp_tb;
struct folio *folio;
for (;;) {
- swp_tb = swap_table_get(__swap_entry_to_cluster(entry),
- swp_cluster_offset(entry));
+ swp_tb = swap_table_get(ci, ci_off);
if (!swp_tb_is_folio(swp_tb))
return NULL;
folio = swp_tb_to_folio(swp_tb);
- if (likely(folio_try_get(folio)))
+ if (!folio_try_get(folio))
+ continue;
+ /*
+ * The folio may have been freed and its memory reused before
+ * we got the reference. Make sure it is still in the slot.
+ */
+ swp_tb = swap_table_get(ci, ci_off);
+ if (likely(swp_tb_is_folio(swp_tb) &&
+ swp_tb_to_folio(swp_tb) == folio))
return folio;
+ folio_put(folio);
}
return NULL;
@@ -306,21 +316,28 @@ static void __swap_cache_do_del_folio(struct swap_cluster_info *ci,
* @folio: The folio.
* @entry: The first swap entry that the folio corresponds to.
* @shadow: shadow value to be filled in the swap cache.
+ * @swapout: whether this folio is being reclaimed after swapout.
*
* Removes a folio from the swap cache and fills a shadow in place.
* This won't put the folio's refcount. The caller has to do that.
*
* Context: Caller must ensure the folio is locked and in the swap cache
* using the index of @entry, and lock the cluster that holds the entries.
+ * If @swapout is set, the folio should be in reclaim path and IRQs
+ * should be disabled.
*/
void __swap_cache_del_folio(struct swap_cluster_info *ci, struct folio *folio,
- swp_entry_t entry, void *shadow)
+ swp_entry_t entry, void *shadow, bool swapout)
{
unsigned long nr_pages = folio_nr_pages(folio);
- __swap_cache_do_del_folio(ci, folio, entry, shadow);
node_stat_mod_folio(folio, NR_FILE_PAGES, -nr_pages);
lruvec_stat_mod_folio(folio, NR_SWAPCACHE, -nr_pages);
+
+ if (swapout)
+ __memcg1_swapout(folio, ci);
+
+ __swap_cache_do_del_folio(ci, folio, entry, shadow);
}
/**
@@ -339,7 +356,7 @@ void swap_cache_del_folio(struct folio *folio)
swp_entry_t entry = folio->swap;
ci = swap_cluster_lock(__swap_entry_to_info(entry), swp_offset(entry));
- __swap_cache_del_folio(ci, folio, entry, NULL);
+ __swap_cache_del_folio(ci, folio, entry, NULL, false);
swap_cluster_unlock(ci);
folio_ref_sub(folio, folio_nr_pages(folio));
diff --git a/mm/truncate.c b/mm/truncate.c
index 7912d3123170..29782aced8ba 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -880,8 +880,7 @@ void pagecache_isize_extended(struct inode *inode, loff_t from, loff_t to)
unsigned int offset, end;
offset = from - folio_pos(folio);
- end = min_t(unsigned int, to - folio_pos(folio),
- folio_size(folio));
+ end = min_t(loff_t, to - folio_pos(folio), folio_size(folio));
folio_zero_segment(folio, offset, end);
}
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 74f04c323c50..500962be1162 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -1663,8 +1663,12 @@ static long move_pages_ptes(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd
pte_unmap(dst_pte);
src_pte = dst_pte = NULL;
ret = split_folio(src_folio);
- if (ret)
+ if (ret) {
+ /* do not retry, extra refs may be held forever */
+ if (ret == -EAGAIN)
+ ret = -EBUSY;
goto out;
+ }
/* have to reacquire the folio after it got split */
folio_unlock(src_folio);
folio_put(src_folio);
diff --git a/mm/vma.c b/mm/vma.c
index 6c35f0d775ab..3d63f73e723b 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -1225,6 +1225,11 @@ static struct vm_area_struct *vma_merge_copied_range(struct vma_merge_struct *vm
vmg->copied_from = vmg->middle;
vmg->middle = NULL;
+
+ /* If not registered for remap, uffd context will be cleared. */
+ if (vma_has_uffd_without_event_remap(vmg->copied_from))
+ return NULL;
+
return vma_merge_new_range(vmg);
}
@@ -2577,14 +2582,17 @@ static int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,
/* Private writable mapping: check memory availability. */
if (accountable_mapping(map)) {
- map->charged = map->pglen;
- map->charged -= vms->nr_accounted;
- if (map->charged) {
- error = security_vm_enough_memory_mm(map->mm, map->charged);
+ const unsigned long uncharged = map->pglen - vms->nr_accounted;
+
+ /* Charge only what's not already charged by mappings we replace. */
+ if (uncharged) {
+ error = security_vm_enough_memory_mm(map->mm, uncharged);
if (error)
return error;
}
+ /* Our new mapping owns the entire charge now. */
+ map->charged = map->pglen;
vms->nr_accounted = 0;
vma_flags_set(&map->vma_flags, VMA_ACCOUNT_BIT);
}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f11491ee9ed5..6a53762d5544 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -735,8 +735,7 @@ static int __remove_mapping(struct address_space *mapping, struct folio *folio,
if (reclaimed && !mapping_exiting(mapping))
shadow = workingset_eviction(folio, target_memcg);
- __memcg1_swapout(folio, ci);
- __swap_cache_del_folio(ci, folio, swap, shadow);
+ __swap_cache_del_folio(ci, folio, swap, shadow, true);
swap_cluster_unlock_irq(ci);
} else {
void (*free_folio)(struct folio *);
@@ -5102,7 +5101,13 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
struct mem_cgroup *memcg = lruvec_memcg(lruvec);
struct pglist_data *pgdat = lruvec_pgdat(lruvec);
- /* lru_gen_age_node() called mem_cgroup_calculate_protection() */
+ /*
+ * For kswapd, mem_cgroup_calculate_protection() has already
+ * been called during the top-down cgroup traversal.
+ */
+ if (!current_is_kswapd())
+ mem_cgroup_calculate_protection_path(NULL, memcg);
+
if (mem_cgroup_below_min(NULL, memcg))
return MEMCG_LRU_YOUNG;
@@ -5253,7 +5258,7 @@ static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *
sc->nr_reclaimed = 0;
if (mem_cgroup_disabled())
- shrink_one(&pgdat->__lruvec, sc);
+ shrink_one(mem_cgroup_lruvec(NULL, pgdat), sc);
else
shrink_many(pgdat, sc);
@@ -7842,7 +7847,8 @@ static unsigned long __node_reclaim(struct pglist_data *pgdat,
sc->gfp_mask);
cond_resched();
- psi_memstall_enter(&pflags);
+ if (!sc->proactive)
+ psi_memstall_enter(&pflags);
delayacct_freepages_start();
fs_reclaim_acquire(sc->gfp_mask);
/*
@@ -7866,7 +7872,8 @@ static unsigned long __node_reclaim(struct pglist_data *pgdat,
memalloc_noreclaim_restore(noreclaim_flag);
fs_reclaim_release(sc->gfp_mask);
delayacct_freepages_end();
- psi_memstall_leave(&pflags);
+ if (!sc->proactive)
+ psi_memstall_leave(&pflags);
trace_mm_vmscan_node_reclaim_end(sc->nr_reclaimed, NULL);
@@ -8034,7 +8041,7 @@ int user_proactive_reclaim(char *buf,
} else {
struct scan_control sc = {
.gfp_mask = current_gfp_context(gfp_mask),
- .reclaim_idx = gfp_zone(gfp_mask),
+ .reclaim_idx = MAX_NR_ZONES - 1,
.proactive_swappiness = swappiness == -1 ? NULL : &swappiness,
.priority = DEF_PRIORITY,
.may_writepage = 1,
diff --git a/mm/workingset.c b/mm/workingset.c
index 7ac2b88c80ae..8412f4840ae3 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -688,10 +688,9 @@ static unsigned long count_shadow_nodes(struct shrinker *shrinker,
mem_cgroup_flush_stats_ratelimited(sc->memcg);
lruvec = mem_cgroup_lruvec(sc->memcg, NODE_DATA(sc->nid));
-
for (pages = 0, i = 0; i < NR_LRU_LISTS; i++)
- pages += lruvec_lru_size(lruvec, i, MAX_NR_ZONES - 1);
-
+ pages += lruvec_page_state_local(lruvec,
+ NR_LRU_BASE + i);
pages += lruvec_page_state_local(
lruvec, NR_SLAB_RECLAIMABLE_B) >> PAGE_SHIFT;
pages += lruvec_page_state_local(
diff --git a/mm/zswap.c b/mm/zswap.c
index 37f34e406c8e..e6e6377a62a0 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -324,6 +324,8 @@ static struct zswap_pool *zswap_pool_create(char *compressor)
zswap_pool_debug("created", pool);
+ static_branch_enable(&zswap_ever_enabled);
+
return pool;
ref_fail:
@@ -1803,9 +1805,8 @@ static int zswap_setup(void)
pool = __zswap_pool_create_fallback();
if (pool) {
pr_info("loaded using pool %s\n", pool->tfm_name);
- list_add(&pool->list, &zswap_pools);
+ list_add_rcu(&pool->list, &zswap_pools);
zswap_has_pool = true;
- static_branch_enable(&zswap_ever_enabled);
} else {
pr_err("pool creation failed\n");
zswap_enabled = false;
diff --git a/net/bluetooth/6lowpan.c b/net/bluetooth/6lowpan.c
index 30f4afa18bc8..20fb20eea8e7 100644
--- a/net/bluetooth/6lowpan.c
+++ b/net/bluetooth/6lowpan.c
@@ -1209,6 +1209,7 @@ static int lowpan_control_open(struct inode *inode, struct file *file)
}
static const struct file_operations lowpan_control_fops = {
+ .owner = THIS_MODULE,
.open = lowpan_control_open,
.read = seq_read,
.write = lowpan_control_write,
diff --git a/net/bluetooth/hci_debugfs.c b/net/bluetooth/hci_debugfs.c
index 2559fb6324d6..cd5ef398f238 100644
--- a/net/bluetooth/hci_debugfs.c
+++ b/net/bluetooth/hci_debugfs.c
@@ -65,6 +65,7 @@ static ssize_t __name ## _write(struct file *file, \
} \
\
static const struct file_operations __name ## _fops = { \
+ .owner = THIS_MODULE, \
.open = simple_open, \
.read = __name ## _read, \
.write = __name ## _write, \
@@ -283,6 +284,7 @@ static ssize_t use_debug_keys_read(struct file *file, char __user *user_buf,
}
static const struct file_operations use_debug_keys_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = use_debug_keys_read,
.llseek = default_llseek,
@@ -301,6 +303,7 @@ static ssize_t sc_only_mode_read(struct file *file, char __user *user_buf,
}
static const struct file_operations sc_only_mode_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = sc_only_mode_read,
.llseek = default_llseek,
@@ -437,6 +440,7 @@ static ssize_t ssp_debug_mode_read(struct file *file, char __user *user_buf,
}
static const struct file_operations ssp_debug_mode_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = ssp_debug_mode_read,
.llseek = default_llseek,
@@ -529,6 +533,7 @@ static ssize_t force_bredr_smp_write(struct file *file,
}
static const struct file_operations force_bredr_smp_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = force_bredr_smp_read,
.write = force_bredr_smp_write,
@@ -778,6 +783,7 @@ static ssize_t force_static_address_write(struct file *file,
}
static const struct file_operations force_static_address_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = force_static_address_read,
.write = force_static_address_write,
@@ -1177,6 +1183,7 @@ static ssize_t force_no_mitm_write(struct file *file,
}
static const struct file_operations force_no_mitm_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = force_no_mitm_read,
.write = force_no_mitm_write,
@@ -1315,6 +1322,7 @@ static ssize_t dut_mode_write(struct file *file, const char __user *user_buf,
}
static const struct file_operations dut_mode_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = dut_mode_read,
.write = dut_mode_write,
@@ -1371,6 +1379,7 @@ static ssize_t vendor_diag_write(struct file *file, const char __user *user_buf,
}
static const struct file_operations vendor_diag_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = vendor_diag_read,
.write = vendor_diag_write,
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index d87d2dfbce68..b86999899f66 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -320,6 +320,10 @@ static int l2cap_seq_list_init(struct l2cap_seq_list *seq_list, u16 size)
{
size_t alloc_size, i;
+ /* roundup_pow_of_two(0) is undefined */
+ if (!size)
+ return -EINVAL;
+
/* Allocated size is a power of 2 to map sequence numbers
* (which may be up to 14 bits) in to a smaller array that is
* sized for the negotiated ERTM transmit windows.
@@ -3649,18 +3653,33 @@ static int l2cap_parse_conf_req(struct l2cap_chan *chan, void *data, size_t data
break;
case L2CAP_MODE_ERTM:
- if (!test_bit(CONF_EWS_RECV, &chan->conf_state))
- chan->remote_tx_win = rfc.txwin_size;
- else
- rfc.txwin_size = L2CAP_DEFAULT_TX_WINDOW;
-
- chan->remote_max_tx = rfc.max_transmit;
-
size = min_t(u16, le16_to_cpu(rfc.max_pdu_size),
chan->conn->mtu - L2CAP_EXT_HDR_SIZE -
L2CAP_SDULEN_SIZE - L2CAP_FCS_SIZE);
rfc.max_pdu_size = cpu_to_le16(size);
- chan->remote_mps = size;
+
+ /* The TxWindow must be within 1 to 63, otherwise
+ * propose the default one and keep the parameters of
+ * an earlier request. This overrides a PENDING result
+ * from the EFS check above, so the request is no
+ * longer pending, nor done if an earlier one was.
+ */
+ if (!rfc.txwin_size ||
+ rfc.txwin_size > L2CAP_DEFAULT_TX_WINDOW) {
+ result = L2CAP_CONF_UNACCEPT;
+ clear_bit(CONF_LOC_CONF_PEND, &chan->conf_state);
+ clear_bit(CONF_OUTPUT_DONE, &chan->conf_state);
+ rfc.txwin_size = L2CAP_DEFAULT_TX_WINDOW;
+ remote_efs = 0;
+ } else {
+ if (!test_bit(CONF_EWS_RECV, &chan->conf_state))
+ chan->remote_tx_win = rfc.txwin_size;
+ else
+ rfc.txwin_size = L2CAP_DEFAULT_TX_WINDOW;
+
+ chan->remote_max_tx = rfc.max_transmit;
+ chan->remote_mps = size;
+ }
__l2cap_set_ertm_timeouts(chan, &rfc);
@@ -3802,7 +3821,11 @@ static int l2cap_parse_conf_rsp(struct l2cap_chan *chan, void *rsp, int len,
}
}
- if (chan->mode == L2CAP_MODE_BASIC && chan->mode != rfc.mode)
+ /* A connected channel cannot change its mode, as the new mode would
+ * not be initialized.
+ */
+ if (chan->mode != rfc.mode &&
+ (chan->mode == L2CAP_MODE_BASIC || chan->state == BT_CONNECTED))
return -ECONNREFUSED;
chan->mode = rfc.mode;
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index 38aedd7b00d3..1eab4478abf4 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -815,6 +815,10 @@ static int l2cap_sock_setsockopt_old(struct socket *sock, int optname,
break;
}
+ /* A zero TxWindow is invalid, use the default instead */
+ if (!opts.txwin_size)
+ opts.txwin_size = L2CAP_DEFAULT_TX_WINDOW;
+
if (!l2cap_valid_mtu(chan, opts.imtu)) {
err = -EINVAL;
break;
diff --git a/net/bluetooth/selftest.c b/net/bluetooth/selftest.c
index ae5b44bb9d3d..cdfc40e0016b 100644
--- a/net/bluetooth/selftest.c
+++ b/net/bluetooth/selftest.c
@@ -188,6 +188,7 @@ static ssize_t test_ecdh_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_ecdh_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_ecdh_read,
.llseek = default_llseek,
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index 2df303f6a38f..81d255525fe0 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -3696,6 +3696,7 @@ static ssize_t test_smp_read(struct file *file, char __user *user_buf,
}
static const struct file_operations test_smp_fops = {
+ .owner = THIS_MODULE,
.open = simple_open,
.read = test_smp_read,
.llseek = default_llseek,
diff --git a/net/ceph/mon_client.c b/net/ceph/mon_client.c
index 59b304eca273..2f0181b2566d 100644
--- a/net/ceph/mon_client.c
+++ b/net/ceph/mon_client.c
@@ -1141,6 +1141,9 @@ static int build_initial_monmap(struct ceph_mon_client *monc)
int num_mon = opt->num_mon;
int i;
+ if (num_mon < 1 || num_mon > CEPH_MAX_MON)
+ return -EINVAL;
+
/* build initial monmap */
monc->monmap = kzalloc_flex(*monc->monmap, mon_inst, num_mon);
if (!monc->monmap)
diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
index f36ce5ae7568..b8e4740d07fe 100644
--- a/net/ceph/osd_client.c
+++ b/net/ceph/osd_client.c
@@ -1108,7 +1108,9 @@ struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *osdc,
req->r_base_oloc.pool = layout->pool_id;
req->r_base_oloc.pool_ns = ceph_try_get_string(layout->pool_ns);
ceph_oid_printf(&req->r_base_oid, "%llx.%08llx", vino.ino, objnum);
- req->r_flags = flags | osdc->client->options->read_from_replica;
+ req->r_flags = flags;
+ if (flags & CEPH_OSD_FLAG_READ)
+ req->r_flags |= osdc->client->options->read_from_replica;
req->r_snapid = vino.snap;
if (flags & CEPH_OSD_FLAG_WRITE)
@@ -3565,6 +3567,7 @@ static int ceph_oloc_decode(void **p, void *end,
ceph_decode_need(p, end, len, e_inval);
struct_end = *p + len;
+ ceph_decode_need(p, end, 8 + 4 + 4, e_inval);
oloc->pool = ceph_decode_64(p);
*p += 4; /* skip preferred */
@@ -3577,7 +3580,7 @@ static int ceph_oloc_decode(void **p, void *end,
if (struct_v >= 5) {
bool changed = false;
- len = ceph_decode_32(p);
+ ceph_decode_32_safe(p, end, len, e_inval);
if (len > 0) {
ceph_decode_need(p, end, len, e_inval);
if (!oloc->pool_ns ||
@@ -3596,7 +3599,9 @@ static int ceph_oloc_decode(void **p, void *end,
}
if (struct_v >= 6) {
- s64 hash = ceph_decode_64(p);
+ s64 hash;
+
+ ceph_decode_64_safe(p, end, hash, e_inval);
if (hash != -1) {
pr_warn("ceph_object_locator::hash is set\n");
goto e_inval;
@@ -3637,7 +3642,7 @@ static int ceph_redirect_decode(void **p, void *end,
if (ret)
goto out;
- len = ceph_decode_32(p);
+ ceph_decode_32_safe(p, end, len, e_inval);
if (len > 0) {
pr_warn("ceph_request_redirect::object_name is set\n");
goto e_inval;
diff --git a/net/ceph/osdmap.c b/net/ceph/osdmap.c
index cf34b35c9a90..e69a2249ddfb 100644
--- a/net/ceph/osdmap.c
+++ b/net/ceph/osdmap.c
@@ -319,7 +319,7 @@ static int decode_choose_arg(void **p, void *end, struct crush_choose_arg *arg)
if (arg->weight_set_size) {
u32 i;
- arg->weight_set = kmalloc_objs(*arg->weight_set,
+ arg->weight_set = kzalloc_objs(*arg->weight_set,
arg->weight_set_size, GFP_NOIO);
if (!arg->weight_set)
return -ENOMEM;
@@ -376,21 +376,33 @@ static int decode_choose_args(void **p, void *end, struct crush_map *c)
ceph_decode_32_safe(p, end, num_buckets, e_inval);
while (num_buckets--) {
struct crush_choose_arg *arg;
+ struct crush_bucket *bucket;
u32 bucket_index;
+ u32 i;
ceph_decode_32_safe(p, end, bucket_index, e_inval);
if (bucket_index >= arg_map->size)
goto e_inval;
arg = &arg_map->args[bucket_index];
+ if (arg->ids || arg->weight_set)
+ goto e_inval;
ret = decode_choose_arg(p, end, arg);
if (ret)
goto fail;
+ bucket = c->buckets[bucket_index];
if (arg->ids_size &&
- (!c->buckets[bucket_index] ||
- arg->ids_size != c->buckets[bucket_index]->size))
+ (!bucket || arg->ids_size != bucket->size))
+ goto e_inval;
+
+ if (arg->weight_set_size && !bucket)
goto e_inval;
+
+ for (i = 0; i < arg->weight_set_size; i++) {
+ if (arg->weight_set[i].size != bucket->size)
+ goto e_inval;
+ }
}
if (!__insert_choose_arg_map(&c->choose_args, arg_map)) {
diff --git a/net/core/dev.c b/net/core/dev.c
index 3f39d953eb48..cc7b32ab5608 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6201,6 +6201,8 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc,
}
if (unlikely(skb_vlan_tag_present(skb)) && !netdev_uses_dsa(skb->dev)) {
+ unsigned int priority_tag_depth = 1;
+
check_vlan_id:
if (skb_vlan_tag_get_id(skb)) {
/* Vlan id is non 0 and vlan_do_receive() above couldn't
@@ -6212,6 +6214,12 @@ static int __netif_receive_skb_core(struct sk_buff **pskb, bool pfmemalloc,
* 802.1Q or 802.1AD and vlan_do_receive() above could
* not find vlan dev for vlan id 0.
*/
+ if (unlikely(priority_tag_depth >= VLAN_MAX_DEPTH)) {
+ drop_reason = SKB_DROP_REASON_RECURSION_LIMIT;
+ goto drop;
+ }
+
+ priority_tag_depth++;
__vlan_hwaccel_clear_tag(skb);
skb = skb_vlan_untag(skb);
if (unlikely(!skb))
diff --git a/net/core/gro.c b/net/core/gro.c
index 29b4d02bf519..ff828a3dce3c 100644
--- a/net/core/gro.c
+++ b/net/core/gro.c
@@ -187,6 +187,10 @@ int skb_gro_receive(struct sk_buff *p, struct sk_buff *skb)
}
merge:
+ /* skb_segment() cannot split the linear part of a frag_list member. */
+ if (unlikely(headlen > offset + skb_shinfo(p)->gso_size))
+ return -E2BIG;
+
/* sk ownership - if any - completely transferred to the aggregated packet */
skb->destructor = NULL;
skb->sk = NULL;
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index da5f881fbd3b..52c2bd44778a 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -447,7 +447,9 @@ static __net_init int setup_net(struct net *net)
down_write(&net_rwsem);
list_add_tail_rcu(&net->list, &net_namespace_list);
up_write(&net_rwsem);
- ns_tree_add_raw(net);
+ /* Without CONFIG_NET_NS init_net isn't exposed as a namespace. */
+ if (IS_ENABLED(CONFIG_NET_NS))
+ ns_tree_add_raw(net);
out:
return error;
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 2bfabde31ea9..ea4c38dee19f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -3961,7 +3961,18 @@ void skb_copy_and_csum_dev(const struct sk_buff *skb, u8 *to)
else
csstart = skb_headlen(skb);
- BUG_ON(csstart > skb_headlen(skb));
+ if (unlikely(csstart < 0 || csstart > skb_headlen(skb) ||
+ (skb->ip_summed == CHECKSUM_PARTIAL &&
+ csstart + skb->csum_offset + sizeof(__sum16) >
+ skb->len))) {
+ DEBUG_NET_WARN_ON_ONCE(1);
+ /* Send the frame without the checksum filled in, or send
+ * zeroes if we can't even copy it.
+ */
+ if (skb_copy_bits(skb, 0, to, skb->len))
+ memset(to, 0, skb->len);
+ return;
+ }
skb_copy_from_linear_data(skb, to, csstart);
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index c6f8011200a0..8e891c8141f8 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1140,10 +1140,14 @@ static void reqsk_timer_handler(struct timer_list *t)
*/
if (nreq) {
__NET_INC_STATS(net, LINUX_MIB_TCPMIGRATEREQFAILURE);
+ /* The timer was not armed, drop its reference here. */
+ reqsk_put(nreq);
no_ownership:
reqsk_migrate_reset(nreq);
reqsk_queue_removed(queue, nreq);
- __reqsk_free(nreq);
+ /* Drop the temporary reference and the one reserved for ehash. */
+ reqsk_put(nreq);
+ reqsk_put(nreq);
}
drop:
@@ -1451,7 +1455,7 @@ struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMIGRATEREQFAILURE);
reqsk_migrate_reset(nreq);
- __reqsk_free(nreq);
+ reqsk_put(nreq);
} else if (inet_csk_reqsk_queue_add(sk, req, child)) {
return child;
}
@@ -1506,7 +1510,7 @@ void inet_csk_listen_stop(struct sock *sk)
__NET_INC_STATS(sock_net(nsk),
LINUX_MIB_TCPMIGRATEREQFAILURE);
reqsk_migrate_reset(nreq);
- __reqsk_free(nreq);
+ reqsk_put(nreq);
}
rcu_read_unlock();
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index b286ee429da8..c18da91c1f03 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -23,6 +23,7 @@
#include <net/inet_ecn.h>
#include <net/ip.h>
#include <net/ipv6.h>
+#include <net/xfrm.h>
/* Use skb->cb to track consecutive/adjacent fragments coming at
* the end of the queue. Nodes in the rb-tree queue will
@@ -441,6 +442,10 @@ int inet_frag_queue_insert(struct inet_frag_queue *q, struct sk_buff *skb,
{
struct sk_buff *last = q->fragments_tail;
+ /* The reassembled skb keeps the first fragment's secpath. */
+ if (last && !secpath_equal(last, skb))
+ return IPFRAG_MIXED_SECPATH;
+
/* An IP fragment is never a GSO packet, but an untrusted source
* (virtio_net_hdr) may have attached GSO metadata to it. Do not let
* that reach the reassembled skb, whose head keeps the first
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index ba0faa9ae2bb..58d54965036a 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -552,8 +552,9 @@ struct sock *__inet_lookup_established(const struct net *net,
if (likely(inet_match(net, sk, acookie, ports, dif, sdif))) {
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
- if (unlikely(!inet_match(net, sk, acookie,
- ports, dif, sdif))) {
+ if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
+ !inet_match(net, sk, acookie, ports, dif,
+ sdif))) {
sock_gen_put(sk);
goto begin;
}
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index c790d2f49487..36105dd19282 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -378,8 +378,9 @@ static int ip_frag_queue(struct ipq *qp, struct sk_buff *skb, int *refs)
err = -EINVAL;
goto err;
}
+ if (err == IPFRAG_OVERLAP)
+ __IP_INC_STATS(net, IPSTATS_MIB_REASM_OVERLAPS);
err = -EINVAL;
- __IP_INC_STATS(net, IPSTATS_MIB_REASM_OVERLAPS);
discard_qp:
inet_frag_kill(&qp->q, refs);
__IP_INC_STATS(net, IPSTATS_MIB_REASMFAILS);
diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index e4878e9aa636..149a4e83b692 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -1367,6 +1367,7 @@ static int erspan_tunnel_init(struct net_device *dev)
dev->features |= GRE_FEATURES;
dev->hw_features |= GRE_FEATURES;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
netif_keep_dst(dev);
return ip_tunnel_init(dev);
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index b9c544d48c45..f51014b37178 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -537,6 +537,12 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb, struct net_device *dev)
};
int err;
+ if (skb->protocol != htons(ETH_P_IP) || !pskb_inet_may_pull(skb)) {
+ DEV_STATS_INC(dev, tx_errors);
+ kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
rcu_read_lock();
err = ipmr_fib_lookup(net, &fl4, &mrt);
if (err < 0) {
@@ -1057,6 +1063,7 @@ static void ipmr_cache_resolve(struct net *net, struct mr_table *mrt,
static int ipmr_cache_report(const struct mr_table *mrt,
struct sk_buff *pkt, vifi_t vifi, int assert)
{
+ unsigned char pkt_cb[sizeof(pkt->cb)];
const int ihl = ip_hdrlen(pkt);
struct sock *mroute_sk;
struct igmphdr *igmp;
@@ -1112,8 +1119,6 @@ static int ipmr_cache_report(const struct mr_table *mrt,
msg = (struct igmpmsg *)skb_network_header(skb);
msg->im_vif = vifi;
msg->im_vif_hi = vifi >> 8;
- ipv4_pktinfo_prepare(mroute_sk, pkt, false);
- memcpy(skb->cb, pkt->cb, sizeof(skb->cb));
/* Add our header.
* Note that code, csum and group fields are cleared.
*/
@@ -1124,6 +1129,15 @@ static int ipmr_cache_report(const struct mr_table *mrt,
skb->transport_header = skb->network_header;
}
+ memset(skb->cb, 0, sizeof(skb->cb));
+ /* Route queries from ipmr_get_route() carry NETLINK_CB, not IPCB. */
+ if (ip_hdr(pkt)->version) {
+ memcpy(pkt_cb, pkt->cb, sizeof(pkt_cb));
+ ipv4_pktinfo_prepare(mroute_sk, pkt, false);
+ *PKTINFO_SKB_CB(skb) = *PKTINFO_SKB_CB(pkt);
+ memcpy(pkt->cb, pkt_cb, sizeof(pkt_cb));
+ }
+
igmpmsg_netlink_event(mrt, skb);
/* Deliver to mrouted */
@@ -2064,7 +2078,8 @@ static void ip_mr_forward(struct net *net, struct mr_table *mrt,
}
/* Wrong interface: drop packet and (maybe) send PIM assert. */
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
if (rt_is_output_route(skb_rtable(skb))) {
/* It is our own packet, looped back.
* Very complicated situation...
@@ -2492,6 +2507,7 @@ int ipmr_get_route(struct net *net, struct sk_buff *skb,
skb_push(skb2, sizeof(struct iphdr));
skb_reset_network_header(skb2);
iph = ip_hdr(skb2);
+ memset(iph, 0, sizeof(*iph));
iph->ihl = sizeof(struct iphdr) >> 2;
iph->saddr = saddr;
iph->daddr = daddr;
diff --git a/net/ipv4/netfilter/nf_reject_ipv4.c b/net/ipv4/netfilter/nf_reject_ipv4.c
index 59ec465a9df9..7cb33af671e2 100644
--- a/net/ipv4/netfilter/nf_reject_ipv4.c
+++ b/net/ipv4/netfilter/nf_reject_ipv4.c
@@ -357,6 +357,11 @@ void nf_send_unreach(struct sk_buff *skb_in, int code, int hook)
if (!skb_valid_dst(skb_in) && nf_reject_fill_skb_dst(skb_in) < 0)
return;
+ if (hook == NF_INET_INGRESS) {
+ memset(IPCB(skb_in), 0, sizeof(*IPCB(skb_in)));
+ IPCB(skb_in)->iif = skb_in->skb_iif;
+ }
+
if (skb_csum_unnecessary(skb_in) ||
!nf_reject_verify_csum(skb_in, dataoff, proto)) {
icmp_send(skb_in, ICMP_DEST_UNREACH, code, 0);
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 268d8821f7e0..1d8fcd19eaf6 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -1365,8 +1365,8 @@ static unsigned int ipv4_default_advmss(const struct dst_entry *dst)
rcu_read_lock();
net = dst_dev_net_rcu(dst);
- advmss = max_t(unsigned int, ip_dst_mtu_configured(dst) - header_size,
- net->ipv4.ip_rt_min_advmss);
+ advmss = max_t(unsigned int, ipv4_mtu(dst) - header_size,
+ net->ipv4.ip_rt_min_advmss);
rcu_read_unlock();
return min(advmss, IPV4_MAX_PMTU - header_size);
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 73e129768184..644030561537 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -218,6 +218,8 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
bh_unlock_sock(child);
sock_put(child);
+ reqsk_put(req);
+ return NULL;
}
__reqsk_free(req);
diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c
index e3760daa3470..bf193e7ec516 100644
--- a/net/ipv4/sysctl_net_ipv4.c
+++ b/net/ipv4/sysctl_net_ipv4.c
@@ -26,6 +26,7 @@ static int ip_local_port_range_max[] = { 65535, 65535 };
static int tcp_adv_win_scale_min = -31;
static int tcp_adv_win_scale_max = 31;
static int tcp_app_win_max = 31;
+static int tcp_tso_rtt_log_max = 31;
static int tcp_min_snd_mss_min = TCP_MIN_SND_MSS;
static int tcp_min_snd_mss_max = 65535;
static int tcp_rto_max_max = TCP_RTO_MAX_SEC * MSEC_PER_SEC;
@@ -1406,6 +1407,7 @@ static const struct ctl_table ipv4_net_table[] = {
.maxlen = sizeof(u8),
.mode = 0644,
.proc_handler = proc_dou8vec_minmax,
+ .extra2 = &tcp_tso_rtt_log_max,
},
{
.procname = "tcp_min_rtt_wlen",
diff --git a/net/ipv4/tcp_cong.c b/net/ipv4/tcp_cong.c
index 8e83ef81fc18..e3f462f161dd 100644
--- a/net/ipv4/tcp_cong.c
+++ b/net/ipv4/tcp_cong.c
@@ -288,12 +288,12 @@ int tcp_set_default_congestion_control(struct net *net, const char *name)
ca = tcp_ca_find_autoload(name);
if (!ca) {
ret = -ENOENT;
- } else if (!bpf_try_module_get(ca, ca->owner)) {
- ret = -EBUSY;
} else if (!net_eq(net, &init_net) &&
!(ca->flags & TCP_CONG_NON_RESTRICTED)) {
/* Only init netns can set default to a restricted algorithm */
ret = -EPERM;
+ } else if (!bpf_try_module_get(ca, ca->owner)) {
+ ret = -EBUSY;
} else {
prev = xchg(&net->ipv4.tcp_congestion_control, ca);
if (prev)
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 99baf14afdfd..3583867487d7 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -7777,7 +7777,9 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
if (!inet_csk_reqsk_queue_add(sk, req, fastopen_sk)) {
bh_unlock_sock(fastopen_sk);
sock_put(fastopen_sk);
- goto drop_and_free;
+ /* tcp_disconnect() already dropped one reqsk ref. */
+ reqsk_put(req);
+ goto drop;
}
READ_ONCE(sk->sk_data_ready)(sk);
bh_unlock_sock(fastopen_sk);
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index e0c392e29de5..9f47b6644ed4 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -2712,7 +2712,7 @@ static int tcp_mtu_probe(struct sock *sk)
mss_now = tcp_current_mss(sk);
probe_size = tcp_mtu_to_mss(sk, (icsk->icsk_mtup.search_high +
icsk->icsk_mtup.search_low) >> 1);
- size_needed = probe_size + (tp->reordering + 1) * (u64)tp->mss_cache;
+ size_needed = probe_size + ((u64)tp->reordering + 1) * tp->mss_cache;
interval = icsk->icsk_mtup.search_high - icsk->icsk_mtup.search_low;
/* When misfortune happens, we are reprobing actively,
* and then reprobe timer has expired. We stick with current
diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
index c90ee6dd7446..a2d3fb187f76 100644
--- a/net/ipv6/addrconf.c
+++ b/net/ipv6/addrconf.c
@@ -1155,10 +1155,19 @@ ipv6_add_addr(struct inet6_dev *idev, struct ifa6_config *cfg,
ifa->flags |= IFA_F_TENTATIVE;
ifa->valid_lft = cfg->valid_lft;
ifa->prefered_lft = cfg->preferred_lft;
- ifa->cstamp = ifa->tstamp = jiffies;
ifa->tokenized = false;
ifa->rt = f6i;
+ ifa->ifpub = cfg->ifpub;
+ if (cfg->ifpub) {
+ ifa->cstamp = cfg->cstamp;
+ ifa->tstamp = cfg->tstamp;
+ } else {
+ unsigned long now = jiffies;
+
+ ifa->cstamp = now;
+ ifa->tstamp = now;
+ }
ifa->idev = idev;
in6_dev_hold(idev);
@@ -1168,14 +1177,18 @@ ipv6_add_addr(struct inet6_dev *idev, struct ifa6_config *cfg,
rcu_read_lock();
- err = ipv6_add_addr_hash(idev->dev, ifa);
+ write_lock_bh(&idev->lock);
+
+ if (idev->dead)
+ err = -ENODEV;
+ else
+ err = ipv6_add_addr_hash(idev->dev, ifa);
if (err < 0) {
+ write_unlock_bh(&idev->lock);
rcu_read_unlock();
goto out;
}
- write_lock_bh(&idev->lock);
-
/* Add to inet6_dev unicast addr list. */
ipv6_link_dev_addr(idev, ifa);
@@ -1487,6 +1500,9 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
cfg.pfx = &addr;
cfg.scope = ipv6_addr_scope(cfg.pfx);
+ cfg.ifpub = ifp;
+ cfg.cstamp = now;
+ cfg.tstamp = tmp_tstamp;
ift = ipv6_add_addr(idev, &cfg, block, NULL);
if (IS_ERR(ift)) {
@@ -1497,12 +1513,6 @@ static int ipv6_create_tempaddr(struct inet6_ifaddr *ifp, bool block)
goto retry;
}
- spin_lock_bh(&ift->lock);
- ift->ifpub = ifp;
- ift->cstamp = now;
- ift->tstamp = tmp_tstamp;
- spin_unlock_bh(&ift->lock);
-
addrconf_dad_start(ift);
in6_ifa_put(ift);
in6_dev_put(idev);
@@ -2188,6 +2198,7 @@ void addrconf_dad_failure(struct sk_buff *skb, struct inet6_ifaddr *ifp)
ifp->idev->dev->name, &ifp->addr, eth_hdr(skb)->h_source);
if (ifp->flags & IFA_F_STABLE_PRIVACY) {
+ bool start_dad = false;
struct in6_addr new_addr;
struct inet6_ifaddr *ifp2;
int retries = ifp->stable_privacy_retry + 1;
@@ -2226,11 +2237,14 @@ void addrconf_dad_failure(struct sk_buff *skb, struct inet6_ifaddr *ifp)
goto lock_errdad;
spin_lock_bh(&ifp2->lock);
- ifp2->stable_privacy_retry = retries;
- ifp2->state = INET6_IFADDR_STATE_PREDAD;
+ if (ifp2->state != INET6_IFADDR_STATE_DEAD) {
+ ifp2->stable_privacy_retry = retries;
+ start_dad = true;
+ }
spin_unlock_bh(&ifp2->lock);
- addrconf_mod_dad_work(ifp2, net->ipv6.sysctl.idgen_delay);
+ if (start_dad)
+ addrconf_mod_dad_work(ifp2, net->ipv6.sysctl.idgen_delay);
in6_ifa_put(ifp2);
lock_errdad:
spin_lock_bh(&ifp->lock);
@@ -3897,7 +3911,9 @@ static int addrconf_ifdown(struct net_device *dev, bool unregister)
* Do not dev_put!
*/
if (unregister) {
+ write_lock_bh(&idev->lock);
WRITE_ONCE(idev->dead, 1);
+ write_unlock_bh(&idev->lock);
/* protected by rtnl_lock */
RCU_INIT_POINTER(dev->ip6_ptr, NULL);
@@ -4019,6 +4035,10 @@ static int addrconf_ifdown(struct net_device *dev, bool unregister)
}
if (!keep) {
+ spin_lock_bh(&net->ipv6.addrconf_hash_lock);
+ hlist_del_init_rcu(&ifa->addr_lst);
+ spin_unlock_bh(&net->ipv6.addrconf_hash_lock);
+
write_lock_bh(&idev->lock);
list_del_rcu(&ifa->if_list);
write_unlock_bh(&idev->lock);
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index b111b51d69fc..aa5fa1dd062f 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -108,7 +108,9 @@ struct sock *__inet6_lookup_established(const struct net *net,
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
- if (unlikely(!inet6_match(net, sk, saddr, daddr, ports, dif, sdif))) {
+ if (unlikely(hlist_nulls_unhashed_lockless(&sk->sk_nulls_node) ||
+ !inet6_match(net, sk, saddr, daddr, ports, dif,
+ sdif))) {
sock_gen_put(sk);
goto begin;
}
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 04f7c70b7320..3429543ec77e 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -1864,6 +1864,7 @@ static int ip6erspan_tap_init(struct net_device *dev)
dev->mtu -= 8;
dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+ dev->lltx = true;
ip6erspan_tnl_link_config(tunnel, 1);
netdev_hold(dev, &tunnel->dev_tracker, GFP_KERNEL);
diff --git a/net/ipv6/ip6mr.c b/net/ipv6/ip6mr.c
index 9d8116b5edb1..ed9d786f1a42 100644
--- a/net/ipv6/ip6mr.c
+++ b/net/ipv6/ip6mr.c
@@ -641,7 +641,7 @@ static netdev_tx_t reg_vif_xmit(struct sk_buff *skb,
.flowi6_mark = skb->mark,
};
- if (!pskb_inet_may_pull(skb))
+ if (skb->protocol != htons(ETH_P_IPV6) || !pskb_inet_may_pull(skb))
goto tx_err;
rcu_read_lock();
@@ -2218,7 +2218,8 @@ static void ip6_mr_forward(struct net *net, struct mr_table *mrt,
/*
* Wrong interface: drop packet and (maybe) send PIM assert.
*/
- if (rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
+ if (true_vifi < 0 ||
+ rcu_access_pointer(mrt->vif_table[vif].dev) != dev) {
atomic_long_inc(&c->_c.mfc_un.res.wrong_if);
if (true_vifi >= 0 && READ_ONCE(mrt->mroute_do_assert) &&
diff --git a/net/ipv6/netfilter/nf_reject_ipv6.c b/net/ipv6/netfilter/nf_reject_ipv6.c
index 07cdaa10da0d..5e08d8529cc8 100644
--- a/net/ipv6/netfilter/nf_reject_ipv6.c
+++ b/net/ipv6/netfilter/nf_reject_ipv6.c
@@ -444,6 +444,11 @@ void nf_send_unreach6(struct net *net, struct sk_buff *skb_in,
if (hooknum == NF_INET_LOCAL_OUT && skb_in->dev == NULL)
skb_in->dev = net->loopback_dev;
+ if (hooknum == NF_INET_INGRESS) {
+ memset(IP6CB(skb_in), 0, sizeof(*IP6CB(skb_in)));
+ IP6CB(skb_in)->iif = skb_in->skb_iif;
+ }
+
if (!skb_valid_dst(skb_in) && nf_reject6_fill_skb_dst(skb_in) < 0)
return;
diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c
index 05c51f669754..8418ed9423f4 100644
--- a/net/ipv6/reassembly.c
+++ b/net/ipv6/reassembly.c
@@ -232,6 +232,7 @@ static int ip6_frag_queue(struct net *net,
}
skb_dst_drop(skb);
+ skb_orphan(skb);
return -EINPROGRESS;
insert_error:
@@ -240,8 +241,9 @@ static int ip6_frag_queue(struct net *net,
err = -EINVAL;
goto err;
}
+ if (err == IPFRAG_OVERLAP)
+ __IP6_INC_STATS(net, idev, IPSTATS_MIB_REASM_OVERLAPS);
err = -EINVAL;
- __IP6_INC_STATS(net, idev, IPSTATS_MIB_REASM_OVERLAPS);
discard_fq:
inet_frag_kill(&fq->q, refs);
__IP6_INC_STATS(net, idev, IPSTATS_MIB_REASMFAILS);
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index a5e955e56e79..bd6bc9ada539 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -3267,7 +3267,7 @@ void ip6_sk_redirect(struct sk_buff *skb, struct sock *sk)
static unsigned int ip6_default_advmss(const struct dst_entry *dst)
{
- unsigned int mtu = ip6_dst_mtu_configured(dst);
+ unsigned int mtu = dst6_mtu(dst);
struct net *net;
mtu -= sizeof(struct ipv6hdr) + sizeof(struct tcphdr);
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 4712cc41881a..7e042a3e648c 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -915,11 +915,9 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb,
* check if we sre sending sequence numbers and if not,
* configure it so.
*/
- if (!session->lns_mode && !session->send_seq) {
+ if (!session->lns_mode && !READ_ONCE(session->send_seq)) {
trace_session_seqnum_lns_enable(session);
- session->send_seq = 1;
- l2tp_session_set_header_len(session, tunnel->version,
- tunnel->encap);
+ WRITE_ONCE(session->send_seq, true);
}
} else {
/* No sequence numbers.
@@ -937,12 +935,10 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb,
* If we're the LNS and we're sending sequence numbers, the
* LAC is broken. Discard the frame.
*/
- if (!session->lns_mode && session->send_seq) {
+ if (!session->lns_mode && READ_ONCE(session->send_seq)) {
trace_session_seqnum_lns_disable(session);
- session->send_seq = 0;
- l2tp_session_set_header_len(session, tunnel->version,
- tunnel->encap);
- } else if (session->send_seq) {
+ WRITE_ONCE(session->send_seq, false);
+ } else if (READ_ONCE(session->send_seq)) {
pr_debug_ratelimited("%s: recv data has no seq numbers when required. Discarding.\n",
session->name);
atomic_long_inc(&session->stats.rx_seq_discards);
@@ -1138,7 +1134,8 @@ static void l2tp_udp_encap_err_recv(struct sock *sk, struct sk_buff *skb, int er
/* Build an L2TP header for the session into the buffer provided.
*/
-static int l2tp_build_l2tpv2_header(struct l2tp_session *session, void *buf)
+static int l2tp_build_l2tpv2_header(struct l2tp_session *session, void *buf,
+ bool send_seq)
{
struct l2tp_tunnel *tunnel = session->tunnel;
__be16 *bufp = buf;
@@ -1147,14 +1144,14 @@ static int l2tp_build_l2tpv2_header(struct l2tp_session *session, void *buf)
u32 tunnel_id = tunnel->peer_tunnel_id;
u32 session_id = session->peer_session_id;
- if (session->send_seq)
+ if (send_seq)
flags |= L2TP_HDRFLAG_S;
/* Setup L2TP header. */
*bufp++ = htons(flags);
*bufp++ = htons(tunnel_id);
*bufp++ = htons(session_id);
- if (session->send_seq) {
+ if (send_seq) {
*bufp++ = htons(session->ns);
*bufp++ = 0;
session->ns++;
@@ -1165,7 +1162,8 @@ static int l2tp_build_l2tpv2_header(struct l2tp_session *session, void *buf)
return bufp - optr;
}
-static int l2tp_build_l2tpv3_header(struct l2tp_session *session, void *buf)
+static int l2tp_build_l2tpv3_header(struct l2tp_session *session, void *buf,
+ bool send_seq)
{
struct l2tp_tunnel *tunnel = session->tunnel;
char *bufp = buf;
@@ -1191,7 +1189,7 @@ static int l2tp_build_l2tpv3_header(struct l2tp_session *session, void *buf)
if (session->l2specific_type == L2TP_L2SPECTYPE_DEFAULT) {
u32 l2h = 0;
- if (session->send_seq) {
+ if (send_seq) {
l2h = 0x40000000 | session->ns;
session->ns++;
session->ns &= 0xffffff;
@@ -1225,19 +1223,24 @@ static int l2tp_xmit_queue(struct l2tp_tunnel *tunnel, struct sk_buff *skb, stru
static int l2tp_xmit_core(struct l2tp_session *session, struct sk_buff *skb, unsigned int *len)
{
struct l2tp_tunnel *tunnel = session->tunnel;
+ bool send_seq = READ_ONCE(session->send_seq);
unsigned int data_len = skb->len;
struct sock *sk = tunnel->sock;
+ u16 hdr_len = session->hdr_len;
int headroom, uhlen, udp_len;
int ret = NET_XMIT_SUCCESS;
struct inet_sock *inet;
struct udphdr *uh;
+ if (tunnel->version == L2TP_HDR_VER_2 && !send_seq)
+ hdr_len -= 4;
+
/* Check that there's enough headroom in the skb to insert IP,
* UDP and L2TP headers. If not enough, expand it to
* make room. Adjust truesize.
*/
uhlen = (tunnel->encap == L2TP_ENCAPTYPE_UDP) ? sizeof(*uh) : 0;
- headroom = NET_SKB_PAD + sizeof(struct iphdr) + uhlen + session->hdr_len;
+ headroom = NET_SKB_PAD + sizeof(struct iphdr) + uhlen + hdr_len;
if (skb_cow_head(skb, headroom)) {
kfree_skb(skb);
return NET_XMIT_DROP;
@@ -1245,9 +1248,9 @@ static int l2tp_xmit_core(struct l2tp_session *session, struct sk_buff *skb, uns
/* Setup L2TP header */
if (tunnel->version == L2TP_HDR_VER_2)
- l2tp_build_l2tpv2_header(session, __skb_push(skb, session->hdr_len));
+ l2tp_build_l2tpv2_header(session, __skb_push(skb, hdr_len), send_seq);
else
- l2tp_build_l2tpv3_header(session, __skb_push(skb, session->hdr_len));
+ l2tp_build_l2tpv3_header(session, __skb_push(skb, hdr_len), send_seq);
/* Reset control buffer */
memset(skb->cb, 0, sizeof(skb->cb));
@@ -1290,7 +1293,7 @@ static int l2tp_xmit_core(struct l2tp_session *session, struct sk_buff *skb, uns
uh = udp_hdr(skb);
uh->source = inet->inet_sport;
uh->dest = inet->inet_dport;
- udp_len = uhlen + session->hdr_len + data_len;
+ udp_len = uhlen + hdr_len + data_len;
if (udp_len > U16_MAX) {
kfree_skb(skb);
ret = NET_XMIT_DROP;
@@ -1759,16 +1762,11 @@ static void l2tp_session_del_work(struct work_struct *work)
l2tp_session_put(session);
}
-/* We come here whenever a session's send_seq, cookie_len or
- * l2specific_type parameters are set.
- */
-void l2tp_session_set_header_len(struct l2tp_session *session, int version,
- enum l2tp_encap_type encap)
+static void l2tp_session_set_header_len(struct l2tp_session *session, int version,
+ enum l2tp_encap_type encap)
{
if (version == L2TP_HDR_VER_2) {
- session->hdr_len = 6;
- if (session->send_seq)
- session->hdr_len += 4;
+ session->hdr_len = 10;
} else {
session->hdr_len = 4 + session->cookie_len;
session->hdr_len += l2tp_get_l2specific_len(session);
@@ -1776,7 +1774,6 @@ void l2tp_session_set_header_len(struct l2tp_session *session, int version,
session->hdr_len += 4;
}
}
-EXPORT_SYMBOL_GPL(l2tp_session_set_header_len);
struct l2tp_session *l2tp_session_create(int priv_size, struct l2tp_tunnel *tunnel, u32 session_id,
u32 peer_session_id, struct l2tp_session_cfg *cfg)
diff --git a/net/l2tp/l2tp_core.h b/net/l2tp/l2tp_core.h
index ffd8ced3a51f..25ae4c25d3ef 100644
--- a/net/l2tp/l2tp_core.h
+++ b/net/l2tp/l2tp_core.h
@@ -76,7 +76,7 @@ struct l2tp_session {
u8 peer_cookie[8];
int peer_cookie_len;
u16 l2specific_type;
- u16 hdr_len;
+ u16 hdr_len; /* maximum L2TP header length */
u32 nr; /* session NR state (receive) */
u32 ns; /* session NR state (send) */
struct sk_buff_head reorder_q; /* receive reorder queue */
@@ -94,8 +94,8 @@ struct l2tp_session {
char name[L2TP_SESSION_NAME_MAX]; /* for logging */
char ifname[IFNAMSIZ];
+ bool send_seq; /* send packets with sequence numbers? */
unsigned int recv_seq:1; /* expect receive packets with sequence numbers? */
- unsigned int send_seq:1; /* send packets with sequence numbers? */
unsigned int lns_mode:1; /* behave as LNS?
* LAC enables sequence numbers under LNS control.
*/
@@ -257,8 +257,6 @@ void l2tp_recv_common(struct l2tp_session *session, struct sk_buff *skb,
int l2tp_udp_encap_recv(struct sock *sk, struct sk_buff *skb);
/* Transmit path helpers for sending packets over the tunnel socket. */
-void l2tp_session_set_header_len(struct l2tp_session *session, int version,
- enum l2tp_encap_type encap);
int l2tp_xmit_skb(struct l2tp_session *session, struct sk_buff *skb);
/* Pseudowire management.
diff --git a/net/l2tp/l2tp_debugfs.c b/net/l2tp/l2tp_debugfs.c
index c12d5e1ee550..58d1d6a7768e 100644
--- a/net/l2tp/l2tp_debugfs.c
+++ b/net/l2tp/l2tp_debugfs.c
@@ -182,12 +182,12 @@ static void l2tp_dfs_seq_session_show(struct seq_file *m, void *v)
session->pwtype == L2TP_PWTYPE_ETH ? "ETH" :
session->pwtype == L2TP_PWTYPE_PPP ? "PPP" :
"");
- if (session->send_seq || session->recv_seq)
+ if (READ_ONCE(session->send_seq) || session->recv_seq)
seq_printf(m, " nr %u, ns %u\n", session->nr, session->ns);
seq_printf(m, " refcnt %d\n", refcount_read(&session->ref_count));
seq_printf(m, " config 0/0/%c/%c/-/%s %08x %u\n",
session->recv_seq ? 'R' : '-',
- session->send_seq ? 'S' : '-',
+ READ_ONCE(session->send_seq) ? 'S' : '-',
session->lns_mode ? "LNS" : "LAC",
0,
jiffies_to_msecs(session->reorder_timeout));
diff --git a/net/l2tp/l2tp_netlink.c b/net/l2tp/l2tp_netlink.c
index 38aac59d052c..89ef04c78787 100644
--- a/net/l2tp/l2tp_netlink.c
+++ b/net/l2tp/l2tp_netlink.c
@@ -700,12 +700,9 @@ static int l2tp_nl_cmd_session_modify(struct sk_buff *skb, struct genl_info *inf
if (info->attrs[L2TP_ATTR_RECV_SEQ])
session->recv_seq = nla_get_u8(info->attrs[L2TP_ATTR_RECV_SEQ]);
- if (info->attrs[L2TP_ATTR_SEND_SEQ]) {
- struct l2tp_tunnel *tunnel = session->tunnel;
-
- session->send_seq = nla_get_u8(info->attrs[L2TP_ATTR_SEND_SEQ]);
- l2tp_session_set_header_len(session, tunnel->version, tunnel->encap);
- }
+ if (info->attrs[L2TP_ATTR_SEND_SEQ])
+ WRITE_ONCE(session->send_seq,
+ nla_get_u8(info->attrs[L2TP_ATTR_SEND_SEQ]) & 1);
if (info->attrs[L2TP_ATTR_LNS_MODE])
session->lns_mode = nla_get_u8(info->attrs[L2TP_ATTR_LNS_MODE]);
@@ -748,7 +745,7 @@ static int l2tp_nl_session_send(struct sk_buff *skb, u32 portid, u32 seq, int fl
(session->peer_cookie_len &&
nla_put(skb, L2TP_ATTR_PEER_COOKIE, session->peer_cookie_len, session->peer_cookie)) ||
nla_put_u8(skb, L2TP_ATTR_RECV_SEQ, session->recv_seq) ||
- nla_put_u8(skb, L2TP_ATTR_SEND_SEQ, session->send_seq) ||
+ nla_put_u8(skb, L2TP_ATTR_SEND_SEQ, READ_ONCE(session->send_seq)) ||
nla_put_u8(skb, L2TP_ATTR_LNS_MODE, session->lns_mode) ||
(l2tp_tunnel_uses_xfrm(tunnel) &&
nla_put_u8(skb, L2TP_ATTR_USING_IPSEC, 1)) ||
diff --git a/net/l2tp/l2tp_ppp.c b/net/l2tp/l2tp_ppp.c
index ef8fdfaf051d..4f311d473a64 100644
--- a/net/l2tp/l2tp_ppp.c
+++ b/net/l2tp/l2tp_ppp.c
@@ -1195,9 +1195,7 @@ static int pppol2tp_session_setsockopt(struct sock *sk,
err = -EINVAL;
break;
}
- session->send_seq = !!val;
- l2tp_session_set_header_len(session, session->tunnel->version,
- session->tunnel->encap);
+ WRITE_ONCE(session->send_seq, !!val);
break;
case PPPOL2TP_SO_LNSMODE:
@@ -1307,7 +1305,7 @@ static int pppol2tp_session_getsockopt(struct sock *sk,
break;
case PPPOL2TP_SO_SENDSEQ:
- *val = session->send_seq;
+ *val = READ_ONCE(session->send_seq);
break;
case PPPOL2TP_SO_LNSMODE:
@@ -1546,7 +1544,7 @@ static void pppol2tp_seq_session_show(struct seq_file *m, void *v)
state, user_data_ok);
seq_printf(m, " 0/0/%c/%c/%s %08x %u\n",
session->recv_seq ? 'R' : '-',
- session->send_seq ? 'S' : '-',
+ READ_ONCE(session->send_seq) ? 'S' : '-',
session->lns_mode ? "LNS" : "LAC",
0,
jiffies_to_msecs(session->reorder_timeout));
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index d51f810dd2c5..5714634c8103 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -269,7 +269,6 @@ ieee80211_tx_h_dynamic_ps(struct ieee80211_tx_data *tx)
static ieee80211_tx_result debug_noinline
ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
{
-
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
bool assoc = false;
@@ -277,20 +276,6 @@ ieee80211_tx_h_check_assoc(struct ieee80211_tx_data *tx)
if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
return TX_CONTINUE;
- if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
- !ieee80211_is_probe_req(hdr->frame_control) &&
- !ieee80211_is_any_nullfunc(hdr->frame_control))
- /*
- * When software scanning only nullfunc frames (to notify
- * the sleep state to the AP) and probe requests (for the
- * active scan) are allowed, all other frames should not be
- * sent and we should not get here, but if we do
- * nonetheless, drop them to avoid sending them
- * off-channel. See __ieee80211_start_scan() for more.
- */
- return TX_DROP;
-
if (tx->sdata->vif.type == NL80211_IFTYPE_OCB)
return TX_CONTINUE;
@@ -807,6 +792,33 @@ ieee80211_tx_h_rate_ctrl(struct ieee80211_tx_data *tx)
return TX_CONTINUE;
}
+static ieee80211_tx_result debug_noinline
+ieee80211_tx_h_check_offchannel(struct ieee80211_tx_data *tx)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)tx->skb->data;
+ struct ieee80211_tx_info *info = IEEE80211_SKB_CB(tx->skb);
+
+ if (unlikely(info->flags & IEEE80211_TX_CTL_INJECTED))
+ return TX_CONTINUE;
+
+ if (unlikely(test_bit(SCAN_SW_SCANNING, &tx->local->scanning)) &&
+ test_bit(SDATA_STATE_OFFCHANNEL, &tx->sdata->state) &&
+ !ieee80211_is_probe_req(hdr->frame_control) &&
+ !ieee80211_is_any_nullfunc(hdr->frame_control)) {
+ /*
+ * When software scanning only nullfunc frames (to notify
+ * the sleep state to the AP) and probe requests (for the
+ * active scan) are allowed, all other frames should not be
+ * sent and we should not get here, but if we do
+ * nonetheless, drop them to avoid sending them
+ * off-channel. See __ieee80211_start_scan() for more.
+ */
+ return TX_DROP;
+ }
+
+ return TX_CONTINUE;
+}
+
static __le16 ieee80211_tx_next_seq(struct sta_info *sta, int tid)
{
u16 *seq = &sta->tid_seq[tid];
@@ -1873,6 +1885,7 @@ static int invoke_tx_handlers_late(struct ieee80211_tx_data *tx)
goto txh_done;
}
+ CALL_TXH(ieee80211_tx_h_check_offchannel);
CALL_TXH(ieee80211_tx_h_michael_mic_add);
CALL_TXH(ieee80211_tx_h_sequence);
CALL_TXH(ieee80211_tx_h_fragment);
@@ -4745,10 +4758,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
queue = ieee80211_select_queue(sdata, sta, skb);
skb_set_queue_mapping(skb, queue);
- if (unlikely(test_bit(SCAN_SW_SCANNING, &local->scanning)) &&
- test_bit(SDATA_STATE_OFFCHANNEL, &sdata->state))
- goto out_free;
-
skb = skb_share_check(skb, GFP_ATOMIC);
if (unlikely(!skb))
return;
@@ -4825,11 +4834,6 @@ static void ieee80211_8023_xmit(struct ieee80211_sub_if_data *sdata,
ieee80211_tpt_led_trig_tx(local, len);
ieee80211_tx_8023(sdata, skb, sta, false);
-
- return;
-
-out_free:
- kfree_skb(skb);
}
static bool ieee80211_check_mcast_offload(struct ieee80211_sub_if_data *sdata,
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e89a69ab927c..05c38564e8fd 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -3706,6 +3706,24 @@ static void mptcp_copy_ip_options(struct sock *newsk, const struct sock *sk)
rcu_read_unlock();
}
+static void mptcp_sk_clone_destroy(struct sock *nsk)
+{
+ struct mptcp_sock *msk = mptcp_sk(nsk);
+
+ mptcp_release_sched(msk);
+ mptcp_set_state(nsk, TCP_CLOSE);
+ /* inet_csk_prepare_forced_close() clears TCP sock_ops state via
+ * tcp_sk(), but nsk is an MPTCP master socket; keep the inet-level
+ * destroy preparation here.
+ */
+ bh_unlock_sock(nsk);
+ sock_put(nsk);
+ sock_set_flag(nsk, SOCK_DEAD);
+ tcp_orphan_count_inc();
+ inet_sk(nsk)->inet_num = 0;
+ inet_csk_destroy_sock(nsk);
+}
+
struct sock *mptcp_sk_clone_init(const struct sock *sk,
const struct mptcp_options_received *mp_opt,
struct sock *ssk,
@@ -3765,11 +3783,6 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
list_add(&subflow->node, &msk->conn_list);
sock_hold(ssk);
- /* new mpc subflow takes ownership of the newly
- * created mptcp socket
- */
- mptcp_token_accept(subflow_req, msk);
-
/* set msk addresses early to ensure mptcp_pm_get_local_id()
* uses the correct data
*/
@@ -3778,6 +3791,14 @@ struct sock *mptcp_sk_clone_init(const struct sock *sk,
mptcp_rcv_space_init(msk, ssk);
msk->rcvq_space.time = mptcp_stamp();
+ if (!mptcp_token_accept(subflow_req, msk)) {
+ list_del_init(&subflow->node);
+ WRITE_ONCE(msk->first, NULL);
+ sock_put(ssk);
+ mptcp_sk_clone_destroy(nsk);
+ return NULL;
+ }
+
if (mp_opt->suboptions & OPTION_MPTCP_MPC_ACK)
__mptcp_subflow_fully_established(msk, subflow, mp_opt);
bh_unlock_sock(nsk);
@@ -4212,9 +4233,16 @@ static int mptcp_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
* subflow_finish_connect()
*/
if (unlikely(err)) {
- /* avoid leaving a dangling token in an unconnected socket */
mptcp_token_destroy(msk);
mptcp_set_state(sk, TCP_CLOSE);
+
+ spin_lock_bh(&msk->fallback_lock);
+ msk->allow_subflows = true;
+ msk->allow_infinite_fallback = true;
+ clear_bit(MPTCP_FALLBACK_DONE, &msk->flags);
+ spin_unlock_bh(&msk->fallback_lock);
+
+ mptcp_subflow_ctx_reset(mptcp_subflow_ctx(ssk));
return err;
}
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 0384d6a023f9..4ed598febbf0 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -583,14 +583,16 @@ struct mptcp_subflow_context {
closing : 1, /* must not pass rx data to msk anymore */
valid_csum_seen : 1, /* at least one csum validated */
is_mptfo : 1, /* subflow is doing TFO */
- close_event_done : 1, /* has done the post-closed part */
mpc_drop : 1, /* the MPC option has been dropped in a rtx */
resetting : 1, /* subflow is resetting */
- __unused : 8;
+ __unused : 9;
bool data_avail;
bool scheduled;
bool pm_listener; /* a listener managed by the kernel PM? */
bool fully_established; /* path validated */
+ bool close_event_done; /* has done the post-closed part,
+ * protected by msk socket lock
+ */
u32 lent_mem_frag;
u32 remote_nonce;
u64 thmac;
@@ -1081,9 +1083,11 @@ static inline void mptcp_token_init_request(struct request_sock *req)
}
int mptcp_token_new_request(struct request_sock *req);
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req);
void mptcp_token_destroy_request(struct request_sock *req);
int mptcp_token_new_connect(struct sock *ssk);
-void mptcp_token_accept(struct mptcp_subflow_request_sock *r,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *r,
struct mptcp_sock *msk);
bool mptcp_token_exists(u32 token);
struct mptcp_sock *mptcp_token_get_sock(struct net *net, u32 token);
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 3139a9ab2853..2c72cb50fbf3 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -78,6 +78,8 @@ void mptcp_subflow_reqsk_clone(struct request_sock *req,
}
new_subflow_req->msk = msk;
+
+ mptcp_token_move_request(req, new_req);
}
static void subflow_generate_hmac(u64 key1, u64 key2, u32 nonce1, u32 nonce2,
@@ -922,8 +924,10 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
if (ctx->mp_capable) {
ctx->conn = mptcp_sk_clone_init(listener->conn, &mp_opt, child, req);
- if (!ctx->conn)
+ if (!ctx->conn) {
+ fallback = true;
goto fallback;
+ }
ctx->subflow_id = 1;
owner = mptcp_sk(ctx->conn);
diff --git a/net/mptcp/token.c b/net/mptcp/token.c
index f1a50f367add..e0f2823f92ef 100644
--- a/net/mptcp/token.c
+++ b/net/mptcp/token.c
@@ -180,6 +180,43 @@ int mptcp_token_new_connect(struct sock *ssk)
return 0;
}
+/**
+ * mptcp_token_move_request - move request token ownership to a clone
+ * @req: original request socket
+ * @new_req: cloned request socket
+ *
+ * Move the token hash entry from the original request to its clone.
+ */
+void mptcp_token_move_request(struct request_sock *req,
+ struct request_sock *new_req)
+{
+ struct mptcp_subflow_request_sock *subflow_req = mptcp_subflow_rsk(req);
+ struct mptcp_subflow_request_sock *new_subflow_req;
+ struct mptcp_subflow_request_sock *pos;
+ struct token_bucket *bucket;
+
+ new_subflow_req = mptcp_subflow_rsk(new_req);
+
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ return;
+ }
+
+ bucket = token_bucket(subflow_req->token);
+ spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node)) {
+ mptcp_token_init_request(new_req);
+ } else {
+ pos = __token_lookup_req(bucket, subflow_req->token);
+ if (pos == subflow_req)
+ hlist_nulls_replace_init_rcu(&subflow_req->token_node,
+ &new_subflow_req->token_node);
+ else
+ mptcp_token_init_request(new_req);
+ }
+ spin_unlock_bh(&bucket->lock);
+}
+
/**
* mptcp_token_accept - replace a req sk with full sock in token hash
* @req: the request socket to be removed
@@ -187,24 +224,36 @@ int mptcp_token_new_connect(struct sock *ssk)
*
* Called when a SYN packet creates a new logical connection, i.e.
* is not a join request.
+ *
+ * Return: true on success.
*/
-void mptcp_token_accept(struct mptcp_subflow_request_sock *req,
+bool mptcp_token_accept(struct mptcp_subflow_request_sock *req,
struct mptcp_sock *msk)
{
struct mptcp_subflow_request_sock *pos;
struct sock *sk = (struct sock *)msk;
struct token_bucket *bucket;
+ bool ret = false;
- sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
bucket = token_bucket(req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&req->token_node))
+ goto unlock;
- /* pedantic lookup check for the moved token */
pos = __token_lookup_req(bucket, req->token);
- if (!WARN_ON_ONCE(pos != req))
- hlist_nulls_del_init_rcu(&req->token_node);
+ if (pos != req)
+ goto unlock;
+
+ hlist_nulls_del_init_rcu(&req->token_node);
__sk_nulls_add_node_rcu((struct sock *)msk, &bucket->msk_chain);
+ ret = true;
+
+unlock:
spin_unlock_bh(&bucket->lock);
+ if (ret)
+ sock_prot_inuse_add(sock_net(sk), sk->sk_prot, 1);
+
+ return ret;
}
bool mptcp_token_exists(u32 token)
@@ -355,16 +404,21 @@ void mptcp_token_destroy_request(struct request_sock *req)
struct mptcp_subflow_request_sock *pos;
struct token_bucket *bucket;
- if (hlist_nulls_unhashed(&subflow_req->token_node))
+ if (hlist_nulls_unhashed_lockless(&subflow_req->token_node))
return;
bucket = token_bucket(subflow_req->token);
spin_lock_bh(&bucket->lock);
+ if (hlist_nulls_unhashed(&subflow_req->token_node))
+ goto unlock;
+
pos = __token_lookup_req(bucket, subflow_req->token);
- if (!WARN_ON_ONCE(pos != subflow_req)) {
+ if (pos == subflow_req) {
hlist_nulls_del_init_rcu(&pos->token_node);
bucket->chain_len--;
}
+
+unlock:
spin_unlock_bh(&bucket->lock);
}
diff --git a/net/mptcp/token_test.c b/net/mptcp/token_test.c
index 4fc39fa2e262..be9acce8a567 100644
--- a/net/mptcp/token_test.c
+++ b/net/mptcp/token_test.c
@@ -99,7 +99,7 @@ static void mptcp_token_test_accept(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
KUNIT_EXPECT_PTR_EQ(test, msk, mptcp_token_get_sock(&init_net, msk->token));
/* this is now a no-op */
@@ -122,7 +122,7 @@ static void mptcp_token_test_destroyed(struct kunit *test)
KUNIT_ASSERT_EQ(test, 0,
mptcp_token_new_request((struct request_sock *)req));
msk->token = req->token;
- mptcp_token_accept(req, msk);
+ KUNIT_EXPECT_TRUE(test, mptcp_token_accept(req, msk));
/* simulate race on removal */
refcount_set(&sk->sk_refcnt, 0);
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index 4e5d7c701436..e8477f792c95 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1098,13 +1098,13 @@ static int ctnetlink_filter_match_tuple(struct nf_conntrack_tuple *filter_tuple,
break;
case NFPROTO_IPV6:
if ((flags & CTA_FILTER_FLAG(CTA_IP_SRC)) &&
- !ipv6_addr_cmp(&filter_tuple->src.u3.in6,
- &ct_tuple->src.u3.in6))
+ !ipv6_addr_equal(&filter_tuple->src.u3.in6,
+ &ct_tuple->src.u3.in6))
return 0;
if ((flags & CTA_FILTER_FLAG(CTA_IP_DST)) &&
- !ipv6_addr_cmp(&filter_tuple->dst.u3.in6,
- &ct_tuple->dst.u3.in6))
+ !ipv6_addr_equal(&filter_tuple->dst.u3.in6,
+ &ct_tuple->dst.u3.in6))
return 0;
break;
}
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 52ae3d875e97..515a3e8dc38b 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -2647,6 +2647,12 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb,
}
offset = offset_in_page(data);
+ if (unlikely(to_write > MAX_SKB_FRAGS * PAGE_SIZE - offset)) {
+ net_warn_ratelimited("Packet exceed the number of skb frags(%u)\n",
+ (unsigned int)MAX_SKB_FRAGS);
+ return -EFAULT;
+ }
+
len_max = PAGE_SIZE - offset;
len = ((to_write > len_max) ? len_max : to_write);
@@ -2658,12 +2664,6 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb,
while (likely(to_write)) {
nr_frags = skb_shinfo(skb)->nr_frags;
- if (unlikely(nr_frags >= MAX_SKB_FRAGS)) {
- pr_err("Packet exceed the number of skb frags(%u)\n",
- (unsigned int)MAX_SKB_FRAGS);
- return -EFAULT;
- }
-
page = pgv_to_page(data);
data += len;
flush_dcache_page(page);
diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index 6e3110a04ae6..5c717120eab5 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -1271,6 +1271,7 @@ void rds_ib_conn_free(void *arg)
{
struct rds_ib_connection *ic = arg;
spinlock_t *lock_ptr;
+ unsigned long flags;
rdsdebug("ic %p\n", ic);
@@ -1281,9 +1282,10 @@ void rds_ib_conn_free(void *arg)
*/
lock_ptr = ic->rds_ibdev ? &ic->rds_ibdev->spinlock : &ib_nodev_conns_lock;
- spin_lock_irq(lock_ptr);
+ /* May be called with IRQs already disabled (see __rds_conn_create()) */
+ spin_lock_irqsave(lock_ptr, flags);
list_del(&ic->ib_node);
- spin_unlock_irq(lock_ptr);
+ spin_unlock_irqrestore(lock_ptr, flags);
rds_ib_recv_free_caches(ic);
diff --git a/net/rds/ib_recv.c b/net/rds/ib_recv.c
index bd6cb3ffaa57..aa50b85b0f99 100644
--- a/net/rds/ib_recv.c
+++ b/net/rds/ib_recv.c
@@ -949,6 +949,12 @@ static void rds_ib_process_recv(struct rds_connection *conn,
}
}
+ if (data_len < min_t(u32, ic->i_recv_data_rem, RDS_FRAG_SIZE)) {
+ rds_ib_conn_error(conn,
+ "fragment shorter than header-declared length; forcing reconnect\n");
+ goto done;
+ }
+
list_add_tail(&recv->r_frag->f_item, &ibinc->ii_frags);
recv->r_frag = NULL;
@@ -1004,7 +1010,12 @@ void rds_ib_recv_cqe_handler(struct rds_ib_connection *ic,
* event is processed.
*/
if (wc->status == IB_WC_SUCCESS) {
- rds_ib_process_recv(conn, recv, wc->byte_len, state);
+ if (wc->opcode != IB_WC_RECV)
+ rds_ib_conn_error(conn,
+ "recv completion had unexpected opcode %u, disconnecting and reconnecting\n",
+ wc->opcode);
+ else
+ rds_ib_process_recv(conn, recv, wc->byte_len, state);
} else {
/* We expect errors as the qp is drained during shutdown */
if (rds_conn_up(conn) || rds_conn_connecting(conn))
diff --git a/net/rds/recv.c b/net/rds/recv.c
index 6204e577a90a..93ee4546ae36 100644
--- a/net/rds/recv.c
+++ b/net/rds/recv.c
@@ -379,7 +379,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
rds_stats_inc(s_recv_ping);
rds_send_pong(cp, inc->i_hdr.h_sport);
/* if this is a handshake ping, start multipath if necessary */
- if (RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
+ if (conn->c_trans->t_mp_capable &&
+ RDS_HS_PROBE(be16_to_cpu(inc->i_hdr.h_sport),
be16_to_cpu(inc->i_hdr.h_dport))) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
rds_start_mprds(cp->cp_conn);
@@ -387,7 +388,8 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
goto out;
}
- if (be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
+ if (conn->c_trans->t_mp_capable &&
+ be16_to_cpu(inc->i_hdr.h_dport) == RDS_FLAG_PROBE_PORT &&
inc->i_hdr.h_sport == 0) {
rds_recv_hs_exthdrs(&inc->i_hdr, cp->cp_conn);
/* if this is a handshake pong, start multipath if necessary */
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 321898f22d80..c616f1609d0c 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -75,7 +75,8 @@ static int sctp_wait_for_sndbuf(struct sctp_association *asoc,
struct sctp_transport *transport,
long *timeo_p, size_t msg_len);
static int sctp_wait_for_packet(struct sock *sk, int *err, long *timeo_p);
-static int sctp_wait_for_connect(struct sctp_association *, long *timeo_p);
+static int sctp_wait_for_connect(struct sctp_association *,
+ struct sctp_transport *, long *timeo_p);
static int sctp_wait_for_accept(struct sock *sk, long timeo);
static void sctp_wait_for_close(struct sock *sk, long timeo);
static void sctp_destruct_sock(struct sock *sk);
@@ -1245,7 +1246,7 @@ static int __sctp_connect(struct sock *sk, struct sockaddr *kaddrs,
*assoc_id = asoc->assoc_id;
timeo = sock_sndtimeo(sk, flags & O_NONBLOCK);
- return sctp_wait_for_connect(asoc, &timeo);
+ return sctp_wait_for_connect(asoc, NULL, &timeo);
out_free:
pr_debug("%s: took out_free path with asoc:%p kaddrs:%p err:%d\n",
@@ -1847,7 +1848,7 @@ static int sctp_sendmsg_to_asoc(struct sctp_association *asoc,
if (asoc->ep->intl_enable) {
timeo = sock_sndtimeo(sk, 0);
- err = sctp_wait_for_connect(asoc, &timeo);
+ err = sctp_wait_for_connect(asoc, transport, &timeo);
if (err || sk->sk_shutdown & RCV_SHUTDOWN) {
err = -ESRCH;
goto err;
@@ -1890,7 +1891,7 @@ static int sctp_sendmsg_to_asoc(struct sctp_association *asoc,
if (unlikely(wait_connect)) {
timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
- sctp_wait_for_connect(asoc, &timeo);
+ sctp_wait_for_connect(asoc, NULL, &timeo);
}
err = msg_len;
@@ -9390,7 +9391,9 @@ static bool sctp_writeable(const struct sock *sk)
/* Wait for an association to go into ESTABLISHED state. If timeout is 0,
* returns immediately with EINPROGRESS.
*/
-static int sctp_wait_for_connect(struct sctp_association *asoc, long *timeo_p)
+static int sctp_wait_for_connect(struct sctp_association *asoc,
+ struct sctp_transport *transport,
+ long *timeo_p)
{
struct sock *sk = asoc->base.sk;
int err = 0;
@@ -9399,12 +9402,16 @@ static int sctp_wait_for_connect(struct sctp_association *asoc, long *timeo_p)
pr_debug("%s: asoc:%p, timeo:%ld\n", __func__, asoc, *timeo_p);
- /* Increment the association's refcnt. */
+ /* Hold references while the socket lock is dropped. */
+ if (transport)
+ sctp_transport_hold(transport);
sctp_association_hold(asoc);
for (;;) {
prepare_to_wait_exclusive(&asoc->wait, &wait,
TASK_INTERRUPTIBLE);
+ if (transport && transport->dead)
+ goto do_transport_dead;
if (!*timeo_p)
goto do_nonblock;
if (sk->sk_shutdown & RCV_SHUTDOWN)
@@ -9433,7 +9440,8 @@ static int sctp_wait_for_connect(struct sctp_association *asoc, long *timeo_p)
out:
finish_wait(&asoc->wait, &wait);
- /* Release the association's refcnt. */
+ if (transport)
+ sctp_transport_put(transport);
sctp_association_put(asoc);
return err;
@@ -9452,6 +9460,10 @@ static int sctp_wait_for_connect(struct sctp_association *asoc, long *timeo_p)
do_nonblock:
err = -EINPROGRESS;
goto out;
+
+do_transport_dead:
+ err = -EAGAIN;
+ goto out;
}
static int sctp_wait_for_accept(struct sock *sk, long timeo)
diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c
index 3144b4b1fe29..495fb1354728 100644
--- a/net/smc/smc_tx.c
+++ b/net/smc/smc_tx.c
@@ -249,6 +249,13 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len)
rc = memcpy_from_msg(sndbuf_base + chunk_off,
msg, chunk_len);
if (rc) {
+ if (chunk) {
+ /* send_done counts the first chunk:
+ * queue it and retry the rest
+ */
+ copylen -= chunk_len;
+ break;
+ }
smc_sndbuf_sync_sg_for_device(conn);
if (send_done)
return send_done;
diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
index 1285f04cdac1..7e5713367291 100644
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -984,12 +984,16 @@ void rpcrdma_reset_cwnd(struct rpcrdma_xprt *r_xprt)
* @rqst: controlling RPC request
* @srcp: points to RPC message payload in receive buffer
* @copy_len: remaining length of receive buffer content
- * @pad: Write chunk pad bytes needed (zero for pure inline)
+ * @writelist: bytes conveyed by the Write chunk (zero for pure inline)
*
* The upper layer has set the maximum number of bytes it can
* receive in each component of rq_rcv_buf. These values are set in
* the head.iov_len, page_len, tail.iov_len, and buflen fields.
*
+ * For READ replies with a Write chunk, the payload is already in
+ * the page list. An overestimated head length and buflen can be
+ * reduced without moving reply data.
+ *
* Unlike the TCP equivalent (xdr_partial_copy_from_skb), in
* many cases this function simply updates iov_base pointers in
* rq_rcv_buf to point directly to the received reply data, to
@@ -998,13 +1002,16 @@ void rpcrdma_reset_cwnd(struct rpcrdma_xprt *r_xprt)
* Returns the count of bytes which had to be memcopied.
*/
static unsigned long
-rpcrdma_inline_fixup(struct rpc_rqst *rqst, char *srcp, int copy_len, int pad)
+rpcrdma_inline_fixup(struct rpc_rqst *rqst, char *srcp, int copy_len,
+ u32 writelist)
{
unsigned long fixup_copy_count;
int i, npages, curlen;
char *destp;
struct page **ppages;
int page_base;
+ int pad = writelist & 3;
+ unsigned int delta;
/* The head iovec is redirected to the RPC reply message
* in the receive buffer, to avoid a memcopy.
@@ -1016,8 +1023,20 @@ rpcrdma_inline_fixup(struct rpc_rqst *rqst, char *srcp, int copy_len, int pad)
* head.iov_len bytes are copied into the page list.
*/
curlen = rqst->rq_rcv_buf.head[0].iov_len;
- if (curlen > copy_len)
+ if (curlen > copy_len) {
+ /* These READ replies have no inline fields after the payload.
+ * Trim the head without moving the directly-placed data.
+ */
+ if (writelist) {
+ delta = curlen - copy_len;
+
+ rqst->rq_rcv_buf.head[0].iov_len = copy_len;
+ rqst->rq_private_buf.head[0].iov_len = copy_len;
+ rqst->rq_rcv_buf.buflen -= delta;
+ rqst->rq_private_buf.buflen -= delta;
+ }
curlen = copy_len;
+ }
srcp += curlen;
copy_len -= curlen;
@@ -1245,7 +1264,7 @@ rpcrdma_decode_msg(struct rpcrdma_xprt *r_xprt, struct rpcrdma_rep *rep,
base = (char *)xdr_inline_decode(xdr, 0);
rpclen = xdr_stream_remaining(xdr);
r_xprt->rx_stats.fixup_copy_count +=
- rpcrdma_inline_fixup(rqst, base, rpclen, writelist & 3);
+ rpcrdma_inline_fixup(rqst, base, rpclen, writelist);
r_xprt->rx_stats.total_rdma_reply += writelist;
return rpclen + xdr_align_size(writelist);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 927269598ac2..f949601b2144 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -471,7 +471,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
newxprt->sc_max_requests = svcrdma_max_requests;
newxprt->sc_max_bc_requests = svcrdma_max_bc_requests;
newxprt->sc_recv_batch = RPCRDMA_MAX_RECV_BATCH;
- newxprt->sc_fc_credits = cpu_to_be32(newxprt->sc_max_requests);
/* Qualify the transport's resource defaults with the
* capabilities of this particular device.
@@ -492,6 +491,8 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
newxprt->sc_max_bc_requests = 2;
}
+ newxprt->sc_fc_credits = cpu_to_be32(newxprt->sc_max_requests);
+
/* Estimate the needed number of rdma_rw contexts. The maximum
* Read and Write chunks have one segment each. Each request
* can involve one Read chunk and either a Write chunk or Reply
diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c
index be335eed329d..54957295a631 100644
--- a/net/sunrpc/xprtrdma/verbs.c
+++ b/net/sunrpc/xprtrdma/verbs.c
@@ -114,9 +114,9 @@ static void rpcrdma_xprt_drain(struct rpcrdma_xprt *r_xprt)
rpcrdma_ep_put(ep);
}
-/* Ensure xprt_force_disconnect() is invoked exactly once when a
- * connection is closed or lost. (The important thing is it needs
- * to be invoked "at least" once).
+/* Ensure xprt_force_disconnect() is invoked at most once for an
+ * unexpected endpoint failure. Local endpoint teardown closes this
+ * gate before it can generate CM events and flushed completions.
*/
void rpcrdma_force_disconnect(struct rpcrdma_ep *ep)
{
@@ -587,6 +587,9 @@ void rpcrdma_xprt_disconnect(struct rpcrdma_xprt *r_xprt)
if (!ep)
return;
+ /* Prevent teardown-generated events from requesting another close. */
+ atomic_xchg(&ep->re_force_disconnect, 1);
+
id = ep->re_id;
rc = rdma_disconnect(id);
trace_xprtrdma_disconnect(r_xprt, rc);
diff --git a/net/tipc/group.c b/net/tipc/group.c
index 74f6d3dac078..0c6a762000b0 100644
--- a/net/tipc/group.c
+++ b/net/tipc/group.c
@@ -74,7 +74,7 @@ struct tipc_member {
u16 window;
u16 bc_rcv_nxt;
u16 bc_syncpt;
- u16 bc_acked;
+ bool bc_pending;
};
struct tipc_group {
@@ -316,7 +316,6 @@ static struct tipc_member *tipc_group_create_member(struct tipc_group *grp,
m->node = node;
m->port = port;
m->instance = instance;
- m->bc_acked = grp->bc_snd_nxt - 1;
ret = tipc_group_add_to_tree(grp, m);
if (ret < 0) {
kfree(m);
@@ -340,9 +339,12 @@ static void tipc_group_delete_member(struct tipc_group *grp,
rb_erase(&m->tree_node, &grp->members);
grp->member_cnt--;
- /* Check if we were waiting for replicast ack from this member */
- if (grp->bc_ackers && less(m->bc_acked, grp->bc_snd_nxt - 1))
- grp->bc_ackers--;
+ /* If this member owed an ack for the current replicast and it
+ * was the last one, restore the open state so blocked broadcast
+ * senders can proceed.
+ */
+ if (grp->bc_ackers && m->bc_pending && !--grp->bc_ackers)
+ *grp->open = true;
list_del_init(&m->list);
list_del_init(&m->small_win);
@@ -394,7 +396,6 @@ void tipc_group_update_member(struct tipc_member *m, int len)
void tipc_group_update_bc_members(struct tipc_group *grp, int len, bool ack)
{
- u16 prev = grp->bc_snd_nxt - 1;
struct tipc_member *m;
struct rb_node *n;
u16 ackers = 0;
@@ -403,8 +404,8 @@ void tipc_group_update_bc_members(struct tipc_group *grp, int len, bool ack)
m = container_of(n, struct tipc_member, tree_node);
if (tipc_group_is_receiver(m)) {
tipc_group_update_member(m, len);
- m->bc_acked = prev;
ackers++;
+ m->bc_pending = ack;
}
}
@@ -800,9 +801,9 @@ void tipc_group_proto_rcv(struct tipc_group *grp, bool *usr_wakeup,
if (!m || !grp->bc_ackers)
return;
acked = msg_grp_bc_acked(hdr);
- if (acked != grp->bc_snd_nxt || m->bc_acked == acked)
+ if (acked != grp->bc_snd_nxt || !m->bc_pending)
return;
- m->bc_acked = acked;
+ m->bc_pending = false;
if (--grp->bc_ackers)
return;
list_del_init(&m->small_win);
diff --git a/net/tipc/socket.c b/net/tipc/socket.c
index d5d70eb230b5..65cc0c4964b3 100644
--- a/net/tipc/socket.c
+++ b/net/tipc/socket.c
@@ -2356,8 +2356,18 @@ static void tipc_sk_filter_rcv(struct sock *sk, struct sk_buff *skb,
if (unlikely(!msg_isdata(hdr)))
tipc_sk_proto_rcv(sk, &inputq, xmitq);
- if (unlikely(grp))
+ if (unlikely(grp)) {
+ bool was_open = tsk->group_is_open;
+
tipc_group_filter_msg(grp, &inputq, xmitq);
+ /* A departing member may have held the last outstanding
+ * replicast ack, which can restore the open state inside
+ * tipc_group_filter_msg(); wake up senders blocked in
+ * poll()/write() if the group just became writable.
+ */
+ if (!was_open && tsk->group_is_open)
+ sk->sk_write_space(sk);
+ }
if (unlikely(!grp) && mtyp == TIPC_MCAST_MSG)
tipc_mcast_filter_msg(net, &tsk->mc_method.deferredq, &inputq);
diff --git a/net/unix/sysctl_net_unix.c b/net/unix/sysctl_net_unix.c
index 47660d5726bb..fa1fbaec3aaf 100644
--- a/net/unix/sysctl_net_unix.c
+++ b/net/unix/sysctl_net_unix.c
@@ -19,7 +19,9 @@ static const struct ctl_table unix_table[] = {
.data = &init_net.unx.sysctl_max_dgram_qlen,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = proc_dointvec
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
};
diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c
index 9d2d94f1fb75..e836eca469b0 100644
--- a/net/xdp/xsk_buff_pool.c
+++ b/net/xdp/xsk_buff_pool.c
@@ -166,6 +166,7 @@ int xp_assign_dev(struct xsk_buff_pool *pool,
{
u32 needed = netdev->mtu + ETH_PAD_LEN;
u32 segs = netdev->xdp_zc_max_segs;
+ u8 umem_flags = pool->umem->flags;
bool mbuf = flags & XDP_USE_SG;
bool force_zc, force_copy;
struct netdev_bpf bpf;
@@ -264,6 +265,7 @@ int xp_assign_dev(struct xsk_buff_pool *pool,
if (!force_zc)
err = 0; /* fallback to copy mode */
if (err) {
+ pool->umem->flags = umem_flags;
xsk_clear_pool_at_qid(netdev, queue_id);
dev_put(netdev);
}
diff --git a/rust/bindgen_parameters b/rust/bindgen_parameters
index 39aae62dca66..5bab3cf06336 100644
--- a/rust/bindgen_parameters
+++ b/rust/bindgen_parameters
@@ -29,6 +29,12 @@
# enums that appear in indirect function calls should specify a cfi type
--newtype-enum fwctl_rpc_scope
--with-attribute-custom-enum=fwctl_rpc_scope='#[cfi_encoding="15fwctl_rpc_scope"]'
+--newtype-enum dma_data_direction
+--with-attribute-custom-enum=dma_data_direction='#[cfi_encoding="18dma_data_direction"]'
+--newtype-enum hrtimer_restart
+--with-attribute-custom-enum=hrtimer_restart='#[cfi_encoding="15hrtimer_restart"]'
+--newtype-enum irqreturn
+--with-attribute-custom-enum=irqreturn='#[cfi_encoding="9irqreturn"]'
--newtype-enum lru_status
--with-attribute-custom-enum=lru_status='#[cfi_encoding="10lru_status"]'
diff --git a/rust/kernel/dma.rs b/rust/kernel/dma.rs
index 2ce09f8e90c6..2ec5fe7d56bc 100644
--- a/rust/kernel/dma.rs
+++ b/rust/kernel/dma.rs
@@ -310,25 +310,25 @@ pub enum DataDirection {
///
/// This is used when the buffer can be both read from and written to by the device.
/// The cache for the corresponding memory region is both flushed and invalidated.
- Bidirectional = Self::const_cast(bindings::dma_data_direction_DMA_BIDIRECTIONAL),
+ Bidirectional = Self::const_cast(bindings::dma_data_direction::DMA_BIDIRECTIONAL),
/// The DMA mapping is for data transfer from memory to the device (write).
///
/// The CPU has prepared data in the buffer, and the device will read it.
/// The cache for the corresponding memory region is flushed before device access.
- ToDevice = Self::const_cast(bindings::dma_data_direction_DMA_TO_DEVICE),
+ ToDevice = Self::const_cast(bindings::dma_data_direction::DMA_TO_DEVICE),
/// The DMA mapping is for data transfer from the device to memory (read).
///
/// The device will write data into the buffer for the CPU to read.
/// The cache for the corresponding memory region is invalidated before CPU access.
- FromDevice = Self::const_cast(bindings::dma_data_direction_DMA_FROM_DEVICE),
+ FromDevice = Self::const_cast(bindings::dma_data_direction::DMA_FROM_DEVICE),
/// The DMA mapping is not for data transfer.
///
/// This is primarily for debugging purposes. With this direction, the DMA mapping API
/// will not perform any cache coherency operations.
- None = Self::const_cast(bindings::dma_data_direction_DMA_NONE),
+ None = Self::const_cast(bindings::dma_data_direction::DMA_NONE),
}
impl DataDirection {
@@ -340,7 +340,7 @@ const fn const_cast(val: bindings::dma_data_direction) -> u32 {
// CAST: The C standard allows compilers to choose different integer types for enums.
// To safely check the value, we cast it to a wide signed integer type (`i128`)
// which can hold any standard C integer enum type without truncation.
- let wide_val = val as i128;
+ let wide_val = val.0 as i128;
// Check if the value is outside the valid range for the target type `u32`.
// CAST: `u32::MAX` is cast to `i128` to match the type of `wide_val` for the comparison.
@@ -358,11 +358,12 @@ const fn const_cast(val: bindings::dma_data_direction) -> u32 {
impl From<DataDirection> for bindings::dma_data_direction {
/// Returns the raw representation of [`enum dma_data_direction`].
fn from(direction: DataDirection) -> Self {
- // CAST: `direction as u32` gets the underlying representation of our `#[repr(u32)]` enum.
- // The subsequent cast to `Self` (the bindgen type) assumes the C enum is compatible
- // with the enum variants of `DataDirection`, which is a valid assumption given our
- // compile-time checks.
- direction as u32 as Self
+ match direction {
+ DataDirection::Bidirectional => Self::DMA_BIDIRECTIONAL,
+ DataDirection::ToDevice => Self::DMA_TO_DEVICE,
+ DataDirection::FromDevice => Self::DMA_FROM_DEVICE,
+ DataDirection::None => Self::DMA_NONE,
+ }
}
}
diff --git a/rust/kernel/irq/request.rs b/rust/kernel/irq/request.rs
index c1c6525a676a..1afa9fcaad1f 100644
--- a/rust/kernel/irq/request.rs
+++ b/rust/kernel/irq/request.rs
@@ -25,10 +25,10 @@
#[repr(u32)]
pub enum IrqReturn {
/// The interrupt was not from this device or was not handled.
- None = bindings::irqreturn_IRQ_NONE,
+ None = bindings::irqreturn::IRQ_NONE.0,
/// The interrupt was handled by this device.
- Handled = bindings::irqreturn_IRQ_HANDLED,
+ Handled = bindings::irqreturn::IRQ_HANDLED.0,
}
/// Callbacks for an IRQ handler.
@@ -240,25 +240,28 @@ fn drop(self: Pin<&mut Self>) {
/// # Safety
///
/// This function should be only used as the callback in `request_irq`.
-unsafe extern "C" fn handle_irq_callback<T: Handler>(_irq: i32, ptr: *mut c_void) -> c_uint {
+unsafe extern "C" fn handle_irq_callback<T: Handler>(
+ _irq: i32,
+ ptr: *mut c_void,
+) -> bindings::irqreturn_t {
let ptr = ptr.cast_const().cast::<Registration<'_, T>>();
// SAFETY: `ptr` is a pointer to `Registration<'_, T>` set in `Registration::new()`.
let registration = unsafe { &*ptr };
- T::handle(®istration.handler) as c_uint
+ bindings::irqreturn_t(T::handle(®istration.handler) as c_uint)
}
/// The value that can be returned from [`ThreadedHandler::handle`].
#[repr(u32)]
pub enum ThreadedIrqReturn {
/// The interrupt was not from this device or was not handled.
- None = bindings::irqreturn_IRQ_NONE,
+ None = bindings::irqreturn::IRQ_NONE.0,
/// The interrupt was handled by this device.
- Handled = bindings::irqreturn_IRQ_HANDLED,
+ Handled = bindings::irqreturn::IRQ_HANDLED.0,
/// The handler wants the handler thread to wake up.
- WakeThread = bindings::irqreturn_IRQ_WAKE_THREAD,
+ WakeThread = bindings::irqreturn::IRQ_WAKE_THREAD.0,
}
/// Callbacks for a threaded IRQ handler.
@@ -461,23 +464,26 @@ fn drop(self: Pin<&mut Self>) {
unsafe extern "C" fn handle_threaded_irq_callback<T: ThreadedHandler>(
_irq: i32,
ptr: *mut c_void,
-) -> c_uint {
+) -> bindings::irqreturn_t {
let ptr = ptr.cast_const().cast::<ThreadedRegistration<'_, T>>();
// SAFETY: `ptr` is a pointer to `ThreadedRegistration<'_, T>` set in
// `ThreadedRegistration::new()`.
let registration = unsafe { &*ptr };
- T::handle(®istration.handler) as c_uint
+ bindings::irqreturn_t(T::handle(®istration.handler) as c_uint)
}
/// # Safety
///
/// This function should be only used as the callback in `request_threaded_irq`.
-unsafe extern "C" fn thread_fn_callback<T: ThreadedHandler>(_irq: i32, ptr: *mut c_void) -> c_uint {
+unsafe extern "C" fn thread_fn_callback<T: ThreadedHandler>(
+ _irq: i32,
+ ptr: *mut c_void,
+) -> bindings::irqreturn_t {
let ptr = ptr.cast_const().cast::<ThreadedRegistration<'_, T>>();
// SAFETY: `ptr` is a pointer to `ThreadedRegistration<'_, T>` set in
// `ThreadedRegistration::new()`.
let registration = unsafe { &*ptr };
- T::handle_threaded(®istration.handler) as c_uint
+ bindings::irqreturn_t(T::handle_threaded(®istration.handler) as c_uint)
}
diff --git a/rust/kernel/time/hrtimer.rs b/rust/kernel/time/hrtimer.rs
index 2d7f1131a813..07344ed5fbd5 100644
--- a/rust/kernel/time/hrtimer.rs
+++ b/rust/kernel/time/hrtimer.rs
@@ -834,14 +834,14 @@ unsafe fn start(this: *const Self, expires: <Self::TimerMode as HrTimerMode>::Ex
#[repr(u32)]
pub enum HrTimerRestart {
/// Timer should not be restarted.
- NoRestart = bindings::hrtimer_restart_HRTIMER_NORESTART,
+ NoRestart = bindings::hrtimer_restart::HRTIMER_NORESTART.0,
/// Timer should be restarted.
- Restart = bindings::hrtimer_restart_HRTIMER_RESTART,
+ Restart = bindings::hrtimer_restart::HRTIMER_RESTART.0,
}
impl HrTimerRestart {
fn into_c(self) -> bindings::hrtimer_restart {
- self as bindings::hrtimer_restart
+ bindings::hrtimer_restart(self as c_uint)
}
}
diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo
index 1442043da139..57a3aba6ffc2 100644
--- a/scripts/Makefile.autofdo
+++ b/scripts/Makefile.autofdo
@@ -22,7 +22,7 @@ ifdef CONFIG_LTO_CLANG_THIN
KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE)
endif
KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto
- KBUILD_LDFLAGS += -plugin-opt=-split-machine-functions
+ KBUILD_LDFLAGS += $(call ld-option,-plugin-opt=-function-splitting=all,-plugin-opt=-split-machine-functions)
endif
export CFLAGS_AUTOFDO_CLANG RUSTFLAGS_AUTOFDO_CLANG
diff --git a/security/apparmor/af_unix.c b/security/apparmor/af_unix.c
index b908e744818c..58c638830afc 100644
--- a/security/apparmor/af_unix.c
+++ b/security/apparmor/af_unix.c
@@ -682,7 +682,7 @@ static void update_sk_ctx(struct sock *sk, struct aa_label *label,
if (old == plabel) {
rcu_assign_pointer(ctx->peer_lastupdate,
aa_get_label(plabel));
- } else if (aa_label_is_subset(plabel, old)) {
+ } else if (!old || aa_label_is_subset(plabel, old)) {
rcu_assign_pointer(ctx->peer_lastupdate,
aa_get_label(plabel));
rcu_assign_pointer(ctx->peer, aa_get_label(plabel));
@@ -700,13 +700,17 @@ static void update_peer_ctx(struct sock *sk, struct aa_sk_ctx *ctx,
spin_lock(&unix_sk(sk)->lock);
old = rcu_dereference_protected(ctx->peer,
lockdep_is_held(&unix_sk(sk)->lock));
- l = aa_label_merge(old, label, GFP_ATOMIC);
- if (l) {
- if (l != old) {
- rcu_assign_pointer(ctx->peer, l);
- aa_put_label(old);
- } else
- aa_put_label(l);
+ if (!old) {
+ rcu_assign_pointer(ctx->peer, aa_get_label(label));
+ } else {
+ l = aa_label_merge(old, label, GFP_ATOMIC);
+ if (l) {
+ if (l != old) {
+ rcu_assign_pointer(ctx->peer, l);
+ aa_put_label(old);
+ } else
+ aa_put_label(l);
+ }
}
spin_unlock(&unix_sk(sk)->lock);
}
diff --git a/security/apparmor/apparmorfs.c b/security/apparmor/apparmorfs.c
index 5b42140e12e8..9df06fdd894f 100644
--- a/security/apparmor/apparmorfs.c
+++ b/security/apparmor/apparmorfs.c
@@ -919,7 +919,7 @@ static void profile_query_cb(const struct aa_profile *profile,
static ssize_t query_data(char *buf, size_t buf_len,
char *query, size_t query_len)
{
- char *out;
+ char *out, *keydup;
const char *key;
struct label_it i;
struct aa_label *label, *curr;
@@ -941,11 +941,18 @@ static ssize_t query_data(char *buf, size_t buf_len,
if (buf_len < sizeof(bytes) + sizeof(blocks))
return -EINVAL; /* not enough space */
+ /* buf overlaps query, keep a copy of the key for the later lookups */
+ keydup = kstrdup(key, GFP_KERNEL);
+ if (!keydup)
+ return -ENOMEM;
+
curr = begin_current_label_crit_section(&needput);
label = aa_label_parse(curr, query, GFP_KERNEL, false, false);
end_current_label_crit_section(curr, needput);
- if (IS_ERR(label))
+ if (IS_ERR(label)) {
+ kfree(keydup);
return PTR_ERR(label);
+ }
/* We are going to leave space for two numbers. The first is the total
* number of bytes we are writing after the first number. This is so
@@ -963,13 +970,14 @@ static ssize_t query_data(char *buf, size_t buf_len,
if (!profile->data)
continue;
- data = rhashtable_lookup_fast(profile->data, &key,
+ data = rhashtable_lookup_fast(profile->data, &keydup,
profile->data->p);
if (data) {
if (out + sizeof(outle32) + data->size > buf +
buf_len) {
aa_put_label(label);
+ kfree(keydup);
return -EINVAL; /* not enough space */
}
outle32 = __cpu_to_le32(data->size);
@@ -981,6 +989,7 @@ static ssize_t query_data(char *buf, size_t buf_len,
}
}
aa_put_label(label);
+ kfree(keydup);
outle32 = __cpu_to_le32(out - buf - sizeof(bytes));
memcpy(buf, &outle32, sizeof(outle32));
diff --git a/security/apparmor/file.c b/security/apparmor/file.c
index 3e74b613db32..baca370ddfc0 100644
--- a/security/apparmor/file.c
+++ b/security/apparmor/file.c
@@ -706,49 +706,50 @@ static void revalidate_tty(const struct cred *subj_cred, struct aa_label *label)
no_tty();
}
-struct cred_label {
- const struct cred *cred;
- struct aa_label *label;
-};
-
-static int match_file(const void *p, struct file *file, unsigned int fd)
+/*
+ * iterate_fd() callback, called with files->file_lock held: grab a ref
+ * on the next file so it can be revalidated without the lock held.
+ */
+static int get_next_file(const void *p, struct file *file, unsigned int fd)
{
- struct cred_label *cl = (struct cred_label *)p;
+ *(struct file **)p = get_file(file);
- if (aa_file_perm(OP_INHERIT, cl->cred, cl->label, file,
- aa_map_file_to_perms(file), IN_ATOMIC))
- return fd + 1;
- return 0;
+ return fd + 1;
}
-
/* based on selinux's flush_unauthorized_files */
void aa_inherit_files(const struct cred *cred, struct files_struct *files)
{
struct aa_label *label = aa_get_newest_cred_label(cred);
- struct cred_label cl = {
- .cred = cred,
- .label = label,
- };
- struct file *devnull = NULL;
- unsigned int n;
+ struct file *devnull = NULL, *file = NULL;
+ bool devnull_tried = false;
+ unsigned int n = 0;
revalidate_tty(cred, label);
- /* Revalidate access to inherited open files. */
- n = iterate_fd(files, 0, match_file, &cl);
- if (!n) /* none found? */
- goto out;
-
- devnull = dentry_open(&aa_null, O_RDWR, cred);
- if (IS_ERR(devnull))
- devnull = NULL;
- /* replace all the matching ones with this */
- do {
+ /*
+ * Revalidate access to inherited open files. Each file is
+ * revalidated holding only a reference to it, with no locks
+ * held, as revalidation may sleep (e.g. unix socket mediation
+ * dropping path refs, or allocating path buffers).
+ */
+ while ((n = iterate_fd(files, n, get_next_file, &file)) != 0) {
+ int error = aa_file_perm(OP_INHERIT, cred, label, file,
+ aa_map_file_to_perms(file), false);
+
+ fput(file);
+ if (!error)
+ continue;
+ if (!devnull_tried) {
+ devnull_tried = true;
+ devnull = dentry_open(&aa_null, O_RDWR, cred);
+ if (IS_ERR(devnull))
+ devnull = NULL;
+ }
+ /* replace the denied fd (n - 1) with devnull, or close it */
replace_fd(n - 1, devnull, 0);
- } while ((n = iterate_fd(files, n, match_file, &cl)) != 0);
+ }
if (devnull)
fput(devnull);
-out:
aa_put_label(label);
}
diff --git a/security/apparmor/include/label.h b/security/apparmor/include/label.h
index 2c869b7dec97..61794b9cd6be 100644
--- a/security/apparmor/include/label.h
+++ b/security/apparmor/include/label.h
@@ -162,7 +162,11 @@ do { \
#define labels_ns(X) (vec_ns(&((X)->vec[0]), (X)->size))
#define labels_set(X) (&labels_ns(X)->labels)
#define labels_view(X) labels_ns(X)
-#define labels_profile(X) ((X)->vec[(X)->size - 1])
+
+static inline struct aa_profile *labels_profile(const struct aa_label *label)
+{
+ return label->vec[label->size - 1];
+}
int aa_label_next_confined(const struct aa_label *l, int i);
diff --git a/security/apparmor/include/policy.h b/security/apparmor/include/policy.h
index 0dcbf8cf1029..bb851654cb80 100644
--- a/security/apparmor/include/policy.h
+++ b/security/apparmor/include/policy.h
@@ -436,6 +436,26 @@ static inline void aa_put_profile(struct aa_profile *p)
kref_put(&p->label.count.count, aa_label_kref);
}
+/**
+ * aa_put_unpublished_profile - put a profile that was never made visible
+ * @p: profile (MAYBE NULL)
+ *
+ * A newly allocated profile and its proxy reference each other
+ * (proxy->label holds a label refcount). That cycle is normally broken
+ * when the profile is replaced or removed, which redirects the proxy.
+ * A profile that is discarded before being added to the policy tree
+ * never goes through either path, so break the cycle here, otherwise
+ * the profile, its proxy and its rawdata are leaked.
+ */
+static inline void aa_put_unpublished_profile(struct aa_profile *p)
+{
+ if (p && p->label.proxy) {
+ aa_put_proxy(p->label.proxy);
+ p->label.proxy = NULL;
+ }
+ aa_put_profile(p);
+}
+
static inline int AUDIT_MODE(const struct aa_profile *profile)
{
if (aa_g_audit != AUDIT_NORMAL)
diff --git a/security/apparmor/label.c b/security/apparmor/label.c
index 82742a471055..42ed476fe89d 100644
--- a/security/apparmor/label.c
+++ b/security/apparmor/label.c
@@ -387,7 +387,6 @@ void aa_label_kref(struct kref *kref)
label_free_switch(label);
return;
}
- /* TODO: update labels_profile macro so it works here */
AA_BUG(label_isprofile(label) &&
on_list_rcu(&label->vec[0]->base.profiles));
AA_BUG(label_isprofile(label) &&
diff --git a/security/apparmor/match.c b/security/apparmor/match.c
index 7713484f6a36..ca30a4292c42 100644
--- a/security/apparmor/match.c
+++ b/security/apparmor/match.c
@@ -397,7 +397,6 @@ struct aa_dfa *aa_dfa_unpack(const void *blob, size_t size, int flags)
/* check for duplicate table entry */
if (dfa->tables[table->td_id])
goto fail;
- dfa->tables[table->td_id] = table;
data += table_size(table->td_lolen, table->td_flags);
size -= table_size(table->td_lolen, table->td_flags);
@@ -414,9 +413,9 @@ struct aa_dfa *aa_dfa_unpack(const void *blob, size_t size, int flags)
if (!table)
goto fail;
}
- dfa->tables[table->td_id] = table;
break;
}
+ dfa->tables[table->td_id] = table;
table = NULL;
}
error = verify_table_headers(dfa->tables, flags);
diff --git a/security/apparmor/policy.c b/security/apparmor/policy.c
index 397f9ff78aeb..63f7821b4dff 100644
--- a/security/apparmor/policy.c
+++ b/security/apparmor/policy.c
@@ -1191,6 +1191,7 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
struct aa_loaddata *rawdata_ent;
const char *op;
ssize_t count, error;
+ bool rawdata_created = false;
LIST_HEAD(lh);
op = mask & AA_MAY_REPLACE_POLICY ? OP_PROF_REPL : OP_PROF_LOAD;
@@ -1330,6 +1331,7 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
ent = NULL;
goto fail_lock;
}
+ rawdata_created = true;
}
list_for_each_entry(ent, &lh, list) {
if (!ent->old) {
@@ -1425,6 +1427,20 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
return udata_sz;
fail_lock:
+ /*
+ * Remove fs entries created by this load. Nothing else references
+ * them yet, and leaving them behind would leak the new profiles
+ * (their dents pin the proxy) and make the next load at the same,
+ * unbumped ns revision fail with -EEXIST on its raw_data dir.
+ * ensure children (which are at the end of the list are removed
+ * before their parent
+ */
+ list_for_each_entry_reverse(tmp, &lh, list) {
+ if (!tmp->old)
+ __aafs_profile_rmdir(tmp->new);
+ }
+ if (rawdata_created)
+ __aa_fs_remove_rawdata(udata);
mutex_unlock(&ns->lock);
/* audit cause of failure */
@@ -1446,6 +1462,9 @@ ssize_t aa_replace_profiles(struct aa_ns *policy_ns, struct aa_label *label,
}
list_for_each_entry_safe(ent, tmp, &lh, list) {
list_del_init(&ent->list);
+ /* new was never added to the policy tree */
+ aa_put_unpublished_profile(ent->new);
+ ent->new = NULL;
aa_load_ent_free(ent);
}
diff --git a/security/apparmor/policy_unpack.c b/security/apparmor/policy_unpack.c
index f1fc48e72d0e..99dd61e3b103 100644
--- a/security/apparmor/policy_unpack.c
+++ b/security/apparmor/policy_unpack.c
@@ -1489,6 +1489,15 @@ static bool verify_dfa_accept_index(const struct aa_dfa *dfa, int table_size)
for (i = 0; i < dfa->tables[YYTD_ID_ACCEPT]->td_lolen; i++) {
if (ACCEPT_TABLE(dfa)[i] >= table_size)
return false;
+ /*
+ * Accept indexes for owner-conditional permissions come in
+ * pairs, so the non-owner entry at index + 1 must also be
+ * in bounds.
+ */
+ if (dfa->tables[YYTD_ID_ACCEPT2] &&
+ (ACCEPT_TABLE2(dfa)[i] & ACCEPT_FLAG_OWNER) &&
+ ACCEPT_TABLE(dfa)[i] + 1 >= table_size)
+ return false;
}
return true;
}
@@ -1807,13 +1816,15 @@ int aa_unpack(struct aa_loaddata *udata, struct list_head *lh,
fail_profile:
kfree(ns_name);
- aa_put_profile(profile);
+ aa_put_unpublished_profile(profile);
fail:
if (compressed_data)
kvfree(compressed_data);
list_for_each_entry_safe(ent, tmp, lh, list) {
list_del_init(&ent->list);
+ aa_put_unpublished_profile(ent->new);
+ ent->new = NULL;
aa_load_ent_free(ent);
}
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index 7c397b1c9231..77503bcdc53b 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -669,13 +669,30 @@ snd_compr_set_params(struct snd_compr_stream *stream, unsigned long arg)
if (retval)
return retval;
- retval = snd_compr_allocate_buffer(stream, params);
- if (retval)
- return -ENOMEM;
+ if (stream->next_track) {
+ /*
+ * total_bytes_available/transferred still refer to the
+ * current buffer; its geometry must not change.
+ */
+ if (params->buffer.fragment_size != stream->runtime->fragment_size ||
+ params->buffer.fragments != stream->runtime->fragments)
+ return -EINVAL;
+ } else {
+ retval = snd_compr_allocate_buffer(stream, params);
+ if (retval)
+ return retval;
+ }
retval = stream->ops->set_params(stream, params);
- if (retval)
+ if (retval) {
+ /* the stream stays OPEN: drop the buffer of the rejected setup */
+ if (stream->runtime->state == SNDRV_PCM_STATE_OPEN &&
+ !stream->runtime->dma_buffer_p) {
+ kfree(stream->runtime->buffer);
+ stream->runtime->buffer = NULL;
+ }
return retval;
+ }
if (stream->next_track)
return retval;
@@ -744,6 +761,9 @@ snd_compr_set_metadata(struct snd_compr_stream *stream, unsigned long arg)
* we should allow parameter change only when stream has been
* opened not in other cases
*/
+ if (stream->runtime->state == SNDRV_PCM_STATE_OPEN)
+ return -EBADFD;
+
if (copy_from_user(&metadata, (void __user *)arg, sizeof(metadata)))
return -EFAULT;
@@ -1136,7 +1156,9 @@ static int snd_compr_task_start_prepare(struct snd_compr_task_runtime *task,
{
if (task == NULL)
return -EINVAL;
- if (task->state >= SND_COMPRESS_TASK_STATE_FINISHED)
+ /* only an unqueued or already finished task may be (re)started */
+ if (task->state != SND_COMPRESS_TASK_STATE_IDLE &&
+ task->state != SND_COMPRESS_TASK_STATE_FINISHED)
return -EBUSY;
if (utask->input_size > task->input->size)
return -EINVAL;
diff --git a/sound/core/init.c b/sound/core/init.c
index 1bcb6a2e7550..a51b71812fd9 100644
--- a/sound/core/init.c
+++ b/sound/core/init.c
@@ -1174,12 +1174,12 @@ EXPORT_SYMBOL(snd_card_file_remove);
int snd_power_ref_and_wait(struct snd_card *card)
{
snd_power_ref(card);
- if (snd_power_get_state(card) == SNDRV_CTL_POWER_D0)
- return 0;
- wait_event_cmd(card->power_sleep,
- card->shutdown ||
- snd_power_get_state(card) == SNDRV_CTL_POWER_D0,
- snd_power_unref(card), snd_power_ref(card));
+ if (snd_power_get_state(card) != SNDRV_CTL_POWER_D0) {
+ wait_event_cmd(card->power_sleep,
+ card->shutdown ||
+ snd_power_get_state(card) == SNDRV_CTL_POWER_D0,
+ snd_power_unref(card), snd_power_ref(card));
+ }
if (card->shutdown) {
snd_power_unref(card);
return -ENODEV;
diff --git a/sound/core/seq/oss/seq_oss_device.h b/sound/core/seq/oss/seq_oss_device.h
index 935cf3df0b30..571186db4fa5 100644
--- a/sound/core/seq/oss/seq_oss_device.h
+++ b/sound/core/seq/oss/seq_oss_device.h
@@ -72,6 +72,7 @@ struct seq_oss_devinfo {
int cseq; /* sequencer client number */
int port; /* sequencer port number */
int queue; /* sequencer queue number */
+ bool initialized;
struct snd_seq_addr addr; /* address of this device */
diff --git a/sound/core/seq/oss/seq_oss_event.c b/sound/core/seq/oss/seq_oss_event.c
index 122735862044..20f4c5ce763c 100644
--- a/sound/core/seq/oss/seq_oss_event.c
+++ b/sound/core/seq/oss/seq_oss_event.c
@@ -425,6 +425,9 @@ snd_seq_oss_event_input(struct snd_seq_event *ev, int direct, void *private_data
struct seq_oss_devinfo *dp = (struct seq_oss_devinfo *)private_data;
union evrec *rec;
+ if (!dp || !smp_load_acquire(&dp->initialized))
+ return 0; /* ignored */
+
if (ev->type != SNDRV_SEQ_EVENT_ECHO)
return snd_seq_oss_midi_input(ev, direct, private_data);
diff --git a/sound/core/seq/oss/seq_oss_init.c b/sound/core/seq/oss/seq_oss_init.c
index 1aece46c8b06..a1063dc22d46 100644
--- a/sound/core/seq/oss/seq_oss_init.c
+++ b/sound/core/seq/oss/seq_oss_init.c
@@ -258,6 +258,7 @@ snd_seq_oss_open(struct file *file, int level)
client_table[dp->index] = dp;
num_clients++;
+ smp_store_release(&dp->initialized, true);
return 0;
@@ -403,6 +404,7 @@ snd_seq_oss_release(struct seq_oss_devinfo *dp)
client_table[dp->index] = NULL;
num_clients--;
+ smp_store_release(&dp->initialized, false);
snd_seq_oss_reset(dp);
diff --git a/sound/core/timer.c b/sound/core/timer.c
index 679b26435670..69a59b25fe0a 100644
--- a/sound/core/timer.c
+++ b/sound/core/timer.c
@@ -2176,6 +2176,7 @@ static const struct file_operations snd_utimer_fops = {
.llseek = noop_llseek,
.release = snd_utimer_release,
.unlocked_ioctl = snd_utimer_ioctl,
+ .compat_ioctl = snd_utimer_ioctl,
};
static int snd_utimer_start(struct snd_timer *t)
diff --git a/sound/core/timer_compat.c b/sound/core/timer_compat.c
index 25ee81c1668b..5bab00466de7 100644
--- a/sound/core/timer_compat.c
+++ b/sound/core/timer_compat.c
@@ -99,6 +99,7 @@ static long __snd_timer_user_ioctl_compat(struct file *file, unsigned int cmd,
case SNDRV_TIMER_IOCTL_PAUSE:
case SNDRV_TIMER_IOCTL_PAUSE_OLD:
case SNDRV_TIMER_IOCTL_NEXT_DEVICE:
+ case SNDRV_TIMER_IOCTL_CREATE:
return __snd_timer_user_ioctl(file, cmd, (unsigned long)argp, true);
case SNDRV_TIMER_IOCTL_GPARAMS32:
return snd_timer_user_gparams_compat(file, argp);
diff --git a/sound/core/ump_convert.c b/sound/core/ump_convert.c
index 85cc67de6330..0a7972d91ec5 100644
--- a/sound/core/ump_convert.c
+++ b/sound/core/ump_convert.c
@@ -323,7 +323,7 @@ static int fill_rpn(struct ump_cvt_to_ump_bank *cc,
cc->cc_data_lsb);
reset_rpn(cc);
- return 1;
+ return 8;
}
/* convert to a MIDI 1.0 Channel Voice message */
diff --git a/sound/drivers/aloop.c b/sound/drivers/aloop.c
index 81cb1f59f703..38fe3f4ce5d8 100644
--- a/sound/drivers/aloop.c
+++ b/sound/drivers/aloop.c
@@ -412,11 +412,14 @@ static int loopback_check_format(struct loopback_cable *cable, int stream)
runtime->channels != cruntime->channels ||
is_access_interleaved(runtime->access) !=
is_access_interleaved(cruntime->access);
- if (!check)
- return 0;
- if (stream == SNDRV_PCM_STREAM_CAPTURE)
+ /*
+ * Matching params also go on to the update below, so that
+ * the setup controls follow a playback that joins an open
+ * capture.
+ */
+ if (check && stream == SNDRV_PCM_STREAM_CAPTURE)
return -EIO;
- else if (cruntime->state == SNDRV_PCM_STATE_RUNNING) {
+ else if (check && cruntime->state == SNDRV_PCM_STATE_RUNNING) {
/* close must not free the peer runtime below */
snd_refcount_get(&cable->stop_count);
stop_capture = true;
@@ -1073,6 +1076,18 @@ static unsigned int get_cable_index(struct snd_pcm_substream *substream)
return !substream->stream;
}
+/*
+ * In notify mode the playback side may change format, rate, channels and,
+ * with a sound timer, the period size. A running capture that no longer
+ * matches is stopped by loopback_check_format().
+ * call in loopback->cable_lock
+ */
+static bool loopback_playback_unconstrained(struct loopback_pcm *dpcm)
+{
+ return dpcm->substream->stream == SNDRV_PCM_STREAM_PLAYBACK &&
+ get_notify(dpcm);
+}
+
static int rule_format(struct snd_pcm_hw_params *params,
struct snd_pcm_hw_rule *rule)
{
@@ -1082,6 +1097,8 @@ static int rule_format(struct snd_pcm_hw_params *params,
snd_mask_none(&m);
scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
m.bits[0] = (u_int32_t)cable->hw.formats;
m.bits[1] = (u_int32_t)(cable->hw.formats >> 32);
}
@@ -1096,6 +1113,8 @@ static int rule_rate(struct snd_pcm_hw_params *params,
struct snd_interval t;
scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
t.min = cable->hw.rate_min;
t.max = cable->hw.rate_max;
}
@@ -1112,6 +1131,8 @@ static int rule_channels(struct snd_pcm_hw_params *params,
struct snd_interval t;
scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
t.min = cable->hw.channels_min;
t.max = cable->hw.channels_max;
}
@@ -1128,6 +1149,8 @@ static int rule_period_bytes(struct snd_pcm_hw_params *params,
struct snd_interval t;
scoped_guard(mutex, &dpcm->loopback->cable_lock) {
+ if (loopback_playback_unconstrained(dpcm))
+ return 0;
t.min = cable->hw.period_bytes_min;
t.max = cable->hw.period_bytes_max;
}
diff --git a/sound/drivers/mpu401/mpu401_uart.c b/sound/drivers/mpu401/mpu401_uart.c
index 6de03d72a908..fcf2b686f9f5 100644
--- a/sound/drivers/mpu401/mpu401_uart.c
+++ b/sound/drivers/mpu401/mpu401_uart.c
@@ -155,20 +155,20 @@ static void snd_mpu401_uart_timer(struct timer_list *t)
scoped_guard(spinlock_irqsave, &mpu->timer_lock) {
/*mpu->mode |= MPU401_MODE_TIMER;*/
- mod_timer(&mpu->timer, 1 + jiffies);
+ if (mpu->timer_invoked)
+ mod_timer(&mpu->timer, 1 + jiffies);
}
if (mpu->rmidi)
_snd_mpu401_uart_interrupt(mpu);
}
/*
- * initialize the timer callback if not programmed yet
+ * arm the timer if it is not already active
*/
static void snd_mpu401_uart_add_timer (struct snd_mpu401 *mpu, int input)
{
guard(spinlock_irqsave)(&mpu->timer_lock);
if (mpu->timer_invoked == 0) {
- timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0);
mod_timer(&mpu->timer, 1 + jiffies);
}
mpu->timer_invoked |= input ? MPU401_MODE_INPUT_TIMER :
@@ -474,6 +474,7 @@ static void snd_mpu401_uart_free(struct snd_rawmidi *rmidi)
struct snd_mpu401 *mpu = rmidi->private_data;
if (mpu->irq >= 0)
free_irq(mpu->irq, (void *) mpu);
+ timer_shutdown_sync(&mpu->timer);
release_and_free_resource(mpu->res);
kfree(mpu);
}
@@ -528,6 +529,7 @@ int snd_mpu401_uart_new(struct snd_card *card, int device,
spin_lock_init(&mpu->input_lock);
spin_lock_init(&mpu->output_lock);
spin_lock_init(&mpu->timer_lock);
+ timer_setup(&mpu->timer, snd_mpu401_uart_timer, 0);
mpu->hardware = hardware;
mpu->irq = -1;
mpu->rmidi = rmidi;
diff --git a/sound/drivers/mtpav.c b/sound/drivers/mtpav.c
index 1ed6d4f0cb62..288a873a27e6 100644
--- a/sound/drivers/mtpav.c
+++ b/sound/drivers/mtpav.c
@@ -443,7 +443,7 @@ static void snd_mtpav_output_trigger(struct snd_rawmidi_substream *substream, in
scoped_guard(spinlock_irqsave, &mtp_card->spinlock) {
if (up) {
- if ((portp->mode & MTPAV_MODE_OUTPUT_TRIGGERED)) {
+ if (!(portp->mode & MTPAV_MODE_OUTPUT_TRIGGERED)) {
if (mtp_card->istimer++ == 0)
snd_mtpav_add_output_timer(mtp_card);
portp->mode |= MTPAV_MODE_OUTPUT_TRIGGERED;
diff --git a/sound/hda/codecs/hdmi/eld.c b/sound/hda/codecs/hdmi/eld.c
index 1464fd1c675b..fc6fe1b0873c 100644
--- a/sound/hda/codecs/hdmi/eld.c
+++ b/sound/hda/codecs/hdmi/eld.c
@@ -155,9 +155,11 @@ void snd_hdmi_write_eld_info(struct hdmi_eld *eld,
e->aud_synch_delay = val;
else if (!strcmp(name, "speakers"))
e->spk_alloc = val;
- else if (!strcmp(name, "sad_count"))
+ else if (!strcmp(name, "sad_count")) {
+ if (val > ELD_MAX_SAD)
+ continue;
e->sad_count = val;
- else if (!strncmp(name, "sad", 3)) {
+ } else if (!strncmp(name, "sad", 3)) {
sname = name + 4;
n = name[3] - '0';
if (name[4] >= '0' && name[4] <= '9') {
diff --git a/sound/hda/codecs/hdmi/hdmi.c b/sound/hda/codecs/hdmi/hdmi.c
index 1e7a05c87733..8020178b6296 100644
--- a/sound/hda/codecs/hdmi/hdmi.c
+++ b/sound/hda/codecs/hdmi/hdmi.c
@@ -983,7 +983,8 @@ static int hdmi_read_pin_conn(struct hda_codec *codec, int pin_idx)
snd_hda_set_dev_select(codec, pin_nid, dev_id);
if (spec->intel_hsw_fixup) {
- conns = spec->num_cvts;
+ /* spec->cvt_nids[] only has ARRAY_SIZE(spec->cvt_nids) slots */
+ conns = min_t(int, spec->num_cvts, ARRAY_SIZE(spec->cvt_nids));
memcpy(per_pin->mux_nids, spec->cvt_nids,
sizeof(hda_nid_t) * conns);
} else {
diff --git a/sound/hda/codecs/realtek/alc269.c b/sound/hda/codecs/realtek/alc269.c
index 1623cf926c71..68650004ce0f 100644
--- a/sound/hda/codecs/realtek/alc269.c
+++ b/sound/hda/codecs/realtek/alc269.c
@@ -4290,6 +4290,7 @@ enum {
ALC236_FIXUP_HP_LIMIT_INT_MIC_BOOST,
ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS,
ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
+ ALC287_FIXUP_LEGION_S7_15IMH05_AMP,
ALC287_FIXUP_YOGA7_14ITL_SPEAKERS,
ALC298_FIXUP_LENOVO_C940_DUET7,
ALC287_FIXUP_LENOVO_YOGA_BOOK_9I,
@@ -4338,9 +4339,12 @@ enum {
ALC245_FIXUP_HP_X360_MUTE_LEDS,
ALC287_FIXUP_THINKPAD_I2S_SPK,
ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_THINKPAD,
+ ALC287_FIXUP_CS35L41_I2C_4_THINKPAD_ACPI,
+ ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_4_THINKPAD,
ALC2XX_FIXUP_HEADSET_MIC,
ALC289_FIXUP_DELL_CS35L41_SPI_2,
ALC256_FIXUP_ACER_SFG16_MICMUTE_LED,
+ ALC256_FIXUP_ACER_HEADSET_MICMUTE_LED,
ALC256_FIXUP_HEADPHONE_AMP_VOL,
ALC245_FIXUP_HP_SPECTRE_X360_EU0XXX,
ALC245_FIXUP_HP_SPECTRE_X360_16_AA0XXX,
@@ -4375,7 +4379,6 @@ enum {
ALC245_FIXUP_BASS_HP_DAC,
ALC245_FIXUP_ACER_MICMUTE_LED,
ALC245_FIXUP_CS35L41_I2C_2_MUTE_LED,
- ALC236_FIXUP_HP_DMIC,
ALC256_FIXUP_HONOR_MRB_XXX_M1020_AUDIO,
ALC245_FIXUP_HP_ENVY_X360_15_FH0XXX,
ALC287_FIXUP_ACER_MICMUTE_LED,
@@ -4437,6 +4440,53 @@ static void alc700_fixup_intel_hades_canyon(struct hda_codec *codec,
alc_update_coef_idx(codec, 0x1b, 0x0400, 0x0400);
}
+static const struct hda_verb legion_amp_verbs[] = {
+ /* left speaker */
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x41 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x1a },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+
+ /* right speaker */
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x42 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2a },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
+ { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
+ {}
+};
+
+static void alc287_fixup_legion_s7_amp(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ if (action != HDA_FIXUP_ACT_INIT)
+ return;
+
+ snd_hda_sequence_write(codec, legion_amp_verbs);
+ /*
+ * The amplifiers accept the setup only a few ms after the first
+ * write (8 ms is the measured minimum on the test machine), so
+ * keep a margin.
+ */
+ msleep(20);
+ snd_hda_sequence_write(codec, legion_amp_verbs);
+}
+
static const struct hda_fixup alc269_fixups[] = {
[ALC298_FIXUP_RAZER_BLADE16_2025_PINS] = {
.type = HDA_FIXUP_PINS,
@@ -6486,41 +6536,13 @@ static const struct hda_fixup alc269_fixups[] = {
},
[ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS] = {
.type = HDA_FIXUP_VERBS,
- //.v.verbs = legion_15imhg05_coefs,
- .v.verbs = (const struct hda_verb[]) {
- // set left speaker Legion 7i.
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x41 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x1a },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- // set right speaker Legion 7i.
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x24 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x42 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xc },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2a },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
-
- { 0x20, AC_VERB_SET_COEF_INDEX, 0x26 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x2 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0x0 },
- { 0x20, AC_VERB_SET_PROC_COEF, 0xb020 },
- {}
- },
+ .v.verbs = legion_amp_verbs,
+ .chained = true,
+ .chain_id = ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
+ },
+ [ALC287_FIXUP_LEGION_S7_15IMH05_AMP] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc287_fixup_legion_s7_amp,
.chained = true,
.chain_id = ALC287_FIXUP_LEGION_15IMHG05_AUTOMUTE,
},
@@ -6886,6 +6908,18 @@ static const struct hda_fixup alc269_fixups[] = {
.chained = true,
.chain_id = ALC287_FIXUP_CS35L41_I2C_2_THINKPAD_ACPI,
},
+ [ALC287_FIXUP_CS35L41_I2C_4_THINKPAD_ACPI] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = cs35l41_fixup_i2c_four,
+ .chained = true,
+ .chain_id = ALC285_FIXUP_THINKPAD_NO_BASS_SPK_HEADSET_JACK,
+ },
+ [ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_4_THINKPAD] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc287_fixup_tb_vmaster_led,
+ .chained = true,
+ .chain_id = ALC287_FIXUP_CS35L41_I2C_4_THINKPAD_ACPI,
+ },
[ALC2XX_FIXUP_HEADSET_MIC] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc2xx_fixup_headset_mic,
@@ -6900,6 +6934,13 @@ static const struct hda_fixup alc269_fixups[] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc256_fixup_acer_sfg16_micmute_led,
},
+ [ALC256_FIXUP_ACER_HEADSET_MICMUTE_LED] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc256_fixup_acer_sfg16_micmute_led,
+ .chained = true,
+ .chained_before = true,
+ .chain_id = ALC2XX_FIXUP_HEADSET_MIC,
+ },
[ALC256_FIXUP_HEADPHONE_AMP_VOL] = {
.type = HDA_FIXUP_FUNC,
.v.func = alc256_decrease_headphone_amp_val,
@@ -7084,13 +7125,6 @@ static const struct hda_fixup alc269_fixups[] = {
.chained = true,
.chain_id = ALC287_FIXUP_CS35L41_I2C_2,
},
- [ALC236_FIXUP_HP_DMIC] = {
- .type = HDA_FIXUP_PINS,
- .v.pins = (const struct hda_pintbl[]) {
- { 0x12, 0x90a60160 }, /* use as internal mic */
- { }
- },
- },
[ALC256_FIXUP_HONOR_MRB_XXX_M1020_AUDIO] = {
.type = HDA_FIXUP_PINS,
.v.pins = (const struct hda_pintbl[]) {
@@ -7230,7 +7264,13 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1025, 0x160e, "Acer PT316-51S", ALC2XX_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1025, 0x1616, "Acer Aspire A515-57", ALC256_FIXUP_ACER_COLDBOOT),
SND_PCI_QUIRK(0x1025, 0x161f, "Acer S40-54", ALC256_FIXUP_ACER_MIC_NO_PRESENCE),
- SND_PCI_QUIRK(0x1025, 0x1640, "Acer Aspire A315-44P", ALC256_FIXUP_ACER_SFG16_MICMUTE_LED),
+ /*
+ * Shared SSID: A315-44P needs GPIO3 micmute LED; AG15-42P needs
+ * the headset pin remap (BIOS leaves 0x19 as 0x411111f0). One
+ * entry applies both. GPIO3 is unused on AG15.
+ */
+ SND_PCI_QUIRK(0x1025, 0x1640, "Acer Aspire A315-44P / AG15-42P",
+ ALC256_FIXUP_ACER_HEADSET_MICMUTE_LED),
SND_PCI_QUIRK(0x1025, 0x166c, "Acer Predator PH16-71", ALC2XX_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1025, 0x1679, "Acer Nitro 16 AN16-41", ALC2XX_FIXUP_HEADSET_MIC),
SND_PCI_QUIRK(0x1025, 0x169a, "Acer Swift SFG16", ALC256_FIXUP_ACER_SFG16_MICMUTE_LED),
@@ -7722,12 +7762,12 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x103c, 0x8d9c, "HP 17 Turbine OmniBook 7 DIS", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8d9d, "HP 17 Turbine OmniBook X UMA", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8d9e, "HP 17 Turbine OmniBook X DIS", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2),
+ SND_PCI_QUIRK(0x103c, 0x8d9f, "HP 14 Cadet (x360)", ALC287_FIXUP_CS35L41_I2C_2_HP_MUTE_LEDS),
SND_PCI_QUIRK(0x103c, 0x8da0, "HP 16 Clipper OmniBook 7(X360)", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da1, "HP 16 Clipper OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da7, "HP 14 Enstrom OmniBook X", ALC287_FIXUP_CS35L41_I2C_2),
SND_PCI_QUIRK(0x103c, 0x8da8, "HP 16 Piston OmniBook X", ALC245_FIXUP_HP_ENVY_X360_15_FH0XXX),
- SND_PCI_QUIRK(0x103c, 0x8dc9, "HP Laptop 15-fc0xxx", ALC236_FIXUP_HP_DMIC),
+ SND_PCI_QUIRK(0x103c, 0x8dc9, "HP Laptop 15-fc0xxx", ALC236_FIXUP_HP_MUTE_LED_MICMUTE_GPIO),
SND_PCI_QUIRK(0x103c, 0x8dd4, "HP EliteStudio 8 AIO", ALC274_FIXUP_HP_AIO_BIND_DACS),
SND_PCI_QUIRK(0x103c, 0x8dd7, "HP Laptop 15-fd0xxx", ALC236_FIXUP_HP_MUTE_LED_MICMUTE_GPIO),
SND_PCI_QUIRK(0x103c, 0x8de8, "HP Gemtree", ALC245_FIXUP_TAS2781_SPI_2),
@@ -7867,7 +7907,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1043, 0x1663, "ASUS GU603ZI/ZJ/ZQ/ZU/ZV", ALC285_FIXUP_ASUS_HEADSET_MIC),
SND_PCI_QUIRK(0x1043, 0x1664, "ASUS ExpertBook PM3406CHA", ALC256_FIXUP_ASUS_PM3406CHA),
SND_PCI_QUIRK(0x1043, 0x1683, "ASUS UM3402YAR", ALC287_FIXUP_CS35L41_I2C_2),
- SND_PCI_QUIRK(0x1043, 0x16a3, "ASUS UX3402VA", ALC245_FIXUP_CS35L41_SPI_2),
+ SND_PCI_QUIRK(0x1043, 0x16a3, "ASUS UX3402VA", ALC294_FIXUP_ASUS_SPI_HEADSET_MIC),
SND_PCI_QUIRK(0x1043, 0x16b2, "ASUS GU603", ALC289_FIXUP_ASUS_GA401),
SND_PCI_QUIRK(0x1043, 0x16d3, "ASUS UX5304VA", ALC245_FIXUP_CS35L41_SPI_2),
SND_PCI_QUIRK(0x1043, 0x16e3, "ASUS UX50", ALC269_FIXUP_STEREO_DMIC),
@@ -8206,7 +8246,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
HDA_CODEC_QUIRK(0x17aa, 0x38d5, "Lenovo Yoga Pro 9 16IMH9", ALC287_FIXUP_TAS2781_I2C),
HDA_CODEC_QUIRK(0x17aa, 0x38d6, "Lenovo Yoga Pro 9 16IMH9", ALC287_FIXUP_TAS2781_I2C),
HDA_CODEC_QUIRK(0x17aa, 0x3855, "Legion 7 16ITHG6", ALC287_FIXUP_LEGION_16ITHG6),
- SND_PCI_QUIRK(0x17aa, 0x3811, "Legion S7 15IMH05", ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS),
+ SND_PCI_QUIRK(0x17aa, 0x3811, "Legion S7 15IMH05", ALC287_FIXUP_LEGION_S7_15IMH05_AMP),
SND_PCI_QUIRK(0x17aa, 0x3813, "Legion 7i 15IMHG05", ALC287_FIXUP_LEGION_15IMHG05_SPEAKERS),
SND_PCI_QUIRK(0x17aa, 0x3818, "Lenovo C940 / Yoga Duet 7", ALC298_FIXUP_LENOVO_C940_DUET7),
SND_PCI_QUIRK(0x17aa, 0x3819, "Lenovo 13s Gen2 ITL", ALC287_FIXUP_13S_GEN2_SPEAKERS),
@@ -8216,6 +8256,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x17aa, 0x3827, "Ideapad S740", ALC285_FIXUP_IDEAPAD_S740_COEF),
SND_PCI_QUIRK(0x17aa, 0x3834, "Lenovo IdeaPad Slim 9i 14ITL5", ALC287_FIXUP_YOGA7_14ITL_SPEAKERS),
SND_PCI_QUIRK(0x17aa, 0x383d, "Legion Y9000X 2019", ALC285_FIXUP_LEGION_Y9000X_SPEAKERS),
+ SND_PCI_QUIRK(0x17aa, 0x3842, "ThinkBook X G2 IAH", ALC287_FIXUP_MG_RTKC_CSAMP_CS35L41_I2C_4_THINKPAD),
SND_PCI_QUIRK(0x17aa, 0x3843, "Lenovo Yoga 9i / Yoga Book 9i", ALC287_FIXUP_LENOVO_YOGA_BOOK_9I),
/* Yoga Pro 7 14IMH9 shares PCI SSID 17aa:3847 with Legion 7 16ACHG6;
* use codec SSID to distinguish them
@@ -8419,6 +8460,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x2782, 0x1701, "Infinix Y4 Max", ALC269VC_FIXUP_INFINIX_Y4_MAX),
SND_PCI_QUIRK(0x2782, 0x1705, "MEDION E15433", ALC269VC_FIXUP_INFINIX_Y4_MAX),
SND_PCI_QUIRK(0x2782, 0x1707, "Vaio VJFE-ADL", ALC298_FIXUP_SPK_VOLUME),
+ SND_PCI_QUIRK(0x2782, 0x1718, "Acer Aspire AL16-31P", ALC269VC_FIXUP_INFINIX_Y4_MAX),
SND_PCI_QUIRK(0x2782, 0x4900, "MEDION E15443", ALC233_FIXUP_MEDION_MTL_SPK),
SND_PCI_QUIRK(0x2782, 0xa128, "Positivo N15RPE-S", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x2782, 0xa212, "Lunnen Ground 14", ALC269VC_FIXUP_LUNNEN_GROUND_14),
diff --git a/sound/hda/codecs/side-codecs/cs35l41_hda.c b/sound/hda/codecs/side-codecs/cs35l41_hda.c
index 62296be45c55..d50c3d8f5303 100644
--- a/sound/hda/codecs/side-codecs/cs35l41_hda.c
+++ b/sound/hda/codecs/side-codecs/cs35l41_hda.c
@@ -1907,7 +1907,6 @@ int cs35l41_hda_parse_acpi(struct cs35l41_hda *cs35l41, struct device *physdev,
hw_cfg->valid = false;
hw_cfg->gpio1.valid = false;
hw_cfg->gpio2.valid = false;
- acpi_dev_put(cs35l41->dacpi);
return ret;
}
@@ -1928,10 +1927,8 @@ static int cs35l41_hda_read_acpi(struct cs35l41_hda *cs35l41, const char *hid, i
cs35l41->dacpi = adev;
struct device *physdev __free(put_device) =
get_device(acpi_get_first_physical_node(adev));
- if (!physdev) {
- acpi_dev_put(adev);
+ if (!physdev)
return -ENODEV;
- }
sub = acpi_get_subsystem_id(ACPI_HANDLE(physdev));
if (IS_ERR(sub))
@@ -1986,8 +1983,10 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
dev_set_drvdata(dev, cs35l41);
ret = cs35l41_hda_read_acpi(cs35l41, device_name, id);
- if (ret)
- return dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ if (ret) {
+ dev_err_probe(cs35l41->dev, ret, "Platform not supported\n");
+ goto err_acpi;
+ }
if (IS_ERR(cs35l41->reset_gpio)) {
ret = PTR_ERR(cs35l41->reset_gpio);
@@ -2074,6 +2073,7 @@ int cs35l41_hda_probe(struct device *dev, const char *device_name, int id, int i
err:
if (cs35l41_safe_reset(cs35l41->regmap, cs35l41->hw_cfg.bst_type))
gpiod_set_value_cansleep(cs35l41->reset_gpio, 0);
+err_acpi:
gpiod_put(cs35l41->reset_gpio);
gpiod_put(cs35l41->cs_gpio);
acpi_dev_put(cs35l41->dacpi);
diff --git a/sound/hda/codecs/side-codecs/tas2781_hda_i2c.c b/sound/hda/codecs/side-codecs/tas2781_hda_i2c.c
index 5e5f46c9fb6b..44855f593daf 100644
--- a/sound/hda/codecs/side-codecs/tas2781_hda_i2c.c
+++ b/sound/hda/codecs/side-codecs/tas2781_hda_i2c.c
@@ -335,10 +335,11 @@ static int tas2563_save_calibration(struct tas2781_hda *h)
}
cd->cali_dat_sz_per_dev = TAS2563_CAL_DATA_SIZE * TASDEV_CALIB_N;
+ cd->total_sz = 0;
/* extra byte for each device is the device number */
- cd->total_sz = (cd->cali_dat_sz_per_dev + 1) * p->ndev;
- data = cd->data = devm_kzalloc(p->dev, cd->total_sz,
+ data = cd->data = devm_kzalloc(p->dev,
+ (cd->cali_dat_sz_per_dev + 1) * p->ndev,
GFP_KERNEL);
if (!data)
return -ENOMEM;
@@ -352,7 +353,6 @@ static int tas2563_save_calibration(struct tas2781_hda *h)
if (ret < 0 || ret >= sizeof(var8) - 1) {
dev_err(p->dev, "%s: Read %s failed\n",
__func__, var8);
- cd->total_sz = 0;
return -EINVAL;
}
/*
@@ -370,7 +370,11 @@ static int tas2563_save_calibration(struct tas2781_hda *h)
dev_warn(p->dev,
"Dev %d: Caldat[%d] read failed %ld\n",
i, j, status);
- cd->total_sz = 0;
+ return -EINVAL;
+ }
+ if (*(uint32_t *)&data[offset] == 0) {
+ dev_warn(p->dev,
+ "Dev %d: Caldat[%d] is zero\n", i, j);
return -EINVAL;
}
bedata = cpu_to_be32(*(uint32_t *)&data[offset]);
@@ -379,12 +383,7 @@ static int tas2563_save_calibration(struct tas2781_hda *h)
}
}
- if (cd->total_sz != offset) {
- dev_err(p->dev, "%s: tot_size(%lu) and offset(%u) mismatch\n",
- __func__, cd->total_sz, offset);
- cd->total_sz = 0;
- return -EINVAL;
- }
+ cd->total_sz = offset;
r->r0_reg = TAS2563_CAL_R0;
r->invr0_reg = TAS2563_CAL_INVR0;
diff --git a/sound/hda/codecs/via.c b/sound/hda/codecs/via.c
index 807312cffd35..0641fc14b346 100644
--- a/sound/hda/codecs/via.c
+++ b/sound/hda/codecs/via.c
@@ -933,6 +933,7 @@ static const struct hda_fixup via_fixups[] = {
static const struct hda_quirk vt2002p_fixups[] = {
SND_PCI_QUIRK(0x1043, 0x13f7, "Asus B23E", VIA_FIXUP_POWER_SAVE),
SND_PCI_QUIRK(0x1043, 0x1487, "Asus G75", VIA_FIXUP_ASUS_G75),
+ SND_PCI_QUIRK(0x1043, 0x1c33, "Asus X55A", VIA_FIXUP_POWER_SAVE),
SND_PCI_QUIRK(0x1043, 0x8532, "Asus X202E", VIA_FIXUP_INTMIC_BOOST),
SND_PCI_QUIRK_VENDOR(0x1558, "Clevo", VIA_FIXUP_POWER_SAVE),
{}
diff --git a/sound/hda/common/codec.c b/sound/hda/common/codec.c
index 5d9b53bd64ea..7c2b967aebe9 100644
--- a/sound/hda/common/codec.c
+++ b/sound/hda/common/codec.c
@@ -1800,6 +1800,12 @@ int snd_hda_codec_reset(struct hda_codec *codec)
{
struct hda_bus *bus = codec->bus;
+ /* ASoC: driver unbind frees the card, incl. the hwdep issuing the reset */
+ if (bus->core.ext_ops) {
+ codec_err(codec, "Codec reset is not supported with ASoC\n");
+ return -EOPNOTSUPP;
+ }
+
if (snd_hda_lock_devices(bus) < 0)
return -EBUSY;
diff --git a/sound/hda/common/sysfs.c b/sound/hda/common/sysfs.c
index bedf10b30885..bf65223fde10 100644
--- a/sound/hda/common/sysfs.c
+++ b/sound/hda/common/sysfs.c
@@ -117,6 +117,8 @@ static int clear_codec(struct hda_codec *codec)
int err;
err = snd_hda_codec_reset(codec);
+ if (err == -EOPNOTSUPP)
+ return err;
if (err < 0) {
codec_err(codec, "The codec is being used, can't free.\n");
return err;
@@ -132,6 +134,8 @@ static int reconfig_codec(struct hda_codec *codec)
CLASS(snd_hda_power, pm)(codec);
codec_info(codec, "hda-codec: reconfiguring\n");
err = snd_hda_codec_reset(codec);
+ if (err == -EOPNOTSUPP)
+ return err;
if (err < 0) {
codec_err(codec,
"The codec is being used, can't reconfigure.\n");
diff --git a/sound/mips/sgio2audio.c b/sound/mips/sgio2audio.c
index 497d84cff12f..35648b0b97b9 100644
--- a/sound/mips/sgio2audio.c
+++ b/sound/mips/sgio2audio.c
@@ -742,7 +742,8 @@ static struct {
/* ALSA driver */
-static int snd_sgio2audio_free(struct snd_sgio2audio *chip)
+static int snd_sgio2audio_free(struct snd_sgio2audio *chip,
+ unsigned int irq_count)
{
int i;
@@ -752,7 +753,7 @@ static int snd_sgio2audio_free(struct snd_sgio2audio *chip)
writeq(0, &mace->perif.audio.control);
/* release IRQ's */
- for (i = 0; i < ARRAY_SIZE(snd_sgio2_isr_table); i++)
+ for (i = 0; i < irq_count; i++)
free_irq(snd_sgio2_isr_table[i].irq,
&chip->channel[snd_sgio2_isr_table[i].idx]);
@@ -768,7 +769,7 @@ static int snd_sgio2audio_dev_free(struct snd_device *device)
{
struct snd_sgio2audio *chip = device->device_data;
- return snd_sgio2audio_free(chip);
+ return snd_sgio2audio_free(chip, ARRAY_SIZE(snd_sgio2_isr_table));
}
static const struct snd_device_ops ops = {
@@ -819,7 +820,7 @@ static int snd_sgio2audio_create(struct snd_card *card,
0,
snd_sgio2_isr_table[i].desc,
&chip->channel[snd_sgio2_isr_table[i].idx])) {
- snd_sgio2audio_free(chip);
+ snd_sgio2audio_free(chip, i);
printk(KERN_ERR "sgio2audio: cannot allocate irq %d\n",
snd_sgio2_isr_table[i].irq);
return -EBUSY;
@@ -843,13 +844,13 @@ static int snd_sgio2audio_create(struct snd_card *card,
/* initialize the AD1843 codec */
err = ad1843_init(&chip->ad1843);
if (err < 0) {
- snd_sgio2audio_free(chip);
+ snd_sgio2audio_free(chip, ARRAY_SIZE(snd_sgio2_isr_table));
return err;
}
err = snd_device_new(card, SNDRV_DEV_LOWLEVEL, chip, &ops);
if (err < 0) {
- snd_sgio2audio_free(chip);
+ snd_sgio2audio_free(chip, ARRAY_SIZE(snd_sgio2_isr_table));
return err;
}
*rchip = chip;
diff --git a/sound/oss/dmasound/dmasound_q40.c b/sound/oss/dmasound/dmasound_q40.c
index e25a78dd1bf2..339311e5d807 100644
--- a/sound/oss/dmasound/dmasound_q40.c
+++ b/sound/oss/dmasound/dmasound_q40.c
@@ -49,9 +49,8 @@ static int Q40SetFormat(int format);
static int Q40SetVolume(int volume);
static void Q40PlayNextFrame(int index);
static void Q40Play(void);
-static irqreturn_t Q40StereoInterrupt(int irq, void *dummy);
-static irqreturn_t Q40MonoInterrupt(int irq, void *dummy);
-static void Q40Interrupt(void);
+static irqreturn_t q40_audio_interrupt(int irq, void *dummy);
+static void q40_frame_done(void);
/*** Mid level stuff *********************************************************/
@@ -372,8 +371,8 @@ static void Q40Free(void *ptr, unsigned int size)
static int __init Q40IrqInit(void)
{
/* Register interrupt handler. */
- if (request_irq(Q40_IRQ_SAMPLE, Q40StereoInterrupt, 0,
- "DMA sound", Q40Interrupt))
+ if (request_irq(Q40_IRQ_SAMPLE, q40_audio_interrupt, 0,
+ "DMA sound", q40_frame_done))
return 0;
return(1);
@@ -384,7 +383,7 @@ static int __init Q40IrqInit(void)
static void Q40IrqCleanUp(void)
{
master_outb(0,SAMPLE_ENABLE_REG);
- free_irq(Q40_IRQ_SAMPLE, Q40Interrupt);
+ free_irq(Q40_IRQ_SAMPLE, q40_frame_done);
}
#endif /* MODULE */
@@ -403,7 +402,6 @@ static void Q40PlayNextFrame(int index)
u_char *start;
u_long size;
u_char speed;
- int error;
/* used by Q40Play() if all doubts whether there really is something
* to be played are already wiped out.
@@ -420,15 +418,6 @@ static void Q40PlayNextFrame(int index)
speed=(dmasound.hard.speed==10000 ? 0 : 1);
master_outb( 0,SAMPLE_ENABLE_REG);
- free_irq(Q40_IRQ_SAMPLE, Q40Interrupt);
- if (dmasound.soft.stereo)
- error = request_irq(Q40_IRQ_SAMPLE, Q40StereoInterrupt, 0,
- "Q40 sound", Q40Interrupt);
- else
- error = request_irq(Q40_IRQ_SAMPLE, Q40MonoInterrupt, 0,
- "Q40 sound", Q40Interrupt);
- if (error && printk_ratelimit())
- pr_err("Couldn't register sound interrupt\n");
master_outb( speed, SAMPLE_RATE_REG);
master_outb( 1,SAMPLE_CLEAR_REG);
@@ -456,31 +445,35 @@ static void Q40Play(void)
spin_unlock_irqrestore(&dmasound.lock, flags);
}
-static irqreturn_t Q40StereoInterrupt(int irq, void *dummy)
-{
- spin_lock(&dmasound.lock);
- if (q40_sc>1){
- *DAC_LEFT=*q40_pp++;
- *DAC_RIGHT=*q40_pp++;
- q40_sc -=2;
- master_outb(1,SAMPLE_CLEAR_REG);
- }else Q40Interrupt();
- spin_unlock(&dmasound.lock);
- return IRQ_HANDLED;
-}
-static irqreturn_t Q40MonoInterrupt(int irq, void *dummy)
+static irqreturn_t q40_audio_interrupt(int irq, void *dummy)
{
+ bool frame_done = false;
+
spin_lock(&dmasound.lock);
- if (q40_sc>0){
- *DAC_LEFT=*q40_pp;
- *DAC_RIGHT=*q40_pp++;
- q40_sc --;
- master_outb(1,SAMPLE_CLEAR_REG);
- }else Q40Interrupt();
+ if (dmasound.hard.stereo) {
+ if (q40_sc > 1) {
+ *DAC_LEFT = *q40_pp++;
+ *DAC_RIGHT = *q40_pp++;
+ q40_sc -= 2;
+ master_outb(1, SAMPLE_CLEAR_REG);
+ } else {
+ frame_done = true;
+ }
+ } else if (q40_sc > 0) {
+ *DAC_LEFT = *q40_pp;
+ *DAC_RIGHT = *q40_pp++;
+ q40_sc--;
+ master_outb(1, SAMPLE_CLEAR_REG);
+ } else {
+ frame_done = true;
+ }
spin_unlock(&dmasound.lock);
+ if (frame_done)
+ q40_frame_done();
return IRQ_HANDLED;
}
-static void Q40Interrupt(void)
+
+static void q40_frame_done(void)
{
if (!write_sq.active) {
/* playing was interrupted and sq_reset() has already cleared
diff --git a/sound/pci/cs46xx/dsp_spos.c b/sound/pci/cs46xx/dsp_spos.c
index 6fd4f629d1d5..a77e6f5a14ad 100644
--- a/sound/pci/cs46xx/dsp_spos.c
+++ b/sound/pci/cs46xx/dsp_spos.c
@@ -393,7 +393,7 @@ int cs46xx_dsp_load_module (struct snd_cs46xx * chip, struct dsp_module_desc * m
dev_dbg(chip->card->dev,
"dsp_spos: module got no code segment\n");
} else {
- if (ins->code.offset + code->size > DSP_CODE_BYTE_SIZE) {
+ if ((u64)ins->code.offset + code->size > DSP_CODE_BYTE_SIZE / 4) {
dev_err(chip->card->dev,
"dsp_spos: no space available in DSP\n");
return -ENOMEM;
diff --git a/sound/soc/amd/acp-config.c b/sound/soc/amd/acp-config.c
index 3d48bb0f100f..f190d3cdc81e 100644
--- a/sound/soc/amd/acp-config.c
+++ b/sound/soc/amd/acp-config.c
@@ -98,6 +98,13 @@ static const struct dmi_system_id acp70_acpi_flag_override_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "8EA2"),
},
},
+ {
+ /* Dell 14S DS14265 */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Dell 14S DS14265"),
+ },
+ },
{}
};
diff --git a/sound/soc/codecs/arizona-jack.c b/sound/soc/codecs/arizona-jack.c
index 819d080b1188..6587d46a1867 100644
--- a/sound/soc/codecs/arizona-jack.c
+++ b/sound/soc/codecs/arizona-jack.c
@@ -699,6 +699,7 @@ static void arizona_start_hpdet_acc_id(struct arizona_priv *info)
snd_soc_jack_report(info->jack, SND_JACK_HEADPHONE,
SND_JACK_LINEOUT | SND_JACK_HEADPHONE);
+ pm_runtime_put_autosuspend(arizona->dev);
info->hpdet_active = false;
}
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index 1c06fdbf0e71..40d602f79314 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -514,6 +514,9 @@ static int hdac_hda_codec_probe(struct snd_soc_component *component)
snd_hdac_display_power(hdev->bus,
HDA_CODEC_IDX_CONTROLLER, false);
+ /* The codec is in D0, the status is RPM_SUSPENDED after a previous unbind */
+ pm_runtime_set_active(&hdev->dev);
+
/* match for forbid call in snd_hda_codec_device_new() */
pm_runtime_allow(&hdev->dev);
@@ -554,6 +557,13 @@ static void hdac_hda_codec_remove(struct snd_soc_component *component)
return;
}
+ /*
+ * Resume the codec to have its link reference (dropped below) and
+ * codec_powered bit in the same state as after the probe.
+ */
+ pm_runtime_get_sync(&hdev->dev);
+
+ /* prevent runtime PM callbacks while the link and the codec are torn down */
pm_runtime_disable(&hdev->dev);
snd_hdac_ext_bus_link_put(hdev->bus, hlink);
@@ -561,6 +571,12 @@ static void hdac_hda_codec_remove(struct snd_soc_component *component)
driver->ops->remove(codec);
snd_hda_codec_cleanup_for_unbind(codec);
+
+ /* Balances the disable above, cleanup_for_unbind keeps runtime PM disabled */
+ pm_runtime_enable(&hdev->dev);
+ pm_runtime_put_noidle(&hdev->dev);
+ /* The codec is cleaned up, system suspend must skip it from now on */
+ pm_runtime_set_suspended(&hdev->dev);
}
static const struct snd_soc_dapm_route hdac_hda_dapm_routes[] = {
diff --git a/sound/soc/codecs/lpass-wsa-macro.c b/sound/soc/codecs/lpass-wsa-macro.c
index 38e37908baf9..f7dad096bfa3 100644
--- a/sound/soc/codecs/lpass-wsa-macro.c
+++ b/sound/soc/codecs/lpass-wsa-macro.c
@@ -1129,27 +1129,35 @@ static const struct regmap_config wsa_regmap_config = {
int wsa_macro_set_spkr_mode(struct snd_soc_component *component, int mode)
{
struct wsa_macro *wsa = snd_soc_component_get_drvdata(component);
+ unsigned int comp1_ctl3, comp1_ctl7;
+ u8 comp_val, boost_val;
wsa->spkr_mode = mode;
- switch (mode) {
- case WSA_MACRO_SPKR_MODE_1:
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL3, 0x80, 0x00);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER1_CTL3, 0x80, 0x00);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL7, 0x01, 0x00);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER1_CTL7, 0x01, 0x00);
- snd_soc_component_update_bits(component, CDC_WSA_BOOST0_BOOST_CTL, 0x7C, 0x44);
- snd_soc_component_update_bits(component, CDC_WSA_BOOST1_BOOST_CTL, 0x7C, 0x44);
- break;
- default:
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL3, 0x80, 0x80);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER1_CTL3, 0x80, 0x80);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL7, 0x01, 0x01);
- snd_soc_component_update_bits(component, CDC_WSA_COMPANDER1_CTL7, 0x01, 0x01);
- snd_soc_component_update_bits(component, CDC_WSA_BOOST0_BOOST_CTL, 0x7C, 0x58);
- snd_soc_component_update_bits(component, CDC_WSA_BOOST1_BOOST_CTL, 0x7C, 0x58);
- break;
+ /* Compander1 CTL3/CTL7 moved on v2.5+ silicon. */
+ if (wsa->codec_version >= LPASS_CODEC_VERSION_2_5) {
+ comp1_ctl3 = CDC_2_5_WSA_COMPANDER1_CTL3;
+ comp1_ctl7 = CDC_2_5_WSA_COMPANDER1_CTL7;
+ } else {
+ comp1_ctl3 = CDC_WSA_COMPANDER1_CTL3;
+ comp1_ctl7 = CDC_WSA_COMPANDER1_CTL7;
}
+
+ if (mode == WSA_MACRO_SPKR_MODE_1) {
+ comp_val = 0x00;
+ boost_val = 0x44;
+ } else {
+ comp_val = 0x80;
+ boost_val = 0x58;
+ }
+
+ snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL3, 0x80, comp_val);
+ snd_soc_component_update_bits(component, comp1_ctl3, 0x80, comp_val);
+ snd_soc_component_update_bits(component, CDC_WSA_COMPANDER0_CTL7, 0x01, comp_val ? 0x01 : 0x00);
+ snd_soc_component_update_bits(component, comp1_ctl7, 0x01, comp_val ? 0x01 : 0x00);
+ snd_soc_component_update_bits(component, CDC_WSA_BOOST0_BOOST_CTL, 0x7C, boost_val);
+ snd_soc_component_update_bits(component, CDC_WSA_BOOST1_BOOST_CTL, 0x7C, boost_val);
+
return 0;
}
EXPORT_SYMBOL(wsa_macro_set_spkr_mode);
diff --git a/sound/soc/codecs/pcm6240.c b/sound/soc/codecs/pcm6240.c
index db85ae2f8aed..7a8c8f53f04c 100644
--- a/sound/soc/codecs/pcm6240.c
+++ b/sound/soc/codecs/pcm6240.c
@@ -2019,8 +2019,6 @@ static const struct regmap_config pcmdevice_i2c_regmap = {
static void pcmdevice_remove(struct pcmdevice_priv *pcm_dev)
{
- if (pcm_dev->irq)
- free_irq(pcm_dev->irq, pcm_dev);
mutex_destroy(&pcm_dev->codec_lock);
}
diff --git a/sound/soc/codecs/rt5682s.c b/sound/soc/codecs/rt5682s.c
index 34997cfb465b..7a196fab623a 100644
--- a/sound/soc/codecs/rt5682s.c
+++ b/sound/soc/codecs/rt5682s.c
@@ -3294,8 +3294,8 @@ static void rt5682s_i2c_shutdown(struct i2c_client *client)
struct rt5682s_priv *rt5682s = i2c_get_clientdata(client);
disable_irq(client->irq);
- cancel_delayed_work_sync(&rt5682s->jack_detect_work);
- cancel_delayed_work_sync(&rt5682s->jd_check_work);
+ disable_delayed_work_sync(&rt5682s->jack_detect_work);
+ disable_delayed_work_sync(&rt5682s->jd_check_work);
rt5682s_reset(rt5682s);
}
diff --git a/sound/soc/codecs/rt712-sdca-dmic.c b/sound/soc/codecs/rt712-sdca-dmic.c
index a9f3aa4e143a..882390890738 100644
--- a/sound/soc/codecs/rt712-sdca-dmic.c
+++ b/sound/soc/codecs/rt712-sdca-dmic.c
@@ -246,7 +246,7 @@ static int rt712_sdca_dmic_set_gain_get(struct snd_kcontrol *kcontrol,
struct snd_ctl_elem_value *ucontrol)
{
struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
- struct rt712_sdca_priv *rt712 = snd_soc_component_get_drvdata(component);
+ struct rt712_sdca_dmic_priv *rt712 = snd_soc_component_get_drvdata(component);
struct rt712_sdca_dmic_kctrl_priv *p =
(struct rt712_sdca_dmic_kctrl_priv *)kcontrol->private_value;
unsigned int regvalue, ctl, i;
@@ -277,7 +277,7 @@ static int rt712_sdca_dmic_set_gain_put(struct snd_kcontrol *kcontrol,
struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
struct rt712_sdca_dmic_kctrl_priv *p =
(struct rt712_sdca_dmic_kctrl_priv *)kcontrol->private_value;
- struct rt712_sdca_priv *rt712 = snd_soc_component_get_drvdata(component);
+ struct rt712_sdca_dmic_priv *rt712 = snd_soc_component_get_drvdata(component);
unsigned int gain_val[4];
unsigned int i, adc_vol_flag = 0, changed = 0;
unsigned int regvalue[4];
diff --git a/sound/soc/codecs/tlv320aic32x4-clk.c b/sound/soc/codecs/tlv320aic32x4-clk.c
index deed61650e09..8fa8e69f1f72 100644
--- a/sound/soc/codecs/tlv320aic32x4-clk.c
+++ b/sound/soc/codecs/tlv320aic32x4-clk.c
@@ -318,6 +318,9 @@ static int clk_aic32x4_div_set_rate(struct clk_hw *hw, unsigned long rate,
struct clk_aic32x4 *div = to_clk_aic32x4(hw);
u8 divisor;
+ if (!rate)
+ return -EINVAL;
+
divisor = DIV_ROUND_UP(parent_rate, rate);
if (divisor > AIC32X4_DIV_MAX)
return -EINVAL;
@@ -331,8 +334,11 @@ static int clk_aic32x4_div_determine_rate(struct clk_hw *hw,
{
unsigned long divisor;
+ if (!req->rate)
+ return -EINVAL;
+
divisor = DIV_ROUND_UP(req->best_parent_rate, req->rate);
- if (divisor > AIC32X4_DIV_MAX)
+ if (!divisor || divisor > AIC32X4_DIV_MAX)
return -EINVAL;
req->rate = DIV_ROUND_UP(req->best_parent_rate, divisor);
diff --git a/sound/soc/codecs/wcd9335.c b/sound/soc/codecs/wcd9335.c
index 40de279a9750..a98d1fd78925 100644
--- a/sound/soc/codecs/wcd9335.c
+++ b/sound/soc/codecs/wcd9335.c
@@ -5096,6 +5096,13 @@ static int wcd9335_slim_probe(struct slim_device *slim)
return 0;
}
+static void wcd9335_put_device_action(void *data)
+{
+ struct device *dev = data;
+
+ put_device(dev);
+}
+
static int wcd9335_slim_status(struct slim_device *sdev,
enum slim_device_status status)
{
@@ -5120,28 +5127,40 @@ static int wcd9335_slim_status(struct slim_device *sdev,
return -EINVAL;
}
+ ret = devm_add_action_or_reset(dev, wcd9335_put_device_action,
+ &wcd->slim_ifc_dev->dev);
+ if (ret)
+ return ret;
+
slim_get_logical_addr(wcd->slim_ifc_dev);
wcd->regmap = regmap_init_slimbus(sdev, &wcd9335_regmap_config);
- if (IS_ERR(wcd->regmap))
+ if (IS_ERR(wcd->regmap)) {
+ put_device(&wcd->slim_ifc_dev->dev);
return dev_err_probe(dev, PTR_ERR(wcd->regmap),
"Failed to allocate slim register map\n");
+ }
wcd->if_regmap = regmap_init_slimbus(wcd->slim_ifc_dev,
&wcd9335_ifc_regmap_config);
- if (IS_ERR(wcd->if_regmap))
+ if (IS_ERR(wcd->if_regmap)) {
+ put_device(&wcd->slim_ifc_dev->dev);
return dev_err_probe(dev, PTR_ERR(wcd->if_regmap),
"Failed to allocate ifc register map\n");
+ }
ret = wcd9335_bring_up(wcd);
if (ret) {
dev_err(dev, "Failed to bringup WCD9335\n");
+ put_device(&wcd->slim_ifc_dev->dev);
return ret;
}
ret = wcd9335_irq_init(wcd);
- if (ret)
+ if (ret) {
+ put_device(&wcd->slim_ifc_dev->dev);
return ret;
+ }
wcd9335_probe(wcd);
diff --git a/sound/soc/codecs/wm8994.c b/sound/soc/codecs/wm8994.c
index 8bf58a6a8af7..58cfdd60d8ab 100644
--- a/sound/soc/codecs/wm8994.c
+++ b/sound/soc/codecs/wm8994.c
@@ -3532,6 +3532,7 @@ int wm8994_mic_detect(struct snd_soc_component *component, struct snd_soc_jack *
break;
default:
dev_warn(component->dev, "Invalid MICBIAS %d\n", micbias);
+ pm_runtime_put(component->dev);
return -EINVAL;
}
@@ -4340,6 +4341,7 @@ static int wm8994_component_probe(struct snd_soc_component *component)
ret = regmap_read(control->regmap, WM8994_GPIO_1, ®);
if (ret < 0) {
dev_err(component->dev, "Failed to read GPIO1 state: %d\n", ret);
+ pm_runtime_put(component->dev);
goto err_irq;
}
if ((reg & WM8994_GPN_FN_MASK) != WM8994_GP_FN_PIN_SPECIFIC) {
@@ -4352,6 +4354,7 @@ static int wm8994_component_probe(struct snd_soc_component *component)
ret = regmap_read(control->regmap, WM8994_GPIO_6, ®);
if (ret < 0) {
dev_err(component->dev, "Failed to read GPIO6 state: %d\n", ret);
+ pm_runtime_put(component->dev);
goto err_irq;
}
if ((reg & WM8994_GPN_FN_MASK) != WM8994_GP_FN_PIN_SPECIFIC) {
diff --git a/sound/soc/codecs/wsa883x.c b/sound/soc/codecs/wsa883x.c
index 442cf9f3eaac..ccd4404c1644 100644
--- a/sound/soc/codecs/wsa883x.c
+++ b/sound/soc/codecs/wsa883x.c
@@ -456,10 +456,11 @@
/*
* Device will report senseless data in many cases, so discard any measurements
- * outside of valid range.
+ * outside of the two-point trim range. Do not narrow this to ambient: a
+ * speaker that has been playing easily runs well above 45 degC.
*/
-#define WSA883X_LOW_TEMP_THRESHOLD 5
-#define WSA883X_HIGH_TEMP_THRESHOLD 45
+#define WSA883X_LOW_TEMP_THRESHOLD WSA883X_T1_TEMP
+#define WSA883X_HIGH_TEMP_THRESHOLD WSA883X_T2_TEMP
struct wsa883x_priv {
struct regmap *regmap;
@@ -1430,7 +1431,7 @@ static int wsa883x_get_temp(struct wsa883x_priv *wsa883x, long *temp)
unsigned int dmeas_msb = 0, dmeas_lsb = 0;
int d1, d2, dmeas;
unsigned int mask;
- int ret, range;
+ int ret;
long val;
guard(mutex)(&wsa883x->sp_lock);
@@ -1493,8 +1494,8 @@ static int wsa883x_get_temp(struct wsa883x_priv *wsa883x, long *temp)
}
val = WSA883X_T1_TEMP + (((dmeas - d1) * (WSA883X_T2_TEMP - WSA883X_T1_TEMP)) / (d2 - d1));
- range = WSA883X_HIGH_TEMP_THRESHOLD - WSA883X_LOW_TEMP_THRESHOLD;
- if (in_range(val, WSA883X_LOW_TEMP_THRESHOLD, range)) {
+ if (val >= WSA883X_LOW_TEMP_THRESHOLD &&
+ val <= WSA883X_HIGH_TEMP_THRESHOLD) {
wsa883x->temperature = val;
*temp = val * 1000;
ret = 0;
diff --git a/sound/soc/codecs/wsa884x.c b/sound/soc/codecs/wsa884x.c
index a367c94bfb4d..f0857c0588d5 100644
--- a/sound/soc/codecs/wsa884x.c
+++ b/sound/soc/codecs/wsa884x.c
@@ -720,10 +720,11 @@
/*
* Device will report senseless data in many cases, so discard any measurements
- * outside of valid range.
+ * outside of the two-point trim range. Do not narrow this to ambient: a
+ * speaker that has been playing easily runs well above 45 degC.
*/
-#define WSA884X_LOW_TEMP_THRESHOLD 5
-#define WSA884X_HIGH_TEMP_THRESHOLD 45
+#define WSA884X_LOW_TEMP_THRESHOLD WSA884X_T1_TEMP
+#define WSA884X_HIGH_TEMP_THRESHOLD WSA884X_T2_TEMP
struct wsa884x_priv {
struct regmap *regmap;
@@ -1929,8 +1930,8 @@ static int wsa884x_get_temp(struct wsa884x_priv *wsa884x, long *temp)
dev_dbg(wsa884x->dev, "Measured temp %ld (dmeas=%d, d1=%d, d2=%d)\n",
val, dmeas, d1, d2);
- if ((val > WSA884X_LOW_TEMP_THRESHOLD) &&
- (val < WSA884X_HIGH_TEMP_THRESHOLD)) {
+ if (val >= WSA884X_LOW_TEMP_THRESHOLD &&
+ val <= WSA884X_HIGH_TEMP_THRESHOLD) {
wsa884x->temperature = val;
*temp = val * 1000;
ret = 0;
diff --git a/sound/soc/fsl/fsl_xcvr.c b/sound/soc/fsl/fsl_xcvr.c
index 982827204351..29929dfcce2a 100644
--- a/sound/soc/fsl/fsl_xcvr.c
+++ b/sound/soc/fsl/fsl_xcvr.c
@@ -3,6 +3,7 @@
#include <linux/bitrev.h>
#include <linux/clk.h>
+#include <linux/devm-helpers.h>
#include <linux/firmware.h>
#include <linux/interrupt.h>
#include <linux/module.h>
@@ -57,6 +58,7 @@ struct fsl_xcvr {
struct snd_aes_iec958 tx_iec958;
u8 cap_ds[FSL_XCVR_CAPDS_SIZE];
struct work_struct work_rst;
+ int irq;
spinlock_t lock; /* Protect hw_reset and trigger */
struct snd_pcm_hw_constraint_list spdif_constr_rates;
u32 spdif_constr_rates_list[SPDIF_NUM_RATES];
@@ -1617,7 +1619,7 @@ static int fsl_xcvr_probe(struct platform_device *pdev)
struct fsl_xcvr *xcvr;
struct resource *rx_res, *tx_res;
void __iomem *regs;
- int ret, irq;
+ int ret;
xcvr = devm_kzalloc(dev, sizeof(*xcvr), GFP_KERNEL);
if (!xcvr)
@@ -1705,12 +1707,22 @@ static int fsl_xcvr_probe(struct platform_device *pdev)
return dev_err_probe(dev, PTR_ERR(xcvr->reset),
"failed to get XCVR reset control\n");
+ /*
+ * irq0_isr() schedules work_rst. Prepare the work and its lock
+ * before the IRQ, and register the cancel action first so a failed
+ * probe frees the IRQ and then cancels any queued work.
+ */
+ spin_lock_init(&xcvr->lock);
+ ret = devm_work_autocancel(dev, &xcvr->work_rst, reset_rx_work);
+ if (ret)
+ return ret;
+
/* get IRQs */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
+ xcvr->irq = platform_get_irq(pdev, 0);
+ if (xcvr->irq < 0)
+ return xcvr->irq;
- ret = devm_request_irq(dev, irq, irq0_isr, 0, pdev->name, xcvr);
+ ret = devm_request_irq(dev, xcvr->irq, irq0_isr, 0, pdev->name, xcvr);
if (ret)
return dev_err_probe(dev, ret, "failed to claim IRQ0\n");
@@ -1751,8 +1763,6 @@ static int fsl_xcvr_probe(struct platform_device *pdev)
fsl_xcvr_comp.name);
}
- INIT_WORK(&xcvr->work_rst, reset_rx_work);
- spin_lock_init(&xcvr->lock);
return ret;
}
@@ -1760,6 +1770,8 @@ static void fsl_xcvr_remove(struct platform_device *pdev)
{
struct fsl_xcvr *xcvr = dev_get_drvdata(&pdev->dev);
+ /* Free the IRQ first so irq0_isr() cannot requeue work_rst. */
+ devm_free_irq(&pdev->dev, xcvr->irq, xcvr);
cancel_work_sync(&xcvr->work_rst);
pm_runtime_disable(&pdev->dev);
}
diff --git a/sound/soc/intel/atom/sst/sst_acpi.c b/sound/soc/intel/atom/sst/sst_acpi.c
index 73624e1b138a..de1ad168008b 100644
--- a/sound/soc/intel/atom/sst/sst_acpi.c
+++ b/sound/soc/intel/atom/sst/sst_acpi.c
@@ -254,6 +254,11 @@ static int sst_platform_get_resources(struct intel_sst_drv *ctx)
return 0;
}
+static void sst_unregister_platform_device(void *data)
+{
+ platform_device_unregister(data);
+}
+
static int sst_acpi_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -342,7 +347,10 @@ static int sst_acpi_probe(struct platform_device *pdev)
pdata->platform);
return PTR_ERR(plat_dev);
}
-
+ ret = devm_add_action_or_reset(dev, sst_unregister_platform_device,
+ plat_dev);
+ if (ret)
+ return ret;
/*
* Create platform device for sst machine driver,
* pass machine info as pdata
@@ -355,6 +363,10 @@ static int sst_acpi_probe(struct platform_device *pdev)
return PTR_ERR(mdev);
}
+ ret = devm_add_action_or_reset(dev, sst_unregister_platform_device,
+ mdev);
+ if (ret)
+ return ret;
/* Fill sst platform data */
ctx->pdata = pdata;
strscpy(ctx->firmware_name, mach->fw_filename);
diff --git a/sound/soc/qcom/qdsp6/q6afe.c b/sound/soc/qcom/qdsp6/q6afe.c
index f0ff1350e9dd..9ac51c304ead 100644
--- a/sound/soc/qcom/qdsp6/q6afe.c
+++ b/sound/soc/qcom/qdsp6/q6afe.c
@@ -393,6 +393,7 @@ struct q6afe {
struct q6core_svc_api_info ainfo;
struct mutex lock;
struct aprv2_ibasic_rsp_result_t result;
+ uint32_t vote_result;
wait_queue_head_t wait;
struct list_head port_list;
spinlock_t port_list_lock;
@@ -1010,13 +1011,14 @@ static int q6afe_callback(struct apr_device *adev, const struct apr_resp_pkt *da
const struct aprv2_ibasic_rsp_result_t *res;
const struct apr_hdr *hdr = &data->hdr;
struct q6afe_port *port;
+ uint32_t *vote_res;
if (!data->payload_size)
return 0;
- res = data->payload;
switch (hdr->opcode) {
case APR_BASIC_RSP_RESULT: {
+ res = data->payload;
if (res->status) {
dev_err(afe->dev, "cmd = 0x%x returned error = 0x%x\n",
res->opcode, res->status);
@@ -1043,8 +1045,10 @@ static int q6afe_callback(struct apr_device *adev, const struct apr_resp_pkt *da
}
break;
case AFE_CMD_RSP_REMOTE_LPASS_CORE_HW_VOTE_REQUEST:
+ vote_res = data->payload;
afe->result.opcode = hdr->opcode;
- afe->result.status = res->status;
+ afe->result.status = 0;
+ afe->vote_result = *vote_res;
wake_up(&afe->wait);
break;
default:
@@ -1955,6 +1959,8 @@ int q6afe_vote_lpass_core_hw(struct device *dev, uint32_t hw_block_id,
AFE_CMD_RSP_REMOTE_LPASS_CORE_HW_VOTE_REQUEST);
if (ret)
dev_err(afe->dev, "AFE failed to vote (%d)\n", hw_block_id);
+ else
+ *client_handle = afe->vote_result;
return ret;
}
diff --git a/sound/soc/qcom/qdsp6/q6apm-dai.c b/sound/soc/qcom/qdsp6/q6apm-dai.c
index 033197bdd870..cac59c101586 100644
--- a/sound/soc/qcom/qdsp6/q6apm-dai.c
+++ b/sound/soc/qcom/qdsp6/q6apm-dai.c
@@ -498,11 +498,15 @@ static snd_pcm_uframes_t q6apm_dai_pointer(struct snd_soc_component *component,
ptr = bytes_to_frames(runtime, index);
return ptr;
}
- ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) * runtime->period_size;
- if (ptr)
- return ptr - 1;
+ /*
+ * The hw pointer counts the period buffers the DSP has returned in
+ * this lap, 1..periods. A full lap is the start of the ring again:
+ * report 0 rather than one frame less, so that the position of a
+ * drained buffer reaches appl_ptr and ALSA sees the underrun.
+ */
+ ptr = q6apm_get_hw_pointer(prtd->graph, substream->stream) % runtime->periods;
- return 0;
+ return ptr * runtime->period_size;
}
static int q6apm_dai_hw_params(struct snd_soc_component *component,
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index f167b9dae3fa..a79bbb34c777 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -283,6 +283,7 @@ int q6apm_alloc_fragments(struct q6apm_graph *graph, unsigned int dir, phys_addr
mutex_lock(&graph->lock);
data->dsp_buf = 0;
+ atomic_set(&data->hw_ptr, 0);
if (data->buf) {
mutex_unlock(&graph->lock);
@@ -319,15 +320,15 @@ int q6apm_alloc_fragments(struct q6apm_graph *graph, unsigned int dir, phys_addr
}
EXPORT_SYMBOL_GPL(q6apm_alloc_fragments);
-static int __q6apm_unmap_memory_fixed_region(struct device *dev, unsigned int graph_id,
+static int __q6apm_unmap_memory_fixed_region(struct q6apm *apm, unsigned int graph_id,
bool is_pos_buf)
{
struct apm_cmd_shared_mem_unmap_regions *cmd;
- struct q6apm *apm = dev_get_drvdata(dev->parent);
struct audioreach_graph_info *info;
uint32_t mem_map_handle;
+ uint32_t pos_mask = is_pos_buf ? APM_MMAP_TOKEN_MAP_TYPE_POS_BUF : 0;
struct gpr_pkt *pkt __free(kfree) = audioreach_alloc_apm_cmd_pkt(sizeof(*cmd),
- APM_CMD_SHARED_MEM_UNMAP_REGIONS, graph_id);
+ APM_CMD_SHARED_MEM_UNMAP_REGIONS, (graph_id | pos_mask));
if (IS_ERR(pkt))
return PTR_ERR(pkt);
@@ -354,13 +355,15 @@ static int __q6apm_unmap_memory_fixed_region(struct device *dev, unsigned int gr
int q6apm_unmap_memory_fixed_region(struct device *dev, unsigned int graph_id)
{
- return __q6apm_unmap_memory_fixed_region(dev, graph_id, false);
+ return __q6apm_unmap_memory_fixed_region(dev_get_drvdata(dev->parent),
+ graph_id, false);
}
EXPORT_SYMBOL_GPL(q6apm_unmap_memory_fixed_region);
int q6apm_unmap_pos_buffer(struct device *dev, unsigned int graph_id)
{
- return __q6apm_unmap_memory_fixed_region(dev, graph_id, true);
+ return __q6apm_unmap_memory_fixed_region(dev_get_drvdata(dev->parent),
+ graph_id, true);
}
EXPORT_SYMBOL_GPL(q6apm_unmap_pos_buffer);
@@ -843,6 +846,21 @@ static int q6apm_audio_probe(struct snd_soc_component *component)
static void q6apm_audio_remove(struct snd_soc_component *component)
{
+ struct q6apm *apm = dev_get_drvdata(component->dev);
+ struct audioreach_graph_info *info;
+ int id;
+
+ /*
+ * The card core removes components before freeing the PCMs, so
+ * q6apm-dai's pcm_free() runs after the topology (and graph_info)
+ * is gone and can no longer unmap. Release the DSP mappings here,
+ * otherwise the next pcm_new() fails to map the same buffer again.
+ */
+ idr_for_each_entry(&apm->graph_info_idr, info, id) {
+ __q6apm_unmap_memory_fixed_region(apm, id, false);
+ __q6apm_unmap_memory_fixed_region(apm, id, true);
+ }
+
/* remove topology */
snd_soc_tplg_component_remove(component);
}
@@ -952,17 +970,23 @@ static int apm_callback(const struct gpr_resp_pkt *data, void *priv, int op)
wake_up(&apm->wait);
break;
case APM_CMD_SHARED_MEM_UNMAP_REGIONS:
- apm->result.opcode = hdr->opcode;
- apm->result.status = 0;
- rsp = data->payload;
-
- info = idr_find(&apm->graph_info_idr, hdr->token);
- if (info)
- info->mem_map_handle = 0;
- else
+ apm->result.opcode = result->opcode;
+ apm->result.status = result->status;
+ if (result->status)
dev_err(dev, "Error (%d) Processing 0x%08x cmd\n", result->status,
result->opcode);
+ graph_id = hdr->token & APM_MMAP_TOKEN_GID_MASK;
+ is_pos_buf = hdr->token & APM_MMAP_TOKEN_MAP_TYPE_POS_BUF;
+
+ info = idr_find(&apm->graph_info_idr, graph_id);
+ if (info && !result->status) {
+ if (is_pos_buf)
+ info->pos_buf_mem_map_handle = 0;
+ else
+ info->mem_map_handle = 0;
+ }
+
wake_up(&apm->wait);
break;
default:
diff --git a/sound/soc/qcom/qdsp6/q6asm-dai.c b/sound/soc/qcom/qdsp6/q6asm-dai.c
index 4f09fdd40905..e3742663c13e 100644
--- a/sound/soc/qcom/qdsp6/q6asm-dai.c
+++ b/sound/soc/qcom/qdsp6/q6asm-dai.c
@@ -485,11 +485,10 @@ static snd_pcm_uframes_t q6asm_dai_pointer(struct snd_soc_component *component,
struct q6asm_dai_rtd *prtd = runtime->private_data;
snd_pcm_uframes_t ptr;
- ptr = q6asm_get_hw_pointer(prtd->audio_client, substream->stream) * runtime->period_size;
- if (ptr)
- return ptr - 1;
+ /* a full lap of period buffers is the start of the ring again */
+ ptr = q6asm_get_hw_pointer(prtd->audio_client, substream->stream) % runtime->periods;
- return 0;
+ return ptr * runtime->period_size;
}
static int q6asm_dai_hw_params(struct snd_soc_component *component,
diff --git a/sound/soc/qcom/qdsp6/topology.c b/sound/soc/qcom/qdsp6/topology.c
index faafbc5c0ad0..612d8f03e6e9 100644
--- a/sound/soc/qcom/qdsp6/topology.c
+++ b/sound/soc/qcom/qdsp6/topology.c
@@ -312,22 +312,37 @@ audioreach_get_module_priv_data(const struct snd_soc_tplg_private *private)
for (sz = 0; sz < le32_to_cpu(private->size); ) {
const struct snd_soc_tplg_vendor_array *mod_array;
+ u32 array_size;
mod_array = (struct snd_soc_tplg_vendor_array *)((u8 *)private->array + sz);
+ array_size = le32_to_cpu(mod_array->size);
+
if (le32_to_cpu(mod_array->type) == SND_SOC_AR_TPLG_MODULE_CFG_TYPE) {
struct audioreach_module_priv_data *pdata;
+ u32 nwords;
+
+ /* size is the byte count of the __le32 data[] */
+ nwords = array_size / sizeof(__le32);
+
+ /* the blob must lie within the private data */
+ if (sizeof(*pdata) + (u64)array_size >
+ le32_to_cpu(private->size) - sz)
+ return ERR_PTR(-EINVAL);
- pdata = kzalloc_flex(*pdata, data,
- le32_to_cpu(mod_array->size));
+ pdata = kzalloc_flex(*pdata, data, nwords);
if (!pdata)
return ERR_PTR(-ENOMEM);
- memcpy(pdata, ((u8 *)private->data + sz), struct_size(pdata, data,
- le32_to_cpu(mod_array->size)));
+ memcpy(pdata, ((u8 *)private->data + sz),
+ struct_size(pdata, data, nwords));
return pdata;
}
- sz = sz + le32_to_cpu(mod_array->size);
+ /* a zero size would never advance the walk */
+ if (!array_size)
+ return ERR_PTR(-EINVAL);
+
+ sz = sz + array_size;
}
return NULL;
@@ -611,6 +626,12 @@ static int audioreach_widget_load_module_common(struct snd_soc_component *compon
return mod ? PTR_ERR(mod) : -ENODEV;
mod->data = audioreach_get_module_priv_data(&tplg_w->priv);
+ if (IS_ERR(mod->data)) {
+ int rc = PTR_ERR(mod->data);
+
+ mod->data = NULL;
+ return rc;
+ }
dobj = &w->dobj;
dobj->private = mod;
diff --git a/sound/soc/sdca/sdca_class.c b/sound/soc/sdca/sdca_class.c
index d7444f442c71..e0e7aff70f93 100644
--- a/sound/soc/sdca/sdca_class.c
+++ b/sound/soc/sdca/sdca_class.c
@@ -27,8 +27,11 @@
static int class_read_prop(struct sdw_slave *sdw)
{
struct sdw_slave_prop *prop = &sdw->prop;
+ int ret;
- sdw_slave_read_prop(sdw);
+ ret = sdw_slave_read_prop(sdw);
+ if (ret)
+ return ret;
prop->use_domain_irq = true;
prop->scp_int1_mask = SDW_SCP_INT1_BUS_CLASH | SDW_SCP_INT1_PARITY |
@@ -178,8 +181,10 @@ static int class_sdw_probe(struct sdw_slave *sdw, const struct sdw_device_id *id
pm_runtime_get_noresume(dev);
ret = devm_pm_runtime_enable(dev);
- if (ret)
+ if (ret) {
+ pm_runtime_put_sync(dev);
return ret;
+ }
queue_work(system_long_wq, &drv->boot_work);
diff --git a/sound/soc/sdw_utils/soc_sdw_rt711.c b/sound/soc/sdw_utils/soc_sdw_rt711.c
index 3a3a66b4b737..4707190e160c 100644
--- a/sound/soc/sdw_utils/soc_sdw_rt711.c
+++ b/sound/soc/sdw_utils/soc_sdw_rt711.c
@@ -124,6 +124,7 @@ int asoc_sdw_rt711_exit(struct snd_soc_card *card, struct snd_soc_dai_link *dai_
device_remove_software_node(ctx->headset_codec_dev);
put_device(ctx->headset_codec_dev);
+ ctx->headset_codec_dev = NULL;
return 0;
}
diff --git a/sound/soc/stm/stm32_i2s.c b/sound/soc/stm/stm32_i2s.c
index 83b51893b37c..7fcf23bd0aff 100644
--- a/sound/soc/stm/stm32_i2s.c
+++ b/sound/soc/stm/stm32_i2s.c
@@ -1185,7 +1185,7 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
struct device_node *np = pdev->dev.of_node;
struct reset_control *rst;
struct resource *res;
- int irq, ret;
+ int ret;
if (!np)
return -ENODEV;
@@ -1231,16 +1231,6 @@ static int stm32_i2s_parse_dt(struct platform_device *pdev,
return ret;
}
- /* Get irqs */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
-
- ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
- dev_name(&pdev->dev), i2s);
- if (ret)
- return ret;
-
/* Reset */
rst = devm_reset_control_get_optional_exclusive(&pdev->dev, NULL);
if (IS_ERR(rst))
@@ -1265,7 +1255,7 @@ static int stm32_i2s_probe(struct platform_device *pdev)
{
struct stm32_i2s_data *i2s;
u32 val;
- int ret;
+ int irq, ret;
i2s = devm_kzalloc(&pdev->dev, sizeof(*i2s), GFP_KERNEL);
if (!i2s)
@@ -1291,6 +1281,15 @@ static int stm32_i2s_probe(struct platform_device *pdev)
return dev_err_probe(&pdev->dev, PTR_ERR(i2s->regmap),
"Regmap init error\n");
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ ret = devm_request_irq(&pdev->dev, irq, stm32_i2s_isr, 0,
+ dev_name(&pdev->dev), i2s);
+ if (ret)
+ return ret;
+
ret = snd_dmaengine_pcm_register(&pdev->dev, &stm32_i2s_pcm_config, 0);
if (ret)
return ret;
diff --git a/sound/soc/ti/davinci-mcasp.c b/sound/soc/ti/davinci-mcasp.c
index 79239131b659..d3c9797c03dd 100644
--- a/sound/soc/ti/davinci-mcasp.c
+++ b/sound/soc/ti/davinci-mcasp.c
@@ -799,6 +799,7 @@ static int __davinci_mcasp_set_clkdiv(struct davinci_mcasp *mcasp, int div_id,
break;
default:
+ pm_runtime_put(mcasp->dev);
return -EINVAL;
}
diff --git a/sound/synth/emux/emux.c b/sound/synth/emux/emux.c
index 450147b1abed..2a42d041e1da 100644
--- a/sound/synth/emux/emux.c
+++ b/sound/synth/emux/emux.c
@@ -71,7 +71,12 @@ static int sf_sample_free(void *private_data, struct snd_sf_sample *sp,
static void sf_sample_reset(void *private_data)
{
struct snd_emux *emu = private_data;
- emu->ops.sample_reset(emu);
+
+ /* stop all voices that might refer to zones and samples to be freed */
+ if (emu->ops.sample_reset)
+ emu->ops.sample_reset(emu);
+ else
+ snd_emux_terminate_all(emu);
}
int snd_emux_register(struct snd_emux *emu, struct snd_card *card, int index, char *name)
@@ -95,8 +100,7 @@ int snd_emux_register(struct snd_emux *emu, struct snd_card *card, int index, ch
sf_cb.private_data = emu;
sf_cb.sample_new = sf_sample_new;
sf_cb.sample_free = sf_sample_free;
- if (emu->ops.sample_reset)
- sf_cb.sample_reset = sf_sample_reset;
+ sf_cb.sample_reset = sf_sample_reset;
emu->sflist = snd_sf_new(&sf_cb, emu->memhdr);
if (emu->sflist == NULL)
return -ENOMEM;
diff --git a/sound/synth/emux/soundfont.c b/sound/synth/emux/soundfont.c
index 039357d96e07..42d1f33fb2b5 100644
--- a/sound/synth/emux/soundfont.c
+++ b/sound/synth/emux/soundfont.c
@@ -443,6 +443,15 @@ remove_info(struct snd_sf_list *sflist, struct snd_soundfont *sf,
struct snd_sf_zone *prev, *next, *p;
int removed = 0;
+ /* stop voices before freeing the zones they might refer to */
+ for (p = sf->zones; p; p = p->next) {
+ if (!p->mapped && p->bank == bank && p->instr == instr) {
+ if (sflist->callback.sample_reset)
+ sflist->callback.sample_reset(sflist->callback.private_data);
+ break;
+ }
+ }
+
prev = NULL;
for (p = sf->zones; p; p = next) {
next = p->next;
diff --git a/sound/usb/line6/capture.c b/sound/usb/line6/capture.c
index 6dbaf30232f9..ceee51eeda0d 100644
--- a/sound/usb/line6/capture.c
+++ b/sound/usb/line6/capture.c
@@ -125,24 +125,21 @@ void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf, int fsize)
line6pcm->in.pos_done -= runtime->buffer_size;
}
-void line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length)
+/* update the capture position and check whether we need period_elapsed call */
+bool line6_capture_check_period(struct snd_line6_pcm *line6pcm, int length)
{
- struct snd_pcm_substream *substream =
- get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE);
-
line6pcm->in.bytes += length;
if (line6pcm->in.bytes >= line6pcm->in.period) {
line6pcm->in.bytes %= line6pcm->in.period;
- spin_unlock(&line6pcm->in.lock);
- snd_pcm_period_elapsed(substream);
- spin_lock(&line6pcm->in.lock);
+ return true;
}
+ return false;
}
/*
* Callback for completed capture URB.
*/
-static void audio_in_callback(struct urb *urb)
+static bool handle_audio_in(struct urb *urb)
{
int i, index, length = 0, shutdown = 0;
struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context;
@@ -207,8 +204,18 @@ static void audio_in_callback(struct urb *urb)
if (!test_bit(LINE6_STREAM_IMPULSE, &line6pcm->in.running) &&
test_bit(LINE6_STREAM_PCM, &line6pcm->in.running))
- line6_capture_check_period(line6pcm, length);
+ return line6_capture_check_period(line6pcm, length);
}
+
+ return false;
+}
+
+static void audio_in_callback(struct urb *urb)
+{
+ struct snd_line6_pcm *line6pcm = (struct snd_line6_pcm *)urb->context;
+
+ if (handle_audio_in(urb))
+ snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE));
}
/* open capture callback */
diff --git a/sound/usb/line6/capture.h b/sound/usb/line6/capture.h
index 90572dae134e..628315389efc 100644
--- a/sound/usb/line6/capture.h
+++ b/sound/usb/line6/capture.h
@@ -17,7 +17,7 @@ extern const struct snd_pcm_ops snd_line6_capture_ops;
extern void line6_capture_copy(struct snd_line6_pcm *line6pcm, char *fbuf,
int fsize);
-extern void line6_capture_check_period(struct snd_line6_pcm *line6pcm,
+extern bool line6_capture_check_period(struct snd_line6_pcm *line6pcm,
int length);
extern int line6_create_audio_in_urbs(struct snd_line6_pcm *line6pcm);
extern int line6_submit_audio_in_all_urbs(struct snd_line6_pcm *line6pcm);
diff --git a/sound/usb/line6/playback.c b/sound/usb/line6/playback.c
index aa6ddf8746a0..e93a16f6958e 100644
--- a/sound/usb/line6/playback.c
+++ b/sound/usb/line6/playback.c
@@ -136,6 +136,8 @@ static void add_monitor_signal(struct urb *urb_out, unsigned char *signal,
/*
Find a free URB, prepare audio data, and submit URB.
must be called in line6pcm->out.lock context
+ Returns 1 if snd_pcm_period_elapsed() needs to be called for the
+ capture stream after releasing the lock, 0 if not, or a negative error
*/
static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
{
@@ -151,6 +153,7 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
line6pcm->properties->rates.rats[0].den *
(line6pcm->line6->intervals_per_second / LINE6_ISO_INTERVAL);
struct urb *urb_out;
+ bool capture_period_elapsed = false;
index = find_first_zero_bit(&line6pcm->out.active_urbs,
line6pcm->line6->iso_buffers);
@@ -260,8 +263,9 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
urb_out->transfer_buffer,
urb_out->
transfer_buffer_length);
- line6_capture_check_period(line6pcm,
- urb_out->transfer_buffer_length);
+ capture_period_elapsed =
+ line6_capture_check_period(line6pcm,
+ urb_out->transfer_buffer_length);
}
} else {
if (!(line6pcm->line6->properties->capabilities & LINE6_CAP_HWMON)
@@ -283,24 +287,26 @@ static int submit_audio_out_urb(struct snd_line6_pcm *line6pcm)
dev_err(line6pcm->line6->ifcdev,
"URB out #%d submission failed (%d)\n", index, ret);
- return 0;
+ return capture_period_elapsed;
}
/*
Submit all currently available playback URBs.
must be called in line6pcm->out.lock context
+ The capture period notification is skipped here since we can't call
+ snd_pcm_period_elapsed() in the lock; it'll be caught up at the next one.
*/
int line6_submit_audio_out_all_urbs(struct snd_line6_pcm *line6pcm)
{
- int ret = 0, i;
+ int ret, i;
for (i = 0; i < line6pcm->line6->iso_buffers; ++i) {
ret = submit_audio_out_urb(line6pcm);
if (ret < 0)
- break;
+ return ret;
}
- return ret;
+ return 0;
}
/*
@@ -316,6 +322,8 @@ static void audio_out_callback(struct urb *urb)
const int bytes_per_frame =
line6pcm->properties->bytes_per_channel *
line6pcm->properties->playback_hw.channels_max;
+ bool capture_period_elapsed = false;
+ bool playback_period_elapsed = false;
#if USE_CLEAR_BUFFER_WORKAROUND
memset(urb->transfer_buffer, 0, urb->transfer_buffer_length);
@@ -358,19 +366,22 @@ static void audio_out_callback(struct urb *urb)
shutdown = 1;
if (!shutdown) {
- submit_audio_out_urb(line6pcm);
+ capture_period_elapsed = submit_audio_out_urb(line6pcm) > 0;
if (test_bit(LINE6_STREAM_PCM, &line6pcm->out.running)) {
line6pcm->out.bytes += length;
if (line6pcm->out.bytes >= line6pcm->out.period) {
line6pcm->out.bytes %= line6pcm->out.period;
- spin_unlock(&line6pcm->out.lock);
- snd_pcm_period_elapsed(substream);
- spin_lock(&line6pcm->out.lock);
+ playback_period_elapsed = true;
}
}
}
spin_unlock_irqrestore(&line6pcm->out.lock, flags);
+
+ if (capture_period_elapsed)
+ snd_pcm_period_elapsed(get_substream(line6pcm, SNDRV_PCM_STREAM_CAPTURE));
+ if (playback_period_elapsed)
+ snd_pcm_period_elapsed(substream);
}
/* open playback callback */
diff --git a/sound/usb/misc/ua101.c b/sound/usb/misc/ua101.c
index 58d8a3cdc8b9..bd8e0ed0ba5d 100644
--- a/sound/usb/misc/ua101.c
+++ b/sound/usb/misc/ua101.c
@@ -285,13 +285,29 @@ static void playback_work(struct work_struct *work)
/* fill packet with data or silence */
urb->iso_frame_desc[0].length =
frames * ua->playback.frame_bytes;
- if (test_bit(ALSA_PLAYBACK_RUNNING, &ua->states))
+ if (test_bit(ALSA_PLAYBACK_RUNNING, &ua->states)) {
+ /*
+ * The packet size comes from the capture stream,
+ * i.e. from the device, while the ring buffer can
+ * be as small as two tiny periods. The single wrap
+ * in copy_playback_data() is only valid for
+ * frames <= buffer_size, so copy at most one
+ * buffer and send silence for the rest.
+ */
+ unsigned int copy_frames = min(frames,
+ ua->playback.substream->runtime->buffer_size);
+
do_period_elapsed |= copy_playback_data(&ua->playback,
urb,
- frames);
- else
+ copy_frames);
+ memset(urb->transfer_buffer +
+ copy_frames * ua->playback.frame_bytes, 0,
+ (frames - copy_frames) *
+ ua->playback.frame_bytes);
+ } else {
memset(urb->transfer_buffer, 0,
urb->iso_frame_desc[0].length);
+ }
/* and off you go ... */
err = usb_submit_urb(urb, GFP_ATOMIC);
@@ -320,6 +336,13 @@ static bool copy_capture_data(struct ua101_stream *stream, struct urb *urb,
runtime = stream->substream->runtime;
frame_bytes = stream->frame_bytes;
+ /*
+ * A packet can carry up to max_packet_bytes, the device's
+ * wMaxPacketSize, which may exceed the whole ring buffer; keep what
+ * fits so the single wrap below stays valid.
+ */
+ if (unlikely(frames > runtime->buffer_size))
+ frames = runtime->buffer_size;
dest = runtime->dma_area + stream->buffer_pos * frame_bytes;
if (stream->buffer_pos + frames <= runtime->buffer_size) {
memcpy(dest, urb->transfer_buffer, frames * frame_bytes);
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 682b6c1fe76b..c2f7b08607ac 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -1343,6 +1343,18 @@ static void retire_capture_urb(struct snd_usb_substream *subs,
bytes -= adj;
subs->stream_offset_adj -= adj;
}
+ /*
+ * A packet can be as large as the endpoint's maxpacksize, which
+ * the device chooses, while the ring buffer can be as small as
+ * two 64-byte periods. The hwptr_done wrap below handles one
+ * wrap only, so never take more than one buffer from a packet.
+ */
+ if (unlikely(bytes > subs->buffer_bytes)) {
+ dev_warn_ratelimited(&subs->dev->dev,
+ "%u-byte packet exceeds %u-byte buffer, truncating\n",
+ bytes, subs->buffer_bytes);
+ bytes = subs->buffer_bytes;
+ }
frames = bytes / stride;
if (!subs->txfr_quirk)
bytes = frames * stride;
@@ -1553,8 +1565,14 @@ static int prepare_playback_urb(struct snd_usb_substream *subs,
for (i = 0; i < ctx->packets; i++) {
counts = snd_usb_endpoint_next_packet_size(ep, ctx, i, avail);
+ /*
+ * Bound by the ring buffer as well as by the URB:
+ * copy_to_urb() wraps hwptr_done once, so one URB must
+ * never carry more than the whole buffer.
+ */
if (counts < 0 ||
- (frames + counts) * stride > ctx->buffer_size)
+ (frames + counts) * stride > ctx->buffer_size ||
+ (frames + counts) * stride > subs->buffer_bytes)
break;
/* set up descriptor */
urb->iso_frame_desc[i].offset = frames * stride;
diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
index dd02c6294edf..0426cbda314b 100644
--- a/sound/usb/quirks-table.h
+++ b/sound/usb/quirks-table.h
@@ -2719,7 +2719,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
.iface = 2,
.altsetting = 1,
.altset_idx = 1,
- .attributes = 0,
+ .attributes = UAC_EP_CS_ATTR_SAMPLE_RATE,
.endpoint = 0x01,
.ep_attr = USB_ENDPOINT_XFER_ISOC,
.rates = SNDRV_PCM_RATE_44100 |
@@ -2740,7 +2740,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
.iface = 3,
.altsetting = 1,
.altset_idx = 1,
- .attributes = 0,
+ .attributes = UAC_EP_CS_ATTR_SAMPLE_RATE,
.endpoint = 0x82,
.ep_attr = USB_ENDPOINT_XFER_ISOC,
.rates = SNDRV_PCM_RATE_44100 |
diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c
index a54fabce7920..b96cb3421cc1 100644
--- a/sound/usb/quirks.c
+++ b/sound/usb/quirks.c
@@ -2251,6 +2251,8 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
QUIRK_FLAG_DISABLE_AUTOSUSPEND),
DEVICE_FLG(0x045e, 0x070f, /* MS LifeChat LX-3000 Headset */
QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
+ DEVICE_FLG(0x046d, 0x0802, /* Logitech Webcam C200 */
+ QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384),
DEVICE_FLG(0x046d, 0x0807, /* Logitech Webcam C500 */
QUIRK_FLAG_CTL_MSG_DELAY_1M | QUIRK_FLAG_MIC_RES_384),
DEVICE_FLG(0x046d, 0x0808, /* Logitech Webcam C600 */
@@ -2460,6 +2462,8 @@ static const struct usb_audio_quirk_flags_table quirk_flags_table[] = {
QUIRK_FLAG_ITF_USB_DSD_DAC | QUIRK_FLAG_CTL_MSG_DELAY),
DEVICE_FLG(0x1901, 0x0191, /* GE B850V3 CP2114 audio interface */
QUIRK_FLAG_GET_SAMPLE_RATE),
+ DEVICE_FLG(0x194f, 0x0301, /* PreSonus AudioBox USB */
+ QUIRK_FLAG_GET_SAMPLE_RATE),
DEVICE_FLG(0x19f7, 0x0003, /* RODE NT-USB */
QUIRK_FLAG_MIXER_PLAYBACK_MIN_MUTE),
DEVICE_FLG(0x19f7, 0x0035, /* RODE NT-USB+ */
diff --git a/tools/dma/Makefile b/tools/dma/Makefile
index 0c239a6b9eae..20ae7f09cd20 100644
--- a/tools/dma/Makefile
+++ b/tools/dma/Makefile
@@ -7,7 +7,13 @@ bindir ?= /usr/bin
# isn't set and when invoked from selftests build, where srctree
# is set to ".". building_out_of_srctree is undefined for in srctree
# builds
+ifeq ($(srctree),)
+update_srctree := 1
+endif
ifndef building_out_of_srctree
+update_srctree := 1
+endif
+ifeq ($(update_srctree),1)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
srctree := $(patsubst %/,%,$(dir $(srctree)))
endif
diff --git a/tools/perf/arch/loongarch/util/header.c b/tools/perf/arch/loongarch/util/header.c
index 0c6d823334a2..46f53a74f71e 100644
--- a/tools/perf/arch/loongarch/util/header.c
+++ b/tools/perf/arch/loongarch/util/header.c
@@ -26,7 +26,7 @@
static char *_get_field(const char *line)
{
- char *line2, *nl;
+ const char *line2, *nl;
line2 = strrchr(line, ' ');
if (!line2)
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index f86b3b7df742..10a88c070814 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -2858,7 +2858,7 @@ static int thread__resolve_callchain_sample(struct thread *thread,
if (chain)
chain_nr = chain->nr;
- if (evsel__has_branch_callstack(evsel)) {
+ if (chain && evsel__has_branch_callstack(evsel)) {
struct perf_env *env = evsel__env(evsel);
err = resolve_lbr_callchain_sample(thread, cursor, sample, parent,
diff --git a/tools/perf/util/powerpc-vpadtl.c b/tools/perf/util/powerpc-vpadtl.c
index af6783cfdb53..032864e3884a 100644
--- a/tools/perf/util/powerpc-vpadtl.c
+++ b/tools/perf/util/powerpc-vpadtl.c
@@ -683,7 +683,7 @@ int powerpc_vpadtl_process_auxtrace_info(union perf_event *event,
struct perf_session *session)
{
struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info;
- size_t min_sz = sizeof(u64) * POWERPC_VPADTL_TYPE;
+ size_t min_sz = sizeof(u64) * (POWERPC_VPADTL_TYPE + 1);
struct powerpc_vpadtl *vpa;
int err;
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index d8a842621cda..c504b83be39f 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -3469,10 +3469,12 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)
perf_evlist__set_maps(evlist__core(evlist), cpus, threads);
if (parse_events(evlist, input, &err)) {
parse_events_error__print(&err, input);
+ parse_events_error__exit(&err);
PyErr_SetFromErrno(PyExc_OSError);
evlist__put(evlist);
return NULL;
}
+ parse_events_error__exit(&err);
result = pyrf_evlist__from_evlist(evlist);
evlist__put(evlist);
return result;
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 7efd5e645767..f7cc9da57b55 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -385,6 +385,9 @@ TEST_MATRIX=(
# Enabling partition with child cpusets is allowed
" C0-1 C1 . C2-3 P1 . . . 0 A1:0-1|A2:1 A1:P1"
+ # Tasks below a new partition belong to it, not to its parent.
+ " C1-2:P1 C1-2 C1-2:T . . P1 . . 0 A1:|A2:1-2|A3:1-2 A1:P1|A2:P1"
+
# A partition root with non-partition root parent is invalid| but it
# can be made valid if its parent becomes a partition root too.
" C0-1 C1 . C2-3 . P2 . . 0 A1:0-1|A2:1 A1:P0|A2:P-2"
diff --git a/tools/testing/selftests/core/unshare_test.c b/tools/testing/selftests/core/unshare_test.c
index ffce75a6c228..d40e963dd520 100644
--- a/tools/testing/selftests/core/unshare_test.c
+++ b/tools/testing/selftests/core/unshare_test.c
@@ -40,6 +40,14 @@ TEST(unshare_EMFILE)
ASSERT_EQ(sscanf(buf, "%d", &nr_open), 1);
+ /*
+ * Cap nr_open for the duration of the test to avoid ENOMEM from a
+ * huge fd table allocation; buf/n keep the real original value so
+ * fs.nr_open can be restored to it once the test is done.
+ */
+ if (nr_open > 1024 * 1024)
+ nr_open = 1024 * 1024;
+
ASSERT_EQ(0, getrlimit(RLIMIT_NOFILE, &rlimit));
/* bump fs.nr_open */
@@ -73,10 +81,13 @@ TEST(unshare_EMFILE)
if (pid == 0) {
int err;
+ char buf3[32];
+ ssize_t n3;
- /* restore fs.nr_open */
+ /* restore fs.nr_open to the (possibly capped) test baseline */
+ n3 = sprintf(buf3, "%d\n", nr_open);
lseek(fd, 0, SEEK_SET);
- write(fd, buf, n);
+ write(fd, buf3, n3);
/* ... and now unshare(CLONE_FILES) must fail with EMFILE */
err = unshare(CLONE_FILES);
EXPECT_EQ(err, -1)
@@ -89,6 +100,10 @@ TEST(unshare_EMFILE)
EXPECT_EQ(waitpid(pid, &status, 0), pid);
EXPECT_EQ(true, WIFEXITED(status));
EXPECT_EQ(0, WEXITSTATUS(status));
+
+ /* restore the real fs.nr_open value */
+ lseek(fd, 0, SEEK_SET);
+ write(fd, buf, n);
}
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/filesystems/idmapped_tmpfile.c b/tools/testing/selftests/filesystems/idmapped_tmpfile.c
index bc411ab8281e..77b96d094357 100644
--- a/tools/testing/selftests/filesystems/idmapped_tmpfile.c
+++ b/tools/testing/selftests/filesystems/idmapped_tmpfile.c
@@ -91,6 +91,13 @@ FIXTURE_SETUP(idmapped_tmpfile)
ASSERT_EQ(mkdir(self->dir, 0777), 0);
/* World-writable so an unmapped caller still passes permission(). */
ASSERT_EQ(chmod(self->dir, 0777), 0);
+ /*
+ * The directory itself must be inside the range the mount maps.
+ * Left owned by 0 it has no mapping in the mount, and
+ * inode_permission() then fails MAY_WRITE via HAS_UNMAPPED_ID()
+ * before any of the tmpfile logic is reached, regardless of mode.
+ */
+ ASSERT_EQ(chown(self->dir, MAP_HOST, MAP_HOST), 0);
}
FIXTURE_TEARDOWN(idmapped_tmpfile)
diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c
index d44b34b05757..22af28aef13b 100644
--- a/tools/testing/selftests/iommu/iommufd.c
+++ b/tools/testing/selftests/iommu/iommufd.c
@@ -2175,9 +2175,9 @@ FIXTURE_SETUP(iommufd_dirty_tracking)
test_ioctl_ioas_alloc(&self->ioas_id);
/*
- * For dirty testing it is important that the page size fed into
- * the iommu page tables matches the size the dirty logic
- * expects, or set_dirty can touch too much stuff.
+ * Disabling huge pages maps in PAGE_SIZE units, which can span
+ * multiple contiguous MOCK_PAGE_SIZE entries. The dirty tests must
+ * expect the entire mapping to be reported dirty.
*/
cmd.object_id = self->ioas_id;
if (!variant->hugepages)
@@ -2335,7 +2335,7 @@ TEST_F(iommufd_dirty_tracking, device_dirty_capability)
TEST_F(iommufd_dirty_tracking, get_dirty_bitmap)
{
- uint32_t page_size = MOCK_PAGE_SIZE;
+ uint32_t page_size = PAGE_SIZE;
uint32_t ioas_id = self->ioas_id;
uint32_t hwpt_id;
@@ -2377,7 +2377,7 @@ TEST_F(iommufd_dirty_tracking, get_dirty_bitmap)
TEST_F(iommufd_dirty_tracking, get_dirty_bitmap_no_clear)
{
- uint32_t page_size = MOCK_PAGE_SIZE;
+ uint32_t page_size = PAGE_SIZE;
uint32_t ioas_id = self->ioas_id;
uint32_t hwpt_id;
diff --git a/tools/testing/selftests/landlock/audit_test.c b/tools/testing/selftests/landlock/audit_test.c
index 72b5612375dd..4bea8c880a4d 100644
--- a/tools/testing/selftests/landlock/audit_test.c
+++ b/tools/testing/selftests/landlock/audit_test.c
@@ -31,7 +31,7 @@ static int matches_log_signal(struct __test_metadata *const _metadata,
log_match_len =
snprintf(log_match, sizeof(log_match), log_template, opid);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/landlock/net_test.c b/tools/testing/selftests/landlock/net_test.c
index 16afbfdf06bb..50b637354b51 100644
--- a/tools/testing/selftests/landlock/net_test.c
+++ b/tools/testing/selftests/landlock/net_test.c
@@ -2878,7 +2878,7 @@ static int matches_auditlog(const int audit_fd, const char *const blockers,
log_match_len = snprintf(log_match, sizeof(log_match),
log_with_addrport_tmpl, blockers,
dir_addr, addr, dir_port, port);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
@@ -3173,7 +3173,7 @@ static int matches_log_connect_bound(int audit_fd, const char *const blockers,
log_match_len = snprintf(log_match, sizeof(log_match), log_template,
blockers, addr, lport, addr, dport);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/landlock/ptrace_test.c b/tools/testing/selftests/landlock/ptrace_test.c
index 2644445a9d02..e0e7f95e889b 100644
--- a/tools/testing/selftests/landlock/ptrace_test.c
+++ b/tools/testing/selftests/landlock/ptrace_test.c
@@ -305,7 +305,7 @@ static int matches_log_ptrace(struct __test_metadata *const _metadata,
log_match_len =
snprintf(log_match, sizeof(log_match), log_template, opid);
- if (log_match_len > sizeof(log_match))
+ if (log_match_len >= sizeof(log_match))
return -E2BIG;
return audit_match_record(audit_fd, AUDIT_LANDLOCK_ACCESS, log_match,
diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c
index 0622bc2acad4..891383a161d0 100644
--- a/tools/testing/selftests/seccomp/seccomp_bpf.c
+++ b/tools/testing/selftests/seccomp/seccomp_bpf.c
@@ -4368,6 +4368,185 @@ TEST(user_notification_addfd_rlimit)
close(memfd);
}
+/* Just take a signal and do nothing else. */
+static void empty_handler(int signo)
+{
+}
+
+/*
+ * Verify that when SECCOMP_IOCTL_NOTIF_ADDFD with SECCOMP_ADDFD_FLAG_SEND is
+ * interrupted by a signal before the tracee dequeues the addfd request,
+ * knotif->state is restored from SECCOMP_NOTIFY_REPLIED back to
+ * SECCOMP_NOTIFY_SENT so that:
+ * 1. The woken tracee sees knotif->state == SECCOMP_NOTIFY_SENT in
+ * do_user_notif() and goes back to sleep instead of prematurely
+ * returning 0 from the trapped syscall without the FD installed.
+ * 2. The supervisor can retry SECCOMP_IOCTL_NOTIF_ADDFD (or
+ * SECCOMP_IOCTL_NOTIF_SEND) instead of failing with -EINPROGRESS.
+ *
+ * To deterministically hit the race window where the supervisor sleeps in
+ * wait_for_completion_interruptible(&kaddfd.completion) after waking the
+ * tracee (complete(&knotif->ready)) but before the tracee runs
+ * seccomp_handle_addfd(), pin all processes to a single CPU and enforce a
+ * strict 3-tier scheduling priority hierarchy on that CPU:
+ * - Supervisor: SCHED_FIFO priority 99 (highest)
+ * - Signal helper (sig_pid): SCHED_FIFO priority 50 (middle)
+ * - Tracee (pid): SCHED_IDLE (lowest)
+ */
+TEST(user_notification_addfd_send_interrupted)
+{
+ /*
+ * Save parent_pid before user_notif_syscall(__NR_getppid, ...) installs
+ * the seccomp filter on the calling process; children inherit that
+ * filter, so sig_pid must not call getppid().
+ */
+ pid_t tracee_pid, sig_pid, parent_pid = getpid();
+ long ret;
+ int status, listener, memfd, err;
+ struct seccomp_notif_addfd addfd = {};
+ struct seccomp_notif req = {};
+ struct sigaction sa = {};
+ struct sched_param sp_zero = { .sched_priority = 0 };
+ struct sched_param sp_fifo_high = { .sched_priority = 99 };
+ struct sched_param sp_fifo_mid = { .sched_priority = 50 };
+ struct timespec delay = { .tv_nsec = 15000000 };
+ cpu_set_t cpuset;
+ int cpu;
+
+ /* Pin the supervisor (and its future child processes) to one CPU. */
+ cpu = sched_getcpu();
+ if (cpu >= 0) {
+ CPU_ZERO(&cpuset);
+ CPU_SET(cpu, &cpuset);
+ sched_setaffinity(0, sizeof(cpuset), &cpuset);
+ }
+
+ sa.sa_handler = empty_handler;
+ ASSERT_EQ(sigaction(SIGUSR1, &sa, NULL), 0);
+
+ memfd = memfd_create("test", 0);
+ ASSERT_GE(memfd, 0);
+
+ ret = prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
+ ASSERT_EQ(0, ret) {
+ TH_LOG("Kernel does not support PR_SET_NO_NEW_PRIVS!");
+ }
+
+ /*
+ * Follow the convention of other user_notification_* tests in this
+ * file by trapping __NR_getppid: because the filter is installed on
+ * the supervisor before fork(), the trapped syscall must be a
+ * side-effect-free syscall that the supervisor itself never invokes.
+ * Even though getppid() does not normally return an FD,
+ * SECCOMP_ADDFD_FLAG_SEND replaces the trapped syscall's return value
+ * with the newly installed FD number (42).
+ */
+ listener = user_notif_syscall(__NR_getppid,
+ SECCOMP_FILTER_FLAG_NEW_LISTENER);
+ ASSERT_GE(listener, 0);
+
+ tracee_pid = fork();
+ ASSERT_GE(tracee_pid, 0);
+
+ if (tracee_pid == 0) {
+ /*
+ * Tracee: invoke __NR_getppid as a dummy trigger syscall to
+ * trap into do_user_notif(). Verify that the syscall returns
+ * the injected FD number (42) and that FD 42 is open.
+ */
+ ret = syscall(__NR_getppid);
+ exit(ret != 42 || fcntl(42, F_GETFD) < 0);
+ }
+
+ /* Wait for the tracee to trap in do_user_notif(). */
+ ASSERT_EQ(ioctl(listener, SECCOMP_IOCTL_NOTIF_RECV, &req), 0);
+
+ /*
+ * Demote the tracee to SCHED_IDLE and promote the supervisor to
+ * SCHED_FIFO(99) on the same CPU.
+ */
+ ASSERT_EQ(sched_setscheduler(tracee_pid, SCHED_IDLE, &sp_zero), 0);
+ if (sched_setscheduler(0, SCHED_FIFO, &sp_fifo_high) != 0) {
+ kill(tracee_pid, SIGKILL);
+ waitpid(tracee_pid, NULL, 0);
+ SKIP(return, "SCHED_FIFO requires CAP_SYS_NICE");
+ }
+
+ addfd.srcfd = memfd;
+ addfd.newfd_flags = O_CLOEXEC;
+ addfd.newfd = 42;
+ addfd.id = req.id;
+ addfd.flags = SECCOMP_ADDFD_FLAG_SETFD | SECCOMP_ADDFD_FLAG_SEND;
+
+ /*
+ * Fork a signal helper on the same CPU and set it to SCHED_FIFO(50).
+ * Because the supervisor is currently running at SCHED_FIFO(99) on
+ * this CPU, sig_pid is queued on the runqueue but cannot run until the
+ * supervisor blocks inside the kernel.
+ *
+ * When the supervisor invokes ioctl(SECCOMP_IOCTL_NOTIF_ADDFD) below:
+ * 1. seccomp_notify_addfd() sets knotif->state = SECCOMP_NOTIFY_REPLIED,
+ * wakes the tracee (SCHED_IDLE), and blocks in
+ * wait_for_completion_interruptible(&kaddfd.completion).
+ * 2. The CPU scheduler immediately runs sig_pid (SCHED_FIFO 50)
+ * ahead of the woken tracee (SCHED_IDLE).
+ * 3. sig_pid sends SIGUSR1 to parent_pid, waking the supervisor
+ * (SCHED_FIFO 99), which immediately preempts sig_pid, aborts the
+ * wait with -ERESTARTSYS (-EINTR), removes kaddfd from
+ * knotif->addfd, and restores knotif->state = SECCOMP_NOTIFY_SENT
+ * before the tracee has executed a single instruction.
+ */
+ sig_pid = fork();
+ if (sig_pid < 0) {
+ sched_setscheduler(0, SCHED_OTHER, &sp_zero);
+ kill(tracee_pid, SIGKILL);
+ waitpid(tracee_pid, NULL, 0);
+ }
+ ASSERT_GE(sig_pid, 0);
+ if (sig_pid == 0) {
+ if (sched_setscheduler(0, SCHED_FIFO, &sp_fifo_mid) != 0)
+ _exit(1);
+ kill(parent_pid, SIGUSR1);
+ _exit(0);
+ }
+
+ ret = ioctl(listener, SECCOMP_IOCTL_NOTIF_ADDFD, &addfd);
+ err = errno;
+
+ /*
+ * Restore normal scheduling ASAP so the supervisor does not remain
+ * SCHED_FIFO on a single-CPU machine/VM/CI, then sleep briefly so the
+ * woken tracee runs in do_user_notif(). With knotif->state restored to
+ * SECCOMP_NOTIFY_SENT, the tracee must loop back to sleep waiting for
+ * the notification reply rather than returning 0 from __NR_getppid.
+ */
+ ASSERT_EQ(sched_setscheduler(0, SCHED_OTHER, &sp_zero), 0);
+ ASSERT_EQ(sched_setscheduler(tracee_pid, SCHED_OTHER, &sp_zero), 0);
+
+ EXPECT_EQ(ret, -1);
+ EXPECT_EQ(err, EINTR);
+ EXPECT_EQ(waitpid(sig_pid, &status, 0), sig_pid);
+ EXPECT_EQ(true, WIFEXITED(status));
+ EXPECT_EQ(0, WEXITSTATUS(status));
+
+ nanosleep(&delay, NULL);
+
+ /*
+ * Retry SECCOMP_IOCTL_NOTIF_ADDFD. Because knotif->state is
+ * SECCOMP_NOTIFY_SENT, the retry succeeds (returns 42) instead of
+ * failing with -EINPROGRESS, installs FD 42 into the tracee, and wakes
+ * the tracee to complete the syscall with return value 42.
+ */
+ EXPECT_EQ(ioctl(listener, SECCOMP_IOCTL_NOTIF_ADDFD, &addfd), 42);
+
+ EXPECT_EQ(waitpid(tracee_pid, &status, 0), tracee_pid);
+ EXPECT_EQ(true, WIFEXITED(status));
+ EXPECT_EQ(0, WEXITSTATUS(status));
+
+ close(listener);
+ close(memfd);
+}
+
#ifndef SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP
#define SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP (1UL << 0)
#define SECCOMP_IOCTL_NOTIF_SET_FLAGS SECCOMP_IOW(4, __u64)
diff --git a/tools/testing/vma/include/stubs.h b/tools/testing/vma/include/stubs.h
index d6136e19a8af..ab441162ac10 100644
--- a/tools/testing/vma/include/stubs.h
+++ b/tools/testing/vma/include/stubs.h
@@ -356,6 +356,11 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma,
return true;
}
+static inline bool vma_has_uffd_without_event_remap(struct vm_area_struct *vma)
+{
+ return false;
+}
+
static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1,
struct anon_vma_name *anon_name2)
{
^ permalink raw reply [flat|nested] 2+ messages in thread