From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEBB55172EA; Thu, 15 Oct 2026 10:37:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792060700; cv=none; b=O2BchCU0VXLURgZQwgx16XVRDjC5bs5Otfuu2LRoKJ2dwoYDWdCNtnrUtGxlAPOLvJX2c90eUkZKL7Wd9HgesLmGsgK0NJoPhdypU2YoOU0uBU92wCGbMCKDGJxd+nJKGXGtevaTFWx1DZHw6atN/nohP1RjnZDteqHzON7TY1Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792060700; c=relaxed/simple; bh=YRcXMqf76dc1gUnFBNMYReL1BDMdmwYwWeSrskaTD9c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r10oU0yqa+TJgq79MMxRjy6xvcJCfEh4P6tne0z2o8DLMo811ZFLF9wcB8yZq+AkrYC9DpgbbxlFJwh7kG4RCMbVIoQ5R+LFOOT2wWmKaZRDYqUoKfaLhXeFewqtuNj51+7vcVnLJoOw9KUoExHae6ifeDkNYxpRmupAthGxD9M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=LTLe0VJ4; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="LTLe0VJ4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E47C21F000C5; Thu, 15 Oct 2026 10:37:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1792060679; bh=TX10mqmfZyAtB5rq3bql2eyOzXZS8HBoRqObT4LM4qA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=LTLe0VJ4lBHnjRYpKaCoHTHyiN8qbCYoDBKH3jcLLX5FqjEmHkyUE4AC3yFbwgqCU TVSI9yZspqtH1s5vswoEsEbKbqlCDBI0R88zRiUebMU7xtyH/zVv9fC+75nNXLT1DU 5BmvmjBcpfRxGJf90GkinsPkQolm2vWqFWQJngRc= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org, stable@vger.kernel.org Cc: lwn@lwn.net, jslaby@suse.cz, Greg Kroah-Hartman Subject: Re: Linux 7.2.11 Date: Thu, 15 Oct 2026 12:37:56 +0200 Message-ID: <2026101556-trifle-onboard-df3a@gregkh> X-Mailer: git-send-email 2.56.0 In-Reply-To: <2026101556-negate-tackle-1db4@gregkh> References: <2026101556-negate-tackle-1db4@gregkh> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit diff --git a/Documentation/networking/strparser.rst b/Documentation/networking/strparser.rst index 372106b61e65..d0804c88ad0a 100644 --- a/Documentation/networking/strparser.rst +++ b/Documentation/networking/strparser.rst @@ -77,7 +77,7 @@ Functions strp_done is called to release any resources held by the stream parser instance. This must be called after the stream processor - has been stopped. + has been stopped. The socket lock must not be held. :: diff --git a/Makefile b/Makefile index be413b167b24..4b60c83ab75c 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 VERSION = 7 PATCHLEVEL = 2 -SUBLEVEL = 10 +SUBLEVEL = 11 EXTRAVERSION = NAME = Baby Opossum Posse diff --git a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi index 1d73abffa6b7..6ead557a8cf1 100644 --- a/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi +++ b/arch/arm64/boot/dts/freescale/fsl-lx2160a.dtsi @@ -1783,7 +1783,7 @@ esdhc0_clksync_pins: iic5-sdhc-clk-pins { }; dspi2_miso_mosi_pins: iic5-spi3-pins { - pinctrl-single,bits = <0x3 (0x2 << 9) (0x7 << 9)>; + pinctrl-single,bits = <0x0 (0x3 << 9) (0x7 << 9)>; }; i2c5_pins: iic6-i2c-pins { @@ -1803,7 +1803,7 @@ fspi_data74_pins: xspi1-data74-pins { }; gpio1_31_28_pins: xspi1-data74-gpio-pins { - pinctrl-single,bits = <0x0 0x1 (0x7 << 15)>; + pinctrl-single,bits = <0x0 (0x1 << 15) (0x7 << 15)>; }; fspi_data30_pins: xspi1-data30-pins { @@ -1811,7 +1811,7 @@ fspi_data30_pins: xspi1-data30-pins { }; gpio1_27_24_pins: xspi1-data30-gpio-pins { - pinctrl-single,bits = <0x0 0x1 (0x7 << 18)>; + pinctrl-single,bits = <0x0 (0x1 << 18) (0x7 << 18)>; }; fspi_dqs_sck_cs10_pins: xspi1-base-pins { @@ -1819,7 +1819,7 @@ fspi_dqs_sck_cs10_pins: xspi1-base-pins { }; gpio1_23_20_pins: xspi1-base-gpio-pins { - pinctrl-single,bits = <0x0 0x1 (0x7 << 21)>; + pinctrl-single,bits = <0x0 (0x1 << 21) (0x7 << 21)>; }; esdhc0_cmd_data30_clk_vsel_pins: sdhc1-base-sdhc-vsel-pins { @@ -1868,12 +1868,19 @@ i2c6_i2c7_pins: iic8-iic7-i2c-pins { }; /* RCWSR14 */ + /* + * IIC1_PMUX is a single bit at RCWSR14[10], but bit-per-mux + * uses the mask for the field to clear before applying the + * value. + * So bits 11 and 12, which are reserved, are written as 0 + * on every mux switch, it is harmless. + */ i2c0_pins: iic1-i2c-pins { - pinctrl-single,bits = <0x8 0x0 (0x1 << 10)>; + pinctrl-single,bits = <0x8 0x0 (0x7 << 10)>; }; gpio0_3_2_pins: iic1-gpio-pins { - pinctrl-single,bits = <0x8 (0x1 << 10) (0x1 << 10)>; + pinctrl-single,bits = <0x8 (0x1 << 10) (0x7 << 10)>; }; }; diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 87560d8b254e..1da277baacf7 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -421,8 +421,9 @@ mov x2, xzr mrs x1, id_aa64dfr0_el1 ubfx x1, x1, #ID_AA64DFR0_EL1_PMUVer_SHIFT, #4 - cmp x1, #ID_AA64DFR0_EL1_PMUVer_V3P9 - b.lt .Lskip_pmuv3p9_\@ + cmp x1, #ID_AA64DFR0_EL1_PMUVer_IMP_DEF + ccmp x1, #ID_AA64DFR0_EL1_PMUVer_V3P9, #8, ne + b.lt .Lskip_pmuv3p9_\@ // Skip if < PMUv3p9 or IMP_DEF orr x0, x0, #HDFGRTR2_EL2_nPMICNTR_EL0 orr x0, x0, #HDFGRTR2_EL2_nPMICFILTR_EL0 diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index d2690c3ec528..ea66339435bf 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -25,6 +25,9 @@ KASAN_SANITIZE_stacktrace.o := n KCOV_INSTRUMENT_entry-common.o := n KCOV_INSTRUMENT_idle.o := n +# Softirq stack switching can run outside interrupt context. +KCOV_INSTRUMENT_irq.o := n + # Object file lists. obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ entry-common.o process.o ptrace.o \ diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index f7028caaeae0..e5166df510bb 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -450,9 +450,10 @@ config SMP config NR_CPUS int "Maximum number of CPUs (2-512)" depends on SMP - range 2 512 if !RISCV_SBI_V01 - range 2 32 if RISCV_SBI_V01 && 32BIT + range 2 512 if !RISCV_SBI_V01 && 64BIT + range 2 128 if !RISCV_SBI_V01 && 32BIT range 2 64 if RISCV_SBI_V01 && 64BIT + range 2 32 if RISCV_SBI_V01 && 32BIT default "32" if 32BIT default "64" if RISCV_SBI_V01 && 64BIT default "256" if !RISCV_SBI_V01 && 64BIT diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata index 3c945d086c7d..6121ba83cfa6 100644 --- a/arch/riscv/Kconfig.errata +++ b/arch/riscv/Kconfig.errata @@ -146,6 +146,7 @@ config ERRATA_THEAD_PMU config ERRATA_THEAD_GHOSTWRITE bool "Apply T-Head Ghostwrite errata" depends on ERRATA_THEAD && RISCV_ISA_XTHEADVECTOR + select RISCV_ALTERNATIVE_EARLY default y help The T-Head C9xx cores have a vulnerability in the xtheadvector diff --git a/arch/riscv/include/asm/futex.h b/arch/riscv/include/asm/futex.h index 90c86b115e00..6f3bc9bdb85e 100644 --- a/arch/riscv/include/asm/futex.h +++ b/arch/riscv/include/asm/futex.h @@ -40,6 +40,7 @@ arch_futex_atomic_op_inuser(int op, int oparg, int *oval, u32 __user *uaddr) if (!access_ok(uaddr, sizeof(u32))) return -EFAULT; + uaddr = untagged_addr(uaddr); switch (op) { case FUTEX_OP_SET: @@ -82,6 +83,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, if (!access_ok(uaddr, sizeof(u32))) return -EFAULT; + uaddr = untagged_addr(uaddr); __enable_user_access(); __asm__ __volatile__ ( diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h index 023a412fe38d..49e40eef4ed7 100644 --- a/arch/riscv/include/asm/scs.h +++ b/arch/riscv/include/asm/scs.h @@ -4,6 +4,7 @@ #ifdef __ASSEMBLER__ #include +#include #ifdef CONFIG_SHADOW_CALL_STACK @@ -22,9 +23,17 @@ REG_L gp, TASK_TI_SCS_SP(tp) .endm -/* Load task_scs_sp(current) to gp, but only if tp has changed. */ -.macro scs_load_current_if_task_changed prev - beq \prev, tp, _skip_scs +/* + * Load task_scs_sp(current) to gp, but only when the trap came from U-mode. + * The source privilege level is taken from the saved sstatus.SPP bit (written + * by hardware on trap entry), not by comparing tp. tp is a user-writable + * register, so gating the shadow call stack reload on it let a user task that + * set tp == ¤t skip the reload and run the kernel with an attacker + * controlled gp (the shadow call stack pointer). + */ +.macro scs_load_current_if_from_user status, tmp + andi \tmp, \status, SR_SPP + bnez \tmp, _skip_scs scs_load_current _skip_scs: .endm @@ -42,7 +51,7 @@ .endm .macro scs_load_current .endm -.macro scs_load_current_if_task_changed prev +.macro scs_load_current_if_from_user status, tmp .endm .macro scs_save_current .endm diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 00cb9c0982b1..8d736e826d84 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -132,8 +132,8 @@ static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) "csrr %0, " __stringify(CSR_VSTART) "\n\t" "csrr %1, " __stringify(CSR_VTYPE) "\n\t" "csrr %2, " __stringify(CSR_VL) "\n\t" - : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), - "=r" (dest->vcsr) : :); + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl) + : :); if (has_xtheadvector()) { unsigned long status; @@ -225,7 +225,7 @@ static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, "add %1, %1, %0\n\t" "vse8.v v24, (%1)\n\t" ".option pop\n\t" - : "=&r" (vl) : "r" (datap) : "memory"); + : "=&r" (vl), "+r" (datap) : : "memory"); } riscv_v_disable(); } @@ -261,7 +261,7 @@ static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_ "add %1, %1, %0\n\t" "vle8.v v24, (%1)\n\t" ".option pop\n\t" - : "=&r" (vl) : "r" (datap) : "memory"); + : "=&r" (vl), "+r" (datap) : : "memory"); } __vstate_csr_restore(restore_from); riscv_v_disable(); diff --git a/arch/riscv/include/uapi/asm/vendor/mips.h b/arch/riscv/include/uapi/asm/vendor/mips.h index e65ab268b265..a31c23afe74f 100644 --- a/arch/riscv/include/uapi/asm/vendor/mips.h +++ b/arch/riscv/include/uapi/asm/vendor/mips.h @@ -1,3 +1,5 @@ /* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ -#define RISCV_HWPROBE_VENDOR_EXT_XMIPSEXECTL BIT(0) +#include + +#define RISCV_HWPROBE_VENDOR_EXT_XMIPSEXECTL _BITULL(0) diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index d799c4e56f80..61b7a7716697 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -208,7 +208,7 @@ SYM_CODE_START(handle_exception) load_global_pointer /* Load the kernel shadow call stack pointer if coming from userspace */ - scs_load_current_if_task_changed s5 + scs_load_current_if_from_user s1, s5 #ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE move a0, sp diff --git a/arch/riscv/kernel/patch.c b/arch/riscv/kernel/patch.c index 16b243376f36..1e8f1ac5a755 100644 --- a/arch/riscv/kernel/patch.c +++ b/arch/riscv/kernel/patch.c @@ -46,7 +46,7 @@ static __always_inline void *patch_map(void *addr, const unsigned int fixmap) if (core_kernel_text(uintaddr) || is_kernel_exittext(uintaddr)) { phys = __pa_symbol(addr); - } else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) { + } else if (IS_ENABLED(CONFIG_EXECMEM)) { struct page *page = vmalloc_to_page(addr); BUG_ON(!page); diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c index f336a183667e..a0055743d3ba 100644 --- a/arch/riscv/kernel/ptrace.c +++ b/arch/riscv/kernel/ptrace.c @@ -230,6 +230,7 @@ static int riscv_vr_set(struct task_struct *target, return -ENODATA; /* Copy rest of the vstate except datap */ + memset(&ptrace_vstate, 0, sizeof(ptrace_vstate)); ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &ptrace_vstate, 0, sizeof(struct __riscv_v_regset_state)); if (unlikely(ret)) @@ -299,6 +300,9 @@ static int riscv_cfi_get(struct task_struct *target, struct user_cfi_state user_cfi; struct pt_regs *regs; + if (!is_user_lpad_enabled() && !is_user_shstk_enabled()) + return -EINVAL; + memset(&user_cfi, 0, sizeof(user_cfi)); regs = task_pt_regs(target); @@ -337,8 +341,12 @@ static int riscv_cfi_set(struct task_struct *target, struct user_cfi_state user_cfi; struct pt_regs *regs; + if (!is_user_lpad_enabled() && !is_user_shstk_enabled()) + return -EINVAL; + regs = task_pt_regs(target); + memset(&user_cfi, 0, sizeof(user_cfi)); ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &user_cfi, 0, -1); if (ret) return ret; diff --git a/arch/riscv/lib/strnlen.S b/arch/riscv/lib/strnlen.S index a8911605c248..5b3bbf0a3098 100644 --- a/arch/riscv/lib/strnlen.S +++ b/arch/riscv/lib/strnlen.S @@ -87,9 +87,25 @@ strnlen_zbb: * Aligned boundary. Use the address of the last valid byte * (s + count - 1) to avoid loading a word past the count * boundary in the loop below. count == 0 is handled above. + * + * Saturate the boundary when s + count would wrap around (very + * large counts, e.g. SIZE_MAX passed by FORTIFY strcat/strlcat + * with a destination whose size is unknown at compile time). + * Without this, the wrapped boundary lands before s and the + * pre-loop guard below always exits, returning a truncated + * length. + * + * Clamping the increment to ~s (== SIZE_MAX - s) keeps the + * computation branchless and wrap-free: + * + * s + min(count - 1, ~s) == saturating_add(s, count - 1) + * + * Saturating makes the scan equivalent to strlen(). */ - add t4, a0, a1 - addi t4, t4, -1 + addi t4, a1, -1 /* count - 1 */ + not t1, a0 /* SIZE_MAX - s */ + minu t4, t4, t1 /* clamp, so s + t4 can never wrap */ + add t4, a0, t4 /* saturated s + count - 1 */ andi t4, t4, -SZREG /* Get the first word. */ diff --git a/arch/riscv/mm/extable.c b/arch/riscv/mm/extable.c index dd1530af3ef1..e252eba55ede 100644 --- a/arch/riscv/mm/extable.c +++ b/arch/riscv/mm/extable.c @@ -68,8 +68,8 @@ ex_handler_load_unaligned_zeropad(const struct exception_table_entry *ex, addr = regs_get_gpr(regs, reg_addr * sizeof(unsigned long)); - offset = addr & 0x7UL; - addr &= ~0x7UL; + offset = addr & (sizeof(data) - 1); + addr &= ~(sizeof(data) - 1); data = *(unsigned long *)addr >> (offset * 8); diff --git a/arch/um/drivers/vector_transports.c b/arch/um/drivers/vector_transports.c index ddd127ee9678..e1fb2a76fdff 100644 --- a/arch/um/drivers/vector_transports.c +++ b/arch/um/drivers/vector_transports.c @@ -209,6 +209,7 @@ static int raw_verify_header( if ((vheader->flags & VIRTIO_NET_HDR_F_DATA_VALID) > 0) return 1; + skb_set_network_header(skb, ETH_HLEN); virtio_net_hdr_to_skb(skb, vheader, virtio_legacy_is_little_endian()); return 0; } diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 1b4a48bff18f..caca69a4e6d5 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -465,6 +465,8 @@ #define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* No Nested Data Breakpoints */ #define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1) /* WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */ #define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* LFENCE always serializing / synchronizes RDTSC */ + +#define X86_FEATURE_TLBSYNC_SFW_NO (20*32+ 4) /* TLBI Erratum #1718 fixed in microcode */ #define X86_FEATURE_VERW_CLEAR (20*32+ 5) /* The memory form of VERW mitigates TSA */ #define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* Null Selector Clears Base */ @@ -575,4 +577,5 @@ #define X86_BUG_TSA X86_BUG( 1*32+ 9) /* "tsa" CPU is affected by Transient Scheduler Attacks */ #define X86_BUG_VMSCAPE X86_BUG( 1*32+10) /* "vmscape" CPU is affected by VMSCAPE attacks from guests */ #define X86_BUG_SEAMRET_INVD_VMCS X86_BUG( 1*32+11) /* "seamret_invd_vmcs" SEAMRET from P-SEAMLDR clears the current VMCS */ +#define X86_BUG_TLBI X86_BUG( 1*32+12) /* "tlbi" AMD TLBI Erratum #1718 */ #endif /* _ASM_X86_CPUFEATURES_H */ diff --git a/arch/x86/include/asm/tlb.h b/arch/x86/include/asm/tlb.h index 866ea78ba156..eba925c85f52 100644 --- a/arch/x86/include/asm/tlb.h +++ b/arch/x86/include/asm/tlb.h @@ -110,6 +110,12 @@ static inline void __tlbsync(void) /* TLBSYNC: supported in binutils >= 0.36. */ asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory"); + + if (!static_cpu_has_bug(X86_BUG_TLBI)) + return; + + /* TLBSYNC: supported in binutils >= 0.36. */ + asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory"); } #else /* Some compilers (I'm looking at you clang!) simply can't do DCE */ diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index d61df70d6875..89e3280475a9 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -1047,6 +1047,10 @@ static void init_amd_zen5(struct cpuinfo_x86 *c) msr_clear_bit(MSR_AMD64_CPUID_FN_7, 18); pr_emerg_once("RDSEED32 is broken. Disabling the corresponding CPUID bit.\n"); } + + if (cpu_has(c, X86_FEATURE_INVLPGB) && + !cpu_has(c, X86_FEATURE_TLBSYNC_SFW_NO)) + set_cpu_bug(c, X86_BUG_TLBI); } static void init_amd(struct cpuinfo_x86 *c) diff --git a/drivers/android/binder/node.rs b/drivers/android/binder/node.rs index c10148e9069f..8f7aa867ce68 100644 --- a/drivers/android/binder/node.rs +++ b/drivers/android/binder/node.rs @@ -49,9 +49,9 @@ /// about to drop the weak reference, then the strong increment could be processed after the /// other thread has already exited, which would be too late. /// -/// Note that trying to create a `ListArc` to the node can succeed even if `has_normal_push` is +/// Note that trying to create a `ListArc` to the node can succeed even if `has_pushed_node` is /// set. This is because another thread might just have popped the node from a todo list, but not -/// yet called `do_work`. However, if `has_normal_push` is false, then creating a `ListArc` should +/// yet called `do_work`. However, if `has_pushed_node` is false, then creating a `ListArc` should /// always succeed. /// /// Like the other fields in `NodeInner`, the delivery state is protected by the process lock. @@ -733,7 +733,21 @@ fn do_work( self.do_work_locked(writer, owner_inner) } - fn cancel(self: DArc) {} + fn cancel(self: DArc) { + let _drop_outside_lock; + let mut owner_inner = self.owner.inner.lock(); + + // We only do something on BINDER_THREAD_EXIT, not process exit. + if owner_inner.is_dead { + return; + } + + // If BINDER_THREAD_EXIT is invoked on a thread with a pending node refcount update, we + // should move ourselves to ensure the refcount update is still delivered. + if let Some(node) = ListArc::try_from_arc_borrow(self.as_arc_borrow()) { + _drop_outside_lock = owner_inner.push_work(node); + } + } fn should_sync_wakeup(&self) -> bool { false diff --git a/drivers/android/binder/node/wrapper.rs b/drivers/android/binder/node/wrapper.rs index 43294c050502..aeda07ff7218 100644 --- a/drivers/android/binder/node/wrapper.rs +++ b/drivers/android/binder/node/wrapper.rs @@ -57,7 +57,39 @@ fn do_work( node.do_work_locked(writer, owner_inner) } - fn cancel(self: DArc) {} + fn cancel(self: DArc) { + let _drop_outside_lock; + let node = &self.node; + let mut owner_inner = node.owner.inner.lock(); + + // We only do something on BINDER_THREAD_EXIT, not process exit. + if owner_inner.is_dead { + return; + } + + // We transfer the responsibility of the node refcount update to the scheduled Node because + // NodeWrapper has no way to re-create the ListArc. + let inner = node.inner.access_mut(&mut owner_inner); + + let ds = &mut inner.delivery_state; + assert!(ds.has_pushed_wrapper); + assert!(ds.has_strong_zero2one); + ds.has_pushed_wrapper = false; + + // We are changing the state to one where the Node is the strong zero2one update instead of + // the wrapper. + ds.has_weak_zero2one = false; + + if !ds.has_pushed_node { + if let Some(node2) = ListArc::try_from_arc_borrow(node.as_arc_borrow()) { + ds.has_pushed_node = true; + _drop_outside_lock = owner_inner.push_work(node2); + } else { + // This can't actually happen. + ds.has_strong_zero2one = false; + } + } + } fn should_sync_wakeup(&self) -> bool { false diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c index 3559485b5034..d06062abca9c 100644 --- a/drivers/ata/libata-scsi.c +++ b/drivers/ata/libata-scsi.c @@ -3026,10 +3026,15 @@ static void atapi_qc_complete(struct ata_queued_cmd *qc) if (qc->cdb[0] == ALLOW_MEDIUM_REMOVAL && qc->dev->sdev) qc->dev->sdev->locked = 0; - if (cmd->result) - ata_scsi_qc_done(qc, false, 0); - else - ata_scsi_qc_done(qc, true, SAM_STAT_CHECK_CONDITION); + /* + * Report CHECK CONDITION unless the command was terminated + * with a host byte set. Only set the status byte, so that the + * SCSI midlayer internal byte, which scsi_check_sense() may + * have set from the sense data, is preserved. + */ + if (get_host_byte(cmd) == DID_OK) + set_status_byte(cmd, SAM_STAT_CHECK_CONDITION); + ata_scsi_qc_done(qc, false, 0); goto schedule_deferred; } diff --git a/drivers/block/zloop.c b/drivers/block/zloop.c index 1333fae05ac5..a011f3e518a5 100644 --- a/drivers/block/zloop.c +++ b/drivers/block/zloop.c @@ -799,6 +799,27 @@ static void zloop_cmd_workfn(struct work_struct *work) current->flags = orig_flags; } +static void zloop_fill_zero_rq(struct request *rq, unsigned int zero_from) +{ + struct req_iterator iter; + struct bio_vec bv; + unsigned int pos = 0; + unsigned int end, off; + + rq_for_each_segment(bv, rq, iter) { + end = pos + bv.bv_len; + if (end > zero_from) { + if (pos < zero_from) + off = zero_from - pos; + else + off = 0; + memzero_page(bv.bv_page, bv.bv_offset + off, + bv.bv_len - off); + } + pos = end; + } +} + static void zloop_complete_rq(struct request *rq) { struct zloop_cmd *cmd = blk_mq_rq_to_pdu(rq); @@ -813,13 +834,9 @@ static void zloop_complete_rq(struct request *rq) pr_err("Zone %u: failed read sector %llu, %llu sectors\n", zone_no, cmd->sector, cmd->nr_sectors); - if (cmd->ret >= 0 && cmd->ret != blk_rq_bytes(rq)) { - /* short read */ - struct bio *bio; - - __rq_for_each_bio(bio, rq) - zero_fill_bio(bio); - } + /* Short read. Zero out the unread tail beyond cmd->ret. */ + if (cmd->ret >= 0 && cmd->ret != blk_rq_bytes(rq)) + zloop_fill_zero_rq(rq, cmd->ret); break; case REQ_OP_WRITE: case REQ_OP_ZONE_APPEND: diff --git a/drivers/bluetooth/btintel_pcie.c b/drivers/bluetooth/btintel_pcie.c index fdc1fd2586b9..ed4ba2e469ac 100644 --- a/drivers/bluetooth/btintel_pcie.c +++ b/drivers/bluetooth/btintel_pcie.c @@ -401,7 +401,7 @@ static int btintel_pcie_send_sync(struct btintel_pcie_data *data, tfd_index = data->ia.tr_hia[BTINTEL_PCIE_TXQ_NUM]; - if (tfd_index > txq->count) + if (tfd_index >= txq->count) return -ERANGE; if (skb->len > BTINTEL_PCIE_BUFFER_SIZE - BTINTEL_PCIE_HCI_TYPE_LEN) { @@ -1715,7 +1715,7 @@ static irqreturn_t btintel_pcie_irq_msix_handler(int irq, void *dev_id) static int btintel_pcie_setup_irq(struct btintel_pcie_data *data) { int err; - int num_irqs, i; + int num_irqs, i, j; for (i = 0; i < BTINTEL_PCIE_MSIX_VEC_MAX; i++) data->msix_entries[i].entry = i; @@ -1744,6 +1744,10 @@ static int btintel_pcie_setup_irq(struct btintel_pcie_data *data) KBUILD_MODNAME, msix_entry); if (err) { + for (j = 0; j < i; j++) + devm_free_irq(&data->pdev->dev, + data->msix_entries[j].vector, + &data->msix_entries[j]); pci_free_irq_vectors(data->pdev); data->alloc_vecs = 0; return err; @@ -1815,6 +1819,13 @@ static int btintel_pcie_config_pcie(struct pci_dev *pdev, if (IS_ERR(data->base_addr)) return PTR_ERR(data->base_addr); + /* Do shared hardware reset to ensure a clean start before + * configuring interrupts. + */ + err = btintel_pcie_reset_bt(data); + if (err) + return err; + err = btintel_pcie_setup_irq(data); if (err) return err; @@ -3003,7 +3014,7 @@ static void btintel_pcie_remove(struct pci_dev *pdev) struct msix_entry *msix_entry; msix_entry = &data->msix_entries[i]; - free_irq(msix_entry->vector, msix_entry); + devm_free_irq(&pdev->dev, msix_entry->vector, msix_entry); } pci_free_irq_vectors(pdev); diff --git a/drivers/clk/qcom/gcc-eliza.c b/drivers/clk/qcom/gcc-eliza.c index 3e26c7a1e5b5..fe18942a59ac 100644 --- a/drivers/clk/qcom/gcc-eliza.c +++ b/drivers/clk/qcom/gcc-eliza.c @@ -3010,8 +3010,6 @@ static const u32 gcc_eliza_critical_cbcrs[] = { 0x26034, /* GCC_CAMERA_XO_CLK */ 0x27004, /* GCC_DISP_AHB_CLK */ 0x71004, /* GCC_GPU_CFG_AHB_CLK */ - 0x52010, /* GCC_PCIE_RSCC_CFG_AHB_CLK */ - 0x52010, /* GCC_PCIE_RSCC_XO_CLK */ 0x32004, /* GCC_VIDEO_AHB_CLK */ 0x32038, /* GCC_VIDEO_XO_CLK */ }; @@ -3045,6 +3043,10 @@ static const struct regmap_config gcc_eliza_regmap_config = { static void clk_eliza_regs_configure(struct device *dev, struct regmap *regmap) { + /* Keep clocks always enabled */ + regmap_update_bits(regmap, 0x52010, BIT(20), BIT(20)); /* GCC_PCIE_RSCC_CFG_AHB_CLK */ + regmap_update_bits(regmap, 0x52010, BIT(21), BIT(21)); /* GCC_PCIE_RSCC_XO_CLK */ + /* FORCE_MEM_CORE_ON for ufs phy ice core and gcc ufs phy axi clocks */ qcom_branch_set_force_mem_core(regmap, gcc_ufs_phy_ice_core_clk, true); qcom_branch_set_force_mem_core(regmap, gcc_ufs_phy_axi_clk, true); diff --git a/drivers/clk/qcom/gcc-kaanapali.c b/drivers/clk/qcom/gcc-kaanapali.c index 842c1a70c691..5bcbcd178657 100644 --- a/drivers/clk/qcom/gcc-kaanapali.c +++ b/drivers/clk/qcom/gcc-kaanapali.c @@ -3464,8 +3464,6 @@ static const u32 gcc_kaanapali_critical_cbcrs[] = { 0x9f004, /* GCC_EVA_AHB_CLK */ 0x9f024, /* GCC_EVA_XO_CLK */ 0x71004, /* GCC_GPU_CFG_AHB_CLK */ - 0x52010, /* GCC_PCIE_RSCC_CFG_AHB_CLK */ - 0x52010, /* GCC_PCIE_RSCC_XO_CLK */ 0x32004, /* GCC_VIDEO_AHB_CLK */ 0x32040, /* GCC_VIDEO_XO_CLK */ }; @@ -3480,6 +3478,10 @@ static const struct regmap_config gcc_kaanapali_regmap_config = { static void clk_kaanapali_regs_configure(struct device *dev, struct regmap *regmap) { + /* Keep clocks always enabled */ + regmap_update_bits(regmap, 0x52010, BIT(20), BIT(20)); /* GCC_PCIE_RSCC_CFG_AHB_CLK */ + regmap_update_bits(regmap, 0x52010, BIT(21), BIT(21)); /* GCC_PCIE_RSCC_XO_CLK */ + /* FORCE_MEM_CORE_ON for ufs phy ice core clocks */ qcom_branch_set_force_mem_core(regmap, gcc_ufs_phy_ice_core_clk, true); } diff --git a/drivers/clk/qcom/gpucc-glymur.c b/drivers/clk/qcom/gpucc-glymur.c index 001b2454786a..cac51de20b29 100644 --- a/drivers/clk/qcom/gpucc-glymur.c +++ b/drivers/clk/qcom/gpucc-glymur.c @@ -510,7 +510,7 @@ static struct gdsc gpu_cc_cx_gdsc = { .name = "gpu_cc_cx_gdsc", }, .pwrsts = PWRSTS_OFF_ON, - .flags = POLL_CFG_GDSCR | RETAIN_FF_ENABLE, + .flags = RETAIN_FF_ENABLE | VOTABLE, }; static struct clk_regmap *gpu_cc_glymur_clocks[] = { diff --git a/drivers/clk/qcom/gpucc-kaanapali.c b/drivers/clk/qcom/gpucc-kaanapali.c index ae5563e516f6..f75f2caa2205 100644 --- a/drivers/clk/qcom/gpucc-kaanapali.c +++ b/drivers/clk/qcom/gpucc-kaanapali.c @@ -380,7 +380,7 @@ static struct gdsc gpu_cc_cx_gdsc = { .name = "gpu_cc_cx_gdsc", }, .pwrsts = PWRSTS_OFF_ON, - .flags = POLL_CFG_GDSCR | RETAIN_FF_ENABLE, + .flags = RETAIN_FF_ENABLE | VOTABLE, }; static struct clk_regmap *gpu_cc_kaanapali_clocks[] = { diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c index 475235c402e8..0c0d11c72925 100644 --- a/drivers/edac/amd64_edac.c +++ b/drivers/edac/amd64_edac.c @@ -2796,13 +2796,13 @@ static inline void decode_bus_error(int node_id, struct mce *m) * the instance_id. For example, instance_id=0xYXXXXX where Y is the channel * number. * - * For DRAM ECC errors, the Chip Select number is given in bits [2:0] of + * For DRAM ECC errors, the Chip Select number is given in bits [1:0] of * the MCA_SYND[ErrorInformation] field. */ static void umc_get_err_info(struct mce *m, struct err_info *err) { err->channel = (m->ipid & GENMASK(31, 0)) >> 20; - err->csrow = m->synd & 0x7; + err->csrow = m->synd & 0x3; } static void decode_umc_error(int node_id, struct mce *m) diff --git a/drivers/gpio/gpio-exar.c b/drivers/gpio/gpio-exar.c index 9053662f1817..f629cc526f3e 100644 --- a/drivers/gpio/gpio-exar.c +++ b/drivers/gpio/gpio-exar.c @@ -199,6 +199,7 @@ static int gpio_exar_probe(struct platform_device *pdev) index = ida_alloc(&ida_index, GFP_KERNEL); if (index < 0) return index; + exar_gpio->index = index; ret = devm_add_action_or_reset(dev, exar_devm_ida_free, exar_gpio); if (ret) @@ -214,7 +215,6 @@ static int gpio_exar_probe(struct platform_device *pdev) exar_gpio->gpio_chip.set = exar_set_value; exar_gpio->gpio_chip.base = -1; exar_gpio->gpio_chip.ngpio = ngpios; - exar_gpio->index = index; exar_gpio->first_pin = first_pin; ret = devm_gpiochip_add_data(dev, &exar_gpio->gpio_chip, exar_gpio); diff --git a/drivers/gpio/gpio-mpsse.c b/drivers/gpio/gpio-mpsse.c index a859deab2bca..5ce822c0ddcf 100644 --- a/drivers/gpio/gpio-mpsse.c +++ b/drivers/gpio/gpio-mpsse.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -24,6 +25,7 @@ struct mpsse_priv { raw_spinlock_t irq_spin; /* protects worker list */ atomic_t irq_type[16]; /* pin -> edge detection type */ atomic_t irq_enabled; + atomic_t dying; /* no new workers after disconnect */ int id; u8 gpio_outputs[2]; /* Output states for GPIOs [L, H] */ @@ -46,6 +48,7 @@ struct mpsse_worker { atomic_t cancelled; struct list_head list; /* linked list */ struct list_head destroy; /* teardown linked list */ + struct rcu_head rcu; /* deferred free for dying path */ }; struct bulk_desc { @@ -525,10 +528,24 @@ static void gpio_mpsse_irq_enable(struct irq_data *irqd) worker->priv = priv; INIT_LIST_HEAD(&worker->list); INIT_WORK(&worker->work, gpio_mpsse_poll); - schedule_work(&worker->work); - scoped_guard(raw_spinlock_irqsave, &priv->irq_spin) + scoped_guard(raw_spinlock_irqsave, &priv->irq_spin) { + if (atomic_read(&priv->dying)) { + /* + * Cannot kfree() here: this callback + * runs with the IRQ descriptor's raw + * spinlock held and kfree() may sleep + * on PREEMPT_RT. The worker is not + * yet published, so the deferred free + * is unobservable. + */ + kfree_rcu(worker, rcu); + return; + } + list_add(&worker->list, &priv->workers); + schedule_work(&worker->work); + } } } @@ -704,6 +721,9 @@ static void gpio_mpsse_disconnect(struct usb_interface *intf) { struct mpsse_priv *priv = usb_get_intfdata(intf); + scoped_guard(raw_spinlock_irqsave, &priv->irq_spin) + atomic_set(&priv->dying, 1); + /* * Lock prevents double-free of worker from here and the teardown * step at the beginning of gpio_mpsse_poll diff --git a/drivers/gpio/gpio-xilinx.c b/drivers/gpio/gpio-xilinx.c index 532205175827..9406ce85f933 100644 --- a/drivers/gpio/gpio-xilinx.c +++ b/drivers/gpio/gpio-xilinx.c @@ -271,14 +271,7 @@ static void xgpio_save_regs(struct xgpio_instance *chip) static int xgpio_request(struct gpio_chip *chip, unsigned int offset) { - int ret; - - ret = pm_runtime_get_sync(chip->parent); - /* - * If the device is already active pm_runtime_get() will return 1 on - * success, but gpio_request still needs to return 0. - */ - return ret < 0 ? ret : 0; + return pm_runtime_resume_and_get(chip->parent); } static void xgpio_free(struct gpio_chip *chip, unsigned int offset) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 4b29e7eee610..299baecb83eb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -597,14 +597,18 @@ int amdgpu_amdkfd_get_dmabuf_info(struct amdgpu_device *adev, int dma_buf_fd, /* first get metadata_size by buffer = NULL */ r = amdgpu_bo_get_metadata(bo, NULL, 0, metadata_size, NULL); + if (r) + goto out_put; /* user buf_size is bigger than bo metadata_size * allocate a buf at kernel space and copy */ if (*metadata_size <= buffer_size) { *metadata_buffer = kzalloc(*metadata_size, GFP_KERNEL); - if (!*metadata_buffer) - return -ENOMEM; + if (!*metadata_buffer) { + r = -ENOMEM; + goto out_put; + } r = amdgpu_bo_get_metadata(bo, *metadata_buffer, *metadata_size, NULL, &metadata_flags); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c index 4536d46b7f21..40b34770414b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c @@ -120,6 +120,20 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, *pos &= (1UL << 22) - 1; + if (size > PAGE_SIZE) + return -EINVAL; + + /* + * Access the user buffer only while none of the locks below are held: + * a fault takes mmap_lock, and grbm_idx_mutex and srbm_mutex nest + * inside it. + */ + u32 *data __free(kfree) = kmalloc(size, GFP_KERNEL); + if (!data) + return -ENOMEM; + if (!read && copy_from_user(data, buf, size)) + return -EFAULT; + r = pm_runtime_get_sync(adev_to_drm(adev)->dev); if (r < 0) { pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); @@ -151,28 +165,16 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, mutex_lock(&adev->pm.mutex); while (size) { - uint32_t value; - - if (read) { - value = RREG32(*pos >> 2); - r = put_user(value, (uint32_t *)buf); - } else { - r = get_user(value, (uint32_t *)buf); - if (!r) - amdgpu_mm_wreg_mmio_rlc(adev, *pos >> 2, value, 0); - } - if (r) { - result = r; - goto end; - } + if (read) + data[result >> 2] = RREG32(*pos >> 2); + else + amdgpu_mm_wreg_mmio_rlc(adev, *pos >> 2, data[result >> 2], 0); result += 4; - buf += 4; *pos += 4; size -= 4; } -end: if (use_bank) { amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff, 0); mutex_unlock(&adev->grbm_idx_mutex); @@ -187,6 +189,10 @@ static int amdgpu_debugfs_process_reg_op(bool read, struct file *f, pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); amdgpu_virt_disable_access_debugfs(adev); + + if (read && copy_to_user(buf, data, result)) + return -EFAULT; + return result; } @@ -237,11 +243,24 @@ static ssize_t amdgpu_debugfs_regs2_op(struct file *f, char __user *buf, u32 off struct amdgpu_device *adev = rd->adev; ssize_t result = 0; int r; - uint32_t value; if (size & 0x3 || offset & 0x3) return -EINVAL; + if (size > PAGE_SIZE) + return -EINVAL; + + /* + * Access the user buffer only while none of the locks below are held: + * a fault takes mmap_lock, and grbm_idx_mutex and srbm_mutex nest + * inside it. + */ + u32 *data __free(kfree) = kmalloc(size, GFP_KERNEL); + if (!data) + return -ENOMEM; + if (write_en && copy_from_user(data, buf, size)) + return -EFAULT; + r = pm_runtime_get_sync(adev_to_drm(adev)->dev); if (r < 0) { pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); @@ -280,24 +299,16 @@ static ssize_t amdgpu_debugfs_regs2_op(struct file *f, char __user *buf, u32 off mutex_lock(&adev->pm.mutex); while (size) { - if (!write_en) { - value = RREG32(offset >> 2); - r = put_user(value, (uint32_t *)buf); - } else { - r = get_user(value, (uint32_t *)buf); - if (!r) - amdgpu_mm_wreg_mmio_rlc(adev, offset >> 2, value, rd->id.xcc_id); - } - if (r) { - result = r; - goto end; - } + if (!write_en) + data[result >> 2] = RREG32(offset >> 2); + else + amdgpu_mm_wreg_mmio_rlc(adev, offset >> 2, data[result >> 2], + rd->id.xcc_id); offset += 4; size -= 4; result += 4; - buf += 4; } -end: + if (rd->id.use_grbm) { amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff, rd->id.xcc_id); mutex_unlock(&adev->grbm_idx_mutex); @@ -316,6 +327,10 @@ static ssize_t amdgpu_debugfs_regs2_op(struct file *f, char __user *buf, u32 off pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); amdgpu_virt_disable_access_debugfs(adev); + + if (!write_en && copy_to_user(buf, data, result)) + return -EFAULT; + return result; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 5d6149ba7ab7..8fb3ebe4c299 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -1089,20 +1089,17 @@ void amdgpu_gmc_init_vga_resv_regions(struct amdgpu_device *adev) case CHIP_RENOIR: adev->mman.keep_stolen_vga_memory = true; break; - case CHIP_POLARIS10: - case CHIP_POLARIS11: - case CHIP_POLARIS12: - /* MacBookPros with switchable graphics put VRAM at 0 when - * the iGPU is enabled which results in cursor issues if - * the cursor ends up at 0. Reserve vram at 0 in that case. - */ - if (adev->gmc.vram_start == 0) - adev->mman.keep_stolen_vga_memory = true; - break; default: adev->mman.keep_stolen_vga_memory = false; break; } + /* MacBookPros with switchable graphics put VRAM at 0 when + * the iGPU is enabled which results in cursor issues if + * the cursor ends up at 0. Reserve vram at 0 in that case. + */ + if ((adev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE) && + (adev->gmc.vram_start == 0)) + adev->mman.keep_stolen_vga_memory = true; if (amdgpu_sriov_vf(adev) || !amdgpu_device_has_display_hardware(adev)) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 8bb9c198ddec..872cce1115a9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -1631,6 +1632,9 @@ u32 amdgpu_get_vblank_counter_kms(struct drm_crtc *crtc) DRM_ERROR("Invalid crtc %u\n", pipe); return -EINVAL; } + if (!down_read_trylock(&adev->reset_domain->sem)) + return drm_crtc_vblank_count(crtc); + /* The hw increments its frame counter at start of vsync, not at start * of vblank, as is required by DRM core vblank counter handling. @@ -1676,6 +1680,7 @@ u32 amdgpu_get_vblank_counter_kms(struct drm_crtc *crtc) DRM_DEBUG_VBL("NULL mode info! Returned count may be wrong.\n"); } + up_read(&adev->reset_domain->sem); return count; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c index 655429c40351..0c3fa2d60555 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c @@ -439,6 +439,7 @@ amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, u64 doorbell_index; struct drm_gem_object *gobj; struct amdgpu_userq_obj *db_obj = db_info->db_obj; + struct amdgpu_bo *abo; int r, db_size; gobj = drm_gem_object_lookup(filp, db_info->doorbell_handle); @@ -447,7 +448,17 @@ amdgpu_userq_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, return -EINVAL; } - db_obj->obj = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj)); + /* + * Pinning a regular BO into the doorbell domain would discard its + * contents, possibly those of a buffer shared by another client. + */ + abo = gem_to_amdgpu_bo(gobj); + if (!(abo->preferred_domains & AMDGPU_GEM_DOMAIN_DOORBELL)) { + drm_gem_object_put(gobj); + return -EINVAL; + } + + db_obj->obj = amdgpu_bo_ref(abo); drm_gem_object_put(gobj); r = amdgpu_bo_reserve(db_obj->obj, true); diff --git a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c index 4e44a581a78a..e4bd6e5509b5 100644 --- a/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +++ b/drivers/gpu/drm/amd/amdgpu/mes_userqueue.c @@ -304,8 +304,7 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, compute_mqd = memdup_user(u64_to_user_ptr(mqd_user->mqd), mqd_user->mqd_size); if (IS_ERR(compute_mqd)) { - DRM_ERROR("Failed to read user MQD\n"); - r = -ENOMEM; + r = PTR_ERR(compute_mqd); goto free_mqd; } @@ -349,8 +348,7 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, mqd_gfx_v11 = memdup_user(u64_to_user_ptr(mqd_user->mqd), mqd_user->mqd_size); if (IS_ERR(mqd_gfx_v11)) { - DRM_ERROR("Failed to read user MQD\n"); - r = -ENOMEM; + r = PTR_ERR(mqd_gfx_v11); goto free_mqd; } @@ -394,8 +392,7 @@ static int mes_userq_mqd_create(struct amdgpu_usermode_queue *queue, mqd_sdma_v11 = memdup_user(u64_to_user_ptr(mqd_user->mqd), mqd_user->mqd_size); if (IS_ERR(mqd_sdma_v11)) { - DRM_ERROR("Failed to read sdma user MQD\n"); - r = -ENOMEM; + r = PTR_ERR(mqd_sdma_v11); goto free_mqd; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c index 487cf746c450..ed9344259079 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c @@ -599,12 +599,32 @@ static int allocate_doorbell(struct qcm_process_device *qpd, * we need the physical sdma engine id in order to get the * correct doorbell offset. */ - uint32_t valid_id = idx_offset[qpd->dqm->dev->node_id * - get_num_all_sdma_engines(qpd->dqm) + - q->properties.sdma_engine_id] - + (q->properties.sdma_queue_id & 1) - * KFD_QUEUE_DOORBELL_MIRROR_OFFSET - + (q->properties.sdma_queue_id >> 1); + u32 engine_base = idx_offset[dev->node_id * + get_num_all_sdma_engines(qpd->dqm) + + q->properties.sdma_engine_id]; + u32 valid_id; + + /* + * With the aqua_vanjaram doorbell layout, a shader doorbell + * write whose 32-byte block starts in the previous engine's + * range rings that engine instead. Engine ranges are 10 + * indices apart (sdma_doorbell_range = 20 dwords, halved for + * 8-byte doorbells, in aqua_vanjaram_doorbell_index_init()), + * which puts odd engines 2 indices into a shared block. + * Aligning costs at most 2 indices, and 8 queues per engine use + * only 4 of the 10, so the aligned base stays in range. The + * 512-index mirror offset keeps the alignment. + */ + if (KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 3) || + KFD_GC_VERSION(dev) == IP_VERSION(9, 4, 4) || + KFD_GC_VERSION(dev) == IP_VERSION(9, 5, 0)) + engine_base = ALIGN(engine_base, + KFD_SDMA_SHADER_DOORBELL_GRANULARITY); + + valid_id = engine_base + + (q->properties.sdma_queue_id & 1) * + KFD_QUEUE_DOORBELL_MIRROR_OFFSET + + (q->properties.sdma_queue_id >> 1); if (restore_id && *restore_id != valid_id) return -EINVAL; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 4d65f94da4d8..993d98374253 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -132,6 +132,13 @@ */ #define KFD_QUEUE_DOORBELL_MIRROR_OFFSET 512 +/* + * On GC 9.4.3, 9.4.4 and 9.5.0, doorbell writes issued from shaders are + * routed to an SDMA engine at 32-byte granularity, i.e. 4 indices with the + * 8-byte SDMA doorbells used on SOC15. + */ +#define KFD_SDMA_SHADER_DOORBELL_GRANULARITY 4 + /** * enum kfd_ioctl_flags - KFD ioctl flags * Various flags that can be set in &amdkfd_ioctl_desc.flags to control how diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 4ee7db8a6a22..b92d129962b2 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -7260,10 +7260,15 @@ decide_crtc_timing_for_drm_display_mode(struct drm_display_mode *drm_mode, const struct drm_display_mode *native_mode, bool scale_enabled) { + /* Inserted modes copy the native sync; a sink mode with equal totals may not */ if (scale_enabled || ( native_mode->clock == drm_mode->clock && native_mode->htotal == drm_mode->htotal && - native_mode->vtotal == drm_mode->vtotal)) { + native_mode->vtotal == drm_mode->vtotal && + native_mode->hsync_start == drm_mode->hsync_start && + native_mode->hsync_end == drm_mode->hsync_end && + native_mode->vsync_start == drm_mode->vsync_start && + native_mode->vsync_end == drm_mode->vsync_end)) { if (native_mode->crtc_clock) copy_crtc_timing_for_drm_display_mode(native_mode, drm_mode); } else { @@ -10397,6 +10402,13 @@ static void amdgpu_dm_enable_self_refresh(struct amdgpu_display_manager *dm, struct amdgpu_dm_connector *aconn = (struct amdgpu_dm_connector *)acrtc_state->stream->dm_stream_context; + if (!acrtc_state->base.async_flip) { + amdgpu_dm_psr_set_event(dm, acrtc_state->stream, false, + psr_event_immediate_flip, false); + amdgpu_dm_replay_set_event(dm, acrtc_state->stream, false, + replay_event_immediate_flip, false); + } + /* Decrement skip count when SR is enabled and we're doing fast updates. */ if (acrtc_state->update_type == UPDATE_TYPE_FAST && (psr->psr_feature_enabled || pr->replay_feature_enabled)) { @@ -10810,6 +10822,18 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_commit *state, spin_unlock_irqrestore(&pcrtc->dev->event_lock, flags); } mutex_lock(&dm->dc_lock); + /* + * Tearing (immediate) flips cannot work with panel self-refresh + * features. The event is cleared by + * amdgpu_dm_enable_self_refresh() once async flips stop. + */ + if (immediate_flip) { + dc_exit_ips_for_hw_access(dm->dc); + amdgpu_dm_psr_set_event(dm, acrtc_state->stream, true, + psr_event_immediate_flip, true); + amdgpu_dm_replay_set_event(dm, acrtc_state->stream, true, + replay_event_immediate_flip, true); + } update_planes_and_stream_adapter(dm->dc, acrtc_state->update_type, planes_count, diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c index cfd76c54f652..1d69ecae10a4 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c @@ -110,25 +110,7 @@ void amdgpu_dm_plane_fill_blending_from_plane_state(const struct drm_plane_state if (plane_state->pixel_blend_mode == DRM_MODE_BLEND_PREMULTI || plane_state->pixel_blend_mode == DRM_MODE_BLEND_COVERAGE) { - static const uint32_t alpha_formats[] = { - DRM_FORMAT_ARGB8888, - DRM_FORMAT_RGBA8888, - DRM_FORMAT_ABGR8888, - DRM_FORMAT_ARGB2101010, - DRM_FORMAT_ABGR2101010, - DRM_FORMAT_ARGB16161616, - DRM_FORMAT_ABGR16161616, - DRM_FORMAT_ARGB16161616F, - }; - uint32_t format = plane_state->fb->format->format; - unsigned int i; - - for (i = 0; i < ARRAY_SIZE(alpha_formats); ++i) { - if (format == alpha_formats[i]) { - *per_pixel_alpha = true; - break; - } - } + *per_pixel_alpha = plane_state->fb->format->has_alpha; if (*per_pixel_alpha && plane_state->pixel_blend_mode == DRM_MODE_BLEND_COVERAGE) *pre_multiplied_alpha = false; diff --git a/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c b/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c index d4577663a1f0..aa36b79d0f6c 100644 --- a/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c +++ b/drivers/gpu/drm/aspeed/aspeed_gfx_drv.c @@ -189,13 +189,12 @@ static int aspeed_gfx_load(struct drm_device *drm) } reset_control_deassert(priv->rst); - priv->clk = devm_clk_get(drm->dev, NULL); + priv->clk = devm_clk_get_enabled(drm->dev, NULL); if (IS_ERR(priv->clk)) { dev_err(&pdev->dev, - "missing or invalid clk device tree entry"); + "failed to get and enable clock\n"); return PTR_ERR(priv->clk); } - clk_prepare_enable(priv->clk); /* Sanitize control registers */ writel(0, priv->base + CRT_CTRL1); diff --git a/drivers/gpu/drm/bridge/th1520-dw-hdmi.c b/drivers/gpu/drm/bridge/th1520-dw-hdmi.c index cbea8b14cd4b..2fa7f2d46b3d 100644 --- a/drivers/gpu/drm/bridge/th1520-dw-hdmi.c +++ b/drivers/gpu/drm/bridge/th1520-dw-hdmi.c @@ -135,8 +135,8 @@ static int th1520_dw_hdmi_probe(struct platform_device *pdev) plat_data->priv_data = hdmi; hdmi->dw_hdmi = dw_hdmi_probe(pdev, plat_data); - if (IS_ERR(hdmi)) - return PTR_ERR(hdmi); + if (IS_ERR(hdmi->dw_hdmi)) + return PTR_ERR(hdmi->dw_hdmi); platform_set_drvdata(pdev, hdmi); @@ -145,9 +145,9 @@ static int th1520_dw_hdmi_probe(struct platform_device *pdev) static void th1520_dw_hdmi_remove(struct platform_device *pdev) { - struct dw_hdmi *hdmi = platform_get_drvdata(pdev); + struct th1520_hdmi *hdmi = platform_get_drvdata(pdev); - dw_hdmi_remove(hdmi); + dw_hdmi_remove(hdmi->dw_hdmi); } static const struct of_device_id th1520_dw_hdmi_of_table[] = { diff --git a/drivers/gpu/drm/display/drm_dp_helper.c b/drivers/gpu/drm/display/drm_dp_helper.c index e623ccb4c1d8..488fd123a01a 100644 --- a/drivers/gpu/drm/display/drm_dp_helper.c +++ b/drivers/gpu/drm/display/drm_dp_helper.c @@ -4634,7 +4634,9 @@ static int dp_aux_backlight_update_status(struct backlight_device *bd) if (!backlight_is_blank(bd)) { if (!bl->enabled) { - drm_edp_backlight_enable(bl->aux, &bl->info, brightness); + ret = drm_edp_backlight_enable(bl->aux, &bl->info, brightness); + if (ret) + return ret; bl->enabled = true; return 0; } diff --git a/drivers/gpu/drm/drm_bridge.c b/drivers/gpu/drm/drm_bridge.c index 687b36eea0c7..9124cbe1ff88 100644 --- a/drivers/gpu/drm/drm_bridge.c +++ b/drivers/gpu/drm/drm_bridge.c @@ -414,8 +414,10 @@ void drm_bridge_add(struct drm_bridge *bridge) * in bridge_lingering_list. Remove it or bridge_lingering_list will be * corrupted when adding this bridge to bridge_list below. */ + mutex_lock(&bridge_lock); if (!list_empty(&bridge->list)) list_del_init(&bridge->list); + mutex_unlock(&bridge_lock); mutex_init(&bridge->hpd_mutex); @@ -556,8 +558,10 @@ int drm_bridge_attach(struct drm_encoder *encoder, struct drm_bridge *bridge, if (!bridge->container) DRM_WARN("DRM bridge corrupted or not allocated by devm_drm_bridge_alloc()\n"); - if (list_empty(&bridge->list)) - DRM_WARN("Missing drm_bridge_add() before attach\n"); + scoped_guard(mutex, &bridge_lock) { + if (list_empty(&bridge->list)) + DRM_WARN("Missing drm_bridge_add() before attach\n"); + } drm_bridge_get(bridge); diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c index 80ca785bdb26..3dfa64dc3109 100644 --- a/drivers/gpu/drm/drm_fb_helper.c +++ b/drivers/gpu/drm/drm_fb_helper.c @@ -1724,10 +1724,10 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper) ret = drm_fb_helper_single_fb_probe(fb_helper); if (ret < 0) { - if (ret == -EAGAIN) { - fb_helper->deferred_setup = true; + /* modesets are probed but fb_helper->fb isn't; defer restore too */ + fb_helper->deferred_setup = true; + if (ret == -EAGAIN) ret = 0; - } mutex_unlock(&fb_helper->lock); goto err_drm_fb_helper_release_info; diff --git a/drivers/gpu/drm/gud/gud_pipe.c b/drivers/gpu/drm/gud/gud_pipe.c index aa7792966287..5aa5ac2f3727 100644 --- a/drivers/gpu/drm/gud/gud_pipe.c +++ b/drivers/gpu/drm/gud/gud_pipe.c @@ -507,6 +507,7 @@ int gud_plane_atomic_check(struct drm_plane *plane, for_each_new_connector_in_state(state, connector, connector_state, i) { if (connector_state->crtc) break; + connector_state = NULL; } /* diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 1f303d4eaa4d..d5aadced7da0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -1766,6 +1766,7 @@ static int eb_reinit_userptr(struct i915_execbuffer *eb) static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb) { + const unsigned int count = eb->buffer_count; bool have_copy = false; struct eb_vma *ev; int err = 0; @@ -1868,7 +1869,6 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb) out: if (have_copy) { - const unsigned int count = eb->buffer_count; unsigned int i; for (i = 0; i < count; i++) { diff --git a/drivers/gpu/drm/i915/gt/intel_rps.c b/drivers/gpu/drm/i915/gt/intel_rps.c index a33b19c04737..0f9b4f1dd3cb 100644 --- a/drivers/gpu/drm/i915/gt/intel_rps.c +++ b/drivers/gpu/drm/i915/gt/intel_rps.c @@ -1904,7 +1904,8 @@ static void rps_work(struct work_struct *work) out: spin_lock_irq(gt->irq_lock); - gen6_gt_pm_unmask_irq(gt, rps->pm_events); + if (intel_rps_is_active(rps)) + gen6_gt_pm_unmask_irq(gt, rps->pm_events); spin_unlock_irq(gt->irq_lock); } diff --git a/drivers/gpu/drm/imagination/pvr_mmu.c b/drivers/gpu/drm/imagination/pvr_mmu.c index 62eae7fcd5a2..67e73d7d5d0e 100644 --- a/drivers/gpu/drm/imagination/pvr_mmu.c +++ b/drivers/gpu/drm/imagination/pvr_mmu.c @@ -2337,7 +2337,7 @@ void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx) * @ctx: MMU context associated with owning VM context. * @sgt: Scatter gather table containing pages pinned for use by this context. * @device_addr: Virtual device address at the start of the requested mapping. - * @sgt_offset: Start offset of the requested device-virtual memory mapping. + * @sgt_offset: Offset into @sgt of the start of the requested mapping. * @size: Size in bytes of the requested device-virtual memory mapping. For an * unmapping, this should be zero so that no page tables are allocated. * @@ -2350,7 +2350,6 @@ struct pvr_mmu_op_context * pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt, u64 device_addr, u64 sgt_offset, u64 size) { - u64 start_addr = device_addr + sgt_offset; int err; struct pvr_mmu_op_context *op_ctx = kzalloc_obj(*op_ctx); @@ -2365,18 +2364,18 @@ pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt, if (size) { /* - * The number of page table objects we need to prealloc is - * indicated by the mapping size, start address and the sizes - * of the areas mapped per PT or PD. The range calculation is - * identical to that for the index into a table for a device - * address, so we reuse those functions here. + * The page tables needed are set by the device-virtual range + * being mapped: one level 1 table per 1GiB region and one + * level 0 table per 2MiB region the range touches. Tables that + * already exist are not consumed, so this is an upper bound. */ - const u32 l1_start_idx = pvr_page_table_l2_idx(start_addr); - const u32 l1_end_idx = pvr_page_table_l2_idx(start_addr + size); - const u32 l1_count = l1_end_idx - l1_start_idx + 1; - const u32 l0_start_idx = pvr_page_table_l1_idx(start_addr); - const u32 l0_end_idx = pvr_page_table_l1_idx(start_addr + size); - const u32 l0_count = l0_end_idx - l0_start_idx + 1; + const u64 last_addr = device_addr + size - 1; + const u64 l1_count = + (last_addr >> ROGUE_MMUCTRL_VADDR_PC_INDEX_SHIFT) - + (device_addr >> ROGUE_MMUCTRL_VADDR_PC_INDEX_SHIFT) + 1; + const u64 l0_count = + (last_addr >> ROGUE_MMUCTRL_VADDR_PD_INDEX_SHIFT) - + (device_addr >> ROGUE_MMUCTRL_VADDR_PD_INDEX_SHIFT) + 1; /* * Alloc and push page table entries until we have enough of diff --git a/drivers/gpu/drm/imagination/pvr_vm.c b/drivers/gpu/drm/imagination/pvr_vm.c index 55cc999f3708..cbdd15ed74fd 100644 --- a/drivers/gpu/drm/imagination/pvr_vm.c +++ b/drivers/gpu/drm/imagination/pvr_vm.c @@ -418,6 +418,8 @@ pvr_vm_gpuva_unmap(struct drm_gpuva_op *op, void *op_ctx) static int pvr_vm_gpuva_remap(struct drm_gpuva_op *op, void *op_ctx) { + /* The split parts belong to the object of the mapping being split. */ + struct drm_gpuvm_bo *vm_bo = op->remap.unmap->va->vm_bo; struct pvr_vm_bind_op *ctx = op_ctx; u64 va_start = 0, va_range = 0; int err; @@ -433,14 +435,12 @@ pvr_vm_gpuva_remap(struct drm_gpuva_op *op, void *op_ctx) drm_gpuva_remap(&ctx->prev_va->base, &ctx->next_va->base, &op->remap); if (op->remap.prev) { - pvr_gem_object_get(gem_to_pvr_gem(ctx->prev_va->base.gem.obj)); - drm_gpuva_link(&ctx->prev_va->base, ctx->gpuvm_bo); + drm_gpuva_link(&ctx->prev_va->base, vm_bo); ctx->prev_va = NULL; } if (op->remap.next) { - pvr_gem_object_get(gem_to_pvr_gem(ctx->next_va->base.gem.obj)); - drm_gpuva_link(&ctx->next_va->base, ctx->gpuvm_bo); + drm_gpuva_link(&ctx->next_va->base, vm_bo); ctx->next_va = NULL; } diff --git a/drivers/gpu/drm/panthor/panthor_sched.c b/drivers/gpu/drm/panthor/panthor_sched.c index 369ee06434bb..765b895619e8 100644 --- a/drivers/gpu/drm/panthor/panthor_sched.c +++ b/drivers/gpu/drm/panthor/panthor_sched.c @@ -3675,6 +3675,9 @@ int panthor_group_create(struct panthor_file *pfile, hweight64(group_args->tiler_core_mask) < group_args->max_tiler_cores) return -EINVAL; + if (group_args->queues.count > sched->cs_slot_count) + return -EINVAL; + group = kzalloc_obj(*group); if (!group) return -ENOMEM; diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c index 49ea2ed0996b..7b8b3364000f 100644 --- a/drivers/gpu/drm/vc4/vc4_bo.c +++ b/drivers/gpu/drm/vc4/vc4_bo.c @@ -204,7 +204,7 @@ static struct list_head *vc4_get_cache_list_for_size(struct drm_device *dev, struct list_head *new_list; uint32_t i; - new_list = kmalloc_objs(struct list_head, new_size); + new_list = kvmalloc_objs(struct list_head, new_size); if (!new_list) return NULL; @@ -224,7 +224,7 @@ static struct list_head *vc4_get_cache_list_for_size(struct drm_device *dev, for (i = vc4->bo_cache.size_list_size; i < new_size; i++) INIT_LIST_HEAD(&new_list[i]); - kfree(vc4->bo_cache.size_list); + kvfree(vc4->bo_cache.size_list); vc4->bo_cache.size_list = new_list; vc4->bo_cache.size_list_size = new_size; } @@ -1052,6 +1052,7 @@ static void vc4_bo_cache_destroy(struct drm_device *dev, void *unused) cancel_work_sync(&vc4->bo_cache.time_work); vc4_bo_cache_purge(dev); + kvfree(vc4->bo_cache.size_list); for (i = 0; i < vc4->num_labels; i++) { if (vc4->bo_labels[i].num_allocated) { diff --git a/drivers/gpu/drm/vc4/vc4_drv.c b/drivers/gpu/drm/vc4/vc4_drv.c index 616caf9d9915..7b64e1ed02d4 100644 --- a/drivers/gpu/drm/vc4/vc4_drv.c +++ b/drivers/gpu/drm/vc4/vc4_drv.c @@ -360,8 +360,11 @@ static int vc4_drm_bind(struct device *dev) } ret = aperture_remove_all_conflicting_devices(driver->name); - if (ret) + if (ret) { + if (firmware) + rpi_firmware_put(firmware); goto err; + } if (firmware) { ret = rpi_firmware_property(firmware, diff --git a/drivers/gpu/drm/vc4/vc4_drv.h b/drivers/gpu/drm/vc4/vc4_drv.h index 0f5958c1e6b6..b0e82c2dc1ba 100644 --- a/drivers/gpu/drm/vc4/vc4_drv.h +++ b/drivers/gpu/drm/vc4/vc4_drv.h @@ -1051,7 +1051,7 @@ void vc4_plane_async_set_fb(struct drm_plane *plane, /* vc4_v3d.c */ extern struct platform_driver vc4_v3d_driver; extern const struct of_device_id vc4_v3d_dt_match[]; -int vc4_v3d_get_bin_slot(struct vc4_dev *vc4); +int vc4_v3d_get_bin_slot(struct vc4_dev *vc4, long timeout); int vc4_v3d_bin_bo_get(struct vc4_dev *vc4, bool *used); void vc4_v3d_bin_bo_put(struct vc4_dev *vc4); int vc4_v3d_pm_get(struct vc4_dev *vc4); diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c index e231c906709c..6ee4bd77d433 100644 --- a/drivers/gpu/drm/vc4/vc4_gem.c +++ b/drivers/gpu/drm/vc4/vc4_gem.c @@ -877,11 +877,17 @@ vc4_complete_exec(struct drm_device *dev, struct vc4_exec_info *exec) drm_gem_object_put(&bo->base.base); } - /* Free up the allocation of any bin slots we used. */ + /* Free up the allocation of any bin slots we used. Jobs that ran to + * completion had their slots released in vc4_irq_finish_render_job(). + * Only jobs that never completed still have slots to be released here. + */ spin_lock_irqsave(&vc4->job_lock, irqflags); vc4->bin_alloc_used &= ~exec->bin_slots; spin_unlock_irqrestore(&vc4->job_lock, irqflags); + /* Let anyone waiting on the binner pool retry. */ + wake_up_all(&vc4->job_wait_queue); + /* Release the reference on the binner BO if needed. */ if (exec->bin_bo_used) vc4_v3d_bin_bo_put(vc4); diff --git a/drivers/gpu/drm/vc4/vc4_irq.c b/drivers/gpu/drm/vc4/vc4_irq.c index 7877d493d80e..dc10d3a4d90f 100644 --- a/drivers/gpu/drm/vc4/vc4_irq.c +++ b/drivers/gpu/drm/vc4/vc4_irq.c @@ -74,7 +74,7 @@ vc4_overflow_mem_work(struct work_struct *work) bo = vc4->bin_bo; - bin_bo_slot = vc4_v3d_get_bin_slot(vc4); + bin_bo_slot = vc4_v3d_get_bin_slot(vc4, msecs_to_jiffies(500)); if (bin_bo_slot < 0) { drm_err(&vc4->base, "Couldn't allocate binner overflow mem\n"); goto complete; @@ -165,6 +165,10 @@ vc4_irq_finish_render_job(struct drm_device *dev) trace_vc4_rcl_end_irq(dev, exec->seqno); vc4->finished_seqno++; + + vc4->bin_alloc_used &= ~exec->bin_slots; + exec->bin_slots = 0; + list_move_tail(&exec->head, &vc4->job_done_list); nextbin = vc4_first_bin_job(vc4); @@ -276,7 +280,7 @@ vc4_irq_disable(struct drm_device *dev) V3D_WRITE(V3D_INTCTL, V3D_DRIVER_IRQS); /* Finish any interrupt handler still in flight. */ - synchronize_irq(vc4->irq); + disable_irq(vc4->irq); cancel_work_sync(&vc4->overflow_mem_work); } @@ -284,7 +288,6 @@ vc4_irq_disable(struct drm_device *dev) int vc4_irq_install(struct drm_device *dev, int irq) { struct vc4_dev *vc4 = to_vc4_dev(dev); - int ret; if (WARN_ON_ONCE(vc4->gen > VC4_GEN_4)) return -ENODEV; @@ -298,19 +301,8 @@ int vc4_irq_install(struct drm_device *dev, int irq) init_waitqueue_head(&vc4->job_wait_queue); INIT_WORK(&vc4->overflow_mem_work, vc4_overflow_mem_work); - /* Clear any pending interrupts someone might have left around - * for us. - */ - V3D_WRITE(V3D_INTCTL, V3D_DRIVER_IRQS); - - ret = devm_request_irq(dev->dev, irq, vc4_irq, 0, - dev_name(dev->dev), dev); - if (ret) - return ret; - - vc4_irq_enable(dev); - - return 0; + return devm_request_irq(dev->dev, irq, vc4_irq, IRQF_NO_AUTOEN, + dev_name(dev->dev), dev); } void vc4_irq_uninstall(struct drm_device *dev) diff --git a/drivers/gpu/drm/vc4/vc4_v3d.c b/drivers/gpu/drm/vc4/vc4_v3d.c index f32410420d3e..6844e8f3948c 100644 --- a/drivers/gpu/drm/vc4/vc4_v3d.c +++ b/drivers/gpu/drm/vc4/vc4_v3d.c @@ -9,6 +9,7 @@ #include #include #include +#include #include @@ -172,46 +173,42 @@ static void vc4_v3d_init_hw(struct drm_device *dev) V3D_WRITE(V3D_VPMBASE, 0); } -int vc4_v3d_get_bin_slot(struct vc4_dev *vc4) +static int bin_slot_try_alloc(struct vc4_dev *vc4) { - struct drm_device *dev = &vc4->base; - unsigned long irqflags; int slot; - uint64_t seqno = 0; - struct vc4_exec_info *exec; - if (WARN_ON_ONCE(vc4->gen > VC4_GEN_4)) - return -ENODEV; + guard(spinlock_irqsave)(&vc4->job_lock); -try_again: - spin_lock_irqsave(&vc4->job_lock, irqflags); slot = ffs(~vc4->bin_alloc_used); if (slot != 0) { /* Switch from ffs() bit index to a 0-based index. */ slot--; vc4->bin_alloc_used |= BIT(slot); - spin_unlock_irqrestore(&vc4->job_lock, irqflags); - return slot; + } else { + slot = -ENOMEM; } - /* Couldn't find an open slot. Wait for render to complete - * and try again. - */ - exec = vc4_last_render_job(vc4); - if (exec) - seqno = exec->seqno; - spin_unlock_irqrestore(&vc4->job_lock, irqflags); + return slot; +} - if (seqno) { - int ret = vc4_wait_for_seqno(dev, seqno, ~0ull, true); +int vc4_v3d_get_bin_slot(struct vc4_dev *vc4, long timeout) +{ + int slot; + long ret; - if (ret == 0) - goto try_again; + if (WARN_ON_ONCE(vc4->gen > VC4_GEN_4)) + return -ENODEV; + /* If the pool is full, wait for a job to release its slots. */ + ret = wait_event_interruptible_timeout(vc4->job_wait_queue, + (slot = bin_slot_try_alloc(vc4)) >= 0, + timeout); + if (ret < 0) return ret; - } + if (ret == 0) + return -ENOMEM; - return -ENOMEM; + return slot; } /* @@ -396,6 +393,7 @@ static int vc4_v3d_runtime_resume(struct device *dev) vc4_v3d_init_hw(&vc4->base); vc4_irq_enable(&vc4->base); + enable_irq(vc4->irq); return 0; } @@ -452,6 +450,12 @@ static int vc4_v3d_bind(struct device *dev, struct device *master, void *data) return ret; vc4->irq = ret; + ret = vc4_irq_install(drm, vc4->irq); + if (ret) { + drm_err(drm, "Failed to install IRQ handler\n"); + return ret; + } + ret = devm_pm_runtime_enable(dev); if (ret) return ret; @@ -473,12 +477,6 @@ static int vc4_v3d_bind(struct device *dev, struct device *master, void *data) V3D_WRITE(V3D_BPOA, 0); V3D_WRITE(V3D_BPOS, 0); - ret = vc4_irq_install(drm, vc4->irq); - if (ret) { - drm_err(drm, "Failed to install IRQ handler\n"); - goto err_put_runtime_pm; - } - pm_runtime_use_autosuspend(dev); pm_runtime_set_autosuspend_delay(dev, 40); /* a little over 2 frames. */ pm_runtime_put_autosuspend(dev); @@ -497,8 +495,15 @@ static void vc4_v3d_unbind(struct device *dev, struct device *master, struct drm_device *drm = data; struct vc4_dev *vc4 = to_vc4_dev(drm); + /* A straggler vc4_reset() re-enables the interrupt. */ + timer_shutdown_sync(&vc4->hangcheck.timer); + cancel_work_sync(&vc4->hangcheck.reset_work); + vc4_irq_uninstall(drm); + /* Flush rather than cancel, so queued completions release their jobs. */ + flush_work(&vc4->job_done_work); + /* Disable the binner's overflow memory address, so the next * driver probe (if any) doesn't try to reuse our old * allocation. diff --git a/drivers/gpu/drm/vc4/vc4_validate.c b/drivers/gpu/drm/vc4/vc4_validate.c index d2a65c968b1f..565227657b43 100644 --- a/drivers/gpu/drm/vc4/vc4_validate.c +++ b/drivers/gpu/drm/vc4/vc4_validate.c @@ -402,7 +402,7 @@ validate_tile_binning_config(VALIDATE_ARGS) return -EINVAL; } - bin_slot = vc4_v3d_get_bin_slot(vc4); + bin_slot = vc4_v3d_get_bin_slot(vc4, MAX_SCHEDULE_TIMEOUT); if (bin_slot < 0) { if (bin_slot != -EINTR && bin_slot != -ERESTARTSYS) { drm_err(dev, "Failed to allocate binner memory: %d\n", diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cursor_plane.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cursor_plane.c index d1e7df500190..5a69606a0593 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cursor_plane.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cursor_plane.c @@ -324,6 +324,7 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf, unsigned long kmap_num; SVGA3dCopyBox *box; u32 box_count; + u64 src_extent; void *virtual; bool is_iomem; struct vmw_dma_cmd { @@ -335,6 +336,8 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf, vmw_surface_get_desc(VMW_CURSOR_SNOOP_FORMAT); const u32 image_pitch = VMW_CURSOR_SNOOP_WIDTH * desc->pitchBytesPerBlock; + BUILD_BUG_ON(sizeof(*srf->snooper.image) != desc->pitchBytesPerBlock); + cmd = container_of(header, struct vmw_dma_cmd, header); /* No snooper installed, nothing to copy */ @@ -372,8 +375,19 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf, return; } + if (box->w == 0 || box->h == 0) + return; + + src_extent = (u64)(box->h - 1) * cmd->dma.guest.pitch + + (u64)box->w * desc->pitchBytesPerBlock; + if (src_extent > bo->base.size) { + DRM_ERROR("Cursor snoop source of %llu bytes exceeds the %zu byte buffer\n", + src_extent, bo->base.size); + return; + } + kmap_offset = cmd->dma.guest.ptr.offset >> PAGE_SHIFT; - kmap_num = (VMW_CURSOR_SNOOP_HEIGHT * image_pitch) >> PAGE_SHIFT; + kmap_num = PFN_UP(src_extent); ret = ttm_bo_reserve(bo, true, false, NULL); if (unlikely(ret != 0)) { @@ -387,14 +401,16 @@ void vmw_kms_cursor_snoop(struct vmw_surface *srf, virtual = ttm_kmap_obj_virtual(&map, &is_iomem); - if (box->w == VMW_CURSOR_SNOOP_WIDTH && cmd->dma.guest.pitch == image_pitch) { + if (box->w == VMW_CURSOR_SNOOP_WIDTH && + box->h == VMW_CURSOR_SNOOP_HEIGHT && + cmd->dma.guest.pitch == image_pitch) { memcpy(srf->snooper.image, virtual, VMW_CURSOR_SNOOP_HEIGHT * image_pitch); } else { - /* Image is unsigned pointer. */ + /* The destination stride is in pixels. */ for (i = 0; i < box->h; i++) - memcpy(srf->snooper.image + i * image_pitch, - virtual + i * cmd->dma.guest.pitch, + memcpy(srf->snooper.image + i * VMW_CURSOR_SNOOP_WIDTH, + virtual + (size_t)i * cmd->dma.guest.pitch, box->w * desc->pitchBytesPerBlock); } srf->snooper.id++; diff --git a/drivers/gpu/drm/xe/display/xe_fb_pin.c b/drivers/gpu/drm/xe/display/xe_fb_pin.c index 73469ea5f333..f1dbdc802df8 100644 --- a/drivers/gpu/drm/xe/display/xe_fb_pin.c +++ b/drivers/gpu/drm/xe/display/xe_fb_pin.c @@ -356,7 +356,7 @@ static struct i915_vma *__xe_pin_fb_vma(struct drm_gem_object *obj, bool is_dpt, drm_exec_retry_on_contention(&exec); xe_validation_retry_on_oom(&ctx, &ret); if (!ret) - ttm_bo_pin(&bo->ttm); + xe_bo_pin_account(bo); } if (ret) goto err; @@ -373,7 +373,7 @@ static struct i915_vma *__xe_pin_fb_vma(struct drm_gem_object *obj, bool is_dpt, err_unpin: ttm_bo_reserve(&bo->ttm, false, false, NULL); - ttm_bo_unpin(&bo->ttm); + xe_bo_unpin_account(bo); ttm_bo_unreserve(&bo->ttm); err: kfree(vma); @@ -393,7 +393,7 @@ static void __xe_unpin_fb_vma(struct i915_vma *vma) xe_ggtt_node_remove(vma->node, false); ttm_bo_reserve(&vma->bo->ttm, false, false, NULL); - ttm_bo_unpin(&vma->bo->ttm); + xe_bo_unpin_account(vma->bo); ttm_bo_unreserve(&vma->bo->ttm); kfree(vma); } diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 2c60e99249f9..f4c5151d61c7 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -443,6 +443,100 @@ static void xe_ttm_tt_account_subtract(struct xe_device *xe, struct ttm_tt *tt) xe_shrinker_mod_pages(xe->mem.shrinker, -(long)tt->num_pages, 0); } +static bool xe_ttm_bo_is_imported(struct ttm_buffer_object *tbo) +{ + dma_resv_assert_held(tbo->base.resv); + + return tbo->ttm && + (tbo->ttm->page_flags & (TTM_TT_FLAG_EXTERNAL | TTM_TT_FLAG_EXTERNAL_MAPPABLE)) == + TTM_TT_FLAG_EXTERNAL; +} + +/* + * Account @bo's pages as pinned for the shrinker. Removes @bo's pages + * from the shrinker's shrinkable / purgeable counts on the transition + * from unpinned to pinned. Must be called with @bo's dma-resv held, + * after &ttm_buffer_object.pin_count has been incremented by + * ttm_bo_pin(). Safe to call unconditionally regardless of which pin + * path (kernel, external, framebuffer, backup object, ...) is pinning + * @bo, since it only acts on the true 0->1 pin_count transition. + * + * Imported bos (xe_ttm_bo_is_imported()) are excluded: for those, + * xe_ttm_tt_populate() never actually populates the tt or calls + * xe_ttm_tt_account_add(), but TTM core's ttm_tt_populate() still + * unconditionally marks the tt as populated on driver-hook success. + * Relying on ttm_tt_is_populated() for these would subtract pages that + * were never added, underflowing the shrinker's counts. + */ +static void xe_bo_account_pin(struct xe_bo *bo) +{ + struct xe_device *xe = xe_bo_device(bo); + + if (bo->ttm.pin_count == 1 && bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm) && + !xe_ttm_bo_is_imported(&bo->ttm)) + xe_ttm_tt_account_subtract(xe, bo->ttm.ttm); +} + +/* + * Counterpart to xe_bo_account_pin(). Must be called with @bo's dma-resv + * held, before &ttm_buffer_object.pin_count is decremented by + * ttm_bo_unpin(), so that the check against the true 1->0 transition sees + * the pin count that is about to be released. See xe_bo_account_pin() for + * why imported bos are excluded. + * + * On the true last unpin, also removes @bo from whichever pinned-bo list + * (external or kernel_bo_present) it may currently be linked on, since a + * bo's final unpin can happen through a pin path (e.g. framebuffer, + * backup object) that has no notion of, or ownership over, that list. + * This is safe and list-agnostic: list_del_init() only needs the node + * itself, not knowledge of which list it is threaded through, and is a + * no-op if @bo is not linked. + */ +static void xe_bo_account_unpin(struct xe_bo *bo) +{ + struct xe_device *xe = xe_bo_device(bo); + bool last_unpin = bo->ttm.pin_count == 1; + + if (last_unpin && !list_empty(&bo->pinned_link)) { + spin_lock(&xe->pinned.lock); + list_del_init(&bo->pinned_link); + spin_unlock(&xe->pinned.lock); + } + + if (last_unpin && bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm) && + !xe_ttm_bo_is_imported(&bo->ttm)) + xe_ttm_tt_account_add(xe, bo->ttm.ttm); +} + +/** + * xe_bo_pin_account() - Pin a bo and account its pages for the shrinker + * @bo: The buffer object to pin. + * + * Pins @bo via ttm_bo_pin() and updates the shrinker's shrinkable / + * purgeable page accounting to match. Must be called with @bo's dma-resv + * held. Safe to call regardless of which pin path (kernel, external, + * framebuffer, backup object, ...) is pinning @bo. + */ +void xe_bo_pin_account(struct xe_bo *bo) +{ + ttm_bo_pin(&bo->ttm); + xe_bo_account_pin(bo); +} + +/** + * xe_bo_unpin_account() - Unpin a bo and account its pages for the shrinker + * @bo: The buffer object to unpin. + * + * Counterpart to xe_bo_pin_account(). Updates the shrinker's shrinkable / + * purgeable page accounting to match, then unpins @bo via ttm_bo_unpin(). + * Must be called with @bo's dma-resv held. + */ +void xe_bo_unpin_account(struct xe_bo *bo) +{ + xe_bo_account_unpin(bo); + ttm_bo_unpin(&bo->ttm); +} + static void update_global_total_pages(struct ttm_device *ttm_dev, long num_pages) { @@ -1377,7 +1471,8 @@ int xe_bo_notifier_prepare_pinned(struct xe_bo *bo) } backup->parent_obj = xe_bo_get(bo); /* Released by bo_destroy */ - ttm_bo_pin(&backup->ttm); + /* Note: XE_BO_FLAG_SYSTEM resolves to XE_PL_TT so already populated. */ + xe_bo_pin_account(backup); bo->backup_obj = backup; } @@ -1397,7 +1492,7 @@ int xe_bo_notifier_unprepare_pinned(struct xe_bo *bo) { xe_bo_lock(bo, false); if (bo->backup_obj) { - ttm_bo_unpin(&bo->backup_obj->ttm); + xe_bo_unpin_account(bo->backup_obj); xe_bo_put(bo->backup_obj); bo->backup_obj = NULL; } @@ -1608,7 +1703,7 @@ int xe_bo_restore_pinned(struct xe_bo *bo) xe_bo_vunmap(backup); if (!bo->backup_obj) { if (xe_bo_is_pinned(backup)) - ttm_bo_unpin(&backup->ttm); + xe_bo_unpin_account(backup); xe_bo_put(backup); } out_unlock_bo: @@ -2000,15 +2095,6 @@ static vm_fault_t xe_err_to_fault_t(int err) return VM_FAULT_SIGBUS; } -static bool xe_ttm_bo_is_imported(struct ttm_buffer_object *tbo) -{ - dma_resv_assert_held(tbo->base.resv); - - return tbo->ttm && - (tbo->ttm->page_flags & (TTM_TT_FLAG_EXTERNAL | TTM_TT_FLAG_EXTERNAL_MAPPABLE)) == - TTM_TT_FLAG_EXTERNAL; -} - static vm_fault_t xe_bo_cpu_fault_fastpath(struct vm_fault *vmf, struct xe_device *xe, struct xe_bo *bo, bool needs_rpm) { @@ -2975,12 +3061,13 @@ uint64_t vram_region_gpu_offset(struct ttm_resource *res) int xe_bo_pin_external(struct xe_bo *bo, bool in_place, struct drm_exec *exec) { struct xe_device *xe = xe_bo_device(bo); + bool first_pin = !xe_bo_is_pinned(bo); int err; xe_assert(xe, !bo->vm); xe_assert(xe, xe_bo_is_user(bo)); - if (!xe_bo_is_pinned(bo)) { + if (first_pin) { if (!in_place) { err = xe_bo_validate(bo, NULL, false, exec); if (err) @@ -2992,9 +3079,7 @@ int xe_bo_pin_external(struct xe_bo *bo, bool in_place, struct drm_exec *exec) spin_unlock(&xe->pinned.lock); } - ttm_bo_pin(&bo->ttm); - if (bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm)) - xe_ttm_tt_account_subtract(xe, bo->ttm.ttm); + xe_bo_pin_account(bo); /* * FIXME: If we always use the reserve / unreserve functions for locking @@ -3050,9 +3135,7 @@ int xe_bo_pin(struct xe_bo *bo, struct drm_exec *exec) spin_unlock(&xe->pinned.lock); } - ttm_bo_pin(&bo->ttm); - if (bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm)) - xe_ttm_tt_account_subtract(xe, bo->ttm.ttm); + xe_bo_pin_account(bo); /* * FIXME: If we always use the reserve / unreserve functions for locking @@ -3081,14 +3164,7 @@ void xe_bo_unpin_external(struct xe_bo *bo) xe_assert(xe, xe_bo_is_pinned(bo)); xe_assert(xe, xe_bo_is_user(bo)); - spin_lock(&xe->pinned.lock); - if (bo->ttm.pin_count == 1 && !list_empty(&bo->pinned_link)) - list_del_init(&bo->pinned_link); - spin_unlock(&xe->pinned.lock); - - ttm_bo_unpin(&bo->ttm); - if (bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm)) - xe_ttm_tt_account_add(xe, bo->ttm.ttm); + xe_bo_unpin_account(bo); /* * FIXME: If we always use the reserve / unreserve functions for locking @@ -3106,21 +3182,16 @@ void xe_bo_unpin(struct xe_bo *bo) xe_assert(xe, xe_bo_is_pinned(bo)); if (mem_type_is_vram(place->mem_type) || bo->flags & XE_BO_FLAG_GGTT) { - spin_lock(&xe->pinned.lock); xe_assert(xe, !list_empty(&bo->pinned_link)); - list_del_init(&bo->pinned_link); - spin_unlock(&xe->pinned.lock); if (bo->backup_obj) { if (xe_bo_is_pinned(bo->backup_obj)) - ttm_bo_unpin(&bo->backup_obj->ttm); + xe_bo_unpin_account(bo->backup_obj); xe_bo_put(bo->backup_obj); bo->backup_obj = NULL; } } - ttm_bo_unpin(&bo->ttm); - if (bo->ttm.ttm && ttm_tt_is_populated(bo->ttm.ttm)) - xe_ttm_tt_account_add(xe, bo->ttm.ttm); + xe_bo_unpin_account(bo); } /** diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h index a9e5e1bb8e4f..f9c8c73a1e69 100644 --- a/drivers/gpu/drm/xe/xe_bo.h +++ b/drivers/gpu/drm/xe/xe_bo.h @@ -228,6 +228,8 @@ int xe_bo_pin_external(struct xe_bo *bo, bool in_place, struct drm_exec *exec); int xe_bo_pin(struct xe_bo *bo, struct drm_exec *exec); void xe_bo_unpin_external(struct xe_bo *bo); void xe_bo_unpin(struct xe_bo *bo); +void xe_bo_pin_account(struct xe_bo *bo); +void xe_bo_unpin_account(struct xe_bo *bo); int xe_bo_validate(struct xe_bo *bo, struct xe_vm *vm, bool allow_res_evict, struct drm_exec *exec); diff --git a/drivers/gpu/drm/xe/xe_i2c.c b/drivers/gpu/drm/xe/xe_i2c.c index b376a9167cc1..3127fb89d965 100644 --- a/drivers/gpu/drm/xe/xe_i2c.c +++ b/drivers/gpu/drm/xe/xe_i2c.c @@ -330,12 +330,15 @@ static void xe_i2c_remove(void *data) xe_i2c_irq_reset(xe); xe_amc_exit(i2c); + /* Stop the notifier from arming the client work before teardown. */ + bus_unregister_notifier(&i2c_bus_type, &i2c->bus_notifier); + cancel_work_sync(&i2c->work); + for (i = 0; i < XE_I2C_MAX_CLIENTS; i++) { i2c_unregister_device(i2c->client[i]); i2c->client[i] = NULL; } - bus_unregister_notifier(&i2c_bus_type, &i2c->bus_notifier); xe_i2c_unregister_adapter(i2c); xe->i2c = NULL; } diff --git a/drivers/gpu/drm/xe/xe_lrc.c b/drivers/gpu/drm/xe/xe_lrc.c index 9f8217ff1904..224e64bccaeb 100644 --- a/drivers/gpu/drm/xe/xe_lrc.c +++ b/drivers/gpu/drm/xe/xe_lrc.c @@ -1099,8 +1099,7 @@ static void xe_lrc_finish(struct xe_lrc *lrc) #define CONTEXT_ACTIVE XE_LRC_CTX_TIMESTAMP_ACTIVE static ssize_t setup_utilization_wa(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, - size_t max_len) + u32 *batch, size_t max_len, bool indirect) { u32 *cmd = batch; @@ -1131,19 +1130,25 @@ static ssize_t setup_utilization_wa(struct xe_lrc *lrc, } static ssize_t setup_timestamp_wa(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_len) + u32 *batch, size_t max_len, bool indirect) { const u32 ts_addr = __xe_lrc_ctx_timestamp_ggtt_addr(lrc); u32 *cmd = batch; - if (!XE_GT_WA(lrc->gt, 16010904313) || - !(hwe->class == XE_ENGINE_CLASS_RENDER || - hwe->class == XE_ENGINE_CLASS_COMPUTE || - hwe->class == XE_ENGINE_CLASS_COPY || + if (!XE_GT_WA(lrc->gt, 16010904313)) + return 0; + + if (!indirect && + !(hwe->class == XE_ENGINE_CLASS_COPY || hwe->class == XE_ENGINE_CLASS_VIDEO_DECODE || hwe->class == XE_ENGINE_CLASS_VIDEO_ENHANCE)) return 0; + if (indirect && + !(hwe->class == XE_ENGINE_CLASS_RENDER || + hwe->class == XE_ENGINE_CLASS_COMPUTE)) + return 0; + if (xe_gt_WARN_ON(lrc->gt, max_len < 12)) return -ENOSPC; @@ -1169,7 +1174,8 @@ static ssize_t setup_timestamp_wa(struct xe_lrc *lrc, struct xe_hw_engine *hwe, static ssize_t setup_configfs_post_ctx_restore_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_len) + u32 *batch, size_t max_len, + bool indirect) { struct xe_device *xe = gt_to_xe(lrc->gt); const u32 *user_batch; @@ -1198,7 +1204,8 @@ static ssize_t setup_configfs_post_ctx_restore_bb(struct xe_lrc *lrc, static ssize_t setup_configfs_mid_ctx_restore_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_len) + u32 *batch, size_t max_len, + bool indirect) { struct xe_device *xe = gt_to_xe(lrc->gt); const u32 *user_batch; @@ -1227,7 +1234,8 @@ static ssize_t setup_configfs_mid_ctx_restore_bb(struct xe_lrc *lrc, static ssize_t setup_invalidate_state_cache_wa(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_len) + u32 *batch, size_t max_len, + bool indirect) { u32 *cmd = batch; @@ -1247,7 +1255,8 @@ static ssize_t setup_invalidate_state_cache_wa(struct xe_lrc *lrc, static ssize_t setup_invalidate_auxccs_wa(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_len) + u32 *batch, size_t max_len, + bool indirect) { struct xe_gt *gt = lrc->gt; u32 *(*emit)(struct xe_gt *gt, u32 *cmd) = @@ -1264,7 +1273,7 @@ static ssize_t setup_invalidate_auxccs_wa(struct xe_lrc *lrc, struct bo_setup { ssize_t (*setup)(struct xe_lrc *lrc, struct xe_hw_engine *hwe, - u32 *batch, size_t max_size); + u32 *batch, size_t max_size, bool indirect); }; struct bo_setup_state { @@ -1273,6 +1282,7 @@ struct bo_setup_state { struct xe_hw_engine *hwe; size_t max_size; size_t reserve_dw; + bool indirect; unsigned int offset; const struct bo_setup *funcs; unsigned int num_funcs; @@ -1298,7 +1308,8 @@ static int setup_bo(struct bo_setup_state *state) for (size_t i = 0; i < state->num_funcs; i++) { ssize_t len = state->funcs[i].setup(state->lrc, state->hwe, - state->ptr, remain); + state->ptr, remain, + state->indirect); remain -= len; @@ -1404,6 +1415,7 @@ setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe) struct bo_setup_state state = { .lrc = lrc, .hwe = hwe, + .indirect = true, .max_size = (63 * 64) /* max 63 cachelines */, .buffer = NULL, .offset = __xe_lrc_indirect_ctx_offset(lrc), diff --git a/drivers/gpu/drm/xe/xe_mmio.c b/drivers/gpu/drm/xe/xe_mmio.c index 78adb303b663..6907c1127d32 100644 --- a/drivers/gpu/drm/xe/xe_mmio.c +++ b/drivers/gpu/drm/xe/xe_mmio.c @@ -288,6 +288,7 @@ u64 xe_mmio_read64_2x32(struct xe_mmio *mmio, struct xe_reg reg) return (u64)udw << 32 | ldw; } +#define __XE_MMIO_WAIT_MAX_BACKOFF_100MS (100 * USEC_PER_MSEC) static int __xe_mmio_wait32(struct xe_mmio *mmio, struct xe_reg reg, u32 mask, u32 val, u32 timeout_us, u32 *out_val, bool atomic, bool expect_match) { @@ -318,10 +319,10 @@ static int __xe_mmio_wait32(struct xe_mmio *mmio, struct xe_reg reg, u32 mask, u wait = ktime_us_delta(end, cur); if (atomic) - udelay(wait); + udelay(min_t(s64, wait, (MAX_UDELAY_MS * 1000))); else usleep_range(wait, wait << 1); - wait <<= 1; + wait = min_t(s64, wait << 1, __XE_MMIO_WAIT_MAX_BACKOFF_100MS); } if (ret != 0) { diff --git a/drivers/gpu/drm/xe/xe_sysctrl_mailbox.c b/drivers/gpu/drm/xe/xe_sysctrl_mailbox.c index 72baf1aa4b3a..a0179e62abaa 100644 --- a/drivers/gpu/drm/xe/xe_sysctrl_mailbox.c +++ b/drivers/gpu/drm/xe/xe_sysctrl_mailbox.c @@ -19,20 +19,20 @@ #include "xe_sysctrl_mailbox_types.h" struct xe_sysctrl_mailbox_msg_hdr { - __le32 data; -} __packed; + u32 data; +}; #define XE_SYSCTRL_HDR_GROUP_ID(hdr) \ - FIELD_GET(SYSCTRL_HDR_GROUP_ID_MASK, le32_to_cpu((hdr)->data)) + FIELD_GET(SYSCTRL_HDR_GROUP_ID_MASK, (hdr)->data) #define XE_SYSCTRL_HDR_COMMAND(hdr) \ - FIELD_GET(SYSCTRL_HDR_COMMAND_MASK, le32_to_cpu((hdr)->data)) + FIELD_GET(SYSCTRL_HDR_COMMAND_MASK, (hdr)->data) #define XE_SYSCTRL_HDR_IS_RESPONSE(hdr) \ - FIELD_GET(SYSCTRL_HDR_IS_RESPONSE, le32_to_cpu((hdr)->data)) + FIELD_GET(SYSCTRL_HDR_IS_RESPONSE, (hdr)->data) #define XE_SYSCTRL_HDR_RESULT(hdr) \ - FIELD_GET(SYSCTRL_HDR_RESULT_MASK, le32_to_cpu((hdr)->data)) + FIELD_GET(SYSCTRL_HDR_RESULT_MASK, (hdr)->data) static bool sysctrl_wait_bit_clear(struct xe_sysctrl *sc, u32 bit_mask, unsigned int timeout_ms) @@ -126,8 +126,8 @@ static int sysctrl_prepare_command(struct xe_device *xe, return -ENOMEM; hdr = (struct xe_sysctrl_mailbox_msg_hdr *)buffer; - hdr->data = cpu_to_le32(FIELD_PREP(SYSCTRL_HDR_GROUP_ID_MASK, group_id) | - FIELD_PREP(SYSCTRL_HDR_COMMAND_MASK, command)); + hdr->data = FIELD_PREP(SYSCTRL_HDR_GROUP_ID_MASK, group_id) | + FIELD_PREP(SYSCTRL_HDR_COMMAND_MASK, command); if (data_in && data_in_len) memcpy(buffer + sizeof(*hdr), data_in, data_in_len); diff --git a/drivers/hid/hid-steam.c b/drivers/hid/hid-steam.c index 80d2eabc930e..84c90091d45f 100644 --- a/drivers/hid/hid-steam.c +++ b/drivers/hid/hid-steam.c @@ -327,12 +327,14 @@ struct steam_device { spinlock_t lock; struct hid_device *hdev, *client_hdev; struct mutex report_mutex; + struct mutex registration_mutex; unsigned long client_opened; struct input_dev __rcu *input; struct input_dev __rcu *sensors; unsigned long quirks; struct work_struct work_connect; bool connected; + bool registered; char serial_no[STEAM_SERIAL_LEN + 1]; struct power_supply_desc battery_desc; struct power_supply __rcu *battery; @@ -346,7 +348,6 @@ struct steam_device { u16 rumble_right; unsigned int sensor_timestamp_us; unsigned int sensor_update_rate_us; - struct work_struct unregister_work; }; static int steam_recv_report(struct steam_device *steam, @@ -510,7 +511,6 @@ static int steam_get_serial(struct steam_device *steam) u8 cmd[] = {ID_GET_STRING_ATTRIBUTE, sizeof(steam->serial_no), ATTRIB_STR_UNIT_SERIAL}; u8 reply[3 + STEAM_SERIAL_LEN + 1]; - guard(mutex)(&steam->report_mutex); ret = steam_send_report(steam, cmd, sizeof(cmd)); if (ret < 0) return ret; @@ -537,7 +537,6 @@ static int steam_get_attributes(struct steam_device *steam) int i; struct steam_controller_attribute *attr; - guard(mutex)(&steam->report_mutex); ret = steam_send_report(steam, cmd, sizeof(cmd)); if (ret < 0) return ret; @@ -599,9 +598,7 @@ static inline int steam_haptic_pulse(struct steam_device *steam, u8 pad, report[8] = count >> 8; report[9] = gain; - mutex_lock(&steam->report_mutex); ret = steam_send_report(steam, report, sizeof(report)); - mutex_unlock(&steam->report_mutex); return ret; } @@ -621,9 +618,7 @@ static inline int steam_haptic_rumble(struct steam_device *steam, report[9] = left_gain; report[10] = right_gain; - mutex_lock(&steam->report_mutex); ret = steam_send_report(steam, report, sizeof(report)); - mutex_unlock(&steam->report_mutex); return ret; } @@ -631,8 +626,11 @@ static void steam_haptic_rumble_cb(struct work_struct *work) { struct steam_device *steam = container_of(work, struct steam_device, rumble_work); + + mutex_lock(&steam->report_mutex); steam_haptic_rumble(steam, 0, steam->rumble_left, steam->rumble_right, 2, 0); + mutex_unlock(&steam->report_mutex); } #ifdef CONFIG_STEAM_FF @@ -653,7 +651,6 @@ static void steam_set_lizard_mode(struct steam_device *steam, bool enable) if (steam->gamepad_mode) enable = false; - mutex_lock(&steam->report_mutex); if (enable) { /* enable esc, enter, cursors */ steam_send_report_byte(steam, ID_SET_DEFAULT_DIGITAL_MAPPINGS); @@ -678,14 +675,13 @@ static void steam_set_lizard_mode(struct steam_device *steam, bool enable) 0); } } - mutex_unlock(&steam->report_mutex); } static int steam_input_open(struct input_dev *dev) { struct steam_device *steam = input_get_drvdata(dev); unsigned long flags; - bool set_lizard_mode; + bool client_opened; /* * Disabling lizard mode automatically is only done on the Steam @@ -694,9 +690,10 @@ static int steam_input_open(struct input_dev *dev) */ if (!(steam->quirks & STEAM_QUIRK_DECK)) { spin_lock_irqsave(&steam->lock, flags); - set_lizard_mode = !steam->client_opened && lizard_mode; + client_opened = steam->client_opened; spin_unlock_irqrestore(&steam->lock, flags); - if (set_lizard_mode) + guard(mutex)(&steam->report_mutex); + if (!client_opened && lizard_mode) steam_set_lizard_mode(steam, false); } @@ -707,13 +704,14 @@ static void steam_input_close(struct input_dev *dev) { struct steam_device *steam = input_get_drvdata(dev); unsigned long flags; - bool set_lizard_mode; + bool client_opened; if (!(steam->quirks & STEAM_QUIRK_DECK)) { spin_lock_irqsave(&steam->lock, flags); - set_lizard_mode = !steam->client_opened && lizard_mode; + client_opened = steam->client_opened; spin_unlock_irqrestore(&steam->lock, flags); - if (set_lizard_mode) + guard(mutex)(&steam->report_mutex); + if (!client_opened && lizard_mode) steam_set_lizard_mode(steam, true); } } @@ -721,14 +719,11 @@ static void steam_input_close(struct input_dev *dev) static int steam_sensor_open(struct input_dev *dev) { struct steam_device *steam = input_get_drvdata(dev); - unsigned long flags; - bool client_opened; - spin_lock_irqsave(&steam->lock, flags); - client_opened = steam->client_opened; - spin_unlock_irqrestore(&steam->lock, flags); - if (client_opened) - return 0; + scoped_guard(spinlock_irqsave, &steam->lock) { + if (steam->client_opened) + return 0; + } guard(mutex)(&steam->report_mutex); steam_write_settings(steam, SETTING_IMU_MODE, @@ -741,14 +736,11 @@ static int steam_sensor_open(struct input_dev *dev) static void steam_sensor_close(struct input_dev *dev) { struct steam_device *steam = input_get_drvdata(dev); - unsigned long flags; - bool client_opened; - spin_lock_irqsave(&steam->lock, flags); - client_opened = steam->client_opened; - spin_unlock_irqrestore(&steam->lock, flags); - if (client_opened) - return; + scoped_guard(spinlock_irqsave, &steam->lock) { + if (steam->client_opened) + return; + } guard(mutex)(&steam->report_mutex); steam_write_settings(steam, SETTING_IMU_MODE, 0, 0); @@ -823,6 +815,7 @@ static int steam_battery_register(struct steam_device *steam) &steam->battery_desc, &battery_cfg); if (IS_ERR(battery)) { ret = PTR_ERR(battery); + devm_kfree(&steam->hdev->dev, steam->battery_desc.name); hid_err(steam->hdev, "%s:power_supply_register failed with error %d\n", __func__, ret); @@ -1082,103 +1075,117 @@ static void steam_battery_unregister(struct steam_device *steam) RCU_INIT_POINTER(steam->battery, NULL); synchronize_rcu(); power_supply_unregister(battery); + devm_kfree(&steam->hdev->dev, steam->battery_desc.name); } static int steam_register(struct steam_device *steam) { int ret; - unsigned long client_opened; - unsigned long flags; + mutex_lock(&steam->registration_mutex); /* * This function can be called several times in a row with the * wireless adaptor, without steam_unregister() between them, because * another client send a get_connection_status command, for example. * The battery and serial number are set just once per device. */ - if (!steam->serial_no[0]) { - /* - * Unlikely, but getting the serial could fail, and it is not so - * important, so make up a serial number and go on. - */ - if (steam_get_serial(steam) < 0) - strscpy(steam->serial_no, "XXXXXXXXXX", - sizeof(steam->serial_no)); + if (steam->registered) { + mutex_unlock(&steam->registration_mutex); + return 0; + } - ret = steam_get_attributes(steam); - if (ret < 0) - hid_err(steam->hdev, - "%s:steam_get_attributes failed with error %d\n", - __func__, ret); + mutex_lock(&steam->report_mutex); + /* + * Unlikely, but getting the serial could fail, and it is not so + * important, so make up a serial number and go on. + */ + if (steam_get_serial(steam) < 0 || !steam->serial_no[0]) + strscpy(steam->serial_no, "XXXXXXXXXX", + sizeof(steam->serial_no)); - hid_info(steam->hdev, "Steam Controller '%s' connected", - steam->serial_no); + ret = steam_get_attributes(steam); + if (ret < 0) + hid_err(steam->hdev, + "%s:steam_get_attributes failed with error %d\n", + __func__, ret); - /* ignore battery errors, we can live without it */ - if (steam->quirks & STEAM_QUIRK_WIRELESS) - steam_battery_register(steam); + hid_info(steam->hdev, "Steam Controller '%s' connected", + steam->serial_no); - mutex_lock(&steam_devices_lock); - if (list_empty(&steam->list)) - list_add(&steam->list, &steam_devices); - mutex_unlock(&steam_devices_lock); - } + steam_set_lizard_mode(steam, lizard_mode); + mutex_unlock(&steam->report_mutex); - spin_lock_irqsave(&steam->lock, flags); - client_opened = steam->client_opened; - spin_unlock_irqrestore(&steam->lock, flags); + /* ignore battery errors, we can live without it */ + if (steam->quirks & STEAM_QUIRK_WIRELESS) + steam_battery_register(steam); - if (!client_opened) { - steam_set_lizard_mode(steam, lizard_mode); - ret = steam_input_register(steam); - if (ret != 0) - goto steam_register_input_fail; - ret = steam_sensors_register(steam); - if (ret != 0) - goto steam_register_sensors_fail; - } + ret = steam_input_register(steam); + if (ret != 0) + goto steam_register_input_fail; + ret = steam_sensors_register(steam); + if (ret != 0) + goto steam_register_sensors_fail; + + steam->registered = true; + mutex_unlock(&steam->registration_mutex); + mutex_lock(&steam_devices_lock); + if (list_empty(&steam->list)) + list_add(&steam->list, &steam_devices); + mutex_unlock(&steam_devices_lock); return 0; steam_register_sensors_fail: steam_input_unregister(steam); steam_register_input_fail: + steam_battery_unregister(steam); + mutex_unlock(&steam->registration_mutex); + cancel_work_sync(&steam->rumble_work); + cancel_delayed_work_sync(&steam->mode_switch); return ret; } static void steam_unregister(struct steam_device *steam) { + if (!steam->registered) + return; + + hid_info(steam->hdev, "Steam Controller '%s' disconnected", + steam->serial_no); + mutex_lock(&steam->registration_mutex); + steam->registered = false; steam_battery_unregister(steam); steam_sensors_unregister(steam); steam_input_unregister(steam); - if (steam->serial_no[0]) { - hid_info(steam->hdev, "Steam Controller '%s' disconnected", - steam->serial_no); - mutex_lock(&steam_devices_lock); - list_del_init(&steam->list); - mutex_unlock(&steam_devices_lock); - steam->serial_no[0] = 0; - } + mutex_unlock(&steam->registration_mutex); + cancel_work_sync(&steam->rumble_work); + cancel_delayed_work_sync(&steam->mode_switch); + mutex_lock(&steam_devices_lock); + list_del_init(&steam->list); + mutex_unlock(&steam_devices_lock); + steam->serial_no[0] = 0; } static void steam_work_connect_cb(struct work_struct *work) { struct steam_device *steam = container_of(work, struct steam_device, work_connect); + unsigned long flags; bool connected; + bool opened; int ret; spin_lock_irqsave(&steam->lock, flags); + opened = steam->client_opened; connected = steam->connected; spin_unlock_irqrestore(&steam->lock, flags); - if (connected) { + if (connected && !opened) { ret = steam_register(steam); - if (ret) { + if (ret) hid_err(steam->hdev, "%s:steam_register failed with error %d\n", __func__, ret); - } } else { steam_unregister(steam); } @@ -1190,54 +1197,32 @@ static void steam_mode_switch_cb(struct work_struct *work) struct steam_device, mode_switch); unsigned long flags; bool client_opened; + bool gamepad_mode; + if (!lizard_mode) return; + spin_lock_irqsave(&steam->lock, flags); steam->gamepad_mode = !steam->gamepad_mode; - hid_dbg(steam->hdev, "%s: switching gamepad mode to %i\n", __func__, steam->gamepad_mode); - if (steam->gamepad_mode) + gamepad_mode = steam->gamepad_mode; + client_opened = steam->client_opened; + spin_unlock_irqrestore(&steam->lock, flags); + + guard(mutex)(&steam->report_mutex); + hid_dbg(steam->hdev, "%s: switching gamepad mode to %i\n", __func__, gamepad_mode); + if (gamepad_mode) steam_set_lizard_mode(steam, false); - else { - spin_lock_irqsave(&steam->lock, flags); - client_opened = steam->client_opened; - spin_unlock_irqrestore(&steam->lock, flags); - if (!client_opened) - steam_set_lizard_mode(steam, lizard_mode); - } + else if (!client_opened) + steam_set_lizard_mode(steam, lizard_mode); steam_haptic_pulse(steam, STEAM_PAD_RIGHT, 0x190, 0, 1, 0); - if (steam->gamepad_mode) { + if (gamepad_mode) { steam_haptic_pulse(steam, STEAM_PAD_LEFT, 0x14D, 0x14D, 0x2D, 0); } else { steam_haptic_pulse(steam, STEAM_PAD_LEFT, 0x1F4, 0x1F4, 0x1E, 0); } } -static void steam_work_unregister_cb(struct work_struct *work) -{ - struct steam_device *steam = container_of(work, struct steam_device, - unregister_work); - unsigned long flags; - bool connected; - bool opened; - - spin_lock_irqsave(&steam->lock, flags); - opened = steam->client_opened; - connected = steam->connected; - spin_unlock_irqrestore(&steam->lock, flags); - - if (connected) { - if (opened) { - steam_sensors_unregister(steam); - steam_input_unregister(steam); - } else { - steam_set_lizard_mode(steam, lizard_mode); - steam_input_register(steam); - steam_sensors_register(steam); - } - } -} - static bool steam_is_valve_interface(struct hid_device *hdev) { struct hid_report_enum *rep_enum; @@ -1283,7 +1268,7 @@ static int steam_client_ll_open(struct hid_device *hdev) steam->client_opened++; spin_unlock_irqrestore(&steam->lock, flags); - schedule_work(&steam->unregister_work); + schedule_work(&steam->work_connect); return 0; } @@ -1298,7 +1283,7 @@ static void steam_client_ll_close(struct hid_device *hdev) steam->client_opened--; spin_unlock_irqrestore(&steam->lock, flags); - schedule_work(&steam->unregister_work); + schedule_work(&steam->work_connect); } static int steam_client_ll_raw_request(struct hid_device *hdev, @@ -1384,6 +1369,7 @@ static int steam_probe(struct hid_device *hdev, hid_set_drvdata(hdev, steam); spin_lock_init(&steam->lock); mutex_init(&steam->report_mutex); + mutex_init(&steam->registration_mutex); steam->quirks = id->driver_data; INIT_WORK(&steam->work_connect, steam_work_connect_cb); INIT_DELAYED_WORK(&steam->mode_switch, steam_mode_switch_cb); @@ -1394,7 +1380,6 @@ static int steam_probe(struct hid_device *hdev, steam->sensor_update_rate_us = 4000; else steam->sensor_update_rate_us = 9000; - INIT_WORK(&steam->unregister_work, steam_work_unregister_cb); /* * With the real steam controller interface, do not connect hidraw. @@ -1455,7 +1440,6 @@ static int steam_probe(struct hid_device *hdev, cancel_work_sync(&steam->work_connect); cancel_delayed_work_sync(&steam->mode_switch); cancel_work_sync(&steam->rumble_work); - cancel_work_sync(&steam->unregister_work); return ret; } @@ -1463,36 +1447,32 @@ static int steam_probe(struct hid_device *hdev, static void steam_remove(struct hid_device *hdev) { struct steam_device *steam = hid_get_drvdata(hdev); + unsigned long flags; if (!steam || hdev->group == HID_GROUP_STEAM) { hid_hw_stop(hdev); return; } + hid_hw_close(hdev); hid_destroy_device(steam->client_hdev); - cancel_delayed_work_sync(&steam->mode_switch); - cancel_work_sync(&steam->work_connect); - cancel_work_sync(&steam->rumble_work); - cancel_work_sync(&steam->unregister_work); - steam->client_hdev = NULL; + spin_lock_irqsave(&steam->lock, flags); steam->client_opened = 0; + spin_unlock_irqrestore(&steam->lock, flags); + cancel_work_sync(&steam->work_connect); if (steam->quirks & STEAM_QUIRK_WIRELESS) { hid_info(hdev, "Steam wireless receiver disconnected"); } - hid_hw_close(hdev); - hid_hw_stop(hdev); steam_unregister(steam); + hid_hw_stop(hdev); } static void steam_do_connect_event(struct steam_device *steam, bool connected) { - unsigned long flags; bool changed; - spin_lock_irqsave(&steam->lock, flags); changed = steam->connected != connected; steam->connected = connected; - spin_unlock_irqrestore(&steam->lock, flags); if (changed && schedule_work(&steam->work_connect) == 0) dbg_hid("%s: connected=%d event already queued\n", @@ -1921,8 +1901,6 @@ static void steam_do_deck_sensors_event(struct steam_device *steam, static void steam_do_battery_event(struct steam_device *steam, struct power_supply *battery, u8 *data) { - unsigned long flags; - s16 volts = steam_le16(data + 12); u8 batt = data[14]; @@ -1930,10 +1908,8 @@ static void steam_do_battery_event(struct steam_device *steam, rcu_read_lock(); battery = rcu_dereference(steam->battery); if (likely(battery)) { - spin_lock_irqsave(&steam->lock, flags); steam->voltage = volts; steam->battery_charge = batt; - spin_unlock_irqrestore(&steam->lock, flags); power_supply_changed(battery); } rcu_read_unlock(); @@ -1951,6 +1927,7 @@ static int steam_raw_event(struct hid_device *hdev, if (!steam) return 0; + guard(spinlock_irqsave)(&steam->lock); if (steam->client_opened) hid_input_report(steam->client_hdev, HID_FEATURE_REPORT, data, size, 0); @@ -2038,6 +2015,8 @@ static int steam_param_set_lizard_mode(const char *val, { struct steam_device *steam; int ret; + bool client_opened; + unsigned long flags; ret = param_set_bool(val, kp); if (ret) @@ -2045,8 +2024,13 @@ static int steam_param_set_lizard_mode(const char *val, mutex_lock(&steam_devices_lock); list_for_each_entry(steam, &steam_devices, list) { - if (!steam->client_opened) + spin_lock_irqsave(&steam->lock, flags); + client_opened = steam->client_opened; + spin_unlock_irqrestore(&steam->lock, flags); + if (!client_opened) { + guard(mutex)(&steam->report_mutex); steam_set_lizard_mode(steam, lizard_mode); + } } mutex_unlock(&steam_devices_lock); return 0; diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c index 6406ceab988a..810e52782ae8 100644 --- a/drivers/input/joystick/xpad.c +++ b/drivers/input/joystick/xpad.c @@ -302,6 +302,8 @@ static const struct xpad_device { { 0x17ef, 0x6182, "Lenovo Legion Controller for Windows", 0, XTYPE_XBOX360 }, { 0x1949, 0x041a, "Amazon Game Controller", 0, XTYPE_XBOX360 }, { 0x1a86, 0xe310, "Legion Go S", 0, XTYPE_XBOX360 }, + { 0x1b1c, 0x2b1f, "Corsair Novablade Pro (Wired)", 0, XTYPE_XBOX360 }, + { 0x1b1c, 0x2b2b, "Corsair Novablade Pro (2.4 GHz Receiver)", 0, XTYPE_XBOX360 }, { 0x1bad, 0x0002, "Harmonix Rock Band Guitar", 0, XTYPE_XBOX360 }, { 0x1bad, 0x0003, "Harmonix Rock Band Drumkit", MAP_DPAD_TO_BUTTONS, XTYPE_XBOX360 }, { 0x1bad, 0x0130, "Ion Drum Rocker", MAP_DPAD_TO_BUTTONS, XTYPE_XBOX360 }, @@ -379,6 +381,7 @@ static const struct xpad_device { { 0x2993, 0x2001, "TECNO Pocket Go", 0, XTYPE_XBOX360 }, { 0x2dc8, 0x2000, "8BitDo Pro 2 Wired Controller fox Xbox", 0, XTYPE_XBOXONE }, { 0x2dc8, 0x200f, "8BitDo Ultimate 3-mode Controller for Xbox", MAP_SHARE_BUTTON, XTYPE_XBOXONE }, + { 0x2dc8, 0x201e, "8BitDo Pro 2 Wired Controller for Xbox", MAP_SHARE_BUTTON, XTYPE_XBOXONE }, { 0x2dc8, 0x3106, "8BitDo Ultimate Wireless / Pro 2 Wired Controller", 0, XTYPE_XBOX360 }, { 0x2dc8, 0x3109, "8BitDo Ultimate Wireless Bluetooth", 0, XTYPE_XBOX360 }, { 0x2dc8, 0x310a, "8BitDo Ultimate 2C Wireless Controller", 0, XTYPE_XBOX360 }, @@ -545,6 +548,7 @@ static const struct usb_device_id xpad_table[] = { XPAD_XBOX360_VENDOR(0x17ef), /* Lenovo */ XPAD_XBOX360_VENDOR(0x1949), /* Amazon controllers */ XPAD_XBOX360_VENDOR(0x1a86), /* Nanjing Qinheng Microelectronics (WCH) */ + XPAD_XBOX360_VENDOR(0x1b1c), /* Corsair controllers */ XPAD_XBOX360_VENDOR(0x1bad), /* Harmonix Rock Band guitar and drums */ XPAD_XBOX360_VENDOR(0x1ee9), /* ZOTAC Technology Limited */ XPAD_XBOX360_VENDOR(0x20bc), /* BETOP wireless dongles */ diff --git a/drivers/input/misc/ati_remote2.c b/drivers/input/misc/ati_remote2.c index 8db2dca84975..3e0f874e9b16 100644 --- a/drivers/input/misc/ati_remote2.c +++ b/drivers/input/misc/ati_remote2.c @@ -672,8 +672,7 @@ static ssize_t ati_remote2_show_channel_mask(struct device *dev, struct device_attribute *attr, char *buf) { - struct usb_device *udev = to_usb_device(dev); - struct usb_interface *intf = usb_ifnum_to_if(udev, 0); + struct usb_interface *intf = to_usb_interface(dev); struct ati_remote2 *ar2 = usb_get_intfdata(intf); return sprintf(buf, "0x%04x\n", ar2->channel_mask); @@ -683,8 +682,7 @@ static ssize_t ati_remote2_store_channel_mask(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct usb_device *udev = to_usb_device(dev); - struct usb_interface *intf = usb_ifnum_to_if(udev, 0); + struct usb_interface *intf = to_usb_interface(dev); struct ati_remote2 *ar2 = usb_get_intfdata(intf); unsigned int mask; int r; @@ -720,8 +718,7 @@ static ssize_t ati_remote2_show_mode_mask(struct device *dev, struct device_attribute *attr, char *buf) { - struct usb_device *udev = to_usb_device(dev); - struct usb_interface *intf = usb_ifnum_to_if(udev, 0); + struct usb_interface *intf = to_usb_interface(dev); struct ati_remote2 *ar2 = usb_get_intfdata(intf); return sprintf(buf, "0x%02x\n", ar2->mode_mask); @@ -731,8 +728,7 @@ static ssize_t ati_remote2_store_mode_mask(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { - struct usb_device *udev = to_usb_device(dev); - struct usb_interface *intf = usb_ifnum_to_if(udev, 0); + struct usb_interface *intf = to_usb_interface(dev); struct ati_remote2 *ar2 = usb_get_intfdata(intf); unsigned int mask; int err; diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index ae814d8c99c0..5f901b4baf21 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1228,9 +1228,9 @@ dma_addr_t iommu_dma_map_phys(struct device *dev, phys_addr_t phys, size_t size, * If both the physical buffer start address and size are page aligned, * we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev, size, dir) && + if (!(attrs & DMA_ATTR_MMIO) && dev_use_swiotlb(dev, size, dir) && iova_unaligned(iovad, phys, size)) { - if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) + if (attrs & DMA_ATTR_REQUIRE_COHERENT) return DMA_MAPPING_ERROR; phys = iommu_dma_map_swiotlb(dev, phys, size, dir, attrs); diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 608b617fb817..ee36789a7f0d 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -2895,8 +2895,13 @@ static int crypt_ctr_cipher_new(struct dm_target *ti, char *cipher_in, char *key } } - if (*ivmode && !strcmp(*ivmode, "lmk")) + if (*ivmode && !strcmp(*ivmode, "lmk")) { + if (crypt_integrity_aead(cc)) { + ti->error = "AEAD transforms not supported for LMK"; + return -EINVAL; + } cc->tfms_count = 64; + } if (*ivmode && !strcmp(*ivmode, "essiv")) { if (!*ivopts) { diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 86fc35e76a0f..4bda7b5dba1a 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -2721,7 +2721,7 @@ static void dm_integrity_inline_recheck(struct work_struct *w) outgoing_data = dio->integrity_payload + PAGE_SIZE; while (dio->bio_details.bi_iter.bi_size) { - char digest[HASH_MAX_DIGESTSIZE]; + char digest[MAX_T(size_t, HASH_MAX_DIGESTSIZE, MAX_TAG_SIZE)]; int r; struct bio_integrity_payload *bip; struct bio_vec bv; @@ -2788,7 +2788,7 @@ static inline bool dm_integrity_check(struct dm_integrity_c *ic, struct dm_integ unsigned pos = 0; while (dio->bio_details.bi_iter.bi_size) { - char digest[HASH_MAX_DIGESTSIZE]; + char digest[MAX_T(size_t, HASH_MAX_DIGESTSIZE, MAX_TAG_SIZE)]; struct bio_vec bv = bio_iter_iovec(bio, dio->bio_details.bi_iter); char *mem = integrity_kmap(ic, bv.bv_page); integrity_sector_checksum(ic, &dio->ahash_req, dio->bio_details.bi_iter.bi_sector, mem, bv.bv_offset, digest); diff --git a/drivers/md/dm-ioctl.c b/drivers/md/dm-ioctl.c index 61af2a437a05..d3430825acad 100644 --- a/drivers/md/dm-ioctl.c +++ b/drivers/md/dm-ioctl.c @@ -658,10 +658,16 @@ static void *get_result_buffer(struct dm_ioctl *param, size_t param_size, { param->data_start = align_ptr(param + 1) - (void *) param; - if (param->data_start < param_size) + if (param->data_start < param_size) { *len = param_size - param->data_start; - else + } else { + /* + * The buffer can be as small as offsetof(*param, data), which + * is less than sizeof(*param), so don't run off the end of it. + */ *len = 0; + param->data_start = param_size; + } return ((void *) param) + param->data_start; } @@ -1289,6 +1295,7 @@ static int do_resume(struct dm_ioctl *param) /* Do we need to load a new map ? */ if (new_map) { sector_t old_size, new_size; + blk_mode_t new_map_mode; dm_ima_context_table_op(md, ima_context, DM_IMA_TABLE_SAVE); /* Suspend if it isn't already suspended */ @@ -1321,6 +1328,7 @@ static int do_resume(struct dm_ioctl *param) } } + new_map_mode = dm_table_get_mode(new_map); old_size = dm_get_size(md); old_map = dm_swap_table(md, new_map); if (IS_ERR(old_map)) { @@ -1336,7 +1344,7 @@ static int do_resume(struct dm_ioctl *param) if (old_size && new_size && old_size != new_size) need_resize_uevent = true; - if (dm_table_get_mode(new_map) & BLK_OPEN_WRITE) + if (new_map_mode & BLK_OPEN_WRITE) set_disk_ro(dm_disk(md), 0); else set_disk_ro(dm_disk(md), 1); diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c index 1489fda9d24a..02e3cc473026 100644 --- a/drivers/md/dm-snap.c +++ b/drivers/md/dm-snap.c @@ -1294,6 +1294,20 @@ static int snapshot_ctr(struct dm_target *ti, unsigned int argc, char **argv) goto bad_store; } + /* + * snapshot-merge requires the optional prepare_merge/commit_merge + * callbacks; the transient exception store does not implement them. + * The handover path checks this in __validate_exception_handover(), + * but a directly constructed snapshot-merge never goes through it. + */ + if (dm_target_is_snapshot_merge(ti) && + (!s->store->type->prepare_merge || + !s->store->type->commit_merge)) { + ti->error = "Snapshot exception store does not support snapshot-merge."; + r = -EINVAL; + goto bad_hash_tables; + } + argv += args_used; argc -= args_used; diff --git a/drivers/memstick/core/memstick.c b/drivers/memstick/core/memstick.c index e03989c4e99e..0d15f48ef51d 100644 --- a/drivers/memstick/core/memstick.c +++ b/drivers/memstick/core/memstick.c @@ -370,9 +370,7 @@ int memstick_set_rw_addr(struct memstick_dev *card) { card->next_request = h_memstick_set_rw_addr; memstick_new_req(card->host); - if (!wait_for_completion_timeout(&card->mrq_complete, - msecs_to_jiffies(500))) - card->current_mrq.error = -ETIMEDOUT; + wait_for_completion(&card->mrq_complete); return card->current_mrq.error; } @@ -405,9 +403,7 @@ static struct memstick_dev *memstick_alloc_card(struct memstick_host *host) card->next_request = h_memstick_read_dev_id; memstick_new_req(host); - if (!wait_for_completion_timeout(&card->mrq_complete, - msecs_to_jiffies(500))) - card->current_mrq.error = -ETIMEDOUT; + wait_for_completion(&card->mrq_complete); if (card->current_mrq.error) goto err_out; diff --git a/drivers/memstick/host/rtsx_usb_ms.c b/drivers/memstick/host/rtsx_usb_ms.c index beadc389f15f..d5b3a96fc609 100644 --- a/drivers/memstick/host/rtsx_usb_ms.c +++ b/drivers/memstick/host/rtsx_usb_ms.c @@ -27,7 +27,6 @@ struct rtsx_usb_ms { struct memstick_host *msh; struct memstick_request *req; - struct mutex host_mutex; struct work_struct handle_req; struct delayed_work poll_card; @@ -514,6 +513,13 @@ static void rtsx_usb_ms_handle_req(struct work_struct *work) struct memstick_host *msh = host->msh; int rc; + /* Fail requests after eject so their waiters are released. */ + if (host->eject) { + while (!memstick_next_req(msh, &host->req)) + host->req->error = -ENOMEDIUM; + return; + } + if (!host->req) { pm_runtime_get_sync(ms_dev(host)); do { @@ -547,8 +553,7 @@ static void rtsx_usb_ms_request(struct memstick_host *msh) dev_dbg(ms_dev(host), "--> %s\n", __func__); - if (!host->eject) - schedule_work(&host->handle_req); + schedule_work(&host->handle_req); } static int rtsx_usb_ms_set_param(struct memstick_host *msh, @@ -781,7 +786,6 @@ static int rtsx_usb_ms_drv_probe(struct platform_device *pdev) host->power_mode = MEMSTICK_POWER_OFF; platform_set_drvdata(pdev, host); - mutex_init(&host->host_mutex); INIT_WORK(&host->handle_req, rtsx_usb_ms_handle_req); INIT_DELAYED_WORK(&host->poll_card, rtsx_usb_ms_poll_card); @@ -812,27 +816,12 @@ static void rtsx_usb_ms_drv_remove(struct platform_device *pdev) { struct rtsx_usb_ms *host = platform_get_drvdata(pdev); struct memstick_host *msh = host->msh; - int err; host->eject = true; msh->removing = true; - cancel_work_sync(&host->handle_req); + flush_work(&host->handle_req); cancel_delayed_work_sync(&host->poll_card); - mutex_lock(&host->host_mutex); - if (host->req) { - dev_dbg(ms_dev(host), - "%s: Controller removed during transfer\n", - dev_name(&msh->dev)); - host->req->error = -ENOMEDIUM; - do { - err = memstick_next_req(msh, &host->req); - if (!err) - host->req->error = -ENOMEDIUM; - } while (!err); - } - mutex_unlock(&host->host_mutex); - /* Balance possible unbalanced usage count * e.g. unconditional module removal */ @@ -841,6 +830,8 @@ static void rtsx_usb_ms_drv_remove(struct platform_device *pdev) pm_runtime_disable(ms_dev(host)); memstick_remove_host(msh); + /* No card, no new requests; wait for the last failed one to finish. */ + cancel_work_sync(&host->handle_req); dev_dbg(ms_dev(host), ": Realtek USB Memstick controller has been removed\n"); memstick_free_host(msh); diff --git a/drivers/mmc/host/cavium-octeon.c b/drivers/mmc/host/cavium-octeon.c index 8a0daddd9200..1c219cf35a8e 100644 --- a/drivers/mmc/host/cavium-octeon.c +++ b/drivers/mmc/host/cavium-octeon.c @@ -301,9 +301,12 @@ static void octeon_mmc_remove(struct platform_device *pdev) u64 dma_cfg; int i; - for (i = 0; i < CAVIUM_MAX_MMC; i++) + for (i = 0; i < CAVIUM_MAX_MMC; i++) { if (host->slot[i]) cvm_mmc_of_slot_remove(host->slot[i]); + if (host->slot_pdev[i]) + of_platform_device_destroy(&host->slot_pdev[i]->dev, NULL); + } dma_cfg = readq(host->dma_base + MIO_EMM_DMA_CFG(host)); dma_cfg &= ~MIO_EMM_DMA_CFG_EN; diff --git a/drivers/mmc/host/cavium-thunderx.c b/drivers/mmc/host/cavium-thunderx.c index 84ff6d82ae3c..2635ac94ec5d 100644 --- a/drivers/mmc/host/cavium-thunderx.c +++ b/drivers/mmc/host/cavium-thunderx.c @@ -173,9 +173,12 @@ static void thunder_mmc_remove(struct pci_dev *pdev) u64 dma_cfg; int i; - for (i = 0; i < CAVIUM_MAX_MMC; i++) + for (i = 0; i < CAVIUM_MAX_MMC; i++) { if (host->slot[i]) cvm_mmc_of_slot_remove(host->slot[i]); + if (host->slot_pdev[i]) + of_platform_device_destroy(&host->slot_pdev[i]->dev, NULL); + } dma_cfg = readq(host->dma_base + MIO_EMM_DMA_CFG(host)); dma_cfg &= ~MIO_EMM_DMA_CFG_EN; diff --git a/drivers/mmc/host/mtk-sd.c b/drivers/mmc/host/mtk-sd.c index b2680cc054bd..d70e37eb1cb5 100644 --- a/drivers/mmc/host/mtk-sd.c +++ b/drivers/mmc/host/mtk-sd.c @@ -3216,6 +3216,7 @@ static void msdc_drv_remove(struct platform_device *pdev) platform_set_drvdata(pdev, NULL); mmc_remove_host(mmc); + cancel_delayed_work_sync(&host->req_timeout); msdc_deinit_hw(host); msdc_gate_clock(host); diff --git a/drivers/mmc/host/sdhci-sprd.c b/drivers/mmc/host/sdhci-sprd.c index 3584a2b314a9..0c8f389fe584 100644 --- a/drivers/mmc/host/sdhci-sprd.c +++ b/drivers/mmc/host/sdhci-sprd.c @@ -890,6 +890,10 @@ static void sdhci_sprd_remove(struct platform_device *pdev) struct sdhci_host *host = platform_get_drvdata(pdev); struct sdhci_sprd_host *sprd_host = TO_SPRD_HOST(host); + pm_runtime_get_sync(&pdev->dev); + pm_runtime_disable(&pdev->dev); + pm_runtime_put_noidle(&pdev->dev); + sdhci_remove_host(host, 0); clk_disable_unprepare(sprd_host->clk_sdio); diff --git a/drivers/net/amt.c b/drivers/net/amt.c index 7645ce3a29ad..189b3a43e4e2 100644 --- a/drivers/net/amt.c +++ b/drivers/net/amt.c @@ -80,15 +80,6 @@ static struct in6_addr mld2_all_node = MLD2_ALL_NODE_INIT; static struct mld2_grec mldv2_zero_grec; #endif -static struct amt_skb_cb *amt_skb_cb(struct sk_buff *skb) -{ - BUILD_BUG_ON(sizeof(struct amt_skb_cb) + sizeof(struct tc_skb_cb) > - sizeof_field(struct sk_buff, cb)); - - return (struct amt_skb_cb *)((void *)skb->cb + - sizeof(struct tc_skb_cb)); -} - static void __amt_source_gc_work(void) { struct amt_source_node *snode; @@ -789,6 +780,11 @@ static void amt_send_request(struct amt_dev *amt, bool v6) rcu_read_unlock(); } +static bool amt_send_membership_query(struct amt_dev *amt, + struct sk_buff *skb, + struct amt_tunnel_list *tunnel, + bool v6); + static void amt_send_igmp_gq(struct amt_dev *amt, struct amt_tunnel_list *tunnel) { @@ -798,8 +794,11 @@ static void amt_send_igmp_gq(struct amt_dev *amt, if (!skb) return; - amt_skb_cb(skb)->tunnel = tunnel; - dev_queue_xmit(skb); + skb_pull(skb, sizeof(struct ethhdr)); + if (amt_send_membership_query(amt, skb, tunnel, false)) { + amt->dev->stats.tx_dropped++; + kfree_skb(skb); + } } #if IS_ENABLED(CONFIG_IPV6) @@ -883,8 +882,11 @@ static void amt_send_mld_gq(struct amt_dev *amt, struct amt_tunnel_list *tunnel) if (!skb) return; - amt_skb_cb(skb)->tunnel = tunnel; - dev_queue_xmit(skb); + skb_pull(skb, sizeof(struct ethhdr)); + if (amt_send_membership_query(amt, skb, tunnel, true)) { + amt->dev->stats.tx_dropped++; + kfree_skb(skb); + } } #else static void amt_send_mld_gq(struct amt_dev *amt, struct amt_tunnel_list *tunnel) @@ -1183,7 +1185,6 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) #endif bool report = false; struct igmphdr *ih; - bool query = false; struct iphdr *iph; bool data = false; bool v6 = false; @@ -1201,9 +1202,6 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) case IGMP_HOST_MEMBERSHIP_REPORT: report = true; break; - case IGMP_HOST_MEMBERSHIP_QUERY: - query = true; - break; default: goto free; } @@ -1225,9 +1223,6 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) case ICMPV6_MLD2_REPORT: report = true; break; - case ICMPV6_MGM_QUERY: - query = true; - break; default: goto free; } @@ -1258,19 +1253,6 @@ static netdev_tx_t amt_dev_xmit(struct sk_buff *skb, struct net_device *dev) goto free; goto unlock; } else if (amt->mode == AMT_MODE_RELAY) { - if (query) { - tunnel = amt_skb_cb(skb)->tunnel; - if (!tunnel) { - WARN_ON(1); - goto free; - } - - /* Do not forward unexpected query */ - if (amt_send_membership_query(amt, skb, tunnel, v6)) - goto free; - goto unlock; - } - if (!data) goto free; list_for_each_entry_rcu(tunnel, &amt->tunnel_list, list) { diff --git a/drivers/net/bareudp.c b/drivers/net/bareudp.c index 5ef841c85526..2bfad8653be6 100644 --- a/drivers/net/bareudp.c +++ b/drivers/net/bareudp.c @@ -355,7 +355,7 @@ static int bareudp_xmit_skb(struct sk_buff *skb, struct net_device *dev, if (err) goto free_dst; - skb_set_inner_protocol(skb, bareudp->ethertype); + skb_set_inner_protocol(skb, skb->protocol); udp_tunnel_xmit_skb(rt, sk, skb, saddr, info->key.u.ipv4.dst, tos, ttl, df, sport, bareudp->port, !net_eq(bareudp->net, dev_net(bareudp->dev)), @@ -423,6 +423,7 @@ static int bareudp6_xmit_skb(struct sk_buff *skb, struct net_device *dev, if (err) goto free_dst; + skb_set_inner_protocol(skb, skb->protocol); daddr = info->key.u.ipv6.dst; udp_tunnel6_xmit_skb(dst, sk, skb, dev, &saddr, &daddr, prio, ttl, diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index 34c5bbbed9ae..507d655eca21 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -5738,12 +5738,12 @@ static int bond_xdp_set(struct net_device *dev, struct bpf_prog *prog, bpf_prog_inc(prog); } - if (prog) { + if (prog && !old_prog) static_branch_inc(&bpf_master_redirect_enabled_key); - } else if (old_prog) { - bpf_prog_put(old_prog); + else if (!prog && old_prog) static_branch_dec(&bpf_master_redirect_enabled_key); - } + if (old_prog) + bpf_prog_put(old_prog); return 0; diff --git a/drivers/net/can/dev/skb.c b/drivers/net/can/dev/skb.c index 95fcdc1026f8..14edec5afb57 100644 --- a/drivers/net/can/dev/skb.c +++ b/drivers/net/can/dev/skb.c @@ -210,6 +210,10 @@ static void init_can_skb(struct sk_buff *skb) { skb->pkt_type = PACKET_BROADCAST; skb->ip_summed = CHECKSUM_UNNECESSARY; + + skb_reset_mac_header(skb); + skb_reset_network_header(skb); + skb_reset_transport_header(skb); } struct sk_buff *alloc_can_skb(struct net_device *dev, struct can_frame **cf) diff --git a/drivers/net/can/usb/f81604.c b/drivers/net/can/usb/f81604.c index f12318268e46..4c147b9d6d69 100644 --- a/drivers/net/can/usb/f81604.c +++ b/drivers/net/can/usb/f81604.c @@ -169,7 +169,7 @@ struct f81604_int_data { u8 rxerr; u8 txerr; u8 val; -} __packed __aligned(4); +} __packed; struct f81604_sff { __be16 id; diff --git a/drivers/net/can/vxcan.c b/drivers/net/can/vxcan.c index e882250180ef..f365deafaf54 100644 --- a/drivers/net/can/vxcan.c +++ b/drivers/net/can/vxcan.c @@ -78,6 +78,9 @@ static netdev_tx_t vxcan_xmit(struct sk_buff *oskb, struct net_device *dev) /* reset CAN GW hop counter */ csx->can_gw_hops = 0; + /* start with new CAN skb UID in the other namespace */ + csx->can_skb_uid = 0; + skb->pkt_type = PACKET_BROADCAST; skb->dev = peer; skb->ip_summed = CHECKSUM_UNNECESSARY; diff --git a/drivers/net/dsa/microchip/ksz_common.h b/drivers/net/dsa/microchip/ksz_common.h index b4a5673ba365..8f6ac13b5b18 100644 --- a/drivers/net/dsa/microchip/ksz_common.h +++ b/drivers/net/dsa/microchip/ksz_common.h @@ -778,7 +778,7 @@ static inline bool ksz_is_sgmii_port(struct ksz_device *dev, int port) #define KSZ88_FAMILY_ID 0x88 #define KSZ8895_FAMILY_ID 0x95 -#define KSZ8_PORT_STATUS_0 0x08 +#define KSZ8_PORT_STATUS_0 0x18 #define KSZ8_PORT_FIBER_MODE BIT(7) #define SW_CHIP_ID_M GENMASK(7, 4) diff --git a/drivers/net/ethernet/airoha/airoha_eth.c b/drivers/net/ethernet/airoha/airoha_eth.c index 2fc8c91f210c..d4faf4503487 100644 --- a/drivers/net/ethernet/airoha/airoha_eth.c +++ b/drivers/net/ethernet/airoha/airoha_eth.c @@ -2716,14 +2716,34 @@ static int airoha_qdma_set_trtcm_param(struct airoha_qdma *qdma, int channel, FIELD_PREP(TRTCM_METER_GROUP_MASK, group) | FIELD_PREP(TRTCM_PARAM_INDEX_MASK, idx) | FIELD_PREP(TRTCM_PARAM_RATE_TYPE_MASK, mode); + int i; - airoha_qdma_wr(qdma, REG_TRTCM_DATA_LOW(addr), val); - airoha_qdma_wr(qdma, REG_TRTCM_CFG_PARAM(addr), config); + /* + * HW might be in the middle of bucket handling and might + * require an additional try to actually apply the new TRTCM + * config. + */ + for (i = 0; i < TRTCM_CONFIG_RETRY_MAX; i++) { + u32 data = 0; - return read_poll_timeout(airoha_qdma_rr, val, - val & TRTCM_PARAM_RW_DONE_MASK, - USEC_PER_MSEC, 10 * USEC_PER_MSEC, true, - qdma, REG_TRTCM_CFG_PARAM(addr)); + airoha_qdma_wr(qdma, REG_TRTCM_DATA_LOW(addr), val); + airoha_qdma_wr(qdma, REG_TRTCM_CFG_PARAM(addr), config); + + if (read_poll_timeout(airoha_qdma_rr, data, + data & TRTCM_PARAM_RW_DONE_MASK, + USEC_PER_MSEC, 10 * USEC_PER_MSEC, + true, qdma, REG_TRTCM_CFG_PARAM(addr))) + continue; + + if (airoha_qdma_get_trtcm_param(qdma, channel, addr, param, + mode, &data, NULL)) + continue; + + if (data == val) + return 0; + } + + return -EBUSY; } static int airoha_qdma_set_trtcm_config(struct airoha_qdma *qdma, int channel, diff --git a/drivers/net/ethernet/airoha/airoha_eth.h b/drivers/net/ethernet/airoha/airoha_eth.h index 70f16189b258..dc0c29001f81 100644 --- a/drivers/net/ethernet/airoha/airoha_eth.h +++ b/drivers/net/ethernet/airoha/airoha_eth.h @@ -171,6 +171,8 @@ enum trtcm_param { #define TRTCM_TOKEN_RATE_MASK GENMASK(23, 6) #define TRTCM_TOKEN_RATE_FRACTION_MASK GENMASK(5, 0) +#define TRTCM_CONFIG_RETRY_MAX 2 + enum airoha_dma_map_type { AIROHA_DMA_UNMAPPED, AIROHA_DMA_MAP_SINGLE, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index ced998399efc..78583cee3414 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -486,7 +486,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) struct netdev_queue *txq; int i; dma_addr_t mapping; - unsigned int length, pad = 0; + unsigned int length; u32 len, free_size, vlan_tag_flags, cfa_action, flags; struct bnxt_ptp_cfg *ptp = bp->ptp_cfg; struct pci_dev *pdev = bp->pdev; @@ -536,6 +536,16 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) return rc < 0 ? NETDEV_TX_BUSY : NETDEV_TX_OK; } + /* Pad after the SW USO branch: bnxt_sw_udp_gso_xmit() would + * otherwise account the padding as UDP payload. + * Must be done before skb_shinfo(skb)->nr_frags is sampled, + * because skb_put_padto() might linearize the skb. + */ + if (skb_put_padto(skb, BNXT_MIN_PKT_SIZE)) { + /* SKB already freed. */ + goto tx_kick_pending; + } + free_size = bnxt_tx_avail(bp, txr); if (unlikely(free_size < skb_shinfo(skb)->nr_frags + 2)) { /* We must have raced with NAPI cleanup */ @@ -672,14 +682,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) } normal_tx: - if (length < BNXT_MIN_PKT_SIZE) { - pad = BNXT_MIN_PKT_SIZE - length; - if (skb_pad(skb, pad)) - /* SKB already freed. */ - goto tx_kick_pending; - length = BNXT_MIN_PKT_SIZE; - } - mapping = dma_map_single(&pdev->dev, skb->data, len, DMA_TO_DEVICE); if (unlikely(dma_mapping_error(&pdev->dev, mapping))) @@ -759,10 +761,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txbd->tx_bd_len_flags_type = cpu_to_le32(flags); } - flags &= ~TX_BD_LEN; - txbd->tx_bd_len_flags_type = - cpu_to_le32(((len + pad) << TX_BD_LEN_SHIFT) | flags | - TX_BD_FLAGS_PACKET_END); + txbd->tx_bd_len_flags_type |= cpu_to_le32(TX_BD_FLAGS_PACKET_END); netdev_tx_sent_queue(txq, skb->len); @@ -15506,7 +15505,7 @@ static void bnxt_fw_reset_task(struct work_struct *work) if (test_and_clear_bit(BNXT_STATE_FW_ACTIVATE_RESET, &bp->state) && !test_bit(BNXT_STATE_FW_ACTIVATE, &bp->state)) bnxt_dl_remote_reload(bp); - if (pci_enable_device(bp->pdev)) { + if (!pci_is_enabled(bp->pdev) && pci_enable_device(bp->pdev)) { netdev_err(bp->dev, "Cannot re-enable PCI device\n"); rc = -ENODEV; goto fw_reset_abort; @@ -17067,6 +17066,51 @@ void bnxt_print_device_info(struct bnxt *bp) pcie_print_link_status(bp->pdev); } +static void bnxt_clear_bars(struct pci_dev *pdev) +{ + int off; + + for (off = PCI_BASE_ADDRESS_0; off <= PCI_BASE_ADDRESS_5; off += 4) + pci_write_config_dword(pdev, off, 0); +} + +/* Clear any pending DMA transactions from crash kernel while loading driver in + * capture kernel. + */ +static int bnxt_kdump_reset(struct pci_dev *pdev) +{ + int rc, i; + u16 cmd; + + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + cmd &= ~(PCI_COMMAND_MASTER | PCI_COMMAND_MEMORY); + pci_write_config_word(pdev, PCI_COMMAND, cmd); + + if (pci_save_state(pdev)) + dev_warn(&pdev->dev, "Failed to save PCI state, PCI restore may be incomplete\n"); + + rc = pcie_flr(pdev); + if (rc) + dev_warn(&pdev->dev, "pcie_flr() failed (rc: %d), trying to continue\n", + rc); + + /* In case device is not returning CRS, wait 5 seconds longer */ + for (i = 0; i < 50; i++) { + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + if (!PCI_POSSIBLE_ERROR(cmd)) + break; + msleep(100); + } + if (PCI_POSSIBLE_ERROR(cmd)) { + dev_err(&pdev->dev, "PCI config space inaccessible after FLR, aborting\n"); + return -ENODEV; + } + + bnxt_clear_bars(pdev); + pci_restore_state(pdev); + return 0; +} + static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) { struct bnxt_hw_resc *hw_resc; @@ -17082,12 +17126,10 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) return -ENODEV; } - /* Clear any pending DMA transactions from crash kernel - * while loading driver in capture kernel. - */ if (is_kdump_kernel()) { - pci_clear_master(pdev); - pcie_flr(pdev); + rc = bnxt_kdump_reset(pdev); + if (rc) + return rc; } max_irqs = bnxt_get_max_irq(pdev); @@ -17515,10 +17557,8 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev, * so we disable bus master to prevent any potential bad DMAs before * freeing kernel memory. */ - if (state == pci_channel_io_frozen) { - set_bit(BNXT_STATE_PCI_CHANNEL_IO_FROZEN, &bp->state); + if (state == pci_channel_io_frozen) bnxt_fw_fatal_close(bp); - } if (netif_running(netdev)) __bnxt_close_nic(bp, true, true); @@ -17548,69 +17588,80 @@ static pci_ers_result_t bnxt_io_slot_reset(struct pci_dev *pdev) struct bnxt *bp = netdev_priv(netdev); int retry = 0; int err = 0; - int off; + u16 cmd; netdev_info(bp->dev, "PCI Slot Reset\n"); - if (test_bit(BNXT_STATE_PCI_CHANNEL_IO_FROZEN, &bp->state)) { - /* After DPC, the chip should return CRS when the vendor ID - * config register is read until it is ready. On all chips, - * this is not happening reliably so add a 5-second delay as a - * workaround. - */ - msleep(5000); - } + /* After a PCIe hot reset, the chip should return CRS when the + * vendor ID config register is read until it is ready. On all + * chips, this is not happening reliably so add a 5-second delay + * as a workaround. + */ + msleep(5000); netdev_lock(netdev); - if (pci_enable_device(pdev)) { + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + if (PCI_POSSIBLE_ERROR(cmd)) { dev_err(&pdev->dev, - "Cannot re-enable PCI device after reset.\n"); - } else { - pci_set_master(pdev); - /* Upon fatal error, our device internal logic that latches to - * BAR value is getting reset and will restore only upon - * rewriting the BARs. - * - * As pci_restore_state() does not re-write the BARs if the - * value is same as saved value earlier, driver needs to - * write the BARs to 0 to force restore, in case of fatal error. - */ - if (test_and_clear_bit(BNXT_STATE_PCI_CHANNEL_IO_FROZEN, - &bp->state)) { - for (off = PCI_BASE_ADDRESS_0; - off <= PCI_BASE_ADDRESS_5; off += 4) - pci_write_config_dword(bp->pdev, off, 0); - } - pci_restore_state(pdev); + "PCI config space inaccessible after reset\n"); + goto reset_exit; + } - bnxt_inv_fw_health_reg(bp); - bnxt_try_map_fw_health_reg(bp); + /* Upon PCIe error, our device internal logic that latches to + * BAR value is getting reset and will restore only upon + * rewriting the BARs. + * + * As pci_restore_state() does not re-write the BARs if the + * value is same as saved value earlier, driver needs to + * write the BARs to 0 to force restore. + */ + pci_clear_master(pdev); + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + cmd &= ~PCI_COMMAND_MEMORY; + pci_write_config_word(pdev, PCI_COMMAND, cmd); - /* In some PCIe AER scenarios, firmware may take up to - * 10 seconds to become ready in the worst case. - */ - do { - err = bnxt_try_recover_fw(bp); - if (!err) - break; - retry++; - } while (retry < BNXT_FW_SLOT_RESET_RETRY); + bnxt_clear_bars(pdev); + pci_restore_state(pdev); - if (err) { - dev_err(&pdev->dev, "Firmware not ready\n"); - goto reset_exit; + if (!pci_is_enabled(pdev) && pci_enable_device(pdev)) { + dev_err(&pdev->dev, + "Cannot re-enable PCI device after reset.\n"); + pci_read_config_word(pdev, PCI_COMMAND, &cmd); + if (!PCI_POSSIBLE_ERROR(cmd)) { + cmd &= ~(PCI_COMMAND_MASTER | PCI_COMMAND_MEMORY); + pci_write_config_word(pdev, PCI_COMMAND, cmd); } + goto reset_exit; + } + pci_set_master(pdev); - err = bnxt_hwrm_func_reset(bp); + bnxt_inv_fw_health_reg(bp); + bnxt_try_map_fw_health_reg(bp); + + /* In some PCIe AER scenarios, firmware may take up to + * 10 seconds to become ready in the worst case. + */ + do { + err = bnxt_try_recover_fw(bp); if (!err) - result = PCI_ERS_RESULT_RECOVERED; + break; + retry++; + } while (retry < BNXT_FW_SLOT_RESET_RETRY); - /* IRQ will be initialized later in bnxt_io_resume */ - bnxt_ulp_irq_stop(bp); - bnxt_clear_int_mode(bp); + if (err) { + dev_err(&pdev->dev, "Firmware not ready\n"); + goto reset_exit; } + err = bnxt_hwrm_func_reset(bp); + if (!err) + result = PCI_ERS_RESULT_RECOVERED; + + /* IRQ will be initialized later in bnxt_io_resume */ + bnxt_ulp_irq_stop(bp); + bnxt_clear_int_mode(bp); + reset_exit: clear_bit(BNXT_STATE_IN_FW_RESET, &bp->state); bnxt_clear_reservations(bp, true); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 4dbcd08ef7c3..a939f4ed1007 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -2470,7 +2470,6 @@ struct bnxt { #define BNXT_STATE_ABORT_ERR 5 #define BNXT_STATE_FW_FATAL_COND 6 #define BNXT_STATE_DRV_REGISTERED 7 -#define BNXT_STATE_PCI_CHANNEL_IO_FROZEN 8 #define BNXT_STATE_NAPI_DISABLED 9 #define BNXT_STATE_FW_ACTIVATE 11 #define BNXT_STATE_RECOVER 12 diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c b/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c index 96d5d4f7f51f..984432952963 100644 --- a/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c @@ -253,6 +253,17 @@ int bcmgenet_wol_power_up_cfg(struct bcmgenet_priv *priv, reg = bcmgenet_umac_readl(priv, UMAC_CMD); reg &= ~CMD_CRC_FWD; bcmgenet_umac_writel(priv, reg, UMAC_CMD); + + /* + * Mirror wol_power_down_cfg(). If only UMAC_RX + * is enabled, then we must place the UMAC back + * into SW_RESET. + */ + reg = bcmgenet_umac_readl(priv, UMAC_CMD); + if ((reg & CMD_RX_EN) && !(reg & CMD_TX_EN)) { + reg |= CMD_SW_RESET; + bcmgenet_umac_writel(priv, reg, UMAC_CMD); + } spin_unlock_bh(&priv->reg_lock); /* Resume link status tracking */ diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c index f5acd4be1e69..dbec95b49777 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c @@ -389,7 +389,7 @@ static void chcr_ktls_dev_del(struct net_device *netdev, cxgb4_remove_tid(&tx_info->adap->tids, tx_info->tx_chan, tx_info->tid, tx_info->ip_family); - xa_erase(&u_ctx->tid_list, tx_info->tid); + xa_erase_bh(&u_ctx->tid_list, tx_info->tid); } port_stats = &tx_info->adap->ch_ktls_stats.ktls_port[tx_info->port_id]; @@ -577,7 +577,7 @@ static int chcr_ktls_dev_add(struct net_device *netdev, struct sock *sk, cxgb4_remove_tid(&tx_info->adap->tids, tx_info->tx_chan, tx_info->tid, tx_info->ip_family); - xa_erase(&u_ctx->tid_list, tx_info->tid); + xa_erase_bh(&u_ctx->tid_list, tx_info->tid); put_module: /* release module refcount */ diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index e4d78ae52daf..0dd04ee30c9a 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -1708,17 +1708,17 @@ static int gve_xsk_pool_disable(struct net_device *dev, } napi_rx = &priv->ntfy_blocks[priv->rx[qid].ntfy_id].napi; - napi_disable(napi_rx); /* make sure current rx poll is done */ + napi_disable_locked(napi_rx); /* make sure current rx poll is done */ tx_qid = gve_xdp_tx_queue_id(priv, qid); napi_tx = &priv->ntfy_blocks[priv->tx[tx_qid].ntfy_id].napi; - napi_disable(napi_tx); /* make sure current tx poll is done */ + napi_disable_locked(napi_tx); /* make sure current tx poll is done */ gve_unreg_xsk_pool(priv, qid); smp_mb(); /* Make sure it is visible to the workers on datapath */ - napi_enable(napi_rx); - napi_enable(napi_tx); + napi_enable_locked(napi_rx); + napi_enable_locked(napi_tx); if (gve_is_gqi(priv)) { if (gve_rx_work_pending(&priv->rx[qid])) napi_schedule(napi_rx); diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c index 0ece2f6fdffb..2e6d6abbddfc 100644 --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c @@ -113,6 +113,12 @@ static void gve_rx_reset_ring_dqo(struct gve_priv *priv, int idx) gve_free_to_page_pool(rx, bs, false); else gve_free_qpl_page_dqo(bs); + + if (gve_buf_state_is_allocated(rx, bs) && + bs->xsk_buff) { + xsk_buff_free(bs->xsk_buff); + bs->xsk_buff = NULL; + } } } @@ -839,7 +845,12 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx, } if (unlikely(compl_desc->rx_error)) { - gve_free_buffer(rx, buf_state); + if (buf_state->xsk_buff) { + xsk_buff_free(buf_state->xsk_buff); + gve_free_buf_state(rx, buf_state); + } else { + gve_free_buffer(rx, buf_state); + } return -EINVAL; } diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c index 73e051d26b9d..abebdb1fc262 100644 --- a/drivers/net/ethernet/ibm/ibmveth.c +++ b/drivers/net/ethernet/ibm/ibmveth.c @@ -553,10 +553,15 @@ static int ibmveth_rxq_harvest_buffer(struct ibmveth_adapter *adapter, static void ibmveth_free_tx_ltb(struct ibmveth_adapter *adapter, int idx) { + void *ptr = adapter->tx_ltb_ptr[idx]; + + if (!ptr) + return; + + adapter->tx_ltb_ptr[idx] = NULL; dma_unmap_single(&adapter->vdev->dev, adapter->tx_ltb_dma[idx], adapter->tx_ltb_size, DMA_TO_DEVICE); - kfree(adapter->tx_ltb_ptr[idx]); - adapter->tx_ltb_ptr[idx] = NULL; + kfree(ptr); } static int ibmveth_allocate_tx_ltb(struct ibmveth_adapter *adapter, int idx) @@ -667,7 +672,7 @@ static int ibmveth_open(struct net_device *netdev) for (i = 0; i < netdev->real_num_tx_queues; i++) { if (ibmveth_allocate_tx_ltb(adapter, i)) - goto out_free_tx_ltb; + goto out_unmap_filter_list; } adapter->rx_queue.index = 0; @@ -718,10 +723,6 @@ static int ibmveth_open(struct net_device *netdev) if (rc != 0) { netdev_err(netdev, "unable to request irq 0x%x, rc %d\n", netdev->irq, rc); - do { - lpar_rc = h_free_logical_lan(adapter->vdev->unit_address); - } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY)); - goto out_free_buffer_pools; } @@ -737,6 +738,9 @@ static int ibmveth_open(struct net_device *netdev) return 0; out_free_buffer_pools: + do { + lpar_rc = h_free_logical_lan(adapter->vdev->unit_address); + } while (H_IS_LONG_BUSY(lpar_rc) || (lpar_rc == H_BUSY)); while (--i >= 0) { if (adapter->rx_buff_pool[i].active) ibmveth_free_buffer_pool(adapter, @@ -746,10 +750,8 @@ static int ibmveth_open(struct net_device *netdev) dma_unmap_single(dev, adapter->filter_list_dma, 4096, DMA_BIDIRECTIONAL); -out_free_tx_ltb: - while (--i >= 0) { + for (i = netdev->real_num_tx_queues - 1; i >= 0; i--) ibmveth_free_tx_ltb(adapter, i); - } out_unmap_buffer_list: dma_unmap_single(dev, adapter->buffer_list_dma, 4096, diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c index 844f31ab37ad..c282f65199e4 100644 --- a/drivers/net/ethernet/intel/e1000e/netdev.c +++ b/drivers/net/ethernet/intel/e1000e/netdev.c @@ -67,6 +67,13 @@ static const struct dmi_system_id disable_k1_list[] = { DMI_MATCH(DMI_PRODUCT_NAME, "Dell Pro 16 Plus PB16250"), }, }, + { + .ident = "Lenovo ThinkPad P14s Gen 5", + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"), + DMI_MATCH(DMI_PRODUCT_VERSION, "ThinkPad P14s Gen 5"), + }, + }, {} }; diff --git a/drivers/net/ethernet/intel/iavf/iavf.h b/drivers/net/ethernet/intel/iavf/iavf.h index 050f8241ef5e..5fcbfa0ca855 100644 --- a/drivers/net/ethernet/intel/iavf/iavf.h +++ b/drivers/net/ethernet/intel/iavf/iavf.h @@ -259,7 +259,6 @@ struct iavf_adapter { struct work_struct adminq_task; struct work_struct finish_config; wait_queue_head_t down_waitqueue; - wait_queue_head_t vc_waitqueue; struct iavf_q_vector *q_vectors; struct list_head vlan_filter_list; int num_vlan_filters; @@ -588,8 +587,9 @@ void iavf_configure_queues(struct iavf_adapter *adapter); void iavf_enable_queues(struct iavf_adapter *adapter); void iavf_disable_queues(struct iavf_adapter *adapter); void iavf_map_queues(struct iavf_adapter *adapter); -void iavf_add_ether_addrs(struct iavf_adapter *adapter); +int iavf_add_ether_addrs(struct iavf_adapter *adapter); void iavf_del_ether_addrs(struct iavf_adapter *adapter); +void iavf_mac_add_reject(struct iavf_adapter *adapter); void iavf_add_vlans(struct iavf_adapter *adapter); void iavf_del_vlans(struct iavf_adapter *adapter); void iavf_set_promiscuous(struct iavf_adapter *adapter); @@ -606,6 +606,13 @@ void iavf_disable_vlan_stripping(struct iavf_adapter *adapter); void iavf_virtchnl_completion(struct iavf_adapter *adapter, enum virtchnl_ops v_opcode, enum iavf_status v_retval, u8 *msg, u16 msglen); +int iavf_poll_virtchnl_response(struct iavf_adapter *adapter, + struct iavf_arq_event_info *event, + bool (*condition)(struct iavf_adapter *adapter, + const void *data, + enum virtchnl_ops v_op), + const void *cond_data, + unsigned int timeout_ms); int iavf_config_rss(struct iavf_adapter *adapter); void iavf_cfg_queues_bw(struct iavf_adapter *adapter); void iavf_cfg_queues_quanta_size(struct iavf_adapter *adapter); diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 29b8403a066b..2f8a80a11336 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1029,6 +1029,60 @@ static bool iavf_is_mac_set_handled(struct net_device *netdev, return ret; } +/** + * iavf_mac_change_done - Check if MAC change completed + * @adapter: board private structure + * @data: MAC address being checked (as const void *) + * @v_op: virtchnl opcode from processed message + * + * Callback for iavf_poll_virtchnl_response() to check if MAC change completed. + * + * Return: true if MAC change completed, false otherwise + */ +static bool iavf_mac_change_done(struct iavf_adapter *adapter, + const void *data, enum virtchnl_ops v_op) +{ + const u8 *addr = data; + + return iavf_is_mac_set_handled(adapter->netdev, addr); +} + +/** + * iavf_set_mac_sync - Synchronously change MAC address + * @adapter: board private structure + * @addr: MAC address to set + * + * Send MAC change request to PF and poll admin queue for response. + * Caller must hold netdev_lock. This can sleep for up to 2.5 seconds. + * Event buffer is allocated before sending to avoid state mismatch if + * allocation fails after message is sent to PF. + * + * Return: 0 on success, negative on failure + */ +static int iavf_set_mac_sync(struct iavf_adapter *adapter, const u8 *addr) +{ + struct iavf_arq_event_info event; + int ret; + + netdev_assert_locked(adapter->netdev); + + event.buf_len = IAVF_MAX_AQ_BUF_SIZE; + event.msg_buf = kzalloc(event.buf_len, GFP_KERNEL); + if (!event.msg_buf) + return -ENOMEM; + + ret = iavf_add_ether_addrs(adapter); + if (ret) + goto out; + + ret = iavf_poll_virtchnl_response(adapter, &event, + iavf_mac_change_done, addr, 2500); + +out: + kfree(event.msg_buf); + return ret; +} + /** * iavf_set_mac - NDO callback to set port MAC address * @netdev: network interface device structure @@ -1046,25 +1100,23 @@ static int iavf_set_mac(struct net_device *netdev, void *p) return -EADDRNOTAVAIL; ret = iavf_replace_primary_mac(adapter, addr->sa_data); - if (ret) return ret; - ret = wait_event_interruptible_timeout(adapter->vc_waitqueue, - iavf_is_mac_set_handled(netdev, addr->sa_data), - msecs_to_jiffies(2500)); - - /* If ret < 0 then it means wait was interrupted. - * If ret == 0 then it means we got a timeout. - * else it means we got response for set MAC from PF, - * check if netdev MAC was updated to requested MAC, - * if yes then set MAC succeeded otherwise it failed return -EACCES - */ - if (ret < 0) + ret = iavf_set_mac_sync(adapter, addr->sa_data); + if (ret) { + /* Rollback only if send failed (message never reached PF). + * Don't rollback on timeout (-EAGAIN) because the message was + * sent and PF will eventually respond. When the response arrives, + * iavf_virtchnl_completion() will handle rollback (on PF error) + * or acceptance (on PF success) automatically. + */ + if (ret != -EAGAIN) { + iavf_mac_add_reject(adapter); + ether_addr_copy(adapter->hw.mac.addr, netdev->dev_addr); + } return ret; - - if (!ret) - return -EAGAIN; + } if (!ether_addr_equal(netdev->dev_addr, addr->sa_data)) return -EACCES; @@ -5394,9 +5446,6 @@ static int iavf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) /* Setup the wait queue for indicating transition to down status */ init_waitqueue_head(&adapter->down_waitqueue); - /* Setup the wait queue for indicating virtchannel events */ - init_waitqueue_head(&adapter->vc_waitqueue); - INIT_LIST_HEAD(&adapter->ptp.aq_cmds); init_waitqueue_head(&adapter->ptp.phc_time_waitqueue); mutex_init(&adapter->ptp.aq_cmd_lock); diff --git a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c index ec234cc8bd9d..e6b7e8f82c7c 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +++ b/drivers/net/ethernet/intel/iavf/iavf_virtchnl.c @@ -2,6 +2,7 @@ /* Copyright(c) 2013 - 2018 Intel Corporation. */ #include +#include #include "iavf.h" #include "iavf_ptp.h" @@ -555,20 +556,23 @@ iavf_set_mac_addr_type(struct virtchnl_ether_addr *virtchnl_ether_addr, * @adapter: adapter structure * * Request that the PF add one or more addresses to our filters. - **/ -void iavf_add_ether_addrs(struct iavf_adapter *adapter) + * + * Return: 0 on success, negative on failure + */ +int iavf_add_ether_addrs(struct iavf_adapter *adapter) { struct virtchnl_ether_addr_list *veal; struct iavf_mac_filter *f; int i = 0, count = 0; bool more = false; size_t len; + int ret; if (adapter->current_op != VIRTCHNL_OP_UNKNOWN) { /* bail because we already have a command pending */ dev_err(&adapter->pdev->dev, "Cannot add filters, command %d pending\n", adapter->current_op); - return; + return -EBUSY; } spin_lock_bh(&adapter->mac_vlan_list_lock); @@ -580,7 +584,7 @@ void iavf_add_ether_addrs(struct iavf_adapter *adapter) if (!count) { adapter->aq_required &= ~IAVF_FLAG_AQ_ADD_MAC_FILTER; spin_unlock_bh(&adapter->mac_vlan_list_lock); - return; + return 0; } adapter->current_op = VIRTCHNL_OP_ADD_ETH_ADDR; @@ -594,8 +598,9 @@ void iavf_add_ether_addrs(struct iavf_adapter *adapter) veal = kzalloc(len, GFP_ATOMIC); if (!veal) { + adapter->current_op = VIRTCHNL_OP_UNKNOWN; spin_unlock_bh(&adapter->mac_vlan_list_lock); - return; + return -ENOMEM; } veal->vsi_id = adapter->vsi_res->vsi_id; @@ -615,8 +620,15 @@ void iavf_add_ether_addrs(struct iavf_adapter *adapter) spin_unlock_bh(&adapter->mac_vlan_list_lock); - iavf_send_pf_msg(adapter, VIRTCHNL_OP_ADD_ETH_ADDR, (u8 *)veal, len); + ret = iavf_send_pf_msg(adapter, VIRTCHNL_OP_ADD_ETH_ADDR, (u8 *)veal, len); kfree(veal); + if (ret) { + dev_err(&adapter->pdev->dev, + "Unable to send ADD_ETH_ADDR message to PF, error %d\n", ret); + adapter->current_op = VIRTCHNL_OP_UNKNOWN; + } + + return ret; } /** @@ -712,8 +724,8 @@ static void iavf_mac_add_ok(struct iavf_adapter *adapter) * @adapter: adapter structure * * Remove filters from list based on PF response. - **/ -static void iavf_mac_add_reject(struct iavf_adapter *adapter) + */ +void iavf_mac_add_reject(struct iavf_adapter *adapter) { struct net_device *netdev = adapter->netdev; struct iavf_mac_filter *f, *ftmp; @@ -2364,7 +2376,6 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter, iavf_mac_add_reject(adapter); /* restore administratively set MAC address */ ether_addr_copy(adapter->hw.mac.addr, netdev->dev_addr); - wake_up(&adapter->vc_waitqueue); break; case VIRTCHNL_OP_DEL_ETH_ADDR: dev_err(&adapter->pdev->dev, "Failed to delete MAC filter, error %s\n", @@ -2555,7 +2566,6 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter, eth_hw_addr_set(netdev, adapter->hw.mac.addr); netif_addr_unlock_bh(netdev); } - wake_up(&adapter->vc_waitqueue); break; case VIRTCHNL_OP_GET_STATS: { struct iavf_eth_stats *stats = @@ -2950,3 +2960,72 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter, } /* switch v_opcode */ adapter->current_op = VIRTCHNL_OP_UNKNOWN; } + +/** + * iavf_poll_virtchnl_response - Poll admin queue for virtchnl response + * @adapter: adapter structure + * @event: pre-allocated event buffer to use for polling + * @condition: callback to check if desired response received + * @cond_data: context data passed to condition callback + * @timeout_ms: maximum time to wait in milliseconds + * + * Polls the admin queue and processes all incoming virtchnl messages. + * After processing each valid message, calls the condition callback to check + * if the expected response has been received. The callback receives the opcode + * of the processed message to identify which response was received. Continues + * polling until the callback returns true or timeout expires. + * + * Caller must allocate event buffer before sending any messages to PF to avoid + * state mismatch if allocation fails after message is sent. + * + * Caller must hold netdev_lock. This can sleep for up to timeout_ms while + * polling hardware. + * + * Return: 0 on success (condition met), -EAGAIN on timeout, or error code + */ +int iavf_poll_virtchnl_response(struct iavf_adapter *adapter, + struct iavf_arq_event_info *event, + bool (*condition)(struct iavf_adapter *adapter, + const void *data, + enum virtchnl_ops v_op), + const void *cond_data, + unsigned int timeout_ms) +{ + struct iavf_hw *hw = &adapter->hw; + enum virtchnl_ops received_op; + unsigned long timeout; + int ret = -EAGAIN; + u16 pending = 0; + u32 v_retval; + + netdev_assert_locked(adapter->netdev); + + timeout = jiffies + msecs_to_jiffies(timeout_ms); + do { + if (!pending) + usleep_range(50, 75); + + if (iavf_clean_arq_element(hw, event, &pending) == IAVF_SUCCESS) { + received_op = (enum virtchnl_ops)le32_to_cpu(event->desc.cookie_high); + if (received_op != VIRTCHNL_OP_UNKNOWN) { + v_retval = le32_to_cpu(event->desc.cookie_low); + + iavf_virtchnl_completion(adapter, received_op, + (enum iavf_status)v_retval, + event->msg_buf, event->msg_len); + + if (condition(adapter, cond_data, received_op)) { + ret = 0; + break; + } + } + + memset(event->msg_buf, 0, IAVF_MAX_AQ_BUF_SIZE); + + if (pending) + continue; + } + } while (time_before(jiffies, timeout)); + + return ret; +} diff --git a/drivers/net/ethernet/intel/ice/devlink/port.c b/drivers/net/ethernet/intel/ice/devlink/port.c index 2a2e56777f9f..3ede24649002 100644 --- a/drivers/net/ethernet/intel/ice/devlink/port.c +++ b/drivers/net/ethernet/intel/ice/devlink/port.c @@ -590,8 +590,8 @@ static void ice_dealloc_dynamic_port(struct ice_dynamic_port *dyn_port) xa_erase(&pf->sf_nums, devlink_port->attrs.pci_sf.sf); ice_eswitch_detach_sf(pf, dyn_port); - ice_vsi_free(dyn_port->vsi); xa_erase(&pf->dyn_ports, dyn_port->vsi->idx); + ice_vsi_free(dyn_port->vsi); kfree(dyn_port); } diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c index b069e6c514fb..6e7bba473898 100644 --- a/drivers/net/ethernet/intel/ice/ice_eswitch.c +++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c @@ -95,7 +95,7 @@ ice_eswitch_release_repr(struct ice_pf *pf, struct ice_repr *repr) return; ice_vsi_update_security(vsi, ice_vsi_ctx_set_antispoof); - metadata_dst_free(repr->dst); + dst_release(&repr->dst->dst); repr->dst = NULL; ice_fltr_add_mac_and_broadcast(vsi, repr->parent_mac, ICE_FWD_TO_VSI); diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index c04c5856dad6..4e26ace793ec 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1561,10 +1561,10 @@ ice_tx_map(struct ice_tx_ring *tx_ring, struct ice_tx_buf *first, } } tstamp_ring->next_to_use = j; - writel_relaxed(j, tstamp_ring->tail); + writel(j, tstamp_ring->tail); } else { ring_kick: - writel_relaxed(i, tx_ring->tail); + writel(i, tx_ring->tail); } return; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c index 4f984f6a2cb9..55ec6387ab28 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/rx.c @@ -3,6 +3,7 @@ #include "rx.h" #include "en/xdp.h" +#include #include #include @@ -156,6 +157,7 @@ int mlx5e_xsk_alloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix) xsk_buff_free(xsk_buffs[batch]); err: + bitmap_fill(wi->skip_release_bitmap, rq->mpwqe.pages_per_wqe); rq->stats->buff_alloc_err++; return -ENOMEM; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 6fc6605d2054..c75b064b0416 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -410,8 +410,11 @@ static inline void mlx5e_free_rx_wqe(struct mlx5e_rq *rq, static void mlx5e_xsk_free_rx_wqe(struct mlx5e_wqe_frag_info *wi) { - if (!(wi->flags & BIT(MLX5E_WQE_FRAG_SKIP_RELEASE))) - xsk_buff_free(*wi->xskp); + if (wi->flags & BIT(MLX5E_WQE_FRAG_SKIP_RELEASE)) + return; + + xsk_buff_free(*wi->xskp); + wi->flags |= BIT(MLX5E_WQE_FRAG_SKIP_RELEASE); } static void mlx5e_dealloc_rx_wqe(struct mlx5e_rq *rq, u16 ix) @@ -905,10 +908,12 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq) } while (!last_wqe); } while ((++i < MLX5E_TX_CQ_POLL_BUDGET) && (cqe = mlx5_cqwq_get_cqe(&cq->wq))); - sq->cc = sqcc; - mlx5_cqwq_update_db_record(&cq->wq); + /* ensure cq space is freed before enabling more cqes */ + dma_wmb(); + + sq->cc = sqcc; return i; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c index 7950b82d8b8a..3624c680a861 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c @@ -11,26 +11,6 @@ int mlx5_esw_adj_vport_modify(struct mlx5_core_dev *dev, u16 vport, lockdep_assert_held(&dev->priv.eswitch->state_lock); - if (MLX5_CAP_ESW(dev, esw_vport_state_max_tx_speed)) { - u8 op_mod = MLX5_VPORT_STATE_OP_MOD_ESW_VPORT; - struct mlx5_vport *esw_vport; - u32 speed = 0; - int err; - - err = mlx5_query_vport_max_tx_speed(dev, op_mod, vport, - true, &speed, NULL); - if (err) { - esw_vport = mlx5_eswitch_get_vport(dev->priv.eswitch, - vport); - speed = IS_ERR(esw_vport) ? 0 : - esw_vport->agg_max_tx_speed; - mlx5_core_dbg(dev, - "Failed to query vport %d max tx speed, err=%d, using cached %u\n", - vport, err, speed); - } - MLX5_SET(modify_vport_state_in, in, max_tx_speed, speed); - } - MLX5_SET(modify_vport_state_in, in, opcode, MLX5_CMD_OP_MODIFY_VPORT_STATE); MLX5_SET(modify_vport_state_in, in, op_mod, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c index 3b34bec559e0..4e173b08cb37 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c @@ -1432,30 +1432,40 @@ static bool mlx5_lag_should_disable_lag(struct mlx5_lag *ldev, bool do_bond) } #ifdef CONFIG_MLX5_ESWITCH -static int -mlx5_lag_sum_devices_speed(struct mlx5_lag *ldev, u32 *sum_speed, - int (*get_speed)(struct mlx5_core_dev *, u32 *)) +static int mlx5_lag_get_devices_oper_speed(struct mlx5_lag *ldev, + u32 *sum_speed) { - struct mlx5_core_dev *pf_mdev; - struct lag_func *pf; int pf_idx; - u32 speed; - int ret; *sum_speed = 0; mlx5_ldev_for_each(pf_idx, 0, ldev) { + u8 opmod = MLX5_VPORT_STATE_OP_MOD_VNIC_VPORT; + struct mlx5_core_dev *pf_mdev; + struct lag_func *pf; + u32 speed; + u8 state; + int ret; + pf = mlx5_lag_pf(ldev, pf_idx); if (!pf) continue; pf_mdev = pf->dev; if (!pf_mdev) continue; + ret = mlx5_query_vport_max_tx_speed(pf_mdev, opmod, 0, 0, + &speed, &state); + if (ret) { + mlx5_core_dbg(pf_mdev, "State query failed (err=%d)\n", + ret); + return ret; + } + if (state != VPORT_STATE_UP) + continue; - ret = get_speed(pf_mdev, &speed); + ret = mlx5_port_oper_linkspeed(pf_mdev, &speed); if (ret) { mlx5_core_dbg(pf_mdev, - "Failed to get device speed using %ps. Device %s speed is not available (err=%d)\n", - get_speed, dev_name(pf_mdev->device), + "Failed to get oper speed (err=%d)\n", ret); return ret; } @@ -1466,17 +1476,42 @@ mlx5_lag_sum_devices_speed(struct mlx5_lag *ldev, u32 *sum_speed, return 0; } -static int mlx5_lag_sum_devices_max_speed(struct mlx5_lag *ldev, u32 *max_speed) +static int mlx5_lag_get_devices_max_speed(struct mlx5_lag *ldev, u32 *max_speed) { - return mlx5_lag_sum_devices_speed(ldev, max_speed, - mlx5_port_max_linkspeed); -} + bool take_max; + int pf_idx; -static int mlx5_lag_sum_devices_oper_speed(struct mlx5_lag *ldev, - u32 *oper_speed) -{ - return mlx5_lag_sum_devices_speed(ldev, oper_speed, - mlx5_port_oper_linkspeed); + take_max = ldev->tracker.tx_type == NETDEV_LAG_TX_TYPE_ACTIVEBACKUP; + if (ldev->mode == MLX5_LAG_MODE_MPESW) + take_max = false; + + *max_speed = 0; + mlx5_ldev_for_each(pf_idx, 0, ldev) { + struct mlx5_core_dev *pf_mdev; + struct lag_func *pf; + u32 speed; + int ret; + + pf = mlx5_lag_pf(ldev, pf_idx); + if (!pf) + continue; + pf_mdev = pf->dev; + if (!pf_mdev) + continue; + + ret = mlx5_port_max_linkspeed(pf_mdev, &speed); + if (ret) { + mlx5_core_dbg(pf_mdev, + "Failed to get max speed (err=%d)\n", + ret); + return ret; + } + + *max_speed = take_max ? + max(*max_speed, speed) : *max_speed + speed; + } + + return 0; } static void mlx5_lag_modify_device_vports_speed(struct mlx5_core_dev *mdev, @@ -1525,7 +1560,7 @@ void mlx5_lag_set_vports_agg_speed(struct mlx5_lag *ldev) int pf_idx; if (ldev->mode == MLX5_LAG_MODE_MPESW) { - if (mlx5_lag_sum_devices_oper_speed(ldev, &speed)) + if (mlx5_lag_get_devices_oper_speed(ldev, &speed)) return; } else { speed = ldev->tracker.bond_speed_mbps; @@ -1533,8 +1568,8 @@ void mlx5_lag_set_vports_agg_speed(struct mlx5_lag *ldev) return; } - /* If speed is not set, use the sum of max speeds of all PFs */ - if (!speed && mlx5_lag_sum_devices_max_speed(ldev, &speed)) + /* If speed is not set, fall back to the max achievable speed */ + if (!speed && mlx5_lag_get_devices_max_speed(ldev, &speed)) return; speed = speed / MLX5_MAX_TX_SPEED_UNIT; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/vport.c b/drivers/net/ethernet/mellanox/mlx5/core/vport.c index 8aa94ec87a0e..edac2d694e0f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/vport.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/vport.c @@ -93,30 +93,6 @@ int mlx5_modify_vport_admin_state(struct mlx5_core_dev *mdev, u8 opmod, lockdep_assert_held(&mdev->priv.eswitch->state_lock); #endif - if (MLX5_CAP_ESW(mdev, esw_vport_state_max_tx_speed) && - opmod == MLX5_VPORT_STATE_OP_MOD_ESW_VPORT && - vport != MLX5_VPORT_UPLINK) { - u32 speed = 0; - int err; - - err = mlx5_query_vport_max_tx_speed(mdev, opmod, vport, - other_vport, &speed, NULL); - if (err) { -#ifdef CONFIG_MLX5_ESWITCH - struct mlx5_vport *esw_vport; - - esw_vport = mlx5_eswitch_get_vport(mdev->priv.eswitch, - vport); - speed = IS_ERR(esw_vport) ? 0 : - esw_vport->agg_max_tx_speed; -#endif - mlx5_core_dbg(mdev, - "Failed to query vport %d max tx speed, err=%d, using cached %u\n", - vport, err, speed); - } - MLX5_SET(modify_vport_state_in, in, max_tx_speed, speed); - } - MLX5_SET(modify_vport_state_in, in, opcode, MLX5_CMD_OP_MODIFY_VPORT_STATE); MLX5_SET(modify_vport_state_in, in, op_mod, opmod); diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h index b03ff9e044f9..10af08ce72ef 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h @@ -989,6 +989,8 @@ void mlxsw_sp_acl_ruleset_prio_get(struct mlxsw_sp_acl_ruleset *ruleset, struct mlxsw_sp_acl_rule_info * mlxsw_sp_acl_rulei_create(struct mlxsw_sp_acl *acl, struct mlxsw_afa_block *afa_block); +void mlxsw_sp_acl_rulei_free(struct mlxsw_sp *mlxsw_sp, + const struct mlxsw_sp_acl_rule_info *rulei); void mlxsw_sp_acl_rulei_destroy(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_acl_rule_info *rulei); int mlxsw_sp_acl_rulei_commit(struct mlxsw_sp_acl_rule_info *rulei); diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c index cb232accb296..f7be4f3a1518 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c @@ -340,8 +340,8 @@ mlxsw_sp_acl_rulei_create(struct mlxsw_sp_acl *acl, return ERR_PTR(err); } -void mlxsw_sp_acl_rulei_destroy(struct mlxsw_sp *mlxsw_sp, - struct mlxsw_sp_acl_rule_info *rulei) +void mlxsw_sp_acl_rulei_free(struct mlxsw_sp *mlxsw_sp, + const struct mlxsw_sp_acl_rule_info *rulei) { if (rulei->action_created) mlxsw_afa_block_destroy(rulei->act_block); @@ -351,6 +351,12 @@ void mlxsw_sp_acl_rulei_destroy(struct mlxsw_sp *mlxsw_sp, if (rulei->dst_port_range_reg_valid) mlxsw_sp_port_range_reg_put(mlxsw_sp, rulei->dst_port_range_reg_index); +} + +void mlxsw_sp_acl_rulei_destroy(struct mlxsw_sp *mlxsw_sp, + struct mlxsw_sp_acl_rule_info *rulei) +{ + mlxsw_sp_acl_rulei_free(mlxsw_sp, rulei); kfree(rulei); } diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c index 353fd9ca89a6..5e4632cb792d 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c @@ -862,14 +862,19 @@ int mlxsw_sp_flower_tmplt_create(struct mlxsw_sp *mlxsw_sp, memset(&rulei, 0, sizeof(rulei)); err = mlxsw_sp_flower_parse(mlxsw_sp, block, &rulei, f); if (err) - return err; + goto out; + ruleset = mlxsw_sp_acl_ruleset_get(mlxsw_sp, block, f->common.chain_index, MLXSW_SP_ACL_PROFILE_FLOWER, &rulei.values.elusage); + err = PTR_ERR_OR_ZERO(ruleset); + +out: + mlxsw_sp_acl_rulei_free(mlxsw_sp, &rulei); /* keep the reference to the ruleset */ - return PTR_ERR_OR_ZERO(ruleset); + return err; } void mlxsw_sp_flower_tmplt_destroy(struct mlxsw_sp *mlxsw_sp, diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_ptp.c b/drivers/net/ethernet/microchip/sparx5/sparx5_ptp.c index 5953e04ef14e..2826e48af103 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_ptp.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_ptp.c @@ -622,18 +622,9 @@ int sparx5_ptp_init(struct sparx5 *sparx5) sparx5->ptp = 1; } - if (!sparx5->ptp) - return 0; - - for (i = 0; i < SPARX5_PHC_COUNT; ++i) { - err = sparx5_ptp_phc_init(sparx5, i, &sparx5_ptp_clock_info); - if (err) - return err; - } - - spin_lock_init(&sparx5->ptp_clock_lock); - spin_lock_init(&sparx5->ptp_ts_id_lock); - mutex_init(&sparx5->ptp_lock); + /* The base, non-PTP-capable lan969x variants need the + * first TOD counter running to forward frames. + */ /* Disable master counters */ spx5_wr(PTP_PTP_DOM_CFG_PTP_ENA_SET(0), sparx5, PTP_PTP_DOM_CFG); @@ -657,6 +648,19 @@ int sparx5_ptp_init(struct sparx5 *sparx5) /* Enable master counters */ spx5_wr(PTP_PTP_DOM_CFG_PTP_ENA_SET(0x7), sparx5, PTP_PTP_DOM_CFG); + if (!sparx5->ptp) + return 0; + + for (i = 0; i < SPARX5_PHC_COUNT; ++i) { + err = sparx5_ptp_phc_init(sparx5, i, &sparx5_ptp_clock_info); + if (err) + return err; + } + + spin_lock_init(&sparx5->ptp_clock_lock); + spin_lock_init(&sparx5->ptp_ts_id_lock); + mutex_init(&sparx5->ptp_lock); + for (i = 0; i < sparx5->data->consts->n_ports; i++) { port = sparx5->ports[i]; if (!port) diff --git a/drivers/net/ethernet/microchip/sparx5/sparx5_tc_matchall.c b/drivers/net/ethernet/microchip/sparx5/sparx5_tc_matchall.c index 702257979462..146d7f28e8f3 100644 --- a/drivers/net/ethernet/microchip/sparx5/sparx5_tc_matchall.c +++ b/drivers/net/ethernet/microchip/sparx5/sparx5_tc_matchall.c @@ -160,6 +160,7 @@ static int sparx5_tc_matchall_destroy(struct net_device *ndev, } list_del(&entry->list); + kfree(entry); return err; } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 010fef79ec01..0da3da7d0d1b 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -6222,6 +6222,13 @@ static netdev_features_t stmmac_fix_features(struct net_device *dev, if (priv->plat->bugged_jumbo && (dev->mtu > ETH_DATA_LEN)) features &= ~NETIF_F_CSUM_MASK; + if (priv->plat->core_type == DWMAC_CORE_XGMAC) { + if (features & NETIF_F_HW_VLAN_CTAG_RX) + features |= NETIF_F_HW_VLAN_STAG_RX; + else + features &= ~NETIF_F_HW_VLAN_STAG_RX; + } + return features; } @@ -8016,14 +8023,18 @@ static int __stmmac_dvr_probe(struct device *device, ndev->watchdog_timeo = msecs_to_jiffies(watchdog); #ifdef STMMAC_VLAN_TAG_USED /* Both mac100 and gmac support receive VLAN tag detection */ - ndev->features |= NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_STAG_RX; + ndev->features |= NETIF_F_HW_VLAN_CTAG_RX; + if (priv->plat->core_type == DWMAC_CORE_XGMAC) + ndev->features |= NETIF_F_HW_VLAN_STAG_RX; + if (dwmac_is_xmac(priv->plat->core_type)) { ndev->hw_features |= NETIF_F_HW_VLAN_CTAG_RX; priv->hw->hw_vlan_en = true; } if (priv->dma_cap.vlhash) { ndev->features |= NETIF_F_HW_VLAN_CTAG_FILTER; - ndev->features |= NETIF_F_HW_VLAN_STAG_FILTER; + if (priv->plat->core_type == DWMAC_CORE_XGMAC) + ndev->features |= NETIF_F_HW_VLAN_STAG_FILTER; } if (priv->dma_cap.vlins) ndev->features |= NETIF_F_HW_VLAN_CTAG_TX; diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet.h b/drivers/net/ethernet/xilinx/xilinx_axienet.h index fcd3aaef27fc..c63d99686d0b 100644 --- a/drivers/net/ethernet/xilinx/xilinx_axienet.h +++ b/drivers/net/ethernet/xilinx/xilinx_axienet.h @@ -523,8 +523,9 @@ struct skbuf_dma_descriptor { * @stats_work: Work for reading the hardware statistics counters often enough * to catch overflows. * @dma_err_task: Work structure to process Axi DMA errors - * @stopping: Set when @dma_err_task shouldn't do anything because we are - * about to stop the device. + * @stopping: Set when we are about to stop the device: makes @dma_err_task + * a no-op (legacy DMA path) and fences RX descriptor + * resubmission in axienet_dma_rx_cb() (dmaengine path). * @tx_irq: Axidma TX IRQ number * @rx_irq: Axidma RX IRQ number * @eth_irq: Ethernet core IRQ number @@ -545,6 +546,7 @@ struct skbuf_dma_descriptor { * @tx_ring_tail: TX skb ring buffer tail index. * @rx_ring_head: RX skb ring buffer head index. * @rx_ring_tail: RX skb ring buffer tail index. + * @rx_submit_lock: Protects RX ring resubmission vs teardown in dmaengine path. */ struct axienet_local { struct net_device *ndev; @@ -626,6 +628,7 @@ struct axienet_local { int tx_ring_tail; int rx_ring_head; int rx_ring_tail; + spinlock_t rx_submit_lock; }; /** diff --git a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c index fcf517069d16..1ae4fd65e5ec 100644 --- a/drivers/net/ethernet/xilinx/xilinx_axienet_main.c +++ b/drivers/net/ethernet/xilinx/xilinx_axienet_main.c @@ -881,6 +881,7 @@ static void axienet_dma_tx_cb(void *data, const struct dmaengine_result *result) u64_stats_update_end(&lp->tx_stat_sync); dma_unmap_sg(lp->dev, skbuf_dma->sgl, skbuf_dma->sg_len, DMA_TO_DEVICE); dev_consume_skb_any(skbuf_dma->skb); + skbuf_dma->skb = NULL; netif_txq_completed_wake(txq, 1, len, CIRC_SPACE(lp->tx_ring_head, lp->tx_ring_tail, TX_BD_NUM_MAX), 2); @@ -1171,6 +1172,7 @@ static void axienet_dma_rx_cb(void *data, const struct dmaengine_result *result) &meta_max_len); dma_unmap_single(lp->dev, skbuf_dma->dma_address, lp->max_frm_size, DMA_FROM_DEVICE); + skbuf_dma->skb = NULL; if (IS_ERR(app_metadata)) { if (net_ratelimit()) @@ -1193,10 +1195,17 @@ static void axienet_dma_rx_cb(void *data, const struct dmaengine_result *result) u64_stats_update_end(&lp->rx_stat_sync); rx_submit: + spin_lock(&lp->rx_submit_lock); + if (lp->stopping) { + spin_unlock(&lp->rx_submit_lock); + return; + } + for (i = 0; i < CIRC_SPACE(lp->rx_ring_head, lp->rx_ring_tail, RX_BUF_NUM_DEFAULT); i++) axienet_rx_submit_desc(lp->ndev); dma_async_issue_pending(lp->rx_chan); + spin_unlock(&lp->rx_submit_lock); } /** @@ -1541,6 +1550,7 @@ static int axienet_init_dmaengine(struct net_device *ndev) lp->tx_ring_head = 0; lp->rx_ring_tail = 0; lp->rx_ring_head = 0; + lp->stopping = false; lp->tx_skb_ring = kzalloc_objs(*lp->tx_skb_ring, TX_BD_NUM_MAX); if (!lp->tx_skb_ring) { ret = -ENOMEM; @@ -1752,20 +1762,42 @@ static int axienet_stop(struct net_device *ndev) free_irq(lp->rx_irq, ndev); axienet_dma_bd_release(ndev); } else { + struct skbuf_dma_descriptor *skbuf_dma; + + spin_lock_bh(&lp->rx_submit_lock); + lp->stopping = true; + spin_unlock_bh(&lp->rx_submit_lock); + dmaengine_terminate_sync(lp->tx_chan); - dmaengine_synchronize(lp->tx_chan); dmaengine_terminate_sync(lp->rx_chan); - dmaengine_synchronize(lp->rx_chan); - - for (i = 0; i < TX_BD_NUM_MAX; i++) - kfree(lp->tx_skb_ring[i]); - kfree(lp->tx_skb_ring); - for (i = 0; i < RX_BUF_NUM_DEFAULT; i++) - kfree(lp->rx_skb_ring[i]); - kfree(lp->rx_skb_ring); dma_release_channel(lp->rx_chan); dma_release_channel(lp->tx_chan); + + /* Unmap and free any buffer the terminate did not reclaim, so it + * is not leaked; a non-NULL skb marks such a slot. + */ + for (i = 0; i < TX_BD_NUM_MAX; i++) { + skbuf_dma = lp->tx_skb_ring[i]; + if (skbuf_dma && skbuf_dma->skb) { + dma_unmap_sg(lp->dev, skbuf_dma->sgl, + skbuf_dma->sg_len, DMA_TO_DEVICE); + dev_kfree_skb_any(skbuf_dma->skb); + } + kfree(skbuf_dma); + } + kfree(lp->tx_skb_ring); + + for (i = 0; i < RX_BUF_NUM_DEFAULT; i++) { + skbuf_dma = lp->rx_skb_ring[i]; + if (skbuf_dma && skbuf_dma->skb) { + dma_unmap_single(lp->dev, skbuf_dma->dma_address, + lp->max_frm_size, DMA_FROM_DEVICE); + dev_kfree_skb_any(skbuf_dma->skb); + } + kfree(skbuf_dma); + } + kfree(lp->rx_skb_ring); } netdev_reset_queue(ndev); @@ -3062,6 +3094,7 @@ static int axienet_probe(struct platform_device *pdev) spin_lock_init(&lp->rx_cr_lock); spin_lock_init(&lp->tx_cr_lock); + spin_lock_init(&lp->rx_submit_lock); INIT_WORK(&lp->rx_dim.work, axienet_rx_dim_work); lp->rx_dim_enabled = true; lp->rx_dim.profile_ix = 1; diff --git a/drivers/net/pfcp.c b/drivers/net/pfcp.c index 8bbaf167db75..08557c038180 100644 --- a/drivers/net/pfcp.c +++ b/drivers/net/pfcp.c @@ -64,6 +64,13 @@ static int pfcp_encap_recv(struct sock *sk, struct sk_buff *skb) goto drop; unparsed = pfcp_hdr(skb); + if (unparsed->flags & PFCP_SEID_FLAG) { + if (unlikely(!pskb_may_pull(skb, PFCP_HLEN + + offsetofend(struct pfcphdr_session, + seid)))) + goto drop; + unparsed = pfcp_hdr(skb); + } ip_tunnel_flags_zero(flags); tun_dst = udp_tun_rx_dst(skb, sk->sk_family, flags, 0, diff --git a/drivers/net/team/team_core.c b/drivers/net/team/team_core.c index feaa75fbf8fc..3bf7c80f1d8e 100644 --- a/drivers/net/team/team_core.c +++ b/drivers/net/team/team_core.c @@ -820,7 +820,8 @@ static struct list_head *__team_get_qom_list(struct team *team, u16 queue_id) /* * note: already called with rcu_read_lock */ -static bool team_queue_override_transmit(struct team *team, struct sk_buff *skb) +static bool team_queue_override_transmit(struct team *team, struct sk_buff *skb, + bool *tx_success) { struct list_head *qom_list; struct team_port *port; @@ -828,11 +829,11 @@ static bool team_queue_override_transmit(struct team *team, struct sk_buff *skb) if (!team->queue_override_enabled || !skb->queue_mapping) return false; qom_list = __team_get_qom_list(team, skb->queue_mapping); - list_for_each_entry_rcu(port, qom_list, qom_list) { - if (!team_dev_queue_xmit(team, port, skb)) - return true; - } - return false; + port = list_first_or_null_rcu(qom_list, struct team_port, qom_list); + if (!port) + return false; + *tx_success = !team_dev_queue_xmit(team, port, skb); + return true; } static void __team_queue_override_port_del(struct team *team, @@ -1864,8 +1865,7 @@ static netdev_tx_t team_xmit(struct sk_buff *skb, struct net_device *dev) bool tx_success; unsigned int len = skb->len; - tx_success = team_queue_override_transmit(team, skb); - if (!tx_success) + if (!team_queue_override_transmit(team, skb, &tx_success)) tx_success = READ_ONCE(team->ops.transmit)(team, skb); if (tx_success) { struct team_pcpu_stats *pcpu_stats; diff --git a/drivers/net/team/team_mode_broadcast.c b/drivers/net/team/team_mode_broadcast.c index 61d7d79f0c36..69dc25a5847a 100644 --- a/drivers/net/team/team_mode_broadcast.c +++ b/drivers/net/team/team_mode_broadcast.c @@ -37,6 +37,8 @@ static bool bc_transmit(struct team *team, struct sk_buff *skb) ret = !team_dev_queue_xmit(team, last, skb); if (!sum_ret) sum_ret = ret; + } else { + dev_kfree_skb_any(skb); } return sum_ret; } diff --git a/drivers/net/tun.c b/drivers/net/tun.c index 5bbe3123979e..dcd39815205a 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -1818,12 +1818,7 @@ static ssize_t tun_get_user(struct tun_struct *tun, struct tun_file *tfile, } } - if (tun_vnet_hdr_tnl_to_skb(tun->flags, features, skb, &hdr)) { - atomic_long_inc(&tun->rx_frame_errors); - err = -EINVAL; - goto free_skb; - } - + skb->dev = tun->dev; switch (tun->flags & TUN_TYPE_MASK) { case IFF_TUN: if (tun->flags & IFF_NO_PI) { @@ -1848,9 +1843,8 @@ static ssize_t tun_get_user(struct tun_struct *tun, struct tun_file *tfile, } } - skb_reset_mac_header(skb); + skb_reset_network_header(skb); skb->protocol = pi.proto; - skb->dev = tun->dev; break; case IFF_TAP: if (!pskb_may_pull(skb, ETH_HLEN)) { @@ -1858,10 +1852,19 @@ static ssize_t tun_get_user(struct tun_struct *tun, struct tun_file *tfile, drop_reason = SKB_DROP_REASON_HDR_TRUNC; goto drop; } - skb->protocol = eth_type_trans(skb, tun->dev); + skb_set_network_header(skb, ETH_HLEN); break; } + if (tun_vnet_hdr_tnl_to_skb(tun->flags, features, skb, &hdr)) { + atomic_long_inc(&tun->rx_frame_errors); + err = -EINVAL; + goto free_skb; + } + + if ((tun->flags & TUN_TYPE_MASK) == IFF_TAP) + skb->protocol = eth_type_trans(skb, tun->dev); + /* copy skb_ubuf_info for callback when skb has no error */ if (zerocopy) { skb_zcopy_init(skb, msg_control); @@ -2469,6 +2472,8 @@ static int tun_xdp_one(struct tun_struct *tun, features = tun_vnet_hdr_guest_features(READ_ONCE(tun->vnet_hdr_sz)); tnl_hdr = (struct virtio_net_hdr_v1_hash_tunnel *)gso; + skb->dev = tun->dev; + skb_set_network_header(skb, ETH_HLEN); if (tun_vnet_hdr_tnl_to_skb(tun->flags, features, skb, tnl_hdr)) { atomic_long_inc(&tun->rx_frame_errors); kfree_skb(skb); diff --git a/drivers/net/usb/ax88179_178a.c b/drivers/net/usb/ax88179_178a.c index 81d8412ce8e2..cba1dc95d359 100644 --- a/drivers/net/usb/ax88179_178a.c +++ b/drivers/net/usb/ax88179_178a.c @@ -1457,11 +1457,11 @@ static int ax88179_rx_fixup(struct usbnet *dev, struct sk_buff *skb) return 1; } - ax_skb = netdev_alloc_skb_ip_align(dev->net, pkt_len); + ax_skb = netdev_alloc_skb_ip_align(dev->net, pkt_len - 2); if (!ax_skb) return 0; - skb_put(ax_skb, pkt_len); - memcpy(ax_skb->data, skb->data + 2, pkt_len); + skb_put(ax_skb, pkt_len - 2); + memcpy(ax_skb->data, skb->data + 2, pkt_len - 2); ax88179_rx_checksum(ax_skb, pkt_hdr); usbnet_skb_return(dev, ax_skb); diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 2780b2dd073d..4f145c5f686e 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1414,6 +1414,7 @@ static int veth_open(struct net_device *dev) } veth_set_xdp_features(dev); + veth_set_xdp_features(peer); return 0; } diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index bf82ef9874ab..daab43ac92ce 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -2515,6 +2515,8 @@ static void virtnet_receive_done(struct virtnet_info *vi, struct receive_queue * goto frame_err; } + skb->dev = dev; + skb_set_network_header(skb, ETH_HLEN); if (virtio_net_hdr_tnl_to_skb(skb, &hdr->tnl_hdr, vi->rx_tnl, vi->rx_tnl_csum, virtio_is_little_endian(vi->vdev))) { diff --git a/drivers/net/wireguard/noise.c b/drivers/net/wireguard/noise.c index 9c0a09bf6c95..ec2513d6dfab 100644 --- a/drivers/net/wireguard/noise.c +++ b/drivers/net/wireguard/noise.c @@ -784,10 +784,11 @@ wg_noise_handshake_consume_response(struct message_handshake_response *src, /* Success! Copy everything to peer */ down_write(&handshake->lock); - /* It's important to check that the state is still the same, while we - * have an exclusive lock. + /* Check that the state is the same and that this is still the + * initiation we started with, while we have an exclusive lock. */ - if (handshake->state != state) { + if (handshake->state != state || + crypto_memneq(handshake->ephemeral_private, ephemeral_private, NOISE_PUBLIC_KEY_LEN)) { up_write(&handshake->lock); goto fail; } diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h index 79b6d70de236..663b19b78395 100644 --- a/drivers/net/wireguard/queueing.h +++ b/drivers/net/wireguard/queueing.h @@ -78,12 +78,14 @@ static inline void wg_reset_packet(struct sk_buff *skb, bool encapsulating) u8 l4_hash = skb->l4_hash; u8 sw_hash = skb->sw_hash; u32 hash = skb->hash; + u8 tstamp_type = skb->tstamp_type; skb_scrub_packet(skb, true); memset(&skb->headers, 0, sizeof(skb->headers)); if (encapsulating) { skb->l4_hash = l4_hash; skb->sw_hash = sw_hash; skb->hash = hash; + skb->tstamp_type = tstamp_type; } skb->queue_mapping = 0; skb->nohdr = 0; diff --git a/drivers/net/wireless/ath/ath12k/core.c b/drivers/net/wireless/ath/ath12k/core.c index 5cdf4973d986..992602965fb4 100644 --- a/drivers/net/wireless/ath/ath12k/core.c +++ b/drivers/net/wireless/ath/ath12k/core.c @@ -1770,29 +1770,6 @@ int ath12k_core_pre_init(struct ath12k_base *ab) return 0; } -static int ath12k_core_panic_handler(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct ath12k_base *ab = container_of(nb, struct ath12k_base, - panic_nb); - - return ath12k_hif_panic_handler(ab); -} - -static int ath12k_core_panic_notifier_register(struct ath12k_base *ab) -{ - ab->panic_nb.notifier_call = ath12k_core_panic_handler; - - return atomic_notifier_chain_register(&panic_notifier_list, - &ab->panic_nb); -} - -static void ath12k_core_panic_notifier_unregister(struct ath12k_base *ab) -{ - atomic_notifier_chain_unregister(&panic_notifier_list, - &ab->panic_nb); -} - static inline bool ath12k_core_hw_group_create_ready(struct ath12k_hw_group *ag) { @@ -2208,18 +2185,13 @@ int ath12k_core_init(struct ath12k_base *ab) struct ath12k_hw_group *ag; int ret; - ret = ath12k_core_panic_notifier_register(ab); - if (ret) - ath12k_warn(ab, "failed to register panic handler: %d\n", ret); - mutex_lock(&ath12k_hw_group_mutex); ag = ath12k_core_hw_group_assign(ab); if (!ag) { mutex_unlock(&ath12k_hw_group_mutex); ath12k_warn(ab, "unable to get hw group\n"); - ret = -ENODEV; - goto err_unregister_notifier; + return -ENODEV; } mutex_unlock(&ath12k_hw_group_mutex); @@ -2244,8 +2216,6 @@ int ath12k_core_init(struct ath12k_base *ab) err_unassign_hw_group: ath12k_core_hw_group_unassign(ab); -err_unregister_notifier: - ath12k_core_panic_notifier_unregister(ab); return ret; } @@ -2254,7 +2224,6 @@ void ath12k_core_deinit(struct ath12k_base *ab) { ath12k_core_hw_group_destroy(ab->ag); ath12k_core_hw_group_unassign(ab); - ath12k_core_panic_notifier_unregister(ab); } void ath12k_core_free(struct ath12k_base *ab) diff --git a/drivers/net/wireless/ath/ath12k/core.h b/drivers/net/wireless/ath/ath12k/core.h index f28d2e90b67b..708479df4b89 100644 --- a/drivers/net/wireless/ath/ath12k/core.h +++ b/drivers/net/wireless/ath/ath12k/core.h @@ -15,7 +15,6 @@ #include #include #include -#include #include #include #include @@ -1146,8 +1145,6 @@ struct ath12k_base { #endif /* CONFIG_ACPI */ - struct notifier_block panic_nb; - struct ath12k_hw_group *ag; struct ath12k_wsi_info wsi_info; enum ath12k_firmware_mode fw_mode; diff --git a/drivers/net/wireless/ath/ath12k/hif.h b/drivers/net/wireless/ath/ath12k/hif.h index 4de8667690e9..aecdb0c5f285 100644 --- a/drivers/net/wireless/ath/ath12k/hif.h +++ b/drivers/net/wireless/ath/ath12k/hif.h @@ -30,7 +30,6 @@ struct ath12k_hif_ops { void (*ce_irq_enable)(struct ath12k_base *ab); void (*ce_irq_disable)(struct ath12k_base *ab); void (*get_ce_msi_idx)(struct ath12k_base *ab, u32 ce_id, u32 *msi_idx); - int (*panic_handler)(struct ath12k_base *ab); void (*coredump_download)(struct ath12k_base *ab); }; @@ -149,14 +148,6 @@ static inline void ath12k_hif_power_down(struct ath12k_base *ab, bool is_suspend ab->hif.ops->power_down(ab, is_suspend); } -static inline int ath12k_hif_panic_handler(struct ath12k_base *ab) -{ - if (!ab->hif.ops->panic_handler) - return NOTIFY_DONE; - - return ab->hif.ops->panic_handler(ab); -} - static inline void ath12k_hif_coredump_download(struct ath12k_base *ab) { if (ab->hif.ops->coredump_download) diff --git a/drivers/net/wireless/ath/ath12k/pci.c b/drivers/net/wireless/ath/ath12k/pci.c index fee4129ea405..a41544665080 100644 --- a/drivers/net/wireless/ath/ath12k/pci.c +++ b/drivers/net/wireless/ath/ath12k/pci.c @@ -1473,13 +1473,6 @@ void ath12k_pci_power_down(struct ath12k_base *ab, bool is_suspend) ath12k_pci_sw_reset(ab_pci->ab, false); } -static int ath12k_pci_panic_handler(struct ath12k_base *ab) -{ - ath12k_pci_sw_reset(ab, false); - - return NOTIFY_OK; -} - static const struct ath12k_hif_ops ath12k_pci_hif_ops = { .start = ath12k_pci_start, .stop = ath12k_pci_stop, @@ -1497,7 +1490,6 @@ static const struct ath12k_hif_ops ath12k_pci_hif_ops = { .ce_irq_enable = ath12k_pci_hif_ce_irq_enable, .ce_irq_disable = ath12k_pci_hif_ce_irq_disable, .get_ce_msi_idx = ath12k_pci_get_ce_msi_idx, - .panic_handler = ath12k_pci_panic_handler, #ifdef CONFIG_ATH12K_COREDUMP .coredump_download = ath12k_pci_coredump_download, #endif diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c index d2ae67985606..92ec269f43b2 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c @@ -1489,8 +1489,31 @@ void brcmf_detach(struct device *dev) brcmf_bus_change_state(bus_if, BRCMF_BUS_DOWN); /* make sure primary interface removed last */ for (i = BRCMF_MAX_IFS - 1; i > -1; i--) { - if (drvr->iflist[i]) - brcmf_remove_interface(drvr->iflist[i], false); + struct brcmf_if *ifp = drvr->iflist[i]; + + if (!ifp) + continue; + + if (ifp->ndev) { + brcmf_remove_interface(ifp, false); + continue; + } + + /* The P2P device interface has no netdev. Its removal can + * race with NL80211_CMD_DEL_INTERFACE issued by a user space + * process that is exiting (e.g. wpa_supplicant), which ends + * in brcmf_p2p_del_vif() under RTNL and the wiphy mutex. If + * both paths remove the interface, the wdev is unregistered + * twice and the vif is used after being freed. Take the same + * locks and re-read iflist, so only one path removes it. + */ + rtnl_lock(); + wiphy_lock(drvr->wiphy); + ifp = drvr->iflist[i]; + if (ifp) + brcmf_remove_interface(ifp, true); + wiphy_unlock(drvr->wiphy); + rtnl_unlock(); } brcmf_bus_stop(drvr->bus_if); diff --git a/drivers/net/wireless/marvell/libertas/tx.c b/drivers/net/wireless/marvell/libertas/tx.c index 13d08022e414..27304a98787d 100644 --- a/drivers/net/wireless/marvell/libertas/tx.c +++ b/drivers/net/wireless/marvell/libertas/tx.c @@ -117,13 +117,6 @@ netdev_tx_t lbs_hard_start_xmit(struct sk_buff *skb, struct net_device *dev) if (priv->wdev->iftype == NL80211_IFTYPE_MONITOR) { struct tx_radiotap_hdr *rtap_hdr = (void *)skb->data; - if (skb->len < sizeof(*rtap_hdr) + 4 + ETH_ALEN) { - lbs_deb_tx("tx err: short monitor frame %u\n", skb->len); - dev->stats.tx_dropped++; - dev->stats.tx_errors++; - goto free; - } - /* set txpd fields from the radiotap header */ txpd->tx_control = cpu_to_le32(convert_radiotap_rate_to_mv(rtap_hdr->rate)); diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h index 476578e2fbf0..9188a27e7865 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76.h +++ b/drivers/net/wireless/mediatek/mt76/mt76.h @@ -539,6 +539,7 @@ struct mt76_hw_cap { #define MT_DRV_HW_MGMT_TXQ BIT(4) #define MT_DRV_AMSDU_OFFLOAD BIT(5) #define MT_DRV_IGNORE_TXS_FAILED BIT(6) +#define MT_DRV_HW_PS_BUFFERING BIT(7) struct mt76_driver_ops { u32 drv_flags; @@ -1424,6 +1425,13 @@ mtxq_to_txq(struct mt76_txq *mtxq) return container_of(ptr, struct ieee80211_txq, drv_priv); } +/* peer-wide state uses the wcid of the primary link */ +static inline struct mt76_wcid * +mt76_wcid_primary(struct mt76_wcid *wcid) +{ + return wcid->def_wcid ? wcid->def_wcid : wcid; +} + static inline struct ieee80211_sta * wcid_to_sta(struct mt76_wcid *wcid) { @@ -1541,6 +1549,8 @@ void mt76_release_buffered_frames(struct ieee80211_hw *hw, u16 tids, int nframes, enum ieee80211_frame_release_type reason, bool more_data); +void mt76_sta_ps_transition(struct mt76_dev *dev, struct mt76_wcid *wcid, + bool ps); bool mt76_has_tx_pending(struct mt76_phy *phy); int mt76_update_channel(struct mt76_phy *phy); void mt76_update_survey(struct mt76_phy *phy); diff --git a/drivers/net/wireless/mediatek/mt76/mt7603/init.c b/drivers/net/wireless/mediatek/mt76/mt7603/init.c index 10f2ec70c792..218ec406d2a8 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7603/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7603/init.c @@ -230,6 +230,7 @@ mt7603_mac_init(struct mt7603_dev *dev) eth_broadcast_addr(bc_addr); mt7603_wtbl_init(dev, MT7603_WTBL_RESERVED, -1, bc_addr); + mt76_wcid_init(&dev->global_sta.wcid, 0); dev->global_sta.wcid.idx = MT7603_WTBL_RESERVED; rcu_assign_pointer(dev->mt76.wcid[MT7603_WTBL_RESERVED], &dev->global_sta.wcid); diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h index a9a4a87ae0a7..fe03bc70e63a 100644 --- a/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h +++ b/drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h @@ -1068,6 +1068,7 @@ enum { MCU_UNI_EVENT_IE_COUNTDOWN = 0x09, MCU_UNI_EVENT_COREDUMP = 0x0a, MCU_UNI_EVENT_BSS_BEACON_LOSS = 0x0c, + MCU_UNI_EVENT_PS_SYNC = 0x0d, MCU_UNI_EVENT_SCAN_DONE = 0x0e, MCU_UNI_EVENT_RDD_REPORT = 0x11, MCU_UNI_EVENT_ROC = 0x27, diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/init.c b/drivers/net/wireless/mediatek/mt76/mt7915/init.c index 2ab38f9e5258..e768f4f72ed8 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/init.c @@ -1208,6 +1208,12 @@ static void mt7915_unregister_ext_phy(struct mt7915_dev *dev) static void mt7915_stop_hardware(struct mt7915_dev *dev) { + int i; + + mt76_for_each_q_rx(&dev->mt76, i) { + if (!mt76_queue_is_wed_rro(&dev->mt76.q_rx[i])) + napi_disable(&dev->mt76.napi[i]); + } mt7915_mcu_exit(dev); mt76_connac2_tx_token_put(&dev->mt76); mt7915_dma_cleanup(dev); diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c index 4783e5f52d22..1830398ff719 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c @@ -805,6 +805,7 @@ __mt7915_drop_sta(void *ptr, u8 *mac, struct ieee80211_vif *vif) msta = (struct mt7915_sta *)sta->drv_priv; mt7915_mcu_add_sta(data->dev, vif, sta, CONN_STATE_DISCONNECT, false); + msta->wcid.tx_info &= ~MT_WCID_TX_INFO_SET; msta->wcid.sta_disabled = 1; msta->wcid.sta = 0; } @@ -871,6 +872,7 @@ int mt7915_mac_sta_event(struct mt76_dev *mdev, struct ieee80211_vif *vif, mutex_unlock(&dev->mt76.mutex); mt7915_mcu_add_sta(dev, vif, sta, CONN_STATE_DISCONNECT, false); + msta->wcid.tx_info &= ~MT_WCID_TX_INFO_SET; msta->wcid.sta_disabled = 1; msta->wcid.sta = 0; return 0; diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c index b52b67833060..8603915ddc2e 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/mac.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/mac.c @@ -781,7 +781,8 @@ mt7925_mac_write_txwi(struct mt76_dev *dev, __le32 *txwi, val = FIELD_PREP(MT_TXD1_WLAN_IDX, wcid->idx) | FIELD_PREP(MT_TXD1_OWN_MAC, omac_idx); - if (band_idx) + /* 0xff means BAND_AUTO to the firmware, and TGID cannot encode it */ + if (band_idx && band_idx != 0xff) val |= FIELD_PREP(MT_TXD1_TGID, band_idx); txwi[1] = cpu_to_le32(val); diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c index 9f080da13341..e6e5c26307a1 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c @@ -2220,6 +2220,18 @@ mt7925_change_vif_links(struct ieee80211_hw *hw, struct ieee80211_vif *vif, mvif->valid_links = new_links; + /* Restore the legacy BSS after disabling MLO */ + if (old_links && !new_links) { + mt792x_mac_link_bss_remove(dev, &mvif->bss_conf, + &mvif->sta.deflink); + err = mt7925_mac_link_bss_add(dev, &vif->bss_conf, + &mvif->sta.deflink); + if (err < 0) { + mt792x_mutex_release(dev); + return err; + } + } + mt792x_mutex_release(dev); return 0; diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.c b/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.c index 946dd7956e4a..ce5d5909d838 100644 --- a/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.c +++ b/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.c @@ -108,68 +108,98 @@ mt792x_asar_acpi_read_mtcl(struct mt792x_dev *dev, u8 **table, u8 *version) return ret; } +/* A table's layout is decided by the version byte in MTCL, but some firmware + * labels a version 2 table as version 1. Accept whichever layout the table + * actually has: for every accepted entry count the two do not overlap, since + * one is 6 + 5n bytes and the other 7 + 7n. + */ +static bool +mt792x_asar_fits(u8 *table, int len, int prelen, int sarlen, int nr_off, + int min, int max) +{ + int tblcnt; + + if (len < prelen || (len - prelen) % sarlen) + return false; + + tblcnt = (len - prelen) / sarlen; + if (tblcnt < min || tblcnt > max) + return false; + + return tblcnt == table[nr_off]; +} + /* MTDS : Dynamic SAR Power Table */ static int -mt792x_asar_acpi_read_mtds(struct mt792x_dev *dev, u8 **table, u8 version) +mt792x_asar_acpi_read_mtds(struct mt792x_dev *dev, u8 **table, u8 version, + u8 *used) { - int len, ret, sarlen, prelen, tblcnt; - bool enable; + int len, ret; + u8 *t; ret = mt792x_acpi_read(dev, MT792x_ACPI_MTDS, table, &len); if (ret) return ret; + t = *table; + /* Table content validation */ - switch (version) { - case 1: - enable = ((struct mt792x_asar_dyn *)*table)->enable; - sarlen = sizeof(struct mt792x_asar_dyn_limit); - prelen = sizeof(struct mt792x_asar_dyn); - break; - case 2: - enable = ((struct mt792x_asar_dyn_v2 *)*table)->enable; - sarlen = sizeof(struct mt792x_asar_dyn_limit_v2); - prelen = sizeof(struct mt792x_asar_dyn_v2); - break; - default: + if (mt792x_asar_fits(t, len, sizeof(struct mt792x_asar_dyn), + sizeof(struct mt792x_asar_dyn_limit), + offsetof(struct mt792x_asar_dyn, nr_tbl), + MT792x_ASAR_MIN_DYN, MT792x_ASAR_MAX_DYN)) + *used = 1; + else if (mt792x_asar_fits(t, len, sizeof(struct mt792x_asar_dyn_v2), + sizeof(struct mt792x_asar_dyn_limit_v2), + offsetof(struct mt792x_asar_dyn_v2, nr_tbl), + MT792x_ASAR_MIN_DYN, MT792x_ASAR_MAX_DYN)) + *used = 2; + else return -EINVAL; - } - tblcnt = (len - prelen) / sarlen; - if (!enable || - tblcnt > MT792x_ASAR_MAX_DYN || tblcnt < MT792x_ASAR_MIN_DYN) + if (!t[offsetof(struct mt792x_asar_dyn, enable)]) return -EINVAL; + if (version && *used != version) + dev_info(dev->mt76.dev, + "MTDS is v%u, MTCL says v%u; using v%u\n", + *used, version, *used); + return 0; } /* MTGS : Geo SAR Power Table */ static int -mt792x_asar_acpi_read_mtgs(struct mt792x_dev *dev, u8 **table, u8 version) +mt792x_asar_acpi_read_mtgs(struct mt792x_dev *dev, u8 **table, u8 version, + u8 *used) { - int len, ret, sarlen, prelen, tblcnt; + int len, ret; + u8 *t; ret = mt792x_acpi_read(dev, MT792x_ACPI_MTGS, table, &len); if (ret) return ret; + t = *table; + /* Table content validation */ - switch (version) { - case 1: - sarlen = sizeof(struct mt792x_asar_geo_limit); - prelen = sizeof(struct mt792x_asar_geo); - break; - case 2: - sarlen = sizeof(struct mt792x_asar_geo_limit_v2); - prelen = sizeof(struct mt792x_asar_geo_v2); - break; - default: + if (mt792x_asar_fits(t, len, sizeof(struct mt792x_asar_geo), + sizeof(struct mt792x_asar_geo_limit), + offsetof(struct mt792x_asar_geo, nr_tbl), + MT792x_ASAR_MIN_GEO, MT792x_ASAR_MAX_GEO)) + *used = 1; + else if (mt792x_asar_fits(t, len, sizeof(struct mt792x_asar_geo_v2), + sizeof(struct mt792x_asar_geo_limit_v2), + offsetof(struct mt792x_asar_geo_v2, nr_tbl), + MT792x_ASAR_MIN_GEO, MT792x_ASAR_MAX_GEO)) + *used = 2; + else return -EINVAL; - } - tblcnt = (len - prelen) / sarlen; - if (tblcnt > MT792x_ASAR_MAX_GEO || tblcnt < MT792x_ASAR_MIN_GEO) - return -EINVAL; + if (version && *used != version) + dev_info(dev->mt76.dev, + "MTGS is v%u, MTCL says v%u; using v%u\n", + *used, version, *used); return 0; } @@ -205,14 +235,16 @@ int mt792x_init_acpi_sar(struct mt792x_dev *dev) asar->countrylist = NULL; } - ret = mt792x_asar_acpi_read_mtds(dev, (u8 **)&asar->dyn, asar->ver); + ret = mt792x_asar_acpi_read_mtds(dev, (u8 **)&asar->dyn, asar->ver, + &asar->dyn_ver); if (ret) { devm_kfree(dev->mt76.dev, asar->dyn); asar->dyn = NULL; } /* MTGS is optional */ - ret = mt792x_asar_acpi_read_mtgs(dev, (u8 **)&asar->geo, asar->ver); + ret = mt792x_asar_acpi_read_mtgs(dev, (u8 **)&asar->geo, asar->ver, + &asar->geo_ver); if (ret) { devm_kfree(dev->mt76.dev, asar->geo); asar->geo = NULL; @@ -254,7 +286,7 @@ mt792x_asar_get_geo_pwr(struct mt792x_phy *phy, break; } - if (asar->ver == 1) { + if (asar->geo_ver == 1) { band_pwr = &asar->geo->tbl[idx].band[0]; max = ARRAY_SIZE(asar->geo->tbl[idx].band); } else { @@ -279,10 +311,18 @@ mt792x_asar_get_geo_pwr(struct mt792x_phy *phy, if (idx >= max) return dyn_power; - geo_power = (band_pwr + idx)->pwr; - dyn_power += (band_pwr + idx)->offset; + band_pwr += idx; + + /* Some OEM tables carry 0xff here; narrowed to s8 that is -1 and + * wins the min() below. Take it as no limit. + */ + if (band_pwr->pwr == 0xff) + return dyn_power; + + geo_power = band_pwr->pwr; - return min(geo_power, dyn_power); + return min_t(int, geo_power, + min_t(int, dyn_power + band_pwr->offset, 127)); } static s8 @@ -297,7 +337,7 @@ mt792x_asar_range_pwr(struct mt792x_phy *phy, if (!capa) return 127; - if (asar->ver == 1) { + if (asar->dyn_ver == 1) { limit = &asar->dyn->tbl[0].frp[0]; max = ARRAY_SIZE(asar->dyn->tbl[0].frp); } else { @@ -315,7 +355,9 @@ mt792x_asar_range_pwr(struct mt792x_phy *phy, else band = NL80211_BAND_2GHZ; - return mt792x_asar_get_geo_pwr(phy, band, limit[idx]); + /* 0xff: no dynamic limit, see mt792x_asar_get_geo_pwr() */ + return mt792x_asar_get_geo_pwr(phy, band, + limit[idx] == 0xff ? 127 : limit[idx]); } int mt792x_init_acpi_sar_power(struct mt792x_phy *phy, bool set_default) diff --git a/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.h b/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.h index 474033073831..1d42f24b05f6 100644 --- a/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.h +++ b/drivers/net/wireless/mediatek/mt76/mt792x_acpi_sar.h @@ -105,6 +105,8 @@ struct mt792x_asar_fg { struct mt792x_acpi_sar { u8 ver; + u8 dyn_ver; + u8 geo_ver; union { struct mt792x_asar_dyn *dyn; struct mt792x_asar_dyn_v2 *dyn_v2; diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/init.c b/drivers/net/wireless/mediatek/mt76/mt7996/init.c index fa74aba42690..95fb30404b85 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7996/init.c +++ b/drivers/net/wireless/mediatek/mt76/mt7996/init.c @@ -536,6 +536,7 @@ mt7996_init_wiphy(struct ieee80211_hw *hw, struct mtk_wed_device *wed) ieee80211_hw_set(hw, NO_VIRTUAL_MONITOR); ieee80211_hw_set(hw, SUPPORTS_MULTI_BSSID); ieee80211_hw_set(hw, CHANCTX_STA_CSA); + ieee80211_hw_set(hw, CONNECTION_MONITOR); hw->max_tx_fragments = 4; wiphy->txq_memory_limit = 32 << 20; /* 32 MiB */ @@ -1237,7 +1238,7 @@ static int mt7996_variant_fem_init(struct mt7996_dev *dev) adie_ver = u32_get_bits(regval, MT_ADIE_VERSION_MASK); idx = MT7976C_EFUSE_OFFSET % MT7996_EEPROM_BLOCK_SIZE; is_7976c = adie_ver == 0x8a10 || adie_ver == 0x8b00 || - adie_ver == 0x8c10 || buf[idx] == 0xc; + adie_ver == 0x8c10 || (!ret && buf[idx] == 0xc); adie_id = u32_get_bits(regval, MT_ADIE_CHIP_ID_MASK); if (adie_id == 0x7975 || adie_id == 0x7979 || diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/main.c b/drivers/net/wireless/mediatek/mt76/mt7996/main.c index c6140412ed32..ae006f29091f 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7996/main.c +++ b/drivers/net/wireless/mediatek/mt76/mt7996/main.c @@ -310,7 +310,14 @@ int mt7996_vif_link_add(struct mt76_phy *mphy, struct ieee80211_vif *vif, if (vif->type == NL80211_IFTYPE_AP) return mt7996_mcu_mld_link_oper(dev, link_conf, link, true); - return 0; + + /* update the link address */ + ret = mt7996_mcu_add_dev_info(phy, vif, link_conf, mlink, true); + if (ret) + return ret; + + return mt7996_mcu_add_bss_info(phy, vif, link_conf, mlink, + msta_link, true); } mlink->idx = __ffs64(~dev->mt76.vif_mask); @@ -1422,6 +1429,7 @@ mt7996_mac_sta_event(struct mt7996_dev *dev, struct ieee80211_vif *vif, else if (sta->mlo && links == BIT(link_id)) /* last link */ mt7996_mcu_teardown_mld_sta(dev, link, msta_link); + msta_link->wcid.tx_info &= ~MT_WCID_TX_INFO_SET; msta_link->wcid.sta_disabled = 1; msta_link->wcid.sta = 0; links = links & ~BIT(link_id); diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h index c673e986ecb5..288b923750c0 100644 --- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h +++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h @@ -284,6 +284,9 @@ struct mt7996_mcu_wed_rro_ba_delete_event { __le16 len; __le16 session_id; + __le16 wlan_id; + u8 tid; + u8 __rsv1; u8 __rsv2[2]; } __packed; diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c index b03be0eb4712..65703dbb185c 100644 --- a/drivers/net/wireless/mediatek/mt76/tx.c +++ b/drivers/net/wireless/mediatek/mt76/tx.c @@ -234,6 +234,8 @@ mt76_tx_check_non_aql(struct mt76_dev *dev, struct mt76_wcid *wcid, if (!wcid || info->tx_time_est) return; + wcid = mt76_wcid_primary(wcid); + pending = atomic_dec_return(&wcid->non_aql_packets); if (pending < 0) atomic_cmpxchg(&wcid->non_aql_packets, pending, 0); @@ -267,6 +269,21 @@ void __mt76_tx_complete_skb(struct mt76_dev *dev, u16 wcid_idx, struct sk_buff * wcid = __mt76_wcid_ptr(dev, wcid_idx); mt76_tx_check_non_aql(dev, wcid, skb); + if (wcid && (dev->drv->drv_flags & MT_DRV_HW_PS_BUFFERING) && + test_bit(MT_WCID_FLAG_PS, &wcid->flags)) { + struct ieee80211_sta *sta = wcid_to_sta(wcid); + + if (sta) { + struct ieee80211_hw *hw = mt76_phy_hw(dev, wcid->phy_idx); + int i; + + for (i = 0; i < ARRAY_SIZE(sta->txq); i++) + if (sta->txq[i]) + ieee80211_schedule_txq(hw, sta->txq[i]); + mt76_worker_schedule(&dev->tx_worker); + } + } + #ifdef CONFIG_NL80211_TESTMODE if (mt76_is_testmode_skb(dev, skb, &hw)) { struct mt76_phy *phy = hw->priv; @@ -329,7 +346,8 @@ __mt76_tx_queue_skb(struct mt76_phy *phy, int qid, struct sk_buff *skb, if (!non_aql) return idx; - pending = atomic_inc_return(&wcid->non_aql_packets); + /* the hardware can report the completion on a different link */ + pending = atomic_inc_return(&mt76_wcid_primary(wcid)->non_aql_packets); if (stop && pending >= MT_MAX_NON_AQL_PKT) *stop = true; @@ -476,10 +494,15 @@ mt76_txq_send_burst(struct mt76_phy *phy, struct mt76_queue *q, bool stop = false; int idx; - if (test_bit(MT_WCID_FLAG_PS, &wcid->flags)) - return 0; + if (test_bit(MT_WCID_FLAG_PS, &wcid->flags)) { + if (!(dev->drv->drv_flags & MT_DRV_HW_PS_BUFFERING)) + return 0; + if (ieee80211_txq_aql_pending(phy->hw, txq)) + return 0; + } - if (atomic_read(&wcid->non_aql_packets) >= MT_MAX_NON_AQL_PKT) + if (atomic_read(&mt76_wcid_primary(wcid)->non_aql_packets) >= + MT_MAX_NON_AQL_PKT) return 0; skb = mt76_txq_dequeue(phy, mtxq); @@ -497,6 +520,9 @@ mt76_txq_send_burst(struct mt76_phy *phy, struct mt76_queue *q, if (idx < 0) return idx; + if (test_bit(MT_WCID_FLAG_PS, &wcid->flags)) + goto out; + do { if (test_bit(MT76_RESET, &phy->state) || phy->offchannel) break; @@ -522,6 +548,7 @@ mt76_txq_send_burst(struct mt76_phy *phy, struct mt76_queue *q, n_frames++; } while (1); +out: spin_lock(&q->lock); dev->queue_ops->kick(dev, q); spin_unlock(&q->lock); @@ -548,10 +575,7 @@ mt76_txq_schedule_list(struct mt76_phy *phy, enum mt76_txq_id qid) mtxq = (struct mt76_txq *)txq->drv_priv; wcid = __mt76_wcid_ptr(dev, mtxq->wcid); - if (!wcid || test_bit(MT_WCID_FLAG_PS, &wcid->flags)) - continue; - - if (atomic_read(&wcid->non_aql_packets) >= MT_MAX_NON_AQL_PKT) + if (!wcid) continue; phy = mt76_dev_phy(dev, wcid->phy_idx); @@ -559,6 +583,25 @@ mt76_txq_schedule_list(struct mt76_phy *phy, enum mt76_txq_id qid) continue; q = phy->q_tx[qid]; + + if (test_bit(MT_WCID_FLAG_PS, &wcid->flags)) { + if (!(dev->drv->drv_flags & MT_DRV_HW_PS_BUFFERING)) + continue; + + if (!mt76_txq_stopped(q)) + n_frames = mt76_txq_send_burst(phy, q, mtxq, wcid); + + ieee80211_return_txq(phy->hw, txq, false); + + if (unlikely(n_frames < 0)) + return n_frames; + ret += n_frames; + continue; + } + + if (atomic_read(&mt76_wcid_primary(wcid)->non_aql_packets) >= + MT_MAX_NON_AQL_PKT) + continue; if (dev->queue_ops->tx_cleanup && q->queued + 2 * MT_TXQ_FREE_THR >= q->ndesc) { dev->queue_ops->tx_cleanup(dev, q, false); @@ -629,7 +672,8 @@ mt76_txq_schedule_pending_wcid(struct mt76_phy *phy, struct mt76_wcid *wcid, if ((dev->drv->drv_flags & MT_DRV_HW_MGMT_TXQ) && !(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) && !ieee80211_is_data_present(hdr->frame_control) && - (!ieee80211_is_bufferable_mmpdu(skb) || + (!(wcid->tx_info & MT_WCID_TX_INFO_SET) || + !ieee80211_is_bufferable_mmpdu(skb) || ieee80211_is_deauth(hdr->frame_control) || ieee80211_is_disassoc(hdr->frame_control) || head == &wcid->tx_offchannel)) @@ -765,6 +809,33 @@ void mt76_stop_tx_queues(struct mt76_phy *phy, struct ieee80211_sta *sta, } EXPORT_SYMBOL_GPL(mt76_stop_tx_queues); +void mt76_sta_ps_transition(struct mt76_dev *dev, struct mt76_wcid *wcid, + bool ps) +{ + struct ieee80211_sta *sta; + struct ieee80211_hw *hw; + int i; + + if (ps) { + set_bit(MT_WCID_FLAG_PS, &wcid->flags); + mt76_worker_schedule(&dev->tx_worker); + return; + } + + clear_bit(MT_WCID_FLAG_PS, &wcid->flags); + + sta = wcid_to_sta(wcid); + if (!sta) + return; + + hw = mt76_phy_hw(dev, wcid->phy_idx); + for (i = 0; i < ARRAY_SIZE(sta->txq); i++) + if (sta->txq[i]) + ieee80211_schedule_txq(hw, sta->txq[i]); + mt76_worker_schedule(&dev->tx_worker); +} +EXPORT_SYMBOL_GPL(mt76_sta_ps_transition); + void mt76_wake_tx_queue(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { struct mt76_phy *phy = hw->priv; diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index d269457e839e..fdcb91042f29 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -1346,8 +1346,10 @@ static int xennet_poll(struct napi_struct *napi, int budget) skb->data_len = rx->status; skb->len += rx->status; - if (unlikely(xennet_fill_frags(queue, skb, &tmpq))) + if (unlikely(xennet_fill_frags(queue, skb, &tmpq))) { + __skb_queue_head(&tmpq, skb); goto err; + } if (rx->flags & XEN_NETRXF_csum_blank) skb->ip_summed = CHECKSUM_PARTIAL; diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c index 2aa9ded7d551..0b1f800c6cc8 100644 --- a/drivers/nvme/target/core.c +++ b/drivers/nvme/target/core.c @@ -1310,7 +1310,7 @@ static int nvmet_req_alloc_p2pmem_sgls(struct pci_dev *p2p_dev, return 0; out_free_sg: - pci_p2pmem_free_sgl(req->p2p_dev, req->sg); + pci_p2pmem_free_sgl(p2p_dev, req->sg); out_err: return -ENOMEM; } diff --git a/drivers/nvme/target/pr.c b/drivers/nvme/target/pr.c index ad07f55e7f17..acb44df15fc7 100644 --- a/drivers/nvme/target/pr.c +++ b/drivers/nvme/target/pr.c @@ -51,6 +51,8 @@ u16 nvmet_set_feat_resv_notif_mask(struct nvmet_req *req, u32 mask) if (nsid != U32_MAX) { status = nvmet_req_find_ns(req); + if (status == (NVME_SC_INVALID_NS | NVME_STATUS_DNR)) + return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR; if (status) return status; if (!req->ns->pr.enable) @@ -75,6 +77,8 @@ u16 nvmet_get_feat_resv_notif_mask(struct nvmet_req *req) u16 status; status = nvmet_req_find_ns(req); + if (status == (NVME_SC_INVALID_NS | NVME_STATUS_DNR)) + return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR; if (status) return status; diff --git a/drivers/pmdomain/imx/imx8m-blk-ctrl.c b/drivers/pmdomain/imx/imx8m-blk-ctrl.c index 99d100e1d923..62c4f829605c 100644 --- a/drivers/pmdomain/imx/imx8m-blk-ctrl.c +++ b/drivers/pmdomain/imx/imx8m-blk-ctrl.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -34,6 +35,12 @@ struct imx8m_blk_ctrl { struct regmap *regmap; struct imx8m_blk_ctrl_domain *domains; struct genpd_onecell_data onecell_data; + /* + * Serializes the blk-ctrl reset/clock sequence across sibling domains; + * their transitions interact through the shared VPUMIX bus domain, + * VPU_NOC and the not-ack-verified ADB400 handshake (ERR050531). + */ + struct mutex power_lock; }; struct imx8m_blk_ctrl_domain_data { @@ -98,6 +105,8 @@ static int imx8m_blk_ctrl_power_on(struct generic_pm_domain *genpd) struct imx8m_blk_ctrl *bc = domain->bc; int ret; + guard(mutex)(&bc->power_lock); + /* make sure bus domain is awake */ ret = pm_runtime_get_sync(bc->bus_power_dev); if (ret < 0) { @@ -164,6 +173,8 @@ static int imx8m_blk_ctrl_power_off(struct generic_pm_domain *genpd) const struct imx8m_blk_ctrl_domain_data *data = domain->data; struct imx8m_blk_ctrl *bc = domain->bc; + guard(mutex)(&bc->power_lock); + /* put devices into reset and disable clocks */ if (data->mipi_phy_rst_mask) regmap_clear_bits(bc->regmap, BLK_MIPI_RESET_DIV, data->mipi_phy_rst_mask); @@ -202,6 +213,10 @@ static int imx8m_blk_ctrl_probe(struct platform_device *pdev) bc->dev = dev; + ret = devm_mutex_init(dev, &bc->power_lock); + if (ret) + return ret; + bc_data = of_device_get_match_data(dev); base = devm_platform_ioremap_resource(pdev, 0); diff --git a/drivers/pmdomain/rockchip/pm-domains.c b/drivers/pmdomain/rockchip/pm-domains.c index 490bbb1d1d8e..b57581b367a9 100644 --- a/drivers/pmdomain/rockchip/pm-domains.c +++ b/drivers/pmdomain/rockchip/pm-domains.c @@ -745,7 +745,19 @@ static int rockchip_pd_attach_dev(struct generic_pm_domain *genpd, } i = 0; - while ((clk = of_clk_get(dev->of_node, i++)) && !IS_ERR(clk)) { + while (1) { + clk = of_clk_get(dev->of_node, i++); + if (IS_ERR(clk)) { + error = PTR_ERR(clk); + if (error == -ENOENT) + break; + + dev_err(dev, "failed to get clock %d: %d\n", i - 1, + error); + pm_clk_destroy(dev); + return error; + } + dev_dbg(dev, "adding clock '%pC' to list of PM clocks\n", clk); error = pm_clk_add_clk(dev, clk); if (error) { @@ -828,7 +840,7 @@ static int rockchip_pm_add_one_domain(struct rockchip_pmu *pmu, dev_err(pmu->dev, "%pOFn: failed to get clk at index %d: %d\n", node, i, error); - return error; + goto err_put_clocks; } } @@ -1004,7 +1016,9 @@ static int rockchip_pm_add_subdomain(struct rockchip_pmu *pmu, parent_domain->name, child_domain->name); } - rockchip_pm_add_subdomain(pmu, np); + error = rockchip_pm_add_subdomain(pmu, np); + if (error) + return error; } return 0; diff --git a/drivers/power/sequencing/pwrseq-pcie-m2.c b/drivers/power/sequencing/pwrseq-pcie-m2.c index ddfe6ca82494..c59c487c43e3 100644 --- a/drivers/power/sequencing/pwrseq-pcie-m2.c +++ b/drivers/power/sequencing/pwrseq-pcie-m2.c @@ -562,6 +562,7 @@ static int pwrseq_pcie_m2_probe(struct platform_device *pdev) mutex_destroy(&ctx->list_lock); err_free_regulators: regulator_bulk_free(ctx->num_vregs, ctx->regs); + kfree(ctx->regs); return ret; } @@ -575,6 +576,7 @@ static void pwrseq_pcie_m2_remove(struct platform_device *pdev) mutex_destroy(&ctx->list_lock); regulator_bulk_free(ctx->num_vregs, ctx->regs); + kfree(ctx->regs); } static const struct of_device_id pwrseq_pcie_m2_of_match[] = { diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index 3d26ec1f7b9e..a8cba2f64b98 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -1825,7 +1825,7 @@ ptp_ocp_watchdog(struct timer_list *t) static void ptp_ocp_estimate_pci_timing(struct ptp_ocp *bp) { - ktime_t start, end, delay = U64_MAX; + ktime_t start, end, delay = KTIME_MAX; u32 ctrl; int i; diff --git a/drivers/regulator/ltc3676.c b/drivers/regulator/ltc3676.c index 768cdce0d4ec..1ec218951829 100644 --- a/drivers/regulator/ltc3676.c +++ b/drivers/regulator/ltc3676.c @@ -232,7 +232,7 @@ static const struct regulator_desc ltc3676_regulators[LTC3676_NUM_REGULATORS] = LTC3676_FIXED_REG(LDO4, ldo4, LDOB, 2), }; -static bool ltc3676_readable_writeable_reg(struct device *dev, unsigned int reg) +static bool ltc3676_writeable_reg(struct device *dev, unsigned int reg) { switch (reg) { case LTC3676_BUCK1 ... LTC3676_IRQSTAT: @@ -243,6 +243,15 @@ static bool ltc3676_readable_writeable_reg(struct device *dev, unsigned int reg) return false; } +static bool ltc3676_readable_reg(struct device *dev, unsigned int reg) +{ + switch (reg) { + case LTC3676_BUCK1 ... LTC3676_IRQSTAT: + return true; + } + return false; +} + static bool ltc3676_volatile_reg(struct device *dev, unsigned int reg) { switch (reg) { @@ -255,8 +264,8 @@ static bool ltc3676_volatile_reg(struct device *dev, unsigned int reg) static const struct regmap_config ltc3676_regmap_config = { .reg_bits = 8, .val_bits = 8, - .writeable_reg = ltc3676_readable_writeable_reg, - .readable_reg = ltc3676_readable_writeable_reg, + .writeable_reg = ltc3676_writeable_reg, + .readable_reg = ltc3676_readable_reg, .volatile_reg = ltc3676_volatile_reg, .max_register = LTC3676_CLIRQ, .use_single_read = true, diff --git a/drivers/regulator/of_regulator.c b/drivers/regulator/of_regulator.c index c0cc6cc0afd8..785b7a11dfc6 100644 --- a/drivers/regulator/of_regulator.c +++ b/drivers/regulator/of_regulator.c @@ -935,15 +935,15 @@ static int is_supply_name(const char *name) int of_regulator_bulk_get_all(struct device *dev, struct device_node *np, struct regulator_bulk_data **consumers) { - int num_consumers = 0; + int num_consumers = 0, names_len = 0; struct regulator *tmp; struct regulator_bulk_data *_consumers = NULL; struct property *prop; + char *names; int i, n = 0, ret; - char name[64]; /* - * first pass: get numbers of xxx-supply + * first pass: get numbers of xxx-supply and the room their names take * second pass: fill consumers */ restart: @@ -953,16 +953,19 @@ int of_regulator_bulk_get_all(struct device *dev, struct device_node *np, continue; if (!_consumers) { num_consumers++; + names_len += i + 1; continue; } else { - memcpy(name, prop->name, i); - name[i] = '\0'; - tmp = regulator_get(dev, name); + memcpy(names, prop->name, i); + names[i] = '\0'; + tmp = regulator_get(dev, names); if (IS_ERR(tmp)) { ret = PTR_ERR(tmp); goto error; } + _consumers[n].supply = names; _consumers[n].consumer = tmp; + names += i + 1; n++; continue; } @@ -973,9 +976,16 @@ int of_regulator_bulk_get_all(struct device *dev, struct device_node *np, } if (num_consumers == 0) return 0; - _consumers = kmalloc_objs(struct regulator_bulk_data, num_consumers); + /* + * The supply names are kept in the same allocation as the array, so + * that they share its lifetime and the caller has nothing extra to + * free. + */ + _consumers = kzalloc(size_add(size_mul(num_consumers, sizeof(*_consumers)), + names_len), GFP_KERNEL); if (!_consumers) return -ENOMEM; + names = (char *)(_consumers + num_consumers); goto restart; error: diff --git a/drivers/regulator/tps6594-regulator.c b/drivers/regulator/tps6594-regulator.c index 31a5218d5510..503b89a2f5f8 100644 --- a/drivers/regulator/tps6594-regulator.c +++ b/drivers/regulator/tps6594-regulator.c @@ -241,21 +241,21 @@ static const struct regulator_desc tps65224_buck_regs[] = { TPS65224_MASK_BUCKS_VSET, TPS6594_REG_BUCKX_CTRL(1), TPS6594_BIT_BUCK_EN, 0, 0, tps65224_bucks_2_3_4_ranges, - 4, 0, 0, NULL, 0, 0), + ARRAY_SIZE(tps65224_bucks_2_3_4_ranges), 0, 0, NULL, 0, 0), TPS6594_REGULATOR("BUCK3", "buck3", TPS6594_BUCK_3, REGULATOR_VOLTAGE, tps6594_bucks_ops, TPS65224_MASK_BUCKS_VSET, TPS6594_REG_BUCKX_VOUT_1(2), TPS65224_MASK_BUCKS_VSET, TPS6594_REG_BUCKX_CTRL(2), TPS6594_BIT_BUCK_EN, 0, 0, tps65224_bucks_2_3_4_ranges, - 4, 0, 0, NULL, 0, 0), + ARRAY_SIZE(tps65224_bucks_2_3_4_ranges), 0, 0, NULL, 0, 0), TPS6594_REGULATOR("BUCK4", "buck4", TPS6594_BUCK_4, REGULATOR_VOLTAGE, tps6594_bucks_ops, TPS65224_MASK_BUCKS_VSET, TPS6594_REG_BUCKX_VOUT_1(3), TPS65224_MASK_BUCKS_VSET, TPS6594_REG_BUCKX_CTRL(3), TPS6594_BIT_BUCK_EN, 0, 0, tps65224_bucks_2_3_4_ranges, - 4, 0, 0, NULL, 0, 0), + ARRAY_SIZE(tps65224_bucks_2_3_4_ranges), 0, 0, NULL, 0, 0), }; static const struct tps6594_regulator_irq_type tps6594_buck1_irq_types[] = { diff --git a/drivers/scsi/ch.c b/drivers/scsi/ch.c index 4010fdbf813c..c36539c83ed1 100644 --- a/drivers/scsi/ch.c +++ b/drivers/scsi/ch.c @@ -112,7 +112,6 @@ typedef struct { int minor; char name[8]; struct scsi_device *device; - struct scsi_device **dt; /* ptrs to data transfer elements */ u_int firsts[CH_TYPES]; u_int counts[CH_TYPES]; u_int voltags; @@ -363,15 +362,10 @@ ch_readconfig(scsi_changer *ch) vendor_labels[i]); } - /* look up the devices of the data transfer elements */ - ch->dt = kzalloc_objs(*ch->dt, ch->counts[CHET_DT]); - - if (!ch->dt) { - kfree(buffer); - return -ENOMEM; - } - + /* report the devices of the data transfer elements */ for (elem = 0; elem < ch->counts[CHET_DT]; elem++) { + struct scsi_device *sdev; + id = -1; lun = 0; if (elem < CH_DT_MAX && -1 != dt_id[elem]) { @@ -387,10 +381,8 @@ ch_readconfig(scsi_changer *ch) VPRINTK(KERN_INFO, "dt 0x%x: ",elem+ch->firsts[CHET_DT]); if (data[6] & 0x80) { VPRINTK(KERN_CONT, "not this SCSI bus\n"); - ch->dt[elem] = NULL; } else if (0 == (data[6] & 0x30)) { VPRINTK(KERN_CONT, "ID/LUN unknown\n"); - ch->dt[elem] = NULL; } else { id = ch->device->id; lun = 0; @@ -400,18 +392,16 @@ ch_readconfig(scsi_changer *ch) } if (-1 != id) { VPRINTK(KERN_CONT, "ID %i, LUN %i, ",id,lun); - ch->dt[elem] = - scsi_device_lookup(ch->device->host, - ch->device->channel, - id,lun); - if (!ch->dt[elem]) { + sdev = scsi_device_lookup(ch->device->host, + ch->device->channel, + id, lun); + if (!sdev) { /* should not happen */ VPRINTK(KERN_CONT, "Huh? device not found!\n"); } else { VPRINTK(KERN_CONT, "name: %8.8s %16.16s %4.4s\n", - ch->dt[elem]->vendor, - ch->dt[elem]->model, - ch->dt[elem]->rev); + sdev->vendor, sdev->model, sdev->rev); + scsi_device_put(sdev); } } } @@ -575,7 +565,6 @@ static void ch_destroy(struct kref *ref) scsi_changer *ch = container_of(ref, scsi_changer, ref); ch->device = NULL; - kfree(ch->dt); kfree(ch); } diff --git a/drivers/scsi/qedi/qedi_main.c b/drivers/scsi/qedi/qedi_main.c index 227ff7bd1bdc..f394fcbb97e7 100644 --- a/drivers/scsi/qedi/qedi_main.c +++ b/drivers/scsi/qedi/qedi_main.c @@ -2652,8 +2652,6 @@ static int __qedi_probe(struct pci_dev *pdev, int mode) snprintf(host_buf, sizeof(host_buf), "host_%d", qedi->shost->host_no); qedi_ops->common->set_name(qedi->cdev, host_buf); - qedi_ops->register_ops(qedi->cdev, &qedi_cb_ops, qedi); - memset(¶ms, 0, sizeof(params)); params.mtu = DEF_PATH_MTU + IPV6_HDR_LEN + TCP_HDR_LEN; qedi->ll2_mtu = DEF_PATH_MTU; @@ -2670,6 +2668,7 @@ static int __qedi_probe(struct pci_dev *pdev, int mode) spin_lock_init(&qedi->task_idx_lock); mutex_init(&qedi->stats_lock); } + qedi_ops->register_ops(qedi->cdev, &qedi_cb_ops, qedi); qedi_ops->ll2->register_cb_ops(qedi->cdev, &qedi_ll2_cb_ops, qedi); qedi_ops->ll2->start(qedi->cdev, ¶ms); diff --git a/drivers/spi/spi-nxp-fspi.c b/drivers/spi/spi-nxp-fspi.c index 4425132432ec..e0c88e9f8183 100644 --- a/drivers/spi/spi-nxp-fspi.c +++ b/drivers/spi/spi-nxp-fspi.c @@ -898,10 +898,10 @@ static int nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, dev_dbg(f->dev, "Target device [CS:%x] selected\n", spi_get_chipselect(spi, 0)); /* - * Per the FlexSPI reference manual (initialization sequence), MCR0 and - * the DLL control registers should be configured while the module is in - * stop mode (MCR0[MDIS] = 1). Enter stop mode before reconfiguring the - * RX sample clock source and the DLL, then exit stop mode afterwards. + * Per the FlexSPI reference manual (initialization sequence), MCR0 + * should be configured while the module is in stop mode (MCR0[MDIS] = 1). + * Enter stop mode before reconfiguring the RX sample clock source, then + * exit stop mode afterwards. */ reg = fspi_readl(f, f->iobase + FSPI_MCR0); fspi_writel(f, reg | FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0); @@ -936,6 +936,16 @@ static int nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, if (ret) return ret; + /* + * Exit stop mode before running the DLL calibration. The DLL + * reference clock is derived from the module clock domain, which is + * gated while MCR0[MDIS] = 1. Calibrating the DLL in stop mode leaves + * the STS2 lock bits stuck at 0 (the DLL never runs), so clear MDIS + * first and let the module and reference clock run. + */ + reg = fspi_readl(f, f->iobase + FSPI_MCR0); + fspi_writel(f, reg & ~FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0); + /* * If clock rate > 100MHz, then switch from DLL override mode to * DLL calibration mode. @@ -945,10 +955,6 @@ static int nxp_fspi_select_mem(struct nxp_fspi *f, struct spi_device *spi, else nxp_fspi_dll_override(f); - /* Exit stop mode now that MCR0 and the DLL have been reconfigured. */ - reg = fspi_readl(f, f->iobase + FSPI_MCR0); - fspi_writel(f, reg & ~FSPI_MCR0_MDIS, f->iobase + FSPI_MCR0); - f->pre_op_rate = op->max_freq; f->selected = spi_get_chipselect(spi, 0); diff --git a/drivers/spi/spi-sg2044-nor.c b/drivers/spi/spi-sg2044-nor.c index f3bcb8a1b92b..bc9a8ee9bce4 100644 --- a/drivers/spi/spi-sg2044-nor.c +++ b/drivers/spi/spi-sg2044-nor.c @@ -307,15 +307,15 @@ static ssize_t sg2044_spifmc_tran_cmd(struct sg2044_spifmc *spifmc, return 0; } -static void sg2044_spifmc_trans(struct sg2044_spifmc *spifmc, - const struct spi_mem_op *op) +static int sg2044_spifmc_trans(struct sg2044_spifmc *spifmc, + const struct spi_mem_op *op) { if (op->data.dir == SPI_MEM_DATA_IN) - sg2044_spifmc_read(spifmc, op); + return sg2044_spifmc_read(spifmc, op); else if (op->data.dir == SPI_MEM_DATA_OUT) - sg2044_spifmc_write(spifmc, op); + return sg2044_spifmc_write(spifmc, op); else - sg2044_spifmc_tran_cmd(spifmc, op); + return sg2044_spifmc_tran_cmd(spifmc, op); } static ssize_t sg2044_spifmc_trans_reg(struct sg2044_spifmc *spifmc, @@ -385,29 +385,64 @@ static ssize_t sg2044_spifmc_trans_reg(struct sg2044_spifmc *spifmc, return 0; } +static int sg2044_spifmc_set_speed(struct sg2044_spifmc *spifmc, + unsigned int speed_hz) +{ + unsigned long rate = clk_get_rate(spifmc->clk); + u64 div; + u32 reg; + + if (!rate) + return -EINVAL; + + /* SCK = HCLK / (2 * (SckDiv + 1)); round down the resulting SCK. */ + div = speed_hz ? DIV_ROUND_UP_ULL(rate, 2ULL * speed_hz) : 1; + if (div > SPIFMC_CTRL_SCK_DIV_MASK + 1) + return -EINVAL; + + reg = readl(spifmc->io_base + SPIFMC_CTRL); + reg &= ~SPIFMC_CTRL_SCK_DIV_MASK; + reg |= div - 1; + writel(reg, spifmc->io_base + SPIFMC_CTRL); + + return 0; +} + static int sg2044_spifmc_exec_op(struct spi_mem *mem, const struct spi_mem_op *op) { struct sg2044_spifmc *spifmc; + int ret; spifmc = spi_controller_get_devdata(mem->spi->controller); mutex_lock(&spifmc->lock); + ret = sg2044_spifmc_set_speed(spifmc, op->max_freq ? + min(op->max_freq, mem->spi->max_speed_hz) : + mem->spi->max_speed_hz); + if (ret) + goto out_unlock; + if (op->addr.nbytes == 0) - sg2044_spifmc_trans_reg(spifmc, op); + ret = sg2044_spifmc_trans_reg(spifmc, op); else - sg2044_spifmc_trans(spifmc, op); + ret = sg2044_spifmc_trans(spifmc, op); +out_unlock: mutex_unlock(&spifmc->lock); - return 0; + return ret; } static const struct spi_controller_mem_ops sg2044_spifmc_mem_ops = { .exec_op = sg2044_spifmc_exec_op, }; +static const struct spi_controller_mem_caps sg2044_spifmc_mem_caps = { + .per_op_freq = true, +}; + static void sg2044_spifmc_init(struct sg2044_spifmc *spifmc) { u32 tran_csr; @@ -447,6 +482,10 @@ static int sg2044_spifmc_probe(struct platform_device *pdev) if (IS_ERR(spifmc->clk)) return dev_err_probe(dev, PTR_ERR(spifmc->clk), "Cannot get and enable AHB clock\n"); + ctrl->max_speed_hz = clk_get_rate(spifmc->clk) / 2; + if (!ctrl->max_speed_hz) + return dev_err_probe(dev, -EINVAL, "Invalid AHB clock rate\n"); + spifmc->dev = &pdev->dev; spifmc->ctrl = ctrl; @@ -458,6 +497,7 @@ static int sg2044_spifmc_probe(struct platform_device *pdev) ctrl->bits_per_word_mask = SPI_BPW_MASK(8); ctrl->auto_runtime_pm = false; ctrl->mem_ops = &sg2044_spifmc_mem_ops; + ctrl->mem_caps = &sg2044_spifmc_mem_caps; ctrl->mode_bits = SPI_RX_DUAL | SPI_TX_DUAL | SPI_RX_QUAD | SPI_TX_QUAD; ret = devm_mutex_init(dev, &spifmc->lock); diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c index 633715b98625..d027fe7ce86f 100644 --- a/drivers/tee/optee/ffa_abi.c +++ b/drivers/tee/optee/ffa_abi.c @@ -198,7 +198,8 @@ static int to_msg_param_ffa_mem(struct optee_msg_param *mp, if (shm) { u64 shm_offs = p->u.memref.shm_offs; - mp->u.fmem.internal_offs = shm->offset; + mp->u.fmem.internal_offs = tee_shm_get_page_offset(shm) & + (FFA_PAGE_SIZE - 1); mp->u.fmem.offs_low = shm_offs; mp->u.fmem.offs_high = shm_offs >> 32; @@ -284,14 +285,26 @@ static int optee_ffa_shm_register(struct tee_context *ctx, struct tee_shm *shm, .nattrs = 1, }; struct sg_table sgt; + size_t ffa_offs; int rc; + if (!num_pages) + return -EINVAL; + rc = optee_check_mem_type(start, num_pages); if (rc) return rc; - rc = sg_alloc_table_from_pages(&sgt, pages, num_pages, 0, - num_pages * PAGE_SIZE, GFP_KERNEL); + /* + * Start the FF-A descriptor at the 4 KiB page containing the shared + * buffer, skipping unused leading 4 KiB pages when PAGE_SIZE is + * larger. Same approach as optee_fill_pages_list() in the SMC ABI. + * This leaves only the offset into that 4 KiB page for internal_offs. + */ + ffa_offs = round_down(tee_shm_get_page_offset(shm), FFA_PAGE_SIZE); + rc = sg_alloc_table_from_pages(&sgt, pages, num_pages, ffa_offs, + num_pages * PAGE_SIZE - ffa_offs, + GFP_KERNEL); if (rc) return rc; args.sg = sgt.sgl; @@ -458,7 +471,8 @@ static void handle_ffa_rpc_func_cmd_shm_alloc(struct tee_context *ctx, .attr = OPTEE_MSG_ATTR_TYPE_FMEM_OUTPUT, .u.fmem.size = tee_shm_get_size(shm), .u.fmem.global_id = shm->sec_world_id, - .u.fmem.internal_offs = shm->offset, + .u.fmem.internal_offs = tee_shm_get_page_offset(shm) & + (FFA_PAGE_SIZE - 1), }; arg->ret = TEEC_SUCCESS; @@ -1123,14 +1137,6 @@ static int optee_ffa_probe(struct ffa_device *ffa_dev) optee_set_dev_group(optee); - rc = tee_device_register(optee->teedev); - if (rc) - goto err_unreg_supp_teedev; - - rc = tee_device_register(optee->supp_teedev); - if (rc) - goto err_unreg_supp_teedev; - rc = rhashtable_init(&optee->ffa.global_ids, &shm_rhash_params); if (rc) goto err_unreg_supp_teedev; @@ -1159,6 +1165,14 @@ static int optee_ffa_probe(struct ffa_device *ffa_dev) if (optee_ffa_protmem_pool_init(optee, sec_caps)) pr_info("Protected memory service not available\n"); + rc = tee_device_register(optee->teedev); + if (rc) + goto err_unregister_devices; + + rc = tee_device_register(optee->supp_teedev); + if (rc) + goto err_unregister_devices; + rc = optee_enumerate_devices(PTA_CMD_GET_DEVICES); if (rc) goto err_unregister_devices; diff --git a/drivers/tee/optee/optee_msg.h b/drivers/tee/optee/optee_msg.h index 7d9b12e71c03..6c3043f8da33 100644 --- a/drivers/tee/optee/optee_msg.h +++ b/drivers/tee/optee/optee_msg.h @@ -136,8 +136,8 @@ struct optee_msg_param_rmem { * struct optee_msg_param_fmem - FF-A memory reference parameter * @offs_low: lower bits of offset into shared memory reference * @offs_high: higher bits of offset into shared memory reference - * @internal_offs: internal offset into the first page of shared memory - * reference + * @internal_offs: offset into the first 4 KiB page of the FF-A shared + * memory region * @size: size of the buffer * @global_id: global identifier of the shared memory */ diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index b8a2bdac3208..51624443359b 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -1849,14 +1849,6 @@ static int optee_probe(struct platform_device *pdev) optee_set_dev_group(optee); - rc = tee_device_register(optee->teedev); - if (rc) - goto err_unreg_supp_teedev; - - rc = tee_device_register(optee->supp_teedev); - if (rc) - goto err_unreg_supp_teedev; - optee_cq_init(&optee->call_queue, thread_count); optee_supp_init(&optee->supp); optee->smc.memremaped_shm = memremaped_shm; @@ -1916,6 +1908,14 @@ static int optee_probe(struct platform_device *pdev) if (optee->smc.sec_caps & OPTEE_SMC_SEC_CAP_DYNAMIC_SHM) pr_info("dynamic shared memory is enabled\n"); + rc = tee_device_register(optee->teedev); + if (rc) + goto err_disable_shm_cache; + + rc = tee_device_register(optee->supp_teedev); + if (rc) + goto err_disable_shm_cache; + rc = optee_enumerate_devices(PTA_CMD_GET_DEVICES); if (rc) goto err_disable_shm_cache; @@ -1942,7 +1942,6 @@ static int optee_probe(struct platform_device *pdev) optee_shm_arg_cache_uninit(optee); optee_supp_uninit(&optee->supp); mutex_destroy(&optee->call_queue.mutex); -err_unreg_supp_teedev: tee_device_unregister(optee->supp_teedev); err_unreg_teedev: tee_device_unregister(optee->teedev); diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c index 6742b3579c86..daa4af1e089e 100644 --- a/drivers/tee/tee_shm.c +++ b/drivers/tee/tee_shm.c @@ -343,6 +343,10 @@ int tee_dyn_shm_alloc_helper(struct tee_shm *shm, size_t size, size_t align, unsigned int i; int rc = 0; + /* get_order(0) is undefined and exceeds MAX_PAGE_ORDER. */ + if (!nr_pages) + return -EINVAL; + /* * Ignore alignment since this is already going to be page aligned * and there's no need for any larger alignment. diff --git a/drivers/ufs/core/ufs-mcq.c b/drivers/ufs/core/ufs-mcq.c index 13b60a2d06db..967b51ca57a0 100644 --- a/drivers/ufs/core/ufs-mcq.c +++ b/drivers/ufs/core/ufs-mcq.c @@ -306,20 +306,24 @@ static int ufshcd_mcq_get_tag(struct ufs_hba *hba, struct cq_entry *cqe) return div_u64(addr, ufshcd_get_ucd_size(hba)); } +static void ufshcd_mcq_compl_cqe(struct ufs_hba *hba, struct cq_entry *cqe) +{ + int tag = ufshcd_mcq_get_tag(hba, cqe); + + ufshcd_compl_one_cqe(hba, tag, cqe); + /* After processing the CQE, mark it as an empty (invalid) entry. */ + cqe->command_desc_base_addr = 0; +} + static void ufshcd_mcq_process_cqe(struct ufs_hba *hba, struct ufs_hw_queue *hwq) { struct cq_entry *cqe = ufshcd_mcq_cur_cqe(hwq); - if (cqe->command_desc_base_addr) { - int tag = ufshcd_mcq_get_tag(hba, cqe); - - ufshcd_compl_one_cqe(hba, tag, cqe); - /* After processed the cqe, mark it empty (invalid) entry */ - cqe->command_desc_base_addr = 0; - } else { + if (cqe->command_desc_base_addr) + ufshcd_mcq_compl_cqe(hba, cqe); + else dev_err(hba->dev, "Abnormal CQ entry!\n"); - } } /* @@ -327,7 +331,7 @@ static void ufshcd_mcq_process_cqe(struct ufs_hba *hba, * controller disabled (HCE = 0). Reading host controller registers, e.g. the * CQ tail pointer (CQTPy), may not be safe with the host controller disabled. * Hence, iterate over all completion queue entries. This won't result in - * double completions because ufshcd_mcq_process_cqe() clears a CQE after it + * double completions because ufshcd_mcq_compl_cqe() clears a CQE after it * has been processed. */ void ufshcd_mcq_compl_all_cqes_lock(struct ufs_hba *hba, @@ -338,13 +342,13 @@ void ufshcd_mcq_compl_all_cqes_lock(struct ufs_hba *hba, spin_lock_irqsave(&hwq->cq_lock, flags); while (entries > 0) { - ufshcd_mcq_process_cqe(hba, hwq); + struct cq_entry *cqe = ufshcd_mcq_cur_cqe(hwq); + + if (cqe->command_desc_base_addr) + ufshcd_mcq_compl_cqe(hba, cqe); ufshcd_mcq_inc_cq_head_slot(hwq); entries--; } - - ufshcd_mcq_update_cq_tail_slot(hwq); - hwq->cq_head_slot = hwq->cq_tail_slot; spin_unlock_irqrestore(&hwq->cq_lock, flags); } diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index bc33da89af26..0a9366f21483 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5975,8 +5975,6 @@ static bool ufshcd_mcq_force_compl_one(struct request *rq, void *priv) if (blk_mq_is_reserved_rq(rq) || !hwq) return true; - ufshcd_mcq_compl_all_cqes_lock(hba, hwq); - /* * For those cmds of which the cqes are not present in the cq, complete * them explicitly. @@ -5992,19 +5990,6 @@ static bool ufshcd_mcq_force_compl_one(struct request *rq, void *priv) return true; } -static bool ufshcd_mcq_compl_one(struct request *rq, void *priv) -{ - struct scsi_device *sdev = rq->q->queuedata; - struct Scsi_Host *shost = sdev->host; - struct ufs_hba *hba = shost_priv(shost); - struct ufs_hw_queue *hwq = ufshcd_mcq_req_to_hwq(hba, rq); - - if (!blk_mq_is_reserved_rq(rq) && hwq) - ufshcd_mcq_poll_cqe_lock(hba, hwq); - - return true; -} - /** * ufshcd_mcq_compl_pending_transfer - MCQ mode function. It is * invoked from the error handler context or ufshcd_host_reset_and_restore() @@ -6019,10 +6004,18 @@ static bool ufshcd_mcq_compl_one(struct request *rq, void *priv) static void ufshcd_mcq_compl_pending_transfer(struct ufs_hba *hba, bool force_compl) { - blk_mq_tagset_busy_iter(&hba->host->tag_set, - force_compl ? ufshcd_mcq_force_compl_one : - ufshcd_mcq_compl_one, - NULL); + int i; + + for (i = 0; i < hba->nr_hw_queues; i++) { + if (force_compl) + ufshcd_mcq_compl_all_cqes_lock(hba, &hba->uhq[i]); + else + ufshcd_mcq_poll_cqe_lock(hba, &hba->uhq[i]); + } + + if (force_compl) + blk_mq_tagset_busy_iter(&hba->host->tag_set, + ufshcd_mcq_force_compl_one, NULL); } /** @@ -9491,6 +9484,7 @@ static void ufshcd_async_scan(void *data, async_cookie_t cookie) ret = ufshcd_add_lus(hba); out: + ufshcd_release(hba); pm_runtime_put_sync(hba->dev); if (ret) @@ -11213,6 +11207,9 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) ufshcd_init_clk_gating(hba); + /* Released by ufshcd_async_scan(), or by out_release on failure. */ + ufshcd_hold(hba); + ufshcd_init_clk_scaling(hba); /* @@ -11233,7 +11230,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) err = devm_request_irq(dev, irq, ufshcd_intr, IRQF_SHARED, UFSHCD, hba); if (err) { dev_err(hba->dev, "request irq failed\n"); - goto out_disable; + goto out_release; } else { hba->is_irq_enabled = true; } @@ -11249,7 +11246,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) dev_err(hba->dev, "Host controller enable failed\n"); ufshcd_print_evt_hist(hba); ufshcd_print_host_state(hba); - goto out_disable; + goto out_release; } INIT_DELAYED_WORK(&hba->rpm_dev_flush_recheck_work, ufshcd_rpm_dev_flush_recheck_work); @@ -11263,7 +11260,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) err = ufshcd_add_scsi_host(hba); if (err) - goto out_disable; + goto out_release; /* Hold auto suspend until async scan completes */ pm_runtime_get_sync(dev); @@ -11283,7 +11280,7 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) err = ufshcd_link_startup(hba); if (err) - goto out_disable; + goto out_release; if (hba->mcq_enabled) ufshcd_config_mcq(hba); @@ -11300,23 +11297,23 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) /* Verify device initialization by sending NOP OUT UPIU */ err = ufshcd_verify_dev_init(hba); if (err) - goto out_disable; + goto out_release; /* Initiate UFS initialization, and waiting until completion */ err = ufshcd_complete_dev_init(hba); if (err) - goto out_disable; + goto out_release; err = ufshcd_device_params_init(hba); if (err) - goto out_disable; + goto out_release; err = ufshcd_post_device_init(hba); initialized: ufshcd_process_probe_result(hba, probe_start, err); if (err) - goto out_disable; + goto out_release; ufs_sysfs_add_nodes(hba->dev); hba->dme_qos_sysfs_handle = sysfs_get_dirent(hba->dev->kobj.sd, @@ -11327,6 +11324,8 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq) ufshcd_pm_qos_init(hba); return 0; +out_release: + ufshcd_release(hba); out_disable: hba->is_irq_enabled = false; ufshcd_hba_exit(hba); diff --git a/drivers/ufs/host/ufs-mediatek.c b/drivers/ufs/host/ufs-mediatek.c index 3991a51263a6..a2defbe3b748 100644 --- a/drivers/ufs/host/ufs-mediatek.c +++ b/drivers/ufs/host/ufs-mediatek.c @@ -509,6 +509,7 @@ static int ufs_mtk_mphy_power_on(struct ufs_hba *hba, bool on) struct ufs_mtk_host *host = ufshcd_get_variant(hba); struct phy *mphy = host->mphy; struct arm_smccc_res res; + int err; int ret = 0; if (!mphy || !(on ^ host->mphy_powered_on)) @@ -523,7 +524,18 @@ static int ufs_mtk_mphy_power_on(struct ufs_hba *hba, bool on) usleep_range(200, 210); ufs_mtk_va09_pwr_ctrl(res, 1); } - phy_power_on(mphy); + ret = phy_power_on(mphy); + if (ret) { + if (ufs_mtk_is_va09_supported(hba)) { + ufs_mtk_va09_pwr_ctrl(res, 0); + err = regulator_disable(host->reg_va09); + if (err) + dev_err(hba->dev, + "failed to unwind va09: %d\n", err); + } + dev_err(hba->dev, "failed to enable mphy: %d\n", ret); + return ret; + } } else { phy_power_off(mphy); if (ufs_mtk_is_va09_supported(hba)) { @@ -1295,7 +1307,9 @@ static int ufs_mtk_init(struct ufs_hba *hba) * * Enable phy clocks specifically here. */ - ufs_mtk_mphy_power_on(hba, true); + err = ufs_mtk_mphy_power_on(hba, true); + if (err) + goto out_variant_clear; if (ufs_mtk_is_rtff_mtcmos(hba)) { /* First Restore here, to avoid backup unexpected value */ diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 7fe0a63d690a..579b82e27b5a 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -9622,7 +9622,6 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode, struct completion sync_reads; unsigned long i = 0; struct btrfs_bio *bbio; - int ret; /* * Fast path for synchronous reads which completes in this call, io_uring @@ -9669,10 +9668,10 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode, if (uring_ctx) { if (refcount_dec_and_test(&priv->pending_refs)) { - ret = blk_status_to_errno(READ_ONCE(priv->status)); - btrfs_uring_read_extent_endio(uring_ctx, ret); + int error = blk_status_to_errno(READ_ONCE(priv->status)); + + btrfs_uring_read_extent_endio(uring_ctx, error); kfree(priv); - return ret; } return -EIOCBQUEUED; diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index a537036303a4..833052306964 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4559,7 +4559,7 @@ static void btrfs_uring_read_finished(struct io_tw_req tw_req, io_tw_token_t tw) size_t page_offset; ssize_t ret; - /* The inode lock has already been acquired in btrfs_uring_read_extent. */ + /* The inode lock has already been acquired in btrfs_encoded_read(). */ btrfs_lockdep_inode_acquire(inode, i_rwsem); if (priv->err) { @@ -4625,7 +4625,6 @@ static int btrfs_uring_read_extent(struct kiocb *iocb, struct iov_iter *iter, struct iovec *iov, struct io_uring_cmd *cmd) { struct btrfs_inode *inode = BTRFS_I(file_inode(iocb->ki_filp)); - struct extent_io_tree *io_tree = &inode->io_tree; struct page **pages = NULL; struct btrfs_uring_priv *priv = NULL; unsigned long nr_pages; @@ -4681,8 +4680,6 @@ static int btrfs_uring_read_extent(struct kiocb *iocb, struct iov_iter *iter, return -EIOCBQUEUED; out_fail: - btrfs_unlock_extent(io_tree, start, lockend, &cached_state); - btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); kfree(priv); for (int i = 0; i < nr_pages; i++) { if (pages[i]) @@ -4795,7 +4792,7 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issue ret = btrfs_encoded_read(&kiocb, &data->iter, &data->args, &cached_state, &disk_bytenr, &disk_io_size); if (ret == -EAGAIN) - goto out_acct; + goto out_free; if (ret < 0 && ret != -EIOCBQUEUED) goto out_free; @@ -4823,8 +4820,10 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issue cached_state, disk_bytenr, disk_io_size, count, data->args.compression, data->iov, cmd); - - goto out_acct; + if (ret == -EIOCBQUEUED) + goto out_acct; + btrfs_unlock_extent(io_tree, start, lockend, &cached_state); + btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); } out_free: @@ -4835,8 +4834,10 @@ static int btrfs_uring_encoded_read(struct io_uring_cmd *cmd, unsigned int issue add_rchar(current, ret); inc_syscr(current); - if (ret != -EIOCBQUEUED && ret != -EAGAIN) + if (ret != -EIOCBQUEUED) { kfree(data); + bc->data = NULL; + } return ret; } @@ -4865,6 +4866,11 @@ static int btrfs_uring_encoded_write(struct io_uring_cmd *cmd, unsigned int issu goto out_acct; } + if (issue_flags & IO_URING_F_NONBLOCK) { + ret = -EAGAIN; + goto out_acct; + } + if (!data) { data = kzalloc_obj(*data, GFP_NOFS); if (!data) { @@ -4933,11 +4939,6 @@ static int btrfs_uring_encoded_write(struct io_uring_cmd *cmd, unsigned int issu } } - if (issue_flags & IO_URING_F_NONBLOCK) { - ret = -EAGAIN; - goto out_acct; - } - pos = data->args.offset; ret = rw_verify_area(WRITE, file, &pos, data->args.len); if (ret < 0) @@ -4963,8 +4964,8 @@ static int btrfs_uring_encoded_write(struct io_uring_cmd *cmd, unsigned int issu add_wchar(current, ret); inc_syscw(current); - if (ret != -EAGAIN) - kfree(data); + kfree(data); + bc->data = NULL; return ret; } diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 3b3f04521369..9db8f8b6d3d5 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -550,8 +550,10 @@ static int scrub_print_warning_inode(u64 inum, u64 offset, u64 num_bytes, } ret = paths_from_inode(inum, ipath); - if (ret < 0) + if (ret < 0) { + btrfs_put_root(local_root); goto err; + } /* * we deliberately ignore the bit ipath might have been too small to diff --git a/fs/btrfs/xattr.c b/fs/btrfs/xattr.c index ab55d10bd71f..cb001a211f55 100644 --- a/fs/btrfs/xattr.c +++ b/fs/btrfs/xattr.c @@ -161,6 +161,7 @@ int btrfs_setxattr(struct btrfs_trans_handle *trans, struct inode *inode, const u16 old_data_len = btrfs_dir_data_len(leaf, di); const u32 item_size = btrfs_item_size(leaf, slot); const u32 data_size = sizeof(*di) + name_len + size; + unsigned long name_ptr; unsigned long data_ptr; char *ptr; @@ -189,8 +190,16 @@ int btrfs_setxattr(struct btrfs_trans_handle *trans, struct inode *inode, ptr = btrfs_item_ptr(leaf, slot, char); ptr += btrfs_item_size(leaf, slot) - data_size; di = (struct btrfs_dir_item *)ptr; + memzero_extent_buffer(leaf, (unsigned long)ptr + + offsetof(struct btrfs_dir_item, location), + sizeof(struct btrfs_disk_key)); + btrfs_set_dir_flags(leaf, di, BTRFS_FT_XATTR); + btrfs_set_dir_transid(leaf, di, trans->transid); + btrfs_set_dir_name_len(leaf, di, name_len); btrfs_set_dir_data_len(leaf, di, size); + name_ptr = (unsigned long)(di + 1); data_ptr = ((unsigned long)(di + 1)) + name_len; + write_extent_buffer(leaf, name, name_ptr, name_len); write_extent_buffer(leaf, value, data_ptr, size); } else { /* @@ -320,10 +329,8 @@ ssize_t btrfs_listxattr(struct dentry *dentry, char *buffer, size_t size) if (!size) goto next; - if (!buffer || (name_len + 1) > size_left) { - iter_ret = -ERANGE; - break; - } + if (!buffer || (name_len + 1) > size_left) + return -ERANGE; read_extent_buffer(leaf, buffer, name_ptr, name_len); buffer[name_len] = '\0'; diff --git a/fs/dcache.c b/fs/dcache.c index 073c2ce2d444..d6fbb4fa7dbc 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1583,6 +1583,8 @@ EXPORT_SYMBOL(path_has_submounts); * * Only one of d_invalidate() and d_set_mounted() must succeed. For * this reason take rename_lock and d_lock on dentry and ancestors. + * Likewise for dont_mount() which marks a dentry that is being removed + * under d_lock. */ int d_set_mounted(struct dentry *dentry) { @@ -1599,7 +1601,7 @@ int d_set_mounted(struct dentry *dentry) spin_unlock(&p->d_lock); } spin_lock(&dentry->d_lock); - if (!d_unlinked(dentry)) { + if (!d_unlinked(dentry) && !cant_mount(dentry)) { ret = -EBUSY; if (!d_mountpoint(dentry)) { dentry->d_flags |= DCACHE_MOUNTED; diff --git a/fs/fcntl.c b/fs/fcntl.c index c158f082f1da..9b71bc9d71d1 100644 --- a/fs/fcntl.c +++ b/fs/fcntl.c @@ -372,6 +372,9 @@ static long fcntl_set_rw_hint(struct file *file, unsigned long arg) u64 __user *argp = (u64 __user *)arg; u64 hint; + /* nothing is ever written to it */ + if (IS_IMMUTABLE(inode)) + return -EPERM; if (!inode_owner_or_capable(file_mnt_idmap(file), inode)) return -EPERM; diff --git a/fs/fs_context.c b/fs/fs_context.c index 23ad66cd94e1..a50584df97be 100644 --- a/fs/fs_context.c +++ b/fs/fs_context.c @@ -315,6 +315,10 @@ struct fs_context *fs_context_for_reconfigure(struct dentry *dentry, unsigned int sb_flags, unsigned int sb_flags_mask) { + /* kernel-internal superblocks are nobody's to reconfigure */ + if (dentry->d_sb->s_flags & SB_NOUSER) + return ERR_PTR(-EINVAL); + return alloc_fs_context(dentry->d_sb->s_type, dentry, sb_flags, sb_flags_mask, FS_CONTEXT_FOR_RECONFIGURE); } diff --git a/fs/inode.c b/fs/inode.c index c6b08e8c5f62..efa801f3eb11 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -763,21 +763,18 @@ void clear_inode(struct inode *inode) fsverity_cleanup_inode(inode); /* - * We have to cycle the i_pages lock here because reclaim can be in the - * process of removing the last page (in __filemap_remove_folio()) - * and we must not free the mapping under it. + * We have to cycle the i_pages lock here because reclaim + * can be in the process of removing the last page (in + * __filemap_remove_folio()) and we must not free the mapping + * under it. We also remove nodes which are empty; these + * can occur in two different ways. The first is that radix + * tree expansion can fail partway and the second is that THP + * collapse_file() can allocate some temporary nodes and not + * clean them up. */ - xa_lock_irq(&inode->i_data.i_pages); + xa_destroy(&inode->i_data.i_pages); + BUG_ON(inode->i_data.nrpages); - /* - * Almost always, mapping_empty(&inode->i_data) here; but there are - * two known and long-standing ways in which nodes may get left behind - * (when deep radix-tree node allocation failed partway; or when THP - * collapse_file() failed). Until those two known cases are cleaned up, - * or a cleanup function is called here, do not BUG_ON(!mapping_empty), - * nor even WARN_ON(!mapping_empty). - */ - xa_unlock_irq(&inode->i_data.i_pages); BUG_ON(!(inode_state_read_once(inode) & I_FREEING)); BUG_ON(inode_state_read_once(inode) & I_CLEAR); BUG_ON(!list_empty(&inode->i_wb_list)); diff --git a/fs/mount.h b/fs/mount.h index 94fcc306d21e..2223fb141499 100644 --- a/fs/mount.h +++ b/fs/mount.h @@ -33,7 +33,8 @@ struct mnt_namespace { } __randomize_layout; struct mnt_pcp { - int mnt_count; + unsigned int mnt_gets; + unsigned int mnt_puts; int mnt_writers; }; @@ -233,6 +234,9 @@ static inline struct mnt_namespace *to_mnt_ns(struct ns_common *ns) #ifdef CONFIG_FSNOTIFY static inline void mnt_notify_add(struct mount *m) { + /* queued already under this namespace_sem hold */ + if (!list_empty(&m->to_notify)) + return; /* Optimize the case where there are no watches */ if ((m->mnt_ns && m->mnt_ns->n_fsnotify_marks) || (m->prev_ns && m->prev_ns->n_fsnotify_marks)) diff --git a/fs/namei.c b/fs/namei.c index 19ce43c9a6e6..27e0637bfdda 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -2086,6 +2086,33 @@ static noinline const char *pick_link(struct nameidata *nd, struct path *link, return NULL; } +/* + * Be careful in case the dentry is unlinked or renamed. Any mounts + * stacked on top of it are going away. We need to make sure that we + * don't reveal the underyling dentry during refwalk. In rcuwalk we + * catch this via d_seq and another lookup for the name. Give the same + * guarantee in refwalk. + */ +static bool unlink_may_reveal(struct nameidata *nd, int flags, + struct path *path, struct dentry *dentry) +{ + /* ".." and LOOKUP_DOWN may land on an unhashed directory */ + if (flags & WALK_NOFOLLOW) + return false; + if (nd->flags & LOOKUP_REVAL) + return false; + /* We crossed onto a mount and the name led us here while it still existed */ + if (path->mnt != nd->path.mnt) + return false; + /* only a name on its way out is flagged */ + if (likely(!cant_mount(dentry))) + return false; + if (!d_unlinked(dentry)) + return false; + dput(no_free_ptr(path->dentry)); + return true; +} + /* * Do we need to follow links? We _really_ want to be able * to do this check without having to look at inode->i_op, @@ -2115,6 +2142,8 @@ static noinline const char *step_into_slowpath(struct nameidata *nd, int flags, if (unlikely(!inode)) return ERR_PTR(-ENOENT); } else { + if (unlikely(unlink_may_reveal(nd, flags, &path, dentry))) + return ERR_PTR(-ESTALE); dput(nd->path.dentry); if (nd->path.mnt != path.mnt) mntput(nd->path.mnt); diff --git a/fs/namespace.c b/fs/namespace.c index 3d5cd5bf3b05..ec93bdac9e78 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -130,7 +130,6 @@ static void mnt_ns_release(struct mnt_namespace *ns) { /* keep alive for {list,stat}mount() */ if (ns && refcount_dec_and_test(&ns->passive)) { - fsnotify_mntns_delete(ns); put_user_ns(ns->user_ns); kfree(ns); } @@ -249,16 +248,24 @@ void mnt_release_group_id(struct mount *mnt) mnt->mnt_group_id = 0; } -/* - * vfsmount lock must be held for read - */ -static inline void mnt_add_count(struct mount *mnt, int n) +static inline void mnt_inc_count(struct mount *mnt) { #ifdef CONFIG_SMP - this_cpu_add(mnt->mnt_pcp->mnt_count, n); + this_cpu_inc(mnt->mnt_pcp->mnt_gets); #else preempt_disable(); - mnt->mnt_count += n; + mnt->mnt_count++; + preempt_enable(); +#endif +} + +static inline void mnt_dec_count(struct mount *mnt) +{ +#ifdef CONFIG_SMP + this_cpu_inc(mnt->mnt_pcp->mnt_puts); +#else + preempt_disable(); + mnt->mnt_count--; preempt_enable(); #endif } @@ -269,14 +276,17 @@ static inline void mnt_add_count(struct mount *mnt, int n) int mnt_get_count(struct mount *mnt) { #ifdef CONFIG_SMP - int count = 0; + unsigned int gets = 0, puts = 0; int cpu; - for_each_possible_cpu(cpu) { - count += per_cpu_ptr(mnt->mnt_pcp, cpu)->mnt_count; - } + /* puts first, so a put counted here has its get counted below */ + for_each_possible_cpu(cpu) + puts += per_cpu_ptr(mnt->mnt_pcp, cpu)->mnt_puts; + smp_mb(); /* pairs with the smp_wmb() in mntput_no_expire() */ + for_each_possible_cpu(cpu) + gets += per_cpu_ptr(mnt->mnt_pcp, cpu)->mnt_gets; - return count; + return gets - puts; #else return mnt->mnt_count; #endif @@ -305,7 +315,7 @@ static struct mount *alloc_vfsmnt(const char *name) if (!mnt->mnt_pcp) goto out_free_devname; - this_cpu_add(mnt->mnt_pcp->mnt_count, 1); + this_cpu_inc(mnt->mnt_pcp->mnt_gets); #else mnt->mnt_count = 1; mnt->mnt_writers = 0; @@ -322,6 +332,9 @@ static struct mount *alloc_vfsmnt(const char *name) INIT_HLIST_NODE(&mnt->mnt_mp_list); INIT_HLIST_HEAD(&mnt->mnt_stuck_children); INIT_HLIST_NODE(&mnt->mnt_ns_visible); +#ifdef CONFIG_FSNOTIFY + INIT_LIST_HEAD(&mnt->to_notify); +#endif RB_CLEAR_NODE(&mnt->mnt_node); mnt->mnt.mnt_idmap = &nop_mnt_idmap; } @@ -746,13 +759,13 @@ int __legitimize_mnt(struct vfsmount *bastard, unsigned seq) if (bastard == NULL) return 0; mnt = real_mount(bastard); - mnt_add_count(mnt, 1); - smp_mb(); // see mntput_no_expire() and do_umount() + mnt_inc_count(mnt); + smp_mb(); /* see mntput_no_expire_slowpath() and do_umount() */ if (likely(!read_seqretry(&mount_lock, seq))) return 0; lock_mount_hash(); if (unlikely(bastard->mnt_flags & (MNT_SYNC_UMOUNT | MNT_DOOMED))) { - mnt_add_count(mnt, -1); + mnt_dec_count(mnt); unlock_mount_hash(); return 1; } @@ -1058,7 +1071,10 @@ static void attach_mnt(struct mount *mnt, struct mount *parent, void mnt_change_mountpoint(struct mount *parent, struct mountpoint *mp, struct mount *mnt) { struct mountpoint *old_mp = mnt->mnt_mp; + struct mount *old_parent = mnt->mnt_parent; + if (old_parent->overmount == mnt) + old_parent->overmount = NULL; list_del_init(&mnt->mnt_child); hlist_del_init(&mnt->mnt_mp_list); hlist_del_init_rcu(&mnt->mnt_hash); @@ -1254,6 +1270,7 @@ static struct mount *clone_mnt(struct mount *old, struct dentry *root, mnt->mnt.mnt_flags = READ_ONCE(old->mnt.mnt_flags) & ~MNT_INTERNAL_FLAGS; + mnt->mnt_t_flags = old->mnt_t_flags & T_UNBINDABLE; if (flag & (CL_SLAVE | CL_PRIVATE)) mnt->mnt_group_id = 0; /* not a peer of original */ @@ -1347,7 +1364,7 @@ static void noinline mntput_no_expire_slowpath(struct mount *mnt) * mount_lock, we'll see their refcount increment here. */ smp_mb(); - mnt_add_count(mnt, -1); + mnt_dec_count(mnt); count = mnt_get_count(mnt); if (count != 0) { WARN_ON(count < 0); @@ -1404,7 +1421,8 @@ static void mntput_no_expire(struct mount *mnt) * non-NULL under rcu_read_lock(), the reference * we are dropping is not the final one. */ - mnt_add_count(mnt, -1); + smp_wmb(); /* pairs with the smp_mb() in mnt_get_count() */ + mnt_dec_count(mnt); rcu_read_unlock(); return; } @@ -1426,7 +1444,7 @@ EXPORT_SYMBOL(mntput); struct vfsmount *mntget(struct vfsmount *mnt) { if (mnt) - mnt_add_count(real_mount(mnt), 1); + mnt_inc_count(real_mount(mnt)); return mnt; } EXPORT_SYMBOL(mntget); @@ -1783,6 +1801,8 @@ static void umount_tree(struct mount *mnt, enum umount_tree_flags how) /* Gather the mounts to umount */ for (p = mnt; p; p = next_mnt(p, mnt)) { + /* A mount is unmounted once. */ + VFS_WARN_ON_ONCE(p->mnt.mnt_flags & MNT_UMOUNT); p->mnt.mnt_flags |= MNT_UMOUNT; if (mnt_ns_attached(p)) move_from_ns(p); @@ -1976,6 +1996,9 @@ static int do_umount(struct mount *mnt, int flags) * detach_mounts allows lazily unmounting those mounts instead of * leaking them. * + * The dentry is unhashed before the mounts go so that no lookup finds + * what they covered. The caller removes it for good afterwards. + * * The caller may hold dentry->d_inode->i_rwsem. */ void __detach_mounts(struct dentry *dentry) @@ -1989,6 +2012,8 @@ void __detach_mounts(struct dentry *dentry) if (!lookup_mountpoint(dentry, &mp)) return; + /* the name goes first, what covered it goes second */ + d_drop(dentry); event++; while (mp.node.next) { mnt = hlist_entry(mp.node.next, struct mount, mnt_mp_list); @@ -2345,6 +2370,18 @@ bool has_locked_children(struct mount *mnt, struct dentry *dentry) return __has_locked_children(mnt, dentry); } +/* locks: namespace_shared && pinned(mnt) || mount_locked_reader */ +static bool __has_children(struct mount *mnt, struct dentry *dentry) +{ + struct mount *child; + + list_for_each_entry(child, &mnt->mnt_mounts, mnt_child) { + if (is_subdir(child->mnt_mountpoint, dentry)) + return true; + } + return false; +} + /* * Check that there aren't references to earlier/same mount namespaces in the * specified subtree. Such references can act as pins for mount namespaces @@ -2564,11 +2601,11 @@ enum mnt_tree_flags_t { static int attach_recursive_mnt(struct mount *source_mnt, const struct pinned_mountpoint *dest) { - struct user_namespace *user_ns = current->nsproxy->mnt_ns->user_ns; struct mount *dest_mnt = dest->parent; struct mountpoint *dest_mp = dest->mp; HLIST_HEAD(tree_list); struct mnt_namespace *ns = dest_mnt->mnt_ns; + struct user_namespace *user_ns = ns->user_ns; struct pinned_mountpoint root = {}; struct mountpoint *shorter = NULL; struct mount *child, *p; @@ -2577,13 +2614,31 @@ static int attach_recursive_mnt(struct mount *source_mnt, int err = 0; bool moving = mnt_has_parent(source_mnt); + /* + * A caller in an unprivileged mount namespaces may trigger an + * automount and propagate locked mounts into privileged mount + * namespaces. Take ownership from the target mount namespace. + * It's equivalent for everything but the automount case. + * + * Detached trees in anonymous mount namespaces by be handed + * over via SCM_RIGHTS or inherited in other ways on purpose + * the attaching task's mount namespace is authoritative, not + * the creator of the detached tree. + */ + if (is_anon_ns(ns)) + user_ns = current->nsproxy->mnt_ns->user_ns; + else + user_ns = ns->user_ns; + /* * Preallocate a mountpoint in case the new mounts need to be * mounted beneath mounts on the same mountpoint. */ - for (top = source_mnt; unlikely(top->overmount); top = top->overmount) { + for (top = source_mnt; ; top = top->overmount) { if (!shorter && is_mnt_ns_file(top->mnt.mnt_root)) shorter = top->mnt_mp; + if (likely(!top->overmount)) + break; } err = get_mountpoint(top->mnt.mnt_root, &root); if (err) @@ -2654,15 +2709,17 @@ static int attach_recursive_mnt(struct mount *source_mnt, /* * If @q was locked it was meant to hide * whatever was under it. Let @child take over - * that job and lock it, then we can unlock @q. - * That'll allow another namespace to shed @q - * and reveal @child. Clearly, that mounter - * consented to this by not severing the mount - * relationship. Otherwise, what's the point. + * that job and lock it. If @child is the mount + * the caller placed we can then unlock @q: + * nothing another namespace does removes it + * again. A propagated copy goes away when the + * mounter of the original unmounts it, so @q + * keeps its lock. */ if (IS_MNT_LOCKED(q)) { child->mnt.mnt_flags |= MNT_LOCKED; - q->mnt.mnt_flags &= ~MNT_LOCKED; + if (child == source_mnt) + q->mnt.mnt_flags &= ~MNT_LOCKED; } mnt_change_mountpoint(r, mp, q); } @@ -2986,6 +3043,13 @@ static struct mount *__do_loopback(const struct path *old_path, if (!may_copy_tree(old_path)) return ERR_PTR(-EINVAL); + /* a pseudo dentry is freed without an RCU delay, no walk may find it */ + if (old_path->dentry->d_flags & DCACHE_NORCU) + return ERR_PTR(-EINVAL); + + if (recurse && !old->mnt_ns) + return ERR_PTR(-EINVAL); + if (!recurse && __has_locked_children(old, old_path->dentry)) return ERR_PTR(-EINVAL); @@ -3025,7 +3089,11 @@ static int do_loopback(const struct path *path, const char *old_name, if (IS_ERR(mnt)) return PTR_ERR(mnt); - err = graft_tree(mnt, &mp); + /* the copy may carry mount namespace files from below the source */ + if (recurse && !check_for_nsfs_mounts(mnt)) + err = -EINVAL; + else + err = graft_tree(mnt, &mp); if (err) { lock_mount_hash(); umount_tree(mnt, UMOUNT_SYNC); @@ -3107,12 +3175,18 @@ static struct mnt_namespace *create_new_namespace(struct path *path, struct mount *mnt; unsigned int copy_flags = 0; bool locked = false, recurse = flags & MOUNT_COPY_RECURSIVE; + bool foreign = user_ns != ns->user_ns; if (unlikely(!d_can_lookup(path->dentry))) return ERR_PTR(-ENOTDIR); - if (user_ns != ns->user_ns) - copy_flags |= CL_SLAVE; + /* + * Without privileges over the mount namespace the copy is made from + * nothing mounted below @path may be left out. It would reveal what + * it covers. That's what unshare() gives such a caller as well. + */ + if (foreign) + copy_flags |= CL_SLAVE | CL_COPY_UNBINDABLE; new_ns = alloc_mnt_ns(user_ns, false); if (IS_ERR(new_ns)) @@ -3146,10 +3220,14 @@ static struct mnt_namespace *create_new_namespace(struct path *path, /* * We don't emulate unshare()ing a mount namespace. We stick to * the restrictions of creating detached bind-mounts. It has a - * lot saner and simpler semantics. + * lot saner and simpler semantics. A caller without privileges + * over the mount namespace can't leave out any child though. */ if (flags & MOUNT_COPY_NEW) mnt = clone_mnt(real_mount(path->mnt), path->dentry, copy_flags); + else if (foreign && !recurse && + __has_children(real_mount(path->mnt), path->dentry)) + mnt = ERR_PTR(-EINVAL); else mnt = __do_loopback(path, recurse, copy_flags); scoped_guard(mount_writer) { @@ -3166,7 +3244,7 @@ static struct mnt_namespace *create_new_namespace(struct path *path, * of the real rootfs we created. */ attach_mnt(mnt, new_ns_root, mp.mp); - if (user_ns != ns->user_ns) + if (foreign) lock_mnt_tree(new_ns_root); } @@ -3464,7 +3542,7 @@ static int do_set_group(const struct path *from_path, const struct path *to_path return -EINVAL; /* Setting sharing groups is only allowed on private mounts */ - if (IS_MNT_SHARED(to) || IS_MNT_SLAVE(to)) + if (IS_MNT_SHARED(to) || IS_MNT_SLAVE(to) || IS_MNT_UNBINDABLE(to)) return -EINVAL; /* From should not be private */ @@ -3736,7 +3814,7 @@ static int do_add_mount(struct mount *newmnt, const struct pinned_mountpoint *mp if (!(mnt_flags & MNT_SHRINKABLE)) return -EINVAL; /* ... and for those we'd better have mountpoint still alive */ - if (!parent->mnt_ns) + if (!is_mounted(&parent->mnt)) return -EINVAL; } @@ -3946,6 +4024,8 @@ void mark_mounts_for_expiry(struct list_head *mounts) list_for_each_entry_safe(mnt, next, mounts, mnt_expire) { if (!is_mounted(&mnt->mnt)) continue; + /* lock_mnt_tree() leaves expirable mounts alone */ + VFS_WARN_ON_ONCE(IS_MNT_LOCKED(mnt)); if (!xchg(&mnt->mnt_expiry_mark, 1) || propagate_mount_busy(mnt, 1)) continue; @@ -3953,6 +4033,11 @@ void mark_mounts_for_expiry(struct list_head *mounts) } while (!list_empty(&graveyard)) { mnt = list_first_entry(&graveyard, struct mount, mnt_expire); + /* an earlier umount_tree() may have moved a busy mount here */ + if (propagate_mount_busy(mnt, 1)) { + list_move(&mnt->mnt_expire, mounts); + continue; + } touch_mnt_namespace(mnt->mnt_ns); umount_tree(mnt, UMOUNT_PROPAGATE|UMOUNT_SYNC); } @@ -3960,17 +4045,38 @@ void mark_mounts_for_expiry(struct list_head *mounts) EXPORT_SYMBOL_GPL(mark_mounts_for_expiry); +/* + * Unmount @mnt if it's a shrinkable mount without children that nobody uses. + * + * mount_lock must be held for write + */ +static bool shrink_submount(struct mount *mnt) +{ + /* not the kernel's to remove either */ + if (IS_MNT_LOCKED(mnt) || propagate_mount_busy(mnt, 1)) + return false; + touch_mnt_namespace(mnt->mnt_ns); + umount_tree(mnt, UMOUNT_PROPAGATE|UMOUNT_SYNC); + return true; +} + /* * Ripoff of 'select_parent()' * - * search the list of submounts for a given mountpoint, and move any - * shrinkable submounts to the 'graveyard' list. + * unmount the shrinkable submounts of @parent that aren't busy, children + * before their parent, and say whether anything went + * + * The cursor into the children of @this_parent survives the umount of a + * child mount without child mounts. The mounts that get umounted together with + * it are located under receiving mounts of @this_parent and never under + * @this_parent itself. The one exception is @this_parent getting unmounted + * then the walk starts over. */ -static int select_submounts(struct mount *parent, struct list_head *graveyard) +static bool __shrink_submounts(struct mount *parent) { struct mount *this_parent = parent; struct list_head *next; - int found = 0; + bool shrunk = false; repeat: next = this_parent->mnt_mounts.next; @@ -3989,42 +4095,45 @@ static int select_submounts(struct mount *parent, struct list_head *graveyard) this_parent = mnt; goto repeat; } - - if (!propagate_mount_busy(mnt, 1)) { - list_move_tail(&mnt->mnt_expire, graveyard); - found++; - } + if (!shrink_submount(mnt)) + continue; + shrunk = true; + if (unlikely(this_parent->mnt.mnt_flags & MNT_UMOUNT)) + return true; } /* * All done at this level ... ascend and resume the search */ if (this_parent != parent) { - next = this_parent->mnt_child.next; - this_parent = this_parent->mnt_parent; + struct mount *mnt = this_parent; + + next = mnt->mnt_child.next; + this_parent = mnt->mnt_parent; + /* its children are gone, maybe it can go as well */ + if (shrink_submount(mnt)) { + shrunk = true; + if (unlikely(this_parent->mnt.mnt_flags & MNT_UMOUNT)) + return true; + } goto resume; } - return found; + return shrunk; } /* - * process a list of expirable mountpoints with the intent of discarding any - * submounts of a specific parent mountpoint + * unmount the shrinkable submounts of @mnt that aren't busy + * + * The busy check and the umount of a mount are adjacent. An umount can + * still empty or move a mount in a part of the tree that was walked + * already, so walk again until nothing goes. * * mount_lock must be held for write */ static void shrink_submounts(struct mount *mnt) { - LIST_HEAD(graveyard); - struct mount *m; - - /* extract submounts of 'mountpoint' from the expiration list */ - while (select_submounts(mnt, &graveyard)) { - while (!list_empty(&graveyard)) { - m = list_first_entry(&graveyard, struct mount, - mnt_expire); - touch_mnt_namespace(m->mnt_ns); - umount_tree(m, UMOUNT_PROPAGATE|UMOUNT_SYNC); - } + for (;;) { + if (!__shrink_submounts(mnt)) + break; } } @@ -4189,6 +4298,8 @@ static void free_mnt_ns(struct mnt_namespace *ns) if (!is_anon_ns(ns)) ns_common_free(ns); dec_mnt_namespaces(ns->ucounts); + /* the last active reference is gone, no mark can show up anymore */ + fsnotify_mntns_delete(ns); mnt_ns_tree_remove(ns); } @@ -4690,14 +4801,15 @@ int path_pivot_root(struct path *new, struct path *old) new_mnt = real_mount(new->mnt); root_mnt = real_mount(root.mnt); + /* only a mounted mount has a parent that namespace_sem pins */ + if (!check_mnt(root_mnt) || !check_mnt(new_mnt)) + return -EINVAL; ex_parent = new_mnt->mnt_parent; root_parent = root_mnt->mnt_parent; if (IS_MNT_SHARED(old_mnt) || IS_MNT_SHARED(ex_parent) || IS_MNT_SHARED(root_parent)) return -EINVAL; - if (!check_mnt(root_mnt) || !check_mnt(new_mnt)) - return -EINVAL; if (new_mnt->mnt.mnt_flags & MNT_LOCKED) return -EINVAL; if (d_unlinked(new->dentry)) @@ -4948,6 +5060,11 @@ static int do_mount_setattr(const struct path *path, struct mount_kattr *kattr) * propagation. */ namespace_lock(); + /* invent_group_ids() walks the tree, only walk a mounted one */ + if (!anon_ns_root(mnt) && !check_mnt(mnt)) { + namespace_unlock(); + return -EINVAL; + } if (kattr->propagation == MS_SHARED) { err = invent_group_ids(mnt, kattr->kflags & MOUNT_KATTR_RECURSE); if (err) { @@ -4960,6 +5077,7 @@ static int do_mount_setattr(const struct path *path, struct mount_kattr *kattr) err = -EINVAL; lock_mount_hash(); + /* Checked under namespace_sem already if the propagation changes. */ if (!anon_ns_root(mnt) && !check_mnt(mnt)) goto out; @@ -5312,15 +5430,25 @@ static void statmount_sb_basic(struct kstatmount *s) s->sm.sb_flags = sb->s_flags & (SB_RDONLY|SB_SYNCHRONOUS|SB_DIRSYNC|SB_LAZYTIME); } +static void statmount_mnt_parent(struct kstatmount *s, const struct mount *m) +{ + s->sm.mnt_parent_id = m->mnt_parent->mnt_id_unique; + s->sm.mnt_parent_id_old = m->mnt_parent->mnt_id; +} + static void statmount_mnt_basic(struct kstatmount *s) { struct mount *m = real_mount(s->mnt); s->sm.mask |= STATMOUNT_MNT_BASIC; s->sm.mnt_id = m->mnt_id_unique; - s->sm.mnt_parent_id = m->mnt_parent->mnt_id_unique; s->sm.mnt_id_old = m->mnt_id; - s->sm.mnt_parent_id_old = m->mnt_parent->mnt_id; + /* An unmounted mount is cut loose from its parent under mount_lock alone. */ + if (likely(is_mounted(s->mnt))) + statmount_mnt_parent(s, m); + else + scoped_guard(mount_locked_reader) + statmount_mnt_parent(s, m); s->sm.mnt_attr = mnt_to_attr_flags(&m->mnt); s->sm.mnt_propagation = mnt_to_propagation_flags(m); s->sm.mnt_peer_group = m->mnt_group_id; diff --git a/fs/nullfs.c b/fs/nullfs.c index fdbd3e5d3d71..20d3aa627a47 100644 --- a/fs/nullfs.c +++ b/fs/nullfs.c @@ -30,8 +30,8 @@ static int nullfs_fs_fill_super(struct super_block *s, struct fs_context *fc) make_empty_dir_inode(inode); simple_inode_init_ts(inode); inode->i_ino = 1; - /* ... and immutable. */ - inode->i_flags |= S_IMMUTABLE; + /* ... and immutable, reading it leaves no trace either. */ + inode->i_flags |= S_IMMUTABLE | S_NOATIME; s->s_root = d_make_root(inode); if (!s->s_root) diff --git a/fs/pnode.c b/fs/pnode.c index 5d91c3e58d2a..2cd667958efe 100644 --- a/fs/pnode.c +++ b/fs/pnode.c @@ -410,19 +410,99 @@ bool propagation_would_overmount(const struct mount *from, return false; } +/* Does @m receive propagation from @parent? */ +static bool receives_from(struct mount *m, struct mount *parent) +{ + if (m == parent) + return false; + for (; m; m = m->mnt_master) + if (m == parent || peers(m, parent)) + return true; + return false; +} + +/* + * Does @m receive propagation from the victim's parent as well? If so, then + * the mount at the victim's mountpoint inside of @m is a umount candidate as + * well. So it's the next candidate in the chain. Otherwise the chain ends at + * @m. + */ +static struct mount *next_candidate(struct mount *m, struct mount *victim) +{ + if (!receives_from(m, victim->mnt_parent)) + return NULL; + return __lookup_mnt(&m->mnt, victim->mnt_mountpoint); +} + +/* + * Would propagate_umount() pull out a mount of the chain of candidates that + * starts at @c, and does that mount have references beyond its own? + * + * This mirrors how trim_one(), trim_ancestors() and handle_locked() handle a + * synchronous umount: + * + * - single victim + * - without children + * - with MNT_LOCKED already cleared on every candidate by propagate_mount_unlock() + * + * A copy of the victim gets unmounted when each of its children is + * the next candidate in the chain or its overmount, unless the next + * unmount candidate is not its overmount and some unmount candidate further + * down has a child outside the chain. Keep this in sync with + * Documentation/filesystems/propagate_umount.txt. + */ +static bool chain_busy(struct mount *c, struct mount *victim) +{ + struct mount *m, *n, *next, *deepest = NULL; + bool above; + + /* the deepest candidate with a child outside the chain */ + for (m = c; m; m = next) { + next = next_candidate(m, victim); + list_for_each_entry(n, &m->mnt_mounts, mnt_child) { + if (n != next && n != victim) { + deepest = m; + break; + } + } + } + + above = deepest != NULL; /* @deepest is at or below @m */ + for (m = c; m; m = next) { + bool goes = true; + + next = next_candidate(m, victim); + list_for_each_entry(n, &m->mnt_mounts, mnt_child) { + if (n != next && n != m->overmount && n != victim) { + goes = false; + break; + } + } + if (goes && next && next != m->overmount && above && m != deepest) + goes = false; + if (m == deepest) + above = false; + if (goes && do_refcount_check(m, 1)) + return true; + } + return false; +} + /* * check if the mount 'mnt' can be unmounted successfully. * @mnt: the mount to be checked for unmount * NOTE: unmounting 'mnt' would naturally propagate to all * other mounts its parent propagates to. - * Check if any of these mounts that **do not have submounts** - * have more references than 'refcnt'. If so return busy. + * Check if any of the mounts that propagate_umount() would pull out + * along with it have more references than their own. If so return busy. * * vfsmount lock must be held for write */ int propagate_mount_busy(struct mount *mnt, int refcnt) { struct mount *parent = mnt->mnt_parent; + struct dentry *mp = mnt->mnt_mountpoint; + struct mount *m; /* * quickly check if the current mount can be unmounted. @@ -435,24 +515,16 @@ int propagate_mount_busy(struct mount *mnt, int refcnt) if (mnt == parent) return 0; - for (struct mount *m = propagation_next(parent, parent); m; - m = propagation_next(m, parent)) { - struct list_head *head; - struct mount *child = __lookup_mnt(&m->mnt, mnt->mnt_mountpoint); + /* the candidates are the mounts at @mp below the receivers */ + for (m = propagation_next(parent, parent); m; + m = propagation_next(m, parent)) { + struct mount *c = __lookup_mnt(&m->mnt, mp); - if (!child) + /* each chain once, from its top: skip receivers that are candidates */ + if (!c || (mnt_has_parent(m) && m->mnt_mountpoint == mp && + receives_from(m->mnt_parent, parent))) continue; - - head = &child->mnt_mounts; - if (!list_empty(head)) { - /* - * a mount that covers child completely wouldn't prevent - * it being pulled out; any other would. - */ - if (!list_is_singular(head) || !child->overmount) - continue; - } - if (do_refcount_check(child, 1)) + if (chain_busy(c, mnt)) return 1; } return 0; diff --git a/fs/verity/fsverity_private.h b/fs/verity/fsverity_private.h index 881d46f25e08..6f85f63861aa 100644 --- a/fs/verity/fsverity_private.h +++ b/fs/verity/fsverity_private.h @@ -11,6 +11,7 @@ #define pr_fmt(fmt) "fs-verity: " fmt #include +#include #include /* @@ -76,7 +77,19 @@ struct merkle_tree_params { struct fsverity_info { struct rhash_head rhash_head; struct merkle_tree_params tree_params; - u8 root_hash[FS_VERITY_MAX_DIGEST_SIZE]; + union { + u8 root_hash[FS_VERITY_MAX_DIGEST_SIZE]; + + /* + * Used for final freeing of the struct, in which case root_hash + * is no longer needed. Specifically, the only fields needed + * besides the rcu_head are those needed by fsverity_free_info() + * itself, and those that can be accessed by fsverity_get_info() + * (rhash_head and inode) during the RCU-protected rhashtable + * chain search trying to find another inode's fsverity_info. + */ + struct rcu_head rcu_head; + }; u8 file_digest[FS_VERITY_MAX_DIGEST_SIZE]; struct inode *inode; unsigned long *hash_block_verified; diff --git a/fs/verity/open.c b/fs/verity/open.c index d0c56a7faa3b..52c8cd014921 100644 --- a/fs/verity/open.c +++ b/fs/verity/open.c @@ -399,11 +399,22 @@ void fsverity_free_info(struct fsverity_info *vi) kmem_cache_free(fsverity_info_cachep, vi); } +static void fsverity_free_info_rcu(struct rcu_head *head) +{ + fsverity_free_info(container_of(head, struct fsverity_info, rcu_head)); +} + void fsverity_remove_info(struct fsverity_info *vi) { rhashtable_remove_fast(&fsverity_info_hash, &vi->rhash_head, fsverity_info_hash_params); - fsverity_free_info(vi); + /* + * The freeing must be RCU-delayed because __fsverity_get_info() for a + * *different* inode's fsverity_info in the same rhashtable hash chain + * can still be accessing *this* fsverity_info's rhash_head and inode + * fields as part of the RCU-protected hash chain search. + */ + call_rcu(&vi->rcu_head, fsverity_free_info_rcu); } void fsverity_cleanup_inode(struct inode *inode) diff --git a/fs/xfs/libxfs/xfs_exchmaps.c b/fs/xfs/libxfs/xfs_exchmaps.c index 49eda8d0994d..37595f579b63 100644 --- a/fs/xfs/libxfs/xfs_exchmaps.c +++ b/fs/xfs/libxfs/xfs_exchmaps.c @@ -49,7 +49,7 @@ struct xfs_exchmaps_adjacent { .right2 = { .br_startblock = HOLESTARTBLOCK }, \ } -/* Information to reset reflink flag / CoW fork state after an exchange. */ +/* Information to reset CoW fork state after an exchange. */ /* * If the reflink flag is set on either inode, make sure it has an incore CoW @@ -127,9 +127,7 @@ xmi_has_more_exchange_work(const struct xfs_exchmaps_intent *xmi) static inline bool xmi_has_postop_work(const struct xfs_exchmaps_intent *xmi) { - return xmi->xmi_flags & (XFS_EXCHMAPS_CLEAR_INO1_REFLINK | - XFS_EXCHMAPS_CLEAR_INO2_REFLINK | - __XFS_EXCHMAPS_INO2_SHORTFORM); + return xmi->xmi_flags & __XFS_EXCHMAPS_INO2_SHORTFORM; } /* Check all mappings to make sure we can actually exchange them. */ @@ -525,18 +523,6 @@ xfs_exchmaps_link_to_sf( return error; } -/* Clear the reflink flag after an exchange. */ -static inline void -xfs_exchmaps_clear_reflink( - struct xfs_trans *tp, - struct xfs_inode *ip) -{ - trace_xfs_reflink_unset_inode_flag(ip); - - ip->i_diflags2 &= ~XFS_DIFLAG2_REFLINK; - xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); -} - /* Finish whatever work might come after an exchange operation. */ static int xfs_exchmaps_do_postop_work( @@ -557,16 +543,6 @@ xfs_exchmaps_do_postop_work( return error; } - if (xmi->xmi_flags & XFS_EXCHMAPS_CLEAR_INO1_REFLINK) { - xfs_exchmaps_clear_reflink(tp, xmi->xmi_ip1); - xmi->xmi_flags &= ~XFS_EXCHMAPS_CLEAR_INO1_REFLINK; - } - - if (xmi->xmi_flags & XFS_EXCHMAPS_CLEAR_INO2_REFLINK) { - xfs_exchmaps_clear_reflink(tp, xmi->xmi_ip2); - xmi->xmi_flags &= ~XFS_EXCHMAPS_CLEAR_INO2_REFLINK; - } - return 0; } @@ -948,56 +924,14 @@ xfs_exchmaps_intent_destroy_cache(void) } /* - * Decide if we will exchange the reflink flags between the two files after the - * exchange. The only time we want to do this is if we're exchanging all - * mappings under EOF and the inode reflink flags have different states. + * Allocate and initialize a new incore intent item from a request. */ -static inline bool -xmi_can_exchange_reflink_flags( - const struct xfs_exchmaps_req *req, - unsigned int reflink_state) -{ - struct xfs_mount *mp = req->ip1->i_mount; - - /* - * The INO1_WRITTEN optimization can skip exchanging hole and - * unwritten mappings, which means we cannot guarantee that all - * shared extents actually moved to the other file. Clearing the - * reflink flag of an inode that still holds shared extents breaks - * the CoW write path, so refuse to exchange the flags in that case. - */ - if (req->flags & XFS_EXCHMAPS_INO1_WRITTEN) - return false; - - /* - * The INO1_WRITTEN optimization can skip exchanging hole and - * unwritten mappings, which means we cannot guarantee that all - * shared extents actually moved to the other file. Clearing the - * reflink flag of an inode that still holds shared extents breaks - * the CoW write path, so refuse to exchange the flags in that case. - */ - if (req->flags & XFS_EXCHMAPS_INO1_WRITTEN) - return false; - - if (hweight32(reflink_state) != 1) - return false; - if (req->startoff1 != 0 || req->startoff2 != 0) - return false; - if (req->blockcount != XFS_B_TO_FSB(mp, req->ip1->i_disk_size)) - return false; - if (req->blockcount != XFS_B_TO_FSB(mp, req->ip2->i_disk_size)) - return false; - return true; -} - - -/* Allocate and initialize a new incore intent item from a request. */ struct xfs_exchmaps_intent * xfs_exchmaps_init_intent( const struct xfs_exchmaps_req *req) { struct xfs_exchmaps_intent *xmi; - unsigned int rs = 0; + struct xfs_mount *mp = req->ip1->i_mount; xmi = kmem_cache_zalloc(xfs_exchmaps_intent_cache, GFP_NOFS | __GFP_NOFAIL); @@ -1015,27 +949,18 @@ xfs_exchmaps_init_intent( return xmi; } - if (req->flags & XFS_EXCHMAPS_SET_SIZES) { - xmi->xmi_flags |= XFS_EXCHMAPS_SET_SIZES; - xmi->xmi_isize1 = req->ip2->i_disk_size; - xmi->xmi_isize2 = req->ip1->i_disk_size; - } - - /* Record the state of each inode's reflink flag before the op. */ - if (xfs_is_reflink_inode(req->ip1)) - rs |= 1; - if (xfs_is_reflink_inode(req->ip2)) - rs |= 2; - /* - * Figure out if we're clearing the reflink flags (which effectively - * exchanges them) after the operation. + * If the caller wanted, set each file's size to that file's exchange + * offset + length exchanged from the other file because the ranges in + * each file might be different lengths. */ - if (xmi_can_exchange_reflink_flags(req, rs)) { - if (rs & 1) - xmi->xmi_flags |= XFS_EXCHMAPS_CLEAR_INO1_REFLINK; - if (rs & 2) - xmi->xmi_flags |= XFS_EXCHMAPS_CLEAR_INO2_REFLINK; + if (req->flags & XFS_EXCHMAPS_SET_SIZES) { + loff_t off1 = XFS_FSB_TO_B(mp, xmi->xmi_startoff1); + loff_t off2 = XFS_FSB_TO_B(mp, xmi->xmi_startoff2); + + xmi->xmi_flags |= XFS_EXCHMAPS_SET_SIZES; + xmi->xmi_isize1 = off1 + (req->ip2->i_disk_size - off2); + xmi->xmi_isize2 = off2 + (req->ip1->i_disk_size - off1); } if (S_ISDIR(VFS_I(xmi->xmi_ip2)->i_mode) || diff --git a/fs/xfs/libxfs/xfs_log_format.h b/fs/xfs/libxfs/xfs_log_format.h index a4e1b3eb425c..6e620380d5dc 100644 --- a/fs/xfs/libxfs/xfs_log_format.h +++ b/fs/xfs/libxfs/xfs_log_format.h @@ -927,10 +927,8 @@ struct xfs_xmi_log_format { */ #define XFS_EXCHMAPS_INO1_WRITTEN (1ULL << 2) -/* Clear the reflink flag from inode1 after the operation. */ +/* No longer set, but older intents may carry them; keep them valid. */ #define XFS_EXCHMAPS_CLEAR_INO1_REFLINK (1ULL << 3) - -/* Clear the reflink flag from inode2 after the operation. */ #define XFS_EXCHMAPS_CLEAR_INO2_REFLINK (1ULL << 4) #define XFS_EXCHMAPS_LOGGED_FLAGS (XFS_EXCHMAPS_ATTR_FORK | \ diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index c88b9ade7389..2ea650767805 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -1514,7 +1514,6 @@ xfs_swap_extents( struct xfs_bstat *sbp = &sxp->sx_stat; int src_log_flags, target_log_flags; int error = 0; - uint64_t f; int resblks = 0; unsigned int flags = 0; struct timespec64 ctime, mtime; @@ -1670,14 +1669,12 @@ xfs_swap_extents( if (error) goto out_trans_cancel; - /* Do we have to swap reflink flags? */ - if ((ip->i_diflags2 & XFS_DIFLAG2_REFLINK) ^ - (tip->i_diflags2 & XFS_DIFLAG2_REFLINK)) { - f = ip->i_diflags2 & XFS_DIFLAG2_REFLINK; - ip->i_diflags2 &= ~XFS_DIFLAG2_REFLINK; - ip->i_diflags2 |= tip->i_diflags2 & XFS_DIFLAG2_REFLINK; - tip->i_diflags2 &= ~XFS_DIFLAG2_REFLINK; - tip->i_diflags2 |= f & XFS_DIFLAG2_REFLINK; + /* Either inode may now own shared blocks; flag both, with CoW forks. */ + if ((ip->i_diflags2 | tip->i_diflags2) & XFS_DIFLAG2_REFLINK) { + ip->i_diflags2 |= XFS_DIFLAG2_REFLINK; + tip->i_diflags2 |= XFS_DIFLAG2_REFLINK; + xfs_ifork_init_cow(ip); + xfs_ifork_init_cow(tip); } /* Swap the cow forks. */ diff --git a/fs/xfs/xfs_exchrange.c b/fs/xfs/xfs_exchrange.c index fafb4e3f065c..935b561dfd3c 100644 --- a/fs/xfs/xfs_exchrange.c +++ b/fs/xfs/xfs_exchrange.c @@ -290,17 +290,20 @@ xfs_exchrange_mappings( goto out_unlock; /* - * If the caller wanted us to exchange the contents of two complete - * files of unequal length, exchange the incore sizes now. This should - * be safe because we flushed both files' page caches, exchanged all - * the mappings, and updated the ondisk sizes. + * If the caller wanted, set each file's size to that file's exchange + * offset + length exchanged from the other file because the ranges in + * each file might be different lengths. This should be safe because + * we flushed both files' page caches, exchanged all the mappings, and + * updated the ondisk sizes. */ if (fxr->flags & XFS_EXCHANGE_RANGE_TO_EOF) { - loff_t temp; + loff_t old_ip1_size = i_size_read(VFS_I(ip1)); + loff_t old_ip2_size = i_size_read(VFS_I(ip2)); - temp = i_size_read(VFS_I(ip2)); - i_size_write(VFS_I(ip2), i_size_read(VFS_I(ip1))); - i_size_write(VFS_I(ip1), temp); + i_size_write(VFS_I(ip2), fxr->file2_offset + + (old_ip1_size - fxr->file1_offset)); + i_size_write(VFS_I(ip1), fxr->file1_offset + + (old_ip2_size - fxr->file2_offset)); } out_unlock: @@ -445,6 +448,30 @@ xfs_exchange_range_checks( return blen == fxr->length ? 0 : -EINVAL; } +/* Flush all relevant dirty pagecache near the range to be exchanged. */ +static inline int +xfs_exchange_range_flush( + struct xfs_inode *ip, + const struct xfs_exchrange *fxr, + loff_t offset) +{ + /* + * If TO_EOF is set, the ondisk file size update logic depends on the + * ondisk file size matching the incore file size. Flush everything. + */ + if (fxr->flags & XFS_EXCHANGE_RANGE_TO_EOF) + return filemap_write_and_wait(VFS_I(ip)->i_mapping); + + /* + * Because we're updating the ondisk mappings, flush all dirty data + * between the ondisk size and the exchange offset to reduce the chance + * of zeroed file contents in that gap after a crash. + */ + return filemap_write_and_wait_range(VFS_I(ip)->i_mapping, + min(offset, ip->i_disk_size), + offset + fxr->length - 1); +} + /* * Check that the two inodes are eligible for range exchanges, the ranges make * sense, and then flush all dirty data. Caller must ensure that the inodes @@ -470,15 +497,11 @@ xfs_exchange_range_prep( if (!same_inode) inode_dio_wait(inode2); - error = filemap_write_and_wait_range(inode1->i_mapping, - fxr->file1_offset, - fxr->file1_offset + fxr->length - 1); + error = xfs_exchange_range_flush(XFS_I(inode1), fxr, fxr->file1_offset); if (error) return error; - error = filemap_write_and_wait_range(inode2->i_mapping, - fxr->file2_offset, - fxr->file2_offset + fxr->length - 1); + error = xfs_exchange_range_flush(XFS_I(inode2), fxr, fxr->file2_offset); if (error) return error; diff --git a/include/linux/can/core.h b/include/linux/can/core.h index 3287232e3cad..2de74c2b78b6 100644 --- a/include/linux/can/core.h +++ b/include/linux/can/core.h @@ -17,6 +17,7 @@ #include #include #include +#include #define DNAME(dev) ((dev) ? (dev)->name : "any") @@ -58,7 +59,7 @@ extern void can_rx_unregister(struct net *net, struct net_device *dev, void *data); extern int can_send(struct sk_buff *skb, int loop); -void can_set_skb_uid(struct sk_buff *skb); +void can_set_skb_uid(struct can_skb_ext *csx); void can_sock_destruct(struct sock *sk); #endif /* !_CAN_CORE_H */ diff --git a/include/linux/can/skb.h b/include/linux/can/skb.h index a70a02967071..5d27843862fc 100644 --- a/include/linux/can/skb.h +++ b/include/linux/can/skb.h @@ -43,8 +43,10 @@ static inline struct can_skb_ext *can_skb_ext_add(struct sk_buff *skb) struct can_skb_ext *csx = skb_ext_add(skb, SKB_EXT_CAN); /* skb_ext_add() returns uninitialized space */ - if (csx) + if (csx) { csx->can_gw_hops = 0; + csx->can_skb_uid = 0; + } return csx; } diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index dac5b2cb8aaa..20116462cf3d 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -316,6 +316,8 @@ enum { IO_RING_F_DRAIN_DISABLED = BIT(10), IO_RING_F_COMPAT = BIT(11), IO_RING_F_IOWQ_LIMITS_SET = BIT(12), + /* restrictions were inherited from the task that created the ring */ + IO_RING_F_RESTRICT_INHERITED = BIT(13), }; struct iou_ctx {}; diff --git a/include/linux/mount.h b/include/linux/mount.h index acfe7ef86a1b..b16ba5d45605 100644 --- a/include/linux/mount.h +++ b/include/linux/mount.h @@ -52,7 +52,7 @@ enum mount_flags { MNT_ATIME_MASK = MNT_NOATIME | MNT_NODIRATIME | MNT_RELATIME, MNT_INTERNAL_FLAGS = MNT_INTERNAL | MNT_DOOMED | - MNT_SYNC_UMOUNT | MNT_LOCKED + MNT_SYNC_UMOUNT | MNT_LOCKED | MNT_UMOUNT }; struct vfsmount { diff --git a/include/linux/nsproxy.h b/include/linux/nsproxy.h index 5a67648721c7..dc2447f5f092 100644 --- a/include/linux/nsproxy.h +++ b/include/linux/nsproxy.h @@ -100,6 +100,7 @@ void exit_cred_namespaces(struct task_struct *tsk); void switch_task_namespaces(struct task_struct *tsk, struct nsproxy *new); int exec_task_namespaces(void); void deactivate_nsproxy(struct nsproxy *ns); +void nsproxy_free(struct nsproxy *ns); int unshare_nsproxy_namespaces(unsigned long, struct nsproxy **, struct cred *, struct fs_struct *); int __init nsproxy_cache_init(void); diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 041f30931a90..03ddd2c6d7fa 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -1266,8 +1266,8 @@ enum { NVME_ZRASF_ZONE_STATE_IMP_OPEN = 0x02, NVME_ZRASF_ZONE_STATE_EXP_OPEN = 0x03, NVME_ZRASF_ZONE_STATE_CLOSED = 0x04, - NVME_ZRASF_ZONE_STATE_READONLY = 0x05, - NVME_ZRASF_ZONE_STATE_FULL = 0x06, + NVME_ZRASF_ZONE_STATE_FULL = 0x05, + NVME_ZRASF_ZONE_STATE_READONLY = 0x06, NVME_ZRASF_ZONE_STATE_OFFLINE = 0x07, NVME_REPORT_ZONE_PARTIAL = 1, }; diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h index 87e353f7e011..67a97429d8a6 100644 --- a/include/linux/sched/ext.h +++ b/include/linux/sched/ext.h @@ -198,6 +198,7 @@ struct sched_ext_entity { s32 sticky_cpu; s32 holding_cpu; s32 selected_cpu; + u32 ops_qseq; /* protected by rq lock */ struct task_struct *kf_tasks[2]; /* see SCX_CALL_OP_TASK() */ struct list_head runnable_node; /* rq->scx.runnable_list */ diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h index c381b916c1b5..d6466f96cdd0 100644 --- a/include/linux/virtio_net.h +++ b/include/linux/virtio_net.h @@ -111,48 +111,38 @@ static inline int __virtio_net_hdr_to_skb(struct sk_buff *skb, p_off = nh_min_len + thlen; if (!pskb_may_pull(skb, p_off)) return -EINVAL; - } else { + } else if (gso_type) { /* gso packets without NEEDS_CSUM do not set transport_offset. * probe and drop if does not match one of the above types. */ - if (gso_type && skb->network_header) { - struct flow_keys_basic keys; - - if (!skb->protocol) { - __be16 protocol = dev_parse_header_protocol(skb); - - if (!protocol) - virtio_net_hdr_set_proto(skb, hdr); - else if (!virtio_net_hdr_match_proto(protocol, - hdr_gso_type)) - return -EINVAL; - else - skb->protocol = protocol; - } + struct flow_keys_basic keys; + + if (!skb->protocol) { + skb->protocol = dev_parse_header_protocol(skb); + if (!skb->protocol) + virtio_net_hdr_set_proto(skb, hdr); + } retry: - if (!skb_flow_dissect_flow_keys_basic(NULL, skb, &keys, - NULL, 0, 0, 0, - 0)) { - /* UFO does not specify ipv4 or 6: try both */ - if (gso_type & SKB_GSO_UDP && - skb->protocol == htons(ETH_P_IP)) { - skb->protocol = htons(ETH_P_IPV6); - goto retry; - } - return -EINVAL; + if (!skb_flow_dissect_flow_keys_basic(NULL, skb, &keys, + NULL, 0, 0, 0, + 0)) { + /* UFO does not specify ipv4 or 6: try both */ + if (gso_type & SKB_GSO_UDP && + skb->protocol == htons(ETH_P_IP)) { + skb->protocol = htons(ETH_P_IPV6); + goto retry; } + return -EINVAL; + } - p_off = keys.control.thoff + thlen; - if (!pskb_may_pull(skb, p_off) || - keys.basic.ip_proto != ip_proto) - return -EINVAL; + p_off = keys.control.thoff + thlen; + if (!pskb_may_pull(skb, p_off) || + keys.basic.ip_proto != ip_proto || + !virtio_net_hdr_match_proto(keys.basic.n_proto, + hdr_gso_type)) + return -EINVAL; - skb_set_transport_header(skb, keys.control.thoff); - } else if (gso_type) { - p_off = nh_min_len + thlen; - if (!pskb_may_pull(skb, p_off)) - return -EINVAL; - } + skb_set_transport_header(skb, keys.control.thoff); } if (hdr_gso_type != VIRTIO_NET_HDR_GSO_NONE) { diff --git a/include/net/amt.h b/include/net/amt.h index a0255491f5b0..2846dde0cadc 100644 --- a/include/net/amt.h +++ b/include/net/amt.h @@ -231,10 +231,6 @@ struct amt_relay_headers { }; } __packed; -struct amt_skb_cb { - struct amt_tunnel_list *tunnel; -}; - struct amt_tunnel_list { struct list_head list; /* Protect All resources under an amt_tunne_list */ diff --git a/include/net/bluetooth/rfcomm.h b/include/net/bluetooth/rfcomm.h index 102c278e3584..9dc5fd4cb8c8 100644 --- a/include/net/bluetooth/rfcomm.h +++ b/include/net/bluetooth/rfcomm.h @@ -207,6 +207,7 @@ struct rfcomm_dlc { #define RFCOMM_AUTH_REJECT 7 #define RFCOMM_DEFER_SETUP 8 #define RFCOMM_ENC_DROP 9 +#define RFCOMM_CLOSED 10 /* Scheduling flags and events */ #define RFCOMM_SCHED_WAKEUP 31 diff --git a/include/net/can.h b/include/net/can.h index 6db9e826f0e0..2b8af2598732 100644 --- a/include/net/can.h +++ b/include/net/can.h @@ -17,12 +17,14 @@ * @can_framelen: cached echo CAN frame length for bql * @can_gw_hops: can-gw CAN frame time-to-live counter * @can_ext_flags: CAN skb extensions flags + * @can_skb_uid: CAN skb UID for raw_rcv and isotp echo handling */ struct can_skb_ext { int can_iif; u16 can_framelen; u8 can_gw_hops; u8 can_ext_flags; + u32 can_skb_uid; }; #endif /* _NET_CAN_H */ diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index f5abf1db7558..7354ba079363 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -3722,6 +3722,7 @@ enum wiphy_params_flags { /* The per TXQ device queue limit in airtime */ #define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L 5000 #define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_H 12000 +#define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_MC 50000 /* The per interface airtime threshold to switch to lower queue limit */ #define IEEE80211_AQL_THRESHOLD 24000 diff --git a/include/net/ip.h b/include/net/ip.h index 6f602df72ee6..d07c2c573024 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -74,6 +74,13 @@ static inline unsigned int ip_hdrlen(const struct sk_buff *skb) return ip_hdr(skb)->ihl * 4; } +static inline int ip_check_csum_start(const struct sk_buff *skb) +{ + if (unlikely(skb->csum_start < skb->network_header + ip_hdrlen(skb))) + return -EINVAL; + return 0; +} + struct ipcm_cookie { struct sockcm_cookie sockc; __be32 addr; diff --git a/include/net/mac80211.h b/include/net/mac80211.h index 36055d40172a..7b610dfee286 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -1317,6 +1317,7 @@ ieee80211_rate_get_vht_nss(const struct ieee80211_tx_rate *rate) * @status_data: internal data for TX status handling, assigned privately, * see also &enum ieee80211_status_data for the internal documentation * @status_data_idr: indicates status data is IDR allocated ID for ack frame + * @tx_time_mc: TX time estimate is for a multicast frame, used internally * @tx_time_est: TX time estimate in units of 4us, used internally * @control: union part for control data * @control.rates: TX rates array to try @@ -1354,8 +1355,8 @@ struct ieee80211_tx_info { status_data_idr:1, status_data:13, hw_queue:4, + tx_time_mc:1, tx_time_est:10; - /* 1 free bit */ union { struct { @@ -6876,6 +6877,17 @@ void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, bool ieee80211_txq_airtime_check(struct ieee80211_hw *hw, struct ieee80211_txq *txq); +/** + * ieee80211_txq_aql_pending - get pending AQL airtime for a txq + * + * @hw: pointer obtained from ieee80211_alloc_hw() + * @txq: pointer obtained from station or virtual interface + * + * Return: pending airtime (in usec) for the given txq. + */ +u32 ieee80211_txq_aql_pending(struct ieee80211_hw *hw, + struct ieee80211_txq *txq); + /** * ieee80211_iter_keys - iterate keys programmed into the device * @hw: pointer obtained from ieee80211_alloc_hw() diff --git a/include/net/strparser.h b/include/net/strparser.h index 0ed73e364faa..881b7ae34498 100644 --- a/include/net/strparser.h +++ b/include/net/strparser.h @@ -91,11 +91,11 @@ static inline struct strp_msg *strp_msg(struct sk_buff *skb) struct strparser { struct sock *sk; - u32 stopped : 1; - u32 paused : 1; - u32 aborted : 1; - u32 interrupted : 1; - u32 unrecov_intr : 1; + u8 stopped; + u8 paused; + u8 aborted; + u8 interrupted; + u8 unrecov_intr; struct sk_buff **skb_nextp; struct sk_buff *skb_head; diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index c5404fbe3f7e..3d8c8a50fd31 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2077,8 +2077,9 @@ int io_submit_sqes(struct io_ring_ctx *ctx, unsigned int nr) static void io_rings_free(struct io_ring_ctx *ctx) { - io_free_region(ctx->user, &ctx->sq_region); - io_free_region(ctx->user, &ctx->ring_region); + /* ring memory is not charged to RLIMIT_MEMLOCK, hence no user */ + io_free_region(NULL, &ctx->sq_region); + io_free_region(NULL, &ctx->ring_region); ctx->rings = NULL; RCU_INIT_POINTER(ctx->rings_rcu, NULL); ctx->sq_sqes = NULL; @@ -2742,7 +2743,8 @@ static __cold int io_allocate_scq_urings(struct io_ring_ctx *ctx, rd.user_addr = p->cq_off.user_addr; rd.flags |= IORING_MEM_REGION_TYPE_USER; } - ret = io_create_region(ctx, &ctx->ring_region, &rd, IORING_OFF_CQ_RING); + /* ring memory is not charged to RLIMIT_MEMLOCK, hence no user */ + ret = io_create_region(NULL, &ctx->ring_region, &rd, IORING_OFF_CQ_RING); if (ret) return ret; ctx->rings = rings = io_region_get_ptr(&ctx->ring_region); @@ -2756,7 +2758,7 @@ static __cold int io_allocate_scq_urings(struct io_ring_ctx *ctx, rd.user_addr = p->sq_off.user_addr; rd.flags |= IORING_MEM_REGION_TYPE_USER; } - ret = io_create_region(ctx, &ctx->sq_region, &rd, IORING_OFF_SQES); + ret = io_create_region(NULL, &ctx->sq_region, &rd, IORING_OFF_SQES); if (ret) { io_rings_free(ctx); return ret; @@ -2967,7 +2969,8 @@ static void io_ctx_restriction_clone(struct io_ring_ctx *ctx, if (dst->op_registered) ctx->int_flags |= IO_RING_F_OP_RESTRICTED; if (dst->reg_registered) - ctx->int_flags |= IO_RING_F_REG_RESTRICTED; + ctx->int_flags |= IO_RING_F_REG_RESTRICTED | + IO_RING_F_RESTRICT_INHERITED; } static __cold int io_uring_create(struct io_ctx_config *config) diff --git a/io_uring/kbuf.c b/io_uring/kbuf.c index 1cf5be62bb65..5065a3eff809 100644 --- a/io_uring/kbuf.c +++ b/io_uring/kbuf.c @@ -458,7 +458,7 @@ static int io_remove_buffers_legacy(struct io_ring_ctx *ctx, static void io_put_bl(struct io_ring_ctx *ctx, struct io_buffer_list *bl) { if (bl->flags & IOBL_BUF_RING) - io_free_region(ctx->user, &bl->region); + io_free_region(NULL, &bl->region); else io_remove_buffers_legacy(ctx, bl, -1U); @@ -675,7 +675,7 @@ int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg) rd.user_addr = reg.ring_addr; rd.flags |= IORING_MEM_REGION_TYPE_USER; } - ret = io_create_region(ctx, &bl->region, &rd, mmap_offset); + ret = io_create_region(NULL, &bl->region, &rd, mmap_offset); if (ret) goto fail; br = io_region_get_ptr(&bl->region); @@ -708,7 +708,7 @@ int io_register_pbuf_ring(struct io_ring_ctx *ctx, void __user *arg) if (!ret) return 0; fail: - io_free_region(ctx->user, &bl->region); + io_free_region(NULL, &bl->region); kfree(bl); return ret; } diff --git a/io_uring/memmap.c b/io_uring/memmap.c index 23e8a85111bc..d4d716946971 100644 --- a/io_uring/memmap.c +++ b/io_uring/memmap.c @@ -16,8 +16,10 @@ #include "zcrx.h" static bool io_mem_alloc_compound(struct page **pages, int nr_pages, - size_t size, gfp_t gfp) + size_t size, gfp_t gfp, + struct user_struct *user) { + unsigned long nr_compound, extra; struct page *page; int i, order; @@ -27,9 +29,22 @@ static bool io_mem_alloc_compound(struct page **pages, int nr_pages, else if (order) gfp |= __GFP_COMP; + /* + * get_order() rounds a non power of two size up, so the allocation + * can hold more pages than the region exposes. Account those too, + * and leave the compound allocation alone if they do not fit. + */ + nr_compound = 1UL << order; + extra = nr_compound - nr_pages; + if (extra && user && __io_account_mem(user, extra)) + return false; + page = alloc_pages(gfp, order); - if (!page) + if (!page) { + if (extra && user) + __io_unaccount_mem(user, extra); return false; + } for (i = 0; i < nr_pages; i++) pages[i] = page + i; @@ -105,8 +120,15 @@ void io_free_region(struct user_struct *user, struct io_mapped_region *mr) } if ((mr->flags & IO_REGION_F_VMAP) && mr->ptr) vunmap(mr->ptr); - if (mr->nr_pages && user) - __io_unaccount_mem(user, mr->nr_pages); + if (mr->nr_pages && user) { + unsigned long nr_accounted = mr->nr_pages; + + /* a compound region was accounted for the whole allocation */ + if (mr->flags & IO_REGION_F_SINGLE_REF) + nr_accounted = 1UL << get_order(io_region_size(mr)); + + __io_unaccount_mem(user, nr_accounted); + } memset(mr, 0, sizeof(*mr)); } @@ -151,7 +173,8 @@ static int io_region_pin_pages(struct io_mapped_region *mr, static int io_region_allocate_pages(struct io_mapped_region *mr, struct io_uring_region_desc *reg, - unsigned long mmap_offset) + unsigned long mmap_offset, + struct user_struct *user) { gfp_t gfp = GFP_KERNEL_ACCOUNT | __GFP_ZERO | __GFP_NOWARN; size_t size = io_region_size(mr); @@ -162,7 +185,7 @@ static int io_region_allocate_pages(struct io_mapped_region *mr, if (!pages) return -ENOMEM; - if (io_mem_alloc_compound(pages, mr->nr_pages, size, gfp)) { + if (io_mem_alloc_compound(pages, mr->nr_pages, size, gfp, user)) { mr->flags |= IO_REGION_F_SINGLE_REF; goto done; } @@ -181,7 +204,7 @@ static int io_region_allocate_pages(struct io_mapped_region *mr, return 0; } -int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr, +int io_create_region(struct user_struct *user, struct io_mapped_region *mr, struct io_uring_region_desc *reg, unsigned long mmap_offset) { @@ -207,8 +230,8 @@ int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr, return -EOVERFLOW; nr_pages = reg->size >> PAGE_SHIFT; - if (ctx->user) { - ret = __io_account_mem(ctx->user, nr_pages); + if (user) { + ret = __io_account_mem(user, nr_pages); if (ret) return ret; } @@ -217,7 +240,7 @@ int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr, if (reg->flags & IORING_MEM_REGION_TYPE_USER) ret = io_region_pin_pages(mr, reg); else - ret = io_region_allocate_pages(mr, reg, mmap_offset); + ret = io_region_allocate_pages(mr, reg, mmap_offset, user); if (ret) goto out_free; @@ -226,7 +249,7 @@ int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr, goto out_free; return 0; out_free: - io_free_region(ctx->user, mr); + io_free_region(user, mr); return ret; } diff --git a/io_uring/memmap.h b/io_uring/memmap.h index f4cfbb6b9a1f..0714bb5a9616 100644 --- a/io_uring/memmap.h +++ b/io_uring/memmap.h @@ -18,7 +18,7 @@ unsigned long io_uring_get_unmapped_area(struct file *file, unsigned long addr, int io_uring_mmap(struct file *file, struct vm_area_struct *vma); void io_free_region(struct user_struct *user, struct io_mapped_region *mr); -int io_create_region(struct io_ring_ctx *ctx, struct io_mapped_region *mr, +int io_create_region(struct user_struct *user, struct io_mapped_region *mr, struct io_uring_region_desc *reg, unsigned long mmap_offset); diff --git a/io_uring/net.c b/io_uring/net.c index e6847fb084ab..37fc7baa55b2 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -1772,7 +1773,8 @@ void io_connect_bpf_populate(struct io_uring_bpf_ctx *bctx, struct io_kiocb *req case AF_INET6: { struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)ss; - if (conn->addr_len < (int)sizeof(*sin6)) + /* The IPv6 stack accepts SIN6_LEN_RFC2133; port and addr are in it. */ + if (conn->addr_len < SIN6_LEN_RFC2133) break; bctx->connect.port = sin6->sin6_port; memcpy(bctx->connect.v6_addr, &sin6->sin6_addr, diff --git a/io_uring/register.c b/io_uring/register.c index 02bc103bcc9d..dab3eb4258df 100644 --- a/io_uring/register.c +++ b/io_uring/register.c @@ -480,8 +480,9 @@ struct io_ring_ctx_rings { static void io_register_free_rings(struct io_ring_ctx *ctx, struct io_ring_ctx_rings *r) { - io_free_region(ctx->user, &r->sq_region); - io_free_region(ctx->user, &r->ring_region); + /* ring memory is not charged to RLIMIT_MEMLOCK, hence no user */ + io_free_region(NULL, &r->sq_region); + io_free_region(NULL, &r->ring_region); } #define swap_old(ctx, o, n, field) \ @@ -529,7 +530,7 @@ static int io_register_resize_rings(struct io_ring_ctx *ctx, void __user *arg) rd.user_addr = p->cq_off.user_addr; rd.flags |= IORING_MEM_REGION_TYPE_USER; } - ret = io_create_region(ctx, &n.ring_region, &rd, IORING_OFF_CQ_RING); + ret = io_create_region(NULL, &n.ring_region, &rd, IORING_OFF_CQ_RING); if (ret) return ret; @@ -559,7 +560,7 @@ static int io_register_resize_rings(struct io_ring_ctx *ctx, void __user *arg) rd.user_addr = p->sq_off.user_addr; rd.flags |= IORING_MEM_REGION_TYPE_USER; } - ret = io_create_region(ctx, &n.sq_region, &rd, IORING_OFF_SQES); + ret = io_create_region(NULL, &n.sq_region, &rd, IORING_OFF_SQES); if (ret) { io_register_free_rings(ctx, &n); return ret; @@ -730,7 +731,8 @@ static int io_register_mem_region(struct io_ring_ctx *ctx, void __user *uarg) !(ctx->flags & IORING_SETUP_R_DISABLED)) return -EINVAL; - ret = io_create_region(ctx, ®ion, &rd, IORING_MAP_OFF_PARAM_REGION); + ret = io_create_region(ctx->user, ®ion, &rd, + IORING_MAP_OFF_PARAM_REGION); if (ret) return ret; if (copy_to_user(rd_uptr, &rd, sizeof(rd))) { @@ -764,7 +766,14 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, if (ctx->submitter_task && ctx->submitter_task != current) return -EEXIST; - if ((ctx->int_flags & IO_RING_F_REG_RESTRICTED) && !(ctx->flags & IORING_SETUP_R_DISABLED)) { + /* + * A disabled ring is exempt from its own register restrictions, so + * that the creator can set them up and then enable the ring. That + * does not apply to restrictions inherited from the task. + */ + if ((ctx->int_flags & IO_RING_F_REG_RESTRICTED) && + (!(ctx->flags & IORING_SETUP_R_DISABLED) || + (ctx->int_flags & IO_RING_F_RESTRICT_INHERITED))) { opcode = array_index_nospec(opcode, IORING_REGISTER_LAST); if (!test_bit(opcode, ctx->restrictions.register_op)) return -EACCES; diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c index 1b6832f6e10f..0a9e8a58b99a 100644 --- a/io_uring/zcrx.c +++ b/io_uring/zcrx.c @@ -398,7 +398,7 @@ static int io_allocate_rbuf_ring(struct io_ring_ctx *ctx, mmap_offset = IORING_MAP_OFF_ZCRX_REGION; mmap_offset += (u64)id << IORING_OFF_ZCRX_SHIFT; - ret = io_create_region(ctx, &ifq->rq_region, rd, mmap_offset); + ret = io_create_region(ctx->user, &ifq->rq_region, rd, mmap_offset); if (ret < 0) return ret; diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index c43f15c9917c..501f12de533f 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -151,6 +151,12 @@ static cpumask_var_t isolated_cpus; /* CSCB */ */ static bool update_housekeeping; /* RWCS */ +/* + * Set if "cpuset_v2_mode" mount option is used + * Cached at bind time and not lock protected; accessed via {READ,WRITE}_ONCE + */ +static bool cpuset_v2_mode; + /* * Copy of isolated_cpus to be passed to housekeeping_update() */ @@ -403,8 +409,7 @@ static inline bool cpuset_v2(void) */ static inline bool is_in_v2_mode(void) { - return cpuset_v2() || - (cpuset_cgrp_subsys.root->flags & CGRP_ROOT_CPUSET_V2_MODE); + return cpuset_v2() || READ_ONCE(cpuset_v2_mode); } /** @@ -478,10 +483,26 @@ static void guarantee_active_cpus(struct task_struct *tsk, rcu_read_lock(); cs = task_cs(tsk); - while (!cpumask_intersects(cs->effective_cpus, pmask)) + while (!cpumask_intersects(cs->effective_cpus, pmask)) { cs = parent_cs(cs); - + if (unlikely(!cs)) { + /* + * The top cpuset doesn't have any active cpu as a + * consequence of a race between its caller and the cpu + * hotplug operation where cpu_active_mask is updated + * asynchronously before cpuset_handle_hotplug() is + * being called to adjust the effective_cpus of the + * affected cpusets. But we know the top cpuset's + * effective_cpus is on its way to be identical to + * cpu_active_mask minus the exclusive CPUs dedicated + * to other valid cpuset partitions. Just pass back + * the filtered cpu_active_mask in this case. + */ + goto out_unlock; + } + } cpumask_and(pmask, pmask, cs->effective_cpus); +out_unlock: rcu_read_unlock(); } @@ -3586,6 +3607,8 @@ static void cpuset_bind(struct cgroup_subsys_state *root_css) mutex_lock(&cpuset_mutex); spin_lock_irq(&callback_lock); + WRITE_ONCE(cpuset_v2_mode, + !!(cpuset_cgrp_subsys.root->flags & CGRP_ROOT_CPUSET_V2_MODE)); if (is_in_v2_mode()) { cpumask_copy(top_cpuset.cpus_allowed, cpu_possible_mask); cpumask_copy(top_cpuset.effective_xcpus, cpu_possible_mask); @@ -3902,15 +3925,16 @@ static void cpuset_handle_hotplug(void) static cpumask_t new_cpus; static nodemask_t new_mems; bool cpus_updated, mems_updated; - bool on_dfl = is_in_v2_mode(); + bool on_dfl; struct tmpmasks tmp, *ptmp = NULL; - if (on_dfl && !alloc_tmpmasks(&tmp)) - ptmp = &tmp; - lockdep_assert_cpus_held(); mutex_lock(&cpuset_mutex); + on_dfl = is_in_v2_mode(); + if (on_dfl && !alloc_tmpmasks(&tmp)) + ptmp = &tmp; + /* fetch the available cpus/mems and find out which changed how */ cpumask_copy(&new_cpus, cpu_active_mask); new_mems = node_states[N_MEMORY]; diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index bd5393133753..08ded023328f 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -1680,10 +1680,10 @@ phys_addr_t default_swiotlb_base(void) phys_addr_t default_swiotlb_limit(void) { #ifdef CONFIG_SWIOTLB_DYNAMIC - return io_tlb_default_mem.phys_limit; -#else - return io_tlb_default_mem.defpool.end - 1; + if (io_tlb_default_mem.can_grow) + return io_tlb_default_mem.phys_limit; #endif + return io_tlb_default_mem.defpool.end - 1; } #ifdef CONFIG_DEBUG_FS diff --git a/kernel/fork.c b/kernel/fork.c index b22b5ba9a543..854448152696 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -3326,8 +3326,9 @@ int ksys_unshare(unsigned long unshare_flags) perf_event_namespaces(current); bad_unshare_cleanup_nsproxy: + /* never installed, so no active references to drop */ if (new_nsproxy) - put_nsproxy(new_nsproxy); + nsproxy_free(new_nsproxy); bad_unshare_cleanup_cred: if (new_cred) put_cred(new_cred); diff --git a/kernel/nsproxy.c b/kernel/nsproxy.c index d9d3d5973bf5..3fb1595a4a59 100644 --- a/kernel/nsproxy.c +++ b/kernel/nsproxy.c @@ -61,7 +61,7 @@ static inline struct nsproxy *create_nsproxy(void) return nsproxy; } -static inline void nsproxy_free(struct nsproxy *ns) +void nsproxy_free(struct nsproxy *ns) { put_mnt_ns(ns->mnt_ns); put_uts_ns(ns->uts_ns); diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index cd7cbd1c99a2..a9f33e2b17ac 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -1953,8 +1953,14 @@ static void do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags, if (unlikely(!SCX_HAS_OP(sch, enqueue))) goto global; - /* DSQ bypass didn't trigger, enqueue on the BPF scheduler */ - qseq = rq->scx.ops_qseq++ << SCX_OPSS_QSEQ_SHIFT; + /* + * DSQ bypass didn't trigger, enqueue on the BPF scheduler. Wrap the + * per-task qseq counter where the QSEQ field wraps and skip 0, which is + * what scx_bpf_dsq_insert() records for a task in NONE or DISPATCHING. + */ + p->scx.ops_qseq = ((p->scx.ops_qseq + 1) & + (SCX_OPSS_QSEQ_MASK >> SCX_OPSS_QSEQ_SHIFT)) ?: 1; + qseq = (unsigned long)p->scx.ops_qseq << SCX_OPSS_QSEQ_SHIFT; WARN_ON_ONCE(atomic_long_read(&p->scx.ops_state) != SCX_OPSS_NONE); atomic_long_set(&p->scx.ops_state, SCX_OPSS_QUEUEING | qseq); @@ -2044,6 +2050,13 @@ static void enqueue_task_scx(struct rq *rq, struct task_struct *p, int core_enq_ int sticky_cpu = p->scx.sticky_cpu; u64 enq_flags = core_enq_flags | rq->scx.extra_enq_flags; + /* + * An SCX-internal migration ends on arrival. Clear sticky_cpu so @p can + * leave custody when inserted into the destination DSQ. + */ + if (sticky_cpu >= 0) + p->scx.sticky_cpu = -1; + /* * SCX_RQ_IN_WAKEUP promises a task_woken_scx() call once this enqueue * returns. Only the core's wakeup path delivers one. The flags stashed @@ -2082,9 +2095,6 @@ static void enqueue_task_scx(struct rq *rq, struct task_struct *p, int core_enq_ dl_server_start(&rq->ext_server); do_enqueue_task(rq, p, enq_flags, sticky_cpu); - - if (sticky_cpu >= 0) - p->scx.sticky_cpu = -1; out: rq->scx.flags &= ~SCX_RQ_IN_WAKEUP; @@ -6690,10 +6700,9 @@ static void scx_dump_cpu(struct scx_sched *sch, struct seq_buf *s, seq_buf_init(&ns, buf, avail); dump_newline(&ns); - dump_line(&ns, "CPU %-4d: nr_run=%u flags=0x%x cpu_rel=%d ops_qseq=%lu ksync=%lu", + dump_line(&ns, "CPU %-4d: nr_run=%u flags=0x%x cpu_rel=%d ksync=%lu", cpu, rq->scx.nr_running, rq->scx.flags, - rq->scx.cpu_released, rq->scx.ops_qseq, - rq->scx.kick_sync); + rq->scx.cpu_released, rq->scx.kick_sync); dump_line(&ns, " curr=%s[%d] class=%ps", rq->curr->comm, rq->curr->pid, rq->curr->sched_class); diff --git a/kernel/sched/ext/internal.h b/kernel/sched/ext/internal.h index b295991b9f10..3cd223c6d763 100644 --- a/kernel/sched/ext/internal.h +++ b/kernel/sched/ext/internal.h @@ -1508,6 +1508,11 @@ enum scx_ops_state { * dequeue/requeue, the dispatcher can tell whether it still has a claim * on the task being dispatched. * + * QSEQ is generated from the per-task p->scx.ops_qseq counter so that + * it doesn't repeat across QUEUED instances of the same task even if + * the task moves between rqs. 0 is never used as a valid QSEQ since + * NONE and DISPATCHING map to this value. + * * As some 32bit archs can't do 64bit store_release/load_acquire, * p->scx.ops_state is atomic_long_t which leaves 30 bits for QSEQ on * 32bit machines. The dispatch race window QSEQ protects is very narrow diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index f76e6f7f4a01..a6c6f8165eec 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -795,7 +795,6 @@ struct scx_rq { struct scx_dispatch_q local_dsq; struct list_head runnable_list; /* runnable tasks on this rq */ struct list_head ddsp_deferred_locals; /* deferred ddsps from enq */ - unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ u32 nr_running; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 53608ac01fab..cb7055aee3af 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2244,7 +2244,7 @@ static bool is_chained_work(struct workqueue_struct *wq) * Return %true iff I'm a worker executing a work item on @wq. If * I'm @worker, it's safe to dereference it without locking. */ - return worker && worker->current_pwq->wq == wq; + return worker && worker->current_pwq && worker->current_pwq->wq == wq; } /* diff --git a/lib/dim/dim.c b/lib/dim/dim.c index 97c3d084ebf0..7f3eebae73cb 100644 --- a/lib/dim/dim.c +++ b/lib/dim/dim.c @@ -69,11 +69,15 @@ bool dim_calc_stats(const struct dim_sample *start, if (!delta_us) return false; - curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us); - curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us); + /* u32 * USEC_PER_MSEC overflows a 32-bit long */ + curr_stats->ppms = DIV_ROUND_UP_ULL((u64)npkts * USEC_PER_MSEC, + delta_us); + curr_stats->bpms = DIV_ROUND_UP_ULL((u64)nbytes * USEC_PER_MSEC, + delta_us); curr_stats->epms = DIV_ROUND_UP(DIM_NEVENTS * USEC_PER_MSEC, delta_us); - curr_stats->cpms = DIV_ROUND_UP(ncomps * USEC_PER_MSEC, delta_us); + curr_stats->cpms = DIV_ROUND_UP_ULL((u64)ncomps * USEC_PER_MSEC, + delta_us); if (curr_stats->epms != 0) curr_stats->cpe_ratio = DIV_ROUND_DOWN_ULL( curr_stats->cpms * 100, curr_stats->epms); diff --git a/mm/mremap.c b/mm/mremap.c index ecd926209560..ff1cf59b26c7 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -1029,6 +1029,7 @@ static void vrm_stat_account(struct vma_remap_struct *vrm, } static bool __check_map_count_against_split(struct mm_struct *mm, + bool pre_split, bool before_unmaps) { const int sys_map_count = get_sysctl_max_map_count(); @@ -1080,26 +1081,42 @@ static bool __check_map_count_against_split(struct mm_struct *mm, * Therefore we must check to ensure we have headroom of 2 additional * VMAs. */ - return map_count + 2 <= sys_map_count; + map_count += 2; + + /* If pre-split, the -1 observed above doesn't apply. */ + if (pre_split) + map_count++; + + return map_count <= sys_map_count; +} + +static bool needs_pre_split(struct vma_remap_struct *vrm) +{ + /* + * An MREMAP_DONTUNMAP of a mlock()'d VMA needs to unlock the + * source VMA, so split in this case. + */ + return (vrm->flags & MREMAP_DONTUNMAP) && + vma_test(vrm->vma, VMA_LOCKED_BIT); } /* Do we violate the map count limit if we split VMAs when moving the VMA? */ -static bool check_map_count_against_split(void) +static bool check_map_count_against_split(struct vma_remap_struct *vrm) { return __check_map_count_against_split(current->mm, - /*before_unmaps=*/false); + needs_pre_split(vrm), /*before_unmaps=*/false); } /* Do we violate the map count limit if we split VMAs prior to early unmaps? */ -static bool check_map_count_against_split_early(void) +static bool check_map_count_against_split_early(struct vma_remap_struct *vrm) { return __check_map_count_against_split(current->mm, - /*before_unmaps=*/true); + vrm->flags & MREMAP_DONTUNMAP, /*before_unmaps=*/true); } /* - * Perform checks before attempting to write a VMA prior to it being - * moved. + * Perform checks and preparation before attempting to write a VMA prior to it + * being moved. */ static unsigned long prep_move_vma(struct vma_remap_struct *vrm) { @@ -1108,19 +1125,17 @@ static unsigned long prep_move_vma(struct vma_remap_struct *vrm) unsigned long old_addr = vrm->addr; unsigned long old_len = vrm->old_len; vm_flags_t dummy = vma->vm_flags; + const bool split_before = vma->vm_start != old_addr; + const bool split_after = vma->vm_end != old_addr + old_len; - /* - * We'd prefer to avoid failure later on in do_munmap: we copy a VMA, - * which may not merge, then (if MREMAP_DONTUNMAP is not set) unmap the - * source, which may split, causing a net increase of 2 mappings. - */ - if (!check_map_count_against_split()) + /* Avoid failure later on. */ + if (!check_map_count_against_split(vrm)) return -ENOMEM; if (vma->vm_ops && vma->vm_ops->may_split) { - if (vma->vm_start != old_addr) + if (split_before) err = vma->vm_ops->may_split(vma, old_addr); - if (!err && vma->vm_end != old_addr + old_len) + if (!err && split_after) err = vma->vm_ops->may_split(vma, old_addr + old_len); if (err) return err; @@ -1138,6 +1153,21 @@ static unsigned long prep_move_vma(struct vma_remap_struct *vrm) if (err) return err; + /* + * To account mlock()'d pages correctly in the MREMAP_DONTUNMAP + * case perform any split ahead of time for an mlock()'d VMA. + */ + if (needs_pre_split(vrm)) { + VMA_ITERATOR(vmi, vma->vm_mm, old_addr); + + if (split_before) + err = split_vma(&vmi, vma, old_addr, 1); + if (!err && split_after) + err = split_vma(&vmi, vma, old_addr + old_len, 0); + vrm->vmi_needs_invalidate = true; + return err; + } + return 0; } @@ -1265,7 +1295,8 @@ static int copy_vma_and_data(struct vma_remap_struct *vrm, PAGETABLE_MOVE(pmc, NULL, NULL, vrm->addr, vrm->new_addr, vrm->old_len); new_vma = copy_vma(&vma, vrm->new_addr, vrm->new_len, new_pgoff, - &pmc.need_rmap_locks); + &pmc.need_rmap_locks, + vrm->flags & MREMAP_DONTUNMAP); if (!new_vma) { vrm_uncharge(vrm); *new_vma_ptr = NULL; @@ -1325,6 +1356,9 @@ static void dontunmap_complete(struct vma_remap_struct *vrm, unsigned long old_start = vma->vm_start; unsigned long old_end = vma->vm_end; + /* Self-merge is disallowed. */ + VM_WARN_ON_ONCE(new_vma == vma); + /* We always clear VM_LOCKED[ONFAULT] on the old VMA. */ vm_flags_clear(vma, VM_LOCKED_MASK); @@ -1332,7 +1366,7 @@ static void dontunmap_complete(struct vma_remap_struct *vrm, * anon_vma links of the old vma is no longer needed after its page * table has been moved. */ - if (new_vma != vma && start == old_start && end == old_end) { + if (start == old_start && end == old_end) { const pgoff_t pgoff_unfaulted = vma->vm_start >> PAGE_SHIFT; unlink_anon_vmas(vma); @@ -1992,7 +2026,7 @@ static unsigned long do_mremap(struct vma_remap_struct *vrm) return -EINTR; vrm->mmap_locked = true; - if (!check_map_count_against_split_early()) { + if (!check_map_count_against_split_early(vrm)) { mmap_write_unlock(mm); return -ENOMEM; } diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index b91b1a98029c..cd227fc05d2d 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -438,12 +438,30 @@ static void kernel_pgtable_work_func(struct work_struct *work) __pagetable_free(pt); } +static void schedule_kernel_pgtable_free(void) +{ + schedule_work(&kernel_pgtable_work.work); +} + void pagetable_free_kernel(struct ptdesc *pt) { spin_lock(&kernel_pgtable_work.lock); list_add(&pt->pt_list, &kernel_pgtable_work.list); spin_unlock(&kernel_pgtable_work.lock); - schedule_work(&kernel_pgtable_work.work); + /* + * The workqueue may not exist yet while the system is booting. + * kernel_pgtable_drain_early() schedules the work once it does. + */ + if (system_state != SYSTEM_BOOTING) + schedule_kernel_pgtable_free(); +} + +static int __init kernel_pgtable_drain_early(void) +{ + /* Free the kernel page tables queued while booting. */ + schedule_kernel_pgtable_free(); + return 0; } +core_initcall(kernel_pgtable_drain_early); #endif diff --git a/mm/vma.c b/mm/vma.c index 8f7f77a4c7ce..07881c0987cf 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -590,8 +590,8 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, * Split a vma into two pieces at address 'addr', a new vma is allocated * either for the first part or the tail. */ -static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long addr, int new_below) +int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, int new_below) { if (vma->vm_mm->map_count >= get_sysctl_max_map_count()) return -ENOMEM; @@ -1863,7 +1863,7 @@ static int vma_link(struct mm_struct *mm, struct vm_area_struct *vma) */ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, unsigned long addr, unsigned long len, pgoff_t pgoff, - bool *need_rmap_locks) + bool *need_rmap_locks, bool keep_source) { struct vm_area_struct *vma = *vmap; unsigned long vma_start = vma->vm_start; @@ -1896,6 +1896,21 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, vmg.pgoff = pgoff; vmg.next = vma_iter_next_rewind(&vmi, NULL); + + /* + * If the original VMA is kept (MREMAP_DONTUNMAP), the source and + * destination VMA must be treated distinctly. + * + * A merge violates this, so in this case disallow a self-merge. + */ + if (!faulted_in_anon_vma && keep_source) { + if (vmg.prev == vma) + vmg.prev = NULL; + if (vmg.next == vma) + vmg.next = NULL; + faulted_in_anon_vma = true; + } + new_vma = vma_merge_copied_range(&vmg); if (new_vma) { diff --git a/mm/vma.h b/mm/vma.h index 8e4b61a7304c..3d2fa4f0d777 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -449,7 +449,7 @@ void unlink_file_vma_batch_add(struct unlink_vma_file_batch *vb, struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, unsigned long addr, unsigned long len, pgoff_t pgoff, - bool *need_rmap_locks); + bool *need_rmap_locks, bool keep_source); struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *vma); @@ -470,6 +470,11 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *brkvma, unsigned long unmapped_area(struct vm_unmapped_area_info *info); unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info); +#ifdef CONFIG_MMU +int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, int new_below); +#endif + static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma) { /* diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c index aea017ee5b0a..b19e68844e1a 100644 --- a/net/bluetooth/hci_core.c +++ b/net/bluetooth/hci_core.c @@ -4113,9 +4113,11 @@ static int hci_send_cmd_sync(struct hci_dev *hdev, struct sk_buff *skb) bt_dev_dbg(hdev, "skb %p", skb); + hci_dev_lock(hdev); kfree_skb(hdev->sent_cmd); hdev->sent_cmd = skb_clone(skb, GFP_KERNEL); + hci_dev_unlock(hdev); if (!hdev->sent_cmd) { skb_queue_head(&hdev->cmd_q, skb); queue_work(hdev->workqueue, &hdev->cmd_work); @@ -4136,8 +4138,10 @@ static int hci_send_cmd_sync(struct hci_dev *hdev, struct sk_buff *skb) if (READ_ONCE(hdev->req_status) == HCI_REQ_PEND && !hci_dev_test_and_set_flag(hdev, HCI_CMD_PENDING)) { + hci_dev_lock(hdev); kfree_skb(hdev->req_skb); hdev->req_skb = skb_get(hdev->sent_cmd); + hci_dev_unlock(hdev); } return err; diff --git a/net/bluetooth/hci_sock.c b/net/bluetooth/hci_sock.c index 6d56c77741e1..81068b585764 100644 --- a/net/bluetooth/hci_sock.c +++ b/net/bluetooth/hci_sock.c @@ -1223,8 +1223,10 @@ static int hci_sock_bind(struct socket *sock, struct sockaddr_unsized *addr, */ hdev = hci_pi(sk)->hdev; if (hdev && hci_dev_test_flag(hdev, HCI_UNREGISTER)) { + write_lock(&hci_sk_list.lock); hci_pi(sk)->hdev = NULL; sk->sk_state = BT_OPEN; + write_unlock(&hci_sk_list.lock); hci_dev_put(hdev); } hdev = NULL; diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c index 17882feeeeee..110135848bd6 100644 --- a/net/bluetooth/hci_sync.c +++ b/net/bluetooth/hci_sync.c @@ -32,8 +32,10 @@ static void hci_cmd_sync_complete(struct hci_dev *hdev, u8 result, u16 opcode, WRITE_ONCE(hdev->req_status, HCI_REQ_DONE); /* Free the request command so it is not used as response */ + hci_dev_lock(hdev); kfree_skb(hdev->req_skb); hdev->req_skb = NULL; + hci_dev_unlock(hdev); if (skb) { struct sock *sk = hci_skb_sk(skb); @@ -3060,15 +3062,21 @@ static int hci_le_set_ext_scan_param_sync(struct hci_dev *hdev, u8 type, */ if (hci_dev_test_flag(hdev, HCI_PA_SYNC)) { struct hci_cp_le_add_to_accept_list *sent; + bdaddr_t bdaddr; + hci_dev_lock(hdev); sent = hci_sent_cmd_data(hdev, HCI_OP_LE_ADD_TO_ACCEPT_LIST); + if (sent) + bacpy(&bdaddr, &sent->bdaddr); + hci_dev_unlock(hdev); + if (sent) { struct hci_conn *conn; rcu_read_lock(); conn = hci_conn_hash_lookup_ba(hdev, PA_LINK, - &sent->bdaddr); + &bdaddr); if (conn) { struct bt_iso_qos *qos = &conn->iso_qos; diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c index 7657c2a0abbf..d6ac5b1f49bc 100644 --- a/net/bluetooth/iso.c +++ b/net/bluetooth/iso.c @@ -61,6 +61,7 @@ enum { BT_SK_BIG_SYNC, BT_SK_PA_SYNC, BT_SK_KILLED, + BT_SK_CONNECTING, }; struct iso_pinfo { @@ -350,6 +351,9 @@ static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, return -EBUSY; } + if (iso_pi(sk)->conn) + return -EISCONN; + if (!conn->hcon) { BT_ERR("conn->hcon missing"); return -EIO; @@ -410,6 +414,11 @@ static int iso_connect_bis(struct sock *sk) hci_dev_lock(hdev); lock_sock(sk); + if (iso_pi(sk)->conn) { + err = -EISCONN; + goto unlock; + } + if (!bis_capable(hdev)) { err = -EOPNOTSUPP; goto unlock; @@ -562,6 +571,13 @@ static int iso_connect_cis(struct sock *sk) lockdep_assert_held(&hcon->hdev->lock); + /* The socket lock keeps the current attachment and its hcon stable. */ + if (iso_pi(sk)->conn && iso_pi(sk)->conn->hcon != hcon) { + hci_conn_drop(hcon); + err = -EISCONN; + goto unlock; + } + conn = iso_conn_add(hcon); if (!conn) { hci_conn_drop(hcon); @@ -1269,17 +1285,26 @@ static int iso_sock_connect(struct socket *sock, struct sockaddr_unsized *addr, addr->sa_family != AF_BLUETOOTH) return -EINVAL; - if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) - return -EBADFD; + lock_sock(sk); - if (sk->sk_type != SOCK_SEQPACKET) - return -EINVAL; + if ((sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) || + test_bit(BT_SK_CONNECTING, &iso_pi(sk)->flags)) { + err = -EBADFD; + goto done; + } + + if (sk->sk_type != SOCK_SEQPACKET) { + err = -EINVAL; + goto done; + } /* Check if the address type is of LE type */ - if (!bdaddr_type_is_le(sa->iso_bdaddr_type)) - return -EINVAL; + if (!bdaddr_type_is_le(sa->iso_bdaddr_type)) { + err = -EINVAL; + goto done; + } - lock_sock(sk); + set_bit(BT_SK_CONNECTING, &iso_pi(sk)->flags); bacpy(&iso_pi(sk)->dst, &sa->iso_bdaddr); iso_pi(sk)->dst_type = sa->iso_bdaddr_type; @@ -1291,16 +1316,18 @@ static int iso_sock_connect(struct socket *sock, struct sockaddr_unsized *addr, else err = iso_connect_bis(sk); - if (err) - return err; - lock_sock(sk); + clear_bit(BT_SK_CONNECTING, &iso_pi(sk)->flags); + if (err) + goto done; + if (!test_bit(BT_SK_DEFER_SETUP, &bt_sk(sk)->flags)) { err = bt_sock_wait_state(sk, BT_CONNECTED, sock_sndtimeo(sk, flags & O_NONBLOCK)); } +done: release_sock(sk); return err; } @@ -1340,6 +1367,11 @@ static int iso_listen_bis(struct sock *sk) hci_dev_lock(hdev); lock_sock(sk); + if (sk->sk_state != BT_BOUND || iso_pi(sk)->conn) { + err = -EBADFD; + goto unlock; + } + /* Fail if user set invalid QoS */ if (iso_pi(sk)->qos_user_set && !check_bcast_qos(&iso_pi(sk)->qos)) { iso_pi(sk)->qos = default_qos; diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c index 2a177cd17a9e..7928d9e45f98 100644 --- a/net/bluetooth/mgmt.c +++ b/net/bluetooth/mgmt.c @@ -1489,7 +1489,7 @@ static void cmd_complete_rsp(struct mgmt_pending_cmd *cmd, void *data) return; } - cmd_status_rsp(cmd, data); + cmd_status_rsp(cmd, &match->mgmt_status); } static int generic_cmd_complete(struct mgmt_pending_cmd *cmd, u8 status) diff --git a/net/bluetooth/rfcomm/core.c b/net/bluetooth/rfcomm/core.c index 3f757198051a..7a7791fe56b0 100644 --- a/net/bluetooth/rfcomm/core.c +++ b/net/bluetooth/rfcomm/core.c @@ -62,10 +62,9 @@ static void rfcomm_make_uih(struct sk_buff *skb, u8 addr); static void rfcomm_process_connect(struct rfcomm_session *s); -static struct rfcomm_session *rfcomm_session_create(bdaddr_t *src, - bdaddr_t *dst, - u8 sec_level, - int *err); +static struct socket *rfcomm_session_connect(bdaddr_t *src, bdaddr_t *dst, + u8 sec_level, int *err); +static struct rfcomm_session *rfcomm_session_add(struct socket *sock, int state); static struct rfcomm_session *rfcomm_session_get(bdaddr_t *src, bdaddr_t *dst); static struct rfcomm_session *rfcomm_session_del(struct rfcomm_session *s); @@ -365,28 +364,17 @@ static int rfcomm_check_channel(u8 channel) return channel < 1 || channel > 30; } -static int __rfcomm_dlc_open(struct rfcomm_dlc *d, bdaddr_t *src, bdaddr_t *dst, u8 channel) +static int __rfcomm_dlc_open(struct rfcomm_dlc *d, struct rfcomm_session *s, + u8 channel) { - struct rfcomm_session *s; - int err = 0; u8 dlci; - BT_DBG("dlc %p state %ld %pMR -> %pMR channel %d", - d, d->state, src, dst, channel); - - if (rfcomm_check_channel(channel)) - return -EINVAL; + BT_DBG("dlc %p state %ld session %p channel %d", + d, d->state, s, channel); if (d->state != BT_OPEN && d->state != BT_CLOSED) return 0; - s = rfcomm_session_get(src, dst); - if (!s) { - s = rfcomm_session_create(src, dst, d->sec_level, &err); - if (!s) - return err; - } - dlci = __dlci(__session_dir(s), channel); /* Check if DLCI already exists */ @@ -421,14 +409,84 @@ static int __rfcomm_dlc_open(struct rfcomm_dlc *d, bdaddr_t *src, bdaddr_t *dst, int rfcomm_dlc_open(struct rfcomm_dlc *d, bdaddr_t *src, bdaddr_t *dst, u8 channel) { - int r; + struct rfcomm_session *s; + struct socket *sock; + int err; + + BT_DBG("dlc %p state %ld %pMR -> %pMR channel %d", + d, d->state, src, dst, channel); + + if (rfcomm_check_channel(channel)) + return -EINVAL; rfcomm_lock(); - r = __rfcomm_dlc_open(d, src, dst, channel); + clear_bit(RFCOMM_CLOSED, &d->flags); + /* Do not page the remote device for a DLC that cannot be opened + * anyway. __rfcomm_dlc_open() looks at the state again once the + * lock has been re-acquired below. + */ + if (d->state != BT_OPEN && d->state != BT_CLOSED) { + rfcomm_unlock(); + return 0; + } + + s = rfcomm_session_get(src, dst); + if (s) { + err = __rfcomm_dlc_open(d, s, channel); + rfcomm_unlock(); + return err; + } rfcomm_unlock(); - return r; + + /* There is no session for this pair yet. kernel_connect() ends up in + * l2cap_chan_connect(), which takes hdev->lock, and the HCI event + * path takes rfcomm_mutex while holding hdev->lock, so the socket has + * to be connected with rfcomm_mutex released. + */ + sock = rfcomm_session_connect(src, dst, d->sec_level, &err); + if (!sock) + return err; + + rfcomm_lock(); + + /* The DLC may have been closed while the socket was connecting. It + * was not on a session, so rfcomm_dlc_close() could only mark it; + * attaching it now would leave it with no owner. + */ + if (test_bit(RFCOMM_CLOSED, &d->flags)) { + rfcomm_unlock(); + sock_release(sock); + return -ECONNRESET; + } + + /* Another opener may have added a session for the same pair in the + * meantime; that one is used and this socket is dropped. + */ + s = rfcomm_session_get(src, dst); + if (!s) { + s = rfcomm_session_add(sock, BT_BOUND); + if (s) { + s->initiator = 1; + sock = NULL; + } + } + + err = s ? __rfcomm_dlc_open(d, s, channel) : -ENOMEM; + + rfcomm_unlock(); + + if (sock) + sock_release(sock); + + /* Over an existing ACL link the connection can complete before the + * session reaches the list, and that wakeup is then lost, so let + * krfcommd look at the socket state now. + */ + rfcomm_schedule(); + + return err; } static void __rfcomm_dlc_disconn(struct rfcomm_dlc *d) @@ -508,8 +566,14 @@ int rfcomm_dlc_close(struct rfcomm_dlc *d, int err) rfcomm_lock(); s = d->session; - if (!s) + if (!s) { + /* Not on a session yet: rfcomm_dlc_open() may be connecting a + * socket for it with rfcomm_mutex released. Leave a mark so + * that it does not attach the DLC once it holds the lock again. + */ + set_bit(RFCOMM_CLOSED, &d->flags); goto no_session; + } /* after waiting on the mutex check the session still exists * then check the dlc still exists @@ -757,12 +821,12 @@ static struct rfcomm_session *rfcomm_session_close(struct rfcomm_session *s, return rfcomm_session_del(s); } -static struct rfcomm_session *rfcomm_session_create(bdaddr_t *src, - bdaddr_t *dst, - u8 sec_level, - int *err) +/* Creates the L2CAP socket a new session will run on and starts connecting + * it. Must be called with rfcomm_mutex released. + */ +static struct socket *rfcomm_session_connect(bdaddr_t *src, bdaddr_t *dst, + u8 sec_level, int *err) { - struct rfcomm_session *s = NULL; struct sockaddr_l2 addr; struct socket *sock; struct sock *sk; @@ -792,24 +856,16 @@ static struct rfcomm_session *rfcomm_session_create(bdaddr_t *src, l2cap_pi(sk)->chan->mode = L2CAP_MODE_ERTM; release_sock(sk); - s = rfcomm_session_add(sock, BT_BOUND); - if (!s) { - *err = -ENOMEM; - goto failed; - } - - s->initiator = 1; - bacpy(&addr.l2_bdaddr, dst); addr.l2_family = AF_BLUETOOTH; addr.l2_psm = cpu_to_le16(L2CAP_PSM_RFCOMM); addr.l2_cid = 0; addr.l2_bdaddr_type = BDADDR_BREDR; *err = kernel_connect(sock, (struct sockaddr_unsized *)&addr, sizeof(addr), O_NONBLOCK); - if (*err == 0 || *err == -EINPROGRESS) - return s; + if (*err && *err != -EINPROGRESS) + goto failed; - return rfcomm_session_del(s); + return sock; failed: sock_release(sock); diff --git a/net/bluetooth/rfcomm/sock.c b/net/bluetooth/rfcomm/sock.c index d2855f9b3abc..0a6ba8ca230f 100644 --- a/net/bluetooth/rfcomm/sock.c +++ b/net/bluetooth/rfcomm/sock.c @@ -48,8 +48,11 @@ static void rfcomm_sock_kill(struct sock *sk); static void rfcomm_sk_data_ready(struct rfcomm_dlc *d, struct sk_buff *skb) { struct sock *sk = d->owner; - if (!sk) + + if (!sk) { + kfree_skb(skb); return; + } atomic_add(skb->len, &sk->sk_rmem_alloc); skb_queue_tail(&sk->sk_receive_queue, skb); diff --git a/net/bluetooth/rfcomm/tty.c b/net/bluetooth/rfcomm/tty.c index dc3cdf614def..198c7dd1a95b 100644 --- a/net/bluetooth/rfcomm/tty.c +++ b/net/bluetooth/rfcomm/tty.c @@ -128,7 +128,7 @@ static void rfcomm_dev_shutdown(struct tty_port *port) { struct rfcomm_dev *dev = container_of(port, struct rfcomm_dev, port); - if (dev->tty_dev->parent) + if (dev->tty_dev && dev->tty_dev->parent) device_move(dev->tty_dev, NULL, DPM_ORDER_DEV_LAST); /* close the dlc */ diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c index 3d4362a09df4..f4a20d7b5f9c 100644 --- a/net/bluetooth/sco.c +++ b/net/bluetooth/sco.c @@ -84,12 +84,14 @@ static void sco_conn_free(struct kref *ref) if (conn->sk) sco_pi(conn->sk)->conn = NULL; - if (conn->hcon) { - conn->hcon->sco_data = NULL; - hci_conn_drop(conn->hcon); - } + /* hcon->sco_data is cleared and the association's reference on the + * sco_conn is dropped in sco_conn_del() under hdev->lock, and the + * hci_conn is now owned by the socket (held in __sco_chan_add() and + * dropped in sco_chan_del()/sco_sock_destruct()), so there is nothing + * left to release towards hcon here. + */ - /* Ensure no more work items will run since hci_conn has been dropped */ + /* Ensure no more work items will run before the connection is freed */ disable_delayed_work_sync(&conn->timeout_work); kfree(conn); @@ -188,25 +190,19 @@ static void sco_sock_clear_timer(struct sock *sk) } /* ---- SCO connections ---- */ -/* Consumes a reference on @hcon, which the returned sco_conn owns until it is - * freed. On failure (NULL return) the reference is left for the caller to drop. +/* Returns a new reference the caller must drop with sco_conn_put(). The + * hcon->sco_data association holds its own reference on the sco_conn for the + * connection's lifetime; it is dropped in sco_conn_del() under hdev->lock. + * @hcon is not consumed: the hci_conn reference is taken and owned by the + * socket in __sco_chan_add(). */ static struct sco_conn *sco_conn_add(struct hci_conn *hcon) { struct sco_conn *conn = hcon->sco_data; conn = sco_conn_hold_unless_zero(conn); - if (conn) { - if (!conn->hcon) { - sco_conn_lock(conn); - conn->hcon = hcon; - sco_conn_unlock(conn); - } else { - /* conn already owns a reference on hcon */ - hci_conn_drop(hcon); - } + if (conn) return conn; - } conn = kzalloc_obj(struct sco_conn); if (!conn) @@ -227,7 +223,10 @@ static struct sco_conn *sco_conn_add(struct hci_conn *hcon) BT_DBG("hcon %p conn %p", hcon, conn); - return conn; + /* kref_init() above set the association reference owned by + * hcon->sco_data; hand the caller its own reference. + */ + return sco_conn_hold(conn); } /* Delete channel. @@ -242,6 +241,19 @@ static void sco_chan_del(struct sock *sk, int err) BT_DBG("sk %p, conn %p, err %d", sk, conn, err); if (conn) { + struct hci_conn *hcon; + + sco_conn_lock(conn); + hcon = conn->hcon; + sco_conn_unlock(conn); + + /* Drop the socket's hci_conn reference BEFORE clearing + * conn->sk, so sco_conn_del() on another CPU cannot free + * the hci_conn while we still hold a pointer to it. + */ + if (hcon) + hci_conn_drop(hcon); + sco_conn_lock(conn); conn->sk = NULL; sco_conn_unlock(conn); @@ -266,6 +278,13 @@ static void sco_conn_del(struct hci_conn *hcon, int err) BT_DBG("hcon %p conn %p, err %d", hcon, conn, err); + /* Detach from the hci_conn and drop the association's reference. + * The caller holds hdev->lock, which serialises this against the + * read of hcon->sco_data in sco_recv_scodata(). + */ + hcon->sco_data = NULL; + sco_conn_put(conn); + sco_conn_lock(conn); sk = sco_sock_hold(conn); sco_conn_unlock(conn); @@ -290,6 +309,11 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk, sco_pi(sk)->conn = sco_conn_hold(conn); conn->sk = sk; + /* The socket owns an hci_conn reference for as long as it stays + * attached; it is dropped in sco_chan_del()/sco_sock_destruct(). + */ + hci_conn_hold(conn->hcon); + if (parent) bt_accept_enqueue(parent, sk, true); } @@ -371,6 +395,7 @@ static int sco_connect(struct sock *sk) if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) { release_sock(sk); sco_conn_put(conn); + hci_conn_drop(hcon); err = -EBADFD; goto unlock; } @@ -379,9 +404,13 @@ static int sco_connect(struct sock *sk) sco_conn_put(conn); if (err) { release_sock(sk); + hci_conn_drop(hcon); goto unlock; } + /* __sco_chan_add() took its own hci_conn reference; drop ours. */ + hci_conn_drop(hcon); + /* Update source addr of the socket */ bacpy(&sco_pi(sk)->src, &hcon->src); @@ -495,9 +524,25 @@ static struct sock *sco_get_sock_listen(bdaddr_t *src) static void sco_sock_destruct(struct sock *sk) { + struct sco_conn *conn = sco_pi(sk)->conn; + BT_DBG("sk %p", sk); - sco_conn_put(sco_pi(sk)->conn); + /* If the channel was not already torn down via sco_chan_del(), drop + * the socket's own references here. + */ + if (conn) { + struct hci_conn *hcon; + + sco_conn_lock(conn); + hcon = conn->hcon; + sco_conn_unlock(conn); + + if (hcon) + hci_conn_drop(hcon); + sco_pi(sk)->conn = NULL; + sco_conn_put(conn); + } skb_queue_purge(&sk->sk_receive_queue); skb_queue_purge(&sk->sk_write_queue); @@ -1511,12 +1556,10 @@ static void sco_connect_cfm(struct hci_conn *hcon, __u8 status) if (!status) { struct sco_conn *conn; - conn = sco_conn_add(hci_conn_hold(hcon)); + conn = sco_conn_add(hcon); if (conn) { sco_conn_ready(conn); sco_conn_put(conn); - } else { - hci_conn_drop(hcon); } } else sco_conn_del(hcon, bt_to_errno(status)); diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c index 2191d35d9fa0..79b6fe54a7cb 100644 --- a/net/bridge/br_multicast.c +++ b/net/bridge/br_multicast.c @@ -81,6 +81,10 @@ __br_multicast_add_group(struct net_bridge_mcast *brmctx, bool blocked); static void br_multicast_find_del_pg(struct net_bridge *br, struct net_bridge_port_group *pg); +static void __br_multicast_del_pg(struct net_bridge_mdb_entry *mp, + struct net_bridge_port_group *pg, + struct net_bridge_port_group __rcu **pp, + bool sg_del_exclude_ports); static void __br_multicast_stop(struct net_bridge_mcast *brmctx); static int br_mc_disabled_update(struct net_device *dev, bool value, @@ -458,7 +462,7 @@ static void br_multicast_sg_del_exclude_ports(struct net_bridge_mdb_entry *sgmp) for (pp = &sgmp->ports; (p = mlock_dereference(*pp, sgmp->br)) != NULL;) { if (!(p->flags & MDB_PG_FLAGS_PERMANENT)) - br_multicast_del_pg(sgmp, p, pp); + __br_multicast_del_pg(sgmp, p, pp, false); else pp = &p->next; } @@ -799,9 +803,10 @@ static void br_multicast_destroy_port_group(struct net_bridge_mcast_gc *gc) kfree_rcu(pg, rcu); } -void br_multicast_del_pg(struct net_bridge_mdb_entry *mp, - struct net_bridge_port_group *pg, - struct net_bridge_port_group __rcu **pp) +static void __br_multicast_del_pg(struct net_bridge_mdb_entry *mp, + struct net_bridge_port_group *pg, + struct net_bridge_port_group __rcu **pp, + bool sg_del_exclude_ports) { struct net_bridge *br = pg->key.port->br; struct net_bridge_group_src *ent; @@ -816,7 +821,8 @@ void br_multicast_del_pg(struct net_bridge_mdb_entry *mp, if (!br_multicast_is_star_g(&mp->addr)) { rhashtable_remove_fast(&br->sg_port_tbl, &pg->rhnode, br_sg_port_rht_params); - br_multicast_sg_del_exclude_ports(mp); + if (sg_del_exclude_ports) + br_multicast_sg_del_exclude_ports(mp); } else { br_multicast_star_g_handle_mode(pg, MCAST_INCLUDE); } @@ -828,6 +834,13 @@ void br_multicast_del_pg(struct net_bridge_mdb_entry *mp, mod_timer(&mp->timer, jiffies); } +void br_multicast_del_pg(struct net_bridge_mdb_entry *mp, + struct net_bridge_port_group *pg, + struct net_bridge_port_group __rcu **pp) +{ + __br_multicast_del_pg(mp, pg, pp, true); +} + static void br_multicast_find_del_pg(struct net_bridge *br, struct net_bridge_port_group *pg) { diff --git a/net/bridge/netfilter/nf_conntrack_bridge.c b/net/bridge/netfilter/nf_conntrack_bridge.c index 7ecb8a26bfa3..b5444335b86f 100644 --- a/net/bridge/netfilter/nf_conntrack_bridge.c +++ b/net/bridge/netfilter/nf_conntrack_bridge.c @@ -39,9 +39,13 @@ static int nf_br_ip_fragment(struct net *net, struct sock *sk, int err = 0; /* for offloaded checksums cleanup checksum before fragmentation */ - if (skb->ip_summed == CHECKSUM_PARTIAL && - (err = skb_checksum_help(skb))) - goto blackhole; + if (skb->ip_summed == CHECKSUM_PARTIAL) { + err = ip_check_csum_start(skb); + if (!err) + err = skb_checksum_help(skb); + if (err) + goto blackhole; + } iph = ip_hdr(skb); diff --git a/net/can/af_can.c b/net/can/af_can.c index 7bc86b176b4d..34fe3b28d576 100644 --- a/net/can/af_can.c +++ b/net/can/af_can.c @@ -641,13 +641,10 @@ static int can_rcv_filter(struct can_dev_rcv_lists *dev_rcv_lists, struct sk_buf return matches; } -void can_set_skb_uid(struct sk_buff *skb) +void can_set_skb_uid(struct can_skb_ext *csx) { - /* create non-zero unique skb identifier together with *skb */ - while (!(skb->hash)) - skb->hash = atomic_inc_return(&skbcounter); - - skb->sw_hash = 1; + while (!(csx->can_skb_uid)) + csx->can_skb_uid = atomic_inc_return(&skbcounter); } EXPORT_SYMBOL(can_set_skb_uid); @@ -662,8 +659,6 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev) atomic_long_inc(&pkg_stats->rx_frames); atomic_long_inc(&pkg_stats->rx_frames_delta); - can_set_skb_uid(skb); - rcu_read_lock(); /* deliver the packet to sockets listening on all devices */ @@ -687,8 +682,10 @@ static void can_receive(struct sk_buff *skb, struct net_device *dev) static int can_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { + struct can_skb_ext *csx = can_skb_ext_find(skb); + if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !can_skb_ext_find(skb) || !can_is_can_skb(skb))) { + !csx || !can_is_can_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -696,6 +693,14 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev, return NET_RX_DROP; } + /* create unshared CAN skb_extension for netem/mirred skb clones */ + csx = skb_ext_add(skb, SKB_EXT_CAN); + if (unlikely(!csx)) { + kfree_skb_reason(skb, SKB_DROP_REASON_NOMEM); + return NET_RX_DROP; + } + + can_set_skb_uid(csx); can_receive(skb, dev); return NET_RX_SUCCESS; } @@ -703,8 +708,10 @@ static int can_rcv(struct sk_buff *skb, struct net_device *dev, static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { + struct can_skb_ext *csx = can_skb_ext_find(skb); + if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !can_skb_ext_find(skb) || !can_is_canfd_skb(skb))) { + !csx || !can_is_canfd_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN FD skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -712,6 +719,14 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, return NET_RX_DROP; } + /* create unshared CAN skb_extension for netem/mirred skb clones */ + csx = skb_ext_add(skb, SKB_EXT_CAN); + if (unlikely(!csx)) { + kfree_skb_reason(skb, SKB_DROP_REASON_NOMEM); + return NET_RX_DROP; + } + + can_set_skb_uid(csx); can_receive(skb, dev); return NET_RX_SUCCESS; } @@ -719,8 +734,10 @@ static int canfd_rcv(struct sk_buff *skb, struct net_device *dev, static int canxl_rcv(struct sk_buff *skb, struct net_device *dev, struct packet_type *pt, struct net_device *orig_dev) { + struct can_skb_ext *csx = can_skb_ext_find(skb); + if (unlikely(dev->type != ARPHRD_CAN || !can_get_ml_priv(dev) || - !can_skb_ext_find(skb) || !can_is_canxl_skb(skb))) { + !csx || !can_is_canxl_skb(skb))) { pr_warn_once("PF_CAN: dropped non conform CAN XL skbuff: dev type %d, len %d\n", dev->type, skb->len); @@ -728,6 +745,14 @@ static int canxl_rcv(struct sk_buff *skb, struct net_device *dev, return NET_RX_DROP; } + /* create unshared CAN skb_extension for netem/mirred skb clones */ + csx = skb_ext_add(skb, SKB_EXT_CAN); + if (unlikely(!csx)) { + kfree_skb_reason(skb, SKB_DROP_REASON_NOMEM); + return NET_RX_DROP; + } + + can_set_skb_uid(csx); can_receive(skb, dev); return NET_RX_SUCCESS; } diff --git a/net/can/gw.c b/net/can/gw.c index 0ec99f68aa45..5793cb2420ed 100644 --- a/net/can/gw.c +++ b/net/can/gw.c @@ -527,6 +527,9 @@ static void can_can_gw_rcv(struct sk_buff *skb, void *data) /* put the incremented hop counter in the cloned skb */ ncsx->can_gw_hops = csx->can_gw_hops + 1; + /* force a new CAN UID generation for the routed frame */ + ncsx->can_skb_uid = 0; + /* first processing of this CAN frame -> adjust to private hop limit */ if (gwj->limit_hops && ncsx->can_gw_hops == 1) ncsx->can_gw_hops = max_hops - gwj->limit_hops + 1; diff --git a/net/can/isotp.c b/net/can/isotp.c index 155530aedce2..f5dc9d04bd68 100644 --- a/net/can/isotp.c +++ b/net/can/isotp.c @@ -891,7 +891,7 @@ static void isotp_send_cframe(struct isotp_sock *so) csx->can_iif = dev->ifindex; /* set uid in tx skb to identify CF echo frames */ - can_set_skb_uid(skb); + can_set_skb_uid(csx); cf = (struct canfd_frame *)skb->data; skb_put_zero(skb, so->ll.mtu); @@ -917,7 +917,7 @@ static void isotp_send_cframe(struct isotp_sock *so) pr_notice_once("can-isotp: cfecho is %08X != 0\n", old_cfecho); /* set consecutive frame echo tag */ - WRITE_ONCE(so->cfecho, skb->hash); + WRITE_ONCE(so->cfecho, csx->can_skb_uid); /* send frame with local echo enabled */ can_send_ret = can_send(skb, 1); @@ -969,18 +969,22 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) { struct sock *sk = (struct sock *)data; struct isotp_sock *so = isotp_sk(sk); + struct can_skb_ext *csx = can_skb_ext_find(skb); /* only handle my own local echo CF/SF skb's (no FF!) */ if (skb->sk != sk) return; + if (WARN_ON_ONCE(!csx)) + return; + /* unlike isotp_rcv_fc()/isotp_rcv_cf(), not already under so->rx_lock * (no isotp_rcv() caller here), so take it ourselves */ spin_lock(&so->rx_lock); /* so->cfecho may since belong to a new transfer; recheck under lock */ - if (READ_ONCE(so->cfecho) != skb->hash) + if (READ_ONCE(so->cfecho) != csx->can_skb_uid) goto out_unlock; /* cancel local echo timeout */ @@ -1222,7 +1226,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) csx->can_iif = dev->ifindex; /* set uid in tx skb to identify CF echo frames */ - can_set_skb_uid(skb); + can_set_skb_uid(csx); so->tx.len = size; so->tx.idx = 0; @@ -1261,7 +1265,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) cf->data[ae] |= size; /* set CF echo tag for isotp_rcv_echo() (SF-mode) */ - WRITE_ONCE(so->cfecho, skb->hash); + WRITE_ONCE(so->cfecho, csx->can_skb_uid); } else { /* send first frame */ @@ -1278,7 +1282,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) so->txfc.bs = 0; /* set CF echo tag for isotp_rcv_echo() (CF-mode) */ - WRITE_ONCE(so->cfecho, skb->hash); + WRITE_ONCE(so->cfecho, csx->can_skb_uid); } else { /* standard flow control check */ new_state = ISOTP_WAIT_FIRST_FC; diff --git a/net/can/raw.c b/net/can/raw.c index 82d9c0499c95..0acd4f6c6dd6 100644 --- a/net/can/raw.c +++ b/net/can/raw.c @@ -77,7 +77,7 @@ MODULE_ALIAS("can-proto-1"); struct uniqframe { const struct sk_buff *skb; - u32 hash; + u32 can_skb_uid; unsigned int join_rx_count; }; @@ -133,12 +133,16 @@ static void raw_rcv(struct sk_buff *oskb, void *data) enum skb_drop_reason reason; struct sockaddr_can *addr; struct sk_buff *skb; + struct can_skb_ext *csx = can_skb_ext_find(oskb); unsigned int *pflags; /* check the received tx sock reference */ if (!ro->recv_own_msgs && oskb->sk == sk) return; + if (WARN_ON_ONCE(!csx)) + return; + /* make sure to not pass oversized frames to the socket */ if (!ro->fd_frames && can_is_canfd_skb(oskb)) return; @@ -165,7 +169,7 @@ static void raw_rcv(struct sk_buff *oskb, void *data) /* eliminate multiple filter matches for the same skb */ if (this_cpu_ptr(ro->uniq)->skb == oskb && - this_cpu_ptr(ro->uniq)->hash == oskb->hash) { + this_cpu_ptr(ro->uniq)->can_skb_uid == csx->can_skb_uid) { if (!ro->join_filters) return; @@ -175,7 +179,7 @@ static void raw_rcv(struct sk_buff *oskb, void *data) return; } else { this_cpu_ptr(ro->uniq)->skb = oskb; - this_cpu_ptr(ro->uniq)->hash = oskb->hash; + this_cpu_ptr(ro->uniq)->can_skb_uid = csx->can_skb_uid; this_cpu_ptr(ro->uniq)->join_rx_count = 1; /* drop first frame to check all enabled filters? */ if (ro->join_filters && ro->count > 1) diff --git a/net/core/dev.c b/net/core/dev.c index 6f75d1d1af32..1a06976e1956 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10356,13 +10356,15 @@ EXPORT_SYMBOL_GPL(dev_xdp_prog_count); u8 dev_xdp_sb_prog_count(struct net_device *dev) { + struct bpf_prog *prog; u8 count = 0; int i; - for (i = 0; i < __MAX_XDP_MODE; i++) - if (dev->xdp_state[i].prog && - !dev->xdp_state[i].prog->aux->xdp_has_frags) + for (i = 0; i < __MAX_XDP_MODE; i++) { + prog = dev_xdp_prog(dev, i); + if (prog && !prog->aux->xdp_has_frags) count++; + } return count; } @@ -10742,11 +10744,12 @@ static int bpf_xdp_link_update(struct bpf_link *link, struct bpf_prog *new_prog, bpf_op = dev_xdp_bpf_op(xdp_link->dev, mode); err = dev_xdp_install(xdp_link->dev, mode, bpf_op, NULL, xdp_link->flags, new_prog); + if (!err) + old_prog = xchg(&link->prog, new_prog); netdev_unlock_ops(xdp_link->dev); if (err) goto out_unlock; - old_prog = xchg(&link->prog, new_prog); bpf_prog_put(old_prog); out_unlock: diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c index 8aa4f9b4df81..1be6c739e3c7 100644 --- a/net/core/flow_dissector.c +++ b/net/core/flow_dissector.c @@ -1023,6 +1023,8 @@ u32 bpf_flow_dissect(struct bpf_prog *prog, struct bpf_flow_dissector *ctx, result = bpf_prog_run_pin_on_cpu(prog, ctx); + /* bpf_flow_keys offsets are u16: do not let @hlen be truncated. */ + hlen = min_t(int, hlen, U16_MAX); flow_keys->nhoff = clamp_t(u16, flow_keys->nhoff, nhoff, hlen); flow_keys->thoff = clamp_t(u16, flow_keys->thoff, flow_keys->nhoff, hlen); @@ -1071,6 +1073,7 @@ bool __skb_flow_dissect(const struct net *net, int mpls_lse = 0; int num_hdrs = 0; u8 ip_proto = 0; + u32 thoff; bool ret; if (!data) { @@ -1692,7 +1695,13 @@ bool __skb_flow_dissect(const struct net *net, ret = true; out: - key_control->thoff = min_t(u16, nhoff, skb ? skb->len : hlen); + thoff = min_t(u32, nhoff, skb ? skb->len : hlen); + if (unlikely(thoff > U16_MAX)) { + /* Cannot be represented in key_control->thoff. */ + thoff = U16_MAX; + ret = false; + } + key_control->thoff = thoff; key_basic->n_proto = proto; key_basic->ip_proto = ip_proto; diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 88febbc5803e..fa5c1c15d417 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -3633,8 +3633,12 @@ __wsum skb_copy_and_csum_bits(const struct sk_buff *skb, int offset, pos = copy; } - if (!skb_frags_readable(skb)) + if (!skb_frags_readable(skb)) { + /* Don't hand the caller a buffer with stale bytes in it. */ + if (len > 0) + memset(to, 0, len); return 0; + } for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { int end; diff --git a/net/devlink/core.c b/net/devlink/core.c index fe9f6a0a67d5..42c16940e4c4 100644 --- a/net/devlink/core.c +++ b/net/devlink/core.c @@ -96,13 +96,19 @@ static void devlink_rel_nested_in_notify_work(struct work_struct *work) return; reschedule_work: - schedule_delayed_work(&rel->nested_in.notify_work, 1); + /* The work may have been queued again meanwhile, which took its own + * reference. Drop ours in that case. + */ + if (!schedule_delayed_work(&rel->nested_in.notify_work, 1)) + __devlink_rel_put(rel); } static void devlink_rel_nested_in_notify_work_schedule(struct devlink_rel *rel) { __devlink_rel_get(rel); - schedule_delayed_work(&rel->nested_in.notify_work, 0); + /* The pending work holds a reference already, drop the new one. */ + if (!schedule_delayed_work(&rel->nested_in.notify_work, 0)) + __devlink_rel_put(rel); } static struct devlink_rel *devlink_rel_alloc(void) diff --git a/net/ipv4/cipso_ipv4.c b/net/ipv4/cipso_ipv4.c index a05aa075de1a..1aacbbeffbc6 100644 --- a/net/ipv4/cipso_ipv4.c +++ b/net/ipv4/cipso_ipv4.c @@ -2287,6 +2287,14 @@ int cipso_v4_skbuff_delattr(struct sk_buff *skb) new_hdr_len - new_hdr_len_actual); opt->optlen -= hdr_len_delta; + if (opt->srr > opt->cipso) + opt->srr -= cipso_len; + if (opt->rr > opt->cipso) + opt->rr -= cipso_len; + if (opt->ts > opt->cipso) + opt->ts -= cipso_len; + if (opt->router_alert > opt->cipso) + opt->router_alert -= cipso_len; opt->cipso = 0; opt->is_changed = 1; if (hdr_len_delta != 0) { diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index 0c25f6dfb8a6..92cfc838e5d3 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -542,6 +542,9 @@ size_t fib_nlmsg_size(struct fib_info *fi) struct fib_nh_common *nhc = fib_info_nhc(fi, i); size_t nhsize; + if (!nhc) + break; + nhsize = fib_nexthop_nlmsg_size(nhc, nhs != 1); if (nhs != 1) diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c index 6badad29593b..58be0752b035 100644 --- a/net/ipv4/fib_trie.c +++ b/net/ipv4/fib_trie.c @@ -1079,8 +1079,6 @@ void fib_alias_hw_flags_set(struct net *net, const struct fib_rt_info *fri) err = fib_dump_info(skb, 0, 0, RTM_NEWROUTE, fri, 0); if (err < 0) { - /* -EMSGSIZE implies BUG in fib_nlmsg_size() */ - WARN_ON(err == -EMSGSIZE); kfree_skb(skb); goto errout; } diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index a24cc8ee11d3..eaa6fabda347 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -771,9 +771,13 @@ int ip_do_fragment(struct net *net, struct sock *sk, struct sk_buff *skb, int err = 0; /* for offloaded checksums cleanup checksum before fragmentation */ - if (skb->ip_summed == CHECKSUM_PARTIAL && - (err = skb_checksum_help(skb))) - goto fail; + if (skb->ip_summed == CHECKSUM_PARTIAL) { + err = ip_check_csum_start(skb); + if (!err) + err = skb_checksum_help(skb); + if (err) + goto fail; + } /* * Point into the IP datagram header. diff --git a/net/ipv4/route.c b/net/ipv4/route.c index f07231447883..196ca7e8acff 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -1080,6 +1080,8 @@ static void __ip_rt_update_pmtu(struct rtable *rt, struct flowi4 *fl4, u32 mtu) for (nhsel = 0; nhsel < fib_info_num_path(res.fi); nhsel++) { nhc = fib_info_nhc(res.fi, nhsel); + if (!nhc) + break; update_or_create_fnhe(nhc, fl4->daddr, 0, mtu, lock, jiffies + net->ipv4.ip_rt_mtu_expires); } @@ -3163,6 +3165,9 @@ int fib_dump_info_fnhe(struct sk_buff *skb, struct netlink_callback *cb, struct fnhe_hash_bucket *bucket; int err; + if (!nhc) + break; + if (nhc->nhc_flags & RTNH_F_DEAD) continue; diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 47a34a993972..93efc948d2be 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1908,6 +1908,8 @@ static bool can_map_frag(const skb_frag_t *frag) if (skb_frag_size(frag) != PAGE_SIZE || skb_frag_off(frag)) return false; + if (skb_frag_is_net_iov(frag)) + return false; page = skb_frag_page(frag); diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index 59afce196079..a50dccfbf420 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -908,6 +908,15 @@ static struct sock *__udp4_lib_err_encap(struct net *net, return sk; } +static void udp_err_update_exception(struct net *net, struct sk_buff *skb, + int type, int code, u32 info) +{ + if (type == ICMP_DEST_UNREACH && code == ICMP_FRAG_NEEDED) + ipv4_update_pmtu(skb, net, info, 0, IPPROTO_UDP); + else if (type == ICMP_REDIRECT) + ipv4_redirect(skb, net, 0, IPPROTO_UDP); +} + /* * This routine is called by the ICMP module when it gets some * sort of error condition. If err < 0 then the socket should @@ -931,6 +940,8 @@ int udp_err(struct sk_buff *skb, u32 info) int harderr; int err; + udp_err_update_exception(net, skb, type, code, info); + uh = (struct udphdr *)(skb->data + (iph->ihl << 2)); sk = __udp4_lib_lookup(net, iph->daddr, uh->dest, iph->saddr, uh->source, skb->dev->ifindex, diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c index e6913a7f6881..6d7c547232a2 100644 --- a/net/ipv6/ip6_gre.c +++ b/net/ipv6/ip6_gre.c @@ -923,7 +923,6 @@ static netdev_tx_t ip6erspan_tunnel_xmit(struct sk_buff *skb, { struct ip_tunnel_info *tun_info = NULL; struct ip6_tnl *t = netdev_priv(dev); - struct dst_entry *dst = skb_dst(skb); IP_TUNNEL_DECLARE_FLAGS(flags) = { }; bool truncate = false; int encap_limit = -1; @@ -1059,12 +1058,6 @@ static netdev_tx_t ip6erspan_tunnel_xmit(struct sk_buff *skb, gre_build_header(skb, 8, flags, proto, 0, htonl(atomic_fetch_inc(&t->o_seqno))); - /* TooBig packet may have updated dst->dev's mtu */ - if (!t->parms.collect_md && dst) { - mtu = READ_ONCE(dst_dev(dst)->mtu); - if (dst_mtu(dst) > mtu) - dst->ops->update_pmtu(dst, NULL, skb, mtu, false); - } err = ip6_tnl_xmit(skb, dev, dsfield, &fl6, encap_limit, &mtu, NEXTHDR_GRE); if (err != 0) { diff --git a/net/ipv6/route.c b/net/ipv6/route.c index 8fddcc631fcb..5d80a025c3e6 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -3261,7 +3261,7 @@ void ip6_redirect_no_header(struct sk_buff *skb, struct net *net, int oif) void ip6_sk_redirect(struct sk_buff *skb, struct sock *sk) { - ip6_redirect(skb, sock_net(sk), sk->sk_bound_dev_if, + ip6_redirect(skb, sock_net(sk), skb->dev->ifindex, READ_ONCE(sk->sk_mark), sk_uid(sk)); } @@ -6474,8 +6474,6 @@ void fib6_info_hw_flags_set(struct net *net, struct fib6_info *f6i, err = rt6_fill_node(net, skb, f6i, NULL, NULL, NULL, 0, RTM_NEWROUTE, 0, 0, 0); if (err < 0) { - /* -EMSGSIZE implies BUG in rt6_nlmsg_size() */ - WARN_ON(err == -EMSGSIZE); kfree_skb(skb); goto errout; } diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 15e032194ecc..b73cf02ac1b6 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -690,6 +690,17 @@ static struct sock *__udp6_lib_err_encap(struct net *net, return sk; } +static void udpv6_err_update_exception(struct net *net, struct sk_buff *skb, + u8 type, __be32 info) +{ + if (type == ICMPV6_PKT_TOOBIG) + ip6_update_pmtu(skb, net, info, skb->dev->ifindex, 0, + sock_net_uid(net, NULL)); + else if (type == NDISC_REDIRECT) + ip6_redirect(skb, net, skb->dev->ifindex, 0, + sock_net_uid(net, NULL)); +} + static int udpv6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, u8 type, u8 code, int offset, __be32 info) { @@ -703,6 +714,8 @@ static int udpv6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, int harderr; int err; + udpv6_err_update_exception(net, skb, type, info); + daddr = seg6_get_daddr(skb, opt) ? : &hdr->daddr; saddr = &hdr->saddr; sk = __udp6_lib_lookup(net, daddr, uh->dest, saddr, uh->source, diff --git a/net/mac80211/debugfs.c b/net/mac80211/debugfs.c index f36cc2590305..3a671225c127 100644 --- a/net/mac80211/debugfs.c +++ b/net/mac80211/debugfs.c @@ -210,11 +210,13 @@ static ssize_t aql_pending_read(struct file *file, "VI %u us\n" "BE %u us\n" "BK %u us\n" + "MC %u us\n" "total %u us\n", atomic_read(&local->aql_ac_pending_airtime[IEEE80211_AC_VO]), atomic_read(&local->aql_ac_pending_airtime[IEEE80211_AC_VI]), atomic_read(&local->aql_ac_pending_airtime[IEEE80211_AC_BE]), atomic_read(&local->aql_ac_pending_airtime[IEEE80211_AC_BK]), + atomic_read(&local->aql_mc_pending_airtime), atomic_read(&local->aql_total_pending_airtime)); return simple_read_from_buffer(user_buf, count, ppos, buf, len); @@ -239,7 +241,8 @@ static ssize_t aql_txq_limit_read(struct file *file, "VO %u %u\n" "VI %u %u\n" "BE %u %u\n" - "BK %u %u\n", + "BK %u %u\n" + "MC %u\n", local->aql_txq_limit_low[IEEE80211_AC_VO], local->aql_txq_limit_high[IEEE80211_AC_VO], local->aql_txq_limit_low[IEEE80211_AC_VI], @@ -247,7 +250,8 @@ static ssize_t aql_txq_limit_read(struct file *file, local->aql_txq_limit_low[IEEE80211_AC_BE], local->aql_txq_limit_high[IEEE80211_AC_BE], local->aql_txq_limit_low[IEEE80211_AC_BK], - local->aql_txq_limit_high[IEEE80211_AC_BK]); + local->aql_txq_limit_high[IEEE80211_AC_BK], + local->aql_txq_limit_mc); return simple_read_from_buffer(user_buf, count, ppos, buf, len); } @@ -273,6 +277,11 @@ static ssize_t aql_txq_limit_write(struct file *file, else buf[count] = '\0'; + if (sscanf(buf, "mcast %u", &q_limit_low) == 1) { + local->aql_txq_limit_mc = q_limit_low; + return count; + } + if (sscanf(buf, "%u %u %u", &ac, &q_limit_low, &q_limit_high) != 3) return -EINVAL; diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 011ec7fbb711..2854dcc86b21 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -1464,10 +1464,12 @@ struct ieee80211_local { spinlock_t handle_wake_tx_queue_lock; u16 airtime_flags; + u32 aql_txq_limit_mc; u32 aql_txq_limit_low[IEEE80211_NUM_ACS]; u32 aql_txq_limit_high[IEEE80211_NUM_ACS]; u32 aql_threshold; atomic_t aql_total_pending_airtime; + atomic_t aql_mc_pending_airtime; atomic_t aql_ac_pending_airtime[IEEE80211_NUM_ACS]; const struct ieee80211_ops *ops; diff --git a/net/mac80211/main.c b/net/mac80211/main.c index 8edfef2b57f8..d5ef44f2b3f2 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -988,6 +988,7 @@ struct ieee80211_hw *ieee80211_alloc_hw_nm(size_t priv_data_len, spin_lock_init(&local->rx_path_lock); spin_lock_init(&local->queue_stop_reason_lock); + local->aql_txq_limit_mc = IEEE80211_DEFAULT_AQL_TXQ_LIMIT_MC; for (i = 0; i < IEEE80211_NUM_ACS; i++) { INIT_LIST_HEAD(&local->active_txqs[i]); spin_lock_init(&local->active_txq_lock[i]); diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index 01d70f71c43e..ce7a3f839ce2 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -2493,13 +2493,28 @@ EXPORT_SYMBOL(ieee80211_sta_recalc_aggregates); void ieee80211_sta_update_pending_airtime(struct ieee80211_local *local, struct sta_info *sta, u8 ac, - u16 tx_airtime, bool tx_completed) + u16 tx_airtime, bool tx_completed, + bool mcast) { int tx_pending; if (!wiphy_ext_feature_isset(local->hw.wiphy, NL80211_EXT_FEATURE_AQL)) return; + if (mcast) { + if (!tx_completed) { + atomic_add(tx_airtime, &local->aql_mc_pending_airtime); + return; + } + + tx_pending = atomic_sub_return(tx_airtime, + &local->aql_mc_pending_airtime); + if (tx_pending < 0) + atomic_cmpxchg(&local->aql_mc_pending_airtime, + tx_pending, 0); + return; + } + if (!tx_completed) { if (sta) atomic_add(tx_airtime, diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index 5da3142d8516..ee0d32877c5b 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -147,7 +147,8 @@ struct airtime_info { void ieee80211_sta_update_pending_airtime(struct ieee80211_local *local, struct sta_info *sta, u8 ac, - u16 tx_airtime, bool tx_completed); + u16 tx_airtime, bool tx_completed, + bool mcast); struct sta_info; diff --git a/net/mac80211/status.c b/net/mac80211/status.c index e10e5bcc279b..a9da9fe87cf1 100644 --- a/net/mac80211/status.c +++ b/net/mac80211/status.c @@ -774,7 +774,7 @@ static void ieee80211_report_used_skb(struct ieee80211_local *local, ieee80211_sta_update_pending_airtime(local, sta, skb_get_queue_mapping(skb), tx_time_est, - true); + true, info->tx_time_mc); rcu_read_unlock(); } @@ -1255,10 +1255,11 @@ void ieee80211_tx_status_ext(struct ieee80211_hw *hw, /* Do this here to avoid the expensive lookup of the sta * in ieee80211_report_used_skb(). */ + bool mcast = IEEE80211_SKB_CB(skb)->tx_time_mc; ieee80211_sta_update_pending_airtime(local, sta, skb_get_queue_mapping(skb), tx_time_est, - true); + true, mcast); ieee80211_info_set_tx_time_est(IEEE80211_SKB_CB(skb), 0); } diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 65c5a6519b89..3e5b1c05f4a3 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -4083,20 +4083,20 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw, encap_out: info->control.vif = vif; - if (tx.sta && - wiphy_ext_feature_isset(local->hw.wiphy, NL80211_EXT_FEATURE_AQL)) { - bool ampdu = txq->ac != IEEE80211_AC_VO; + if (wiphy_ext_feature_isset(local->hw.wiphy, NL80211_EXT_FEATURE_AQL)) { + bool ampdu = txq->sta && txq->ac != IEEE80211_AC_VO; u32 airtime; airtime = ieee80211_calc_expected_tx_airtime(hw, vif, txq->sta, skb->len, ampdu); - if (airtime) { - airtime = ieee80211_info_set_tx_time_est(info, airtime); - ieee80211_sta_update_pending_airtime(local, tx.sta, - txq->ac, - airtime, - false); - } + if (!airtime) + return skb; + + airtime = ieee80211_info_set_tx_time_est(info, airtime); + info->tx_time_mc = !tx.sta; + ieee80211_sta_update_pending_airtime(local, tx.sta, txq->ac, + airtime, false, + info->tx_time_mc); } return skb; @@ -4148,6 +4148,7 @@ struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac) struct ieee80211_txq *ret = NULL; struct txq_info *txqi = NULL, *head = NULL; bool found_eligible_txq = false; + bool aql_check; spin_lock_bh(&local->active_txq_lock[ac]); @@ -4171,26 +4172,28 @@ struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac) if (!head) head = txqi; + aql_check = ieee80211_txq_airtime_check(hw, &txqi->txq); + if (aql_check) + found_eligible_txq = true; + if (txqi->txq.sta) { struct sta_info *sta = container_of(txqi->txq.sta, struct sta_info, sta); - bool aql_check = ieee80211_txq_airtime_check(hw, &txqi->txq); - s32 deficit = ieee80211_sta_deficit(sta, txqi->txq.ac); - if (aql_check) - found_eligible_txq = true; - - if (deficit < 0) + if (ieee80211_sta_deficit(sta, txqi->txq.ac) < 0) { sta->airtime[txqi->txq.ac].deficit += sta->airtime_weight; - if (deficit < 0 || !aql_check) { - list_move_tail(&txqi->schedule_order, - &local->active_txqs[txqi->txq.ac]); - goto begin; + aql_check = false; } } + if (!aql_check) { + list_move_tail(&txqi->schedule_order, + &local->active_txqs[txqi->txq.ac]); + goto begin; + } + if (txqi->schedule_round == local->schedule_round[ac]) goto out; @@ -4257,7 +4260,8 @@ bool ieee80211_txq_airtime_check(struct ieee80211_hw *hw, return true; if (!txq->sta) - return true; + return atomic_read(&local->aql_mc_pending_airtime) < + local->aql_txq_limit_mc; if (unlikely(txq->tid == IEEE80211_NUM_TIDS)) return true; @@ -4277,6 +4281,24 @@ bool ieee80211_txq_airtime_check(struct ieee80211_hw *hw, } EXPORT_SYMBOL(ieee80211_txq_airtime_check); +u32 ieee80211_txq_aql_pending(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) +{ + struct ieee80211_local *local = hw_to_local(hw); + struct sta_info *sta; + + if (unlikely(txq->tid == IEEE80211_NUM_TIDS)) + return 0; + + if (!txq->sta) + return atomic_read(&local->aql_mc_pending_airtime); + + sta = container_of(txq->sta, struct sta_info, sta); + + return atomic_read(&sta->airtime[txq->ac].aql_tx_pending); +} +EXPORT_SYMBOL(ieee80211_txq_aql_pending); + static bool ieee80211_txq_schedule_airtime_check(struct ieee80211_local *local, u8 ac) { @@ -4306,15 +4328,15 @@ bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw, spin_lock_bh(&local->active_txq_lock[ac]); - if (!txqi->txq.sta) - goto out; - if (list_empty(&txqi->schedule_order)) goto out; if (!ieee80211_txq_schedule_airtime_check(local, ac)) goto out; + if (!txqi->txq.sta) + goto out; + list_for_each_entry_safe(iter, tmp, &local->active_txqs[ac], schedule_order) { if (iter == txqi) diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c index ec5e7716df8b..8a5240a9be22 100644 --- a/net/netfilter/nfnetlink_queue.c +++ b/net/netfilter/nfnetlink_queue.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -667,6 +668,12 @@ static int nfqnl_put_bridge(struct nf_queue_entry *entry, struct sk_buff *skb) static int nf_queue_checksum_help(struct sk_buff *entskb) { + if (entskb->protocol == htons(ETH_P_IP)) { + if (!pskb_network_may_pull(entskb, sizeof(struct iphdr)) || + ip_check_csum_start(entskb)) + return -EINVAL; + } + if (skb_csum_is_sctp(entskb)) return skb_crc32c_csum_help(entskb); diff --git a/net/netfilter/xt_CHECKSUM.c b/net/netfilter/xt_CHECKSUM.c index 9d99f5a3d176..1fb0f8118404 100644 --- a/net/netfilter/xt_CHECKSUM.c +++ b/net/netfilter/xt_CHECKSUM.c @@ -15,6 +15,7 @@ #include #include +#include MODULE_LICENSE("GPL"); MODULE_AUTHOR("Michael S. Tsirkin "); @@ -25,8 +26,11 @@ MODULE_ALIAS("ip6t_CHECKSUM"); static unsigned int checksum_tg(struct sk_buff *skb, const struct xt_action_param *par) { - if (skb->ip_summed == CHECKSUM_PARTIAL && !skb_is_gso(skb)) + if (skb->ip_summed == CHECKSUM_PARTIAL && !skb_is_gso(skb)) { + if (xt_family(par) == NFPROTO_IPV4 && ip_check_csum_start(skb)) + return NF_DROP; skb_checksum_help(skb); + } return XT_CONTINUE; } diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c index dc5ff859f114..27152bca8a8c 100644 --- a/net/openvswitch/actions.c +++ b/net/openvswitch/actions.c @@ -358,7 +358,12 @@ static void set_ip_addr(struct sk_buff *skb, struct iphdr *nh, static void update_ipv6_checksum(struct sk_buff *skb, u8 l4_proto, __be32 addr[4], const __be32 new_addr[4]) { - int transport_len = skb->len - skb_transport_offset(skb); + int transport_len; + + if (l4_proto == NEXTHDR_FRAGMENT) + return; + + transport_len = skb->len - skb_transport_offset(skb); if (l4_proto == NEXTHDR_TCP) { if (likely(transport_len >= sizeof(struct tcphdr))) diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index b601e4859de9..21bc56e79627 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -1529,6 +1529,12 @@ static int ovs_flow_cmd_dump(struct sk_buff *skb, struct netlink_callback *cb) return -ENODEV; } + /* + * Not needed for safety. Stops every spin_unlock_bh() in + * ovs_flow_stats_get() from running the softirq backlog. + */ + local_bh_disable(); + ti = rcu_dereference(dp->table.ti); for (;;) { struct sw_flow *flow; @@ -1549,6 +1555,8 @@ static int ovs_flow_cmd_dump(struct sk_buff *skb, struct netlink_callback *cb) cb->args[0] = bucket; cb->args[1] = obj; } + + local_bh_enable(); rcu_read_unlock(); return skb->len; } diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 8865c1a42667..52ae3d875e97 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -1911,7 +1911,8 @@ static int packet_rcv_spkt(struct sk_buff *skb, struct net_device *dev, spkt = &PACKET_SKB_CB(skb)->sa.pkt; - skb_push(skb, skb->data - skb_mac_header(skb)); + if (dev_has_header(dev)) + skb_push(skb, skb->data - skb_mac_header(skb)); /* * The SOCK_PACKET socket receives _all_ frames. @@ -1942,8 +1943,7 @@ static void packet_parse_headers(struct sk_buff *skb, struct socket *sock) /* On TX skb->data is the L2 header; anchor it for all socket types. */ skb_reset_mac_header(skb); - if ((!skb->protocol || skb->protocol == htons(ETH_P_ALL)) && - sock->type == SOCK_RAW) + if (!skb->protocol && sock->type == SOCK_RAW) skb->protocol = dev_parse_header_protocol(skb); skb_probe_transport_header(skb); @@ -2613,7 +2613,8 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, struct page *page; int err; - skb->protocol = proto; + if (proto != htons(ETH_P_ALL)) + skb->protocol = proto; skb->dev = dev; skb->priority = sockc->priority; skb->mark = sockc->mark; @@ -2677,8 +2678,6 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, if (unlikely(!skb->len)) return -EINVAL; - packet_parse_headers(skb, sock); - return tp_len; } @@ -2918,9 +2917,13 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) tp_len = -EINVAL; goto tpacket_error; } - virtio_net_hdr_set_proto(skb, &vnet_hdr); } + packet_parse_headers(skb, po->sk.sk_socket); + + if (has_vnet_hdr) + virtio_net_hdr_set_proto(skb, &vnet_hdr); + uarg = kmalloc(sizeof(*uarg), GFP_KERNEL); if (unlikely(!uarg)) { if (likely(len_sum > 0)) @@ -3121,7 +3124,8 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_free; } - skb->protocol = proto; + if (proto != htons(ETH_P_ALL)) + skb->protocol = proto; skb->dev = dev; skb->priority = sockc.priority; skb->mark = sockc.mark; @@ -3130,16 +3134,18 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) if (unlikely(extra_len == 4)) skb->no_fcs = 1; - packet_parse_headers(skb, sock); - if (vnet_hdr_sz) { err = virtio_net_hdr_to_skb(skb, &vnet_hdr, vio_le()); if (err) goto out_free; len += vnet_hdr_sz; - virtio_net_hdr_set_proto(skb, &vnet_hdr); } + packet_parse_headers(skb, sock); + + if (vnet_hdr_sz) + virtio_net_hdr_set_proto(skb, &vnet_hdr); + err = packet_xmit(po, skb); if (unlikely(err != 0)) { diff --git a/net/sched/cls_route.c b/net/sched/cls_route.c index 0f211f030fd9..dca812a75269 100644 --- a/net/sched/cls_route.c +++ b/net/sched/cls_route.c @@ -434,6 +434,12 @@ static int route4_set_parms(struct net *net, struct tcf_proto *tp, if (handle && (!fold || nhandle == (handle & ~0x7F00))) nhandle |= handle & 0x7F00; + if (fold && !tb[TCA_ROUTE4_TO] && !tb[TCA_ROUTE4_FROM] && + !tb[TCA_ROUTE4_IIF] && nhandle != fold->handle) { + NL_SET_ERR_MSG(extack, "Routing attributes required"); + return -EINVAL; + } + if (handle && !fold && nhandle != handle) { NL_SET_ERR_MSG_FMT(extack, "Handle mismatch constructed: %x (expected: %x)", diff --git a/net/sched/em_text.c b/net/sched/em_text.c index 4132f8c3c5fc..bc45edb8c03b 100644 --- a/net/sched/em_text.c +++ b/net/sched/em_text.c @@ -58,6 +58,9 @@ static int em_text_change(struct net *net, void *data, int len, if (len < sizeof(*conf) || len < (sizeof(*conf) + conf->pattern_len)) return -EINVAL; + if (!memchr(conf->algo, '\0', sizeof(conf->algo))) + return -EINVAL; + if (conf->from_layer > conf->to_layer) return -EINVAL; diff --git a/net/sctp/socket.c b/net/sctp/socket.c index 4652fd90d9a6..e35692dd8d30 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -7569,11 +7569,7 @@ static int sctp_getsockopt_pr_streamstatus(struct sock *sk, int len, /* Not allocated yet, means all stats are 0 */ params.sprstat_abandoned_unsent = 0; params.sprstat_abandoned_sent = 0; - retval = 0; - goto out; - } - - if (policy == SCTP_PR_SCTP_ALL) { + } else if (policy == SCTP_PR_SCTP_ALL) { params.sprstat_abandoned_unsent = 0; params.sprstat_abandoned_sent = 0; for (policy = 0; policy <= SCTP_PR_INDEX(MAX); policy++) { diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index e9f93b3ab435..8b3ee70f8433 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -2784,6 +2784,7 @@ int smc_getname(struct socket *sock, struct sockaddr *addr, int peer) { struct smc_sock *smc; + int rc = -EBADF; if (peer && (sock->sk->sk_state != SMC_ACTIVE) && (sock->sk->sk_state != SMC_APPCLOSEWAIT1)) @@ -2791,7 +2792,11 @@ int smc_getname(struct socket *sock, struct sockaddr *addr, smc = smc_sk(sock->sk); - return smc->clcsock->ops->getname(smc->clcsock, addr, peer); + mutex_lock(&smc->clcsock_release_lock); + if (smc->clcsock) + rc = smc->clcsock->ops->getname(smc->clcsock, addr, peer); + mutex_unlock(&smc->clcsock_release_lock); + return rc; } int smc_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) diff --git a/net/strparser/strparser.c b/net/strparser/strparser.c index a23f4b4dfc67..d5b17d099a0e 100644 --- a/net/strparser/strparser.c +++ b/net/strparser/strparser.c @@ -505,12 +505,16 @@ void strp_unpause(struct strparser *strp) EXPORT_SYMBOL_GPL(strp_unpause); /* strp must already be stopped so that strp_recv will no longer be called. - * Note that strp_done is not called with the lower socket held. + * Note that strp_done must not be called with the lower socket held. */ void strp_done(struct strparser *strp) { WARN_ON(!strp->stopped); + lock_sock(strp->sk); + /* sync with pending strp_recv */ + release_sock(strp->sk); + cancel_delayed_work_sync(&strp->msg_timer_work); cancel_work_sync(&strp->work); diff --git a/net/tipc/node.c b/net/tipc/node.c index 683a136e53ef..5a7487dab31d 100644 --- a/net/tipc/node.c +++ b/net/tipc/node.c @@ -2419,8 +2419,11 @@ static struct tipc_node *tipc_node_find_by_name(struct net *net, } } tipc_node_read_unlock(n); - if (found_node) + if (found_node) { + if (!kref_get_unless_zero(&found_node->kref)) + found_node = NULL; break; + } } rcu_read_unlock(); @@ -2505,6 +2508,7 @@ int tipc_nl_node_set_link(struct sk_buff *skb, struct genl_info *info) tipc_node_read_unlock(node); tipc_bearer_xmit(net, bearer_id, &xmitq, &node->links[bearer_id].maddr, NULL); + tipc_node_put(node); return res; } @@ -2556,12 +2560,14 @@ int tipc_nl_node_get_link(struct sk_buff *skb, struct genl_info *info) link = node->links[bearer_id].link; if (!link) { tipc_node_read_unlock(node); + tipc_node_put(node); err = -EINVAL; goto err_free; } err = __tipc_nl_add_link(net, &msg, link, 0); tipc_node_read_unlock(node); + tipc_node_put(node); if (err) goto err_free; } @@ -2632,11 +2638,13 @@ int tipc_nl_node_reset_link_stats(struct sk_buff *skb, struct genl_info *info) if (!link) { spin_unlock_bh(&le->lock); tipc_node_read_unlock(node); + tipc_node_put(node); return -EINVAL; } tipc_link_reset_stats(link); spin_unlock_bh(&le->lock); tipc_node_read_unlock(node); + tipc_node_put(node); return 0; } diff --git a/net/tipc/topsrv.c b/net/tipc/topsrv.c index af530c9ed840..82e9f44e2fe4 100644 --- a/net/tipc/topsrv.c +++ b/net/tipc/topsrv.c @@ -55,7 +55,7 @@ /** * struct tipc_topsrv - TIPC server structure * @conn_idr: identifier set of connection - * @idr_lock: protect the connection identifier set + * @idr_lock: protect the connection identifier set and listener * @idr_in_use: amount of allocated identifier entry * @net: network namespace instance * @awork: accept work item @@ -218,6 +218,10 @@ static struct tipc_conn *tipc_conn_lookup(struct tipc_topsrv *s, int conid) struct tipc_conn *con; spin_lock_bh(&s->idr_lock); + if (!s->listener) { + spin_unlock_bh(&s->idr_lock); + return NULL; + } con = idr_find(&s->conn_idr, conid); if (!connected(con) || !kref_get_unless_zero(&con->kref)) con = NULL; @@ -301,10 +305,20 @@ static void tipc_conn_send_to_sock(struct tipc_conn *con) static void tipc_conn_send_work(struct work_struct *work) { struct tipc_conn *con = container_of(work, struct tipc_conn, swork); + struct tipc_topsrv *srv; + + srv = con->server; + spin_lock_bh(&srv->idr_lock); + if (!srv->listener) { + spin_unlock_bh(&srv->idr_lock); + goto out; + } + spin_unlock_bh(&srv->idr_lock); if (connected(con)) tipc_conn_send_to_sock(con); +out: conn_put(con); } @@ -334,8 +348,14 @@ void tipc_topsrv_queue_evt(struct net *net, int conid, list_add_tail(&e->list, &con->outqueue); spin_unlock_bh(&con->outqueue_lock); - if (queue_work(srv->send_wq, &con->swork)) - return; + spin_lock_bh(&srv->idr_lock); + if (srv->listener) { + if (queue_work(srv->send_wq, &con->swork)) { + spin_unlock_bh(&srv->idr_lock); + return; + } + } + spin_unlock_bh(&srv->idr_lock); err: conn_put(con); } @@ -346,14 +366,20 @@ void tipc_topsrv_queue_evt(struct net *net, int conid, */ static void tipc_conn_write_space(struct sock *sk) { + struct tipc_topsrv *srv; struct tipc_conn *con; read_lock_bh(&sk->sk_callback_lock); con = sk->sk_user_data; if (connected(con)) { - conn_get(con); - if (!queue_work(con->server->send_wq, &con->swork)) - conn_put(con); + srv = con->server; + spin_lock_bh(&srv->idr_lock); + if (srv->listener) { + conn_get(con); + if (!queue_work(srv->send_wq, &con->swork)) + conn_put(con); + } + spin_unlock_bh(&srv->idr_lock); } read_unlock_bh(&sk->sk_callback_lock); } @@ -418,8 +444,17 @@ static int tipc_conn_rcv_from_sock(struct tipc_conn *con) static void tipc_conn_recv_work(struct work_struct *work) { struct tipc_conn *con = container_of(work, struct tipc_conn, rwork); + struct tipc_topsrv *srv; int count = 0; + srv = con->server; + spin_lock_bh(&srv->idr_lock); + if (!srv->listener) { + spin_unlock_bh(&srv->idr_lock); + goto out; + } + spin_unlock_bh(&srv->idr_lock); + while (connected(con)) { if (tipc_conn_rcv_from_sock(con)) break; @@ -430,6 +465,7 @@ static void tipc_conn_recv_work(struct work_struct *work) count = 0; } } +out: conn_put(con); } @@ -438,6 +474,7 @@ static void tipc_conn_recv_work(struct work_struct *work) */ static void tipc_conn_data_ready(struct sock *sk) { + struct tipc_topsrv *srv; struct tipc_conn *con; trace_sk_data_ready(sk); @@ -445,9 +482,14 @@ static void tipc_conn_data_ready(struct sock *sk) read_lock_bh(&sk->sk_callback_lock); con = sk->sk_user_data; if (connected(con)) { - conn_get(con); - if (!queue_work(con->server->rcv_wq, &con->rwork)) - conn_put(con); + srv = con->server; + spin_lock_bh(&srv->idr_lock); + if (srv->listener) { + conn_get(con); + if (!queue_work(srv->rcv_wq, &con->rwork)) + conn_put(con); + } + spin_unlock_bh(&srv->idr_lock); } read_unlock_bh(&sk->sk_callback_lock); } @@ -503,8 +545,12 @@ static void tipc_topsrv_listener_data_ready(struct sock *sk) read_lock_bh(&sk->sk_callback_lock); srv = sk->sk_user_data; - if (srv) - queue_work(srv->rcv_wq, &srv->awork); + if (srv) { + spin_lock_bh(&srv->idr_lock); + if (srv->listener) + queue_work(srv->rcv_wq, &srv->awork); + spin_unlock_bh(&srv->idr_lock); + } read_unlock_bh(&sk->sk_callback_lock); } @@ -700,23 +746,27 @@ static void tipc_topsrv_stop(struct net *net) struct tipc_conn *con; int id; + spin_lock_bh(&srv->idr_lock); + srv->listener = NULL; + spin_unlock_bh(&srv->idr_lock); + tipc_topsrv_work_stop(srv); + spin_lock_bh(&srv->idr_lock); for (id = 0; srv->idr_in_use; id++) { con = idr_find(&srv->conn_idr, id); if (con) { - conn_get(con); spin_unlock_bh(&srv->idr_lock); tipc_conn_close(con); - conn_put(con); spin_lock_bh(&srv->idr_lock); + continue; } + spin_unlock_bh(&srv->idr_lock); + spin_lock_bh(&srv->idr_lock); } __module_get(lsock->ops->owner); __module_get(lsock->sk->sk_prot_creator->owner); - srv->listener = NULL; spin_unlock_bh(&srv->idr_lock); - tipc_topsrv_work_stop(srv); sock_release(lsock); idr_destroy(&srv->conn_idr); kfree(srv); diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 4a1dfde8414f..f271a095996c 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -1021,6 +1021,8 @@ static void vsock_sk_destruct(struct sock *sk) { struct vsock_sock *vsk = vsock_sk(sk); + /* Socket buffers may remain in the queue in VMCI datagram mode */ + __skb_queue_purge(&sk->sk_receive_queue); /* Flush MSG_ZEROCOPY leftovers. */ __skb_queue_purge(&sk->sk_error_queue); diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 8becad81279c..1f3fd0895f7c 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -1916,6 +1916,7 @@ int virtio_transport_read_skb(struct vsock_sock *vsk, skb_read_actor_t recv_acto struct sock *sk = sk_vsock(vsk); struct virtio_vsock_hdr *hdr; struct sk_buff *skb; + u32 bytes_read; u32 pkt_len; int off = 0; int err; @@ -1935,7 +1936,8 @@ int virtio_transport_read_skb(struct vsock_sock *vsk, skb_read_actor_t recv_acto vvs->msg_count--; pkt_len = le32_to_cpu(hdr->len); - virtio_transport_dec_rx_pkt(vvs, pkt_len, pkt_len); + bytes_read = skb->len - VIRTIO_VSOCK_SKB_CB(skb)->offset; + virtio_transport_dec_rx_pkt(vvs, bytes_read, pkt_len); spin_unlock_bh(&vvs->rx_lock); virtio_transport_send_credit_update(vsk); diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c index e305ba32e356..f1f612cd7b43 100644 --- a/net/xfrm/xfrm_output.c +++ b/net/xfrm/xfrm_output.c @@ -823,16 +823,22 @@ int xfrm_output(struct sock *sk, struct sk_buff *skb) } if (skb->ip_summed == CHECKSUM_PARTIAL) { - err = skb_checksum_help(skb); - if (err) { - XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR); - kfree_skb(skb); - return err; + if (skb->protocol == htons(ETH_P_IP)) { + err = ip_check_csum_start(skb); + if (err) + goto error; } + err = skb_checksum_help(skb); + if (err) + goto error; } out: return xfrm_output2(net, sk, skb); +error: + XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR); + kfree_skb(skb); + return err; } EXPORT_SYMBOL_GPL(xfrm_output); diff --git a/security/keys/key.c b/security/keys/key.c index b34a64d81d47..a438c4508595 100644 --- a/security/keys/key.c +++ b/security/keys/key.c @@ -840,9 +840,6 @@ static key_ref_t __key_create_or_update(key_ref_t keyring_ref, key_check(keyring); - if (!(flags & KEY_ALLOC_BYPASS_RESTRICTION)) - restrict_link = keyring->restrict_link; - key_ref = ERR_PTR(-ENOTDIR); if (keyring->type != &key_type_keyring) goto error_put_type; @@ -880,6 +877,9 @@ static key_ref_t __key_create_or_update(key_ref_t keyring_ref, goto error_link_end; } + if (!(flags & KEY_ALLOC_BYPASS_RESTRICTION)) + restrict_link = keyring->restrict_link; + if (restrict_link && restrict_link->check) { ret = restrict_link->check(keyring, index_key.type, &prep.payload, restrict_link->key); diff --git a/security/keys/persistent.c b/security/keys/persistent.c index 97af230aa4b2..bdab99539326 100644 --- a/security/keys/persistent.c +++ b/security/keys/persistent.c @@ -33,25 +33,31 @@ static int key_create_persistent_register(struct user_namespace *ns) } /* - * Create the persistent keyring for the specified user. + * Get or create the persistent keyring for the specified user. * * Called with the namespace's sem locked for writing. + * + * Returns a boolean indicating whether the keyring already existed, + * or a negative error. On success, *persistent_ref holds a reference + * to the keyring. */ -static key_ref_t key_create_persistent(struct user_namespace *ns, kuid_t uid, - struct keyring_index_key *index_key) +static int key_get_or_create_persistent(struct user_namespace *ns, kuid_t uid, + struct keyring_index_key *index_key, + key_ref_t *persistent_ref) { struct key *persistent; - key_ref_t reg_ref, persistent_ref; + key_ref_t reg_ref; if (!ns->persistent_keyring_register) { - long err = key_create_persistent_register(ns); + int err = key_create_persistent_register(ns); + if (err < 0) - return ERR_PTR(err); + return err; } else { reg_ref = make_key_ref(ns->persistent_keyring_register, true); - persistent_ref = find_key_to_update(reg_ref, index_key); - if (persistent_ref) - return persistent_ref; + *persistent_ref = find_key_to_update(reg_ref, index_key); + if (*persistent_ref) + return 1; } persistent = keyring_alloc(index_key->description, @@ -61,9 +67,10 @@ static key_ref_t key_create_persistent(struct user_namespace *ns, kuid_t uid, KEY_ALLOC_NOT_IN_QUOTA, NULL, ns->persistent_keyring_register); if (IS_ERR(persistent)) - return ERR_CAST(persistent); + return PTR_ERR(persistent); - return make_key_ref(persistent, true); + *persistent_ref = make_key_ref(persistent, true); + return 0; } /* @@ -78,6 +85,7 @@ static long key_get_persistent(struct user_namespace *ns, kuid_t uid, key_ref_t reg_ref, persistent_ref; char buf[32]; long ret; + bool created = false; /* Look in the register if it exists */ memset(&index_key, 0, sizeof(index_key)); @@ -100,23 +108,24 @@ static long key_get_persistent(struct user_namespace *ns, kuid_t uid, * also need to create the register. */ down_write(&ns->keyring_sem); - persistent_ref = key_create_persistent(ns, uid, &index_key); + ret = key_get_or_create_persistent(ns, uid, &index_key, + &persistent_ref); up_write(&ns->keyring_sem); - if (!IS_ERR(persistent_ref)) - goto found; - - return PTR_ERR(persistent_ref); + if (ret < 0) + return ret; + created = ret == 0; found: + persistent = key_ref_to_ptr(persistent_ref); ret = key_task_permission(persistent_ref, current_cred(), KEY_NEED_LINK); - if (ret == 0) { - persistent = key_ref_to_ptr(persistent_ref); + if (ret == 0) ret = key_link(key_ref_to_ptr(dest_ref), persistent); - if (ret == 0) { - key_set_timeout(persistent, persistent_keyring_expiry); - ret = persistent->serial; - } - } + + if (ret == 0 || created) + key_set_timeout(persistent, persistent_keyring_expiry); + + if (ret == 0) + ret = persistent->serial; key_ref_put(persistent_ref); return ret; diff --git a/security/selinux/avc.c b/security/selinux/avc.c index 221c41978dc5..436b9af7b92c 100644 --- a/security/selinux/avc.c +++ b/security/selinux/avc.c @@ -579,7 +579,7 @@ static int avc_latest_notif_update(u32 seqno, int is_insert) } } else { if (seqno > selinux_avc.avc_cache.latest_notif) - selinux_avc.avc_cache.latest_notif = seqno; + WRITE_ONCE(selinux_avc.avc_cache.latest_notif, seqno); } spin_unlock_irqrestore(¬if_lock, flag); @@ -1207,5 +1207,6 @@ int avc_has_perm(u32 ssid, u32 tsid, u16 tclass, u32 avc_policy_seqno(void) { - return selinux_avc.avc_cache.latest_notif; + /* Writers hold notif_lock; readers only sample the sequence number. */ + return READ_ONCE(selinux_avc.avc_cache.latest_notif); } diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 5832131f29c4..cf7297072ec0 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -688,8 +688,12 @@ static int selinux_set_mnt_opts(struct super_block *sb, * will be used for both mounts) */ if ((sbsec->flags & SE_SBINITIALIZED) && (sb->s_type->fs_flags & FS_BINARY_MOUNTDATA) - && !opts) + && !opts) { + if ((kern_flags & SECURITY_LSM_NATIVE_LABELS) && + sbsec->behavior == SECURITY_FS_USE_NATIVE) + *set_kern_flags |= SECURITY_LSM_NATIVE_LABELS; goto out; + } root_isec = backing_inode_security_novalidate(root); @@ -733,6 +737,9 @@ static int selinux_set_mnt_opts(struct super_block *sb, /* previously mounted with options, but not on this attempt? */ if ((sbsec->flags & SE_MNTMASK) && !opts) goto out_double_mount; + if ((kern_flags & SECURITY_LSM_NATIVE_LABELS) && + sbsec->behavior == SECURITY_FS_USE_NATIVE) + *set_kern_flags |= SECURITY_LSM_NATIVE_LABELS; rc = 0; goto out; } diff --git a/sound/hda/codecs/side-codecs/tas2781_hda.c b/sound/hda/codecs/side-codecs/tas2781_hda.c index b22f93424c62..e0da299df780 100644 --- a/sound/hda/codecs/side-codecs/tas2781_hda.c +++ b/sound/hda/codecs/side-codecs/tas2781_hda.c @@ -139,9 +139,21 @@ static void tas2781_apply_calib(struct tasdevice_priv *p) */ crc = crc32(~0, data, 84) ^ ~0; if (crc != tmp_val[21]) { - cali_data->total_sz = 0; - dev_err(p->dev, "%s: V1 CRC error\n", __func__); - return; + /* + * Some Lenovo BIOSes (e.g. Legion Pro 7 16IRX9H, + * SSID 17aa:38cd) store valid V1 calibration data + * but leave TimeStamp and CRC zeroed. + */ + if (!tmp_val[20] && !tmp_val[21] && + tmp_val[0] && tmp_val[1]) { + dev_warn(p->dev, + "%s: V1 data without CRC, accepting\n", + __func__); + } else { + cali_data->total_sz = 0; + dev_err(p->dev, "%s: V1 CRC error\n", __func__); + return; + } } for (j = p->ndev - 1; j >= 0; j--) { diff --git a/sound/pci/ctxfi/ctdaio.c b/sound/pci/ctxfi/ctdaio.c index 69aacd06716c..59ed96fb938a 100644 --- a/sound/pci/ctxfi/ctdaio.c +++ b/sound/pci/ctxfi/ctdaio.c @@ -221,14 +221,18 @@ static int dao_set_right_input(struct dao *dao, struct rsc *input) static int dao_clear_input(struct dao *dao, unsigned int start, unsigned int end) { + struct imapper *entry; unsigned int i; - if (!dao->imappers[start]) + entry = dao->imappers[start]; + if (!entry) return 0; + for (i = start; i < end; i++) { dao->mgr->imap_delete(dao->mgr, dao->imappers[i]); dao->imappers[i] = NULL; } + kfree(entry); return 0; } diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c index 79858fc94806..7eaf15a27b7b 100644 --- a/sound/soc/amd/yc/acp6x-mach.c +++ b/sound/soc/amd/yc/acp6x-mach.c @@ -759,6 +759,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = { DMI_MATCH(DMI_BOARD_NAME, "Herbag_MDU"), } }, + { + .driver_data = &acp6x_card, + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "MDC"), + DMI_MATCH(DMI_BOARD_NAME, "Herbag2_MDU"), + } + }, { .driver_data = &acp6x_card, .matches = { diff --git a/sound/soc/codecs/adau1372.c b/sound/soc/codecs/adau1372.c index cc174ec3a1f8..c3d3be898ba6 100644 --- a/sound/soc/codecs/adau1372.c +++ b/sound/soc/codecs/adau1372.c @@ -798,7 +798,7 @@ static int adau1372_set_power(struct adau1372 *adau1372, bool enable) if (ret) return ret; if (adau1372->pd_gpio) - gpiod_set_value(adau1372->pd_gpio, 0); + gpiod_set_value_cansleep(adau1372->pd_gpio, 0); if (adau1372->switch_mode) adau1372->switch_mode(adau1372->dev); @@ -818,8 +818,7 @@ static int adau1372_set_power(struct adau1372 *adau1372, bool enable) ADAU1372_CLK_CTRL_PLL_EN, 0); regcache_cache_only(adau1372->regmap, true); - if (adau1372->pd_gpio) - gpiod_set_value(adau1372->pd_gpio, 1); + gpiod_set_value_cansleep(adau1372->pd_gpio, 1); clk_disable_unprepare(adau1372->mclk); return ret; } @@ -836,7 +835,7 @@ static int adau1372_set_power(struct adau1372 *adau1372, bool enable) * map. No need to do any register writes to manually * turn things off. */ - gpiod_set_value(adau1372->pd_gpio, 1); + gpiod_set_value_cansleep(adau1372->pd_gpio, 1); regcache_mark_dirty(adau1372->regmap); } else { regmap_update_bits(adau1372->regmap, ADAU1372_REG_CLK_CTRL, @@ -1003,6 +1002,17 @@ int adau1372_probe(struct device *dev, struct regmap *regmap, regmap_write(regmap, ADAU1372_REG_OP_STAGE_MUTE, 0x0); + /* + * Set sane default values for the Output ASRC and DAC input muxes, + * since the power-on reset defaults are invalid "Reserved" states. + */ + regmap_write(regmap, ADAU1372_REG_ASRCO_SOURCE_0_1, + 0x54); /* Decimator0, Decimator1 */ + regmap_write(regmap, ADAU1372_REG_ASRCO_SOURCE_2_3, + 0x76); /* Decimator2, Decimator3 */ + regmap_write(regmap, ADAU1372_REG_DAC_SOURCE, + 0xdc); /* Input ASRC0, Input ASRC1 */ + regmap_write(regmap, 0x7, 0x01); /* CLOCK OUT */ return devm_snd_soc_register_component(dev, &adau1372_driver, &adau1372_dai_driver, 1); diff --git a/sound/soc/samsung/i2s.c b/sound/soc/samsung/i2s.c index f80e8d498156..525126ecb0c3 100644 --- a/sound/soc/samsung/i2s.c +++ b/sound/soc/samsung/i2s.c @@ -1518,9 +1518,16 @@ static int samsung_i2s_probe(struct platform_device *pdev) goto err_disable_pm; priv->op_clk = clk_get_parent(priv->clk_table[CLK_I2S_RCLK_SRC]); + ret = clk_prepare_enable(priv->op_clk); + if (ret < 0) { + priv->op_clk = NULL; + goto err_unregister_clk; + } return 0; +err_unregister_clk: + i2s_unregister_clock_provider(priv); err_disable_pm: pm_runtime_disable(&pdev->dev); err_del_sec: @@ -1543,6 +1550,7 @@ static void samsung_i2s_remove(struct platform_device *pdev) i2s_unregister_clock_provider(priv); i2s_delete_secondary_device(priv); + clk_disable_unprepare(priv->op_clk); clk_disable_unprepare(priv->clk); pm_runtime_put_noidle(&pdev->dev); diff --git a/sound/soc/sdw_utils/soc_sdw_utils.c b/sound/soc/sdw_utils/soc_sdw_utils.c index 5266720e58b3..a47f34afd1d1 100644 --- a/sound/soc/sdw_utils/soc_sdw_utils.c +++ b/sound/soc/sdw_utils/soc_sdw_utils.c @@ -797,7 +797,7 @@ struct asoc_sdw_codec_info codec_info_list[] = { }, { .direction = {false, true}, - .dai_name = "cs35l56-sdw1c", + .dai_name = "cs35l56-ot25", .dai_type = SOC_SDW_DAI_TYPE_AMP, .dailink = {SOC_SDW_UNUSED_DAI_ID, SOC_SDW_AMP_IN_DAI_ID}, .rtd_init = asoc_sdw_cs_spk_feedback_rtd_init, @@ -826,7 +826,36 @@ struct asoc_sdw_codec_info codec_info_list[] = { }, { .direction = {false, true}, - .dai_name = "cs35l56-sdw1c", + .dai_name = "cs35l56-ot25", + .dai_type = SOC_SDW_DAI_TYPE_AMP, + .dailink = {SOC_SDW_UNUSED_DAI_ID, SOC_SDW_AMP_IN_DAI_ID}, + .rtd_init = asoc_sdw_cs_spk_feedback_rtd_init, + }, + }, + .dai_num = 2, + }, + { + .vendor_id = 0x01fa, + .part_id = 0x3562, + .name_prefix = "AMP", + .is_amp = true, + .dais = { + { + .direction = {true, false}, + .dai_name = "cs35l56-sdw1", + .component_name = "cs35l56", + .dai_type = SOC_SDW_DAI_TYPE_AMP, + .dailink = {SOC_SDW_AMP_OUT_DAI_ID, SOC_SDW_UNUSED_DAI_ID}, + .init = asoc_sdw_cs_amp_init, + .rtd_init = asoc_sdw_cs_spk_rtd_init, + .controls = generic_spk_controls, + .num_controls = ARRAY_SIZE(generic_spk_controls), + .widgets = generic_spk_widgets, + .num_widgets = ARRAY_SIZE(generic_spk_widgets), + }, + { + .direction = {false, true}, + .dai_name = "cs35l56-ot25", .dai_type = SOC_SDW_DAI_TYPE_AMP, .dailink = {SOC_SDW_UNUSED_DAI_ID, SOC_SDW_AMP_IN_DAI_ID}, .rtd_init = asoc_sdw_cs_spk_feedback_rtd_init, @@ -855,7 +884,7 @@ struct asoc_sdw_codec_info codec_info_list[] = { }, { .direction = {false, true}, - .dai_name = "cs35l56-sdw1c", + .dai_name = "cs35l56-ot25", .dai_type = SOC_SDW_DAI_TYPE_AMP, .dailink = {SOC_SDW_UNUSED_DAI_ID, SOC_SDW_AMP_IN_DAI_ID}, .rtd_init = asoc_sdw_cs_spk_feedback_rtd_init, diff --git a/tools/testing/selftests/mm/hugetlb-mmap.c b/tools/testing/selftests/mm/hugetlb-mmap.c index 0f2aad1b7dbd..2edbc992e6bc 100644 --- a/tools/testing/selftests/mm/hugetlb-mmap.c +++ b/tools/testing/selftests/mm/hugetlb-mmap.c @@ -122,7 +122,7 @@ int main(int argc, char **argv) hugepage_size = default_huge_page_size(); if (!hugepage_size) ksft_exit_skip("Could not detect default hugetlb page size."); - ksft_print_msg("Default size hugepages (%lu kB)\n", hugepage_size >> 10); + ksft_print_msg("Default size hugepages (%zu kB)\n", hugepage_size >> 10); } /* munmap will fail if the length is not page aligned */ @@ -131,7 +131,7 @@ int main(int argc, char **argv) hugetlb_set_nr_pages(hugepage_size, nr); if (hugetlb_free_pages(hugepage_size) < nr) - ksft_exit_skip("Not enough %lu Kb pages\n", hugepage_size >> 10); + ksft_exit_skip("Not enough %zu Kb pages\n", hugepage_size >> 10); ksft_set_plan(2); ksft_print_msg("Mapping %lu Mbytes\n", (unsigned long)length >> 20); diff --git a/tools/testing/vma/tests/vma.c b/tools/testing/vma/tests/vma.c index 754a2da06321..b7563ff20277 100644 --- a/tools/testing/vma/tests/vma.c +++ b/tools/testing/vma/tests/vma.c @@ -38,7 +38,7 @@ static bool test_copy_vma(void) /* Move backwards and do not merge. */ vma = alloc_and_link_vma(&mm, 0x3000, 0x5000, 3, vma_flags); - vma_new = copy_vma(&vma, 0, 0x2000, 0, &need_locks); + vma_new = copy_vma(&vma, 0, 0x2000, 0, &need_locks, false); ASSERT_NE(vma_new, vma); ASSERT_EQ(vma_new->vm_start, 0); ASSERT_EQ(vma_new->vm_end, 0x2000); @@ -51,7 +51,7 @@ static bool test_copy_vma(void) vma = alloc_and_link_vma(&mm, 0, 0x2000, 0, vma_flags); vma_next = alloc_and_link_vma(&mm, 0x6000, 0x8000, 6, vma_flags); - vma_new = copy_vma(&vma, 0x4000, 0x2000, 4, &need_locks); + vma_new = copy_vma(&vma, 0x4000, 0x2000, 4, &need_locks, false); vma_assert_attached(vma_new); ASSERT_EQ(vma_new, vma_next);