From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
torvalds@linux-foundation.org, stable@vger.kernel.org
Cc: lwn@lwn.net, jslaby@suse.cz,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Subject: Re: Linux 6.12.110
Date: Mon, 14 Sep 2026 13:58:12 +0200 [thread overview]
Message-ID: <2026091412-sniff-drank-bc23@gregkh> (raw)
In-Reply-To: <2026091412-pavilion-curfew-0637@gregkh>
diff --git a/Documentation/ABI/testing/configfs-usb-gadget-uac1_legacy b/Documentation/ABI/testing/configfs-usb-gadget-uac1_legacy
index b2eaefd9bc49..6a681d219f43 100644
--- a/Documentation/ABI/testing/configfs-usb-gadget-uac1_legacy
+++ b/Documentation/ABI/testing/configfs-usb-gadget-uac1_legacy
@@ -5,8 +5,5 @@ Description:
The attributes:
audio_buf_size - audio buffer size
- fn_cap - capture pcm device file name
- fn_cntl - control device file name
- fn_play - playback pcm device file name
req_buf_size - ISO OUT endpoint request buffer size
req_count - ISO OUT endpoint request count
diff --git a/Documentation/admin-guide/cgroup-v1/memory.rst b/Documentation/admin-guide/cgroup-v1/memory.rst
index 270501db9f4e..88c483b665ec 100644
--- a/Documentation/admin-guide/cgroup-v1/memory.rst
+++ b/Documentation/admin-guide/cgroup-v1/memory.rst
@@ -47,7 +47,6 @@ Features:
- pages are linked to per-memcg LRU exclusively, and there is no global LRU.
- optionally, memory+swap usage can be accounted and limited.
- hierarchical accounting
- - soft limit
- moving (recharging) account at moving a task is selectable.
- usage threshold notifier
- memory pressure notifier
@@ -76,10 +75,9 @@ Brief summary of control files.
memory.memsw.failcnt show the number of memory+Swap hits limits
memory.max_usage_in_bytes show max memory usage recorded
memory.memsw.max_usage_in_bytes show max memory+Swap usage recorded
- memory.soft_limit_in_bytes set/show soft limit of memory usage
- This knob is not available on CONFIG_PREEMPT_RT systems.
- This knob is deprecated and shouldn't be
- used.
+ memory.soft_limit_in_bytes This knob is deprecated and has no effect.
+ Writes are ignored and reads always
+ return the maximum value.
memory.stat show various statistics
memory.use_hierarchy set/show hierarchical account enabled
This knob is deprecated and shouldn't be
@@ -346,9 +344,6 @@ memory.kmem.usage_in_bytes, or in a separate counter when it makes sense.
The main "kmem" counter is fed into the main counter, so kmem charges will
also be visible from the user counter.
-Currently no soft limit is implemented for kernel memory. It is future work
-to trigger slab reclaim when those limits are reached.
-
2.7.1 Current Kernel Memory resources accounted
-----------------------------------------------
@@ -712,42 +707,10 @@ For compatibility reasons writing 1 to memory.use_hierarchy will always pass::
THIS IS DEPRECATED!
-Soft limits allow for greater sharing of memory. The idea behind soft limits
-is to allow control groups to use as much of the memory as needed, provided
-
-a. There is no memory contention
-b. They do not exceed their hard limit
-
-When the system detects memory contention or low memory, control groups
-are pushed back to their soft limits. If the soft limit of each control
-group is very high, they are pushed back as much as possible to make
-sure that one control group does not starve the others of memory.
-
-Please note that soft limits is a best-effort feature; it comes with
-no guarantees, but it does its best to make sure that when memory is
-heavily contended for, memory is allocated based on the soft limit
-hints/setup. Currently soft limit based reclaim is set up such that
-it gets invoked from balance_pgdat (kswapd).
-
-7.1 Interface
--------------
-
-Soft limits can be setup by using the following commands (in this example we
-assume a soft limit of 256 MiB)::
-
- # echo 256M > memory.soft_limit_in_bytes
-
-If we want to change this to 1G, we can at any time use::
+Writing to memory.soft_limit_in_bytes has no effect and reading it will
+always return the maximum value.
- # echo 1G > memory.soft_limit_in_bytes
-
-.. note::
- Soft limits take effect over a long period of time, since they involve
- reclaiming memory for balancing between memory cgroups
-
-.. note::
- It is recommended to set the soft limit always below the hard limit,
- otherwise the hard limit will take precedence.
+Use memory.low and memory.min in cgroup v2 instead.
.. _cgroup-v1-memory-move-charges:
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 811345669be4..886095f45290 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3362,8 +3362,8 @@
[KNL] Set the initial state for the memory hotplug
onlining policy. If not specified, the default value is
set according to the
- CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE kernel config
- option.
+ CONFIG_MHP_DEFAULT_ONLINE_TYPE kernel config
+ options.
See Documentation/admin-guide/mm/memory-hotplug.rst.
memmap=exactmap [KNL,X86,EARLY] Enable setting of an exact
@@ -4860,6 +4860,14 @@
Format: <bool>
default: 0 (auto_verbose is enabled)
+ printk.debug_non_panic_cpus=
+ Allows storing messages from non-panic CPUs into
+ the printk log buffer during panic(). They are
+ flushed to consoles by the panic-CPU on
+ a best-effort basis.
+ Format: <bool> (1/Y/y=enable, 0/N/n=disable)
+ Default: disabled
+
printk.devkmsg={on,off,ratelimit}
Control writing to /dev/kmsg.
on - unlimited logging to /dev/kmsg from userspace
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
index cb2c080f400c..33c886f3d198 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -280,8 +280,8 @@ The following files are currently defined:
blocks; configure auto-onlining.
The default value depends on the
- CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE kernel configuration
- option.
+ CONFIG_MHP_DEFAULT_ONLINE_TYPE kernel configuration
+ options.
See the ``state`` property of memory blocks for details.
``block_size_bytes`` read-only: the size in bytes of a memory block.
diff --git a/Documentation/admin-guide/pm/amd-pstate.rst b/Documentation/admin-guide/pm/amd-pstate.rst
index 210a808b74ec..412423c54f25 100644
--- a/Documentation/admin-guide/pm/amd-pstate.rst
+++ b/Documentation/admin-guide/pm/amd-pstate.rst
@@ -251,9 +251,7 @@ performance supported in `AMD CPPC Performance Capability <perf_cap_>`_).
In some ASICs, the highest CPPC performance is not the one in the ``_CPC``
table, so we need to expose it to sysfs. If boost is not active, but
still supported, this maximum frequency will be larger than the one in
-``cpuinfo``. On systems that support preferred core, the driver will have
-different values for some cores than others and this will reflect the values
-advertised by the platform at bootup.
+``cpuinfo``.
This attribute is read-only.
``amd_pstate_lowest_nonlinear_freq``
diff --git a/Documentation/hwmon/emc1403.rst b/Documentation/hwmon/emc1403.rst
index 57f833b1a800..ebf2435a76a6 100644
--- a/Documentation/hwmon/emc1403.rst
+++ b/Documentation/hwmon/emc1403.rst
@@ -71,10 +71,10 @@ and EMC14x8 support eight sensors (one internal, seven external).
The chips implement three limits for each sensor: low (tempX_min), high
(tempX_max) and critical (tempX_crit.) The chips also implement an
-hysteresis mechanism which applies to all limits. The relative difference
-is stored in a single register on the chip, which means that the relative
-difference between the limit and its hysteresis is always the same for
-all three limits.
+hysteresis mechanism which applies to high and critical limits. The relative
+difference is stored in a single register on the chip, which means that the
+relative difference between the limit and its hysteresis is always the same
+for high and critical limits.
This implementation detail implies the following:
diff --git a/Documentation/usb/gadget-testing.rst b/Documentation/usb/gadget-testing.rst
index bf555c2270f5..3ef109ad12d5 100644
--- a/Documentation/usb/gadget-testing.rst
+++ b/Documentation/usb/gadget-testing.rst
@@ -712,9 +712,6 @@ The uac1 function provides these attributes in its function directory:
=============== ====================================
audio_buf_size audio buffer size
- fn_cap capture pcm device file name
- fn_cntl control device file name
- fn_play playback pcm device file name
req_buf_size ISO OUT endpoint request buffer size
req_count ISO OUT endpoint request count
=============== ====================================
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-colorimetry.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-colorimetry.rst
index 1e7265155715..fb0fe51ad0f5 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-colorimetry.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-colorimetry.rst
@@ -79,15 +79,25 @@ Colorimetry Control IDs
- ``white_point_x``
- Specifies the normalized x chromaticity coordinate of the white
point of the mastering display in increments of 0.00002.
+ When both ``white_point_x`` and ``white_point_y`` are zero,
+ the white point chromaticity is unknown. If either coordinate is
+ non-zero, both coordinates shall be within their valid ranges.
* - __u16
- ``white_point_y``
- Specifies the normalized y chromaticity coordinate of the white
point of the mastering display in increments of 0.00002.
+ When both ``white_point_x`` and ``white_point_y`` are zero,
+ the white point chromaticity is unknown. If either coordinate is
+ non-zero, both coordinates shall be within their valid ranges.
* - __u32
- ``max_luminance``
- Specifies the nominal maximum display luminance of the mastering
display in units of 0.0001 cd/m\ :sup:`2`.
+ A value of zero indicates that the nominal maximum display
+ luminance is unknown.
* - __u32
- ``min_luminance``
- - specifies the nominal minimum display luminance of the mastering
+ - Specifies the nominal minimum display luminance of the mastering
display in units of 0.0001 cd/m\ :sup:`2`.
+ A value of zero indicates that the nominal minimum display
+ luminance is unknown.
diff --git a/MAINTAINERS b/MAINTAINERS
index d5d394a5ad6b..a9a03029ecb8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11257,6 +11257,7 @@ R: Eric Snowberg <eric.snowberg@oracle.com>
L: linux-integrity@vger.kernel.org
S: Supported
T: git git://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity.git
+F: include/linux/secure_boot.h
F: security/integrity/
F: security/integrity/ima/
diff --git a/Makefile b/Makefile
index 42de57a52201..679576d1fecd 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 6
PATCHLEVEL = 12
-SUBLEVEL = 109
+SUBLEVEL = 110
EXTRAVERSION =
NAME = Baby Opossum Posse
diff --git a/arch/Kconfig b/arch/Kconfig
index 1812e4e4d714..cdb220f6c2cb 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1684,4 +1684,7 @@ config CC_HAS_SANE_FUNCTION_ALIGNMENT
config ARCH_NEED_CMPXCHG_1_EMU
bool
+config HAVE_ARCH_GET_SECUREBOOT
+ def_bool EFI
+
endmenu
diff --git a/arch/arm/boot/dts/allwinner/sun4i-a10.dtsi b/arch/arm/boot/dts/allwinner/sun4i-a10.dtsi
index 51a6464aab9a..cabf619c2e21 100644
--- a/arch/arm/boot/dts/allwinner/sun4i-a10.dtsi
+++ b/arch/arm/boot/dts/allwinner/sun4i-a10.dtsi
@@ -185,7 +185,7 @@ de: display-engine {
pmu {
compatible = "arm,cortex-a8-pmu";
- interrupts = <3>;
+ interrupts = <66>;
};
reserved-memory {
diff --git a/arch/arm/boot/dts/marvell/armada-388-helios4.dts b/arch/arm/boot/dts/marvell/armada-388-helios4.dts
index ec134e22bae3..003b0708e965 100644
--- a/arch/arm/boot/dts/marvell/armada-388-helios4.dts
+++ b/arch/arm/boot/dts/marvell/armada-388-helios4.dts
@@ -169,6 +169,7 @@ expander0: gpio-expander@20 {
gpio-controller;
#gpio-cells = <2>;
reg = <0x20>;
+ vcc-supply = <®_3p3v>;
pinctrl-names = "default";
pinctrl-0 = <&pca0_pins>;
interrupt-parent = <&gpio0>;
@@ -201,6 +202,10 @@ temp_sensor: temp@4c {
reg = <0x4c>;
vcc-supply = <®_3p3v>;
};
+
+ eeprom@53 {
+ vcc-supply = <®_3p3v>;
+ };
};
i2c@11100 {
@@ -217,13 +222,17 @@ sata@a8000 {
status = "okay";
#address-cells = <1>;
#size-cells = <0>;
+ ahci-supply = <®_3p3v>;
+ phy-supply = <®_3p3v>;
sata0: sata-port@0 {
reg = <0>;
+ target-supply = <®_5p0v_hdd>;
};
sata1: sata-port@1 {
reg = <1>;
+ target-supply = <®_5p0v_hdd>;
};
};
@@ -231,13 +240,17 @@ sata@e0000 {
status = "okay";
#address-cells = <1>;
#size-cells = <0>;
+ ahci-supply = <®_3p3v>;
+ phy-supply = <®_3p3v>;
sata2: sata-port@0 {
reg = <0>;
+ target-supply = <®_5p0v_hdd>;
};
sata3: sata-port@1 {
reg = <1>;
+ target-supply = <®_5p0v_hdd>;
};
};
diff --git a/arch/arm/kernel/hw_breakpoint.c b/arch/arm/kernel/hw_breakpoint.c
index a12efd0f43e8..6819d4c1f352 100644
--- a/arch/arm/kernel/hw_breakpoint.c
+++ b/arch/arm/kernel/hw_breakpoint.c
@@ -929,10 +929,6 @@ static void hw_breakpoint_cfi_handler(struct pt_regs *regs)
break;
}
}
-#else
-static void hw_breakpoint_cfi_handler(struct pt_regs *regs)
-{
-}
#endif
/*
@@ -964,9 +960,14 @@ static int hw_breakpoint_pending(unsigned long addr, unsigned int fsr,
case ARM_ENTRY_SYNC_WATCHPOINT:
watchpoint_handler(addr, fsr, regs);
break;
+#ifdef CONFIG_CFI
case ARM_ENTRY_CFI_BREAKPOINT:
- hw_breakpoint_cfi_handler(regs);
+ if (user_mode(regs))
+ ret = 1; /* Don't handle userspace BKPT */
+ else
+ hw_breakpoint_cfi_handler(regs);
break;
+#endif
default:
ret = 1; /* Unhandled fault. */
}
diff --git a/arch/arm/mach-imx/src.c b/arch/arm/mach-imx/src.c
index 59a8e8cc4469..c3c80b4c3d53 100644
--- a/arch/arm/mach-imx/src.c
+++ b/arch/arm/mach-imx/src.c
@@ -171,6 +171,7 @@ void __init imx_src_init(void)
if (!np)
return;
src_base = of_iomap(np, 0);
+ of_node_put(np);
WARN_ON(!src_base);
/*
@@ -195,6 +196,7 @@ void __init imx7_src_init(void)
return;
src_base = of_iomap(np, 0);
+ of_node_put(np);
if (!src_base)
return;
@@ -203,6 +205,7 @@ void __init imx7_src_init(void)
return;
gpc_base = of_iomap(np, 0);
+ of_node_put(np);
if (!gpc_base)
return;
}
diff --git a/arch/arm/mach-lpc32xx/common.c b/arch/arm/mach-lpc32xx/common.c
index 304ea61a0716..35ed3569c5a3 100644
--- a/arch/arm/mach-lpc32xx/common.c
+++ b/arch/arm/mach-lpc32xx/common.c
@@ -106,7 +106,7 @@ void __init lpc32xx_map_io(void)
iotable_init(lpc32xx_io_desc, ARRAY_SIZE(lpc32xx_io_desc));
}
-static int __init lpc32xx_check_uid(void)
+void __init lpc32xx_check_uid(void)
{
u32 uid[4];
@@ -119,7 +119,4 @@ static int __init lpc32xx_check_uid(void)
system_serial_low = uid[0];
system_serial_high = uid[1];
}
-
- return 1;
}
-arch_initcall(lpc32xx_check_uid);
diff --git a/arch/arm/mach-lpc32xx/common.h b/arch/arm/mach-lpc32xx/common.h
index 32f0ad217807..06b20bea324e 100644
--- a/arch/arm/mach-lpc32xx/common.h
+++ b/arch/arm/mach-lpc32xx/common.h
@@ -16,6 +16,8 @@
* Other arch specific structures and functions
*/
extern void __init lpc32xx_map_io(void);
+extern void __init lpc32xx_check_uid(void);
+extern void __init lpc32xx_pm_init(void);
extern void __init lpc32xx_serial_init(void);
/*
diff --git a/arch/arm/mach-lpc32xx/phy3250.c b/arch/arm/mach-lpc32xx/phy3250.c
index 66701bf43248..ddc6333ca55d 100644
--- a/arch/arm/mach-lpc32xx/phy3250.c
+++ b/arch/arm/mach-lpc32xx/phy3250.c
@@ -71,6 +71,8 @@ static const struct of_dev_auxdata lpc32xx_auxdata_lookup[] __initconst = {
static void __init lpc3250_machine_init(void)
{
+ lpc32xx_check_uid();
+ lpc32xx_pm_init();
lpc32xx_serial_init();
of_platform_default_populate(NULL, lpc32xx_auxdata_lookup, NULL);
diff --git a/arch/arm/mach-lpc32xx/pm.c b/arch/arm/mach-lpc32xx/pm.c
index 2572bd89a5e8..9b5c5e1462ed 100644
--- a/arch/arm/mach-lpc32xx/pm.c
+++ b/arch/arm/mach-lpc32xx/pm.c
@@ -120,7 +120,7 @@ static const struct platform_suspend_ops lpc32xx_pm_ops = {
#define EMC_DYN_MEM_CTRL_OFS 0x20
#define EMC_SRMMC (1 << 3)
#define EMC_CTRL_REG io_p2v(LPC32XX_EMC_BASE + EMC_DYN_MEM_CTRL_OFS)
-static int __init lpc32xx_pm_init(void)
+void __init lpc32xx_pm_init(void)
{
/*
* Setup SDRAM self-refresh clock to automatically disable o
@@ -129,7 +129,4 @@ static int __init lpc32xx_pm_init(void)
__raw_writel(__raw_readl(EMC_CTRL_REG) | EMC_SRMMC, EMC_CTRL_REG);
suspend_set_ops(&lpc32xx_pm_ops);
-
- return 0;
}
-arch_initcall(lpc32xx_pm_init);
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index 16b5a7d21480..6763a7c66078 100644
--- a/arch/arm/mm/fault.c
+++ b/arch/arm/mm/fault.c
@@ -205,7 +205,11 @@ __do_user_fault(unsigned long addr, unsigned int fsr, unsigned int sig,
pr_err("8<--- cut here ---\n");
pr_err("%s: unhandled page fault (%d) at 0x%08lx, code 0x%03x\n",
tsk->comm, sig, addr, fsr);
- show_pte(KERN_ERR, tsk->mm, addr);
+ if (likely(addr < TASK_SIZE)) {
+ mmap_write_lock(tsk->mm);
+ show_pte(KERN_ERR, tsk->mm, addr);
+ mmap_write_unlock(tsk->mm);
+ }
show_regs(regs);
}
#endif
@@ -649,7 +653,15 @@ do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
pr_alert("8<--- cut here ---\n");
pr_alert("Unhandled fault: %s (0x%03x) at 0x%08lx\n",
inf->name, fsr, addr);
- show_pte(KERN_ALERT, current->mm, addr);
+ if (likely(user_mode(regs))) {
+ if (addr < TASK_SIZE) {
+ mmap_write_lock(current->mm);
+ show_pte(KERN_ALERT, current->mm, addr);
+ mmap_write_unlock(current->mm);
+ }
+ } else {
+ show_pte(KERN_ALERT, current->mm, addr);
+ }
arm_notify_die("", regs, inf->sig, inf->code, (void __user *)addr,
fsr, 0);
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index dd98fa97aad0..938bf2f96b1a 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -38,6 +38,8 @@ config ARM64
select ARCH_HAS_MEM_ENCRYPT
select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS
select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
+ select ARCH_HAS_NONLEAF_PMD_YOUNG if ARM64_HAFT
+ select ARCH_HAS_PTDUMP
select ARCH_HAS_PTE_DEVMAP
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_HW_PTE_YOUNG
@@ -152,7 +154,6 @@ config ARM64
select GENERIC_IRQ_SHOW_LEVEL
select GENERIC_LIB_DEVMEM_IS_ALLOWED
select GENERIC_PCI_IOMAP
- select GENERIC_PTDUMP
select GENERIC_SCHED_CLOCK
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
diff --git a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
index b844759f52c0..82f959aace10 100644
--- a/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
+++ b/arch/arm64/boot/dts/allwinner/sun50i-a64-pinephone.dtsi
@@ -212,8 +212,8 @@ accelerometer@68 {
interrupts = <7 5 IRQ_TYPE_EDGE_RISING>; /* PH5 */
vdd-supply = <®_dldo1>;
vddio-supply = <®_dldo1>;
- mount-matrix = "0", "1", "0",
- "-1", "0", "0",
+ mount-matrix = "0", "-1", "0",
+ "1", "0", "0",
"0", "0", "1";
};
};
diff --git a/arch/arm64/boot/dts/amlogic/meson-axg-s400.dts b/arch/arm64/boot/dts/amlogic/meson-axg-s400.dts
index 9611775b81ee..7c8a8529dbc5 100644
--- a/arch/arm64/boot/dts/amlogic/meson-axg-s400.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-axg-s400.dts
@@ -432,6 +432,10 @@ gpio_speaker: gpio-controller@1f {
};
};
+&mipi_pcie_analog_dphy {
+ status = "okay";
+};
+
&pdm {
pinctrl-0 = <&pdm_dclk_a14_pins>, <&pdm_din0_pins>,
<&pdm_din1_pins>, <&pdm_din2_pins>, <&pdm_din3_pins>;
diff --git a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
index 4717c9666f2a..ff05b997f6fe 100644
--- a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
@@ -454,7 +454,8 @@ mux {
"nand_ale",
"nand_cle",
"nand_wen_clk",
- "nand_ren_wr";
+ "nand_ren_wr",
+ "nand_rb0";
function = "nand";
input-enable;
bias-pull-up;
diff --git a/arch/arm64/boot/dts/freescale/imx8-ss-audio.dtsi b/arch/arm64/boot/dts/freescale/imx8-ss-audio.dtsi
index ff5df0fed9e9..741a311614dd 100644
--- a/arch/arm64/boot/dts/freescale/imx8-ss-audio.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8-ss-audio.dtsi
@@ -124,10 +124,10 @@ asrc0: asrc@59000000 {
compatible = "fsl,imx8qm-asrc";
reg = <0x59000000 0x10000>;
interrupts = <GIC_SPI 372 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&asrc0_lpcg IMX_LPCG_CLK_0>,
- <&asrc0_lpcg IMX_LPCG_CLK_0>,
- <&aud_pll_div0_lpcg IMX_LPCG_CLK_4>,
- <&aud_pll_div1_lpcg IMX_LPCG_CLK_4>,
+ clocks = <&asrc0_lpcg IMX_LPCG_CLK_4>,
+ <&asrc0_lpcg IMX_LPCG_CLK_4>,
+ <&aud_pll_div0_lpcg IMX_LPCG_CLK_0>,
+ <&aud_pll_div1_lpcg IMX_LPCG_CLK_0>,
<&acm IMX_ADMA_ACM_AUD_CLK0_SEL>,
<&acm IMX_ADMA_ACM_AUD_CLK1_SEL>,
<&clk_dummy>,
diff --git a/arch/arm64/boot/dts/freescale/imx93-kontron-osm-s.dtsi b/arch/arm64/boot/dts/freescale/imx93-kontron-osm-s.dtsi
index 47c1363a2f99..dd5da2789bc7 100644
--- a/arch/arm64/boot/dts/freescale/imx93-kontron-osm-s.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx93-kontron-osm-s.dtsi
@@ -15,9 +15,9 @@ aliases {
rtc1 = &bbnsm_rtc;
};
- memory@40000000 {
+ memory@80000000 {
device_type = "memory";
- reg = <0x0 0x40000000 0 0x80000000>;
+ reg = <0x0 0x80000000 0 0x40000000>;
};
chosen {
diff --git a/arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts b/arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts
index 54453b0a91f9..2c4b8f3f43e0 100644
--- a/arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts
+++ b/arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts
@@ -290,7 +290,8 @@ connector {
&usb3 {
status = "okay";
- phys = <&comphy2 0>;
+ phys = <&usb2_utmi_otg_phy>, <&comphy2 0>;
+ phy-names = "usb2-phy", "usb3-phy";
};
&mdio {
diff --git a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-gemini.dts b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-gemini.dts
index dbad8f57f2fa..f145e8ee5f80 100644
--- a/arch/arm64/boot/dts/qcom/msm8996-xiaomi-gemini.dts
+++ b/arch/arm64/boot/dts/qcom/msm8996-xiaomi-gemini.dts
@@ -39,7 +39,7 @@ &blsp2_i2c3 {
haptics: drv2604@5a {
compatible = "ti,drv2604";
reg = <0x5a>;
- enable-gpio = <&tlmm 93 0x00>;
+ enable-gpios = <&tlmm 93 GPIO_ACTIVE_HIGH>;
mode = <DRV260X_LRA_MODE>;
library-sel = <DRV260X_LIB_LRA>;
pinctrl-names = "default","sleep";
diff --git a/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi b/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
index d8cc0d729e99..0b6b71b5ec5d 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998-sony-xperia-yoshino.dtsi
@@ -229,10 +229,6 @@ rmi4-f11@11 {
};
};
-&blsp1_i2c5_sleep {
- bias-disable;
-};
-
&blsp1_uart3 {
status = "okay";
diff --git a/arch/arm64/boot/dts/qcom/msm8998-xiaomi-sagit.dts b/arch/arm64/boot/dts/qcom/msm8998-xiaomi-sagit.dts
index 0cac06f25a77..2ceeec827f8e 100644
--- a/arch/arm64/boot/dts/qcom/msm8998-xiaomi-sagit.dts
+++ b/arch/arm64/boot/dts/qcom/msm8998-xiaomi-sagit.dts
@@ -217,11 +217,6 @@ rmi4-f1a@1a {
};
};
-&blsp1_i2c5_sleep {
- /delete-property/ bias-pull-up;
- bias-disable;
-};
-
&blsp1_uart3 {
status = "okay";
diff --git a/arch/arm64/boot/dts/qcom/msm8998.dtsi b/arch/arm64/boot/dts/qcom/msm8998.dtsi
index 9aa9c5cee355..6524a182277d 100644
--- a/arch/arm64/boot/dts/qcom/msm8998.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998.dtsi
@@ -1166,7 +1166,7 @@ blsp1_i2c1_sleep: blsp1-i2c1-sleep-state-state {
pins = "gpio2", "gpio3";
function = "blsp_i2c1";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_i2c2_default: blsp1-i2c2-default-state {
@@ -1180,7 +1180,7 @@ blsp1_i2c2_sleep: blsp1-i2c2-sleep-state-state {
pins = "gpio32", "gpio33";
function = "blsp_i2c2";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_i2c3_default: blsp1-i2c3-default-state {
@@ -1194,7 +1194,7 @@ blsp1_i2c3_sleep: blsp1-i2c3-sleep-state {
pins = "gpio47", "gpio48";
function = "blsp_i2c3";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_i2c4_default: blsp1-i2c4-default-state {
@@ -1208,7 +1208,7 @@ blsp1_i2c4_sleep: blsp1-i2c4-sleep-state {
pins = "gpio10", "gpio11";
function = "blsp_i2c4";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_i2c5_default: blsp1-i2c5-default-state {
@@ -1222,7 +1222,7 @@ blsp1_i2c5_sleep: blsp1-i2c5-sleep-state {
pins = "gpio87", "gpio88";
function = "blsp_i2c5";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_i2c6_default: blsp1-i2c6-default-state {
@@ -1236,7 +1236,7 @@ blsp1_i2c6_sleep: blsp1-i2c6-sleep-state {
pins = "gpio43", "gpio44";
function = "blsp_i2c6";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp1_spi_b_default: blsp1-spi-b-default-state {
@@ -1301,7 +1301,7 @@ blsp2_i2c1_sleep: blsp2-i2c1-sleep-state {
pins = "gpio55", "gpio56";
function = "blsp_i2c7";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_i2c2_default: blsp2-i2c2-default-state {
@@ -1315,7 +1315,7 @@ blsp2_i2c2_sleep: blsp2-i2c2-sleep-state {
pins = "gpio6", "gpio7";
function = "blsp_i2c8";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_i2c3_default: blsp2-i2c3-default-state {
@@ -1329,7 +1329,7 @@ blsp2_i2c3_sleep: blsp2-i2c3-sleep-state {
pins = "gpio51", "gpio52";
function = "blsp_i2c9";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_i2c4_default: blsp2-i2c4-default-state {
@@ -1343,7 +1343,7 @@ blsp2_i2c4_sleep: blsp2-i2c4-sleep-state {
pins = "gpio67", "gpio68";
function = "blsp_i2c10";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_i2c5_default: blsp2-i2c5-default-state {
@@ -1357,7 +1357,7 @@ blsp2_i2c5_sleep: blsp2-i2c5-sleep-state {
pins = "gpio60", "gpio61";
function = "blsp_i2c11";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_i2c6_default: blsp2-i2c6-default-state {
@@ -1371,7 +1371,7 @@ blsp2_i2c6_sleep: blsp2-i2c6-sleep-state {
pins = "gpio83", "gpio84";
function = "blsp_i2c12";
drive-strength = <2>;
- bias-pull-up;
+ bias-disable;
};
blsp2_spi1_default: blsp2-spi1-default-state {
diff --git a/arch/arm64/boot/dts/qcom/qcm2290.dtsi b/arch/arm64/boot/dts/qcom/qcm2290.dtsi
index dbc78b2d5095..d50b36143ebd 100644
--- a/arch/arm64/boot/dts/qcom/qcm2290.dtsi
+++ b/arch/arm64/boot/dts/qcom/qcm2290.dtsi
@@ -1798,6 +1798,7 @@ dispcc: clock-controller@5f00000 {
"gcc_disp_gpll0_div_clk_src",
"dsi0_phy_pll_out_byteclk",
"dsi0_phy_pll_out_dsiclk";
+ power-domains = <&rpmpd QCM2290_VDDCX>;
#power-domain-cells = <1>;
#clock-cells = <1>;
#reset-cells = <1>;
diff --git a/arch/arm64/boot/dts/qcom/qcs404.dtsi b/arch/arm64/boot/dts/qcom/qcs404.dtsi
index 81a161c0cc5a..3ba456d73fb2 100644
--- a/arch/arm64/boot/dts/qcom/qcs404.dtsi
+++ b/arch/arm64/boot/dts/qcom/qcs404.dtsi
@@ -677,11 +677,11 @@ usb3: usb@7678800 {
assigned-clock-rates = <19200000>, <200000000>;
interrupts = <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 319 IRQ_TYPE_LEVEL_HIGH>;
+ <GIC_SPI 319 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "pwr_event",
- "hs_phy_irq",
- "qusb2_phy";
+ "qusb2_phy",
+ "hs_phy_irq";
status = "disabled";
@@ -714,11 +714,11 @@ usb2: usb@79b8800 {
assigned-clock-rates = <19200000>, <133333333>;
interrupts = <GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI 318 IRQ_TYPE_LEVEL_HIGH>;
+ <GIC_SPI 318 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "pwr_event",
- "hs_phy_irq",
- "qusb2_phy";
+ "qusb2_phy",
+ "hs_phy_irq";
status = "disabled";
diff --git a/arch/arm64/boot/dts/qcom/qcs8550-aim300.dtsi b/arch/arm64/boot/dts/qcom/qcs8550-aim300.dtsi
index e6ac529e6b72..3ead078246f4 100644
--- a/arch/arm64/boot/dts/qcom/qcs8550-aim300.dtsi
+++ b/arch/arm64/boot/dts/qcom/qcs8550-aim300.dtsi
@@ -396,8 +396,8 @@ &usb_1_hsphy {
};
&usb_dp_qmpphy {
- vdda-phy-supply = <&vreg_l3e_1p2>;
- vdda-pll-supply = <&vreg_l3f_0p88>;
+ vdda-phy-supply = <&vreg_l3f_0p88>;
+ vdda-pll-supply = <&vreg_l3e_1p2>;
};
&xo_board {
diff --git a/arch/arm64/boot/dts/qcom/sc8180x-lenovo-flex-5g.dts b/arch/arm64/boot/dts/qcom/sc8180x-lenovo-flex-5g.dts
index 62de4774c556..debe9fa74724 100644
--- a/arch/arm64/boot/dts/qcom/sc8180x-lenovo-flex-5g.dts
+++ b/arch/arm64/boot/dts/qcom/sc8180x-lenovo-flex-5g.dts
@@ -26,6 +26,7 @@ backlight: backlight {
compatible = "pwm-backlight";
pwms = <&pmc8180c_lpg 4 1000000>;
enable-gpios = <&pmc8180c_gpios 8 GPIO_ACTIVE_HIGH>;
+ power-supply = <&vled_bl_pw>;
pinctrl-0 = <&bl_pwm_default>;
pinctrl-names = "default";
@@ -162,14 +163,39 @@ cdsp_mem: cdsp-region@98900000 {
};
};
- vph_pwr: vph-pwr-regulator {
+ vled_bl_pw: regulator-vled-bl-pw {
compatible = "regulator-fixed";
- regulator-name = "vph_pwr";
- regulator-min-microvolt = <3700000>;
- regulator-max-microvolt = <3700000>;
+
+ regulator-name = "VLED_BL_PW";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+
+ gpio = <&pmc8180_2_gpios 1 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+
+ pinctrl-0 = <&bl_pwr_en>;
+ pinctrl-names = "default";
+
+ regulator-boot-on;
};
- vreg_s4a_1p8: pm8150-s4-regulator {
+ vreg_lcm_3v3: regulator-edp-3p3 {
+ compatible = "regulator-fixed";
+
+ regulator-name = "VREG_LCM_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+
+ gpio = <&tlmm 130 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+
+ pinctrl-0 = <&lcm_3v3_en>;
+ pinctrl-names = "default";
+
+ regulator-boot-on;
+ };
+
+ vreg_s4a_1p8: regulator-pm8150-s4 {
compatible = "regulator-fixed";
regulator-name = "vreg_s4a_1p8";
@@ -182,6 +208,13 @@ vreg_s4a_1p8: pm8150-s4-regulator {
vin-supply = <&vph_pwr>;
};
+ vph_pwr: regulator-vph-pwr {
+ compatible = "regulator-fixed";
+ regulator-name = "vph_pwr";
+ regulator-min-microvolt = <3700000>;
+ regulator-max-microvolt = <3700000>;
+ };
+
usbprim-sbu-mux {
compatible = "pericom,pi3usb102", "gpio-sbu-mux";
@@ -446,6 +479,7 @@ &mdss_edp {
aux-bus {
panel {
compatible = "edp-panel";
+ power-supply = <&vreg_lcm_3v3>;
no-hpd;
backlight = <&backlight>;
@@ -484,6 +518,13 @@ &pcie3_phy {
status = "okay";
};
+&pmc8180_2_gpios {
+ bl_pwr_en: bl-pwr-en-state {
+ pins = "gpio1";
+ function = "normal";
+ };
+};
+
&pmc8180_pwrkey {
status = "okay";
};
@@ -781,6 +822,12 @@ ts_int_default: ts-int-default-state {
drive-strength = <2>;
};
+ lcm_3v3_en: lcm-3v3-en-state {
+ pins = "gpio130";
+ function = "gpio";
+ bias-disable;
+ };
+
usbprim_sbu_default: usbprim-sbu-state {
oe-n-pins {
pins = "gpio152";
diff --git a/arch/arm64/boot/dts/qcom/sc8180x-primus.dts b/arch/arm64/boot/dts/qcom/sc8180x-primus.dts
index 79b4d293ea1e..4d1d079a1f37 100644
--- a/arch/arm64/boot/dts/qcom/sc8180x-primus.dts
+++ b/arch/arm64/boot/dts/qcom/sc8180x-primus.dts
@@ -27,9 +27,10 @@ backlight: backlight {
compatible = "pwm-backlight";
pwms = <&pmc8180c_lpg 4 1000000>;
enable-gpios = <&pmc8180c_gpios 8 GPIO_ACTIVE_HIGH>;
+ power-supply = <&vled_bl_pw>;
- pinctrl-names = "default";
pinctrl-0 = <&bl_pwm_default>;
+ pinctrl-names = "default";
};
chosen {
@@ -165,7 +166,39 @@ reserved-region@9a500000 {
};
};
- vreg_nvme_0p9: nvme-0p9-regulator {
+ vled_bl_pw: regulator-vled-bl-pw {
+ compatible = "regulator-fixed";
+
+ regulator-name = "VLED_BL_PW";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+
+ gpio = <&pmc8180_2_gpios 1 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+
+ pinctrl-0 = <&bl_pwr_en>;
+ pinctrl-names = "default";
+
+ regulator-boot-on;
+ };
+
+ vreg_lcm_3v3: regulator-edp-3p3 {
+ compatible = "regulator-fixed";
+
+ regulator-name = "VREG_LCM_3V3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+
+ gpio = <&tlmm 130 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+
+ pinctrl-0 = <&lcm_3v3_en>;
+ pinctrl-names = "default";
+
+ regulator-boot-on;
+ };
+
+ vreg_nvme_0p9: regulator-nvme-0p9 {
compatible = "regulator-fixed";
regulator-name = "vreg_nvme_0p9";
@@ -175,7 +208,7 @@ vreg_nvme_0p9: nvme-0p9-regulator {
regulator-always-on;
};
- vreg_nvme_3p3: nvme-3p3-regulator {
+ vreg_nvme_3p3: regulator-nvme-3p3 {
compatible = "regulator-fixed";
regulator-name = "vreg_nvme_3p3";
@@ -188,7 +221,7 @@ vreg_nvme_3p3: nvme-3p3-regulator {
regulator-always-on;
};
- vdd_kb_tp_3v3: vdd-kb-tp-3v3-regulator {
+ vdd_kb_tp_3v3: regulator-vdd-kb-tp-3v3 {
compatible = "regulator-fixed";
regulator-name = "vdd_kb_tp_3v3";
regulator-min-microvolt = <3300000>;
@@ -203,7 +236,7 @@ vdd_kb_tp_3v3: vdd-kb-tp-3v3-regulator {
pinctrl-0 = <&kb_tp_3v3_en_active_state>;
};
- vph_pwr: vph-pwr-regulator {
+ vph_pwr: regulator-vph-pwr {
compatible = "regulator-fixed";
regulator-name = "vph_pwr";
regulator-min-microvolt = <3700000>;
@@ -541,6 +574,7 @@ &mdss_edp {
aux-bus {
panel {
compatible = "edp-panel";
+ power-supply = <&vreg_lcm_3v3>;
backlight = <&backlight>;
@@ -778,6 +812,12 @@ &xo_board_clk {
};
/* PINCTRL */
+&pmc8180_2_gpios {
+ bl_pwr_en: bl-pwr-en-state {
+ pins = "gpio1";
+ function = "normal";
+ };
+};
&pmc8180c_gpios {
bl_pwm_default: bl-pwm-default-state {
@@ -959,4 +999,10 @@ rx-pins {
bias-pull-up;
};
};
+
+ lcm_3v3_en: lcm-3v3-en-state {
+ pins = "gpio130";
+ function = "gpio";
+ bias-disable;
+ };
};
diff --git a/arch/arm64/boot/dts/qcom/sc8180x.dtsi b/arch/arm64/boot/dts/qcom/sc8180x.dtsi
index 4715e26fc264..d3aed296e32c 100644
--- a/arch/arm64/boot/dts/qcom/sc8180x.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc8180x.dtsi
@@ -1761,8 +1761,8 @@ pcie0: pcie@1c00000 {
assigned-clocks = <&gcc GCC_PCIE_0_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1d80 0x1>,
- <0x100 &apps_smmu 0x1d81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1d80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1d81 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -1882,8 +1882,8 @@ pcie3: pcie@1c08000 {
assigned-clocks = <&gcc GCC_PCIE_3_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1e00 0x1>,
- <0x100 &apps_smmu 0x1e01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1e00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1e01 0x0 0x1>;
resets = <&gcc GCC_PCIE_3_BCR>;
reset-names = "pci";
@@ -2004,8 +2004,8 @@ pcie1: pcie@1c10000 {
assigned-clocks = <&gcc GCC_PCIE_1_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1c80 0x1>,
- <0x100 &apps_smmu 0x1c81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c81 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
@@ -2126,8 +2126,8 @@ pcie2: pcie@1c18000 {
assigned-clocks = <&gcc GCC_PCIE_2_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1d00 0x1>,
- <0x100 &apps_smmu 0x1d01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1d00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1d01 0x0 0x1>;
resets = <&gcc GCC_PCIE_2_BCR>;
reset-names = "pci";
diff --git a/arch/arm64/boot/dts/qcom/sc8280xp-crd.dts b/arch/arm64/boot/dts/qcom/sc8280xp-crd.dts
index 6020582b0a59..36beb71d7d2f 100644
--- a/arch/arm64/boot/dts/qcom/sc8280xp-crd.dts
+++ b/arch/arm64/boot/dts/qcom/sc8280xp-crd.dts
@@ -167,7 +167,7 @@ vreg_misc_3p3: regulator-misc-3p3 {
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
- gpio = <&pmc8280_1_gpios 2 GPIO_ACTIVE_HIGH>;
+ gpio = <&pmc8280_1_gpios 1 GPIO_ACTIVE_HIGH>;
enable-active-high;
pinctrl-names = "default";
@@ -759,7 +759,7 @@ edp_bl_reg_en: edp-bl-reg-en-state {
};
misc_3p3_reg_en: misc-3p3-reg-en-state {
- pins = "gpio2";
+ pins = "gpio1";
function = "normal";
};
};
diff --git a/arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts b/arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts
index 8e5951da5920..4dfe29caa52d 100644
--- a/arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts
+++ b/arch/arm64/boot/dts/qcom/sc8280xp-lenovo-thinkpad-x13s.dts
@@ -1488,7 +1488,7 @@ cam_rgb_default: cam-rgb-default-state {
mclk-pins {
pins = "gpio17";
function = "cam_mclk";
- drive-strength = <16>;
+ drive-strength = <6>;
bias-disable;
};
diff --git a/arch/arm64/boot/dts/qcom/sdm632-motorola-ocean.dts b/arch/arm64/boot/dts/qcom/sdm632-motorola-ocean.dts
index 2f55db0c8ce3..9ea3e5e76bf9 100644
--- a/arch/arm64/boot/dts/qcom/sdm632-motorola-ocean.dts
+++ b/arch/arm64/boot/dts/qcom/sdm632-motorola-ocean.dts
@@ -130,7 +130,7 @@ led-controller@36 {
led: led@1 {
reg = <1>;
- default-trigger = "backlight";
+ linux,default-trigger = "backlight";
function = LED_FUNCTION_BACKLIGHT;
led-sources = <0 1 2>;
};
diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
index 54956ae1f674..0653bccf5049 100644
--- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
@@ -2350,22 +2350,22 @@ pcie0: pcie@1c00000 {
"slave_q2a",
"tbu";
- iommu-map = <0x0 &apps_smmu 0x1c10 0x1>,
- <0x100 &apps_smmu 0x1c11 0x1>,
- <0x200 &apps_smmu 0x1c12 0x1>,
- <0x300 &apps_smmu 0x1c13 0x1>,
- <0x400 &apps_smmu 0x1c14 0x1>,
- <0x500 &apps_smmu 0x1c15 0x1>,
- <0x600 &apps_smmu 0x1c16 0x1>,
- <0x700 &apps_smmu 0x1c17 0x1>,
- <0x800 &apps_smmu 0x1c18 0x1>,
- <0x900 &apps_smmu 0x1c19 0x1>,
- <0xa00 &apps_smmu 0x1c1a 0x1>,
- <0xb00 &apps_smmu 0x1c1b 0x1>,
- <0xc00 &apps_smmu 0x1c1c 0x1>,
- <0xd00 &apps_smmu 0x1c1d 0x1>,
- <0xe00 &apps_smmu 0x1c1e 0x1>,
- <0xf00 &apps_smmu 0x1c1f 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c10 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c11 0x0 0x1>,
+ <0x200 &apps_smmu 0x1c12 0x0 0x1>,
+ <0x300 &apps_smmu 0x1c13 0x0 0x1>,
+ <0x400 &apps_smmu 0x1c14 0x0 0x1>,
+ <0x500 &apps_smmu 0x1c15 0x0 0x1>,
+ <0x600 &apps_smmu 0x1c16 0x0 0x1>,
+ <0x700 &apps_smmu 0x1c17 0x0 0x1>,
+ <0x800 &apps_smmu 0x1c18 0x0 0x1>,
+ <0x900 &apps_smmu 0x1c19 0x0 0x1>,
+ <0xa00 &apps_smmu 0x1c1a 0x0 0x1>,
+ <0xb00 &apps_smmu 0x1c1b 0x0 0x1>,
+ <0xc00 &apps_smmu 0x1c1c 0x0 0x1>,
+ <0xd00 &apps_smmu 0x1c1d 0x0 0x1>,
+ <0xe00 &apps_smmu 0x1c1e 0x0 0x1>,
+ <0xf00 &apps_smmu 0x1c1f 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -2464,22 +2464,22 @@ pcie1: pcie@1c08000 {
assigned-clocks = <&gcc GCC_PCIE_1_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1c00 0x1>,
- <0x100 &apps_smmu 0x1c01 0x1>,
- <0x200 &apps_smmu 0x1c02 0x1>,
- <0x300 &apps_smmu 0x1c03 0x1>,
- <0x400 &apps_smmu 0x1c04 0x1>,
- <0x500 &apps_smmu 0x1c05 0x1>,
- <0x600 &apps_smmu 0x1c06 0x1>,
- <0x700 &apps_smmu 0x1c07 0x1>,
- <0x800 &apps_smmu 0x1c08 0x1>,
- <0x900 &apps_smmu 0x1c09 0x1>,
- <0xa00 &apps_smmu 0x1c0a 0x1>,
- <0xb00 &apps_smmu 0x1c0b 0x1>,
- <0xc00 &apps_smmu 0x1c0c 0x1>,
- <0xd00 &apps_smmu 0x1c0d 0x1>,
- <0xe00 &apps_smmu 0x1c0e 0x1>,
- <0xf00 &apps_smmu 0x1c0f 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c01 0x0 0x1>,
+ <0x200 &apps_smmu 0x1c02 0x0 0x1>,
+ <0x300 &apps_smmu 0x1c03 0x0 0x1>,
+ <0x400 &apps_smmu 0x1c04 0x0 0x1>,
+ <0x500 &apps_smmu 0x1c05 0x0 0x1>,
+ <0x600 &apps_smmu 0x1c06 0x0 0x1>,
+ <0x700 &apps_smmu 0x1c07 0x0 0x1>,
+ <0x800 &apps_smmu 0x1c08 0x0 0x1>,
+ <0x900 &apps_smmu 0x1c09 0x0 0x1>,
+ <0xa00 &apps_smmu 0x1c0a 0x0 0x1>,
+ <0xb00 &apps_smmu 0x1c0b 0x0 0x1>,
+ <0xc00 &apps_smmu 0x1c0c 0x0 0x1>,
+ <0xd00 &apps_smmu 0x1c0d 0x0 0x1>,
+ <0xe00 &apps_smmu 0x1c0e 0x0 0x1>,
+ <0xf00 &apps_smmu 0x1c0f 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
diff --git a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
index 83dac3ca5331..dfa3b111ae10 100644
--- a/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
+++ b/arch/arm64/boot/dts/qcom/sm7225-fairphone-fp4.dts
@@ -47,7 +47,7 @@ chosen {
stdout-path = "serial0:115200n8";
- framebuffer0: framebuffer@a000000 {
+ framebuffer0: framebuffer@a0000000 {
compatible = "simple-framebuffer";
reg = <0 0xa0000000 0 (2340 * 1080 * 4)>;
width = <1080>;
@@ -1143,8 +1143,8 @@ &usb_1_hsphy {
};
&usb_1_qmpphy {
- vdda-phy-supply = <&vreg_l22a>;
- vdda-pll-supply = <&vreg_l16a>;
+ vdda-phy-supply = <&vreg_l16a>;
+ vdda-pll-supply = <&vreg_l22a>;
status = "okay";
};
diff --git a/arch/arm64/boot/dts/qcom/sm8150.dtsi b/arch/arm64/boot/dts/qcom/sm8150.dtsi
index 0aeceffd872c..e06f7782b551 100644
--- a/arch/arm64/boot/dts/qcom/sm8150.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8150.dtsi
@@ -1885,8 +1885,8 @@ pcie0: pcie@1c00000 {
"tbu",
"ref";
- iommu-map = <0x0 &apps_smmu 0x1d80 0x1>,
- <0x100 &apps_smmu 0x1d81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1d80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1d81 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -2005,8 +2005,8 @@ pcie1: pcie@1c08000 {
assigned-clocks = <&gcc GCC_PCIE_1_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1e00 0x1>,
- <0x100 &apps_smmu 0x1e01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1e00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1e01 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
diff --git a/arch/arm64/boot/dts/qcom/sm8250-xiaomi-elish-common.dtsi b/arch/arm64/boot/dts/qcom/sm8250-xiaomi-elish-common.dtsi
index 3596dd328c31..00e90080d212 100644
--- a/arch/arm64/boot/dts/qcom/sm8250-xiaomi-elish-common.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8250-xiaomi-elish-common.dtsi
@@ -28,7 +28,7 @@ / {
/* required for bootloader to select correct board */
qcom,msm-id = <QCOM_ID_SM8250 0x20001>; /* SM8250 v2.1 */
- qcom,board-id = <0x10008 0>;
+ qcom,board-id = <0x2f 0>;
chosen {
#address-cells = <2>;
diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi
index 4cb91f31340d..f1851cbcd48b 100644
--- a/arch/arm64/boot/dts/qcom/sm8250.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi
@@ -2187,8 +2187,8 @@ pcie0: pcie@1c00000 {
"tbu",
"ddrss_sf_tbu";
- iommu-map = <0x0 &apps_smmu 0x1c00 0x1>,
- <0x100 &apps_smmu 0x1c01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c01 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -2312,8 +2312,8 @@ pcie1: pcie@1c08000 {
assigned-clocks = <&gcc GCC_PCIE_1_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1c80 0x1>,
- <0x100 &apps_smmu 0x1c81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c81 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
@@ -2437,8 +2437,8 @@ pcie2: pcie@1c10000 {
assigned-clocks = <&gcc GCC_PCIE_2_AUX_CLK>;
assigned-clock-rates = <19200000>;
- iommu-map = <0x0 &apps_smmu 0x1d00 0x1>,
- <0x100 &apps_smmu 0x1d01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1d00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1d01 0x0 0x1>;
resets = <&gcc GCC_PCIE_2_BCR>;
reset-names = "pci";
@@ -4310,8 +4310,12 @@ venus: video-codec@aa00000 {
interrupts = <GIC_SPI 174 IRQ_TYPE_LEVEL_HIGH>;
power-domains = <&videocc MVS0C_GDSC>,
<&videocc MVS0_GDSC>,
- <&rpmhpd RPMHPD_MX>;
- power-domain-names = "venus", "vcodec0", "mx";
+ <&rpmhpd RPMHPD_MX>,
+ <&rpmhpd RPMHPD_MMCX>;
+ power-domain-names = "venus",
+ "vcodec0",
+ "mx",
+ "mmcx";
operating-points-v2 = <&venus_opp_table>;
clocks = <&gcc GCC_VIDEO_AXI0_CLK>,
@@ -4343,24 +4347,28 @@ video-encoder {
venus_opp_table: opp-table {
compatible = "operating-points-v2";
- opp-720000000 {
- opp-hz = /bits/ 64 <720000000>;
- required-opps = <&rpmhpd_opp_low_svs>;
+ opp-240000000 {
+ opp-hz = /bits/ 64 <240000000>;
+ required-opps = <&rpmhpd_opp_svs>,
+ <&rpmhpd_opp_low_svs>;
};
- opp-1014000000 {
- opp-hz = /bits/ 64 <1014000000>;
- required-opps = <&rpmhpd_opp_svs>;
+ opp-338000000 {
+ opp-hz = /bits/ 64 <338000000>;
+ required-opps = <&rpmhpd_opp_svs>,
+ <&rpmhpd_opp_svs>;
};
- opp-1098000000 {
- opp-hz = /bits/ 64 <1098000000>;
- required-opps = <&rpmhpd_opp_svs_l1>;
+ opp-366000000 {
+ opp-hz = /bits/ 64 <366000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>,
+ <&rpmhpd_opp_svs_l1>;
};
- opp-1332000000 {
- opp-hz = /bits/ 64 <1332000000>;
- required-opps = <&rpmhpd_opp_nom>;
+ opp-444000000 {
+ opp-hz = /bits/ 64 <444000000>;
+ required-opps = <&rpmhpd_opp_svs_l1>,
+ <&rpmhpd_opp_nom>;
};
};
};
diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
index 1870ec4176da..6f175791de90 100644
--- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
@@ -1571,8 +1571,8 @@ pcie0: pcie@1c00000 {
"aggre1",
"aggre0";
- iommu-map = <0x0 &apps_smmu 0x1c00 0x1>,
- <0x100 &apps_smmu 0x1c01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c01 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -1678,8 +1678,8 @@ pcie1: pcie@1c08000 {
"ddrss_sf_tbu",
"aggre1";
- iommu-map = <0x0 &apps_smmu 0x1c80 0x1>,
- <0x100 &apps_smmu 0x1c81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c81 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi
index 59172db1e3d9..521b12a56e55 100644
--- a/arch/arm64/boot/dts/qcom/sm8450.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi
@@ -1834,8 +1834,8 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
"aggre0",
"aggre1";
- iommu-map = <0x0 &apps_smmu 0x1c00 0x1>,
- <0x100 &apps_smmu 0x1c01 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c00 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c01 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -1994,8 +1994,8 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
"ddrss_sf_tbu",
"aggre1";
- iommu-map = <0x0 &apps_smmu 0x1c80 0x1>,
- <0x100 &apps_smmu 0x1c81 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1c80 0x0 0x1>,
+ <0x100 &apps_smmu 0x1c81 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>;
reset-names = "pci";
diff --git a/arch/arm64/boot/dts/qcom/sm8550.dtsi b/arch/arm64/boot/dts/qcom/sm8550.dtsi
index 38b15db0676c..7f06bea09d0a 100644
--- a/arch/arm64/boot/dts/qcom/sm8550.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8550.dtsi
@@ -1771,8 +1771,8 @@ pcie0: pcie@1c00000 {
msi-map = <0x0 &gic_its 0x1400 0x1>,
<0x100 &gic_its 0x1401 0x1>;
- iommu-map = <0x0 &apps_smmu 0x1400 0x1>,
- <0x100 &apps_smmu 0x1401 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1400 0x0 0x1>,
+ <0x100 &apps_smmu 0x1401 0x0 0x1>;
resets = <&gcc GCC_PCIE_0_BCR>;
reset-names = "pci";
@@ -1892,8 +1892,8 @@ pcie1: pcie@1c08000 {
msi-map = <0x0 &gic_its 0x1480 0x1>,
<0x100 &gic_its 0x1481 0x1>;
- iommu-map = <0x0 &apps_smmu 0x1480 0x1>,
- <0x100 &apps_smmu 0x1481 0x1>;
+ iommu-map = <0x0 &apps_smmu 0x1480 0x0 0x1>,
+ <0x100 &apps_smmu 0x1481 0x0 0x1>;
resets = <&gcc GCC_PCIE_1_BCR>,
<&gcc GCC_PCIE_1_LINK_DOWN_BCR>;
diff --git a/arch/arm64/boot/dts/qcom/sm8650.dtsi b/arch/arm64/boot/dts/qcom/sm8650.dtsi
index b4d9900e1776..d28dfbf57e63 100644
--- a/arch/arm64/boot/dts/qcom/sm8650.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8650.dtsi
@@ -2279,8 +2279,10 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
power-domains = <&gcc PCIE_0_GDSC>;
- iommu-map = <0 &apps_smmu 0x1400 0x1>,
- <0x100 &apps_smmu 0x1401 0x1>;
+ operating-points-v2 = <&pcie0_opp_table>;
+
+ iommu-map = <0 &apps_smmu 0x1400 0x0 0x1>,
+ <0x100 &apps_smmu 0x1401 0x0 0x1>;
interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>,
<0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>,
@@ -2309,6 +2311,45 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
status = "disabled";
+ pcie0_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ /* GEN 1 x1 */
+ opp-2500000 {
+ opp-hz = /bits/ 64 <2500000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <250000 1>;
+ };
+
+ /* GEN 1 x2 and GEN 2 x1 */
+ opp-5000000 {
+ opp-hz = /bits/ 64 <5000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <500000 1>;
+ };
+
+ /* GEN 2 x2 */
+ opp-10000000 {
+ opp-hz = /bits/ 64 <10000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <1000000 1>;
+ };
+
+ /* GEN 3 x1 */
+ opp-8000000 {
+ opp-hz = /bits/ 64 <8000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ opp-peak-kBps = <984500 1>;
+ };
+
+ /* GEN 3 x2 */
+ opp-16000000 {
+ opp-hz = /bits/ 64 <16000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ opp-peak-kBps = <1969000 1>;
+ };
+ };
+
pcieport0: pcie@0 {
device_type = "pci";
reg = <0x0 0x0 0x0 0x0 0x0>;
@@ -2416,8 +2457,10 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
power-domains = <&gcc PCIE_1_GDSC>;
- iommu-map = <0 &apps_smmu 0x1480 0x1>,
- <0x100 &apps_smmu 0x1481 0x1>;
+ operating-points-v2 = <&pcie1_opp_table>;
+
+ iommu-map = <0 &apps_smmu 0x1480 0x0 0x1>,
+ <0x100 &apps_smmu 0x1481 0x0 0x1>;
interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>,
<0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>,
@@ -2446,6 +2489,52 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
status = "disabled";
+ pcie1_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ /* GEN 1 x1 */
+ opp-2500000 {
+ opp-hz = /bits/ 64 <2500000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <250000 1>;
+ };
+
+ /* GEN 1 x2 and GEN 2 x1 */
+ opp-5000000 {
+ opp-hz = /bits/ 64 <5000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <500000 1>;
+ };
+
+ /* GEN 2 x2 */
+ opp-10000000 {
+ opp-hz = /bits/ 64 <10000000>;
+ required-opps = <&rpmhpd_opp_low_svs>;
+ opp-peak-kBps = <1000000 1>;
+ };
+
+ /* GEN 3 x1 */
+ opp-8000000 {
+ opp-hz = /bits/ 64 <8000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ opp-peak-kBps = <984500 1>;
+ };
+
+ /* GEN 3 x2 and GEN 4 x1 */
+ opp-16000000 {
+ opp-hz = /bits/ 64 <16000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ opp-peak-kBps = <1969000 1>;
+ };
+
+ /* GEN 4 x2 */
+ opp-32000000 {
+ opp-hz = /bits/ 64 <32000000>;
+ required-opps = <&rpmhpd_opp_nom>;
+ opp-peak-kBps = <3938000 1>;
+ };
+ };
+
pcie@0 {
device_type = "pci";
reg = <0x0 0x0 0x0 0x0 0x0>;
diff --git a/arch/arm64/boot/dts/qcom/x1e80100.dtsi b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
index 7e9de1b7c7a3..d456190ed76d 100644
--- a/arch/arm64/boot/dts/qcom/x1e80100.dtsi
+++ b/arch/arm64/boot/dts/qcom/x1e80100.dtsi
@@ -2596,7 +2596,7 @@ usb_1_ss0_hsphy: phy@fd3000 {
reg = <0 0x00fd3000 0 0x154>;
#phy-cells = <0>;
- clocks = <&tcsr TCSR_USB2_1_CLKREF_EN>;
+ clocks = <&rpmhcc RPMH_CXO_CLK>;
clock-names = "ref";
resets = <&gcc GCC_QUSB2PHY_PRIM_BCR>;
@@ -2736,7 +2736,7 @@ usb_1_ss2_hsphy: phy@fde000 {
reg = <0 0x00fde000 0 0x154>;
#phy-cells = <0>;
- clocks = <&tcsr TCSR_USB2_1_CLKREF_EN>;
+ clocks = <&tcsr TCSR_USB2_2_CLKREF_EN>;
clock-names = "ref";
resets = <&gcc GCC_QUSB2PHY_TERT_BCR>;
diff --git a/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
index cacbad35cfc8..cf9fdb7d0cfb 100644
--- a/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3399-gru-chromebook.dtsi
@@ -510,8 +510,7 @@ &pci_rootport {
mvl_wifi: wifi@0,0 {
compatible = "pci1b4b,2b42";
reg = <0x0000 0x0 0x0 0x0 0x0>;
- interrupt-parent = <&gpio0>;
- interrupts = <8 IRQ_TYPE_LEVEL_LOW>;
+ interrupts-extended = <&gpio0 8 IRQ_TYPE_LEVEL_LOW>;
pinctrl-names = "default";
pinctrl-0 = <&wlan_host_wake_l>;
wakeup-source;
diff --git a/arch/arm64/boot/dts/ti/k3-am64-main.dtsi b/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
index 7eae18399caa..7c6ba86cf8ee 100644
--- a/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-am64-main.dtsi
@@ -1349,7 +1349,7 @@ tx_pru0_1: txpru@c000 {
icssg0_mdio: mdio@32400 {
compatible = "ti,davinci_mdio";
reg = <0x32400 0x100>;
- clocks = <&k3_clks 62 3>;
+ clocks = <&k3_clks 81 0>;
clock-names = "fck";
#address-cells = <1>;
#size-cells = <0>;
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 2de5a70a47ce..a66a474949c8 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -1266,7 +1266,7 @@ static inline int __ptep_clear_flush_young(struct vm_area_struct *vma,
return young;
}
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG)
#define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG
static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long address,
@@ -1274,7 +1274,7 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,
{
return __ptep_test_and_clear_young(vma, address, (pte_t *)pmdp);
}
-#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG */
static inline pte_t __ptep_get_and_clear(struct mm_struct *mm,
unsigned long address, pte_t *ptep)
@@ -1509,6 +1509,10 @@ static inline void update_mmu_cache_range(struct vm_fault *vmf,
*/
#define arch_has_hw_pte_young cpu_has_hw_af
+#ifdef CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
+#define arch_has_hw_nonleaf_pmd_young system_supports_haft
+#endif
+
/*
* Experimentally, it's cheap to set the access flag in hardware and we
* benefit from prefaulting mappings as 'old' to start with.
diff --git a/arch/arm64/include/asm/ptdump.h b/arch/arm64/include/asm/ptdump.h
index 6cf4aae05219..8bf87ac9bb4b 100644
--- a/arch/arm64/include/asm/ptdump.h
+++ b/arch/arm64/include/asm/ptdump.h
@@ -7,7 +7,7 @@
#include <linux/ptdump.h>
-#ifdef CONFIG_PTDUMP_CORE
+#ifdef CONFIG_PTDUMP
#include <linux/mm_types.h>
#include <linux/seq_file.h>
@@ -50,6 +50,8 @@ struct ptdump_pg_state {
const struct addr_marker *marker;
const struct mm_struct *mm;
unsigned long start_address;
+ /* exclusive end, ULONG_MAX represents an end at 1 << 64 */
+ unsigned long end_address;
int level;
u64 current_prot;
bool check_wx;
@@ -59,7 +61,13 @@ struct ptdump_pg_state {
void ptdump_walk(struct seq_file *s, struct ptdump_info *info);
void note_page(struct ptdump_state *pt_st, unsigned long addr, int level,
- u64 val);
+ pteval_t val);
+void note_page_pte(struct ptdump_state *st, unsigned long addr, pte_t pte);
+void note_page_pmd(struct ptdump_state *st, unsigned long addr, pmd_t pmd);
+void note_page_pud(struct ptdump_state *st, unsigned long addr, pud_t pud);
+void note_page_p4d(struct ptdump_state *st, unsigned long addr, p4d_t p4d);
+void note_page_pgd(struct ptdump_state *st, unsigned long addr, pgd_t pgd);
+void note_page_flush(struct ptdump_state *st);
#ifdef CONFIG_PTDUMP_DEBUGFS
#define EFI_RUNTIME_MAP_END DEFAULT_MAP_WINDOW_64
void __init ptdump_debugfs_register(struct ptdump_info *info, const char *name);
@@ -69,7 +77,13 @@ static inline void ptdump_debugfs_register(struct ptdump_info *info,
#endif /* CONFIG_PTDUMP_DEBUGFS */
#else
static inline void note_page(struct ptdump_state *pt_st, unsigned long addr,
- int level, u64 val) { }
-#endif /* CONFIG_PTDUMP_CORE */
+ int level, pteval_t val) { }
+static inline void note_page_pte(struct ptdump_state *st, unsigned long addr, pte_t pte) { }
+static inline void note_page_pmd(struct ptdump_state *st, unsigned long addr, pmd_t pmd) { }
+static inline void note_page_pud(struct ptdump_state *st, unsigned long addr, pud_t pud) { }
+static inline void note_page_p4d(struct ptdump_state *st, unsigned long addr, p4d_t p4d) { }
+static inline void note_page_pgd(struct ptdump_state *st, unsigned long addr, pgd_t pgd) { }
+static inline void note_page_flush(struct ptdump_state *st) { }
+#endif /* CONFIG_PTDUMP */
#endif /* __ASM_PTDUMP_H */
diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 945983735815..44d31b019efd 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -342,8 +342,10 @@ int swsusp_arch_suspend(void)
crash_prepare_suspend();
ret = swsusp_mte_save_tags();
- if (ret)
+ if (ret) {
+ local_daif_restore(flags);
return ret;
+ }
sleep_cpu = smp_processor_id();
ret = swsusp_save();
@@ -459,9 +461,21 @@ int __nocfi swsusp_arch_resume(void)
if (el2_reset_needed())
__hyp_set_vectors(el2_vectors);
+ /*
+ * It is necessary to mask all DAIF exceptions here as:
+ *
+ * - The copy of swsusp_arch_suspend_exit() in the hibernation
+ * text cannot handle taking any exceptions.
+ *
+ * - The suspended kernel masked all DAIF exceptions in
+ * swsusp_arch_resume(), and expects to be re-entered in the
+ * same state : with all DAIF exceptions masked.
+ */
+ local_daif_save();
hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1,
resume_hdr.reenter_kernel, restore_pblist,
resume_hdr.__hyp_stub_vectors, virt_to_phys(zero_page));
+ unreachable();
return 0;
}
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index ead632ad01b4..096e45acadb2 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -71,8 +71,8 @@ config PTDUMP_STAGE2_DEBUGFS
depends on KVM
depends on DEBUG_KERNEL
depends on DEBUG_FS
- depends on GENERIC_PTDUMP
- select PTDUMP_CORE
+ depends on ARCH_HAS_PTDUMP
+ select PTDUMP
default n
help
Say Y here if you want to show the stage-2 kernel pagetables
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index 13e6a2829116..68d91c2021f1 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -1,4 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
+
+# KCSAN uses udelay for introducing watchpoint delay; avoid recursion.
+KCSAN_SANITIZE_delay.o := n
+
lib-y := clear_user.o delay.o copy_from_user.o \
copy_to_user.o copy_page.o \
clear_page.o csum.o insn.o memchr.o memcpy.o \
diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile
index 2fc8c6dd0407..ea6cd8b4e434 100644
--- a/arch/arm64/mm/Makefile
+++ b/arch/arm64/mm/Makefile
@@ -5,7 +5,7 @@ obj-y := dma-mapping.o extable.o fault.o init.o \
context.o proc.o pageattr.o fixmap.o
obj-$(CONFIG_ARM64_CONTPTE) += contpte.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
-obj-$(CONFIG_PTDUMP_CORE) += ptdump.o
+obj-$(CONFIG_PTDUMP) += ptdump.o
obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o
obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o
obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 9ee5a2d2b321..9a33856683af 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -15,6 +15,7 @@
#include <linux/mm.h>
#include <linux/hardirq.h>
#include <linux/init.h>
+#include <linux/irqflags.h>
#include <linux/kasan.h>
#include <linux/kprobes.h>
#include <linux/uaccess.h>
@@ -151,6 +152,9 @@ static void show_pte(unsigned long addr)
pr_alert("%s pgtable: %luk pages, %llu-bit VAs, pgdp=%016lx\n",
mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K,
vabits_actual, mm_to_pgd_phys(mm));
+
+ guard(irqsave)();
+
pgdp = pgd_offset(mm, addr);
pgd = READ_ONCE(*pgdp);
pr_alert("[%016lx] pgd=%016llx", addr, pgd_val(pgd));
@@ -164,25 +168,25 @@ static void show_pte(unsigned long addr)
if (pgd_none(pgd) || pgd_bad(pgd))
break;
- p4dp = p4d_offset(pgdp, addr);
+ p4dp = p4d_offset_lockless(pgdp, pgd, addr);
p4d = READ_ONCE(*p4dp);
pr_cont(", p4d=%016llx", p4d_val(p4d));
if (p4d_none(p4d) || p4d_bad(p4d))
break;
- pudp = pud_offset(p4dp, addr);
+ pudp = pud_offset_lockless(p4dp, p4d, addr);
pud = READ_ONCE(*pudp);
pr_cont(", pud=%016llx", pud_val(pud));
if (pud_none(pud) || pud_bad(pud))
break;
- pmdp = pmd_offset(pudp, addr);
+ pmdp = pmd_offset_lockless(pudp, pud, addr);
pmd = READ_ONCE(*pmdp);
pr_cont(", pmd=%016llx", pmd_val(pmd));
if (pmd_none(pmd) || pmd_bad(pmd))
break;
- ptep = pte_offset_map(pmdp, addr);
+ ptep = pte_offset_map(&pmd, addr);
if (!ptep)
break;
diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c
index 264c5f9b97d8..93a872582ede 100644
--- a/arch/arm64/mm/ptdump.c
+++ b/arch/arm64/mm/ptdump.c
@@ -189,7 +189,7 @@ static void note_prot_wx(struct ptdump_pg_state *st, unsigned long addr)
}
void note_page(struct ptdump_state *pt_st, unsigned long addr, int level,
- u64 val)
+ pteval_t val)
{
struct ptdump_pg_state *st = container_of(pt_st, struct ptdump_pg_state, ptdump);
struct ptdump_pg_level *pg_level = st->pg_level;
@@ -251,6 +251,48 @@ void note_page(struct ptdump_state *pt_st, unsigned long addr, int level,
}
+void note_page_pte(struct ptdump_state *pt_st, unsigned long addr, pte_t pte)
+{
+ note_page(pt_st, addr, 4, pte_val(pte));
+}
+
+void note_page_pmd(struct ptdump_state *pt_st, unsigned long addr, pmd_t pmd)
+{
+ note_page(pt_st, addr, 3, pmd_val(pmd));
+}
+
+void note_page_pud(struct ptdump_state *pt_st, unsigned long addr, pud_t pud)
+{
+ note_page(pt_st, addr, 2, pud_val(pud));
+}
+
+void note_page_p4d(struct ptdump_state *pt_st, unsigned long addr, p4d_t p4d)
+{
+ note_page(pt_st, addr, 1, p4d_val(p4d));
+}
+
+void note_page_pgd(struct ptdump_state *pt_st, unsigned long addr, pgd_t pgd)
+{
+ note_page(pt_st, addr, 0, pgd_val(pgd));
+}
+
+void note_page_flush(struct ptdump_state *pt_st)
+{
+ struct ptdump_pg_state *st = container_of(pt_st, struct ptdump_pg_state, ptdump);
+ unsigned long end = st->end_address;
+ pte_t pte_zero = {0};
+
+ /*
+ * Address spaces that end at 1 << 64 have end_address == ULONG_MAX,
+ * but note_page() expects the exclusive end. In this case adjust end
+ * to the wraparound value 0.
+ */
+ if (end == ULONG_MAX)
+ end = 0;
+
+ note_page(pt_st, end, -1, pte_val(pte_zero));
+}
+
void ptdump_walk(struct seq_file *s, struct ptdump_info *info)
{
unsigned long end = ~0UL;
@@ -264,9 +306,15 @@ void ptdump_walk(struct seq_file *s, struct ptdump_info *info)
.marker = info->markers,
.mm = info->mm,
.pg_level = &kernel_pg_levels[0],
+ .end_address = end,
.level = -1,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]){
{info->base_addr, end},
{0, 0}
@@ -300,10 +348,16 @@ bool ptdump_check_wx(void)
{ -1, NULL},
},
.pg_level = &kernel_pg_levels[0],
+ .end_address = ~0UL,
.level = -1,
.check_wx = true,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]) {
{_PAGE_OFFSET(vabits_actual), ~0UL},
{0, 0}
diff --git a/arch/csky/kernel/entry.S b/arch/csky/kernel/entry.S
index c68cdcc76d60..3261f46f2244 100644
--- a/arch/csky/kernel/entry.S
+++ b/arch/csky/kernel/entry.S
@@ -93,11 +93,11 @@ csky_syscall_trace:
ldw a2, (sp, LSAVE_A2)
ldw a3, (sp, LSAVE_A3)
#if defined(__CSKYABIV2__)
- subi sp, 8
ldw r9, (sp, LSAVE_A4)
+ ldw r10, (sp, LSAVE_A5)
+ subi sp, 8
stw r9, (sp, 0x0)
- ldw r9, (sp, LSAVE_A5)
- stw r9, (sp, 0x4)
+ stw r10, (sp, 0x4)
jsr syscallid /* Do system call */
addi sp, 8
#else
diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig
index d402fcdf0861..3048c7c527b6 100644
--- a/arch/loongarch/Kconfig
+++ b/arch/loongarch/Kconfig
@@ -161,7 +161,7 @@ config LOONGARCH
select HAVE_RELIABLE_STACKTRACE if UNWINDER_ORC
select HAVE_RETHOOK
select HAVE_RSEQ
- select HAVE_RUST
+ select HAVE_RUST if !KASAN
select HAVE_SAMPLE_FTRACE_DIRECT
select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
select HAVE_SETUP_PER_CPU_AREA if NUMA
diff --git a/arch/loongarch/configs/loongson3_defconfig b/arch/loongarch/configs/loongson3_defconfig
index 75b366407a60..60424d59a4ec 100644
--- a/arch/loongarch/configs/loongson3_defconfig
+++ b/arch/loongarch/configs/loongson3_defconfig
@@ -99,7 +99,10 @@ CONFIG_ZBUD=y
CONFIG_ZSMALLOC=m
# CONFIG_COMPAT_BRK is not set
CONFIG_MEMORY_HOTPLUG=y
-CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y
+# CONFIG_MHP_DEFAULT_ONLINE_TYPE_OFFLINE is not set
+CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_AUTO=y
+# CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_KERNEL is not set
+# CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_MOVABLE is not set
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_KSM=y
CONFIG_TRANSPARENT_HUGEPAGE=y
diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index 4f813fd89ac5..7d123a7d9794 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -110,6 +110,7 @@ struct kvm_arch {
unsigned int pte_shifts[MAX_PGTABLE_LEVELS];
unsigned int root_level;
spinlock_t phyid_map_lock;
+ spinlock_t pv_setting_lock;
struct kvm_phyid_map *phyid_map;
/* Enabled PV features */
unsigned long pv_features;
diff --git a/arch/loongarch/kernel/acpi.c b/arch/loongarch/kernel/acpi.c
index df9c0f8058d8..309fb7b61489 100644
--- a/arch/loongarch/kernel/acpi.c
+++ b/arch/loongarch/kernel/acpi.c
@@ -12,6 +12,7 @@
#include <linux/efi-bgrt.h>
#include <linux/irq.h>
#include <linux/irqdomain.h>
+#include <linux/kvm_host.h>
#include <linux/memblock.h>
#include <linux/of_fdt.h>
#include <linux/serial_core.h>
@@ -201,7 +202,7 @@ static void __init acpi_process_madt(void)
int pptt_enabled;
static int acpi_nr_packages;
-static int acpi_package_ids[MAX_PACKAGES];
+static int acpi_package_ids[MAX(MAX_PACKAGES, KVM_MAX_VCPUS)];
int __init parse_acpi_topology(void)
{
diff --git a/arch/loongarch/kernel/kprobes.c b/arch/loongarch/kernel/kprobes.c
index bd783ebed4cc..ad6b28bbd4d3 100644
--- a/arch/loongarch/kernel/kprobes.c
+++ b/arch/loongarch/kernel/kprobes.c
@@ -273,6 +273,9 @@ bool kprobe_singlestep_handler(struct pt_regs *regs)
struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
unsigned long addr = instruction_pointer(regs);
+ if (!cur)
+ return false;
+
if (cur && (kcb->kprobe_status & (KPROBE_HIT_SS | KPROBE_REENTER)) &&
((unsigned long)&cur->ainsn.insn[1] == addr)) {
restore_local_irqflag(kcb, regs);
diff --git a/arch/loongarch/kernel/rethook_trampoline.S b/arch/loongarch/kernel/rethook_trampoline.S
index d4ceb2fa2a5c..0298ebced64a 100644
--- a/arch/loongarch/kernel/rethook_trampoline.S
+++ b/arch/loongarch/kernel/rethook_trampoline.S
@@ -24,7 +24,6 @@
cfi_st t6, PT_R18
cfi_st t7, PT_R19
cfi_st t8, PT_R20
- cfi_st u0, PT_R21
cfi_st fp, PT_R22
cfi_st s0, PT_R23
cfi_st s1, PT_R24
@@ -59,7 +58,6 @@
cfi_ld t6, PT_R18
cfi_ld t7, PT_R19
cfi_ld t8, PT_R20
- cfi_ld u0, PT_R21
cfi_ld fp, PT_R22
cfi_ld s0, PT_R23
cfi_ld s1, PT_R24
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 550ce2b4df70..ee6b97102ba3 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -377,7 +377,11 @@ static int kvm_loongarch_init(void)
if (r)
return r;
- return kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+ r = kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODULE);
+ if (r)
+ kvm_loongarch_env_exit();
+
+ return r;
}
static void kvm_loongarch_exit(void)
diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c
index f854e74676af..d316cbc6e5e9 100644
--- a/arch/loongarch/kvm/mmu.c
+++ b/arch/loongarch/kvm/mmu.c
@@ -383,6 +383,16 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, const struct kvm_memory_slot
hva_t hva_start;
size_t size, gpa_offset, hva_offset;
+ /*
+ * The generic code allocates a fresh, zeroed memslot for every change,
+ * so the arch flags computed below must be carried over when only the
+ * userspace flags change, e.g. when dirty logging is toggled.
+ */
+ if (change == KVM_MR_FLAGS_ONLY) {
+ new->arch = old->arch;
+ return 0;
+ }
+
if ((change != KVM_MR_MOVE) && (change != KVM_MR_CREATE))
return 0;
/*
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index c9f16c27e87d..59eee474e818 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -1088,10 +1088,14 @@ static int kvm_loongarch_cpucfg_set_attr(struct kvm_vcpu *vcpu,
return -EINVAL;
/* All vCPUs need set the same PV features */
+ spin_lock(&kvm->arch.pv_setting_lock);
if ((kvm->arch.pv_features & LOONGARCH_PV_FEAT_UPDATED)
- && ((kvm->arch.pv_features & valid) != val))
+ && ((kvm->arch.pv_features & valid) != val)) {
+ spin_unlock(&kvm->arch.pv_setting_lock);
return -EINVAL;
+ }
kvm->arch.pv_features = val | LOONGARCH_PV_FEAT_UPDATED;
+ spin_unlock(&kvm->arch.pv_setting_lock);
return 0;
default:
return -ENXIO;
diff --git a/arch/loongarch/kvm/vm.c b/arch/loongarch/kvm/vm.c
index 9cb4433c7bcb..2bde2e1840a0 100644
--- a/arch/loongarch/kvm/vm.c
+++ b/arch/loongarch/kvm/vm.c
@@ -38,6 +38,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
return -ENOMEM;
}
spin_lock_init(&kvm->arch.phyid_map_lock);
+ spin_lock_init(&kvm->arch.pv_setting_lock);
kvm_init_vmcs(kvm);
diff --git a/arch/m68k/emu/nfcon.c b/arch/m68k/emu/nfcon.c
index d41260672e24..f4657d9f4f15 100644
--- a/arch/m68k/emu/nfcon.c
+++ b/arch/m68k/emu/nfcon.c
@@ -49,7 +49,7 @@ static void nfcon_write(struct console *con, const char *str,
static struct tty_driver *nfcon_device(struct console *con, int *index)
{
*index = 0;
- return console_is_registered(con) ? nfcon_tty_driver : NULL;
+ return nfcon_tty_driver;
}
static struct console nf_console = {
diff --git a/arch/m68k/kernel/traps.c b/arch/m68k/kernel/traps.c
index 53d0cf343d90..c871916ea80f 100644
--- a/arch/m68k/kernel/traps.c
+++ b/arch/m68k/kernel/traps.c
@@ -936,8 +936,8 @@ void show_stack(struct task_struct *task, unsigned long *stack,
int i;
if (!stack) {
- if (task)
- stack = (unsigned long *)task->thread.esp0;
+ if (task && task != current)
+ stack = (unsigned long *)task->thread.ksp;
else
stack = (unsigned long *)&stack;
}
diff --git a/arch/mips/kernel/ptrace.c b/arch/mips/kernel/ptrace.c
index 61503a36067e..2016dd5e8fae 100644
--- a/arch/mips/kernel/ptrace.c
+++ b/arch/mips/kernel/ptrace.c
@@ -1319,8 +1319,12 @@ long arch_ptrace(struct task_struct *child, long request,
*/
asmlinkage long syscall_trace_enter(struct pt_regs *regs)
{
+ long syscall;
+
user_exit();
+ syscall = current_thread_info()->syscall;
+
if (test_thread_flag(TIF_SYSCALL_TRACE)) {
if (ptrace_report_syscall_entry(regs))
return -1;
@@ -1356,7 +1360,7 @@ asmlinkage long syscall_trace_enter(struct pt_regs *regs)
* Negative syscall numbers are mistaken for rejected syscalls, but
* won't have had the return value set appropriately, so we do so now.
*/
- if (current_thread_info()->syscall < 0)
+ if (syscall < 0)
syscall_set_return_value(current, regs, -ENOSYS, 0);
return current_thread_info()->syscall;
}
diff --git a/arch/parisc/kernel/head.S b/arch/parisc/kernel/head.S
index 9188c8d87437..e84d7d6e1eb9 100644
--- a/arch/parisc/kernel/head.S
+++ b/arch/parisc/kernel/head.S
@@ -105,6 +105,7 @@ $iodc_panic:
or %r10,%r10,%r10 /* qemu idle sleep */
msg1: .ascii "Can't boot kernel which was built for PA8x00 CPUs on this machine.\r\n"
msg1_end:
+ .align 4
$cpu_ok:
#endif
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index c7b420d6787c..43515128c9d5 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -145,6 +145,8 @@ config PPC
select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
select ARCH_HAS_PHYS_TO_DMA
select ARCH_HAS_PMEM_API
+ select ARCH_HAS_PREEMPT_LAZY
+ select ARCH_HAS_PTDUMP
select ARCH_HAS_PTE_DEVMAP if PPC_BOOK3S_64
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE && PPC_BOOK3S_64
@@ -203,7 +205,6 @@ config PPC
select GENERIC_IRQ_SHOW
select GENERIC_IRQ_SHOW_LEVEL
select GENERIC_PCI_IOMAP if PCI
- select GENERIC_PTDUMP
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
select GENERIC_VDSO_TIME_NS
@@ -1043,6 +1044,7 @@ config PPC_SECURE_BOOT
depends on IMA_ARCH_POLICY
imply IMA_SECURE_AND_OR_TRUSTED_BOOT
select PSERIES_PLPKS if PPC_PSERIES
+ select HAVE_ARCH_GET_SECUREBOOT
help
Systems with firmware secure boot enabled need to define security
policies to extend secure boot to the OS. This config allows a user
diff --git a/arch/powerpc/configs/85xx-hw.config b/arch/powerpc/configs/85xx-hw.config
index 8aff83217397..b655c36e56da 100644
--- a/arch/powerpc/configs/85xx-hw.config
+++ b/arch/powerpc/configs/85xx-hw.config
@@ -90,6 +90,7 @@ CONFIG_PPC_EPAPR_HV_BYTECHAN=y
CONFIG_QE_GPIO=y
CONFIG_QUICC_ENGINE=y
CONFIG_RAPIDIO=y
+CONFIG_RAS=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_DRV_CMOS=y
CONFIG_RTC_DRV_DS1307=y
diff --git a/arch/powerpc/include/asm/papr-watchdog.h b/arch/powerpc/include/asm/papr-watchdog.h
new file mode 100644
index 000000000000..bf876fc2caae
--- /dev/null
+++ b/arch/powerpc/include/asm/papr-watchdog.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef _ASM_POWERPC_PAPR_WATCHDOG_H
+#define _ASM_POWERPC_PAPR_WATCHDOG_H
+
+/*
+ * H_WATCHDOG Input
+ *
+ * R4: "flags":
+ *
+ * Bits 48-55: "operation"
+ */
+#define PSERIES_WDTF_OP_START 0x100UL /* start timer */
+#define PSERIES_WDTF_OP_STOP 0x200UL /* stop timer */
+#define PSERIES_WDTF_OP_QUERY 0x300UL /* query timer capabilities */
+
+/*
+ * Bits 56-63: "timeoutAction" (for "Start Watchdog" only)
+ */
+#define PSERIES_WDTF_ACTION_HARD_POWEROFF 0x1UL /* poweroff */
+#define PSERIES_WDTF_ACTION_HARD_RESTART 0x2UL /* restart */
+#define PSERIES_WDTF_ACTION_DUMP_RESTART 0x3UL /* dump + restart */
+
+/*
+ * R5: "watchdogNumber":
+ * PAPR says use -1 (all ones) to stop all watchdogs.
+ */
+#define PSERIES_WDT_NUM_ALL ((unsigned long)-1)
+
+/*
+ * H_WATCHDOG Output
+ *
+ * R3: Return code
+ *
+ * H_SUCCESS The operation completed.
+ *
+ * H_BUSY The hypervisor is too busy; retry the operation.
+ *
+ * H_PARAMETER The given "flags" are somehow invalid. Either the
+ * "operation" or "timeoutAction" is invalid, or a
+ * reserved bit is set.
+ *
+ * H_P2 The given "watchdogNumber" is zero or exceeds the
+ * supported maximum value.
+ *
+ * H_P3 The given "timeoutInMs" is below the supported
+ * minimum value.
+ *
+ * H_NOOP The given "watchdogNumber" is already stopped.
+ *
+ * H_HARDWARE The operation failed for ineffable reasons.
+ *
+ * H_FUNCTION The H_WATCHDOG hypercall is not supported by this
+ * hypervisor.
+ *
+ * R4:
+ *
+ * - For the "Query Watchdog Capabilities" operation, a 64-bit
+ * structure:
+ */
+#define PSERIES_WDTQ_MIN_TIMEOUT(cap) (((cap) >> 48) & 0xffff)
+#define PSERIES_WDTQ_MAX_NUMBER(cap) (((cap) >> 32) & 0xffff)
+
+#endif /* _ASM_POWERPC_PAPR_WATCHDOG_H */
diff --git a/arch/powerpc/include/asm/thread_info.h b/arch/powerpc/include/asm/thread_info.h
index 6ebca2996f18..2785c7462ebf 100644
--- a/arch/powerpc/include/asm/thread_info.h
+++ b/arch/powerpc/include/asm/thread_info.h
@@ -103,6 +103,7 @@ void arch_setup_new_exec(void);
#define TIF_PATCH_PENDING 6 /* pending live patching update */
#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */
#define TIF_SINGLESTEP 8 /* singlestepping active */
+#define TIF_NEED_RESCHED_LAZY 9 /* Scheduler driven lazy preemption */
#define TIF_SECCOMP 10 /* secure computing */
#define TIF_RESTOREALL 11 /* Restore all regs (implies NOERROR) */
#define TIF_NOERROR 12 /* Force successful syscall return */
@@ -122,6 +123,7 @@ void arch_setup_new_exec(void);
#define _TIF_SYSCALL_TRACE (1<<TIF_SYSCALL_TRACE)
#define _TIF_SIGPENDING (1<<TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1<<TIF_NEED_RESCHED)
+#define _TIF_NEED_RESCHED_LAZY (1<<TIF_NEED_RESCHED_LAZY)
#define _TIF_NOTIFY_SIGNAL (1<<TIF_NOTIFY_SIGNAL)
#define _TIF_POLLING_NRFLAG (1<<TIF_POLLING_NRFLAG)
#define _TIF_32BIT (1<<TIF_32BIT)
@@ -142,9 +144,10 @@ void arch_setup_new_exec(void);
_TIF_SYSCALL_EMU)
#define _TIF_USER_WORK_MASK (_TIF_SIGPENDING | _TIF_NEED_RESCHED | \
- _TIF_NOTIFY_RESUME | _TIF_UPROBE | \
- _TIF_RESTORE_TM | _TIF_PATCH_PENDING | \
- _TIF_NOTIFY_SIGNAL)
+ _TIF_NEED_RESCHED_LAZY | _TIF_NOTIFY_RESUME | \
+ _TIF_UPROBE | _TIF_RESTORE_TM | \
+ _TIF_PATCH_PENDING | _TIF_NOTIFY_SIGNAL)
+
#define _TIF_PERSYSCALL_MASK (_TIF_RESTOREALL|_TIF_NOERROR)
/* Bits in local_flags */
diff --git a/arch/powerpc/include/asm/vdso_datapage.h b/arch/powerpc/include/asm/vdso_datapage.h
index 248dee138f7b..33069afccb3c 100644
--- a/arch/powerpc/include/asm/vdso_datapage.h
+++ b/arch/powerpc/include/asm/vdso_datapage.h
@@ -82,8 +82,9 @@ struct vdso_arch_data {
__u32 syscall_map[SYSCALL_MAP_SIZE]; /* Map of syscalls */
__u32 compat_syscall_map[SYSCALL_MAP_SIZE]; /* Map of compat syscalls */
- struct vdso_data data[CS_BASES];
struct vdso_rng_data rng_data;
+
+ struct vdso_data data[CS_BASES] __aligned(1 << CONFIG_PAGE_SHIFT);
};
#else /* CONFIG_PPC64 */
@@ -95,8 +96,9 @@ struct vdso_arch_data {
__u64 tb_ticks_per_sec; /* Timebase tics / sec 0x38 */
__u32 syscall_map[SYSCALL_MAP_SIZE]; /* Map of syscalls */
__u32 compat_syscall_map[0]; /* No compat syscalls on PPC32 */
- struct vdso_data data[CS_BASES];
struct vdso_rng_data rng_data;
+
+ struct vdso_data data[CS_BASES] __aligned(1 << CONFIG_PAGE_SHIFT);
};
#endif /* CONFIG_PPC64 */
@@ -105,29 +107,17 @@ extern struct vdso_arch_data *vdso_data;
#else /* __ASSEMBLY__ */
-.macro get_datapage ptr
+.macro get_datapage ptr offset=0
bcl 20, 31, .+4
999:
mflr \ptr
- addis \ptr, \ptr, (_vdso_datapage - 999b)@ha
- addi \ptr, \ptr, (_vdso_datapage - 999b)@l
+ addis \ptr, \ptr, (_vdso_datapage - 999b + \offset)@ha
+ addi \ptr, \ptr, (_vdso_datapage - 999b + \offset)@l
.endm
#include <asm/asm-offsets.h>
#include <asm/page.h>
-.macro get_realdatapage ptr scratch
- get_datapage \ptr
-#ifdef CONFIG_TIME_NS
- lwz \scratch, VDSO_CLOCKMODE_OFFSET(\ptr)
- xoris \scratch, \scratch, VDSO_CLOCKMODE_TIMENS@h
- xori \scratch, \scratch, VDSO_CLOCKMODE_TIMENS@l
- cntlzw \scratch, \scratch
- rlwinm \scratch, \scratch, PAGE_SHIFT - 5, 1 << PAGE_SHIFT
- add \ptr, \ptr, \scratch
-#endif
-.endm
-
#endif /* __ASSEMBLY__ */
#endif /* __KERNEL__ */
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index cbeeda45c00a..757ba2a3d4c8 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -348,8 +348,6 @@ int main(void)
#else
OFFSET(CFG_SYSCALL_MAP32, vdso_arch_data, syscall_map);
#endif
- OFFSET(VDSO_CLOCKMODE_OFFSET, vdso_arch_data, data[0].clock_mode);
- DEFINE(VDSO_CLOCKMODE_TIMENS, VDSO_CLOCKMODE_TIMENS);
#ifdef CONFIG_BUG
DEFINE(BUG_ENTRY_SIZE, sizeof(struct bug_entry));
diff --git a/arch/powerpc/kernel/ima_arch.c b/arch/powerpc/kernel/ima_arch.c
index b7029beed847..0d8892a03526 100644
--- a/arch/powerpc/kernel/ima_arch.c
+++ b/arch/powerpc/kernel/ima_arch.c
@@ -7,11 +7,6 @@
#include <linux/ima.h>
#include <asm/secure_boot.h>
-bool arch_ima_get_secureboot(void)
-{
- return is_ppc_secureboot_enabled();
-}
-
/*
* The "secure_rules" are enabled only on "secureboot" enabled systems.
* These rules verify the file signatures against known good values.
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index af62ec974b97..8f4acc55407b 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -185,7 +185,7 @@ interrupt_exit_user_prepare_main(unsigned long ret, struct pt_regs *regs)
ti_flags = read_thread_flags();
while (unlikely(ti_flags & (_TIF_USER_WORK_MASK & ~_TIF_RESTORE_TM))) {
local_irq_enable();
- if (ti_flags & _TIF_NEED_RESCHED) {
+ if (ti_flags & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)) {
schedule();
} else {
/*
@@ -396,7 +396,7 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs)
/* Returning to a kernel context with local irqs enabled. */
WARN_ON_ONCE(!(regs->msr & MSR_EE));
again:
- if (IS_ENABLED(CONFIG_PREEMPT)) {
+ if (IS_ENABLED(CONFIG_PREEMPTION)) {
/* Return to preemptible kernel context */
if (unlikely(read_thread_flags() & _TIF_NEED_RESCHED)) {
if (preempt_count() == 0)
diff --git a/arch/powerpc/kernel/irq.c b/arch/powerpc/kernel/irq.c
index 2e1600a8bbbb..7ce69e7d6b11 100644
--- a/arch/powerpc/kernel/irq.c
+++ b/arch/powerpc/kernel/irq.c
@@ -217,8 +217,12 @@ static __always_inline void call_do_softirq(const void *sp)
[sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE),
[callee] "i" (__do_softirq)
: // Clobbers
- "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
- "cr7", "r0", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10",
+ "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6", "cr7", "r0",
+ /* r2 may be clobbered by the callee when using PCREL mode in the ELFv2 ABI. */
+#ifdef CONFIG_PPC_KERNEL_PCREL
+ "r2",
+#endif
+ "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10",
"r11", "r12"
);
}
@@ -275,8 +279,12 @@ static __always_inline void call_do_irq(struct pt_regs *regs, void *sp)
[sp] "b" (sp), [offset] "i" (THREAD_SIZE - STACK_FRAME_MIN_SIZE),
[callee] "i" (__do_irq)
: // Clobbers
- "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6",
- "cr7", "r0", "r4", "r5", "r6", "r7", "r8", "r9", "r10",
+ "lr", "xer", "ctr", "memory", "cr0", "cr1", "cr5", "cr6", "cr7", "r0",
+ /* r2 may be clobbered by the callee when using PCREL mode in the ELFv2 ABI. */
+#ifdef CONFIG_PPC_KERNEL_PCREL
+ "r2",
+#endif
+ "r4", "r5", "r6", "r7", "r8", "r9", "r10",
"r11", "r12"
);
}
diff --git a/arch/powerpc/kernel/secure_boot.c b/arch/powerpc/kernel/secure_boot.c
index 9e0efb657f39..28436c1599e0 100644
--- a/arch/powerpc/kernel/secure_boot.c
+++ b/arch/powerpc/kernel/secure_boot.c
@@ -5,6 +5,8 @@
*/
#include <linux/types.h>
#include <linux/of.h>
+#include <linux/secure_boot.h>
+#include <linux/string_choices.h>
#include <asm/secure_boot.h>
static struct device_node *get_ppc_fw_sb_node(void)
@@ -38,11 +40,16 @@ bool is_ppc_secureboot_enabled(void)
of_node_put(node);
out:
- pr_info("Secure boot mode %s\n", enabled ? "enabled" : "disabled");
+ pr_info("Secure boot mode %s\n", str_enabled_disabled(enabled));
return enabled;
}
+bool arch_get_secureboot(void)
+{
+ return is_ppc_secureboot_enabled();
+}
+
bool is_ppc_trustedboot_enabled(void)
{
struct device_node *node;
@@ -62,7 +69,7 @@ bool is_ppc_trustedboot_enabled(void)
of_node_put(node);
out:
- pr_info("Trusted boot mode %s\n", enabled ? "enabled" : "disabled");
+ pr_info("Trusted boot mode %s\n", str_enabled_disabled(enabled));
return enabled;
}
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 09b2ccf38711..6bf939d15f1f 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -288,6 +288,9 @@ void smp_muxed_ipi_set_message(int cpu, int msg)
void smp_muxed_ipi_message_pass(int cpu, int msg)
{
+ if (!smp_ops->cause_ipi)
+ return;
+
smp_muxed_ipi_set_message(cpu, msg);
/*
diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c
index ee4b9d676cff..6166c1862c06 100644
--- a/arch/powerpc/kernel/vdso.c
+++ b/arch/powerpc/kernel/vdso.c
@@ -48,12 +48,13 @@ long sys_ni_syscall(void);
*/
static union {
struct vdso_arch_data data;
- u8 page[PAGE_SIZE];
+ u8 page[2 * PAGE_SIZE];
} vdso_data_store __page_aligned_data;
struct vdso_arch_data *vdso_data = &vdso_data_store.data;
enum vvar_pages {
- VVAR_DATA_PAGE_OFFSET,
+ VVAR_BASE_PAGE_OFFSET,
+ VVAR_TIME_PAGE_OFFSET,
VVAR_TIMENS_PAGE_OFFSET,
VVAR_NR_PAGES,
};
@@ -119,7 +120,7 @@ static struct vm_special_mapping vdso64_spec __ro_after_init = {
#ifdef CONFIG_TIME_NS
struct vdso_data *arch_get_vdso_data(void *vvar_page)
{
- return ((struct vdso_arch_data *)vvar_page)->data;
+ return vvar_page;
}
/*
@@ -153,11 +154,14 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
unsigned long pfn;
switch (vmf->pgoff) {
- case VVAR_DATA_PAGE_OFFSET:
+ case VVAR_BASE_PAGE_OFFSET:
+ pfn = virt_to_pfn(vdso_data);
+ break;
+ case VVAR_TIME_PAGE_OFFSET:
if (timens_page)
pfn = page_to_pfn(timens_page);
else
- pfn = virt_to_pfn(vdso_data);
+ pfn = virt_to_pfn(vdso_data->data);
break;
#ifdef CONFIG_TIME_NS
case VVAR_TIMENS_PAGE_OFFSET:
@@ -170,7 +174,7 @@ static vm_fault_t vvar_fault(const struct vm_special_mapping *sm,
*/
if (!timens_page)
return VM_FAULT_SIGBUS;
- pfn = virt_to_pfn(vdso_data);
+ pfn = virt_to_pfn(vdso_data->data);
break;
#endif /* CONFIG_TIME_NS */
default:
diff --git a/arch/powerpc/kernel/vdso/cacheflush.S b/arch/powerpc/kernel/vdso/cacheflush.S
index 3b2479bd2f9a..0085ae464dac 100644
--- a/arch/powerpc/kernel/vdso/cacheflush.S
+++ b/arch/powerpc/kernel/vdso/cacheflush.S
@@ -30,7 +30,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_COHERENT_ICACHE)
#ifdef CONFIG_PPC64
mflr r12
.cfi_register lr,r12
- get_realdatapage r10, r11
+ get_datapage r10
mtlr r12
.cfi_restore lr
#endif
diff --git a/arch/powerpc/kernel/vdso/datapage.S b/arch/powerpc/kernel/vdso/datapage.S
index 2b19b6201a33..db8e167f0166 100644
--- a/arch/powerpc/kernel/vdso/datapage.S
+++ b/arch/powerpc/kernel/vdso/datapage.S
@@ -28,7 +28,7 @@ V_FUNCTION_BEGIN(__kernel_get_syscall_map)
mflr r12
.cfi_register lr,r12
mr. r4,r3
- get_realdatapage r3, r11
+ get_datapage r3
mtlr r12
#ifdef __powerpc64__
addi r3,r3,CFG_SYSCALL_MAP64
@@ -52,7 +52,7 @@ V_FUNCTION_BEGIN(__kernel_get_tbfreq)
.cfi_startproc
mflr r12
.cfi_register lr,r12
- get_realdatapage r3, r11
+ get_datapage r3
#ifndef __powerpc64__
lwz r4,(CFG_TB_TICKS_PER_SEC + 4)(r3)
#endif
diff --git a/arch/powerpc/kernel/vdso/getrandom.S b/arch/powerpc/kernel/vdso/getrandom.S
index f3bbf931931c..3deddcf89f99 100644
--- a/arch/powerpc/kernel/vdso/getrandom.S
+++ b/arch/powerpc/kernel/vdso/getrandom.S
@@ -31,8 +31,7 @@
PPC_STL r2, PPC_MIN_STKFRM + STK_GOT(r1)
.cfi_rel_offset r2, PPC_MIN_STKFRM + STK_GOT
#endif
- get_realdatapage r8, r11
- addi r8, r8, VDSO_RNG_DATA_OFFSET
+ get_datapage r8 VDSO_RNG_DATA_OFFSET
bl CFUNC(DOTSYM(\funct))
PPC_LL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1)
#ifdef __powerpc64__
diff --git a/arch/powerpc/kernel/vdso/gettimeofday.S b/arch/powerpc/kernel/vdso/gettimeofday.S
index 5540d7021fa2..5333848322ca 100644
--- a/arch/powerpc/kernel/vdso/gettimeofday.S
+++ b/arch/powerpc/kernel/vdso/gettimeofday.S
@@ -32,11 +32,10 @@
PPC_STL r2, PPC_MIN_STKFRM + STK_GOT(r1)
.cfi_rel_offset r2, PPC_MIN_STKFRM + STK_GOT
#endif
- get_datapage r5
.ifeq \call_time
- addi r5, r5, VDSO_DATA_OFFSET
+ get_datapage r5 VDSO_DATA_OFFSET
.else
- addi r4, r5, VDSO_DATA_OFFSET
+ get_datapage r4 VDSO_DATA_OFFSET
.endif
bl CFUNC(DOTSYM(\funct))
PPC_LL r0, PPC_MIN_STKFRM + PPC_LR_STKOFF(r1)
diff --git a/arch/powerpc/kernel/vdso/vdso32.lds.S b/arch/powerpc/kernel/vdso/vdso32.lds.S
index 7b41d5d256e8..1a1b0b6d681a 100644
--- a/arch/powerpc/kernel/vdso/vdso32.lds.S
+++ b/arch/powerpc/kernel/vdso/vdso32.lds.S
@@ -16,7 +16,7 @@ OUTPUT_ARCH(powerpc:common)
SECTIONS
{
- PROVIDE(_vdso_datapage = . - 2 * PAGE_SIZE);
+ PROVIDE(_vdso_datapage = . - 3 * PAGE_SIZE);
. = SIZEOF_HEADERS;
.hash : { *(.hash) } :text
diff --git a/arch/powerpc/kernel/vdso/vdso64.lds.S b/arch/powerpc/kernel/vdso/vdso64.lds.S
index 9481e4b892ed..e21b5506cad6 100644
--- a/arch/powerpc/kernel/vdso/vdso64.lds.S
+++ b/arch/powerpc/kernel/vdso/vdso64.lds.S
@@ -16,7 +16,7 @@ OUTPUT_ARCH(powerpc:common64)
SECTIONS
{
- PROVIDE(_vdso_datapage = . - 2 * PAGE_SIZE);
+ PROVIDE(_vdso_datapage = . - 3 * PAGE_SIZE);
. = SIZEOF_HEADERS;
.hash : { *(.hash) } :text
diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
index e6539f213b3d..a520f851c3a6 100644
--- a/arch/powerpc/kexec/crash.c
+++ b/arch/powerpc/kexec/crash.c
@@ -502,7 +502,7 @@ static void update_crash_elfcorehdr(struct kimage *image, struct memory_notify *
ret = get_crash_memory_ranges(&cmem);
if (ret) {
pr_err("Failed to get crash mem range\n");
- return;
+ goto out;
}
/*
diff --git a/arch/powerpc/kexec/file_load_64.c b/arch/powerpc/kexec/file_load_64.c
index 276233f73a53..cb14de058de2 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -847,7 +847,7 @@ unsigned int kexec_extra_fdt_size_ppc64(struct kimage *image, struct crash_mem *
extra_size += (cpu_nodes - boot_cpu_node_count) * cpu_node_size();
/* Consider extra space for reserved memory ranges if any */
- if (rmem->nr_ranges > 0)
+ if (rmem && rmem->nr_ranges > 0)
extra_size += sizeof(struct fdt_reserve_entry) * rmem->nr_ranges;
return extra_size + kdump_extra_fdt_size_ppc64(image, cpu_nodes);
diff --git a/arch/powerpc/kexec/ranges.c b/arch/powerpc/kexec/ranges.c
index 426bdca4667e..8815ab961752 100644
--- a/arch/powerpc/kexec/ranges.c
+++ b/arch/powerpc/kexec/ranges.c
@@ -21,6 +21,7 @@
#include <linux/of.h>
#include <linux/slab.h>
#include <linux/memblock.h>
+#include <linux/minmax.h>
#include <linux/crash_core.h>
#include <asm/sections.h>
#include <asm/kexec_ranges.h>
@@ -105,19 +106,16 @@ static void __merge_memory_ranges(struct crash_mem *mem_rngs)
struct range *ranges;
int i, idx;
- if (!mem_rngs)
+ if (!mem_rngs || mem_rngs->nr_ranges <= 1)
return;
idx = 0;
- ranges = &(mem_rngs->ranges[0]);
+ ranges = mem_rngs->ranges;
for (i = 1; i < mem_rngs->nr_ranges; i++) {
- if (ranges[i].start <= (ranges[i-1].end + 1))
- ranges[idx].end = ranges[i].end;
+ if (ranges[i].start <= (ranges[idx].end + 1))
+ ranges[idx].end = max(ranges[idx].end, ranges[i].end);
else {
idx++;
- if (i == idx)
- continue;
-
ranges[idx] = ranges[i];
}
}
diff --git a/arch/powerpc/mm/Makefile b/arch/powerpc/mm/Makefile
index 0fe2f085c05a..8c1582b2987d 100644
--- a/arch/powerpc/mm/Makefile
+++ b/arch/powerpc/mm/Makefile
@@ -15,5 +15,5 @@ obj-$(CONFIG_NUMA) += numa.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
obj-$(CONFIG_NOT_COHERENT_CACHE) += dma-noncoherent.o
obj-$(CONFIG_PPC_COPRO_BASE) += copro_fault.o
-obj-$(CONFIG_PTDUMP_CORE) += ptdump/
+obj-$(CONFIG_PTDUMP) += ptdump/
obj-$(CONFIG_KASAN) += kasan/
diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index 0d807bf2328d..cc82d9aa53cd 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -1318,7 +1318,6 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
* covering out both edges.
*/
unsigned long addr;
- unsigned long addr_pfn = start_pfn;
unsigned long next;
pgd_t *pgd;
p4d_t *p4d;
@@ -1339,7 +1338,6 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
if (pmd_leaf(READ_ONCE(*pmd))) {
/* existing huge mapping. Skip the range */
- addr_pfn += (PMD_SIZE >> PAGE_SHIFT);
next = pmd_addr_end(addr, end);
continue;
}
@@ -1352,11 +1350,11 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
* page whose VMEMMAP_RESERVE_NR pages were mapped and
* this request fall in those pages.
*/
- addr_pfn += 1;
next = addr + PAGE_SIZE;
continue;
} else {
unsigned long nr_pages = pgmap_vmemmap_nr(pgmap);
+ unsigned long addr_pfn = page_to_pfn((struct page *)addr);
unsigned long pfn_offset = addr_pfn - ALIGN_DOWN(addr_pfn, nr_pages);
pte_t *tail_page_pte;
@@ -1380,7 +1378,6 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
if (!pte)
return -ENOMEM;
- addr_pfn += 2;
next = addr + 2 * PAGE_SIZE;
continue;
}
@@ -1396,7 +1393,6 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
return -ENOMEM;
vmemmap_verify(pte, node, addr, addr + PAGE_SIZE);
- addr_pfn += 1;
next = addr + PAGE_SIZE;
continue;
}
@@ -1406,7 +1402,6 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
return -ENOMEM;
vmemmap_verify(pte, node, addr, addr + PAGE_SIZE);
- addr_pfn += 1;
next = addr + PAGE_SIZE;
continue;
}
diff --git a/arch/powerpc/mm/ptdump/ptdump.c b/arch/powerpc/mm/ptdump/ptdump.c
index 9dc239967b77..b2358d794855 100644
--- a/arch/powerpc/mm/ptdump/ptdump.c
+++ b/arch/powerpc/mm/ptdump/ptdump.c
@@ -298,6 +298,38 @@ static void populate_markers(void)
#endif
}
+static void note_page_pte(struct ptdump_state *pt_st, unsigned long addr, pte_t pte)
+{
+ note_page(pt_st, addr, 4, pte_val(pte));
+}
+
+static void note_page_pmd(struct ptdump_state *pt_st, unsigned long addr, pmd_t pmd)
+{
+ note_page(pt_st, addr, 3, pmd_val(pmd));
+}
+
+static void note_page_pud(struct ptdump_state *pt_st, unsigned long addr, pud_t pud)
+{
+ note_page(pt_st, addr, 2, pud_val(pud));
+}
+
+static void note_page_p4d(struct ptdump_state *pt_st, unsigned long addr, p4d_t p4d)
+{
+ note_page(pt_st, addr, 1, p4d_val(p4d));
+}
+
+static void note_page_pgd(struct ptdump_state *pt_st, unsigned long addr, pgd_t pgd)
+{
+ note_page(pt_st, addr, 0, pgd_val(pgd));
+}
+
+static void note_page_flush(struct ptdump_state *pt_st)
+{
+ pte_t pte_zero = {0};
+
+ note_page(pt_st, 0, -1, pte_val(pte_zero));
+}
+
static int ptdump_show(struct seq_file *m, void *v)
{
struct pg_state st = {
@@ -305,7 +337,12 @@ static int ptdump_show(struct seq_file *m, void *v)
.marker = address_markers,
.level = -1,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = ptdump_range,
}
};
@@ -338,7 +375,12 @@ bool ptdump_check_wx(void)
.level = -1,
.check_wx = true,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = ptdump_range,
}
};
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index b10a25325238..d9ee2e7735ec 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -76,6 +76,7 @@
#include <asm/dtl.h>
#include <asm/hvconsole.h>
#include <asm/setup.h>
+#include <asm/papr-watchdog.h>
#include "pseries.h"
@@ -187,14 +188,37 @@ static void __init fwnmi_init(void)
#endif
}
+static void pseries_crash_stop_watchdogs(void)
+{
+ long rc;
+
+ rc = plpar_hcall_norets_notrace(H_WATCHDOG, PSERIES_WDTF_OP_STOP,
+ PSERIES_WDT_NUM_ALL);
+ if (rc != H_SUCCESS && rc != H_NOOP)
+ pr_warn("Could not stop watchdogs before kdump rc=%ld\n", rc);
+}
+
/*
* Affix a device for the first timer to the platform bus if
* we have firmware support for the H_WATCHDOG hypercall.
*/
static __init int pseries_wdt_init(void)
{
- if (firmware_has_feature(FW_FEATURE_WATCHDOG))
- platform_device_register_simple("pseries-wdt", 0, NULL, 0);
+ struct platform_device *pdev;
+
+ if (!firmware_has_feature(FW_FEATURE_WATCHDOG))
+ return 0;
+
+ pdev = platform_device_register_simple("pseries-wdt", 0, NULL, 0);
+
+ if (IS_ERR(pdev)) {
+ pr_err("Failed to register pseries-wdt platform device\n");
+ return PTR_ERR(pdev);
+ }
+
+ if (crash_shutdown_register(pseries_crash_stop_watchdogs))
+ pr_warn("Could not register watchdog crash shutdown handler\n");
+
return 0;
}
machine_subsys_initcall(pseries, pseries_wdt_init);
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 1c7575816795..dcb2fef2fb54 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -40,6 +40,7 @@ config RISCV
select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
select ARCH_HAS_PMEM_API
select ARCH_HAS_PREPARE_SYNC_CORE_CMD
+ select ARCH_HAS_PTDUMP if MMU
select ARCH_HAS_PTE_DEVMAP if 64BIT && MMU
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SET_DIRECT_MAP if MMU
@@ -108,7 +109,6 @@ config RISCV
select GENERIC_IRQ_SHOW_LEVEL
select GENERIC_LIB_DEVMEM_IS_ALLOWED
select GENERIC_PCI_IOMAP
- select GENERIC_PTDUMP if MMU
select GENERIC_SCHED_CLOCK
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL if MMU && 64BIT
diff --git a/arch/riscv/boot/dts/sophgo/cv18xx.dtsi b/arch/riscv/boot/dts/sophgo/cv18xx.dtsi
index b8063ba6d6d7..e519a72c093d 100644
--- a/arch/riscv/boot/dts/sophgo/cv18xx.dtsi
+++ b/arch/riscv/boot/dts/sophgo/cv18xx.dtsi
@@ -298,7 +298,7 @@ sdhci0: mmc@4310000 {
};
dmac: dma-controller@4330000 {
- compatible = "snps,axi-dma-1.01a";
+ compatible = "sophgo,cv1800b-axi-dma", "snps,axi-dma-1.01a";
reg = <0x04330000 0x1000>;
interrupts = <29 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk CLK_SDMA_AXI>, <&clk CLK_SDMA_AXI>;
diff --git a/arch/riscv/include/asm/kexec.h b/arch/riscv/include/asm/kexec.h
index 2b56769cb530..518825fe4160 100644
--- a/arch/riscv/include/asm/kexec.h
+++ b/arch/riscv/include/asm/kexec.h
@@ -67,6 +67,11 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
struct kimage;
int arch_kimage_file_post_load_cleanup(struct kimage *image);
#define arch_kimage_file_post_load_cleanup arch_kimage_file_post_load_cleanup
+
+int load_extra_segments(struct kimage *image, unsigned long kernel_start,
+ unsigned long kernel_len, char *initrd,
+ unsigned long initrd_len, char *cmdline,
+ unsigned long cmdline_len);
#endif
#endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 6b3b5255c889..0678650817d4 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -106,7 +106,7 @@ obj-$(CONFIG_HOTPLUG_CPU) += cpu-hotplug.o
obj-$(CONFIG_PARAVIRT) += paravirt.o
obj-$(CONFIG_KGDB) += kgdb.o
obj-$(CONFIG_KEXEC_CORE) += kexec_relocate.o crash_save_regs.o machine_kexec.o
-obj-$(CONFIG_KEXEC_FILE) += elf_kexec.o machine_kexec_file.o
+obj-$(CONFIG_KEXEC_FILE) += kexec_elf.o machine_kexec_file.o
obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
obj-$(CONFIG_VMCORE_INFO) += vmcore_info.o
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 308430af3e8f..472de1ed1ad7 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -263,7 +263,8 @@ static const unsigned int riscv_c_exts[] = {
/*
* The canonical order of ISA extension names in the ISA string is defined in
- * chapter 27 of the unprivileged specification.
+ * Chapter 27 of the RISC-V Instruction Set Manual Volume I Unprivileged ISA
+ * (Document Version 20191213).
*
* Ordinarily, for in-kernel data structures, this order is unimportant but
* isa_ext_arr defines the order of the ISA string in /proc/cpuinfo.
diff --git a/arch/riscv/kernel/elf_kexec.c b/arch/riscv/kernel/elf_kexec.c
deleted file mode 100644
index e783a72d051f..000000000000
--- a/arch/riscv/kernel/elf_kexec.c
+++ /dev/null
@@ -1,485 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Load ELF vmlinux file for the kexec_file_load syscall.
- *
- * Copyright (C) 2021 Huawei Technologies Co, Ltd.
- *
- * Author: Liao Chang (liaochang1@huawei.com)
- *
- * Based on kexec-tools' kexec-elf-riscv.c, heavily modified
- * for kernel.
- */
-
-#define pr_fmt(fmt) "kexec_image: " fmt
-
-#include <linux/elf.h>
-#include <linux/kexec.h>
-#include <linux/slab.h>
-#include <linux/of.h>
-#include <linux/libfdt.h>
-#include <linux/types.h>
-#include <linux/memblock.h>
-#include <linux/vmalloc.h>
-#include <asm/setup.h>
-
-int arch_kimage_file_post_load_cleanup(struct kimage *image)
-{
- kvfree(image->arch.fdt);
- image->arch.fdt = NULL;
-
- vfree(image->elf_headers);
- image->elf_headers = NULL;
- image->elf_headers_sz = 0;
-
- return kexec_image_post_load_cleanup_default(image);
-}
-
-static int riscv_kexec_elf_load(struct kimage *image, struct elfhdr *ehdr,
- struct kexec_elf_info *elf_info, unsigned long old_pbase,
- unsigned long new_pbase)
-{
- int i;
- int ret = 0;
- size_t size;
- struct kexec_buf kbuf;
- const struct elf_phdr *phdr;
-
- kbuf.image = image;
-
- for (i = 0; i < ehdr->e_phnum; i++) {
- phdr = &elf_info->proghdrs[i];
- if (phdr->p_type != PT_LOAD)
- continue;
-
- size = phdr->p_filesz;
- if (size > phdr->p_memsz)
- size = phdr->p_memsz;
-
- kbuf.buffer = (void *) elf_info->buffer + phdr->p_offset;
- kbuf.bufsz = size;
- kbuf.buf_align = phdr->p_align;
- kbuf.mem = phdr->p_paddr - old_pbase + new_pbase;
- kbuf.memsz = phdr->p_memsz;
- kbuf.top_down = false;
- ret = kexec_add_buffer(&kbuf);
- if (ret)
- break;
- }
-
- return ret;
-}
-
-/*
- * Go through the available phsyical memory regions and find one that hold
- * an image of the specified size.
- */
-static int elf_find_pbase(struct kimage *image, unsigned long kernel_len,
- struct elfhdr *ehdr, struct kexec_elf_info *elf_info,
- unsigned long *old_pbase, unsigned long *new_pbase)
-{
- int i;
- int ret;
- struct kexec_buf kbuf;
- const struct elf_phdr *phdr;
- unsigned long lowest_paddr = ULONG_MAX;
- unsigned long lowest_vaddr = ULONG_MAX;
-
- for (i = 0; i < ehdr->e_phnum; i++) {
- phdr = &elf_info->proghdrs[i];
- if (phdr->p_type != PT_LOAD)
- continue;
-
- if (lowest_paddr > phdr->p_paddr)
- lowest_paddr = phdr->p_paddr;
-
- if (lowest_vaddr > phdr->p_vaddr)
- lowest_vaddr = phdr->p_vaddr;
- }
-
- kbuf.image = image;
- kbuf.buf_min = lowest_paddr;
- kbuf.buf_max = ULONG_MAX;
-
- /*
- * Current riscv boot protocol requires 2MB alignment for
- * RV64 and 4MB alignment for RV32
- *
- */
- kbuf.buf_align = PMD_SIZE;
- kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
- kbuf.memsz = ALIGN(kernel_len, PAGE_SIZE);
- kbuf.top_down = false;
- ret = arch_kexec_locate_mem_hole(&kbuf);
- if (!ret) {
- *old_pbase = lowest_paddr;
- *new_pbase = kbuf.mem;
- image->start = ehdr->e_entry - lowest_vaddr + kbuf.mem;
- }
- return ret;
-}
-
-#ifdef CONFIG_CRASH_DUMP
-static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
-{
- unsigned int *nr_ranges = arg;
-
- (*nr_ranges)++;
- return 0;
-}
-
-static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
-{
- struct crash_mem *cmem = arg;
-
- cmem->ranges[cmem->nr_ranges].start = res->start;
- cmem->ranges[cmem->nr_ranges].end = res->end;
- cmem->nr_ranges++;
-
- return 0;
-}
-
-static int prepare_elf_headers(void **addr, unsigned long *sz)
-{
- struct crash_mem *cmem;
- unsigned int nr_ranges;
- int ret;
-
- nr_ranges = 1; /* For exclusion of crashkernel region */
- walk_system_ram_res(0, -1, &nr_ranges, get_nr_ram_ranges_callback);
-
- cmem = kmalloc(struct_size(cmem, ranges, nr_ranges), GFP_KERNEL);
- if (!cmem)
- return -ENOMEM;
-
- cmem->max_nr_ranges = nr_ranges;
- cmem->nr_ranges = 0;
- ret = walk_system_ram_res(0, -1, cmem, prepare_elf64_ram_headers_callback);
- if (ret)
- goto out;
-
- /* Exclude crashkernel region */
- ret = crash_exclude_mem_range(cmem, crashk_res.start, crashk_res.end);
- if (!ret)
- ret = crash_prepare_elf64_headers(cmem, true, addr, sz);
-
-out:
- kfree(cmem);
- return ret;
-}
-
-static char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
- unsigned long cmdline_len)
-{
- int elfcorehdr_strlen;
- char *cmdline_ptr;
-
- cmdline_ptr = kzalloc(COMMAND_LINE_SIZE, GFP_KERNEL);
- if (!cmdline_ptr)
- return NULL;
-
- elfcorehdr_strlen = sprintf(cmdline_ptr, "elfcorehdr=0x%lx ",
- image->elf_load_addr);
-
- if (elfcorehdr_strlen + cmdline_len > COMMAND_LINE_SIZE) {
- pr_err("Appending elfcorehdr=<addr> exceeds cmdline size\n");
- kfree(cmdline_ptr);
- return NULL;
- }
-
- memcpy(cmdline_ptr + elfcorehdr_strlen, cmdline, cmdline_len);
- /* Ensure it's nul terminated */
- cmdline_ptr[COMMAND_LINE_SIZE - 1] = '\0';
- return cmdline_ptr;
-}
-#endif
-
-static void *elf_kexec_load(struct kimage *image, char *kernel_buf,
- unsigned long kernel_len, char *initrd,
- unsigned long initrd_len, char *cmdline,
- unsigned long cmdline_len)
-{
- int ret;
- void *fdt;
- unsigned long old_kernel_pbase = ULONG_MAX;
- unsigned long new_kernel_pbase = 0UL;
- unsigned long initrd_pbase = 0UL;
- unsigned long kernel_start;
- struct elfhdr ehdr;
- struct kexec_buf kbuf;
- struct kexec_elf_info elf_info;
- char *modified_cmdline = NULL;
-
- ret = kexec_build_elf_info(kernel_buf, kernel_len, &ehdr, &elf_info);
- if (ret)
- return ERR_PTR(ret);
-
- ret = elf_find_pbase(image, kernel_len, &ehdr, &elf_info,
- &old_kernel_pbase, &new_kernel_pbase);
- if (ret)
- goto out;
- kernel_start = image->start;
-
- /* Add the kernel binary to the image */
- ret = riscv_kexec_elf_load(image, &ehdr, &elf_info,
- old_kernel_pbase, new_kernel_pbase);
- if (ret)
- goto out;
-
- kbuf.image = image;
- kbuf.buf_min = new_kernel_pbase + kernel_len;
- kbuf.buf_max = ULONG_MAX;
-
-#ifdef CONFIG_CRASH_DUMP
- /* Add elfcorehdr */
- if (image->type == KEXEC_TYPE_CRASH) {
- void *headers;
- unsigned long headers_sz;
- ret = prepare_elf_headers(&headers, &headers_sz);
- if (ret) {
- pr_err("Preparing elf core header failed\n");
- goto out;
- }
-
- kbuf.buffer = headers;
- kbuf.bufsz = headers_sz;
- kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
- kbuf.memsz = headers_sz;
- kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
- kbuf.top_down = true;
-
- ret = kexec_add_buffer(&kbuf);
- if (ret) {
- vfree(headers);
- goto out;
- }
- image->elf_headers = headers;
- image->elf_load_addr = kbuf.mem;
- image->elf_headers_sz = headers_sz;
-
- kexec_dprintk("Loaded elf core header at 0x%lx bufsz=0x%lx memsz=0x%lx\n",
- image->elf_load_addr, kbuf.bufsz, kbuf.memsz);
-
- /* Setup cmdline for kdump kernel case */
- modified_cmdline = setup_kdump_cmdline(image, cmdline,
- cmdline_len);
- if (!modified_cmdline) {
- pr_err("Setting up cmdline for kdump kernel failed\n");
- ret = -EINVAL;
- goto out;
- }
- cmdline = modified_cmdline;
- }
-#endif
-
-#ifdef CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY
- /* Add purgatory to the image */
- kbuf.top_down = true;
- kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
- ret = kexec_load_purgatory(image, &kbuf);
- if (ret) {
- pr_err("Error loading purgatory ret=%d\n", ret);
- goto out;
- }
- kexec_dprintk("Loaded purgatory at 0x%lx\n", kbuf.mem);
-
- ret = kexec_purgatory_get_set_symbol(image, "riscv_kernel_entry",
- &kernel_start,
- sizeof(kernel_start), 0);
- if (ret)
- pr_err("Error update purgatory ret=%d\n", ret);
-#endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */
-
- /* Add the initrd to the image */
- if (initrd != NULL) {
- kbuf.buffer = initrd;
- kbuf.bufsz = kbuf.memsz = initrd_len;
- kbuf.buf_align = PAGE_SIZE;
- kbuf.top_down = true;
- kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
- ret = kexec_add_buffer(&kbuf);
- if (ret)
- goto out;
- initrd_pbase = kbuf.mem;
- kexec_dprintk("Loaded initrd at 0x%lx\n", initrd_pbase);
- }
-
- /* Add the DTB to the image */
- fdt = of_kexec_alloc_and_setup_fdt(image, initrd_pbase,
- initrd_len, cmdline, 0);
- if (!fdt) {
- pr_err("Error setting up the new device tree.\n");
- ret = -EINVAL;
- goto out;
- }
-
- fdt_pack(fdt);
- kbuf.buffer = fdt;
- kbuf.bufsz = kbuf.memsz = fdt_totalsize(fdt);
- kbuf.buf_align = PAGE_SIZE;
- kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
- kbuf.top_down = true;
- ret = kexec_add_buffer(&kbuf);
- if (ret) {
- pr_err("Error add DTB kbuf ret=%d\n", ret);
- goto out_free_fdt;
- }
- /* Cache the fdt buffer address for memory cleanup */
- image->arch.fdt = fdt;
- kexec_dprintk("Loaded device tree at 0x%lx\n", kbuf.mem);
- goto out;
-
-out_free_fdt:
- kvfree(fdt);
-out:
- kfree(modified_cmdline);
- kexec_free_elf_info(&elf_info);
- return ret ? ERR_PTR(ret) : NULL;
-}
-
-#define RV_X(x, s, n) (((x) >> (s)) & ((1 << (n)) - 1))
-#define RISCV_IMM_BITS 12
-#define RISCV_IMM_REACH (1LL << RISCV_IMM_BITS)
-#define RISCV_CONST_HIGH_PART(x) \
- (((x) + (RISCV_IMM_REACH >> 1)) & ~(RISCV_IMM_REACH - 1))
-#define RISCV_CONST_LOW_PART(x) ((x) - RISCV_CONST_HIGH_PART(x))
-
-#define ENCODE_ITYPE_IMM(x) \
- (RV_X(x, 0, 12) << 20)
-#define ENCODE_BTYPE_IMM(x) \
- ((RV_X(x, 1, 4) << 8) | (RV_X(x, 5, 6) << 25) | \
- (RV_X(x, 11, 1) << 7) | (RV_X(x, 12, 1) << 31))
-#define ENCODE_UTYPE_IMM(x) \
- (RV_X(x, 12, 20) << 12)
-#define ENCODE_JTYPE_IMM(x) \
- ((RV_X(x, 1, 10) << 21) | (RV_X(x, 11, 1) << 20) | \
- (RV_X(x, 12, 8) << 12) | (RV_X(x, 20, 1) << 31))
-#define ENCODE_CBTYPE_IMM(x) \
- ((RV_X(x, 1, 2) << 3) | (RV_X(x, 3, 2) << 10) | (RV_X(x, 5, 1) << 2) | \
- (RV_X(x, 6, 2) << 5) | (RV_X(x, 8, 1) << 12))
-#define ENCODE_CJTYPE_IMM(x) \
- ((RV_X(x, 1, 3) << 3) | (RV_X(x, 4, 1) << 11) | (RV_X(x, 5, 1) << 2) | \
- (RV_X(x, 6, 1) << 7) | (RV_X(x, 7, 1) << 6) | (RV_X(x, 8, 2) << 9) | \
- (RV_X(x, 10, 1) << 8) | (RV_X(x, 11, 1) << 12))
-#define ENCODE_UJTYPE_IMM(x) \
- (ENCODE_UTYPE_IMM(RISCV_CONST_HIGH_PART(x)) | \
- (ENCODE_ITYPE_IMM(RISCV_CONST_LOW_PART(x)) << 32))
-#define ENCODE_UITYPE_IMM(x) \
- (ENCODE_UTYPE_IMM(x) | (ENCODE_ITYPE_IMM(x) << 32))
-
-#define CLEAN_IMM(type, x) \
- ((~ENCODE_##type##_IMM((uint64_t)(-1))) & (x))
-
-int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
- Elf_Shdr *section,
- const Elf_Shdr *relsec,
- const Elf_Shdr *symtab)
-{
- const char *strtab, *name, *shstrtab;
- const Elf_Shdr *sechdrs;
- Elf64_Rela *relas;
- int i, r_type;
-
- /* String & section header string table */
- sechdrs = (void *)pi->ehdr + pi->ehdr->e_shoff;
- strtab = (char *)pi->ehdr + sechdrs[symtab->sh_link].sh_offset;
- shstrtab = (char *)pi->ehdr + sechdrs[pi->ehdr->e_shstrndx].sh_offset;
-
- relas = (void *)pi->ehdr + relsec->sh_offset;
-
- for (i = 0; i < relsec->sh_size / sizeof(*relas); i++) {
- const Elf_Sym *sym; /* symbol to relocate */
- unsigned long addr; /* final location after relocation */
- unsigned long val; /* relocated symbol value */
- unsigned long sec_base; /* relocated symbol value */
- void *loc; /* tmp location to modify */
-
- sym = (void *)pi->ehdr + symtab->sh_offset;
- sym += ELF64_R_SYM(relas[i].r_info);
-
- if (sym->st_name)
- name = strtab + sym->st_name;
- else
- name = shstrtab + sechdrs[sym->st_shndx].sh_name;
-
- loc = pi->purgatory_buf;
- loc += section->sh_offset;
- loc += relas[i].r_offset;
-
- if (sym->st_shndx == SHN_ABS)
- sec_base = 0;
- else if (sym->st_shndx >= pi->ehdr->e_shnum) {
- pr_err("Invalid section %d for symbol %s\n",
- sym->st_shndx, name);
- return -ENOEXEC;
- } else
- sec_base = pi->sechdrs[sym->st_shndx].sh_addr;
-
- val = sym->st_value;
- val += sec_base;
- val += relas[i].r_addend;
-
- addr = section->sh_addr + relas[i].r_offset;
-
- r_type = ELF64_R_TYPE(relas[i].r_info);
-
- switch (r_type) {
- case R_RISCV_BRANCH:
- *(u32 *)loc = CLEAN_IMM(BTYPE, *(u32 *)loc) |
- ENCODE_BTYPE_IMM(val - addr);
- break;
- case R_RISCV_JAL:
- *(u32 *)loc = CLEAN_IMM(JTYPE, *(u32 *)loc) |
- ENCODE_JTYPE_IMM(val - addr);
- break;
- /*
- * With no R_RISCV_PCREL_LO12_S, R_RISCV_PCREL_LO12_I
- * sym is expected to be next to R_RISCV_PCREL_HI20
- * in purgatory relsec. Handle it like R_RISCV_CALL
- * sym, instead of searching the whole relsec.
- */
- case R_RISCV_PCREL_HI20:
- case R_RISCV_CALL_PLT:
- case R_RISCV_CALL:
- *(u64 *)loc = CLEAN_IMM(UITYPE, *(u64 *)loc) |
- ENCODE_UJTYPE_IMM(val - addr);
- break;
- case R_RISCV_RVC_BRANCH:
- *(u32 *)loc = CLEAN_IMM(CBTYPE, *(u32 *)loc) |
- ENCODE_CBTYPE_IMM(val - addr);
- break;
- case R_RISCV_RVC_JUMP:
- *(u32 *)loc = CLEAN_IMM(CJTYPE, *(u32 *)loc) |
- ENCODE_CJTYPE_IMM(val - addr);
- break;
- case R_RISCV_ADD16:
- *(u16 *)loc += val;
- break;
- case R_RISCV_SUB16:
- *(u16 *)loc -= val;
- break;
- case R_RISCV_ADD32:
- *(u32 *)loc += val;
- break;
- case R_RISCV_SUB32:
- *(u32 *)loc -= val;
- break;
- /* It has been applied by R_RISCV_PCREL_HI20 sym */
- case R_RISCV_PCREL_LO12_I:
- case R_RISCV_ALIGN:
- case R_RISCV_RELAX:
- break;
- case R_RISCV_64:
- *(u64 *)loc = val;
- break;
- default:
- pr_err("Unknown rela relocation: %d\n", r_type);
- return -ENOEXEC;
- }
- }
- return 0;
-}
-
-const struct kexec_file_ops elf_kexec_ops = {
- .probe = kexec_elf_probe,
- .load = elf_kexec_load,
-};
diff --git a/arch/riscv/kernel/kexec_elf.c b/arch/riscv/kernel/kexec_elf.c
new file mode 100644
index 000000000000..f4755d49b89e
--- /dev/null
+++ b/arch/riscv/kernel/kexec_elf.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Load ELF vmlinux file for the kexec_file_load syscall.
+ *
+ * Copyright (C) 2021 Huawei Technologies Co, Ltd.
+ *
+ * Author: Liao Chang (liaochang1@huawei.com)
+ *
+ * Based on kexec-tools' kexec-elf-riscv.c, heavily modified
+ * for kernel.
+ */
+
+#define pr_fmt(fmt) "kexec_image: " fmt
+
+#include <linux/elf.h>
+#include <linux/kexec.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/libfdt.h>
+#include <linux/types.h>
+#include <linux/memblock.h>
+#include <asm/setup.h>
+
+static int riscv_kexec_elf_load(struct kimage *image, struct elfhdr *ehdr,
+ struct kexec_elf_info *elf_info, unsigned long old_pbase,
+ unsigned long new_pbase)
+{
+ int i;
+ int ret = 0;
+ size_t size;
+ struct kexec_buf kbuf;
+ const struct elf_phdr *phdr;
+
+ kbuf.image = image;
+
+ for (i = 0; i < ehdr->e_phnum; i++) {
+ phdr = &elf_info->proghdrs[i];
+ if (phdr->p_type != PT_LOAD)
+ continue;
+
+ size = phdr->p_filesz;
+ if (size > phdr->p_memsz)
+ size = phdr->p_memsz;
+
+ kbuf.buffer = (void *) elf_info->buffer + phdr->p_offset;
+ kbuf.bufsz = size;
+ kbuf.buf_align = phdr->p_align;
+ kbuf.mem = phdr->p_paddr - old_pbase + new_pbase;
+ kbuf.memsz = phdr->p_memsz;
+ kbuf.top_down = false;
+ ret = kexec_add_buffer(&kbuf);
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
+
+/*
+ * Go through the available phsyical memory regions and find one that hold
+ * an image of the specified size.
+ */
+static int elf_find_pbase(struct kimage *image, unsigned long kernel_len,
+ struct elfhdr *ehdr, struct kexec_elf_info *elf_info,
+ unsigned long *old_pbase, unsigned long *new_pbase)
+{
+ int i;
+ int ret;
+ struct kexec_buf kbuf;
+ const struct elf_phdr *phdr;
+ unsigned long lowest_paddr = ULONG_MAX;
+ unsigned long lowest_vaddr = ULONG_MAX;
+
+ for (i = 0; i < ehdr->e_phnum; i++) {
+ phdr = &elf_info->proghdrs[i];
+ if (phdr->p_type != PT_LOAD)
+ continue;
+
+ if (lowest_paddr > phdr->p_paddr)
+ lowest_paddr = phdr->p_paddr;
+
+ if (lowest_vaddr > phdr->p_vaddr)
+ lowest_vaddr = phdr->p_vaddr;
+ }
+
+ kbuf.image = image;
+ kbuf.buf_min = lowest_paddr;
+ kbuf.buf_max = ULONG_MAX;
+
+ /*
+ * Current riscv boot protocol requires 2MB alignment for
+ * RV64 and 4MB alignment for RV32
+ *
+ */
+ kbuf.buf_align = PMD_SIZE;
+ kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+ kbuf.memsz = ALIGN(kernel_len, PAGE_SIZE);
+ kbuf.top_down = false;
+ ret = arch_kexec_locate_mem_hole(&kbuf);
+ if (!ret) {
+ *old_pbase = lowest_paddr;
+ *new_pbase = kbuf.mem;
+ image->start = ehdr->e_entry - lowest_vaddr + kbuf.mem;
+ }
+ return ret;
+}
+
+static void *elf_kexec_load(struct kimage *image, char *kernel_buf,
+ unsigned long kernel_len, char *initrd,
+ unsigned long initrd_len, char *cmdline,
+ unsigned long cmdline_len)
+{
+ int ret;
+ unsigned long old_kernel_pbase = ULONG_MAX;
+ unsigned long new_kernel_pbase = 0UL;
+ struct elfhdr ehdr;
+ struct kexec_elf_info elf_info;
+
+ ret = kexec_build_elf_info(kernel_buf, kernel_len, &ehdr, &elf_info);
+ if (ret)
+ return ERR_PTR(ret);
+
+ ret = elf_find_pbase(image, kernel_len, &ehdr, &elf_info,
+ &old_kernel_pbase, &new_kernel_pbase);
+ if (ret)
+ goto out;
+
+ /* Add the kernel binary to the image */
+ ret = riscv_kexec_elf_load(image, &ehdr, &elf_info,
+ old_kernel_pbase, new_kernel_pbase);
+ if (ret)
+ goto out;
+
+ ret = load_extra_segments(image, image->start, kernel_len,
+ initrd, initrd_len, cmdline, cmdline_len);
+out:
+ kexec_free_elf_info(&elf_info);
+ return ret ? ERR_PTR(ret) : NULL;
+}
+
+const struct kexec_file_ops elf_kexec_ops = {
+ .probe = kexec_elf_probe,
+ .load = elf_kexec_load,
+};
diff --git a/arch/riscv/kernel/machine_kexec_file.c b/arch/riscv/kernel/machine_kexec_file.c
index b0bf8c1722c0..fc506f95aeaa 100644
--- a/arch/riscv/kernel/machine_kexec_file.c
+++ b/arch/riscv/kernel/machine_kexec_file.c
@@ -7,8 +7,376 @@
* Author: Liao Chang (liaochang1@huawei.com)
*/
#include <linux/kexec.h>
+#include <linux/elf.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+#include <linux/libfdt.h>
+#include <linux/types.h>
+#include <linux/memblock.h>
+#include <linux/vmalloc.h>
+#include <asm/setup.h>
const struct kexec_file_ops * const kexec_file_loaders[] = {
&elf_kexec_ops,
NULL
};
+
+int arch_kimage_file_post_load_cleanup(struct kimage *image)
+{
+ kvfree(image->arch.fdt);
+ image->arch.fdt = NULL;
+
+ vfree(image->elf_headers);
+ image->elf_headers = NULL;
+ image->elf_headers_sz = 0;
+
+ return kexec_image_post_load_cleanup_default(image);
+}
+
+#ifdef CONFIG_CRASH_DUMP
+static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
+{
+ unsigned int *nr_ranges = arg;
+
+ (*nr_ranges)++;
+ return 0;
+}
+
+static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
+{
+ struct crash_mem *cmem = arg;
+
+ cmem->ranges[cmem->nr_ranges].start = res->start;
+ cmem->ranges[cmem->nr_ranges].end = res->end;
+ cmem->nr_ranges++;
+
+ return 0;
+}
+
+static int prepare_elf_headers(void **addr, unsigned long *sz)
+{
+ struct crash_mem *cmem;
+ unsigned int nr_ranges;
+ int ret;
+
+ nr_ranges = 2; /* For exclusion of crashkernel region */
+ walk_system_ram_res(0, -1, &nr_ranges, get_nr_ram_ranges_callback);
+
+ cmem = kmalloc(struct_size(cmem, ranges, nr_ranges), GFP_KERNEL);
+ if (!cmem)
+ return -ENOMEM;
+
+ cmem->max_nr_ranges = nr_ranges;
+ cmem->nr_ranges = 0;
+ ret = walk_system_ram_res(0, -1, cmem, prepare_elf64_ram_headers_callback);
+ if (ret)
+ goto out;
+
+ /* Exclude crashkernel region */
+ ret = crash_exclude_mem_range(cmem, crashk_res.start, crashk_res.end);
+ if (ret)
+ goto out;
+
+ if (crashk_low_res.end) {
+ ret = crash_exclude_mem_range(cmem, crashk_low_res.start, crashk_low_res.end);
+ if (ret)
+ goto out;
+ }
+
+ ret = crash_prepare_elf64_headers(cmem, true, addr, sz);
+
+out:
+ kfree(cmem);
+ return ret;
+}
+
+static char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
+ unsigned long cmdline_len)
+{
+ int elfcorehdr_strlen;
+ char *cmdline_ptr;
+
+ cmdline_ptr = kzalloc(COMMAND_LINE_SIZE, GFP_KERNEL);
+ if (!cmdline_ptr)
+ return NULL;
+
+ elfcorehdr_strlen = sprintf(cmdline_ptr, "elfcorehdr=0x%lx ",
+ image->elf_load_addr);
+
+ if (elfcorehdr_strlen + cmdline_len > COMMAND_LINE_SIZE) {
+ pr_err("Appending elfcorehdr=<addr> exceeds cmdline size\n");
+ kfree(cmdline_ptr);
+ return NULL;
+ }
+
+ memcpy(cmdline_ptr + elfcorehdr_strlen, cmdline, cmdline_len);
+ /* Ensure it's nul terminated */
+ cmdline_ptr[COMMAND_LINE_SIZE - 1] = '\0';
+ return cmdline_ptr;
+}
+#endif
+
+#define RV_X(x, s, n) (((x) >> (s)) & ((1 << (n)) - 1))
+#define RISCV_IMM_BITS 12
+#define RISCV_IMM_REACH (1LL << RISCV_IMM_BITS)
+#define RISCV_CONST_HIGH_PART(x) \
+ (((x) + (RISCV_IMM_REACH >> 1)) & ~(RISCV_IMM_REACH - 1))
+#define RISCV_CONST_LOW_PART(x) ((x) - RISCV_CONST_HIGH_PART(x))
+
+#define ENCODE_ITYPE_IMM(x) \
+ (RV_X(x, 0, 12) << 20)
+#define ENCODE_BTYPE_IMM(x) \
+ ((RV_X(x, 1, 4) << 8) | (RV_X(x, 5, 6) << 25) | \
+ (RV_X(x, 11, 1) << 7) | (RV_X(x, 12, 1) << 31))
+#define ENCODE_UTYPE_IMM(x) \
+ (RV_X(x, 12, 20) << 12)
+#define ENCODE_JTYPE_IMM(x) \
+ ((RV_X(x, 1, 10) << 21) | (RV_X(x, 11, 1) << 20) | \
+ (RV_X(x, 12, 8) << 12) | (RV_X(x, 20, 1) << 31))
+#define ENCODE_CBTYPE_IMM(x) \
+ ((RV_X(x, 1, 2) << 3) | (RV_X(x, 3, 2) << 10) | (RV_X(x, 5, 1) << 2) | \
+ (RV_X(x, 6, 2) << 5) | (RV_X(x, 8, 1) << 12))
+#define ENCODE_CJTYPE_IMM(x) \
+ ((RV_X(x, 1, 3) << 3) | (RV_X(x, 4, 1) << 11) | (RV_X(x, 5, 1) << 2) | \
+ (RV_X(x, 6, 1) << 7) | (RV_X(x, 7, 1) << 6) | (RV_X(x, 8, 2) << 9) | \
+ (RV_X(x, 10, 1) << 8) | (RV_X(x, 11, 1) << 12))
+#define ENCODE_UJTYPE_IMM(x) \
+ (ENCODE_UTYPE_IMM(RISCV_CONST_HIGH_PART(x)) | \
+ (ENCODE_ITYPE_IMM(RISCV_CONST_LOW_PART(x)) << 32))
+#define ENCODE_UITYPE_IMM(x) \
+ (ENCODE_UTYPE_IMM(x) | (ENCODE_ITYPE_IMM(x) << 32))
+
+#define CLEAN_IMM(type, x) \
+ ((~ENCODE_##type##_IMM((uint64_t)(-1))) & (x))
+
+int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
+ Elf_Shdr *section,
+ const Elf_Shdr *relsec,
+ const Elf_Shdr *symtab)
+{
+ const char *strtab, *name, *shstrtab;
+ const Elf_Shdr *sechdrs;
+ Elf64_Rela *relas;
+ int i, r_type;
+
+ /* String & section header string table */
+ sechdrs = (void *)pi->ehdr + pi->ehdr->e_shoff;
+ strtab = (char *)pi->ehdr + sechdrs[symtab->sh_link].sh_offset;
+ shstrtab = (char *)pi->ehdr + sechdrs[pi->ehdr->e_shstrndx].sh_offset;
+
+ relas = (void *)pi->ehdr + relsec->sh_offset;
+
+ for (i = 0; i < relsec->sh_size / sizeof(*relas); i++) {
+ const Elf_Sym *sym; /* symbol to relocate */
+ unsigned long addr; /* final location after relocation */
+ unsigned long val; /* relocated symbol value */
+ unsigned long sec_base; /* relocated symbol value */
+ void *loc; /* tmp location to modify */
+
+ sym = (void *)pi->ehdr + symtab->sh_offset;
+ sym += ELF64_R_SYM(relas[i].r_info);
+
+ if (sym->st_name)
+ name = strtab + sym->st_name;
+ else
+ name = shstrtab + sechdrs[sym->st_shndx].sh_name;
+
+ loc = pi->purgatory_buf;
+ loc += section->sh_offset;
+ loc += relas[i].r_offset;
+
+ if (sym->st_shndx == SHN_ABS)
+ sec_base = 0;
+ else if (sym->st_shndx >= pi->ehdr->e_shnum) {
+ pr_err("Invalid section %d for symbol %s\n",
+ sym->st_shndx, name);
+ return -ENOEXEC;
+ } else
+ sec_base = pi->sechdrs[sym->st_shndx].sh_addr;
+
+ val = sym->st_value;
+ val += sec_base;
+ val += relas[i].r_addend;
+
+ addr = section->sh_addr + relas[i].r_offset;
+
+ r_type = ELF64_R_TYPE(relas[i].r_info);
+
+ switch (r_type) {
+ case R_RISCV_BRANCH:
+ *(u32 *)loc = CLEAN_IMM(BTYPE, *(u32 *)loc) |
+ ENCODE_BTYPE_IMM(val - addr);
+ break;
+ case R_RISCV_JAL:
+ *(u32 *)loc = CLEAN_IMM(JTYPE, *(u32 *)loc) |
+ ENCODE_JTYPE_IMM(val - addr);
+ break;
+ /*
+ * With no R_RISCV_PCREL_LO12_S, R_RISCV_PCREL_LO12_I
+ * sym is expected to be next to R_RISCV_PCREL_HI20
+ * in purgatory relsec. Handle it like R_RISCV_CALL
+ * sym, instead of searching the whole relsec.
+ */
+ case R_RISCV_PCREL_HI20:
+ case R_RISCV_CALL_PLT:
+ case R_RISCV_CALL:
+ *(u64 *)loc = CLEAN_IMM(UITYPE, *(u64 *)loc) |
+ ENCODE_UJTYPE_IMM(val - addr);
+ break;
+ case R_RISCV_RVC_BRANCH:
+ *(u32 *)loc = CLEAN_IMM(CBTYPE, *(u32 *)loc) |
+ ENCODE_CBTYPE_IMM(val - addr);
+ break;
+ case R_RISCV_RVC_JUMP:
+ *(u32 *)loc = CLEAN_IMM(CJTYPE, *(u32 *)loc) |
+ ENCODE_CJTYPE_IMM(val - addr);
+ break;
+ case R_RISCV_ADD16:
+ *(u16 *)loc += val;
+ break;
+ case R_RISCV_SUB16:
+ *(u16 *)loc -= val;
+ break;
+ case R_RISCV_ADD32:
+ *(u32 *)loc += val;
+ break;
+ case R_RISCV_SUB32:
+ *(u32 *)loc -= val;
+ break;
+ /* It has been applied by R_RISCV_PCREL_HI20 sym */
+ case R_RISCV_PCREL_LO12_I:
+ case R_RISCV_ALIGN:
+ case R_RISCV_RELAX:
+ break;
+ case R_RISCV_64:
+ *(u64 *)loc = val;
+ break;
+ default:
+ pr_err("Unknown rela relocation: %d\n", r_type);
+ return -ENOEXEC;
+ }
+ }
+ return 0;
+}
+
+
+int load_extra_segments(struct kimage *image, unsigned long kernel_start,
+ unsigned long kernel_len, char *initrd,
+ unsigned long initrd_len, char *cmdline,
+ unsigned long cmdline_len)
+{
+ int ret;
+ void *fdt;
+ unsigned long initrd_pbase = 0UL;
+ struct kexec_buf kbuf;
+ char *modified_cmdline = NULL;
+
+ kbuf.image = image;
+ kbuf.buf_min = kernel_start + kernel_len;
+ kbuf.buf_max = ULONG_MAX;
+
+#ifdef CONFIG_CRASH_DUMP
+ /* Add elfcorehdr */
+ if (image->type == KEXEC_TYPE_CRASH) {
+ void *headers;
+ unsigned long headers_sz;
+ ret = prepare_elf_headers(&headers, &headers_sz);
+ if (ret) {
+ pr_err("Preparing elf core header failed\n");
+ goto out;
+ }
+
+ kbuf.buffer = headers;
+ kbuf.bufsz = headers_sz;
+ kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+ kbuf.memsz = headers_sz;
+ kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
+ kbuf.top_down = true;
+
+ ret = kexec_add_buffer(&kbuf);
+ if (ret) {
+ vfree(headers);
+ goto out;
+ }
+ image->elf_headers = headers;
+ image->elf_load_addr = kbuf.mem;
+ image->elf_headers_sz = headers_sz;
+
+ kexec_dprintk("Loaded elf core header at 0x%lx bufsz=0x%lx memsz=0x%lx\n",
+ image->elf_load_addr, kbuf.bufsz, kbuf.memsz);
+
+ /* Setup cmdline for kdump kernel case */
+ modified_cmdline = setup_kdump_cmdline(image, cmdline,
+ cmdline_len);
+ if (!modified_cmdline) {
+ pr_err("Setting up cmdline for kdump kernel failed\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ cmdline = modified_cmdline;
+ }
+#endif
+
+#ifdef CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY
+ /* Add purgatory to the image */
+ kbuf.top_down = true;
+ kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+ ret = kexec_load_purgatory(image, &kbuf);
+ if (ret) {
+ pr_err("Error loading purgatory ret=%d\n", ret);
+ goto out;
+ }
+ kexec_dprintk("Loaded purgatory at 0x%lx\n", kbuf.mem);
+
+ ret = kexec_purgatory_get_set_symbol(image, "riscv_kernel_entry",
+ &kernel_start,
+ sizeof(kernel_start), 0);
+ if (ret)
+ pr_err("Error update purgatory ret=%d\n", ret);
+#endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */
+
+ /* Add the initrd to the image */
+ if (initrd != NULL) {
+ kbuf.buffer = initrd;
+ kbuf.bufsz = kbuf.memsz = initrd_len;
+ kbuf.buf_align = PAGE_SIZE;
+ kbuf.top_down = true;
+ kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+ ret = kexec_add_buffer(&kbuf);
+ if (ret)
+ goto out;
+ initrd_pbase = kbuf.mem;
+ kexec_dprintk("Loaded initrd at 0x%lx\n", initrd_pbase);
+ }
+
+ /* Add the DTB to the image */
+ fdt = of_kexec_alloc_and_setup_fdt(image, initrd_pbase,
+ initrd_len, cmdline, 0);
+ if (!fdt) {
+ pr_err("Error setting up the new device tree.\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ fdt_pack(fdt);
+ kbuf.buffer = fdt;
+ kbuf.bufsz = kbuf.memsz = fdt_totalsize(fdt);
+ kbuf.buf_align = PAGE_SIZE;
+ kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+ kbuf.top_down = true;
+ ret = kexec_add_buffer(&kbuf);
+ if (ret) {
+ pr_err("Error add DTB kbuf ret=%d\n", ret);
+ goto out_free_fdt;
+ }
+ /* Cache the fdt buffer address for memory cleanup */
+ image->arch.fdt = fdt;
+ kexec_dprintk("Loaded device tree at 0x%lx\n", kbuf.mem);
+ goto out;
+
+out_free_fdt:
+ kvfree(fdt);
+out:
+ kfree(modified_cmdline);
+ return ret;
+}
diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile
index cbe4d775ef56..b916a68d324a 100644
--- a/arch/riscv/mm/Makefile
+++ b/arch/riscv/mm/Makefile
@@ -19,7 +19,7 @@ obj-y += context.o
obj-y += pmem.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
-obj-$(CONFIG_PTDUMP_CORE) += ptdump.o
+obj-$(CONFIG_PTDUMP) += ptdump.o
obj-$(CONFIG_KASAN) += kasan_init.o
ifdef CONFIG_KASAN
diff --git a/arch/riscv/mm/ptdump.c b/arch/riscv/mm/ptdump.c
index 4bb09cadfb85..34299c2b231f 100644
--- a/arch/riscv/mm/ptdump.c
+++ b/arch/riscv/mm/ptdump.c
@@ -317,6 +317,38 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr,
}
}
+static void note_page_pte(struct ptdump_state *pt_st, unsigned long addr, pte_t pte)
+{
+ note_page(pt_st, addr, 4, pte_val(pte));
+}
+
+static void note_page_pmd(struct ptdump_state *pt_st, unsigned long addr, pmd_t pmd)
+{
+ note_page(pt_st, addr, 3, pmd_val(pmd));
+}
+
+static void note_page_pud(struct ptdump_state *pt_st, unsigned long addr, pud_t pud)
+{
+ note_page(pt_st, addr, 2, pud_val(pud));
+}
+
+static void note_page_p4d(struct ptdump_state *pt_st, unsigned long addr, p4d_t p4d)
+{
+ note_page(pt_st, addr, 1, p4d_val(p4d));
+}
+
+static void note_page_pgd(struct ptdump_state *pt_st, unsigned long addr, pgd_t pgd)
+{
+ note_page(pt_st, addr, 0, pgd_val(pgd));
+}
+
+static void note_page_flush(struct ptdump_state *pt_st)
+{
+ pte_t pte_zero = {0};
+
+ note_page(pt_st, 0, -1, pte_val(pte_zero));
+}
+
static void ptdump_walk(struct seq_file *s, struct ptd_mm_info *pinfo)
{
struct pg_state st = {
@@ -324,7 +356,12 @@ static void ptdump_walk(struct seq_file *s, struct ptd_mm_info *pinfo)
.marker = pinfo->markers,
.level = -1,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]) {
{pinfo->base_addr, pinfo->end},
{0, 0}
@@ -346,7 +383,12 @@ bool ptdump_check_wx(void)
.level = -1,
.check_wx = true,
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]) {
{KERN_VIRT_START, ULONG_MAX},
{0, 0}
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index 43a58e5e123f..55c2f85090d4 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1617,9 +1617,10 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
for (idx = 0; idx < fm->nr_args; idx++) {
u8 reg = bpf_to_rv_reg(BPF_REG_1 + idx, ctx);
+ bool sign = fm->arg_flags[idx] & BTF_FMODEL_SIGNED_ARG;
- if (fm->arg_size[idx] == sizeof(int))
- emit_sextw(reg, reg, ctx);
+ if (sign_extend(reg, reg, fm->arg_size[idx], sign, ctx))
+ return -EINVAL;
}
}
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 2f2af89ab4df..43cba6b657b7 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -259,6 +259,7 @@ void bpf_jit_free(struct bpf_prog *prog)
*/
if (jit_data) {
bpf_jit_binary_pack_finalize(jit_data->ro_header, jit_data->header);
+ kfree(jit_data->ctx.offset);
kfree(jit_data);
}
hdr = bpf_jit_binary_pack_hdr(prog);
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 28e900e5212c..cf353f5f8a26 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -57,7 +57,6 @@ config S390
#
# Note: keep this list sorted alphabetically
#
- imply IMA_SECURE_AND_OR_TRUSTED_BOOT
select ALTERNATE_USER_ADDRESS_SPACE
select ARCH_32BIT_USTAT_F_TINODE
select ARCH_BINFMT_ELF_STATE
@@ -81,6 +80,8 @@ config S390
select ARCH_HAS_MEMBARRIER_SYNC_CORE
select ARCH_HAS_MEM_ENCRYPT
select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS
+ select ARCH_HAS_PREEMPT_LAZY
+ select ARCH_HAS_PTDUMP
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SCALED_CPUTIME
select ARCH_HAS_SET_DIRECT_MAP
@@ -146,13 +147,13 @@ config S390
select GENERIC_CPU_VULNERABILITIES
select GENERIC_ENTRY
select GENERIC_GETTIMEOFDAY
- select GENERIC_PTDUMP
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
select GENERIC_VDSO_TIME_NS
select GENERIC_IOREMAP if PCI
select HAVE_ALIGNED_STRUCT_PAGE
select HAVE_ARCH_AUDITSYSCALL
+ select HAVE_ARCH_GET_SECUREBOOT
select HAVE_ARCH_JUMP_LABEL
select HAVE_ARCH_JUMP_LABEL_RELATIVE
select HAVE_ARCH_KASAN
diff --git a/arch/s390/include/asm/irqflags.h b/arch/s390/include/asm/irqflags.h
index bcab456dfb80..6eb6de6a99f2 100644
--- a/arch/s390/include/asm/irqflags.h
+++ b/arch/s390/include/asm/irqflags.h
@@ -37,18 +37,24 @@ static __always_inline void __arch_local_irq_ssm(unsigned long flags)
asm volatile("ssm %0" : : "Q" (flags) : "memory");
}
-#ifdef CONFIG_KMSAN
-#define arch_local_irq_attributes noinline notrace __no_sanitize_memory __maybe_unused
+#if defined(CONFIG_KMSAN) && !defined(__DECOMPRESSOR)
+unsigned long arch_local_save_flags(void);
+unsigned long arch_local_irq_save(void);
+void arch_local_irq_enable_external(void);
+void arch_local_irq_enable(void);
#else
-#define arch_local_irq_attributes __always_inline
+#define arch_local_save_flags __arch_local_save_flags
+#define arch_local_irq_save __arch_local_irq_save
+#define arch_local_irq_enable_external __arch_local_irq_enable_external
+#define arch_local_irq_enable __arch_local_irq_enable
#endif
-static arch_local_irq_attributes unsigned long arch_local_save_flags(void)
+static __always_inline unsigned long __arch_local_save_flags(void)
{
return __arch_local_irq_stnsm(0xff);
}
-static arch_local_irq_attributes unsigned long arch_local_irq_save(void)
+static __always_inline unsigned long __arch_local_irq_save(void)
{
return __arch_local_irq_stnsm(0xfc);
}
@@ -58,12 +64,12 @@ static __always_inline void arch_local_irq_disable(void)
arch_local_irq_save();
}
-static arch_local_irq_attributes void arch_local_irq_enable_external(void)
+static __always_inline void __arch_local_irq_enable_external(void)
{
__arch_local_irq_stosm(0x01);
}
-static arch_local_irq_attributes void arch_local_irq_enable(void)
+static __always_inline void __arch_local_irq_enable(void)
{
__arch_local_irq_stosm(0x03);
}
diff --git a/arch/s390/include/asm/thread_info.h b/arch/s390/include/asm/thread_info.h
index 00ac01874a12..c33f7144d1b9 100644
--- a/arch/s390/include/asm/thread_info.h
+++ b/arch/s390/include/asm/thread_info.h
@@ -61,44 +61,45 @@ void arch_setup_new_exec(void);
/*
* thread information flags bit numbers
*/
-/* _TIF_WORK bits */
#define TIF_NOTIFY_RESUME 0 /* callback before returning to user */
#define TIF_SIGPENDING 1 /* signal pending */
#define TIF_NEED_RESCHED 2 /* rescheduling necessary */
-#define TIF_UPROBE 3 /* breakpointed or single-stepping */
-#define TIF_GUARDED_STORAGE 4 /* load guarded storage control block */
+#define TIF_NEED_RESCHED_LAZY 3 /* lazy rescheduling needed */
+#define TIF_UPROBE 4 /* breakpointed or single-stepping */
#define TIF_PATCH_PENDING 5 /* pending live patching update */
#define TIF_PGSTE 6 /* New mm's will use 4K page tables */
#define TIF_NOTIFY_SIGNAL 7 /* signal notifications exist */
+#define TIF_GUARDED_STORAGE 8 /* load guarded storage control block */
#define TIF_ISOLATE_BP_GUEST 9 /* Run KVM guests with isolated BP */
#define TIF_PER_TRAP 10 /* Need to handle PER trap on exit to usermode */
-
#define TIF_31BIT 16 /* 32bit process */
#define TIF_MEMDIE 17 /* is terminating due to OOM killer */
#define TIF_RESTORE_SIGMASK 18 /* restore signal mask in do_signal() */
#define TIF_SINGLE_STEP 19 /* This task is single stepped */
#define TIF_BLOCK_STEP 20 /* This task is block stepped */
#define TIF_UPROBE_SINGLESTEP 21 /* This task is uprobe single stepped */
-
-/* _TIF_TRACE bits */
#define TIF_SYSCALL_TRACE 24 /* syscall trace active */
#define TIF_SYSCALL_AUDIT 25 /* syscall auditing active */
#define TIF_SECCOMP 26 /* secure computing */
#define TIF_SYSCALL_TRACEPOINT 27 /* syscall tracepoint instrumentation */
#define _TIF_NOTIFY_RESUME BIT(TIF_NOTIFY_RESUME)
-#define _TIF_NOTIFY_SIGNAL BIT(TIF_NOTIFY_SIGNAL)
#define _TIF_SIGPENDING BIT(TIF_SIGPENDING)
#define _TIF_NEED_RESCHED BIT(TIF_NEED_RESCHED)
+#define _TIF_NEED_RESCHED_LAZY BIT(TIF_NEED_RESCHED_LAZY)
#define _TIF_UPROBE BIT(TIF_UPROBE)
-#define _TIF_GUARDED_STORAGE BIT(TIF_GUARDED_STORAGE)
#define _TIF_PATCH_PENDING BIT(TIF_PATCH_PENDING)
+#define _TIF_PGSTE BIT(TIF_PGSTE)
+#define _TIF_NOTIFY_SIGNAL BIT(TIF_NOTIFY_SIGNAL)
+#define _TIF_GUARDED_STORAGE BIT(TIF_GUARDED_STORAGE)
#define _TIF_ISOLATE_BP_GUEST BIT(TIF_ISOLATE_BP_GUEST)
#define _TIF_PER_TRAP BIT(TIF_PER_TRAP)
-
#define _TIF_31BIT BIT(TIF_31BIT)
+#define _TIF_MEMDIE BIT(TIF_MEMDIE)
+#define _TIF_RESTORE_SIGMASK BIT(TIF_RESTORE_SIGMASK)
#define _TIF_SINGLE_STEP BIT(TIF_SINGLE_STEP)
-
+#define _TIF_BLOCK_STEP BIT(TIF_BLOCK_STEP)
+#define _TIF_UPROBE_SINGLESTEP BIT(TIF_UPROBE_SINGLESTEP)
#define _TIF_SYSCALL_TRACE BIT(TIF_SYSCALL_TRACE)
#define _TIF_SYSCALL_AUDIT BIT(TIF_SYSCALL_AUDIT)
#define _TIF_SECCOMP BIT(TIF_SECCOMP)
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index 48caae8c7e10..0a1692a0e85c 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -73,7 +73,7 @@ obj-$(CONFIG_JUMP_LABEL) += jump_label.o
obj-$(CONFIG_KEXEC_FILE) += machine_kexec_file.o kexec_image.o
obj-$(CONFIG_KEXEC_FILE) += kexec_elf.o
obj-$(CONFIG_CERT_STORE) += cert_store.o
-obj-$(CONFIG_IMA_SECURE_AND_OR_TRUSTED_BOOT) += ima_arch.o
+obj-$(CONFIG_KMSAN) += irqflags.o
obj-$(CONFIG_PERF_EVENTS) += perf_event.o
obj-$(CONFIG_PERF_EVENTS) += perf_cpum_cf.o perf_cpum_sf.o
diff --git a/arch/s390/kernel/debug.c b/arch/s390/kernel/debug.c
index 16bb554a07fe..f275fa8b3873 100644
--- a/arch/s390/kernel/debug.c
+++ b/arch/s390/kernel/debug.c
@@ -746,8 +746,8 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
mutex_unlock(&debug_mutex);
}
-/* Remove debugfs entries and remove from internal list. */
-static void _debug_unregister(debug_info_t *id)
+/* Remove debugfs entries. */
+static void _debug_unregister_debugfs(debug_info_t *id)
{
int i;
@@ -757,6 +757,11 @@ static void _debug_unregister(debug_info_t *id)
debugfs_remove(id->debugfs_entries[i]);
}
debugfs_remove(id->debugfs_root_entry);
+}
+
+/* Remove from internal list. */
+static void _debug_unregister(debug_info_t *id)
+{
if (id == debug_area_first)
debug_area_first = id->next;
if (id == debug_area_last)
@@ -782,6 +787,7 @@ void debug_unregister(debug_info_t *id)
mutex_lock(&debug_mutex);
_debug_unregister(id);
mutex_unlock(&debug_mutex);
+ _debug_unregister_debugfs(id);
debug_info_put(id);
}
diff --git a/arch/s390/kernel/ima_arch.c b/arch/s390/kernel/ima_arch.c
deleted file mode 100644
index f3c3e6e1c5d3..000000000000
--- a/arch/s390/kernel/ima_arch.c
+++ /dev/null
@@ -1,14 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include <linux/ima.h>
-#include <asm/boot_data.h>
-
-bool arch_ima_get_secureboot(void)
-{
- return ipl_secure_flag;
-}
-
-const char * const *arch_get_ima_policy(void)
-{
- return NULL;
-}
diff --git a/arch/s390/kernel/ipl.c b/arch/s390/kernel/ipl.c
index f5a56c341501..c262d493a4cb 100644
--- a/arch/s390/kernel/ipl.c
+++ b/arch/s390/kernel/ipl.c
@@ -21,6 +21,7 @@
#include <linux/crash_dump.h>
#include <linux/debug_locks.h>
#include <linux/vmalloc.h>
+#include <linux/secure_boot.h>
#include <asm/asm-extable.h>
#include <asm/diag.h>
#include <asm/ipl.h>
@@ -2369,6 +2370,11 @@ void s390_reset_system(void)
diag_amode31_ops.diag308_reset();
}
+bool arch_get_secureboot(void)
+{
+ return ipl_secure_flag;
+}
+
#ifdef CONFIG_KEXEC_FILE
int ipl_report_add_component(struct ipl_report *report, struct kexec_buf *kbuf,
diff --git a/arch/s390/kernel/irqflags.c b/arch/s390/kernel/irqflags.c
new file mode 100644
index 000000000000..e192f59f8918
--- /dev/null
+++ b/arch/s390/kernel/irqflags.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/export.h>
+#include <asm/irqflags.h>
+
+noinstr unsigned long arch_local_save_flags(void)
+{
+ return __arch_local_save_flags();
+}
+EXPORT_SYMBOL(arch_local_save_flags);
+
+noinstr unsigned long arch_local_irq_save(void)
+{
+ return __arch_local_irq_save();
+}
+EXPORT_SYMBOL(arch_local_irq_save);
+
+noinstr void arch_local_irq_enable_external(void)
+{
+ __arch_local_irq_enable_external();
+}
+EXPORT_SYMBOL(arch_local_irq_enable_external);
+
+noinstr void arch_local_irq_enable(void)
+{
+ __arch_local_irq_enable();
+}
+EXPORT_SYMBOL(arch_local_irq_enable);
diff --git a/arch/s390/kvm/guestdbg.c b/arch/s390/kvm/guestdbg.c
index 80879fc73c90..5644d0680b77 100644
--- a/arch/s390/kvm/guestdbg.c
+++ b/arch/s390/kvm/guestdbg.c
@@ -184,7 +184,7 @@ static int __import_wp_info(struct kvm_vcpu *vcpu,
if (wp_info->len < 0 || wp_info->len > MAX_WP_SIZE)
return -EINVAL;
- wp_info->old_data = kmalloc(bp_data->len, GFP_KERNEL_ACCOUNT);
+ wp_info->old_data = kmalloc(wp_info->len, GFP_KERNEL_ACCOUNT);
if (!wp_info->old_data)
return -ENOMEM;
/* try to backup the original value */
@@ -256,7 +256,7 @@ int kvm_s390_import_bp_data(struct kvm_vcpu *vcpu,
ret = __import_wp_info(vcpu, &bp_data[i],
&wp_info[nr_wp]);
if (ret)
- goto error;
+ goto error_wp;
nr_wp++;
break;
case KVM_HW_BP:
@@ -271,7 +271,12 @@ int kvm_s390_import_bp_data(struct kvm_vcpu *vcpu,
vcpu->arch.guestdbg.hw_bp_info = bp_info;
vcpu->arch.guestdbg.nr_hw_wp = nr_wp;
vcpu->arch.guestdbg.hw_wp_info = wp_info;
+ kfree(bp_data);
return 0;
+
+error_wp:
+ while (nr_wp--)
+ kfree(wp_info[nr_wp].old_data);
error:
kfree(bp_data);
kfree(wp_info);
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 6f1314490641..3fc07da7fd7a 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -2863,7 +2863,7 @@ void kvm_s390_reinject_machine_check(struct kvm_vcpu *vcpu,
struct mcck_volatile_info *mcck_info)
{
struct kvm_s390_interrupt_info inti;
- struct kvm_s390_irq irq;
+ struct kvm_s390_irq irq = {};
struct kvm_s390_mchk_info *mchk;
union mci mci;
__u64 cr14 = 0; /* upper bits are not used */
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 70d7bbd62120..f2e2e9385d72 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -3440,6 +3440,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
trace_kvm_s390_destroy_vcpu(vcpu->vcpu_id);
kvm_s390_clear_local_irqs(vcpu);
kvm_clear_async_pf_completion_queue(vcpu);
+ kvm_s390_clear_bp_data(vcpu);
if (!kvm_is_ucontrol(vcpu->kvm))
sca_del_vcpu(vcpu);
kvm_s390_update_topology_change_report(vcpu->kvm, 1);
@@ -4392,8 +4393,10 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
/* enforce guest PER */
kvm_s390_set_cpuflags(vcpu, CPUSTAT_P);
- if (dbg->control & KVM_GUESTDBG_USE_HW_BP)
- rc = kvm_s390_import_bp_data(vcpu, dbg);
+ if (dbg->control & KVM_GUESTDBG_USE_HW_BP) {
+ scoped_guard(srcu, &vcpu->kvm->srcu)
+ rc = kvm_s390_import_bp_data(vcpu, dbg);
+ }
} else {
kvm_s390_clear_cpuflags(vcpu, CPUSTAT_P);
vcpu->arch.guestdbg.last_bp = 0;
@@ -5064,7 +5067,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
pr_err_ratelimited("can't run stopped vcpu %d\n",
vcpu->vcpu_id);
rc = -EINVAL;
- goto out;
+ goto out_sigset;
}
kernel_fpu_begin(&fpu, KERNEL_FPC | KERNEL_VXR);
@@ -5093,9 +5096,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
store_regs(vcpu);
kernel_fpu_end(&fpu, KERNEL_FPC | KERNEL_VXR);
+ vcpu->stat.exit_userspace++;
+
+out_sigset:
kvm_sigset_deactivate(vcpu);
- vcpu->stat.exit_userspace++;
out:
vcpu_put(vcpu);
return rc;
@@ -5715,7 +5720,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
r = kvm_s390_handle_pv_vcpu_dump(vcpu, &cmd);
/* Always copy over UV rc / rrc data */
- if (copy_to_user((__u8 __user *)argp, &cmd.rc,
+ if (copy_to_user(argp + offsetof(struct kvm_pv_cmd, rc), &cmd.rc,
sizeof(cmd.rc) + sizeof(cmd.rrc)))
r = -EFAULT;
break;
diff --git a/arch/s390/mm/Makefile b/arch/s390/mm/Makefile
index f6c2db7a8669..9726b91fe7e4 100644
--- a/arch/s390/mm/Makefile
+++ b/arch/s390/mm/Makefile
@@ -9,6 +9,6 @@ obj-y += page-states.o pageattr.o pgtable.o pgalloc.o extable.o
obj-$(CONFIG_CMM) += cmm.o
obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
-obj-$(CONFIG_PTDUMP_CORE) += dump_pagetables.o
+obj-$(CONFIG_PTDUMP) += dump_pagetables.o
obj-$(CONFIG_PGSTE) += gmap.o
obj-$(CONFIG_PFAULT) += pfault.o
diff --git a/arch/s390/mm/dump_pagetables.c b/arch/s390/mm/dump_pagetables.c
index 965af3dabade..70872ea07201 100644
--- a/arch/s390/mm/dump_pagetables.c
+++ b/arch/s390/mm/dump_pagetables.c
@@ -145,11 +145,48 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level,
}
}
+static void note_page_pte(struct ptdump_state *pt_st, unsigned long addr, pte_t pte)
+{
+ note_page(pt_st, addr, 4, pte_val(pte));
+}
+
+static void note_page_pmd(struct ptdump_state *pt_st, unsigned long addr, pmd_t pmd)
+{
+ note_page(pt_st, addr, 3, pmd_val(pmd));
+}
+
+static void note_page_pud(struct ptdump_state *pt_st, unsigned long addr, pud_t pud)
+{
+ note_page(pt_st, addr, 2, pud_val(pud));
+}
+
+static void note_page_p4d(struct ptdump_state *pt_st, unsigned long addr, p4d_t p4d)
+{
+ note_page(pt_st, addr, 1, p4d_val(p4d));
+}
+
+static void note_page_pgd(struct ptdump_state *pt_st, unsigned long addr, pgd_t pgd)
+{
+ note_page(pt_st, addr, 0, pgd_val(pgd));
+}
+
+static void note_page_flush(struct ptdump_state *pt_st)
+{
+ pte_t pte_zero = {0};
+
+ note_page(pt_st, 0, -1, pte_val(pte_zero));
+}
+
bool ptdump_check_wx(void)
{
struct pg_state st = {
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]) {
{.start = 0, .end = max_addr},
{.start = 0, .end = 0},
@@ -188,7 +225,12 @@ static int ptdump_show(struct seq_file *m, void *v)
{
struct pg_state st = {
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.range = (struct ptdump_range[]) {
{.start = 0, .end = max_addr},
{.start = 0, .end = 0},
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
index c3ad3cf86ca6..55419813c570 100644
--- a/arch/s390/net/bpf_jit_comp.c
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -782,6 +782,8 @@ static void bpf_jit_probe_atomic_pre(struct bpf_jit *jit,
struct bpf_insn *insn,
struct bpf_jit_probe *probe)
{
+ int load_reg;
+
if (BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)
return;
@@ -791,6 +793,14 @@ static void bpf_jit_probe_atomic_pre(struct bpf_jit *jit,
EMIT4(0xb9080000, REG_W1, insn->dst_reg);
probe->arena_reg = REG_W1;
probe->prg = jit->prg;
+ /*
+ * A read-modify-write carrying BPF_FETCH reads the old value into
+ * src_reg, or into r0 for a BPF_CMPXCHG. Clear that register on
+ * fault, the remaining atomics only write memory.
+ */
+ load_reg = bpf_atomic_load_reg(insn);
+ if (load_reg >= 0)
+ probe->reg = reg2hex[load_reg];
}
static int bpf_jit_probe_post(struct bpf_jit *jit, struct bpf_prog *fp,
@@ -1653,6 +1663,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
if (load_probe.prg != -1) {
probe.prg = jit->prg;
probe.arena_reg = load_probe.arena_reg;
+ probe.reg = load_probe.reg;
}
loop_start = jit->prg;
/* 0: {csy|csg} %w0,%src,off(%arena) */
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index d1c73d5ed32f..f717d9c93aa2 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -93,6 +93,7 @@ config X86
select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS
select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
select ARCH_HAS_PMEM_API if X86_64
+ select ARCH_HAS_PTDUMP
select ARCH_HAS_PTE_DEVMAP if X86_64
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_HW_PTE_YOUNG
@@ -169,7 +170,6 @@ config X86
select GENERIC_IRQ_RESERVATION_MODE
select GENERIC_IRQ_SHOW
select GENERIC_PENDING_IRQ if SMP
- select GENERIC_PTDUMP
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
index 74777a97e394..47a2b1f00f54 100644
--- a/arch/x86/Kconfig.debug
+++ b/arch/x86/Kconfig.debug
@@ -59,7 +59,7 @@ config EARLY_PRINTK_USB_XDBC
config EFI_PGT_DUMP
bool "Dump the EFI pagetable"
depends on EFI
- select PTDUMP_CORE
+ select PTDUMP
help
Enable this if you want to dump the EFI page table before
enabling virtual mode. This can be used to debug miscellaneous
diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S
index 0ca9db540b16..3c1bf47af56c 100644
--- a/arch/x86/entry/entry_64_fred.S
+++ b/arch/x86/entry/entry_64_fred.S
@@ -6,6 +6,7 @@
#include <linux/export.h>
#include <asm/asm.h>
+#include <asm/frame.h>
#include <asm/fred.h>
#include <asm/segment.h>
@@ -18,6 +19,7 @@
UNWIND_HINT_END_OF_STACK
ANNOTATE_NOENDBR
PUSH_AND_CLEAR_REGS
+ ENCODE_FRAME_POINTER
movq %rsp, %rdi /* %rdi -> pt_regs */
.endm
diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c
index cad240e88b4f..06c5e485fd6e 100644
--- a/arch/x86/events/amd/uncore.c
+++ b/arch/x86/events/amd/uncore.c
@@ -206,6 +206,29 @@ static void amd_uncore_del(struct perf_event *event, int flags)
hwc->idx = -1;
}
+static bool amd_uncore_group_valid(struct perf_event *event)
+{
+ struct amd_uncore_pmu *pmu = event_to_amd_uncore_pmu(event);
+ struct perf_event *leader = event->group_leader;
+ struct perf_event *sibling;
+ int counters = 0;
+
+ if (leader->pmu == event->pmu)
+ counters++;
+
+ for_each_sibling_event(sibling, leader) {
+ if (sibling->pmu == event->pmu &&
+ sibling->state > PERF_EVENT_STATE_OFF)
+ counters++;
+ }
+
+ /*
+ * When pmu->event_init() is called, the event is yet to be linked to
+ * its leader's sibling list, so it is counted separately
+ */
+ return (counters + 1) <= pmu->num_counters;
+}
+
static int amd_uncore_event_init(struct perf_event *event)
{
struct amd_uncore_pmu *pmu;
@@ -223,6 +246,14 @@ static int amd_uncore_event_init(struct perf_event *event)
if (!ctx)
return -ENODEV;
+ /*
+ * Ensure that all events in a group can be scheduled together so that
+ * a failure can be reported at perf_event_open() time rather than
+ * silently at pmu->add() time when no free counter is found
+ */
+ if (event->group_leader != event && !amd_uncore_group_valid(event))
+ return -EINVAL;
+
/*
* NB and Last level cache counters (MSRs) are shared across all cores
* that share the same NB / Last level cache. On family 16h and below,
diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c
index dc641b50814e..1ef937577082 100644
--- a/arch/x86/events/intel/lbr.c
+++ b/arch/x86/events/intel/lbr.c
@@ -1202,7 +1202,7 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
{
u64 from, to;
int br_sel = cpuc->br_sel;
- int i, j, type, to_plm;
+ int i, j, type, from_plm, to_plm;
bool compress = false;
/* if sampling all branches, then nothing to filter */
@@ -1234,8 +1234,14 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc)
type |= X86_BR_NO_TX;
}
- /* if type does not correspond, then discard */
- if (type == X86_BR_NONE || (br_sel & type) != type) {
+ from_plm = kernel_ip(from) ? X86_BR_KERNEL : X86_BR_USER;
+ /*
+ * If type does not correspond, then discard.
+ * Specifically reject entries whose from address is in
+ * kernel space when only X86_BR_USER is requested.
+ */
+ if (type == X86_BR_NONE || (br_sel & type) != type ||
+ (!(br_sel & X86_BR_KERNEL) && (from_plm & X86_BR_KERNEL))) {
cpuc->lbr_entries[i].from = 0;
compress = true;
}
diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
index a087bc0c5498..86f0b17446e6 100644
--- a/arch/x86/events/intel/pt.c
+++ b/arch/x86/events/intel/pt.c
@@ -418,6 +418,9 @@ static void pt_config_start(struct perf_event *event)
struct pt *pt = this_cpu_ptr(&pt_ctx);
u64 ctl = event->hw.aux_config;
+ if (READ_ONCE(event->hw.aux_paused))
+ return;
+
ctl |= RTIT_CTL_TRACEEN;
if (READ_ONCE(pt->vmx_on))
perf_aux_output_flag(&pt->handle, PERF_AUX_FLAG_PARTIAL);
@@ -496,6 +499,29 @@ static u64 pt_config_filters(struct perf_event *event)
return rtit_ctl;
}
+static void pt_config_enable(struct perf_event *event)
+{
+ struct pt *pt = this_cpu_ptr(&pt_ctx);
+
+ /*
+ * Allow resume before starting so as not to overwrite a value set by a
+ * PMI.
+ */
+ barrier();
+ WRITE_ONCE(pt->resume_allowed, 1);
+ /* Configuration is complete, it is now OK to handle an NMI */
+ barrier();
+ WRITE_ONCE(pt->handle_nmi, 1);
+ barrier();
+ pt_config_start(event);
+ barrier();
+ /*
+ * Allow pause after starting so its pt_config_stop() doesn't race with
+ * pt_config_start().
+ */
+ WRITE_ONCE(pt->pause_allowed, 1);
+}
+
static void pt_config(struct perf_event *event)
{
struct pt *pt = this_cpu_ptr(&pt_ctx);
@@ -534,7 +560,8 @@ static void pt_config(struct perf_event *event)
reg |= (event->attr.config & PT_CONFIG_MASK);
event->hw.aux_config = reg;
- pt_config_start(event);
+
+ pt_config_enable(event);
}
static void pt_config_stop(struct perf_event *event)
@@ -1510,12 +1537,15 @@ void intel_pt_interrupt(void)
perf_aux_output_end(&pt->handle, local_xchg(&buf->data_size, 0));
- if (!event->hw.state) {
+ event->hw.state |= PERF_HES_UPTODATE;
+
+ if (!(event->hw.state & PERF_HES_STOPPED)) {
int ret;
buf = perf_aux_output_begin(&pt->handle, event);
if (!buf) {
- event->hw.state = PERF_HES_STOPPED;
+ event->hw.state |= PERF_HES_STOPPED;
+ WRITE_ONCE(pt->resume_allowed, 0);
return;
}
@@ -1524,11 +1554,14 @@ void intel_pt_interrupt(void)
ret = pt_buffer_reset_markers(buf, &pt->handle);
if (ret) {
perf_aux_output_end(&pt->handle, 0);
+ WRITE_ONCE(pt->resume_allowed, 0);
return;
}
pt_config_buffer(buf);
pt_config_start(event);
+
+ event->hw.state &= ~PERF_HES_UPTODATE;
}
}
@@ -1578,6 +1611,38 @@ static void pt_event_start(struct perf_event *event, int mode)
struct pt *pt = this_cpu_ptr(&pt_ctx);
struct pt_buffer *buf;
+ if (mode & PERF_EF_RESUME) {
+ if (READ_ONCE(pt->resume_allowed)) {
+ u64 status;
+
+ /*
+ * Only if the trace is not active and the error and
+ * stopped bits are clear, is it safe to start, but a
+ * PMI might have just cleared these, so resume_allowed
+ * must be checked again also.
+ */
+ rdmsrl(MSR_IA32_RTIT_STATUS, status);
+ if (!(status & (RTIT_STATUS_TRIGGEREN |
+ RTIT_STATUS_ERROR |
+ RTIT_STATUS_STOPPED)) &&
+ READ_ONCE(pt->resume_allowed))
+ pt_config_start(event);
+ }
+ return;
+ }
+
+ /*
+ * Re-start subsequent to a call to pt_event_stop() without the
+ * PERF_EF_UPDATE flag. Absence of PERF_HES_UPTODATE indicates that
+ * perf_aux_output_begin() has already been called. This path can
+ * come about only in snapshot/overwrite mode - see pt_event_stop().
+ */
+ if (!(hwc->state & PERF_HES_UPTODATE)) {
+ hwc->state &= ~PERF_HES_STOPPED;
+ pt_config_enable(event);
+ return;
+ }
+
buf = perf_aux_output_begin(&pt->handle, event);
if (!buf)
goto fail_stop;
@@ -1588,8 +1653,7 @@ static void pt_event_start(struct perf_event *event, int mode)
goto fail_end_stop;
}
- WRITE_ONCE(pt->handle_nmi, 1);
- hwc->state = 0;
+ hwc->state &= ~(PERF_HES_STOPPED | PERF_HES_UPTODATE);
pt_config_buffer(buf);
pt_config(event);
@@ -1599,12 +1663,19 @@ static void pt_event_start(struct perf_event *event, int mode)
fail_end_stop:
perf_aux_output_end(&pt->handle, 0);
fail_stop:
- hwc->state = PERF_HES_STOPPED;
+ hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
}
static void pt_event_stop(struct perf_event *event, int mode)
{
struct pt *pt = this_cpu_ptr(&pt_ctx);
+ struct pt_buffer *buf;
+
+ if (mode & PERF_EF_PAUSE) {
+ if (READ_ONCE(pt->pause_allowed))
+ pt_config_stop(event);
+ return;
+ }
/*
* Protect against the PMI racing with disabling wrmsr,
@@ -1613,19 +1684,35 @@ static void pt_event_stop(struct perf_event *event, int mode)
WRITE_ONCE(pt->handle_nmi, 0);
barrier();
- pt_config_stop(event);
+ /*
+ * Prevent a resume from attempting to restart tracing, or a pause
+ * during a subsequent start. Do this after clearing handle_nmi so that
+ * pt_event_snapshot_aux() will not re-allow them.
+ */
+ WRITE_ONCE(pt->pause_allowed, 0);
+ WRITE_ONCE(pt->resume_allowed, 0);
+ barrier();
- if (event->hw.state == PERF_HES_STOPPED)
- return;
+ pt_config_stop(event);
- event->hw.state = PERF_HES_STOPPED;
+ event->hw.state |= PERF_HES_STOPPED;
- if (mode & PERF_EF_UPDATE) {
- struct pt_buffer *buf = perf_get_aux(&pt->handle);
+ if (event->hw.state & PERF_HES_UPTODATE)
+ return;
- if (!buf)
- return;
+ buf = perf_get_aux(&pt->handle);
+ if (!buf)
+ return;
+ /*
+ * When not in snapshot/overwrite mode, there is a possibility that the
+ * buffer has run out of space. The accounting for that is handled by
+ * the update, so always update in that case. Snapshot/overwrite mode is
+ * treated differently to allow for pt_event_snapshot_aux() which can
+ * still get called if the AUX-sampling event is not stopped until after
+ * PT is stopped.
+ */
+ if ((mode & PERF_EF_UPDATE) || !buf->snapshot) {
if (WARN_ON_ONCE(pt->handle.event != event))
return;
@@ -1640,6 +1727,7 @@ static void pt_event_stop(struct perf_event *event, int mode)
local_xchg(&buf->data_size,
buf->nr_pages << PAGE_SHIFT);
perf_aux_output_end(&pt->handle, local_xchg(&buf->data_size, 0));
+ event->hw.state |= PERF_HES_UPTODATE;
}
}
@@ -1662,6 +1750,10 @@ static long pt_event_snapshot_aux(struct perf_event *event,
if (WARN_ON_ONCE(!buf->snapshot))
return 0;
+ /* Prevent pause/resume from attempting to start/stop tracing */
+ WRITE_ONCE(pt->pause_allowed, 0);
+ WRITE_ONCE(pt->resume_allowed, 0);
+ barrier();
/*
* There is no PT interrupt in this mode, so stop the trace and it will
* remain stopped while the buffer is copied.
@@ -1681,8 +1773,13 @@ static long pt_event_snapshot_aux(struct perf_event *event,
* Here, handle_nmi tells us if the tracing was on.
* If the tracing was on, restart it.
*/
- if (READ_ONCE(pt->handle_nmi))
+ if (READ_ONCE(pt->handle_nmi)) {
+ WRITE_ONCE(pt->resume_allowed, 1);
+ barrier();
pt_config_start(event);
+ barrier();
+ WRITE_ONCE(pt->pause_allowed, 1);
+ }
return ret;
}
@@ -1701,13 +1798,15 @@ static int pt_event_add(struct perf_event *event, int mode)
if (pt->handle.event)
goto fail;
+ event->hw.state |= PERF_HES_UPTODATE;
+
if (mode & PERF_EF_START) {
pt_event_start(event, 0);
ret = -EINVAL;
- if (hwc->state == PERF_HES_STOPPED)
+ if (hwc->state & PERF_HES_STOPPED)
goto fail;
} else {
- hwc->state = PERF_HES_STOPPED;
+ hwc->state |= PERF_HES_STOPPED;
}
ret = 0;
@@ -1798,7 +1897,9 @@ static __init int pt_init(void)
if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries))
pt_pmu.pmu.capabilities = PERF_PMU_CAP_AUX_NO_SG;
- pt_pmu.pmu.capabilities |= PERF_PMU_CAP_EXCLUSIVE | PERF_PMU_CAP_ITRACE;
+ pt_pmu.pmu.capabilities |= PERF_PMU_CAP_EXCLUSIVE |
+ PERF_PMU_CAP_ITRACE |
+ PERF_PMU_CAP_AUX_PAUSE;
pt_pmu.pmu.attr_groups = pt_attr_groups;
pt_pmu.pmu.task_ctx_nr = perf_sw_context;
pt_pmu.pmu.event_init = pt_event_init;
diff --git a/arch/x86/events/intel/pt.h b/arch/x86/events/intel/pt.h
index a1b6c04b7f68..7ee94fc6d7cb 100644
--- a/arch/x86/events/intel/pt.h
+++ b/arch/x86/events/intel/pt.h
@@ -119,6 +119,8 @@ struct pt_filters {
* @filters: last configured filters
* @handle_nmi: do handle PT PMI on this cpu, there's an active event
* @vmx_on: 1 if VMX is ON on this cpu
+ * @pause_allowed: PERF_EF_PAUSE is allowed to stop tracing
+ * @resume_allowed: PERF_EF_RESUME is allowed to start tracing
* @output_base: cached RTIT_OUTPUT_BASE MSR value
* @output_mask: cached RTIT_OUTPUT_MASK MSR value
*/
@@ -127,6 +129,8 @@ struct pt {
struct pt_filters filters;
int handle_nmi;
int vmx_on;
+ int pause_allowed;
+ int resume_allowed;
u64 output_base;
u64 output_mask;
};
diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c
index 96607174a0a9..2736815e651c 100644
--- a/arch/x86/events/intel/uncore.c
+++ b/arch/x86/events/intel/uncore.c
@@ -1629,8 +1629,6 @@ static int uncore_event_cpu_online(unsigned int cpu)
die = topology_logical_die_id(cpu);
msr_ret = uncore_box_ref(uncore_msr_uncores, die, cpu);
mmio_ret = uncore_box_ref(uncore_mmio_uncores, die, cpu);
- if (msr_ret && mmio_ret)
- return -ENOMEM;
/*
* Check if there is an online cpu in the package
diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index 69920800da36..222410af815c 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -398,9 +398,9 @@ extern int __init efi_memmap_split_count(efi_memory_desc_t *md,
extern void __init efi_memmap_insert(struct efi_memory_map *old_memmap,
void *buf, struct efi_mem_range *mem);
-extern enum efi_secureboot_mode __x86_ima_efi_boot_mode(void);
+enum efi_secureboot_mode __x86_efi_boot_mode(void);
-#define arch_ima_efi_boot_mode __x86_ima_efi_boot_mode()
+#define arch_efi_boot_mode __x86_efi_boot_mode()
#ifdef CONFIG_EFI_RUNTIME_MAP
int efi_get_runtime_map_size(void);
diff --git a/arch/x86/include/asm/linkage.h b/arch/x86/include/asm/linkage.h
index dc31b13b87a0..e9daba896111 100644
--- a/arch/x86/include/asm/linkage.h
+++ b/arch/x86/include/asm/linkage.h
@@ -103,7 +103,7 @@
.byte 0xb8 ASM_NL \
.long __kcfi_typeid_##name ASM_NL \
CFI_POST_PADDING \
- SYM_FUNC_END(__cfi_##name)
+ SYM_END(__cfi_##name, SYM_T_FUNC)
/* UML needs to be able to override memcpy() and friends for KASAN. */
#ifdef CONFIG_UML
diff --git a/arch/x86/include/asm/pkeys.h b/arch/x86/include/asm/pkeys.h
index 2e6c04d8a45b..32d327609cc4 100644
--- a/arch/x86/include/asm/pkeys.h
+++ b/arch/x86/include/asm/pkeys.h
@@ -89,6 +89,9 @@ int mm_pkey_alloc(struct mm_struct *mm)
u16 all_pkeys_mask = ((1U << arch_max_pkey()) - 1);
int ret;
+ if (!arch_pkeys_enabled())
+ return -1;
+
/*
* Are we out of pkeys? We must handle this specially
* because ffz() behavior is undefined if there are no
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index f85c51241b3d..40660da6b773 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -622,6 +622,18 @@ static enum hrtimer_restart stimer_timer_callback(struct hrtimer *timer)
return HRTIMER_NORESTART;
}
+/*
+ * Translate a stimer expiry given in 100ns reference ticks into an
+ * an absolute deadline. Saturates on overflow.
+ */
+static ktime_t stimer_add_delta(ktime_t now, u64 delta_100ns)
+{
+ if (delta_100ns >= KTIME_MAX / 100)
+ return KTIME_MAX;
+
+ return ktime_add_safe(now, 100 * delta_100ns);
+}
+
/*
* stimer_start() assumptions:
* a) stimer->count is not equal to 0
@@ -631,6 +643,7 @@ static int stimer_start(struct kvm_vcpu_hv_stimer *stimer)
{
u64 time_now;
ktime_t ktime_now;
+ ktime_t deadline;
time_now = get_time_ref_counter(hv_stimer_to_vcpu(stimer)->kvm);
ktime_now = ktime_get();
@@ -653,10 +666,8 @@ static int stimer_start(struct kvm_vcpu_hv_stimer *stimer)
stimer->index,
time_now, stimer->exp_time);
- hrtimer_start(&stimer->timer,
- ktime_add_ns(ktime_now,
- 100 * (stimer->exp_time - time_now)),
- HRTIMER_MODE_ABS);
+ deadline = stimer_add_delta(ktime_now, stimer->exp_time - time_now);
+ hrtimer_start(&stimer->timer, deadline, HRTIMER_MODE_ABS);
return 0;
}
stimer->exp_time = stimer->count;
@@ -675,9 +686,9 @@ static int stimer_start(struct kvm_vcpu_hv_stimer *stimer)
stimer->index,
time_now, stimer->count);
- hrtimer_start(&stimer->timer,
- ktime_add_ns(ktime_now, 100 * (stimer->count - time_now)),
- HRTIMER_MODE_ABS);
+ deadline = stimer_add_delta(ktime_now, stimer->count - time_now);
+ hrtimer_start(&stimer->timer, deadline, HRTIMER_MODE_ABS);
+
return 0;
}
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index b5a5e5c3cc5a..ed0cc781f1c7 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -7107,23 +7107,6 @@ static void kvm_mmu_zap_memslot_pages_and_flush(struct kvm *kvm,
kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush);
}
-static void kvm_mmu_zap_memslot(struct kvm *kvm,
- struct kvm_memory_slot *slot)
-{
- struct kvm_gfn_range range = {
- .slot = slot,
- .start = slot->base_gfn,
- .end = slot->base_gfn + slot->npages,
- .may_block = true,
- };
- bool flush;
-
- write_lock(&kvm->mmu_lock);
- flush = kvm_unmap_gfn_range(kvm, &range);
- kvm_mmu_zap_memslot_pages_and_flush(kvm, slot, flush);
- write_unlock(&kvm->mmu_lock);
-}
-
static inline bool kvm_memslot_flush_zap_all(struct kvm *kvm)
{
return kvm->arch.vm_type == KVM_X86_DEFAULT_VM &&
@@ -7133,10 +7116,22 @@ static inline bool kvm_memslot_flush_zap_all(struct kvm *kvm)
void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
struct kvm_memory_slot *slot)
{
- if (kvm_memslot_flush_zap_all(kvm))
+ struct kvm_gfn_range range = {
+ .slot = slot,
+ .start = slot->base_gfn,
+ .end = slot->base_gfn + slot->npages,
+ .may_block = true,
+ };
+ bool flush;
+
+ if (kvm_memslot_flush_zap_all(kvm)) {
kvm_mmu_zap_all_fast(kvm);
- else
- kvm_mmu_zap_memslot(kvm, slot);
+ } else {
+ write_lock(&kvm->mmu_lock);
+ flush = kvm_unmap_gfn_range(kvm, &range);
+ kvm_mmu_zap_memslot_pages_and_flush(kvm, slot, flush);
+ write_unlock(&kvm->mmu_lock);
+ }
}
void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen)
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 40078d5e6b29..5a5da0009bda 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1235,6 +1235,9 @@ static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu,
* is the VPID incorporated into the MMU context. I.e. KVM must assume
* that the new vpid12 has never been used and thus represents a new
* guest ASID that cannot have entries in the TLB.
+ *
+ * Note, last_vpid is initialized as 0, so the first nested VM-Enter
+ * after VMXON will always flush the TLB to avoid using stale entries.
*/
if (is_vmenter && vmcs12->virtual_processor_id != vmx->nested.last_vpid) {
vmx->nested.last_vpid = vmcs12->virtual_processor_id;
@@ -3667,6 +3670,14 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
vmentry_fail_vmexit_guest_mode:
if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING)
vcpu->arch.tsc_offset -= vmcs12->tsc_offset;
+
+ /*
+ * Handle any TLB flush requests that were queued for L2 if KVM made it
+ * far enough along to switch to L2 context. Note, loading host state
+ * will generate any flushes for L1 required by VM-Exit.
+ */
+ kvm_service_local_tlb_flush_requests(vcpu);
+
leave_guest_mode(vcpu);
vmentry_fail_vmexit:
@@ -4959,8 +4970,9 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
/* trying to cancel vmlaunch/vmresume is a bug */
WARN_ON_ONCE(vmx->nested.nested_run_pending);
-#ifdef CONFIG_KVM_HYPERV
+ /* Note, "checking" the request also clears the request. */
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu)) {
+#ifdef CONFIG_KVM_HYPERV
/*
* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map
* Enlightened VMCS after migration and we still need to
@@ -4968,8 +4980,8 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
* the first L2 run.
*/
(void)nested_get_evmcs_page(vcpu);
- }
#endif
+ }
/* Service pending TLB flush requests for L2 before switching to L1. */
kvm_service_local_tlb_flush_requests(vcpu);
@@ -5343,6 +5355,13 @@ static int enter_vmx_operation(struct kvm_vcpu *vcpu)
vmx->nested.vpid02 = allocate_vpid();
+ /*
+ * Clear last_vpid to ensure that the VPID is flushed on the first
+ * nested VM-Enter. Otherwise, stale TLB entries from a previous life of
+ * the VPID (e.g. different vCPU or even different VM) could be used.
+ */
+ vmx->nested.last_vpid = 0;
+
vmx->nested.vmcs02_initialized = false;
vmx->nested.vmxon = true;
@@ -5970,8 +5989,8 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
u64 vpid;
u64 gla;
} operand;
- u16 vpid02;
int r, gpr_index;
+ int cpu;
if (!(vmx->nested.msrs.secondary_ctls_high &
SECONDARY_EXEC_ENABLE_VPID) ||
@@ -6005,42 +6024,34 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
return kvm_handle_memory_failure(vcpu, r, &e);
if (operand.vpid >> 16)
- return nested_vmx_fail(vcpu,
- VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+ return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+
+ if (type != VMX_VPID_EXTENT_ALL_CONTEXT && !operand.vpid)
+ return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
+
+ /* LAM doesn't apply to addresses that are inputs to TLB invalidation. */
+ if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR &&
+ is_noncanonical_invlpg_address(operand.gla, vcpu))
+ return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
/*
* Always flush the effective vpid02, i.e. never flush the current VPID
* and never explicitly flush vpid01. INVVPID targets a VPID, not a
* VMCS, and so whether or not the current vmcs12 has VPID enabled is
* irrelevant (and there may not be a loaded vmcs12).
+ *
+ * If vmcs02 was last loaded on a different pCPU, then defer the flush
+ * by invalidating the nested VPID tracking to ensure that KVM performs
+ * the invalidation on the correct pCPU.
*/
- vpid02 = nested_get_vpid02(vcpu);
- switch (type) {
- case VMX_VPID_EXTENT_INDIVIDUAL_ADDR:
- /*
- * LAM doesn't apply to addresses that are inputs to TLB
- * invalidation.
- */
- if (!operand.vpid ||
- is_noncanonical_invlpg_address(operand.gla, vcpu))
- return nested_vmx_fail(vcpu,
- VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
- vpid_sync_vcpu_addr(vpid02, operand.gla);
- break;
- case VMX_VPID_EXTENT_SINGLE_CONTEXT:
- case VMX_VPID_EXTENT_SINGLE_NON_GLOBAL:
- if (!operand.vpid)
- return nested_vmx_fail(vcpu,
- VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
- vpid_sync_context(vpid02);
- break;
- case VMX_VPID_EXTENT_ALL_CONTEXT:
- vpid_sync_context(vpid02);
- break;
- default:
- WARN_ON_ONCE(1);
- return kvm_skip_emulated_instruction(vcpu);
- }
+ cpu = get_cpu();
+ if (cpu != vmx->nested.vmcs02.cpu)
+ vmx->nested.last_vpid = 0;
+ else if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR)
+ vpid_sync_vcpu_addr(nested_get_vpid02(vcpu), operand.gla);
+ else
+ vpid_sync_context(nested_get_vpid02(vcpu));
+ put_cpu();
/*
* Sync the shadow page tables if EPT is disabled, L1 is invalidating
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 9e7c9e3edb5b..d0c7a7098143 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6552,7 +6552,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
break;
fallthrough;
case KVM_CAP_DISABLE_QUIRKS:
- kvm->arch.disabled_quirks |= cap->args[0] & kvm_caps.supported_quirks;
+ mutex_lock(&kvm->lock);
+ WRITE_ONCE(kvm->arch.disabled_quirks,
+ kvm->arch.disabled_quirks | (cap->args[0] & kvm_caps.supported_quirks));
+ mutex_unlock(&kvm->lock);
r = 0;
break;
case KVM_CAP_SPLIT_IRQCHIP: {
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index a1bd382232f4..69145d116aa0 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -357,7 +357,7 @@ static inline void kvm_register_write(struct kvm_vcpu *vcpu,
static inline bool kvm_check_has_quirk(struct kvm *kvm, u64 quirk)
{
- return !(kvm->arch.disabled_quirks & quirk);
+ return !(READ_ONCE(kvm->arch.disabled_quirks) & quirk);
}
void kvm_inject_realmode_interrupt(struct kvm_vcpu *vcpu, int irq, int inc_eip);
diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile
index 60a53baa0427..1a2a0821c016 100644
--- a/arch/x86/mm/Makefile
+++ b/arch/x86/mm/Makefile
@@ -41,7 +41,7 @@ CFLAGS_fault.o := -I $(src)/../include/asm/trace
obj-$(CONFIG_X86_32) += pgtable_32.o iomap_32.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
-obj-$(CONFIG_PTDUMP_CORE) += dump_pagetables.o
+obj-$(CONFIG_PTDUMP) += dump_pagetables.o
obj-$(CONFIG_PTDUMP_DEBUGFS) += debug_pagetables.o
obj-$(CONFIG_HIGHMEM) += highmem_32.o
diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c
index 89079ea73e65..2e1c2d006ace 100644
--- a/arch/x86/mm/dump_pagetables.c
+++ b/arch/x86/mm/dump_pagetables.c
@@ -362,6 +362,38 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level,
}
}
+static void note_page_pte(struct ptdump_state *pt_st, unsigned long addr, pte_t pte)
+{
+ note_page(pt_st, addr, 4, pte_val(pte));
+}
+
+static void note_page_pmd(struct ptdump_state *pt_st, unsigned long addr, pmd_t pmd)
+{
+ note_page(pt_st, addr, 3, pmd_val(pmd));
+}
+
+static void note_page_pud(struct ptdump_state *pt_st, unsigned long addr, pud_t pud)
+{
+ note_page(pt_st, addr, 2, pud_val(pud));
+}
+
+static void note_page_p4d(struct ptdump_state *pt_st, unsigned long addr, p4d_t p4d)
+{
+ note_page(pt_st, addr, 1, p4d_val(p4d));
+}
+
+static void note_page_pgd(struct ptdump_state *pt_st, unsigned long addr, pgd_t pgd)
+{
+ note_page(pt_st, addr, 0, pgd_val(pgd));
+}
+
+static void note_page_flush(struct ptdump_state *pt_st)
+{
+ pte_t pte_zero = {0};
+
+ note_page(pt_st, 0, -1, pte_val(pte_zero));
+}
+
bool ptdump_walk_pgd_level_core(struct seq_file *m,
struct mm_struct *mm, pgd_t *pgd,
bool checkwx, bool dmesg)
@@ -378,7 +410,12 @@ bool ptdump_walk_pgd_level_core(struct seq_file *m,
struct pg_state st = {
.ptdump = {
- .note_page = note_page,
+ .note_page_pte = note_page_pte,
+ .note_page_pmd = note_page_pmd,
+ .note_page_pud = note_page_pud,
+ .note_page_p4d = note_page_p4d,
+ .note_page_pgd = note_page_pgd,
+ .note_page_flush = note_page_flush,
.effective_prot = effective_prot,
.range = ptdump_ranges
},
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 5e1518218f1d..ca882b8d7073 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2948,11 +2948,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
WARN_ON_ONCE((flags & BPF_TRAMP_F_INDIRECT) &&
(flags & ~(BPF_TRAMP_F_INDIRECT | BPF_TRAMP_F_RET_FENTRY_RET)));
- /* extra registers for struct arguments */
- for (i = 0; i < m->nr_args; i++) {
- if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG)
- nr_regs += (m->arg_size[i] + 7) / 8 - 1;
- }
+ for (i = 0; i < m->nr_args; i++)
+ nr_regs += (m->arg_size[i] + 7) / 8 - 1;
/* x86-64 supports up to MAX_BPF_FUNC_ARGS arguments. 1-6
* are passed through regs, the remains are through stack.
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 6727cff19d92..e005d0a4143e 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -949,7 +949,7 @@ umode_t efi_attr_is_visible(struct kobject *kobj, struct attribute *attr, int n)
return attr->mode;
}
-enum efi_secureboot_mode __x86_ima_efi_boot_mode(void)
+enum efi_secureboot_mode __x86_efi_boot_mode(void)
{
return boot_params.secure_boot;
}
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index d95104f0fcdc..8e7e2db4d1c2 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -1573,6 +1573,8 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
if (blkg->pd[pol->plid])
continue;
+ if (hlist_unhashed(&blkg->blkcg_node))
+ continue;
/* If prealloc matches, use it; otherwise try GFP_NOWAIT */
if (blkg == pinned_blkg) {
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index a5894ec9696e..61a347f504d7 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -1603,7 +1603,7 @@ static void ioc_lat_stat(struct ioc *ioc, u32 *missed_ppm_ar, u32 *rq_wait_pct_p
u64 rq_wait_ns = 0;
int cpu, rw;
- for_each_online_cpu(cpu) {
+ for_each_possible_cpu(cpu) {
struct ioc_pcpu_stat *stat = per_cpu_ptr(ioc->pcpu_stat, cpu);
u64 this_rq_wait_ns;
diff --git a/block/blk-stat.c b/block/blk-stat.c
index eaf60097bbe1..0149abb59b76 100644
--- a/block/blk-stat.c
+++ b/block/blk-stat.c
@@ -83,7 +83,7 @@ static void blk_stat_timer_fn(struct timer_list *t)
for (bucket = 0; bucket < cb->buckets; bucket++)
blk_rq_stat_init(&cb->stat[bucket]);
- for_each_online_cpu(cpu) {
+ for_each_possible_cpu(cpu) {
struct blk_rq_stat *cpu_stat;
cpu_stat = per_cpu_ptr(cb->cpu_stat, cpu);
diff --git a/block/genhd.c b/block/genhd.c
index 425dcb99bcd0..f4f18ca70aa0 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -436,6 +436,13 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk,
if (disk->fops->submit_bio)
bdev_set_flag(disk->part0, BD_HAS_SUBMIT_BIO);
+ /*
+ * We do not support partitions with zoned block devices, so do not try
+ * to scan the partitions table.
+ */
+ if (blk_queue_is_zoned(disk->queue))
+ disk->flags |= GENHD_FL_NO_PART;
+
/*
* If the driver provides an explicit major number it also must provide
* the number of minors numbers supported, and those will be used to
diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c
index ccfefa6a3669..869e83054c48 100644
--- a/block/kyber-iosched.c
+++ b/block/kyber-iosched.c
@@ -279,7 +279,7 @@ static void kyber_timer_fn(struct timer_list *t)
bool bad = false;
/* Sum all of the per-cpu latency histograms. */
- for_each_online_cpu(cpu) {
+ for_each_possible_cpu(cpu) {
struct kyber_cpu_latency *cpu_latency;
cpu_latency = per_cpu_ptr(kqd->cpu_latency, cpu);
diff --git a/crypto/lskcipher.c b/crypto/lskcipher.c
index cdb4897c63e6..b4537be3b2a8 100644
--- a/crypto/lskcipher.c
+++ b/crypto/lskcipher.c
@@ -95,7 +95,6 @@ static int crypto_lskcipher_crypt_unaligned(
while (len >= bs) {
unsigned chunk = min((unsigned)PAGE_SIZE, len);
- int err;
if (chunk > cs)
chunk &= ~(cs - 1);
diff --git a/drivers/accessibility/speakup/keyhelp.c b/drivers/accessibility/speakup/keyhelp.c
index 822ceac83068..e632c53d6246 100644
--- a/drivers/accessibility/speakup/keyhelp.c
+++ b/drivers/accessibility/speakup/keyhelp.c
@@ -8,6 +8,7 @@
*/
#include <linux/keyboard.h>
+#include <linux/ctype.h>
#include "spk_priv.h"
#include "speakup.h"
@@ -111,7 +112,7 @@ static void say_key(int key)
spk_msg_get(MSG_KEYNAMES_START + (key - 1)));
}
-static int help_init(void)
+static void help_init(void)
{
char start = SPACE;
int i;
@@ -120,13 +121,19 @@ static int help_init(void)
state_tbl = spk_our_keys[0] + SHIFT_TBL_SIZE + 2;
for (i = 0; i < num_funcs; i++) {
char *cur_funcname = spk_msg_get(MSG_FUNCNAMES_START + i);
+ char first_letter;
- if (start == *cur_funcname)
+ first_letter = tolower(*cur_funcname);
+
+ /* Accept only 'a'..'z' to index letter_offsets[] safely */
+ if (first_letter < 'a' || first_letter > 'z')
+ continue;
+
+ if (start == first_letter)
continue;
- start = *cur_funcname;
+ start = first_letter;
letter_offsets[(start & 31) - 1] = i;
}
- return 0;
}
int spk_handle_help(struct vc_data *vc, u_char type, u_char ch, u_short key)
@@ -144,7 +151,7 @@ int spk_handle_help(struct vc_data *vc, u_char type, u_char ch, u_short key)
synth_printf("%s\n", spk_msg_get(MSG_LEAVING_HELP));
return 1;
}
- ch |= 32; /* lower case */
+ ch = tolower(ch);
if (ch < 'a' || ch > 'z')
return -1;
if (letter_offsets[ch - 'a'] == -1) {
diff --git a/drivers/accessibility/speakup/kobjects.c b/drivers/accessibility/speakup/kobjects.c
index 0dfdb6608e02..943ef71b1329 100644
--- a/drivers/accessibility/speakup/kobjects.c
+++ b/drivers/accessibility/speakup/kobjects.c
@@ -92,9 +92,9 @@ static void report_char_chartab_status(int reset, int received, int used,
if (reset) {
pr_info("%s reset to defaults\n", object_type[do_characters]);
} else if (received) {
- len = snprintf(buf, sizeof(buf),
- " updated %d of %d %s\n",
- used, received, object_type[do_characters]);
+ len = scnprintf(buf, sizeof(buf),
+ " updated %d of %d %s\n",
+ used, received, object_type[do_characters]);
if (rejected)
snprintf(buf + (len - 1), sizeof(buf) - (len - 1),
" with %d reject%s\n",
diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c
index a33d60e625f8..c69970ba1843 100644
--- a/drivers/acpi/battery.c
+++ b/drivers/acpi/battery.c
@@ -151,27 +151,28 @@ static int acpi_battery_technology(struct acpi_battery *battery)
static int acpi_battery_get_state(struct acpi_battery *battery);
-static int acpi_battery_is_charged(struct acpi_battery *battery)
+static bool acpi_battery_is_full(struct acpi_battery *battery)
{
- /* charging, discharging, critical low or charge limited */
- if (battery->state != 0)
- return 0;
-
/* battery not reporting charge */
if (battery->capacity_now == ACPI_BATTERY_VALUE_UNKNOWN ||
battery->capacity_now == 0)
- return 0;
+ return false;
/* good batteries update full_charge as the batteries degrade */
if (battery->full_charge_capacity == battery->capacity_now)
- return 1;
+ return true;
/* fallback to using design values for broken batteries */
- if (battery->design_capacity <= battery->capacity_now)
- return 1;
+ return battery->design_capacity <= battery->capacity_now;
+}
- /* we don't do any sort of metric based on percentages */
- return 0;
+static int acpi_battery_is_charged(struct acpi_battery *battery)
+{
+ /* charging, discharging, critical low or charge limited */
+ if (battery->state != 0)
+ return 0;
+
+ return acpi_battery_is_full(battery);
}
static bool acpi_battery_is_degraded(struct acpi_battery *battery)
@@ -212,13 +213,14 @@ static int acpi_battery_get_property(struct power_supply *psy,
if (battery->state & ACPI_BATTERY_STATE_DISCHARGING)
val->intval = acpi_battery_handle_discharging(battery);
else if (battery->state & ACPI_BATTERY_STATE_CHARGING)
- /* Validate the status by checking the current. */
- if (battery->rate_now != ACPI_BATTERY_VALUE_UNKNOWN &&
- battery->rate_now == 0) {
- /* On charge but no current (0W/0mA). */
- val->intval = POWER_SUPPLY_STATUS_NOT_CHARGING;
- } else {
+ /* Check the rate and capacity to validate the status. */
+ if (!acpi_battery_is_full(battery) ||
+ (battery->rate_now != ACPI_BATTERY_VALUE_UNKNOWN &&
+ battery->rate_now > 0)) {
val->intval = POWER_SUPPLY_STATUS_CHARGING;
+ } else {
+ /* Full and zero rate. */
+ val->intval = POWER_SUPPLY_STATUS_NOT_CHARGING;
}
else if (battery->state & ACPI_BATTERY_STATE_CHARGE_LIMITING)
val->intval = POWER_SUPPLY_STATUS_NOT_CHARGING;
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index fd6a83e3d430..8ad6d12b7772 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -129,6 +129,20 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
#define CPC_SUPPORTED(cpc) ((cpc)->type == ACPI_TYPE_INTEGER ? \
!!(cpc)->cpc_entry.int_value : \
!IS_NULL_REG(&(cpc)->cpc_entry.reg))
+
+/*
+ * Each bit indicates the optionality of the register in per-cpu
+ * cpc_regs[] with the corresponding index. 0 means mandatory and 1
+ * means optional.
+ */
+#define REG_OPTIONAL (0x1FC7D0)
+
+/*
+ * Use the index of the register in per-cpu cpc_regs[] to check if
+ * it's an optional one.
+ */
+#define IS_OPTIONAL_CPC_REG(reg_idx) (REG_OPTIONAL & (1U << (reg_idx)))
+
/*
* Arbitrary Retries in case the remote processor is slow to respond
* to PCC commands. Keeping it high enough to cover emulators where
@@ -1176,32 +1190,44 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
return ret_val;
}
-static int cppc_get_perf(int cpunum, enum cppc_regs reg_idx, u64 *perf)
+static int cppc_get_reg_val(int cpu, enum cppc_regs reg_idx, u64 *val)
{
- struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpunum);
+ struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
struct cpc_register_resource *reg;
+ if (val == NULL)
+ return -EINVAL;
+
if (!cpc_desc) {
- pr_debug("No CPC descriptor for CPU:%d\n", cpunum);
+ pr_debug("No CPC descriptor for CPU:%d\n", cpu);
return -ENODEV;
}
reg = &cpc_desc->cpc_regs[reg_idx];
+ if ((reg->type == ACPI_TYPE_INTEGER && IS_OPTIONAL_CPC_REG(reg_idx) &&
+ !reg->cpc_entry.int_value) || (reg->type != ACPI_TYPE_INTEGER &&
+ IS_NULL_REG(®->cpc_entry.reg))) {
+ pr_debug("CPC register is not supported\n");
+ return -EOPNOTSUPP;
+ }
+
if (CPC_IN_PCC(reg)) {
- int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpunum);
+ int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
struct cppc_pcc_data *pcc_ss_data = NULL;
- int ret = 0;
+ int ret;
- if (pcc_ss_id < 0)
- return -EIO;
+ if (pcc_ss_id < 0) {
+ pr_debug("Invalid pcc_ss_id\n");
+ return -ENODEV;
+ }
pcc_ss_data = pcc_data[pcc_ss_id];
down_write(&pcc_ss_data->pcc_lock);
if (send_pcc_cmd(pcc_ss_id, CMD_READ) >= 0)
- cpc_read(cpunum, reg, perf);
+ ret = cpc_read(cpu, reg, val);
else
ret = -EIO;
@@ -1210,9 +1236,56 @@ static int cppc_get_perf(int cpunum, enum cppc_regs reg_idx, u64 *perf)
return ret;
}
- cpc_read(cpunum, reg, perf);
+ return cpc_read(cpu, reg, val);
+}
- return 0;
+static int cppc_set_reg_val_in_pcc(int cpu, struct cpc_register_resource *reg, u64 val)
+{
+ int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
+ struct cppc_pcc_data *pcc_ss_data = NULL;
+ int ret;
+
+ if (pcc_ss_id < 0) {
+ pr_debug("Invalid pcc_ss_id\n");
+ return -ENODEV;
+ }
+
+ ret = cpc_write(cpu, reg, val);
+ if (ret)
+ return ret;
+
+ pcc_ss_data = pcc_data[pcc_ss_id];
+
+ down_write(&pcc_ss_data->pcc_lock);
+ /* after writing CPC, transfer the ownership of PCC to platform */
+ ret = send_pcc_cmd(pcc_ss_id, CMD_WRITE);
+ up_write(&pcc_ss_data->pcc_lock);
+
+ return ret;
+}
+
+static int cppc_set_reg_val(int cpu, enum cppc_regs reg_idx, u64 val)
+{
+ struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
+ struct cpc_register_resource *reg;
+
+ if (!cpc_desc) {
+ pr_debug("No CPC descriptor for CPU:%d\n", cpu);
+ return -ENODEV;
+ }
+
+ reg = &cpc_desc->cpc_regs[reg_idx];
+
+ /* if a register is writeable, it must be a buffer and not null */
+ if ((reg->type != ACPI_TYPE_BUFFER) || IS_NULL_REG(®->cpc_entry.reg)) {
+ pr_debug("CPC register is not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (CPC_IN_PCC(reg))
+ return cppc_set_reg_val_in_pcc(cpu, reg, val);
+
+ return cpc_write(cpu, reg, val);
}
/**
@@ -1224,7 +1297,7 @@ static int cppc_get_perf(int cpunum, enum cppc_regs reg_idx, u64 *perf)
*/
int cppc_get_desired_perf(int cpunum, u64 *desired_perf)
{
- return cppc_get_perf(cpunum, DESIRED_PERF, desired_perf);
+ return cppc_get_reg_val(cpunum, DESIRED_PERF, desired_perf);
}
EXPORT_SYMBOL_GPL(cppc_get_desired_perf);
@@ -1237,7 +1310,7 @@ EXPORT_SYMBOL_GPL(cppc_get_desired_perf);
*/
int cppc_get_nominal_perf(int cpunum, u64 *nominal_perf)
{
- return cppc_get_perf(cpunum, NOMINAL_PERF, nominal_perf);
+ return cppc_get_reg_val(cpunum, NOMINAL_PERF, nominal_perf);
}
/**
@@ -1249,7 +1322,7 @@ int cppc_get_nominal_perf(int cpunum, u64 *nominal_perf)
*/
int cppc_get_highest_perf(int cpunum, u64 *highest_perf)
{
- return cppc_get_perf(cpunum, HIGHEST_PERF, highest_perf);
+ return cppc_get_reg_val(cpunum, HIGHEST_PERF, highest_perf);
}
EXPORT_SYMBOL_GPL(cppc_get_highest_perf);
@@ -1262,7 +1335,7 @@ EXPORT_SYMBOL_GPL(cppc_get_highest_perf);
*/
int cppc_get_epp_perf(int cpunum, u64 *epp_perf)
{
- return cppc_get_perf(cpunum, ENERGY_PERF, epp_perf);
+ return cppc_get_reg_val(cpunum, ENERGY_PERF, epp_perf);
}
EXPORT_SYMBOL_GPL(cppc_get_epp_perf);
@@ -1537,53 +1610,27 @@ int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable)
EXPORT_SYMBOL_GPL(cppc_set_epp_perf);
/**
- * cppc_get_auto_sel_caps - Read autonomous selection register.
- * @cpunum : CPU from which to read register.
- * @perf_caps : struct where autonomous selection register value is updated.
+ * cppc_get_auto_sel() - Read autonomous selection register.
+ * @cpu: CPU from which to read register.
+ * @enable: Return address.
*/
-int cppc_get_auto_sel_caps(int cpunum, struct cppc_perf_caps *perf_caps)
+int cppc_get_auto_sel(int cpu, bool *enable)
{
- struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpunum);
- struct cpc_register_resource *auto_sel_reg;
- u64 auto_sel;
-
- if (!cpc_desc) {
- pr_debug("No CPC descriptor for CPU:%d\n", cpunum);
- return -ENODEV;
- }
-
- auto_sel_reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
-
- if (!CPC_SUPPORTED(auto_sel_reg))
- pr_warn_once("Autonomous mode is not unsupported!\n");
-
- if (CPC_IN_PCC(auto_sel_reg)) {
- int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpunum);
- struct cppc_pcc_data *pcc_ss_data = NULL;
- int ret = 0;
-
- if (pcc_ss_id < 0)
- return -ENODEV;
-
- pcc_ss_data = pcc_data[pcc_ss_id];
-
- down_write(&pcc_ss_data->pcc_lock);
-
- if (send_pcc_cmd(pcc_ss_id, CMD_READ) >= 0) {
- cpc_read(cpunum, auto_sel_reg, &auto_sel);
- perf_caps->auto_sel = (bool)auto_sel;
- } else {
- ret = -EIO;
- }
+ u64 auto_sel;
+ int ret;
- up_write(&pcc_ss_data->pcc_lock);
+ if (enable == NULL)
+ return -EINVAL;
+ ret = cppc_get_reg_val(cpu, AUTO_SEL_ENABLE, &auto_sel);
+ if (ret)
return ret;
- }
+
+ *enable = (bool)auto_sel;
return 0;
}
-EXPORT_SYMBOL_GPL(cppc_get_auto_sel_caps);
+EXPORT_SYMBOL_GPL(cppc_get_auto_sel);
/**
* cppc_set_auto_sel - Write autonomous selection register.
@@ -1592,43 +1639,7 @@ EXPORT_SYMBOL_GPL(cppc_get_auto_sel_caps);
*/
int cppc_set_auto_sel(int cpu, bool enable)
{
- int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
- struct cpc_register_resource *auto_sel_reg;
- struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
- struct cppc_pcc_data *pcc_ss_data = NULL;
- int ret = -EINVAL;
-
- if (!cpc_desc) {
- pr_debug("No CPC descriptor for CPU:%d\n", cpu);
- return -ENODEV;
- }
-
- auto_sel_reg = &cpc_desc->cpc_regs[AUTO_SEL_ENABLE];
-
- if (CPC_IN_PCC(auto_sel_reg)) {
- if (pcc_ss_id < 0) {
- pr_debug("Invalid pcc_ss_id\n");
- return -ENODEV;
- }
-
- if (CPC_SUPPORTED(auto_sel_reg)) {
- ret = cpc_write(cpu, auto_sel_reg, enable);
- if (ret)
- return ret;
- }
-
- pcc_ss_data = pcc_data[pcc_ss_id];
-
- down_write(&pcc_ss_data->pcc_lock);
- /* after writing CPC, transfer the ownership of PCC to platform */
- ret = send_pcc_cmd(pcc_ss_id, CMD_WRITE);
- up_write(&pcc_ss_data->pcc_lock);
- } else {
- ret = -ENOTSUPP;
- pr_debug("_CPC in PCC is not supported\n");
- }
-
- return ret;
+ return cppc_set_reg_val(cpu, AUTO_SEL_ENABLE, enable);
}
EXPORT_SYMBOL_GPL(cppc_set_auto_sel);
@@ -1642,38 +1653,7 @@ EXPORT_SYMBOL_GPL(cppc_set_auto_sel);
*/
int cppc_set_enable(int cpu, bool enable)
{
- int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, cpu);
- struct cpc_register_resource *enable_reg;
- struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu);
- struct cppc_pcc_data *pcc_ss_data = NULL;
- int ret = -EINVAL;
-
- if (!cpc_desc) {
- pr_debug("No CPC descriptor for CPU:%d\n", cpu);
- return -EINVAL;
- }
-
- enable_reg = &cpc_desc->cpc_regs[ENABLE];
-
- if (CPC_IN_PCC(enable_reg)) {
-
- if (pcc_ss_id < 0)
- return -EIO;
-
- ret = cpc_write(cpu, enable_reg, enable);
- if (ret)
- return ret;
-
- pcc_ss_data = pcc_data[pcc_ss_id];
-
- down_write(&pcc_ss_data->pcc_lock);
- /* after writing CPC, transfer the ownership of PCC to platfrom */
- ret = send_pcc_cmd(pcc_ss_id, CMD_WRITE);
- up_write(&pcc_ss_data->pcc_lock);
- return ret;
- }
-
- return cpc_write(cpu, enable_reg, enable);
+ return cppc_set_reg_val(cpu, ENABLE, enable);
}
EXPORT_SYMBOL_GPL(cppc_set_enable);
diff --git a/drivers/acpi/ec.c b/drivers/acpi/ec.c
index 10bd2942c4be..2e65d254f296 100644
--- a/drivers/acpi/ec.c
+++ b/drivers/acpi/ec.c
@@ -1510,6 +1510,24 @@ static bool install_gpio_irq_event_handler(struct acpi_ec *ec)
IRQF_SHARED | IRQF_ONESHOT, "ACPI EC", ec) >= 0;
}
+static int ec_prepare_gpio_irq(struct acpi_ec *ec, struct acpi_device *device)
+{
+ int irq;
+
+ if (!device || ec->gpe >= 0 || ec->irq >= 0)
+ return 0;
+
+ /* ACPI reduced hardware platforms use a GpioInt from _CRS. */
+ irq = acpi_dev_gpio_irq_get(device, 0);
+ if (irq == -EPROBE_DEFER)
+ return irq;
+
+ if (irq >= 0)
+ ec->irq = irq;
+
+ return 0;
+}
+
/**
* ec_install_handlers - Install service callbacks and register query methods.
* @ec: Target EC.
@@ -1524,7 +1542,6 @@ static bool install_gpio_irq_event_handler(struct acpi_ec *ec)
* Return:
* -ENODEV if the address space handler cannot be installed, which means
* "unable to handle transactions",
- * -EPROBE_DEFER if GPIO IRQ acquisition needs to be deferred,
* or 0 (success) otherwise.
*/
static int ec_install_handlers(struct acpi_ec *ec, struct acpi_device *device,
@@ -1557,19 +1574,6 @@ static int ec_install_handlers(struct acpi_ec *ec, struct acpi_device *device,
if (!device)
return 0;
- if (ec->gpe < 0) {
- /* ACPI reduced hardware platforms use a GpioInt from _CRS. */
- int irq = acpi_dev_gpio_irq_get(device, 0);
- /*
- * Bail out right away for deferred probing or complete the
- * initialization regardless of any other errors.
- */
- if (irq == -EPROBE_DEFER)
- return -EPROBE_DEFER;
- else if (irq >= 0)
- ec->irq = irq;
- }
-
if (!test_bit(EC_FLAGS_QUERY_METHODS_INSTALLED, &ec->flags)) {
/* Find and register all query methods */
acpi_walk_namespace(ACPI_TYPE_METHOD, ec->handle, 1,
@@ -1647,6 +1651,14 @@ static int acpi_ec_setup(struct acpi_ec *ec, struct acpi_device *device, bool ca
{
int ret;
+ /*
+ * GPIO IRQ lookup can defer. Do it before publishing the EC
+ * OpRegion to AML to avoid a spurious _REG(disconnect).
+ */
+ ret = ec_prepare_gpio_irq(ec, device);
+ if (ret)
+ return ret;
+
/* First EC capable of handling transactions */
if (!first_ec)
first_ec = ec;
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index d0bfb3706801..eb36cb77dfa4 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -730,7 +730,6 @@ static int acpi_pci_root_add(struct acpi_device *device,
dev_err(&device->dev,
"Bus %04x:%02x not present in PCI namespace\n",
root->segment, (unsigned int)root->secondary.start);
- device->driver_data = NULL;
result = -ENODEV;
goto remove_dmar;
}
@@ -766,6 +765,7 @@ static int acpi_pci_root_add(struct acpi_device *device,
if (hotadd)
dmar_device_remove(handle);
end:
+ device->driver_data = NULL;
kfree(root);
return result;
}
@@ -789,6 +789,7 @@ static void acpi_pci_root_remove(struct acpi_device *device)
pci_unlock_rescan_remove();
+ device->driver_data = NULL;
kfree(root);
}
diff --git a/drivers/acpi/processor_core.c b/drivers/acpi/processor_core.c
index a4498357bd16..3bf076c150fa 100644
--- a/drivers/acpi/processor_core.c
+++ b/drivers/acpi/processor_core.c
@@ -336,11 +336,26 @@ int acpi_get_cpuid(acpi_handle handle, int type, u32 acpi_id)
EXPORT_SYMBOL_GPL(acpi_get_cpuid);
#ifdef CONFIG_ACPI_HOTPLUG_IOAPIC
-static int get_ioapic_id(struct acpi_subtable_header *entry, u32 gsi_base,
+static bool madt_entry_is_valid(struct acpi_subtable_header *entry,
+ unsigned long end)
+{
+ unsigned long start = (unsigned long)entry;
+
+ if (start >= end || end - start < sizeof(*entry))
+ return false;
+
+ return entry->length >= sizeof(*entry) && entry->length <= end - start;
+}
+
+static int get_ioapic_id(struct acpi_subtable_header *entry,
+ const unsigned long end, u32 gsi_base,
u64 *phys_addr, int *ioapic_id)
{
struct acpi_madt_io_apic *ioapic = (struct acpi_madt_io_apic *)entry;
+ if (!madt_entry_is_valid(entry, end) || BAD_MADT_ENTRY(ioapic, end))
+ return 0;
+
if (ioapic->global_irq_base != gsi_base)
return 0;
@@ -361,17 +376,19 @@ static int parse_madt_ioapic_entry(u32 gsi_base, u64 *phys_addr)
return apic_id;
entry = (unsigned long)madt;
+ if (madt->header.length < sizeof(*madt))
+ return apic_id;
madt_end = entry + madt->header.length;
/* Parse all entries looking for a match. */
entry += sizeof(struct acpi_table_madt);
- while (entry + sizeof(struct acpi_subtable_header) < madt_end) {
+ while (madt_entry_is_valid((struct acpi_subtable_header *)entry,
+ madt_end)) {
hdr = (struct acpi_subtable_header *)entry;
if (hdr->type == ACPI_MADT_TYPE_IO_APIC &&
- get_ioapic_id(hdr, gsi_base, phys_addr, &apic_id))
+ get_ioapic_id(hdr, madt_end, gsi_base, phys_addr, &apic_id))
break;
- else
- entry += hdr->length;
+ entry += hdr->length;
}
return apic_id;
@@ -398,7 +415,9 @@ static int parse_mat_ioapic_entry(acpi_handle handle, u32 gsi_base,
header = (struct acpi_subtable_header *)obj->buffer.pointer;
if (header->type == ACPI_MADT_TYPE_IO_APIC)
- get_ioapic_id(header, gsi_base, phys_addr, &apic_id);
+ get_ioapic_id(header,
+ (unsigned long)header + obj->buffer.length,
+ gsi_base, phys_addr, &apic_id);
exit:
kfree(buffer.pointer);
diff --git a/drivers/acpi/processor_driver.c b/drivers/acpi/processor_driver.c
index 3b281bc1e73c..2f5bfd00bfce 100644
--- a/drivers/acpi/processor_driver.c
+++ b/drivers/acpi/processor_driver.c
@@ -166,7 +166,7 @@ static int __acpi_processor_start(struct acpi_device *device)
if (result && !IS_ENABLED(CONFIG_ACPI_CPU_FREQ_PSS))
dev_dbg(&device->dev, "CPPC data invalid or not present\n");
- if (!cpuidle_get_driver() || cpuidle_get_driver() == &acpi_idle_driver)
+ if (cpuidle_get_driver() == &acpi_idle_driver)
acpi_processor_power_init(pr);
acpi_pss_perf_init(pr);
@@ -259,9 +259,11 @@ static int __init acpi_processor_driver_init(void)
acpi_processor_ignore_ppc_init();
}
+ acpi_processor_register_idle_driver();
+
result = driver_register(&acpi_processor_driver);
if (result < 0)
- return result;
+ goto unregister_idle_drv;
result = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
"acpi/cpu-drv:online",
@@ -280,8 +282,19 @@ static int __init acpi_processor_driver_init(void)
*/
acpi_processor_init_invariance_cppc();
return 0;
+
err:
driver_unregister(&acpi_processor_driver);
+
+unregister_idle_drv:
+ acpi_processor_unregister_idle_driver();
+
+ if (acpi_processor_cpufreq_init) {
+ cpufreq_unregister_notifier(&acpi_processor_notifier_block,
+ CPUFREQ_POLICY_NOTIFIER);
+ acpi_processor_cpufreq_init = false;
+ }
+
return result;
}
@@ -299,6 +312,7 @@ static void __exit acpi_processor_driver_exit(void)
cpuhp_remove_state_nocalls(hp_online);
cpuhp_remove_state_nocalls(CPUHP_ACPI_CPUDRV_DEAD);
driver_unregister(&acpi_processor_driver);
+ acpi_processor_unregister_idle_driver();
}
module_init(acpi_processor_driver_init);
diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c
index b8a8cbf8fafd..c0dfae3f835b 100644
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -950,6 +950,13 @@ static int acpi_processor_evaluate_lpi(acpi_handle handle,
if (obj->type == ACPI_TYPE_BUFFER) {
struct acpi_power_register *reg;
+ if (obj->buffer.length < sizeof(*reg)) {
+ acpi_handle_debug(handle,
+ "Invalid register data for _LPI state %d\n",
+ state_idx);
+ continue;
+ }
+
reg = (struct acpi_power_register *)obj->buffer.pointer;
if (reg->space_id != ACPI_ADR_SPACE_SYSTEM_IO &&
reg->space_id != ACPI_ADR_SPACE_FIXED_HARDWARE)
@@ -966,13 +973,6 @@ static int acpi_processor_evaluate_lpi(acpi_handle handle,
continue;
}
- /* elements[7,8] skipped for now i.e. Residency/Usage counter*/
-
- obj = pkg_elem + 9;
- if (obj->type == ACPI_TYPE_STRING)
- strscpy(lpi_state->desc, obj->string.pointer,
- ACPI_CX_DESC_LEN);
-
lpi_state->index = state_idx;
if (obj_get_integer(pkg_elem + 0, &lpi_state->min_residency)) {
pr_debug("No min. residency found, assuming 10 us\n");
@@ -995,6 +995,20 @@ static int acpi_processor_evaluate_lpi(acpi_handle handle,
if (obj_get_integer(pkg_elem + 5, &lpi_state->enable_parent_state))
lpi_state->enable_parent_state = 0;
+
+ /* Skip elements [7-8] i.e. Residency/Usage counters. */
+
+ /*
+ * Avoid out-of-bounds access if the size of the package is less
+ * than expected.
+ */
+ if (element->package.count < 10)
+ continue;
+
+ obj = pkg_elem + 9;
+ if (obj->type == ACPI_TYPE_STRING)
+ strscpy(lpi_state->desc, obj->string.pointer,
+ ACPI_CX_DESC_LEN);
}
acpi_handle_debug(handle, "Found %d power states\n", state_idx);
@@ -1365,7 +1379,58 @@ int acpi_processor_power_state_has_changed(struct acpi_processor *pr)
return 0;
}
-static int acpi_processor_registered;
+void acpi_processor_register_idle_driver(void)
+{
+ struct acpi_processor *pr;
+ int ret = -ENODEV;
+ int cpu;
+
+ /*
+ * If a cpuidle driver is already registered, there is no need to
+ * evaluate _CST or attempt to register the ACPI idle driver.
+ */
+ if (cpuidle_get_driver()) {
+ pr_debug("cpuidle driver %pS already registered.\n", cpuidle_get_driver());
+ return;
+ }
+
+ /*
+ * ACPI idle driver is used by all possible CPUs.
+ * Use the processor power info of one in them to set up idle states.
+ * Note that the existing idle handler will be used on platforms that
+ * only support C1.
+ */
+ for_each_possible_cpu(cpu) {
+ pr = per_cpu(processors, cpu);
+ if (!pr)
+ continue;
+
+ acpi_processor_cstate_first_run_checks();
+ ret = acpi_processor_get_power_info(pr);
+ if (!ret) {
+ pr->flags.power_setup_done = 1;
+ acpi_processor_setup_cpuidle_states(pr);
+ break;
+ }
+ }
+
+ if (ret) {
+ pr_debug("No ACPI power information from any CPUs.\n");
+ return;
+ }
+
+ ret = cpuidle_register_driver(&acpi_idle_driver);
+ if (ret) {
+ pr_debug("register %s failed.\n", acpi_idle_driver.name);
+ return;
+ }
+ pr_debug("%s registered with cpuidle.\n", acpi_idle_driver.name);
+}
+
+void acpi_processor_unregister_idle_driver(void)
+{
+ cpuidle_unregister_driver(&acpi_idle_driver);
+}
int acpi_processor_power_init(struct acpi_processor *pr)
{
@@ -1380,22 +1445,7 @@ int acpi_processor_power_init(struct acpi_processor *pr)
if (!acpi_processor_get_power_info(pr))
pr->flags.power_setup_done = 1;
- /*
- * Install the idle handler if processor power management is supported.
- * Note that we use previously set idle handler will be used on
- * platforms that only support C1.
- */
if (pr->flags.power) {
- /* Register acpi_idle_driver if not already registered */
- if (!acpi_processor_registered) {
- acpi_processor_setup_cpuidle_states(pr);
- retval = cpuidle_register_driver(&acpi_idle_driver);
- if (retval)
- return retval;
- pr_debug("%s registered with cpuidle\n",
- acpi_idle_driver.name);
- }
-
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (!dev)
return -ENOMEM;
@@ -1408,14 +1458,11 @@ int acpi_processor_power_init(struct acpi_processor *pr)
*/
retval = cpuidle_register_device(dev);
if (retval) {
- if (acpi_processor_registered == 0)
- cpuidle_unregister_driver(&acpi_idle_driver);
per_cpu(acpi_cpuidle_device, pr->id) = NULL;
kfree(dev);
return retval;
}
- acpi_processor_registered++;
}
return 0;
}
@@ -1429,10 +1476,6 @@ int acpi_processor_power_exit(struct acpi_processor *pr)
if (pr->flags.power) {
cpuidle_unregister_device(dev);
- acpi_processor_registered--;
- if (acpi_processor_registered == 0)
- cpuidle_unregister_driver(&acpi_idle_driver);
-
kfree(dev);
}
diff --git a/drivers/acpi/riscv/irq.c b/drivers/acpi/riscv/irq.c
index cced960c2aef..9c8185ca9f8c 100644
--- a/drivers/acpi/riscv/irq.c
+++ b/drivers/acpi/riscv/irq.c
@@ -230,6 +230,7 @@ static acpi_status riscv_acpi_irq_get_parent(struct acpi_resource *ares, void *c
return AE_OK;
ctx->handle = riscv_acpi_get_gsi_handle(eirq->interrupts[ctx->index]);
+ ctx->rc = 0;
return AE_CTRL_TERMINATE;
}
@@ -245,10 +246,22 @@ static int riscv_acpi_irq_get_dep(acpi_handle handle, unsigned int index, acpi_h
acpi_walk_resources(handle, METHOD_NAME__CRS, riscv_acpi_irq_get_parent, &ctx);
*gsi_handle = ctx.handle;
- if (*gsi_handle)
- return 1;
- return 0;
+ return ctx.rc;
+}
+
+static bool acpi_prt_entry_valid(void *prt_entry)
+{
+ struct acpi_pci_routing_table *entry = prt_entry;
+
+ return entry && entry->length > 0;
+}
+
+static void *acpi_prt_next_entry(void *prt_entry)
+{
+ struct acpi_pci_routing_table *entry = prt_entry;
+
+ return prt_entry + entry->length;
}
static u32 riscv_acpi_add_prt_dep(acpi_handle handle)
@@ -269,9 +282,11 @@ static u32 riscv_acpi_add_prt_dep(acpi_handle handle)
}
entry = buffer.pointer;
- while (entry && (entry->length > 0)) {
+ for (; acpi_prt_entry_valid(entry); entry = acpi_prt_next_entry(entry)) {
if (entry->source[0]) {
- acpi_get_handle(handle, entry->source, &link_handle);
+ status = acpi_get_handle(handle, entry->source, &link_handle);
+ if (ACPI_FAILURE(status))
+ continue;
dep_devices.count = 1;
dep_devices.handles = kcalloc(1, sizeof(*dep_devices.handles), GFP_KERNEL);
if (!dep_devices.handles) {
@@ -293,9 +308,6 @@ static u32 riscv_acpi_add_prt_dep(acpi_handle handle)
dep_devices.handles[0] = gsi_handle;
count += acpi_scan_add_dep(handle, &dep_devices);
}
-
- entry = (struct acpi_pci_routing_table *)
- ((unsigned long)entry + entry->length);
}
kfree(buffer.pointer);
@@ -310,8 +322,11 @@ static u32 riscv_acpi_add_irq_dep(acpi_handle handle)
int i;
for (i = 0;
- riscv_acpi_irq_get_dep(handle, i, &gsi_handle);
+ !riscv_acpi_irq_get_dep(handle, i, &gsi_handle);
i++) {
+ if (!gsi_handle)
+ continue;
+
dep_devices.count = 1;
dep_devices.handles = kcalloc(1, sizeof(*dep_devices.handles), GFP_KERNEL);
if (!dep_devices.handles) {
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index 4b9c81fbdc3b..42f0840212e8 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -521,12 +521,10 @@ static void acpi_device_release(struct device *dev)
kfree(acpi_dev);
}
-static void acpi_device_del(struct acpi_device *device)
+static void acpi_device_cleanup(struct acpi_device *device)
{
struct acpi_device_bus_id *acpi_device_bus_id;
- mutex_lock(&acpi_device_lock);
-
list_for_each_entry(acpi_device_bus_id, &acpi_bus_id_list, node)
if (!strcmp(acpi_device_bus_id->bus_id,
acpi_device_hid(device))) {
@@ -541,6 +539,13 @@ static void acpi_device_del(struct acpi_device *device)
}
list_del(&device->wakeup_list);
+}
+
+static void acpi_device_del(struct acpi_device *device)
+{
+ mutex_lock(&acpi_device_lock);
+
+ acpi_device_cleanup(device);
mutex_unlock(&acpi_device_lock);
@@ -803,7 +808,7 @@ int acpi_device_add(struct acpi_device *device)
err:
mutex_lock(&acpi_device_lock);
- list_del(&device->wakeup_list);
+ acpi_device_cleanup(device);
err_unlock:
mutex_unlock(&acpi_device_lock);
diff --git a/drivers/acpi/video_detect.c b/drivers/acpi/video_detect.c
index 2c120ade8f51..2a562cb42508 100644
--- a/drivers/acpi/video_detect.c
+++ b/drivers/acpi/video_detect.c
@@ -137,8 +137,10 @@ static int video_detect_portege_r100(const struct dmi_system_id *d)
struct pci_dev *dev;
/* Search for Trident CyberBlade XP4m32 to confirm Portégé R100 */
dev = pci_get_device(PCI_VENDOR_ID_TRIDENT, 0x2100, NULL);
- if (dev)
+ if (dev) {
acpi_backlight_dmi = acpi_backlight_vendor;
+ pci_dev_put(dev);
+ }
return 0;
}
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 57a7a2b68aaa..e12d89765502 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -1676,6 +1676,51 @@ static irqreturn_t ahci_thunderx_irq_handler(int irq, void *dev_instance)
}
#endif
+/*
+ * The Marvell 88SE6111/6121/6145 ("Thor") family stops reporting interrupts
+ * for a port when HOST_IRQ_STAT is cleared while PxIS still holds bits: PxIS
+ * keeps its content, HOST_IRQ_STAT reads back as 0, the port is never looked
+ * at again and the command in flight only ends in a timeout. On a 88SE6121
+ * this makes every SATA-2 or SATA-3 disk fail to IDENTIFY, while SATA-1 disks
+ * happen to win the race often enough to work.
+ *
+ * Clearing the host status before servicing the ports avoids it. Marvell's
+ * own driver for these chips does the same and says so ("clear global before
+ * channel"), and ahci_xgene handles its broken edge latch the same way. The
+ * price is at most one spurious interrupt per valid one, which is why this is
+ * not the generic behaviour - see AHCI 1.1 section 10.6.2.
+ *
+ * Link: https://bugzilla.kernel.org/show_bug.cgi?id=216094
+ */
+static irqreturn_t ahci_mv_irq_handler(int irq, void *dev_instance)
+{
+ struct ata_host *host = dev_instance;
+ struct ahci_host_priv *hpriv = host->private_data;
+ void __iomem *mmio = hpriv->mmio;
+ unsigned int rc;
+ u32 irq_stat, irq_masked;
+
+ irq_stat = readl(mmio + HOST_IRQ_STAT);
+ if (!irq_stat)
+ return IRQ_NONE;
+
+ irq_masked = irq_stat & hpriv->port_map;
+
+ spin_lock(&host->lock);
+
+ /*
+ * Use the unmasked value to clear the interrupt, as a spurious pending
+ * event on a dummy port might cause a screaming IRQ.
+ */
+ writel(irq_stat, mmio + HOST_IRQ_STAT);
+
+ rc = ahci_handle_port_intr(host, irq_masked);
+
+ spin_unlock(&host->lock);
+
+ return IRQ_RETVAL(rc);
+}
+
static void ahci_remap_check(struct pci_dev *pdev, int bar,
struct ahci_host_priv *hpriv)
{
@@ -1948,6 +1993,10 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
return -ENOMEM;
hpriv->flags |= (unsigned long)pi.private_data;
+ /* the Marvell "Thor" family needs HOST_IRQ_STAT cleared first */
+ if (board_id == board_ahci_mv)
+ hpriv->irq_handler = ahci_mv_irq_handler;
+
/* MCP65 revision A1 and A2 can't do MSI */
if (board_id == board_ahci_mcp65 &&
(pdev->revision == 0xa1 || pdev->revision == 0xa2))
diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c
index 89410127089b..2af13612a40a 100644
--- a/drivers/base/cacheinfo.c
+++ b/drivers/base/cacheinfo.c
@@ -332,9 +332,14 @@ static int cache_setup_properties(unsigned int cpu)
else if (!acpi_disabled)
ret = cache_setup_acpi(cpu);
- // Assume there is no cache information available in DT/ACPI from now.
- if (ret && use_arch_cache_info())
+ /*
+ * No DT/ACPI cache nodes; fall back to arch-derived topology (e.g.
+ * arm64 CLIDR_EL1) and clear the error to avoid a spurious warning.
+ */
+ if (ret && use_arch_cache_info()) {
use_arch_info = true;
+ ret = 0;
+ }
return ret;
}
diff --git a/drivers/base/firmware_loader/fallback.c b/drivers/base/firmware_loader/fallback.c
index 3ef0b312ae71..00d6b6e59461 100644
--- a/drivers/base/firmware_loader/fallback.c
+++ b/drivers/base/firmware_loader/fallback.c
@@ -95,6 +95,16 @@ static int fw_load_sysfs_fallback(struct fw_sysfs *fw_sysfs, long timeout)
retval = -EINTR;
goto out;
}
+
+ /*
+ * device_add() exposes the loading interface before pending_list is
+ * linked into pending_fw_head, so fw_state_done() may run first.
+ */
+ if (fw_state_is_done(fw_priv)) {
+ mutex_unlock(&fw_lock);
+ goto out;
+ }
+
list_add(&fw_priv->pending_list, &pending_fw_head);
mutex_unlock(&fw_lock);
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 2f7e8bab3d10..889e65378145 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -160,7 +160,7 @@ static ssize_t state_show(struct device *dev, struct device_attribute *attr,
break;
default:
WARN_ON(1);
- return sysfs_emit(buf, "ERROR-UNKNOWN-%ld\n", mem->state);
+ return sysfs_emit(buf, "ERROR-UNKNOWN-%d\n", mem->state);
}
return sysfs_emit(buf, "%s\n", output);
@@ -435,7 +435,7 @@ static ssize_t phys_device_show(struct device *dev,
static int print_allowed_zone(char *buf, int len, int nid,
struct memory_group *group,
unsigned long start_pfn, unsigned long nr_pages,
- int online_type, struct zone *default_zone)
+ enum mmop online_type, struct zone *default_zone)
{
struct zone *zone;
@@ -512,7 +512,7 @@ static ssize_t auto_online_blocks_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
return sysfs_emit(buf, "%s\n",
- online_type_to_str[mhp_default_online_type]);
+ online_type_to_str[mhp_get_default_online_type()]);
}
static ssize_t auto_online_blocks_store(struct device *dev,
@@ -524,7 +524,7 @@ static ssize_t auto_online_blocks_store(struct device *dev,
if (online_type < 0)
return -EINVAL;
- mhp_default_online_type = online_type;
+ mhp_set_default_online_type(online_type);
return count;
}
diff --git a/drivers/base/soc.c b/drivers/base/soc.c
index 282c38aece0d..c43940940aa3 100644
--- a/drivers/base/soc.c
+++ b/drivers/base/soc.c
@@ -194,6 +194,7 @@ EXPORT_SYMBOL_GPL(soc_device_unregister);
static int __init soc_bus_register(void)
{
+ struct soc_device *soc_dev;
int ret;
ret = bus_register(&soc_bus_type);
@@ -201,10 +202,20 @@ static int __init soc_bus_register(void)
return ret;
soc_bus_registered = true;
- if (early_soc_dev_attr)
- return PTR_ERR(soc_device_register(early_soc_dev_attr));
+ if (early_soc_dev_attr) {
+ soc_dev = soc_device_register(early_soc_dev_attr);
+ if (IS_ERR(soc_dev)) {
+ ret = PTR_ERR(soc_dev);
+ goto err_unregister_bus;
+ }
+ }
return 0;
+
+err_unregister_bus:
+ soc_bus_registered = false;
+ bus_unregister(&soc_bus_type);
+ return ret;
}
core_initcall(soc_bus_register);
diff --git a/drivers/base/swnode.c b/drivers/base/swnode.c
index 53b3f0061ad1..49295a1a3ea9 100644
--- a/drivers/base/swnode.c
+++ b/drivers/base/swnode.c
@@ -529,7 +529,7 @@ software_node_get_reference_args(const struct fwnode_handle *fwnode,
if (prop->is_inline)
return -EINVAL;
- if ((index + 1) * sizeof(*ref) > prop->length)
+ if (index >= prop->length / sizeof(*ref))
return -ENOENT;
ref_array = prop->pointer;
diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c
index 223faa9d5ffd..86a9c557a5d6 100644
--- a/drivers/block/mtip32xx/mtip32xx.c
+++ b/drivers/block/mtip32xx/mtip32xx.c
@@ -3038,6 +3038,8 @@ static int mtip_block_ioctl(struct block_device *dev,
if (!dd)
return -ENOTTY;
+ guard(mutex)(&dd->ioctl_mutex);
+
if (unlikely(test_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag)))
return -ENOTTY;
@@ -3076,6 +3078,8 @@ static int mtip_block_compat_ioctl(struct block_device *dev,
if (!dd)
return -ENOTTY;
+ guard(mutex)(&dd->ioctl_mutex);
+
if (unlikely(test_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag)))
return -ENOTTY;
@@ -3700,6 +3704,7 @@ static int mtip_pci_probe(struct pci_dev *pdev,
dd = kzalloc_node(sizeof(struct driver_data), GFP_KERNEL, my_node);
if (!dd)
return -ENOMEM;
+ mutex_init(&dd->ioctl_mutex);
/* Attach the private data to this PCI device. */
pci_set_drvdata(pdev, dd);
@@ -3872,6 +3877,7 @@ static void mtip_pci_remove(struct pci_dev *pdev)
}
set_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag);
+ mutex_lock(&dd->ioctl_mutex);
if (test_bit(MTIP_DDF_INIT_DONE_BIT, &dd->dd_flag))
del_gendisk(dd->disk);
@@ -3900,6 +3906,7 @@ static void mtip_pci_remove(struct pci_dev *pdev)
/* De-initialize the protocol layer. */
mtip_hw_exit(dd);
+ mutex_unlock(&dd->ioctl_mutex);
if (dd->isr_workq) {
destroy_workqueue(dd->isr_workq);
diff --git a/drivers/block/mtip32xx/mtip32xx.h b/drivers/block/mtip32xx/mtip32xx.h
index f7328f19ac5c..0963c07b5845 100644
--- a/drivers/block/mtip32xx/mtip32xx.h
+++ b/drivers/block/mtip32xx/mtip32xx.h
@@ -12,6 +12,7 @@
#define __MTIP32XX_H__
#include <linux/spinlock.h>
+#include <linux/mutex.h>
#include <linux/rwsem.h>
#include <linux/ata.h>
#include <linux/interrupt.h>
@@ -432,6 +433,7 @@ struct driver_data {
struct request_queue *queue; /* Our request queue. */
struct blk_mq_tag_set tags; /* blk_mq tags */
+ struct mutex ioctl_mutex;
struct mtip_port *port; /* Pointer to the port data structure. */
diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 74adee7ac386..a0086307b460 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -66,7 +66,7 @@ struct nullb_page {
#define NULLB_PAGE_FREE (MAP_SZ - 2)
static LIST_HEAD(nullb_list);
-static struct mutex lock;
+static DEFINE_MUTEX(lock);
static int null_major;
static DEFINE_IDA(nullb_indexes);
static struct blk_mq_tag_set tag_set;
@@ -336,7 +336,15 @@ static ssize_t nullb_device_bool_attr_store(bool *val, const char *page,
return count;
}
-/* The following macro should only be used with TYPE = {uint, ulong, bool}. */
+/*
+ * The following macro should only be used with TYPE = {uint, ulong, bool}.
+ *
+ * The device configuration is modified under the global lock to serialize
+ * attribute changes against null_add_dev() and null_del_dev(): without this,
+ * an attribute could be changed while null_add_dev() is running, that is,
+ * before NULLB_DEV_FL_CONFIGURED is set, which would let null_add_dev()
+ * observe inconsistent values for the device configuration.
+ */
#define NULLB_DEVICE_ATTR(NAME, TYPE, APPLY) \
static ssize_t \
nullb_device_##NAME##_show(struct config_item *item, char *page) \
@@ -356,6 +364,7 @@ nullb_device_##NAME##_store(struct config_item *item, const char *page, \
ret = nullb_device_##TYPE##_attr_store(&new_value, page, count);\
if (ret < 0) \
return ret; \
+ guard(mutex)(&lock); \
if (apply_fn) \
ret = apply_fn(dev, new_value); \
else if (test_bit(NULLB_DEV_FL_CONFIGURED, &dev->flags)) \
@@ -375,9 +384,20 @@ static int nullb_update_nr_hw_queues(struct nullb_device *dev,
struct blk_mq_tag_set *set;
int ret, nr_hw_queues;
+ lockdep_assert_held(&lock);
+
if (!dev->nullb)
return 0;
+ /*
+ * A shared tag_set is mapped via the module-wide queue counts, so a
+ * per-device resize is meaningless. On shrink it would also leave
+ * mq_map[] pointing at NULLed hctx slots, causing a NULL deref in
+ * blk_mq_map_swqueue(). Reject it.
+ */
+ if (dev->shared_tags)
+ return -EINVAL;
+
/*
* Make sure at least one submit queue exists.
*/
@@ -417,25 +437,13 @@ static int nullb_update_nr_hw_queues(struct nullb_device *dev,
static int nullb_apply_submit_queues(struct nullb_device *dev,
unsigned int submit_queues)
{
- int ret;
-
- mutex_lock(&lock);
- ret = nullb_update_nr_hw_queues(dev, submit_queues, dev->poll_queues);
- mutex_unlock(&lock);
-
- return ret;
+ return nullb_update_nr_hw_queues(dev, submit_queues, dev->poll_queues);
}
static int nullb_apply_poll_queues(struct nullb_device *dev,
unsigned int poll_queues)
{
- int ret;
-
- mutex_lock(&lock);
- ret = nullb_update_nr_hw_queues(dev, dev->submit_queues, poll_queues);
- mutex_unlock(&lock);
-
- return ret;
+ return nullb_update_nr_hw_queues(dev, dev->submit_queues, poll_queues);
}
NULLB_DEVICE_ATTR(size, ulong, NULL);
@@ -572,8 +580,13 @@ static ssize_t nullb_device_zone_readonly_store(struct config_item *item,
const char *page, size_t count)
{
struct nullb_device *dev = to_nullb_device(item);
+ ssize_t ret;
- return zone_cond_store(dev, page, count, BLK_ZONE_COND_READONLY);
+ mutex_lock(&lock);
+ ret = zone_cond_store(dev, page, count, BLK_ZONE_COND_READONLY);
+ mutex_unlock(&lock);
+
+ return ret;
}
CONFIGFS_ATTR_WO(nullb_device_, zone_readonly);
@@ -581,8 +594,13 @@ static ssize_t nullb_device_zone_offline_store(struct config_item *item,
const char *page, size_t count)
{
struct nullb_device *dev = to_nullb_device(item);
+ ssize_t ret;
+
+ mutex_lock(&lock);
+ ret = zone_cond_store(dev, page, count, BLK_ZONE_COND_OFFLINE);
+ mutex_unlock(&lock);
- return zone_cond_store(dev, page, count, BLK_ZONE_COND_OFFLINE);
+ return ret;
}
CONFIGFS_ATTR_WO(nullb_device_, zone_offline);
@@ -812,7 +830,6 @@ static void null_free_dev(struct nullb_device *dev)
if (!dev)
return;
- null_free_zoned_dev(dev);
badblocks_exit(&dev->badblocks);
kfree(dev);
}
@@ -1721,6 +1738,7 @@ static void null_del_dev(struct nullb *nullb)
}
put_disk(nullb->disk);
+ null_free_zoned_dev(dev);
if (nullb->tag_set == &nullb->__tag_set)
blk_mq_free_tag_set(nullb->tag_set);
kfree(nullb->queues);
@@ -2105,17 +2123,9 @@ static int __init null_init(void)
config_group_init(&nullb_subsys.su_group);
mutex_init(&nullb_subsys.su_mutex);
- ret = configfs_register_subsystem(&nullb_subsys);
- if (ret)
- return ret;
-
- mutex_init(&lock);
-
null_major = register_blkdev(0, "nullb");
- if (null_major < 0) {
- ret = null_major;
- goto err_conf;
- }
+ if (null_major < 0)
+ return null_major;
for (i = 0; i < nr_devices; i++) {
ret = null_create_dev();
@@ -2123,6 +2133,10 @@ static int __init null_init(void)
goto err_dev;
}
+ ret = configfs_register_subsystem(&nullb_subsys);
+ if (ret)
+ goto err_dev;
+
pr_info("module loaded\n");
return 0;
@@ -2132,8 +2146,8 @@ static int __init null_init(void)
null_destroy_dev(nullb);
}
unregister_blkdev(null_major, "nullb");
-err_conf:
- configfs_unregister_subsystem(&nullb_subsys);
+ if (tag_set.ops)
+ blk_mq_free_tag_set(&tag_set);
return ret;
}
@@ -2154,8 +2168,6 @@ static void __exit null_exit(void)
if (tag_set.ops)
blk_mq_free_tag_set(&tag_set);
-
- mutex_destroy(&lock);
}
module_init(null_init);
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 404cb44dd708..6d5523058bed 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -543,7 +543,7 @@ static int ublk_validate_params(const struct ublk_device *ub)
if (p->max_sectors < PAGE_SECTORS)
return -EINVAL;
- if (ublk_dev_is_zoned(ub) && !p->chunk_sectors)
+ if (ublk_dev_is_zoned(ub) && !is_power_of_2(p->chunk_sectors))
return -EINVAL;
} else
return -EINVAL;
@@ -1072,8 +1072,14 @@ static inline void __ublk_complete_rq(struct request *req)
*
* Re-read simply for this unlikely case.
*/
- if (unlikely(unmapped_bytes < io->res))
+ if (unlikely(unmapped_bytes < io->res)) {
+ if (unlikely(!unmapped_bytes)) {
+ res = BLK_STS_IOERR;
+ goto exit;
+ }
+
io->res = unmapped_bytes;
+ }
/*
* Run bio->bi_end_io() with softirqs disabled. If the final fput
@@ -1421,6 +1427,12 @@ static int ublk_ch_mmap(struct file *filp, struct vm_area_struct *vma)
if (vma->vm_flags & VM_WRITE)
return -EPERM;
+ /*
+ * The per-queue command buffer is kernel-written ABI; prevent
+ * the daemon from upgrading to writable via mprotect().
+ */
+ vm_flags_clear(vma, VM_MAYWRITE);
+
end = UBLKSRV_CMD_BUF_OFFSET + ub->dev_info.nr_hw_queues * max_sz;
if (phys_off < UBLKSRV_CMD_BUF_OFFSET || phys_off >= end)
return -EINVAL;
diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c
index cb948423a9bd..c63246a60c6b 100644
--- a/drivers/bluetooth/btintel.c
+++ b/drivers/bluetooth/btintel.c
@@ -3399,7 +3399,7 @@ int btintel_recv_event(struct hci_dev *hdev, struct sk_buff *skb)
/* Handle all diagnostics events separately. May still call
* hci_recv_frame.
*/
- if (len >= sizeof(diagnostics_hdr) &&
+ if (len + 1 >= sizeof(diagnostics_hdr) &&
memcmp(&skb->data[2], diagnostics_hdr,
sizeof(diagnostics_hdr)) == 0) {
return btintel_diagnostics(hdev, skb);
diff --git a/drivers/bluetooth/btmtk.c b/drivers/bluetooth/btmtk.c
index 22b05628e52e..e7d01f1e26b8 100644
--- a/drivers/bluetooth/btmtk.c
+++ b/drivers/bluetooth/btmtk.c
@@ -112,7 +112,11 @@ static void btmtk_coredump_notify(struct hci_dev *hdev, int state)
void btmtk_fw_get_filename(char *buf, size_t size, u32 dev_id, u32 fw_ver,
u32 fw_flavor)
{
- if (dev_id == 0x7925)
+ if (dev_id == 0x6639)
+ snprintf(buf, size,
+ "mediatek/mt7927/BT_RAM_CODE_MT%04x_2_%x_hdr.bin",
+ dev_id & 0xffff, (fw_ver & 0xff) + 1);
+ else if (dev_id == 0x7925)
snprintf(buf, size,
"mediatek/mt%04x/BT_RAM_CODE_MT%04x_1_%x_hdr.bin",
dev_id & 0xffff, dev_id & 0xffff, (fw_ver & 0xff) + 1);
@@ -128,7 +132,8 @@ void btmtk_fw_get_filename(char *buf, size_t size, u32 dev_id, u32 fw_ver,
EXPORT_SYMBOL_GPL(btmtk_fw_get_filename);
int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
- wmt_cmd_sync_func_t wmt_cmd_sync)
+ wmt_cmd_sync_func_t wmt_cmd_sync,
+ u32 dev_id)
{
struct btmtk_hci_wmt_params wmt_params;
struct btmtk_patch_header *hdr;
@@ -166,6 +171,14 @@ int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
section_offset = le32_to_cpu(sectionmap->secoffset);
dl_size = le32_to_cpu(sectionmap->bin_info_spec.dlsize);
+ /* MT6639: only download sections where dlmode byte0 == 0x01,
+ * matching the Windows driver behavior which skips WiFi/other
+ * sections that would cause the chip to hang.
+ */
+ if (dev_id == 0x6639 && dl_size > 0 &&
+ (le32_to_cpu(sectionmap->bin_info_spec.dlmodecrctype) & 0xff) != 0x01)
+ continue;
+
if (dl_size > 0) {
retry = 20;
while (retry > 0) {
@@ -442,6 +455,22 @@ int btmtk_process_coredump(struct hci_dev *hdev, struct sk_buff *skb)
EXPORT_SYMBOL_GPL(btmtk_process_coredump);
#if IS_ENABLED(CONFIG_BT_HCIBTUSB_MTK)
+/* Known MT6639 (MT7927) Bluetooth USB devices.
+ * Used to scope the zero-CHIPID workaround to real MT6639 hardware,
+ * since some boards return 0x0000 from the MMIO chip ID register.
+ */
+static const struct {
+ u16 vendor;
+ u16 product;
+} btmtk_mt6639_devs[] = {
+ { 0x0489, 0xe13a }, /* ASUS ROG Crosshair X870E Hero */
+ { 0x0489, 0xe0fa }, /* Lenovo Legion Pro 7 16ARX9 */
+ { 0x0489, 0xe10f }, /* Gigabyte Z790 AORUS MASTER X */
+ { 0x0489, 0xe110 }, /* MSI X870E Ace Max */
+ { 0x0489, 0xe116 }, /* TP-Link Archer TBE550E */
+ { 0x13d3, 0x3588 }, /* ASUS ROG STRIX X870E-E */
+};
+
static void btmtk_usb_wmt_recv(struct urb *urb)
{
struct hci_dev *hdev = urb->context;
@@ -786,27 +815,21 @@ static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val)
static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val)
{
struct btmtk_data *data = hci_get_priv(hdev);
- int pipe, err, size = sizeof(u32);
- void *buf;
-
- buf = kzalloc(size, GFP_KERNEL);
- if (!buf)
- return -ENOMEM;
+ u8 buf[sizeof(u32)];
+ int err;
- pipe = usb_rcvctrlpipe(data->udev, 0);
- err = usb_control_msg(data->udev, pipe, 0x63,
- USB_TYPE_VENDOR | USB_DIR_IN,
- reg >> 16, reg & 0xffff,
- buf, size, USB_CTRL_GET_TIMEOUT);
+ *val = 0;
+ err = usb_control_msg_recv(data->udev, 0, 0x63,
+ USB_TYPE_VENDOR | USB_DIR_IN,
+ reg >> 16, reg & 0xffff,
+ buf, sizeof(buf), USB_CTRL_GET_TIMEOUT,
+ GFP_KERNEL);
if (err < 0)
- goto err_free_buf;
+ return err;
*val = get_unaligned_le32(buf);
-err_free_buf:
- kfree(buf);
-
- return err;
+ return 0;
}
static int btmtk_usb_id_get(struct hci_dev *hdev, u32 reg, u32 *id)
@@ -825,6 +848,7 @@ static u32 btmtk_usb_reset_done(struct hci_dev *hdev)
int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id)
{
+ int reset_err = 0;
u32 val;
int err;
@@ -847,7 +871,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id)
if (err < 0)
return err;
msleep(100);
- } else if (dev_id == 0x7925) {
+ } else if (dev_id == 0x7925 || dev_id == 0x6639) {
err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val);
if (err)
return err;
@@ -923,8 +947,10 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id)
err = readx_poll_timeout(btmtk_usb_reset_done, hdev, val,
val & MTK_BT_RST_DONE, 20000, 1000000);
- if (err < 0)
+ if (err < 0) {
bt_dev_err(hdev, "Reset timeout");
+ reset_err = err;
+ }
if (dev_id == 0x7922) {
err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT, 0x000000FF);
@@ -933,10 +959,12 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id)
}
err = btmtk_usb_id_get(hdev, 0x70010200, &val);
- if (err < 0 || !val)
+ if (err || (!val && dev_id != 0x6639)) {
bt_dev_err(hdev, "Can't get device id, subsys reset fail.");
+ return err ? err : -ENODEV;
+ }
- return err;
+ return reset_err;
}
EXPORT_SYMBOL_GPL(btmtk_usb_subsys_reset);
@@ -1295,30 +1323,48 @@ int btmtk_usb_setup(struct hci_dev *hdev)
calltime = ktime_get();
err = btmtk_usb_id_get(hdev, 0x80000008, &dev_id);
- if (err < 0) {
+ if (err) {
bt_dev_err(hdev, "Failed to get device id (%d)", err);
return err;
}
if (!dev_id || dev_id != 0x7663) {
err = btmtk_usb_id_get(hdev, 0x70010200, &dev_id);
- if (err < 0) {
+ if (err) {
bt_dev_err(hdev, "Failed to get device id (%d)", err);
return err;
}
err = btmtk_usb_id_get(hdev, 0x80021004, &fw_version);
- if (err < 0) {
+ if (err) {
bt_dev_err(hdev, "Failed to get fw version (%d)", err);
return err;
}
err = btmtk_usb_id_get(hdev, 0x70010020, &fw_flavor);
- if (err < 0) {
+ if (err) {
bt_dev_err(hdev, "Failed to get fw flavor (%d)", err);
return err;
}
fw_flavor = (fw_flavor & 0x00000080) >> 7;
}
+ if (!dev_id) {
+ u16 vid = le16_to_cpu(btmtk_data->udev->descriptor.idVendor);
+ u16 pid = le16_to_cpu(btmtk_data->udev->descriptor.idProduct);
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(btmtk_mt6639_devs); i++) {
+ if (vid == btmtk_mt6639_devs[i].vendor &&
+ pid == btmtk_mt6639_devs[i].product) {
+ dev_id = 0x6639;
+ break;
+ }
+ }
+
+ if (dev_id)
+ bt_dev_info(hdev, "MT6639: CHIPID=0x0000 with VID=%04x PID=%04x, using 0x6639",
+ vid, pid);
+ }
+
btmtk_data->dev_id = dev_id;
err = btmtk_register_coredump(hdev, btmtk_data->drv_name, fw_version);
@@ -1335,11 +1381,14 @@ int btmtk_usb_setup(struct hci_dev *hdev)
case 0x7922:
case 0x7925:
case 0x7961:
+ case 0x7902:
+ case 0x6639:
btmtk_fw_get_filename(fw_bin_name, sizeof(fw_bin_name), dev_id,
fw_version, fw_flavor);
err = btmtk_setup_firmware_79xx(hdev, fw_bin_name,
- btmtk_usb_hci_wmt_sync);
+ btmtk_usb_hci_wmt_sync,
+ dev_id);
if (err < 0) {
bt_dev_err(hdev, "Failed to set up firmware (%d)", err);
return err;
@@ -1506,3 +1555,4 @@ MODULE_FIRMWARE(FIRMWARE_MT7668);
MODULE_FIRMWARE(FIRMWARE_MT7922);
MODULE_FIRMWARE(FIRMWARE_MT7961);
MODULE_FIRMWARE(FIRMWARE_MT7925);
+MODULE_FIRMWARE(FIRMWARE_MT7927);
diff --git a/drivers/bluetooth/btmtk.h b/drivers/bluetooth/btmtk.h
index 5df7c3296624..8f40180ca92e 100644
--- a/drivers/bluetooth/btmtk.h
+++ b/drivers/bluetooth/btmtk.h
@@ -5,8 +5,10 @@
#define FIRMWARE_MT7663 "mediatek/mt7663pr2h.bin"
#define FIRMWARE_MT7668 "mediatek/mt7668pr2h.bin"
#define FIRMWARE_MT7922 "mediatek/BT_RAM_CODE_MT7922_1_1_hdr.bin"
+#define FIRMWARE_MT7902 "mediatek/BT_RAM_CODE_MT7902_1_1_hdr.bin"
#define FIRMWARE_MT7961 "mediatek/BT_RAM_CODE_MT7961_1_2_hdr.bin"
#define FIRMWARE_MT7925 "mediatek/mt7925/BT_RAM_CODE_MT7925_1_1_hdr.bin"
+#define FIRMWARE_MT7927 "mediatek/mt7927/BT_RAM_CODE_MT6639_2_1_hdr.bin"
#define HCI_EV_WMT 0xe4
#define HCI_WMT_MAX_EVENT_SIZE 64
@@ -187,7 +189,8 @@ typedef int (*wmt_cmd_sync_func_t)(struct hci_dev *,
int btmtk_set_bdaddr(struct hci_dev *hdev, const bdaddr_t *bdaddr);
int btmtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwname,
- wmt_cmd_sync_func_t wmt_cmd_sync);
+ wmt_cmd_sync_func_t wmt_cmd_sync,
+ u32 dev_id);
int btmtk_setup_firmware(struct hci_dev *hdev, const char *fwname,
wmt_cmd_sync_func_t wmt_cmd_sync);
@@ -226,7 +229,8 @@ static inline int btmtk_set_bdaddr(struct hci_dev *hdev,
static inline int btmtk_setup_firmware_79xx(struct hci_dev *hdev,
const char *fwname,
- wmt_cmd_sync_func_t wmt_cmd_sync)
+ wmt_cmd_sync_func_t wmt_cmd_sync,
+ u32 dev_id)
{
return -EOPNOTSUPP;
}
diff --git a/drivers/bluetooth/btmtksdio.c b/drivers/bluetooth/btmtksdio.c
index bd22a35c84eb..0e8fb0198143 100644
--- a/drivers/bluetooth/btmtksdio.c
+++ b/drivers/bluetooth/btmtksdio.c
@@ -42,24 +42,35 @@ struct btmtksdio_data {
const char *fwname;
u16 chipid;
bool lp_mbox_supported;
+ bool pm_runtime_supported;
};
static const struct btmtksdio_data mt7663_data = {
.fwname = FIRMWARE_MT7663,
.chipid = 0x7663,
.lp_mbox_supported = false,
+ .pm_runtime_supported = true,
};
static const struct btmtksdio_data mt7668_data = {
.fwname = FIRMWARE_MT7668,
.chipid = 0x7668,
.lp_mbox_supported = false,
+ .pm_runtime_supported = true,
};
static const struct btmtksdio_data mt7921_data = {
.fwname = FIRMWARE_MT7961,
.chipid = 0x7921,
.lp_mbox_supported = true,
+ .pm_runtime_supported = true,
+};
+
+static const struct btmtksdio_data mt7902_data = {
+ .fwname = FIRMWARE_MT7902,
+ .chipid = 0x7902,
+ .lp_mbox_supported = false,
+ .pm_runtime_supported = false,
};
static const struct sdio_device_id btmtksdio_table[] = {
@@ -69,6 +80,8 @@ static const struct sdio_device_id btmtksdio_table[] = {
.driver_data = (kernel_ulong_t)&mt7668_data },
{SDIO_DEVICE(SDIO_VENDOR_ID_MEDIATEK, SDIO_DEVICE_ID_MEDIATEK_MT7961),
.driver_data = (kernel_ulong_t)&mt7921_data },
+ {SDIO_DEVICE(SDIO_VENDOR_ID_MEDIATEK, SDIO_DEVICE_ID_MEDIATEK_MT7902),
+ .driver_data = (kernel_ulong_t)&mt7902_data },
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE(sdio, btmtksdio_table);
@@ -259,12 +272,24 @@ static int btmtksdio_tx_packet(struct btmtksdio_dev *bdev,
struct sk_buff *skb)
{
struct mtkbtsdio_hdr *sdio_hdr;
+ unsigned int len, pad_len;
int err;
- /* Make sure that there are enough rooms for SDIO header */
- if (unlikely(skb_headroom(skb) < sizeof(*sdio_hdr))) {
- err = pskb_expand_head(skb, sizeof(*sdio_hdr), 0,
- GFP_ATOMIC);
+ /* Make sure that the data buffer is not shared with anyone else and
+ * that there is enough room for the SDIO header
+ */
+ err = skb_cow_head(skb, sizeof(*sdio_hdr));
+ if (err < 0)
+ return err;
+
+ /* The transfer is rounded up to the SDIO block size, so the buffer
+ * has to provide tailroom for the padding as well
+ */
+ len = skb->len + sizeof(*sdio_hdr);
+ pad_len = round_up(len, MTK_SDIO_BLOCK_SIZE) - len;
+
+ if (unlikely(skb_tailroom(skb) < pad_len)) {
+ err = pskb_expand_head(skb, 0, pad_len, GFP_ATOMIC);
if (err < 0)
return err;
}
@@ -277,19 +302,22 @@ static int btmtksdio_tx_packet(struct btmtksdio_dev *bdev,
sdio_hdr->reserved = cpu_to_le16(0);
sdio_hdr->bt_type = hci_skb_pkt_type(skb);
+ /* Zero the padding so that no uninitialised memory is sent out */
+ skb_put_zero(skb, pad_len);
+
clear_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state);
- err = sdio_writesb(bdev->func, MTK_REG_CTDR, skb->data,
- round_up(skb->len, MTK_SDIO_BLOCK_SIZE));
+ err = sdio_writesb(bdev->func, MTK_REG_CTDR, skb->data, skb->len);
if (err < 0)
- goto err_skb_pull;
+ goto err_skb_restore;
- bdev->hdev->stat.byte_tx += skb->len;
+ bdev->hdev->stat.byte_tx += len;
kfree_skb(skb);
return 0;
-err_skb_pull:
+err_skb_restore:
+ skb_trim(skb, len);
skb_pull(skb, sizeof(*sdio_hdr));
return err;
@@ -871,7 +899,7 @@ static int mt79xx_setup(struct hci_dev *hdev, const char *fwname)
u8 param = 0x1;
int err;
- err = btmtk_setup_firmware_79xx(hdev, fwname, mtk_hci_wmt_sync);
+ err = btmtk_setup_firmware_79xx(hdev, fwname, mtk_hci_wmt_sync, 0);
if (err < 0) {
bt_dev_err(hdev, "Failed to setup 79xx firmware (%d)", err);
return err;
@@ -1091,6 +1119,7 @@ static int btmtksdio_setup(struct hci_dev *hdev)
set_bit(BTMTKSDIO_HW_TX_READY, &bdev->tx_state);
switch (bdev->data->chipid) {
+ case 0x7902:
case 0x7921:
if (test_bit(BTMTKSDIO_HW_RESET_ACTIVE, &bdev->tx_state)) {
err = btmtksdio_mtk_reg_read(hdev, MT7921_DLSTATUS,
@@ -1168,22 +1197,24 @@ static int btmtksdio_setup(struct hci_dev *hdev)
delta = ktime_sub(rettime, calltime);
duration = (unsigned long long)ktime_to_ns(delta) >> 10;
- pm_runtime_set_autosuspend_delay(bdev->dev,
- MTKBTSDIO_AUTOSUSPEND_DELAY);
- pm_runtime_use_autosuspend(bdev->dev);
+ if (bdev->data->pm_runtime_supported) {
+ pm_runtime_set_autosuspend_delay(bdev->dev,
+ MTKBTSDIO_AUTOSUSPEND_DELAY);
+ pm_runtime_use_autosuspend(bdev->dev);
- err = pm_runtime_set_active(bdev->dev);
- if (err < 0)
- return err;
+ err = pm_runtime_set_active(bdev->dev);
+ if (err < 0)
+ return err;
- /* Default forbid runtime auto suspend, that can be allowed by
- * enable_autosuspend flag or the PM runtime entry under sysfs.
- */
- pm_runtime_forbid(bdev->dev);
- pm_runtime_enable(bdev->dev);
+ /* Default forbid runtime auto suspend, that can be allowed by
+ * enable_autosuspend flag or the PM runtime entry under sysfs.
+ */
+ pm_runtime_forbid(bdev->dev);
+ pm_runtime_enable(bdev->dev);
- if (enable_autosuspend)
- pm_runtime_allow(bdev->dev);
+ if (enable_autosuspend)
+ pm_runtime_allow(bdev->dev);
+ }
bt_dev_info(hdev, "Device setup in %llu usecs", duration);
@@ -1452,6 +1483,9 @@ static void btmtksdio_remove(struct sdio_func *func)
if (test_bit(BTMTKSDIO_FUNC_ENABLED, &bdev->tx_state))
btmtksdio_close(hdev);
+ if (bdev->data->pm_runtime_supported)
+ pm_runtime_dont_use_autosuspend(bdev->dev);
+
/* Be consistent the state in btmtksdio_probe */
pm_runtime_get_noresume(bdev->dev);
diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
index 083a599eec74..c6709791d59c 100644
--- a/drivers/bluetooth/btqca.c
+++ b/drivers/bluetooth/btqca.c
@@ -996,8 +996,7 @@ int qca_set_bdaddr(struct hci_dev *hdev, const bdaddr_t *bdaddr)
baswap(&bdaddr_swapped, bdaddr);
skb = __hci_cmd_sync_ev(hdev, EDL_WRITE_BD_ADDR_OPCODE, 6,
- &bdaddr_swapped, HCI_EV_VENDOR,
- HCI_INIT_TIMEOUT);
+ &bdaddr_swapped, 0, HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
err = PTR_ERR(skb);
bt_dev_err(hdev, "QCA Change address cmd failed (%d)", err);
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index 047652598ab6..941f2968bced 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -949,6 +949,7 @@ struct btusb_data {
bool usb_alt6_packet_flow;
int isoc_altsetting;
int suspend_count;
+ const struct usb_device_id *match_id;
int (*recv_event)(struct hci_dev *hdev, struct sk_buff *skb);
int (*recv_acl)(struct hci_dev *hdev, struct sk_buff *skb);
@@ -2990,14 +2991,15 @@ static int btusb_set_bdaddr_ath3012(struct hci_dev *hdev,
static int btusb_set_bdaddr_wcn6855(struct hci_dev *hdev,
const bdaddr_t *bdaddr)
{
+ bdaddr_t bdaddr_swapped;
struct sk_buff *skb;
- u8 buf[6];
long ret;
- memcpy(buf, bdaddr, sizeof(bdaddr_t));
+ baswap(&bdaddr_swapped, bdaddr);
- skb = __hci_cmd_sync_ev(hdev, 0xfc14, sizeof(buf), buf,
- HCI_EV_CMD_COMPLETE, HCI_INIT_TIMEOUT);
+ skb = __hci_cmd_sync_ev(hdev, 0xfc14, sizeof(bdaddr_swapped),
+ &bdaddr_swapped, HCI_EV_CMD_COMPLETE,
+ HCI_INIT_TIMEOUT);
if (IS_ERR(skb)) {
ret = PTR_ERR(skb);
bt_dev_err(hdev, "Change address command failed (%ld)", ret);
@@ -3568,8 +3570,10 @@ static int btusb_setup_qca(struct hci_dev *hdev)
if (err)
return err;
- btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version);
- btdata->qca_dump.controller_id = le32_to_cpu(ver.rom_version);
+ if (btdata->match_id->driver_info & BTUSB_QCA_WCN6855) {
+ btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version);
+ btdata->qca_dump.controller_id = le32_to_cpu(ver.rom_version);
+ }
if (!(status & QCA_SYSCFG_UPDATED)) {
err = btusb_setup_qca_load_nvm(hdev, &ver, info);
@@ -3596,31 +3600,14 @@ static inline int __set_diag_interface(struct hci_dev *hdev)
{
struct btusb_data *data = hci_get_drvdata(hdev);
struct usb_interface *intf = data->diag;
- int i;
+ int ret;
if (!data->diag)
return -ENODEV;
- data->diag_tx_ep = NULL;
- data->diag_rx_ep = NULL;
-
- for (i = 0; i < intf->cur_altsetting->desc.bNumEndpoints; i++) {
- struct usb_endpoint_descriptor *ep_desc;
-
- ep_desc = &intf->cur_altsetting->endpoint[i].desc;
-
- if (!data->diag_tx_ep && usb_endpoint_is_bulk_out(ep_desc)) {
- data->diag_tx_ep = ep_desc;
- continue;
- }
-
- if (!data->diag_rx_ep && usb_endpoint_is_bulk_in(ep_desc)) {
- data->diag_rx_ep = ep_desc;
- continue;
- }
- }
-
- if (!data->diag_tx_ep || !data->diag_rx_ep) {
+ ret = usb_find_common_endpoints(intf->cur_altsetting, &data->diag_rx_ep,
+ &data->diag_tx_ep, NULL, NULL);
+ if (ret) {
bt_dev_err(hdev, "invalid diagnostic descriptors");
return -ENODEV;
}
@@ -3821,12 +3808,11 @@ static const struct file_operations force_poll_sync_fops = {
static int btusb_probe(struct usb_interface *intf,
const struct usb_device_id *id)
{
- struct usb_endpoint_descriptor *ep_desc;
struct gpio_desc *reset_gpio;
struct btusb_data *data;
struct hci_dev *hdev;
unsigned ifnum_base;
- int i, err, priv_size;
+ int err, priv_size;
BT_DBG("intf %p id %p", intf, id);
@@ -3863,26 +3849,10 @@ static int btusb_probe(struct usb_interface *intf,
if (!data)
return -ENOMEM;
- for (i = 0; i < intf->cur_altsetting->desc.bNumEndpoints; i++) {
- ep_desc = &intf->cur_altsetting->endpoint[i].desc;
-
- if (!data->intr_ep && usb_endpoint_is_int_in(ep_desc)) {
- data->intr_ep = ep_desc;
- continue;
- }
-
- if (!data->bulk_tx_ep && usb_endpoint_is_bulk_out(ep_desc)) {
- data->bulk_tx_ep = ep_desc;
- continue;
- }
-
- if (!data->bulk_rx_ep && usb_endpoint_is_bulk_in(ep_desc)) {
- data->bulk_rx_ep = ep_desc;
- continue;
- }
- }
-
- if (!data->intr_ep || !data->bulk_tx_ep || !data->bulk_rx_ep) {
+ data->match_id = id;
+ err = usb_find_common_endpoints(intf->cur_altsetting, &data->bulk_rx_ep,
+ &data->bulk_tx_ep, &data->intr_ep, NULL);
+ if (err) {
kfree(data);
return -ENODEV;
}
diff --git a/drivers/bluetooth/virtio_bt.c b/drivers/bluetooth/virtio_bt.c
index 0a602f4cbccc..f9b124fe109b 100644
--- a/drivers/bluetooth/virtio_bt.c
+++ b/drivers/bluetooth/virtio_bt.c
@@ -120,9 +120,13 @@ static int virtbt_setup_zephyr(struct hci_dev *hdev)
if (IS_ERR(skb))
return PTR_ERR(skb);
- bt_dev_info(hdev, "%s", (char *)(skb->data + 1));
+ /* Bounded print: the backend controls skb->len. */
+ if (skb->len > 1) {
+ int len = skb->len - 1;
- hci_set_fw_info(hdev, "%s", skb->data + 1);
+ bt_dev_info(hdev, "%.*s", len, (char *)(skb->data + 1));
+ hci_set_fw_info(hdev, "%.*s", len, skb->data + 1);
+ }
kfree_skb(skb);
return 0;
diff --git a/drivers/bus/mhi/ep/main.c b/drivers/bus/mhi/ep/main.c
index 2f58ad0f14b6..fd1d7c84fb1c 100644
--- a/drivers/bus/mhi/ep/main.c
+++ b/drivers/bus/mhi/ep/main.c
@@ -1340,14 +1340,19 @@ static int mhi_ep_create_device(struct mhi_ep_cntrl *mhi_cntrl, u32 ch_id)
ret = dev_set_name(&mhi_dev->dev, "%s_%s",
dev_name(&mhi_cntrl->mhi_dev->dev),
mhi_dev->name);
- if (ret) {
- put_device(&mhi_dev->dev);
- return ret;
- }
+ if (ret)
+ goto err_put_channels;
ret = device_add(&mhi_dev->dev);
if (ret)
- put_device(&mhi_dev->dev);
+ goto err_put_channels;
+
+ return 0;
+
+err_put_channels:
+ put_device(&mhi_dev->dev); /* DL channel reference */
+ put_device(&mhi_dev->dev); /* UL channel reference */
+ put_device(&mhi_dev->dev); /* device_initialize() reference */
return ret;
}
diff --git a/drivers/bus/mhi/host/init.c b/drivers/bus/mhi/host/init.c
index 223fc54c45b3..9a917f14ab1b 100644
--- a/drivers/bus/mhi/host/init.c
+++ b/drivers/bus/mhi/host/init.c
@@ -1042,7 +1042,7 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl,
if (mhi_cntrl->edl_trigger) {
ret = sysfs_create_file(&mhi_dev->dev.kobj, &dev_attr_trigger_edl.attr);
if (ret)
- goto err_release_dev;
+ goto err_del_dev;
}
mhi_cntrl->mhi_dev = mhi_dev;
@@ -1051,6 +1051,8 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl,
return 0;
+err_del_dev:
+ device_del(&mhi_dev->dev);
err_release_dev:
put_device(&mhi_dev->dev);
error_setup_irq:
diff --git a/drivers/bus/mhi/host/main.c b/drivers/bus/mhi/host/main.c
index 45ec1b585577..846aee2a42a0 100644
--- a/drivers/bus/mhi/host/main.c
+++ b/drivers/bus/mhi/host/main.c
@@ -170,6 +170,9 @@ EXPORT_SYMBOL_GPL(mhi_get_mhi_state);
void mhi_soc_reset(struct mhi_controller *mhi_cntrl)
{
+ int __maybe_unused ret;
+ u32 tmp;
+
if (mhi_cntrl->reset) {
mhi_cntrl->reset(mhi_cntrl);
return;
@@ -178,6 +181,9 @@ void mhi_soc_reset(struct mhi_controller *mhi_cntrl)
/* Generic MHI SoC reset */
mhi_write_reg(mhi_cntrl, mhi_cntrl->regs, MHI_SOC_RESET_REQ_OFFSET,
MHI_SOC_RESET_REQ);
+ /* Flush the posted write to the device (ignore return value) */
+ ret = mhi_read_reg(mhi_cntrl, mhi_cntrl->regs, MHI_SOC_RESET_REQ_OFFSET,
+ &tmp);
}
EXPORT_SYMBOL_GPL(mhi_soc_reset);
diff --git a/drivers/bus/qcom-ebi2.c b/drivers/bus/qcom-ebi2.c
index c1fef1b4bd89..8d2eb955dc92 100644
--- a/drivers/bus/qcom-ebi2.c
+++ b/drivers/bus/qcom-ebi2.c
@@ -292,7 +292,6 @@ static void qcom_ebi2_setup_chipselect(struct device_node *np,
static int qcom_ebi2_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
- struct device_node *child;
struct device *dev = &pdev->dev;
struct resource *res;
void __iomem *ebi2_base;
@@ -303,41 +302,23 @@ static int qcom_ebi2_probe(struct platform_device *pdev)
u32 val;
int ret;
- ebi2xclk = devm_clk_get(dev, "ebi2x");
+ ebi2xclk = devm_clk_get_enabled(dev, "ebi2x");
if (IS_ERR(ebi2xclk))
return PTR_ERR(ebi2xclk);
- ret = clk_prepare_enable(ebi2xclk);
- if (ret) {
- dev_err(dev, "could not enable EBI2X clk (%d)\n", ret);
- return ret;
- }
-
- ebi2clk = devm_clk_get(dev, "ebi2");
- if (IS_ERR(ebi2clk)) {
- ret = PTR_ERR(ebi2clk);
- goto err_disable_2x_clk;
- }
-
- ret = clk_prepare_enable(ebi2clk);
- if (ret) {
- dev_err(dev, "could not enable EBI2 clk\n");
- goto err_disable_2x_clk;
- }
+ ebi2clk = devm_clk_get_enabled(dev, "ebi2");
+ if (IS_ERR(ebi2clk))
+ return PTR_ERR(ebi2clk);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
ebi2_base = devm_ioremap_resource(dev, res);
- if (IS_ERR(ebi2_base)) {
- ret = PTR_ERR(ebi2_base);
- goto err_disable_clk;
- }
+ if (IS_ERR(ebi2_base))
+ return PTR_ERR(ebi2_base);
res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
ebi2_xmem = devm_ioremap_resource(dev, res);
- if (IS_ERR(ebi2_xmem)) {
- ret = PTR_ERR(ebi2_xmem);
- goto err_disable_clk;
- }
+ if (IS_ERR(ebi2_xmem))
+ return PTR_ERR(ebi2_xmem);
/* Allegedly this turns the power save mode off */
writel(0UL, ebi2_xmem + EBI2_XMEM_CFG);
@@ -348,15 +329,13 @@ static int qcom_ebi2_probe(struct platform_device *pdev)
writel(val, ebi2_base);
/* Walk over the child nodes and see what chipselects we use */
- for_each_available_child_of_node(np, child) {
+ for_each_available_child_of_node_scoped(np, child) {
u32 csindex;
/* Figure out the chipselect */
ret = of_property_read_u32(child, "reg", &csindex);
- if (ret) {
- of_node_put(child);
+ if (ret)
return ret;
- }
if (csindex > 5) {
dev_err(dev,
@@ -376,15 +355,9 @@ static int qcom_ebi2_probe(struct platform_device *pdev)
}
if (have_children)
- return of_platform_default_populate(np, NULL, dev);
- return 0;
-
-err_disable_clk:
- clk_disable_unprepare(ebi2clk);
-err_disable_2x_clk:
- clk_disable_unprepare(ebi2xclk);
+ return devm_of_platform_populate(dev);
- return ret;
+ return 0;
}
static const struct of_device_id qcom_ebi2_of_match[] = {
diff --git a/drivers/bus/ti-sysc.c b/drivers/bus/ti-sysc.c
index 27149eb29afb..382f7bc2e1eb 100644
--- a/drivers/bus/ti-sysc.c
+++ b/drivers/bus/ti-sysc.c
@@ -682,6 +682,7 @@ static struct device_node *stdout_path;
static void sysc_init_stdout_path(struct sysc *ddata)
{
+ struct device_node *chosen;
struct device_node *np = NULL;
const char *uart;
@@ -691,15 +692,18 @@ static void sysc_init_stdout_path(struct sysc *ddata)
if (stdout_path)
return;
- np = of_find_node_by_path("/chosen");
- if (!np)
+ chosen = of_find_node_by_path("/chosen");
+ if (!chosen)
goto err;
- uart = of_get_property(np, "stdout-path", NULL);
- if (!uart)
+ uart = of_get_property(chosen, "stdout-path", NULL);
+ if (!uart) {
+ of_node_put(chosen);
goto err;
+ }
np = of_find_node_by_path(uart);
+ of_node_put(chosen);
if (!np)
goto err;
diff --git a/drivers/char/hw_random/core.c b/drivers/char/hw_random/core.c
index c6e913497e1d..008f81a82131 100644
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -597,11 +597,13 @@ int hwrng_register(struct hwrng *rng)
*/
err = set_current_rng(rng);
if (err)
- goto out_unlock;
+ goto out_list_del;
}
}
mutex_unlock(&rng_mutex);
return 0;
+out_list_del:
+ list_del_init(&rng->list);
out_unlock:
mutex_unlock(&rng_mutex);
out:
diff --git a/drivers/char/hw_random/ks-sa-rng.c b/drivers/char/hw_random/ks-sa-rng.c
index 3c514b4fbc8a..64369d4f1644 100644
--- a/drivers/char/hw_random/ks-sa-rng.c
+++ b/drivers/char/hw_random/ks-sa-rng.c
@@ -242,7 +242,14 @@ static int ks_sa_rng_probe(struct platform_device *pdev)
return dev_err_probe(dev, ret, "Failed to enable SA power-domain\n");
}
- return devm_hwrng_register(&pdev->dev, &ks_sa_rng->rng);
+ ret = devm_hwrng_register(dev, &ks_sa_rng->rng);
+ if (ret) {
+ pm_runtime_put_sync(dev);
+ pm_runtime_disable(dev);
+ return ret;
+ }
+
+ return 0;
}
static void ks_sa_rng_remove(struct platform_device *pdev)
diff --git a/drivers/char/ppdev.c b/drivers/char/ppdev.c
index d1dfbd8d4d42..3dec6516a5eb 100644
--- a/drivers/char/ppdev.c
+++ b/drivers/char/ppdev.c
@@ -340,15 +340,17 @@ static enum ieee1284_phase init_phase(int mode)
return IEEE1284_PH_FWD_IDLE;
}
-static int pp_set_timeout(struct pardevice *pdev, long tv_sec, int tv_usec)
+static int pp_set_timeout(struct pardevice *pdev, s64 tv_sec, s64 tv_usec)
{
+ struct timespec64 ts;
long to_jiffies;
- if ((tv_sec < 0) || (tv_usec < 0))
+ if (tv_sec < 0 || tv_usec < 0 || tv_usec >= USEC_PER_SEC)
return -EINVAL;
- to_jiffies = usecs_to_jiffies(tv_usec);
- to_jiffies += tv_sec * HZ;
+ ts.tv_sec = tv_sec;
+ ts.tv_nsec = tv_usec * NSEC_PER_USEC;
+ to_jiffies = timespec64_to_jiffies(&ts);
if (to_jiffies <= 0)
return -EINVAL;
diff --git a/drivers/char/tpm/st33zp24/st33zp24.c b/drivers/char/tpm/st33zp24/st33zp24.c
index 06caf53a42ee..e3513015acae 100644
--- a/drivers/char/tpm/st33zp24/st33zp24.c
+++ b/drivers/char/tpm/st33zp24/st33zp24.c
@@ -93,7 +93,9 @@ static u8 st33zp24_status(struct tpm_chip *chip)
struct st33zp24_dev *tpm_dev = dev_get_drvdata(&chip->dev);
u8 data;
- tpm_dev->ops->recv(tpm_dev->phy_id, TPM_STS, &data, 1);
+ if (tpm_dev->ops->recv(tpm_dev->phy_id, TPM_STS, &data, 1) != 1)
+ return 0;
+
return data;
}
@@ -104,10 +106,10 @@ static bool check_locality(struct tpm_chip *chip)
{
struct st33zp24_dev *tpm_dev = dev_get_drvdata(&chip->dev);
u8 data;
- u8 status;
+ int status;
status = tpm_dev->ops->recv(tpm_dev->phy_id, TPM_ACCESS, &data, 1);
- if (status && (data &
+ if (status == 1 && (data &
(TPM_ACCESS_ACTIVE_LOCALITY | TPM_ACCESS_VALID)) ==
(TPM_ACCESS_ACTIVE_LOCALITY | TPM_ACCESS_VALID))
return true;
diff --git a/drivers/char/xilinx_hwicap/xilinx_hwicap.c b/drivers/char/xilinx_hwicap/xilinx_hwicap.c
index 4f6c3cb8aa41..259900a9ee21 100644
--- a/drivers/char/xilinx_hwicap/xilinx_hwicap.c
+++ b/drivers/char/xilinx_hwicap/xilinx_hwicap.c
@@ -760,7 +760,7 @@ static int __init hwicap_module_init(void)
HWICAP_DEVICES,
DRIVER_NAME);
if (retval < 0)
- return retval;
+ goto failed_class;
retval = platform_driver_register(&hwicap_platform_driver);
if (retval)
@@ -771,6 +771,9 @@ static int __init hwicap_module_init(void)
failed:
unregister_chrdev_region(devt, HWICAP_DEVICES);
+ failed_class:
+ class_unregister(&icap_class);
+
return retval;
}
diff --git a/drivers/clk/clk-devres.c b/drivers/clk/clk-devres.c
index 82ae1f26e634..a9ceb84a45f8 100644
--- a/drivers/clk/clk-devres.c
+++ b/drivers/clk/clk-devres.c
@@ -99,6 +99,11 @@ struct clk *devm_clk_get_optional_enabled(struct device *dev, const char *id)
}
EXPORT_SYMBOL_GPL(devm_clk_get_optional_enabled);
+static void devm_clk_disable_unprepare(void *data)
+{
+ clk_disable_unprepare(data);
+}
+
struct clk *devm_clk_get_optional_enabled_with_rate(struct device *dev,
const char *id,
unsigned long rate)
@@ -106,8 +111,7 @@ struct clk *devm_clk_get_optional_enabled_with_rate(struct device *dev,
struct clk *clk;
int ret;
- clk = __devm_clk_get(dev, id, clk_get_optional, NULL,
- clk_disable_unprepare);
+ clk = devm_clk_get_optional(dev, id);
if (IS_ERR(clk))
return ERR_CAST(clk);
@@ -119,6 +123,10 @@ struct clk *devm_clk_get_optional_enabled_with_rate(struct device *dev,
if (ret)
goto out_put_clk;
+ ret = devm_add_action_or_reset(dev, devm_clk_disable_unprepare, clk);
+ if (ret)
+ goto out_put_clk;
+
return clk;
out_put_clk:
diff --git a/drivers/clk/clk-moxart.c b/drivers/clk/clk-moxart.c
index 3786a0153ad1..40663ef3ef0a 100644
--- a/drivers/clk/clk-moxart.c
+++ b/drivers/clk/clk-moxart.c
@@ -17,7 +17,6 @@ static void __init moxart_of_pll_clk_init(struct device_node *node)
{
void __iomem *base;
struct clk_hw *hw;
- struct clk *ref_clk;
unsigned int mul;
const char *name = node->name;
const char *parent_name;
@@ -34,12 +33,6 @@ static void __init moxart_of_pll_clk_init(struct device_node *node)
mul = readl(base + 0x30) >> 3 & 0x3f;
iounmap(base);
- ref_clk = of_clk_get(node, 0);
- if (IS_ERR(ref_clk)) {
- pr_err("%pOF: of_clk_get failed\n", node);
- return;
- }
-
hw = clk_hw_register_fixed_factor(NULL, name, parent_name, 0, mul, 1);
if (IS_ERR(hw)) {
pr_err("%pOF: failed to register clock\n", node);
@@ -56,7 +49,6 @@ static void __init moxart_of_apb_clk_init(struct device_node *node)
{
void __iomem *base;
struct clk_hw *hw;
- struct clk *pll_clk;
unsigned int div, val;
unsigned int div_idx[] = { 2, 3, 4, 6, 8};
const char *name = node->name;
@@ -78,12 +70,6 @@ static void __init moxart_of_apb_clk_init(struct device_node *node)
val = 0;
div = div_idx[val] * 2;
- pll_clk = of_clk_get(node, 0);
- if (IS_ERR(pll_clk)) {
- pr_err("%pOF: of_clk_get failed\n", node);
- return;
- }
-
hw = clk_hw_register_fixed_factor(NULL, name, parent_name, 0, 1, div);
if (IS_ERR(hw)) {
pr_err("%pOF: failed to register clock\n", node);
diff --git a/drivers/clk/clk-palmas.c b/drivers/clk/clk-palmas.c
index 39049f62dbbb..86a51edac827 100644
--- a/drivers/clk/clk-palmas.c
+++ b/drivers/clk/clk-palmas.c
@@ -194,6 +194,13 @@ static void palmas_clks_get_clk_data(struct platform_device *pdev,
cinfo->ext_control_pin = prop;
}
+static void palmas_clks_unprepare_ext_control(void *data)
+{
+ struct palmas_clock_info *cinfo = data;
+
+ clk_unprepare(cinfo->hw.clk);
+}
+
static int palmas_clks_init_configure(struct palmas_clock_info *cinfo)
{
int ret;
@@ -214,13 +221,18 @@ static int palmas_clks_init_configure(struct palmas_clock_info *cinfo)
return ret;
}
+ ret = devm_add_action_or_reset(cinfo->dev,
+ palmas_clks_unprepare_ext_control,
+ cinfo);
+ if (ret)
+ return ret;
+
ret = palmas_ext_control_req_config(cinfo->palmas,
cinfo->clk_desc->sleep_reqstr_id,
cinfo->ext_control_pin, true);
if (ret < 0) {
dev_err(cinfo->dev, "Ext config for %s failed, %d\n",
cinfo->clk_desc->clk_name, ret);
- clk_unprepare(cinfo->hw.clk);
return ret;
}
}
diff --git a/drivers/clk/clk-versaclock7.c b/drivers/clk/clk-versaclock7.c
index f323263e32c3..e1d6d88b6635 100644
--- a/drivers/clk/clk-versaclock7.c
+++ b/drivers/clk/clk-versaclock7.c
@@ -1191,7 +1191,7 @@ static int vc7_probe(struct i2c_client *client)
if (ret) {
dev_err_probe(&client->dev, ret,
"unable to register output %d\n", i);
- return ret;
+ goto err_clk;
}
switch (bank_src_map.type) {
diff --git a/drivers/clk/hisilicon/reset.c b/drivers/clk/hisilicon/reset.c
index 93cee17db8b1..c3b7daac9313 100644
--- a/drivers/clk/hisilicon/reset.c
+++ b/drivers/clk/hisilicon/reset.c
@@ -91,7 +91,7 @@ struct hisi_reset_controller *hisi_reset_init(struct platform_device *pdev)
{
struct hisi_reset_controller *rstc;
- rstc = devm_kmalloc(&pdev->dev, sizeof(*rstc), GFP_KERNEL);
+ rstc = devm_kzalloc(&pdev->dev, sizeof(*rstc), GFP_KERNEL);
if (!rstc)
return NULL;
diff --git a/drivers/clk/imx/clk-scu.c b/drivers/clk/imx/clk-scu.c
index 75a6d91c0711..ffbb05fb91bc 100644
--- a/drivers/clk/imx/clk-scu.c
+++ b/drivers/clk/imx/clk-scu.c
@@ -483,7 +483,6 @@ struct clk_hw *__imx_clk_scu(struct device *dev, const char *name,
clk->clk_type = clk_type;
init.name = name;
- init.ops = &clk_scu_ops;
if (rsrc_id == IMX_SC_R_A35 || rsrc_id == IMX_SC_R_A53 || rsrc_id == IMX_SC_R_A72)
init.ops = &clk_scu_cpu_ops;
else if (rsrc_id == IMX_SC_R_PI_0_PLL)
diff --git a/drivers/clk/mediatek/clk-mt8135.c b/drivers/clk/mediatek/clk-mt8135.c
index 084e48a554c2..1d20e15608f7 100644
--- a/drivers/clk/mediatek/clk-mt8135.c
+++ b/drivers/clk/mediatek/clk-mt8135.c
@@ -409,6 +409,9 @@ static const struct mtk_gate_regs infra_cg_regs = {
GATE_MTK_FLAGS(_id, _name, _parent, &infra_cg_regs, _shift, \
&mtk_clk_gate_ops_setclr, CLK_IS_CRITICAL)
+#define GATE_ICG_INV(_id, _name, _parent, _shift) \
+ GATE_MTK(_id, _name, _parent, &infra_cg_regs, _shift, &mtk_clk_gate_ops_setclr_inv)
+
static const struct mtk_gate infra_clks[] = {
GATE_DUMMY(CLK_DUMMY, "infra_dummy"),
GATE_ICG(CLK_INFRA_PMIC_WRAP, "pmic_wrap_ck", "axi_sel", 23),
@@ -419,7 +422,7 @@ static const struct mtk_gate infra_clks[] = {
GATE_ICG(CLK_INFRA_CPUM, "cpum_ck", "cpum_tck_in", 15),
GATE_ICG_AO(CLK_INFRA_M4U, "m4u_ck", "mem_sel", 8),
GATE_ICG(CLK_INFRA_MFGAXI, "mfgaxi_ck", "axi_sel", 7),
- GATE_ICG(CLK_INFRA_DEVAPC, "devapc_ck", "axi_sel", 6),
+ GATE_ICG_INV(CLK_INFRA_DEVAPC, "devapc_ck", "axi_sel", 6),
GATE_ICG(CLK_INFRA_AUDIO, "audio_ck", "aud_intbus_sel", 5),
GATE_ICG(CLK_INFRA_MFG_BUS, "mfg_bus_ck", "axi_sel", 2),
GATE_ICG(CLK_INFRA_SMI, "smi_ck", "smi_sel", 1),
diff --git a/drivers/clk/meson/gxbb.c b/drivers/clk/meson/gxbb.c
index 2bfb0ab9c93c..39a6a31e796f 100644
--- a/drivers/clk/meson/gxbb.c
+++ b/drivers/clk/meson/gxbb.c
@@ -1305,7 +1305,7 @@ static struct clk_regmap gxbb_32k_clk_sel = {
.name = "32k_clk_sel",
.ops = &clk_regmap_mux_ops,
.parent_data = gxbb_32k_clk_parent_data,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gxbb_32k_clk_parent_data),
.flags = CLK_SET_RATE_PARENT,
},
};
diff --git a/drivers/clk/nuvoton/clk-ma35d1-pll.c b/drivers/clk/nuvoton/clk-ma35d1-pll.c
index ff3fb8b87c24..c7c0dc91a012 100644
--- a/drivers/clk/nuvoton/clk-ma35d1-pll.c
+++ b/drivers/clk/nuvoton/clk-ma35d1-pll.c
@@ -48,7 +48,7 @@
#define PLL_CTL1_PD BIT(0)
#define PLL_CTL1_BP BIT(1)
#define PLL_CTL1_OUTDIV GENMASK(6, 4)
-#define PLL_CTL1_FRAC GENMASK(31, 24)
+#define PLL_CTL1_FRAC GENMASK(31, 8)
#define PLL_CTL2_SLOPE GENMASK(23, 0)
#define INDIV_MIN 1
@@ -92,7 +92,7 @@ static unsigned long ma35d1_calc_smic_pll_freq(u32 pll0_ctl0,
p = FIELD_GET(SPLL0_CTL0_OUTDIV, pll0_ctl0);
outdiv = 1 << p;
pll_freq = (u64)parent_rate * n;
- div_u64(pll_freq, m * outdiv);
+ pll_freq = div_u64(pll_freq, m * outdiv);
return pll_freq;
}
@@ -110,12 +110,12 @@ static unsigned long ma35d1_calc_pll_freq(u8 mode, u32 *reg_ctl, unsigned long p
if (mode == PLL_MODE_INT) {
pll_freq = (u64)parent_rate * n;
- div_u64(pll_freq, m * p);
+ pll_freq = div_u64(pll_freq, m * p);
} else {
x = FIELD_GET(PLL_CTL1_FRAC, reg_ctl[1]);
- /* 2 decimal places floating to integer (ex. 1.23 to 123) */
- n = n * 100 + ((x * 100) / FIELD_MAX(PLL_CTL1_FRAC));
- pll_freq = div_u64(parent_rate * n, 100 * m * p);
+ /* convert 24-bit fraction to 3 decimal digits, rounding to closest */
+ n = n * 1000 + DIV_ROUND_CLOSEST_ULL((u64)x * 1000, 1ULL << 24);
+ pll_freq = div_u64((u64)parent_rate * n, 1000 * m * p);
}
return pll_freq;
}
@@ -244,35 +244,43 @@ static unsigned long ma35d1_clk_pll_recalc_rate(struct clk_hw *hw, unsigned long
return 0;
}
-static long ma35d1_clk_pll_round_rate(struct clk_hw *hw, unsigned long rate,
- unsigned long *parent_rate)
+static int ma35d1_clk_pll_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
{
struct ma35d1_clk_pll *pll = to_ma35d1_clk_pll(hw);
u32 reg_ctl[3] = { 0 };
unsigned long pll_freq;
long ret;
- if (*parent_rate < PLL_FREF_MIN_FREQ || *parent_rate > PLL_FREF_MAX_FREQ)
+ if (req->best_parent_rate < PLL_FREF_MIN_FREQ || req->best_parent_rate > PLL_FREF_MAX_FREQ)
return -EINVAL;
- ret = ma35d1_pll_find_closest(pll, rate, *parent_rate, reg_ctl, &pll_freq);
- if (ret < 0)
- return ret;
-
switch (pll->id) {
case CAPLL:
- reg_ctl[0] = readl_relaxed(pll->ctl0_base);
- pll_freq = ma35d1_calc_smic_pll_freq(reg_ctl[0], *parent_rate);
- return pll_freq;
case DDRPLL:
+ /* Read-only PLLs: return current rate */
+ reg_ctl[0] = readl_relaxed(pll->ctl0_base);
+ if (pll->id == CAPLL) {
+ pll_freq = ma35d1_calc_smic_pll_freq(reg_ctl[0], req->best_parent_rate);
+ } else {
+ reg_ctl[1] = readl_relaxed(pll->ctl1_base);
+ pll_freq = ma35d1_calc_pll_freq(pll->mode, reg_ctl, req->best_parent_rate);
+ }
+ req->rate = pll_freq;
+ return 0;
case APLL:
case EPLL:
case VPLL:
- reg_ctl[0] = readl_relaxed(pll->ctl0_base);
- reg_ctl[1] = readl_relaxed(pll->ctl1_base);
- pll_freq = ma35d1_calc_pll_freq(pll->mode, reg_ctl, *parent_rate);
- return pll_freq;
+ /* Configurable PLLs: find closest achievable rate */
+ ret = ma35d1_pll_find_closest(pll, req->rate, req->best_parent_rate,
+ reg_ctl, &pll_freq);
+ if (ret < 0)
+ return ret;
+ req->rate = pll_freq;
+ return 0;
}
+
+ req->rate = 0;
return 0;
}
@@ -311,12 +319,12 @@ static const struct clk_ops ma35d1_clk_pll_ops = {
.unprepare = ma35d1_clk_pll_unprepare,
.set_rate = ma35d1_clk_pll_set_rate,
.recalc_rate = ma35d1_clk_pll_recalc_rate,
- .round_rate = ma35d1_clk_pll_round_rate,
+ .determine_rate = ma35d1_clk_pll_determine_rate,
};
static const struct clk_ops ma35d1_clk_fixed_pll_ops = {
.recalc_rate = ma35d1_clk_pll_recalc_rate,
- .round_rate = ma35d1_clk_pll_round_rate,
+ .determine_rate = ma35d1_clk_pll_determine_rate,
};
struct clk_hw *ma35d1_reg_clk_pll(struct device *dev, u32 id, u8 u8mode, const char *name,
diff --git a/drivers/clk/qcom/camcc-sc8280xp.c b/drivers/clk/qcom/camcc-sc8280xp.c
index 479964f91608..cf170fc7baf1 100644
--- a/drivers/clk/qcom/camcc-sc8280xp.c
+++ b/drivers/clk/qcom/camcc-sc8280xp.c
@@ -1754,24 +1754,6 @@ static struct clk_branch camcc_csiphy3_clk = {
},
};
-static struct clk_branch camcc_gdsc_clk = {
- .halt_reg = 0xc1e4,
- .halt_check = BRANCH_HALT,
- .clkr = {
- .enable_reg = 0xc1e4,
- .enable_mask = BIT(0),
- .hw.init = &(struct clk_init_data){
- .name = "camcc_gdsc_clk",
- .parent_hws = (const struct clk_hw*[]){
- &camcc_xo_clk_src.clkr.hw,
- },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
- .ops = &clk_branch2_ops,
- },
- },
-};
-
static struct clk_branch camcc_icp_ahb_clk = {
.halt_reg = 0xc0d8,
.halt_check = BRANCH_HALT,
@@ -2840,7 +2822,6 @@ static struct clk_regmap *camcc_sc8280xp_clocks[] = {
[CAMCC_CSIPHY2_CLK] = &camcc_csiphy2_clk.clkr,
[CAMCC_CSIPHY3_CLK] = &camcc_csiphy3_clk.clkr,
[CAMCC_FAST_AHB_CLK_SRC] = &camcc_fast_ahb_clk_src.clkr,
- [CAMCC_GDSC_CLK] = &camcc_gdsc_clk.clkr,
[CAMCC_ICP_AHB_CLK] = &camcc_icp_ahb_clk.clkr,
[CAMCC_ICP_CLK] = &camcc_icp_clk.clkr,
[CAMCC_ICP_CLK_SRC] = &camcc_icp_clk_src.clkr,
diff --git a/drivers/clk/qcom/common.c b/drivers/clk/qcom/common.c
index e1a83ec48150..8061d0bf590a 100644
--- a/drivers/clk/qcom/common.c
+++ b/drivers/clk/qcom/common.c
@@ -165,7 +165,7 @@ static int _qcom_cc_register_board_clk(struct device *dev, const char *path,
if (!node) {
fixed = devm_kzalloc(dev, sizeof(*fixed), GFP_KERNEL);
if (!fixed)
- return -EINVAL;
+ return -ENOMEM;
fixed->fixed_rate = rate;
fixed->hw.init = &init_data;
@@ -182,7 +182,7 @@ static int _qcom_cc_register_board_clk(struct device *dev, const char *path,
if (add_factor) {
factor = devm_kzalloc(dev, sizeof(*factor), GFP_KERNEL);
if (!factor)
- return -EINVAL;
+ return -ENOMEM;
factor->mult = factor->div = 1;
factor->hw.init = &init_data;
diff --git a/drivers/clk/qcom/gcc-mdm9607.c b/drivers/clk/qcom/gcc-mdm9607.c
index 07f1b78d737a..4b749c99586b 100644
--- a/drivers/clk/qcom/gcc-mdm9607.c
+++ b/drivers/clk/qcom/gcc-mdm9607.c
@@ -26,7 +26,6 @@
enum {
P_XO,
- P_BIMC,
P_GPLL0,
P_GPLL1,
P_GPLL2,
@@ -121,7 +120,7 @@ static struct clk_alpha_pll gpll2_early = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_DEFAULT],
.clkr = {
.enable_reg = 0x45000,
- .enable_mask = BIT(3), /* Yeah, apparently it's not 2 */
+ .enable_mask = BIT(3), /* BIT(2) is used for BIMC PLL */
.hw.init = &(struct clk_init_data)
{
.name = "gpll2_early",
@@ -158,20 +157,6 @@ static const struct clk_parent_data gcc_xo_gpll0_gpll2[] = {
{ .hw = &gpll2.clkr.hw },
};
-static const struct parent_map gcc_xo_gpll0_gpll1_gpll2_map[] = {
- { P_XO, 0 },
- { P_GPLL0, 1 },
- { P_GPLL1, 2 },
- { P_GPLL2, 3 },
-};
-
-static const struct clk_parent_data gcc_xo_gpll0_gpll1_gpll2[] = {
- { .fw_name = "xo" },
- { .hw = &gpll0.clkr.hw },
- { .hw = &gpll1_vote.hw },
- { .hw = &gpll2.clkr.hw },
-};
-
static const struct freq_tbl ftbl_apss_ahb_clk[] = {
F(19200000, P_XO, 1, 0, 0),
F(50000000, P_GPLL0, 16, 0, 0),
@@ -192,80 +177,6 @@ static struct clk_rcg2 apss_ahb_clk_src = {
},
};
-static struct clk_pll bimc_pll = {
- .l_reg = 0x23004,
- .m_reg = 0x23008,
- .n_reg = 0x2300c,
- .config_reg = 0x23010,
- .mode_reg = 0x23000,
- .status_reg = 0x2301c,
- .status_bit = 17,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "bimc_pll",
- .parent_data = &(const struct clk_parent_data){
- .fw_name = "xo",
- },
- .num_parents = 1,
- .ops = &clk_pll_ops,
- },
-};
-
-static struct clk_regmap bimc_pll_vote = {
- .enable_reg = 0x45000,
- .enable_mask = BIT(3),
- .hw.init = &(struct clk_init_data){
- .name = "bimc_pll_vote",
- .parent_hws = (const struct clk_hw *[]){ &bimc_pll.clkr.hw },
- .num_parents = 1,
- .ops = &clk_pll_vote_ops,
- },
-};
-
-static const struct parent_map gcc_xo_gpll0_bimc_map[] = {
- { P_XO, 0 },
- { P_GPLL0, 1 },
- { P_BIMC, 2 },
-};
-
-static const struct clk_parent_data gcc_xo_gpll0_bimc[] = {
- { .fw_name = "xo" },
- { .hw = &gpll0.clkr.hw },
- { .hw = &bimc_pll_vote.hw },
-};
-
-static const struct freq_tbl ftbl_pcnoc_bfdcd_clk_src[] = {
- F(19200000, P_XO, 1, 0, 0),
- F(50000000, P_GPLL0, 16, 0, 0),
- F(100000000, P_GPLL0, 8, 0, 0),
- { }
-};
-
-static struct clk_rcg2 pcnoc_bfdcd_clk_src = {
- .cmd_rcgr = 0x27000,
- .freq_tbl = ftbl_pcnoc_bfdcd_clk_src,
- .hid_width = 5,
- .parent_map = gcc_xo_gpll0_bimc_map,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "pcnoc_bfdcd_clk_src",
- .parent_data = gcc_xo_gpll0_bimc,
- .num_parents = ARRAY_SIZE(gcc_xo_gpll0_bimc),
- .ops = &clk_rcg2_ops,
- .flags = CLK_IS_CRITICAL,
- },
-};
-
-static struct clk_rcg2 system_noc_bfdcd_clk_src = {
- .cmd_rcgr = 0x26004,
- .hid_width = 5,
- .parent_map = gcc_xo_gpll0_bimc_map,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "system_noc_bfdcd_clk_src",
- .parent_data = gcc_xo_gpll0_bimc,
- .num_parents = ARRAY_SIZE(gcc_xo_gpll0_bimc),
- .ops = &clk_rcg2_ops,
- },
-};
-
static const struct freq_tbl ftbl_gcc_blsp1_qup1_6_i2c_apps_clk[] = {
F(19200000, P_XO, 1, 0, 0),
F(50000000, P_GPLL0, 16, 0, 0),
@@ -674,26 +585,6 @@ static struct clk_rcg2 sdcc2_apps_clk_src = {
},
};
-static const struct freq_tbl ftbl_gcc_apss_tcu_clk[] = {
- F(155000000, P_GPLL2, 6, 0, 0),
- F(310000000, P_GPLL2, 3, 0, 0),
- F(400000000, P_GPLL0, 2, 0, 0),
- { }
-};
-
-static struct clk_rcg2 apss_tcu_clk_src = {
- .cmd_rcgr = 0x1207c,
- .hid_width = 5,
- .parent_map = gcc_xo_gpll0_gpll1_gpll2_map,
- .freq_tbl = ftbl_gcc_apss_tcu_clk,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "apss_tcu_clk_src",
- .parent_data = gcc_xo_gpll0_gpll1_gpll2,
- .num_parents = 4,
- .ops = &clk_rcg2_ops,
- },
-};
-
static const struct freq_tbl ftbl_gcc_usb_hs_system_clk[] = {
F(19200000, P_XO, 1, 0, 0),
F(57140000, P_GPLL0, 14, 0, 0),
@@ -781,8 +672,6 @@ static struct clk_branch gcc_blsp1_ahb_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_blsp1_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
.ops = &clk_branch2_ops,
},
},
@@ -790,9 +679,10 @@ static struct clk_branch gcc_blsp1_ahb_clk = {
static struct clk_branch gcc_blsp1_sleep_clk = {
.halt_reg = 0x1004,
+ .halt_check = BRANCH_HALT_VOTED,
.clkr = {
- .enable_reg = 0x1004,
- .enable_mask = BIT(0),
+ .enable_reg = 0x45004,
+ .enable_mask = BIT(9),
.hw.init = &(struct clk_init_data){
.name = "gcc_blsp1_sleep_clk",
.parent_data = &(const struct clk_parent_data){
@@ -1083,8 +973,6 @@ static struct clk_branch gcc_boot_rom_ahb_clk = {
.enable_mask = BIT(7),
.hw.init = &(struct clk_init_data){
.name = "gcc_boot_rom_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
.ops = &clk_branch2_ops,
},
},
@@ -1098,9 +986,6 @@ static struct clk_branch gcc_crypto_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_crypto_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1114,9 +999,6 @@ static struct clk_branch gcc_crypto_axi_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_crypto_axi_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1190,9 +1072,6 @@ static struct clk_branch gcc_mss_cfg_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_mss_cfg_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1220,9 +1099,6 @@ static struct clk_branch gcc_pdm_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1236,9 +1112,6 @@ static struct clk_branch gcc_prng_ahb_clk = {
.enable_mask = BIT(8),
.hw.init = &(struct clk_init_data){
.name = "gcc_prng_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1251,9 +1124,6 @@ static struct clk_branch gcc_sdcc1_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1281,9 +1151,6 @@ static struct clk_branch gcc_sdcc2_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1304,19 +1171,6 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
},
};
-static struct clk_rcg2 bimc_ddr_clk_src = {
- .cmd_rcgr = 0x32004,
- .hid_width = 5,
- .parent_map = gcc_xo_gpll0_bimc_map,
- .clkr.hw.init = &(struct clk_init_data){
- .name = "bimc_ddr_clk_src",
- .parent_data = gcc_xo_gpll0_bimc,
- .num_parents = 3,
- .ops = &clk_rcg2_ops,
- .flags = CLK_GET_RATE_NOCACHE,
- },
-};
-
static struct clk_branch gcc_mss_q6_bimc_axi_clk = {
.halt_reg = 0x49004,
.clkr = {
@@ -1324,9 +1178,6 @@ static struct clk_branch gcc_mss_q6_bimc_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_mss_q6_bimc_axi_clk",
- .parent_hws = (const struct clk_hw *[]){ &bimc_ddr_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1340,8 +1191,6 @@ static struct clk_branch gcc_apss_tcu_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_apss_tcu_clk",
- .parent_hws = (const struct clk_hw *[]){ &bimc_ddr_clk_src.clkr.hw },
- .num_parents = 1,
.ops = &clk_branch2_ops,
},
},
@@ -1355,9 +1204,6 @@ static struct clk_branch gcc_smmu_cfg_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_smmu_cfg_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1405,9 +1251,6 @@ static struct clk_branch gcc_usb_hs_phy_cfg_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb_hs_phy_cfg_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1420,9 +1263,6 @@ static struct clk_branch gcc_usb_hs_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb_hs_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -1451,23 +1291,19 @@ static struct clk_branch gcc_apss_ahb_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_apss_ahb_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
.ops = &clk_branch2_ops,
},
},
};
static struct clk_branch gcc_apss_axi_clk = {
- .halt_reg = 0x4601c,
+ .halt_reg = 0x46020,
.halt_check = BRANCH_HALT_VOTED,
.clkr = {
.enable_reg = 0x45004,
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_apss_axi_clk",
- .parent_hws = (const struct clk_hw *[]){ &pcnoc_bfdcd_clk_src.clkr.hw },
- .num_parents = 1,
.ops = &clk_branch2_ops,
},
},
@@ -1480,11 +1316,6 @@ static struct clk_regmap *gcc_mdm9607_clocks[] = {
[GPLL1_VOTE] = &gpll1_vote,
[GPLL2] = &gpll2.clkr,
[GPLL2_EARLY] = &gpll2_early.clkr,
- [BIMC_PLL] = &bimc_pll.clkr,
- [BIMC_PLL_VOTE] = &bimc_pll_vote,
- [BIMC_DDR_CLK_SRC] = &bimc_ddr_clk_src.clkr,
- [PCNOC_BFDCD_CLK_SRC] = &pcnoc_bfdcd_clk_src.clkr,
- [SYSTEM_NOC_BFDCD_CLK_SRC] = &system_noc_bfdcd_clk_src.clkr,
[APSS_AHB_CLK_SRC] = &apss_ahb_clk_src.clkr,
[BLSP1_QUP1_I2C_APPS_CLK_SRC] = &blsp1_qup1_i2c_apps_clk_src.clkr,
[BLSP1_QUP1_SPI_APPS_CLK_SRC] = &blsp1_qup1_spi_apps_clk_src.clkr,
@@ -1511,7 +1342,6 @@ static struct clk_regmap *gcc_mdm9607_clocks[] = {
[PDM2_CLK_SRC] = &pdm2_clk_src.clkr,
[SDCC1_APPS_CLK_SRC] = &sdcc1_apps_clk_src.clkr,
[SDCC2_APPS_CLK_SRC] = &sdcc2_apps_clk_src.clkr,
- [APSS_TCU_CLK_SRC] = &apss_tcu_clk_src.clkr,
[USB_HS_SYSTEM_CLK_SRC] = &usb_hs_system_clk_src.clkr,
[GCC_BLSP1_AHB_CLK] = &gcc_blsp1_ahb_clk.clkr,
[GCC_BLSP1_SLEEP_CLK] = &gcc_blsp1_sleep_clk.clkr,
@@ -1567,7 +1397,7 @@ static const struct qcom_reset_map gcc_mdm9607_resets[] = {
[USB_HS_HSIC_BCR] = { 0x3d05c },
[GCC_MSS_RESTART] = { 0x3e000 },
[USB_HS_BCR] = { 0x41000 },
- [USB2_HS_PHY_ONLY_BCR] = { 0x41034 },
+ [USB2_HS_PHY_ONLY_BCR] = { .reg = 0x41034, .udelay = 15 },
[QUSB2_PHY_BCR] = { 0x4103c },
};
diff --git a/drivers/clk/qcom/gcc-msm8916.c b/drivers/clk/qcom/gcc-msm8916.c
index 9c7c6b23ce32..e91406e6f527 100644
--- a/drivers/clk/qcom/gcc-msm8916.c
+++ b/drivers/clk/qcom/gcc-msm8916.c
@@ -1589,9 +1589,10 @@ static struct clk_branch gcc_blsp1_ahb_clk = {
static struct clk_branch gcc_blsp1_sleep_clk = {
.halt_reg = 0x01004,
+ .halt_check = BRANCH_HALT_VOTED,
.clkr = {
- .enable_reg = 0x01004,
- .enable_mask = BIT(0),
+ .enable_reg = 0x45004,
+ .enable_mask = BIT(9),
.hw.init = &(struct clk_init_data){
.name = "gcc_blsp1_sleep_clk",
.parent_data = &(const struct clk_parent_data){
diff --git a/drivers/clk/qcom/gcc-msm8939.c b/drivers/clk/qcom/gcc-msm8939.c
index 45193b3d714b..ffd7f14fcbaf 100644
--- a/drivers/clk/qcom/gcc-msm8939.c
+++ b/drivers/clk/qcom/gcc-msm8939.c
@@ -1929,9 +1929,10 @@ static struct clk_branch gcc_blsp1_ahb_clk = {
static struct clk_branch gcc_blsp1_sleep_clk = {
.halt_reg = 0x01004,
+ .halt_check = BRANCH_HALT_VOTED,
.clkr = {
- .enable_reg = 0x01004,
- .enable_mask = BIT(0),
+ .enable_reg = 0x45004,
+ .enable_mask = BIT(9),
.hw.init = &(struct clk_init_data){
.name = "gcc_blsp1_sleep_clk",
.ops = &clk_branch2_ops,
diff --git a/drivers/clk/qcom/gcc-qcm2290.c b/drivers/clk/qcom/gcc-qcm2290.c
index 9a6703365e61..bbcbe847fd95 100644
--- a/drivers/clk/qcom/gcc-qcm2290.c
+++ b/drivers/clk/qcom/gcc-qcm2290.c
@@ -1082,7 +1082,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s0_clk_src_init = {
.name = "gcc_qupv3_wrap0_s0_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
@@ -1098,7 +1098,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s1_clk_src_init = {
.name = "gcc_qupv3_wrap0_s1_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s1_clk_src = {
@@ -1114,7 +1114,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s2_clk_src_init = {
.name = "gcc_qupv3_wrap0_s2_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
@@ -1130,7 +1130,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s3_clk_src_init = {
.name = "gcc_qupv3_wrap0_s3_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
@@ -1146,7 +1146,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s4_clk_src_init = {
.name = "gcc_qupv3_wrap0_s4_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
@@ -1162,7 +1162,7 @@ static struct clk_init_data gcc_qupv3_wrap0_s5_clk_src_init = {
.name = "gcc_qupv3_wrap0_s5_clk_src",
.parent_data = gcc_parents_1,
.num_parents = ARRAY_SIZE(gcc_parents_1),
- .ops = &clk_rcg2_shared_ops,
+ .ops = &clk_rcg2_shared_no_init_park_ops,
};
static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
diff --git a/drivers/clk/qcom/gdsc.c b/drivers/clk/qcom/gdsc.c
index 208fc430ec98..83320559d218 100644
--- a/drivers/clk/qcom/gdsc.c
+++ b/drivers/clk/qcom/gdsc.c
@@ -103,14 +103,21 @@ static int gdsc_hwctrl(struct gdsc *sc, bool en)
static int gdsc_poll_status(struct gdsc *sc, enum gdsc_status status)
{
ktime_t start;
+ int ret;
start = ktime_get();
do {
- if (gdsc_check_status(sc, status))
+ ret = gdsc_check_status(sc, status);
+ if (ret < 0)
+ return ret;
+ if (ret)
return 0;
} while (ktime_us_delta(ktime_get(), start) < STATUS_POLL_TIMEOUT_US);
- if (gdsc_check_status(sc, status))
+ ret = gdsc_check_status(sc, status);
+ if (ret < 0)
+ return ret;
+ if (ret)
return 0;
return -ETIMEDOUT;
@@ -474,7 +481,9 @@ static int gdsc_init(struct gdsc *sc)
} else if (sc->flags & ALWAYS_ON) {
/* If ALWAYS_ON GDSCs are not ON, turn them ON */
- gdsc_enable(&sc->pd);
+ ret = gdsc_enable(&sc->pd);
+ if (ret)
+ return ret;
on = true;
}
@@ -591,10 +600,18 @@ int gdsc_register(struct gdsc_desc *desc,
void gdsc_unregister(struct gdsc_desc *desc)
{
struct device *dev = desc->dev;
+ struct gdsc **scs = desc->scs;
size_t num = desc->num;
+ int i;
- gdsc_pm_subdomain_remove(desc, num);
of_genpd_del_provider(dev->of_node);
+ gdsc_pm_subdomain_remove(desc, num);
+
+ for (i = 0; i < num; i++) {
+ if (!scs[i])
+ continue;
+ pm_genpd_remove(&scs[i]->pd);
+ }
}
/*
diff --git a/drivers/clk/qcom/gpucc-qcm2290.c b/drivers/clk/qcom/gpucc-qcm2290.c
index dc369dff882e..3ccab8f3b4e0 100644
--- a/drivers/clk/qcom/gpucc-qcm2290.c
+++ b/drivers/clk/qcom/gpucc-qcm2290.c
@@ -144,7 +144,7 @@ static struct clk_rcg2 gpu_cc_gx_gfx3d_clk_src = {
.parent_data = gpu_cc_parent_data_1,
.num_parents = ARRAY_SIZE(gpu_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
diff --git a/drivers/clk/rockchip/clk-pll.c b/drivers/clk/rockchip/clk-pll.c
index fe76756e592e..dfe02f463e24 100644
--- a/drivers/clk/rockchip/clk-pll.c
+++ b/drivers/clk/rockchip/clk-pll.c
@@ -893,6 +893,13 @@ static void rockchip_rk3588_pll_get_params(struct rockchip_clk_pll *pll,
rate->k = ((pllcon >> RK3588_PLLCON2_K_SHIFT) & RK3588_PLLCON2_K_MASK);
}
+/*
+ * 2250 MHz <= Fvco <= 4500 MHz
+ * For Fvco > 3 GHz: period jitter +-1% frac PLL, +-0.75% int PLL
+ * For Fvco < 3 GHz: period jitter +-2% frac PLL, +-1.50% int PLL
+ * Fvco = ((m + k / 65536) * Fin) / p
+ * Fout = ((m + k / 65536) * Fin) / (p * 2^s)
+ */
static unsigned long rockchip_rk3588_pll_recalc_rate(struct clk_hw *hw, unsigned long prate)
{
struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
@@ -908,7 +915,7 @@ static unsigned long rockchip_rk3588_pll_recalc_rate(struct clk_hw *hw, unsigned
/* fractional mode */
u64 frac_rate64 = prate * cur.k;
- postdiv = cur.p * 65535;
+ postdiv = cur.p * 65536;
do_div(frac_rate64, postdiv);
rate64 += frac_rate64;
}
diff --git a/drivers/clk/rockchip/clk-rk3576.c b/drivers/clk/rockchip/clk-rk3576.c
index 595e010341f7..0746d2d07095 100644
--- a/drivers/clk/rockchip/clk-rk3576.c
+++ b/drivers/clk/rockchip/clk-rk3576.c
@@ -313,6 +313,7 @@ PNAME(mux_100m_24m_lclk0_p) = { "clk_cpll_div10", "xin24m", "lclk_asrc_src_0" }
PNAME(mux_100m_24m_lclk1_p) = { "clk_cpll_div10", "xin24m", "lclk_asrc_src_1" };
PNAME(mux_150m_100m_50m_24m_p) = { "clk_gpll_div8", "clk_cpll_div10", "clk_cpll_div20", "xin24m" };
PNAME(mux_200m_100m_50m_24m_p) = { "clk_gpll_div6", "clk_cpll_div10", "clk_cpll_div20", "xin24m" };
+PNAME(mux_200m_150m_100m_24m_p) = { "clk_gpll_div6", "clk_gpll_div8", "clk_cpll_div10", "xin24m" };
PNAME(mux_400m_200m_100m_24m_p) = { "clk_gpll_div3", "clk_gpll_div6", "clk_cpll_div10", "xin24m" };
PNAME(mux_500m_250m_100m_24m_p) = { "clk_cpll_div2", "clk_cpll_div4", "clk_cpll_div10", "xin24m" };
PNAME(mux_600m_400m_300m_24m_p) = { "clk_gpll_div2", "clk_gpll_div3", "clk_gpll_div4", "xin24m" };
@@ -702,19 +703,19 @@ static struct rockchip_clk_branch rk3576_clk_branches[] __initdata = {
RK3576_CLKGATE_CON(16), 0, GFLAGS),
GATE(PCLK_SPI4, "pclk_spi4", "pclk_bus_root", 0,
RK3576_CLKGATE_CON(16), 1, GFLAGS),
- COMPOSITE_NODIV(CLK_SPI0, "clk_spi0", mux_200m_100m_50m_24m_p, 0,
+ COMPOSITE_NODIV(CLK_SPI0, "clk_spi0", mux_200m_150m_100m_24m_p, 0,
RK3576_CLKSEL_CON(70), 13, 2, MFLAGS,
RK3576_CLKGATE_CON(16), 2, GFLAGS),
- COMPOSITE_NODIV(CLK_SPI1, "clk_spi1", mux_200m_100m_50m_24m_p, 0,
+ COMPOSITE_NODIV(CLK_SPI1, "clk_spi1", mux_200m_150m_100m_24m_p, 0,
RK3576_CLKSEL_CON(71), 0, 2, MFLAGS,
RK3576_CLKGATE_CON(16), 3, GFLAGS),
- COMPOSITE_NODIV(CLK_SPI2, "clk_spi2", mux_200m_100m_50m_24m_p, 0,
+ COMPOSITE_NODIV(CLK_SPI2, "clk_spi2", mux_200m_150m_100m_24m_p, 0,
RK3576_CLKSEL_CON(71), 2, 2, MFLAGS,
RK3576_CLKGATE_CON(16), 4, GFLAGS),
- COMPOSITE_NODIV(CLK_SPI3, "clk_spi3", mux_200m_100m_50m_24m_p, 0,
+ COMPOSITE_NODIV(CLK_SPI3, "clk_spi3", mux_200m_150m_100m_24m_p, 0,
RK3576_CLKSEL_CON(71), 4, 2, MFLAGS,
RK3576_CLKGATE_CON(16), 5, GFLAGS),
- COMPOSITE_NODIV(CLK_SPI4, "clk_spi4", mux_200m_100m_50m_24m_p, 0,
+ COMPOSITE_NODIV(CLK_SPI4, "clk_spi4", mux_200m_150m_100m_24m_p, 0,
RK3576_CLKSEL_CON(71), 6, 2, MFLAGS,
RK3576_CLKGATE_CON(16), 6, GFLAGS),
GATE(PCLK_WDT0, "pclk_wdt0", "pclk_bus_root", 0,
diff --git a/drivers/clk/rockchip/clk-rk3588.c b/drivers/clk/rockchip/clk-rk3588.c
index 0ffaf639f807..8f029c2b73db 100644
--- a/drivers/clk/rockchip/clk-rk3588.c
+++ b/drivers/clk/rockchip/clk-rk3588.c
@@ -2113,7 +2113,7 @@ static struct rockchip_clk_branch rk3588_clk_branches[] __initdata = {
COMPOSITE(DCLK_VOP1_SRC, "dclk_vop1_src", gpll_cpll_v0pll_aupll_p, 0,
RK3588_CLKSEL_CON(111), 14, 2, MFLAGS, 9, 5, DFLAGS,
RK3588_CLKGATE_CON(52), 11, GFLAGS),
- COMPOSITE(DCLK_VOP2_SRC, "dclk_vop2_src", gpll_cpll_v0pll_aupll_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ COMPOSITE(DCLK_VOP2_SRC, "dclk_vop2_src", gpll_cpll_v0pll_aupll_p, CLK_SET_RATE_NO_REPARENT,
RK3588_CLKSEL_CON(112), 5, 2, MFLAGS, 0, 5, DFLAGS,
RK3588_CLKGATE_CON(52), 12, GFLAGS),
COMPOSITE_NODIV(DCLK_VOP0, "dclk_vop0", dclk_vop0_p,
diff --git a/drivers/clk/tegra/clk-tegra124-emc.c b/drivers/clk/tegra/clk-tegra124-emc.c
index 5f1af6dfe715..674aef278539 100644
--- a/drivers/clk/tegra/clk-tegra124-emc.c
+++ b/drivers/clk/tegra/clk-tegra124-emc.c
@@ -539,6 +539,7 @@ struct clk *tegra124_clk_register_emc(void __iomem *base, struct device_node *np
clk = clk_register(NULL, &tegra->hw);
if (IS_ERR(clk)) {
+ of_node_put(tegra->emc_node);
kfree(tegra);
return clk;
}
diff --git a/drivers/clk/ti/adpll.c b/drivers/clk/ti/adpll.c
index e305fcbac647..8885d28face5 100644
--- a/drivers/clk/ti/adpll.c
+++ b/drivers/clk/ti/adpll.c
@@ -483,7 +483,7 @@ static const struct clk_ops ti_adpll_ops = {
static int ti_adpll_init_dco(struct ti_adpll_data *d)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct clk *clock;
const char *postfix;
int width, err;
@@ -576,7 +576,7 @@ static int ti_adpll_init_clkout(struct ti_adpll_data *d,
struct clk *clk1)
{
struct ti_adpll_clkout_data *co;
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct clk_ops *ops;
const char *parent_names[2];
const char *child_name;
diff --git a/drivers/clk/ti/divider.c b/drivers/clk/ti/divider.c
index ade99ab6cfa9..3f30cb0806c1 100644
--- a/drivers/clk/ti/divider.c
+++ b/drivers/clk/ti/divider.c
@@ -309,7 +309,7 @@ static struct clk *_register_divider(struct device_node *node,
u32 flags,
struct clk_omap_divider *div)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
const char *parent_name;
const char *name;
diff --git a/drivers/clk/ti/mux.c b/drivers/clk/ti/mux.c
index 216d85d6aac6..1a0e12b3b3fd 100644
--- a/drivers/clk/ti/mux.c
+++ b/drivers/clk/ti/mux.c
@@ -119,14 +119,14 @@ const struct clk_ops ti_clk_mux_ops = {
};
static struct clk *_register_mux(struct device_node *node, const char *name,
- const char * const *parent_names,
+ const struct clk_parent_data *parent_data,
u8 num_parents, unsigned long flags,
struct clk_omap_reg *reg, u8 shift, u32 mask,
s8 latch, u8 clk_mux_flags, u32 *table)
{
+ struct clk_init_data init = {};
struct clk_omap_mux *mux;
struct clk *clk;
- struct clk_init_data init;
/* allocate the mux */
mux = kzalloc(sizeof(*mux), GFP_KERNEL);
@@ -136,7 +136,7 @@ static struct clk *_register_mux(struct device_node *node, const char *name,
init.name = name;
init.ops = &ti_clk_mux_ops;
init.flags = flags;
- init.parent_names = parent_names;
+ init.parent_data = parent_data;
init.num_parents = num_parents;
/* struct clk_mux assignments */
@@ -167,24 +167,26 @@ static void of_mux_clk_setup(struct device_node *node)
struct clk *clk;
struct clk_omap_reg reg;
unsigned int num_parents;
- const char **parent_names;
+ struct clk_parent_data *parent_data;
const char *name;
u8 clk_mux_flags = 0;
u32 mask = 0;
u32 shift = 0;
s32 latch = -EINVAL;
u32 flags = CLK_SET_RATE_NO_REPARENT;
+ int i;
num_parents = of_clk_get_parent_count(node);
if (num_parents < 2) {
pr_err("mux-clock %pOFn must have parents\n", node);
return;
}
- parent_names = kzalloc((sizeof(char *) * num_parents), GFP_KERNEL);
- if (!parent_names)
- goto cleanup;
+ parent_data = kcalloc(num_parents, sizeof(*parent_data), GFP_KERNEL);
+ if (!parent_data)
+ return;
- of_clk_parent_fill(node, parent_names, num_parents);
+ for (i = 0; i < num_parents; i++)
+ parent_data[i].index = i;
if (ti_clk_get_reg_addr(node, 0, ®))
goto cleanup;
@@ -207,7 +209,7 @@ static void of_mux_clk_setup(struct device_node *node)
mask = (1 << fls(mask)) - 1;
name = ti_dt_clk_name(node);
- clk = _register_mux(node, name, parent_names, num_parents,
+ clk = _register_mux(node, name, parent_data, num_parents,
flags, ®, shift, mask, latch, clk_mux_flags,
NULL);
@@ -215,7 +217,7 @@ static void of_mux_clk_setup(struct device_node *node)
of_clk_add_provider(node, of_clk_src_simple_get, clk);
cleanup:
- kfree(parent_names);
+ kfree(parent_data);
}
CLK_OF_DECLARE(mux_clk, "ti,mux-clock", of_mux_clk_setup);
diff --git a/drivers/clk/visconti/clkc.c b/drivers/clk/visconti/clkc.c
index d0b193b5d0b3..4018d1298880 100644
--- a/drivers/clk/visconti/clkc.c
+++ b/drivers/clk/visconti/clkc.c
@@ -81,9 +81,9 @@ static struct clk_hw *visconti_clk_register_gate(struct device *dev,
u8 rs_idx,
spinlock_t *lock)
{
+ struct clk_init_data init = {};
struct visconti_clk_gate *gate;
struct clk_parent_data *pdata;
- struct clk_init_data init;
struct clk_hw *hw;
int ret;
diff --git a/drivers/clk/x86/clk-pmc-atom.c b/drivers/clk/x86/clk-pmc-atom.c
index 99291ba65da7..08c83e0abc41 100644
--- a/drivers/clk/x86/clk-pmc-atom.c
+++ b/drivers/clk/x86/clk-pmc-atom.c
@@ -160,6 +160,9 @@ static struct clk_plt *plt_clk_register(struct platform_device *pdev, int id,
return ERR_PTR(-ENOMEM);
init.name = kasprintf(GFP_KERNEL, "%s_%d", PLT_CLK_NAME_BASE, id);
+ if (!init.name)
+ return ERR_PTR(-ENOMEM);
+
init.ops = &plt_clk_ops;
init.flags = 0;
init.parent_names = parent_names;
diff --git a/drivers/clocksource/clps711x-timer.c b/drivers/clocksource/clps711x-timer.c
index bbceb0289d45..95f8a50ba1eb 100644
--- a/drivers/clocksource/clps711x-timer.c
+++ b/drivers/clocksource/clps711x-timer.c
@@ -94,7 +94,7 @@ static int __init clps711x_timer_init(struct device_node *np)
switch (of_alias_get_id(np, "timer")) {
case CLPS711X_CLKSRC_CLOCKSOURCE:
clps711x_clksrc_init(clock, base);
- break;
+ return 0;
case CLPS711X_CLKSRC_CLOCKEVENT:
ret = _clps711x_clkevt_init(clock, base, irq);
break;
diff --git a/drivers/clocksource/timer-armada-370-xp.c b/drivers/clocksource/timer-armada-370-xp.c
index 6ec565d6939a..49735ced52bb 100644
--- a/drivers/clocksource/timer-armada-370-xp.c
+++ b/drivers/clocksource/timer-armada-370-xp.c
@@ -346,7 +346,11 @@ static int __init armada_xp_timer_init(struct device_node *np)
timer_clk = clk_get_rate(clk);
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_xp, "marvell,armada-xp-timer",
armada_xp_timer_init);
@@ -384,7 +388,11 @@ static int __init armada_375_timer_init(struct device_node *np)
timer25Mhz = false;
}
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_375, "marvell,armada-375-timer",
armada_375_timer_init);
@@ -407,7 +415,11 @@ static int __init armada_370_timer_init(struct device_node *np)
timer_clk = clk_get_rate(clk) / TIMER_DIVIDER;
timer25Mhz = false;
- return armada_370_xp_timer_common_init(np);
+ ret = armada_370_xp_timer_common_init(np);
+ if (ret)
+ clk_disable_unprepare(clk);
+
+ return ret;
}
TIMER_OF_DECLARE(armada_370, "marvell,armada-370-timer",
armada_370_timer_init);
diff --git a/drivers/cpufreq/amd-pstate.c b/drivers/cpufreq/amd-pstate.c
index c0e073b0425e..c0ddf9f5a8b0 100644
--- a/drivers/cpufreq/amd-pstate.c
+++ b/drivers/cpufreq/amd-pstate.c
@@ -368,15 +368,19 @@ static inline int amd_pstate_enable(bool enable)
static int pstate_init_perf(struct amd_cpudata *cpudata)
{
- u64 cap1;
+ u64 cap1, numerator;
int ret = rdmsrl_safe_on_cpu(cpudata->cpu, MSR_AMD_CPPC_CAP1,
&cap1);
if (ret)
return ret;
- WRITE_ONCE(cpudata->highest_perf, AMD_CPPC_HIGHEST_PERF(cap1));
- WRITE_ONCE(cpudata->max_limit_perf, AMD_CPPC_HIGHEST_PERF(cap1));
+ ret = amd_get_boost_ratio_numerator(cpudata->cpu, &numerator);
+ if (ret)
+ return ret;
+
+ WRITE_ONCE(cpudata->highest_perf, numerator);
+ WRITE_ONCE(cpudata->max_limit_perf, numerator);
WRITE_ONCE(cpudata->nominal_perf, AMD_CPPC_NOMINAL_PERF(cap1));
WRITE_ONCE(cpudata->lowest_nonlinear_perf, AMD_CPPC_LOWNONLIN_PERF(cap1));
WRITE_ONCE(cpudata->lowest_perf, AMD_CPPC_LOWEST_PERF(cap1));
@@ -388,13 +392,19 @@ static int pstate_init_perf(struct amd_cpudata *cpudata)
static int cppc_init_perf(struct amd_cpudata *cpudata)
{
struct cppc_perf_caps cppc_perf;
+ u64 numerator;
+ bool auto_sel;
int ret = cppc_get_perf_caps(cpudata->cpu, &cppc_perf);
if (ret)
return ret;
- WRITE_ONCE(cpudata->highest_perf, cppc_perf.highest_perf);
- WRITE_ONCE(cpudata->max_limit_perf, cppc_perf.highest_perf);
+ ret = amd_get_boost_ratio_numerator(cpudata->cpu, &numerator);
+ if (ret)
+ return ret;
+
+ WRITE_ONCE(cpudata->highest_perf, numerator);
+ WRITE_ONCE(cpudata->max_limit_perf, numerator);
WRITE_ONCE(cpudata->nominal_perf, cppc_perf.nominal_perf);
WRITE_ONCE(cpudata->lowest_nonlinear_perf,
cppc_perf.lowest_nonlinear_perf);
@@ -402,10 +412,7 @@ static int cppc_init_perf(struct amd_cpudata *cpudata)
WRITE_ONCE(cpudata->prefcore_ranking, cppc_perf.highest_perf);
WRITE_ONCE(cpudata->min_limit_perf, cppc_perf.lowest_perf);
- if (cppc_state == AMD_PSTATE_ACTIVE)
- return 0;
-
- ret = cppc_get_auto_sel_caps(cpudata->cpu, &cppc_perf);
+ ret = cppc_get_auto_sel(cpudata->cpu, &auto_sel);
if (ret) {
pr_warn("failed to get auto_sel, ret: %d\n", ret);
return 0;
@@ -773,7 +780,7 @@ static void amd_pstate_init_prefcore(struct amd_cpudata *cpudata)
* sched_set_itmt_support(true) has been called and it is valid to
* update them at any time after it has been called.
*/
- sched_set_itmt_core_prio((int)READ_ONCE(cpudata->highest_perf), cpudata->cpu);
+ sched_set_itmt_core_prio((int)READ_ONCE(cpudata->prefcore_ranking), cpudata->cpu);
schedule_work(&sched_prefcore_work);
}
@@ -867,7 +874,6 @@ static int amd_pstate_init_freq(struct amd_cpudata *cpudata)
{
int ret;
u32 min_freq, max_freq;
- u64 numerator;
u32 nominal_perf, nominal_freq;
u32 lowest_nonlinear_perf, lowest_nonlinear_freq;
u32 boost_ratio, lowest_nonlinear_ratio;
@@ -889,10 +895,7 @@ static int amd_pstate_init_freq(struct amd_cpudata *cpudata)
nominal_perf = READ_ONCE(cpudata->nominal_perf);
- ret = amd_get_boost_ratio_numerator(cpudata->cpu, &numerator);
- if (ret)
- return ret;
- boost_ratio = div_u64(numerator << SCHED_CAPACITY_SHIFT, nominal_perf);
+ boost_ratio = div_u64(cpudata->highest_perf << SCHED_CAPACITY_SHIFT, nominal_perf);
max_freq = (nominal_freq * boost_ratio >> SCHED_CAPACITY_SHIFT) * 1000;
lowest_nonlinear_perf = READ_ONCE(cpudata->lowest_nonlinear_perf);
diff --git a/drivers/cpufreq/imx6q-cpufreq.c b/drivers/cpufreq/imx6q-cpufreq.c
index 33c1df7f683e..51520307c4b0 100644
--- a/drivers/cpufreq/imx6q-cpufreq.c
+++ b/drivers/cpufreq/imx6q-cpufreq.c
@@ -55,7 +55,6 @@ static unsigned int max_freq;
static unsigned int transition_latency;
static u32 *imx6_soc_volt;
-static u32 soc_opp_count;
static int imx6q_set_target(struct cpufreq_policy *policy, unsigned int index)
{
@@ -331,6 +330,7 @@ static int imx6q_cpufreq_probe(struct platform_device *pdev)
const struct property *prop;
const __be32 *val;
u32 nr, i, j;
+ u32 soc_opp_count = 0;
cpu_dev = get_cpu_device(0);
if (!cpu_dev) {
@@ -401,7 +401,7 @@ static int imx6q_cpufreq_probe(struct platform_device *pdev)
}
/* Make imx6_soc_volt array's size same as arm opp number */
- imx6_soc_volt = devm_kcalloc(cpu_dev, num, sizeof(*imx6_soc_volt),
+ imx6_soc_volt = devm_kcalloc(&pdev->dev, num, sizeof(*imx6_soc_volt),
GFP_KERNEL);
if (imx6_soc_volt == NULL) {
ret = -ENOMEM;
@@ -486,6 +486,7 @@ static int imx6q_cpufreq_probe(struct platform_device *pdev)
return 0;
free_freq_table:
+ imx6_soc_volt = NULL;
dev_pm_opp_free_cpufreq_table(cpu_dev, &freq_table);
out_free_opp:
dev_pm_opp_of_remove_table(cpu_dev);
@@ -507,6 +508,7 @@ static int imx6q_cpufreq_probe(struct platform_device *pdev)
static void imx6q_cpufreq_remove(struct platform_device *pdev)
{
cpufreq_unregister_driver(&imx6q_cpufreq_driver);
+ imx6_soc_volt = NULL;
dev_pm_opp_free_cpufreq_table(cpu_dev, &freq_table);
dev_pm_opp_of_remove_table(cpu_dev);
regulator_put(arm_reg);
diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 818507586cd9..f1910f9109f0 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -2300,8 +2300,6 @@ static void intel_pstate_get_cpu_pstates(struct cpudata *cpu)
if (pstate_funcs.get_vid)
pstate_funcs.get_vid(cpu);
-
- intel_pstate_set_min_pstate(cpu);
}
/*
@@ -3008,6 +3006,7 @@ static int __intel_pstate_cpu_init(struct cpufreq_policy *policy)
static int intel_pstate_cpu_init(struct cpufreq_policy *policy)
{
int ret = __intel_pstate_cpu_init(policy);
+ struct cpudata *cpu;
if (ret)
return ret;
@@ -3018,11 +3017,11 @@ static int intel_pstate_cpu_init(struct cpufreq_policy *policy)
*/
policy->policy = CPUFREQ_POLICY_POWERSAVE;
- if (hwp_active) {
- struct cpudata *cpu = all_cpu_data[policy->cpu];
-
+ cpu = all_cpu_data[policy->cpu];
+ if (hwp_active)
cpu->epp_cached = intel_pstate_get_epp(cpu, 0);
- }
+ else
+ intel_pstate_set_min_pstate(cpu);
return 0;
}
@@ -3246,8 +3245,6 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy)
return ret;
policy->cpuinfo.transition_latency = INTEL_CPUFREQ_TRANSITION_LATENCY;
- /* This reflects the intel_pstate_get_cpu_pstates() setting. */
- policy->cur = policy->cpuinfo.min_freq;
req = kcalloc(2, sizeof(*req), GFP_KERNEL);
if (!req) {
@@ -3268,9 +3265,15 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy)
WRITE_ONCE(cpu->hwp_req_cached, value);
cpu->epp_cached = intel_pstate_get_epp(cpu, value);
+
+ intel_cpufreq_hwp_update(cpu, cpu->pstate.min_pstate,
+ cpu->pstate.max_pstate,
+ cpu->pstate.min_pstate, false);
} else {
policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY;
+ intel_pstate_set_min_pstate(cpu);
}
+ policy->cur = policy->cpuinfo.min_freq;
freq = DIV_ROUND_UP(cpu->pstate.turbo_freq * global.min_perf_pct, 100);
diff --git a/drivers/cpufreq/spear-cpufreq.c b/drivers/cpufreq/spear-cpufreq.c
index c032dd5d12d3..ae038eec35bc 100644
--- a/drivers/cpufreq/spear-cpufreq.c
+++ b/drivers/cpufreq/spear-cpufreq.c
@@ -79,9 +79,9 @@ static int spear1340_set_cpu_rate(struct clk *sys_pclk, unsigned long newfreq)
int ret = 0;
sys_clk = clk_get_parent(spear_cpufreq.clk);
- if (IS_ERR(sys_clk)) {
+ if (!sys_clk) {
pr_err("failed to get cpu's parent (sys) clock\n");
- return PTR_ERR(sys_clk);
+ return -EINVAL;
}
/* Set the rate of the source clock before changing the parent */
diff --git a/drivers/cpuidle/dt_idle_genpd.c b/drivers/cpuidle/dt_idle_genpd.c
index 203e9b754aea..82ff79177358 100644
--- a/drivers/cpuidle/dt_idle_genpd.c
+++ b/drivers/cpuidle/dt_idle_genpd.c
@@ -99,7 +99,7 @@ struct generic_pm_domain *dt_idle_pd_alloc(struct device_node *np,
if (!pd)
goto out;
- pd->name = kasprintf(GFP_KERNEL, "%pOF", np);
+ pd->name = kstrdup(kbasename(of_node_full_name(np)), GFP_KERNEL);
if (!pd->name)
goto free_pd;
@@ -112,7 +112,6 @@ struct generic_pm_domain *dt_idle_pd_alloc(struct device_node *np,
goto free_name;
pd->free_states = pd_free_states;
- pd->name = kbasename(pd->name);
pd->states = states;
pd->state_count = state_count;
diff --git a/drivers/crypto/atmel-ecc.c b/drivers/crypto/atmel-ecc.c
index 9da5a0388080..c64431380d30 100644
--- a/drivers/crypto/atmel-ecc.c
+++ b/drivers/crypto/atmel-ecc.c
@@ -27,15 +27,14 @@ static struct atmel_ecc_driver_data driver_data;
/**
* struct atmel_ecdh_ctx - transformation context
- * @client : pointer to i2c client device
- * @fallback : used for unsupported curves or when user wants to use its own
- * private key.
- * @public_key : generated when calling set_secret(). It's the responsibility
- * of the user to not call set_secret() while
- * generate_public_key() or compute_shared_secret() are in flight.
- * @curve_id : elliptic curve id
- * @do_fallback: true when the device doesn't support the curve or when the user
- * wants to use its own private key.
+ * @client: I2C client device
+ * @fallback: ECDH fallback used for caller-provided private keys
+ * @public_key: cached public key for the device-generated private key
+ * @curve_id: elliptic curve id
+ * @do_fallback: true when ECDH operations should use @fallback
+ *
+ * The caller must not invoke set_secret() while generate_public_key()
+ * or compute_shared_secret() are in flight.
*/
struct atmel_ecdh_ctx {
struct i2c_client *client;
@@ -55,8 +54,8 @@ static void atmel_ecdh_done(struct atmel_i2c_work_data *work_data, void *areq,
if (status)
goto free_work_data;
- /* might want less than we've got */
- n_sz = min_t(size_t, ATMEL_ECC_NIST_P256_N_SIZE, req->dst_len);
+ /* copy only as much as requested, capped at 32 bytes */
+ n_sz = min(ATMEL_ECC_NIST_P256_N_SIZE, req->dst_len);
/* copy the shared secret */
copied = sg_copy_from_buffer(req->dst, sg_nents_for_len(req->dst, n_sz),
@@ -64,15 +63,15 @@ static void atmel_ecdh_done(struct atmel_i2c_work_data *work_data, void *areq,
if (copied != n_sz)
status = -EINVAL;
- /* fall through */
free_work_data:
kfree_sensitive(work_data);
kpp_request_complete(req, status);
}
/*
- * A random private key is generated and stored in the device. The device
- * returns the pair public key.
+ * If no private key is provided, generate one in the device and cache
+ * the corresponding public key. The generated private key never leaves
+ * the device.
*/
static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
unsigned int len)
@@ -83,9 +82,7 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
struct ecdh params;
int ret = -ENOMEM;
- /* free the old public key, if any */
kfree(ctx->public_key);
- /* make sure you don't free the old public key twice */
ctx->public_key = NULL;
if (crypto_ecdh_decode_key(buf, len, ¶ms) < 0) {
@@ -94,7 +91,6 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
}
if (params.key_size) {
- /* fallback to ecdh software implementation */
ctx->do_fallback = true;
return crypto_kpp_set_secret(ctx->fallback, buf, len);
}
@@ -103,11 +99,6 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
if (!cmd)
return -ENOMEM;
- /*
- * The device only supports NIST P256 ECC keys. The public key size will
- * always be the same. Use a macro for the key size to avoid unnecessary
- * computations.
- */
public_key = kmalloc(ATMEL_ECC_PUBKEY_SIZE, GFP_KERNEL);
if (!public_key)
goto free_cmd;
@@ -120,7 +111,6 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
if (ret)
goto free_public_key;
- /* save the public key */
memcpy(public_key, &cmd->data[RSP_DATA_IDX], ATMEL_ECC_PUBKEY_SIZE);
ctx->public_key = public_key;
@@ -149,8 +139,8 @@ static int atmel_ecdh_generate_public_key(struct kpp_request *req)
if (!ctx->public_key)
return -EINVAL;
- /* might want less than we've got */
- nbytes = min_t(size_t, ATMEL_ECC_PUBKEY_SIZE, req->dst_len);
+ /* copy only as much as requested, capped at 64 bytes */
+ nbytes = min(ATMEL_ECC_PUBKEY_SIZE, req->dst_len);
/* public key was saved at private key generation */
copied = sg_copy_from_buffer(req->dst,
@@ -175,7 +165,10 @@ static int atmel_ecdh_compute_shared_secret(struct kpp_request *req)
return crypto_kpp_compute_shared_secret(req);
}
- /* must have exactly two points to be on the curve */
+ if (!ctx->public_key)
+ return -EINVAL;
+
+ /* A P-256 public key must contain two 32-byte coordinates */
if (req->src_len != ATMEL_ECC_PUBKEY_SIZE)
return -EINVAL;
diff --git a/drivers/crypto/atmel-sha204a.c b/drivers/crypto/atmel-sha204a.c
index 8a3520c9a0ff..d2031abbd8ef 100644
--- a/drivers/crypto/atmel-sha204a.c
+++ b/drivers/crypto/atmel-sha204a.c
@@ -31,10 +31,14 @@ static void atmel_sha204a_rng_done(struct atmel_i2c_work_data *work_data,
struct atmel_i2c_client_priv *i2c_priv = work_data->ctx;
struct hwrng *rng = areq;
- if (status)
+ if (status) {
dev_warn_ratelimited(&i2c_priv->client->dev,
"i2c transaction failed (%d)\n",
status);
+ kfree(work_data);
+ atomic_dec(&i2c_priv->tfm_count);
+ return;
+ }
rng->priv = (unsigned long)work_data;
atomic_dec(&i2c_priv->tfm_count);
diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index d6a0773d6da8..6dc380537ece 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -1244,7 +1244,7 @@ static int __sev_platform_init_handle_init_ex_path(struct sev_device *sev)
if (sev_init_ex_buffer)
return 0;
- page = alloc_pages(GFP_KERNEL, get_order(NV_LENGTH));
+ page = alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(NV_LENGTH));
if (!page) {
dev_err(sev->dev, "SEV: INIT_EX NV memory allocation failed\n");
return -ENOMEM;
@@ -1254,7 +1254,7 @@ static int __sev_platform_init_handle_init_ex_path(struct sev_device *sev)
rc = sev_read_init_ex_file();
if (rc)
- return rc;
+ goto err_free;
/* If SEV-SNP is initialized, transition to firmware page. */
if (sev->snp_initialized) {
@@ -1263,11 +1263,22 @@ static int __sev_platform_init_handle_init_ex_path(struct sev_device *sev)
npages = 1UL << get_order(NV_LENGTH);
if (rmp_mark_pages_firmware(__pa(sev_init_ex_buffer), npages, false)) {
dev_err(sev->dev, "SEV: INIT_EX NV memory page state change failed.\n");
- return -ENOMEM;
+ rc = -ENOMEM;
+ /*
+ * Pages can be in an inconsistent state, don't release them back to the
+ * system.
+ */
+ goto err_reset;
}
}
return 0;
+
+err_free:
+ __free_pages(page, get_order(NV_LENGTH));
+err_reset:
+ sev_init_ex_buffer = NULL;
+ return rc;
}
static int __sev_platform_init_locked(int *error)
diff --git a/drivers/crypto/gemini/sl3516-ce-cipher.c b/drivers/crypto/gemini/sl3516-ce-cipher.c
index 583010b2d007..02ec4282333b 100644
--- a/drivers/crypto/gemini/sl3516-ce-cipher.c
+++ b/drivers/crypto/gemini/sl3516-ce-cipher.c
@@ -56,10 +56,6 @@ static bool sl3516_ce_need_fallback(struct skcipher_request *areq)
ce->fallback_mod16++;
return true;
}
- if ((sg_dma_len(sg) % 16) != 0) {
- ce->fallback_mod16++;
- return true;
- }
if (!IS_ALIGNED(sg->offset, 16)) {
ce->fallback_align16++;
return true;
@@ -72,10 +68,6 @@ static bool sl3516_ce_need_fallback(struct skcipher_request *areq)
ce->fallback_mod16++;
return true;
}
- if ((sg_dma_len(sg) % 16) != 0) {
- ce->fallback_mod16++;
- return true;
- }
if (!IS_ALIGNED(sg->offset, 16)) {
ce->fallback_align16++;
return true;
diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hisilicon/sec2/sec_crypto.c
index cdd485fcbc5b..760b2513db14 100644
--- a/drivers/crypto/hisilicon/sec2/sec_crypto.c
+++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c
@@ -98,7 +98,6 @@
#define IV_FLAGS_OFFSET 0x6
#define IV_CM_OFFSET 0x3
#define IV_LAST_BYTE1 1
-#define IV_LAST_BYTE2 2
#define IV_LAST_BYTE_MASK 0xFF
#define IV_CTR_INIT 0x1
#define IV_BYTE_OFFSET 0x8
@@ -1465,7 +1464,7 @@ static void set_aead_auth_iv(struct sec_ctx *ctx, struct sec_req *req)
struct sec_cipher_req *c_req = &req->c_req;
u32 data_size = aead_req->cryptlen;
u8 flage = 0;
- u8 cm, cl;
+ u8 cm, cl, i;
/* the specification has been checked in aead_iv_demension_check() */
cl = c_req->c_ivin[0] + 1;
@@ -1489,15 +1488,16 @@ static void set_aead_auth_iv(struct sec_ctx *ctx, struct sec_req *req)
* the last 32bit is counter's initial number,
* but the nonce uses the first 16bit
* the tail 16bit fill with the cipher length
+ * When CL is 3, the tail 24bit fill with the cipher length.
*/
if (!c_req->encrypt)
data_size = aead_req->cryptlen - authsize;
- a_req->a_ivin[ctx->c_ctx.ivsize - IV_LAST_BYTE1] =
- data_size & IV_LAST_BYTE_MASK;
- data_size >>= IV_BYTE_OFFSET;
- a_req->a_ivin[ctx->c_ctx.ivsize - IV_LAST_BYTE2] =
+ for (i = 1; i <= cl; i++) {
+ a_req->a_ivin[ctx->c_ctx.ivsize - i] =
data_size & IV_LAST_BYTE_MASK;
+ data_size >>= IV_BYTE_OFFSET;
+ }
}
static void sec_aead_set_iv(struct sec_ctx *ctx, struct sec_req *req)
diff --git a/drivers/crypto/intel/keembay/keembay-ocs-aes-core.c b/drivers/crypto/intel/keembay/keembay-ocs-aes-core.c
index 9b2d098e5eb2..9ef6a6d58c68 100644
--- a/drivers/crypto/intel/keembay/keembay-ocs-aes-core.c
+++ b/drivers/crypto/intel/keembay/keembay-ocs-aes-core.c
@@ -1541,7 +1541,7 @@ static int register_aes_algs(struct ocs_aes_dev *aes_dev)
/*
* If any algorithm fails to register, all preceding algorithms that
- * were successfully registered will be automatically unregistered.
+ * were registered in the same call are automatically unregistered.
*/
ret = crypto_engine_register_aeads(algs_aead, ARRAY_SIZE(algs_aead));
if (ret)
@@ -1549,7 +1549,7 @@ static int register_aes_algs(struct ocs_aes_dev *aes_dev)
ret = crypto_engine_register_skciphers(algs, ARRAY_SIZE(algs));
if (ret)
- crypto_engine_unregister_aeads(algs_aead, ARRAY_SIZE(algs));
+ crypto_engine_unregister_aeads(algs_aead, ARRAY_SIZE(algs_aead));
return ret;
}
@@ -1561,6 +1561,7 @@ static const struct of_device_id kmb_ocs_aes_of_match[] = {
},
{}
};
+MODULE_DEVICE_TABLE(of, kmb_ocs_aes_of_match);
static void kmb_ocs_aes_remove(struct platform_device *pdev)
{
@@ -1602,6 +1603,8 @@ static int kmb_ocs_aes_probe(struct platform_device *pdev)
if (IS_ERR(aes_dev->base_reg))
return PTR_ERR(aes_dev->base_reg);
+ init_completion(&aes_dev->irq_completion);
+
/* Get and request IRQ */
aes_dev->irq = platform_get_irq(pdev, 0);
if (aes_dev->irq < 0)
@@ -1619,8 +1622,6 @@ static int kmb_ocs_aes_probe(struct platform_device *pdev)
list_add_tail(&aes_dev->list, &ocs_aes.dev_list);
spin_unlock(&ocs_aes.lock);
- init_completion(&aes_dev->irq_completion);
-
/* Initialize crypto engine */
aes_dev->engine = crypto_engine_alloc_init(dev, true);
if (!aes_dev->engine) {
diff --git a/drivers/crypto/intel/qat/qat_common/adf_aer.c b/drivers/crypto/intel/qat/qat_common/adf_aer.c
index bd81458bcf32..403e53347f8e 100644
--- a/drivers/crypto/intel/qat/qat_common/adf_aer.c
+++ b/drivers/crypto/intel/qat/qat_common/adf_aer.c
@@ -140,6 +140,8 @@ static void adf_device_reset_worker(struct work_struct *work)
queue_work(device_sriov_wq, &sriov_data.sriov_work);
if (wait_for_completion_timeout(&sriov_data.compl, wait_jiffies))
adf_pf2vf_notify_restarted(accel_dev);
+ else
+ cancel_work_sync(&sriov_data.sriov_work);
adf_dev_restarted_notify(accel_dev);
clear_bit(ADF_STATUS_RESTARTING, &accel_dev->status);
diff --git a/drivers/crypto/intel/qat/qat_common/qat_algs.c b/drivers/crypto/intel/qat/qat_common/qat_algs.c
index d69cc1e5e023..274bcc83a1fc 100644
--- a/drivers/crypto/intel/qat/qat_common/qat_algs.c
+++ b/drivers/crypto/intel/qat/qat_common/qat_algs.c
@@ -481,6 +481,7 @@ static void qat_alg_xts_reverse_key(const u8 *key_forward, unsigned int keylen,
memcpy(key_reverse + AES_BLOCK_SIZE, key - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
}
+ memzero_explicit(&aes_expanded, sizeof(aes_expanded));
}
static void qat_alg_skcipher_init_dec(struct qat_alg_skcipher_ctx *ctx,
diff --git a/drivers/crypto/rockchip/rk3288_crypto_ahash.c b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
index ba26b4db7a60..f726e53df9fe 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_ahash.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
@@ -322,7 +322,12 @@ static int rk_hash_run(struct crypto_engine *engine, void *breq)
* efficiency, and make it response quickly when dma
* complete.
*/
- readl_poll_timeout(rkc->reg + RK_CRYPTO_HASH_STS, v, v == 0, 10, 1000);
+ err = readl_poll_timeout(rkc->reg + RK_CRYPTO_HASH_STS, v,
+ v == 0, 10, 1000);
+ if (err) {
+ dev_err(rkc->dev, "HASH idle timeout\n");
+ goto theend;
+ }
for (i = 0; i < crypto_ahash_digestsize(tfm) / 4; i++) {
v = readl(rkc->reg + RK_CRYPTO_HASH_DOUT_0 + i * 4);
diff --git a/drivers/crypto/sa2ul.c b/drivers/crypto/sa2ul.c
index d1c2d129ad41..a8f5f475acbe 100644
--- a/drivers/crypto/sa2ul.c
+++ b/drivers/crypto/sa2ul.c
@@ -2426,7 +2426,10 @@ static int sa_ul_probe(struct platform_device *pdev)
return ret;
}
- sa_init_mem(dev_data);
+ ret = sa_init_mem(dev_data);
+ if (ret)
+ goto disable_pm;
+
ret = sa_dma_init(dev_data);
if (ret)
goto destroy_dma_pool;
@@ -2461,6 +2464,7 @@ static int sa_ul_probe(struct platform_device *pdev)
destroy_dma_pool:
dma_pool_destroy(dev_data->sc_pool);
+disable_pm:
pm_runtime_put_sync(dev);
pm_runtime_disable(dev);
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 0c62b4069ba0..2433deaf08f1 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -4,6 +4,8 @@
#ifndef __CXL_CORE_H__
#define __CXL_CORE_H__
+#include <cxl/mailbox.h>
+
extern const struct device_type cxl_nvdimm_bridge_type;
extern const struct device_type cxl_nvdimm_type;
extern const struct device_type cxl_pmu_type;
@@ -65,9 +67,9 @@ static inline void cxl_region_exit(void)
struct cxl_send_command;
struct cxl_mem_query_commands;
-int cxl_query_cmd(struct cxl_memdev *cxlmd,
+int cxl_query_cmd(struct cxl_mailbox *cxl_mbox,
struct cxl_mem_query_commands __user *q);
-int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s);
+int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s);
void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
resource_size_t length);
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 5175138c4fb7..b094aac78ca1 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -349,40 +349,35 @@ static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in)
return true;
}
-static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox,
- struct cxl_memdev_state *mds, u16 opcode,
+static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd,
+ struct cxl_mailbox *cxl_mbox, u16 opcode,
size_t in_size, size_t out_size, u64 in_payload)
{
- struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
- *mbox = (struct cxl_mbox_cmd) {
+ *mbox_cmd = (struct cxl_mbox_cmd) {
.opcode = opcode,
.size_in = in_size,
};
if (in_size) {
- mbox->payload_in = vmemdup_user(u64_to_user_ptr(in_payload),
- in_size);
- if (IS_ERR(mbox->payload_in))
- return PTR_ERR(mbox->payload_in);
+ mbox_cmd->payload_in = vmemdup_user(u64_to_user_ptr(in_payload),
+ in_size);
+ if (IS_ERR(mbox_cmd->payload_in))
+ return PTR_ERR(mbox_cmd->payload_in);
- if (!cxl_payload_from_user_allowed(opcode, mbox->payload_in)) {
- dev_dbg(mds->cxlds.dev, "%s: input payload not allowed\n",
+ if (!cxl_payload_from_user_allowed(opcode, mbox_cmd->payload_in)) {
+ dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n",
cxl_mem_opcode_to_name(opcode));
- kvfree(mbox->payload_in);
+ kvfree(mbox_cmd->payload_in);
return -EBUSY;
}
}
- /* Prepare to handle a full payload for variable sized output */
- if (out_size == CXL_VARIABLE_PAYLOAD)
- mbox->size_out = cxl_mbox->payload_size;
- else
- mbox->size_out = out_size;
+ mbox_cmd->size_out = min_t(size_t, out_size, cxl_mbox->payload_size);
- if (mbox->size_out) {
- mbox->payload_out = kvzalloc(mbox->size_out, GFP_KERNEL);
- if (!mbox->payload_out) {
- kvfree(mbox->payload_in);
+ if (mbox_cmd->size_out) {
+ mbox_cmd->payload_out = kvzalloc(mbox_cmd->size_out, GFP_KERNEL);
+ if (!mbox_cmd->payload_out) {
+ kvfree(mbox_cmd->payload_in);
return -ENOMEM;
}
}
@@ -397,10 +392,8 @@ static void cxl_mbox_cmd_dtor(struct cxl_mbox_cmd *mbox)
static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd,
const struct cxl_send_command *send_cmd,
- struct cxl_memdev_state *mds)
+ struct cxl_mailbox *cxl_mbox)
{
- struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
-
if (send_cmd->raw.rsvd)
return -EINVAL;
@@ -415,7 +408,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd,
if (!cxl_mem_raw_command_allowed(send_cmd->raw.opcode))
return -EPERM;
- dev_WARN_ONCE(mds->cxlds.dev, true, "raw command path used\n");
+ dev_WARN_ONCE(cxl_mbox->host, true, "raw command path used\n");
*mem_cmd = (struct cxl_mem_command) {
.info = {
@@ -431,7 +424,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd,
static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd,
const struct cxl_send_command *send_cmd,
- struct cxl_memdev_state *mds)
+ struct cxl_mailbox *cxl_mbox)
{
struct cxl_mem_command *c = &cxl_mem_commands[send_cmd->id];
const struct cxl_command_info *info = &c->info;
@@ -446,11 +439,11 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd,
return -EINVAL;
/* Check that the command is enabled for hardware */
- if (!test_bit(info->id, mds->enabled_cmds))
+ if (!test_bit(info->id, cxl_mbox->enabled_cmds))
return -ENOTTY;
/* Check that the command is not claimed for exclusive kernel use */
- if (test_bit(info->id, mds->exclusive_cmds))
+ if (test_bit(info->id, cxl_mbox->exclusive_cmds))
return -EBUSY;
/* Check the input buffer is the expected size */
@@ -479,7 +472,7 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd,
/**
* cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND.
* @mbox_cmd: Sanitized and populated &struct cxl_mbox_cmd.
- * @mds: The driver data for the operation
+ * @cxl_mbox: CXL mailbox context
* @send_cmd: &struct cxl_send_command copied in from userspace.
*
* Return:
@@ -494,10 +487,9 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd,
* safe to send to the hardware.
*/
static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd,
- struct cxl_memdev_state *mds,
+ struct cxl_mailbox *cxl_mbox,
const struct cxl_send_command *send_cmd)
{
- struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
struct cxl_mem_command mem_cmd;
int rc;
@@ -514,24 +506,23 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd,
/* Sanitize and construct a cxl_mem_command */
if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW)
- rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, mds);
+ rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, cxl_mbox);
else
- rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, mds);
+ rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, cxl_mbox);
if (rc)
return rc;
/* Sanitize and construct a cxl_mbox_cmd */
- return cxl_mbox_cmd_ctor(mbox_cmd, mds, mem_cmd.opcode,
+ return cxl_mbox_cmd_ctor(mbox_cmd, cxl_mbox, mem_cmd.opcode,
mem_cmd.info.size_in, mem_cmd.info.size_out,
send_cmd->in.payload);
}
-int cxl_query_cmd(struct cxl_memdev *cxlmd,
+int cxl_query_cmd(struct cxl_mailbox *cxl_mbox,
struct cxl_mem_query_commands __user *q)
{
- struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
- struct device *dev = &cxlmd->dev;
+ struct device *dev = cxl_mbox->host;
struct cxl_mem_command *cmd;
u32 n_commands;
int j = 0;
@@ -552,9 +543,9 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd,
cxl_for_each_cmd(cmd) {
struct cxl_command_info info = cmd->info;
- if (test_bit(info.id, mds->enabled_cmds))
+ if (test_bit(info.id, cxl_mbox->enabled_cmds))
info.flags |= CXL_MEM_COMMAND_FLAG_ENABLED;
- if (test_bit(info.id, mds->exclusive_cmds))
+ if (test_bit(info.id, cxl_mbox->exclusive_cmds))
info.flags |= CXL_MEM_COMMAND_FLAG_EXCLUSIVE;
if (copy_to_user(&q->commands[j++], &info, sizeof(info)))
@@ -569,7 +560,7 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd,
/**
* handle_mailbox_cmd_from_user() - Dispatch a mailbox command for userspace.
- * @mds: The driver data for the operation
+ * @cxl_mbox: The mailbox context for the operation.
* @mbox_cmd: The validated mailbox command.
* @out_payload: Pointer to userspace's output payload.
* @size_out: (Input) Max payload size to copy out.
@@ -590,13 +581,12 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd,
*
* See cxl_send_cmd().
*/
-static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds,
+static int handle_mailbox_cmd_from_user(struct cxl_mailbox *cxl_mbox,
struct cxl_mbox_cmd *mbox_cmd,
u64 out_payload, s32 *size_out,
u32 *retval)
{
- struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
- struct device *dev = mds->cxlds.dev;
+ struct device *dev = cxl_mbox->host;
int rc;
dev_dbg(dev,
@@ -633,10 +623,9 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds,
return rc;
}
-int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s)
+int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s)
{
- struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
- struct device *dev = &cxlmd->dev;
+ struct device *dev = cxl_mbox->host;
struct cxl_send_command send;
struct cxl_mbox_cmd mbox_cmd;
int rc;
@@ -646,11 +635,11 @@ int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s)
if (copy_from_user(&send, s, sizeof(send)))
return -EFAULT;
- rc = cxl_validate_cmd_from_user(&mbox_cmd, mds, &send);
+ rc = cxl_validate_cmd_from_user(&mbox_cmd, cxl_mbox, &send);
if (rc)
return rc;
- rc = handle_mailbox_cmd_from_user(mds, &mbox_cmd, send.out.payload,
+ rc = handle_mailbox_cmd_from_user(cxl_mbox, &mbox_cmd, send.out.payload,
&send.out.size, &send.retval);
if (rc)
return rc;
@@ -724,6 +713,7 @@ static int cxl_xfer_log(struct cxl_memdev_state *mds, uuid_t *uuid,
*/
static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel)
{
+ struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
struct cxl_cel_entry *cel_entry;
const int cel_entries = size / sizeof(*cel_entry);
struct device *dev = mds->cxlds.dev;
@@ -737,7 +727,7 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel)
int enabled = 0;
if (cmd) {
- set_bit(cmd->info.id, mds->enabled_cmds);
+ set_bit(cmd->info.id, cxl_mbox->enabled_cmds);
enabled++;
}
@@ -807,6 +797,7 @@ static const uuid_t log_uuid[] = {
*/
int cxl_enumerate_cmds(struct cxl_memdev_state *mds)
{
+ struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
struct cxl_mbox_get_supported_logs *gsl;
struct device *dev = mds->cxlds.dev;
struct cxl_mem_command *cmd;
@@ -845,7 +836,7 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds)
/* In case CEL was bogus, enable some default commands. */
cxl_for_each_cmd(cmd)
if (cmd->flags & CXL_CMD_FLAG_FORCE_ENABLE)
- set_bit(cmd->info.id, mds->enabled_cmds);
+ set_bit(cmd->info.id, cxl_mbox->enabled_cmds);
/* Found the required CEL */
rc = 0;
@@ -1367,6 +1358,11 @@ int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len,
if (rc)
break;
+ if (!le16_to_cpu(po->count)) {
+ dev_dbg(&cxlmd->dev, "Poison empty payload!\n");
+ break;
+ }
+
for (int i = 0; i < le16_to_cpu(po->count); i++)
trace_cxl_poison(cxlmd, cxlr, &po->record[i],
po->flags, po->overflow_ts,
@@ -1448,6 +1444,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev)
mutex_init(&mds->event.log_lock);
mds->cxlds.dev = dev;
mds->cxlds.reg_map.host = dev;
+ mds->cxlds.cxl_mbox.host = dev;
mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE;
mds->cxlds.type = CXL_DEVTYPE_CLASSMEM;
mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID;
diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
index 84fefb76dafa..d2a554c2f92b 100644
--- a/drivers/cxl/core/memdev.c
+++ b/drivers/cxl/core/memdev.c
@@ -564,9 +564,11 @@ EXPORT_SYMBOL_NS_GPL(is_cxl_memdev, CXL);
void set_exclusive_cxl_commands(struct cxl_memdev_state *mds,
unsigned long *cmds)
{
+ struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
+
down_write(&cxl_memdev_rwsem);
- bitmap_or(mds->exclusive_cmds, mds->exclusive_cmds, cmds,
- CXL_MEM_COMMAND_ID_MAX);
+ bitmap_or(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds,
+ cmds, CXL_MEM_COMMAND_ID_MAX);
up_write(&cxl_memdev_rwsem);
}
EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, CXL);
@@ -579,9 +581,11 @@ EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, CXL);
void clear_exclusive_cxl_commands(struct cxl_memdev_state *mds,
unsigned long *cmds)
{
+ struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
+
down_write(&cxl_memdev_rwsem);
- bitmap_andnot(mds->exclusive_cmds, mds->exclusive_cmds, cmds,
- CXL_MEM_COMMAND_ID_MAX);
+ bitmap_andnot(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds,
+ cmds, CXL_MEM_COMMAND_ID_MAX);
up_write(&cxl_memdev_rwsem);
}
EXPORT_SYMBOL_NS_GPL(clear_exclusive_cxl_commands, CXL);
@@ -656,11 +660,14 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds,
static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd,
unsigned long arg)
{
+ struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds);
+ struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
+
switch (cmd) {
case CXL_MEM_QUERY_COMMANDS:
- return cxl_query_cmd(cxlmd, (void __user *)arg);
+ return cxl_query_cmd(cxl_mbox, (void __user *)arg);
case CXL_MEM_SEND_COMMAND:
- return cxl_send_cmd(cxlmd, (void __user *)arg);
+ return cxl_send_cmd(cxl_mbox, (void __user *)arg);
default:
return -ENOTTY;
}
@@ -838,7 +845,7 @@ static enum fw_upload_err cxl_fw_prepare(struct fw_upload *fwl, const u8 *data,
if (!size)
return FW_UPLOAD_ERR_INVALID_SIZE;
- mds->fw.oneshot = struct_size(transfer, data, size) <
+ mds->fw.oneshot = struct_size(transfer, data, size) <=
cxl_mbox->payload_size;
if (cxl_mem_get_fw_info(mds))
@@ -994,10 +1001,11 @@ static void cxl_remove_fw_upload(void *fwl)
int devm_cxl_setup_fw_upload(struct device *host, struct cxl_memdev_state *mds)
{
struct cxl_dev_state *cxlds = &mds->cxlds;
+ struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox;
struct device *dev = &cxlds->cxlmd->dev;
struct fw_upload *fwl;
- if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, mds->enabled_cmds))
+ if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, cxl_mbox->enabled_cmds))
return 0;
fwl = firmware_upload_register(THIS_MODULE, dev, dev_name(dev),
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index f5429666822f..009c0a262ee8 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -1779,14 +1779,13 @@ static int find_pos_and_ways(struct cxl_port *port, struct range *range,
break;
}
}
- put_device(dev);
-
if (rc)
dev_err(port->uport_dev,
"failed to find %s:%s in target list of %s\n",
dev_name(&port->dev),
- dev_name(port->parent_dport->dport_dev),
- dev_name(&cxlsd->cxld.dev));
+ dev_name(port->parent_dport->dport_dev), dev_name(dev));
+
+ put_device(dev);
return rc;
}
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 2a25d1957ddb..a0a49809cd76 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -106,42 +106,6 @@ static inline struct cxl_ep *cxl_ep_load(struct cxl_port *port,
return xa_load(&port->endpoints, (unsigned long)&cxlmd->dev);
}
-/**
- * struct cxl_mbox_cmd - A command to be submitted to hardware.
- * @opcode: (input) The command set and command submitted to hardware.
- * @payload_in: (input) Pointer to the input payload.
- * @payload_out: (output) Pointer to the output payload. Must be allocated by
- * the caller.
- * @size_in: (input) Number of bytes to load from @payload_in.
- * @size_out: (input) Max number of bytes loaded into @payload_out.
- * (output) Number of bytes generated by the device. For fixed size
- * outputs commands this is always expected to be deterministic. For
- * variable sized output commands, it tells the exact number of bytes
- * written.
- * @min_out: (input) internal command output payload size validation
- * @poll_count: (input) Number of timeouts to attempt.
- * @poll_interval_ms: (input) Time between mailbox background command polling
- * interval timeouts.
- * @return_code: (output) Error code returned from hardware.
- *
- * This is the primary mechanism used to send commands to the hardware.
- * All the fields except @payload_* correspond exactly to the fields described in
- * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and
- * @payload_out are written to, and read from the Command Payload Registers
- * defined in CXL 2.0 8.2.8.4.8.
- */
-struct cxl_mbox_cmd {
- u16 opcode;
- void *payload_in;
- void *payload_out;
- size_t size_in;
- size_t size_out;
- size_t min_out;
- int poll_count;
- int poll_interval_ms;
- u16 return_code;
-};
-
/*
* Per CXL 3.0 Section 8.2.8.4.5.1
*/
@@ -461,8 +425,6 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
* @lsa_size: Size of Label Storage Area
* (CXL 2.0 8.2.9.5.1.1 Identify Memory Device)
* @firmware_version: Firmware version for the memory device.
- * @enabled_cmds: Hardware commands found enabled in CEL.
- * @exclusive_cmds: Commands that are kernel-internal only
* @total_bytes: sum of all possible capacities
* @volatile_only_bytes: hard volatile capacity
* @persistent_only_bytes: hard persistent capacity
@@ -485,8 +447,6 @@ struct cxl_memdev_state {
struct cxl_dev_state cxlds;
size_t lsa_size;
char firmware_version[0x10];
- DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX);
- DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX);
u64 total_bytes;
u64 volatile_only_bytes;
u64 persistent_only_bytes;
diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 09a2bc817d64..82cbacb3e2ca 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -794,12 +794,6 @@ static int cxl_pci_type3_init_mailbox(struct cxl_dev_state *cxlds)
{
int rc;
- /*
- * Fail the init if there's no mailbox. For a type3 this is out of spec.
- */
- if (!cxlds->reg_map.device_map.mbox.valid)
- return -ENODEV;
-
rc = cxl_mailbox_init(&cxlds->cxl_mbox, cxlds->dev);
if (rc)
return rc;
@@ -857,10 +851,13 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
*/
rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT,
&cxlds->reg_map);
- if (rc)
+ if (rc) {
+ if (rc == -EPROBE_DEFER)
+ return rc;
dev_warn(&pdev->dev, "No component registers (%d)\n", rc);
- else if (!cxlds->reg_map.component_map.ras.valid)
+ } else if (!cxlds->reg_map.component_map.ras.valid) {
dev_dbg(&pdev->dev, "RAS registers not found\n");
+ }
rc = cxl_map_component_regs(&cxlds->reg_map, &cxlds->regs.component,
BIT(CXL_CM_CAP_CAP_ID_RAS));
diff --git a/drivers/dax/super.c b/drivers/dax/super.c
index e16d1d40d773..6eb7ecd23384 100644
--- a/drivers/dax/super.c
+++ b/drivers/dax/super.c
@@ -233,6 +233,7 @@ EXPORT_SYMBOL_GPL(dax_recovery_write);
int dax_holder_notify_failure(struct dax_device *dax_dev, u64 off,
u64 len, int mf_flags)
{
+ const struct dax_holder_operations *ops;
int rc, id;
id = dax_read_lock();
@@ -241,12 +242,19 @@ int dax_holder_notify_failure(struct dax_device *dax_dev, u64 off,
goto out;
}
- if (!dax_dev->holder_ops) {
+ /*
+ * Read holder_ops once: a concurrent fs_put_dax() can clear it without
+ * synchronizing against readers. Without the single fetch the compiler
+ * could reload between the NULL check and the call and dereference a
+ * NULL ops.
+ */
+ ops = READ_ONCE(dax_dev->holder_ops);
+ if (!ops) {
rc = -EOPNOTSUPP;
goto out;
}
- rc = dax_dev->holder_ops->notify_failure(dax_dev, off, len, mf_flags);
+ rc = ops->notify_failure(dax_dev, off, len, mf_flags);
out:
dax_read_unlock(id);
return rc;
diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
index 9ae789d4aca7..1fd2dbc5d717 100644
--- a/drivers/dma/dw-edma/dw-edma-core.c
+++ b/drivers/dma/dw-edma/dw-edma-core.c
@@ -7,6 +7,7 @@
*/
#include <linux/module.h>
+#include <linux/delay.h>
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/dmaengine.h>
@@ -212,6 +213,35 @@ static int dw_edma_start_transfer(struct dw_edma_chan *chan)
return 1;
}
+static void dw_edma_terminate_vdesc(struct virt_dma_desc *vd)
+{
+ list_del(&vd->node);
+ dma_cookie_complete(&vd->tx);
+ vchan_terminate_vdesc(vd);
+}
+
+static void dw_edma_terminate_vdesc_list(struct list_head *head)
+{
+ struct virt_dma_desc *vd, *_vd;
+
+ list_for_each_entry_safe(vd, _vd, head, node)
+ dw_edma_terminate_vdesc(vd);
+}
+
+/* Must be called with vc.lock held. */
+static void dw_edma_terminate_all_descs(struct dw_edma_chan *chan)
+{
+ /*
+ * This order must not be reversed. Cookies are assigned when
+ * descriptors are submitted, so desc_issued contains older cookies
+ * than desc_submitted. Completing desc_submitted first could move
+ * chan->vc.chan.completed_cookie backwards when desc_issued is
+ * terminated afterwards.
+ */
+ dw_edma_terminate_vdesc_list(&chan->vc.desc_issued);
+ dw_edma_terminate_vdesc_list(&chan->vc.desc_submitted);
+}
+
static void dw_edma_device_caps(struct dma_chan *dchan,
struct dma_slave_caps *caps)
{
@@ -246,6 +276,8 @@ static int dw_edma_device_pause(struct dma_chan *dchan)
struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
int err = 0;
+ guard(spinlock_irqsave)(&chan->vc.lock);
+
if (!chan->configured)
err = -EPERM;
else if (chan->status != EDMA_ST_BUSY)
@@ -263,6 +295,8 @@ static int dw_edma_device_resume(struct dma_chan *dchan)
struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
int err = 0;
+ guard(spinlock_irqsave)(&chan->vc.lock);
+
if (!chan->configured) {
err = -EPERM;
} else if (chan->status != EDMA_ST_PAUSE) {
@@ -271,7 +305,8 @@ static int dw_edma_device_resume(struct dma_chan *dchan)
err = -EPERM;
} else {
chan->status = EDMA_ST_BUSY;
- dw_edma_start_transfer(chan);
+ if (!dw_edma_start_transfer(chan))
+ chan->status = EDMA_ST_IDLE;
}
return err;
@@ -282,25 +317,29 @@ static int dw_edma_device_terminate_all(struct dma_chan *dchan)
struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
int err = 0;
+ guard(spinlock_irqsave)(&chan->vc.lock);
+
if (!chan->configured) {
- /* Do nothing */
+ dw_edma_terminate_all_descs(chan);
} else if (chan->status == EDMA_ST_PAUSE) {
+ dw_edma_terminate_all_descs(chan);
chan->status = EDMA_ST_IDLE;
- chan->configured = false;
} else if (chan->status == EDMA_ST_IDLE) {
- chan->configured = false;
+ dw_edma_terminate_all_descs(chan);
} else if (dw_edma_core_ch_status(chan) == DMA_COMPLETE) {
/*
* The channel is in a false BUSY state, probably didn't
* receive or lost an interrupt
*/
+ dw_edma_terminate_all_descs(chan);
chan->status = EDMA_ST_IDLE;
- chan->configured = false;
} else if (chan->request > EDMA_REQ_PAUSE) {
err = -EPERM;
} else {
chan->request = EDMA_REQ_STOP;
}
+ if (chan->status == EDMA_ST_IDLE)
+ chan->request = EDMA_REQ_NONE;
return err;
}
@@ -310,11 +349,9 @@ static void dw_edma_device_issue_pending(struct dma_chan *dchan)
struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
unsigned long flags;
- if (!chan->configured)
- return;
-
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc) && chan->request == EDMA_REQ_NONE &&
+ if (chan->configured && vchan_issue_pending(&chan->vc) &&
+ chan->request == EDMA_REQ_NONE &&
chan->status == EDMA_ST_IDLE) {
chan->status = EDMA_ST_BUSY;
dw_edma_start_transfer(chan);
@@ -621,10 +658,16 @@ static void dw_edma_done_interrupt(struct dw_edma_chan *chan)
unsigned long flags;
spin_lock_irqsave(&chan->vc.lock, flags);
+ if (chan->status == EDMA_ST_PAUSE) {
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
+ return;
+ }
+
vd = vchan_next_desc(&chan->vc);
if (vd) {
switch (chan->request) {
case EDMA_REQ_NONE:
+ case EDMA_REQ_PAUSE:
desc = vd2dw_edma_desc(vd);
if (!desc->chunks_alloc) {
dw_hdma_set_callback_result(vd,
@@ -633,23 +676,23 @@ static void dw_edma_done_interrupt(struct dw_edma_chan *chan)
vchan_cookie_complete(vd);
}
+ if (chan->request == EDMA_REQ_PAUSE) {
+ chan->request = EDMA_REQ_NONE;
+ chan->status = EDMA_ST_PAUSE;
+ break;
+ }
+
/* Continue transferring if there are remaining chunks or issued requests.
*/
chan->status = dw_edma_start_transfer(chan) ? EDMA_ST_BUSY : EDMA_ST_IDLE;
break;
case EDMA_REQ_STOP:
- list_del(&vd->node);
- vchan_cookie_complete(vd);
+ dw_edma_terminate_all_descs(chan);
chan->request = EDMA_REQ_NONE;
chan->status = EDMA_ST_IDLE;
break;
- case EDMA_REQ_PAUSE:
- chan->request = EDMA_REQ_NONE;
- chan->status = EDMA_ST_PAUSE;
- break;
-
default:
break;
}
@@ -664,14 +707,16 @@ static void dw_edma_abort_interrupt(struct dw_edma_chan *chan)
spin_lock_irqsave(&chan->vc.lock, flags);
vd = vchan_next_desc(&chan->vc);
- if (vd) {
+ if (vd && chan->request == EDMA_REQ_STOP) {
+ dw_edma_terminate_all_descs(chan);
+ } else if (vd) {
dw_hdma_set_callback_result(vd, DMA_TRANS_ABORTED);
list_del(&vd->node);
vchan_cookie_complete(vd);
}
- spin_unlock_irqrestore(&chan->vc.lock, flags);
chan->request = EDMA_REQ_NONE;
chan->status = EDMA_ST_IDLE;
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
}
static inline irqreturn_t dw_edma_interrupt_write(int irq, void *data)
@@ -712,21 +757,49 @@ static int dw_edma_alloc_chan_resources(struct dma_chan *dchan)
return 0;
}
-static void dw_edma_free_chan_resources(struct dma_chan *dchan)
+static void dw_edma_wait_termination(struct dma_chan *dchan)
{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
unsigned long timeout = jiffies + msecs_to_jiffies(5000);
- int ret;
+ bool stopping;
+ /*
+ * A STOP may be deferred to a later interrupt while the channel is still
+ * running. Wait until that handler completes the termination.
+ */
while (time_before(jiffies, timeout)) {
- ret = dw_edma_device_terminate_all(dchan);
- if (!ret)
- break;
+ scoped_guard(spinlock_irqsave, &chan->vc.lock)
+ stopping = chan->request == EDMA_REQ_STOP;
- if (time_after_eq(jiffies, timeout))
+ if (!stopping)
return;
- cpu_relax();
+ fsleep(1000);
}
+
+ dev_warn(chan->dw->chip->dev,
+ "timeout waiting for channel termination\n");
+}
+
+static void dw_edma_device_synchronize(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+
+ dw_edma_wait_termination(dchan);
+ vchan_synchronize(&chan->vc);
+}
+
+static void dw_edma_free_chan_resources(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+
+ dw_edma_device_terminate_all(dchan);
+ dw_edma_device_synchronize(dchan);
+
+ scoped_guard(spinlock_irqsave, &chan->vc.lock)
+ chan->configured = false;
+
+ vchan_free_chan_resources(&chan->vc);
}
static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
@@ -785,7 +858,6 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
else
irq->rd_mask |= BIT(chan->id);
- irq->dw = dw;
memcpy(&chan->msi, &irq->msi, sizeof(chan->msi));
dev_vdbg(dev, "MSI:\t\tChannel %s[%u] addr=0x%.8x%.8x, data=0x%.8x\n",
@@ -823,6 +895,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
dma->device_pause = dw_edma_device_pause;
dma->device_resume = dw_edma_device_resume;
dma->device_terminate_all = dw_edma_device_terminate_all;
+ dma->device_synchronize = dw_edma_device_synchronize;
dma->device_issue_pending = dw_edma_device_issue_pending;
dma->device_tx_status = dw_edma_device_tx_status;
dma->device_prep_slave_sg = dw_edma_device_prep_slave_sg;
@@ -872,6 +945,7 @@ static int dw_edma_irq_request(struct dw_edma *dw,
if (chip->nr_irqs == 1) {
/* Common IRQ shared among all channels */
irq = chip->ops->irq_vector(dev, 0);
+ dw->irq[0].dw = dw;
err = request_irq(irq, dw_edma_interrupt_common,
IRQF_SHARED, dw->name, &dw->irq[0]);
if (err) {
@@ -897,6 +971,7 @@ static int dw_edma_irq_request(struct dw_edma *dw,
for (i = 0; i < (*wr_alloc + *rd_alloc); i++) {
irq = chip->ops->irq_vector(dev, i);
+ dw->irq[i].dw = dw;
err = request_irq(irq,
i < *wr_alloc ?
dw_edma_interrupt_write :
diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw-hdma-v0-core.c
index ce8f7254bab2..d2e5ddbb7b66 100644
--- a/drivers/dma/dw-edma/dw-hdma-v0-core.c
+++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c
@@ -79,7 +79,7 @@ static enum dma_status dw_hdma_v0_core_ch_status(struct dw_edma_chan *chan)
u32 tmp;
tmp = FIELD_GET(HDMA_V0_CH_STATUS_MASK,
- GET_CH_32(dw, chan->id, chan->dir, ch_stat));
+ GET_CH_32(dw, chan->dir, chan->id, ch_stat));
if (tmp == 1)
return DMA_IN_PROGRESS;
diff --git a/drivers/dma/fsl-edma-trace.h b/drivers/dma/fsl-edma-trace.h
index d3541301a247..45d964a3726d 100644
--- a/drivers/dma/fsl-edma-trace.h
+++ b/drivers/dma/fsl-edma-trace.h
@@ -19,14 +19,16 @@ DECLARE_EVENT_CLASS(edma_log_io,
__field(struct fsl_edma_engine *, edma)
__field(void __iomem *, addr)
__field(u32, value)
+ __field(void __iomem *, membase)
),
TP_fast_assign(
__entry->edma = edma;
__entry->addr = addr;
__entry->value = value;
+ __entry->membase = edma->membase;
),
TP_printk("offset %08x: value %08x",
- (u32)(__entry->addr - __entry->edma->membase), __entry->value)
+ (u32)(__entry->addr - __entry->membase), __entry->value)
);
DEFINE_EVENT(edma_log_io, edma_readl,
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 25a4134be36b..ede094e0d090 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -983,7 +983,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
GFP_KERNEL);
if (!hdma_dev)
- return -EINVAL;
+ return -ENOMEM;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
index 3e7f8acf41dd..0c439e85c46d 100644
--- a/drivers/dma/mediatek/mtk-uart-apdma.c
+++ b/drivers/dma/mediatek/mtk-uart-apdma.c
@@ -530,7 +530,7 @@ static int mtk_uart_apdma_probe(struct platform_device *pdev)
for (i = 0; i < mtkd->dma_requests; i++) {
c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
if (!c) {
- rc = -ENODEV;
+ rc = -ENOMEM;
goto err_no_dma;
}
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 348bb9a5c87b..cf647662ae81 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1391,6 +1391,7 @@ static void bam_dma_remove(struct platform_device *pdev)
struct bam_device *bdev = platform_get_drvdata(pdev);
u32 i;
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
pm_runtime_force_suspend(&pdev->dev);
of_dma_controller_free(pdev->dev.of_node);
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index a0361bcee120..bea55dc99673 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -1869,8 +1869,10 @@ static irqreturn_t xilinx_mcdma_irq_handler(int irq, void *data)
if (status & XILINX_MCDMA_IRQ_IOC_MASK) {
spin_lock(&chan->lock);
xilinx_dma_complete_descriptor(chan);
- chan->idle = true;
- chan->start_transfer(chan);
+ if (list_empty(&chan->active_list)) {
+ chan->idle = true;
+ chan->start_transfer(chan);
+ }
spin_unlock(&chan->lock);
}
@@ -1926,8 +1928,10 @@ static irqreturn_t xilinx_dma_irq_handler(int irq, void *data)
XILINX_DMA_DMASR_DLY_CNT_IRQ)) {
spin_lock(&chan->lock);
xilinx_dma_complete_descriptor(chan);
- chan->idle = true;
- chan->start_transfer(chan);
+ if (list_empty(&chan->active_list)) {
+ chan->idle = true;
+ chan->start_transfer(chan);
+ }
spin_unlock(&chan->lock);
}
diff --git a/drivers/firewire/.kunitconfig b/drivers/firewire/.kunitconfig
index 21b7e9eef63d..7406acb00478 100644
--- a/drivers/firewire/.kunitconfig
+++ b/drivers/firewire/.kunitconfig
@@ -6,3 +6,4 @@ CONFIG_FIREWIRE_KUNIT_DEVICE_ATTRIBUTE_TEST=y
CONFIG_FIREWIRE_KUNIT_PACKET_SERDES_TEST=y
CONFIG_FIREWIRE_KUNIT_SELF_ID_SEQUENCE_HELPER_TEST=y
CONFIG_FIREWIRE_KUNIT_OHCI_SERDES_TEST=y
+CONFIG_FIREWIRE_KUNIT_NODE_TREE_TEST=y
diff --git a/drivers/firewire/Kconfig b/drivers/firewire/Kconfig
index 905c82e26ce7..cfbac21660ab 100644
--- a/drivers/firewire/Kconfig
+++ b/drivers/firewire/Kconfig
@@ -81,6 +81,21 @@ config FIREWIRE_KUNIT_SELF_ID_SEQUENCE_HELPER_TEST
For more information on KUnit and unit tests in general, refer
to the KUnit documentation in Documentation/dev-tools/kunit/.
+config FIREWIRE_KUNIT_NODE_TREE_TEST
+ tristate "KUnit tests for node tree" if !KUNIT_ALL_TESTS
+ depends on FIREWIRE && KUNIT
+ default KUNIT_ALL_TESTS
+ help
+ This builds the KUnit tests for node tree.
+
+ KUnit tests run during boot and output the results to the debug
+ log in TAP format (https://testanything.org/). Only useful for
+ kernel devs running KUnit test harness and are not for inclusion
+ into a production build.
+
+ For more information on KUnit and unit tests in general, refer
+ to the KUnit documentation in Documentation/dev-tools/kunit/.
+
config FIREWIRE_OHCI
tristate "OHCI-1394 controllers"
depends on PCI && FIREWIRE && MMU
diff --git a/drivers/firewire/core-topology.c b/drivers/firewire/core-topology.c
index 892b94cfd626..c4a0a3a52b4e 100644
--- a/drivers/firewire/core-topology.c
+++ b/drivers/firewire/core-topology.c
@@ -88,6 +88,17 @@ static inline struct fw_node *fw_node(struct list_head *l)
return list_entry(l, struct fw_node, link);
}
+typedef void (*fw_node_callback_t)(struct fw_card *card, struct fw_node *node,
+ struct fw_node *parent);
+
+static void for_each_fw_node(struct fw_card *card, struct fw_node *root,
+ fw_node_callback_t callback);
+
+static void free_fw_node(struct fw_card *card, struct fw_node *node, struct fw_node *parent)
+{
+ kfree(node);
+}
+
/*
* This function builds the tree representation of the topology given
* by the self IDs from the latest bus reset. During the construction
@@ -119,8 +130,8 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
while (enumerator.quadlet_count > 0) {
unsigned int child_port_count = 0;
+ unsigned int parent_port_count = 0;
unsigned int total_port_count = 0;
- unsigned int parent_count = 0;
unsigned int quadlet_count;
const u32 *self_id_sequence;
unsigned int port_capacity;
@@ -134,7 +145,7 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
if (PTR_ERR(self_id_sequence) != -ENODATA) {
fw_err(card, "inconsistent extended self IDs: %ld\n",
PTR_ERR(self_id_sequence));
- return NULL;
+ goto error;
}
break;
}
@@ -148,26 +159,38 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
switch (port_status) {
case PHY_PACKET_SELF_ID_PORT_STATUS_CHILD:
++child_port_count;
- fallthrough;
+ break;
case PHY_PACKET_SELF_ID_PORT_STATUS_PARENT:
+ ++parent_port_count;
+ break;
case PHY_PACKET_SELF_ID_PORT_STATUS_NCONN:
++total_port_count;
- fallthrough;
+ break;
case PHY_PACKET_SELF_ID_PORT_STATUS_NONE:
default:
break;
}
}
+ total_port_count += child_port_count + parent_port_count;
+
+ // Check that the node reports exactly one parent port, except for the root, which
+ // of course should have no parents.
+ if ((enumerator.quadlet_count == 0 && parent_port_count != 0) ||
+ (enumerator.quadlet_count > 0 && parent_port_count != 1)) {
+ fw_err(card, "parent port inconsistency for node %d: parent_count=%d\n",
+ phy_id, parent_port_count);
+ goto error;
+ }
if (phy_id != phy_packet_self_id_get_phy_id(self_id_sequence[0])) {
fw_err(card, "PHY ID mismatch in self ID: %d != %d\n",
phy_id, phy_packet_self_id_get_phy_id(self_id_sequence[0]));
- return NULL;
+ goto error;
}
if (child_port_count > stack_depth) {
fw_err(card, "topology stack underflow\n");
- return NULL;
+ goto error;
}
/*
@@ -185,7 +208,7 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
node = fw_node_create(self_id_sequence[0], total_port_count, card->color);
if (node == NULL) {
fw_err(card, "out of memory while building topology\n");
- return NULL;
+ goto error;
}
if (phy_id == (card->node_id & 0x3f))
@@ -203,7 +226,6 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
// we temporarily abuse node->color for remembering the entry in
// the node->ports array where the parent node should be. Later,
// when we handle the parent node, we fix up the reference.
- ++parent_count;
node->color = port_index;
break;
@@ -221,21 +243,12 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
}
}
- // Check that the node reports exactly one parent port, except for the root, which
- // of course should have no parents.
- if ((enumerator.quadlet_count == 0 && parent_count != 0) ||
- (enumerator.quadlet_count > 0 && parent_count != 1)) {
- fw_err(card, "parent port inconsistency for node %d: "
- "parent_count=%d\n", phy_id, parent_count);
- return NULL;
- }
-
/* Pop the child nodes off the stack and push the new node. */
__list_del(h->prev, &stack);
list_add_tail(&node->link, &stack);
stack_depth += 1 - child_port_count;
- if (node->phy_speed == SCODE_BETA && parent_count + child_port_count > 1)
+ if (node->phy_speed == SCODE_BETA && parent_port_count + child_port_count > 1)
beta_repeaters_present = true;
// If PHYs report different gap counts, set an invalid count which will force a gap
@@ -254,12 +267,13 @@ static struct fw_node *build_tree(struct fw_card *card, const u32 *sid, int self
card->beta_repeaters_present = beta_repeaters_present;
return local_node;
+error:
+ ++card->color;
+ list_for_each_entry_safe(node, child, &stack, link)
+ for_each_fw_node(card, node, free_fw_node);
+ return NULL;
}
-typedef void (*fw_node_callback_t)(struct fw_card * card,
- struct fw_node * node,
- struct fw_node * parent);
-
static void for_each_fw_node(struct fw_card *card, struct fw_node *root,
fw_node_callback_t callback)
{
@@ -501,3 +515,7 @@ void fw_core_handle_bus_reset(struct fw_card *card, int node_id, int generation,
}
}
EXPORT_SYMBOL(fw_core_handle_bus_reset);
+
+#ifdef CONFIG_FIREWIRE_KUNIT_NODE_TREE_TEST
+#include "node-tree-test.c"
+#endif
diff --git a/drivers/firewire/node-tree-test.c b/drivers/firewire/node-tree-test.c
new file mode 100644
index 000000000000..5e6bb58537e9
--- /dev/null
+++ b/drivers/firewire/node-tree-test.c
@@ -0,0 +1,607 @@
+// SPDX-License-Identifier: GPL-2.0-only
+//
+// node-tree-test.c - An application of Kunit to test node tree.
+//
+// Copyright (c) 2026 Takashi Sakamoto
+//
+// This file can not be built independently since it is intentionally included in core-topology.c.
+
+#include <kunit/test.h>
+#include <kunit/test-bug.h>
+#include <kunit/device.h>
+
+struct private_data {
+ struct fw_card *card;
+ unsigned int release_count;
+};
+
+static int node_tree_test_init(struct kunit *test)
+{
+ struct private_data *data;
+
+ data = kunit_kzalloc(test, sizeof(*data), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, data);
+
+ data->card = kunit_kzalloc(test, sizeof(struct fw_card), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, data->card);
+
+ data->card->device = kunit_device_register(test, "dummy-device");
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, data->card->device);
+
+ test->priv = data;
+
+ return 0;
+}
+
+static void node_tree_test_exit(struct kunit *test)
+{
+ struct private_data *data = test->priv;
+
+ kunit_device_unregister(test, data->card->device);
+ kunit_kfree(test, data->card);
+ kunit_kfree(test, data);
+}
+
+static void release_fw_node(struct fw_card *card, struct fw_node *node, struct fw_node *parent)
+{
+ struct private_data *data = kunit_get_current_test()->priv;
+
+ fw_node_put(node);
+ ++data->release_count;
+}
+
+static void node_tree_test_two_nodes(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 1 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==|==++
+ // |
+ // +-----+
+ // |
+ // ++===|==x==x==++
+ // || P0 P1 P2 ||
+ // || phy 0 ||
+ // ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000080,
+ 0x8100005e,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x01;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 1);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 2);
+}
+
+static void node_tree_test_two_nodes_1394a(struct kunit *test)
+{
+ // root
+ // ++===============++
+ // || phy 0 ||
+ // || P0 P1 P2 P3 ||
+ // ++===|==|==|==|==++
+ // |
+ // +--+
+ // |
+ // ++===|==|==|==|==|==++
+ // || P0 P1 P2 P3 P4 ||
+ // || phy 1 ||
+ // ++==================++
+ //
+ // NOTE: Just for Self-ID Packets Zero and One.
+ static const u32 self_id_sequence[] = {
+ 0x80000065, 0x80814000,
+ 0x8100005d, 0x81810000,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x01;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 4);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+ KUNIT_EXPECT_NULL(test, node->ports[3]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 5);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[1], parent);
+ KUNIT_EXPECT_NULL(test, node->ports[2]);
+ KUNIT_EXPECT_NULL(test, node->ports[3]);
+ KUNIT_EXPECT_NULL(test, node->ports[4]);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 2);
+}
+
+static void node_tree_test_three_nodes_case0(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 2 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==|==++
+ // | |
+ // +--+ +-----------------+
+ // | |
+ // ++===|==|==x==++ ++===|==|==|==++
+ // || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 0 || || phy 1 ||
+ // ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000060,
+ 0x81000058,
+ 0x820000dc,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x02;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[0];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[1], parent);
+
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[2], parent);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 3);
+}
+
+static void node_tree_test_three_nodes_case1(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 2 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==x==++
+ // |
+ // | +-----------+
+ // | | |
+ // ++===|==|==|==++ ++===|==x==x==++
+ // || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 1 || || phy 0 ||
+ // ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000080,
+ 0x8100006c,
+ 0x82000070,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x02;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[1], parent);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 1);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 3);
+}
+
+static void node_tree_test_four_nodes_case0(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 3 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==|==++
+ // |
+ // | +-----------+ +--------------+
+ // | | | | |
+ // ++===|==|==|==++ ++===|==|==x==++ ++===|==x==x==++
+ // || P0 P1 P2 || || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 2 || || phy 1 || || phy 0 ||
+ // ++============++ ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000080,
+ 0x810000b0,
+ 0x8200006c,
+ 0x83000074,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x03);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NULL(test, node->ports[2]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[1], parent);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+
+ parent = node;
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 1);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 4);
+}
+
+static void node_tree_test_four_nodes_case1(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 3 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==x==++
+ // |
+ // | +--------------------------------+
+ // | | +-----------+ |
+ // ++===|==|==|==++ ++===|==x==x==++ ++===|==|==|==++
+ // || P0 P1 P2 || || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 2 || || phy 1 || || phy 0 ||
+ // ++============++ ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x81000080,
+ 0x820000bc,
+ 0x830000d0,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x03);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[0];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 1);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NULL(test, node->ports[2]);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 4);
+}
+
+static void node_tree_test_four_nodes_case2(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 3 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==|==++
+ // | |
+ // | +-----------------------------+
+ // | +--------------+ |
+ // ++===|==|==x==++ ++===|==|==|==++ ++===|==x==x==++
+ // || P0 P1 P2 || || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 1 || || phy 0 || || phy 2 ||
+ // ++============++ ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x810000b0,
+ 0x82000080,
+ 0x830000dc,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x03);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 1);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+
+ node = parent->ports[0];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+
+ parent = node;
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NULL(test, node->ports[2]);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 4);
+}
+
+static void node_tree_test_four_nodes_case3(struct kunit *test)
+{
+ // root
+ // ++============++
+ // || phy 3 ||
+ // || P0 P1 P2 ||
+ // ++===|==|==|==++
+ // | | +--------------------------------+
+ // | +--------------------+ |
+ // | | |
+ // ++===|==|==x==++ ++===|==|==|==++ ++===|==|==x==++
+ // || P0 P1 P2 || || P0 P1 P2 || || P0 P1 P2 ||
+ // || phy 0 || || phy 1 || || phy 2 ||
+ // ++============++ ++============++ ++============++
+ //
+ static const u32 self_id_sequence[] = {
+ 0x80000090,
+ 0x81000058,
+ 0x82000060,
+ 0x830000fc,
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NOT_NULL(test, card->local_node);
+ KUNIT_EXPECT_PTR_EQ(test, card->local_node, card->root_node);
+
+ struct fw_node *node = card->root_node;
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x03);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_NOT_NULL(test, node->ports[2]);
+
+ struct fw_node *parent = node;
+ node = parent->ports[2];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x02);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[1], parent);
+
+ node = parent->ports[1];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x01);
+ KUNIT_EXPECT_EQ(test, node->port_count, 3);
+ KUNIT_EXPECT_NULL(test, node->ports[0]);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[2], parent);
+
+ node = parent->ports[0];
+ KUNIT_EXPECT_EQ(test, node->node_id, LOCAL_BUS | 0x00);
+ KUNIT_EXPECT_EQ(test, node->port_count, 2);
+ KUNIT_EXPECT_PTR_EQ(test, node->ports[0], parent);
+ KUNIT_EXPECT_NULL(test, node->ports[1]);
+
+ ++card->color;
+ for_each_fw_node(card, card->root_node, release_fw_node);
+ KUNIT_EXPECT_EQ(test, data->release_count, 4);
+}
+
+static void node_tree_test_invalid_extended_self_id_sequence(struct kunit *test)
+{
+ // Use the same node tree as node_tree_test_four_nodes_case1, except for the invalid
+ // content of self ID packet for the phy 3.
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x81000080,
+ 0x820000bc,
+ 0x830000d1, // Invalid.
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NULL(test, card->local_node);
+}
+
+static void node_tree_test_invalid_phy_id(struct kunit *test)
+{
+ // Use the same node tree as node_tree_test_four_nodes_case1, except for the invalid
+ // phy ID for phy 3.
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x81000080,
+ 0x820000bc,
+ 0x8f0000d0, // Invalid.
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NULL(test, card->local_node);
+}
+
+static void node_tree_test_invalid_child_port_count(struct kunit *test)
+{
+ // Use the same node tree as node_tree_test_four_nodes_case1, except for the invalid
+ // count of child ports for phy 3.
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x81000080,
+ 0x820000bc,
+ 0x830000fc, // Invalid.
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NULL(test, card->local_node);
+}
+
+static void node_tree_test_invalid_parent_port_count(struct kunit *test)
+{
+ // Use the same node tree as node_tree_test_four_nodes_case1, except for the invalid
+ // count of parent ports for phy 3.
+ static const u32 self_id_sequence[] = {
+ 0x80000094,
+ 0x81000080,
+ 0x820000bc,
+ 0x830000e8, // Invalid.
+ };
+ struct private_data *data = test->priv;
+ struct fw_card *card = data->card;
+
+ card->node_id = LOCAL_BUS | 0x03;
+
+ card->local_node = build_tree(card, self_id_sequence, ARRAY_SIZE(self_id_sequence), 123);
+ KUNIT_EXPECT_NULL(test, card->local_node);
+}
+
+static struct kunit_case node_tree_test_cases[] = {
+ KUNIT_CASE(node_tree_test_two_nodes),
+ KUNIT_CASE(node_tree_test_two_nodes_1394a),
+ KUNIT_CASE(node_tree_test_three_nodes_case0),
+ KUNIT_CASE(node_tree_test_three_nodes_case1),
+ KUNIT_CASE(node_tree_test_four_nodes_case0),
+ KUNIT_CASE(node_tree_test_four_nodes_case1),
+ KUNIT_CASE(node_tree_test_four_nodes_case2),
+ KUNIT_CASE(node_tree_test_four_nodes_case3),
+ KUNIT_CASE(node_tree_test_invalid_extended_self_id_sequence),
+ KUNIT_CASE(node_tree_test_invalid_phy_id),
+ KUNIT_CASE(node_tree_test_invalid_child_port_count),
+ KUNIT_CASE(node_tree_test_invalid_parent_port_count),
+ {}
+};
+
+static struct kunit_suite node_tree_test_suite = {
+ .name = "firewire-node-tree",
+ .init = node_tree_test_init,
+ .exit = node_tree_test_exit,
+ .test_cases = node_tree_test_cases,
+};
+kunit_test_suite(node_tree_test_suite);
diff --git a/drivers/firmware/arm_scmi/bus.c b/drivers/firmware/arm_scmi/bus.c
index 73a6ab4a224d..771b6da2d83b 100644
--- a/drivers/firmware/arm_scmi/bus.c
+++ b/drivers/firmware/arm_scmi/bus.c
@@ -134,17 +134,6 @@ static int scmi_protocol_device_request(const struct scmi_device_id *id_table)
return ret;
}
-static int scmi_protocol_table_register(const struct scmi_device_id *id_table)
-{
- int ret = 0;
- const struct scmi_device_id *entry;
-
- for (entry = id_table; entry->name && ret == 0; entry++)
- ret = scmi_protocol_device_request(entry);
-
- return ret;
-}
-
/**
* scmi_protocol_device_unrequest - Helper to unrequest a device
*
@@ -157,6 +146,7 @@ static int scmi_protocol_table_register(const struct scmi_device_id *id_table)
*/
static void scmi_protocol_device_unrequest(const struct scmi_device_id *id_table)
{
+ struct scmi_requested_dev *rdev, *victim = NULL;
struct list_head *phead;
pr_debug("Unrequesting SCMI device (%s) for protocol %x\n",
@@ -165,29 +155,48 @@ static void scmi_protocol_device_unrequest(const struct scmi_device_id *id_table
mutex_lock(&scmi_requested_devices_mtx);
phead = idr_find(&scmi_requested_devices, id_table->protocol_id);
if (phead) {
- struct scmi_requested_dev *victim, *tmp;
-
- list_for_each_entry_safe(victim, tmp, phead, node) {
- if (!strcmp(victim->id_table->name, id_table->name)) {
- list_del(&victim->node);
-
- mutex_unlock(&scmi_requested_devices_mtx);
- blocking_notifier_call_chain(&scmi_requested_devices_nh,
- SCMI_BUS_NOTIFY_DEVICE_UNREQUEST,
- (void *)victim->id_table);
- kfree(victim);
- mutex_lock(&scmi_requested_devices_mtx);
+ list_for_each_entry(rdev, phead, node) {
+ if (!strcmp(rdev->id_table->name, id_table->name)) {
+ victim = rdev;
+ list_del(&rdev->node);
break;
}
}
- if (list_empty(phead)) {
+ if (victim && list_empty(phead)) {
idr_remove(&scmi_requested_devices,
id_table->protocol_id);
kfree(phead);
}
}
mutex_unlock(&scmi_requested_devices_mtx);
+
+ if (victim) {
+ blocking_notifier_call_chain(&scmi_requested_devices_nh,
+ SCMI_BUS_NOTIFY_DEVICE_UNREQUEST,
+ (void *)victim->id_table);
+ kfree(victim);
+ }
+}
+
+static int scmi_protocol_table_register(const struct scmi_device_id *id_table)
+{
+ const struct scmi_device_id *entry;
+ int ret;
+
+ for (entry = id_table; entry->name; entry++) {
+ ret = scmi_protocol_device_request(entry);
+ if (ret)
+ goto err_unrequest;
+ }
+
+ return 0;
+
+err_unrequest:
+ while (entry != id_table)
+ scmi_protocol_device_unrequest(--entry);
+
+ return ret;
}
static void
@@ -305,10 +314,14 @@ int scmi_driver_register(struct scmi_driver *driver, struct module *owner,
driver->driver.mod_name = mod_name;
retval = driver_register(&driver->driver);
- if (!retval)
- pr_debug("Registered new scmi driver %s\n", driver->name);
+ if (retval) {
+ scmi_protocol_table_unregister(driver->id_table);
+ return retval;
+ }
- return retval;
+ pr_debug("Registered new scmi driver %s\n", driver->name);
+
+ return 0;
}
EXPORT_SYMBOL_GPL(scmi_driver_register);
diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c
index 79866f3b6b3e..ce441bbb5d5d 100644
--- a/drivers/firmware/arm_scmi/driver.c
+++ b/drivers/firmware/arm_scmi/driver.c
@@ -31,6 +31,7 @@
#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/processor.h>
+#include <linux/rcupdate.h>
#include <linux/refcount.h>
#include <linux/slab.h>
#include <linux/xarray.h>
@@ -2505,21 +2506,31 @@ static int scmi_handle_put(const struct scmi_handle *handle)
return 0;
}
-static void scmi_device_link_add(struct device *consumer,
+static bool scmi_device_link_add(struct device *consumer,
struct device *supplier)
{
struct device_link *link;
link = device_link_add(consumer, supplier, DL_FLAG_AUTOREMOVE_CONSUMER);
- WARN_ON(!link);
+ return !WARN_ON(!link);
+}
+
+static void scmi_clear_handle(struct scmi_device *scmi_dev)
+{
+ if (!scmi_dev->handle)
+ return;
+
+ scmi_handle_put(scmi_dev->handle);
+ scmi_dev->handle = NULL;
}
static void scmi_set_handle(struct scmi_device *scmi_dev)
{
scmi_dev->handle = scmi_handle_get(&scmi_dev->dev);
- if (scmi_dev->handle)
- scmi_device_link_add(&scmi_dev->dev, scmi_dev->handle->dev);
+ if (scmi_dev->handle &&
+ !scmi_device_link_add(&scmi_dev->dev, scmi_dev->handle->dev))
+ scmi_clear_handle(scmi_dev);
}
static int __scmi_xfer_info_init(struct scmi_info *sinfo,
@@ -2657,6 +2668,7 @@ static int scmi_chan_setup(struct scmi_info *info, struct device_node *of_node,
cinfo->id = prot_id;
cinfo->dev = &tdev->dev;
+ cinfo->handle = &info->handle;
ret = info->desc->ops->chan_setup(cinfo, info->dev, tx);
if (ret) {
of_node_put(of_node);
@@ -2682,6 +2694,7 @@ static int scmi_chan_setup(struct scmi_info *info, struct device_node *of_node,
"unable to allocate SCMI idr slot err %d\n", ret);
/* Destroy channel and device only if created by this call. */
if (tdev) {
+ info->desc->ops->chan_free(prot_id, cinfo, idr);
of_node_put(of_node);
scmi_device_destroy(info->dev, prot_id, name);
devm_kfree(info->dev, cinfo);
@@ -2689,7 +2702,6 @@ static int scmi_chan_setup(struct scmi_info *info, struct device_node *of_node,
return ret;
}
- cinfo->handle = &info->handle;
return 0;
}
@@ -2747,9 +2759,11 @@ static int scmi_channels_setup(struct scmi_info *info)
if (of_property_read_u32(child, "reg", &prot_id))
continue;
- if (!FIELD_FIT(MSG_PROTOCOL_ID_MASK, prot_id))
+ if (!FIELD_FIT(MSG_PROTOCOL_ID_MASK, prot_id)) {
dev_err(info->dev,
"Out of range protocol %d\n", prot_id);
+ continue;
+ }
ret = scmi_txrx_setup(info, child, prot_id);
if (ret)
@@ -2759,7 +2773,7 @@ static int scmi_channels_setup(struct scmi_info *info)
return 0;
}
-static int scmi_chan_destroy(int id, void *p, void *idr)
+static int scmi_chan_destroy(int id, void *p, void *data)
{
struct scmi_chan_info *cinfo = p;
@@ -2768,12 +2782,10 @@ static int scmi_chan_destroy(int id, void *p, void *idr)
struct scmi_device *sdev = to_scmi_dev(cinfo->dev);
of_node_put(cinfo->dev->of_node);
- scmi_device_destroy(info->dev, id, sdev->name);
+ scmi_device_destroy(info->dev, cinfo->id, sdev->name);
cinfo->dev = NULL;
}
- idr_remove(idr, id);
-
return 0;
}
@@ -2800,6 +2812,7 @@ static int scmi_bus_notifier(struct notifier_block *nb,
{
struct scmi_info *info = bus_nb_to_scmi_info(nb);
struct scmi_device *sdev = to_scmi_dev(data);
+ const char *status;
/* Skip transport devices and devices of different SCMI instances */
if (!strncmp(sdev->name, "__scmi_transport_device", 23) ||
@@ -2810,18 +2823,22 @@ static int scmi_bus_notifier(struct notifier_block *nb,
case BUS_NOTIFY_BIND_DRIVER:
/* setup handle now as the transport is ready */
scmi_set_handle(sdev);
+ status = "about to be BOUND.";
+ break;
+ case BUS_NOTIFY_DRIVER_NOT_BOUND:
+ scmi_clear_handle(sdev);
+ status = "NOT BOUND.";
break;
case BUS_NOTIFY_UNBOUND_DRIVER:
- scmi_handle_put(sdev->handle);
- sdev->handle = NULL;
+ scmi_clear_handle(sdev);
+ status = "UNBOUND.";
break;
default:
return NOTIFY_DONE;
}
dev_dbg(info->dev, "Device %s (%s) is now %s\n", dev_name(&sdev->dev),
- sdev->name, action == BUS_NOTIFY_BIND_DRIVER ?
- "about to be BOUND." : "UNBOUND.");
+ sdev->name, status);
return NOTIFY_OK;
}
@@ -2833,7 +2850,9 @@ static int scmi_device_request_notifier(struct notifier_block *nb,
struct scmi_device_id *id_table = data;
struct scmi_info *info = req_nb_to_scmi_info(nb);
+ rcu_read_lock();
np = idr_find(&info->active_protocols, id_table->protocol_id);
+ rcu_read_unlock();
if (!np)
return NOTIFY_DONE;
@@ -3114,7 +3133,7 @@ static int scmi_probe(struct platform_device *pdev)
ret = scmi_channels_setup(info);
if (ret) {
err_str = "failed to setup channels\n";
- goto clear_ida;
+ goto clear_txrx_setup;
}
ret = bus_register_notifier(&scmi_bus_type, &info->bus_nb);
@@ -3176,7 +3195,7 @@ static int scmi_probe(struct platform_device *pdev)
dev_err(dev, "%s", err_str);
return 0;
}
- goto notification_exit;
+ goto raw_mode_cleanup;
}
mutex_lock(&scmi_list_mutex);
@@ -3189,8 +3208,10 @@ static int scmi_probe(struct platform_device *pdev)
if (of_property_read_u32(child, "reg", &prot_id))
continue;
- if (!FIELD_FIT(MSG_PROTOCOL_ID_MASK, prot_id))
+ if (!FIELD_FIT(MSG_PROTOCOL_ID_MASK, prot_id)) {
dev_err(dev, "Out of range protocol %d\n", prot_id);
+ continue;
+ }
if (!scmi_is_protocol_implemented(handle, prot_id)) {
dev_err(dev, "SCMI protocol %d not implemented\n",
@@ -3216,18 +3237,18 @@ static int scmi_probe(struct platform_device *pdev)
return 0;
-notification_exit:
+raw_mode_cleanup:
if (IS_ENABLED(CONFIG_ARM_SCMI_RAW_MODE_SUPPORT))
scmi_raw_mode_cleanup(info->raw);
- scmi_notification_exit(&info->handle);
clear_dev_req_notifier:
blocking_notifier_chain_unregister(&scmi_requested_devices_nh,
&info->dev_req_nb);
clear_bus_notifier:
bus_unregister_notifier(&scmi_bus_type, &info->bus_nb);
clear_txrx_setup:
+ scmi_notification_quiesce(&info->handle);
scmi_cleanup_txrx_channels(info);
-clear_ida:
+ scmi_notification_exit(&info->handle);
ida_free(&scmi_id, info->id);
out_err:
@@ -3250,6 +3271,12 @@ static void scmi_remove(struct platform_device *pdev)
list_del(&info->node);
mutex_unlock(&scmi_list_mutex);
+ blocking_notifier_chain_unregister(&scmi_requested_devices_nh,
+ &info->dev_req_nb);
+
+ /* Stop transport callbacks before tearing down notifications. */
+ scmi_notification_quiesce(&info->handle);
+ scmi_cleanup_txrx_channels(info);
scmi_notification_exit(&info->handle);
mutex_lock(&info->protocols_mtx);
@@ -3260,13 +3287,8 @@ static void scmi_remove(struct platform_device *pdev)
of_node_put(child);
idr_destroy(&info->active_protocols);
- blocking_notifier_chain_unregister(&scmi_requested_devices_nh,
- &info->dev_req_nb);
bus_unregister_notifier(&scmi_bus_type, &info->bus_nb);
- /* Safe to free channels since no more users */
- scmi_cleanup_txrx_channels(info);
-
ida_free(&scmi_id, info->id);
}
diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
index f042fb8a0502..c4d4337ac3a3 100644
--- a/drivers/firmware/arm_scmi/notify.c
+++ b/drivers/firmware/arm_scmi/notify.c
@@ -1685,6 +1685,25 @@ int scmi_notification_init(struct scmi_handle *handle)
return -ENOMEM;
}
+/**
+ * scmi_notification_quiesce() - Stop notification late initialization
+ * @handle: The handle identifying the platform instance to quiesce
+ *
+ * Prevent new late-init work from being queued and wait for any already queued
+ * or running late-init work to complete before transport channels are torn
+ * down.
+ */
+void scmi_notification_quiesce(struct scmi_handle *handle)
+{
+ struct scmi_notify_instance *ni;
+
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
+ return;
+
+ disable_work_sync(&ni->init_work);
+}
+
/**
* scmi_notification_exit() - Shutdown and clean Notification core
* @handle: The handle identifying the platform instance to shutdown
@@ -1696,6 +1715,8 @@ void scmi_notification_exit(struct scmi_handle *handle)
ni = scmi_notification_instance_data_get(handle);
if (!ni)
return;
+
+ scmi_notification_quiesce(handle);
scmi_notification_instance_data_set(handle, NULL);
/* Destroy while letting pending work complete */
diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
index 76758a736cf4..f18f98c5ab3b 100644
--- a/drivers/firmware/arm_scmi/notify.h
+++ b/drivers/firmware/arm_scmi/notify.h
@@ -82,6 +82,7 @@ struct scmi_protocol_events {
};
int scmi_notification_init(struct scmi_handle *handle);
+void scmi_notification_quiesce(struct scmi_handle *handle);
void scmi_notification_exit(struct scmi_handle *handle);
int scmi_register_protocol_events(const struct scmi_handle *handle, u8 proto_id,
const struct scmi_protocol_handle *ph,
diff --git a/drivers/firmware/arm_scmi/transports/mailbox.c b/drivers/firmware/arm_scmi/transports/mailbox.c
index e3d5f7560990..f43e00c46115 100644
--- a/drivers/firmware/arm_scmi/transports/mailbox.c
+++ b/drivers/firmware/arm_scmi/transports/mailbox.c
@@ -208,13 +208,18 @@ static int mailbox_chan_setup(struct scmi_chan_info *cinfo, struct device *dev,
cl->tx_block = false;
cl->knows_txdone = tx;
+ cinfo->transport_info = smbox;
+ smbox->cinfo = cinfo;
+ mutex_init(&smbox->chan_lock);
+
smbox->chan = mbox_request_channel(cl, tx ? 0 : p2a_chan);
if (IS_ERR(smbox->chan)) {
ret = PTR_ERR(smbox->chan);
+ smbox->chan = NULL;
if (ret != -EPROBE_DEFER)
dev_err(cdev,
"failed to request SCMI %s mailbox\n", desc);
- return ret;
+ goto err_clear_cinfo;
}
/* Additional unidirectional channel for TX if needed */
@@ -222,9 +227,10 @@ static int mailbox_chan_setup(struct scmi_chan_info *cinfo, struct device *dev,
smbox->chan_receiver = mbox_request_channel(cl, a2p_rx_chan);
if (IS_ERR(smbox->chan_receiver)) {
ret = PTR_ERR(smbox->chan_receiver);
+ smbox->chan_receiver = NULL;
if (ret != -EPROBE_DEFER)
dev_err(cdev, "failed to request SCMI Tx Receiver mailbox\n");
- return ret;
+ goto err_free_chan;
}
}
@@ -232,17 +238,23 @@ static int mailbox_chan_setup(struct scmi_chan_info *cinfo, struct device *dev,
smbox->chan_platform_receiver = mbox_request_channel(cl, p2a_rx_chan);
if (IS_ERR(smbox->chan_platform_receiver)) {
ret = PTR_ERR(smbox->chan_platform_receiver);
+ smbox->chan_platform_receiver = NULL;
if (ret != -EPROBE_DEFER)
dev_err(cdev, "failed to request SCMI P2A Receiver mailbox\n");
- return ret;
+ goto err_free_chan;
}
}
- cinfo->transport_info = smbox;
- smbox->cinfo = cinfo;
- mutex_init(&smbox->chan_lock);
-
return 0;
+
+err_free_chan:
+ mbox_free_channel(smbox->chan);
+err_clear_cinfo:
+ cinfo->transport_info = NULL;
+ smbox->cinfo = NULL;
+ devm_iounmap(dev, smbox->shmem);
+ devm_kfree(dev, smbox);
+ return ret;
}
static int mailbox_chan_free(int id, void *p, void *data)
diff --git a/drivers/firmware/arm_scmi/transports/smc.c b/drivers/firmware/arm_scmi/transports/smc.c
index f8dd108777f9..2a35434a2b7f 100644
--- a/drivers/firmware/arm_scmi/transports/smc.c
+++ b/drivers/firmware/arm_scmi/transports/smc.c
@@ -169,6 +169,13 @@ static int smc_chan_setup(struct scmi_chan_info *cinfo, struct device *dev,
scmi_info->param_page = SHMEM_PAGE(res.start);
scmi_info->param_offset = SHMEM_OFFSET(res.start);
}
+
+ scmi_info->func_id = func_id;
+ scmi_info->cap_id = cap_id;
+ scmi_info->cinfo = cinfo;
+ smc_channel_lock_init(scmi_info);
+ cinfo->transport_info = scmi_info;
+
/*
* If there is an interrupt named "a2p", then the service and
* completion of a message is signaled by an interrupt rather than by
@@ -180,18 +187,14 @@ static int smc_chan_setup(struct scmi_chan_info *cinfo, struct device *dev,
IRQF_NO_SUSPEND, dev_name(dev), scmi_info);
if (ret) {
dev_err(dev, "failed to setup SCMI smc irq\n");
+ cinfo->transport_info = NULL;
+ scmi_info->cinfo = NULL;
return ret;
}
} else {
cinfo->no_completion_irq = true;
}
- scmi_info->func_id = func_id;
- scmi_info->cap_id = cap_id;
- scmi_info->cinfo = cinfo;
- smc_channel_lock_init(scmi_info);
- cinfo->transport_info = scmi_info;
-
return 0;
}
diff --git a/drivers/firmware/google/coreboot_table.c b/drivers/firmware/google/coreboot_table.c
index 208652a8087c..3a7b9b432b9a 100644
--- a/drivers/firmware/google/coreboot_table.c
+++ b/drivers/firmware/google/coreboot_table.c
@@ -101,16 +101,20 @@ void coreboot_driver_unregister(struct coreboot_driver *driver)
}
EXPORT_SYMBOL(coreboot_driver_unregister);
-static int coreboot_table_populate(struct device *dev, void *ptr)
+static int coreboot_table_populate(struct device *dev, void *ptr, resource_size_t len)
{
int i, ret;
void *ptr_entry;
struct coreboot_device *device;
struct coreboot_table_entry *entry;
struct coreboot_table_header *header = ptr;
+ void *ptr_end;
+ ptr_end = ptr + len;
ptr_entry = ptr + header->header_bytes;
for (i = 0; i < header->table_entries; i++) {
+ if (ptr_entry + sizeof(*entry) > ptr_end)
+ return -EINVAL;
entry = ptr_entry;
if (entry->size < sizeof(*entry)) {
@@ -118,6 +122,9 @@ static int coreboot_table_populate(struct device *dev, void *ptr)
return -EINVAL;
}
+ if (ptr_entry + entry->size > ptr_end)
+ return -EINVAL;
+
device = kzalloc(sizeof(device->dev) + entry->size, GFP_KERNEL);
if (!device)
return -ENOMEM;
@@ -152,6 +159,7 @@ static int coreboot_table_populate(struct device *dev, void *ptr)
static int coreboot_table_probe(struct platform_device *pdev)
{
resource_size_t len;
+ resource_size_t table_span;
struct coreboot_table_header *header;
struct resource *res;
struct device *dev = &pdev->dev;
@@ -163,7 +171,7 @@ static int coreboot_table_probe(struct platform_device *pdev)
return -EINVAL;
len = resource_size(res);
- if (!res->start || !len)
+ if (!res->start || len < sizeof(*header))
return -EINVAL;
/* Check just the header first to make sure things are sane */
@@ -171,19 +179,27 @@ static int coreboot_table_probe(struct platform_device *pdev)
if (!header)
return -ENOMEM;
- len = header->header_bytes + header->table_bytes;
ret = strncmp(header->signature, "LBIO", sizeof(header->signature));
+
+ if (!ret &&
+ (header->header_bytes < sizeof(*header) ||
+ check_add_overflow((resource_size_t)header->header_bytes,
+ (resource_size_t)header->table_bytes,
+ &table_span) ||
+ table_span > len))
+ ret = -EINVAL;
+
memunmap(header);
if (ret) {
dev_warn(dev, "coreboot table missing or corrupt!\n");
return -ENODEV;
}
- ptr = memremap(res->start, len, MEMREMAP_WB);
+ ptr = memremap(res->start, table_span, MEMREMAP_WB);
if (!ptr)
return -ENOMEM;
- ret = coreboot_table_populate(dev, ptr);
+ ret = coreboot_table_populate(dev, ptr, table_span);
memunmap(ptr);
diff --git a/drivers/firmware/qcom/Makefile b/drivers/firmware/qcom/Makefile
index 0be40a1abc13..b679d3fc2c26 100644
--- a/drivers/firmware/qcom/Makefile
+++ b/drivers/firmware/qcom/Makefile
@@ -5,6 +5,7 @@
obj-$(CONFIG_QCOM_SCM) += qcom-scm.o
qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+CFLAGS_qcom_scm-smc.o := -I$(src)
obj-$(CONFIG_QCOM_TZMEM) += qcom_tzmem.o
obj-$(CONFIG_QCOM_QSEECOM) += qcom_qseecom.o
obj-$(CONFIG_QCOM_QSEECOM_UEFISECAPP) += qcom_qseecom_uefisecapp.o
diff --git a/drivers/firmware/qcom/qcom_scm-smc.c b/drivers/firmware/qcom/qcom_scm-smc.c
index 3f10b23ec941..59cd6cfa75e1 100644
--- a/drivers/firmware/qcom/qcom_scm-smc.c
+++ b/drivers/firmware/qcom/qcom_scm-smc.c
@@ -24,6 +24,9 @@ struct arm_smccc_args {
unsigned long args[8];
};
+#define CREATE_TRACE_POINTS
+#include "qcom_scm_trace.h"
+
static DEFINE_MUTEX(qcom_scm_lock);
#define QCOM_SCM_EBUSY_WAIT_MS 30
@@ -44,6 +47,7 @@ static void __scm_smc_do_quirk(const struct arm_smccc_args *smc,
quirk.state.a6 = 0;
do {
+ trace_scm_smc_request(a0, smc);
arm_smccc_smc_quirk(a0, smc->args[1], smc->args[2],
smc->args[3], smc->args[4], smc->args[5],
quirk.state.a6, smc->args[7], res, &quirk);
@@ -83,6 +87,7 @@ int scm_get_wq_ctx(u32 *wq_ctx, u32 *flags, u32 *more_pending)
if (ret)
return ret;
+ trace_scm_waitq_get_wq_ctx(get_wq_res.a1, get_wq_res.a2, get_wq_res.a3);
*wq_ctx = get_wq_res.a1;
*flags = get_wq_res.a2;
*more_pending = get_wq_res.a3;
@@ -105,10 +110,12 @@ static int __scm_smc_do_quirk_handle_waitq(struct device *dev, struct arm_smccc_
wq_ctx = res->a1;
smc_call_ctx = res->a2;
- ret = qcom_scm_wait_for_wq_completion(wq_ctx);
+ trace_scm_waitq_sleep(wq_ctx, smc_call_ctx);
+ ret = qcom_scm_wait_for_wq_completion(dev, wq_ctx);
if (ret)
return ret;
+ trace_scm_waitq_resume(smc_call_ctx);
fill_wq_resume_args(&resume, smc_call_ctx);
smc = &resume;
}
@@ -200,6 +207,9 @@ int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
}
ret = __scm_smc_do(dev, &smc, &smc_res, atomic);
+
+ trace_scm_smc_done(ret, smc.args[0], &smc_res);
+
if (ret)
return ret;
diff --git a/drivers/firmware/qcom/qcom_scm.c b/drivers/firmware/qcom/qcom_scm.c
index 1da16bc79391..afbf0ad80c85 100644
--- a/drivers/firmware/qcom/qcom_scm.c
+++ b/drivers/firmware/qcom/qcom_scm.c
@@ -30,18 +30,25 @@
#include <linux/sizes.h>
#include <linux/types.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+
#include "qcom_scm.h"
#include "qcom_tzmem.h"
static u32 download_mode;
+#define GIC_SPI_BASE 32
+#define GIC_MAX_SPI 1019 // SPIs in GICv3 spec range from 32..1019
+#define GIC_ESPI_BASE 4096
+#define GIC_MAX_ESPI 5119 // ESPIs in GICv3 spec range from 4096..5119
+
struct qcom_scm {
struct device *dev;
struct clk *core_clk;
struct clk *iface_clk;
struct clk *bus_clk;
struct icc_path *path;
- struct completion waitq_comp;
+ struct completion *waitq_comps;
struct reset_controller_dev reset;
/* control access to the interconnect path */
@@ -51,6 +58,7 @@ struct qcom_scm {
u64 dload_mode_addr;
struct qcom_tzmem_pool *mempool;
+ unsigned int wq_cnt;
};
struct qcom_scm_current_perm_info {
@@ -130,6 +138,8 @@ static const u8 qcom_scm_cpu_warm_bits[QCOM_SCM_BOOT_MAX_CPUS] = {
#define QCOM_DLOAD_MINIDUMP 2
#define QCOM_DLOAD_BOTHDUMP 3
+#define QCOM_SCM_DEFAULT_WAITQ_COUNT 1
+
static const char * const qcom_scm_convention_names[] = {
[SMC_CONVENTION_UNKNOWN] = "unknown",
[SMC_CONVENTION_ARM_32] = "smc arm 32",
@@ -1847,42 +1857,104 @@ bool qcom_scm_is_available(void)
}
EXPORT_SYMBOL_GPL(qcom_scm_is_available);
-static int qcom_scm_assert_valid_wq_ctx(u32 wq_ctx)
+static int qcom_scm_fill_irq_fwspec_params(struct irq_fwspec *fwspec, u32 hwirq)
{
- /* FW currently only supports a single wq_ctx (zero).
- * TODO: Update this logic to include dynamic allocation and lookup of
- * completion structs when FW supports more wq_ctx values.
- */
- if (wq_ctx != 0) {
- dev_err(__scm->dev, "Firmware unexpectedly passed non-zero wq_ctx\n");
- return -EINVAL;
+ if (hwirq >= GIC_SPI_BASE && hwirq <= GIC_MAX_SPI) {
+ fwspec->param[0] = GIC_SPI;
+ fwspec->param[1] = hwirq - GIC_SPI_BASE;
+ } else if (hwirq >= GIC_ESPI_BASE && hwirq <= GIC_MAX_ESPI) {
+ fwspec->param[0] = GIC_ESPI;
+ fwspec->param[1] = hwirq - GIC_ESPI_BASE;
+ } else {
+ WARN(1, "Unexpected hwirq: %d\n", hwirq);
+ return -ENXIO;
}
+ fwspec->param[2] = IRQ_TYPE_EDGE_RISING;
+ fwspec->param_count = 3;
+
return 0;
}
-int qcom_scm_wait_for_wq_completion(u32 wq_ctx)
+static int qcom_scm_query_waitq_count(struct qcom_scm *scm)
{
+ struct qcom_scm_desc desc = {
+ .svc = QCOM_SCM_SVC_WAITQ,
+ .cmd = QCOM_SCM_WAITQ_GET_INFO,
+ .owner = ARM_SMCCC_OWNER_SIP
+ };
+ struct qcom_scm_res res;
int ret;
- ret = qcom_scm_assert_valid_wq_ctx(wq_ctx);
+ ret = qcom_scm_call_atomic(scm->dev, &desc, &res);
if (ret)
return ret;
- wait_for_completion(&__scm->waitq_comp);
-
- return 0;
+ return res.result[0] & GENMASK(7, 0);
}
-static int qcom_scm_waitq_wakeup(unsigned int wq_ctx)
+static int qcom_scm_get_waitq_irq(struct qcom_scm *scm)
{
+ struct qcom_scm_desc desc = {
+ .svc = QCOM_SCM_SVC_WAITQ,
+ .cmd = QCOM_SCM_WAITQ_GET_INFO,
+ .owner = ARM_SMCCC_OWNER_SIP
+ };
+ struct device_node *parent_irq_node;
+ struct irq_fwspec fwspec;
+ struct qcom_scm_res res;
+ u32 hwirq;
int ret;
- ret = qcom_scm_assert_valid_wq_ctx(wq_ctx);
+ ret = qcom_scm_call_atomic(scm->dev, &desc, &res);
if (ret)
return ret;
- complete(&__scm->waitq_comp);
+ hwirq = res.result[1] & GENMASK(15, 0);
+ ret = qcom_scm_fill_irq_fwspec_params(&fwspec, hwirq);
+ if (ret)
+ return ret;
+
+ parent_irq_node = of_irq_find_parent(scm->dev->of_node);
+ if (!parent_irq_node)
+ return -ENODEV;
+
+ fwspec.fwnode = of_fwnode_handle(parent_irq_node);
+
+ return irq_create_fwspec_mapping(&fwspec);
+}
+
+static struct completion *qcom_scm_get_completion(struct qcom_scm *scm, u32 wq_ctx)
+{
+ if (WARN_ON_ONCE(wq_ctx >= scm->wq_cnt))
+ return ERR_PTR(-EINVAL);
+
+ return &scm->waitq_comps[wq_ctx];
+}
+
+int qcom_scm_wait_for_wq_completion(struct device *dev, u32 wq_ctx)
+{
+ struct qcom_scm *scm = dev_get_drvdata(dev);
+ struct completion *wq;
+
+ wq = qcom_scm_get_completion(scm, wq_ctx);
+ if (IS_ERR(wq))
+ return PTR_ERR(wq);
+
+ wait_for_completion(wq);
+
+ return 0;
+}
+
+static int qcom_scm_waitq_wakeup(struct qcom_scm *scm, unsigned int wq_ctx)
+{
+ struct completion *wq;
+
+ wq = qcom_scm_get_completion(scm, wq_ctx);
+ if (IS_ERR(wq))
+ return PTR_ERR(wq);
+
+ complete(wq);
return 0;
}
@@ -1905,7 +1977,7 @@ static irqreturn_t qcom_scm_irq_handler(int irq, void *data)
goto out;
}
- ret = qcom_scm_waitq_wakeup(wq_ctx);
+ ret = qcom_scm_waitq_wakeup(scm, wq_ctx);
if (ret)
goto out;
} while (more_pending);
@@ -1958,17 +2030,18 @@ static int qcom_scm_probe(struct platform_device *pdev)
struct qcom_tzmem_pool_config pool_config;
struct qcom_scm *scm;
int irq, ret;
+ int i;
scm = devm_kzalloc(&pdev->dev, sizeof(*scm), GFP_KERNEL);
if (!scm)
return -ENOMEM;
scm->dev = &pdev->dev;
+ platform_set_drvdata(pdev, scm);
ret = qcom_scm_find_dload_address(&pdev->dev, &scm->dload_mode_addr);
if (ret < 0)
return ret;
- init_completion(&scm->waitq_comp);
mutex_init(&scm->scm_bw_lock);
scm->path = devm_of_icc_get(&pdev->dev, NULL);
@@ -2003,7 +2076,20 @@ static int qcom_scm_probe(struct platform_device *pdev)
/* Paired with smp_load_acquire() in qcom_scm_is_available(). */
smp_store_release(&__scm, scm);
- irq = platform_get_irq_optional(pdev, 0);
+ ret = qcom_scm_query_waitq_count(scm);
+ scm->wq_cnt = ret < 0 ? QCOM_SCM_DEFAULT_WAITQ_COUNT : ret;
+ scm->waitq_comps = devm_kcalloc(&pdev->dev, scm->wq_cnt, sizeof(*scm->waitq_comps),
+ GFP_KERNEL);
+ if (!scm->waitq_comps)
+ return -ENOMEM;
+
+ for (i = 0; i < scm->wq_cnt; i++)
+ init_completion(&scm->waitq_comps[i]);
+
+ irq = qcom_scm_get_waitq_irq(scm);
+ if (irq < 0)
+ irq = platform_get_irq_optional(pdev, 0);
+
if (irq < 0) {
if (irq != -ENXIO) {
ret = irq;
diff --git a/drivers/firmware/qcom/qcom_scm.h b/drivers/firmware/qcom/qcom_scm.h
index 685b8f59e7a6..3a072b861d7f 100644
--- a/drivers/firmware/qcom/qcom_scm.h
+++ b/drivers/firmware/qcom/qcom_scm.h
@@ -63,7 +63,7 @@ struct qcom_scm_res {
u64 result[MAX_QCOM_SCM_RETS];
};
-int qcom_scm_wait_for_wq_completion(u32 wq_ctx);
+int qcom_scm_wait_for_wq_completion(struct device *dev, u32 wq_ctx);
int scm_get_wq_ctx(u32 *wq_ctx, u32 *flags, u32 *more_pending);
#define SCM_SMC_FNID(s, c) ((((s) & 0xFF) << 8) | ((c) & 0xFF))
@@ -143,6 +143,7 @@ struct qcom_tzmem_pool *qcom_scm_get_tzmem_pool(void);
#define QCOM_SCM_SVC_WAITQ 0x24
#define QCOM_SCM_WAITQ_RESUME 0x02
#define QCOM_SCM_WAITQ_GET_WQ_CTX 0x03
+#define QCOM_SCM_WAITQ_GET_INFO 0x04
#define QCOM_SCM_SVC_GPU 0x28
#define QCOM_SCM_SVC_GPU_INIT_REGS 0x01
diff --git a/drivers/firmware/qcom/qcom_scm_trace.h b/drivers/firmware/qcom/qcom_scm_trace.h
new file mode 100644
index 000000000000..6c911124fc56
--- /dev/null
+++ b/drivers/firmware/qcom/qcom_scm_trace.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM qcom_scm
+
+#if !defined(_TRACE_SCM_SMC_INTERFACE_H) || defined(TRACE_HEADER_MULTI_READ)
+
+#define _TRACE_SCM_SMC_INTERFACE_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(scm_smc_request,
+
+ TP_PROTO(unsigned long a0, const struct arm_smccc_args *smc),
+
+ TP_ARGS(a0, smc),
+
+ TP_STRUCT__entry(
+ __field(u64, smc_id)
+ __field(u8, svc_id)
+ __field(u8, cmd_id)
+ __field(u8, args_cnt)
+ __dynamic_array(unsigned long, args,
+ min_t(u8, (smc->args[1] & 0xF), (u8)6))
+ ),
+
+ TP_fast_assign(
+ __entry->smc_id = a0;
+ __entry->svc_id = (smc->args[0] >> 8) & 0xFF;
+ __entry->cmd_id = smc->args[0] & 0xFF;
+ u8 n = min_t(u8, (smc->args[1] & 0xF), (u8)6);
+
+ __entry->args_cnt = n;
+
+ unsigned long *dst = __get_dynamic_array(args);
+
+ for (int i = 0; i < n; i++)
+ dst[i] = smc->args[2 + i];
+ ),
+
+ TP_printk("smc_id:0x%08llx svc_id:0x%02x cmd_id:0x%02x args_cnt:%u args:%s",
+ __entry->smc_id, __entry->svc_id, __entry->cmd_id, __entry->args_cnt,
+ __print_dynamic_array(args, sizeof(unsigned long)))
+);
+
+TRACE_EVENT(scm_waitq_sleep,
+
+ TP_PROTO(u32 wq_ctx, u32 smc_ctx),
+
+ TP_ARGS(wq_ctx, smc_ctx),
+
+ TP_STRUCT__entry(
+ __field(u32, wq_ctx)
+ __field(u32, smc_call_ctx)
+ ),
+
+ TP_fast_assign(
+ __entry->wq_ctx = wq_ctx;
+ __entry->smc_call_ctx = smc_ctx;
+ ),
+
+ TP_printk("wq_ctx:%u, smc_call_ctx:%u", __entry->wq_ctx, __entry->smc_call_ctx)
+);
+
+TRACE_EVENT(scm_waitq_resume,
+
+ TP_PROTO(u32 smc_ctx),
+
+ TP_ARGS(smc_ctx),
+
+ TP_STRUCT__entry(
+ __field(u32, smc_call_ctx)
+ ),
+
+ TP_fast_assign(
+ __entry->smc_call_ctx = smc_ctx;
+ ),
+
+ TP_printk("smc_call_ctx:%u", __entry->smc_call_ctx)
+);
+
+TRACE_EVENT(scm_waitq_get_wq_ctx,
+
+ TP_PROTO(u32 wq_ctx, u32 flags, u32 pending),
+
+ TP_ARGS(wq_ctx, flags, pending),
+
+ TP_STRUCT__entry(
+ __field(u32, wq_ctx)
+ __field(u32, flags)
+ __field(u32, more_pending)
+ ),
+
+ TP_fast_assign(
+ __entry->wq_ctx = wq_ctx;
+ __entry->flags = flags;
+ __entry->more_pending = pending;
+ ),
+
+ TP_printk("wq_ctx:%u, flags:%u, more_pending:%u",
+ __entry->wq_ctx, __entry->flags, __entry->more_pending)
+);
+
+TRACE_EVENT(scm_smc_done,
+
+ TP_PROTO(int ret, u64 smc_id, struct arm_smccc_res *smc_res),
+
+ TP_ARGS(ret, smc_id, smc_res),
+
+ TP_STRUCT__entry(
+ __field(int, ret)
+ __field(u64, smc_id)
+ __field(unsigned long, res)
+ __field(unsigned long, res0)
+ __field(unsigned long, res1)
+ __field(unsigned long, res2)
+ ),
+
+ TP_fast_assign(
+ __entry->ret = ret;
+ __entry->smc_id = smc_id;
+ __entry->res = smc_res->a0;
+ __entry->res0 = smc_res->a1;
+ __entry->res1 = smc_res->a2;
+ __entry->res2 = smc_res->a3;
+ ),
+
+ TP_printk("smc_id:0x%08llx, ret:%d res_to_callee:0x%lx res0:0x%lx res1:0x%lx res2:0x%lx",
+ __entry->smc_id, __entry->ret, __entry->res,
+ __entry->res0, __entry->res1, __entry->res2)
+);
+
+#endif /* _TRACE_SCM_SMC_INTERFACE_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE qcom_scm_trace
+
+#include <trace/define_trace.h>
+
diff --git a/drivers/firmware/qcom/qcom_tzmem.c b/drivers/firmware/qcom/qcom_tzmem.c
index f8fef622e48f..91f1efd83d71 100644
--- a/drivers/firmware/qcom/qcom_tzmem.c
+++ b/drivers/firmware/qcom/qcom_tzmem.c
@@ -15,6 +15,7 @@
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/mm.h>
+#include <linux/once.h>
#include <linux/radix-tree.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
@@ -465,14 +466,18 @@ phys_addr_t qcom_tzmem_to_phys(void *vaddr)
}
EXPORT_SYMBOL_GPL(qcom_tzmem_to_phys);
+static void qcom_tzmem_do_init(int *result)
+{
+ *result = qcom_tzmem_init();
+}
+
int qcom_tzmem_enable(struct device *dev)
{
- if (qcom_tzmem_dev)
- return -EBUSY;
+ static int result;
qcom_tzmem_dev = dev;
-
- return qcom_tzmem_init();
+ DO_ONCE(qcom_tzmem_do_init, &result);
+ return result;
}
EXPORT_SYMBOL_GPL(qcom_tzmem_enable);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index d81af7dcef44..850f1b61694f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1797,6 +1797,12 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
size >>= 1;
aligned_size = PAGE_ALIGN(size);
+ /* reject AQL queue with size < 2 */
+ if (!aligned_size) {
+ ret = -EINVAL;
+ goto err_alignment_size;
+ }
+
(*mem)->alloc_flags = flags;
amdgpu_sync_create(&(*mem)->sync);
@@ -1888,6 +1894,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
amdgpu_amdkfd_unreserve_mem_limit(adev, aligned_size, flags, xcp_id);
err_reserve_limit:
amdgpu_sync_free(&(*mem)->sync);
+err_alignment_size:
mutex_destroy(&(*mem)->lock);
if (gobj)
drm_gem_object_put(gobj);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 20a13db7cd88..908f4539c779 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -4168,6 +4168,14 @@ static void amdgpu_device_set_mcbp(struct amdgpu_device *adev)
DRM_INFO("MCBP is enabled\n");
}
+static bool
+amdgpu_device_should_register_switcheroo(struct amdgpu_device *adev, bool px)
+{
+ return !pci_is_thunderbolt_attached(adev->pdev) &&
+ (px || (!dev_is_removable(&adev->pdev->dev) &&
+ apple_gmux_detect(NULL, NULL)));
+}
+
/**
* amdgpu_device_init - initialize the driver
*
@@ -4628,8 +4636,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
px = amdgpu_device_supports_px(ddev);
- if (px || (!dev_is_removable(&adev->pdev->dev) &&
- apple_gmux_detect(NULL, NULL)))
+ if (amdgpu_device_should_register_switcheroo(adev, px))
vga_switcheroo_register_client(adev->pdev,
&amdgpu_switcheroo_ops, px);
@@ -4798,8 +4805,7 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev)
px = amdgpu_device_supports_px(adev_to_drm(adev));
- if (px || (!dev_is_removable(&adev->pdev->dev) &&
- apple_gmux_detect(NULL, NULL)))
+ if (amdgpu_device_should_register_switcheroo(adev, px))
vga_switcheroo_unregister_client(adev->pdev);
if (px)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index fb26a4c1089c..1f47004ec776 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -2461,6 +2461,7 @@ amdgpu_pci_remove(struct pci_dev *pdev)
if (adev->pm.rpm_mode != AMDGPU_RUNPM_NONE) {
pm_runtime_get_sync(dev->dev);
pm_runtime_forbid(dev->dev);
+ pm_runtime_dont_use_autosuspend(dev->dev);
}
amdgpu_driver_unload_kms(dev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 382125c64e4c..b964b1a2f9b5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -358,6 +358,12 @@ static bool psp_get_runtime_db_entry(struct amdgpu_device *adev,
bool ret = false;
int i;
+ /*
+ * Runtime DB is for dGPUs only.
+ */
+ if (adev->flags & AMD_IS_APU)
+ return false;
+
if (amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(13, 0, 6) ||
amdgpu_ip_version(adev, MP0_HWIP, 0) == IP_VERSION(13, 0, 14))
return false;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index ad3b71bc0b23..a5d651b68729 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1933,7 +1933,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
after->start = eaddr + 1;
after->last = tmp->last;
after->offset = tmp->offset;
- after->offset += (after->start - tmp->start) << PAGE_SHIFT;
+ after->offset += (after->start - tmp->start) << AMDGPU_GPU_PAGE_SHIFT;
after->flags = tmp->flags;
after->bo_va = tmp->bo_va;
list_add(&after->list, &tmp->bo_va->invalids);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
index 90c426ee877b..9b294ef10fd7 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
@@ -1859,11 +1859,13 @@ static int gfx_v6_0_ring_test_ring(struct amdgpu_ring *ring)
return r;
}
-static void gfx_v6_0_ring_emit_vgt_flush(struct amdgpu_ring *ring)
+static void gfx_v6_0_ring_emit_event_write(struct amdgpu_ring *ring,
+ uint32_t event_type,
+ uint32_t event_index)
{
amdgpu_ring_write(ring, PACKET3(PACKET3_EVENT_WRITE, 0));
- amdgpu_ring_write(ring, EVENT_TYPE(VGT_FLUSH) |
- EVENT_INDEX(0));
+ amdgpu_ring_write(ring, EVENT_TYPE(event_type) |
+ EVENT_INDEX(event_index));
}
static void gfx_v6_0_ring_emit_fence(struct amdgpu_ring *ring, u64 addr,
@@ -1902,12 +1904,6 @@ static void gfx_v6_0_ring_emit_ib(struct amdgpu_ring *ring,
unsigned vmid = AMDGPU_JOB_GET_VMID(job);
u32 header, control = 0;
- /* insert SWITCH_BUFFER packet before first IB in the ring frame */
- if (flags & AMDGPU_HAVE_CTX_SWITCH) {
- amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
- amdgpu_ring_write(ring, 0);
- }
-
if (ib->flags & AMDGPU_IB_FLAG_CE)
header = PACKET3(PACKET3_INDIRECT_BUFFER_CONST, 2);
else
@@ -2329,7 +2325,7 @@ static int gfx_v6_0_cp_resume(struct amdgpu_device *adev)
static void gfx_v6_0_ring_emit_pipeline_sync(struct amdgpu_ring *ring)
{
- int usepfp = (ring->funcs->type == AMDGPU_RING_TYPE_GFX);
+ int usepfp = 1;
uint32_t seq = ring->fence_drv.sync_seq;
uint64_t addr = ring->fence_drv.gpu_addr;
@@ -2342,20 +2338,12 @@ static void gfx_v6_0_ring_emit_pipeline_sync(struct amdgpu_ring *ring)
amdgpu_ring_write(ring, seq);
amdgpu_ring_write(ring, 0xffffffff);
amdgpu_ring_write(ring, 4); /* poll interval */
-
- if (usepfp) {
- /* synce CE with ME to prevent CE fetch CEIB before context switch done */
- amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
- amdgpu_ring_write(ring, 0);
- amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
- amdgpu_ring_write(ring, 0);
- }
}
static void gfx_v6_0_ring_emit_vm_flush(struct amdgpu_ring *ring,
unsigned vmid, uint64_t pd_addr)
{
- int usepfp = (ring->funcs->type == AMDGPU_RING_TYPE_GFX);
+ int usepfp = 1;
amdgpu_gmc_emit_flush_gpu_tlb(ring, vmid, pd_addr);
@@ -2373,19 +2361,13 @@ static void gfx_v6_0_ring_emit_vm_flush(struct amdgpu_ring *ring,
/* sync PFP to ME, otherwise we might get invalid PFP reads */
amdgpu_ring_write(ring, PACKET3(PACKET3_PFP_SYNC_ME, 0));
amdgpu_ring_write(ring, 0x0);
-
- /* synce CE with ME to prevent CE fetch CEIB before context switch done */
- amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
- amdgpu_ring_write(ring, 0);
- amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
- amdgpu_ring_write(ring, 0);
}
}
static void gfx_v6_0_ring_emit_wreg(struct amdgpu_ring *ring,
uint32_t reg, uint32_t val)
{
- int usepfp = (ring->funcs->type == AMDGPU_RING_TYPE_GFX);
+ int usepfp = 1;
amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3));
amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(usepfp) |
@@ -2998,12 +2980,30 @@ static uint64_t gfx_v6_0_get_gpu_clock_counter(struct amdgpu_device *adev)
return clock;
}
+static void gfx_v6_0_ring_emit_sb(struct amdgpu_ring *ring)
+{
+ amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
+ amdgpu_ring_write(ring, 0);
+}
+
static void gfx_v6_ring_emit_cntxcntl(struct amdgpu_ring *ring, uint32_t flags)
{
- if (flags & AMDGPU_HAVE_CTX_SWITCH)
- gfx_v6_0_ring_emit_vgt_flush(ring);
+ u32 dw2 = 0x80000000; /* set load_enable otherwise this package is just NOPs */
+
+ if (flags & AMDGPU_HAVE_CTX_SWITCH) {
+ gfx_v6_0_ring_emit_event_write(ring, VS_PARTIAL_FLUSH, 4);
+ gfx_v6_0_ring_emit_event_write(ring, VGT_FLUSH, 0);
+
+ /* set load_global_config (load_global_uconfig doesn't exist on GFX6) */
+ dw2 |= 0x1;
+ /* set load_cs_sh_regs */
+ dw2 |= 0x01000000;
+ /* set load_per_context_state & load_gfx_sh_regs */
+ dw2 |= 0x10002;
+ }
+
amdgpu_ring_write(ring, PACKET3(PACKET3_CONTEXT_CONTROL, 1));
- amdgpu_ring_write(ring, 0x80000000);
+ amdgpu_ring_write(ring, dw2);
amdgpu_ring_write(ring, 0);
}
@@ -3537,11 +3537,12 @@ static const struct amdgpu_ring_funcs gfx_v6_0_ring_funcs_gfx = {
.emit_frame_size =
5 + 5 + /* hdp flush / invalidate */
14 + 14 + 14 + /* gfx_v6_0_ring_emit_fence x3 for user fence, vm fence */
- 7 + 4 + /* gfx_v6_0_ring_emit_pipeline_sync */
- SI_FLUSH_GPU_TLB_NUM_WREG * 5 + 7 + 6 + /* gfx_v6_0_ring_emit_vm_flush */
- 3 + 2 + /* gfx_v6_ring_emit_cntxcntl including vgt flush */
+ 7 + /* gfx_v6_0_ring_emit_pipeline_sync */
+ SI_FLUSH_GPU_TLB_NUM_WREG * 5 + 7 + 2 + /* gfx_v6_0_ring_emit_vm_flush */
+ 3 * 2 + /* gfx_v6_0_ring_emit_sb x3 (from amdgpu_vm_flush, amdgpu_ib_schedule) */
+ 3 + 2 + 2 + /* gfx_v6_ring_emit_cntxcntl including VGT flush */
5, /* SURFACE_SYNC */
- .emit_ib_size = 6, /* gfx_v6_0_ring_emit_ib */
+ .emit_ib_size = 4, /* gfx_v6_0_ring_emit_ib */
.emit_ib = gfx_v6_0_ring_emit_ib,
.emit_fence = gfx_v6_0_ring_emit_fence,
.emit_pipeline_sync = gfx_v6_0_ring_emit_pipeline_sync,
@@ -3549,6 +3550,7 @@ static const struct amdgpu_ring_funcs gfx_v6_0_ring_funcs_gfx = {
.test_ring = gfx_v6_0_ring_test_ring,
.test_ib = gfx_v6_0_ring_test_ib,
.insert_nop = amdgpu_ring_insert_nop,
+ .emit_switch_buffer = gfx_v6_0_ring_emit_sb,
.emit_cntxcntl = gfx_v6_ring_emit_cntxcntl,
.emit_wreg = gfx_v6_0_ring_emit_wreg,
.emit_mem_sync = gfx_v6_0_emit_mem_sync,
@@ -3564,10 +3566,11 @@ static const struct amdgpu_ring_funcs gfx_v6_0_ring_funcs_compute = {
.emit_frame_size =
5 + 5 + /* hdp flush / invalidate */
7 + /* gfx_v6_0_ring_emit_pipeline_sync */
- SI_FLUSH_GPU_TLB_NUM_WREG * 5 + 7 + /* gfx_v6_0_ring_emit_vm_flush */
+ SI_FLUSH_GPU_TLB_NUM_WREG * 5 + 7 + 2 + /* gfx_v6_0_ring_emit_vm_flush */
14 + 14 + 14 + /* gfx_v6_0_ring_emit_fence x3 for user fence, vm fence */
+ 3 * 2 + /* gfx_v6_0_ring_emit_sb x3 (from amdgpu_vm_flush, amdgpu_ib_schedule) */
5, /* SURFACE_SYNC */
- .emit_ib_size = 6, /* gfx_v6_0_ring_emit_ib */
+ .emit_ib_size = 4, /* gfx_v6_0_ring_emit_ib */
.emit_ib = gfx_v6_0_ring_emit_ib,
.emit_fence = gfx_v6_0_ring_emit_fence,
.emit_pipeline_sync = gfx_v6_0_ring_emit_pipeline_sync,
@@ -3575,6 +3578,7 @@ static const struct amdgpu_ring_funcs gfx_v6_0_ring_funcs_compute = {
.test_ring = gfx_v6_0_ring_test_ring,
.test_ib = gfx_v6_0_ring_test_ib,
.insert_nop = amdgpu_ring_insert_nop,
+ .emit_switch_buffer = gfx_v6_0_ring_emit_sb,
.emit_wreg = gfx_v6_0_ring_emit_wreg,
.emit_mem_sync = gfx_v6_0_emit_mem_sync,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
index 09f8324fdb99..c175c58c6073 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
@@ -1898,9 +1898,13 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
len_dw = msg[1] / 4;
num_buffers = msg[2];
- /* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
- if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
- DRM_ERROR("VCN message has too many buffers!\n");
+ /* Verify that all indices fit within the claimed length.
+ * There are 6 dwords in the header before the first buffer.
+ * Each buffer has 4 dwords. Any trailing dwords after the
+ * last buffer are ignored.
+ */
+ if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
+ DRM_ERROR("Invalid VCN message!\n");
r = -EINVAL;
goto out;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
index 80373143949d..d86de234765d 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c
@@ -1821,9 +1821,13 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
len_dw = msg[1] / 4;
num_buffers = msg[2];
- /* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
- if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
- DRM_ERROR("VCN message has too many buffers!\n");
+ /* Verify that all indices fit within the claimed length.
+ * There are 6 dwords in the header before the first buffer.
+ * Each buffer has 4 dwords. Any trailing dwords after the
+ * last buffer are ignored.
+ */
+ if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
+ DRM_ERROR("Invalid VCN message!\n");
r = -EINVAL;
goto out;
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 697325a8f1c6..45bb957421aa 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -1167,7 +1167,8 @@ static int kfd_ioctl_alloc_memory_of_gpu(struct file *filep,
if (flags & KFD_IOC_ALLOC_MEM_FLAGS_AQL_QUEUE_MEM)
size >>= 1;
- atomic64_add(PAGE_ALIGN(size), &pdd->vram_usage);
+ size = PAGE_ALIGN(size);
+ atomic64_add(size, &pdd->vram_usage);
}
mutex_unlock(&p->mutex);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 9035eebe9cd6..5240b5c442a0 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -617,6 +617,11 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
q->properties.type)];
+ if (qd && !mqd_mgr->restore_mqd) {
+ pr_debug("restore_mqd not implemented for this GPU\n");
+ retval = -EOPNOTSUPP;
+ goto deallocate_vmid;
+ }
if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE) {
retval = allocate_hqd(dqm, q);
if (retval)
@@ -1967,6 +1972,11 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
q->properties.type)];
+ if (qd && !mqd_mgr->restore_mqd) {
+ pr_debug("restore_mqd not implemented for this GPU\n");
+ retval = -EOPNOTSUPP;
+ goto out_deallocate_doorbell;
+ }
if (q->properties.type == KFD_QUEUE_TYPE_SDMA ||
q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
index e9c765e1c17c..9153abe7a677 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_color.c
@@ -360,6 +360,12 @@ static bool __is_lut_linear(const struct drm_color_lut *lut, uint32_t size)
uint32_t expected;
int delta;
+ /* A LUT with fewer than two entries can't be interpolated and would
+ * divide by zero below (size - 1); it can't be treated as linear.
+ */
+ if (size < 2)
+ return false;
+
for (i = 0; i < size; i++) {
/* All color values should equal */
if ((lut[i].red != lut[i].green) || (lut[i].green != lut[i].blue))
@@ -1143,6 +1149,13 @@ __set_dm_plane_degamma(struct drm_plane_state *plane_state,
degamma_lut = __extract_blob_lut(dm_plane_state->degamma_lut,
°amma_size);
+ if (degamma_lut && degamma_size != MAX_COLOR_LUT_ENTRIES) {
+ drm_dbg(plane_state->state->dev,
+ "Invalid Plane Degamma LUT size. Should be %u but got %u.\n",
+ MAX_COLOR_LUT_ENTRIES, degamma_size);
+ return -EINVAL;
+ }
+
has_degamma_lut = degamma_lut &&
!__is_lut_linear(degamma_lut, degamma_size);
diff --git a/drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c b/drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c
index da963f73829f..c8ae19b37d84 100644
--- a/drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/hubp/dcn10/dcn10_hubp.c
@@ -755,8 +755,7 @@ bool hubp1_is_flip_pending(struct hubp *hubp)
if (flip_pending)
return true;
- if (hubp &&
- earliest_inuse_address.grph.addr.quad_part != hubp->request_address.grph.addr.quad_part)
+ if (earliest_inuse_address.grph.addr.quad_part != hubp->request_address.grph.addr.quad_part)
return true;
return false;
diff --git a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c
index 0f1c411523a2..e4b2cfb2b556 100644
--- a/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c
+++ b/drivers/gpu/drm/amd/display/dc/link/protocols/link_dp_dpia_bw.c
@@ -102,6 +102,11 @@ static int get_estimated_bw(struct dc_link *link)
{
uint8_t bw_estimated_bw = 0;
+ if (link->dpia_bw_alloc_config.bw_granularity == 0) {
+ DC_LOG_ERROR("%s: BW granularity is zero!\n", __func__);
+ return 0;
+ }
+
core_link_read_dpcd(
link,
ESTIMATED_BW,
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
index 80e34a7748d7..2f46c185b528 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c
@@ -2216,12 +2216,24 @@ static int smu7_patch_voltage_dependency_tables_with_lookup_table(
if (data->vdd_gfx_control == SMU7_VOLTAGE_CONTROL_BY_SVID2) {
for (entry_id = 0; entry_id < sclk_table->count; ++entry_id) {
voltage_id = sclk_table->entries[entry_id].vddInd;
+ if (voltage_id >= table_info->vddgfx_lookup_table->count) {
+ pr_err("amdgpu: sclk[%u] vddgfx index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddgfx_lookup_table->count);
+ return -EINVAL;
+ }
sclk_table->entries[entry_id].vddgfx =
table_info->vddgfx_lookup_table->entries[voltage_id].us_vdd;
}
} else {
for (entry_id = 0; entry_id < sclk_table->count; ++entry_id) {
voltage_id = sclk_table->entries[entry_id].vddInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: sclk[%u] vddc index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
sclk_table->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
}
@@ -2229,12 +2241,24 @@ static int smu7_patch_voltage_dependency_tables_with_lookup_table(
for (entry_id = 0; entry_id < mclk_table->count; ++entry_id) {
voltage_id = mclk_table->entries[entry_id].vddInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: mclk[%u] vddc index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
mclk_table->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
}
for (entry_id = 0; entry_id < mm_table->count; ++entry_id) {
voltage_id = mm_table->entries[entry_id].vddcInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: mm[%u] vddc index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
mm_table->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
}
diff --git a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
index 9ace863792d4..b3fed2a478aa 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c
@@ -685,10 +685,18 @@ static int vega10_patch_voltage_dependency_tables_with_lookup_table(
case 3: vdt = table_info->vdd_dep_on_pixclk; break;
case 4: vdt = table_info->vdd_dep_on_dispclk; break;
case 5: vdt = table_info->vdd_dep_on_phyclk; break;
+ default:
+ continue;
}
for (entry_id = 0; entry_id < vdt->count; entry_id++) {
voltage_id = vdt->entries[entry_id].vddInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: clk_dep[%u][%u] vddc index %u out of bounds (%u)\n",
+ i, entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
vdt->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
}
@@ -696,23 +704,48 @@ static int vega10_patch_voltage_dependency_tables_with_lookup_table(
for (entry_id = 0; entry_id < mm_table->count; ++entry_id) {
voltage_id = mm_table->entries[entry_id].vddcInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: mm[%u] vddc index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
mm_table->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
}
for (entry_id = 0; entry_id < mclk_table->count; ++entry_id) {
voltage_id = mclk_table->entries[entry_id].vddInd;
+ if (voltage_id >= table_info->vddc_lookup_table->count) {
+ pr_err("amdgpu: mclk[%u] vddc index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddc_lookup_table->count);
+ return -EINVAL;
+ }
mclk_table->entries[entry_id].vddc =
table_info->vddc_lookup_table->entries[voltage_id].us_vdd;
+
voltage_id = mclk_table->entries[entry_id].vddciInd;
+ if (voltage_id >= table_info->vddci_lookup_table->count) {
+ pr_err("amdgpu: mclk[%u] vddci index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddci_lookup_table->count);
+ return -EINVAL;
+ }
mclk_table->entries[entry_id].vddci =
table_info->vddci_lookup_table->entries[voltage_id].us_vdd;
+
voltage_id = mclk_table->entries[entry_id].mvddInd;
+ if (voltage_id >= table_info->vddmem_lookup_table->count) {
+ pr_err("amdgpu: mclk[%u] vddmem index %u out of bounds (%u)\n",
+ entry_id, voltage_id,
+ table_info->vddmem_lookup_table->count);
+ return -EINVAL;
+ }
mclk_table->entries[entry_id].mvdd =
table_info->vddmem_lookup_table->entries[voltage_id].us_vdd;
}
-
return 0;
}
diff --git a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
index 3c4ea9bdefbe..322794eef41a 100644
--- a/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
+++ b/drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c
@@ -2158,7 +2158,7 @@ cdns_mhdp_bridge_atomic_reset(struct drm_bridge *bridge)
cdns_mhdp_state = kzalloc(sizeof(*cdns_mhdp_state), GFP_KERNEL);
if (!cdns_mhdp_state)
- return NULL;
+ return ERR_PTR(-ENOMEM);
__drm_atomic_helper_bridge_reset(bridge, &cdns_mhdp_state->base);
diff --git a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
index 0031f3c54882..94ae886231c5 100644
--- a/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
+++ b/drivers/gpu/drm/bridge/synopsys/dw-hdmi.c
@@ -3383,7 +3383,8 @@ struct dw_hdmi *dw_hdmi_probe(struct platform_device *pdev,
break;
default:
dev_err(dev, "reg-io-width must be 1 or 4\n");
- return ERR_PTR(-EINVAL);
+ ret = -EINVAL;
+ goto err_res;
}
iores = platform_get_resource(pdev, IORESOURCE_MEM, 0);
diff --git a/drivers/gpu/drm/bridge/tc358767.c b/drivers/gpu/drm/bridge/tc358767.c
index 47189587643a..47c9b2d4369e 100644
--- a/drivers/gpu/drm/bridge/tc358767.c
+++ b/drivers/gpu/drm/bridge/tc358767.c
@@ -519,7 +519,7 @@ static ssize_t tc_aux_transfer(struct drm_dp_aux *aux,
* address-only transfer
*/
if (size)
- size = FIELD_GET(AUX_BYTES, auxstatus);
+ size = min_t(size_t, size, FIELD_GET(AUX_BYTES, auxstatus));
msg->reply = FIELD_GET(AUX_STATUS, auxstatus);
switch (request) {
diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
index fd36b8fd54e9..fc20b039bed4 100644
--- a/drivers/gpu/drm/drm_atomic_uapi.c
+++ b/drivers/gpu/drm/drm_atomic_uapi.c
@@ -1327,6 +1327,8 @@ static void complete_signaling(struct drm_device *dev,
* to prevent a double free in drm_atomic_state_clear.
*/
if (event && (event->base.fence || event->base.file_priv)) {
+ if (crtc_state->commit && crtc_state->commit->abort_completion)
+ drm_crtc_commit_put(crtc_state->commit);
drm_event_cancel_free(dev, &event->base);
crtc_state->event = NULL;
}
diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c
index ac30b0ec9d93..8a82e6c6e2d2 100644
--- a/drivers/gpu/drm/drm_drv.c
+++ b/drivers/gpu/drm/drm_drv.c
@@ -469,6 +469,22 @@ void drm_dev_exit(int idx)
}
EXPORT_SYMBOL(drm_dev_exit);
+/*
+ * Mark the device as unplugged and wait for any in-flight drm_dev_enter()
+ * critical sections to complete.
+ */
+static void drm_dev_synchronize_unplug(struct drm_device *dev)
+{
+ /*
+ * After synchronizing any critical read section is guaranteed to see
+ * the new value of ->unplugged, and any critical section which might
+ * still have seen the old value of ->unplugged is guaranteed to have
+ * finished.
+ */
+ dev->unplugged = true;
+ synchronize_srcu(&drm_unplug_srcu);
+}
+
/**
* drm_dev_unplug - unplug a DRM device
* @dev: DRM device
@@ -481,15 +497,7 @@ EXPORT_SYMBOL(drm_dev_exit);
*/
void drm_dev_unplug(struct drm_device *dev)
{
- /*
- * After synchronizing any critical read section is guaranteed to see
- * the new value of ->unplugged, and any critical section which might
- * still have seen the old value of ->unplugged is guaranteed to have
- * finished.
- */
- dev->unplugged = true;
- synchronize_srcu(&drm_unplug_srcu);
-
+ drm_dev_synchronize_unplug(dev);
drm_dev_unregister(dev);
/* Clear all CPU mappings pointing to this device */
@@ -931,6 +939,7 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags)
goto err_minors;
dev->registered = true;
+ dev->unplugged = false;
if (driver->load) {
ret = driver->load(dev, flags);
@@ -957,6 +966,13 @@ int drm_dev_register(struct drm_device *dev, unsigned long flags)
if (dev->driver->unload)
dev->driver->unload(dev);
err_minors:
+ /*
+ * If a minor was registered before the failure, userspace could have
+ * opened it and entered a drm_dev_enter() critical section. Ensure all
+ * such sections complete before we clean up.
+ */
+ drm_dev_synchronize_unplug(dev);
+
remove_compat_control_link(dev);
drm_minor_unregister(dev, DRM_MINOR_ACCEL);
drm_minor_unregister(dev, DRM_MINOR_PRIMARY);
diff --git a/drivers/gpu/drm/drm_dumb_buffers.c b/drivers/gpu/drm/drm_dumb_buffers.c
index 70032bba1c97..7b4ba9fab299 100644
--- a/drivers/gpu/drm/drm_dumb_buffers.c
+++ b/drivers/gpu/drm/drm_dumb_buffers.c
@@ -29,7 +29,6 @@
#include <drm/drm_mode.h>
#include "drm_crtc_internal.h"
-#include "drm_internal.h"
/**
* DOC: overview
diff --git a/drivers/gpu/drm/gud/gud_connector.c b/drivers/gpu/drm/gud/gud_connector.c
index 0f07d77c5d52..e1f695da4fae 100644
--- a/drivers/gpu/drm/gud/gud_connector.c
+++ b/drivers/gpu/drm/gud/gud_connector.c
@@ -397,8 +397,16 @@ static int gud_connector_add_tv_mode(struct gud_device *gdrm, struct drm_connect
}
num_modes = ret / GUD_CONNECTOR_TV_MODE_NAME_LEN;
- for (i = 0; i < num_modes; i++)
- modes[i] = &buf[i * GUD_CONNECTOR_TV_MODE_NAME_LEN];
+ for (i = 0; i < num_modes; i++) {
+ char *mode = &buf[i * GUD_CONNECTOR_TV_MODE_NAME_LEN];
+
+ if (!memchr(mode, '\0', GUD_CONNECTOR_TV_MODE_NAME_LEN)) {
+ ret = -EIO;
+ goto free;
+ }
+
+ modes[i] = mode;
+ }
ret = drm_mode_create_tv_properties_legacy(connector->dev, num_modes, modes);
free:
diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
index db7fce4e8cc3..165e7e26e689 100644
--- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
+++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c
@@ -72,46 +72,28 @@ static int hibmc_get_best_clock_idx(const struct drm_display_mode *mode)
static int hibmc_plane_atomic_check(struct drm_plane *plane,
struct drm_atomic_state *state)
{
- struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
- plane);
- struct drm_framebuffer *fb = new_plane_state->fb;
- struct drm_crtc *crtc = new_plane_state->crtc;
- struct drm_crtc_state *crtc_state;
- u32 src_w = new_plane_state->src_w >> 16;
- u32 src_h = new_plane_state->src_h >> 16;
-
- if (!crtc || !fb)
- return 0;
+ struct drm_plane_state *new_plane_state =
+ drm_atomic_get_new_plane_state(state, plane);
+ struct drm_crtc_state *new_crtc_state = NULL;
+ int ret;
- crtc_state = drm_atomic_get_crtc_state(state, crtc);
- if (IS_ERR(crtc_state))
- return PTR_ERR(crtc_state);
+ if (new_plane_state->crtc)
+ new_crtc_state = drm_atomic_get_new_crtc_state(state, new_plane_state->crtc);
- if (src_w != new_plane_state->crtc_w || src_h != new_plane_state->crtc_h) {
- drm_dbg_atomic(plane->dev, "scale not support\n");
- return -EINVAL;
- }
-
- if (new_plane_state->crtc_x < 0 || new_plane_state->crtc_y < 0) {
- drm_dbg_atomic(plane->dev, "crtc_x/y of drm_plane state is invalid\n");
- return -EINVAL;
- }
-
- if (!crtc_state->enable)
+ ret = drm_atomic_helper_check_plane_state(new_plane_state, new_crtc_state,
+ DRM_PLANE_NO_SCALING,
+ DRM_PLANE_NO_SCALING,
+ false, true);
+ if (ret)
+ return ret;
+ else if (!new_plane_state->visible)
return 0;
- if (new_plane_state->crtc_x + new_plane_state->crtc_w >
- crtc_state->adjusted_mode.hdisplay ||
- new_plane_state->crtc_y + new_plane_state->crtc_h >
- crtc_state->adjusted_mode.vdisplay) {
- drm_dbg_atomic(plane->dev, "visible portion of plane is invalid\n");
- return -EINVAL;
- }
-
if (new_plane_state->fb->pitches[0] % 128 != 0) {
drm_dbg_atomic(plane->dev, "wrong stride with 128-byte aligned\n");
return -EINVAL;
}
+
return 0;
}
@@ -153,10 +135,8 @@ static void hibmc_plane_atomic_update(struct drm_plane *plane,
}
static const u32 channel_formats1[] = {
- DRM_FORMAT_RGB565, DRM_FORMAT_BGR565, DRM_FORMAT_RGB888,
- DRM_FORMAT_BGR888, DRM_FORMAT_XRGB8888, DRM_FORMAT_XBGR8888,
- DRM_FORMAT_RGBA8888, DRM_FORMAT_BGRA8888, DRM_FORMAT_ARGB8888,
- DRM_FORMAT_ABGR8888
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_RGB565,
};
static const struct drm_plane_funcs hibmc_plane_funcs = {
diff --git a/drivers/gpu/drm/i915/i915_pci.c b/drivers/gpu/drm/i915/i915_pci.c
index d37bb3a704d0..659cb3ab2611 100644
--- a/drivers/gpu/drm/i915/i915_pci.c
+++ b/drivers/gpu/drm/i915/i915_pci.c
@@ -961,6 +961,9 @@ static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
(struct intel_device_info *) ent->driver_data;
int err;
+ if (!intel_info)
+ return -ENODEV;
+
if (intel_info->require_force_probe && !id_forced(pdev->device)) {
dev_info(&pdev->dev,
"Your graphics device %04x is not properly supported by i915 in this\n"
diff --git a/drivers/gpu/drm/lima/lima_device.c b/drivers/gpu/drm/lima/lima_device.c
index 0bf7105c8748..7c873e62c16d 100644
--- a/drivers/gpu/drm/lima/lima_device.c
+++ b/drivers/gpu/drm/lima/lima_device.c
@@ -368,12 +368,6 @@ int lima_device_init(struct lima_device *ldev)
if (err)
goto err_out0;
- ldev->empty_vm = lima_vm_create(ldev);
- if (!ldev->empty_vm) {
- err = -ENOMEM;
- goto err_out1;
- }
-
ldev->va_start = 0;
if (ldev->id == lima_gpu_mali450) {
ldev->va_end = LIMA_VA_RESERVE_START;
@@ -387,6 +381,12 @@ int lima_device_init(struct lima_device *ldev)
} else
ldev->va_end = LIMA_VA_RESERVE_END;
+ ldev->empty_vm = lima_vm_create(ldev);
+ if (!ldev->empty_vm) {
+ err = -ENOMEM;
+ goto err_out1;
+ }
+
ldev->iomem = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(ldev->iomem)) {
dev_err(ldev->dev, "fail to ioremap iomem\n");
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c
index a763fe5d6e2d..8bb23012994c 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c
@@ -1117,7 +1117,7 @@ static const struct adreno_reglist a730_hwcg[] = {
{ REG_A6XX_RBBM_CLOCK_DELAY_SP0, 0x00000080 },
{ REG_A6XX_RBBM_CLOCK_CNTL_TP0, 0x22222220 },
{ REG_A6XX_RBBM_CLOCK_CNTL2_TP0, 0x22222222 },
- { REG_A6XX_RBBM_CLOCK_CNTL3_TP0, 0x22222222 },
+ { REG_A6XX_RBBM_CLOCK_CNTL3_TP0, 0x22220222 },
{ REG_A6XX_RBBM_CLOCK_CNTL4_TP0, 0x00222222 },
{ REG_A6XX_RBBM_CLOCK_HYST_TP0, 0x77777777 },
{ REG_A6XX_RBBM_CLOCK_HYST2_TP0, 0x77777777 },
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index 3e36cec3801e..99c3abd90c19 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -514,7 +514,7 @@ static void a6xx_rpmh_stop(struct a6xx_gmu *gmu)
int ret;
u32 val;
- if (test_and_clear_bit(GMU_STATUS_FW_START, &gmu->status))
+ if (!test_and_clear_bit(GMU_STATUS_FW_START, &gmu->status))
return;
gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1);
@@ -1186,6 +1186,9 @@ static void a6xx_gmu_shutdown(struct a6xx_gmu *gmu)
/* Stop the interrupts and mask the hardware */
a6xx_gmu_irq_disable(gmu);
+ /* Halt the gmu cm3 core */
+ gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 1);
+
/* Tell RPMh to power off the GPU */
a6xx_rpmh_stop(gmu);
diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_core_perf.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_core_perf.c
index 260accc151d4..d29df73c02c5 100644
--- a/drivers/gpu/drm/msm/disp/dpu1/dpu_core_perf.c
+++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_core_perf.c
@@ -383,6 +383,10 @@ int dpu_core_perf_crtc_update(struct drm_crtc *crtc,
trace_dpu_core_perf_update_clk(kms->dev, !crtc->enabled, clk_rate);
+ /* If we're going offline, PM callbacks will disable the clocks instead */
+ if (!clk_rate)
+ return 0;
+
clk_rate = min(clk_rate, kms->perf.max_core_clk_rate);
ret = dev_pm_opp_set_rate(&kms->pdev->dev, clk_rate);
if (ret) {
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index c9b580771609..77173183509f 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -499,8 +499,6 @@ int dsi_link_clk_enable_v2(struct msm_dsi_host *msm_host)
void dsi_link_clk_disable_6g(struct msm_dsi_host *msm_host)
{
- /* Drop the performance state vote */
- dev_pm_opp_set_rate(&msm_host->pdev->dev, 0);
clk_disable_unprepare(msm_host->esc_clk);
clk_disable_unprepare(msm_host->pixel_clk);
clk_disable_unprepare(msm_host->byte_intf_clk);
@@ -621,12 +619,24 @@ static void dsi_calc_pclk(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
int dsi_calc_clk_rate_6g(struct msm_dsi_host *msm_host, bool is_bonded_dsi)
{
+ long rounded_byte_clk_rate;
+
if (!msm_host->mode) {
pr_err("%s: mode not set\n", __func__);
return -EINVAL;
}
dsi_calc_pclk(msm_host, is_bonded_dsi);
+
+ rounded_byte_clk_rate = clk_round_rate(msm_host->byte_clk,
+ msm_host->byte_clk_rate);
+ if (rounded_byte_clk_rate < 0) {
+ pr_err("%s: failed to round byte clock rate, %ld\n",
+ __func__, rounded_byte_clk_rate);
+ return rounded_byte_clk_rate;
+ }
+
+ msm_host->byte_clk_rate = rounded_byte_clk_rate;
msm_host->esc_clk_rate = clk_get_rate(msm_host->esc_clk);
return 0;
}
diff --git a/drivers/gpu/drm/mxsfb/lcdif_kms.c b/drivers/gpu/drm/mxsfb/lcdif_kms.c
index dbd42cc1da87..19884359ce1d 100644
--- a/drivers/gpu/drm/mxsfb/lcdif_kms.c
+++ b/drivers/gpu/drm/mxsfb/lcdif_kms.c
@@ -373,14 +373,23 @@ static void lcdif_disable_controller(struct lcdif_drm_private *lcdif)
int ret;
reg = readl(lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ /* Disable the layer for DMA. */
reg &= ~CTRLDESCL0_5_EN;
+ /*
+ * It is necessary to wait for the full frame to finish streaming
+ * through the DMA engine before we can safely disable it by removing
+ * the DISP_PARA_DISP_ON bit. Disabling it in-flight can leave the
+ * hardware confused and unable to resume streaming for the next frame.
+ */
+ reg |= CTRLDESCL0_5_SHADOW_LOAD_EN;
writel(reg, lcdif->base + LCDC_V8_CTRLDESCL0_5);
+ /* Wait for the frame to finish or timeout after 50 ms. */
ret = readl_poll_timeout(lcdif->base + LCDC_V8_CTRLDESCL0_5,
- reg, !(reg & CTRLDESCL0_5_EN),
- 0, 36000); /* Wait ~2 frame times max */
+ reg, !(reg & CTRLDESCL0_5_SHADOW_LOAD_EN),
+ 200, 50000);
if (ret)
- drm_err(lcdif->drm, "Failed to disable controller!\n");
+ drm_err(lcdif->drm, "Timed out waiting for final vblank!\n");
reg = readl(lcdif->base + LCDC_V8_DISP_PARA);
reg &= ~DISP_PARA_DISP_ON;
diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.c b/drivers/gpu/drm/nouveau/nouveau_chan.c
index 2cb2e5675807..debf7aef6ec2 100644
--- a/drivers/gpu/drm/nouveau/nouveau_chan.c
+++ b/drivers/gpu/drm/nouveau/nouveau_chan.c
@@ -90,6 +90,14 @@ nouveau_channel_del(struct nouveau_channel **pchan)
{
struct nouveau_channel *chan = *pchan;
if (chan) {
+ /*
+ * Drop the kill-event subscription first. Its handler
+ * dereferences chan->fence, which the fence context teardown
+ * below frees, so leaving it armed across the teardown leaves
+ * a window for a use-after-free.
+ */
+ nvif_event_dtor(&chan->kill);
+
if (chan->fence)
nouveau_fence(chan->cli->drm)->context_del(chan);
@@ -100,7 +108,6 @@ nouveau_channel_del(struct nouveau_channel **pchan)
nvif_object_dtor(&chan->nvsw);
nvif_object_dtor(&chan->gart);
nvif_object_dtor(&chan->vram);
- nvif_event_dtor(&chan->kill);
nvif_object_dtor(&chan->user);
nvif_mem_dtor(&chan->mem_userd);
nvif_object_dtor(&chan->push.ctxdma);
diff --git a/drivers/gpu/drm/nouveau/nouveau_sgdma.c b/drivers/gpu/drm/nouveau/nouveau_sgdma.c
index bd870028514b..f3a1902bc7e0 100644
--- a/drivers/gpu/drm/nouveau/nouveau_sgdma.c
+++ b/drivers/gpu/drm/nouveau/nouveau_sgdma.c
@@ -72,9 +72,7 @@ nouveau_sgdma_create_ttm(struct ttm_buffer_object *bo, uint32_t page_flags)
struct nouveau_sgdma_be *nvbe;
enum ttm_caching caching;
- if (nvbo->force_coherent)
- caching = ttm_uncached;
- else if (drm->agp.bridge)
+ if (nvbo->force_coherent || drm->agp.bridge)
caching = ttm_write_combined;
else
caching = ttm_cached;
diff --git a/drivers/gpu/drm/nouveau/nouveau_uvmm.c b/drivers/gpu/drm/nouveau/nouveau_uvmm.c
index f2e332b08eda..a68b7f0b013b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_uvmm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_uvmm.c
@@ -1251,6 +1251,7 @@ nouveau_uvmm_bind_job_submit(struct nouveau_job *job,
op->va.range);
if (!op->reg || op->reg->dirty) {
ret = -ENOENT;
+ op->reg = NULL;
goto unwind_continue;
}
@@ -1259,6 +1260,7 @@ nouveau_uvmm_bind_job_submit(struct nouveau_job *job,
op->va.range);
if (IS_ERR(op->ops)) {
ret = PTR_ERR(op->ops);
+ op->reg = NULL;
goto unwind_continue;
}
@@ -1402,6 +1404,7 @@ nouveau_uvmm_bind_job_submit(struct nouveau_job *job,
op->va.range);
break;
case OP_UNMAP_SPARSE:
+ op->reg->dirty = false;
__nouveau_uvma_region_insert(uvmm, op->reg);
nouveau_uvmm_sm_unmap_prepare_unwind(uvmm, &op->new,
op->ops);
@@ -1418,7 +1421,8 @@ nouveau_uvmm_bind_job_submit(struct nouveau_job *job,
break;
}
- drm_gpuva_ops_free(&uvmm->base, op->ops);
+ if (!IS_ERR_OR_NULL(op->ops))
+ drm_gpuva_ops_free(&uvmm->base, op->ops);
op->ops = NULL;
op->reg = NULL;
}
diff --git a/drivers/gpu/drm/omapdrm/dss/dsi.c b/drivers/gpu/drm/omapdrm/dss/dsi.c
index ea63c64d3a1a..b9b013ba80e2 100644
--- a/drivers/gpu/drm/omapdrm/dss/dsi.c
+++ b/drivers/gpu/drm/omapdrm/dss/dsi.c
@@ -455,15 +455,10 @@ static irqreturn_t omap_dsi_irq_handler(int irq, void *arg)
del_timer(&dsi->te_timer);
#endif
- /* make a copy and unlock, so that isrs can unregister
- * themselves */
- memcpy(&dsi->isr_tables_copy, &dsi->isr_tables,
- sizeof(dsi->isr_tables));
+ dsi_handle_isrs(&dsi->isr_tables, irqstatus, vcstatus, ciostatus);
spin_unlock(&dsi->irq_lock);
- dsi_handle_isrs(&dsi->isr_tables_copy, irqstatus, vcstatus, ciostatus);
-
dsi_handle_irq_errors(dsi, irqstatus, vcstatus, ciostatus);
dsi_collect_irq_stats(dsi, irqstatus, vcstatus, ciostatus);
diff --git a/drivers/gpu/drm/omapdrm/dss/dsi.h b/drivers/gpu/drm/omapdrm/dss/dsi.h
index 601707c0ecc4..2b25247ea893 100644
--- a/drivers/gpu/drm/omapdrm/dss/dsi.h
+++ b/drivers/gpu/drm/omapdrm/dss/dsi.h
@@ -379,8 +379,6 @@ struct dsi_data {
spinlock_t irq_lock;
struct dsi_isr_tables isr_tables;
- /* space for a copy used by the interrupt handler */
- struct dsi_isr_tables isr_tables_copy;
int update_vc;
#ifdef DSI_PERF_MEASURE
diff --git a/drivers/gpu/drm/panel/panel-edp.c b/drivers/gpu/drm/panel/panel-edp.c
index d6806956f63e..98537e1ac810 100644
--- a/drivers/gpu/drm/panel/panel-edp.c
+++ b/drivers/gpu/drm/panel/panel-edp.c
@@ -831,6 +831,13 @@ static int generic_edp_panel_probe(struct device *dev, struct panel_edp *panel)
return 0;
}
+static void panel_edp_put_adapter(void *_adap)
+{
+ struct i2c_adapter *adap = _adap;
+
+ put_device(&adap->dev);
+}
+
static int panel_edp_probe(struct device *dev, const struct panel_desc *desc,
struct drm_dp_aux *aux)
{
@@ -877,6 +884,11 @@ static int panel_edp_probe(struct device *dev, const struct panel_desc *desc,
if (!panel->ddc)
return -EPROBE_DEFER;
+
+ err = devm_add_action_or_reset(dev, panel_edp_put_adapter,
+ panel->ddc);
+ if (err)
+ return err;
} else if (aux) {
panel->ddc = &aux->ddc;
}
@@ -890,7 +902,7 @@ static int panel_edp_probe(struct device *dev, const struct panel_desc *desc,
err = drm_panel_of_backlight(&panel->base);
if (err)
- goto err_finished_ddc_init;
+ return err;
/*
* We use runtime PM for prepare / unprepare since those power the panel
@@ -937,9 +949,6 @@ static int panel_edp_probe(struct device *dev, const struct panel_desc *desc,
err_finished_pm_runtime:
pm_runtime_dont_use_autosuspend(dev);
pm_runtime_disable(dev);
-err_finished_ddc_init:
- if (panel->ddc && (!panel->aux || panel->ddc != &panel->aux->ddc))
- put_device(&panel->ddc->dev);
return err;
}
@@ -983,8 +992,6 @@ static void panel_edp_remove(struct device *dev)
pm_runtime_dont_use_autosuspend(dev);
pm_runtime_disable(dev);
- if (panel->ddc && (!panel->aux || panel->ddc != &panel->aux->ddc))
- put_device(&panel->ddc->dev);
drm_edid_free(panel->drm_edid);
panel->drm_edid = NULL;
diff --git a/drivers/gpu/drm/panel/panel-samsung-s6d16d0.c b/drivers/gpu/drm/panel/panel-samsung-s6d16d0.c
index 79f611963c61..f73b603be0b0 100644
--- a/drivers/gpu/drm/panel/panel-samsung-s6d16d0.c
+++ b/drivers/gpu/drm/panel/panel-samsung-s6d16d0.c
@@ -89,16 +89,22 @@ static int s6d16d0_prepare(struct drm_panel *panel)
MIPI_DSI_DCS_TEAR_MODE_VBLANK);
if (ret) {
dev_err(s6->dev, "failed to enable vblank TE (%d)\n", ret);
- return ret;
+ goto err_power_off;
}
/* Exit sleep mode and power on */
ret = mipi_dsi_dcs_exit_sleep_mode(dsi);
if (ret) {
dev_err(s6->dev, "failed to exit sleep mode (%d)\n", ret);
- return ret;
+ goto err_power_off;
}
return 0;
+
+err_power_off:
+ gpiod_set_value_cansleep(s6->reset_gpio, 1);
+ regulator_disable(s6->supply);
+
+ return ret;
}
static int s6d16d0_enable(struct drm_panel *panel)
diff --git a/drivers/gpu/drm/panthor/panthor_drv.c b/drivers/gpu/drm/panthor/panthor_drv.c
index 9d4bf7406e24..46aa5ed81589 100644
--- a/drivers/gpu/drm/panthor/panthor_drv.c
+++ b/drivers/gpu/drm/panthor/panthor_drv.c
@@ -1427,7 +1427,7 @@ static int panthor_probe(struct platform_device *pdev)
ptdev = devm_drm_dev_alloc(&pdev->dev, &panthor_drm_driver,
struct panthor_device, base);
if (IS_ERR(ptdev))
- return -ENOMEM;
+ return PTR_ERR(ptdev);
platform_set_drvdata(pdev, ptdev);
diff --git a/drivers/gpu/drm/panthor/panthor_fw.c b/drivers/gpu/drm/panthor/panthor_fw.c
index 2df3f57bcb86..65c8d62fe3d8 100644
--- a/drivers/gpu/drm/panthor/panthor_fw.c
+++ b/drivers/gpu/drm/panthor/panthor_fw.c
@@ -797,14 +797,15 @@ static int panthor_init_cs_iface(struct panthor_device *ptdev,
struct panthor_fw_csg_iface *csg_iface = panthor_fw_get_csg_iface(ptdev, csg_idx);
struct panthor_fw_cs_iface *cs_iface = &ptdev->fw->iface.streams[csg_idx][cs_idx];
u64 shared_section_sz = panthor_kernel_bo_size(ptdev->fw->shared_section->mem);
- u32 iface_offset = CSF_GROUP_CONTROL_OFFSET +
- (csg_idx * glb_iface->control->group_stride) +
+ u64 iface_offset = CSF_GROUP_CONTROL_OFFSET +
+ ((u64)csg_idx * glb_iface->control->group_stride) +
CSF_STREAM_CONTROL_OFFSET +
- (cs_idx * csg_iface->control->stream_stride);
+ ((u64)cs_idx * csg_iface->control->stream_stride);
struct panthor_fw_cs_iface *first_cs_iface =
panthor_fw_get_cs_iface(ptdev, 0, 0);
- if (iface_offset + sizeof(*cs_iface) >= shared_section_sz)
+ if (iface_offset > shared_section_sz ||
+ sizeof(*cs_iface->control) > shared_section_sz - iface_offset)
return -EINVAL;
spin_lock_init(&cs_iface->lock);
@@ -854,10 +855,12 @@ static int panthor_init_csg_iface(struct panthor_device *ptdev,
struct panthor_fw_global_iface *glb_iface = panthor_fw_get_glb_iface(ptdev);
struct panthor_fw_csg_iface *csg_iface = &ptdev->fw->iface.groups[csg_idx];
u64 shared_section_sz = panthor_kernel_bo_size(ptdev->fw->shared_section->mem);
- u32 iface_offset = CSF_GROUP_CONTROL_OFFSET + (csg_idx * glb_iface->control->group_stride);
+ u64 iface_offset = CSF_GROUP_CONTROL_OFFSET +
+ ((u64)csg_idx * glb_iface->control->group_stride);
unsigned int i;
- if (iface_offset + sizeof(*csg_iface) >= shared_section_sz)
+ if (iface_offset > shared_section_sz ||
+ sizeof(*csg_iface->control) > shared_section_sz - iface_offset)
return -EINVAL;
spin_lock_init(&csg_iface->lock);
@@ -909,11 +912,15 @@ static u32 panthor_get_instr_features(struct panthor_device *ptdev)
static int panthor_fw_init_ifaces(struct panthor_device *ptdev)
{
struct panthor_fw_global_iface *glb_iface = &ptdev->fw->iface.global;
+ u64 shared_section_sz = panthor_kernel_bo_size(ptdev->fw->shared_section->mem);
unsigned int i;
if (!ptdev->fw->shared_section->mem->kmap)
return -EINVAL;
+ if (sizeof(*glb_iface->control) > shared_section_sz)
+ return -EINVAL;
+
spin_lock_init(&glb_iface->lock);
glb_iface->control = ptdev->fw->shared_section->mem->kmap;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
index a551458ad434..42521bd2ac16 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop2.c
@@ -514,6 +514,8 @@ static bool is_yuv_output(u32 bus_format)
switch (bus_format) {
case MEDIA_BUS_FMT_YUV8_1X24:
case MEDIA_BUS_FMT_YUV10_1X30:
+ case MEDIA_BUS_FMT_YUYV10_1X20:
+ case MEDIA_BUS_FMT_UYVY10_1X20:
case MEDIA_BUS_FMT_UYYVYY8_0_5X24:
case MEDIA_BUS_FMT_UYYVYY10_0_5X30:
case MEDIA_BUS_FMT_YUYV8_2X8:
diff --git a/drivers/gpu/drm/solomon/ssd130x.c b/drivers/gpu/drm/solomon/ssd130x.c
index cd8347396082..23abff914303 100644
--- a/drivers/gpu/drm/solomon/ssd130x.c
+++ b/drivers/gpu/drm/solomon/ssd130x.c
@@ -863,12 +863,13 @@ static int ssd132x_update_rect(struct ssd130x_device *ssd130x,
*/
/* Set column start and end */
- ret = ssd130x_write_cmd(ssd130x, 3, SSD132X_SET_COL_RANGE, x / segment_width, columns - 1);
+ ret = ssd130x_write_cmd(ssd130x, 3, SSD132X_SET_COL_RANGE, x / segment_width,
+ x / segment_width + columns - 1);
if (ret < 0)
return ret;
/* Set row start and end */
- ret = ssd130x_write_cmd(ssd130x, 3, SSD132X_SET_ROW_RANGE, y, rows - 1);
+ ret = ssd130x_write_cmd(ssd130x, 3, SSD132X_SET_ROW_RANGE, y, y + rows - 1);
if (ret < 0)
return ret;
@@ -914,12 +915,12 @@ static int ssd133x_update_rect(struct ssd130x_device *ssd130x,
*/
/* Set column start and end */
- ret = ssd130x_write_cmd(ssd130x, 3, SSD133X_SET_COL_RANGE, x, columns - 1);
+ ret = ssd130x_write_cmd(ssd130x, 3, SSD133X_SET_COL_RANGE, x, x + columns - 1);
if (ret < 0)
return ret;
/* Set row start and end */
- ret = ssd130x_write_cmd(ssd130x, 3, SSD133X_SET_ROW_RANGE, y, rows - 1);
+ ret = ssd130x_write_cmd(ssd130x, 3, SSD133X_SET_ROW_RANGE, y, y + rows - 1);
if (ret < 0)
return ret;
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
index 329e2a2384e8..bb7d47cc5d0c 100644
--- a/drivers/gpu/drm/sun4i/sun4i_backend.c
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -685,7 +685,7 @@ static int sun4i_backend_init_sat(struct device *dev) {
ret = clk_prepare_enable(backend->sat_clk);
if (ret) {
dev_err(dev, "Couldn't enable the SAT clock\n");
- return ret;
+ goto err_assert_reset;
}
return 0;
diff --git a/drivers/gpu/drm/sun4i/sun4i_crtc.c b/drivers/gpu/drm/sun4i/sun4i_crtc.c
index 18e74047b0f5..3e90aeb68c48 100644
--- a/drivers/gpu/drm/sun4i/sun4i_crtc.c
+++ b/drivers/gpu/drm/sun4i/sun4i_crtc.c
@@ -208,7 +208,7 @@ struct sun4i_crtc *sun4i_crtc_init(struct drm_device *drm,
planes = sunxi_engine_layers_init(drm, engine);
if (IS_ERR(planes)) {
dev_err(drm->dev, "Couldn't create the planes\n");
- return NULL;
+ return ERR_CAST(planes);
}
/* find primary and cursor planes for drm_crtc_init_with_planes */
diff --git a/drivers/gpu/drm/sun4i/sun4i_hdmi_enc.c b/drivers/gpu/drm/sun4i/sun4i_hdmi_enc.c
index b3649449de30..a48bef891f05 100644
--- a/drivers/gpu/drm/sun4i/sun4i_hdmi_enc.c
+++ b/drivers/gpu/drm/sun4i/sun4i_hdmi_enc.c
@@ -146,7 +146,7 @@ static void sun4i_hdmi_enable(struct drm_encoder *encoder,
drm_atomic_helper_connector_hdmi_update_infoframes(connector, state);
- val |= SUN4I_HDMI_PKT_CTRL_TYPE(0, SUN4I_HDMI_PKT_AVI);
+ val = SUN4I_HDMI_PKT_CTRL_TYPE(0, SUN4I_HDMI_PKT_AVI);
val |= SUN4I_HDMI_PKT_CTRL_TYPE(1, SUN4I_HDMI_PKT_END);
writel(val, hdmi->base + SUN4I_HDMI_PKT_CTRL_REG(0));
diff --git a/drivers/gpu/drm/sun4i/sun4i_tcon.c b/drivers/gpu/drm/sun4i/sun4i_tcon.c
index a1a2c845ade0..ab4409dcbf30 100644
--- a/drivers/gpu/drm/sun4i/sun4i_tcon.c
+++ b/drivers/gpu/drm/sun4i/sun4i_tcon.c
@@ -717,9 +717,11 @@ void sun4i_tcon_mode_set(struct sun4i_tcon *tcon,
case DRM_MODE_ENCODER_DSI:
/* DSI is tied to special case of CPU interface */
sun4i_tcon0_mode_set_cpu(tcon, encoder, mode);
+ sun4i_tcon_set_mux(tcon, 0, encoder);
break;
case DRM_MODE_ENCODER_LVDS:
sun4i_tcon0_mode_set_lvds(tcon, encoder, mode);
+ sun4i_tcon_set_mux(tcon, 0, encoder);
break;
case DRM_MODE_ENCODER_NONE:
sun4i_tcon0_mode_set_rgb(tcon, encoder, mode);
@@ -970,6 +972,7 @@ static int sun4i_tcon_of_get_id_from_port(struct device_node *port)
continue;
ret = of_property_read_u32(remote, "reg", ®);
+ of_node_put(remote);
if (ret)
continue;
@@ -1407,7 +1410,7 @@ static int sun8i_r40_tcon_tv_set_mux(struct sun4i_tcon *tcon,
{
struct device_node *port, *remote;
struct platform_device *pdev;
- int id, ret;
+ int id, ret = 0;
/* find TCON TOP platform device and TCON id */
@@ -1430,21 +1433,20 @@ static int sun8i_r40_tcon_tv_set_mux(struct sun4i_tcon *tcon,
if (IS_ENABLED(CONFIG_DRM_SUN8I_TCON_TOP) &&
encoder->encoder_type == DRM_MODE_ENCODER_TMDS) {
ret = sun8i_tcon_top_set_hdmi_src(&pdev->dev, id);
- if (ret) {
- put_device(&pdev->dev);
- return ret;
- }
+ if (ret)
+ goto out_put_device;
}
if (IS_ENABLED(CONFIG_DRM_SUN8I_TCON_TOP)) {
ret = sun8i_tcon_top_de_config(&pdev->dev, tcon->id, id);
- if (ret) {
- put_device(&pdev->dev);
- return ret;
- }
+ if (ret)
+ goto out_put_device;
}
- return 0;
+out_put_device:
+ put_device(&pdev->dev);
+
+ return ret;
}
static const struct sun4i_tcon_quirks sun4i_a10_quirks = {
diff --git a/drivers/gpu/drm/sun4i/sun8i_dw_hdmi.c b/drivers/gpu/drm/sun4i/sun8i_dw_hdmi.c
index 4727dfaa8fb9..af9be9ba8b96 100644
--- a/drivers/gpu/drm/sun4i/sun8i_dw_hdmi.c
+++ b/drivers/gpu/drm/sun4i/sun8i_dw_hdmi.c
@@ -83,6 +83,8 @@ static u32 sun8i_dw_hdmi_find_possible_crtcs(struct drm_device *drm,
of_node_put(remote_port);
}
}
+
+ of_node_put(port);
} else {
crtcs = drm_of_find_possible_crtcs(drm, node);
}
diff --git a/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c b/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c
index 4fa69c463dc4..489ea94693ff 100644
--- a/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c
+++ b/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c
@@ -91,7 +91,7 @@ static const struct dw_hdmi_mpll_config sun50i_h6_mpll_cfg[] = {
},
}, {
594000000, {
- { 0x1a40, 0x0003 },
+ { 0x1a7c, 0x0003 },
{ 0x3b4c, 0x0003 },
{ 0x5a64, 0x0003 },
},
diff --git a/drivers/gpu/drm/sun4i/sun8i_vi_scaler.c b/drivers/gpu/drm/sun4i/sun8i_vi_scaler.c
index 7ba75011adf9..328b84df2ee5 100644
--- a/drivers/gpu/drm/sun4i/sun8i_vi_scaler.c
+++ b/drivers/gpu/drm/sun4i/sun8i_vi_scaler.c
@@ -891,20 +891,26 @@ static void sun8i_vi_scaler_set_coeff(struct regmap *map, u32 base,
lan3coefftab32_left[offset + i]);
regmap_write(map, SUN8I_SCALER_VSU_YHCOEFF1(base, i),
lan3coefftab32_right[offset + i]);
+ }
+ offset = sun8i_vi_scaler_coef_index(vstep) *
+ SUN8I_VI_SCALER_COEFF_COUNT;
+ for (i = 0; i < SUN8I_VI_SCALER_COEFF_COUNT; i++)
+ regmap_write(map, SUN8I_SCALER_VSU_YVCOEFF(base, i),
+ lan2coefftab32[offset + i]);
+
+ offset = sun8i_vi_scaler_coef_index(hstep / format->hsub) *
+ SUN8I_VI_SCALER_COEFF_COUNT;
+ for (i = 0; i < SUN8I_VI_SCALER_COEFF_COUNT; i++) {
regmap_write(map, SUN8I_SCALER_VSU_CHCOEFF0(base, i),
ch_left[offset + i]);
regmap_write(map, SUN8I_SCALER_VSU_CHCOEFF1(base, i),
ch_right[offset + i]);
}
-
- offset = sun8i_vi_scaler_coef_index(hstep) *
+ offset = sun8i_vi_scaler_coef_index(vstep / format->vsub) *
SUN8I_VI_SCALER_COEFF_COUNT;
- for (i = 0; i < SUN8I_VI_SCALER_COEFF_COUNT; i++) {
- regmap_write(map, SUN8I_SCALER_VSU_YVCOEFF(base, i),
- lan2coefftab32[offset + i]);
+ for (i = 0; i < SUN8I_VI_SCALER_COEFF_COUNT; i++)
regmap_write(map, SUN8I_SCALER_VSU_CVCOEFF(base, i),
cy[offset + i]);
- }
}
void sun8i_vi_scaler_enable(struct sun8i_mixer *mixer, int layer, bool enable)
diff --git a/drivers/gpu/drm/tve200/tve200_drv.c b/drivers/gpu/drm/tve200/tve200_drv.c
index acce210e2554..0f55ead3ddbb 100644
--- a/drivers/gpu/drm/tve200/tve200_drv.c
+++ b/drivers/gpu/drm/tve200/tve200_drv.c
@@ -261,6 +261,7 @@ static const struct of_device_id tve200_of_match[] = {
},
{},
};
+MODULE_DEVICE_TABLE(of, tve200_of_match);
static struct platform_driver tve200_driver = {
.driver = {
diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h
index b6e11968fba4..a15a17a4dbc6 100644
--- a/drivers/gpu/drm/v3d/v3d_drv.h
+++ b/drivers/gpu/drm/v3d/v3d_drv.h
@@ -62,6 +62,7 @@ struct v3d_queue_state {
/* Currently active job for this queue */
struct v3d_job *active_job;
+ spinlock_t queue_lock;
};
/* Performance monitor object. The perform lifetime is controlled by userspace
@@ -154,11 +155,6 @@ struct v3d_dev {
struct v3d_queue_state queue[V3D_MAX_QUEUES];
- /* Spinlock used to synchronize the overflow memory
- * management against bin job submission.
- */
- spinlock_t job_lock;
-
/* Used to track the active perfmon if any. */
struct v3d_perfmon *active_perfmon;
@@ -532,7 +528,7 @@ void v3d_get_stats(const struct v3d_stats *stats, u64 timestamp,
/* v3d_fence.c */
extern const struct dma_fence_ops v3d_fence_ops;
-struct dma_fence *v3d_fence_create(struct v3d_dev *v3d, enum v3d_queue queue);
+struct dma_fence *v3d_fence_create(struct v3d_dev *v3d, enum v3d_queue q);
/* v3d_gem.c */
int v3d_gem_init(struct drm_device *dev);
diff --git a/drivers/gpu/drm/v3d/v3d_fence.c b/drivers/gpu/drm/v3d/v3d_fence.c
index 89840ed212c0..8f8471adae34 100644
--- a/drivers/gpu/drm/v3d/v3d_fence.c
+++ b/drivers/gpu/drm/v3d/v3d_fence.c
@@ -3,8 +3,9 @@
#include "v3d_drv.h"
-struct dma_fence *v3d_fence_create(struct v3d_dev *v3d, enum v3d_queue queue)
+struct dma_fence *v3d_fence_create(struct v3d_dev *v3d, enum v3d_queue q)
{
+ struct v3d_queue_state *queue = &v3d->queue[q];
struct v3d_fence *fence;
fence = kzalloc(sizeof(*fence), GFP_KERNEL);
@@ -12,10 +13,10 @@ struct dma_fence *v3d_fence_create(struct v3d_dev *v3d, enum v3d_queue queue)
return ERR_PTR(-ENOMEM);
fence->dev = &v3d->drm;
- fence->queue = queue;
- fence->seqno = ++v3d->queue[queue].emit_seqno;
- dma_fence_init(&fence->base, &v3d_fence_ops, &v3d->job_lock,
- v3d->queue[queue].fence_context, fence->seqno);
+ fence->queue = q;
+ fence->seqno = ++queue->emit_seqno;
+ dma_fence_init(&fence->base, &v3d_fence_ops, &queue->queue_lock,
+ queue->fence_context, fence->seqno);
return &fence->base;
}
diff --git a/drivers/gpu/drm/v3d/v3d_gem.c b/drivers/gpu/drm/v3d/v3d_gem.c
index cf3b93101429..175304977daf 100644
--- a/drivers/gpu/drm/v3d/v3d_gem.c
+++ b/drivers/gpu/drm/v3d/v3d_gem.c
@@ -254,10 +254,11 @@ v3d_gem_init(struct drm_device *dev)
queue->fence_context = dma_fence_context_alloc(1);
memset(&queue->stats, 0, sizeof(queue->stats));
seqcount_init(&queue->stats.lock);
+
+ spin_lock_init(&queue->queue_lock);
}
spin_lock_init(&v3d->mm_lock);
- spin_lock_init(&v3d->job_lock);
ret = drmm_mutex_init(dev, &v3d->bo_lock);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/v3d/v3d_irq.c b/drivers/gpu/drm/v3d/v3d_irq.c
index 2464ea4d935d..6831eeccac37 100644
--- a/drivers/gpu/drm/v3d/v3d_irq.c
+++ b/drivers/gpu/drm/v3d/v3d_irq.c
@@ -62,17 +62,17 @@ v3d_overflow_mem_work(struct work_struct *work)
* bin job got scheduled, that's fine. We'll just give them
* some binner pool anyway.
*/
- spin_lock_irqsave(&v3d->job_lock, irqflags);
+ spin_lock_irqsave(&queue->queue_lock, irqflags);
bin_job = (struct v3d_bin_job *)queue->active_job;
if (!bin_job) {
- spin_unlock_irqrestore(&v3d->job_lock, irqflags);
+ spin_unlock_irqrestore(&queue->queue_lock, irqflags);
goto out;
}
drm_gem_object_get(obj);
list_add_tail(&bo->unref_head, &bin_job->render->unref_list);
- spin_unlock_irqrestore(&v3d->job_lock, irqflags);
+ spin_unlock_irqrestore(&queue->queue_lock, irqflags);
v3d_mmu_flush_all(v3d);
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index f2bac920af89..5bc656509ee9 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -204,33 +204,30 @@ static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job)
{
struct v3d_bin_job *job = to_bin_job(sched_job);
struct v3d_dev *v3d = job->base.v3d;
+ struct v3d_queue_state *queue = &v3d->queue[V3D_BIN];
struct drm_device *dev = &v3d->drm;
- struct dma_fence *fence;
+ struct dma_fence *fence = NULL;
unsigned long irqflags;
- if (unlikely(job->base.base.s_fence->finished.error)) {
- spin_lock_irqsave(&v3d->job_lock, irqflags);
- v3d->queue[V3D_BIN].active_job = NULL;
- spin_unlock_irqrestore(&v3d->job_lock, irqflags);
- return NULL;
- }
+ if (unlikely(job->base.base.s_fence->finished.error))
+ goto out_clean_job;
/* Lock required around bin_job update vs
* v3d_overflow_mem_work().
*/
- spin_lock_irqsave(&v3d->job_lock, irqflags);
- v3d->queue[V3D_BIN].active_job = &job->base;
+ spin_lock_irqsave(&queue->queue_lock, irqflags);
+ queue->active_job = &job->base;
/* Clear out the overflow allocation, so we don't
* reuse the overflow attached to a previous job.
*/
V3D_CORE_WRITE(0, V3D_PTB_BPOS, 0);
- spin_unlock_irqrestore(&v3d->job_lock, irqflags);
+ spin_unlock_irqrestore(&queue->queue_lock, irqflags);
v3d_invalidate_caches(v3d);
fence = v3d_fence_create(v3d, V3D_BIN);
if (IS_ERR(fence))
- return NULL;
+ goto out_clean_job;
if (job->base.irq_fence)
dma_fence_put(job->base.irq_fence);
@@ -258,6 +255,12 @@ static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job)
V3D_CORE_WRITE(0, V3D_CLE_CT0QEA, job->end);
return fence;
+
+out_clean_job:
+ spin_lock_irqsave(&queue->queue_lock, irqflags);
+ queue->active_job = NULL;
+ spin_unlock_irqrestore(&queue->queue_lock, irqflags);
+ return fence;
}
static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job)
@@ -265,12 +268,10 @@ static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job)
struct v3d_render_job *job = to_render_job(sched_job);
struct v3d_dev *v3d = job->base.v3d;
struct drm_device *dev = &v3d->drm;
- struct dma_fence *fence;
+ struct dma_fence *fence = NULL;
- if (unlikely(job->base.base.s_fence->finished.error)) {
- v3d->queue[V3D_RENDER].active_job = NULL;
- return NULL;
- }
+ if (unlikely(job->base.base.s_fence->finished.error))
+ goto out_clean_job;
v3d->queue[V3D_RENDER].active_job = &job->base;
@@ -284,7 +285,7 @@ static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job)
fence = v3d_fence_create(v3d, V3D_RENDER);
if (IS_ERR(fence))
- return NULL;
+ goto out_clean_job;
if (job->base.irq_fence)
dma_fence_put(job->base.irq_fence);
@@ -305,6 +306,10 @@ static struct dma_fence *v3d_render_job_run(struct drm_sched_job *sched_job)
V3D_CORE_WRITE(0, V3D_CLE_CT1QEA, job->end);
return fence;
+
+out_clean_job:
+ v3d->queue[V3D_RENDER].active_job = NULL;
+ return fence;
}
static struct dma_fence *
@@ -313,18 +318,16 @@ v3d_tfu_job_run(struct drm_sched_job *sched_job)
struct v3d_tfu_job *job = to_tfu_job(sched_job);
struct v3d_dev *v3d = job->base.v3d;
struct drm_device *dev = &v3d->drm;
- struct dma_fence *fence;
+ struct dma_fence *fence = NULL;
- if (unlikely(job->base.base.s_fence->finished.error)) {
- v3d->queue[V3D_TFU].active_job = NULL;
- return NULL;
- }
+ if (unlikely(job->base.base.s_fence->finished.error))
+ goto out_clean_job;
v3d->queue[V3D_TFU].active_job = &job->base;
fence = v3d_fence_create(v3d, V3D_TFU);
if (IS_ERR(fence))
- return NULL;
+ goto out_clean_job;
if (job->base.irq_fence)
dma_fence_put(job->base.irq_fence);
@@ -352,6 +355,10 @@ v3d_tfu_job_run(struct drm_sched_job *sched_job)
V3D_WRITE(V3D_TFU_ICFG(v3d->ver), job->args.icfg | V3D_TFU_ICFG_IOC);
return fence;
+
+out_clean_job:
+ v3d->queue[V3D_TFU].active_job = NULL;
+ return fence;
}
static struct dma_fence *
@@ -360,13 +367,11 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
struct v3d_csd_job *job = to_csd_job(sched_job);
struct v3d_dev *v3d = job->base.v3d;
struct drm_device *dev = &v3d->drm;
- struct dma_fence *fence;
+ struct dma_fence *fence = NULL;
int i, csd_cfg0_reg;
- if (unlikely(job->base.base.s_fence->finished.error)) {
- v3d->queue[V3D_CSD].active_job = NULL;
- return NULL;
- }
+ if (unlikely(job->base.base.s_fence->finished.error))
+ goto out_clean_job;
/* The HW interprets a workgroup size of 0 as 65536; however, the
* user-space driver exposes a maximum of 65535. Therefore, a 0 in
@@ -384,7 +389,7 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
fence = v3d_fence_create(v3d, V3D_CSD);
if (IS_ERR(fence))
- return NULL;
+ goto out_clean_job;
if (job->base.irq_fence)
dma_fence_put(job->base.irq_fence);
@@ -411,6 +416,10 @@ v3d_csd_job_run(struct drm_sched_job *sched_job)
V3D_CORE_WRITE(0, csd_cfg0_reg, job->args.cfg[0]);
return fence;
+
+out_clean_job:
+ v3d->queue[V3D_CSD].active_job = NULL;
+ return fence;
}
static void
diff --git a/drivers/gpu/drm/xe/tests/xe_migrate.c b/drivers/gpu/drm/xe/tests/xe_migrate.c
index 3bbdb362d6f0..40007f0ddc0d 100644
--- a/drivers/gpu/drm/xe/tests/xe_migrate.c
+++ b/drivers/gpu/drm/xe/tests/xe_migrate.c
@@ -194,8 +194,7 @@ static void xe_migrate_sanity_test(struct xe_migrate *m, struct kunit *test)
err = xe_bo_vmap(bo);
if (err) {
- KUNIT_FAIL(test, "Failed to vmap our pagetables: %li\n",
- PTR_ERR(bo));
+ KUNIT_FAIL(test, "Failed to vmap our pagetables: %d\n", err);
return;
}
diff --git a/drivers/gpu/host1x/debug.c b/drivers/gpu/host1x/debug.c
index a18cc8d8caf5..ec819164e84f 100644
--- a/drivers/gpu/host1x/debug.c
+++ b/drivers/gpu/host1x/debug.c
@@ -31,7 +31,7 @@ void host1x_debug_output(struct output *o, const char *fmt, ...)
int len;
va_start(args, fmt);
- len = vsnprintf(o->buf, sizeof(o->buf), fmt, args);
+ len = vscnprintf(o->buf, sizeof(o->buf), fmt, args);
va_end(args);
o->fn(o->ctx, o->buf, len, false);
@@ -43,7 +43,7 @@ void host1x_debug_cont(struct output *o, const char *fmt, ...)
int len;
va_start(args, fmt);
- len = vsnprintf(o->buf, sizeof(o->buf), fmt, args);
+ len = vscnprintf(o->buf, sizeof(o->buf), fmt, args);
va_end(args);
o->fn(o->ctx, o->buf, len, true);
diff --git a/drivers/gpu/host1x/hw/channel_hw.c b/drivers/gpu/host1x/hw/channel_hw.c
index d44b8de890be..46b6494768b4 100644
--- a/drivers/gpu/host1x/hw/channel_hw.c
+++ b/drivers/gpu/host1x/hw/channel_hw.c
@@ -36,10 +36,9 @@ static void trace_write_gather(struct host1x_cdma *cdma, struct host1x_bo *bo,
for (i = 0; i < words; i += TRACE_MAX_LENGTH) {
u32 num_words = min(words - i, TRACE_MAX_LENGTH);
- offset += i * sizeof(u32);
-
trace_host1x_cdma_push_gather(dev_name(dev), bo,
- num_words, offset,
+ num_words,
+ offset + i * sizeof(u32),
mem);
}
diff --git a/drivers/hid/bpf/hid_bpf_struct_ops.c b/drivers/hid/bpf/hid_bpf_struct_ops.c
index 702c22fae136..049df0647761 100644
--- a/drivers/hid/bpf/hid_bpf_struct_ops.c
+++ b/drivers/hid/bpf/hid_bpf_struct_ops.c
@@ -250,6 +250,11 @@ static void hid_bpf_unreg(void *kdata, struct bpf_link *link)
mutex_lock(&hdev->bpf.prog_list_lock);
+ if (!ops->hdev) {
+ mutex_unlock(&hdev->bpf.prog_list_lock);
+ return;
+ }
+
list_del_rcu(&ops->list);
synchronize_srcu(&hdev->bpf.srcu);
ops->hdev = NULL;
@@ -310,13 +315,17 @@ static struct bpf_struct_ops bpf_hid_bpf_ops = {
void __hid_bpf_ops_destroy_device(struct hid_device *hdev)
{
struct hid_bpf_ops *e;
+ int count = 0;
- rcu_read_lock();
- list_for_each_entry_rcu(e, &hdev->bpf.prog_list, list) {
- hid_put_device(hdev);
+ mutex_lock(&hdev->bpf.prog_list_lock);
+ list_for_each_entry(e, &hdev->bpf.prog_list, list) {
e->hdev = NULL;
+ count++;
}
- rcu_read_unlock();
+ mutex_unlock(&hdev->bpf.prog_list_lock);
+
+ while (count--)
+ hid_put_device(hdev);
}
static int __init hid_bpf_struct_ops_init(void)
diff --git a/drivers/hid/hid-core.c b/drivers/hid/hid-core.c
index d44b43e2438a..df2e43b27388 100644
--- a/drivers/hid/hid-core.c
+++ b/drivers/hid/hid-core.c
@@ -2423,9 +2423,16 @@ EXPORT_SYMBOL_GPL(hid_hw_start);
*
* This is usually called from remove function or from probe when something
* failed and hid_hw_start was called already.
+ *
+ * If the caller enabled HID input via hid_device_io_start() and is unwinding
+ * without an explicit hid_device_io_stop(), quiesce input first so that
+ * in-flight reports cannot reach handlers (e.g. hidraw_report_event) whose
+ * backing objects hid_disconnect() is about to free.
*/
void hid_hw_stop(struct hid_device *hdev)
{
+ if (hdev->io_started)
+ hid_device_io_stop(hdev);
hid_disconnect(hdev);
hdev->ll_driver->stop(hdev);
}
@@ -2773,6 +2780,8 @@ static int __hid_device_probe(struct hid_device *hdev, struct hid_driver *hdrv)
*/
if (ret) {
+ if (hdev->io_started)
+ hid_device_io_stop(hdev);
devres_release_group(&hdev->dev, hdev->devres_group_id);
hid_close_report(hdev);
hdev->driver = NULL;
diff --git a/drivers/hid/hid-lg4ff.c b/drivers/hid/hid-lg4ff.c
index e3fcf1353fb3..71f168eff29b 100644
--- a/drivers/hid/hid-lg4ff.c
+++ b/drivers/hid/hid-lg4ff.c
@@ -336,6 +336,8 @@ int lg4ff_raw_event(struct hid_device *hdev, struct hid_report *report,
if (entry->wdata.combine) {
switch (entry->wdata.product_id) {
case USB_DEVICE_ID_LOGITECH_WHEEL:
+ if (size < 7)
+ return 0;
rd[5] = rd[3];
rd[6] = 0x7F;
return 1;
@@ -343,10 +345,14 @@ int lg4ff_raw_event(struct hid_device *hdev, struct hid_report *report,
case USB_DEVICE_ID_LOGITECH_WINGMAN_FFG:
case USB_DEVICE_ID_LOGITECH_MOMO_WHEEL:
case USB_DEVICE_ID_LOGITECH_MOMO_WHEEL2:
+ if (size < 6)
+ return 0;
rd[4] = rd[3];
rd[5] = 0x7F;
return 1;
case USB_DEVICE_ID_LOGITECH_DFP_WHEEL:
+ if (size < 7)
+ return 0;
rd[5] = rd[4];
rd[6] = 0x7F;
return 1;
@@ -366,6 +372,8 @@ int lg4ff_raw_event(struct hid_device *hdev, struct hid_report *report,
}
/* Compute a combined axis when wheel does not supply it */
+ if (size <= offset + 1)
+ return 0;
rd[offset] = (0xFF + rd[offset] - rd[offset+1]) >> 1;
rd[offset+1] = 0x7F;
return 1;
diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c
index b8e6cc67c6b6..80423adf2c65 100644
--- a/drivers/hid/hid-multitouch.c
+++ b/drivers/hid/hid-multitouch.c
@@ -2426,7 +2426,7 @@ static const struct hid_device_id mt_devices[] = {
HID_ANY_ID) },
/* Hantick */
- { .driver_data = MT_CLS_NSMU,
+ { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU,
HID_DEVICE(BUS_I2C, HID_GROUP_MULTITOUCH_WIN_8,
I2C_VENDOR_ID_HANTICK, I2C_PRODUCT_ID_HANTICK_5288) },
diff --git a/drivers/hid/hid-nintendo.c b/drivers/hid/hid-nintendo.c
index 54a8bdcc0ef7..ea7751b3555c 100644
--- a/drivers/hid/hid-nintendo.c
+++ b/drivers/hid/hid-nintendo.c
@@ -1457,7 +1457,6 @@ static void joycon_parse_imu_report(struct joycon_ctlr *ctlr,
dropped_threshold = ctlr->imu_avg_delta_ms * 3 / 2;
dropped_pkts = (delta - min(delta, dropped_threshold)) /
ctlr->imu_avg_delta_ms;
- ctlr->imu_timestamp_us += 1000 * ctlr->imu_avg_delta_ms;
if (dropped_pkts > JC_IMU_DROPPED_PKT_WARNING) {
hid_warn(ctlr->hdev,
"compensating for %u dropped IMU reports\n",
diff --git a/drivers/hid/hid-rmi.c b/drivers/hid/hid-rmi.c
index d4af17fdba46..d55a0388895f 100644
--- a/drivers/hid/hid-rmi.c
+++ b/drivers/hid/hid-rmi.c
@@ -235,7 +235,23 @@ static int rmi_hid_read_block(struct rmi_transport_dev *xport, u16 addr,
break;
}
- read_input_count = data->readReport[1];
+ read_input_count = min_t(int, data->readReport[1],
+ data->input_report_size - 2);
+ if (!read_input_count) {
+ /*
+ * A zero length reply advances neither
+ * bytes_read nor bytes_needed, and because a
+ * reply did arrive the wait above does not
+ * time out either, so a device answering 0
+ * forever would spin here indefinitely with
+ * page_mutex held.
+ */
+ hid_warn(hdev, "%s: zero-length read reply\n",
+ __func__);
+ clear_bit(RMI_READ_DATA_PENDING, &data->flags);
+ ret = -EIO;
+ break;
+ }
memcpy(buf + bytes_read, &data->readReport[2],
min(read_input_count, bytes_needed));
@@ -271,6 +287,11 @@ static int rmi_hid_write_block(struct rmi_transport_dev *xport, u16 addr,
goto exit;
}
+ if (len + 4 > data->output_report_size) {
+ ret = -EINVAL;
+ goto exit;
+ }
+
data->writeReport[0] = RMI_WRITE_REPORT_ID;
data->writeReport[1] = len;
data->writeReport[2] = addr & 0xFF;
@@ -666,8 +687,16 @@ static int rmi_probe(struct hid_device *hdev, const struct hid_device_id *id)
return ret;
}
- if (id->driver_data)
- data->device_flags = id->driver_data;
+ /*
+ * RMI_DEVICE can only mean "this probe validated the RMI reports and
+ * allocated writeReport": every bail-out to start below skips that
+ * allocation, and device_flags left carrying RMI_DEVICE from
+ * driver_data would send rmi_input_configured() into rmi_set_page()
+ * with writeReport still NULL. A bind through the new_id sysfs
+ * attribute can supply driver_data with the bit set, so do not let
+ * driver_data grant it.
+ */
+ data->device_flags = id->driver_data & ~RMI_DEVICE;
/*
* Check for the RMI specific report ids. If they are misisng
@@ -696,6 +725,17 @@ static int rmi_probe(struct hid_device *hdev, const struct hid_device_id *id)
data->output_report_size = hid_report_len(output_report);
+ /*
+ * The write reports built by this driver occupy 6 bytes and the read
+ * handshake looks at the first 3 bytes of an input report, so refuse
+ * to drive a device whose reports cannot hold them.
+ */
+ if (data->output_report_size < 6 || data->input_report_size < 3) {
+ hid_err(hdev, "rmi reports too small (out=%u in=%u)\n",
+ data->output_report_size, data->input_report_size);
+ goto start;
+ }
+
data->device_flags |= RMI_DEVICE;
alloc_size = data->output_report_size + data->input_report_size;
diff --git a/drivers/hid/hid-roccat-kone.c b/drivers/hid/hid-roccat-kone.c
index 3f8f459edcf3..dd206531fc79 100644
--- a/drivers/hid/hid-roccat-kone.c
+++ b/drivers/hid/hid-roccat-kone.c
@@ -36,6 +36,8 @@ static uint profile_numbers[5] = {0, 1, 2, 3, 4};
static void kone_profile_activated(struct kone_device *kone, uint new_profile)
{
+ if (new_profile < 1 || new_profile > ARRAY_SIZE(kone->profiles))
+ new_profile = 1;
kone->actual_profile = new_profile;
kone->actual_dpi = kone->profiles[new_profile - 1].startup_dpi;
}
@@ -793,8 +795,10 @@ static void kone_keep_values_up_to_date(struct kone_device *kone,
{
switch (event->event) {
case kone_mouse_event_switch_profile:
- kone->actual_dpi = kone->profiles[event->value - 1].
- startup_dpi;
+ if (event->value >= 1 &&
+ event->value <= ARRAY_SIZE(kone->profiles))
+ kone->actual_dpi =
+ kone->profiles[event->value - 1].startup_dpi;
fallthrough;
case kone_mouse_event_osd_profile:
kone->actual_profile = event->value;
diff --git a/drivers/hid/i2c-hid/Makefile b/drivers/hid/i2c-hid/Makefile
index 55bd5e0f35af..38d5d827f3ce 100644
--- a/drivers/hid/i2c-hid/Makefile
+++ b/drivers/hid/i2c-hid/Makefile
@@ -8,7 +8,7 @@ obj-$(CONFIG_I2C_HID_CORE) += i2c-hid.o
i2c-hid-objs = i2c-hid-core.o
i2c-hid-$(CONFIG_DMI) += i2c-hid-dmi-quirks.o
-obj-$(CONFIG_I2C_HID_ACPI) += i2c-hid-acpi.o
+obj-$(CONFIG_I2C_HID_ACPI) += i2c-hid-acpi.o i2c-hid-acpi-prp0001.o
obj-$(CONFIG_I2C_HID_OF) += i2c-hid-of.o
obj-$(CONFIG_I2C_HID_OF_ELAN) += i2c-hid-of-elan.o
obj-$(CONFIG_I2C_HID_OF_GOODIX) += i2c-hid-of-goodix.o
diff --git a/drivers/hid/i2c-hid/i2c-hid-acpi-prp0001.c b/drivers/hid/i2c-hid/i2c-hid-acpi-prp0001.c
new file mode 100644
index 000000000000..d2cf4714ae7f
--- /dev/null
+++ b/drivers/hid/i2c-hid/i2c-hid-acpi-prp0001.c
@@ -0,0 +1,104 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * HID over I2C driver for PRP0001 devices missing hid-descr-addr
+ *
+ * Some devices, for example the Lenovo KaiTian N60d and Inspur CP300L3, use
+ * _HID "PRP0001" with _DSD compatible "hid-over-i2c" but lack "hid-descr-addr"
+ * from the _DSD. The HID descriptor address is provided only through an ACPI
+ * _DSM. The TPD0 node in the DSDT shows _DSM Function 1 returning 0x20.
+ *
+ * Copyright (C) 2026 谢致邦 (XIE Zhibang) <Yeking@Red54.com>
+ */
+
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#include "i2c-hid.h"
+#include "i2c-hid-acpi.h"
+
+static int i2c_hid_acpi_prp0001_power_up(struct i2chid_ops *ops)
+{
+ /* give the device time to power up */
+ msleep(750);
+ return 0;
+}
+
+static struct i2chid_ops i2c_hid_acpi_prp0001_ops = {
+ .power_up = i2c_hid_acpi_prp0001_power_up,
+ /*
+ * No .restore_sequence needed: the _DSM on these devices returns a
+ * constant (0x20) with no side effects, unlike some PNP0C50 _DSM
+ * implementations that switch the hardware between PS/2 and I2C modes.
+ */
+};
+
+static int i2c_hid_acpi_prp0001_probe(struct i2c_client *client)
+{
+ struct device *dev = &client->dev;
+ struct acpi_device *adev;
+ u16 hid_descriptor_address;
+ int ret;
+
+ /* If hid-descr-addr is present, let i2c-hid-of handle it */
+ if (device_property_present(dev, "hid-descr-addr"))
+ return -ENODEV;
+
+ adev = ACPI_COMPANION(dev);
+ if (!adev)
+ return -ENODEV;
+
+ ret = i2c_hid_acpi_get_descriptor(adev);
+ if (ret < 0)
+ return ret;
+ dev_warn(dev,
+ "hid-descr-addr device property NOT found, using ACPI _DSM fallback. Contact vendor for firmware update!\n");
+ hid_descriptor_address = ret;
+
+ /*
+ * No acpi_device_fix_up_power() needed: TPD0 has no _PS0, _PS3, _PSC
+ * or _PRx methods and follows I2C bus power.
+ */
+ return i2c_hid_core_probe(client, &i2c_hid_acpi_prp0001_ops,
+ hid_descriptor_address, 0);
+}
+
+static const struct of_device_id i2c_hid_acpi_prp0001_of_match[] = {
+ { .compatible = "hid-over-i2c" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, i2c_hid_acpi_prp0001_of_match);
+
+static const struct i2c_device_id i2c_hid_acpi_prp0001_id[] = {
+ { .name = "hid-over-i2c" },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, i2c_hid_acpi_prp0001_id);
+
+static struct i2c_driver i2c_hid_acpi_prp0001_driver = {
+ .driver = {
+ .name = "i2c_hid_acpi_prp0001",
+ .pm = &i2c_hid_core_pm,
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+ /*
+ * of_match_ptr() makes this NULL when CONFIG_OF=n, but that's
+ * fine: the I2C id_table with "hid-over-i2c" handles matching
+ * via client->name (set by acpi_set_modalias() from the _DSD
+ * compatible property).
+ */
+ .of_match_table = of_match_ptr(i2c_hid_acpi_prp0001_of_match),
+ },
+
+ .probe = i2c_hid_acpi_prp0001_probe,
+ .remove = i2c_hid_core_remove,
+ .shutdown = i2c_hid_core_shutdown,
+ .id_table = i2c_hid_acpi_prp0001_id,
+};
+
+module_i2c_driver(i2c_hid_acpi_prp0001_driver);
+
+MODULE_DESCRIPTION("HID over I2C driver for PRP0001 devices missing hid-descr-addr");
+MODULE_AUTHOR("谢致邦 (XIE Zhibang) <Yeking@Red54.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/hid/i2c-hid/i2c-hid-acpi.c b/drivers/hid/i2c-hid/i2c-hid-acpi.c
index abd700a101f4..13f977d6aab6 100644
--- a/drivers/hid/i2c-hid/i2c-hid-acpi.c
+++ b/drivers/hid/i2c-hid/i2c-hid-acpi.c
@@ -25,9 +25,9 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pm.h>
-#include <linux/uuid.h>
#include "i2c-hid.h"
+#include "i2c-hid-acpi.h"
struct i2c_hid_acpi {
struct i2chid_ops ops;
@@ -48,39 +48,11 @@ static const struct acpi_device_id i2c_hid_acpi_blacklist[] = {
{ }
};
-/* HID I²C Device: 3cdff6f7-4267-4555-ad05-b30a3d8938de */
-static guid_t i2c_hid_guid =
- GUID_INIT(0x3CDFF6F7, 0x4267, 0x4555,
- 0xAD, 0x05, 0xB3, 0x0A, 0x3D, 0x89, 0x38, 0xDE);
-
-static int i2c_hid_acpi_get_descriptor(struct i2c_hid_acpi *ihid_acpi)
-{
- struct acpi_device *adev = ihid_acpi->adev;
- acpi_handle handle = acpi_device_handle(adev);
- union acpi_object *obj;
- u16 hid_descriptor_address;
-
- if (acpi_match_device_ids(adev, i2c_hid_acpi_blacklist) == 0)
- return -ENODEV;
-
- obj = acpi_evaluate_dsm_typed(handle, &i2c_hid_guid, 1, 1, NULL,
- ACPI_TYPE_INTEGER);
- if (!obj) {
- acpi_handle_err(handle, "Error _DSM call to get HID descriptor address failed\n");
- return -ENODEV;
- }
-
- hid_descriptor_address = obj->integer.value;
- ACPI_FREE(obj);
-
- return hid_descriptor_address;
-}
-
static void i2c_hid_acpi_restore_sequence(struct i2chid_ops *ops)
{
struct i2c_hid_acpi *ihid_acpi = container_of(ops, struct i2c_hid_acpi, ops);
- i2c_hid_acpi_get_descriptor(ihid_acpi);
+ i2c_hid_acpi_get_descriptor(ihid_acpi->adev);
}
static void i2c_hid_acpi_shutdown_tail(struct i2chid_ops *ops)
@@ -93,24 +65,28 @@ static void i2c_hid_acpi_shutdown_tail(struct i2chid_ops *ops)
static int i2c_hid_acpi_probe(struct i2c_client *client)
{
struct device *dev = &client->dev;
+ struct acpi_device *adev = ACPI_COMPANION(dev);
struct i2c_hid_acpi *ihid_acpi;
u16 hid_descriptor_address;
int ret;
- ihid_acpi = devm_kzalloc(&client->dev, sizeof(*ihid_acpi), GFP_KERNEL);
+ if (acpi_match_device_ids(adev, i2c_hid_acpi_blacklist) == 0)
+ return -ENODEV;
+
+ ret = i2c_hid_acpi_get_descriptor(adev);
+ if (ret < 0)
+ return ret;
+ hid_descriptor_address = ret;
+
+ ihid_acpi = devm_kzalloc(dev, sizeof(*ihid_acpi), GFP_KERNEL);
if (!ihid_acpi)
return -ENOMEM;
- ihid_acpi->adev = ACPI_COMPANION(dev);
+ ihid_acpi->adev = adev;
ihid_acpi->ops.shutdown_tail = i2c_hid_acpi_shutdown_tail;
ihid_acpi->ops.restore_sequence = i2c_hid_acpi_restore_sequence;
- ret = i2c_hid_acpi_get_descriptor(ihid_acpi);
- if (ret < 0)
- return ret;
- hid_descriptor_address = ret;
-
- acpi_device_fix_up_power(ihid_acpi->adev);
+ acpi_device_fix_up_power(adev);
return i2c_hid_core_probe(client, &ihid_acpi->ops,
hid_descriptor_address, 0);
diff --git a/drivers/hid/i2c-hid/i2c-hid-acpi.h b/drivers/hid/i2c-hid/i2c-hid-acpi.h
new file mode 100644
index 000000000000..0bbed1853313
--- /dev/null
+++ b/drivers/hid/i2c-hid/i2c-hid-acpi.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+#ifndef _I2C_HID_ACPI_H
+#define _I2C_HID_ACPI_H
+
+#include <linux/acpi.h>
+#include <linux/uuid.h>
+
+static inline int i2c_hid_acpi_get_descriptor(struct acpi_device *adev)
+{
+ /* HID I²C Device: 3cdff6f7-4267-4555-ad05-b30a3d8938de */
+ static const guid_t i2c_hid_guid =
+ GUID_INIT(0x3CDFF6F7, 0x4267, 0x4555,
+ 0xAD, 0x05, 0xB3, 0x0A, 0x3D, 0x89, 0x38, 0xDE);
+
+ acpi_handle handle = acpi_device_handle(adev);
+ union acpi_object *obj;
+ u16 addr;
+
+ obj = acpi_evaluate_dsm_typed(handle, &i2c_hid_guid,
+ 1, 1, NULL, ACPI_TYPE_INTEGER);
+ if (!obj) {
+ acpi_handle_err(handle,
+ "Error _DSM call to get HID descriptor address failed\n");
+ return -ENODEV;
+ }
+
+ addr = obj->integer.value;
+ ACPI_FREE(obj);
+ return addr;
+}
+
+#endif
diff --git a/drivers/hid/i2c-hid/i2c-hid-core.c b/drivers/hid/i2c-hid/i2c-hid-core.c
index 8ce0535fc42d..0e1adfa9b293 100644
--- a/drivers/hid/i2c-hid/i2c-hid-core.c
+++ b/drivers/hid/i2c-hid/i2c-hid-core.c
@@ -784,7 +784,7 @@ static int i2c_hid_parse(struct hid_device *hid)
ihid->hdesc.wReportDescRegister,
rdesc, rsize);
if (ret) {
- hid_err(hid, "reading report descriptor failed\n");
+ dev_err(&client->dev, "reading report descriptor failed\n");
goto out;
}
}
diff --git a/drivers/hid/i2c-hid/i2c-hid-of-goodix.c b/drivers/hid/i2c-hid/i2c-hid-of-goodix.c
index f1597ad67e7c..f4dbcd1d1d47 100644
--- a/drivers/hid/i2c-hid/i2c-hid-of-goodix.c
+++ b/drivers/hid/i2c-hid/i2c-hid-of-goodix.c
@@ -51,8 +51,10 @@ static int goodix_i2c_hid_power_up(struct i2chid_ops *ops)
return ret;
ret = regulator_enable(ihid_goodix->vddio);
- if (ret)
+ if (ret) {
+ regulator_disable(ihid_goodix->vdd);
return ret;
+ }
if (ihid_goodix->timings->post_power_delay_ms)
msleep(ihid_goodix->timings->post_power_delay_ms);
diff --git a/drivers/hid/wacom_wac.c b/drivers/hid/wacom_wac.c
index 5b97df856d3e..ccbf2a6c7a7f 100644
--- a/drivers/hid/wacom_wac.c
+++ b/drivers/hid/wacom_wac.c
@@ -1548,6 +1548,19 @@ static int wacom_intuos_pro2_bt_irq(struct wacom_wac *wacom, size_t len)
return 0;
}
+ if (wacom->features.type == INTUOSP2_BT ||
+ wacom->features.type == INTUOSP2S_BT) {
+ if (len < 286) {
+ dev_warn(wacom->pen_input->dev.parent,
+ "Pro2 BT report too short: %zu bytes\n", len);
+ return 0;
+ }
+ } else if (len < 46) {
+ dev_warn(wacom->pen_input->dev.parent,
+ "Pro2 BT report too short: %zu bytes\n", len);
+ return 0;
+ }
+
wacom_intuos_pro2_bt_pen(wacom);
if (wacom->features.type == INTUOSP2_BT ||
wacom->features.type == INTUOSP2S_BT) {
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 1b9203b20d70..afa481b08c8e 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -656,6 +656,7 @@ static void coretemp_device_remove(int zoneid)
struct platform_data *pdata = platform_get_drvdata(pdev);
ida_destroy(&pdata->ida);
+ kfree(pdata->core_data);
kfree(pdata);
platform_device_unregister(pdev);
}
diff --git a/drivers/hwmon/emc1403.c b/drivers/hwmon/emc1403.c
index eca33220d34a..39f69adb88c2 100644
--- a/drivers/hwmon/emc1403.c
+++ b/drivers/hwmon/emc1403.c
@@ -17,7 +17,6 @@
#include <linux/hwmon-sysfs.h>
#include <linux/err.h>
#include <linux/sysfs.h>
-#include <linux/mutex.h>
#include <linux/regmap.h>
#include <linux/util_macros.h>
@@ -30,7 +29,6 @@ enum emc1403_chip { emc1402, emc1403, emc1404, emc1428 };
struct thermal_data {
enum emc1403_chip chip;
struct regmap *regmap;
- struct mutex mutex;
};
static ssize_t power_state_show(struct device *dev, struct device_attribute *attr, char *buf)
@@ -268,8 +266,8 @@ static s8 emc1403_temp_regs_low[][4] = {
},
};
-static int __emc1403_get_temp(struct thermal_data *data, int channel,
- enum emc1403_reg_map map, long *val)
+static int emc1403_get_temp(struct thermal_data *data, int channel,
+ enum emc1403_reg_map map, long *val)
{
unsigned int regvalh;
unsigned int regvall = 0;
@@ -295,38 +293,22 @@ static int __emc1403_get_temp(struct thermal_data *data, int channel,
return 0;
}
-static int emc1403_get_temp(struct thermal_data *data, int channel,
- enum emc1403_reg_map map, long *val)
-{
- int ret;
-
- mutex_lock(&data->mutex);
- ret = __emc1403_get_temp(data, channel, map, val);
- mutex_unlock(&data->mutex);
-
- return ret;
-}
-
static int emc1403_get_hyst(struct thermal_data *data, int channel,
enum emc1403_reg_map map, long *val)
{
int hyst, ret;
long limit;
- mutex_lock(&data->mutex);
- ret = __emc1403_get_temp(data, channel, map, &limit);
+ ret = emc1403_get_temp(data, channel, map, &limit);
if (ret < 0)
- goto unlock;
+ return ret;
ret = regmap_read(data->regmap, 0x21, &hyst);
if (ret < 0)
- goto unlock;
- if (map == temp_min)
- *val = limit + hyst * 1000;
- else
- *val = limit - hyst * 1000;
-unlock:
- mutex_unlock(&data->mutex);
- return ret;
+ return ret;
+
+ *val = limit - hyst * 1000;
+
+ return 0;
}
static int emc1403_temp_read(struct thermal_data *data, u32 attr, int channel, long *val)
@@ -341,9 +323,6 @@ static int emc1403_temp_read(struct thermal_data *data, u32 attr, int channel, l
case hwmon_temp_input:
ret = emc1403_get_temp(data, channel, ema1403_temp_map[attr], val);
break;
- case hwmon_temp_min_hyst:
- ret = emc1403_get_hyst(data, channel, temp_min, val);
- break;
case hwmon_temp_max_hyst:
ret = emc1403_get_hyst(data, channel, temp_max, val);
break;
@@ -451,20 +430,16 @@ static int emc1403_set_hyst(struct thermal_data *data, long val)
else
val = clamp_val(val, 0, 255000);
- mutex_lock(&data->mutex);
- ret = __emc1403_get_temp(data, 0, temp_crit, &limit);
+ ret = emc1403_get_temp(data, 0, temp_crit, &limit);
if (ret < 0)
- goto unlock;
+ return ret;
hyst = limit - val;
if (data->chip == emc1428)
hyst = clamp_val(DIV_ROUND_CLOSEST(hyst, 1000), 0, 127);
else
hyst = clamp_val(DIV_ROUND_CLOSEST(hyst, 1000), 0, 255);
- ret = regmap_write(data->regmap, 0x21, hyst);
-unlock:
- mutex_unlock(&data->mutex);
- return ret;
+ return regmap_write(data->regmap, 0x21, hyst);
}
static int emc1403_set_temp(struct thermal_data *data, int channel,
@@ -478,7 +453,6 @@ static int emc1403_set_temp(struct thermal_data *data, int channel,
regh = emc1403_temp_regs[channel][map];
regl = emc1403_temp_regs_low[channel][map];
- mutex_lock(&data->mutex);
if (regl >= 0) {
if (data->chip == emc1428)
val = clamp_val(val, -128000, 127875);
@@ -487,7 +461,7 @@ static int emc1403_set_temp(struct thermal_data *data, int channel,
regval = DIV_ROUND_CLOSEST(val, 125);
ret = regmap_write(data->regmap, regh, (regval >> 3) & 0xff);
if (ret < 0)
- goto unlock;
+ return ret;
ret = regmap_write(data->regmap, regl, (regval & 0x07) << 5);
} else {
if (data->chip == emc1428)
@@ -497,8 +471,6 @@ static int emc1403_set_temp(struct thermal_data *data, int channel,
regval = DIV_ROUND_CLOSEST(val, 1000);
ret = regmap_write(data->regmap, regh, regval);
}
-unlock:
- mutex_unlock(&data->mutex);
return ret;
}
@@ -572,7 +544,6 @@ static umode_t emc1403_temp_is_visible(const void *_data, u32 attr, int channel)
case hwmon_temp_max_alarm:
case hwmon_temp_crit_alarm:
case hwmon_temp_fault:
- case hwmon_temp_min_hyst:
case hwmon_temp_max_hyst:
return 0444;
case hwmon_temp_min:
@@ -615,35 +586,35 @@ static const struct hwmon_channel_info * const emc1403_info[] = {
HWMON_CHANNEL_INFO(chip, HWMON_C_UPDATE_INTERVAL),
HWMON_CHANNEL_INFO(temp,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT,
HWMON_T_INPUT | HWMON_T_MIN | HWMON_T_MAX |
- HWMON_T_CRIT | HWMON_T_MIN_HYST | HWMON_T_MAX_HYST |
+ HWMON_T_CRIT | HWMON_T_MAX_HYST |
HWMON_T_CRIT_HYST | HWMON_T_MIN_ALARM |
HWMON_T_MAX_ALARM | HWMON_T_CRIT_ALARM | HWMON_T_FAULT
),
@@ -695,8 +666,6 @@ static int emc1403_probe(struct i2c_client *client)
if (IS_ERR(data->regmap))
return PTR_ERR(data->regmap);
- mutex_init(&data->mutex);
-
hwmon_dev = devm_hwmon_device_register_with_info(&client->dev,
client->name, data,
&emc1403_chip_info,
diff --git a/drivers/hwspinlock/hwspinlock_core.c b/drivers/hwspinlock/hwspinlock_core.c
index 6505261e6068..b27a2e33a611 100644
--- a/drivers/hwspinlock/hwspinlock_core.c
+++ b/drivers/hwspinlock/hwspinlock_core.c
@@ -472,7 +472,7 @@ static int hwspin_lock_register_single(struct hwspinlock *hwlock, int id)
out:
mutex_unlock(&hwspinlock_tree_lock);
- return 0;
+ return ret;
}
static struct hwspinlock *hwspin_lock_unregister_single(unsigned int id)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
index c302072b293a..e1a59b434505 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-cfg.c
@@ -76,7 +76,7 @@ static int etm4_cfg_map_reg_offset(struct etmv4_drvdata *drvdata,
} else if ((offset & GENMASK(11, 4)) == TRCSEQEVRn(0)) {
/* sequencer state control registers */
idx = (offset & GENMASK(3, 0)) / 4;
- if (idx < ETM_MAX_SEQ_STATES) {
+ if (idx < ETM_MAX_SEQ_TRANSITIONS) {
reg_csdev->driver_regval = &drvcfg->seq_ctrl[idx];
err = 0;
}
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
index 05958827b3bb..312abd57ddce 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
@@ -87,7 +87,7 @@ static int etm4_probe_cpu(unsigned int cpu);
static inline bool etm4x_sspcicrn_present(struct etmv4_drvdata *drvdata, int n)
{
return (n < drvdata->nr_ss_cmp) &&
- drvdata->nr_pe &&
+ drvdata->nr_pe_cmp &&
(drvdata->config.ss_status[n] & TRCSSCSRn_PC);
}
@@ -527,7 +527,8 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
etm4x_relaxed_write32(csa, config->vissctlr, TRCVISSCTLR);
if (drvdata->nr_pe_cmp)
etm4x_relaxed_write32(csa, config->vipcssctlr, TRCVIPCSSCTLR);
- for (i = 0; i < drvdata->nrseqstate - 1; i++)
+
+ for (i = 0; i < drvdata->nr_seq_ctrls; i++)
etm4x_relaxed_write32(csa, config->seq_ctrl[i], TRCSEQEVRn(i));
if (drvdata->nrseqstate) {
etm4x_relaxed_write32(csa, config->seq_rst, TRCSEQRSTEVR);
@@ -853,8 +854,10 @@ static int etm4_enable_sysfs(struct coresight_device *csdev)
cscfg_config_sysfs_get_active_cfg(&cfg_hash, &preset);
if (cfg_hash) {
ret = cscfg_csdev_enable_active_config(csdev, cfg_hash, preset);
- if (ret)
+ if (ret) {
+ etm4_release_trace_id(drvdata);
return ret;
+ }
}
spin_lock(&drvdata->spinlock);
@@ -1415,6 +1418,8 @@ static void etm4_init_arch_data(void *info)
drvdata->lpoverride = (etmidr5 & TRCIDR5_LPOVERRIDE) && (!drvdata->skip_power_up);
/* NUMSEQSTATE, bits[27:25] number of sequencer states implemented */
drvdata->nrseqstate = FIELD_GET(TRCIDR5_NUMSEQSTATE_MASK, etmidr5);
+ if (drvdata->nrseqstate)
+ drvdata->nr_seq_ctrls = ETM_MAX_SEQ_TRANSITIONS;
/* NUMCNTR, bits[30:28] number of counters available for tracing */
drvdata->nr_cntr = FIELD_GET(TRCIDR5_NUMCNTR_MASK, etmidr5);
etm4_cs_lock(drvdata, csa);
@@ -1824,7 +1829,7 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata)
if (drvdata->nr_pe_cmp)
state->trcvipcssctlr = etm4x_read32(csa, TRCVIPCSSCTLR);
- for (i = 0; i < drvdata->nrseqstate - 1; i++)
+ for (i = 0; i < drvdata->nr_seq_ctrls; i++)
state->trcseqevr[i] = etm4x_read32(csa, TRCSEQEVRn(i));
if (drvdata->nrseqstate) {
@@ -1958,7 +1963,7 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata)
if (drvdata->nr_pe_cmp)
etm4x_relaxed_write32(csa, state->trcvipcssctlr, TRCVIPCSSCTLR);
- for (i = 0; i < drvdata->nrseqstate - 1; i++)
+ for (i = 0; i < drvdata->nr_seq_ctrls; i++)
etm4x_relaxed_write32(csa, state->trcseqevr[i], TRCSEQEVRn(i));
if (drvdata->nrseqstate) {
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
index a9f19629f3f8..2a96a4c43a63 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
@@ -221,7 +221,7 @@ static ssize_t reset_store(struct device *dev,
config->vipcssctlr = 0x0;
/* Disable seq events */
- for (i = 0; i < drvdata->nrseqstate-1; i++)
+ for (i = 0; i < drvdata->nr_seq_ctrls; i++)
config->seq_ctrl[i] = 0x0;
config->seq_rst = 0x0;
config->seq_state = 0x0;
@@ -1393,9 +1393,11 @@ static ssize_t seq_idx_store(struct device *dev,
struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
struct etmv4_config *config = &drvdata->config;
+ if (!drvdata->nr_seq_ctrls)
+ return -ENOTSUPP;
if (kstrtoul(buf, 16, &val))
return -EINVAL;
- if (val >= drvdata->nrseqstate - 1)
+ if (val >= drvdata->nr_seq_ctrls)
return -EINVAL;
/*
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h
index 3683966bd060..e8fe685a7601 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x.h
+++ b/drivers/hwtracing/coresight/coresight-etm4x.h
@@ -570,6 +570,7 @@
#define ETM_MAX_NR_PE 8
#define ETMv4_MAX_CNTR 4
#define ETM_MAX_SEQ_STATES 4
+#define ETM_MAX_SEQ_TRANSITIONS 3
#define ETM_MAX_EXT_INP_SEL 4
#define ETM_MAX_EXT_INP 256
#define ETM_MAX_EXT_OUT 4
@@ -825,7 +826,6 @@ struct etmv4_config {
u32 eventctrl1;
u32 stall_ctrl;
u32 ts_ctrl;
- u32 syncfreq;
u32 ccctlr;
u32 bb_ctrl;
u32 vinst_ctrl;
@@ -833,7 +833,8 @@ struct etmv4_config {
u32 vissctlr;
u32 vipcssctlr;
u8 seq_idx;
- u32 seq_ctrl[ETM_MAX_SEQ_STATES];
+ u8 syncfreq;
+ u32 seq_ctrl[ETM_MAX_SEQ_TRANSITIONS];
u32 seq_rst;
u32 seq_state;
u8 cntr_idx;
@@ -885,7 +886,7 @@ struct etmv4_save_state {
u32 trcvissctlr;
u32 trcvipcssctlr;
- u32 trcseqevr[ETM_MAX_SEQ_STATES];
+ u32 trcseqevr[ETM_MAX_SEQ_TRANSITIONS];
u32 trcseqrstevr;
u32 trcseqstr;
u32 trcextinselr;
@@ -938,6 +939,7 @@ struct etmv4_save_state {
* @numcidc: Number of contextID comparators.
* @numvmidc: Number of VMID comparators.
* @nrseqstate: The number of sequencer states that are implemented.
+ * @nr_seq_ctrls: The number of sequence state transition control registers.
* @nr_event: Indicates how many events the trace unit support.
* @nr_resource:The number of resource selection pairs available for tracing.
* @nr_ss_cmp: Number of single-shot comparator controls that are available.
@@ -1004,6 +1006,7 @@ struct etmv4_drvdata {
u8 numextinsel;
u8 numvmidc;
u8 nrseqstate;
+ u8 nr_seq_ctrls;
u8 nr_event;
u8 nr_resource;
u8 nr_ss_cmp;
diff --git a/drivers/i2c/busses/i2c-ocores.c b/drivers/i2c/busses/i2c-ocores.c
index 482b37c8a129..d62a664a94d5 100644
--- a/drivers/i2c/busses/i2c-ocores.c
+++ b/drivers/i2c/busses/i2c-ocores.c
@@ -761,7 +761,11 @@ static int ocores_i2c_resume(struct device *dev)
rate = clk_get_rate(i2c->clk) / 1000;
if (rate)
i2c->ip_clock_khz = rate;
- return ocores_init(dev, i2c);
+ ret = ocores_init(dev, i2c);
+ if (ret)
+ clk_disable_unprepare(i2c->clk);
+
+ return ret;
}
static DEFINE_NOIRQ_DEV_PM_OPS(ocores_i2c_pm,
diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c
index 7cc8ca5671c4..bdab2151e617 100644
--- a/drivers/i2c/i2c-core-base.c
+++ b/drivers/i2c/i2c-core-base.c
@@ -1805,6 +1805,12 @@ void i2c_del_adapter(struct i2c_adapter *adap)
return;
}
+ /*
+ * This drains any in-flight writers, so all
+ * clients will be caught by i2c_deregister_clients().
+ */
+ device_remove_file(&adap->dev, &dev_attr_new_device);
+
i2c_acpi_remove_space_handler(adap);
i2c_deregister_clients(adap);
diff --git a/drivers/i2c/i2c-mux.c b/drivers/i2c/i2c-mux.c
index fda72e8be885..8604b288c676 100644
--- a/drivers/i2c/i2c-mux.c
+++ b/drivers/i2c/i2c-mux.c
@@ -411,6 +411,7 @@ int i2c_mux_add_adapter(struct i2c_mux_core *muxc,
return 0;
err_free_priv:
+ of_node_put(priv->adap.dev.of_node);
kfree(priv);
return ret;
}
diff --git a/drivers/i2c/muxes/i2c-demux-pinctrl.c b/drivers/i2c/muxes/i2c-demux-pinctrl.c
index cec7f3447e19..ff4dd0063afc 100644
--- a/drivers/i2c/muxes/i2c-demux-pinctrl.c
+++ b/drivers/i2c/muxes/i2c-demux-pinctrl.c
@@ -247,6 +247,7 @@ static int i2c_demux_pinctrl_probe(struct platform_device *pdev)
props[i].value = devm_kstrdup(&pdev->dev, "ok", GFP_KERNEL);
if (!props[i].name || !props[i].value) {
err = -ENOMEM;
+ of_node_put(adap_np);
goto err_rollback;
}
props[i].length = 3;
diff --git a/drivers/i3c/master.c b/drivers/i3c/master.c
index 6eec1e6fb139..48a91eac5ae5 100644
--- a/drivers/i3c/master.c
+++ b/drivers/i3c/master.c
@@ -1689,7 +1689,9 @@ i3c_master_register_new_i3c_devs(struct i3c_master_controller *master)
if (ret) {
dev_err(&master->dev,
"Failed to add I3C device (err = %d)\n", ret);
+ desc->dev->desc = NULL;
put_device(&desc->dev->dev);
+ desc->dev = NULL;
}
}
}
diff --git a/drivers/i3c/master/dw-i3c-master.c b/drivers/i3c/master/dw-i3c-master.c
index a60eb86bddba..c68637ddd05e 100644
--- a/drivers/i3c/master/dw-i3c-master.c
+++ b/drivers/i3c/master/dw-i3c-master.c
@@ -889,7 +889,15 @@ static int dw_i3c_master_daa(struct i3c_master_controller *m)
if (!wait_for_completion_timeout(&xfer->comp, XFER_TIMEOUT))
dw_i3c_master_dequeue_xfer(master, xfer);
- newdevs = GENMASK(master->maxdevs - cmd->rx_len - 1, 0);
+ /*
+ * cmd->rx_len holds the number of addresses ENTDAA left unassigned.
+ * On an empty bus rx_len == maxdevs, so avoid GENMASK(-1, 0).
+ */
+ if (cmd->rx_len >= master->maxdevs)
+ newdevs = 0;
+ else
+ newdevs = GENMASK(master->maxdevs - cmd->rx_len - 1, 0);
+
newdevs &= ~olddevs;
for (pos = 0; pos < master->maxdevs; pos++) {
diff --git a/drivers/i3c/master/mipi-i3c-hci/core.c b/drivers/i3c/master/mipi-i3c-hci/core.c
index 0fdd74704b4e..51780354be3b 100644
--- a/drivers/i3c/master/mipi-i3c-hci/core.c
+++ b/drivers/i3c/master/mipi-i3c-hci/core.c
@@ -663,8 +663,8 @@ static int i3c_hci_init(struct i3c_hci *hci)
hci->DAT_entry_size = FIELD_GET(DAT_ENTRY_SIZE, regval) ? 0 : 8;
if (size_in_dwords)
hci->DAT_entries = 4 * hci->DAT_entries / hci->DAT_entry_size;
- dev_info(&hci->master.dev, "DAT: %u %u-bytes entries at offset %#x\n",
- hci->DAT_entries, hci->DAT_entry_size, offset);
+ dev_dbg(&hci->master.dev, "DAT: %u %u-bytes entries at offset %#x\n",
+ hci->DAT_entries, hci->DAT_entry_size, offset);
regval = reg_read(DCT_SECTION);
offset = FIELD_GET(DCT_TABLE_OFFSET, regval);
@@ -673,23 +673,23 @@ static int i3c_hci_init(struct i3c_hci *hci)
hci->DCT_entry_size = FIELD_GET(DCT_ENTRY_SIZE, regval) ? 0 : 16;
if (size_in_dwords)
hci->DCT_entries = 4 * hci->DCT_entries / hci->DCT_entry_size;
- dev_info(&hci->master.dev, "DCT: %u %u-bytes entries at offset %#x\n",
- hci->DCT_entries, hci->DCT_entry_size, offset);
+ dev_dbg(&hci->master.dev, "DCT: %u %u-bytes entries at offset %#x\n",
+ hci->DCT_entries, hci->DCT_entry_size, offset);
regval = reg_read(RING_HEADERS_SECTION);
offset = FIELD_GET(RING_HEADERS_OFFSET, regval);
hci->RHS_regs = offset ? hci->base_regs + offset : NULL;
- dev_info(&hci->master.dev, "Ring Headers at offset %#x\n", offset);
+ dev_dbg(&hci->master.dev, "Ring Headers at offset %#x\n", offset);
regval = reg_read(PIO_SECTION);
offset = FIELD_GET(PIO_REGS_OFFSET, regval);
hci->PIO_regs = offset ? hci->base_regs + offset : NULL;
- dev_info(&hci->master.dev, "PIO section at offset %#x\n", offset);
+ dev_dbg(&hci->master.dev, "PIO section at offset %#x\n", offset);
regval = reg_read(EXT_CAPS_SECTION);
offset = FIELD_GET(EXT_CAPS_OFFSET, regval);
hci->EXTCAPS_regs = offset ? hci->base_regs + offset : NULL;
- dev_info(&hci->master.dev, "Extended Caps at offset %#x\n", offset);
+ dev_dbg(&hci->master.dev, "Extended Caps at offset %#x\n", offset);
ret = i3c_hci_parse_ext_caps(hci);
if (ret)
@@ -766,7 +766,7 @@ static int i3c_hci_init(struct i3c_hci *hci)
ret = -EIO;
} else {
hci->io = &mipi_i3c_hci_dma;
- dev_info(&hci->master.dev, "Using DMA\n");
+ dev_dbg(&hci->master.dev, "Using DMA\n");
}
}
@@ -778,7 +778,7 @@ static int i3c_hci_init(struct i3c_hci *hci)
ret = -EIO;
} else {
hci->io = &mipi_i3c_hci_pio;
- dev_info(&hci->master.dev, "Using PIO\n");
+ dev_dbg(&hci->master.dev, "Using PIO\n");
}
}
diff --git a/drivers/i3c/master/mipi-i3c-hci/dma.c b/drivers/i3c/master/mipi-i3c-hci/dma.c
index b9496e8c4784..09bc109c4ab1 100644
--- a/drivers/i3c/master/mipi-i3c-hci/dma.c
+++ b/drivers/i3c/master/mipi-i3c-hci/dma.c
@@ -201,7 +201,7 @@ static int hci_dma_init(struct i3c_hci *hci)
regval = rhs_reg_read(CONTROL);
nr_rings = FIELD_GET(MAX_HEADER_COUNT_CAP, regval);
- dev_info(&hci->master.dev, "%d DMA rings available\n", nr_rings);
+ dev_dbg(&hci->master.dev, "%d DMA rings available\n", nr_rings);
if (unlikely(nr_rings > 8)) {
dev_err(&hci->master.dev, "number of rings should be <= 8\n");
nr_rings = 8;
@@ -220,7 +220,7 @@ static int hci_dma_init(struct i3c_hci *hci)
for (i = 0; i < rings->total; i++) {
u32 offset = rhs_reg_read(RHn_OFFSET(i));
- dev_info(&hci->master.dev, "Ring %d at offset %#x\n", i, offset);
+ dev_dbg(&hci->master.dev, "Ring %d at offset %#x\n", i, offset);
ret = -EINVAL;
if (!offset)
goto err_out;
diff --git a/drivers/i3c/master/mipi-i3c-hci/ext_caps.c b/drivers/i3c/master/mipi-i3c-hci/ext_caps.c
index 2e9b23efdc45..4e5519628178 100644
--- a/drivers/i3c/master/mipi-i3c-hci/ext_caps.c
+++ b/drivers/i3c/master/mipi-i3c-hci/ext_caps.c
@@ -27,9 +27,9 @@ static int hci_extcap_hardware_id(struct i3c_hci *hci, void __iomem *base)
hci->vendor_version_id = readl(base + 0x08);
hci->vendor_product_id = readl(base + 0x0c);
- dev_info(&hci->master.dev, "vendor MIPI ID: %#x\n", hci->vendor_mipi_id);
- dev_info(&hci->master.dev, "vendor version ID: %#x\n", hci->vendor_version_id);
- dev_info(&hci->master.dev, "vendor product ID: %#x\n", hci->vendor_product_id);
+ dev_dbg(&hci->master.dev, "vendor MIPI ID: %#x\n", hci->vendor_mipi_id);
+ dev_dbg(&hci->master.dev, "vendor version ID: %#x\n", hci->vendor_version_id);
+ dev_dbg(&hci->master.dev, "vendor product ID: %#x\n", hci->vendor_product_id);
/* ought to go in a table if this grows too much */
switch (hci->vendor_mipi_id) {
@@ -49,7 +49,7 @@ static int hci_extcap_master_config(struct i3c_hci *hci, void __iomem *base)
static const char * const functionality[] = {
"(unknown)", "master only", "target only",
"primary/secondary master" };
- dev_info(&hci->master.dev, "operation mode: %s\n", functionality[operation_mode]);
+ dev_dbg(&hci->master.dev, "operation mode: %s\n", functionality[operation_mode]);
if (operation_mode & 0x1)
return 0;
dev_err(&hci->master.dev, "only master mode is currently supported\n");
@@ -61,7 +61,7 @@ static int hci_extcap_multi_bus(struct i3c_hci *hci, void __iomem *base)
u32 bus_instance = readl(base + 0x04);
unsigned int count = FIELD_GET(GENMASK(3, 0), bus_instance);
- dev_info(&hci->master.dev, "%d bus instances\n", count);
+ dev_dbg(&hci->master.dev, "%d bus instances\n", count);
return 0;
}
@@ -71,8 +71,7 @@ static int hci_extcap_xfer_modes(struct i3c_hci *hci, void __iomem *base)
u32 entries = FIELD_GET(CAP_HEADER_LENGTH, header) - 1;
unsigned int index;
- dev_info(&hci->master.dev, "transfer mode table has %d entries\n",
- entries);
+ dev_dbg(&hci->master.dev, "transfer mode table has %d entries\n", entries);
base += 4; /* skip header */
for (index = 0; index < entries; index++) {
u32 mode_entry = readl(base);
@@ -94,19 +93,19 @@ static int hci_extcap_xfer_rates(struct i3c_hci *hci, void __iomem *base)
base += 4; /* skip header */
- dev_info(&hci->master.dev, "available data rates:\n");
+ dev_dbg(&hci->master.dev, "available data rates:\n");
for (index = 0; index < entries; index++) {
rate_entry = readl(base);
DBG("entry %d: 0x%08x", index, rate_entry);
rate = FIELD_GET(XFERRATE_ACTUAL_RATE_KHZ, rate_entry);
rate_id = FIELD_GET(XFERRATE_RATE_ID, rate_entry);
mode_id = FIELD_GET(XFERRATE_MODE_ID, rate_entry);
- dev_info(&hci->master.dev, "rate %d for %s = %d kHz\n",
- rate_id,
- mode_id == XFERRATE_MODE_I3C ? "I3C" :
- mode_id == XFERRATE_MODE_I2C ? "I2C" :
- "unknown mode",
- rate);
+ dev_dbg(&hci->master.dev, "rate %d for %s = %d kHz\n",
+ rate_id,
+ mode_id == XFERRATE_MODE_I3C ? "I3C" :
+ mode_id == XFERRATE_MODE_I2C ? "I2C" :
+ "unknown mode",
+ rate);
base += 4;
}
@@ -120,8 +119,8 @@ static int hci_extcap_auto_command(struct i3c_hci *hci, void __iomem *base)
u32 autocmd_ext_config = readl(base + 0x08);
unsigned int count = FIELD_GET(GENMASK(3, 0), autocmd_ext_config);
- dev_info(&hci->master.dev, "%d/%d active auto-command entries\n",
- count, max_count);
+ dev_dbg(&hci->master.dev, "%d/%d active auto-command entries\n",
+ count, max_count);
/* remember auto-command register location for later use */
hci->AUTOCMD_regs = base;
return 0;
@@ -129,46 +128,46 @@ static int hci_extcap_auto_command(struct i3c_hci *hci, void __iomem *base)
static int hci_extcap_debug(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "debug registers present\n");
+ dev_dbg(&hci->master.dev, "debug registers present\n");
hci->DEBUG_regs = base;
return 0;
}
static int hci_extcap_scheduled_cmd(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "scheduled commands available\n");
+ dev_dbg(&hci->master.dev, "scheduled commands available\n");
/* hci->schedcmd_regs = base; */
return 0;
}
static int hci_extcap_non_curr_master(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "Non-Current Master support available\n");
+ dev_dbg(&hci->master.dev, "Non-Current Master support available\n");
/* hci->NCM_regs = base; */
return 0;
}
static int hci_extcap_ccc_resp_conf(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "CCC Response Configuration available\n");
+ dev_dbg(&hci->master.dev, "CCC Response Configuration available\n");
return 0;
}
static int hci_extcap_global_DAT(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "Global DAT available\n");
+ dev_dbg(&hci->master.dev, "Global DAT available\n");
return 0;
}
static int hci_extcap_multilane(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "Master Multi-Lane support available\n");
+ dev_dbg(&hci->master.dev, "Master Multi-Lane support available\n");
return 0;
}
static int hci_extcap_ncm_multilane(struct i3c_hci *hci, void __iomem *base)
{
- dev_info(&hci->master.dev, "NCM Multi-Lane support available\n");
+ dev_dbg(&hci->master.dev, "NCM Multi-Lane support available\n");
return 0;
}
@@ -201,7 +200,7 @@ static const struct hci_ext_caps ext_capabilities[] = {
static int hci_extcap_vendor_NXP(struct i3c_hci *hci, void __iomem *base)
{
hci->vendor_data = (__force void *)base;
- dev_info(&hci->master.dev, "Build Date Info = %#x\n", readl(base + 1*4));
+ dev_dbg(&hci->master.dev, "Build Date Info = %#x\n", readl(base + 1 * 4));
/* reset the FPGA */
writel(0xdeadbeef, base + 1*4);
return 0;
@@ -239,9 +238,8 @@ static int hci_extcap_vendor_specific(struct i3c_hci *hci, void __iomem *base,
}
if (!vendor_cap_entry) {
- dev_notice(&hci->master.dev,
- "unknown ext_cap 0x%02x for vendor 0x%02x\n",
- cap_id, hci->vendor_mipi_id);
+ dev_dbg(&hci->master.dev, "unknown ext_cap 0x%02x for vendor 0x%02x\n",
+ cap_id, hci->vendor_mipi_id);
return 0;
}
if (cap_length < vendor_cap_entry->min_length) {
@@ -293,8 +291,7 @@ int i3c_hci_parse_ext_caps(struct i3c_hci *hci)
}
}
if (!cap_entry) {
- dev_notice(&hci->master.dev,
- "unknown ext_cap 0x%02x\n", cap_id);
+ dev_dbg(&hci->master.dev, "unknown ext_cap 0x%02x\n", cap_id);
} else if (cap_length < cap_entry->min_length) {
dev_err(&hci->master.dev,
"ext_cap 0x%02x has size %d (expecting >= %d)\n",
diff --git a/drivers/i3c/master/mipi-i3c-hci/pio.c b/drivers/i3c/master/mipi-i3c-hci/pio.c
index d0272aa93599..d8ef51cea6e2 100644
--- a/drivers/i3c/master/mipi-i3c-hci/pio.c
+++ b/drivers/i3c/master/mipi-i3c-hci/pio.c
@@ -136,27 +136,14 @@ struct hci_pio_data {
u32 enabled_irqs;
};
-static int hci_pio_init(struct i3c_hci *hci)
+static void __hci_pio_init(struct i3c_hci *hci, u32 *size_val_ptr)
{
- struct hci_pio_data *pio;
u32 val, size_val, rx_thresh, tx_thresh, ibi_val;
-
- pio = kzalloc(sizeof(*pio), GFP_KERNEL);
- if (!pio)
- return -ENOMEM;
-
- hci->io_data = pio;
- spin_lock_init(&pio->lock);
+ struct hci_pio_data *pio = hci->io_data;
size_val = pio_reg_read(QUEUE_SIZE);
- dev_info(&hci->master.dev, "CMD/RESP FIFO = %ld entries\n",
- FIELD_GET(CR_QUEUE_SIZE, size_val));
- dev_info(&hci->master.dev, "IBI FIFO = %ld bytes\n",
- 4 * FIELD_GET(IBI_STATUS_SIZE, size_val));
- dev_info(&hci->master.dev, "RX data FIFO = %d bytes\n",
- 4 * (2 << FIELD_GET(RX_DATA_BUFFER_SIZE, size_val)));
- dev_info(&hci->master.dev, "TX data FIFO = %d bytes\n",
- 4 * (2 << FIELD_GET(TX_DATA_BUFFER_SIZE, size_val)));
+ if (size_val_ptr)
+ *size_val_ptr = size_val;
/*
* Let's initialize data thresholds to half of the actual FIFO size.
@@ -200,8 +187,35 @@ static int hci_pio_init(struct i3c_hci *hci)
pio_reg_write(INTR_SIGNAL_ENABLE, 0x0);
pio_reg_write(INTR_STATUS_ENABLE, 0xffffffff);
- /* Always accept error interrupts (will be activated on first xfer) */
- pio->enabled_irqs = STAT_ALL_ERRORS;
+ /*
+ * Always accept error interrupts and IBI threshold interrupt
+ * (will be activated on first xfer).
+ */
+ pio->enabled_irqs = STAT_ALL_ERRORS | STAT_IBI_STATUS_THLD;
+}
+
+static int hci_pio_init(struct i3c_hci *hci)
+{
+ struct hci_pio_data *pio;
+ u32 size_val;
+
+ pio = devm_kzalloc(hci->master.dev.parent, sizeof(*pio), GFP_KERNEL);
+ if (!pio)
+ return -ENOMEM;
+
+ hci->io_data = pio;
+ spin_lock_init(&pio->lock);
+
+ __hci_pio_init(hci, &size_val);
+
+ dev_dbg(&hci->master.dev, "CMD/RESP FIFO = %ld entries\n",
+ FIELD_GET(CR_QUEUE_SIZE, size_val));
+ dev_dbg(&hci->master.dev, "IBI FIFO = %ld bytes\n",
+ 4 * FIELD_GET(IBI_STATUS_SIZE, size_val));
+ dev_dbg(&hci->master.dev, "RX data FIFO = %d bytes\n",
+ 4 * (2 << FIELD_GET(RX_DATA_BUFFER_SIZE, size_val)));
+ dev_dbg(&hci->master.dev, "TX data FIFO = %d bytes\n",
+ 4 * (2 << FIELD_GET(TX_DATA_BUFFER_SIZE, size_val)));
return 0;
}
@@ -219,8 +233,6 @@ static void hci_pio_cleanup(struct i3c_hci *hci)
BUG_ON(pio->curr_rx);
BUG_ON(pio->curr_tx);
BUG_ON(pio->curr_resp);
- kfree(pio);
- hci->io_data = NULL;
}
}
diff --git a/drivers/iio/accel/dmard09.c b/drivers/iio/accel/dmard09.c
index 4ec70ca6910d..117cebbeb138 100644
--- a/drivers/iio/accel/dmard09.c
+++ b/drivers/iio/accel/dmard09.c
@@ -8,6 +8,7 @@
#include <linux/unaligned.h>
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/units.h>
#include <linux/iio/iio.h>
#define DMARD09_DRV_NAME "dmard09"
@@ -79,6 +80,12 @@ static int dmard09_read_raw(struct iio_dev *indio_dev,
*val = accel;
return IIO_VAL_INT;
+ case IIO_CHAN_INFO_SCALE:
+ *val = 0;
+ /* 1 g / 32 LSB, in m/s^2 */
+ *val2 = IIO_G_TO_M_S_2(NANO / 32);
+
+ return IIO_VAL_INT_PLUS_NANO;
default:
return -EINVAL;
}
diff --git a/drivers/iio/adc/Kconfig b/drivers/iio/adc/Kconfig
index 57fc8f0de7b0..7484dd98fa1e 100644
--- a/drivers/iio/adc/Kconfig
+++ b/drivers/iio/adc/Kconfig
@@ -841,6 +841,7 @@ config MAX1363
config MAX34408
tristate "Maxim max34408/max344089 ADC driver"
depends on I2C
+ select REGMAP_I2C
help
Say yes here to build support for Maxim max34408/max34409 current sense
monitor with 8-bits ADC interface with overcurrent delay/threshold and
diff --git a/drivers/iio/adc/pac1921.c b/drivers/iio/adc/pac1921.c
index fe1d9e07fce2..5b33e0543a86 100644
--- a/drivers/iio/adc/pac1921.c
+++ b/drivers/iio/adc/pac1921.c
@@ -1020,7 +1020,7 @@ static irqreturn_t pac1921_trigger_handler(int irq, void *p)
iio_for_each_active_channel(idev, bit) {
u16 val;
- ret = pac1921_read_res(priv, idev->channels[ch].address, &val);
+ ret = pac1921_read_res(priv, idev->channels[bit].address, &val);
if (ret)
goto done;
diff --git a/drivers/iio/chemical/atlas-sensor.c b/drivers/iio/chemical/atlas-sensor.c
index baf93e5e3ca7..a80ed077f3a1 100644
--- a/drivers/iio/chemical/atlas-sensor.c
+++ b/drivers/iio/chemical/atlas-sensor.c
@@ -13,7 +13,6 @@
#include <linux/mutex.h>
#include <linux/err.h>
#include <linux/irq.h>
-#include <linux/irq_work.h>
#include <linux/i2c.h>
#include <linux/mod_devicetable.h>
#include <linux/regmap.h>
@@ -89,7 +88,6 @@ struct atlas_data {
struct iio_trigger *trig;
const struct atlas_device *chip;
struct regmap *regmap;
- struct irq_work work;
unsigned int interrupt_enabled;
/* 96-bit data + 32-bit pad + 64-bit timestamp */
__be32 buffer[6] __aligned(8);
@@ -414,7 +412,11 @@ static int atlas_buffer_postenable(struct iio_dev *indio_dev)
if (ret)
return ret;
- return atlas_set_interrupt(data, true);
+ ret = atlas_set_interrupt(data, true);
+ if (ret)
+ pm_runtime_put_autosuspend(&data->client->dev);
+
+ return ret;
}
static int atlas_buffer_predisable(struct iio_dev *indio_dev)
@@ -439,13 +441,6 @@ static const struct iio_buffer_setup_ops atlas_buffer_setup_ops = {
.predisable = atlas_buffer_predisable,
};
-static void atlas_work_handler(struct irq_work *work)
-{
- struct atlas_data *data = container_of(work, struct atlas_data, work);
-
- iio_trigger_poll(data->trig);
-}
-
static irqreturn_t atlas_trigger_handler(int irq, void *private)
{
struct iio_poll_func *pf = private;
@@ -471,7 +466,7 @@ static irqreturn_t atlas_interrupt_handler(int irq, void *private)
struct iio_dev *indio_dev = private;
struct atlas_data *data = iio_priv(indio_dev);
- irq_work_queue(&data->work);
+ iio_trigger_poll_nested(data->trig);
return IRQ_HANDLED;
}
@@ -669,8 +664,6 @@ static int atlas_probe(struct i2c_client *client)
goto unregister_trigger;
}
- init_irq_work(&data->work, atlas_work_handler);
-
if (client->irq > 0) {
/* interrupt pin toggles on new conversion */
ret = devm_request_threaded_irq(&client->dev, client->irq,
diff --git a/drivers/iio/chemical/sgp30.c b/drivers/iio/chemical/sgp30.c
index 21730d62b5c8..cb5e7bb3f191 100644
--- a/drivers/iio/chemical/sgp30.c
+++ b/drivers/iio/chemical/sgp30.c
@@ -548,6 +548,9 @@ static int sgp_probe(struct i2c_client *client)
data->iaq_thread = kthread_run(sgp_iaq_threadfn, data,
"%s-iaq", data->client->name);
+ if (IS_ERR(data->iaq_thread))
+ return dev_err_probe(dev, PTR_ERR(data->iaq_thread),
+ "failed to start IAQ thread\n");
return 0;
}
diff --git a/drivers/iio/dac/m62332.c b/drivers/iio/dac/m62332.c
index ae53baccec91..61c35dd8f092 100644
--- a/drivers/iio/dac/m62332.c
+++ b/drivers/iio/dac/m62332.c
@@ -32,6 +32,7 @@ static int m62332_set_value(struct iio_dev *indio_dev, u8 val, int channel)
{
struct m62332_data *data = iio_priv(indio_dev);
struct i2c_client *client = data->client;
+ bool enabling, disabling;
u8 outbuf[2];
int res;
@@ -43,7 +44,10 @@ static int m62332_set_value(struct iio_dev *indio_dev, u8 val, int channel)
mutex_lock(&data->mutex);
- if (val) {
+ enabling = val && !data->raw[channel];
+ disabling = !val && data->raw[channel];
+
+ if (enabling) {
res = regulator_enable(data->vcc);
if (res)
goto out;
@@ -52,14 +56,17 @@ static int m62332_set_value(struct iio_dev *indio_dev, u8 val, int channel)
res = i2c_master_send(client, outbuf, ARRAY_SIZE(outbuf));
if (res >= 0 && res != ARRAY_SIZE(outbuf))
res = -EIO;
- if (res < 0)
+ if (res < 0) {
+ if (enabling)
+ regulator_disable(data->vcc);
goto out;
+ }
- data->raw[channel] = val;
-
- if (!val)
+ if (disabling)
regulator_disable(data->vcc);
+ data->raw[channel] = val;
+
mutex_unlock(&data->mutex);
return 0;
diff --git a/drivers/iio/gyro/mpu3050-core.c b/drivers/iio/gyro/mpu3050-core.c
index 90518f39092f..3f24c75b6560 100644
--- a/drivers/iio/gyro/mpu3050-core.c
+++ b/drivers/iio/gyro/mpu3050-core.c
@@ -356,7 +356,7 @@ static int mpu3050_read_raw(struct iio_dev *indio_dev,
goto out_read_raw_unlock;
}
- *val = be16_to_cpu(raw_val);
+ *val = (s16)be16_to_cpu(raw_val);
ret = IIO_VAL_INT;
goto out_read_raw_unlock;
diff --git a/drivers/iio/industrialio-buffer.c b/drivers/iio/industrialio-buffer.c
index 1fb3abe75edc..5c0e7788b90d 100644
--- a/drivers/iio/industrialio-buffer.c
+++ b/drivers/iio/industrialio-buffer.c
@@ -47,9 +47,6 @@ struct iio_dmabuf_priv {
u64 context;
- /* Spinlock used for locking the dma_fence */
- spinlock_t lock;
-
struct dma_buf_attachment *attach;
struct sg_table *sgt;
enum dma_data_direction dir;
@@ -57,7 +54,12 @@ struct iio_dmabuf_priv {
};
struct iio_dma_fence {
+ /*
+ * Must remain the first member so the default release callback can pass
+ * the fence directly to dma_fence_free().
+ */
struct dma_fence base;
+ spinlock_t lock; /* protects base */
struct iio_dmabuf_priv *priv;
struct work_struct work;
};
@@ -1599,12 +1601,16 @@ static int iio_buffer_chrdev_release(struct inode *inode, struct file *filep)
wake_up(&buffer->pollq);
- guard(mutex)(&buffer->dmabufs_mutex);
-
- /* Close all attached DMABUFs */
- list_for_each_entry_safe(priv, tmp, &buffer->dmabufs, entry) {
- list_del_init(&priv->entry);
- iio_buffer_dmabuf_put(priv->attach);
+ /*
+ * The mutex must be unlocked before iio_device_put(), which might drop the
+ * last reference and free the buffer.
+ */
+ scoped_guard(mutex, &buffer->dmabufs_mutex) {
+ /* Close all attached DMABUFs */
+ list_for_each_entry_safe(priv, tmp, &buffer->dmabufs, entry) {
+ list_del_init(&priv->entry);
+ iio_buffer_dmabuf_put(priv->attach);
+ }
}
kfree(ib);
@@ -1682,7 +1688,6 @@ static int iio_buffer_attach_dmabuf(struct iio_dev_buffer_pair *ib,
if (!priv)
return -ENOMEM;
- spin_lock_init(&priv->lock);
priv->context = dma_fence_context_alloc(1);
dmabuf = dma_buf_get(fd);
@@ -1807,18 +1812,9 @@ iio_buffer_dma_fence_get_driver_name(struct dma_fence *fence)
return "iio";
}
-static void iio_buffer_dma_fence_release(struct dma_fence *fence)
-{
- struct iio_dma_fence *iio_fence =
- container_of(fence, struct iio_dma_fence, base);
-
- kfree(iio_fence);
-}
-
static const struct dma_fence_ops iio_buffer_dma_fence_ops = {
.get_driver_name = iio_buffer_dma_fence_get_driver_name,
.get_timeline_name = iio_buffer_dma_fence_get_driver_name,
- .release = iio_buffer_dma_fence_release,
};
static int iio_buffer_enqueue_dmabuf(struct iio_dev_buffer_pair *ib,
@@ -1872,6 +1868,8 @@ static int iio_buffer_enqueue_dmabuf(struct iio_dev_buffer_pair *ib,
goto err_attachment_put;
}
+ spin_lock_init(&fence->lock);
+
fence->priv = priv;
seqno = atomic_add_return(1, &priv->seqno);
@@ -1882,7 +1880,7 @@ static int iio_buffer_enqueue_dmabuf(struct iio_dev_buffer_pair *ib,
* the dma_fence.
*/
dma_fence_init(&fence->base, &iio_buffer_dma_fence_ops,
- &priv->lock, priv->context, seqno);
+ &fence->lock, priv->context, seqno);
ret = iio_dma_resv_lock(dmabuf, nonblock);
if (ret)
diff --git a/drivers/iio/light/cm32181.c b/drivers/iio/light/cm32181.c
index 9df85b3999fa..f708be3389bc 100644
--- a/drivers/iio/light/cm32181.c
+++ b/drivers/iio/light/cm32181.c
@@ -370,7 +370,7 @@ static int cm32181_write_raw(struct iio_dev *indio_dev,
switch (mask) {
case IIO_CHAN_INFO_CALIBSCALE:
cm32181->calibscale = val;
- return val;
+ return 0;
case IIO_CHAN_INFO_INT_TIME:
ret = cm32181_write_als_it(cm32181, val2);
return ret;
diff --git a/drivers/iio/light/gp2ap002.c b/drivers/iio/light/gp2ap002.c
index 161776a9468f..e11aa1c7acf4 100644
--- a/drivers/iio/light/gp2ap002.c
+++ b/drivers/iio/light/gp2ap002.c
@@ -343,6 +343,10 @@ static int gp2ap002_write_event_config(struct iio_dev *indio_dev,
int state)
{
struct gp2ap002 *gp2ap002 = iio_priv(indio_dev);
+ int ret;
+
+ if (state == gp2ap002->enabled)
+ return 0;
if (state) {
/*
@@ -350,14 +354,17 @@ static int gp2ap002_write_event_config(struct iio_dev *indio_dev,
* already) and reintialize the sensor by using runtime_pm
* callbacks.
*/
- pm_runtime_get_sync(gp2ap002->dev);
- gp2ap002->enabled = true;
+ ret = pm_runtime_resume_and_get(gp2ap002->dev);
+ if (ret)
+ return ret;
+
} else {
pm_runtime_mark_last_busy(gp2ap002->dev);
pm_runtime_put_autosuspend(gp2ap002->dev);
- gp2ap002->enabled = false;
}
+ gp2ap002->enabled = state;
+
return 0;
}
@@ -644,6 +651,7 @@ static int gp2ap002_runtime_suspend(struct device *dev)
/* Disable chip and IRQ, everything off */
ret = regmap_write(gp2ap002->map, GP2AP002_OPMOD, 0x00);
if (ret) {
+ enable_irq(gp2ap002->irq);
dev_err(gp2ap002->dev, "error setting up operation mode\n");
return ret;
}
@@ -671,7 +679,7 @@ static int gp2ap002_runtime_resume(struct device *dev)
ret = regulator_enable(gp2ap002->vio);
if (ret) {
dev_err(dev, "failed to enable VIO regulator in resume path\n");
- return ret;
+ goto out_disable_vdd;
}
msleep(20);
@@ -679,13 +687,19 @@ static int gp2ap002_runtime_resume(struct device *dev)
ret = gp2ap002_init(gp2ap002);
if (ret) {
dev_err(dev, "re-initialization failed\n");
- return ret;
+ goto out_disable_vio;
}
/* Re-activate the IRQ */
enable_irq(gp2ap002->irq);
return 0;
+
+out_disable_vio:
+ regulator_disable(gp2ap002->vio);
+out_disable_vdd:
+ regulator_disable(gp2ap002->vdd);
+ return ret;
}
static DEFINE_RUNTIME_DEV_PM_OPS(gp2ap002_dev_pm_ops, gp2ap002_runtime_suspend,
diff --git a/drivers/iio/light/isl29028.c b/drivers/iio/light/isl29028.c
index 95bfb3ffa519..a23b2af287d5 100644
--- a/drivers/iio/light/isl29028.c
+++ b/drivers/iio/light/isl29028.c
@@ -414,7 +414,7 @@ static int isl29028_write_raw(struct iio_dev *indio_dev,
if (ret < 0)
return ret;
- return ret;
+ return 0;
}
static int isl29028_read_raw(struct iio_dev *indio_dev,
diff --git a/drivers/iio/light/ltrf216a.c b/drivers/iio/light/ltrf216a.c
index 37eecff571b9..3e3487a19057 100644
--- a/drivers/iio/light/ltrf216a.c
+++ b/drivers/iio/light/ltrf216a.c
@@ -249,11 +249,10 @@ static int ltrf216a_get_lux(struct ltrf216a_data *data)
return ret;
greendata = ltrf216a_read_data(data, LTRF216A_ALS_DATA_0);
+ ltrf216a_set_power_state(data, false);
if (greendata < 0)
return greendata;
- ltrf216a_set_power_state(data, false);
-
lux = greendata * data->info->lux_multiplier * LTRF216A_WIN_FAC;
return lux;
diff --git a/drivers/iio/light/opt4001.c b/drivers/iio/light/opt4001.c
index 6cf60151b3d8..6d159255ff12 100644
--- a/drivers/iio/light/opt4001.c
+++ b/drivers/iio/light/opt4001.c
@@ -39,7 +39,7 @@
#define OPT4001_CTRL_OPER_MODE_MASK GENMASK(5, 4)
#define OPT4001_CTRL_LATCH_MASK GENMASK(3, 3)
#define OPT4001_CTRL_INT_POL_MASK GENMASK(2, 2)
-#define OPT4001_CTRL_FAULT_COUNT GENMASK(0, 1)
+#define OPT4001_CTRL_FAULT_COUNT_MASK GENMASK(1, 0)
/* OPT4001 constants */
#define OPT4001_DEVICE_ID_VAL 0x121
@@ -173,6 +173,7 @@ static int opt4001_read_lux_value(struct iio_dev *indio_dev,
u8 crc;
u8 calc_crc;
u64 lux_raw;
+ u32 rem;
int ret;
ret = regmap_read(chip->regmap, OPT4001_LIGHT1_MSB, &light1);
@@ -199,8 +200,8 @@ static int opt4001_read_lux_value(struct iio_dev *indio_dev,
lux_raw = lux_raw << exp;
lux_raw = lux_raw * chip->chip_info->mul;
- *val = div_u64_rem(lux_raw, chip->chip_info->div, val2);
- *val2 = *val2 * 100;
+ *val = div_u64_rem(lux_raw, chip->chip_info->div, &rem);
+ *val2 = rem * 100;
return IIO_VAL_INT_PLUS_NANO;
}
@@ -222,33 +223,14 @@ static int opt4001_set_conf(struct opt4001_chip *chip)
return ret;
}
-static int opt4001_power_down(struct opt4001_chip *chip)
-{
- struct device *dev = &chip->client->dev;
- int ret;
- unsigned int reg;
-
- ret = regmap_read(chip->regmap, OPT4001_DEVICE_ID, ®);
- if (ret) {
- dev_err(dev, "Failed to read configuration\n");
- return ret;
- }
-
- /* MODE_OFF is 0x0 so just set bits to 0 */
- reg &= ~OPT4001_CTRL_OPER_MODE_MASK;
-
- ret = regmap_write(chip->regmap, OPT4001_CTRL, reg);
- if (ret)
- dev_err(dev, "Failed to set configuration to power down\n");
-
- return ret;
-}
-
static void opt4001_chip_off_action(void *data)
{
struct opt4001_chip *chip = data;
+ int ret;
- opt4001_power_down(chip);
+ ret = regmap_clear_bits(chip->regmap, OPT4001_CTRL, OPT4001_CTRL_OPER_MODE_MASK);
+ if (ret)
+ dev_err(&chip->client->dev, "Failed to power down\n");
}
static const struct iio_chan_spec opt4001_channels[] = {
@@ -287,6 +269,9 @@ static int opt4001_write_raw(struct iio_dev *indio_dev,
switch (mask) {
case IIO_CHAN_INFO_INT_TIME:
+ if (val)
+ return -EINVAL;
+
int_time = opt4001_als_time_to_index(val2);
if (int_time < 0)
return int_time;
diff --git a/drivers/iio/light/tsl2583.c b/drivers/iio/light/tsl2583.c
index 02ad11611b9c..dfcb8228ada1 100644
--- a/drivers/iio/light/tsl2583.c
+++ b/drivers/iio/light/tsl2583.c
@@ -800,7 +800,7 @@ static int tsl2583_write_raw(struct iio_dev *indio_dev,
if (ret < 0)
return ret;
- return ret;
+ return 0;
}
static const struct iio_info tsl2583_info = {
diff --git a/drivers/iio/light/tsl2772.c b/drivers/iio/light/tsl2772.c
index cab468a82b61..fffed029c24d 100644
--- a/drivers/iio/light/tsl2772.c
+++ b/drivers/iio/light/tsl2772.c
@@ -1263,7 +1263,7 @@ static int tsl2772_read_raw(struct iio_dev *indio_dev,
}
break;
case IIO_CHAN_INFO_CALIBSCALE:
- if (chan->type == IIO_LIGHT)
+ if (chan->type == IIO_INTENSITY)
*val = tsl2772_als_gain[chip->settings.als_gain];
else
*val = tsl2772_prox_gain[chip->settings.prox_gain];
diff --git a/drivers/iio/orientation/hid-sensor-rotation.c b/drivers/iio/orientation/hid-sensor-rotation.c
index a0cf5d2a87fb..2ee34e677aa9 100644
--- a/drivers/iio/orientation/hid-sensor-rotation.c
+++ b/drivers/iio/orientation/hid-sensor-rotation.c
@@ -337,12 +337,6 @@ static int hid_dev_rot_probe(struct platform_device *pdev)
return ret;
}
- ret = iio_device_register(indio_dev);
- if (ret) {
- dev_err(&pdev->dev, "device register failed\n");
- goto error_remove_trigger;
- }
-
rot_state->callbacks.send_event = dev_rot_proc_event;
rot_state->callbacks.capture_sample = dev_rot_capture_sample;
rot_state->callbacks.pdev = pdev;
@@ -350,13 +344,19 @@ static int hid_dev_rot_probe(struct platform_device *pdev)
&rot_state->callbacks);
if (ret) {
dev_err(&pdev->dev, "callback reg failed\n");
- goto error_iio_unreg;
+ goto error_remove_trigger;
+ }
+
+ ret = iio_device_register(indio_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "device register failed\n");
+ goto error_remove_callback;
}
return 0;
-error_iio_unreg:
- iio_device_unregister(indio_dev);
+error_remove_callback:
+ sensor_hub_remove_callback(hsdev, hsdev->usage);
error_remove_trigger:
hid_sensor_remove_trigger(indio_dev, &rot_state->common_attributes);
return ret;
@@ -369,8 +369,8 @@ static void hid_dev_rot_remove(struct platform_device *pdev)
struct iio_dev *indio_dev = platform_get_drvdata(pdev);
struct dev_rot_state *rot_state = iio_priv(indio_dev);
- sensor_hub_remove_callback(hsdev, hsdev->usage);
iio_device_unregister(indio_dev);
+ sensor_hub_remove_callback(hsdev, hsdev->usage);
hid_sensor_remove_trigger(indio_dev, &rot_state->common_attributes);
}
diff --git a/drivers/iio/pressure/dps310.c b/drivers/iio/pressure/dps310.c
index c6f44f0f4d2e..5a986d5d6b66 100644
--- a/drivers/iio/pressure/dps310.c
+++ b/drivers/iio/pressure/dps310.c
@@ -845,7 +845,6 @@ static const struct iio_info dps310_info = {
static int dps310_probe(struct i2c_client *client)
{
- const struct i2c_device_id *id = i2c_client_get_device_id(client);
struct dps310_data *data;
struct iio_dev *iio;
int rc;
@@ -858,7 +857,7 @@ static int dps310_probe(struct i2c_client *client)
data->client = client;
mutex_init(&data->lock);
- iio->name = id->name;
+ iio->name = DPS310_DEV_NAME;
iio->channels = dps310_channels;
iio->num_channels = ARRAY_SIZE(dps310_channels);
iio->info = &dps310_info;
diff --git a/drivers/iio/pressure/mpl115.c b/drivers/iio/pressure/mpl115.c
index 11ee879f4da4..5646666a978c 100644
--- a/drivers/iio/pressure/mpl115.c
+++ b/drivers/iio/pressure/mpl115.c
@@ -203,9 +203,9 @@ int mpl115_probe(struct device *dev, const char *name,
if (data->shutdown) {
/* Enable runtime PM */
- pm_runtime_get_noresume(dev);
- pm_runtime_set_active(dev);
- pm_runtime_enable(dev);
+ ret = pm_runtime_set_active(dev);
+ if (ret)
+ return ret;
/*
* As the device takes 3 ms to come up with a fresh
@@ -215,7 +215,10 @@ int mpl115_probe(struct device *dev, const char *name,
*/
pm_runtime_set_autosuspend_delay(dev, 2000);
pm_runtime_use_autosuspend(dev);
- pm_runtime_put(dev);
+
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ return ret;
dev_dbg(dev, "low-power mode enabled");
} else
diff --git a/drivers/iio/proximity/srf04.c b/drivers/iio/proximity/srf04.c
index 86c57672fc7e..7c3f817760d1 100644
--- a/drivers/iio/proximity/srf04.c
+++ b/drivers/iio/proximity/srf04.c
@@ -335,6 +335,7 @@ static int srf04_probe(struct platform_device *pdev)
if (ret) {
dev_err(data->dev, "pm_runtime_set_active: %d\n", ret);
iio_device_unregister(indio_dev);
+ return ret;
}
pm_runtime_enable(data->dev);
diff --git a/drivers/iio/temperature/hid-sensor-temperature.c b/drivers/iio/temperature/hid-sensor-temperature.c
index 0143fd478933..3cfd51236e4c 100644
--- a/drivers/iio/temperature/hid-sensor-temperature.c
+++ b/drivers/iio/temperature/hid-sensor-temperature.c
@@ -243,7 +243,7 @@ static int hid_temperature_probe(struct platform_device *pdev)
if (ret)
goto error_remove_trigger;
- ret = devm_iio_device_register(indio_dev->dev.parent, indio_dev);
+ ret = iio_device_register(indio_dev);
if (ret)
goto error_remove_callback;
@@ -263,6 +263,7 @@ static void hid_temperature_remove(struct platform_device *pdev)
struct iio_dev *indio_dev = platform_get_drvdata(pdev);
struct temperature_state *temp_st = iio_priv(indio_dev);
+ iio_device_unregister(indio_dev);
sensor_hub_remove_callback(hsdev, HID_USAGE_SENSOR_TEMPERATURE);
hid_sensor_remove_trigger(indio_dev, &temp_st->common_attributes);
}
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index a225eae78d3e..5a27dad94ec5 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -3517,10 +3517,12 @@ static void addr_handler(int status, struct sockaddr *src_addr,
memcpy(addr, src_addr, rdma_addr_size(src_addr));
if (!status && !id_priv->cma_dev) {
status = cma_acquire_dev_by_src_ip(id_priv);
- if (status)
+ if (status) {
pr_debug_ratelimited("RDMA CM: ADDR_ERROR: failed to acquire device. status %d\n",
status);
- rdma_restrack_add(&id_priv->res);
+ } else {
+ rdma_restrack_add(&id_priv->res);
+ }
} else if (status) {
pr_debug_ratelimited("RDMA CM: ADDR_ERROR: failed to resolve IP. status %d\n", status);
}
diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c
index d41af5ab6403..8cbca897fc53 100644
--- a/drivers/infiniband/core/counters.c
+++ b/drivers/infiniband/core/counters.c
@@ -217,7 +217,6 @@ static void rdma_counter_free(struct rdma_counter *counter)
mutex_unlock(&port_counter->lock);
- rdma_restrack_del(&counter->res);
rdma_free_hw_stats_struct(counter->stats);
kfree(counter);
}
@@ -312,6 +311,7 @@ static void counter_release(struct kref *kref)
counter = container_of(kref, struct rdma_counter, kref);
counter_history_stat_update(counter);
+ rdma_restrack_del(&counter->res);
counter->device->ops.counter_dealloc(counter);
rdma_counter_free(counter);
}
@@ -472,7 +472,8 @@ static struct rdma_counter *rdma_get_counter_by_id(struct ib_device *dev,
return NULL;
counter = container_of(res, struct rdma_counter, res);
- kref_get(&counter->kref);
+ if (!kref_get_unless_zero(&counter->kref))
+ counter = NULL;
rdma_restrack_put(res);
return counter;
diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c
index a70876a0a231..4ea0dfcff95e 100644
--- a/drivers/infiniband/core/cq.c
+++ b/drivers/infiniband/core/cq.c
@@ -317,13 +317,19 @@ EXPORT_SYMBOL(__ib_alloc_cq_any);
*/
void ib_free_cq(struct ib_cq *cq)
{
- int ret;
+ int ret = 0;
if (WARN_ON_ONCE(atomic_read(&cq->usecnt)))
return;
if (WARN_ON_ONCE(cq->cqe_used))
return;
+ rdma_restrack_del(&cq->res);
+ if (cq->device->ops.pre_destroy_cq) {
+ ret = cq->device->ops.pre_destroy_cq(cq);
+ WARN_ONCE(ret, "Disable of kernel CQ shouldn't fail");
+ }
+
switch (cq->poll_ctx) {
case IB_POLL_DIRECT:
break;
@@ -340,9 +346,11 @@ void ib_free_cq(struct ib_cq *cq)
rdma_dim_destroy(cq);
trace_cq_free(cq);
- ret = cq->device->ops.destroy_cq(cq, NULL);
+ if (cq->device->ops.post_destroy_cq)
+ cq->device->ops.post_destroy_cq(cq);
+ else
+ ret = cq->device->ops.destroy_cq(cq, NULL);
WARN_ONCE(ret, "Destroy of kernel CQ shouldn't fail");
- rdma_restrack_del(&cq->res);
kfree(cq->wc);
kfree(cq);
}
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 666ce83cc35c..d927270bab47 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -2783,8 +2783,10 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, modify_srq);
SET_DEVICE_OP(dev_ops, modify_wq);
SET_DEVICE_OP(dev_ops, peek_cq);
+ SET_DEVICE_OP(dev_ops, pre_destroy_cq);
SET_DEVICE_OP(dev_ops, poll_cq);
SET_DEVICE_OP(dev_ops, port_groups);
+ SET_DEVICE_OP(dev_ops, post_destroy_cq);
SET_DEVICE_OP(dev_ops, post_recv);
SET_DEVICE_OP(dev_ops, post_send);
SET_DEVICE_OP(dev_ops, post_srq_recv);
@@ -3017,6 +3019,7 @@ static void __exit ib_core_cleanup(void)
/* Make sure that any pending umem accounting work is done. */
destroy_workqueue(ib_wq);
destroy_workqueue(ib_unreg_wq);
+ rcu_barrier();
WARN_ON(!xa_empty(&clients));
WARN_ON(!xa_empty(&devices));
}
diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index fef11a80647c..98d33ad6a397 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -678,7 +678,7 @@ static int fill_res_mr_entry(struct sk_buff *msg, bool has_cap_net_admin,
struct rdma_restrack_entry *res, uint32_t port)
{
struct ib_mr *mr = container_of(res, struct ib_mr, res);
- struct ib_device *dev = mr->pd->device;
+ struct ib_device *dev = mr->device;
if (has_cap_net_admin) {
if (nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_RKEY, mr->rkey))
@@ -694,9 +694,12 @@ static int fill_res_mr_entry(struct sk_buff *msg, bool has_cap_net_admin,
if (nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_MRN, res->id))
return -EMSGSIZE;
- if (!rdma_is_kernel_res(res) &&
- nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_PDN, mr->pd->res.id))
- return -EMSGSIZE;
+ if (!rdma_is_kernel_res(res)) {
+ struct ib_pd *pd = READ_ONCE(mr->pd);
+
+ if (nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_PDN, pd->res.id))
+ return -EMSGSIZE;
+ }
if (fill_res_name_pid(msg, res))
return -EMSGSIZE;
@@ -710,7 +713,7 @@ static int fill_res_mr_raw_entry(struct sk_buff *msg, bool has_cap_net_admin,
struct rdma_restrack_entry *res, uint32_t port)
{
struct ib_mr *mr = container_of(res, struct ib_mr, res);
- struct ib_device *dev = mr->pd->device;
+ struct ib_device *dev = mr->device;
if (!dev->ops.fill_res_mr_entry_raw)
return -EINVAL;
@@ -1000,7 +1003,7 @@ static int fill_stat_mr_entry(struct sk_buff *msg, bool has_cap_net_admin,
struct rdma_restrack_entry *res, uint32_t port)
{
struct ib_mr *mr = container_of(res, struct ib_mr, res);
- struct ib_device *dev = mr->pd->device;
+ struct ib_device *dev = mr->device;
if (nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_MRN, res->id))
goto err;
@@ -2093,6 +2096,11 @@ static int nldev_stat_set_counter_dynamic_doit(struct nlattr *tb[],
nla_for_each_nested(entry_attr, tb[RDMA_NLDEV_ATTR_STAT_HWCOUNTERS],
rem) {
+ if (nla_len(entry_attr) != sizeof(u32)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
index = nla_get_u32(entry_attr);
if ((index >= stats->num_counters) ||
!(stats->descs[index].flags & IB_STAT_FLAG_OPTIONAL)) {
diff --git a/drivers/infiniband/core/restrack.c b/drivers/infiniband/core/restrack.c
index 3313410014cd..c03042dc1ca5 100644
--- a/drivers/infiniband/core/restrack.c
+++ b/drivers/infiniband/core/restrack.c
@@ -71,6 +71,8 @@ int rdma_restrack_count(struct ib_device *dev, enum rdma_restrack_type type,
xa_lock(&rt->xa);
xas_for_each(&xas, e, U32_MAX) {
+ if (xa_is_zero(e))
+ continue;
if (xa_get_mark(&rt->xa, e->id, RESTRACK_DD) && !show_details)
continue;
cnt++;
@@ -125,6 +127,46 @@ static void rdma_restrack_attach_task(struct rdma_restrack_entry *res,
res->user = true;
}
+static struct rdma_restrack_root *res_to_rt(struct rdma_restrack_entry *res)
+{
+ struct ib_device *dev = res_to_dev(res);
+
+ if (WARN_ON(!dev))
+ return NULL;
+
+ return &dev->res[res->type];
+}
+
+static void restrack_drain_res(struct rdma_restrack_root *rt,
+ struct rdma_restrack_entry *res)
+{
+ if (rt) {
+ struct rdma_restrack_entry *old;
+
+ old = xa_cmpxchg(&rt->xa, res->id, res, XA_ZERO_ENTRY,
+ GFP_KERNEL);
+ WARN_ON(old != res);
+ }
+
+ rdma_restrack_put(res);
+ wait_for_completion(&res->comp);
+}
+
+static void restrack_restore_res(struct rdma_restrack_root *rt,
+ struct rdma_restrack_entry *res)
+{
+ reinit_completion(&res->comp);
+ kref_init(&res->kref);
+
+ if (rt) {
+ struct rdma_restrack_entry *old;
+
+ old = xa_cmpxchg(&rt->xa, res->id, XA_ZERO_ENTRY, res,
+ GFP_KERNEL);
+ WARN_ON(old);
+ }
+}
+
/**
* rdma_restrack_set_name() - set the task for this resource
* @res: resource entry
@@ -173,22 +215,23 @@ void rdma_restrack_new(struct rdma_restrack_entry *res,
EXPORT_SYMBOL(rdma_restrack_new);
/**
- * rdma_restrack_add() - add object to the reource tracking database
+ * rdma_restrack_add() - add object to the resource tracking database.
+ * If this resource reuses an ID of a resource that was already destroyed
+ * after calling rdma_restrack_begin() but didn't yet call
+ * rdma_restrack_commit_del() it can result in an untracked QP.
* @res: resource entry
*/
void rdma_restrack_add(struct rdma_restrack_entry *res)
{
- struct ib_device *dev = res_to_dev(res);
struct rdma_restrack_root *rt;
int ret = 0;
- if (!dev)
- return;
-
if (res->no_track)
goto out;
- rt = &dev->res[res->type];
+ rt = res_to_rt(res);
+ if (!rt)
+ return;
if (res->type == RDMA_RESTRACK_QP) {
/* Special case to ensure that LQPN points to right QP */
@@ -225,6 +268,28 @@ void rdma_restrack_add(struct rdma_restrack_entry *res)
}
EXPORT_SYMBOL(rdma_restrack_add);
+/**
+ * rdma_restrack_abort_del() - re-add object to the resource tracking database
+ * it can only be used after rdma_restrack_begin_del().
+ * @res: resource entry
+ */
+void rdma_restrack_abort_del(struct rdma_restrack_entry *res)
+{
+ struct rdma_restrack_root *rt = NULL;
+
+ if (!res->valid)
+ return;
+
+ if (!res->no_track) {
+ rt = res_to_rt(res);
+ if (!rt)
+ return;
+ }
+
+ restrack_restore_res(rt, res);
+}
+EXPORT_SYMBOL(rdma_restrack_abort_del);
+
int __must_check rdma_restrack_get(struct rdma_restrack_entry *res)
{
return kref_get_unless_zero(&res->kref);
@@ -261,7 +326,7 @@ static void restrack_release(struct kref *kref)
struct rdma_restrack_entry *res;
res = container_of(kref, struct rdma_restrack_entry, kref);
- if (res->task) {
+ if (res->task && !res->valid) {
put_task_struct(res->task);
res->task = NULL;
}
@@ -275,14 +340,43 @@ int rdma_restrack_put(struct rdma_restrack_entry *res)
EXPORT_SYMBOL(rdma_restrack_put);
/**
- * rdma_restrack_del() - delete object from the reource tracking database
+ * rdma_restrack_sync() - Fence concurrent netlink dumps on an entry
+ * @res: resource entry
+ *
+ * After this returns any concurrent netlink dump threads will see the current
+ * value of the object. This is useful if the object has to be changed and there
+ * is not locking to protect the nl side. Eg for mr->pd. This effectively
+ * destroys the object from a kref/xarray perspective and then immediately
+ * restores it. The kref is acting like a lock to barrier concurrent nl threads.
+ * Callers must ensure rdma_restrack_del() is not concurrently called.
+ */
+void rdma_restrack_sync(struct rdma_restrack_entry *res)
+{
+ struct rdma_restrack_root *rt;
+
+ if (!res->valid || res->no_track)
+ return;
+
+ rt = res_to_rt(res);
+ if (!rt)
+ return;
+
+ if (WARN_ON(xa_get_mark(&rt->xa, res->id, RESTRACK_DD)))
+ return;
+
+ restrack_drain_res(rt, res);
+ restrack_restore_res(rt, res);
+}
+EXPORT_SYMBOL(rdma_restrack_sync);
+
+/**
+ * rdma_restrack_del() - delete object from the resource tracking database
* @res: resource entry
*/
void rdma_restrack_del(struct rdma_restrack_entry *res)
{
struct rdma_restrack_entry *old;
struct rdma_restrack_root *rt;
- struct ib_device *dev;
if (!res->valid) {
if (res->task) {
@@ -295,12 +389,10 @@ void rdma_restrack_del(struct rdma_restrack_entry *res)
if (res->no_track)
goto out;
- dev = res_to_dev(res);
- if (WARN_ON(!dev))
+ rt = res_to_rt(res);
+ if (!rt)
return;
- rt = &dev->res[res->type];
-
old = xa_erase(&rt->xa, res->id);
WARN_ON(old != res);
@@ -308,5 +400,61 @@ void rdma_restrack_del(struct rdma_restrack_entry *res)
res->valid = false;
rdma_restrack_put(res);
wait_for_completion(&res->comp);
+ if (res->task) {
+ put_task_struct(res->task);
+ res->task = NULL;
+ }
}
EXPORT_SYMBOL(rdma_restrack_del);
+
+/**
+ * rdma_restrack_begin_del() - invalidate the object from the resource tracking
+ * database but preserve its index in the array.
+ * Since this preserves the index in the array until rdma_restrack_commit_del()
+ * is called, if rdma_restrack_add() is called in between with an old QP ID it
+ * can result in an untracked QP.
+ * @res: resource entry
+ */
+void rdma_restrack_begin_del(struct rdma_restrack_entry *res)
+{
+ struct rdma_restrack_root *rt = NULL;
+
+ if (!res->valid)
+ return;
+
+ if (!res->no_track) {
+ rt = res_to_rt(res);
+ if (!rt)
+ return;
+ }
+
+ restrack_drain_res(rt, res);
+}
+EXPORT_SYMBOL(rdma_restrack_begin_del);
+
+/**
+ * rdma_restrack_commit_del() - delete object from the resource tracking
+ * database and free the task.
+ * @res: resource entry
+ */
+void rdma_restrack_commit_del(struct rdma_restrack_entry *res)
+{
+ struct rdma_restrack_root *rt;
+
+ if (!res->valid || res->no_track)
+ goto out;
+
+ rt = res_to_rt(res);
+ if (!rt)
+ return;
+
+ xa_erase(&rt->xa, res->id);
+
+out:
+ res->valid = false;
+ if (res->task) {
+ put_task_struct(res->task);
+ res->task = NULL;
+ }
+}
+EXPORT_SYMBOL(rdma_restrack_commit_del);
diff --git a/drivers/infiniband/core/restrack.h b/drivers/infiniband/core/restrack.h
index 6a04fc41f738..2df78e084e10 100644
--- a/drivers/infiniband/core/restrack.h
+++ b/drivers/infiniband/core/restrack.h
@@ -26,7 +26,11 @@ struct rdma_restrack_root {
int rdma_restrack_init(struct ib_device *dev);
void rdma_restrack_clean(struct ib_device *dev);
void rdma_restrack_add(struct rdma_restrack_entry *res);
+void rdma_restrack_abort_del(struct rdma_restrack_entry *res);
void rdma_restrack_del(struct rdma_restrack_entry *res);
+void rdma_restrack_sync(struct rdma_restrack_entry *res);
+void rdma_restrack_begin_del(struct rdma_restrack_entry *res);
+void rdma_restrack_commit_del(struct rdma_restrack_entry *res);
void rdma_restrack_new(struct rdma_restrack_entry *res,
enum rdma_restrack_type type);
void rdma_restrack_set_name(struct rdma_restrack_entry *res,
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index b27079992488..5abab926e335 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -922,7 +922,7 @@ static ssize_t ucma_query_path(struct ucma_context *ctx,
resp->num_paths = ctx->cm_id->route.num_pri_alt_paths;
for (i = 0, out_len -= sizeof(*resp);
- i < resp->num_paths && out_len > sizeof(struct ib_path_rec_data);
+ i < resp->num_paths && out_len >= sizeof(struct ib_path_rec_data);
i++, out_len -= sizeof(struct ib_path_rec_data)) {
struct sa_path_rec *rec = &ctx->cm_id->route.path_rec[i];
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index ac81b7d1eec9..4c258f4c4fb2 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -46,6 +46,7 @@
#include "uverbs.h"
#include "core_priv.h"
+#include "restrack.h"
/*
* Copy a response to userspace. If the provided 'resp' is larger than the
@@ -813,6 +814,10 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
ret = PTR_ERR(new_pd);
goto put_uobjs;
}
+ if (new_pd == orig_pd) {
+ uobj_put_obj_read(new_pd);
+ cmd.flags &= ~IB_MR_REREG_PD;
+ }
} else {
new_pd = mr->pd;
}
@@ -858,9 +863,10 @@ static int ib_uverbs_rereg_mr(struct uverbs_attr_bundle *attrs)
mr = new_mr;
} else {
if (cmd.flags & IB_MR_REREG_PD) {
- atomic_dec(&orig_pd->usecnt);
- mr->pd = new_pd;
atomic_inc(&new_pd->usecnt);
+ WRITE_ONCE(mr->pd, new_pd);
+ rdma_restrack_sync(&mr->res);
+ atomic_dec(&orig_pd->usecnt);
}
if (cmd.flags & IB_MR_REREG_TRANS) {
mr->iova = cmd.hca_va;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index dff87b5980aa..d0bd57ac7c6a 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -335,6 +335,7 @@ int ib_dealloc_pd_user(struct ib_pd *pd, struct ib_udata *udata)
{
int ret;
+ rdma_restrack_begin_del(&pd->res);
if (pd->__internal_mr) {
ret = pd->device->ops.dereg_mr(pd->__internal_mr, NULL);
WARN_ON(ret);
@@ -342,10 +343,12 @@ int ib_dealloc_pd_user(struct ib_pd *pd, struct ib_udata *udata)
}
ret = pd->device->ops.dealloc_pd(pd, udata);
- if (ret)
+ if (ret) {
+ rdma_restrack_abort_del(&pd->res);
return ret;
+ }
- rdma_restrack_del(&pd->res);
+ rdma_restrack_commit_del(&pd->res);
kfree(pd);
return ret;
}
@@ -1083,16 +1086,20 @@ int ib_destroy_srq_user(struct ib_srq *srq, struct ib_udata *udata)
if (atomic_read(&srq->usecnt))
return -EBUSY;
+ rdma_restrack_begin_del(&srq->res);
+
ret = srq->device->ops.destroy_srq(srq, udata);
- if (ret)
+ if (ret) {
+ rdma_restrack_abort_del(&srq->res);
return ret;
+ }
atomic_dec(&srq->pd->usecnt);
if (srq->srq_type == IB_SRQT_XRC && srq->ext.xrc.xrcd)
atomic_dec(&srq->ext.xrc.xrcd->usecnt);
if (ib_srq_has_cq(srq->srq_type))
atomic_dec(&srq->ext.cq->usecnt);
- rdma_restrack_del(&srq->res);
+ rdma_restrack_commit_del(&srq->res);
kfree(srq);
return ret;
@@ -2098,6 +2105,8 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
if (qp->real_qp != qp)
return __ib_destroy_shared_qp(qp);
+ rdma_restrack_begin_del(&qp->res);
+
sec = qp->qp_sec;
if (sec)
ib_destroy_qp_security_begin(sec);
@@ -2110,6 +2119,7 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
if (ret) {
if (sec)
ib_destroy_qp_security_abort(sec);
+ rdma_restrack_abort_del(&qp->res);
return ret;
}
@@ -2122,7 +2132,7 @@ int ib_destroy_qp_user(struct ib_qp *qp, struct ib_udata *udata)
if (sec)
ib_destroy_qp_security_end(sec);
- rdma_restrack_del(&qp->res);
+ rdma_restrack_commit_del(&qp->res);
kfree(qp);
return ret;
}
@@ -2187,11 +2197,15 @@ int ib_destroy_cq_user(struct ib_cq *cq, struct ib_udata *udata)
if (atomic_read(&cq->usecnt))
return -EBUSY;
+ rdma_restrack_begin_del(&cq->res);
+
ret = cq->device->ops.destroy_cq(cq, udata);
- if (ret)
+ if (ret) {
+ rdma_restrack_abort_del(&cq->res);
return ret;
+ }
- rdma_restrack_del(&cq->res);
+ rdma_restrack_commit_del(&cq->res);
kfree(cq);
return ret;
}
diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
index 7ad5d88358b0..467bdfea6c92 100644
--- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c
+++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c
@@ -4389,11 +4389,13 @@ int bnxt_re_mmap(struct ib_ucontext *ib_uctx, struct vm_area_struct *vma)
case BNXT_RE_MMAP_DBR_PAGE:
case BNXT_RE_MMAP_TOGGLE_PAGE:
/* Driver doesn't expect write access for user space */
- if (vma->vm_flags & VM_WRITE)
+ if (vma->vm_flags & VM_WRITE) {
ret = -EFAULT;
- else
+ } else {
+ vm_flags_clear(vma, VM_MAYWRITE);
ret = vm_insert_page(vma, vma->vm_start,
virt_to_page((void *)bnxt_entry->mem_offset));
+ }
break;
default:
ret = -EINVAL;
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index ac86a77c712e..d5101cabbd4b 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -935,6 +935,7 @@ static void c4iw_rdev_close(struct c4iw_rdev *rdev)
void c4iw_dealloc(struct uld_ctx *ctx)
{
+ debugfs_remove_recursive(ctx->dev->debugfs_root);
c4iw_rdev_close(&ctx->dev->rdev);
WARN_ON(!xa_empty(&ctx->dev->cqs));
WARN_ON(!xa_empty(&ctx->dev->qps));
@@ -959,7 +960,6 @@ static void c4iw_remove(struct uld_ctx *ctx)
if (!ctx->dev)
return;
- debugfs_remove_recursive(ctx->dev->debugfs_root);
c4iw_unregister_device(ctx->dev);
c4iw_dealloc(ctx);
}
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 88db7e527728..9c24789bbb54 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -277,6 +277,7 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
int err;
struct fw_ri_tpte *tpt;
u32 stag_idx;
+ bool stag_idx_allocated = false;
static atomic_t key;
if (c4iw_fatal_error(rdev))
@@ -299,6 +300,7 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
return -ENOMEM;
}
mutex_lock(&rdev->stats.lock);
+ stag_idx_allocated = true;
rdev->stats.stag.cur += 32;
if (rdev->stats.stag.cur > rdev->stats.stag.max)
rdev->stats.stag.max = rdev->stats.stag.cur;
@@ -333,7 +335,7 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
(rdev->lldi.vr->stag.start >> 5),
sizeof(*tpt), tpt, skb, wr_waitp);
- if (reset_tpt_entry) {
+ if (reset_tpt_entry || (err && stag_idx_allocated)) {
c4iw_put_resource(&rdev->resource.tpt_table, stag_idx);
mutex_lock(&rdev->stats.lock);
rdev->stats.stag.cur -= 32;
diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
index ff36b7994af9..03f4e97d2614 100644
--- a/drivers/infiniband/hw/efa/efa_verbs.c
+++ b/drivers/infiniband/hw/efa/efa_verbs.c
@@ -1313,9 +1313,11 @@ static int pbl_chunk_list_create(struct efa_dev *dev, struct pbl_context *pbl)
chunk_list->chunks[i].length = EFA_CHUNK_USED_SIZE;
}
- chunk_list->chunks[chunk_list_size - 1].length =
- ((page_cnt % EFA_PTRS_PER_CHUNK) * EFA_CHUNK_PAYLOAD_PTR_SIZE) +
- EFA_CHUNK_PTR_SIZE;
+
+ if (page_cnt % EFA_PTRS_PER_CHUNK != 0)
+ chunk_list->chunks[chunk_list_size - 1].length =
+ ((page_cnt % EFA_PTRS_PER_CHUNK) * EFA_CHUNK_PAYLOAD_PTR_SIZE) +
+ EFA_CHUNK_PTR_SIZE;
/* fill the dma addresses of sg list pages to chunks: */
chunk_idx = 0;
@@ -1327,9 +1329,12 @@ static int pbl_chunk_list_create(struct efa_dev *dev, struct pbl_context *pbl)
rdma_block_iter_dma_address(&biter);
if (payload_idx == EFA_PTRS_PER_CHUNK) {
+ payload_idx = 0;
chunk_idx++;
+ if (chunk_idx >= chunk_list_size)
+ break;
+
cur_chunk_buf = chunk_list->chunks[chunk_idx].buf;
- payload_idx = 0;
}
}
diff --git a/drivers/infiniband/hw/erdma/Kconfig b/drivers/infiniband/hw/erdma/Kconfig
index 169038e3ceb1..745e5551773f 100644
--- a/drivers/infiniband/hw/erdma/Kconfig
+++ b/drivers/infiniband/hw/erdma/Kconfig
@@ -1,11 +1,11 @@
# SPDX-License-Identifier: GPL-2.0-only
config INFINIBAND_ERDMA
tristate "Alibaba Elastic RDMA Adapter (ERDMA) support"
- depends on PCI_MSI && 64BIT
+ depends on PCI_MSI && 64BIT && !CPU_BIG_ENDIAN
depends on INFINIBAND_ADDR_TRANS
depends on INFINIBAND_USER_ACCESS
help
- This is a RDMA/iWarp driver for Alibaba Elastic RDMA Adapter(ERDMA),
+ This is a RDMA driver for Alibaba Elastic RDMA Adapter(ERDMA),
which supports RDMA features in Alibaba cloud environment.
To compile this driver as module, choose M here. The module will be
diff --git a/drivers/infiniband/hw/erdma/erdma.h b/drivers/infiniband/hw/erdma/erdma.h
index 3c166359448d..f7ed1e4cff99 100644
--- a/drivers/infiniband/hw/erdma/erdma.h
+++ b/drivers/infiniband/hw/erdma/erdma.h
@@ -16,7 +16,7 @@
#include "erdma_hw.h"
#define DRV_MODULE_NAME "erdma"
-#define ERDMA_NODE_DESC "Elastic RDMA(iWARP) stack"
+#define ERDMA_NODE_DESC "Elastic RDMA Adapter stack"
struct erdma_eq {
void *qbuf;
@@ -101,8 +101,6 @@ struct erdma_cmdq {
struct erdma_comp_wait *wait_pool;
spinlock_t lock;
- bool use_event;
-
struct erdma_cmdq_sq sq;
struct erdma_cmdq_cq cq;
struct erdma_eq eq;
@@ -148,6 +146,8 @@ struct erdma_devattr {
u32 max_mr;
u32 max_pd;
u32 max_mw;
+ u32 max_gid;
+ u32 max_ah;
u32 local_dma_key;
};
@@ -177,7 +177,8 @@ struct erdma_resource_cb {
enum {
ERDMA_RES_TYPE_PD = 0,
ERDMA_RES_TYPE_STAG_IDX = 1,
- ERDMA_RES_CNT = 2,
+ ERDMA_RES_TYPE_AH = 2,
+ ERDMA_RES_CNT = 3,
};
struct erdma_dev {
@@ -215,6 +216,7 @@ struct erdma_dev {
struct dma_pool *db_pool;
struct dma_pool *resp_pool;
+ enum erdma_proto_type proto;
};
static inline void *get_queue_entry(void *qbuf, u32 idx, u32 depth, u32 shift)
@@ -265,7 +267,7 @@ void erdma_cmdq_destroy(struct erdma_dev *dev);
void erdma_cmdq_build_reqhdr(u64 *hdr, u32 mod, u32 op);
int erdma_post_cmd_wait(struct erdma_cmdq *cmdq, void *req, u32 req_size,
- u64 *resp0, u64 *resp1);
+ u64 *resp0, u64 *resp1, bool sleepable);
void erdma_cmdq_completion_handler(struct erdma_cmdq *cmdq);
int erdma_ceqs_init(struct erdma_dev *dev);
diff --git a/drivers/infiniband/hw/erdma/erdma_cm.c b/drivers/infiniband/hw/erdma/erdma_cm.c
index e349e8d2fb50..f0c8ebf62a05 100644
--- a/drivers/infiniband/hw/erdma/erdma_cm.c
+++ b/drivers/infiniband/hw/erdma/erdma_cm.c
@@ -567,7 +567,8 @@ static int erdma_proc_mpareq(struct erdma_cep *cep)
static int erdma_proc_mpareply(struct erdma_cep *cep)
{
- struct erdma_qp_attrs qp_attrs;
+ enum erdma_qpa_mask_iwarp to_modify_attrs = 0;
+ struct erdma_mod_qp_params_iwarp params;
struct erdma_qp *qp = cep->qp;
struct mpa_rr *rep;
int ret;
@@ -597,26 +598,29 @@ static int erdma_proc_mpareply(struct erdma_cep *cep)
return -EINVAL;
}
- memset(&qp_attrs, 0, sizeof(qp_attrs));
- qp_attrs.irq_size = cep->ird;
- qp_attrs.orq_size = cep->ord;
- qp_attrs.state = ERDMA_QP_STATE_RTS;
+ memset(¶ms, 0, sizeof(params));
+ params.state = ERDMA_QPS_IWARP_RTS;
+ params.irq_size = cep->ird;
+ params.orq_size = cep->ord;
down_write(&qp->state_lock);
- if (qp->attrs.state > ERDMA_QP_STATE_RTR) {
+ if (qp->attrs.iwarp.state > ERDMA_QPS_IWARP_RTR) {
ret = -EINVAL;
up_write(&qp->state_lock);
goto out_err;
}
- qp->attrs.qp_type = ERDMA_QP_ACTIVE;
- if (__mpa_ext_cc(cep->mpa.ext_data.bits) != qp->attrs.cc)
- qp->attrs.cc = COMPROMISE_CC;
+ to_modify_attrs = ERDMA_QPA_IWARP_STATE | ERDMA_QPA_IWARP_LLP_HANDLE |
+ ERDMA_QPA_IWARP_MPA | ERDMA_QPA_IWARP_IRD |
+ ERDMA_QPA_IWARP_ORD;
- ret = erdma_modify_qp_internal(qp, &qp_attrs,
- ERDMA_QP_ATTR_STATE |
- ERDMA_QP_ATTR_LLP_HANDLE |
- ERDMA_QP_ATTR_MPA);
+ params.qp_type = ERDMA_QP_ACTIVE;
+ if (__mpa_ext_cc(cep->mpa.ext_data.bits) != qp->attrs.cc) {
+ to_modify_attrs |= ERDMA_QPA_IWARP_CC;
+ params.cc = COMPROMISE_CC;
+ }
+
+ ret = erdma_modify_qp_state_iwarp(qp, ¶ms, to_modify_attrs);
up_write(&qp->state_lock);
@@ -721,7 +725,7 @@ static int erdma_newconn_connected(struct erdma_cep *cep)
__mpa_rr_set_revision(&cep->mpa.hdr.params.bits, MPA_REVISION_EXT_1);
memcpy(cep->mpa.hdr.key, MPA_KEY_REQ, MPA_KEY_SIZE);
- cep->mpa.ext_data.cookie = cpu_to_be32(cep->qp->attrs.cookie);
+ cep->mpa.ext_data.cookie = cpu_to_be32(cep->qp->attrs.iwarp.cookie);
__mpa_ext_set_cc(&cep->mpa.ext_data.bits, cep->qp->attrs.cc);
ret = erdma_send_mpareqrep(cep, cep->private_data, cep->pd_len);
@@ -1017,10 +1021,9 @@ int erdma_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
if (laddr->sa_family != AF_INET || raddr->sa_family != AF_INET)
return -EAFNOSUPPORT;
- qp = find_qp_by_qpn(dev, params->qpn);
+ qp = erdma_qp_get_by_qpn(dev, params->qpn);
if (!qp)
return -ENOENT;
- erdma_qp_get(qp);
ret = sock_create(AF_INET, SOCK_STREAM, IPPROTO_TCP, &s);
if (ret < 0)
@@ -1125,10 +1128,11 @@ int erdma_connect(struct iw_cm_id *id, struct iw_cm_conn_param *params)
int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
{
- struct erdma_dev *dev = to_edev(id->device);
struct erdma_cep *cep = (struct erdma_cep *)id->provider_data;
+ struct erdma_mod_qp_params_iwarp mod_qp_params;
+ enum erdma_qpa_mask_iwarp to_modify_attrs = 0;
+ struct erdma_dev *dev = to_edev(id->device);
struct erdma_qp *qp;
- struct erdma_qp_attrs qp_attrs;
int ret;
erdma_cep_set_inuse(cep);
@@ -1149,13 +1153,12 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
return -ECONNRESET;
}
- qp = find_qp_by_qpn(dev, params->qpn);
+ qp = erdma_qp_get_by_qpn(dev, params->qpn);
if (!qp)
return -ENOENT;
- erdma_qp_get(qp);
down_write(&qp->state_lock);
- if (qp->attrs.state > ERDMA_QP_STATE_RTR) {
+ if (qp->attrs.iwarp.state > ERDMA_QPS_IWARP_RTR) {
ret = -EINVAL;
up_write(&qp->state_lock);
goto error;
@@ -1180,11 +1183,11 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
cep->cm_id = id;
id->add_ref(id);
- memset(&qp_attrs, 0, sizeof(qp_attrs));
- qp_attrs.orq_size = params->ord;
- qp_attrs.irq_size = params->ird;
+ memset(&mod_qp_params, 0, sizeof(mod_qp_params));
- qp_attrs.state = ERDMA_QP_STATE_RTS;
+ mod_qp_params.irq_size = params->ird;
+ mod_qp_params.orq_size = params->ord;
+ mod_qp_params.state = ERDMA_QPS_IWARP_RTS;
/* Associate QP with CEP */
erdma_cep_get(cep);
@@ -1193,19 +1196,21 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
cep->state = ERDMA_EPSTATE_RDMA_MODE;
- qp->attrs.qp_type = ERDMA_QP_PASSIVE;
- qp->attrs.pd_len = params->private_data_len;
+ mod_qp_params.qp_type = ERDMA_QP_PASSIVE;
+ mod_qp_params.pd_len = params->private_data_len;
- if (qp->attrs.cc != __mpa_ext_cc(cep->mpa.ext_data.bits))
- qp->attrs.cc = COMPROMISE_CC;
+ to_modify_attrs = ERDMA_QPA_IWARP_STATE | ERDMA_QPA_IWARP_ORD |
+ ERDMA_QPA_IWARP_LLP_HANDLE | ERDMA_QPA_IWARP_IRD |
+ ERDMA_QPA_IWARP_MPA;
+
+ if (qp->attrs.cc != __mpa_ext_cc(cep->mpa.ext_data.bits)) {
+ to_modify_attrs |= ERDMA_QPA_IWARP_CC;
+ mod_qp_params.cc = COMPROMISE_CC;
+ }
/* move to rts */
- ret = erdma_modify_qp_internal(qp, &qp_attrs,
- ERDMA_QP_ATTR_STATE |
- ERDMA_QP_ATTR_ORD |
- ERDMA_QP_ATTR_LLP_HANDLE |
- ERDMA_QP_ATTR_IRD |
- ERDMA_QP_ATTR_MPA);
+ ret = erdma_modify_qp_state_iwarp(qp, &mod_qp_params, to_modify_attrs);
+
up_write(&qp->state_lock);
if (ret)
@@ -1213,7 +1218,7 @@ int erdma_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
cep->mpa.ext_data.bits = 0;
__mpa_ext_set_cc(&cep->mpa.ext_data.bits, qp->attrs.cc);
- cep->mpa.ext_data.cookie = cpu_to_be32(cep->qp->attrs.cookie);
+ cep->mpa.ext_data.cookie = cpu_to_be32(cep->qp->attrs.iwarp.cookie);
ret = erdma_send_mpareqrep(cep, params->private_data,
params->private_data_len);
diff --git a/drivers/infiniband/hw/erdma/erdma_cmdq.c b/drivers/infiniband/hw/erdma/erdma_cmdq.c
index a3d8922d1ad1..b867aefe83b2 100644
--- a/drivers/infiniband/hw/erdma/erdma_cmdq.c
+++ b/drivers/infiniband/hw/erdma/erdma_cmdq.c
@@ -182,7 +182,6 @@ int erdma_cmdq_init(struct erdma_dev *dev)
int err;
cmdq->max_outstandings = ERDMA_CMDQ_MAX_OUTSTANDING;
- cmdq->use_event = false;
sema_init(&cmdq->credits, cmdq->max_outstandings);
@@ -223,8 +222,6 @@ int erdma_cmdq_init(struct erdma_dev *dev)
void erdma_finish_cmdq_init(struct erdma_dev *dev)
{
- /* after device init successfully, change cmdq to event mode. */
- dev->cmdq.use_event = true;
arm_cmdq_cq(&dev->cmdq);
}
@@ -312,8 +309,7 @@ static int erdma_poll_single_cmd_completion(struct erdma_cmdq *cmdq)
/* Copy 16B comp data after cqe hdr to outer */
be32_to_cpu_array(comp_wait->comp_data, cqe + 2, 4);
- if (cmdq->use_event)
- complete(&comp_wait->wait_event);
+ complete(&comp_wait->wait_event);
return 0;
}
@@ -332,9 +328,6 @@ static void erdma_polling_cmd_completions(struct erdma_cmdq *cmdq)
if (erdma_poll_single_cmd_completion(cmdq))
break;
- if (comp_num && cmdq->use_event)
- arm_cmdq_cq(cmdq);
-
spin_unlock_irqrestore(&cmdq->cq.lock, flags);
}
@@ -342,8 +335,7 @@ void erdma_cmdq_completion_handler(struct erdma_cmdq *cmdq)
{
int got_event = 0;
- if (!test_bit(ERDMA_CMDQ_STATE_OK_BIT, &cmdq->state) ||
- !cmdq->use_event)
+ if (!test_bit(ERDMA_CMDQ_STATE_OK_BIT, &cmdq->state))
return;
while (get_next_valid_eqe(&cmdq->eq)) {
@@ -354,6 +346,7 @@ void erdma_cmdq_completion_handler(struct erdma_cmdq *cmdq)
if (got_event) {
cmdq->cq.cmdsn++;
erdma_polling_cmd_completions(cmdq);
+ arm_cmdq_cq(cmdq);
}
notify_eq(&cmdq->eq);
@@ -372,7 +365,7 @@ static int erdma_poll_cmd_completion(struct erdma_comp_wait *comp_ctx,
if (time_is_before_jiffies(comp_timeout))
return -ETIME;
- msleep(20);
+ udelay(20);
}
return 0;
@@ -403,7 +396,7 @@ void erdma_cmdq_build_reqhdr(u64 *hdr, u32 mod, u32 op)
}
int erdma_post_cmd_wait(struct erdma_cmdq *cmdq, void *req, u32 req_size,
- u64 *resp0, u64 *resp1)
+ u64 *resp0, u64 *resp1, bool sleepable)
{
struct erdma_comp_wait *comp_wait;
int ret;
@@ -411,7 +404,12 @@ int erdma_post_cmd_wait(struct erdma_cmdq *cmdq, void *req, u32 req_size,
if (!test_bit(ERDMA_CMDQ_STATE_OK_BIT, &cmdq->state))
return -ENODEV;
- down(&cmdq->credits);
+ if (!sleepable) {
+ while (down_trylock(&cmdq->credits))
+ ;
+ } else {
+ down(&cmdq->credits);
+ }
comp_wait = get_comp_wait(cmdq);
if (IS_ERR(comp_wait)) {
@@ -425,7 +423,7 @@ int erdma_post_cmd_wait(struct erdma_cmdq *cmdq, void *req, u32 req_size,
push_cmdq_sqe(cmdq, req, req_size, comp_wait);
spin_unlock(&cmdq->sq.lock);
- if (cmdq->use_event)
+ if (sleepable)
ret = erdma_wait_cmd_completion(comp_wait, cmdq,
ERDMA_CMDQ_TIMEOUT_MS);
else
diff --git a/drivers/infiniband/hw/erdma/erdma_cq.c b/drivers/infiniband/hw/erdma/erdma_cq.c
index 70f89f0162aa..eada882472a3 100644
--- a/drivers/infiniband/hw/erdma/erdma_cq.c
+++ b/drivers/infiniband/hw/erdma/erdma_cq.c
@@ -201,3 +201,48 @@ int erdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
return npolled;
}
+
+void erdma_remove_cqes_of_qp(struct ib_cq *ibcq, u32 qpn)
+{
+ struct erdma_cq *cq = to_ecq(ibcq);
+ struct erdma_cqe *cqe, *dst_cqe;
+ u32 prev_cq_ci, cur_cq_ci;
+ u32 ncqe = 0, nqp_cqe = 0;
+ unsigned long flags;
+ u8 owner;
+
+ spin_lock_irqsave(&cq->kern_cq.lock, flags);
+
+ prev_cq_ci = cq->kern_cq.ci;
+
+ while (ncqe < cq->depth && (cqe = get_next_valid_cqe(cq)) != NULL) {
+ ++cq->kern_cq.ci;
+ ++ncqe;
+ }
+
+ while (ncqe > 0) {
+ cur_cq_ci = prev_cq_ci + ncqe - 1;
+ cqe = get_queue_entry(cq->kern_cq.qbuf, cur_cq_ci, cq->depth,
+ CQE_SHIFT);
+
+ if (be32_to_cpu(cqe->qpn) == qpn) {
+ ++nqp_cqe;
+ } else if (nqp_cqe) {
+ dst_cqe = get_queue_entry(cq->kern_cq.qbuf,
+ cur_cq_ci + nqp_cqe,
+ cq->depth, CQE_SHIFT);
+ owner = FIELD_GET(ERDMA_CQE_HDR_OWNER_MASK,
+ be32_to_cpu(dst_cqe->hdr));
+ cqe->hdr = cpu_to_be32(
+ (be32_to_cpu(cqe->hdr) &
+ ~ERDMA_CQE_HDR_OWNER_MASK) |
+ FIELD_PREP(ERDMA_CQE_HDR_OWNER_MASK, owner));
+ memcpy(dst_cqe, cqe, sizeof(*cqe));
+ }
+
+ --ncqe;
+ }
+
+ cq->kern_cq.ci = prev_cq_ci + nqp_cqe;
+ spin_unlock_irqrestore(&cq->kern_cq.lock, flags);
+}
diff --git a/drivers/infiniband/hw/erdma/erdma_eq.c b/drivers/infiniband/hw/erdma/erdma_eq.c
index 9a72fec6d5cc..e4161e85422c 100644
--- a/drivers/infiniband/hw/erdma/erdma_eq.c
+++ b/drivers/infiniband/hw/erdma/erdma_eq.c
@@ -52,7 +52,7 @@ void erdma_aeq_event_handler(struct erdma_dev *dev)
if (FIELD_GET(ERDMA_AEQE_HDR_TYPE_MASK,
le32_to_cpu(aeqe->hdr)) == ERDMA_AE_TYPE_CQ_ERR) {
cqn = le32_to_cpu(aeqe->event_data0);
- cq = find_cq_by_cqn(dev, cqn);
+ cq = erdma_cq_get_by_cqn(dev, cqn);
if (!cq)
continue;
@@ -62,9 +62,10 @@ void erdma_aeq_event_handler(struct erdma_dev *dev)
if (cq->ibcq.event_handler)
cq->ibcq.event_handler(&event,
cq->ibcq.cq_context);
+ erdma_cq_put(cq);
} else {
qpn = le32_to_cpu(aeqe->event_data0);
- qp = find_qp_by_qpn(dev, qpn);
+ qp = erdma_qp_get_by_qpn(dev, qpn);
if (!qp)
continue;
@@ -74,6 +75,7 @@ void erdma_aeq_event_handler(struct erdma_dev *dev)
if (qp->ibqp.event_handler)
qp->ibqp.event_handler(&event,
qp->ibqp.qp_context);
+ erdma_qp_put(qp);
}
}
@@ -157,7 +159,7 @@ void erdma_ceq_completion_handler(struct erdma_eq_cb *ceq_cb)
poll_cnt++;
cqn = FIELD_GET(ERDMA_CEQE_HDR_CQN_MASK, READ_ONCE(*ceqe));
- cq = find_cq_by_cqn(dev, cqn);
+ cq = erdma_cq_get_by_cqn(dev, cqn);
if (!cq)
continue;
@@ -166,6 +168,7 @@ void erdma_ceq_completion_handler(struct erdma_eq_cb *ceq_cb)
if (cq->ibcq.comp_handler)
cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
+ erdma_cq_put(cq);
}
notify_eq(&ceq_cb->eq);
@@ -219,6 +222,7 @@ static void erdma_free_ceq_irq(struct erdma_dev *dev, u16 ceqn)
irq_set_affinity_hint(eqc->irq.msix_vector, NULL);
free_irq(eqc->irq.msix_vector, eqc);
+ tasklet_kill(&eqc->tasklet);
}
static int create_eq_cmd(struct erdma_dev *dev, u32 eqn, struct erdma_eq *eq)
@@ -236,7 +240,8 @@ static int create_eq_cmd(struct erdma_dev *dev, u32 eqn, struct erdma_eq *eq)
req.db_dma_addr_l = lower_32_bits(eq->dbrec_dma);
req.db_dma_addr_h = upper_32_bits(eq->dbrec_dma);
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ false);
}
static int erdma_ceq_init_one(struct erdma_dev *dev, u16 ceqn)
@@ -278,7 +283,8 @@ static void erdma_ceq_uninit_one(struct erdma_dev *dev, u16 ceqn)
req.qtype = ERDMA_EQ_TYPE_CEQ;
req.vector_idx = ceqn + 1;
- err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ false);
if (err)
return;
diff --git a/drivers/infiniband/hw/erdma/erdma_hw.h b/drivers/infiniband/hw/erdma/erdma_hw.h
index 05978f3b1475..809e77dde271 100644
--- a/drivers/infiniband/hw/erdma/erdma_hw.h
+++ b/drivers/infiniband/hw/erdma/erdma_hw.h
@@ -9,6 +9,7 @@
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/if_ether.h>
/* PCIe device related definition. */
#define ERDMA_PCI_WIDTH 64
@@ -21,8 +22,21 @@
#define ERDMA_NUM_MSIX_VEC 32U
#define ERDMA_MSIX_VECTOR_CMDQ 0
+/* RoCEv2 related */
+#define ERDMA_ROCEV2_GID_SIZE 16
+#define ERDMA_MAX_PKEYS 1
+#define ERDMA_DEFAULT_PKEY 0xFFFF
+
+/* erdma device protocol type */
+enum erdma_proto_type {
+ ERDMA_PROTO_IWARP = 0,
+ ERDMA_PROTO_ROCEV2 = 1,
+ ERDMA_PROTO_COUNT = 2,
+};
+
/* PCIe Bar0 Registers. */
#define ERDMA_REGS_VERSION_REG 0x0
+#define ERDMA_REGS_DEV_PROTO_REG 0xC
#define ERDMA_REGS_DEV_CTRL_REG 0x10
#define ERDMA_REGS_DEV_ST_REG 0x14
#define ERDMA_REGS_NETDEV_MAC_L_REG 0x18
@@ -136,7 +150,11 @@ enum CMDQ_RDMA_OPCODE {
CMDQ_OPCODE_DESTROY_CQ = 5,
CMDQ_OPCODE_REFLUSH = 6,
CMDQ_OPCODE_REG_MR = 8,
- CMDQ_OPCODE_DEREG_MR = 9
+ CMDQ_OPCODE_DEREG_MR = 9,
+ CMDQ_OPCODE_SET_GID = 14,
+ CMDQ_OPCODE_CREATE_AH = 15,
+ CMDQ_OPCODE_DESTROY_AH = 16,
+ CMDQ_OPCODE_QUERY_QP = 17,
};
enum CMDQ_COMMON_OPCODE {
@@ -284,6 +302,36 @@ struct erdma_cmdq_dereg_mr_req {
u32 cfg;
};
+/* create_av cfg0 */
+#define ERDMA_CMD_CREATE_AV_FL_MASK GENMASK(19, 0)
+#define ERDMA_CMD_CREATE_AV_NTYPE_MASK BIT(20)
+
+struct erdma_av_cfg {
+ u32 cfg0;
+ u8 traffic_class;
+ u8 hop_limit;
+ u8 sl;
+ u8 rsvd;
+ u16 udp_sport;
+ u16 sgid_index;
+ u8 dmac[ETH_ALEN];
+ u8 padding[2];
+ u8 dgid[ERDMA_ROCEV2_GID_SIZE];
+};
+
+struct erdma_cmdq_create_ah_req {
+ u64 hdr;
+ u32 pdn;
+ u32 ahn;
+ struct erdma_av_cfg av_cfg;
+};
+
+struct erdma_cmdq_destroy_ah_req {
+ u64 hdr;
+ u32 pdn;
+ u32 ahn;
+};
+
/* modify qp cfg */
#define ERDMA_CMD_MODIFY_QP_STATE_MASK GENMASK(31, 24)
#define ERDMA_CMD_MODIFY_QP_CC_MASK GENMASK(23, 20)
@@ -301,6 +349,31 @@ struct erdma_cmdq_modify_qp_req {
u32 recv_nxt;
};
+/* modify qp cfg1 for roce device */
+#define ERDMA_CMD_MODIFY_QP_DQPN_MASK GENMASK(19, 0)
+
+struct erdma_cmdq_mod_qp_req_rocev2 {
+ u64 hdr;
+ u32 cfg0;
+ u32 cfg1;
+ u32 attr_mask;
+ u32 qkey;
+ u32 rq_psn;
+ u32 sq_psn;
+ struct erdma_av_cfg av_cfg;
+};
+
+/* query qp response mask */
+#define ERDMA_CMD_QUERY_QP_RESP_SQ_PSN_MASK GENMASK_ULL(23, 0)
+#define ERDMA_CMD_QUERY_QP_RESP_RQ_PSN_MASK GENMASK_ULL(47, 24)
+#define ERDMA_CMD_QUERY_QP_RESP_QP_STATE_MASK GENMASK_ULL(55, 48)
+#define ERDMA_CMD_QUERY_QP_RESP_SQ_DRAINING_MASK GENMASK_ULL(56, 56)
+
+struct erdma_cmdq_query_qp_req_rocev2 {
+ u64 hdr;
+ u32 qpn;
+};
+
/* create qp cfg0 */
#define ERDMA_CMD_CREATE_QP_SQ_DEPTH_MASK GENMASK(31, 20)
#define ERDMA_CMD_CREATE_QP_QPN_MASK GENMASK(19, 0)
@@ -394,10 +467,33 @@ struct erdma_cmdq_query_stats_resp {
u64 rx_pps_meter_drop_packets_cnt;
};
+enum erdma_network_type {
+ ERDMA_NETWORK_TYPE_IPV4 = 0,
+ ERDMA_NETWORK_TYPE_IPV6 = 1,
+};
+
+enum erdma_set_gid_op {
+ ERDMA_SET_GID_OP_ADD = 0,
+ ERDMA_SET_GID_OP_DEL = 1,
+};
+
+/* set gid cfg */
+#define ERDMA_CMD_SET_GID_SGID_IDX_MASK GENMASK(15, 0)
+#define ERDMA_CMD_SET_GID_NTYPE_MASK BIT(16)
+#define ERDMA_CMD_SET_GID_OP_MASK BIT(31)
+
+struct erdma_cmdq_set_gid_req {
+ u64 hdr;
+ u32 cfg;
+ u8 gid[ERDMA_ROCEV2_GID_SIZE];
+};
+
/* cap qword 0 definition */
+#define ERDMA_CMD_DEV_CAP_MAX_GID_MASK GENMASK_ULL(51, 48)
#define ERDMA_CMD_DEV_CAP_MAX_CQE_MASK GENMASK_ULL(47, 40)
#define ERDMA_CMD_DEV_CAP_FLAGS_MASK GENMASK_ULL(31, 24)
#define ERDMA_CMD_DEV_CAP_MAX_RECV_WR_MASK GENMASK_ULL(23, 16)
+#define ERDMA_CMD_DEV_CAP_MAX_AH_MASK GENMASK_ULL(15, 8)
#define ERDMA_CMD_DEV_CAP_MAX_MR_SIZE_MASK GENMASK_ULL(7, 0)
/* cap qword 1 definition */
diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c
index 62f497a71004..37723054f064 100644
--- a/drivers/infiniband/hw/erdma/erdma_main.c
+++ b/drivers/infiniband/hw/erdma/erdma_main.c
@@ -172,6 +172,8 @@ static int erdma_device_init(struct erdma_dev *dev, struct pci_dev *pdev)
{
int ret;
+ dev->proto = erdma_reg_read32(dev, ERDMA_REGS_DEV_PROTO_REG);
+
dev->resp_pool = dma_pool_create("erdma_resp_pool", &pdev->dev,
ERDMA_HW_RESP_SIZE, ERDMA_HW_RESP_SIZE,
0);
@@ -390,7 +392,7 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev)
CMDQ_OPCODE_QUERY_DEVICE);
err = erdma_post_cmd_wait(&dev->cmdq, &req_hdr, sizeof(req_hdr), &cap0,
- &cap1);
+ &cap1, true);
if (err)
return err;
@@ -398,6 +400,8 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev)
dev->attrs.max_mr_size = 1ULL << ERDMA_GET_CAP(MAX_MR_SIZE, cap0);
dev->attrs.max_mw = 1 << ERDMA_GET_CAP(MAX_MW, cap1);
dev->attrs.max_recv_wr = 1 << ERDMA_GET_CAP(MAX_RECV_WR, cap0);
+ dev->attrs.max_gid = 1 << ERDMA_GET_CAP(MAX_GID, cap0);
+ dev->attrs.max_ah = 1 << ERDMA_GET_CAP(MAX_AH, cap0);
dev->attrs.local_dma_key = ERDMA_GET_CAP(DMA_LOCAL_KEY, cap1);
dev->attrs.cc = ERDMA_GET_CAP(DEFAULT_CC, cap1);
dev->attrs.max_qp = ERDMA_NQP_PER_QBLOCK * ERDMA_GET_CAP(QBLOCK, cap1);
@@ -415,12 +419,13 @@ static int erdma_dev_attrs_init(struct erdma_dev *dev)
dev->res_cb[ERDMA_RES_TYPE_PD].max_cap = ERDMA_MAX_PD;
dev->res_cb[ERDMA_RES_TYPE_STAG_IDX].max_cap = dev->attrs.max_mr;
+ dev->res_cb[ERDMA_RES_TYPE_AH].max_cap = dev->attrs.max_ah;
erdma_cmdq_build_reqhdr(&req_hdr, CMDQ_SUBMOD_COMMON,
CMDQ_OPCODE_QUERY_FW_INFO);
err = erdma_post_cmd_wait(&dev->cmdq, &req_hdr, sizeof(req_hdr), &cap0,
- &cap1);
+ &cap1, true);
if (!err)
dev->attrs.fw_version =
FIELD_GET(ERDMA_CMD_INFO0_FW_VER_MASK, cap0);
@@ -441,7 +446,8 @@ static int erdma_device_config(struct erdma_dev *dev)
req.cfg = FIELD_PREP(ERDMA_CMD_CONFIG_DEVICE_PGSHIFT_MASK, PAGE_SHIFT) |
FIELD_PREP(ERDMA_CMD_CONFIG_DEVICE_PS_EN_MASK, 1);
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
}
static int erdma_res_cb_init(struct erdma_dev *dev)
@@ -474,6 +480,26 @@ static void erdma_res_cb_free(struct erdma_dev *dev)
bitmap_free(dev->res_cb[i].bitmap);
}
+static const struct ib_device_ops erdma_device_ops_rocev2 = {
+ .get_link_layer = erdma_get_link_layer,
+ .add_gid = erdma_add_gid,
+ .del_gid = erdma_del_gid,
+ .query_pkey = erdma_query_pkey,
+ .create_ah = erdma_create_ah,
+ .destroy_ah = erdma_destroy_ah,
+};
+
+static const struct ib_device_ops erdma_device_ops_iwarp = {
+ .iw_accept = erdma_accept,
+ .iw_add_ref = erdma_qp_get_ref,
+ .iw_connect = erdma_connect,
+ .iw_create_listen = erdma_create_listen,
+ .iw_destroy_listen = erdma_destroy_listen,
+ .iw_get_qp = erdma_get_ibqp,
+ .iw_reject = erdma_reject,
+ .iw_rem_ref = erdma_qp_put_ref,
+};
+
static const struct ib_device_ops erdma_device_ops = {
.owner = THIS_MODULE,
.driver_id = RDMA_DRIVER_ERDMA,
@@ -494,18 +520,9 @@ static const struct ib_device_ops erdma_device_ops = {
.get_dma_mr = erdma_get_dma_mr,
.get_hw_stats = erdma_get_hw_stats,
.get_port_immutable = erdma_get_port_immutable,
- .iw_accept = erdma_accept,
- .iw_add_ref = erdma_qp_get_ref,
- .iw_connect = erdma_connect,
- .iw_create_listen = erdma_create_listen,
- .iw_destroy_listen = erdma_destroy_listen,
- .iw_get_qp = erdma_get_ibqp,
- .iw_reject = erdma_reject,
- .iw_rem_ref = erdma_qp_put_ref,
.map_mr_sg = erdma_map_mr_sg,
.mmap = erdma_mmap,
.mmap_free = erdma_mmap_free,
- .modify_qp = erdma_modify_qp,
.post_recv = erdma_post_recv,
.post_send = erdma_post_send,
.poll_cq = erdma_poll_cq,
@@ -515,6 +532,7 @@ static const struct ib_device_ops erdma_device_ops = {
.query_qp = erdma_query_qp,
.req_notify_cq = erdma_req_notify_cq,
.reg_user_mr = erdma_reg_user_mr,
+ .modify_qp = erdma_modify_qp,
INIT_RDMA_OBJ_SIZE(ib_cq, erdma_cq, ibcq),
INIT_RDMA_OBJ_SIZE(ib_pd, erdma_pd, ibpd),
@@ -537,7 +555,14 @@ static int erdma_ib_device_add(struct pci_dev *pdev)
if (ret)
return ret;
- ibdev->node_type = RDMA_NODE_RNIC;
+ if (erdma_device_iwarp(dev)) {
+ ibdev->node_type = RDMA_NODE_RNIC;
+ ib_set_device_ops(ibdev, &erdma_device_ops_iwarp);
+ } else {
+ ibdev->node_type = RDMA_NODE_IB_CA;
+ ib_set_device_ops(ibdev, &erdma_device_ops_rocev2);
+ }
+
memcpy(ibdev->node_desc, ERDMA_NODE_DESC, sizeof(ERDMA_NODE_DESC));
/*
diff --git a/drivers/infiniband/hw/erdma/erdma_qp.c b/drivers/infiniband/hw/erdma/erdma_qp.c
index 4dfab0c49e51..b1bba54d8735 100644
--- a/drivers/infiniband/hw/erdma/erdma_qp.c
+++ b/drivers/infiniband/hw/erdma/erdma_qp.c
@@ -11,20 +11,20 @@
void erdma_qp_llp_close(struct erdma_qp *qp)
{
- struct erdma_qp_attrs qp_attrs;
+ struct erdma_mod_qp_params_iwarp params;
down_write(&qp->state_lock);
- switch (qp->attrs.state) {
- case ERDMA_QP_STATE_RTS:
- case ERDMA_QP_STATE_RTR:
- case ERDMA_QP_STATE_IDLE:
- case ERDMA_QP_STATE_TERMINATE:
- qp_attrs.state = ERDMA_QP_STATE_CLOSING;
- erdma_modify_qp_internal(qp, &qp_attrs, ERDMA_QP_ATTR_STATE);
+ switch (qp->attrs.iwarp.state) {
+ case ERDMA_QPS_IWARP_RTS:
+ case ERDMA_QPS_IWARP_RTR:
+ case ERDMA_QPS_IWARP_IDLE:
+ case ERDMA_QPS_IWARP_TERMINATE:
+ params.state = ERDMA_QPS_IWARP_CLOSING;
+ erdma_modify_qp_state_iwarp(qp, ¶ms, ERDMA_QPA_IWARP_STATE);
break;
- case ERDMA_QP_STATE_CLOSING:
- qp->attrs.state = ERDMA_QP_STATE_IDLE;
+ case ERDMA_QPS_IWARP_CLOSING:
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_IDLE;
break;
default:
break;
@@ -48,9 +48,10 @@ struct ib_qp *erdma_get_ibqp(struct ib_device *ibdev, int id)
return NULL;
}
-static int erdma_modify_qp_state_to_rts(struct erdma_qp *qp,
- struct erdma_qp_attrs *attrs,
- enum erdma_qp_attr_mask mask)
+static int
+erdma_modify_qp_state_to_rts(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_iwarp *params,
+ enum erdma_qpa_mask_iwarp mask)
{
int ret;
struct erdma_dev *dev = qp->dev;
@@ -59,12 +60,15 @@ static int erdma_modify_qp_state_to_rts(struct erdma_qp *qp,
struct erdma_cep *cep = qp->cep;
struct sockaddr_storage local_addr, remote_addr;
- if (!(mask & ERDMA_QP_ATTR_LLP_HANDLE))
+ if (!(mask & ERDMA_QPA_IWARP_LLP_HANDLE))
return -EINVAL;
- if (!(mask & ERDMA_QP_ATTR_MPA))
+ if (!(mask & ERDMA_QPA_IWARP_MPA))
return -EINVAL;
+ if (!(mask & ERDMA_QPA_IWARP_CC))
+ params->cc = qp->attrs.cc;
+
ret = getname_local(cep->sock, &local_addr);
if (ret < 0)
return ret;
@@ -73,18 +77,16 @@ static int erdma_modify_qp_state_to_rts(struct erdma_qp *qp,
if (ret < 0)
return ret;
- qp->attrs.state = ERDMA_QP_STATE_RTS;
-
tp = tcp_sk(qp->cep->sock->sk);
erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
CMDQ_OPCODE_MODIFY_QP);
- req.cfg = FIELD_PREP(ERDMA_CMD_MODIFY_QP_STATE_MASK, qp->attrs.state) |
- FIELD_PREP(ERDMA_CMD_MODIFY_QP_CC_MASK, qp->attrs.cc) |
+ req.cfg = FIELD_PREP(ERDMA_CMD_MODIFY_QP_STATE_MASK, params->state) |
+ FIELD_PREP(ERDMA_CMD_MODIFY_QP_CC_MASK, params->cc) |
FIELD_PREP(ERDMA_CMD_MODIFY_QP_QPN_MASK, QP_ID(qp));
- req.cookie = be32_to_cpu(qp->cep->mpa.ext_data.cookie);
+ req.cookie = be32_to_cpu(cep->mpa.ext_data.cookie);
req.dip = to_sockaddr_in(remote_addr).sin_addr.s_addr;
req.sip = to_sockaddr_in(local_addr).sin_addr.s_addr;
req.dport = to_sockaddr_in(remote_addr).sin_port;
@@ -92,33 +94,57 @@ static int erdma_modify_qp_state_to_rts(struct erdma_qp *qp,
req.send_nxt = tp->snd_nxt;
/* rsvd tcp seq for mpa-rsp in server. */
- if (qp->attrs.qp_type == ERDMA_QP_PASSIVE)
- req.send_nxt += MPA_DEFAULT_HDR_LEN + qp->attrs.pd_len;
+ if (params->qp_type == ERDMA_QP_PASSIVE)
+ req.send_nxt += MPA_DEFAULT_HDR_LEN + params->pd_len;
req.recv_nxt = tp->rcv_nxt;
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+ if (ret)
+ return ret;
+
+ if (mask & ERDMA_QPA_IWARP_IRD)
+ qp->attrs.irq_size = params->irq_size;
+
+ if (mask & ERDMA_QPA_IWARP_ORD)
+ qp->attrs.orq_size = params->orq_size;
+
+ if (mask & ERDMA_QPA_IWARP_CC)
+ qp->attrs.cc = params->cc;
+
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_RTS;
+
+ return 0;
}
-static int erdma_modify_qp_state_to_stop(struct erdma_qp *qp,
- struct erdma_qp_attrs *attrs,
- enum erdma_qp_attr_mask mask)
+static int
+erdma_modify_qp_state_to_stop(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_iwarp *params,
+ enum erdma_qpa_mask_iwarp mask)
{
struct erdma_dev *dev = qp->dev;
struct erdma_cmdq_modify_qp_req req;
-
- qp->attrs.state = attrs->state;
+ int ret;
erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
CMDQ_OPCODE_MODIFY_QP);
- req.cfg = FIELD_PREP(ERDMA_CMD_MODIFY_QP_STATE_MASK, attrs->state) |
+ req.cfg = FIELD_PREP(ERDMA_CMD_MODIFY_QP_STATE_MASK, params->state) |
FIELD_PREP(ERDMA_CMD_MODIFY_QP_QPN_MASK, QP_ID(qp));
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+ if (ret)
+ return ret;
+
+ qp->attrs.iwarp.state = params->state;
+
+ return 0;
}
-int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,
- enum erdma_qp_attr_mask mask)
+int erdma_modify_qp_state_iwarp(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_iwarp *params,
+ int mask)
{
bool need_reflush = false;
int drop_conn, ret = 0;
@@ -126,31 +152,31 @@ int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,
if (!mask)
return 0;
- if (!(mask & ERDMA_QP_ATTR_STATE))
+ if (!(mask & ERDMA_QPA_IWARP_STATE))
return 0;
- switch (qp->attrs.state) {
- case ERDMA_QP_STATE_IDLE:
- case ERDMA_QP_STATE_RTR:
- if (attrs->state == ERDMA_QP_STATE_RTS) {
- ret = erdma_modify_qp_state_to_rts(qp, attrs, mask);
- } else if (attrs->state == ERDMA_QP_STATE_ERROR) {
- qp->attrs.state = ERDMA_QP_STATE_ERROR;
+ switch (qp->attrs.iwarp.state) {
+ case ERDMA_QPS_IWARP_IDLE:
+ case ERDMA_QPS_IWARP_RTR:
+ if (params->state == ERDMA_QPS_IWARP_RTS) {
+ ret = erdma_modify_qp_state_to_rts(qp, params, mask);
+ } else if (params->state == ERDMA_QPS_IWARP_ERROR) {
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_ERROR;
need_reflush = true;
if (qp->cep) {
erdma_cep_put(qp->cep);
qp->cep = NULL;
}
- ret = erdma_modify_qp_state_to_stop(qp, attrs, mask);
+ ret = erdma_modify_qp_state_to_stop(qp, params, mask);
}
break;
- case ERDMA_QP_STATE_RTS:
+ case ERDMA_QPS_IWARP_RTS:
drop_conn = 0;
- if (attrs->state == ERDMA_QP_STATE_CLOSING ||
- attrs->state == ERDMA_QP_STATE_TERMINATE ||
- attrs->state == ERDMA_QP_STATE_ERROR) {
- ret = erdma_modify_qp_state_to_stop(qp, attrs, mask);
+ if (params->state == ERDMA_QPS_IWARP_CLOSING ||
+ params->state == ERDMA_QPS_IWARP_TERMINATE ||
+ params->state == ERDMA_QPS_IWARP_ERROR) {
+ ret = erdma_modify_qp_state_to_stop(qp, params, mask);
drop_conn = 1;
need_reflush = true;
}
@@ -159,17 +185,17 @@ int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,
erdma_qp_cm_drop(qp);
break;
- case ERDMA_QP_STATE_TERMINATE:
- if (attrs->state == ERDMA_QP_STATE_ERROR)
- qp->attrs.state = ERDMA_QP_STATE_ERROR;
+ case ERDMA_QPS_IWARP_TERMINATE:
+ if (params->state == ERDMA_QPS_IWARP_ERROR)
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_ERROR;
break;
- case ERDMA_QP_STATE_CLOSING:
- if (attrs->state == ERDMA_QP_STATE_IDLE) {
- qp->attrs.state = ERDMA_QP_STATE_IDLE;
- } else if (attrs->state == ERDMA_QP_STATE_ERROR) {
- ret = erdma_modify_qp_state_to_stop(qp, attrs, mask);
- qp->attrs.state = ERDMA_QP_STATE_ERROR;
- } else if (attrs->state != ERDMA_QP_STATE_CLOSING) {
+ case ERDMA_QPS_IWARP_CLOSING:
+ if (params->state == ERDMA_QPS_IWARP_IDLE) {
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_IDLE;
+ } else if (params->state == ERDMA_QPS_IWARP_ERROR) {
+ ret = erdma_modify_qp_state_to_stop(qp, params, mask);
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_ERROR;
+ } else if (params->state != ERDMA_QPS_IWARP_CLOSING) {
return -ECONNABORTED;
}
break;
@@ -186,6 +212,98 @@ int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,
return ret;
}
+static int modify_qp_cmd_rocev2(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_rocev2 *params,
+ enum erdma_qpa_mask_rocev2 attr_mask)
+{
+ struct erdma_cmdq_mod_qp_req_rocev2 req;
+
+ memset(&req, 0, sizeof(req));
+
+ erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
+ CMDQ_OPCODE_MODIFY_QP);
+
+ req.cfg0 = FIELD_PREP(ERDMA_CMD_MODIFY_QP_QPN_MASK, QP_ID(qp));
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_STATE)
+ req.cfg0 |= FIELD_PREP(ERDMA_CMD_MODIFY_QP_STATE_MASK,
+ params->state);
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_DST_QPN)
+ req.cfg1 = FIELD_PREP(ERDMA_CMD_MODIFY_QP_DQPN_MASK,
+ params->dst_qpn);
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_QKEY)
+ req.qkey = params->qkey;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_AV)
+ erdma_set_av_cfg(&req.av_cfg, ¶ms->av);
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_SQ_PSN)
+ req.sq_psn = params->sq_psn;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_RQ_PSN)
+ req.rq_psn = params->rq_psn;
+
+ req.attr_mask = attr_mask;
+
+ return erdma_post_cmd_wait(&qp->dev->cmdq, &req, sizeof(req), NULL,
+ NULL, true);
+}
+
+static void erdma_reset_qp(struct erdma_qp *qp)
+{
+ qp->kern_qp.sq_pi = 0;
+ qp->kern_qp.sq_ci = 0;
+ qp->kern_qp.rq_pi = 0;
+ qp->kern_qp.rq_ci = 0;
+ memset(qp->kern_qp.swr_tbl, 0, qp->attrs.sq_size * sizeof(u64));
+ memset(qp->kern_qp.rwr_tbl, 0, qp->attrs.rq_size * sizeof(u64));
+ memset(qp->kern_qp.sq_buf, 0, qp->attrs.sq_size << SQEBB_SHIFT);
+ memset(qp->kern_qp.rq_buf, 0, qp->attrs.rq_size << RQE_SHIFT);
+ erdma_remove_cqes_of_qp(&qp->scq->ibcq, QP_ID(qp));
+ if (qp->rcq != qp->scq)
+ erdma_remove_cqes_of_qp(&qp->rcq->ibcq, QP_ID(qp));
+}
+
+int erdma_modify_qp_state_rocev2(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_rocev2 *params,
+ int attr_mask)
+{
+ struct erdma_dev *dev = to_edev(qp->ibqp.device);
+ int ret;
+
+ ret = modify_qp_cmd_rocev2(qp, params, attr_mask);
+ if (ret)
+ return ret;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_STATE)
+ qp->attrs.rocev2.state = params->state;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_QKEY)
+ qp->attrs.rocev2.qkey = params->qkey;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_DST_QPN)
+ qp->attrs.rocev2.dst_qpn = params->dst_qpn;
+
+ if (attr_mask & ERDMA_QPA_ROCEV2_AV)
+ memcpy(&qp->attrs.rocev2.av, ¶ms->av,
+ sizeof(struct erdma_av));
+
+ if (rdma_is_kernel_res(&qp->ibqp.res) &&
+ params->state == ERDMA_QPS_ROCEV2_RESET)
+ erdma_reset_qp(qp);
+
+ if (rdma_is_kernel_res(&qp->ibqp.res) &&
+ params->state == ERDMA_QPS_ROCEV2_ERROR) {
+ qp->flags |= ERDMA_QP_IN_FLUSHING;
+ mod_delayed_work(dev->reflush_wq, &qp->reflush_dwork,
+ usecs_to_jiffies(100));
+ }
+
+ return 0;
+}
+
static void erdma_qp_safe_free(struct kref *ref)
{
struct erdma_qp *qp = container_of(ref, struct erdma_qp, ref);
diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c
index bf5627e3f237..057efff532b4 100644
--- a/drivers/infiniband/hw/erdma/erdma_verbs.c
+++ b/drivers/infiniband/hw/erdma/erdma_verbs.c
@@ -119,10 +119,10 @@ static int create_qp_cmd(struct erdma_ucontext *uctx, struct erdma_qp *qp)
}
}
- err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), &resp0,
- &resp1);
- if (!err)
- qp->attrs.cookie =
+ err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), &resp0, &resp1,
+ true);
+ if (!err && erdma_device_iwarp(dev))
+ qp->attrs.iwarp.cookie =
FIELD_GET(ERDMA_CMDQ_CREATE_QP_RESP_COOKIE_MASK, resp0);
return err;
@@ -178,7 +178,8 @@ static int regmr_cmd(struct erdma_dev *dev, struct erdma_mr *mr)
}
post_cmd:
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
}
static int create_cq_cmd(struct erdma_ucontext *uctx, struct erdma_cq *cq)
@@ -240,7 +241,8 @@ static int create_cq_cmd(struct erdma_ucontext *uctx, struct erdma_cq *cq)
}
}
- return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
}
static int erdma_alloc_idx(struct erdma_resource_cb *res_cb)
@@ -336,6 +338,11 @@ int erdma_query_device(struct ib_device *ibdev, struct ib_device_attr *attr,
attr->max_fast_reg_page_list_len = ERDMA_MAX_FRMR_PA;
attr->page_size_cap = ERDMA_PAGE_SIZE_SUPPORT;
+ if (erdma_device_rocev2(dev)) {
+ attr->max_pkeys = ERDMA_MAX_PKEYS;
+ attr->max_ah = dev->attrs.max_ah;
+ }
+
if (dev->attrs.cap_flags & ERDMA_DEV_CAP_FLAGS_ATOMIC)
attr->atomic_cap = IB_ATOMIC_GLOB;
@@ -367,7 +374,14 @@ int erdma_query_port(struct ib_device *ibdev, u32 port,
memset(attr, 0, sizeof(*attr));
- attr->gid_tbl_len = 1;
+ if (erdma_device_iwarp(dev)) {
+ attr->gid_tbl_len = 1;
+ } else {
+ attr->gid_tbl_len = dev->attrs.max_gid;
+ attr->ip_gids = true;
+ attr->pkey_tbl_len = ERDMA_MAX_PKEYS;
+ }
+
attr->port_cap_flags = IB_PORT_CM_SUP | IB_PORT_DEVICE_MGMT_SUP;
attr->max_msg_sz = -1;
@@ -395,8 +409,18 @@ int erdma_query_port(struct ib_device *ibdev, u32 port,
int erdma_get_port_immutable(struct ib_device *ibdev, u32 port,
struct ib_port_immutable *port_immutable)
{
- port_immutable->gid_tbl_len = 1;
- port_immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+ struct erdma_dev *dev = to_edev(ibdev);
+
+ if (erdma_device_iwarp(dev)) {
+ port_immutable->core_cap_flags = RDMA_CORE_PORT_IWARP;
+ port_immutable->gid_tbl_len = 1;
+ } else {
+ port_immutable->core_cap_flags =
+ RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+ port_immutable->max_mad_size = IB_MGMT_MAD_SIZE;
+ port_immutable->gid_tbl_len = dev->attrs.max_gid;
+ port_immutable->pkey_tbl_len = ERDMA_MAX_PKEYS;
+ }
return 0;
}
@@ -438,7 +462,8 @@ static void erdma_flush_worker(struct work_struct *work)
req.qpn = QP_ID(qp);
req.sq_pi = qp->kern_qp.sq_pi;
req.rq_pi = qp->kern_qp.rq_pi;
- erdma_post_cmd_wait(&qp->dev->cmdq, &req, sizeof(req), NULL, NULL);
+ erdma_post_cmd_wait(&qp->dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
}
static int erdma_qp_validate_cap(struct erdma_dev *dev,
@@ -998,7 +1023,12 @@ int erdma_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *attrs,
qp->attrs.max_send_sge = attrs->cap.max_send_sge;
qp->attrs.max_recv_sge = attrs->cap.max_recv_sge;
- qp->attrs.state = ERDMA_QP_STATE_IDLE;
+
+ if (erdma_device_iwarp(qp->dev))
+ qp->attrs.iwarp.state = ERDMA_QPS_IWARP_IDLE;
+ else
+ qp->attrs.rocev2.state = ERDMA_QPS_ROCEV2_RESET;
+
INIT_DELAYED_WORK(&qp->reflush_dwork, erdma_flush_worker);
ret = create_qp_cmd(uctx, qp);
@@ -1222,9 +1252,17 @@ int erdma_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
req.cfg = FIELD_PREP(ERDMA_CMD_MR_MPT_IDX_MASK, ibmr->lkey >> 8) |
FIELD_PREP(ERDMA_CMD_MR_KEY_MASK, ibmr->lkey & 0xFF);
- ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+ /*
+ * A timeout disables the command queue, so retry cannot succeed. Treat
+ * terminal command failures as diagnostic; propagating them can make
+ * forced uverbs cleanup discard the last software resource pointers.
+ */
if (ret)
- return ret;
+ ibdev_warn_ratelimited(&dev->ibdev,
+ "failed to deregister MR 0x%x: %d\n",
+ ibmr->lkey, ret);
erdma_free_idx(&dev->res_cb[ERDMA_RES_TYPE_STAG_IDX], ibmr->lkey >> 8);
@@ -1240,6 +1278,7 @@ int erdma_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
struct erdma_dev *dev = to_edev(ibcq->device);
struct erdma_ucontext *ctx = rdma_udata_to_drv_context(
udata, struct erdma_ucontext, ibucontext);
+ unsigned long flags;
int err;
struct erdma_cmdq_destroy_cq_req req;
@@ -1247,9 +1286,19 @@ int erdma_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
CMDQ_OPCODE_DESTROY_CQ);
req.cqn = cq->cqn;
- err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
if (err)
- return err;
+ ibdev_warn_ratelimited(&dev->ibdev,
+ "failed to destroy CQ %u: %d\n",
+ cq->cqn, err);
+
+ xa_lock_irqsave(&dev->cq_xa, flags);
+ __xa_erase(&dev->cq_xa, cq->cqn);
+ xa_unlock_irqrestore(&dev->cq_xa, flags);
+
+ erdma_cq_put(cq);
+ wait_for_completion(&cq->free);
if (rdma_is_kernel_res(&cq->ibcq.res)) {
dma_free_coherent(&dev->pdev->dev, cq->depth << CQE_SHIFT,
@@ -1261,8 +1310,6 @@ int erdma_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
put_mtt_entries(dev, &cq->user_cq.qbuf_mem);
}
- xa_erase(&dev->cq_xa, cq->cqn);
-
return 0;
}
@@ -1272,13 +1319,21 @@ int erdma_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata)
struct erdma_dev *dev = to_edev(ibqp->device);
struct erdma_ucontext *ctx = rdma_udata_to_drv_context(
udata, struct erdma_ucontext, ibucontext);
- struct erdma_qp_attrs qp_attrs;
- int err;
struct erdma_cmdq_destroy_qp_req req;
+ union erdma_mod_qp_params params;
+ unsigned long flags;
+ int err;
down_write(&qp->state_lock);
- qp_attrs.state = ERDMA_QP_STATE_ERROR;
- erdma_modify_qp_internal(qp, &qp_attrs, ERDMA_QP_ATTR_STATE);
+ if (erdma_device_iwarp(dev)) {
+ params.iwarp.state = ERDMA_QPS_IWARP_ERROR;
+ erdma_modify_qp_state_iwarp(qp, ¶ms.iwarp,
+ ERDMA_QPA_IWARP_STATE);
+ } else {
+ params.rocev2.state = ERDMA_QPS_ROCEV2_ERROR;
+ erdma_modify_qp_state_rocev2(qp, ¶ms.rocev2,
+ ERDMA_QPA_ROCEV2_STATE);
+ }
up_write(&qp->state_lock);
cancel_delayed_work_sync(&qp->reflush_dwork);
@@ -1287,9 +1342,16 @@ int erdma_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata)
CMDQ_OPCODE_DESTROY_QP);
req.qpn = QP_ID(qp);
- err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
if (err)
- return err;
+ ibdev_warn_ratelimited(&dev->ibdev,
+ "failed to destroy QP %u: %d\n",
+ QP_ID(qp), err);
+
+ xa_lock_irqsave(&dev->qp_xa, flags);
+ __xa_erase(&dev->qp_xa, QP_ID(qp));
+ xa_unlock_irqrestore(&dev->qp_xa, flags);
erdma_qp_put(qp);
wait_for_completion(&qp->safe_free);
@@ -1304,7 +1366,6 @@ int erdma_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata)
if (qp->cep)
erdma_cep_put(qp->cep);
- xa_erase(&dev->qp_xa, QP_ID(qp));
return 0;
}
@@ -1385,7 +1446,8 @@ static int alloc_db_resources(struct erdma_dev *dev, struct erdma_ucontext *ctx,
FIELD_PREP(ERDMA_CMD_EXT_DB_RQ_EN_MASK, 1) |
FIELD_PREP(ERDMA_CMD_EXT_DB_SQ_EN_MASK, 1);
- ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), &val0, &val1);
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), &val0, &val1,
+ true);
if (ret)
return ret;
@@ -1420,7 +1482,8 @@ static void free_db_resources(struct erdma_dev *dev, struct erdma_ucontext *ctx)
req.rdb_off = ctx->ext_db.rdb_off;
req.cdb_off = ctx->ext_db.cdb_off;
- ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
if (ret)
ibdev_err_ratelimited(&dev->ibdev,
"free db resources failed %d", ret);
@@ -1509,69 +1572,248 @@ void erdma_dealloc_ucontext(struct ib_ucontext *ibctx)
atomic_dec(&dev->num_ctx);
}
-static int ib_qp_state_to_erdma_qp_state[IB_QPS_ERR + 1] = {
- [IB_QPS_RESET] = ERDMA_QP_STATE_IDLE,
- [IB_QPS_INIT] = ERDMA_QP_STATE_IDLE,
- [IB_QPS_RTR] = ERDMA_QP_STATE_RTR,
- [IB_QPS_RTS] = ERDMA_QP_STATE_RTS,
- [IB_QPS_SQD] = ERDMA_QP_STATE_CLOSING,
- [IB_QPS_SQE] = ERDMA_QP_STATE_TERMINATE,
- [IB_QPS_ERR] = ERDMA_QP_STATE_ERROR
+static void erdma_attr_to_av(const struct rdma_ah_attr *ah_attr,
+ struct erdma_av *av, u16 sport)
+{
+ const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
+
+ av->port = rdma_ah_get_port_num(ah_attr);
+ av->sgid_index = grh->sgid_index;
+ av->hop_limit = grh->hop_limit;
+ av->traffic_class = grh->traffic_class;
+ av->sl = rdma_ah_get_sl(ah_attr);
+
+ av->flow_label = grh->flow_label;
+ av->udp_sport = sport;
+
+ ether_addr_copy(av->dmac, ah_attr->roce.dmac);
+ memcpy(av->dgid, grh->dgid.raw, ERDMA_ROCEV2_GID_SIZE);
+
+ if (ipv6_addr_v4mapped((struct in6_addr *)&grh->dgid))
+ av->ntype = ERDMA_NETWORK_TYPE_IPV4;
+ else
+ av->ntype = ERDMA_NETWORK_TYPE_IPV6;
+}
+
+static void erdma_av_to_attr(struct erdma_av *av, struct rdma_ah_attr *ah_attr)
+{
+ ah_attr->type = RDMA_AH_ATTR_TYPE_ROCE;
+
+ rdma_ah_set_sl(ah_attr, av->sl);
+ rdma_ah_set_port_num(ah_attr, av->port);
+ rdma_ah_set_ah_flags(ah_attr, IB_AH_GRH);
+
+ rdma_ah_set_grh(ah_attr, NULL, av->flow_label, av->sgid_index,
+ av->hop_limit, av->traffic_class);
+ rdma_ah_set_dgid_raw(ah_attr, av->dgid);
+}
+
+static int ib_qps_to_erdma_qps[ERDMA_PROTO_COUNT][IB_QPS_ERR + 1] = {
+ [ERDMA_PROTO_IWARP] = {
+ [IB_QPS_RESET] = ERDMA_QPS_IWARP_IDLE,
+ [IB_QPS_INIT] = ERDMA_QPS_IWARP_IDLE,
+ [IB_QPS_RTR] = ERDMA_QPS_IWARP_RTR,
+ [IB_QPS_RTS] = ERDMA_QPS_IWARP_RTS,
+ [IB_QPS_SQD] = ERDMA_QPS_IWARP_CLOSING,
+ [IB_QPS_SQE] = ERDMA_QPS_IWARP_TERMINATE,
+ [IB_QPS_ERR] = ERDMA_QPS_IWARP_ERROR,
+ },
+ [ERDMA_PROTO_ROCEV2] = {
+ [IB_QPS_RESET] = ERDMA_QPS_ROCEV2_RESET,
+ [IB_QPS_INIT] = ERDMA_QPS_ROCEV2_INIT,
+ [IB_QPS_RTR] = ERDMA_QPS_ROCEV2_RTR,
+ [IB_QPS_RTS] = ERDMA_QPS_ROCEV2_RTS,
+ [IB_QPS_SQD] = ERDMA_QPS_ROCEV2_SQD,
+ [IB_QPS_SQE] = ERDMA_QPS_ROCEV2_SQE,
+ [IB_QPS_ERR] = ERDMA_QPS_ROCEV2_ERROR,
+ },
};
-int erdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask,
- struct ib_udata *udata)
+static int erdma_qps_to_ib_qps[ERDMA_PROTO_COUNT][ERDMA_QPS_ROCEV2_COUNT] = {
+ [ERDMA_PROTO_IWARP] = {
+ [ERDMA_QPS_IWARP_IDLE] = IB_QPS_INIT,
+ [ERDMA_QPS_IWARP_RTR] = IB_QPS_RTR,
+ [ERDMA_QPS_IWARP_RTS] = IB_QPS_RTS,
+ [ERDMA_QPS_IWARP_CLOSING] = IB_QPS_ERR,
+ [ERDMA_QPS_IWARP_TERMINATE] = IB_QPS_ERR,
+ [ERDMA_QPS_IWARP_ERROR] = IB_QPS_ERR,
+ },
+ [ERDMA_PROTO_ROCEV2] = {
+ [ERDMA_QPS_ROCEV2_RESET] = IB_QPS_RESET,
+ [ERDMA_QPS_ROCEV2_INIT] = IB_QPS_INIT,
+ [ERDMA_QPS_ROCEV2_RTR] = IB_QPS_RTR,
+ [ERDMA_QPS_ROCEV2_RTS] = IB_QPS_RTS,
+ [ERDMA_QPS_ROCEV2_SQD] = IB_QPS_SQD,
+ [ERDMA_QPS_ROCEV2_SQE] = IB_QPS_SQE,
+ [ERDMA_QPS_ROCEV2_ERROR] = IB_QPS_ERR,
+ },
+};
+
+static inline enum erdma_qps_iwarp ib_to_iwarp_qps(enum ib_qp_state state)
{
- struct erdma_qp_attrs new_attrs;
- enum erdma_qp_attr_mask erdma_attr_mask = 0;
- struct erdma_qp *qp = to_eqp(ibqp);
- int ret = 0;
+ return ib_qps_to_erdma_qps[ERDMA_PROTO_IWARP][state];
+}
- if (attr_mask & ~IB_QP_ATTR_STANDARD_BITS)
- return -EOPNOTSUPP;
+static inline enum erdma_qps_rocev2 ib_to_rocev2_qps(enum ib_qp_state state)
+{
+ return ib_qps_to_erdma_qps[ERDMA_PROTO_ROCEV2][state];
+}
- memset(&new_attrs, 0, sizeof(new_attrs));
+static inline enum ib_qp_state iwarp_to_ib_qps(enum erdma_qps_iwarp state)
+{
+ return erdma_qps_to_ib_qps[ERDMA_PROTO_IWARP][state];
+}
- if (attr_mask & IB_QP_STATE) {
- new_attrs.state = ib_qp_state_to_erdma_qp_state[attr->qp_state];
+static inline enum ib_qp_state rocev2_to_ib_qps(enum erdma_qps_rocev2 state)
+{
+ return erdma_qps_to_ib_qps[ERDMA_PROTO_ROCEV2][state];
+}
- erdma_attr_mask |= ERDMA_QP_ATTR_STATE;
+static int erdma_check_qp_attrs(struct erdma_qp *qp, struct ib_qp_attr *attr,
+ int attr_mask)
+{
+ enum ib_qp_state cur_state, nxt_state;
+ struct erdma_dev *dev = qp->dev;
+ int ret = -EINVAL;
+
+ if (attr_mask & ~IB_QP_ATTR_STANDARD_BITS) {
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+
+ if ((attr_mask & IB_QP_PORT) &&
+ !rdma_is_port_valid(&dev->ibdev, attr->port_num))
+ goto out;
+
+ if (erdma_device_rocev2(dev)) {
+ cur_state = (attr_mask & IB_QP_CUR_STATE) ?
+ attr->cur_qp_state :
+ rocev2_to_ib_qps(qp->attrs.rocev2.state);
+
+ nxt_state = (attr_mask & IB_QP_STATE) ? attr->qp_state :
+ cur_state;
+
+ if (!ib_modify_qp_is_ok(cur_state, nxt_state, qp->ibqp.qp_type,
+ attr_mask))
+ goto out;
+
+ if ((attr_mask & IB_QP_AV) &&
+ erdma_check_gid_attr(
+ rdma_ah_read_grh(&attr->ah_attr)->sgid_attr))
+ goto out;
+
+ if ((attr_mask & IB_QP_PKEY_INDEX) &&
+ attr->pkey_index >= ERDMA_MAX_PKEYS)
+ goto out;
+ }
+
+ return 0;
+
+out:
+ return ret;
+}
+
+static void erdma_init_mod_qp_params_rocev2(
+ struct erdma_qp *qp, struct erdma_mod_qp_params_rocev2 *params,
+ int *erdma_attr_mask, struct ib_qp_attr *attr, int ib_attr_mask)
+{
+ enum erdma_qpa_mask_rocev2 to_modify_attrs = 0;
+ enum erdma_qps_rocev2 cur_state, nxt_state;
+ u16 udp_sport;
+
+ if (ib_attr_mask & IB_QP_CUR_STATE)
+ cur_state = ib_to_rocev2_qps(attr->cur_qp_state);
+ else
+ cur_state = qp->attrs.rocev2.state;
+
+ if (ib_attr_mask & IB_QP_STATE)
+ nxt_state = ib_to_rocev2_qps(attr->qp_state);
+ else
+ nxt_state = cur_state;
+
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_STATE;
+ params->state = nxt_state;
+
+ if (ib_attr_mask & IB_QP_QKEY) {
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_QKEY;
+ params->qkey = attr->qkey;
+ }
+
+ if (ib_attr_mask & IB_QP_SQ_PSN) {
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_SQ_PSN;
+ params->sq_psn = attr->sq_psn;
+ }
+
+ if (ib_attr_mask & IB_QP_RQ_PSN) {
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_RQ_PSN;
+ params->rq_psn = attr->rq_psn;
+ }
+
+ if (ib_attr_mask & IB_QP_DEST_QPN) {
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_DST_QPN;
+ params->dst_qpn = attr->dest_qp_num;
+ }
+
+ if (ib_attr_mask & IB_QP_AV) {
+ to_modify_attrs |= ERDMA_QPA_ROCEV2_AV;
+ udp_sport = rdma_get_udp_sport(attr->ah_attr.grh.flow_label,
+ QP_ID(qp), params->dst_qpn);
+ erdma_attr_to_av(&attr->ah_attr, ¶ms->av, udp_sport);
}
+ *erdma_attr_mask = to_modify_attrs;
+}
+
+int erdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask,
+ struct ib_udata *udata)
+{
+ struct erdma_qp *qp = to_eqp(ibqp);
+ union erdma_mod_qp_params params;
+ int ret = 0, erdma_attr_mask = 0;
+
down_write(&qp->state_lock);
- ret = erdma_modify_qp_internal(qp, &new_attrs, erdma_attr_mask);
+ ret = erdma_check_qp_attrs(qp, attr, attr_mask);
+ if (ret)
+ goto out;
- up_write(&qp->state_lock);
+ if (erdma_device_iwarp(qp->dev)) {
+ if (attr_mask & IB_QP_STATE) {
+ erdma_attr_mask |= ERDMA_QPA_IWARP_STATE;
+ params.iwarp.state = ib_to_iwarp_qps(attr->qp_state);
+ }
+ ret = erdma_modify_qp_state_iwarp(qp, ¶ms.iwarp,
+ erdma_attr_mask);
+ } else {
+ erdma_init_mod_qp_params_rocev2(
+ qp, ¶ms.rocev2, &erdma_attr_mask, attr, attr_mask);
+
+ ret = erdma_modify_qp_state_rocev2(qp, ¶ms.rocev2,
+ erdma_attr_mask);
+ }
+
+out:
+ up_write(&qp->state_lock);
return ret;
}
static enum ib_qp_state query_qp_state(struct erdma_qp *qp)
{
- switch (qp->attrs.state) {
- case ERDMA_QP_STATE_IDLE:
- return IB_QPS_INIT;
- case ERDMA_QP_STATE_RTR:
- return IB_QPS_RTR;
- case ERDMA_QP_STATE_RTS:
- return IB_QPS_RTS;
- case ERDMA_QP_STATE_CLOSING:
- return IB_QPS_ERR;
- case ERDMA_QP_STATE_TERMINATE:
- return IB_QPS_ERR;
- case ERDMA_QP_STATE_ERROR:
- return IB_QPS_ERR;
- default:
- return IB_QPS_ERR;
- }
+ if (erdma_device_iwarp(qp->dev))
+ return iwarp_to_ib_qps(qp->attrs.iwarp.state);
+ else
+ return rocev2_to_ib_qps(qp->attrs.rocev2.state);
}
int erdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr)
{
+ struct erdma_cmdq_query_qp_req_rocev2 req;
struct erdma_dev *dev;
struct erdma_qp *qp;
+ u64 resp0, resp1;
+ int ret;
if (ibqp && qp_attr && qp_init_attr) {
qp = to_eqp(ibqp);
@@ -1598,8 +1840,37 @@ int erdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
qp_init_attr->cap = qp_attr->cap;
- qp_attr->qp_state = query_qp_state(qp);
- qp_attr->cur_qp_state = query_qp_state(qp);
+ if (erdma_device_rocev2(dev)) {
+ /* Query hardware to get some attributes */
+ erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
+ CMDQ_OPCODE_QUERY_QP);
+ req.qpn = QP_ID(qp);
+
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), &resp0,
+ &resp1, true);
+ if (ret)
+ return ret;
+
+ qp_attr->sq_psn =
+ FIELD_GET(ERDMA_CMD_QUERY_QP_RESP_SQ_PSN_MASK, resp0);
+ qp_attr->rq_psn =
+ FIELD_GET(ERDMA_CMD_QUERY_QP_RESP_RQ_PSN_MASK, resp0);
+ qp_attr->qp_state = rocev2_to_ib_qps(FIELD_GET(
+ ERDMA_CMD_QUERY_QP_RESP_QP_STATE_MASK, resp0));
+ qp_attr->cur_qp_state = qp_attr->qp_state;
+ qp_attr->sq_draining = FIELD_GET(
+ ERDMA_CMD_QUERY_QP_RESP_SQ_DRAINING_MASK, resp0);
+
+ qp_attr->pkey_index = 0;
+ qp_attr->dest_qp_num = qp->attrs.rocev2.dst_qpn;
+
+ if (qp->ibqp.qp_type == IB_QPT_RC)
+ erdma_av_to_attr(&qp->attrs.rocev2.av,
+ &qp_attr->ah_attr);
+ } else {
+ qp_attr->qp_state = query_qp_state(qp);
+ qp_attr->cur_qp_state = qp_attr->qp_state;
+ }
return 0;
}
@@ -1671,6 +1942,8 @@ int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
cq->ibcq.cqe = depth;
cq->depth = depth;
cq->assoc_eqn = attr->comp_vector + 1;
+ refcount_set(&cq->refcount, 1);
+ init_completion(&cq->free);
ret = xa_alloc_cyclic(&dev->cq_xa, &cq->cqn, cq,
XA_LIMIT(1, dev->attrs.max_cq - 1),
@@ -1739,7 +2012,7 @@ void erdma_set_mtu(struct erdma_dev *dev, u32 mtu)
CMDQ_OPCODE_CONF_MTU);
req.mtu = mtu;
- erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL, true);
}
void erdma_port_event(struct erdma_dev *dev, enum ib_event_type reason)
@@ -1809,7 +2082,8 @@ static int erdma_query_hw_stats(struct erdma_dev *dev,
req.target_addr = dma_addr;
req.target_length = ERDMA_HW_RESP_SIZE;
- err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL);
+ err = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
if (err)
goto out;
@@ -1842,3 +2116,161 @@ int erdma_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,
return stats->num_counters;
}
+
+enum rdma_link_layer erdma_get_link_layer(struct ib_device *ibdev, u32 port_num)
+{
+ return IB_LINK_LAYER_ETHERNET;
+}
+
+static int erdma_set_gid(struct erdma_dev *dev, u8 op, u32 idx,
+ const union ib_gid *gid)
+{
+ struct erdma_cmdq_set_gid_req req;
+ u8 ntype;
+
+ req.cfg = FIELD_PREP(ERDMA_CMD_SET_GID_SGID_IDX_MASK, idx) |
+ FIELD_PREP(ERDMA_CMD_SET_GID_OP_MASK, op);
+
+ if (op == ERDMA_SET_GID_OP_ADD) {
+ if (ipv6_addr_v4mapped((struct in6_addr *)gid))
+ ntype = ERDMA_NETWORK_TYPE_IPV4;
+ else
+ ntype = ERDMA_NETWORK_TYPE_IPV6;
+
+ req.cfg |= FIELD_PREP(ERDMA_CMD_SET_GID_NTYPE_MASK, ntype);
+
+ memcpy(&req.gid, gid, ERDMA_ROCEV2_GID_SIZE);
+ }
+
+ erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
+ CMDQ_OPCODE_SET_GID);
+ return erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+}
+
+int erdma_add_gid(const struct ib_gid_attr *attr, void **context)
+{
+ struct erdma_dev *dev = to_edev(attr->device);
+ int ret;
+
+ ret = erdma_check_gid_attr(attr);
+ if (ret)
+ return ret;
+
+ return erdma_set_gid(dev, ERDMA_SET_GID_OP_ADD, attr->index,
+ &attr->gid);
+}
+
+int erdma_del_gid(const struct ib_gid_attr *attr, void **context)
+{
+ return erdma_set_gid(to_edev(attr->device), ERDMA_SET_GID_OP_DEL,
+ attr->index, NULL);
+}
+
+int erdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey)
+{
+ if (index >= ERDMA_MAX_PKEYS)
+ return -EINVAL;
+
+ *pkey = ERDMA_DEFAULT_PKEY;
+ return 0;
+}
+
+void erdma_set_av_cfg(struct erdma_av_cfg *av_cfg, struct erdma_av *av)
+{
+ av_cfg->cfg0 = FIELD_PREP(ERDMA_CMD_CREATE_AV_FL_MASK, av->flow_label) |
+ FIELD_PREP(ERDMA_CMD_CREATE_AV_NTYPE_MASK, av->ntype);
+
+ av_cfg->traffic_class = av->traffic_class;
+ av_cfg->hop_limit = av->hop_limit;
+ av_cfg->sl = av->sl;
+
+ av_cfg->udp_sport = av->udp_sport;
+ av_cfg->sgid_index = av->sgid_index;
+
+ ether_addr_copy(av_cfg->dmac, av->dmac);
+ memcpy(av_cfg->dgid, av->dgid, ERDMA_ROCEV2_GID_SIZE);
+}
+
+int erdma_create_ah(struct ib_ah *ibah, struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ const struct ib_global_route *grh =
+ rdma_ah_read_grh(init_attr->ah_attr);
+ struct erdma_dev *dev = to_edev(ibah->device);
+ struct erdma_pd *pd = to_epd(ibah->pd);
+ struct erdma_ah *ah = to_eah(ibah);
+ struct erdma_cmdq_create_ah_req req;
+ u32 udp_sport;
+ int ret;
+
+ ret = erdma_check_gid_attr(grh->sgid_attr);
+ if (ret)
+ return ret;
+
+ ret = erdma_alloc_idx(&dev->res_cb[ERDMA_RES_TYPE_AH]);
+ if (ret < 0)
+ return ret;
+
+ ah->ahn = ret;
+
+ if (grh->flow_label)
+ udp_sport = rdma_flow_label_to_udp_sport(grh->flow_label);
+ else
+ udp_sport =
+ IB_ROCE_UDP_ENCAP_VALID_PORT_MIN + (ah->ahn & 0x3FFF);
+
+ erdma_attr_to_av(init_attr->ah_attr, &ah->av, udp_sport);
+
+ erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
+ CMDQ_OPCODE_CREATE_AH);
+
+ req.pdn = pd->pdn;
+ req.ahn = ah->ahn;
+ erdma_set_av_cfg(&req.av_cfg, &ah->av);
+
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+ if (ret) {
+ erdma_free_idx(&dev->res_cb[ERDMA_RES_TYPE_AH], ah->ahn);
+ return ret;
+ }
+
+ return 0;
+}
+
+int erdma_destroy_ah(struct ib_ah *ibah, u32 flags)
+{
+ struct erdma_dev *dev = to_edev(ibah->device);
+ struct erdma_pd *pd = to_epd(ibah->pd);
+ struct erdma_ah *ah = to_eah(ibah);
+ struct erdma_cmdq_destroy_ah_req req;
+ int ret;
+
+ erdma_cmdq_build_reqhdr(&req.hdr, CMDQ_SUBMOD_RDMA,
+ CMDQ_OPCODE_DESTROY_AH);
+
+ req.pdn = pd->pdn;
+ req.ahn = ah->ahn;
+
+ ret = erdma_post_cmd_wait(&dev->cmdq, &req, sizeof(req), NULL, NULL,
+ true);
+ if (ret)
+ ibdev_warn_ratelimited(&dev->ibdev,
+ "failed to destroy AH %u: %d\n",
+ ah->ahn, ret);
+
+ erdma_free_idx(&dev->res_cb[ERDMA_RES_TYPE_AH], ah->ahn);
+
+ return 0;
+}
+
+int erdma_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr)
+{
+ struct erdma_ah *ah = to_eah(ibah);
+
+ memset(ah_attr, 0, sizeof(*ah_attr));
+ erdma_av_to_attr(&ah->av, ah_attr);
+
+ return 0;
+}
diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h
index c998acd39a78..a752bf7c4c1e 100644
--- a/drivers/infiniband/hw/erdma/erdma_verbs.h
+++ b/drivers/infiniband/hw/erdma/erdma_verbs.h
@@ -7,6 +7,9 @@
#ifndef __ERDMA_VERBS_H__
#define __ERDMA_VERBS_H__
+#include <linux/completion.h>
+#include <linux/refcount.h>
+
#include "erdma.h"
/* RDMA Capability. */
@@ -136,6 +139,25 @@ struct erdma_user_dbrecords_page {
int refcnt;
};
+struct erdma_av {
+ u8 port;
+ u8 hop_limit;
+ u8 traffic_class;
+ u8 sl;
+ u8 sgid_index;
+ u16 udp_sport;
+ u32 flow_label;
+ u8 dmac[ETH_ALEN];
+ u8 dgid[ERDMA_ROCEV2_GID_SIZE];
+ enum erdma_network_type ntype;
+};
+
+struct erdma_ah {
+ struct ib_ah ibah;
+ struct erdma_av av;
+ u32 ahn;
+};
+
struct erdma_uqp {
struct erdma_mem sq_mem;
struct erdma_mem rq_mem;
@@ -176,33 +198,91 @@ struct erdma_kqp {
u8 sig_all;
};
-enum erdma_qp_state {
- ERDMA_QP_STATE_IDLE = 0,
- ERDMA_QP_STATE_RTR = 1,
- ERDMA_QP_STATE_RTS = 2,
- ERDMA_QP_STATE_CLOSING = 3,
- ERDMA_QP_STATE_TERMINATE = 4,
- ERDMA_QP_STATE_ERROR = 5,
- ERDMA_QP_STATE_UNDEF = 7,
- ERDMA_QP_STATE_COUNT = 8
+enum erdma_qps_iwarp {
+ ERDMA_QPS_IWARP_IDLE = 0,
+ ERDMA_QPS_IWARP_RTR = 1,
+ ERDMA_QPS_IWARP_RTS = 2,
+ ERDMA_QPS_IWARP_CLOSING = 3,
+ ERDMA_QPS_IWARP_TERMINATE = 4,
+ ERDMA_QPS_IWARP_ERROR = 5,
+ ERDMA_QPS_IWARP_UNDEF = 6,
+ ERDMA_QPS_IWARP_COUNT = 7,
+};
+
+enum erdma_qpa_mask_iwarp {
+ ERDMA_QPA_IWARP_STATE = (1 << 0),
+ ERDMA_QPA_IWARP_LLP_HANDLE = (1 << 2),
+ ERDMA_QPA_IWARP_ORD = (1 << 3),
+ ERDMA_QPA_IWARP_IRD = (1 << 4),
+ ERDMA_QPA_IWARP_SQ_SIZE = (1 << 5),
+ ERDMA_QPA_IWARP_RQ_SIZE = (1 << 6),
+ ERDMA_QPA_IWARP_MPA = (1 << 7),
+ ERDMA_QPA_IWARP_CC = (1 << 8),
};
-enum erdma_qp_attr_mask {
- ERDMA_QP_ATTR_STATE = (1 << 0),
- ERDMA_QP_ATTR_LLP_HANDLE = (1 << 2),
- ERDMA_QP_ATTR_ORD = (1 << 3),
- ERDMA_QP_ATTR_IRD = (1 << 4),
- ERDMA_QP_ATTR_SQ_SIZE = (1 << 5),
- ERDMA_QP_ATTR_RQ_SIZE = (1 << 6),
- ERDMA_QP_ATTR_MPA = (1 << 7)
+enum erdma_qps_rocev2 {
+ ERDMA_QPS_ROCEV2_RESET = 0,
+ ERDMA_QPS_ROCEV2_INIT = 1,
+ ERDMA_QPS_ROCEV2_RTR = 2,
+ ERDMA_QPS_ROCEV2_RTS = 3,
+ ERDMA_QPS_ROCEV2_SQD = 4,
+ ERDMA_QPS_ROCEV2_SQE = 5,
+ ERDMA_QPS_ROCEV2_ERROR = 6,
+ ERDMA_QPS_ROCEV2_COUNT = 7,
+};
+
+enum erdma_qpa_mask_rocev2 {
+ ERDMA_QPA_ROCEV2_STATE = (1 << 0),
+ ERDMA_QPA_ROCEV2_QKEY = (1 << 1),
+ ERDMA_QPA_ROCEV2_AV = (1 << 2),
+ ERDMA_QPA_ROCEV2_SQ_PSN = (1 << 3),
+ ERDMA_QPA_ROCEV2_RQ_PSN = (1 << 4),
+ ERDMA_QPA_ROCEV2_DST_QPN = (1 << 5),
};
enum erdma_qp_flags {
ERDMA_QP_IN_FLUSHING = (1 << 0),
};
+#define ERDMA_QP_ACTIVE 0
+#define ERDMA_QP_PASSIVE 1
+
+struct erdma_mod_qp_params_iwarp {
+ enum erdma_qps_iwarp state;
+ enum erdma_cc_alg cc;
+ u8 qp_type;
+ u8 pd_len;
+ u32 irq_size;
+ u32 orq_size;
+};
+
+struct erdma_qp_attrs_iwarp {
+ enum erdma_qps_iwarp state;
+ u32 cookie;
+};
+
+struct erdma_mod_qp_params_rocev2 {
+ enum erdma_qps_rocev2 state;
+ u32 qkey;
+ u32 sq_psn;
+ u32 rq_psn;
+ u32 dst_qpn;
+ struct erdma_av av;
+};
+
+union erdma_mod_qp_params {
+ struct erdma_mod_qp_params_iwarp iwarp;
+ struct erdma_mod_qp_params_rocev2 rocev2;
+};
+
+struct erdma_qp_attrs_rocev2 {
+ enum erdma_qps_rocev2 state;
+ u32 qkey;
+ u32 dst_qpn;
+ struct erdma_av av;
+};
+
struct erdma_qp_attrs {
- enum erdma_qp_state state;
enum erdma_cc_alg cc; /* Congestion control algorithm */
u32 sq_size;
u32 rq_size;
@@ -210,11 +290,10 @@ struct erdma_qp_attrs {
u32 irq_size;
u32 max_send_sge;
u32 max_recv_sge;
- u32 cookie;
-#define ERDMA_QP_ACTIVE 0
-#define ERDMA_QP_PASSIVE 1
- u8 qp_type;
- u8 pd_len;
+ union {
+ struct erdma_qp_attrs_iwarp iwarp;
+ struct erdma_qp_attrs_rocev2 rocev2;
+ };
};
struct erdma_qp {
@@ -265,6 +344,8 @@ struct erdma_cq {
u32 depth;
u32 assoc_eqn;
+ refcount_t refcount;
+ struct completion free;
union {
struct erdma_kcq_info kern_cq;
@@ -279,18 +360,63 @@ static inline struct erdma_qp *find_qp_by_qpn(struct erdma_dev *dev, int id)
return (struct erdma_qp *)xa_load(&dev->qp_xa, id);
}
-static inline struct erdma_cq *find_cq_by_cqn(struct erdma_dev *dev, int id)
+static inline struct erdma_qp *erdma_qp_get_by_qpn(struct erdma_dev *dev,
+ int id)
+{
+ struct erdma_qp *qp;
+ unsigned long flags;
+
+ xa_lock_irqsave(&dev->qp_xa, flags);
+ qp = xa_load(&dev->qp_xa, id);
+ if (qp && !kref_get_unless_zero(&qp->ref))
+ qp = NULL;
+ xa_unlock_irqrestore(&dev->qp_xa, flags);
+
+ return qp;
+}
+
+static inline struct erdma_cq *erdma_cq_get_by_cqn(struct erdma_dev *dev,
+ int id)
+{
+ struct erdma_cq *cq;
+ unsigned long flags;
+
+ xa_lock_irqsave(&dev->cq_xa, flags);
+ cq = xa_load(&dev->cq_xa, id);
+ if (cq && !refcount_inc_not_zero(&cq->refcount))
+ cq = NULL;
+ xa_unlock_irqrestore(&dev->cq_xa, flags);
+
+ return cq;
+}
+
+static inline void erdma_cq_put(struct erdma_cq *cq)
{
- return (struct erdma_cq *)xa_load(&dev->cq_xa, id);
+ if (refcount_dec_and_test(&cq->refcount))
+ complete(&cq->free);
}
void erdma_qp_get(struct erdma_qp *qp);
void erdma_qp_put(struct erdma_qp *qp);
-int erdma_modify_qp_internal(struct erdma_qp *qp, struct erdma_qp_attrs *attrs,
- enum erdma_qp_attr_mask mask);
+int erdma_modify_qp_state_iwarp(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_iwarp *params,
+ int mask);
+int erdma_modify_qp_state_rocev2(struct erdma_qp *qp,
+ struct erdma_mod_qp_params_rocev2 *params,
+ int attr_mask);
void erdma_qp_llp_close(struct erdma_qp *qp);
void erdma_qp_cm_drop(struct erdma_qp *qp);
+static inline bool erdma_device_iwarp(struct erdma_dev *dev)
+{
+ return dev->proto == ERDMA_PROTO_IWARP;
+}
+
+static inline bool erdma_device_rocev2(struct erdma_dev *dev)
+{
+ return dev->proto == ERDMA_PROTO_ROCEV2;
+}
+
static inline struct erdma_ucontext *to_ectx(struct ib_ucontext *ibctx)
{
return container_of(ibctx, struct erdma_ucontext, ibucontext);
@@ -316,6 +442,21 @@ static inline struct erdma_cq *to_ecq(struct ib_cq *ibcq)
return container_of(ibcq, struct erdma_cq, ibcq);
}
+static inline struct erdma_ah *to_eah(struct ib_ah *ibah)
+{
+ return container_of(ibah, struct erdma_ah, ibah);
+}
+
+static inline int erdma_check_gid_attr(const struct ib_gid_attr *attr)
+{
+ u8 ntype = rdma_gid_attr_network_type(attr);
+
+ if (ntype != RDMA_NETWORK_IPV4 && ntype != RDMA_NETWORK_IPV6)
+ return -EINVAL;
+
+ return 0;
+}
+
static inline struct erdma_user_mmap_entry *
to_emmap(struct rdma_user_mmap_entry *ibmmap)
{
@@ -360,6 +501,7 @@ int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr,
int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr,
const struct ib_recv_wr **bad_recv_wr);
int erdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
+void erdma_remove_cqes_of_qp(struct ib_cq *ibcq, u32 qpn);
struct ib_mr *erdma_ib_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type,
u32 max_num_sg);
int erdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
@@ -370,5 +512,15 @@ struct rdma_hw_stats *erdma_alloc_hw_port_stats(struct ib_device *device,
u32 port_num);
int erdma_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,
u32 port, int index);
+enum rdma_link_layer erdma_get_link_layer(struct ib_device *ibdev,
+ u32 port_num);
+int erdma_add_gid(const struct ib_gid_attr *attr, void **context);
+int erdma_del_gid(const struct ib_gid_attr *attr, void **context);
+int erdma_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey);
+void erdma_set_av_cfg(struct erdma_av_cfg *av_cfg, struct erdma_av *av);
+int erdma_create_ah(struct ib_ah *ibah, struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata);
+int erdma_destroy_ah(struct ib_ah *ibah, u32 flags);
+int erdma_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);
#endif
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index c52e6b2c9914..db96a7547fb7 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -15065,7 +15065,7 @@ int hfi1_init_dd(struct hfi1_devdata *dd)
*/
ret = hfi1_pcie_ddinit(dd, pdev);
if (ret < 0)
- goto bail_free;
+ goto bail;
/* Save PCI space registers to rewrite after device reset */
ret = save_pci_variables(dd);
@@ -15320,8 +15320,6 @@ int hfi1_init_dd(struct hfi1_devdata *dd)
bail_cleanup:
hfi1_free_rx(dd);
hfi1_pcie_ddcleanup(dd);
-bail_free:
- hfi1_free_devdata(dd);
bail:
return ret;
}
diff --git a/drivers/infiniband/hw/hfi1/hfi.h b/drivers/infiniband/hw/hfi1/hfi.h
index eb38f81aeeb1..67df5c65c6e5 100644
--- a/drivers/infiniband/hw/hfi1/hfi.h
+++ b/drivers/infiniband/hw/hfi1/hfi.h
@@ -2044,9 +2044,7 @@ struct cc_state *get_cc_state_protected(struct hfi1_pportdata *ppd)
/* waiting for an urgent packet to arrive */
#define HFI1_CTXT_WAITING_URG 4
-/* free up any allocated data at closes */
int hfi1_init_dd(struct hfi1_devdata *dd);
-void hfi1_free_devdata(struct hfi1_devdata *dd);
/* LED beaconing functions */
void hfi1_start_led_override(struct hfi1_pportdata *ppd, unsigned int timeon,
diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c
index cbac4a442d9e..c862ae679c9c 100644
--- a/drivers/infiniband/hw/hfi1/init.c
+++ b/drivers/infiniband/hw/hfi1/init.c
@@ -630,8 +630,6 @@ void hfi1_init_pportdata(struct pci_dev *pdev, struct hfi1_pportdata *ppd,
ppd->sm_trap_qp = 0x0;
ppd->sa_qp = 0x1;
- ppd->hfi1_wq = NULL;
-
spin_lock_init(&ppd->cca_timer_lock);
for (i = 0; i < OPA_MAX_SLS; i++) {
@@ -1164,7 +1162,7 @@ static void finalize_asic_data(struct hfi1_devdata *dd,
* It cleans up and frees all data structures set up by
* by hfi1_alloc_devdata().
*/
-void hfi1_free_devdata(struct hfi1_devdata *dd)
+static void hfi1_free_devdata(struct hfi1_devdata *dd)
{
struct hfi1_asic_data *ad;
unsigned long flags;
@@ -1227,8 +1225,9 @@ static struct hfi1_devdata *hfi1_alloc_devdata(struct pci_dev *pdev,
GFP_KERNEL);
if (ret < 0) {
dev_err(&pdev->dev,
- "Could not allocate unit ID: error %d\n", -ret);
- goto bail;
+ "Could not allocate unit ID: error %pe\n", ERR_PTR(ret));
+ rvt_dealloc_device(&dd->verbs_dev.rdi);
+ return ERR_PTR(ret);
}
rvt_set_ibdev_name(&dd->verbs_dev.rdi, "%s_%d", class_name(), dd->unit);
/*
@@ -1562,15 +1561,6 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
/* First, lock the non-writable module parameters */
HFI1_CAP_LOCK();
- /* Validate dev ids */
- if (!(ent->device == PCI_DEVICE_ID_INTEL0 ||
- ent->device == PCI_DEVICE_ID_INTEL1)) {
- dev_err(&pdev->dev, "Failing on unknown Intel deviceid 0x%x\n",
- ent->device);
- ret = -ENODEV;
- goto bail;
- }
-
/* Allocate the dd so we can get to work */
dd = hfi1_alloc_devdata(pdev, NUM_IB_PORTS *
sizeof(struct hfi1_pportdata));
@@ -1624,17 +1614,17 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto bail;
+ ret = create_workqueues(dd);
+ if (ret)
+ goto free_devdata;
+
/*
* Do device-specific initialization, function table setup, dd
* allocation, etc.
*/
ret = hfi1_init_dd(dd);
if (ret)
- goto clean_bail; /* error already printed */
-
- ret = create_workqueues(dd);
- if (ret)
- goto clean_bail;
+ goto destroy_workqueues; /* error already printed */
/* do the generic initialization */
initfail = hfi1_init(dd, 0);
@@ -1647,11 +1637,8 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
* we still create devices, so diags, etc. can be used
* to determine cause of problem.
*/
- if (!initfail && !ret) {
+ if (!initfail && !ret)
dd->flags |= HFI1_INITTED;
- /* create debufs files after init and ib register */
- hfi1_dbg_ibdev_init(&dd->verbs_dev);
- }
j = hfi1_device_create(dd);
if (j)
@@ -1660,7 +1647,6 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (initfail || ret) {
msix_clean_up_interrupts(dd);
stop_timers(dd);
- flush_workqueue(ib_wq);
for (pidx = 0; pidx < dd->num_pports; ++pidx) {
hfi1_quiet_serdes(dd->pport + pidx);
ppd = dd->pport + pidx;
@@ -1677,6 +1663,7 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
hfi1_device_remove(dd);
if (!ret)
hfi1_unregister_ib_device(dd);
+ hfi1_free_rx(dd);
postinit_cleanup(dd);
if (initfail)
ret = initfail;
@@ -1684,10 +1671,14 @@ static int init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
}
sdma_start(dd);
+ hfi1_dbg_ibdev_init(&dd->verbs_dev);
return 0;
-clean_bail:
+destroy_workqueues:
+ destroy_workqueues(dd);
+free_devdata:
+ hfi1_free_devdata(dd);
hfi1_pcie_cleanup(pdev);
bail:
return ret;
@@ -1733,9 +1724,6 @@ static void remove_one(struct pci_dev *pdev)
stop_timers(dd);
- /* wait until all of our (qsfp) queue_work() calls complete */
- flush_workqueue(ib_wq);
-
postinit_cleanup(dd);
}
diff --git a/drivers/infiniband/hw/hfi1/user_sdma.c b/drivers/infiniband/hw/hfi1/user_sdma.c
index b72625283fcf..d2cbece4eb0f 100644
--- a/drivers/infiniband/hw/hfi1/user_sdma.c
+++ b/drivers/infiniband/hw/hfi1/user_sdma.c
@@ -1028,6 +1028,7 @@ static int set_txreq_header_ahg(struct user_sdma_request *req,
struct user_sdma_txreq *tx, u32 datalen)
{
u32 ahg[AHG_KDETH_ARRAY_SIZE];
+ int ret;
int idx = 0;
u8 omfactor; /* KDETH.OM */
struct hfi1_user_sdma_pkt_q *pq = req->pq;
@@ -1132,11 +1133,13 @@ static int set_txreq_header_ahg(struct user_sdma_request *req,
trace_hfi1_sdma_user_header_ahg(pq->dd, pq->ctxt, pq->subctxt,
req->info.comp_idx, req->sde->this_idx,
req->ahg_idx, ahg, idx, tidval);
- sdma_txinit_ahg(&tx->txreq,
- SDMA_TXREQ_F_USE_AHG,
- datalen, req->ahg_idx, idx,
- ahg, sizeof(req->hdr),
- user_sdma_txreq_cb);
+ ret = sdma_txinit_ahg(&tx->txreq,
+ SDMA_TXREQ_F_USE_AHG,
+ datalen, req->ahg_idx, idx,
+ ahg, sizeof(req->hdr),
+ user_sdma_txreq_cb);
+ if (ret)
+ return ret;
return idx;
}
diff --git a/drivers/infiniband/hw/mlx5/cong.c b/drivers/infiniband/hw/mlx5/cong.c
index a78a067e3ce7..e3c4b1c92816 100644
--- a/drivers/infiniband/hw/mlx5/cong.c
+++ b/drivers/infiniband/hw/mlx5/cong.c
@@ -361,7 +361,7 @@ static int mlx5_ib_set_cc_params(struct mlx5_ib_dev *dev, u32 port_num,
MLX5_SET(field_select_r_roce_rp, field, field_select_r_roce_rp,
attr_mask);
- err = mlx5_cmd_exec_in(dev->mdev, modify_cong_params, in);
+ err = mlx5_cmd_exec_in(mdev, modify_cong_params, in);
kvfree(in);
alloc_err:
mlx5_ib_put_native_port_mdev(dev, port_num + 1);
@@ -399,15 +399,13 @@ static ssize_t get_param(struct file *filp, char __user *buf, size_t count,
int offset = param->offset;
u32 var = 0;
int ret;
- char lbuf[11];
+ char lbuf[12];
ret = mlx5_ib_get_cc_params(param->dev, param->port_num, offset, &var);
if (ret)
return ret;
- ret = snprintf(lbuf, sizeof(lbuf), "%d\n", var);
- if (ret < 0)
- return ret;
+ ret = scnprintf(lbuf, sizeof(lbuf), "%u\n", var);
return simple_read_from_buffer(buf, count, pos, lbuf, ret);
}
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index edd38510c025..a0b308638195 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -5101,6 +5101,7 @@ static int __init mlx5_ib_init(void)
dd_err:
mlx5r_rep_cleanup();
rep_err:
+ rcu_barrier();
mlx5_ib_qp_event_cleanup();
qp_event_err:
destroy_workqueue(mlx5_ib_event_wq);
@@ -5114,6 +5115,7 @@ static void __exit mlx5_ib_cleanup(void)
auxiliary_driver_unregister(&mlx5r_driver);
auxiliary_driver_unregister(&mlx5r_mp_driver);
mlx5r_rep_cleanup();
+ rcu_barrier();
mlx5_ib_qp_event_cleanup();
destroy_workqueue(mlx5_ib_event_wq);
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index badec2e33972..2b0047e6fd49 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -637,6 +637,7 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
struct ib_qp_init_attr *attr)
{
int desc_sz = 1 << qp->sq.wqe_shift;
+ int rq_buf_size, sq_buf_size;
if (desc_sz > MLX5_CAP_GEN(dev->mdev, max_wqe_sz_sq)) {
mlx5_ib_warn(dev, "desc_sz %d, max_sq_desc_sz %d\n",
@@ -661,11 +662,21 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
if (attr->qp_type == IB_QPT_RAW_PACKET ||
qp->flags & IB_QP_CREATE_SOURCE_QPN) {
- base->ubuffer.buf_size = qp->rq.wqe_cnt << qp->rq.wqe_shift;
- qp->raw_packet_qp.sq.ubuffer.buf_size = qp->sq.wqe_cnt << 6;
+ if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+ &base->ubuffer.buf_size))
+ return -EINVAL;
+ if (check_shl_overflow(qp->sq.wqe_cnt, 6,
+ &qp->raw_packet_qp.sq.ubuffer.buf_size))
+ return -EINVAL;
} else {
- base->ubuffer.buf_size = (qp->rq.wqe_cnt << qp->rq.wqe_shift) +
- (qp->sq.wqe_cnt << 6);
+ if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+ &rq_buf_size))
+ return -EINVAL;
+ if (check_shl_overflow(qp->sq.wqe_cnt, 6, &sq_buf_size))
+ return -EINVAL;
+ if (check_add_overflow(rq_buf_size, sq_buf_size,
+ &base->ubuffer.buf_size))
+ return -EINVAL;
}
return 0;
@@ -989,7 +1000,11 @@ static int _create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,
qp->rq.offset = 0;
qp->sq.wqe_shift = ilog2(MLX5_SEND_WQE_BB);
- qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ if (check_shl_overflow(qp->rq.wqe_cnt, qp->rq.wqe_shift,
+ &qp->sq.offset)) {
+ err = -EINVAL;
+ goto err_bfreg;
+ }
err = set_user_buf_size(dev, qp, ucmd, base, attr);
if (err)
diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index d08ebb048cb0..3a4fa0ca36a7 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -702,6 +702,21 @@ int rxe_requester(struct rxe_qp *qp)
if (unlikely(!wqe))
goto exit;
+ /*
+ * Don't trust user space data: a user QP's WQE comes from an mmap'd
+ * ring, so num_sge/cur_sge are attacker-controlled. Bound num_sge like
+ * get_srq_wqe(); bound cur_sge only when payload exists (dma.resid),
+ * since copy_data() skips dma->sge[] on a zero-length copy (all a
+ * max_sge == 0 QP can post).
+ */
+ if (unlikely(wqe->dma.num_sge > qp->sq.max_sge ||
+ (wqe->dma.resid &&
+ wqe->dma.cur_sge >= qp->sq.max_sge))) {
+ rxe_dbg_qp(qp, "invalid num_sge/cur_sge in send wqe\n");
+ wqe->status = IB_WC_LOC_QP_OP_ERR;
+ goto err;
+ }
+
if (rxe_wqe_is_fenced(qp, wqe)) {
qp->req.wait_fence = 1;
goto exit;
diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c
index 5611f63a1115..abf6cf3ce986 100644
--- a/drivers/infiniband/sw/rxe/rxe_resp.c
+++ b/drivers/infiniband/sw/rxe/rxe_resp.c
@@ -1188,7 +1188,14 @@ static enum resp_states do_complete(struct rxe_qp *qp,
spin_lock_irqsave(&qp->state_lock, flags);
if (unlikely(qp_state(qp) == IB_QPS_ERR)) {
spin_unlock_irqrestore(&qp->state_lock, flags);
- return RESPST_CHK_RESOURCE;
+ /* The packet was executed and completed before the QP
+ * moved to ERROR; it must be consumed exactly once.
+ * Re-entering the request chain with the stale packet
+ * would copy it into every remaining recv WQE as a new
+ * completion. Remaining WQEs are flushed by the drain
+ * path at rxe_receiver() entry.
+ */
+ return pkt ? RESPST_CLEANUP : RESPST_CHK_RESOURCE;
}
spin_unlock_irqrestore(&qp->state_lock, flags);
diff --git a/drivers/infiniband/sw/siw/siw_cm.c b/drivers/infiniband/sw/siw/siw_cm.c
index 49ff121f77fe..bb7d90963907 100644
--- a/drivers/infiniband/sw/siw/siw_cm.c
+++ b/drivers/infiniband/sw/siw/siw_cm.c
@@ -1701,7 +1701,7 @@ int siw_accept(struct iw_cm_id *id, struct iw_cm_conn_param *params)
cep->state = SIW_EPSTATE_CLOSED;
siw_free_cm_id(cep);
- if (qp->cep) {
+ if (qp->cep == cep) {
siw_cep_put(cep);
qp->cep = NULL;
}
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index d65f3a596365..bbb00fdc12db 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -2703,6 +2703,7 @@ static int __init ipoib_init_module(void)
err_sa:
ib_sa_unregister_client(&ipoib_sa_client);
destroy_workqueue(ipoib_workqueue);
+ rcu_barrier();
err_fs:
ipoib_unregister_debugfs();
@@ -2720,6 +2721,7 @@ static void __exit ipoib_cleanup_module(void)
ib_sa_unregister_client(&ipoib_sa_client);
ipoib_unregister_debugfs();
destroy_workqueue(ipoib_workqueue);
+ rcu_barrier();
}
module_init(ipoib_init_module);
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index 640634f96d72..6483a55170cd 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -59,6 +59,8 @@ static void isert_recv_done(struct ib_cq *cq, struct ib_wc *wc);
static void isert_send_done(struct ib_cq *cq, struct ib_wc *wc);
static void isert_login_recv_done(struct ib_cq *cq, struct ib_wc *wc);
static void isert_login_send_done(struct ib_cq *cq, struct ib_wc *wc);
+static void isert_unmap_tx_desc(struct iser_tx_desc *tx_desc,
+ struct ib_device *ib_dev);
static int isert_sg_tablesize_set(const char *val, const struct kernel_param *kp)
{
@@ -498,6 +500,8 @@ isert_connect_release(struct isert_conn *isert_conn)
if (isert_conn->qp)
isert_destroy_qp(isert_conn);
+ isert_unmap_tx_desc(&isert_conn->login_tx_desc, device->ib_device);
+
if (isert_conn->login_desc)
isert_free_login_buf(isert_conn);
@@ -945,27 +949,16 @@ isert_put_login_tx(struct iscsit_conn *conn, struct iscsi_login *login,
}
if (!login->login_failed) {
if (login->login_complete) {
- ret = isert_alloc_rx_descriptors(isert_conn);
- if (ret)
- return ret;
-
- ret = isert_post_recvm(isert_conn,
- ISERT_QP_MAX_RECV_DTOS);
- if (ret)
- return ret;
-
- /* Now we are in FULL_FEATURE phase */
- mutex_lock(&isert_conn->mutex);
- isert_conn->state = ISER_CONN_FULL_FEATURE;
- mutex_unlock(&isert_conn->mutex);
- goto post_send;
+ /* Posted and sent from isert_get_rx_pdu(). */
+ isert_conn->login_rsp_pending = true;
+ return 0;
}
ret = isert_login_post_recv(isert_conn);
if (ret)
return ret;
}
-post_send:
+
ret = isert_login_post_send(isert_conn, tx_desc);
if (ret)
return ret;
@@ -973,6 +966,21 @@ isert_put_login_tx(struct iscsit_conn *conn, struct iscsi_login *login,
return 0;
}
+static int
+isert_check_login_req(struct isert_conn *isert_conn)
+{
+ struct iscsi_hdr *hdr = isert_get_iscsi_hdr(isert_conn->login_desc);
+ u32 dlength = ntoh24(hdr->dlength);
+
+ if (unlikely(dlength > (u32)isert_conn->login_req_len)) {
+ isert_dbg("login PDU declares %u data bytes but only %d were received\n",
+ dlength, isert_conn->login_req_len);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static void
isert_rx_login_req(struct isert_conn *isert_conn)
{
@@ -1335,6 +1343,21 @@ isert_recv_done(struct ib_cq *cq, struct ib_wc *wc)
ib_dma_sync_single_for_cpu(ib_dev, rx_desc->dma_addr,
ISER_RX_SIZE, DMA_FROM_DEVICE);
+ /*
+ * The data segment length declared in the BHS is attacker controlled
+ * and is used further down to read that many bytes out of the fixed
+ * size receive descriptor, so it has to be checked against the number
+ * of bytes that were actually received. Comparing without subtracting
+ * also rejects PDUs shorter than the iSER and iSCSI headers, which
+ * would otherwise be parsed out of stale descriptor contents.
+ */
+ if (unlikely(wc->byte_len < ISER_HEADERS_LEN + ntoh24(hdr->dlength))) {
+ isert_err("PDU declares %u data bytes but only %u bytes were received\n",
+ ntoh24(hdr->dlength), wc->byte_len);
+ iscsit_cause_connection_reinstatement(isert_conn->conn, 0);
+ return;
+ }
+
isert_dbg("DMA: 0x%llx, iSCSI opcode: 0x%02x, ITT: 0x%08x, flags: 0x%02x dlen: %d\n",
rx_desc->dma_addr, hdr->opcode, hdr->itt, hdr->flags,
(int)(wc->byte_len - ISER_HEADERS_LEN));
@@ -1396,8 +1419,12 @@ isert_login_recv_done(struct ib_cq *cq, struct ib_wc *wc)
if (isert_conn->conn) {
struct iscsi_login *login = isert_conn->conn->conn_login;
- if (login && !login->first_request)
+ if (login && !login->first_request) {
+ if (isert_check_login_req(isert_conn))
+ return;
+
isert_rx_login_req(isert_conn);
+ }
}
mutex_lock(&isert_conn->mutex);
@@ -2362,6 +2389,10 @@ isert_get_login_rx(struct iscsit_conn *conn, struct iscsi_login *login)
if (!login->first_request)
return 0;
+ ret = isert_check_login_req(isert_conn);
+ if (ret)
+ return ret;
+
isert_rx_login_req(isert_conn);
isert_info("before login_comp conn: %p\n", conn);
@@ -2587,8 +2618,30 @@ static void isert_free_conn(struct iscsit_conn *conn)
static void isert_get_rx_pdu(struct iscsit_conn *conn)
{
+ struct isert_conn *isert_conn = conn->context;
struct completion comp;
+ /* The login timeout timer can fail the login after isert_put_login_tx(). */
+ if (!isert_conn->login_rsp_pending)
+ return;
+
+ isert_conn->login_rsp_pending = false;
+
+ /* The session is registered by now; see isert_put_login_tx(). */
+ if (isert_alloc_rx_descriptors(isert_conn))
+ return;
+
+ if (isert_post_recvm(isert_conn, ISERT_QP_MAX_RECV_DTOS))
+ return;
+
+ /* Now we are in FULL_FEATURE phase */
+ mutex_lock(&isert_conn->mutex);
+ isert_conn->state = ISER_CONN_FULL_FEATURE;
+ mutex_unlock(&isert_conn->mutex);
+
+ if (isert_login_post_send(isert_conn, &isert_conn->login_tx_desc))
+ return;
+
init_completion(&comp);
wait_for_completion_interruptible(&comp);
diff --git a/drivers/infiniband/ulp/isert/ib_isert.h b/drivers/infiniband/ulp/isert/ib_isert.h
index 0b2dfd6e7e27..0bac5aa66c80 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.h
+++ b/drivers/infiniband/ulp/isert/ib_isert.h
@@ -178,6 +178,7 @@ struct isert_conn {
struct completion login_comp;
struct completion login_req_comp;
struct iser_tx_desc login_tx_desc;
+ bool login_rsp_pending;
struct rdma_cm_id *cm_id;
struct ib_qp *qp;
struct ib_cq *cq;
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index ce1505a8ee8d..13eddbe0841a 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -1943,7 +1943,8 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp,
ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
if (rsp->tag == ch->tsk_mgmt_tag) {
ch->tsk_mgmt_status = -1;
- if (be32_to_cpu(rsp->resp_data_len) >= 4)
+ if (be32_to_cpu(rsp->resp_data_len) >= 4 &&
+ byte_len >= sizeof(*rsp) + 4)
ch->tsk_mgmt_status = rsp->data[3];
complete(&ch->tsk_mgmt_done);
} else {
@@ -2043,13 +2044,20 @@ static int srp_response_common(struct srp_rdma_ch *ch, s32 req_delta,
}
static void srp_process_cred_req(struct srp_rdma_ch *ch,
- struct srp_cred_req *req)
+ struct srp_cred_req *req, u32 byte_len)
{
- struct srp_cred_rsp rsp = {
- .opcode = SRP_CRED_RSP,
- .tag = req->tag,
- };
- s32 delta = be32_to_cpu(req->req_lim_delta);
+ struct srp_cred_rsp rsp = { .opcode = SRP_CRED_RSP };
+ s32 delta;
+
+ if (byte_len < sizeof(*req)) {
+ shost_printk(KERN_ERR, ch->target->scsi_host, PFX
+ "dropping truncated SRP_CRED_REQ (%u bytes received, %zu expected)\n",
+ byte_len, sizeof(*req));
+ return;
+ }
+
+ rsp.tag = req->tag;
+ delta = be32_to_cpu(req->req_lim_delta);
if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
shost_printk(KERN_ERR, ch->target->scsi_host, PFX
@@ -2057,14 +2065,21 @@ static void srp_process_cred_req(struct srp_rdma_ch *ch,
}
static void srp_process_aer_req(struct srp_rdma_ch *ch,
- struct srp_aer_req *req)
+ struct srp_aer_req *req, u32 byte_len)
{
struct srp_target_port *target = ch->target;
- struct srp_aer_rsp rsp = {
- .opcode = SRP_AER_RSP,
- .tag = req->tag,
- };
- s32 delta = be32_to_cpu(req->req_lim_delta);
+ struct srp_aer_rsp rsp = { .opcode = SRP_AER_RSP };
+ s32 delta;
+
+ if (byte_len < sizeof(*req)) {
+ shost_printk(KERN_ERR, target->scsi_host, PFX
+ "dropping truncated SRP_AER_REQ (%u bytes received, %zu expected)\n",
+ byte_len, sizeof(*req));
+ return;
+ }
+
+ rsp.tag = req->tag;
+ delta = be32_to_cpu(req->req_lim_delta);
shost_printk(KERN_ERR, target->scsi_host, PFX
"ignoring AER for LUN %llu\n", scsilun_to_int(&req->lun));
@@ -2106,11 +2121,11 @@ static void srp_recv_done(struct ib_cq *cq, struct ib_wc *wc)
break;
case SRP_CRED_REQ:
- srp_process_cred_req(ch, iu->buf);
+ srp_process_cred_req(ch, iu->buf, wc->byte_len);
break;
case SRP_AER_REQ:
- srp_process_aer_req(ch, iu->buf);
+ srp_process_aer_req(ch, iu->buf, wc->byte_len);
break;
case SRP_T_LOGOUT:
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 128e4e836470..a289512560c5 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -959,6 +959,7 @@ static int srpt_alloc_rw_ctxs(struct srpt_send_ioctx *ioctx,
struct srpt_rdma_ch *ch = ioctx->ch;
struct scatterlist *prev = NULL;
unsigned prev_nents;
+ u8 n_rdma, n_rw_ctx;
int ret, i;
if (nbufs == 1) {
@@ -970,6 +971,9 @@ static int srpt_alloc_rw_ctxs(struct srpt_send_ioctx *ioctx,
return -ENOMEM;
}
+ n_rw_ctx = ioctx->n_rw_ctx;
+ n_rdma = ioctx->n_rdma;
+
for (i = ioctx->n_rw_ctx; i < nbufs; i++, db++) {
struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
u64 remote_addr = be64_to_cpu(db->va);
@@ -1016,6 +1020,9 @@ static int srpt_alloc_rw_ctxs(struct srpt_send_ioctx *ioctx,
}
if (ioctx->rw_ctxs != &ioctx->s_rw_ctx)
kfree(ioctx->rw_ctxs);
+ ioctx->rw_ctxs = NULL;
+ ioctx->n_rw_ctx = n_rw_ctx;
+ ioctx->n_rdma = n_rdma;
return ret;
}
@@ -1596,7 +1603,7 @@ static void srpt_handle_cmd(struct srpt_rdma_ch *ch,
rc = target_init_cmd(cmd, ch->sess, &send_ioctx->sense_data[0],
scsilun_to_int(&srp_cmd->lun), data_len,
- TCM_SIMPLE_TAG, dir, TARGET_SCF_ACK_KREF);
+ cmd->sam_task_attr, dir, TARGET_SCF_ACK_KREF);
if (rc != 0) {
pr_debug("target_submit_cmd() returned %d for tag %#llx\n", rc,
srp_cmd->tag);
diff --git a/drivers/input/tablet/aiptek.c b/drivers/input/tablet/aiptek.c
index 2b3fbb0455d5..9e78105bace7 100644
--- a/drivers/input/tablet/aiptek.c
+++ b/drivers/input/tablet/aiptek.c
@@ -658,6 +658,8 @@ static void aiptek_irq(struct urb *urb)
pck = (data[1] & aiptek->curSetting.stylusButtonUpper) != 0 ? 1 : 0;
macro = dv && p && tip && !(data[3] & 1) ? (data[3] >> 1) : -1;
+ if (macro >= ARRAY_SIZE(macroKeyEvents))
+ macro = -1;
z = get_unaligned_le16(data + 4);
if (dv) {
@@ -699,7 +701,9 @@ static void aiptek_irq(struct urb *urb)
left = (data[1]& aiptek->curSetting.mouseButtonLeft) != 0 ? 1 : 0;
right = (data[1] & aiptek->curSetting.mouseButtonRight) != 0 ? 1 : 0;
middle = (data[1] & aiptek->curSetting.mouseButtonMiddle) != 0 ? 1 : 0;
- macro = dv && p && left && !(data[3] & 1) ? (data[3] >> 1) : 0;
+ macro = dv && p && left && !(data[3] & 1) ? (data[3] >> 1) : -1;
+ if (macro >= ARRAY_SIZE(macroKeyEvents))
+ macro = -1;
if (dv) {
/* If the selected tool changed, reset the old
@@ -737,11 +741,11 @@ static void aiptek_irq(struct urb *urb)
*/
else if (data[0] == 6) {
macro = get_unaligned_le16(data + 1);
- if (macro > 0) {
+ if (macro > 0 && macro - 1 < ARRAY_SIZE(macroKeyEvents)) {
input_report_key(inputdev, macroKeyEvents[macro - 1],
0);
}
- if (macro < 25) {
+ if (macro + 1 < ARRAY_SIZE(macroKeyEvents)) {
input_report_key(inputdev, macroKeyEvents[macro + 1],
0);
}
@@ -760,7 +764,8 @@ static void aiptek_irq(struct urb *urb)
aiptek->curSetting.toolMode;
}
- input_report_key(inputdev, macroKeyEvents[macro], 1);
+ if (macro < ARRAY_SIZE(macroKeyEvents))
+ input_report_key(inputdev, macroKeyEvents[macro], 1);
input_report_abs(inputdev, ABS_MISC,
1 | AIPTEK_REPORT_TOOL_UNKNOWN);
input_sync(inputdev);
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 54f3c17520d5..b9a14c3ef440 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -2966,14 +2966,31 @@ static void __init free_iommu_resources(void)
/* SB IOAPIC is always on this device in AMD systems */
#define IOAPIC_SB_DEVID ((0x00 << 8) | PCI_DEVFN(0x14, 0))
+/* SB IOAPIC for Hygon family 18h model 4h is on the device 0xb */
+#define IOAPIC_SB_DEVID_FAM18H_M4H ((0x00 << 8) | PCI_DEVFN(0xb, 0))
+
+/*
+ * The Southbridge IOAPIC is assigned a GSI Base of 0 (handling interrupts
+ * 0 through 23).
+ */
+static int __init get_sb_ioapic_id(void)
+{
+ int idx = mp_find_ioapic(0);
+
+ if (idx < 0)
+ return -ENODEV;
+
+ return mpc_ioapic_id(idx);
+}
+
static bool __init check_ioapic_information(void)
{
const char *fw_bug = FW_BUG;
bool ret, has_sb_ioapic;
- int idx;
+ int idx, sb_apicid;
has_sb_ioapic = false;
- ret = false;
+ ret = true;
/*
* If we have map overrides on the kernel command line the
@@ -2983,6 +3000,16 @@ static bool __init check_ioapic_information(void)
if (cmdline_maps)
fw_bug = "";
+ sb_apicid = get_sb_ioapic_id();
+ if (sb_apicid < 0) {
+ /*
+ * Lack of SB IOAPIC registration is not a firmware bug,
+ * e.g. kernel booted with noapic or noacpi.
+ */
+ fw_bug = "";
+ goto out;
+ }
+
for (idx = 0; idx < nr_ioapics; idx++) {
int devid, id = mpc_ioapic_id(idx);
@@ -2991,12 +3018,16 @@ static bool __init check_ioapic_information(void)
pr_err("%s: IOAPIC[%d] not in IVRS table\n",
fw_bug, id);
ret = false;
- } else if (devid == IOAPIC_SB_DEVID) {
+ } else if (id == sb_apicid && (devid == IOAPIC_SB_DEVID ||
+ (boot_cpu_data.x86_vendor == X86_VENDOR_HYGON &&
+ boot_cpu_data.x86 == 0x18 &&
+ boot_cpu_data.x86_model >= 0x4 &&
+ boot_cpu_data.x86_model <= 0xf &&
+ devid == IOAPIC_SB_DEVID_FAM18H_M4H))) {
has_sb_ioapic = true;
- ret = true;
}
}
-
+out:
if (!has_sb_ioapic) {
/*
* We expect the SB IOAPIC to be listed in the IVRS
@@ -3007,6 +3038,7 @@ static bool __init check_ioapic_information(void)
* device id for the IOAPIC in the system.
*/
pr_err("%s: No southbridge IOAPIC found\n", fw_bug);
+ ret = false;
}
if (!ret)
diff --git a/drivers/iommu/amd/ppr.c b/drivers/iommu/amd/ppr.c
index 00d9a52ccb6f..0e46a47ea4c4 100644
--- a/drivers/iommu/amd/ppr.c
+++ b/drivers/iommu/amd/ppr.c
@@ -140,7 +140,7 @@ static void iommu_call_iopf_notifier(struct amd_iommu *iommu, u64 *raw)
if (event.fault.prm.pasid == 0 ||
event.fault.prm.pasid >= dev_data->max_pasids) {
pr_info_ratelimited("Invalid PASID : 0x%x, device : 0x%x\n",
- event.fault.prm.pasid, pdev->dev.id);
+ event.fault.prm.pasid, dev_data->devid);
goto out;
}
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 0a8a6cf1f033..28331d9118b6 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3774,8 +3774,9 @@ static int arm_smmu_write_reg_sync(struct arm_smmu_device *smmu, u32 val,
u32 reg;
writel_relaxed(val, smmu->base + reg_off);
- return readl_relaxed_poll_timeout(smmu->base + ack_off, reg, reg == val,
- 1, ARM_SMMU_POLL_TIMEOUT_US);
+ return readl_relaxed_poll_timeout_atomic(smmu->base + ack_off, reg,
+ reg == val, 1,
+ ARM_SMMU_POLL_TIMEOUT_US);
}
/* GBPA is "special" */
diff --git a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
index a76839cf571d..8d42c63717b6 100644
--- a/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
+++ b/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
@@ -659,6 +659,14 @@ static void tegra241_cmdqv_remove(struct arm_smmu_device *smmu)
container_of(smmu, struct tegra241_cmdqv, smmu);
u16 idx;
+ /*
+ * Free the IRQ before tearing down the VINTFs. free_irq() waits for any
+ * in-flight tegra241_cmdqv_isr() to finish and blocks new ones, so the
+ * ISR cannot dereference a VINTF that is freed by the loop below.
+ */
+ if (cmdqv->irq > 0)
+ free_irq(cmdqv->irq, cmdqv);
+
/* Remove VINTF resources */
for (idx = 0; idx < cmdqv->num_vintfs; idx++) {
if (cmdqv->vintfs[idx]) {
@@ -671,8 +679,6 @@ static void tegra241_cmdqv_remove(struct arm_smmu_device *smmu)
/* Remove cmdqv resources */
ida_destroy(&cmdqv->vintf_ids);
- if (cmdqv->irq > 0)
- free_irq(cmdqv->irq, cmdqv);
iounmap(cmdqv->base);
kfree(cmdqv->vintfs);
put_device(cmdqv->dev); /* smmu->impl_dev */
@@ -824,16 +830,6 @@ __tegra241_cmdqv_probe(struct arm_smmu_device *smmu, struct resource *res,
cmdqv->base = base;
cmdqv->dev = smmu->impl_dev;
- if (cmdqv->irq > 0) {
- ret = request_irq(irq, tegra241_cmdqv_isr, 0, "tegra241-cmdqv",
- cmdqv);
- if (ret) {
- dev_err(cmdqv->dev, "failed to request irq (%d): %d\n",
- cmdqv->irq, ret);
- goto iounmap;
- }
- }
-
regval = readl_relaxed(REG_CMDQV(cmdqv, PARAM));
cmdqv->num_vintfs = 1 << FIELD_GET(CMDQV_NUM_VINTF_LOG2, regval);
cmdqv->num_vcmdqs = 1 << FIELD_GET(CMDQV_NUM_VCMDQ_LOG2, regval);
@@ -842,10 +838,25 @@ __tegra241_cmdqv_probe(struct arm_smmu_device *smmu, struct resource *res,
cmdqv->vintfs =
kcalloc(cmdqv->num_vintfs, sizeof(*cmdqv->vintfs), GFP_KERNEL);
if (!cmdqv->vintfs)
- goto free_irq;
+ goto iounmap;
ida_init(&cmdqv->vintf_ids);
+ /*
+ * Request the IRQ only after cmdqv->vintfs is allocated and zeroed, so
+ * the ISR would not walk an uninitialized array.
+ */
+ if (cmdqv->irq > 0) {
+ ret = request_threaded_irq(irq, NULL, tegra241_cmdqv_isr,
+ IRQF_ONESHOT, "tegra241-cmdqv",
+ cmdqv);
+ if (ret) {
+ dev_err(cmdqv->dev, "failed to request irq (%d): %d\n",
+ cmdqv->irq, ret);
+ goto free_vintfs;
+ }
+ }
+
#ifdef CONFIG_IOMMU_DEBUGFS
if (!cmdqv_debugfs_dir) {
cmdqv_debugfs_dir =
@@ -860,9 +871,9 @@ __tegra241_cmdqv_probe(struct arm_smmu_device *smmu, struct resource *res,
return new_smmu;
-free_irq:
- if (cmdqv->irq > 0)
- free_irq(cmdqv->irq, cmdqv);
+free_vintfs:
+ ida_destroy(&cmdqv->vintf_ids);
+ kfree(cmdqv->vintfs);
iounmap:
iounmap(base);
return NULL;
diff --git a/drivers/iommu/arm/arm-smmu/qcom_iommu.c b/drivers/iommu/arm/arm-smmu/qcom_iommu.c
index 8a76b4f46a32..09d847666a68 100644
--- a/drivers/iommu/arm/arm-smmu/qcom_iommu.c
+++ b/drivers/iommu/arm/arm-smmu/qcom_iommu.c
@@ -200,7 +200,7 @@ static irqreturn_t qcom_iommu_fault(int irq, void *dev)
fsynr = iommu_readl(ctx, ARM_SMMU_CB_FSYNR0);
iova = iommu_readq(ctx, ARM_SMMU_CB_FAR);
- if (!report_iommu_fault(ctx->domain, ctx->dev, iova, 0)) {
+ if (report_iommu_fault(ctx->domain, ctx->dev, iova, 0)) {
dev_err_ratelimited(ctx->dev,
"Unhandled context fault: fsr=0x%x, "
"iova=0x%016llx, fsynr=0x%x, cb=%d\n",
@@ -860,7 +860,7 @@ static int qcom_iommu_device_probe(struct platform_device *pdev)
ret = iommu_device_register(&qcom_iommu->iommu, &qcom_iommu_ops, dev);
if (ret) {
dev_err(dev, "Failed to register iommu\n");
- goto err_pm_disable;
+ goto err_sysfs_remove;
}
if (qcom_iommu->local_base) {
@@ -871,6 +871,8 @@ static int qcom_iommu_device_probe(struct platform_device *pdev)
return 0;
+err_sysfs_remove:
+ iommu_device_sysfs_remove(&qcom_iommu->iommu);
err_pm_disable:
pm_runtime_disable(dev);
return ret;
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 0f0caf59023c..059463664b41 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -1648,13 +1648,16 @@ void *iommu_dma_alloc(struct device *dev, size_t size, dma_addr_t *handle,
}
if (IS_ENABLED(CONFIG_DMA_DIRECT_REMAP) &&
- !gfpflags_allow_blocking(gfp) && !coherent)
+ !gfpflags_allow_blocking(gfp) && !coherent) {
page = dma_alloc_from_pool(dev, PAGE_ALIGN(size), &cpu_addr,
- gfp, NULL);
- else
+ gfp, NULL);
+ if (!page)
+ return NULL;
+ } else {
cpu_addr = iommu_dma_alloc_pages(dev, size, &page, gfp, attrs);
- if (!cpu_addr)
- return NULL;
+ if (!cpu_addr)
+ return NULL;
+ }
*handle = __iommu_dma_map(dev, page_to_phys(page), size, ioprot,
dev->coherent_dma_mask);
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 85a6a74f02da..6c1e33b84bd5 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1607,7 +1607,8 @@ static void copied_context_tear_down(struct intel_iommu *iommu,
assert_spin_locked(&iommu->lock);
did_old = context_domain_id(context);
- context_clear_entry(context);
+ context_clear_present(context);
+ __iommu_flush_cache(iommu, context, sizeof(*context));
if (did_old < cap_ndoms(iommu->cap)) {
iommu->flush.flush_context(iommu, did_old,
@@ -1618,6 +1619,9 @@ static void copied_context_tear_down(struct intel_iommu *iommu,
DMA_TLB_DSI_FLUSH);
}
+ context_clear_entry(context);
+ __iommu_flush_cache(iommu, context, sizeof(*context));
+
clear_context_copied(iommu, bus, devfn);
}
@@ -2111,7 +2115,7 @@ static int copy_context_table(struct intel_iommu *iommu,
struct context_entry **tbl,
int bus, bool ext)
{
- int tbl_idx, pos = 0, idx, devfn, ret = 0, did;
+ int tbl_idx, tbl_slot = 0, idx, devfn, ret = 0, did;
struct context_entry *new_ce = NULL, ce;
struct context_entry *old_ce = NULL;
struct root_entry re;
@@ -2127,10 +2131,9 @@ static int copy_context_table(struct intel_iommu *iommu,
if (idx == 0) {
/* First save what we may have and clean up */
if (new_ce) {
- tbl[tbl_idx] = new_ce;
+ tbl[tbl_idx + tbl_slot] = new_ce;
__iommu_flush_cache(iommu, new_ce,
VTD_PAGE_SIZE);
- pos = 1;
}
if (old_ce)
@@ -2152,6 +2155,9 @@ static int copy_context_table(struct intel_iommu *iommu,
}
}
+ /* Track if saving UCTP or LCTP entries in scalable mode */
+ tbl_slot = ext && devfn >= 0x80 ? 1 : 0;
+
ret = -ENOMEM;
old_ce = memremap(old_ce_phys, PAGE_SIZE,
MEMREMAP_WB);
@@ -2179,7 +2185,7 @@ static int copy_context_table(struct intel_iommu *iommu,
new_ce[idx] = ce;
}
- tbl[tbl_idx + pos] = new_ce;
+ tbl[tbl_idx + tbl_slot] = new_ce;
__iommu_flush_cache(iommu, new_ce, VTD_PAGE_SIZE);
@@ -3991,6 +3997,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
return &iommu->iommu;
free_table:
+ intel_pasid_teardown_sm_context(dev);
intel_pasid_free_table(dev);
clear_rbtree:
device_rbtree_remove(info);
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index aed260d4a93c..8ec251acd923 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -10,6 +10,7 @@
#include <linux/iommu.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
+#include <linux/sizes.h>
#include <linux/xarray.h>
#include <uapi/linux/iommufd.h>
@@ -1359,6 +1360,9 @@ static int iommufd_test_dirty(struct iommufd_ucmd *ucmd, unsigned int mockpt_id,
if (!page_size || !length || iova % page_size || length % page_size ||
!uptr)
return -EINVAL;
+ max = length / page_size;
+ if (max > SZ_16M * BITS_PER_BYTE)
+ return -EOVERFLOW;
hwpt = get_md_pagetable(ucmd, mockpt_id, &mock);
if (IS_ERR(hwpt))
@@ -1369,7 +1373,6 @@ static int iommufd_test_dirty(struct iommufd_ucmd *ucmd, unsigned int mockpt_id,
goto out_put;
}
- max = length / page_size;
tmp = kvzalloc(DIV_ROUND_UP(max, BITS_PER_LONG) * sizeof(unsigned long),
GFP_KERNEL_ACCOUNT);
if (!tmp) {
diff --git a/drivers/iommu/msm_iommu.c b/drivers/iommu/msm_iommu.c
index 989e0869d805..036bacd162c7 100644
--- a/drivers/iommu/msm_iommu.c
+++ b/drivers/iommu/msm_iommu.c
@@ -720,7 +720,7 @@ static int msm_iommu_probe(struct platform_device *pdev)
iommu = devm_kzalloc(&pdev->dev, sizeof(*iommu), GFP_KERNEL);
if (!iommu)
- return -ENODEV;
+ return -ENOMEM;
iommu->dev = &pdev->dev;
INIT_LIST_HEAD(&iommu->ctx_list);
diff --git a/drivers/iommu/mtk_iommu_v1.c b/drivers/iommu/mtk_iommu_v1.c
index 5a3eb7b587b4..378a09faa04c 100644
--- a/drivers/iommu/mtk_iommu_v1.c
+++ b/drivers/iommu/mtk_iommu_v1.c
@@ -78,7 +78,7 @@
/* MTK generation one iommu HW only support 4K size mapping */
#define MT2701_IOMMU_PAGE_SHIFT 12
#define MT2701_IOMMU_PAGE_SIZE (1UL << MT2701_IOMMU_PAGE_SHIFT)
-#define MT2701_LARB_NR_MAX 3
+#define MT2701_LARB_NR_MAX 4
/*
* MTK m4u support 4GB iova address space, and only support 4K page
diff --git a/drivers/ipack/devices/ipoctal.c b/drivers/ipack/devices/ipoctal.c
index ba2e9e52d72b..474b58858dca 100644
--- a/drivers/ipack/devices/ipoctal.c
+++ b/drivers/ipack/devices/ipoctal.c
@@ -10,6 +10,7 @@
#include <linux/device.h>
#include <linux/module.h>
#include <linux/interrupt.h>
+#include <linux/kref.h>
#include <linux/sched.h>
#include <linux/tty.h>
#include <linux/serial.h>
@@ -25,6 +26,8 @@
static const struct tty_operations ipoctal_fops;
+static void ipoctal_release(struct kref *kref);
+
struct ipoctal_channel {
struct ipoctal_stats stats;
unsigned int nb_bytes;
@@ -49,6 +52,9 @@ struct ipoctal {
struct tty_driver *tty_drv;
u8 __iomem *mem8_space;
u8 __iomem *int_space;
+ struct kref kref;
+ struct module *carrier_owner;
+ bool removed;
};
static inline struct ipoctal *chan_to_ipoctal(struct ipoctal_channel *chan,
@@ -70,8 +76,14 @@ static void ipoctal_reset_channel(struct ipoctal_channel *channel)
static int ipoctal_port_activate(struct tty_port *port, struct tty_struct *tty)
{
struct ipoctal_channel *channel;
+ struct ipoctal *ipoctal;
channel = dev_get_drvdata(tty->dev);
+ ipoctal = chan_to_ipoctal(channel, tty->index);
+
+
+ if (ipoctal->removed)
+ return -ENODEV;
/*
* Enable RX. TX will be enabled when
@@ -95,6 +107,7 @@ static int ipoctal_install(struct tty_driver *driver, struct tty_struct *tty)
if (res)
goto err_put_carrier;
+ kref_get(&ipoctal->kref);
tty->driver_data = channel;
return 0;
@@ -460,8 +473,13 @@ static ssize_t ipoctal_write_tty(struct tty_struct *tty, const u8 *buf,
size_t count)
{
struct ipoctal_channel *channel = tty->driver_data;
+ struct ipoctal *ipoctal = chan_to_ipoctal(channel, tty->index);
size_t char_copied;
+
+ if (ipoctal->removed || !channel->tty_port.xmit_buf)
+ return 0;
+
char_copied = ipoctal_copy_write_buffer(channel, buf, count);
/* As the IP-OCTAL 485 only supports half duplex, do it manually */
@@ -501,8 +519,13 @@ static void ipoctal_set_termios(struct tty_struct *tty,
unsigned char mr2 = 0;
unsigned char csr = 0;
struct ipoctal_channel *channel = tty->driver_data;
+ struct ipoctal *ipoctal = chan_to_ipoctal(channel, tty->index);
speed_t baud;
+
+ if (ipoctal->removed)
+ return;
+
cflag = tty->termios.c_cflag;
/* Disable and reset everything before change the setup */
@@ -631,10 +654,16 @@ static void ipoctal_hangup(struct tty_struct *tty)
{
unsigned long flags;
struct ipoctal_channel *channel = tty->driver_data;
+ struct ipoctal *ipoctal;
if (channel == NULL)
return;
+ ipoctal = chan_to_ipoctal(channel, tty->index);
+
+ if (ipoctal->removed)
+ return;
+
spin_lock_irqsave(&channel->lock, flags);
channel->nb_bytes = 0;
channel->pointer_read = 0;
@@ -651,10 +680,16 @@ static void ipoctal_hangup(struct tty_struct *tty)
static void ipoctal_shutdown(struct tty_struct *tty)
{
struct ipoctal_channel *channel = tty->driver_data;
+ struct ipoctal *ipoctal;
if (channel == NULL)
return;
+ ipoctal = chan_to_ipoctal(channel, tty->index);
+
+ if (ipoctal->removed)
+ return;
+
ipoctal_reset_channel(channel);
tty_port_set_initialized(&channel->tty_port, false);
}
@@ -664,8 +699,9 @@ static void ipoctal_cleanup(struct tty_struct *tty)
struct ipoctal_channel *channel = tty->driver_data;
struct ipoctal *ipoctal = chan_to_ipoctal(channel, tty->index);
- /* release the carrier driver */
- ipack_put_carrier(ipoctal->dev);
+ /* release the carrier driver via cached owner */
+ module_put(ipoctal->carrier_owner);
+ kref_put(&ipoctal->kref, ipoctal_release);
}
static const struct tty_operations ipoctal_fops = {
@@ -683,6 +719,13 @@ static const struct tty_operations ipoctal_fops = {
.cleanup = ipoctal_cleanup,
};
+static void ipoctal_release(struct kref *kref)
+{
+ struct ipoctal *ipoctal = container_of(kref, struct ipoctal, kref);
+
+ kfree(ipoctal);
+}
+
static int ipoctal_probe(struct ipack_device *dev)
{
int res;
@@ -692,7 +735,10 @@ static int ipoctal_probe(struct ipack_device *dev)
if (ipoctal == NULL)
return -ENOMEM;
+ kref_init(&ipoctal->kref);
+
ipoctal->dev = dev;
+ ipoctal->carrier_owner = dev->bus->owner;
res = ipoctal_inst_slot(ipoctal, dev->bus->bus_nr, dev->slot);
if (res)
goto out_uninst;
@@ -701,7 +747,7 @@ static int ipoctal_probe(struct ipack_device *dev)
return 0;
out_uninst:
- kfree(ipoctal);
+ kref_put(&ipoctal->kref, ipoctal_release);
return res;
}
@@ -709,6 +755,8 @@ static void __ipoctal_remove(struct ipoctal *ipoctal)
{
int i;
+ ipoctal->removed = true;
+
ipoctal->dev->bus->ops->free_irq(ipoctal->dev);
for (i = 0; i < NR_CHANNELS; i++) {
@@ -725,7 +773,7 @@ static void __ipoctal_remove(struct ipoctal *ipoctal)
tty_unregister_driver(ipoctal->tty_drv);
kfree(ipoctal->tty_drv->name);
tty_driver_kref_put(ipoctal->tty_drv);
- kfree(ipoctal);
+ kref_put(&ipoctal->kref, ipoctal_release);
}
static void ipoctal_remove(struct ipack_device *idev)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 95d337978c75..b489594df329 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -4503,6 +4503,13 @@ static int its_vpe_init(struct its_vpe *vpe)
static void its_vpe_teardown(struct its_vpe *vpe)
{
+ /*
+ * If vpt_page is NULL, then its_vpe_init() has failed, and
+ * there is nothing to do as no resource has been allocated.
+ */
+ if (vpe->vpt_page == NULL)
+ return;
+
its_vpe_db_proxy_unmap(vpe);
its_vpe_id_free(vpe->vpe_id);
its_free_pending_table(vpe->vpt_page);
@@ -4583,8 +4590,10 @@ static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq
irqd_set_resend_when_in_progress(irq_get_irq_data(virq + i));
}
- if (err)
+ if (err) {
+ its_vpe_teardown(vm->vpes[i]);
its_vpe_irq_domain_free(domain, virq, i);
+ }
return err;
}
@@ -5216,7 +5225,7 @@ static int __init its_probe_one(struct its_node *its)
err = its_init_domain(its);
if (err)
- goto out_free_tables;
+ goto out_free_collection;
raw_spin_lock(&its_lock);
list_add(&its->entry, &its_nodes);
@@ -5224,6 +5233,8 @@ static int __init its_probe_one(struct its_node *its)
return 0;
+out_free_collection:
+ kfree(its->collections);
out_free_tables:
its_free_tables(its);
out_free_cmd:
@@ -5636,9 +5647,13 @@ static int __init gic_acpi_parse_madt_its(union acpi_subtable_headers *header,
its->flags |= ITS_FLAGS_FORCE_NON_SHAREABLE;
err = its_probe_one(its);
- if (!err)
- return 0;
+ if (err)
+ goto probe_err;
+
+ return 0;
+probe_err:
+ its_node_destroy(its);
node_err:
iort_deregister_domain_token(its_entry->translation_id);
dom_err:
diff --git a/drivers/irqchip/irq-mbigen.c b/drivers/irqchip/irq-mbigen.c
index 6f69f4e5dbac..12919836dadb 100644
--- a/drivers/irqchip/irq-mbigen.c
+++ b/drivers/irqchip/irq-mbigen.c
@@ -64,20 +64,6 @@ struct mbigen_device {
void __iomem *base;
};
-static inline unsigned int get_mbigen_node_offset(unsigned int nid)
-{
- unsigned int offset = nid * MBIGEN_NODE_OFFSET;
-
- /*
- * To avoid touched clear register in unexpected way, we need to directly
- * skip clear register when access to more than 10 mbigen nodes.
- */
- if (nid >= (REG_MBIGEN_CLEAR_OFFSET / MBIGEN_NODE_OFFSET))
- offset += MBIGEN_NODE_OFFSET;
-
- return offset;
-}
-
static inline unsigned int get_mbigen_vec_reg(irq_hw_number_t hwirq)
{
unsigned int nid, pin;
@@ -86,7 +72,8 @@ static inline unsigned int get_mbigen_vec_reg(irq_hw_number_t hwirq)
nid = hwirq / IRQS_PER_MBIGEN_NODE + 1;
pin = hwirq % IRQS_PER_MBIGEN_NODE;
- return pin * 4 + get_mbigen_node_offset(nid) + REG_MBIGEN_VEC_OFFSET;
+ return pin * 4 + nid * MBIGEN_NODE_OFFSET
+ + REG_MBIGEN_VEC_OFFSET;
}
static inline void get_mbigen_type_reg(irq_hw_number_t hwirq,
@@ -101,7 +88,8 @@ static inline void get_mbigen_type_reg(irq_hw_number_t hwirq,
*mask = 1 << (irq_ofst % 32);
ofst = irq_ofst / 32 * 4;
- *addr = ofst + get_mbigen_node_offset(nid) + REG_MBIGEN_TYPE_OFFSET;
+ *addr = ofst + nid * MBIGEN_NODE_OFFSET
+ + REG_MBIGEN_TYPE_OFFSET;
}
static inline void get_mbigen_clear_reg(irq_hw_number_t hwirq,
diff --git a/drivers/irqchip/irq-renesas-irqc.c b/drivers/irqchip/irq-renesas-irqc.c
index 76026e0b8e20..53ab8662a63b 100644
--- a/drivers/irqchip/irq-renesas-irqc.c
+++ b/drivers/irqchip/irq-renesas-irqc.c
@@ -176,6 +176,7 @@ static int irqc_probe(struct platform_device *pdev)
goto err_runtime_pm_disable;
}
+ p->irq_domain->flags |= IRQ_DOMAIN_FLAG_DESTROY_GC;
ret = irq_alloc_domain_generic_chips(p->irq_domain, p->number_of_irqs,
1, "irqc", handle_level_irq,
0, 0, IRQ_GC_INIT_NESTED_LOCK);
diff --git a/drivers/irqchip/irq-renesas-rzg2l.c b/drivers/irqchip/irq-renesas-rzg2l.c
index d83dfc10ff49..ac88d5aa46c2 100644
--- a/drivers/irqchip/irq-renesas-rzg2l.c
+++ b/drivers/irqchip/irq-renesas-rzg2l.c
@@ -104,7 +104,7 @@ static void rzg2l_clear_irq_int(struct rzg2l_irqc_priv *priv, unsigned int hwirq
* falling/rising-edge.
*/
if ((iscr & bit) && (iitsr & IITSR_IITSEL_MASK(hw_irq))) {
- writel_relaxed(iscr & ~bit, priv->base + ISCR);
+ writel_relaxed(~bit, priv->base + ISCR);
/*
* Enforce that the posted write is flushed to prevent that the
* just handled interrupt is raised again.
@@ -120,7 +120,7 @@ static void rzg2l_clear_tint_int(struct rzg2l_irqc_priv *priv, unsigned int hwir
reg = readl_relaxed(priv->base + TSCR);
if (reg & bit) {
- writel_relaxed(reg & ~bit, priv->base + TSCR);
+ writel_relaxed(~bit, priv->base + TSCR);
/*
* Enforce that the posted write is flushed to prevent that the
* just handled interrupt is raised again.
diff --git a/drivers/irqchip/irq-stm32mp-exti.c b/drivers/irqchip/irq-stm32mp-exti.c
index 33e0cfdea654..85de5bb49fb1 100644
--- a/drivers/irqchip/irq-stm32mp-exti.c
+++ b/drivers/irqchip/irq-stm32mp-exti.c
@@ -23,7 +23,7 @@
#define IRQS_PER_BANK 32
-#define HWSPNLCK_TIMEOUT 1000 /* usec */
+#define HWSPNLCK_TIMEOUT_MS 1
#define EXTI_EnCIDCFGR(n) (0x180 + (n) * 4)
#define EXTI_HWCFGR1 0x3f0
@@ -377,7 +377,7 @@ static int stm32mp_exti_set_type(struct irq_data *d, unsigned int type)
raw_spin_lock(&chip_data->rlock);
if (hwlock) {
- err = hwspin_lock_timeout_in_atomic(hwlock, HWSPNLCK_TIMEOUT);
+ err = hwspin_lock_timeout_in_atomic(hwlock, HWSPNLCK_TIMEOUT_MS);
if (err) {
pr_err("%s can't get hwspinlock (%d)\n", __func__, err);
goto unlock;
diff --git a/drivers/leds/leds-pca9532.c b/drivers/leds/leds-pca9532.c
index 1b47acf54720..4ecdfee14809 100644
--- a/drivers/leds/leds-pca9532.c
+++ b/drivers/leds/leds-pca9532.c
@@ -324,9 +324,9 @@ static void pca9532_gpio_set_value(struct gpio_chip *gc, unsigned offset, int va
struct pca9532_led *led = &data->leds[offset];
if (val)
- led->state = PCA9532_ON;
- else
led->state = PCA9532_OFF;
+ else
+ led->state = PCA9532_ON;
pca9532_setled(led);
}
@@ -344,7 +344,7 @@ static int pca9532_gpio_get_value(struct gpio_chip *gc, unsigned offset)
static int pca9532_gpio_direction_input(struct gpio_chip *gc, unsigned offset)
{
/* To use as input ensure pin is not driven */
- pca9532_gpio_set_value(gc, offset, 0);
+ pca9532_gpio_set_value(gc, offset, 1);
return 0;
}
@@ -394,10 +394,14 @@ static int pca9532_configure(struct i2c_client *client,
for (i = 0; i < 2; i++) {
data->pwm[i] = pdata->pwm[i];
data->psc[i] = pdata->psc[i];
- i2c_smbus_write_byte_data(client, PCA9532_REG_PWM(maxleds, i),
- data->pwm[i]);
- i2c_smbus_write_byte_data(client, PCA9532_REG_PSC(maxleds, i),
- data->psc[i]);
+ err = i2c_smbus_write_byte_data(client, PCA9532_REG_PWM(maxleds, i),
+ data->pwm[i]);
+ if (err < 0)
+ return err;
+ err = i2c_smbus_write_byte_data(client, PCA9532_REG_PSC(maxleds, i),
+ data->psc[i]);
+ if (err < 0)
+ return err;
}
data->hw_blink = true;
diff --git a/drivers/mailbox/pcc.c b/drivers/mailbox/pcc.c
index 1d4191cb9138..6efeadf89db2 100644
--- a/drivers/mailbox/pcc.c
+++ b/drivers/mailbox/pcc.c
@@ -93,12 +93,11 @@ struct pcc_chan_reg {
* @plat_irq: platform interrupt
* @type: PCC subspace type
* @plat_irq_flags: platform interrupt flags
- * @chan_in_use: this flag is used just to check if the interrupt needs
- * handling when it is shared. Since only one transfer can occur
- * at a time and mailbox takes care of locking, this flag can be
- * accessed without a lock. Note: the type only support the
- * communication from OSPM to Platform, like type3, use it, and
- * other types completely ignore it.
+ * @chan_in_use: lockless flag used by type 3 initiator subspaces to filter
+ * platform interrupts. Only one transfer can occur at a time, but
+ * the interrupt handler may sample the flag on another CPU, so all
+ * accesses must use READ_ONCE() or WRITE_ONCE(). Other subspace
+ * types do not test it.
*/
struct pcc_chan_info {
struct pcc_mbox_chan chan;
@@ -333,8 +332,13 @@ static irqreturn_t pcc_mbox_irq(int irq, void *p)
if (pcc_chan_reg_read_modify_write(&pchan->plat_irq_ack))
return IRQ_NONE;
+ /*
+ * Initiator subspaces use this flag to filter shared interrupts. Use
+ * READ_ONCE() to sample the lockless flag written by pcc_send_data()
+ * on another CPU.
+ */
if (pchan->type == ACPI_PCCT_TYPE_EXT_PCC_MASTER_SUBSPACE &&
- !pchan->chan_in_use)
+ !READ_ONCE(pchan->chan_in_use))
return IRQ_NONE;
if (!pcc_mbox_cmd_complete_check(pchan))
@@ -344,12 +348,12 @@ static irqreturn_t pcc_mbox_irq(int irq, void *p)
return IRQ_NONE;
/*
- * Clear this flag after updating interrupt ack register and just
- * before mbox_chan_received_data() which might call pcc_send_data()
- * where the flag is set again to start new transfer. This is
- * required to avoid any possible race in updatation of this flag.
+ * Clear this flag after updating the interrupt ack register and before
+ * notifying the client and mailbox core. mbox_chan_txdone() may submit
+ * the next queued transfer and set the flag again. Use WRITE_ONCE() for
+ * the lockless update observed by the send and interrupt paths.
*/
- pchan->chan_in_use = false;
+ WRITE_ONCE(pchan->chan_in_use, false);
mbox_chan_received_data(chan, NULL);
check_and_ack(pchan, chan);
@@ -372,6 +376,7 @@ static irqreturn_t pcc_mbox_irq(int irq, void *p)
struct pcc_mbox_chan *
pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id)
{
+ struct pcc_mbox_chan *pcc_mchan;
struct pcc_chan_info *pchan;
struct mbox_chan *chan;
int rc;
@@ -390,7 +395,14 @@ pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id)
if (rc)
return ERR_PTR(rc);
- return &pchan->chan;
+ pcc_mchan = &pchan->chan;
+ pcc_mchan->shmem = acpi_os_ioremap(pcc_mchan->shmem_base_addr,
+ pcc_mchan->shmem_size);
+ if (pcc_mchan->shmem)
+ return pcc_mchan;
+
+ mbox_free_channel(chan);
+ return ERR_PTR(-ENXIO);
}
EXPORT_SYMBOL_GPL(pcc_mbox_request_channel);
@@ -419,25 +431,6 @@ void pcc_mbox_free_channel(struct pcc_mbox_chan *pchan)
}
EXPORT_SYMBOL_GPL(pcc_mbox_free_channel);
-int pcc_mbox_ioremap(struct mbox_chan *chan)
-{
- struct pcc_chan_info *pchan_info;
- struct pcc_mbox_chan *pcc_mbox_chan;
-
- if (!chan || !chan->cl)
- return -1;
- pchan_info = chan->con_priv;
- pcc_mbox_chan = &pchan_info->chan;
-
- pcc_mbox_chan->shmem = acpi_os_ioremap(pcc_mbox_chan->shmem_base_addr,
- pcc_mbox_chan->shmem_size);
- if (!pcc_mbox_chan->shmem)
- return -ENXIO;
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(pcc_mbox_ioremap);
-
/**
* pcc_send_data - Called from Mailbox Controller code. Used
* here only to ring the channel doorbell. The PCC client
@@ -459,9 +452,18 @@ static int pcc_send_data(struct mbox_chan *chan, void *data)
if (ret)
return ret;
+ /*
+ * Set chan_in_use before ringing the doorbell so a fast completion
+ * interrupt is not mistaken for a shared interrupt from another
+ * subspace. Use WRITE_ONCE() for the lockless flag update. The
+ * ordered I/O accessor used to ring the doorbell orders this store
+ * before the platform is notified.
+ */
+ if (pchan->plat_irq > 0)
+ WRITE_ONCE(pchan->chan_in_use, true);
ret = pcc_chan_reg_read_modify_write(&pchan->db);
- if (!ret && pchan->plat_irq > 0)
- pchan->chan_in_use = true;
+ if (ret && pchan->plat_irq > 0)
+ WRITE_ONCE(pchan->chan_in_use, false);
return ret;
}
diff --git a/drivers/mailbox/qcom-cpucp-mbox.c b/drivers/mailbox/qcom-cpucp-mbox.c
index 44f4ed15f818..57f731418faf 100644
--- a/drivers/mailbox/qcom-cpucp-mbox.c
+++ b/drivers/mailbox/qcom-cpucp-mbox.c
@@ -56,14 +56,25 @@ static irqreturn_t qcom_cpucp_mbox_irq_fn(int irq, void *data)
for_each_set_bit(i, (unsigned long *)&status, APSS_CPUCP_IPC_CHAN_SUPPORTED) {
u32 val = readl(cpucp->rx_base + APSS_CPUCP_RX_MBOX_CMD(i) + APSS_CPUCP_MBOX_CMD_OFF);
struct mbox_chan *chan = &cpucp->chans[i];
+ struct mbox_client *cl;
unsigned long flags;
- /* Provide mutual exclusion with changes to chan->cl */
+ /*
+ * Provide mutual exclusion with changes to chan->cl.
+ * Save cl locally and clear the HW interrupt inside the lock,
+ * then invoke mbox_chan_received_data() outside the lock to
+ * avoid a PREEMPT_RT self-deadlock: mbox_chan_received_data()
+ * can call back into mbox_send_message() via scmi_rx_callback()
+ * -> mailbox_clear_channel(), which re-acquires chan->lock
+ * (converted to an rt_spinlock under PREEMPT_RT).
+ */
spin_lock_irqsave(&chan->lock, flags);
- if (chan->cl)
- mbox_chan_received_data(chan, &val);
+ cl = chan->cl;
writeq(BIT(i), cpucp->rx_base + APSS_CPUCP_RX_MBOX_CLEAR);
spin_unlock_irqrestore(&chan->lock, flags);
+
+ if (cl)
+ mbox_chan_received_data(chan, &val);
}
return IRQ_HANDLED;
@@ -99,6 +110,14 @@ static int qcom_cpucp_mbox_send_data(struct mbox_chan *chan, void *data)
unsigned long chan_id = channel_number(chan);
u32 *val = data;
+ /*
+ * mailbox_clear_channel() calls mbox_send_message() with NULL data to
+ * signal the remote side that the channel has been cleared. Nothing
+ * needs to be written to the TX register in that case, so just return.
+ */
+ if (!val)
+ return 0;
+
writel(*val, cpucp->tx_base + APSS_CPUCP_TX_MBOX_CMD(chan_id) + APSS_CPUCP_MBOX_CMD_OFF);
return 0;
diff --git a/drivers/mailbox/rockchip-mailbox.c b/drivers/mailbox/rockchip-mailbox.c
index 4d966cb2ed03..5775362de508 100644
--- a/drivers/mailbox/rockchip-mailbox.c
+++ b/drivers/mailbox/rockchip-mailbox.c
@@ -201,19 +201,10 @@ static int rockchip_mbox_probe(struct platform_device *pdev)
/* Each channel has two buffers for A2B and B2A */
mb->buf_size = (size_t)resource_size(res) / (drv_data->num_chans * 2);
- mb->pclk = devm_clk_get(&pdev->dev, "pclk_mailbox");
- if (IS_ERR(mb->pclk)) {
- ret = PTR_ERR(mb->pclk);
- dev_err(&pdev->dev, "failed to get pclk_mailbox clock: %d\n",
- ret);
- return ret;
- }
-
- ret = clk_prepare_enable(mb->pclk);
- if (ret) {
- dev_err(&pdev->dev, "failed to enable pclk: %d\n", ret);
- return ret;
- }
+ mb->pclk = devm_clk_get_enabled(&pdev->dev, "pclk_mailbox");
+ if (IS_ERR(mb->pclk))
+ return dev_err_probe(&pdev->dev, PTR_ERR(mb->pclk),
+ "failed to get and enable pclk_mailbox clock\n");
for (i = 0; i < mb->mbox.num_chans; i++) {
irq = platform_get_irq(pdev, i);
diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c
index b7f1cd81ab05..507862369029 100644
--- a/drivers/md/dm-raid.c
+++ b/drivers/md/dm-raid.c
@@ -438,7 +438,7 @@ static bool rs_is_reshapable(struct raid_set *rs)
/* Return true, if raid set in @rs is recovering */
static bool rs_is_recovering(struct raid_set *rs)
{
- return rs->md.recovery_cp < rs->md.dev_sectors;
+ return rs->md.resync_offset < rs->md.dev_sectors;
}
/* Return true, if raid set in @rs is reshaping */
@@ -768,7 +768,7 @@ static struct raid_set *raid_set_alloc(struct dm_target *ti, struct raid_type *r
rs->md.layout = raid_type->algorithm;
rs->md.new_layout = rs->md.layout;
rs->md.delta_disks = 0;
- rs->md.recovery_cp = MaxSector;
+ rs->md.resync_offset = MaxSector;
for (i = 0; i < raid_devs; i++)
md_rdev_init(&rs->dev[i].rdev);
@@ -912,7 +912,7 @@ static int parse_dev_params(struct raid_set *rs, struct dm_arg_set *as)
rs->md.external = 0;
rs->md.persistent = 1;
rs->md.major_version = 2;
- } else if (rebuild && !rs->md.recovery_cp) {
+ } else if (rebuild && !rs->md.resync_offset) {
/*
* Without metadata, we will not be able to tell if the array
* is in-sync or not - we must assume it is not. Therefore,
@@ -1699,20 +1699,20 @@ static void rs_setup_recovery(struct raid_set *rs, sector_t dev_sectors)
{
/* raid0 does not recover */
if (rs_is_raid0(rs))
- rs->md.recovery_cp = MaxSector;
+ rs->md.resync_offset = MaxSector;
/*
* A raid6 set has to be recovered either
* completely or for the grown part to
* ensure proper parity and Q-Syndrome
*/
else if (rs_is_raid6(rs))
- rs->md.recovery_cp = dev_sectors;
+ rs->md.resync_offset = dev_sectors;
/*
* Other raid set types may skip recovery
* depending on the 'nosync' flag.
*/
else
- rs->md.recovery_cp = test_bit(__CTR_FLAG_NOSYNC, &rs->ctr_flags)
+ rs->md.resync_offset = test_bit(__CTR_FLAG_NOSYNC, &rs->ctr_flags)
? MaxSector : dev_sectors;
}
@@ -2147,7 +2147,7 @@ static void super_sync(struct mddev *mddev, struct md_rdev *rdev)
sb->events = cpu_to_le64(mddev->events);
sb->disk_recovery_offset = cpu_to_le64(rdev->recovery_offset);
- sb->array_resync_offset = cpu_to_le64(mddev->recovery_cp);
+ sb->array_resync_offset = cpu_to_le64(mddev->resync_offset);
sb->level = cpu_to_le32(mddev->level);
sb->layout = cpu_to_le32(mddev->layout);
@@ -2340,18 +2340,18 @@ static int super_init_validation(struct raid_set *rs, struct md_rdev *rdev)
}
if (!test_bit(__CTR_FLAG_NOSYNC, &rs->ctr_flags))
- mddev->recovery_cp = le64_to_cpu(sb->array_resync_offset);
+ mddev->resync_offset = le64_to_cpu(sb->array_resync_offset);
/*
* During load, we set FirstUse if a new superblock was written.
* There are two reasons we might not have a superblock:
* 1) The raid set is brand new - in which case, all of the
* devices must have their In_sync bit set. Also,
- * recovery_cp must be 0, unless forced.
+ * resync_offset must be 0, unless forced.
* 2) This is a new device being added to an old raid set
* and the new device needs to be rebuilt - in which
* case the In_sync bit will /not/ be set and
- * recovery_cp must be MaxSector.
+ * resync_offset must be MaxSector.
* 3) This is/are a new device(s) being added to an old
* raid set during takeover to a higher raid level
* to provide capacity for redundancy or during reshape
@@ -2396,8 +2396,8 @@ static int super_init_validation(struct raid_set *rs, struct md_rdev *rdev)
new_devs > 1 ? "s" : "");
return -EINVAL;
} else if (!test_bit(__CTR_FLAG_REBUILD, &rs->ctr_flags) && rs_is_recovering(rs)) {
- DMERR("'rebuild' specified while raid set is not in-sync (recovery_cp=%llu)",
- (unsigned long long) mddev->recovery_cp);
+ DMERR("'rebuild' specified while raid set is not in-sync (resync_offset=%llu)",
+ (unsigned long long) mddev->resync_offset);
return -EINVAL;
} else if (rs_is_reshaping(rs)) {
DMERR("'rebuild' specified while raid set is being reshaped (reshape_position=%llu)",
@@ -2702,11 +2702,11 @@ static int rs_adjust_data_offsets(struct raid_set *rs)
}
out:
/*
- * Raise recovery_cp in case data_offset != 0 to
+ * Raise resync_offset in case data_offset != 0 to
* avoid false recovery positives in the constructor.
*/
- if (rs->md.recovery_cp < rs->md.dev_sectors)
- rs->md.recovery_cp += rs->dev[0].rdev.data_offset;
+ if (rs->md.resync_offset < rs->md.dev_sectors)
+ rs->md.resync_offset += rs->dev[0].rdev.data_offset;
/* Adjust data offsets on all rdevs but on any raid4/5/6 journal device */
rdev_for_each(rdev, &rs->md) {
@@ -2761,7 +2761,7 @@ static int rs_setup_takeover(struct raid_set *rs)
}
clear_bit(MD_ARRAY_FIRST_USE, &mddev->flags);
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
while (d--) {
rdev = &rs->dev[d].rdev;
@@ -2769,7 +2769,7 @@ static int rs_setup_takeover(struct raid_set *rs)
if (test_bit(d, (void *) rs->rebuild_disks)) {
clear_bit(In_sync, &rdev->flags);
clear_bit(Faulty, &rdev->flags);
- mddev->recovery_cp = rdev->recovery_offset = 0;
+ mddev->resync_offset = rdev->recovery_offset = 0;
/* Bitmap has to be created when we do an "up" takeover */
set_bit(MD_ARRAY_FIRST_USE, &mddev->flags);
}
@@ -3227,7 +3227,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv)
if (r)
goto bad;
- rs_setup_recovery(rs, rs->md.recovery_cp < rs->md.dev_sectors ? rs->md.recovery_cp : rs->md.dev_sectors);
+ rs_setup_recovery(rs, rs->md.resync_offset < rs->md.dev_sectors ? rs->md.resync_offset : rs->md.dev_sectors);
} else {
/* This is no size change or it is shrinking, update size and record in superblocks */
r = rs_set_dev_and_array_sectors(rs, rs->ti->len, false);
@@ -3449,7 +3449,7 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery,
} else {
if (state == st_idle && !test_bit(MD_RECOVERY_INTR, &recovery))
- r = mddev->recovery_cp;
+ r = mddev->resync_offset;
else
r = mddev->curr_resync_completed;
@@ -4079,9 +4079,9 @@ static int raid_preresume(struct dm_target *ti)
}
/* Check for any resize/reshape on @rs and adjust/initiate */
- if (mddev->recovery_cp && mddev->recovery_cp < MaxSector) {
+ if (mddev->resync_offset && mddev->resync_offset < MaxSector) {
set_bit(MD_RECOVERY_REQUESTED, &mddev->recovery);
- mddev->resync_min = mddev->recovery_cp;
+ mddev->resync_min = mddev->resync_offset;
if (test_bit(RT_FLAG_RS_GROW, &rs->runtime_flags))
mddev->resync_max_sectors = mddev->dev_sectors;
}
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 269959c9204f..20f6c6ebace0 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -2608,9 +2608,10 @@ int dm_setup_md_queue(struct mapped_device *md, struct dm_table *t)
*/
mutex_lock(&md->table_devices_lock);
r = add_disk(md->disk);
- mutex_unlock(&md->table_devices_lock);
- if (r)
+ if (r) {
+ mutex_unlock(&md->table_devices_lock);
return r;
+ }
/*
* Register the holder relationship for devices added before the disk
@@ -2621,18 +2622,21 @@ int dm_setup_md_queue(struct mapped_device *md, struct dm_table *t)
if (r)
goto out_undo_holders;
}
+ mutex_unlock(&md->table_devices_lock);
r = dm_sysfs_init(md);
if (r)
- goto out_undo_holders;
+ goto lock_out_undo_holders;
md->type = type;
+
return 0;
+lock_out_undo_holders:
+ mutex_lock(&md->table_devices_lock);
out_undo_holders:
list_for_each_entry_continue_reverse(td, &md->table_devices, list)
bd_unlink_disk_holder(td->dm_dev.bdev, md->disk);
- mutex_lock(&md->table_devices_lock);
del_gendisk(md->disk);
mutex_unlock(&md->table_devices_lock);
return r;
@@ -3108,7 +3112,7 @@ int dm_resume(struct mapped_device *md)
r = -EINVAL;
mutex_lock_nested(&md->suspend_lock, SINGLE_DEPTH_NESTING);
- if (!dm_suspended_md(md))
+ if (!dm_suspended_md(md) || test_bit(DMF_FREEING, &md->flags))
goto out;
if (dm_suspended_internally_md(md)) {
diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c
index 21cdf48815f8..61271053e8ba 100644
--- a/drivers/md/md-bitmap.c
+++ b/drivers/md/md-bitmap.c
@@ -1976,12 +1976,12 @@ static void bitmap_dirty_bits(struct mddev *mddev, unsigned long s,
md_bitmap_set_memory_bits(bitmap, sec, 1);
md_bitmap_file_set_bit(bitmap, sec);
- if (sec < bitmap->mddev->recovery_cp)
+ if (sec < bitmap->mddev->resync_offset)
/* We are asserting that the array is dirty,
- * so move the recovery_cp address back so
+ * so move the resync_offset address back so
* that it is obvious that it is dirty
*/
- bitmap->mddev->recovery_cp = sec;
+ bitmap->mddev->resync_offset = sec;
}
}
@@ -2097,7 +2097,7 @@ static void bitmap_destroy(struct mddev *mddev)
return;
bitmap_wait_behind_writes(mddev);
- if (!mddev->serialize_policy)
+ if (!test_bit(MD_SERIALIZE_POLICY, &mddev->flags))
mddev_destroy_serial_pool(mddev, NULL);
mutex_lock(&mddev->bitmap_info.mutex);
@@ -2247,7 +2247,7 @@ static int bitmap_load(struct mddev *mddev)
|| bitmap->events_cleared == mddev->events)
/* no need to keep dirty bits to optimise a
* re-add of a missing device */
- start = mddev->recovery_cp;
+ start = mddev->resync_offset;
mutex_lock(&mddev->bitmap_info.mutex);
err = md_bitmap_init_from_disk(bitmap, start);
@@ -2612,10 +2612,12 @@ static ssize_t
location_store(struct mddev *mddev, const char *buf, size_t len)
{
int rv;
+ unsigned int noio_flags;
rv = mddev_suspend_and_lock(mddev);
if (rv)
return rv;
+ noio_flags = memalloc_noio_save();
if (mddev->pers) {
if (mddev->recovery || mddev->sync_thread) {
@@ -2689,6 +2691,7 @@ location_store(struct mddev *mddev, const char *buf, size_t len)
}
rv = 0;
out:
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
if (rv)
return rv;
@@ -2816,14 +2819,14 @@ backlog_store(struct mddev *mddev, const char *buf, size_t len)
if (!has_write_mostly) {
pr_warn_ratelimited("%s: can't set backlog, no write mostly device available\n",
mdname(mddev));
- mddev_unlock(mddev);
+ mddev_unlock_and_resume(mddev);
return -EINVAL;
}
mddev->bitmap_info.max_write_behind = backlog;
if (!backlog && mddev->serial_info_pool) {
/* serial_info_pool is not needed if backlog is zero */
- if (!mddev->serialize_policy)
+ if (!test_bit(MD_SERIALIZE_POLICY, &mddev->flags))
mddev_destroy_serial_pool(mddev, NULL);
} else if (backlog && !mddev->serial_info_pool) {
/* serial_info_pool is needed since backlog is not zero */
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index bf9ab478df2c..7ee6edccd20a 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -337,11 +337,11 @@ static void recover_bitmaps(struct md_thread *thread)
md_wakeup_thread(mddev->sync_thread);
if (hi > 0) {
- if (lo < mddev->recovery_cp)
- mddev->recovery_cp = lo;
+ if (lo < mddev->resync_offset)
+ mddev->resync_offset = lo;
/* wake up thread to continue resync in case resync
* is not finished */
- if (mddev->recovery_cp != MaxSector) {
+ if (mddev->resync_offset != MaxSector) {
/*
* clear the REMOTE flag since we will launch
* resync thread in current node.
@@ -868,9 +868,9 @@ static int gather_all_resync_info(struct mddev *mddev, int total_slots)
lockres_free(bm_lockres);
continue;
}
- if ((hi > 0) && (lo < mddev->recovery_cp)) {
+ if ((hi > 0) && (lo < mddev->resync_offset)) {
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
- mddev->recovery_cp = lo;
+ mddev->resync_offset = lo;
md_check_recovery(mddev);
}
@@ -1032,7 +1032,7 @@ static int leave(struct mddev *mddev)
* Also, we should send BITMAP_NEEDS_SYNC message in
* case reshaping is interrupted.
*/
- if ((cinfo->slot_number > 0 && mddev->recovery_cp != MaxSector) ||
+ if ((cinfo->slot_number > 0 && mddev->resync_offset != MaxSector) ||
(mddev->reshape_position != MaxSector &&
test_bit(MD_CLOSING, &mddev->flags)))
resync_bitmap(mddev);
@@ -1610,8 +1610,8 @@ static int gather_bitmaps(struct md_rdev *rdev)
pr_warn("md-cluster: Could not gather bitmaps from slot %d", sn);
goto out;
}
- if ((hi > 0) && (lo < mddev->recovery_cp))
- mddev->recovery_cp = lo;
+ if ((hi > 0) && (lo < mddev->resync_offset))
+ mddev->resync_offset = lo;
}
out:
return err;
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 02bf5d37888b..b3ba51d7da60 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -105,7 +105,7 @@ struct workqueue_struct *md_bitmap_wq;
static int remove_and_add_spares(struct mddev *mddev,
struct md_rdev *this);
static void mddev_detach(struct mddev *mddev);
-static void export_rdev(struct md_rdev *rdev, struct mddev *mddev);
+static void export_rdev(struct md_rdev *rdev);
static void md_wakeup_thread_directly(struct md_thread __rcu **thread);
/*
@@ -226,23 +226,22 @@ static int rdev_need_serial(struct md_rdev *rdev)
void mddev_create_serial_pool(struct mddev *mddev, struct md_rdev *rdev)
{
int ret = 0;
+ unsigned int noio_flags;
- if (rdev && !rdev_need_serial(rdev) &&
+ if (!test_bit(MD_SERIALIZE_POLICY, &mddev->flags) &&
+ rdev && !rdev_need_serial(rdev) &&
!test_bit(CollisionCheck, &rdev->flags))
return;
+ noio_flags = memalloc_noio_save();
if (!rdev)
ret = rdevs_init_serial(mddev);
else
ret = rdev_init_serial(rdev);
if (ret)
- return;
+ goto out;
if (mddev->serial_info_pool == NULL) {
- /*
- * already in memalloc noio context by
- * mddev_suspend()
- */
mddev->serial_info_pool =
mempool_create_kmalloc_pool(NR_SERIAL_INFOS,
sizeof(struct serial_info));
@@ -251,6 +250,8 @@ void mddev_create_serial_pool(struct mddev *mddev, struct md_rdev *rdev)
pr_err("can't alloc memory pool for serialization\n");
}
}
+out:
+ memalloc_noio_restore(noio_flags);
}
/*
@@ -270,7 +271,8 @@ void mddev_destroy_serial_pool(struct mddev *mddev, struct md_rdev *rdev)
rdev_for_each(temp, mddev) {
if (!rdev) {
- if (!mddev->serialize_policy ||
+ if (!test_bit(MD_SERIALIZE_POLICY,
+ &mddev->flags) ||
!rdev_need_serial(temp))
rdev_uninit_serial(temp);
else
@@ -500,9 +502,6 @@ int mddev_suspend(struct mddev *mddev, bool interruptible)
*/
WRITE_ONCE(mddev->suspended, mddev->suspended + 1);
- /* restrict memory reclaim I/O during raid array is suspend */
- mddev->noio_flag = memalloc_noio_save();
-
mutex_unlock(&mddev->suspend_mutex);
return 0;
}
@@ -519,9 +518,6 @@ static void __mddev_resume(struct mddev *mddev, bool recovery_needed)
return;
}
- /* entred the memalloc scope from mddev_suspend() */
- memalloc_noio_restore(mddev->noio_flag);
-
percpu_ref_resurrect(&mddev->active_io);
wake_up(&mddev->sb_wait);
@@ -865,7 +861,7 @@ void mddev_unlock(struct mddev *mddev)
list_for_each_entry_safe(rdev, tmp, &delete, same_set) {
list_del_init(&rdev->same_set);
kobject_del(&rdev->kobj);
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
}
}
EXPORT_SYMBOL_GPL(mddev_unlock);
@@ -1372,13 +1368,13 @@ static int super_90_validate(struct mddev *mddev, struct md_rdev *freshest, stru
mddev->layout = -1;
if (sb->state & (1<<MD_SB_CLEAN))
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
else {
if (sb->events_hi == sb->cp_events_hi &&
sb->events_lo == sb->cp_events_lo) {
- mddev->recovery_cp = sb->recovery_cp;
+ mddev->resync_offset = sb->recovery_cp;
} else
- mddev->recovery_cp = 0;
+ mddev->resync_offset = 0;
}
memcpy(mddev->uuid+0, &sb->set_uuid0, 4);
@@ -1504,10 +1500,10 @@ static void super_90_sync(struct mddev *mddev, struct md_rdev *rdev)
mddev->minor_version = sb->minor_version;
if (mddev->in_sync)
{
- sb->recovery_cp = mddev->recovery_cp;
+ sb->recovery_cp = mddev->resync_offset;
sb->cp_events_hi = (mddev->events>>32);
sb->cp_events_lo = (u32)mddev->events;
- if (mddev->recovery_cp == MaxSector)
+ if (mddev->resync_offset == MaxSector)
sb->state = (1<< MD_SB_CLEAN);
} else
sb->recovery_cp = 0;
@@ -1865,7 +1861,7 @@ static int super_1_validate(struct mddev *mddev, struct md_rdev *freshest, struc
mddev->bitmap_info.default_space = (4096-1024) >> 9;
mddev->reshape_backwards = 0;
- mddev->recovery_cp = le64_to_cpu(sb->resync_offset);
+ mddev->resync_offset = le64_to_cpu(sb->resync_offset);
memcpy(mddev->uuid, sb->set_uuid, 16);
mddev->max_disks = (4096-256)/2;
@@ -2051,7 +2047,7 @@ static void super_1_sync(struct mddev *mddev, struct md_rdev *rdev)
sb->utime = cpu_to_le64((__u64)mddev->utime);
sb->events = cpu_to_le64(mddev->events);
if (mddev->in_sync)
- sb->resync_offset = cpu_to_le64(mddev->recovery_cp);
+ sb->resync_offset = cpu_to_le64(mddev->resync_offset);
else if (test_bit(MD_JOURNAL_CLEAN, &mddev->flags))
sb->resync_offset = cpu_to_le64(MaxSector);
else
@@ -2498,7 +2494,7 @@ void md_autodetect_dev(dev_t dev);
/* just for claiming the bdev */
static struct md_rdev claim_rdev;
-static void export_rdev(struct md_rdev *rdev, struct mddev *mddev)
+static void export_rdev(struct md_rdev *rdev)
{
pr_debug("md: export_rdev(%pg)\n", rdev->bdev);
md_rdev_clear(rdev);
@@ -2744,7 +2740,7 @@ void md_update_sb(struct mddev *mddev, int force_change)
/* If this is just a dirty<->clean transition, and the array is clean
* and 'events' is odd, we can roll back to the previous clean state */
if (nospares
- && (mddev->in_sync && mddev->recovery_cp == MaxSector)
+ && (mddev->in_sync && mddev->resync_offset == MaxSector)
&& mddev->can_decrease_events
&& mddev->events != 1) {
mddev->events--;
@@ -3898,6 +3894,7 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
char clevel[16];
ssize_t rv;
size_t slen = len;
+ unsigned int noio_flags;
struct md_personality *pers, *oldpers;
long level;
void *priv, *oldpriv;
@@ -3909,6 +3906,7 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
rv = mddev_suspend_and_lock(mddev);
if (rv)
return rv;
+ noio_flags = memalloc_noio_save();
if (mddev->pers == NULL) {
memcpy(mddev->clevel, buf, slen);
@@ -4088,6 +4086,7 @@ level_store(struct mddev *mddev, const char *buf, size_t len)
md_new_event();
rv = len;
out_unlock:
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
return rv;
}
@@ -4187,6 +4186,7 @@ static ssize_t
raid_disks_store(struct mddev *mddev, const char *buf, size_t len)
{
unsigned int n;
+ unsigned int noio_flags;
int err;
err = kstrtouint(buf, 10, &n);
@@ -4196,9 +4196,11 @@ raid_disks_store(struct mddev *mddev, const char *buf, size_t len)
err = mddev_suspend_and_lock(mddev);
if (err)
return err;
- if (mddev->pers)
- err = update_raid_disks(mddev, n);
- else if (mddev->reshape_position != MaxSector) {
+ noio_flags = memalloc_noio_save();
+ if (mddev->pers) {
+ if (n != mddev->raid_disks)
+ err = update_raid_disks(mddev, n);
+ } else if (mddev->reshape_position != MaxSector) {
struct md_rdev *rdev;
int olddisks = mddev->raid_disks - mddev->delta_disks;
@@ -4218,6 +4220,7 @@ raid_disks_store(struct mddev *mddev, const char *buf, size_t len)
} else
mddev->raid_disks = n;
out_unlock:
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
return err ? err : len;
}
@@ -4281,9 +4284,9 @@ __ATTR(chunk_size, S_IRUGO|S_IWUSR, chunk_size_show, chunk_size_store);
static ssize_t
resync_start_show(struct mddev *mddev, char *page)
{
- if (mddev->recovery_cp == MaxSector)
+ if (mddev->resync_offset == MaxSector)
return sprintf(page, "none\n");
- return sprintf(page, "%llu\n", (unsigned long long)mddev->recovery_cp);
+ return sprintf(page, "%llu\n", (unsigned long long)mddev->resync_offset);
}
static ssize_t
@@ -4309,7 +4312,7 @@ resync_start_store(struct mddev *mddev, const char *buf, size_t len)
err = -EBUSY;
if (!err) {
- mddev->recovery_cp = n;
+ mddev->resync_offset = n;
if (mddev->pers)
set_bit(MD_SB_CHANGE_CLEAN, &mddev->sb_flags);
}
@@ -4598,6 +4601,7 @@ new_dev_store(struct mddev *mddev, const char *buf, size_t len)
int minor;
dev_t dev;
struct md_rdev *rdev;
+ unsigned int noio_flags;
int err;
if (!*buf || *e != ':' || !e[1] || e[1] == '\n')
@@ -4613,6 +4617,7 @@ new_dev_store(struct mddev *mddev, const char *buf, size_t len)
err = mddev_suspend_and_lock(mddev);
if (err)
return err;
+ noio_flags = memalloc_noio_save();
if (mddev->persistent) {
rdev = md_import_device(dev, mddev->major_version,
mddev->minor_version);
@@ -4631,13 +4636,15 @@ new_dev_store(struct mddev *mddev, const char *buf, size_t len)
rdev = md_import_device(dev, -1, -1);
if (IS_ERR(rdev)) {
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
return PTR_ERR(rdev);
}
err = bind_rdev_to_array(rdev, mddev);
out:
if (err)
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
if (!err)
md_new_event();
@@ -4813,9 +4820,42 @@ metadata_store(struct mddev *mddev, const char *buf, size_t len)
static struct md_sysfs_entry md_metadata =
__ATTR_PREALLOC(metadata_version, S_IRUGO|S_IWUSR, metadata_show, metadata_store);
+static bool rdev_needs_recovery(struct md_rdev *rdev, sector_t sectors)
+{
+ return rdev->raid_disk >= 0 &&
+ !test_bit(Journal, &rdev->flags) &&
+ !test_bit(Faulty, &rdev->flags) &&
+ !test_bit(In_sync, &rdev->flags) &&
+ rdev->recovery_offset < sectors;
+}
+
+static enum sync_action md_get_active_sync_action(struct mddev *mddev)
+{
+ struct md_rdev *rdev;
+ bool is_recover = false;
+
+ if (mddev->resync_offset < MaxSector)
+ return ACTION_RESYNC;
+
+ if (mddev->reshape_position != MaxSector)
+ return ACTION_RESHAPE;
+
+ rcu_read_lock();
+ rdev_for_each_rcu(rdev, mddev) {
+ if (rdev_needs_recovery(rdev, MaxSector)) {
+ is_recover = true;
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ return is_recover ? ACTION_RECOVER : ACTION_IDLE;
+}
+
enum sync_action md_sync_action(struct mddev *mddev)
{
unsigned long recovery = mddev->recovery;
+ enum sync_action active_action;
/*
* frozen has the highest priority, means running sync_thread will be
@@ -4839,8 +4879,17 @@ enum sync_action md_sync_action(struct mddev *mddev)
!test_bit(MD_RECOVERY_NEEDED, &recovery))
return ACTION_IDLE;
- if (test_bit(MD_RECOVERY_RESHAPE, &recovery) ||
- mddev->reshape_position != MaxSector)
+ /*
+ * Check if any sync operation (resync/recover/reshape) is
+ * currently active. This ensures that only one sync operation
+ * can run at a time. Returns the type of active operation, or
+ * ACTION_IDLE if none are active.
+ */
+ active_action = md_get_active_sync_action(mddev);
+ if (active_action != ACTION_IDLE)
+ return active_action;
+
+ if (test_bit(MD_RECOVERY_RESHAPE, &recovery))
return ACTION_RESHAPE;
if (test_bit(MD_RECOVERY_RECOVER, &recovery))
@@ -5577,11 +5626,11 @@ __ATTR(consistency_policy, S_IRUGO | S_IWUSR, consistency_policy_show,
static ssize_t fail_last_dev_show(struct mddev *mddev, char *page)
{
- return sprintf(page, "%d\n", mddev->fail_last_dev);
+ return sprintf(page, "%d\n", test_bit(MD_FAILLAST_DEV, &mddev->flags));
}
/*
- * Setting fail_last_dev to true to allow last device to be forcibly removed
+ * Setting MD_FAILLAST_DEV to allow last device to be forcibly removed
* from RAID1/RAID10.
*/
static ssize_t
@@ -5594,8 +5643,10 @@ fail_last_dev_store(struct mddev *mddev, const char *buf, size_t len)
if (ret)
return ret;
- if (value != mddev->fail_last_dev)
- mddev->fail_last_dev = value;
+ if (value)
+ set_bit(MD_FAILLAST_DEV, &mddev->flags);
+ else
+ clear_bit(MD_FAILLAST_DEV, &mddev->flags);
return len;
}
@@ -5608,11 +5659,12 @@ static ssize_t serialize_policy_show(struct mddev *mddev, char *page)
if (mddev->pers == NULL || (mddev->pers->level != 1))
return sprintf(page, "n/a\n");
else
- return sprintf(page, "%d\n", mddev->serialize_policy);
+ return sprintf(page, "%d\n",
+ test_bit(MD_SERIALIZE_POLICY, &mddev->flags));
}
/*
- * Setting serialize_policy to true to enforce write IO is not reordered
+ * Setting MD_SERIALIZE_POLICY enforce write IO is not reordered
* for raid1.
*/
static ssize_t
@@ -5625,7 +5677,7 @@ serialize_policy_store(struct mddev *mddev, const char *buf, size_t len)
if (err)
return err;
- if (value == mddev->serialize_policy)
+ if (value == test_bit(MD_SERIALIZE_POLICY, &mddev->flags))
return len;
err = mddev_suspend_and_lock(mddev);
@@ -5637,11 +5689,13 @@ serialize_policy_store(struct mddev *mddev, const char *buf, size_t len)
goto unlock;
}
- if (value)
+ if (value) {
mddev_create_serial_pool(mddev, NULL);
- else
+ set_bit(MD_SERIALIZE_POLICY, &mddev->flags);
+ } else {
mddev_destroy_serial_pool(mddev, NULL);
- mddev->serialize_policy = value;
+ clear_bit(MD_SERIALIZE_POLICY, &mddev->flags);
+ }
unlock:
mddev_unlock_and_resume(mddev);
return err ?: len;
@@ -6058,7 +6112,7 @@ int md_run(struct mddev *mddev)
* the only valid external interface is through the md
* device.
*/
- mddev->has_superblocks = false;
+ clear_bit(MD_HAS_SUPERBLOCK, &mddev->flags);
rdev_for_each(rdev, mddev) {
if (test_bit(Faulty, &rdev->flags))
continue;
@@ -6071,7 +6125,7 @@ int md_run(struct mddev *mddev)
}
if (rdev->sb_page)
- mddev->has_superblocks = true;
+ set_bit(MD_HAS_SUPERBLOCK, &mddev->flags);
/* perform some consistency tests on the device.
* We don't want the data to overlap the metadata,
@@ -6375,7 +6429,7 @@ static void md_clean(struct mddev *mddev)
mddev->external_size = 0;
mddev->dev_sectors = 0;
mddev->raid_disks = 0;
- mddev->recovery_cp = 0;
+ mddev->resync_offset = 0;
mddev->resync_min = 0;
mddev->resync_max = MaxSector;
mddev->reshape_position = MaxSector;
@@ -6446,7 +6500,7 @@ static void __md_stop_writes(struct mddev *mddev)
md_update_sb(mddev, 1);
}
/* disable policy to guarantee rdevs free resources for serialization */
- mddev->serialize_policy = 0;
+ clear_bit(MD_SERIALIZE_POLICY, &mddev->flags);
mddev_destroy_serial_pool(mddev, NULL);
}
@@ -6718,7 +6772,7 @@ static void autorun_devices(int part)
rdev_for_each_list(rdev, tmp, &candidates) {
list_del_init(&rdev->same_set);
if (bind_rdev_to_array(rdev, mddev))
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
}
autorun_array(mddev);
mddev_unlock_and_resume(mddev);
@@ -6728,7 +6782,7 @@ static void autorun_devices(int part)
*/
rdev_for_each_list(rdev, tmp, &candidates) {
list_del_init(&rdev->same_set);
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
}
mddev_put(mddev);
}
@@ -6916,13 +6970,13 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
pr_warn("md: %pg has different UUID to %pg\n",
rdev->bdev,
rdev0->bdev);
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return -EINVAL;
}
}
err = bind_rdev_to_array(rdev, mddev);
if (err)
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return err;
}
@@ -6965,7 +7019,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
/* This was a hot-add request, but events doesn't
* match, so reject it.
*/
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return -EINVAL;
}
@@ -6991,7 +7045,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
}
}
if (has_journal || mddev->bitmap) {
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return -EBUSY;
}
set_bit(Journal, &rdev->flags);
@@ -7006,7 +7060,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
/* --add initiated by this node */
err = md_cluster_ops->add_new_disk(mddev, rdev);
if (err) {
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return err;
}
}
@@ -7016,7 +7070,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
err = bind_rdev_to_array(rdev, mddev);
if (err)
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
if (mddev_is_clustered(mddev)) {
if (info->state & (1 << MD_DISK_CANDIDATE)) {
@@ -7079,7 +7133,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info)
err = bind_rdev_to_array(rdev, mddev);
if (err) {
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return err;
}
}
@@ -7192,7 +7246,7 @@ static int hot_add_disk(struct mddev *mddev, dev_t dev)
return 0;
abort_export:
- export_rdev(rdev, mddev);
+ export_rdev(rdev);
return err;
}
@@ -7330,9 +7384,9 @@ int md_set_array_info(struct mddev *mddev, struct mdu_array_info_s *info)
* openned
*/
if (info->state & (1<<MD_SB_CLEAN))
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
else
- mddev->recovery_cp = 0;
+ mddev->resync_offset = 0;
mddev->persistent = ! info->not_persistent;
mddev->external = 0;
@@ -7708,8 +7762,10 @@ static int md_ioctl(struct block_device *bdev, blk_mode_t mode,
unsigned int cmd, unsigned long arg)
{
int err = 0;
+ unsigned int noio_flags = 0;
void __user *argp = (void __user *)arg;
struct mddev *mddev = NULL;
+ bool suspend;
err = md_ioctl_valid(cmd);
if (err)
@@ -7759,13 +7815,15 @@ static int md_ioctl(struct block_device *bdev, blk_mode_t mode,
if (!md_is_rdwr(mddev))
flush_work(&mddev->sync_work);
- err = md_ioctl_need_suspend(cmd) ? mddev_suspend_and_lock(mddev) :
- mddev_lock(mddev);
+ suspend = md_ioctl_need_suspend(cmd);
+ err = suspend ? mddev_suspend_and_lock(mddev) : mddev_lock(mddev);
if (err) {
pr_debug("md: ioctl lock interrupted, reason %d, cmd %d\n",
err, cmd);
goto out;
}
+ if (suspend)
+ noio_flags = memalloc_noio_save();
if (cmd == SET_ARRAY_INFO) {
err = __md_set_array_info(mddev, argp);
@@ -7890,8 +7948,12 @@ static int md_ioctl(struct block_device *bdev, blk_mode_t mode,
err != -EINVAL)
mddev->hold_active = 0;
- md_ioctl_need_suspend(cmd) ? mddev_unlock_and_resume(mddev) :
- mddev_unlock(mddev);
+ if (suspend) {
+ memalloc_noio_restore(noio_flags);
+ mddev_unlock_and_resume(mddev);
+ } else {
+ mddev_unlock(mddev);
+ }
out:
if (cmd == STOP_ARRAY_RO || (err && cmd == STOP_ARRAY))
@@ -8244,7 +8306,7 @@ static int status_resync(struct seq_file *seq, struct mddev *mddev)
seq_printf(seq, "\tresync=REMOTE");
return 1;
}
- if (mddev->recovery_cp < MaxSector) {
+ if (mddev->resync_offset < MaxSector) {
seq_printf(seq, "\tresync=PENDING");
return 1;
}
@@ -8709,7 +8771,7 @@ void md_write_start(struct mddev *mddev, struct bio *bi)
rcu_read_unlock();
if (did_change)
sysfs_notify_dirent_safe(mddev->sysfs_state);
- if (!mddev->has_superblocks)
+ if (!test_bit(MD_HAS_SUPERBLOCK, &mddev->flags))
return;
wait_event(mddev->sb_wait,
!test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags));
@@ -8816,6 +8878,8 @@ static void md_clone_bio(struct mddev *mddev, struct bio **bio)
md_io_clone->mddev = mddev;
if (blk_queue_io_stat(bdev->bd_disk->queue))
md_io_clone->start_time = bio_start_io_acct(*bio);
+ else
+ md_io_clone->start_time = 0;
if (bio_data_dir(*bio) == WRITE && mddev->bitmap) {
md_io_clone->offset = (*bio)->bi_iter.bi_sector;
@@ -8907,6 +8971,39 @@ static sector_t md_sync_max_sectors(struct mddev *mddev,
}
}
+/*
+ * If lazy recovery is requested and all rdevs are in sync, select the rdev with
+ * the higest index to perfore recovery to build initial xor data, this is the
+ * same as old bitmap.
+ */
+static bool mddev_select_lazy_recover_rdev(struct mddev *mddev)
+{
+ struct md_rdev *recover_rdev = NULL;
+ struct md_rdev *rdev;
+ bool ret = false;
+
+ rcu_read_lock();
+ rdev_for_each_rcu(rdev, mddev) {
+ if (rdev->raid_disk < 0)
+ continue;
+
+ if (test_bit(Faulty, &rdev->flags) ||
+ !test_bit(In_sync, &rdev->flags))
+ break;
+
+ if (!recover_rdev || recover_rdev->raid_disk < rdev->raid_disk)
+ recover_rdev = rdev;
+ }
+
+ if (recover_rdev) {
+ clear_bit(In_sync, &recover_rdev->flags);
+ ret = true;
+ }
+
+ rcu_read_unlock();
+ return ret;
+}
+
static sector_t md_sync_position(struct mddev *mddev, enum sync_action action)
{
sector_t start = 0;
@@ -8918,7 +9015,7 @@ static sector_t md_sync_position(struct mddev *mddev, enum sync_action action)
return mddev->resync_min;
case ACTION_RESYNC:
if (!mddev->bitmap)
- return mddev->recovery_cp;
+ return mddev->resync_offset;
return 0;
case ACTION_RESHAPE:
/*
@@ -8934,14 +9031,18 @@ static sector_t md_sync_position(struct mddev *mddev, enum sync_action action)
start = MaxSector;
rcu_read_lock();
rdev_for_each_rcu(rdev, mddev)
- if (rdev->raid_disk >= 0 &&
- !test_bit(Journal, &rdev->flags) &&
- !test_bit(Faulty, &rdev->flags) &&
- !test_bit(In_sync, &rdev->flags) &&
- rdev->recovery_offset < start)
+ if (rdev_needs_recovery(rdev, start))
start = rdev->recovery_offset;
rcu_read_unlock();
+ /*
+ * If there are no spares, and raid456 lazy initial recover is
+ * requested.
+ */
+ if (test_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery) &&
+ start == MaxSector && mddev_select_lazy_recover_rdev(mddev))
+ start = 0;
+
/* If there is a bitmap, we need to make sure all
* writes that started before we added a spare
* complete before we start doing a recovery.
@@ -9144,8 +9245,8 @@ void md_do_sync(struct md_thread *thread)
atomic_read(&mddev->recovery_active) == 0);
mddev->curr_resync_completed = j;
if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery) &&
- j > mddev->recovery_cp)
- mddev->recovery_cp = j;
+ j > mddev->resync_offset)
+ mddev->resync_offset = j;
update_time = jiffies;
set_bit(MD_SB_CHANGE_CLEAN, &mddev->sb_flags);
sysfs_notify_dirent_safe(mddev->sysfs_completed);
@@ -9264,19 +9365,19 @@ void md_do_sync(struct md_thread *thread)
mddev->curr_resync > MD_RESYNC_ACTIVE) {
if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
if (test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
- if (mddev->curr_resync >= mddev->recovery_cp) {
+ if (mddev->curr_resync >= mddev->resync_offset) {
pr_debug("md: checkpointing %s of %s.\n",
desc, mdname(mddev));
if (test_bit(MD_RECOVERY_ERROR,
&mddev->recovery))
- mddev->recovery_cp =
+ mddev->resync_offset =
mddev->curr_resync_completed;
else
- mddev->recovery_cp =
+ mddev->resync_offset =
mddev->curr_resync;
}
} else
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
} else {
if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery))
mddev->curr_resync = MaxSector;
@@ -9284,12 +9385,8 @@ void md_do_sync(struct md_thread *thread)
test_bit(MD_RECOVERY_RECOVER, &mddev->recovery)) {
rcu_read_lock();
rdev_for_each_rcu(rdev, mddev)
- if (rdev->raid_disk >= 0 &&
- mddev->delta_disks >= 0 &&
- !test_bit(Journal, &rdev->flags) &&
- !test_bit(Faulty, &rdev->flags) &&
- !test_bit(In_sync, &rdev->flags) &&
- rdev->recovery_offset < mddev->curr_resync)
+ if (mddev->delta_disks >= 0 &&
+ rdev_needs_recovery(rdev, mddev->curr_resync))
rdev->recovery_offset = mddev->curr_resync;
rcu_read_unlock();
}
@@ -9424,17 +9521,11 @@ static bool md_spares_need_change(struct mddev *mddev)
return false;
}
-static int remove_and_add_spares(struct mddev *mddev,
- struct md_rdev *this)
+static int remove_spares(struct mddev *mddev, struct md_rdev *this)
{
struct md_rdev *rdev;
- int spares = 0;
int removed = 0;
- if (this && test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
- /* Mustn't remove devices when resync thread is running */
- return 0;
-
rdev_for_each(rdev, mddev) {
if ((this == NULL || rdev == this) && rdev_removeable(rdev) &&
!mddev->pers->hot_remove_disk(mddev, rdev)) {
@@ -9448,6 +9539,21 @@ static int remove_and_add_spares(struct mddev *mddev,
if (removed && mddev->kobj.sd)
sysfs_notify_dirent_safe(mddev->sysfs_degraded);
+ return removed;
+}
+
+static int remove_and_add_spares(struct mddev *mddev,
+ struct md_rdev *this)
+{
+ struct md_rdev *rdev;
+ int spares = 0;
+ int removed = 0;
+
+ if (this && test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
+ /* Mustn't remove devices when resync thread is running */
+ return 0;
+
+ removed = remove_spares(mddev, this);
if (this && removed)
goto no_add;
@@ -9485,6 +9591,16 @@ static bool md_choose_sync_action(struct mddev *mddev, int *spares)
set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
+ clear_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery);
+ return true;
+ }
+
+ /* Check if resync is in progress. */
+ if (mddev->resync_offset < MaxSector) {
+ remove_spares(mddev, NULL);
+ set_bit(MD_RECOVERY_SYNC, &mddev->recovery);
+ clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
+ clear_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery);
return true;
}
@@ -9494,7 +9610,7 @@ static bool md_choose_sync_action(struct mddev *mddev, int *spares)
* re-add.
*/
*spares = remove_and_add_spares(mddev, NULL);
- if (*spares) {
+ if (*spares || test_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery)) {
clear_bit(MD_RECOVERY_SYNC, &mddev->recovery);
clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
clear_bit(MD_RECOVERY_REQUESTED, &mddev->recovery);
@@ -9504,13 +9620,6 @@ static bool md_choose_sync_action(struct mddev *mddev, int *spares)
return true;
}
- /* Check if recovery is in progress. */
- if (mddev->recovery_cp < MaxSector) {
- set_bit(MD_RECOVERY_SYNC, &mddev->recovery);
- clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
- return true;
- }
-
/* Delay to choose resync/check/repair in md_do_sync(). */
if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery))
return true;
@@ -9524,19 +9633,34 @@ static void md_start_sync(struct work_struct *ws)
struct mddev *mddev = container_of(ws, struct mddev, sync_work);
int spares = 0;
bool suspend = false;
+ unsigned int noio_flags = 0;
char *name;
/*
* If reshape is still in progress, spares won't be added or removed
* from conf until reshape is done.
*/
- if (mddev->reshape_position == MaxSector &&
+ if ((mddev->reshape_position == MaxSector || !md_is_rdwr(mddev)) &&
md_spares_need_change(mddev)) {
suspend = true;
mddev_suspend(mddev, false);
+ noio_flags = memalloc_noio_save();
}
mddev_lock_nointr(mddev);
+
+ /*
+ * The spare configuration can change before reconfig_mutex is acquired.
+ * Recheck while holding the lock and suspend if needed.
+ */
+ if (!suspend && (mddev->reshape_position == MaxSector || !md_is_rdwr(mddev)) &&
+ md_spares_need_change(mddev)) {
+ mddev_unlock(mddev);
+ mddev_suspend_and_lock_nointr(mddev);
+ suspend = true;
+ noio_flags = memalloc_noio_save();
+ }
+
if (!md_is_rdwr(mddev)) {
/*
* On a read-only array we can:
@@ -9580,8 +9704,10 @@ static void md_start_sync(struct work_struct *ws)
* https://bugzilla.kernel.org/show_bug.cgi?id=218200
* Therefore, use __mddev_resume(mddev, false).
*/
- if (suspend)
+ if (suspend) {
+ memalloc_noio_restore(noio_flags);
__mddev_resume(mddev, false);
+ }
md_wakeup_thread(mddev->sync_thread);
sysfs_notify_dirent_safe(mddev->sysfs_action);
md_new_event();
@@ -9600,8 +9726,10 @@ static void md_start_sync(struct work_struct *ws)
* https://bugzilla.kernel.org/show_bug.cgi?id=218200
* Therefore, use __mddev_resume(mddev, false).
*/
- if (suspend)
+ if (suspend) {
+ memalloc_noio_restore(noio_flags);
__mddev_resume(mddev, false);
+ }
wake_up(&resync_wait);
if (test_and_clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery) &&
@@ -9669,7 +9797,7 @@ void md_check_recovery(struct mddev *mddev)
test_bit(MD_RECOVERY_DONE, &mddev->recovery) ||
(mddev->external == 0 && mddev->safemode == 1) ||
(mddev->safemode == 2
- && !mddev->in_sync && mddev->recovery_cp == MaxSector)
+ && !mddev->in_sync && mddev->resync_offset == MaxSector)
))
return;
@@ -9714,6 +9842,7 @@ void md_check_recovery(struct mddev *mddev)
}
clear_bit(MD_RECOVERY_RECOVER, &mddev->recovery);
+ clear_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery);
clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
clear_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags);
@@ -9824,6 +9953,7 @@ void md_reap_sync_thread(struct mddev *mddev)
clear_bit(MD_RECOVERY_RESHAPE, &mddev->recovery);
clear_bit(MD_RECOVERY_REQUESTED, &mddev->recovery);
clear_bit(MD_RECOVERY_CHECK, &mddev->recovery);
+ clear_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery);
/*
* We call md_cluster_ops->update_size here because sync_size could
* be changed by md_update_sb, and MD_RECOVERY_RESHAPE is cleared,
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 20857b898462..bd5d95b1bac7 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -312,6 +312,9 @@ struct md_cluster_info;
* array is ready yet.
* @MD_BROKEN: This is used to stop writes and mark array as failed.
* @MD_DELETED: This device is being deleted
+ * @MD_HAS_SUPERBLOCK: There is persistence sb in member disks.
+ * @MD_FAILLAST_DEV: Allow last rdev to be removed.
+ * @MD_SERIALIZE_POLICY: Enforce write IO is not reordered, just used by raid1.
*
* change UNSUPPORTED_MDDEV_FLAGS for each array type if new flag is added
*/
@@ -327,6 +330,9 @@ enum mddev_flags {
MD_NOT_READY,
MD_BROKEN,
MD_DELETED,
+ MD_HAS_SUPERBLOCK,
+ MD_FAILLAST_DEV,
+ MD_SERIALIZE_POLICY,
};
enum mddev_sb_flags {
@@ -493,7 +499,7 @@ struct mddev {
atomic_t recovery_active; /* blocks scheduled, but not written */
wait_queue_head_t recovery_wait;
- sector_t recovery_cp;
+ sector_t resync_offset;
sector_t resync_min; /* user requested sync
* starts here */
sector_t resync_max; /* resync should pause
@@ -577,7 +583,6 @@ struct mddev {
void (*sync_super)(struct mddev *mddev, struct md_rdev *rdev);
struct md_cluster_info *cluster_info;
unsigned int good_device_nr; /* good device num within cluster raid */
- unsigned int noio_flag; /* for memalloc scope API */
/*
* Temporarily store rdev that will be finally removed when
@@ -587,10 +592,6 @@ struct mddev {
/* The sequence number for sync thread */
atomic_t sync_seq;
-
- bool has_superblocks:1;
- bool fail_last_dev:1;
- bool serialize_policy:1;
};
enum recovery_flags {
@@ -634,6 +635,8 @@ enum recovery_flags {
MD_RECOVERY_RESHAPE,
/* remote node is running resync thread */
MD_RESYNCING_REMOTE,
+ /* raid456 lazy initial recover */
+ MD_RECOVERY_LAZY_RECOVER,
};
enum md_ro_state {
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index db1ab214250f..efff5c4629c1 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -27,7 +27,9 @@ module_param(default_layout, int, 0644);
(1L << MD_JOURNAL_CLEAN) | \
(1L << MD_FAILFAST_SUPPORTED) |\
(1L << MD_HAS_PPL) | \
- (1L << MD_HAS_MULTIPLE_PPLS))
+ (1L << MD_HAS_MULTIPLE_PPLS) | \
+ (1L << MD_FAILLAST_DEV) | \
+ (1L << MD_SERIALIZE_POLICY))
/*
* inform the user of the raid configuration
@@ -676,7 +678,7 @@ static void *raid0_takeover_raid45(struct mddev *mddev)
mddev->raid_disks--;
mddev->delta_disks = -1;
/* make sure it will be not marked as dirty */
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
@@ -719,7 +721,7 @@ static void *raid0_takeover_raid10(struct mddev *mddev)
mddev->raid_disks += mddev->delta_disks;
mddev->degraded = 0;
/* make sure it will be not marked as dirty */
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
@@ -762,7 +764,7 @@ static void *raid0_takeover_raid1(struct mddev *mddev)
mddev->delta_disks = 1 - mddev->raid_disks;
mddev->raid_disks = 1;
/* make sure it will be not marked as dirty */
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS);
create_strip_zones(mddev, &priv_conf);
diff --git a/drivers/md/raid1-10.c b/drivers/md/raid1-10.c
index f3750ceaa582..41a559262210 100644
--- a/drivers/md/raid1-10.c
+++ b/drivers/md/raid1-10.c
@@ -283,7 +283,7 @@ static inline int raid1_check_read_range(struct md_rdev *rdev,
static inline bool raid1_should_read_first(struct mddev *mddev,
sector_t this_sector, int len)
{
- if ((mddev->recovery_cp < this_sector + len))
+ if ((mddev->resync_offset < this_sector + len))
return true;
if (mddev_is_clustered(mddev) &&
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 4b18151cd813..32b2eb2a169c 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -542,7 +542,7 @@ static void raid1_end_write_request(struct bio *bio)
call_bio_endio(r1_bio);
}
}
- } else if (rdev->mddev->serialize_policy)
+ } else if (test_bit(MD_SERIALIZE_POLICY, &rdev->mddev->flags))
remove_serial(rdev, lo, hi);
if (r1_bio->bios[mirror] == NULL)
rdev_dec_pending(rdev, conf->mddev);
@@ -1637,7 +1637,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio,
mbio = bio_alloc_clone(rdev->bdev, bio, GFP_NOIO,
&mddev->bio_set);
- if (mddev->serialize_policy)
+ if (test_bit(MD_SERIALIZE_POLICY, &mddev->flags))
wait_for_serialization(rdev, r1_bio);
}
@@ -1741,7 +1741,7 @@ static void raid1_status(struct seq_file *seq, struct mddev *mddev)
* - &mddev->degraded is bumped.
*
* @rdev is marked as &Faulty excluding case when array is failed and
- * &mddev->fail_last_dev is off.
+ * MD_FAILLAST_DEV is not set.
*/
static void raid1_error(struct mddev *mddev, struct md_rdev *rdev)
{
@@ -1754,7 +1754,7 @@ static void raid1_error(struct mddev *mddev, struct md_rdev *rdev)
(conf->raid_disks - mddev->degraded) == 1) {
set_bit(MD_BROKEN, &mddev->flags);
- if (!mddev->fail_last_dev) {
+ if (!test_bit(MD_FAILLAST_DEV, &mddev->flags)) {
conf->recovery_disabled = mddev->recovery_disabled;
spin_unlock_irqrestore(&conf->device_lock, flags);
return;
@@ -2815,7 +2815,7 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr,
}
if (mddev->bitmap == NULL &&
- mddev->recovery_cp == MaxSector &&
+ mddev->resync_offset == MaxSector &&
!test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery) &&
conf->fullsync == 0) {
*skipped = 1;
@@ -3277,9 +3277,9 @@ static int raid1_run(struct mddev *mddev)
}
if (conf->raid_disks - mddev->degraded == 1)
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
- if (mddev->recovery_cp != MaxSector)
+ if (mddev->resync_offset != MaxSector)
pr_info("md/raid1:%s: not clean -- starting background reconstruction\n",
mdname(mddev));
pr_info("md/raid1:%s: active with %d out of %d mirrors\n",
@@ -3340,8 +3340,8 @@ static int raid1_resize(struct mddev *mddev, sector_t sectors)
md_set_array_sectors(mddev, newsize);
if (sectors > mddev->dev_sectors &&
- mddev->recovery_cp > mddev->dev_sectors) {
- mddev->recovery_cp = mddev->dev_sectors;
+ mddev->resync_offset > mddev->dev_sectors) {
+ mddev->resync_offset = mddev->dev_sectors;
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
}
mddev->dev_sectors = sectors;
@@ -3482,8 +3482,6 @@ static void *raid1_takeover(struct mddev *mddev)
mddev->new_chunk_sectors = 0;
conf = setup_conf(mddev);
if (!IS_ERR(conf)) {
- /* Array must appear to be quiesced */
- conf->array_frozen = 1;
mddev_clear_unsupported_flags(mddev,
UNSUPPORTED_MDDEV_FLAGS);
}
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 68f9861fa57f..06189fa95490 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1994,7 +1994,7 @@ static int enough(struct r10conf *conf, int ignore)
* - &mddev->degraded is bumped.
*
* @rdev is marked as &Faulty excluding case when array is failed and
- * &mddev->fail_last_dev is off.
+ * MD_FAILLAST_DEV is not set.
*/
static void raid10_error(struct mddev *mddev, struct md_rdev *rdev)
{
@@ -2006,7 +2006,7 @@ static void raid10_error(struct mddev *mddev, struct md_rdev *rdev)
if (test_bit(In_sync, &rdev->flags) && !enough(conf, rdev->raid_disk)) {
set_bit(MD_BROKEN, &mddev->flags);
- if (!mddev->fail_last_dev) {
+ if (!test_bit(MD_FAILLAST_DEV, &mddev->flags)) {
spin_unlock_irqrestore(&conf->device_lock, flags);
return;
}
@@ -2115,7 +2115,7 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
int last = conf->geo.raid_disks - 1;
struct raid10_info *p;
- if (mddev->recovery_cp < MaxSector)
+ if (mddev->resync_offset < MaxSector)
/* only hot-add to in-sync arrays, as recovery is
* very different from resync
*/
@@ -3192,7 +3192,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr,
* of a clean array, like RAID1 does.
*/
if (mddev->bitmap == NULL &&
- mddev->recovery_cp == MaxSector &&
+ mddev->resync_offset == MaxSector &&
mddev->reshape_position == MaxSector &&
!test_bit(MD_RECOVERY_SYNC, &mddev->recovery) &&
!test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery) &&
@@ -4154,7 +4154,7 @@ static int raid10_run(struct mddev *mddev)
disk->recovery_disabled = mddev->recovery_disabled - 1;
}
- if (mddev->recovery_cp != MaxSector)
+ if (mddev->resync_offset != MaxSector)
pr_notice("md/raid10:%s: not clean -- starting background reconstruction\n",
mdname(mddev));
pr_info("md/raid10:%s: active with %d out of %d devices\n",
@@ -4254,8 +4254,8 @@ static int raid10_resize(struct mddev *mddev, sector_t sectors)
md_set_array_sectors(mddev, size);
if (sectors > mddev->dev_sectors &&
- mddev->recovery_cp > oldsize) {
- mddev->recovery_cp = oldsize;
+ mddev->resync_offset > oldsize) {
+ mddev->resync_offset = oldsize;
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
}
calc_sectors(conf, sectors);
@@ -4284,7 +4284,7 @@ static void *raid10_takeover_raid0(struct mddev *mddev, sector_t size, int devs)
mddev->delta_disks = mddev->raid_disks;
mddev->raid_disks *= 2;
/* make sure it will be not marked as dirty */
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
mddev->dev_sectors = size;
conf = setup_conf(mddev);
@@ -5098,8 +5098,8 @@ static void raid10_finish_reshape(struct mddev *mddev)
return;
if (mddev->delta_disks > 0) {
- if (mddev->recovery_cp > mddev->resync_max_sectors) {
- mddev->recovery_cp = mddev->resync_max_sectors;
+ if (mddev->resync_offset > mddev->resync_max_sectors) {
+ mddev->resync_offset = mddev->resync_max_sectors;
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
}
mddev->resync_max_sectors = mddev->array_sectors;
diff --git a/drivers/md/raid5-ppl.c b/drivers/md/raid5-ppl.c
index a70cbec12ed0..0c630cfbf3a0 100644
--- a/drivers/md/raid5-ppl.c
+++ b/drivers/md/raid5-ppl.c
@@ -643,8 +643,10 @@ static void ppl_do_flush(struct ppl_io_unit *io)
log->disk_flush_bitmap = 0;
for (i = flushed_disks ; i < raid_disks; i++) {
- if (atomic_dec_and_test(&io->pending_flushes))
+ if (atomic_dec_and_test(&io->pending_flushes)) {
ppl_io_unit_finished(io);
+ break;
+ }
}
}
@@ -1163,7 +1165,7 @@ static int ppl_load_distributed(struct ppl_log *log)
le64_to_cpu(pplhdr->generation));
/* attempt to recover from log if we are starting a dirty array */
- if (pplhdr && !mddev->pers && mddev->recovery_cp != MaxSector)
+ if (pplhdr && !mddev->pers && mddev->resync_offset != MaxSector)
ret = ppl_recover(log, pplhdr, pplhdr_offset);
/* write empty header if we are starting the array */
@@ -1422,14 +1424,14 @@ int ppl_init_log(struct r5conf *conf)
if (ret) {
goto err;
- } else if (!mddev->pers && mddev->recovery_cp == 0 &&
+ } else if (!mddev->pers && mddev->resync_offset == 0 &&
ppl_conf->recovered_entries > 0 &&
ppl_conf->mismatch_count == 0) {
/*
* If we are starting a dirty array and the recovery succeeds
* without any issues, set the array as clean.
*/
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
set_bit(MD_SB_CHANGE_CLEAN, &mddev->sb_flags);
} else if (mddev->pers && ppl_conf->mismatch_count > 0) {
/* no mismatch allowed when enabling PPL for a running array */
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index b315d36b7f50..bdd620d89c67 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -56,7 +56,11 @@
#include "md-bitmap.h"
#include "raid5-log.h"
-#define UNSUPPORTED_MDDEV_FLAGS (1L << MD_FAILFAST_SUPPORTED)
+#define UNSUPPORTED_MDDEV_FLAGS \
+ ((1L << MD_FAILFAST_SUPPORTED) | \
+ (1L << MD_FAILLAST_DEV) | \
+ (1L << MD_SERIALIZE_POLICY))
+
#define cpu_to_group(cpu) cpu_to_node(cpu)
#define ANY_GROUP NUMA_NO_NODE
@@ -2460,11 +2464,6 @@ static int scribble_alloc(struct raid5_percpu *percpu,
sizeof(unsigned int) * (num + 2);
void *scribble;
- /*
- * If here is in raid array suspend context, it is in memalloc noio
- * context as well, there is no potential recursive memory reclaim
- * I/Os with the GFP_KERNEL flag.
- */
scribble = kvmalloc_array(cnt, obj_size, GFP_KERNEL);
if (!scribble)
return -ENOMEM;
@@ -2479,6 +2478,7 @@ static int scribble_alloc(struct raid5_percpu *percpu,
static int resize_chunks(struct r5conf *conf, int new_disks, int new_sectors)
{
unsigned long cpu;
+ unsigned int noio_flags;
int err = 0;
/* Never shrink. */
@@ -2487,6 +2487,7 @@ static int resize_chunks(struct r5conf *conf, int new_disks, int new_sectors)
return 0;
raid5_quiesce(conf->mddev, true);
+ noio_flags = memalloc_noio_save();
cpus_read_lock();
for_each_present_cpu(cpu) {
@@ -2500,6 +2501,7 @@ static int resize_chunks(struct r5conf *conf, int new_disks, int new_sectors)
}
cpus_read_unlock();
+ memalloc_noio_restore(noio_flags);
raid5_quiesce(conf->mddev, false);
if (!err) {
@@ -3545,7 +3547,7 @@ static void __add_stripe_bio(struct stripe_head *sh, struct bio *bi,
sh->dev[dd_idx].sector);
if (conf->mddev->bitmap && firstwrite && !sh->batch_head) {
- sh->bm_seq = conf->seq_flush+1;
+ sh->bm_seq = READ_ONCE(conf->seq_flush) + 1;
set_bit(STRIPE_BIT_DELAY, &sh->state);
}
}
@@ -3744,11 +3746,10 @@ static int want_replace(struct stripe_head *sh, int disk_idx)
int rv = 0;
rdev = sh->raid_conf->disks[disk_idx].replacement;
- if (rdev
- && !test_bit(Faulty, &rdev->flags)
- && !test_bit(In_sync, &rdev->flags)
- && (rdev->recovery_offset <= sh->sector
- || rdev->mddev->recovery_cp <= sh->sector))
+ if (rdev && !test_bit(Faulty, &rdev->flags) &&
+ !test_bit(In_sync, &rdev->flags) &&
+ (READ_ONCE(rdev->recovery_offset) <= sh->sector ||
+ rdev->mddev->resync_offset <= sh->sector))
rv = 1;
return rv;
}
@@ -3840,7 +3841,7 @@ static int need_this_block(struct stripe_head *sh, struct stripe_head_state *s,
* is missing/faulty, then we need to read everything we can.
*/
if (!force_rcw &&
- sh->sector < sh->raid_conf->mddev->recovery_cp)
+ sh->sector < sh->raid_conf->mddev->resync_offset)
/* reconstruct-write isn't being forced */
return 0;
for (i = 0; i < s->failed && i < 2; i++) {
@@ -4105,7 +4106,7 @@ static int handle_stripe_dirtying(struct r5conf *conf,
int disks)
{
int rmw = 0, rcw = 0, i;
- sector_t recovery_cp = conf->mddev->recovery_cp;
+ sector_t resync_offset = conf->mddev->resync_offset;
/* Check whether resync is now happening or should start.
* If yes, then the array is dirty (after unclean shutdown or
@@ -4115,14 +4116,14 @@ static int handle_stripe_dirtying(struct r5conf *conf,
* generate correct data from the parity.
*/
if (conf->rmw_level == PARITY_DISABLE_RMW ||
- (recovery_cp < MaxSector && sh->sector >= recovery_cp &&
+ (resync_offset < MaxSector && sh->sector >= resync_offset &&
s->failed == 0)) {
/* Calculate the real rcw later - for now make it
* look like rcw is cheaper
*/
rcw = 1; rmw = 2;
- pr_debug("force RCW rmw_level=%u, recovery_cp=%llu sh->sector=%llu\n",
- conf->rmw_level, (unsigned long long)recovery_cp,
+ pr_debug("force RCW rmw_level=%u, resync_offset=%llu sh->sector=%llu\n",
+ conf->rmw_level, (unsigned long long)resync_offset,
(unsigned long long)sh->sector);
} else for (i = disks; i--; ) {
/* would I have to read this buffer for read_modify_write */
@@ -4665,7 +4666,8 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s)
*/
rdev = conf->disks[i].replacement;
if (rdev && !test_bit(Faulty, &rdev->flags) &&
- rdev->recovery_offset >= sh->sector + RAID5_STRIPE_SECTORS(conf) &&
+ READ_ONCE(rdev->recovery_offset) >=
+ sh->sector + RAID5_STRIPE_SECTORS(conf) &&
!rdev_has_badblock(rdev, sh->sector,
RAID5_STRIPE_SECTORS(conf)))
set_bit(R5_ReadRepl, &dev->flags);
@@ -4707,10 +4709,21 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s)
}
} else if (test_bit(In_sync, &rdev->flags))
set_bit(R5_Insync, &dev->flags);
- else if (sh->sector + RAID5_STRIPE_SECTORS(conf) <= rdev->recovery_offset)
- /* in sync if before recovery_offset */
- set_bit(R5_Insync, &dev->flags);
- else if (test_bit(R5_UPTODATE, &dev->flags) &&
+ else if (sh->sector + RAID5_STRIPE_SECTORS(conf) <=
+ READ_ONCE(rdev->recovery_offset)) {
+ /*
+ * in sync if:
+ * - normal IO, or
+ * - resync IO that is not lazy recovery
+ *
+ * For lazy recovery, we have to mark the rdev without
+ * In_sync as failed, to build initial xor data.
+ */
+ if (!test_bit(STRIPE_SYNCING, &sh->state) ||
+ !test_bit(MD_RECOVERY_LAZY_RECOVER,
+ &conf->mddev->recovery))
+ set_bit(R5_Insync, &dev->flags);
+ } else if (test_bit(R5_UPTODATE, &dev->flags) &&
test_bit(R5_Expanded, &dev->flags))
/* If we've reshaped into here, we assume it is Insync.
* We will shortly update recovery_offset to make
@@ -4779,14 +4792,14 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s)
if (test_bit(STRIPE_SYNCING, &sh->state)) {
/* If there is a failed device being replaced,
* we must be recovering.
- * else if we are after recovery_cp, we must be syncing
+ * else if we are after resync_offset, we must be syncing
* else if MD_RECOVERY_REQUESTED is set, we also are syncing.
* else we can only be replacing
* sync and recovery both need to read all devices, and so
* use the same flag.
*/
if (do_recovery ||
- sh->sector >= conf->mddev->recovery_cp ||
+ sh->sector >= conf->mddev->resync_offset ||
test_bit(MD_RECOVERY_REQUESTED, &(conf->mddev->recovery)))
s->syncing = 1;
else
@@ -5430,13 +5443,13 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio)
rdev = conf->disks[dd_idx].replacement;
if (!rdev || test_bit(Faulty, &rdev->flags) ||
- rdev->recovery_offset < end_sector) {
+ READ_ONCE(rdev->recovery_offset) < end_sector) {
rdev = conf->disks[dd_idx].rdev;
if (!rdev)
return 0;
if (test_bit(Faulty, &rdev->flags) ||
!(test_bit(In_sync, &rdev->flags) ||
- rdev->recovery_offset >= end_sector))
+ READ_ONCE(rdev->recovery_offset) >= end_sector))
return 0;
}
@@ -5751,7 +5764,7 @@ static void make_discard_request(struct mddev *mddev, struct bio *bi)
}
spin_unlock_irq(&sh->stripe_lock);
if (conf->mddev->bitmap) {
- sh->bm_seq = conf->seq_flush + 1;
+ sh->bm_seq = READ_ONCE(conf->seq_flush) + 1;
set_bit(STRIPE_BIT_DELAY, &sh->state);
}
@@ -5905,8 +5918,11 @@ static void raid5_bitmap_sector(struct mddev *mddev, sector_t *offset,
sectors_per_chunk = conf->chunk_sectors *
(conf->raid_disks - conf->max_degraded);
- start = round_down(start, sectors_per_chunk);
- end = round_up(end, sectors_per_chunk);
+ sector_div(start, sectors_per_chunk);
+ start *= sectors_per_chunk;
+ if (sector_div(end, sectors_per_chunk))
+ end++;
+ end *= sectors_per_chunk;
start = raid5_compute_sector(conf, start, 0, &dd_idx, NULL);
end = raid5_compute_sector(conf, end, 0, &dd_idx, NULL);
@@ -5924,8 +5940,10 @@ static void raid5_bitmap_sector(struct mddev *mddev, sector_t *offset,
sectors_per_chunk = conf->prev_chunk_sectors *
(conf->previous_raid_disks - conf->max_degraded);
- prev_start = round_down(prev_start, sectors_per_chunk);
- prev_end = round_down(prev_end, sectors_per_chunk);
+ sector_div(prev_start, sectors_per_chunk);
+ prev_start *= sectors_per_chunk;
+ sector_div(prev_end, sectors_per_chunk);
+ prev_end *= sectors_per_chunk;
prev_start = raid5_compute_sector(conf, prev_start, 1, &dd_idx, NULL);
prev_end = raid5_compute_sector(conf, prev_end, 1, &dd_idx, NULL);
@@ -6365,8 +6383,8 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk
if (rdev->raid_disk >= 0 &&
!test_bit(Journal, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
- rdev->recovery_offset < sector_nr)
- rdev->recovery_offset = sector_nr;
+ READ_ONCE(rdev->recovery_offset) < sector_nr)
+ WRITE_ONCE(rdev->recovery_offset, sector_nr);
conf->reshape_checkpoint = jiffies;
set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags);
@@ -6474,8 +6492,8 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, int *sk
if (rdev->raid_disk >= 0 &&
!test_bit(Journal, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags) &&
- rdev->recovery_offset < sector_nr)
- rdev->recovery_offset = sector_nr;
+ READ_ONCE(rdev->recovery_offset) < sector_nr)
+ WRITE_ONCE(rdev->recovery_offset, sector_nr);
conf->reshape_checkpoint = jiffies;
set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags);
md_wakeup_thread(mddev->thread);
@@ -6788,11 +6806,13 @@ static void raid5d(struct md_thread *thread)
if (
!list_empty(&conf->bitmap_list)) {
/* Now is a good time to flush some bitmap updates */
- conf->seq_flush++;
+ int seq = conf->seq_flush + 1;
+
+ WRITE_ONCE(conf->seq_flush, seq);
spin_unlock_irq(&conf->device_lock);
mddev->bitmap_ops->unplug(mddev, true);
spin_lock_irq(&conf->device_lock);
- conf->seq_write = conf->seq_flush;
+ conf->seq_write = seq;
activate_bit_delay(conf, conf->temp_inactive_list);
}
raid5_activate_delayed(conf);
@@ -6977,6 +6997,7 @@ raid5_store_stripe_size(struct mddev *mddev, const char *page, size_t len)
{
struct r5conf *conf;
unsigned long new;
+ unsigned int noio_flags = 0;
int err;
int size;
@@ -7017,6 +7038,7 @@ raid5_store_stripe_size(struct mddev *mddev, const char *page, size_t len)
goto out_unlock;
}
+ noio_flags = memalloc_noio_save();
mutex_lock(&conf->cache_size_mutex);
size = conf->max_nr_stripes;
@@ -7033,6 +7055,7 @@ raid5_store_stripe_size(struct mddev *mddev, const char *page, size_t len)
mutex_unlock(&conf->cache_size_mutex);
out_unlock:
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
return err ?: len;
}
@@ -7809,7 +7832,7 @@ static int raid5_run(struct mddev *mddev)
int first = 1;
int ret = -EIO;
- if (mddev->recovery_cp != MaxSector)
+ if (mddev->resync_offset != MaxSector)
pr_notice("md/raid:%s: not clean -- starting background reconstruction\n",
mdname(mddev));
@@ -7950,7 +7973,7 @@ static int raid5_run(struct mddev *mddev)
mdname(mddev));
mddev->ro = 1;
set_disk_ro(mddev->gendisk, 1);
- } else if (mddev->recovery_cp == MaxSector)
+ } else if (mddev->resync_offset == MaxSector)
set_bit(MD_JOURNAL_CLEAN, &mddev->flags);
}
@@ -7983,9 +8006,9 @@ static int raid5_run(struct mddev *mddev)
/* Hack because v0.91 doesn't store recovery_offset properly. */
if (mddev->major_version == 0 &&
mddev->minor_version > 90)
- rdev->recovery_offset = reshape_offset;
+ WRITE_ONCE(rdev->recovery_offset, reshape_offset);
- if (rdev->recovery_offset < reshape_offset) {
+ if (READ_ONCE(rdev->recovery_offset) < reshape_offset) {
/* We need to check old and new layout */
if (!only_parity(rdev->raid_disk,
conf->algorithm,
@@ -8017,7 +8040,7 @@ static int raid5_run(struct mddev *mddev)
mddev->resync_max_sectors = mddev->dev_sectors;
if (mddev->degraded > dirty_parity_disks &&
- mddev->recovery_cp != MaxSector) {
+ mddev->resync_offset != MaxSector) {
if (test_bit(MD_HAS_PPL, &mddev->flags))
pr_crit("md/raid:%s: starting dirty degraded array with PPL.\n",
mdname(mddev));
@@ -8136,10 +8159,10 @@ static int raid5_spare_active(struct mddev *mddev)
for (i = 0; i < conf->raid_disks; i++) {
rdev = conf->disks[i].rdev;
replacement = conf->disks[i].replacement;
- if (replacement
- && replacement->recovery_offset == MaxSector
- && !test_bit(Faulty, &replacement->flags)
- && !test_and_set_bit(In_sync, &replacement->flags)) {
+ if (replacement &&
+ READ_ONCE(replacement->recovery_offset) == MaxSector &&
+ !test_bit(Faulty, &replacement->flags) &&
+ !test_and_set_bit(In_sync, &replacement->flags)) {
/* Replacement has just become active. */
if (!rdev
|| !test_and_clear_bit(In_sync, &rdev->flags))
@@ -8154,10 +8177,10 @@ static int raid5_spare_active(struct mddev *mddev)
rdev->sysfs_state);
}
sysfs_notify_dirent_safe(replacement->sysfs_state);
- } else if (rdev
- && rdev->recovery_offset == MaxSector
- && !test_bit(Faulty, &rdev->flags)
- && !test_and_set_bit(In_sync, &rdev->flags)) {
+ } else if (rdev &&
+ READ_ONCE(rdev->recovery_offset) == MaxSector &&
+ !test_bit(Faulty, &rdev->flags) &&
+ !test_and_set_bit(In_sync, &rdev->flags)) {
count++;
sysfs_notify_dirent_safe(rdev->sysfs_state);
}
@@ -8358,8 +8381,8 @@ static int raid5_resize(struct mddev *mddev, sector_t sectors)
md_set_array_sectors(mddev, newsize);
if (sectors > mddev->dev_sectors &&
- mddev->recovery_cp > mddev->dev_sectors) {
- mddev->recovery_cp = mddev->dev_sectors;
+ mddev->resync_offset > mddev->dev_sectors) {
+ mddev->resync_offset = mddev->dev_sectors;
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
}
mddev->dev_sectors = sectors;
@@ -8453,7 +8476,7 @@ static int raid5_start_reshape(struct mddev *mddev)
return -EINVAL;
/* raid5 can't handle concurrent reshape and recovery */
- if (mddev->recovery_cp < MaxSector)
+ if (mddev->resync_offset < MaxSector)
return -EBUSY;
for (i = 0; i < conf->raid_disks; i++)
if (conf->disks[i].replacement)
@@ -8527,7 +8550,7 @@ static int raid5_start_reshape(struct mddev *mddev)
>= conf->previous_raid_disks)
set_bit(In_sync, &rdev->flags);
else
- rdev->recovery_offset = 0;
+ WRITE_ONCE(rdev->recovery_offset, 0);
/* Failure here is OK */
sysfs_link_rdev(mddev, rdev);
@@ -8579,7 +8602,7 @@ static void end_reshape(struct r5conf *conf)
if (rdev->raid_disk >= 0 &&
!test_bit(Journal, &rdev->flags) &&
!test_bit(In_sync, &rdev->flags))
- rdev->recovery_offset = MaxSector;
+ WRITE_ONCE(rdev->recovery_offset, MaxSector);
spin_unlock_irq(&conf->device_lock);
wake_up(&conf->wait_for_reshape);
@@ -8678,7 +8701,7 @@ static void *raid45_takeover_raid0(struct mddev *mddev, int level)
mddev->raid_disks += 1;
mddev->delta_disks = 1;
/* make sure it will be not marked as dirty */
- mddev->recovery_cp = MaxSector;
+ mddev->resync_offset = MaxSector;
return setup_conf(mddev);
}
@@ -8899,6 +8922,7 @@ static void *raid6_takeover(struct mddev *mddev)
static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf)
{
struct r5conf *conf;
+ unsigned int noio_flags;
int err;
err = mddev_suspend_and_lock(mddev);
@@ -8910,6 +8934,7 @@ static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf)
return -ENODEV;
}
+ noio_flags = memalloc_noio_save();
if (strncmp(buf, "ppl", 3) == 0) {
/* ppl only works with RAID 5 */
if (!raid5_has_ppl(conf) && conf->level == 5) {
@@ -8949,6 +8974,7 @@ static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf)
if (!err)
md_update_sb(mddev, 1);
+ memalloc_noio_restore(noio_flags);
mddev_unlock_and_resume(mddev);
return err;
diff --git a/drivers/media/cec/core/cec-adap.c b/drivers/media/cec/core/cec-adap.c
index c7d36010c890..c932c29386a7 100644
--- a/drivers/media/cec/core/cec-adap.c
+++ b/drivers/media/cec/core/cec-adap.c
@@ -964,7 +964,7 @@ int cec_transmit_msg_fh(struct cec_adapter *adap, struct cec_msg *msg,
*/
mutex_unlock(&adap->lock);
err = wait_for_completion_killable(&data->c);
- cancel_delayed_work_sync(&data->work);
+ disable_delayed_work_sync(&data->work);
mutex_lock(&adap->lock);
if (err)
@@ -2209,9 +2209,13 @@ static int cec_receive_notify(struct cec_adapter *adap, struct cec_msg *msg,
* Unprocessed messages are aborted if userspace isn't doing
* any processing either.
*/
+ mutex_lock(&adap->lock);
if (!is_broadcast && !is_reply && !adap->follower_cnt &&
- !adap->cec_follower && msg->msg[1] != CEC_MSG_FEATURE_ABORT)
+ !adap->cec_follower && msg->msg[1] != CEC_MSG_FEATURE_ABORT) {
+ mutex_unlock(&adap->lock);
return cec_feature_abort(adap, msg);
+ }
+ mutex_unlock(&adap->lock);
break;
}
@@ -2224,10 +2228,12 @@ static int cec_receive_notify(struct cec_adapter *adap, struct cec_msg *msg,
* Send to the exclusive follower if there is one, otherwise send
* to all followers.
*/
+ mutex_lock(&adap->lock);
if (adap->cec_follower)
cec_queue_msg_fh(adap->cec_follower, msg);
else
cec_queue_msg_followers(adap, msg);
+ mutex_unlock(&adap->lock);
return 0;
}
diff --git a/drivers/media/cec/core/cec-core.c b/drivers/media/cec/core/cec-core.c
index b3c0710eb5b1..17607975d969 100644
--- a/drivers/media/cec/core/cec-core.c
+++ b/drivers/media/cec/core/cec-core.c
@@ -370,9 +370,7 @@ void cec_unregister_adapter(struct cec_adapter *adap)
return;
#ifdef CONFIG_MEDIA_CEC_RC
- /* Note: rc_unregister also calls rc_free */
rc_unregister_device(adap->rc);
- adap->rc = NULL;
#endif
debugfs_remove_recursive(adap->cec_dir);
#ifdef CONFIG_CEC_NOTIFIER
diff --git a/drivers/media/cec/core/cec-pin.c b/drivers/media/cec/core/cec-pin.c
index 330d5d5d86ab..4c91f96351ca 100644
--- a/drivers/media/cec/core/cec-pin.c
+++ b/drivers/media/cec/core/cec-pin.c
@@ -114,7 +114,7 @@ static void cec_pin_update(struct cec_pin *pin, bool v, bool force)
return;
pin->adap->cec_pin_is_high = v;
- if (atomic_read(&pin->work_pin_num_events) < CEC_NUM_PIN_EVENTS) {
+ if (atomic_read_acquire(&pin->work_pin_num_events) < CEC_NUM_PIN_EVENTS) {
u8 ev = v;
if (pin->work_pin_events_dropped) {
@@ -125,7 +125,7 @@ static void cec_pin_update(struct cec_pin *pin, bool v, bool force)
pin->work_pin_ts[pin->work_pin_events_wr] = ktime_get();
pin->work_pin_events_wr =
(pin->work_pin_events_wr + 1) % CEC_NUM_PIN_EVENTS;
- atomic_inc(&pin->work_pin_num_events);
+ atomic_inc_return_release(&pin->work_pin_num_events);
} else {
pin->work_pin_events_dropped = true;
pin->work_pin_events_dropped_cnt++;
@@ -1074,7 +1074,7 @@ static int cec_pin_thread_func(void *_adap)
pin->work_tx_ts);
}
- while (atomic_read(&pin->work_pin_num_events)) {
+ while (atomic_read_acquire(&pin->work_pin_num_events)) {
unsigned int idx = pin->work_pin_events_rd;
u8 v = pin->work_pin_events[idx];
@@ -1083,7 +1083,7 @@ static int cec_pin_thread_func(void *_adap)
v & CEC_PIN_EVENT_FL_DROPPED,
pin->work_pin_ts[idx]);
pin->work_pin_events_rd = (idx + 1) % CEC_NUM_PIN_EVENTS;
- atomic_dec(&pin->work_pin_num_events);
+ atomic_dec_return_release(&pin->work_pin_num_events);
}
switch (atomic_xchg(&pin->work_irq_change,
diff --git a/drivers/media/cec/platform/meson/ao-cec-g12a.c b/drivers/media/cec/platform/meson/ao-cec-g12a.c
index 51294b9b6cd5..42dab3ca8774 100644
--- a/drivers/media/cec/platform/meson/ao-cec-g12a.c
+++ b/drivers/media/cec/platform/meson/ao-cec-g12a.c
@@ -405,6 +405,7 @@ static int meson_ao_cec_g12a_write(void *context, unsigned int addr,
}
static const struct regmap_config meson_ao_cec_g12a_cec_regmap_conf = {
+ .name = "core",
.reg_bits = 8,
.val_bits = 8,
.reg_read = meson_ao_cec_g12a_read,
diff --git a/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c b/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
index cfbfc4c1b2e6..ac1013e11430 100644
--- a/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
+++ b/drivers/media/cec/usb/extron-da-hd-4k-plus/extron-da-hd-4k-plus.c
@@ -656,7 +656,8 @@ static void extron_process_received(struct extron_port *port, const char *data)
if (!port || port->disconnected)
return;
- if (len < 5 || (len - 2) % 3 || data[len - 2] != '*')
+ if (len < 5 || ((len - 2) / 3 > sizeof(msg.msg)) ||
+ (len - 2) % 3 || data[len - 2] != '*')
goto malformed;
while (*data != '*') {
diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
index 91c16917d7ae..a4d1a6442c49 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.c
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
@@ -906,9 +906,12 @@ static int rtl2832_sdr_start_streaming(struct vb2_queue *vq, unsigned int count)
goto err;
mutex_unlock(&dev->v4l2_lock);
+
return 0;
err:
+ rtl2832_sdr_free_urbs(dev);
+ rtl2832_sdr_free_stream_bufs(dev);
rtl2832_sdr_cleanup_queued_bufs(dev, VB2_BUF_STATE_QUEUED);
mutex_unlock(&dev->v4l2_lock);
@@ -1479,14 +1482,22 @@ static void rtl2832_sdr_remove(struct platform_device *pdev)
dev_dbg(&pdev->dev, "\n");
- mutex_lock(&dev->vb_queue_lock);
+ /*
+ * vb2_video_unregister_device() releases the vb2 queue, which
+ * triggers rtl2832_sdr_stop_streaming() if streaming is active.
+ * stop_streaming() uses dev->udev to free URBs and coherent DMA
+ * stream buffers via usb_free_coherent(), so it must run before
+ * dev->udev is cleared. vb2_video_unregister_device() locks
+ * vb_queue_lock internally and stop_streaming() locks v4l2_lock,
+ * so neither may be held by the caller.
+ */
+ v4l2_device_disconnect(&dev->v4l2_dev);
+ vb2_video_unregister_device(&dev->vdev);
+
mutex_lock(&dev->v4l2_lock);
- /* No need to keep the urbs around after disconnection */
dev->udev = NULL;
- v4l2_device_disconnect(&dev->v4l2_dev);
- video_unregister_device(&dev->vdev);
mutex_unlock(&dev->v4l2_lock);
- mutex_unlock(&dev->vb_queue_lock);
+
v4l2_device_put(&dev->v4l2_dev);
module_put(pdev->dev.parent->driver->owner);
}
diff --git a/drivers/media/i2c/alvium-csi2.c b/drivers/media/i2c/alvium-csi2.c
index 2081714cba0f..5f365674b91c 100644
--- a/drivers/media/i2c/alvium-csi2.c
+++ b/drivers/media/i2c/alvium-csi2.c
@@ -1291,7 +1291,7 @@ static int alvium_set_ctrl_auto_exposure(struct alvium_dev *alvium, bool on)
struct device *dev = &alvium->i2c_client->dev;
int ret;
- ret = alvium_write_hshake(alvium, REG_BCRM_WHITE_BALANCE_AUTO_RW,
+ ret = alvium_write_hshake(alvium, REG_BCRM_EXPOSURE_AUTO_RW,
on ? 0x02 : 0x00);
if (ret) {
dev_err(dev, "Fail to set autoexposure reg\n");
diff --git a/drivers/media/i2c/imx415.c b/drivers/media/i2c/imx415.c
index a20b0db330d3..59e15eacfcc0 100644
--- a/drivers/media/i2c/imx415.c
+++ b/drivers/media/i2c/imx415.c
@@ -721,7 +721,7 @@ static int imx415_set_testpattern(struct imx415 *sensor, int val)
cci_write(sensor->regmap, IMX415_DIG_CLP_MODE, 0x01, &ret);
cci_write(sensor->regmap, IMX415_WRJ_OPEN, 0x01, &ret);
}
- return 0;
+ return ret;
}
static int imx415_s_ctrl(struct v4l2_ctrl *ctrl)
diff --git a/drivers/media/i2c/ov02a10.c b/drivers/media/i2c/ov02a10.c
index 6c30e1a0d814..c93d060a5482 100644
--- a/drivers/media/i2c/ov02a10.c
+++ b/drivers/media/i2c/ov02a10.c
@@ -822,18 +822,16 @@ static int ov02a10_check_hwcfg(struct device *dev, struct ov02a10 *ov02a10)
if (!ep)
return -ENXIO;
+ /* Optional indication of MIPI clock voltage unit */
+ if (!fwnode_property_read_u32(ep, "ovti,mipi-clock-voltage",
+ &clk_volt))
+ ov02a10->mipi_clock_voltage = clk_volt;
+
ret = v4l2_fwnode_endpoint_alloc_parse(ep, &bus_cfg);
fwnode_handle_put(ep);
if (ret)
return ret;
- /* Optional indication of MIPI clock voltage unit */
- ret = fwnode_property_read_u32(ep, "ovti,mipi-clock-voltage",
- &clk_volt);
-
- if (!ret)
- ov02a10->mipi_clock_voltage = clk_volt;
-
for (i = 0; i < ARRAY_SIZE(link_freq_menu_items); i++) {
for (j = 0; j < bus_cfg.nr_of_link_frequencies; j++) {
if (link_freq_menu_items[i] ==
diff --git a/drivers/media/i2c/ov7740.c b/drivers/media/i2c/ov7740.c
index 0830676e5d5a..7b87d855dffb 100644
--- a/drivers/media/i2c/ov7740.c
+++ b/drivers/media/i2c/ov7740.c
@@ -1119,10 +1119,8 @@ static void ov7740_remove(struct i2c_client *client)
struct v4l2_subdev *sd = i2c_get_clientdata(client);
struct ov7740 *ov7740 = container_of(sd, struct ov7740, subdev);
- mutex_destroy(&ov7740->mutex);
- v4l2_ctrl_handler_free(ov7740->subdev.ctrl_handler);
- media_entity_cleanup(&ov7740->subdev.entity);
v4l2_async_unregister_subdev(sd);
+ media_entity_cleanup(&ov7740->subdev.entity);
ov7740_free_controls(ov7740);
pm_runtime_get_sync(&client->dev);
diff --git a/drivers/media/i2c/rdacm21.c b/drivers/media/i2c/rdacm21.c
index 3e22df36354f..e8196ea73739 100644
--- a/drivers/media/i2c/rdacm21.c
+++ b/drivers/media/i2c/rdacm21.c
@@ -589,10 +589,12 @@ static int rdacm21_probe(struct i2c_client *client)
ret = v4l2_async_register_subdev(&dev->sd);
if (ret)
- goto error_free_ctrls;
+ goto error_entity_cleanup;
return 0;
+error_entity_cleanup:
+ media_entity_cleanup(&dev->sd.entity);
error_free_ctrls:
v4l2_ctrl_handler_free(&dev->ctrls);
error:
@@ -607,6 +609,7 @@ static void rdacm21_remove(struct i2c_client *client)
v4l2_async_unregister_subdev(&dev->sd);
v4l2_ctrl_handler_free(&dev->ctrls);
+ media_entity_cleanup(&dev->sd.entity);
i2c_unregister_device(dev->isp);
}
diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c
index 56dbe07a1c99..e6b932e32ed7 100644
--- a/drivers/media/i2c/video-i2c.c
+++ b/drivers/media/i2c/video-i2c.c
@@ -533,8 +533,12 @@ static int start_streaming(struct vb2_queue *vq, unsigned int count)
data->kthread_vid_cap = kthread_run(video_i2c_thread_vid_cap, data,
"%s-vid-cap", data->v4l2_dev.name);
ret = PTR_ERR_OR_ZERO(data->kthread_vid_cap);
- if (!ret)
- return 0;
+ if (ret) {
+ data->kthread_vid_cap = NULL;
+ goto error_rpm_put;
+ }
+
+ return 0;
error_rpm_put:
pm_runtime_mark_last_busy(dev);
diff --git a/drivers/media/pci/cobalt/cobalt-alsa-main.c b/drivers/media/pci/cobalt/cobalt-alsa-main.c
index c57f87a68269..1356353d931c 100644
--- a/drivers/media/pci/cobalt/cobalt-alsa-main.c
+++ b/drivers/media/pci/cobalt/cobalt-alsa-main.c
@@ -135,7 +135,6 @@ int cobalt_alsa_init(struct cobalt_stream *s)
err_exit_free:
if (sc != NULL)
snd_card_free(sc);
- kfree(cobsc);
err_exit:
return ret;
}
diff --git a/drivers/media/pci/cx23885/cimax2.c b/drivers/media/pci/cx23885/cimax2.c
index 06e41f92092d..a4a319087238 100644
--- a/drivers/media/pci/cx23885/cimax2.c
+++ b/drivers/media/pci/cx23885/cimax2.c
@@ -528,6 +528,7 @@ void netup_ci_exit(struct cx23885_tsport *port)
if (NULL == state->ca.data)
return;
+ cancel_work_sync(&state->work);
dvb_ca_en50221_release(&state->ca);
kfree(state);
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c
index 17bc8cabcbdb..bf02d94da49d 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c
@@ -761,6 +761,7 @@ static int isys_notifier_init(struct ipu6_isys *isys)
err_parse:
fwnode_handle_put(ep);
+ v4l2_async_nf_cleanup(&isys->notifier);
return ret;
}
diff --git a/drivers/media/pci/intel/ipu6/ipu6.c b/drivers/media/pci/intel/ipu6/ipu6.c
index 40566b652b2d..8b45ff0c22b6 100644
--- a/drivers/media/pci/intel/ipu6/ipu6.c
+++ b/drivers/media/pci/intel/ipu6/ipu6.c
@@ -399,7 +399,6 @@ ipu6_isys_init(struct pci_dev *pdev, struct device *parent,
&ipdata->hw_variant);
if (IS_ERR(isys_adev->mmu)) {
put_device(&isys_adev->auxdev.dev);
- kfree(pdata);
return dev_err_cast_probe(dev, isys_adev->mmu,
"ipu6_mmu_init(isys_adev->mmu) failed\n");
}
@@ -407,10 +406,8 @@ ipu6_isys_init(struct pci_dev *pdev, struct device *parent,
isys_adev->mmu->dev = &isys_adev->auxdev.dev;
ret = ipu6_bus_add_device(isys_adev);
- if (ret) {
- kfree(pdata);
+ if (ret)
return ERR_PTR(ret);
- }
return isys_adev;
}
@@ -443,7 +440,6 @@ ipu6_psys_init(struct pci_dev *pdev, struct device *parent,
&ipdata->hw_variant);
if (IS_ERR(psys_adev->mmu)) {
put_device(&psys_adev->auxdev.dev);
- kfree(pdata);
return dev_err_cast_probe(&pdev->dev, psys_adev->mmu,
"ipu6_mmu_init(psys_adev->mmu) failed\n");
}
@@ -451,10 +447,8 @@ ipu6_psys_init(struct pci_dev *pdev, struct device *parent,
psys_adev->mmu->dev = &psys_adev->auxdev.dev;
ret = ipu6_bus_add_device(psys_adev);
- if (ret) {
- kfree(pdata);
+ if (ret)
return ERR_PTR(ret);
- }
return psys_adev;
}
diff --git a/drivers/media/pci/saa7164/saa7164-core.c b/drivers/media/pci/saa7164/saa7164-core.c
index ac290f546413..49dca57181d8 100644
--- a/drivers/media/pci/saa7164/saa7164-core.c
+++ b/drivers/media/pci/saa7164/saa7164-core.c
@@ -878,6 +878,9 @@ static int get_resources(struct saa7164_dev *dev)
if (request_mem_region(pci_resource_start(dev->pci, 2),
pci_resource_len(dev->pci, 2), dev->name))
return 0;
+
+ release_mem_region(pci_resource_start(dev->pci, 0),
+ pci_resource_len(dev->pci, 0));
}
printk(KERN_ERR "%s: can't get MMIO memory @ 0x%llx or 0x%llx\n",
@@ -1000,8 +1003,7 @@ static int saa7164_dev_setup(struct saa7164_dev *dev)
dev->name, dev->pci->subsystem_vendor,
dev->pci->subsystem_device);
- saa7164_devcount--;
- return -ENODEV;
+ goto err_devlist;
}
/* PCI/e allocations */
@@ -1039,7 +1041,7 @@ static int saa7164_dev_setup(struct saa7164_dev *dev)
iounmap(dev->lmmio);
err_ioremap_bar0:
release_resources(dev);
-
+err_devlist:
scoped_guard(mutex, &devlist) {
list_del(&dev->devlist);
}
diff --git a/drivers/media/pci/zoran/zoran_card.c b/drivers/media/pci/zoran/zoran_card.c
index 38a083ffe6c3..16af0e74b4dc 100644
--- a/drivers/media/pci/zoran/zoran_card.c
+++ b/drivers/media/pci/zoran/zoran_card.c
@@ -889,7 +889,7 @@ static int zoran_init_video_device(struct zoran *zr, struct video_device *video_
static void zoran_exit_video_devices(struct zoran *zr)
{
video_unregister_device(zr->video_dev);
- kfree(zr->video_dev);
+ zr->video_dev = NULL;
}
static int zoran_init_video_devices(struct zoran *zr)
diff --git a/drivers/media/platform/amphion/venc.c b/drivers/media/platform/amphion/venc.c
index 351b4edc8742..d810730d86ad 100644
--- a/drivers/media/platform/amphion/venc.c
+++ b/drivers/media/platform/amphion/venc.c
@@ -967,8 +967,6 @@ static int venc_start_session(struct vpu_inst *inst, u32 type)
venc->ready_count = 0;
venc->stopped = false;
vpu_process_output_buffer(inst);
- if (venc->frame_count == 0)
- dev_err(inst->dev, "[%d] there is no input when starting\n", inst->id);
return 0;
error:
diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
index 9f81e1582a30..d94703ac1022 100644
--- a/drivers/media/platform/broadcom/bcm2835-unicam.c
+++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
@@ -2570,6 +2570,7 @@ static int unicam_async_nf_init(struct unicam_device *unicam)
return 0;
error:
+ v4l2_async_nf_cleanup(&unicam->notifier);
fwnode_handle_put(ep_handle);
return ret;
}
@@ -2702,6 +2703,7 @@ static void unicam_remove(struct platform_device *pdev)
v4l2_device_unregister(&unicam->v4l2_dev);
media_device_unregister(&unicam->mdev);
v4l2_async_nf_unregister(&unicam->notifier);
+ v4l2_async_nf_cleanup(&unicam->notifier);
unicam_subdev_cleanup(unicam);
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
index 2685ef393eaa..72b3cab56730 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
@@ -1430,6 +1430,7 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
} else if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
struct dec_initial_info *initial_info =
&inst->codec_info->dec_info.initial_info;
+ struct dec_info *p_dec_info = &inst->codec_info->dec_info;
if (inst->state == VPU_INST_STATE_STOP)
ret = switch_state(inst, VPU_INST_STATE_INIT_SEQ);
@@ -1437,6 +1438,7 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
goto return_buffers;
if (inst->state == VPU_INST_STATE_INIT_SEQ &&
+ p_dec_info->initial_info_obtained &&
inst->dev->product_code == WAVE521C_CODE) {
if (initial_info->luma_bitdepth != 8) {
dev_info(inst->dev->dev, "%s: no support for %d bit depth",
@@ -1445,7 +1447,6 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
goto return_buffers;
}
}
-
}
return ret;
diff --git a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-core.c b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-core.c
index afa47649dd4c..29d65326e4a6 100644
--- a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-core.c
+++ b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-core.c
@@ -311,7 +311,11 @@ static int mdp_probe(struct platform_device *pdev)
goto err_destroy_clock_wq;
}
mdp->scp = platform_get_drvdata(mm_pdev);
- put_device(&mm_pdev->dev);
+ }
+
+ if (!mdp->scp) {
+ ret = -EPROBE_DEFER;
+ goto err_destroy_clock_wq;
}
mdp->rproc_handle = scp_get_rproc(mdp->scp);
diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
index 880275de0038..ddbe9342679e 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
@@ -107,7 +107,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
return ret;
/* The memory input can be routed to the first pipeline only. */
- for_each_active_route(&state->routing, route) {
+ for_each_active_route(routing, route) {
if (route->sink_pad == xbar->num_sinks - 1 &&
route->source_pad != xbar->num_sinks) {
dev_dbg(xbar->isi->dev,
@@ -145,10 +145,10 @@ mxc_isi_crossbar_xlate_streams(struct mxc_isi_crossbar *xbar,
*/
for_each_active_route(&state->routing, route) {
if (route->source_pad != source_pad ||
- !(source_streams & BIT(route->source_stream)))
+ !(source_streams & BIT_ULL(route->source_stream)))
continue;
- sink_streams |= BIT(route->sink_stream);
+ sink_streams |= BIT_ULL(route->sink_stream);
sink_pad = route->sink_pad;
}
diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c
index 3f6c5f472b2c..5cc7622b8f2a 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c
@@ -151,7 +151,7 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
.fourcc = V4L2_PIX_FMT_XBGR32,
.type = MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
| MXC_ISI_VIDEO_M2M_CAP,
- .isi_in_format = CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+ .isi_in_format = CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
.isi_out_format = CHNL_IMG_CTRL_FORMAT_XRGB888,
.mem_planes = 1,
.color_planes = 1,
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/venus/hfi_parser.c
index 1b3db2caa99f..522bac7ba154 100644
--- a/drivers/media/platform/qcom/venus/hfi_parser.c
+++ b/drivers/media/platform/qcom/venus/hfi_parser.c
@@ -85,7 +85,7 @@ parse_alloc_mode(struct venus_core *core, u32 codecs, u32 domain, void *data)
type++;
}
- return sizeof(*mode);
+ return mode->num_entries * sizeof(u32) + sizeof(*mode);
}
static void fill_profile_level(struct hfi_plat_caps *cap, const void *data,
@@ -146,7 +146,7 @@ parse_caps(struct venus_core *core, u32 codecs, u32 domain, void *data)
for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
fill_caps, caps_arr, num_caps);
- return sizeof(*caps);
+ return num_caps * sizeof(*cap) + sizeof(u32);
}
static void fill_raw_fmts(struct hfi_plat_caps *cap, const void *fmts,
@@ -171,7 +171,7 @@ parse_raw_formats(struct venus_core *core, u32 codecs, u32 domain, void *data)
u32 entries = fmt->format_entries;
unsigned int i = 0;
u32 num_planes = 0;
- u32 size;
+ u32 size = 2 * sizeof(u32);
while (entries) {
num_planes = pinfo->num_planes;
@@ -186,6 +186,7 @@ parse_raw_formats(struct venus_core *core, u32 codecs, u32 domain, void *data)
if (pinfo->num_planes > MAX_PLANES)
break;
+ size += sizeof(*constr) * num_planes + 2 * sizeof(u32);
pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
2 * sizeof(u32);
entries--;
@@ -193,8 +194,6 @@ parse_raw_formats(struct venus_core *core, u32 codecs, u32 domain, void *data)
for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
fill_raw_fmts, rawfmts, i);
- size = fmt->format_entries * (sizeof(*constr) * num_planes + 2 * sizeof(u32))
- + 2 * sizeof(u32);
return size;
}
diff --git a/drivers/media/rc/sunxi-cir.c b/drivers/media/rc/sunxi-cir.c
index 448d453cfda9..77dbff77b539 100644
--- a/drivers/media/rc/sunxi-cir.c
+++ b/drivers/media/rc/sunxi-cir.c
@@ -344,22 +344,25 @@ static int sunxi_ir_probe(struct platform_device *pdev)
ir->irq = platform_get_irq(pdev, 0);
if (ir->irq < 0) {
ret = ir->irq;
- goto exit_free_dev;
+ goto exit_unregister_dev;
}
ret = devm_request_irq(dev, ir->irq, sunxi_ir_irq, 0, SUNXI_IR_DEV, ir);
if (ret) {
dev_err(dev, "failed request irq\n");
- goto exit_free_dev;
+ goto exit_unregister_dev;
}
ret = sunxi_ir_hw_init(dev);
if (ret)
- goto exit_free_dev;
+ goto exit_unregister_dev;
dev_info(dev, "initialized sunXi IR driver\n");
return 0;
+exit_unregister_dev:
+ rc_unregister_device(ir->rc);
+
exit_free_dev:
rc_free_device(ir->rc);
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index 89506ae00901..d0ba0d5613a4 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -177,8 +177,8 @@ static int vimc_capture_enum_framesizes(struct file *file, void *fh,
if (fsize->index)
return -EINVAL;
- /* Only accept code in the pix map table */
- vpix = vimc_pix_map_by_code(fsize->pixel_format);
+ /* Only accept pixel_format in the pix map table */
+ vpix = vimc_pix_map_by_pixelformat(fsize->pixel_format);
if (!vpix)
return -EINVAL;
diff --git a/drivers/media/tuners/tda18250.c b/drivers/media/tuners/tda18250.c
index 68d0275f29e1..d32dffe4a573 100644
--- a/drivers/media/tuners/tda18250.c
+++ b/drivers/media/tuners/tda18250.c
@@ -440,8 +440,8 @@ static int tda18250_pll_calc(struct dvb_frontend *fe, u8 *rdiv,
goto err;
exp = (uval & 0x70) >> 4;
- if (exp > 5)
- exp = 0;
+ if (exp == 0 || exp > 5)
+ exp = 1;
lopd = 1 << (exp - 1);
scale = uval & 0x0f;
fvco = lopd * scale * ((c->frequency / 1000) + dev->if_frequency);
diff --git a/drivers/media/usb/airspy/airspy.c b/drivers/media/usb/airspy/airspy.c
index 8cdaf8dd4f88..e9607cc68991 100644
--- a/drivers/media/usb/airspy/airspy.c
+++ b/drivers/media/usb/airspy/airspy.c
@@ -464,14 +464,21 @@ static void airspy_disconnect(struct usb_interface *intf)
dev_dbg(s->dev, "\n");
- mutex_lock(&s->vb_queue_lock);
+ /*
+ * vb2_video_unregister_device() releases the vb2 queue, which
+ * triggers airspy_stop_streaming() if streaming is active.
+ * stop_streaming() dereferences s->udev via airspy_ctrl_msg() and
+ * airspy_free_stream_bufs(), so it must run before s->udev is
+ * cleared. vb2_video_unregister_device() locks vb_queue_lock
+ * internally and stop_streaming() locks v4l2_lock, so neither may
+ * be held by the caller.
+ */
+ v4l2_device_disconnect(&s->v4l2_dev);
+ vb2_video_unregister_device(&s->vdev);
+
mutex_lock(&s->v4l2_lock);
- /* No need to keep the urbs around after disconnection */
s->udev = NULL;
- v4l2_device_disconnect(&s->v4l2_dev);
- video_unregister_device(&s->vdev);
mutex_unlock(&s->v4l2_lock);
- mutex_unlock(&s->vb_queue_lock);
v4l2_device_put(&s->v4l2_dev);
}
diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
index 435eb0b32cb1..6529829670e5 100644
--- a/drivers/media/usb/cx231xx/cx231xx-video.c
+++ b/drivers/media/usb/cx231xx/cx231xx-video.c
@@ -900,7 +900,7 @@ static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
if (rc)
return rc;
- if (vb2_is_busy(&dev->vidq)) {
+ if (vb2_is_busy(&dev->vidq) || vb2_is_busy(&dev->vbiq)) {
dev_err(dev->dev, "%s: queue busy\n", __func__);
return -EBUSY;
}
@@ -935,7 +935,7 @@ static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id norm)
if (dev->norm == norm)
return 0;
- if (vb2_is_busy(&dev->vidq))
+ if (vb2_is_busy(&dev->vidq) || vb2_is_busy(&dev->vbiq))
return -EBUSY;
dev->norm = norm;
diff --git a/drivers/media/usb/em28xx/em28xx-cards.c b/drivers/media/usb/em28xx/em28xx-cards.c
index a51cbcf429e1..b8e207443770 100644
--- a/drivers/media/usb/em28xx/em28xx-cards.c
+++ b/drivers/media/usb/em28xx/em28xx-cards.c
@@ -3448,6 +3448,7 @@ static void request_module_async(struct work_struct *work)
* intf. Don't register extensions twice on those devices.
*/
if (dev->is_audio_only) {
+ em28xx_init_extension(dev);
#if defined(CONFIG_MODULES) && defined(MODULE)
request_module("em28xx-alsa");
#endif
@@ -3676,8 +3677,6 @@ static int em28xx_init_dev(struct em28xx *dev, struct usb_device *udev,
retval = -ENODEV;
goto err_deinit_media;
}
- em28xx_init_extension(dev);
-
return 0;
}
diff --git a/drivers/media/usb/em28xx/em28xx-core.c b/drivers/media/usb/em28xx/em28xx-core.c
index 29a7f3f19b56..1012bfb18646 100644
--- a/drivers/media/usb/em28xx/em28xx-core.c
+++ b/drivers/media/usb/em28xx/em28xx-core.c
@@ -1134,6 +1134,8 @@ void em28xx_close_extension(struct em28xx *dev)
ops->fini(dev);
}
}
+ if (dev->dev_next)
+ list_del(&dev->dev_next->devlist);
list_del(&dev->devlist);
mutex_unlock(&em28xx_devlist_mutex);
}
diff --git a/drivers/media/usb/go7007/snd-go7007.c b/drivers/media/usb/go7007/snd-go7007.c
index 9a6bd87fce03..64bf668b7ca0 100644
--- a/drivers/media/usb/go7007/snd-go7007.c
+++ b/drivers/media/usb/go7007/snd-go7007.c
@@ -195,6 +195,13 @@ static const struct snd_device_ops go7007_snd_device_ops = {
.dev_free = go7007_snd_free,
};
+static void go7007_snd_card_free(struct snd_card *card)
+{
+ struct go7007 *go = card->private_data;
+
+ v4l2_device_put(&go->v4l2_dev);
+}
+
int go7007_snd_init(struct go7007 *go)
{
static int dev;
@@ -245,6 +252,8 @@ int go7007_snd_init(struct go7007 *go)
gosnd->substream = NULL;
go->snd_context = gosnd;
v4l2_device_get(&go->v4l2_dev);
+ gosnd->card->private_data = go;
+ gosnd->card->private_free = go7007_snd_card_free;
++dev;
return 0;
@@ -263,7 +272,6 @@ int go7007_snd_remove(struct go7007 *go)
snd_card_disconnect(gosnd->card);
snd_card_free_when_closed(gosnd->card);
- v4l2_device_put(&go->v4l2_dev);
return 0;
}
EXPORT_SYMBOL(go7007_snd_remove);
diff --git a/drivers/media/usb/s2255/s2255drv.c b/drivers/media/usb/s2255/s2255drv.c
index a6e450181fd0..2fa9f4a95223 100644
--- a/drivers/media/usb/s2255/s2255drv.c
+++ b/drivers/media/usb/s2255/s2255drv.c
@@ -617,6 +617,12 @@ static void s2255_fillbuff(struct s2255_vc *vc,
break;
case V4L2_PIX_FMT_JPEG:
case V4L2_PIX_FMT_MJPEG:
+ if (jpgsize < 0 ||
+ jpgsize > vb2_plane_size(&buf->vb.vb2_buf, 0)) {
+ dprintk(dev, 1, "bad JPEG frame size %d\n",
+ jpgsize);
+ break;
+ }
vb2_set_plane_payload(&buf->vb.vb2_buf, 0, jpgsize);
memcpy(vbuf, tmpbuf, jpgsize);
break;
@@ -2283,6 +2289,11 @@ static int s2255_probe(struct usb_interface *interface,
}
/* check the firmware is valid */
fw_size = dev->fw_data->fw->size;
+ if (fw_size < 8) {
+ dev_err(&interface->dev, "Firmware invalid: too small.\n");
+ retval = -ENODEV;
+ goto errorFWMARKER;
+ }
pdata = (__le32 *) &dev->fw_data->fw->data[fw_size - 8];
if (*pdata != S2255_FW_MARKER) {
diff --git a/drivers/media/usb/usbtv/usbtv-audio.c b/drivers/media/usb/usbtv/usbtv-audio.c
index 333bd305a4f9..ae0a14e5ed2a 100644
--- a/drivers/media/usb/usbtv/usbtv-audio.c
+++ b/drivers/media/usb/usbtv/usbtv-audio.c
@@ -317,6 +317,13 @@ static const struct snd_pcm_ops snd_usbtv_pcm_ops = {
.pointer = snd_usbtv_pointer,
};
+static void usbtv_audio_card_free(struct snd_card *card)
+{
+ struct usbtv *usbtv = card->private_data;
+
+ v4l2_device_put(&usbtv->v4l2_dev);
+}
+
int usbtv_audio_init(struct usbtv *usbtv)
{
int rv;
@@ -331,6 +338,10 @@ int usbtv_audio_init(struct usbtv *usbtv)
if (rv < 0)
return rv;
+ v4l2_device_get(&usbtv->v4l2_dev);
+ card->private_data = usbtv;
+ card->private_free = usbtv_audio_card_free;
+
strscpy(card->driver, usbtv->dev->driver->name, sizeof(card->driver));
strscpy(card->shortname, "usbtv", sizeof(card->shortname));
snprintf(card->longname, sizeof(card->longname),
diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
index 1c08bba9ecb9..5f6685e34d90 100644
--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -392,7 +392,6 @@ static int v4l2_async_match_notify(struct v4l2_async_notifier *notifier,
err_call_unbind:
v4l2_async_nf_call_unbind(notifier, sd, asc);
- list_del(&asc->asc_subdev_entry);
err_unregister_subdev:
if (registered)
@@ -898,9 +897,18 @@ void v4l2_async_unregister_subdev(struct v4l2_subdev *sd)
sd->subdev_notifier = NULL;
if (sd->asc_list.next) {
- list_for_each_entry_safe(asc, asc_tmp, &sd->asc_list,
- asc_subdev_entry) {
- v4l2_async_unbind_subdev_one(asc->notifier, asc);
+ if (list_empty(&sd->asc_list)) {
+ /*
+ * If the sub-device was registered through other means
+ * than v4l2-async, there are no async connections but
+ * the sub-device may still well be registered.
+ * Unregister it now.
+ */
+ v4l2_device_unregister_subdev(sd);
+ } else {
+ list_for_each_entry_safe(asc, asc_tmp, &sd->asc_list,
+ asc_subdev_entry)
+ v4l2_async_unbind_subdev_one(asc->notifier, asc);
}
}
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 1a47cf18b56f..2adf14cbe27c 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -1133,24 +1133,41 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
return -EINVAL;
}
- if (p_hdr10_mastering->white_point_x <
- V4L2_HDR10_MASTERING_WHITE_POINT_X_LOW ||
- p_hdr10_mastering->white_point_x >
- V4L2_HDR10_MASTERING_WHITE_POINT_X_HIGH ||
- p_hdr10_mastering->white_point_y <
- V4L2_HDR10_MASTERING_WHITE_POINT_Y_LOW ||
- p_hdr10_mastering->white_point_y >
- V4L2_HDR10_MASTERING_WHITE_POINT_Y_HIGH)
+ /*
+ * SMPTE ST 2086 Annex A documents that CTA 861-G uses
+ * (0, 0) to indicate that the white point chromaticity
+ * is unknown.
+ */
+ if (p_hdr10_mastering->white_point_x ||
+ p_hdr10_mastering->white_point_y) {
+ if (p_hdr10_mastering->white_point_x <
+ V4L2_HDR10_MASTERING_WHITE_POINT_X_LOW ||
+ p_hdr10_mastering->white_point_x >
+ V4L2_HDR10_MASTERING_WHITE_POINT_X_HIGH ||
+ p_hdr10_mastering->white_point_y <
+ V4L2_HDR10_MASTERING_WHITE_POINT_Y_LOW ||
+ p_hdr10_mastering->white_point_y >
+ V4L2_HDR10_MASTERING_WHITE_POINT_Y_HIGH)
+ return -EINVAL;
+ }
+
+ /*
+ * SMPTE ST 2086 Annex A documents that CTA 861-G uses zero
+ * maximum and minimum luminance values to indicate that
+ * the corresponding values are unknown.
+ */
+ if (p_hdr10_mastering->max_display_mastering_luminance &&
+ (p_hdr10_mastering->max_display_mastering_luminance <
+ V4L2_HDR10_MASTERING_MAX_LUMA_LOW ||
+ p_hdr10_mastering->max_display_mastering_luminance >
+ V4L2_HDR10_MASTERING_MAX_LUMA_HIGH))
return -EINVAL;
- if (p_hdr10_mastering->max_display_mastering_luminance <
- V4L2_HDR10_MASTERING_MAX_LUMA_LOW ||
- p_hdr10_mastering->max_display_mastering_luminance >
- V4L2_HDR10_MASTERING_MAX_LUMA_HIGH ||
- p_hdr10_mastering->min_display_mastering_luminance <
- V4L2_HDR10_MASTERING_MIN_LUMA_LOW ||
- p_hdr10_mastering->min_display_mastering_luminance >
- V4L2_HDR10_MASTERING_MIN_LUMA_HIGH)
+ if (p_hdr10_mastering->min_display_mastering_luminance &&
+ (p_hdr10_mastering->min_display_mastering_luminance <
+ V4L2_HDR10_MASTERING_MIN_LUMA_LOW ||
+ p_hdr10_mastering->min_display_mastering_luminance >
+ V4L2_HDR10_MASTERING_MIN_LUMA_HIGH))
return -EINVAL;
/* The following restriction comes from ITU-T Rec. H.265 spec */
diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c b/drivers/media/v4l2-core/v4l2-fwnode.c
index 354162eb9325..50afd78649f7 100644
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -582,6 +582,8 @@ int v4l2_fwnode_parse_link(struct fwnode_handle *fwnode,
if (!link->remote_node)
goto err_put_remote_endpoint;
+ fwnode_handle_put(fwnode);
+
return 0;
err_put_remote_endpoint:
diff --git a/drivers/mfd/iqs62x.c b/drivers/mfd/iqs62x.c
index ee017617d1d1..412ae7777f72 100644
--- a/drivers/mfd/iqs62x.c
+++ b/drivers/mfd/iqs62x.c
@@ -237,7 +237,7 @@ static int iqs62x_firmware_parse(struct iqs62x_core *iqs62x,
fw_rec = (struct iqs62x_fw_rec *)(fw->data + pos);
pos += sizeof(*fw_rec);
- if (pos + fw_rec->len - 1 > fw->size) {
+ if (!fw_rec->len || fw_rec->len - 1 > fw->size - pos) {
ret = -EINVAL;
break;
}
diff --git a/drivers/mfd/rave-sp.c b/drivers/mfd/rave-sp.c
index c1b78d127a26..05d26d92df03 100644
--- a/drivers/mfd/rave-sp.c
+++ b/drivers/mfd/rave-sp.c
@@ -63,6 +63,12 @@
#define RAVE_SP_TX_BUFFER_SIZE \
(RAVE_SP_STX_ETX_SIZE + 2 * RAVE_SP_RX_BUFFER_SIZE)
+enum rave_sp_frame_offset {
+ RAVE_SP_FRAME_CODE_OFFSET,
+ RAVE_SP_FRAME_ACK_ID_OFFSET,
+ RAVE_SP_FRAME_DATA_OFFSET,
+};
+
/**
* enum rave_sp_deframer_state - Possible state for de-framer
*
@@ -352,7 +358,7 @@ int rave_sp_exec(struct rave_sp *sp,
int command, ret = 0;
u8 ackid;
- command = sp->variant->cmd.translate(data[0]);
+ command = sp->variant->cmd.translate(data[RAVE_SP_FRAME_CODE_OFFSET]);
if (command < 0)
return command;
@@ -366,8 +372,8 @@ int rave_sp_exec(struct rave_sp *sp,
sp->reply = &reply;
mutex_unlock(&sp->reply_lock);
- data[0] = command;
- data[1] = ackid;
+ data[RAVE_SP_FRAME_CODE_OFFSET] = command;
+ data[RAVE_SP_FRAME_ACK_ID_OFFSET] = ackid;
rave_sp_write(sp, data, data_size);
@@ -388,16 +394,23 @@ EXPORT_SYMBOL_GPL(rave_sp_exec);
static void rave_sp_receive_event(struct rave_sp *sp,
const unsigned char *data, size_t length)
{
- u8 cmd[] = {
- [0] = rave_sp_reply_code(data[0]),
- [1] = data[1],
- };
+ unsigned long action;
+ u8 cmd[RAVE_SP_FRAME_DATA_OFFSET];
+
+ if (length < RAVE_SP_FRAME_DATA_OFFSET + 1) {
+ dev_warn(&sp->serdev->dev, "Dropping short event frame\n");
+ return;
+ }
+
+ cmd[RAVE_SP_FRAME_CODE_OFFSET] =
+ rave_sp_reply_code(data[RAVE_SP_FRAME_CODE_OFFSET]);
+ cmd[RAVE_SP_FRAME_ACK_ID_OFFSET] = data[RAVE_SP_FRAME_ACK_ID_OFFSET];
rave_sp_write(sp, cmd, sizeof(cmd));
- blocking_notifier_call_chain(&sp->event_notifier_list,
- rave_sp_action_pack(data[0], data[2]),
- NULL);
+ action = rave_sp_action_pack(data[RAVE_SP_FRAME_CODE_OFFSET],
+ data[RAVE_SP_FRAME_DATA_OFFSET]);
+ blocking_notifier_call_chain(&sp->event_notifier_list, action, NULL);
}
static void rave_sp_receive_reply(struct rave_sp *sp,
@@ -405,27 +418,35 @@ static void rave_sp_receive_reply(struct rave_sp *sp,
{
struct device *dev = &sp->serdev->dev;
struct rave_sp_reply *reply;
- const size_t payload_length = length - 2;
+ size_t payload_length;
+
+ if (length < RAVE_SP_FRAME_DATA_OFFSET) {
+ dev_warn(dev, "Dropping short reply frame\n");
+ return;
+ }
+ payload_length = length - RAVE_SP_FRAME_DATA_OFFSET;
mutex_lock(&sp->reply_lock);
reply = sp->reply;
if (reply) {
- if (reply->code == data[0] && reply->ackid == data[1] &&
+ if (reply->code == data[RAVE_SP_FRAME_CODE_OFFSET] &&
+ reply->ackid == data[RAVE_SP_FRAME_ACK_ID_OFFSET] &&
payload_length >= reply->length) {
/*
* We are relying on memcpy(dst, src, 0) to be a no-op
* when handling commands that have a no-payload reply
*/
- memcpy(reply->data, &data[2], reply->length);
+ memcpy(reply->data, &data[RAVE_SP_FRAME_DATA_OFFSET],
+ reply->length);
complete(&reply->received);
sp->reply = NULL;
} else {
dev_err(dev, "Ignoring incorrect reply\n");
dev_dbg(dev, "Code: expected = 0x%08x received = 0x%08x\n",
- reply->code, data[0]);
+ reply->code, data[RAVE_SP_FRAME_CODE_OFFSET]);
dev_dbg(dev, "ACK ID: expected = 0x%08x received = 0x%08x\n",
- reply->ackid, data[1]);
+ reply->ackid, data[RAVE_SP_FRAME_ACK_ID_OFFSET]);
dev_dbg(dev, "Length: expected = %zu received = %zu\n",
reply->length, payload_length);
}
@@ -439,10 +460,10 @@ static void rave_sp_receive_frame(struct rave_sp *sp,
size_t length)
{
const size_t checksum_length = sp->variant->checksum->length;
- const size_t payload_length = length - checksum_length;
- const u8 *crc_reported = &data[payload_length];
struct device *dev = &sp->serdev->dev;
u8 crc_calculated[RAVE_SP_CHECKSUM_SIZE];
+ const u8 *crc_reported;
+ size_t payload_length;
if (unlikely(checksum_length > sizeof(crc_calculated))) {
dev_warn(dev, "Checksum too long, dropping\n");
@@ -457,6 +478,9 @@ static void rave_sp_receive_frame(struct rave_sp *sp,
return;
}
+ payload_length = length - checksum_length;
+ crc_reported = &data[payload_length];
+
sp->variant->checksum->subroutine(data, payload_length,
crc_calculated);
@@ -465,10 +489,10 @@ static void rave_sp_receive_frame(struct rave_sp *sp,
return;
}
- if (rave_sp_id_is_event(data[0]))
- rave_sp_receive_event(sp, data, length);
+ if (rave_sp_id_is_event(data[RAVE_SP_FRAME_CODE_OFFSET]))
+ rave_sp_receive_event(sp, data, payload_length);
else
- rave_sp_receive_reply(sp, data, length);
+ rave_sp_receive_reply(sp, data, payload_length);
}
static size_t rave_sp_receive_buf(struct serdev_device *serdev,
diff --git a/drivers/misc/ad525x_dpot-i2c.c b/drivers/misc/ad525x_dpot-i2c.c
index 469478f7a1d3..896ad61bb9e1 100644
--- a/drivers/misc/ad525x_dpot-i2c.c
+++ b/drivers/misc/ad525x_dpot-i2c.c
@@ -105,6 +105,7 @@ MODULE_DEVICE_TABLE(i2c, ad_dpot_id);
static struct i2c_driver ad_dpot_i2c_driver = {
.driver = {
.name = "ad_dpot",
+ .dev_groups = ad_dpot_groups,
},
.probe = ad_dpot_i2c_probe,
.remove = ad_dpot_i2c_remove,
diff --git a/drivers/misc/ad525x_dpot-spi.c b/drivers/misc/ad525x_dpot-spi.c
index 263055bda48b..1ebe629715a8 100644
--- a/drivers/misc/ad525x_dpot-spi.c
+++ b/drivers/misc/ad525x_dpot-spi.c
@@ -131,6 +131,7 @@ MODULE_DEVICE_TABLE(spi, ad_dpot_spi_id);
static struct spi_driver ad_dpot_spi_driver = {
.driver = {
.name = "ad_dpot",
+ .dev_groups = ad_dpot_groups,
},
.probe = ad_dpot_spi_probe,
.remove = ad_dpot_spi_remove,
diff --git a/drivers/misc/ad525x_dpot.c b/drivers/misc/ad525x_dpot.c
index 756ef6912b5a..3ceb1f1caa38 100644
--- a/drivers/misc/ad525x_dpot.c
+++ b/drivers/misc/ad525x_dpot.c
@@ -631,66 +631,132 @@ static struct attribute *ad525x_attributes_commands[] = {
NULL
};
-static const struct attribute_group ad525x_group_commands = {
- .attrs = ad525x_attributes_commands,
+static struct attribute *ad525x_attributes[] = {
+ &dev_attr_rdac0.attr,
+ &dev_attr_rdac1.attr,
+ &dev_attr_rdac2.attr,
+ &dev_attr_rdac3.attr,
+ &dev_attr_rdac4.attr,
+ &dev_attr_rdac5.attr,
+ &dev_attr_eeprom0.attr,
+ &dev_attr_eeprom1.attr,
+ &dev_attr_eeprom2.attr,
+ &dev_attr_eeprom3.attr,
+ &dev_attr_eeprom4.attr,
+ &dev_attr_eeprom5.attr,
+ &dev_attr_tolerance0.attr,
+ &dev_attr_tolerance1.attr,
+ &dev_attr_tolerance2.attr,
+ &dev_attr_tolerance3.attr,
+ &dev_attr_tolerance4.attr,
+ &dev_attr_tolerance5.attr,
+ &dev_attr_otp0.attr,
+ &dev_attr_otp1.attr,
+ &dev_attr_otp2.attr,
+ &dev_attr_otp3.attr,
+ &dev_attr_otp4.attr,
+ &dev_attr_otp5.attr,
+ &dev_attr_otp0en.attr,
+ &dev_attr_otp1en.attr,
+ &dev_attr_otp2en.attr,
+ &dev_attr_otp3en.attr,
+ &dev_attr_otp4en.attr,
+ &dev_attr_otp5en.attr,
+ &dev_attr_inc_all.attr,
+ &dev_attr_dec_all.attr,
+ &dev_attr_inc_all_6db.attr,
+ &dev_attr_dec_all_6db.attr,
+ NULL
};
-static int ad_dpot_add_files(struct device *dev,
- unsigned int features, unsigned int rdac)
+static int ad525x_attr_index(struct attribute *attr,
+ const struct attribute * const *attrs)
{
- int err = sysfs_create_file(&dev->kobj,
- dpot_attrib_wipers[rdac]);
- if (features & F_CMD_EEP)
- err |= sysfs_create_file(&dev->kobj,
- dpot_attrib_eeprom[rdac]);
- if (features & F_CMD_TOL)
- err |= sysfs_create_file(&dev->kobj,
- dpot_attrib_tolerance[rdac]);
- if (features & F_CMD_OTP) {
- err |= sysfs_create_file(&dev->kobj,
- dpot_attrib_otp_en[rdac]);
- err |= sysfs_create_file(&dev->kobj,
- dpot_attrib_otp[rdac]);
- }
+ int i;
- if (err)
- dev_err(dev, "failed to register sysfs hooks for RDAC%d\n",
- rdac);
+ for (i = 0; attrs[i]; i++)
+ if (attr == attrs[i])
+ return i;
- return err;
+ return -ENOENT;
}
-static inline void ad_dpot_remove_files(struct device *dev,
- unsigned int features, unsigned int rdac)
+static bool ad525x_is_command_attr(struct attribute *attr)
{
- sysfs_remove_file(&dev->kobj,
- dpot_attrib_wipers[rdac]);
- if (features & F_CMD_EEP)
- sysfs_remove_file(&dev->kobj,
- dpot_attrib_eeprom[rdac]);
- if (features & F_CMD_TOL)
- sysfs_remove_file(&dev->kobj,
- dpot_attrib_tolerance[rdac]);
- if (features & F_CMD_OTP) {
- sysfs_remove_file(&dev->kobj,
- dpot_attrib_otp_en[rdac]);
- sysfs_remove_file(&dev->kobj,
- dpot_attrib_otp[rdac]);
+ int i;
+
+ for (i = 0; ad525x_attributes_commands[i]; i++) {
+ if (attr == ad525x_attributes_commands[i])
+ return true;
}
+
+ return false;
+}
+
+static umode_t ad525x_is_visible(struct kobject *kobj, struct attribute *attr,
+ int n)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct dpot_data *data = dev_get_drvdata(dev);
+ int rdac;
+
+ if (!data)
+ return 0;
+
+ rdac = ad525x_attr_index(attr, dpot_attrib_wipers);
+ if (rdac >= 0)
+ return data->wipers & BIT(rdac) ? attr->mode : 0;
+
+ rdac = ad525x_attr_index(attr, dpot_attrib_eeprom);
+ if (rdac >= 0)
+ return (data->wipers & BIT(rdac)) && (data->feat & F_CMD_EEP) ?
+ attr->mode : 0;
+
+ rdac = ad525x_attr_index(attr, dpot_attrib_tolerance);
+ if (rdac >= 0)
+ return (data->wipers & BIT(rdac)) && (data->feat & F_CMD_TOL) ?
+ attr->mode : 0;
+
+ rdac = ad525x_attr_index(attr, dpot_attrib_otp);
+ if (rdac >= 0)
+ return (data->wipers & BIT(rdac)) && (data->feat & F_CMD_OTP) ?
+ attr->mode : 0;
+
+ rdac = ad525x_attr_index(attr, dpot_attrib_otp_en);
+ if (rdac >= 0)
+ return (data->wipers & BIT(rdac)) && (data->feat & F_CMD_OTP) ?
+ attr->mode : 0;
+
+ if (ad525x_is_command_attr(attr))
+ return data->feat & F_CMD_INC ? attr->mode : 0;
+
+ return attr->mode;
}
+static const struct attribute_group ad525x_group = {
+ .attrs = ad525x_attributes,
+ .is_visible = ad525x_is_visible,
+};
+
+const struct attribute_group *ad_dpot_groups[] = {
+ &ad525x_group,
+ NULL
+};
+EXPORT_SYMBOL(ad_dpot_groups);
+
int ad_dpot_probe(struct device *dev,
struct ad_dpot_bus_data *bdata, unsigned long devid,
const char *name)
{
struct dpot_data *data;
- int i, err = 0;
+ int i;
data = kzalloc(sizeof(struct dpot_data), GFP_KERNEL);
if (!data) {
- err = -ENOMEM;
- goto exit;
+ dev_err(dev, "failed to create client for %s ID 0x%lX\n",
+ name, devid);
+ return -ENOMEM;
}
dev_set_drvdata(dev, data);
@@ -706,51 +772,22 @@ int ad_dpot_probe(struct device *dev,
data->wipers = DPOT_WIPERS(devid);
for (i = DPOT_RDAC0; i < MAX_RDACS; i++)
- if (data->wipers & (1 << i)) {
- err = ad_dpot_add_files(dev, data->feat, i);
- if (err)
- goto exit_remove_files;
+ if (data->wipers & BIT(i)) {
/* power-up midscale */
if (data->feat & F_RDACS_WONLY)
data->rdac_cache[i] = data->max_pos / 2;
}
- if (data->feat & F_CMD_INC)
- err = sysfs_create_group(&dev->kobj, &ad525x_group_commands);
-
- if (err) {
- dev_err(dev, "failed to register sysfs hooks\n");
- goto exit_free;
- }
-
dev_info(dev, "%s %d-Position Digital Potentiometer registered\n",
name, data->max_pos);
return 0;
-
-exit_remove_files:
- for (i = DPOT_RDAC0; i < MAX_RDACS; i++)
- if (data->wipers & (1 << i))
- ad_dpot_remove_files(dev, data->feat, i);
-
-exit_free:
- kfree(data);
- dev_set_drvdata(dev, NULL);
-exit:
- dev_err(dev, "failed to create client for %s ID 0x%lX\n",
- name, devid);
- return err;
}
EXPORT_SYMBOL(ad_dpot_probe);
void ad_dpot_remove(struct device *dev)
{
struct dpot_data *data = dev_get_drvdata(dev);
- int i;
-
- for (i = DPOT_RDAC0; i < MAX_RDACS; i++)
- if (data->wipers & (1 << i))
- ad_dpot_remove_files(dev, data->feat, i);
kfree(data);
}
diff --git a/drivers/misc/ad525x_dpot.h b/drivers/misc/ad525x_dpot.h
index 72a9d6801937..2e877c89523b 100644
--- a/drivers/misc/ad525x_dpot.h
+++ b/drivers/misc/ad525x_dpot.h
@@ -10,6 +10,8 @@
#include <linux/types.h>
+struct attribute_group;
+
#define DPOT_CONF(features, wipers, max_pos, uid) \
(((features) << 18) | (((wipers) & 0xFF) << 10) | \
((max_pos & 0xF) << 6) | (uid & 0x3F))
@@ -210,5 +212,6 @@ struct ad_dpot_bus_data {
int ad_dpot_probe(struct device *dev, struct ad_dpot_bus_data *bdata,
unsigned long devid, const char *name);
void ad_dpot_remove(struct device *dev);
+extern const struct attribute_group *ad_dpot_groups[];
#endif
diff --git a/drivers/misc/bcm-vk/bcm_vk_msg.c b/drivers/misc/bcm-vk/bcm_vk_msg.c
index 665a3888708a..a36a6c7d384a 100644
--- a/drivers/misc/bcm-vk/bcm_vk_msg.c
+++ b/drivers/misc/bcm-vk/bcm_vk_msg.c
@@ -108,7 +108,8 @@ u32 msgq_avail_space(const struct bcm_vk_msgq __iomem *msgq,
bool bcm_vk_drv_access_ok(struct bcm_vk *vk)
{
- return (!!atomic_read(&vk->msgq_inited));
+ /* Pair with the release store after message queue initialization. */
+ return !!atomic_read_acquire(&vk->msgq_inited);
}
void bcm_vk_set_host_alert(struct bcm_vk *vk, u32 bit_mask)
@@ -501,7 +502,8 @@ int bcm_vk_sync_msgq(struct bcm_vk *vk, bool force_sync)
msgq++;
}
}
- atomic_set(&vk->msgq_inited, 1);
+ /* Publish message queue info before allowing driver access. */
+ atomic_set_release(&vk->msgq_inited, 1);
return ret;
}
diff --git a/drivers/misc/cardreader/rtsx_pcr.c b/drivers/misc/cardreader/rtsx_pcr.c
index be3d4e0e50cc..6a1cfa948bdb 100644
--- a/drivers/misc/cardreader/rtsx_pcr.c
+++ b/drivers/misc/cardreader/rtsx_pcr.c
@@ -1242,6 +1242,8 @@ static int rtsx_pci_init_hw(struct rtsx_pcr *pcr)
/* Gating real mcu clock */
err = rtsx_pci_write_register(pcr, RTS5261_FW_CFG1,
RTS5261_MCU_CLOCK_GATING, 0);
+ if (err < 0)
+ return err;
err = rtsx_pci_write_register(pcr, RTS5261_REG_FPDCTL,
SSC_POWER_DOWN, 0);
} else {
diff --git a/drivers/misc/cardreader/rtsx_usb.c b/drivers/misc/cardreader/rtsx_usb.c
index f546b050cb49..0a585be7f086 100644
--- a/drivers/misc/cardreader/rtsx_usb.c
+++ b/drivers/misc/cardreader/rtsx_usb.c
@@ -312,6 +312,9 @@ int rtsx_usb_get_card_status(struct rtsx_ucr *ucr, u16 *status)
if (ret < 0)
return ret;
+ ucr->card_status_cache = *status;
+ ucr->card_status_valid = true;
+
return 0;
}
EXPORT_SYMBOL_GPL(rtsx_usb_get_card_status);
@@ -619,6 +622,7 @@ static int rtsx_usb_probe(struct usb_interface *intf,
{
struct usb_device *usb_dev = interface_to_usbdev(intf);
struct rtsx_ucr *ucr;
+ u16 status;
int ret;
dev_dbg(&intf->dev,
@@ -655,6 +659,9 @@ static int rtsx_usb_probe(struct usb_interface *intf,
if (ret)
goto out_init_fail;
+ /* Prime cached status for runtime autosuspend decisions. */
+ rtsx_usb_get_card_status(ucr, &status);
+
/* initialize USB SG transfer timer */
timer_setup(&ucr->sg_timer, rtsx_usb_sg_timed_out, 0);
@@ -709,17 +716,27 @@ static int rtsx_usb_suspend(struct usb_interface *intf, pm_message_t message)
struct rtsx_ucr *ucr =
(struct rtsx_ucr *)usb_get_intfdata(intf);
u16 val = 0;
+ bool valid = false;
dev_dbg(&intf->dev, "%s called with pm message 0x%04x\n",
__func__, message.event);
if (PMSG_IS_AUTO(message)) {
if (mutex_trylock(&ucr->dev_mutex)) {
- rtsx_usb_get_card_status(ucr, &val);
+ valid = ucr->card_status_valid;
+ if (valid)
+ val = ucr->card_status_cache;
mutex_unlock(&ucr->dev_mutex);
- /* Defer the autosuspend if card exists */
- if (val & (SD_CD | MS_CD)) {
+ /*
+ * Do not issue USB commands from runtime autosuspend.
+ * Raw SD_CD is not authoritative on tray-based readers,
+ * while a real SD card is protected by the SD/MMC child
+ * runtime-PM reference once the card is powered. Keep
+ * the historical Memory Stick autosuspend deferral when
+ * the cached status says MS media is present.
+ */
+ if (valid && (val & MS_CD)) {
device_for_each_child(&intf->dev, NULL, rtsx_usb_resume_child);
return -EAGAIN;
}
diff --git a/drivers/misc/sgi-gru/grufault.c b/drivers/misc/sgi-gru/grufault.c
index 3557d78ee47a..5a87c12f444a 100644
--- a/drivers/misc/sgi-gru/grufault.c
+++ b/drivers/misc/sgi-gru/grufault.c
@@ -166,13 +166,8 @@ static void get_clear_fault_map(struct gru_state *gru,
}
/*
- * Atomic (interrupt context) & non-atomic (user context) functions to
- * convert a vaddr into a physical address. The size of the page
- * is returned in pageshift.
- * returns:
- * 0 - successful
- * < 0 - error code
- * 1 - (atomic only) try again in non-atomic context
+ * Convert a user virtual address to a physical address in process context.
+ * The size of the page is returned in pageshift.
*/
static int non_atomic_pte_lookup(struct vm_area_struct *vma,
unsigned long vaddr, int write,
@@ -192,87 +187,25 @@ static int non_atomic_pte_lookup(struct vm_area_struct *vma,
return 0;
}
-/*
- * atomic_pte_lookup
- *
- * Convert a user virtual address to a physical address
- * Only supports Intel large pages (2MB only) on x86_64.
- * ZZZ - hugepage support is incomplete
- *
- * NOTE: mmap_lock is already held on entry to this function. This
- * guarantees existence of the page tables.
- */
-static int atomic_pte_lookup(struct vm_area_struct *vma, unsigned long vaddr,
- int write, unsigned long *paddr, int *pageshift)
-{
- pgd_t *pgdp;
- p4d_t *p4dp;
- pud_t *pudp;
- pmd_t *pmdp;
- pte_t pte;
-
- pgdp = pgd_offset(vma->vm_mm, vaddr);
- if (unlikely(pgd_none(*pgdp)))
- goto err;
-
- p4dp = p4d_offset(pgdp, vaddr);
- if (unlikely(p4d_none(*p4dp)))
- goto err;
-
- pudp = pud_offset(p4dp, vaddr);
- if (unlikely(pud_none(*pudp)))
- goto err;
-
- pmdp = pmd_offset(pudp, vaddr);
- if (unlikely(pmd_none(*pmdp)))
- goto err;
-#ifdef CONFIG_X86_64
- if (unlikely(pmd_leaf(*pmdp)))
- pte = ptep_get((pte_t *)pmdp);
- else
-#endif
- pte = *pte_offset_kernel(pmdp, vaddr);
-
- if (unlikely(!pte_present(pte) ||
- (write && (!pte_write(pte) || !pte_dirty(pte)))))
- return 1;
-
- *paddr = pte_pfn(pte) << PAGE_SHIFT;
-#ifdef CONFIG_HUGETLB_PAGE
- *pageshift = is_vm_hugetlb_page(vma) ? HPAGE_SHIFT : PAGE_SHIFT;
-#else
- *pageshift = PAGE_SHIFT;
-#endif
- return 0;
-
-err:
- return 1;
-}
-
static int gru_vtop(struct gru_thread_state *gts, unsigned long vaddr,
int write, int atomic, unsigned long *gpa, int *pageshift)
{
struct mm_struct *mm = gts->ts_mm;
struct vm_area_struct *vma;
unsigned long paddr;
- int ret, ps;
+ int ps;
vma = find_vma(mm, vaddr);
if (!vma)
goto inval;
- /*
- * Atomic lookup is faster & usually works even if called in non-atomic
- * context.
- */
- rmb(); /* Must/check ms_range_active before loading PTEs */
- ret = atomic_pte_lookup(vma, vaddr, write, &paddr, &ps);
- if (ret) {
- if (atomic)
- goto upm;
- if (non_atomic_pte_lookup(vma, vaddr, write, &paddr, &ps))
- goto inval;
- }
+ if (atomic)
+ goto upm;
+
+ /* Order the caller's ms_range_active check before loading PTEs. */
+ rmb();
+ if (non_atomic_pte_lookup(vma, vaddr, write, &paddr, &ps))
+ goto inval;
if (is_gru_paddr(paddr))
goto inval;
paddr = paddr & ~((1UL << ps) - 1);
@@ -569,19 +502,9 @@ static irqreturn_t gru_intr(int chiplet, int blade)
continue;
}
- /*
- * This is running in interrupt context. Trylock the mmap_lock.
- * If it fails, retry the fault in user context.
- */
+ /* Address translation may sleep, so retry the fault in user context. */
gts->ustats.fmm_tlbmiss++;
- if (!gts->ts_force_cch_reload &&
- mmap_read_trylock(gts->ts_mm)) {
- gru_try_dropin(gru, gts, tfh, NULL);
- mmap_read_unlock(gts->ts_mm);
- } else {
- tfh_user_polling_mode(tfh);
- STAT(intr_mm_lock_failed);
- }
+ tfh_user_polling_mode(tfh);
}
return IRQ_HANDLED;
}
diff --git a/drivers/misc/sgi-gru/gruprocfs.c b/drivers/misc/sgi-gru/gruprocfs.c
index 97b8b38ab47d..b8139c27bc7f 100644
--- a/drivers/misc/sgi-gru/gruprocfs.c
+++ b/drivers/misc/sgi-gru/gruprocfs.c
@@ -54,7 +54,6 @@ static int statistics_show(struct seq_file *s, void *p)
printstat(s, intr_cbr);
printstat(s, intr_tfh);
printstat(s, intr_spurious);
- printstat(s, intr_mm_lock_failed);
printstat(s, call_os);
printstat(s, call_os_wait_queue);
printstat(s, user_flush_tlb);
diff --git a/drivers/misc/sgi-gru/grutables.h b/drivers/misc/sgi-gru/grutables.h
index 640daf1994df..3348552925c6 100644
--- a/drivers/misc/sgi-gru/grutables.h
+++ b/drivers/misc/sgi-gru/grutables.h
@@ -182,7 +182,6 @@ struct gru_stats_s {
atomic_long_t intr_cbr;
atomic_long_t intr_tfh;
atomic_long_t intr_spurious;
- atomic_long_t intr_mm_lock_failed;
atomic_long_t call_os;
atomic_long_t call_os_wait_queue;
atomic_long_t user_flush_tlb;
diff --git a/drivers/misc/vmw_vmci/vmci_event.c b/drivers/misc/vmw_vmci/vmci_event.c
index 9a41ab65378d..89ab9f05b88c 100644
--- a/drivers/misc/vmw_vmci/vmci_event.c
+++ b/drivers/misc/vmw_vmci/vmci_event.c
@@ -179,16 +179,16 @@ int vmci_event_subscribe(u32 event,
}
}
+ *new_subscription_id = sub->id;
if (have_new_id) {
list_add_rcu(&sub->node, &subscriber_array[event]);
retval = VMCI_SUCCESS;
} else {
+ kfree(sub);
retval = VMCI_ERROR_NO_RESOURCES;
}
mutex_unlock(&subscriber_mutex);
-
- *new_subscription_id = sub->id;
return retval;
}
EXPORT_SYMBOL_GPL(vmci_event_subscribe);
diff --git a/drivers/mtd/mtdoops.c b/drivers/mtd/mtdoops.c
index 7bf3777e1f13..fb787e9e2495 100644
--- a/drivers/mtd/mtdoops.c
+++ b/drivers/mtd/mtdoops.c
@@ -393,6 +393,9 @@ static void mtdoops_notify_remove(struct mtd_info *mtd)
cxt->mtd = NULL;
flush_work(&cxt->work_erase);
flush_work(&cxt->work_write);
+ vfree(cxt->oops_page_used);
+ cxt->oops_page_used = NULL;
+ cxt->oops_pages = 0;
}
diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c
index 208532d86cc9..c3ba57215c86 100644
--- a/drivers/mtd/mtdpart.c
+++ b/drivers/mtd/mtdpart.c
@@ -254,7 +254,8 @@ int mtd_add_partition(struct mtd_info *parent, const char *name,
/* the direct offset is expected */
if (offset == MTDPART_OFS_APPEND ||
- offset == MTDPART_OFS_NXTBLK)
+ offset == MTDPART_OFS_NXTBLK ||
+ offset == MTDPART_OFS_RETAIN)
return -EINVAL;
if (length == MTDPART_SIZ_FULL)
diff --git a/drivers/mtd/mtdswap.c b/drivers/mtd/mtdswap.c
index 4d695875ea1b..b7d5fe7eef4b 100644
--- a/drivers/mtd/mtdswap.c
+++ b/drivers/mtd/mtdswap.c
@@ -1452,6 +1452,7 @@ static void mtdswap_add_mtd(struct mtd_blktrans_ops *tr, struct mtd_info *mtd)
debugfs_failed:
del_mtd_blktrans_dev(mbd_dev);
+ mbd_dev = NULL;
cleanup:
mtdswap_cleanup(d);
diff --git a/drivers/mtd/nand/raw/nand_onfi.c b/drivers/mtd/nand/raw/nand_onfi.c
index 861975e44b55..f03553eb1324 100644
--- a/drivers/mtd/nand/raw/nand_onfi.c
+++ b/drivers/mtd/nand/raw/nand_onfi.c
@@ -35,16 +35,21 @@ static int nand_flash_detect_ext_param_page(struct nand_chip *chip,
struct nand_onfi_params *p)
{
struct nand_device *base = &chip->base;
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_props requirements;
struct onfi_ext_param_page *ep;
struct onfi_ext_section *s;
struct onfi_ext_ecc_info *ecc;
+ size_t remaining, section_len;
uint8_t *cursor;
int ret;
int len;
int i;
len = le16_to_cpu(p->ext_param_page_length) * 16;
+ if (len < sizeof(*ep))
+ return -EINVAL;
+
ep = kmalloc(len, GFP_KERNEL);
if (!ep)
return -ENOMEM;
@@ -77,11 +82,29 @@ static int nand_flash_detect_ext_param_page(struct nand_chip *chip,
/* find the ECC section. */
cursor = (uint8_t *)(ep + 1);
+ remaining = len - sizeof(*ep);
for (i = 0; i < ONFI_EXT_SECTION_MAX; i++) {
s = ep->sections + i;
- if (s->type == ONFI_SECTION_TYPE_2)
+ section_len = s->length * 16;
+ if (section_len > remaining) {
+ dev_dbg(&mtd->dev,
+ "ONFI extended parameter section %d exceeds page\n",
+ i);
+ goto ext_out;
+ }
+
+ if (s->type == ONFI_SECTION_TYPE_2) {
+ if (section_len < sizeof(*ecc)) {
+ dev_dbg(&mtd->dev,
+ "ONFI extended parameter ECC section %d is too short\n",
+ i);
+ goto ext_out;
+ }
break;
- cursor += s->length * 16;
+ }
+
+ cursor += section_len;
+ remaining -= section_len;
}
if (i == ONFI_EXT_SECTION_MAX) {
pr_debug("We can not find the ECC section.\n");
diff --git a/drivers/mtd/parsers/afs.c b/drivers/mtd/parsers/afs.c
index 26116694c821..7ab3d50f565e 100644
--- a/drivers/mtd/parsers/afs.c
+++ b/drivers/mtd/parsers/afs.c
@@ -235,6 +235,9 @@ static int afs_parse_v2_partition(struct mtd_info *mtd,
pr_debug("Parsing v2 partition @%08x-%08x\n",
off, off + mtd->erasesize);
+ if (mtd->erasesize < sizeof(footer))
+ return -EINVAL;
+
/* First read the footer */
ptr = off + mtd->erasesize - sizeof(footer);
ret = mtd_read(mtd, ptr, sizeof(footer), &sz, (u_char *)footer);
@@ -245,6 +248,8 @@ static int afs_parse_v2_partition(struct mtd_info *mtd,
}
name = (char *) &footer[0];
version = footer[9];
+ if (footer[8] > mtd->erasesize - sizeof(footer))
+ return -EINVAL;
ptr = off + mtd->erasesize - sizeof(footer) - footer[8];
pr_debug("found image \"%s\", version %08x, info @%08x\n",
@@ -278,6 +283,8 @@ static int afs_parse_v2_partition(struct mtd_info *mtd,
entrypoint = imginfo[pad];
attributes = imginfo[pad+1];
region_count = imginfo[pad+2];
+ if (region_count > (ARRAY_SIZE(imginfo) - pad - 3) / 4)
+ return -EINVAL;
block_start = imginfo[20];
block_end = imginfo[21];
diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c
index 884171871d0e..2b1629fdb41b 100644
--- a/drivers/mtd/ubi/attach.c
+++ b/drivers/mtd/ubi/attach.c
@@ -771,7 +771,7 @@ void ubi_remove_av(struct ubi_attach_info *ai, struct ubi_ainf_volume *av)
static int early_erase_peb(struct ubi_device *ubi,
const struct ubi_attach_info *ai, int pnum, int ec)
{
- int err;
+ int err, torture = 0;
struct ubi_ec_hdr *ec_hdr;
if ((long long)ec >= UBI_MAX_ERASECOUNTER) {
@@ -790,7 +790,7 @@ static int early_erase_peb(struct ubi_device *ubi,
ec_hdr->ec = cpu_to_be64(ec);
- err = ubi_io_sync_erase(ubi, pnum, 0);
+ err = ubi_io_sync_erase(ubi, pnum, &torture);
if (err < 0)
goto out_free;
diff --git a/drivers/mtd/ubi/build.c b/drivers/mtd/ubi/build.c
index ef6a22f372f9..fb121b6d0848 100644
--- a/drivers/mtd/ubi/build.c
+++ b/drivers/mtd/ubi/build.c
@@ -1105,6 +1105,7 @@ int ubi_detach_mtd_dev(int ubi_num, int anyway)
ubi->ref_count -= 1;
if (ubi->ref_count) {
if (!anyway) {
+ put_device(&ubi->dev);
spin_unlock(&ubi_devices_lock);
return -EBUSY;
}
@@ -1317,10 +1318,10 @@ static int __init ubi_init_attach(void)
return 0;
out_detach:
- for (k = 0; k < i; k++)
+ for (k = 0; k < UBI_MAX_DEVICES; k++)
if (ubi_devices[k]) {
mutex_lock(&ubi_devices_mutex);
- ubi_detach_mtd_dev(ubi_devices[k]->ubi_num, 1);
+ ubi_detach_mtd_dev(k, 1);
mutex_unlock(&ubi_devices_mutex);
}
return err;
@@ -1574,7 +1575,7 @@ static int ubi_mtd_param_parse(const char *val, const struct kernel_param *kp)
}
module_param_call(mtd, ubi_mtd_param_parse, NULL, NULL, 0400);
-MODULE_PARM_DESC(mtd, "MTD devices to attach. Parameter format: mtd=<name|num|path>[,<vid_hdr_offs>[,max_beb_per1024[,ubi_num]]].\n"
+MODULE_PARM_DESC(mtd, "MTD devices to attach. Parameter format: mtd=<name|num|path>[,<vid_hdr_offs>[,max_beb_per1024[,ubi_num[,enable_fm[,need_resv_pool]]]]].\n"
"Multiple \"mtd\" parameters may be specified.\n"
"MTD devices may be specified by their number, name, or path to the MTD character device node.\n"
"Optional \"vid_hdr_offs\" parameter specifies UBI VID header position to be used by UBI. (default value if 0)\n"
@@ -1587,9 +1588,9 @@ MODULE_PARM_DESC(mtd, "MTD devices to attach. Parameter format: mtd=<name|num|pa
"Example 1: mtd=/dev/mtd0 - attach MTD device /dev/mtd0.\n"
"Example 2: mtd=content,1984 mtd=4 - attach MTD device with name \"content\" using VID header offset 1984, and MTD device number 4 with default VID header offset.\n"
"Example 3: mtd=/dev/mtd1,0,25 - attach MTD device /dev/mtd1 using default VID header offset and reserve 25*nand_size_in_blocks/1024 erase blocks for bad block handling.\n"
+ "\t(e.g. if the NAND *chipset* has 4096 PEB, 100 will be reserved for this UBI device).\n"
"Example 4: mtd=/dev/mtd1,0,0,5 - attach MTD device /dev/mtd1 to UBI 5 and using default values for the other fields.\n"
- "example 5: mtd=1,0,0,5 mtd=2,0,0,6,1 - attach MTD device /dev/mtd1 to UBI 5 and disable fastmap; attach MTD device /dev/mtd2 to UBI 6 and enable fastmap.(only works when fastmap is enabled and fm_autoconvert=Y).\n"
- "\t(e.g. if the NAND *chipset* has 4096 PEB, 100 will be reserved for this UBI device).");
+ "example 5: mtd=1,0,0,5 mtd=2,0,0,6,1 - attach MTD device /dev/mtd1 to UBI 5 and disable fastmap; attach MTD device /dev/mtd2 to UBI 6 and enable fastmap.(only works when fastmap is enabled and fm_autoconvert=Y).\n");
#ifdef CONFIG_MTD_UBI_FASTMAP
module_param(fm_autoconvert, bool, 0644);
MODULE_PARM_DESC(fm_autoconvert, "Set this parameter to enable fastmap automatically on images without a fastmap.");
diff --git a/drivers/mtd/ubi/fastmap.c b/drivers/mtd/ubi/fastmap.c
index 9a4940874be5..12dd05fd3778 100644
--- a/drivers/mtd/ubi/fastmap.c
+++ b/drivers/mtd/ubi/fastmap.c
@@ -1526,7 +1526,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
if (!tmp_e) {
if (old_fm && old_fm->e[i]) {
- ret = ubi_sync_erase(ubi, old_fm->e[i], 0);
+ ret = ubi_sync_erase(ubi, old_fm->e[i],
+ &old_fm->to_be_tortured[i]);
if (ret < 0) {
ubi_err(ubi, "could not erase old fastmap PEB");
@@ -1578,7 +1579,8 @@ int ubi_update_fastmap(struct ubi_device *ubi)
if (old_fm) {
/* no fresh anchor PEB was found, reuse the old one */
if (!tmp_e) {
- ret = ubi_sync_erase(ubi, old_fm->e[0], 0);
+ ret = ubi_sync_erase(ubi, old_fm->e[0],
+ &old_fm->to_be_tortured[0]);
if (ret < 0) {
ubi_err(ubi, "could not erase old anchor PEB");
diff --git a/drivers/mtd/ubi/io.c b/drivers/mtd/ubi/io.c
index 915eb64cb001..d70d1313ad4e 100644
--- a/drivers/mtd/ubi/io.c
+++ b/drivers/mtd/ubi/io.c
@@ -519,8 +519,10 @@ static int nor_erase_prepare(struct ubi_device *ubi, int pnum)
/**
* ubi_io_sync_erase - synchronously erase a physical eraseblock.
* @ubi: UBI device description object
- * @pnum: physical eraseblock number to erase
- * @torture: if this physical eraseblock has to be tortured
+ * @pnum: the physical eraseblock number to erase
+ * @torture: if this physical eraseblock has to be tortured; cleared to zero
+ * once the torture test has completed successfully so that a retry
+ * of the erase does not torture the physical eraseblock again
*
* This function synchronously erases physical eraseblock @pnum. If @torture
* flag is not zero, the physical eraseblock is checked by means of writing
@@ -532,7 +534,7 @@ static int nor_erase_prepare(struct ubi_device *ubi, int pnum)
* codes in case of other errors. Note, %-EIO means that the physical
* eraseblock is bad.
*/
-int ubi_io_sync_erase(struct ubi_device *ubi, int pnum, int torture)
+int ubi_io_sync_erase(struct ubi_device *ubi, int pnum, int *torture)
{
int err, ret = 0;
@@ -560,10 +562,11 @@ int ubi_io_sync_erase(struct ubi_device *ubi, int pnum, int torture)
return err;
}
- if (torture) {
+ if (*torture) {
ret = torture_peb(ubi, pnum);
if (ret < 0)
return ret;
+ *torture = 0;
}
err = do_sync_erase(ubi, pnum);
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index 450a7b9a5bd7..515e47876900 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -902,7 +902,7 @@ int self_check_eba(struct ubi_device *ubi, struct ubi_attach_info *ai_fastmap,
struct ubi_attach_info *ai_scan);
/* wl.c */
-int ubi_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int torture);
+int ubi_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int *torture);
int ubi_wl_get_peb(struct ubi_device *ubi);
int ubi_wl_put_peb(struct ubi_device *ubi, int vol_id, int lnum,
int pnum, int torture);
@@ -924,7 +924,7 @@ int ubi_io_read(const struct ubi_device *ubi, void *buf, int pnum, int offset,
int len);
int ubi_io_write(struct ubi_device *ubi, const void *buf, int pnum, int offset,
int len);
-int ubi_io_sync_erase(struct ubi_device *ubi, int pnum, int torture);
+int ubi_io_sync_erase(struct ubi_device *ubi, int pnum, int *torture);
int ubi_io_is_bad(const struct ubi_device *ubi, int pnum);
int ubi_io_mark_bad(const struct ubi_device *ubi, int pnum);
int ubi_io_read_ec_hdr(struct ubi_device *ubi, int pnum,
diff --git a/drivers/mtd/ubi/wl.c b/drivers/mtd/ubi/wl.c
index fbd399cf6503..fefcbaf25002 100644
--- a/drivers/mtd/ubi/wl.c
+++ b/drivers/mtd/ubi/wl.c
@@ -443,12 +443,14 @@ static int prot_queue_del(struct ubi_device *ubi, int pnum)
* ubi_sync_erase - synchronously erase a physical eraseblock.
* @ubi: UBI device description object
* @e: the physical eraseblock to erase
- * @torture: if the physical eraseblock has to be tortured
+ * @torture: if the physical eraseblock has to be tortured; cleared to zero
+ * once the torture test has completed successfully so that a retry
+ * of the erase does not torture the physical eraseblock again
*
* This function returns zero in case of success and a negative error code in
* case of failure.
*/
-int ubi_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int torture)
+int ubi_sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int *torture)
{
int err;
struct ubi_ec_hdr *ec_hdr;
@@ -1113,7 +1115,7 @@ static int __erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk)
dbg_wl("erase PEB %d EC %d LEB %d:%d",
pnum, e->ec, wl_wrk->vol_id, wl_wrk->lnum);
- err = ubi_sync_erase(ubi, e, wl_wrk->torture);
+ err = ubi_sync_erase(ubi, e, &wl_wrk->torture);
if (!err) {
spin_lock(&ubi->wl_lock);
@@ -1150,7 +1152,8 @@ static int __erase_worker(struct ubi_device *ubi, struct ubi_work *wl_wrk)
int err1;
/* Re-schedule the LEB for erasure */
- err1 = schedule_erase(ubi, e, vol_id, lnum, 0, true);
+ err1 = schedule_erase(ubi, e, vol_id, lnum, wl_wrk->torture,
+ true);
if (err1) {
spin_lock(&ubi->wl_lock);
wl_entry_destroy(ubi, e);
@@ -1757,7 +1760,7 @@ static void shutdown_work(struct ubi_device *ubi)
static int erase_aeb(struct ubi_device *ubi, struct ubi_ainf_peb *aeb, bool sync)
{
struct ubi_wl_entry *e;
- int err;
+ int err, torture = 0;
e = kmem_cache_alloc(ubi_wl_entry_slab, GFP_KERNEL);
if (!e)
@@ -1768,7 +1771,7 @@ static int erase_aeb(struct ubi_device *ubi, struct ubi_ainf_peb *aeb, bool sync
ubi->lookuptbl[e->pnum] = e;
if (sync) {
- err = ubi_sync_erase(ubi, e, false);
+ err = ubi_sync_erase(ubi, e, &torture);
if (err)
goto out_free;
diff --git a/drivers/net/bonding/bond_netlink.c b/drivers/net/bonding/bond_netlink.c
index a754bad4c5d6..ff0c04cfe971 100644
--- a/drivers/net/bonding/bond_netlink.c
+++ b/drivers/net/bonding/bond_netlink.c
@@ -218,7 +218,7 @@ static int bond_changelink(struct net_device *bond_dev, struct nlattr *tb[],
struct bonding *bond = netdev_priv(bond_dev);
struct bond_opt_value newval;
int miimon = 0;
- int err;
+ int err = 0;
if (!data)
return 0;
diff --git a/drivers/net/dsa/b53/b53_common.c b/drivers/net/dsa/b53/b53_common.c
index 0b666a77ea97..04027cbefddb 100644
--- a/drivers/net/dsa/b53/b53_common.c
+++ b/drivers/net/dsa/b53/b53_common.c
@@ -2020,7 +2020,7 @@ int b53_fdb_dump(struct dsa_switch *ds, int port,
mutex_unlock(&priv->arl_mutex);
- return 0;
+ return ret;
}
EXPORT_SYMBOL(b53_fdb_dump);
diff --git a/drivers/net/dsa/mv88e6xxx/pcs-6352.c b/drivers/net/dsa/mv88e6xxx/pcs-6352.c
index 88f624b65470..f0b42786d587 100644
--- a/drivers/net/dsa/mv88e6xxx/pcs-6352.c
+++ b/drivers/net/dsa/mv88e6xxx/pcs-6352.c
@@ -305,13 +305,16 @@ static bool mv88e6352_pcs_link_check(struct marvell_c22_pcs *mpcs)
struct mv88e6xxx_port *port = mpcs->port;
struct mv88e6xxx_chip *chip = port->chip;
u8 cmode;
+ int err;
/* Port 4 can be in auto-media mode. Check that the port is
* associated with the mpcs.
*/
mv88e6xxx_reg_lock(chip);
- chip->info->ops->port_get_cmode(chip, port->port, &cmode);
+ err = chip->info->ops->port_get_cmode(chip, port->port, &cmode);
mv88e6xxx_reg_unlock(chip);
+ if (err)
+ return false;
return cmode == MV88E6XXX_PORT_STS_CMODE_100BASEX ||
cmode == MV88E6XXX_PORT_STS_CMODE_1000BASEX ||
diff --git a/drivers/net/ethernet/allwinner/sun4i-emac.c b/drivers/net/ethernet/allwinner/sun4i-emac.c
index d761c08fe5c1..e351ba1de844 100644
--- a/drivers/net/ethernet/allwinner/sun4i-emac.c
+++ b/drivers/net/ethernet/allwinner/sun4i-emac.c
@@ -996,9 +996,9 @@ static int emac_probe(struct platform_device *pdev)
/* fill in parameters for net-dev structure */
ndev->base_addr = (unsigned long)db->membase;
ndev->irq = irq_of_parse_and_map(np, 0);
- if (ndev->irq == -ENXIO) {
+ if (!ndev->irq) {
netdev_err(ndev, "No irq resource\n");
- ret = ndev->irq;
+ ret = -ENXIO;
goto out_iounmap;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwmon.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwmon.c
index 669d24ba0e87..a53815bbe272 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwmon.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwmon.c
@@ -40,7 +40,7 @@ void bnxt_hwmon_notify_event(struct bnxt *bp)
return;
}
- hwmon_notify_event(&bp->pdev->dev, hwmon_temp, attr, 0);
+ hwmon_notify_event(bp->hwmon_dev, hwmon_temp, attr, 0);
}
static int bnxt_hwrm_temp_query(struct bnxt *bp, u8 *temp)
diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c
index 22cd47d19ce2..1af74bbf385f 100644
--- a/drivers/net/ethernet/freescale/fec_main.c
+++ b/drivers/net/ethernet/freescale/fec_main.c
@@ -4601,7 +4601,8 @@ fec_probe(struct platform_device *pdev)
failed_irq:
fec_enet_deinit(ndev);
failed_init:
- fec_ptp_stop(pdev);
+ if (fep->bufdesc_ex)
+ fec_ptp_stop(pdev);
failed_reset:
pm_runtime_put_noidle(&pdev->dev);
pm_runtime_disable(&pdev->dev);
@@ -4643,7 +4644,8 @@ fec_drv_remove(struct platform_device *pdev)
ERR_PTR(ret));
cancel_work_sync(&fep->tx_timeout_work);
- fec_ptp_stop(pdev);
+ if (fep->bufdesc_ex)
+ fec_ptp_stop(pdev);
unregister_netdev(ndev);
fec_enet_mii_remove(fep);
if (fep->reg_phy)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 4ad21c21c5c5..7d982a2b001c 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -2803,6 +2803,9 @@ int ice_vsi_release(struct ice_vsi *vsi)
return -ENODEV;
pf = vsi->back;
+ if (ice_is_vsi_dflt_vsi(vsi))
+ ice_clear_dflt_vsi(vsi);
+
if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
ice_rss_clean(vsi);
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c b/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c
index d39d86e694cc..006d3258fd4e 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/mcs_rvu_if.c
@@ -854,7 +854,7 @@ int rvu_mbox_handler_mcs_ctrl_pkt_rule_write(struct rvu *rvu,
static void rvu_mcs_set_lmac_bmap(struct rvu *rvu)
{
struct mcs *mcs = mcs_get_pdata(0);
- unsigned long lmac_bmap;
+ unsigned long lmac_bmap = 0;
int cgx, lmac, port;
for (port = 0; port < mcs->hw->lmac_cnt; port++) {
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
index 53c2a9d6250a..f4a4fe5f4a79 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
@@ -1659,6 +1659,12 @@ static int rvu_dbg_nix_tm_tree_display(struct seq_file *m, void *unused)
return -EINVAL;
pfvf = rvu_get_pfvf(rvu, pcifunc);
+
+ if (!pfvf->sq_ctx) {
+ seq_printf(m, "SQ context is not initialized for pcifunc 0x%x\n", pcifunc);
+ return -EINVAL;
+ }
+
max_id = pfvf->sq_ctx->qsize;
memset(&aq_req, 0, sizeof(struct nix_aq_enq_req));
diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
index f9b7263eeb37..74be2012ec04 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
+++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c
@@ -2370,8 +2370,8 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr,
int pf = rvu_get_pf(pcifunc);
u8 cgx_id = 0, lmac_id = 0;
u16 tl2_tl3_link_schq;
- u8 link, link_level;
u64 cfg, bmap = 0;
+ u8 link_level;
if (!is_rvu_otx2(rvu)) {
/* Skip SMQ flush if pkt count is zero */
@@ -2403,7 +2403,6 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr,
link_level = rvu_read64(rvu, blkaddr, NIX_AF_PSE_CHANNEL_LEVEL) & 0x01 ?
NIX_TXSCH_LVL_TL3 : NIX_TXSCH_LVL_TL2;
tl2_tl3_link_schq = smq_flush_ctx->smq_tree_ctx[link_level].schq;
- link = smq_flush_ctx->smq_tree_ctx[NIX_TXSCH_LVL_TL1].schq;
/* SMQ set enqueue xoff */
cfg = rvu_read64(rvu, blkaddr, NIX_AF_SMQX_CFG(smq));
@@ -2413,13 +2412,13 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr,
/* Clear all NIX_AF_TL3_TL2_LINK_CFG[ENA] for the TL3/TL2 queue */
for (i = 0; i < (rvu->hw->cgx_links + rvu->hw->lbk_links); i++) {
cfg = rvu_read64(rvu, blkaddr,
- NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, link));
+ NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, i));
if (!(cfg & BIT_ULL(12)))
continue;
bmap |= BIT_ULL(i);
cfg &= ~BIT_ULL(12);
rvu_write64(rvu, blkaddr,
- NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, link), cfg);
+ NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, i), cfg);
}
/* Do SMQ flush and set enqueue xoff */
@@ -2440,10 +2439,10 @@ static int nix_smq_flush(struct rvu *rvu, int blkaddr,
if (!(bmap & BIT_ULL(i)))
continue;
cfg = rvu_read64(rvu, blkaddr,
- NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, link));
+ NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, i));
cfg |= BIT_ULL(12);
rvu_write64(rvu, blkaddr,
- NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, link), cfg);
+ NIX_AF_TL3_TL2X_LINKX_CFG(tl2_tl3_link_schq, i), cfg);
}
/* clear XOFF on TL2s */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index 5873fde65c2e..e9912369e361 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -245,7 +245,7 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
}
if (unlikely(aff_change && busy_xsk)) {
- mlx5e_trigger_irq(&c->icosq);
+ mlx5e_trigger_napi_icosq(c);
ch_stats->force_irq++;
}
diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c
index c9fb3be4fca9..f28ea729ebb4 100644
--- a/drivers/net/ethernet/nvidia/forcedeth.c
+++ b/drivers/net/ethernet/nvidia/forcedeth.c
@@ -2754,7 +2754,7 @@ static void nv_tx_timeout(struct net_device *dev, unsigned int txqueue)
netdev_info(dev, "Ring at %lx\n", (unsigned long)np->ring_addr);
netdev_info(dev, "Dumping tx registers\n");
- for (i = 0; i <= np->register_size; i += 32) {
+ for (i = 0; i + 32 <= np->register_size; i += 32) {
netdev_info(dev,
"%3x: %08x %08x %08x %08x "
"%08x %08x %08x %08x\n",
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
index c55348e53c45..14d5961f78fd 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
@@ -706,11 +706,7 @@ static void ionic_rx_clean(struct ionic_queue *q,
__le64 *cq_desc_hwstamp;
u64 hwstamp;
- cq_desc_hwstamp =
- (void *)comp +
- qcq->cq.desc_size -
- sizeof(struct ionic_rxq_comp) -
- IONIC_HWSTAMP_CQ_NEGOFFSET;
+ cq_desc_hwstamp = (void *)comp - IONIC_HWSTAMP_CQ_NEGOFFSET;
hwstamp = le64_to_cpu(*cq_desc_hwstamp);
@@ -734,7 +730,12 @@ static bool __ionic_rx_service(struct ionic_cq *cq, struct bpf_prog *xdp_prog)
struct ionic_queue *q = cq->bound_q;
struct ionic_rxq_comp *comp;
- comp = &((struct ionic_rxq_comp *)cq->base)[cq->tail_idx];
+ if (likely(cq->desc_size == sizeof(*comp)))
+ comp = &((struct ionic_rxq_comp *)cq->base)[cq->tail_idx];
+ else
+ comp = cq->base +
+ cq->desc_size * cq->tail_idx +
+ cq->desc_size - sizeof(*comp);
if (!color_match(comp->pkt_type_color, cq->done_color))
return false;
@@ -1185,7 +1186,6 @@ static void ionic_tx_clean(struct ionic_queue *q,
bool in_napi)
{
struct ionic_tx_stats *stats = q_to_tx_stats(q);
- struct ionic_qcq *qcq = q_to_qcq(q);
struct sk_buff *skb;
if (desc_info->xdpf) {
@@ -1210,11 +1210,7 @@ static void ionic_tx_clean(struct ionic_queue *q,
__le64 *cq_desc_hwstamp;
u64 hwstamp;
- cq_desc_hwstamp =
- (void *)comp +
- qcq->cq.desc_size -
- sizeof(struct ionic_txq_comp) -
- IONIC_HWSTAMP_CQ_NEGOFFSET;
+ cq_desc_hwstamp = (void *)comp - IONIC_HWSTAMP_CQ_NEGOFFSET;
hwstamp = le64_to_cpu(*cq_desc_hwstamp);
@@ -1249,7 +1245,12 @@ static bool ionic_tx_service(struct ionic_cq *cq,
unsigned int pkts = 0;
u16 index;
- comp = &((struct ionic_txq_comp *)cq->base)[cq->tail_idx];
+ if (likely(cq->desc_size == sizeof(*comp)))
+ comp = &((struct ionic_txq_comp *)cq->base)[cq->tail_idx];
+ else
+ comp = cq->base +
+ cq->desc_size * cq->tail_idx +
+ cq->desc_size - sizeof(*comp);
if (!color_match(comp->color, cq->done_color))
return false;
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_init.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_init.c
index 09f20c794754..d26c26c21cbe 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_init.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_init.c
@@ -6,6 +6,7 @@
#include "qlcnic.h"
#include "qlcnic_hw.h"
+#include <linux/unaligned.h>
struct crb_addr_pair {
u32 addr;
@@ -740,159 +741,212 @@ qlcnic_has_mn(struct qlcnic_adapter *adapter)
return 0;
}
-static
-struct uni_table_desc *qlcnic_get_table_desc(const u8 *unirom, int section)
-{
- u32 i, entries;
- struct uni_table_desc *directory = (struct uni_table_desc *) &unirom[0];
- entries = le32_to_cpu(directory->num_entries);
-
- for (i = 0; i < entries; i++) {
+#define FILEHEADER_SIZE (14 * 4)
+#define QLCNIC_UNI_DIR_TYPE_OFF (8 * sizeof(__le32))
+#define QLCNIC_UNI_DIR_ENTRY_MIN_SIZE (9 * sizeof(__le32))
+#define QLCNIC_UNI_PRODUCT_ENTRY_MIN_SIZE \
+ ((QLCNIC_UNI_FIRMWARE_IDX_OFF + 1) * sizeof(__le32))
+#define QLCNIC_UNI_VERSION_TAIL_SIZE 17
+#define QLCNIC_UNI_BOOTLD_SIZE \
+ (QLCNIC_IMAGE_START - QLCNIC_BOOTLD_START)
+
+struct qlcnic_uni_data {
+ u32 offset;
+ u32 size;
+};
- u32 offs = le32_to_cpu(directory->findex) +
- i * le32_to_cpu(directory->entry_size);
- u32 tab_type = le32_to_cpu(*((__le32 *)&unirom[offs] + 8));
+static bool qlcnic_rom_range_valid(size_t size, size_t offset, size_t len)
+{
+ return offset <= size && len <= size - offset;
+}
- if (tab_type == section)
- return (struct uni_table_desc *) &unirom[offs];
- }
+static bool qlcnic_rom_table_valid(size_t size, u32 offset, u32 entries,
+ u32 entry_size, u32 min_entry_size)
+{
+ if (entry_size < min_entry_size || offset > size)
+ return false;
- return NULL;
+ return entries <= (size - offset) / entry_size;
}
-#define FILEHEADER_SIZE (14 * 4)
-
-static int
-qlcnic_validate_header(struct qlcnic_adapter *adapter)
+static int qlcnic_get_directory(struct qlcnic_adapter *adapter,
+ size_t *offset, u32 *entries, u32 *entry_size)
{
- const u8 *unirom = adapter->fw->data;
- struct uni_table_desc *directory = (struct uni_table_desc *) &unirom[0];
- u32 entries, entry_size, tab_size, fw_file_size;
-
- fw_file_size = adapter->fw->size;
+ const struct firmware *fw = adapter->fw;
+ const u8 *directory = fw->data;
- if (fw_file_size < FILEHEADER_SIZE)
+ if (fw->size < FILEHEADER_SIZE)
return -EINVAL;
- entries = le32_to_cpu(directory->num_entries);
- entry_size = le32_to_cpu(directory->entry_size);
- tab_size = le32_to_cpu(directory->findex) + (entries * entry_size);
+ *offset = get_unaligned_le32(directory +
+ offsetof(struct uni_table_desc, findex));
+ *entries = get_unaligned_le32(directory +
+ offsetof(struct uni_table_desc, num_entries));
+ *entry_size = get_unaligned_le32(directory +
+ offsetof(struct uni_table_desc, entry_size));
- if (fw_file_size < tab_size)
+ if (!qlcnic_rom_table_valid(fw->size, *offset, *entries, *entry_size,
+ QLCNIC_UNI_DIR_ENTRY_MIN_SIZE))
return -EINVAL;
return 0;
}
-static int
-qlcnic_validate_bootld(struct qlcnic_adapter *adapter)
+static int qlcnic_get_table_desc(struct qlcnic_adapter *adapter, int section,
+ size_t *desc_offset)
{
- struct uni_table_desc *tab_desc;
- struct uni_data_desc *descr;
- u32 offs, tab_size, data_size, idx;
const u8 *unirom = adapter->fw->data;
- __le32 temp;
-
- temp = *((__le32 *)&unirom[adapter->file_prd_off] +
- QLCNIC_UNI_BOOTLD_IDX_OFF);
- idx = le32_to_cpu(temp);
- tab_desc = qlcnic_get_table_desc(unirom, QLCNIC_UNI_DIR_SECT_BOOTLD);
-
- if (!tab_desc)
- return -EINVAL;
-
- tab_size = le32_to_cpu(tab_desc->findex) +
- le32_to_cpu(tab_desc->entry_size) * (idx + 1);
-
- if (adapter->fw->size < tab_size)
- return -EINVAL;
+ size_t directory_offset;
+ u32 entries, entry_size;
+ size_t i;
+ int ret;
- offs = le32_to_cpu(tab_desc->findex) +
- le32_to_cpu(tab_desc->entry_size) * idx;
- descr = (struct uni_data_desc *)&unirom[offs];
+ ret = qlcnic_get_directory(adapter, &directory_offset, &entries,
+ &entry_size);
+ if (ret)
+ return ret;
- data_size = le32_to_cpu(descr->findex) + le32_to_cpu(descr->size);
+ for (i = 0; i < entries; i++) {
+ size_t offset = directory_offset + i * entry_size;
+ u32 table_type;
- if (adapter->fw->size < data_size)
- return -EINVAL;
+ table_type = get_unaligned_le32(unirom + offset +
+ QLCNIC_UNI_DIR_TYPE_OFF);
+ if (table_type == section) {
+ *desc_offset = offset;
+ return 0;
+ }
+ }
- return 0;
+ return -ENOENT;
}
static int
-qlcnic_validate_fw(struct qlcnic_adapter *adapter)
+qlcnic_validate_header(struct qlcnic_adapter *adapter)
{
- struct uni_table_desc *tab_desc;
- struct uni_data_desc *descr;
- const u8 *unirom = adapter->fw->data;
- u32 offs, tab_size, data_size, idx;
- __le32 temp;
+ u32 entries, entry_size;
+ size_t offset;
+
+ return qlcnic_get_directory(adapter, &offset, &entries, &entry_size);
+}
- temp = *((__le32 *)&unirom[adapter->file_prd_off] +
- QLCNIC_UNI_FIRMWARE_IDX_OFF);
- idx = le32_to_cpu(temp);
- tab_desc = qlcnic_get_table_desc(unirom, QLCNIC_UNI_DIR_SECT_FW);
+static int qlcnic_get_data_desc(struct qlcnic_adapter *adapter, u32 section,
+ u32 index_offset, struct qlcnic_uni_data *data)
+{
+ size_t table_desc_offset, table_offset, desc_offset;
+ const struct firmware *fw = adapter->fw;
+ const u8 *unirom = fw->data;
+ size_t product_index_offset;
+ u32 entries, entry_size, idx;
+ int ret;
- if (!tab_desc)
+ product_index_offset = adapter->file_prd_off +
+ (size_t)index_offset * sizeof(__le32);
+ if (!qlcnic_rom_range_valid(fw->size, product_index_offset,
+ sizeof(__le32)))
return -EINVAL;
- tab_size = le32_to_cpu(tab_desc->findex) +
- le32_to_cpu(tab_desc->entry_size) * (idx + 1);
+ idx = get_unaligned_le32(unirom + product_index_offset);
+ ret = qlcnic_get_table_desc(adapter, section, &table_desc_offset);
+ if (ret)
+ return ret;
- if (adapter->fw->size < tab_size)
+ table_offset = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, findex));
+ entries = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, num_entries));
+ entry_size = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, entry_size));
+ if (!qlcnic_rom_table_valid(fw->size, table_offset, entries,
+ entry_size, sizeof(struct uni_data_desc)) ||
+ idx >= entries)
return -EINVAL;
- offs = le32_to_cpu(tab_desc->findex) +
- le32_to_cpu(tab_desc->entry_size) * idx;
- descr = (struct uni_data_desc *)&unirom[offs];
- data_size = le32_to_cpu(descr->findex) + le32_to_cpu(descr->size);
+ desc_offset = table_offset + (size_t)idx * entry_size;
+ data->offset = get_unaligned_le32(unirom + desc_offset +
+ offsetof(struct uni_data_desc, findex));
+ data->size = get_unaligned_le32(unirom + desc_offset +
+ offsetof(struct uni_data_desc, size));
- if (adapter->fw->size < data_size)
+ if (!qlcnic_rom_range_valid(fw->size, data->offset, data->size))
return -EINVAL;
return 0;
}
+static int qlcnic_validate_bootld(struct qlcnic_adapter *adapter)
+{
+ struct qlcnic_uni_data data;
+ int ret;
+
+ ret = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_BOOTLD,
+ QLCNIC_UNI_BOOTLD_IDX_OFF, &data);
+ if (ret)
+ return ret;
+
+ return data.size < QLCNIC_UNI_BOOTLD_SIZE ? -EINVAL : 0;
+}
+
+static int qlcnic_validate_fw(struct qlcnic_adapter *adapter)
+{
+ struct qlcnic_uni_data data;
+ int ret;
+
+ ret = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
+ QLCNIC_UNI_FIRMWARE_IDX_OFF, &data);
+ if (ret)
+ return ret;
+
+ return data.size < QLCNIC_UNI_VERSION_TAIL_SIZE ? -EINVAL : 0;
+}
+
static int
qlcnic_validate_product_offs(struct qlcnic_adapter *adapter)
{
- struct uni_table_desc *ptab_descr;
+ size_t table_desc_offset, table_offset;
const u8 *unirom = adapter->fw->data;
int mn_present = qlcnic_has_mn(adapter);
- u32 entries, entry_size, tab_size, i;
- __le32 temp;
-
- ptab_descr = qlcnic_get_table_desc(unirom,
- QLCNIC_UNI_DIR_SECT_PRODUCT_TBL);
- if (!ptab_descr)
- return -EINVAL;
+ u32 entries, entry_size;
+ size_t i;
+ int ret;
- entries = le32_to_cpu(ptab_descr->num_entries);
- entry_size = le32_to_cpu(ptab_descr->entry_size);
- tab_size = le32_to_cpu(ptab_descr->findex) + (entries * entry_size);
+ ret = qlcnic_get_table_desc(adapter, QLCNIC_UNI_DIR_SECT_PRODUCT_TBL,
+ &table_desc_offset);
+ if (ret)
+ return ret;
- if (adapter->fw->size < tab_size)
+ table_offset = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, findex));
+ entries = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, num_entries));
+ entry_size = get_unaligned_le32(unirom + table_desc_offset +
+ offsetof(struct uni_table_desc, entry_size));
+ if (!qlcnic_rom_table_valid(adapter->fw->size, table_offset, entries,
+ entry_size,
+ QLCNIC_UNI_PRODUCT_ENTRY_MIN_SIZE))
return -EINVAL;
nomn:
for (i = 0; i < entries; i++) {
-
- u32 flags, file_chiprev, offs;
+ size_t offset = table_offset + i * entry_size;
u8 chiprev = adapter->ahw->revision_id;
+ u32 flags, file_chiprev;
u32 flagbit;
- offs = le32_to_cpu(ptab_descr->findex) +
- i * le32_to_cpu(ptab_descr->entry_size);
- temp = *((__le32 *)&unirom[offs] + QLCNIC_UNI_FLAGS_OFF);
- flags = le32_to_cpu(temp);
- temp = *((__le32 *)&unirom[offs] + QLCNIC_UNI_CHIP_REV_OFF);
- file_chiprev = le32_to_cpu(temp);
+ flags = get_unaligned_le32(unirom + offset +
+ QLCNIC_UNI_FLAGS_OFF * sizeof(__le32));
+ file_chiprev = get_unaligned_le32(unirom + offset +
+ QLCNIC_UNI_CHIP_REV_OFF *
+ sizeof(__le32));
flagbit = mn_present ? 1 : 2;
if ((chiprev == file_chiprev) &&
((1ULL << flagbit) & flags)) {
- adapter->file_prd_off = offs;
+ if (offset > U32_MAX)
+ return -EINVAL;
+
+ adapter->file_prd_off = offset;
return 0;
}
}
@@ -933,93 +987,81 @@ qlcnic_validate_unified_romimage(struct qlcnic_adapter *adapter)
return 0;
}
-static
-struct uni_data_desc *qlcnic_get_data_desc(struct qlcnic_adapter *adapter,
- u32 section, u32 idx_offset)
-{
- const u8 *unirom = adapter->fw->data;
- struct uni_table_desc *tab_desc;
- u32 offs, idx;
- __le32 temp;
-
- temp = *((__le32 *)&unirom[adapter->file_prd_off] + idx_offset);
- idx = le32_to_cpu(temp);
-
- tab_desc = qlcnic_get_table_desc(unirom, section);
-
- if (tab_desc == NULL)
- return NULL;
-
- offs = le32_to_cpu(tab_desc->findex) +
- le32_to_cpu(tab_desc->entry_size) * idx;
-
- return (struct uni_data_desc *)&unirom[offs];
-}
-
-static u8 *
-qlcnic_get_bootld_offs(struct qlcnic_adapter *adapter)
+static int qlcnic_get_bootld_data(struct qlcnic_adapter *adapter,
+ const u8 **bootld)
{
u32 offs = QLCNIC_BOOTLD_START;
- struct uni_data_desc *data_desc;
-
- data_desc = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_BOOTLD,
- QLCNIC_UNI_BOOTLD_IDX_OFF);
+ struct qlcnic_uni_data data;
+ int ret;
- if (adapter->ahw->fw_type == QLCNIC_UNIFIED_ROMIMAGE)
- offs = le32_to_cpu(data_desc->findex);
+ if (adapter->ahw->fw_type == QLCNIC_UNIFIED_ROMIMAGE) {
+ ret = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_BOOTLD,
+ QLCNIC_UNI_BOOTLD_IDX_OFF, &data);
+ if (ret || data.size < QLCNIC_UNI_BOOTLD_SIZE)
+ return -EINVAL;
+ offs = data.offset;
+ } else if (!qlcnic_rom_range_valid(adapter->fw->size, offs,
+ QLCNIC_UNI_BOOTLD_SIZE)) {
+ return -EINVAL;
+ }
- return (u8 *)&adapter->fw->data[offs];
+ *bootld = adapter->fw->data + offs;
+ return 0;
}
-static u8 *
-qlcnic_get_fw_offs(struct qlcnic_adapter *adapter)
+static int qlcnic_get_fw_data(struct qlcnic_adapter *adapter,
+ const u8 **image, u32 *image_size)
{
u32 offs = QLCNIC_IMAGE_START;
- struct uni_data_desc *data_desc;
-
- data_desc = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
- QLCNIC_UNI_FIRMWARE_IDX_OFF);
- if (adapter->ahw->fw_type == QLCNIC_UNIFIED_ROMIMAGE)
- offs = le32_to_cpu(data_desc->findex);
-
- return (u8 *)&adapter->fw->data[offs];
-}
+ struct qlcnic_uni_data data;
+ int ret;
-static u32 qlcnic_get_fw_size(struct qlcnic_adapter *adapter)
-{
- struct uni_data_desc *data_desc;
- const u8 *unirom = adapter->fw->data;
+ if (adapter->ahw->fw_type == QLCNIC_UNIFIED_ROMIMAGE) {
+ ret = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
+ QLCNIC_UNI_FIRMWARE_IDX_OFF, &data);
+ if (ret)
+ return ret;
+ offs = data.offset;
+ *image_size = data.size;
+ } else {
+ if (!qlcnic_rom_range_valid(adapter->fw->size,
+ QLCNIC_FW_SIZE_OFFSET,
+ sizeof(__le32)))
+ return -EINVAL;
+ *image_size = get_unaligned_le32(adapter->fw->data +
+ QLCNIC_FW_SIZE_OFFSET);
+ }
- data_desc = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
- QLCNIC_UNI_FIRMWARE_IDX_OFF);
+ if (!qlcnic_rom_range_valid(adapter->fw->size, offs, *image_size))
+ return -EINVAL;
- if (adapter->ahw->fw_type == QLCNIC_UNIFIED_ROMIMAGE)
- return le32_to_cpu(data_desc->size);
- else
- return le32_to_cpu(*(__le32 *)&unirom[QLCNIC_FW_SIZE_OFFSET]);
+ *image = adapter->fw->data + offs;
+ return 0;
}
static u32 qlcnic_get_fw_version(struct qlcnic_adapter *adapter)
{
- struct uni_data_desc *fw_data_desc;
+ char ver_str[QLCNIC_UNI_VERSION_TAIL_SIZE + 1];
const struct firmware *fw = adapter->fw;
+ struct qlcnic_uni_data data;
u32 major, minor, sub;
- __le32 version_offset;
- const u8 *ver_str;
int i, ret;
- if (adapter->ahw->fw_type != QLCNIC_UNIFIED_ROMIMAGE) {
- version_offset = *(__le32 *)&fw->data[QLCNIC_FW_VERSION_OFFSET];
- return le32_to_cpu(version_offset);
- }
+ if (adapter->ahw->fw_type != QLCNIC_UNIFIED_ROMIMAGE)
+ return get_unaligned_le32(fw->data + QLCNIC_FW_VERSION_OFFSET);
+
+ ret = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
+ QLCNIC_UNI_FIRMWARE_IDX_OFF, &data);
+ if (ret || data.size < QLCNIC_UNI_VERSION_TAIL_SIZE)
+ return 0;
- fw_data_desc = qlcnic_get_data_desc(adapter, QLCNIC_UNI_DIR_SECT_FW,
- QLCNIC_UNI_FIRMWARE_IDX_OFF);
- ver_str = fw->data + le32_to_cpu(fw_data_desc->findex) +
- le32_to_cpu(fw_data_desc->size) - 17;
+ memcpy(ver_str, fw->data + data.offset + data.size -
+ QLCNIC_UNI_VERSION_TAIL_SIZE,
+ QLCNIC_UNI_VERSION_TAIL_SIZE);
+ ver_str[QLCNIC_UNI_VERSION_TAIL_SIZE] = '\0';
for (i = 0; i < 12; i++) {
- if (!strncmp(&ver_str[i], "REV=", 4)) {
+ if (!strncmp(ver_str + i, "REV=", 4)) {
ret = sscanf(&ver_str[i+4], "%u.%u.%u ",
&major, &minor, &sub);
if (ret != 3)
@@ -1034,18 +1076,15 @@ static u32 qlcnic_get_fw_version(struct qlcnic_adapter *adapter)
static u32 qlcnic_get_bios_version(struct qlcnic_adapter *adapter)
{
- const struct firmware *fw = adapter->fw;
u32 bios_ver, prd_off = adapter->file_prd_off;
- u8 *version_offset;
- __le32 temp;
+ const struct firmware *fw = adapter->fw;
- if (adapter->ahw->fw_type != QLCNIC_UNIFIED_ROMIMAGE) {
- version_offset = (u8 *)&fw->data[QLCNIC_BIOS_VERSION_OFFSET];
- return le32_to_cpu(*(__le32 *)version_offset);
- }
+ if (adapter->ahw->fw_type != QLCNIC_UNIFIED_ROMIMAGE)
+ return get_unaligned_le32(fw->data + QLCNIC_BIOS_VERSION_OFFSET);
- temp = *((__le32 *)(&fw->data[prd_off]) + QLCNIC_UNI_BIOS_VERSION_OFF);
- bios_ver = le32_to_cpu(temp);
+ bios_ver = get_unaligned_le32(fw->data + prd_off +
+ QLCNIC_UNI_BIOS_VERSION_OFF *
+ sizeof(__le32));
return (bios_ver << 16) + ((bios_ver >> 8) & 0xff00) + (bios_ver >> 24);
}
@@ -1106,24 +1145,26 @@ static const char *fw_name[] = {
int
qlcnic_load_firmware(struct qlcnic_adapter *adapter)
{
- __le64 *ptr64;
- u32 i, flashaddr, size;
const struct firmware *fw = adapter->fw;
struct pci_dev *pdev = adapter->pdev;
+ const u8 *bootld, *image;
+ u32 i, flashaddr, image_size;
+ int ret;
dev_info(&pdev->dev, "loading firmware from %s\n",
fw_name[adapter->ahw->fw_type]);
if (fw) {
+ u32 words, remainder;
u64 data;
- size = (QLCNIC_IMAGE_START - QLCNIC_BOOTLD_START) / 8;
-
- ptr64 = (__le64 *)qlcnic_get_bootld_offs(adapter);
+ ret = qlcnic_get_bootld_data(adapter, &bootld);
+ if (ret)
+ return ret;
flashaddr = QLCNIC_BOOTLD_START;
- for (i = 0; i < size; i++) {
- data = le64_to_cpu(ptr64[i]);
+ for (i = 0; i < QLCNIC_UNI_BOOTLD_SIZE / sizeof(u64); i++) {
+ data = get_unaligned_le64(bootld + i * sizeof(u64));
if (qlcnic_pci_mem_write_2M(adapter, flashaddr, data))
return -EIO;
@@ -1131,13 +1172,15 @@ qlcnic_load_firmware(struct qlcnic_adapter *adapter)
flashaddr += 8;
}
- size = qlcnic_get_fw_size(adapter) / 8;
-
- ptr64 = (__le64 *)qlcnic_get_fw_offs(adapter);
+ ret = qlcnic_get_fw_data(adapter, &image, &image_size);
+ if (ret)
+ return ret;
+ words = image_size / sizeof(u64);
+ remainder = image_size % sizeof(u64);
flashaddr = QLCNIC_IMAGE_START;
- for (i = 0; i < size; i++) {
- data = le64_to_cpu(ptr64[i]);
+ for (i = 0; i < words; i++) {
+ data = get_unaligned_le64(image + i * sizeof(u64));
if (qlcnic_pci_mem_write_2M(adapter,
flashaddr, data))
@@ -1146,9 +1189,11 @@ qlcnic_load_firmware(struct qlcnic_adapter *adapter)
flashaddr += 8;
}
- size = qlcnic_get_fw_size(adapter) % 8;
- if (size) {
- data = le64_to_cpu(ptr64[i]);
+ if (remainder) {
+ __le64 tail = 0;
+
+ memcpy(&tail, image + words * sizeof(u64), remainder);
+ data = le64_to_cpu(tail);
if (qlcnic_pci_mem_write_2M(adapter,
flashaddr, data))
@@ -1156,10 +1201,9 @@ qlcnic_load_firmware(struct qlcnic_adapter *adapter)
}
} else {
- u64 data;
- u32 hi, lo;
- int ret;
struct qlcnic_flt_entry bootld_entry;
+ u32 hi, lo, size;
+ u64 data;
ret = qlcnic_get_flt_entry(adapter, QLCNIC_BOOTLD_REGION,
&bootld_entry);
@@ -1200,26 +1244,33 @@ qlcnic_load_firmware(struct qlcnic_adapter *adapter)
static int
qlcnic_validate_firmware(struct qlcnic_adapter *adapter)
{
- u32 val;
- u32 ver, bios, min_size;
- struct pci_dev *pdev = adapter->pdev;
const struct firmware *fw = adapter->fw;
+ struct pci_dev *pdev = adapter->pdev;
u8 fw_type = adapter->ahw->fw_type;
+ u32 ver, bios, min_size;
+ const u8 *data;
+ u32 data_size;
+ u32 val;
+
+ if (fw_type == QLCNIC_UNIFIED_ROMIMAGE)
+ min_size = QLCNIC_UNI_FW_MIN_SIZE;
+ else
+ min_size = QLCNIC_FW_MIN_SIZE;
+
+ if (fw->size < min_size)
+ return -EINVAL;
if (fw_type == QLCNIC_UNIFIED_ROMIMAGE) {
if (qlcnic_validate_unified_romimage(adapter))
return -EINVAL;
-
- min_size = QLCNIC_UNI_FW_MIN_SIZE;
} else {
- val = le32_to_cpu(*(__le32 *)&fw->data[QLCNIC_FW_MAGIC_OFFSET]);
+ val = get_unaligned_le32(fw->data + QLCNIC_FW_MAGIC_OFFSET);
if (val != QLCNIC_BDINFO_MAGIC)
return -EINVAL;
-
- min_size = QLCNIC_FW_MIN_SIZE;
}
- if (fw->size < min_size)
+ if (qlcnic_get_bootld_data(adapter, &data) ||
+ qlcnic_get_fw_data(adapter, &data, &data_size))
return -EINVAL;
val = qlcnic_get_fw_version(adapter);
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
index 8471157a431b..615bf82eed51 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c
@@ -394,6 +394,7 @@ struct sk_buff *rmnet_map_deaggregate(struct sk_buff *skb,
if (!skbn)
return NULL;
+ skbn->dev = skb->dev;
skb_reserve(skbn, RMNET_MAP_DEAGGR_HEADROOM);
skb_put(skbn, packet_len);
memcpy(skbn->data, skb->data, packet_len);
diff --git a/drivers/net/ethernet/realtek/8139cp.c b/drivers/net/ethernet/realtek/8139cp.c
index 5652da8a178c..9016527e229a 100644
--- a/drivers/net/ethernet/realtek/8139cp.c
+++ b/drivers/net/ethernet/realtek/8139cp.c
@@ -2066,7 +2066,7 @@ static int __maybe_unused cp_suspend(struct device *device)
/* Disable Rx and Tx */
cpw16 (IntrMask, 0);
- cpw8 (Cmd, cpr8 (Cmd) & (~RxOn | ~TxOn));
+ cpw8 (Cmd, cpr8 (Cmd) & ~(RxOn | TxOn));
spin_unlock_irqrestore (&cp->lock, flags);
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
index d413d76a8936..83fbd8fa759e 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
@@ -185,7 +185,7 @@ static void dwmac1000_set_filter(struct mac_device_info *hw,
dwmac1000_set_mchash(ioaddr, mc_filter, mcbitslog2);
/* Handle multiple unicast addresses (perfect filtering) */
- if (netdev_uc_count(dev) > perfect_addr_number)
+ if (netdev_uc_count(dev) + 1 > perfect_addr_number)
/* Switch to promiscuous mode if more than unicast
* addresses are requested than supported by hardware.
*/
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
index e65a65666cc1..8ce5281634fe 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
@@ -680,7 +680,7 @@ static void dwmac4_set_filter(struct mac_device_info *hw,
value |= GMAC_PACKET_FILTER_HPF;
/* Handle multiple unicast addresses */
- if (netdev_uc_count(dev) > hw->unicast_filter_entries) {
+ if (netdev_uc_count(dev) + 1 > hw->unicast_filter_entries) {
/* Switch to promiscuous mode if more than 128 addrs
* are required
*/
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
index 445259f2ee93..d1a6cc7bd9bb 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
@@ -540,7 +540,7 @@ static void dwxgmac2_set_filter(struct mac_device_info *hw,
dwxgmac2_set_mchash(ioaddr, mc_filter, mcbitslog2);
/* Handle multiple unicast addresses */
- if (netdev_uc_count(dev) > hw->unicast_filter_entries) {
+ if (netdev_uc_count(dev) + 1 > hw->unicast_filter_entries) {
value |= XGMAC_FILTER_PR;
} else {
struct netdev_hw_addr *ha;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 44fd0fadc137..3074e322a2cc 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -1183,6 +1183,10 @@ static int stmmac_init_phy(struct net_device *dev)
struct phy_device *phydev;
if (addr < 0) {
+ /* If a custom PCS is in use, no PHY is needed */
+ if (priv->hw->phylink_pcs)
+ return 0;
+
netdev_err(priv->dev, "no phy found\n");
return -ENODEV;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
index 3ca1c2a816ff..266211a40139 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_selftests.c
@@ -415,11 +415,17 @@ static int stmmac_test_mmc(struct stmmac_priv *priv)
stmmac_mmc_read(priv, priv->mmcaddr, &final);
/*
- * The number of MMC counters available depends on HW configuration
- * so we just use this one to validate the feature. I hope there is
- * not a version without this counter.
+ * The number of MMC counters available depends on HW configuration,
+ * and there doesn't seem to be a way to enumerate the implemented
+ * counters.
+ *
+ * Let's check a hand-picked set of counters, knowing that :
+ * - Starfive JH7110 doesn't implement mmc_tx_framecount_g
+ * - Amlogic SM1 doesn't implement any mmc_tx_*
+ *
*/
- if (final.mmc_tx_framecount_g <= initial.mmc_tx_framecount_g)
+ if (final.mmc_tx_framecount_g <= initial.mmc_tx_framecount_g &&
+ final.mmc_rx_framecount_gb <= initial.mmc_rx_framecount_gb)
return -EINVAL;
return 0;
@@ -493,6 +499,21 @@ static int stmmac_filter_check(struct stmmac_priv *priv)
return -EOPNOTSUPP;
}
+static int stmmac_uc_filter_check(struct stmmac_priv *priv)
+{
+ /* For tests involving the UC filter, we need at least one empty
+ * slot in the UC filter. The UC filters contains netdev_uc_count() + 1
+ * entries: The dev->uc list + one entry for the HW address.
+ *
+ * Having an empty slot therefore means netdev_uc_count() + 2 entries
+ * can fit in the filter
+ */
+ if (netdev_uc_count(priv->dev) + 2 > priv->hw->unicast_filter_entries)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
static bool stmmac_hash_check(struct stmmac_priv *priv, unsigned char *addr)
{
int mc_offset = 32 - priv->hw->mcast_bits_log2;
@@ -584,7 +605,7 @@ static int stmmac_test_pfilt(struct stmmac_priv *priv)
if (stmmac_filter_check(priv))
return -EOPNOTSUPP;
- if (netdev_uc_count(priv->dev) >= priv->hw->unicast_filter_entries)
+ if (stmmac_uc_filter_check(priv))
return -EOPNOTSUPP;
while (--tries) {
@@ -628,7 +649,7 @@ static int stmmac_test_mcfilt(struct stmmac_priv *priv)
if (stmmac_filter_check(priv))
return -EOPNOTSUPP;
- if (netdev_uc_count(priv->dev) >= priv->hw->unicast_filter_entries)
+ if (stmmac_uc_filter_check(priv))
return -EOPNOTSUPP;
if (netdev_mc_count(priv->dev) >= priv->hw->multicast_filter_bins)
return -EOPNOTSUPP;
@@ -674,7 +695,7 @@ static int stmmac_test_ucfilt(struct stmmac_priv *priv)
if (stmmac_filter_check(priv))
return -EOPNOTSUPP;
- if (netdev_uc_count(priv->dev) >= priv->hw->unicast_filter_entries)
+ if (stmmac_uc_filter_check(priv))
return -EOPNOTSUPP;
if (netdev_mc_count(priv->dev) >= priv->hw->multicast_filter_bins)
return -EOPNOTSUPP;
@@ -1452,11 +1473,11 @@ static int __stmmac_test_l4filt(struct stmmac_priv *priv, u32 dst, u32 src,
struct {
struct flow_dissector_key_basic bkey;
struct flow_dissector_key_ports key;
- } __aligned(BITS_PER_LONG / 8) keys;
+ } __aligned(BITS_PER_LONG / 8) keys = { };
struct {
struct flow_dissector_key_basic bmask;
struct flow_dissector_key_ports mask;
- } __aligned(BITS_PER_LONG / 8) masks;
+ } __aligned(BITS_PER_LONG / 8) masks = { };
unsigned long dummy_cookie = 0xdeadbeef;
struct stmmac_packet_attrs attr = { };
struct flow_dissector *dissector;
@@ -1509,6 +1530,8 @@ static int __stmmac_test_l4filt(struct stmmac_priv *priv, u32 dst, u32 src,
keys.bkey.ip_proto = udp ? IPPROTO_UDP : IPPROTO_TCP;
keys.key.src = htons(src);
keys.key.dst = htons(dst);
+ /* Match the full IP proto field */
+ masks.bmask.ip_proto = 0xff;
masks.mask.src = src_mask;
masks.mask.dst = dst_mask;
diff --git a/drivers/net/ethernet/wangxun/libwx/wx_type.h b/drivers/net/ethernet/wangxun/libwx/wx_type.h
index 627dcb0151c7..96839972d1f6 100644
--- a/drivers/net/ethernet/wangxun/libwx/wx_type.h
+++ b/drivers/net/ethernet/wangxun/libwx/wx_type.h
@@ -1134,7 +1134,7 @@ struct wx {
};
#define WX_INTR_ALL (~0ULL)
-#define WX_INTR_Q(i) BIT((i))
+#define WX_INTR_Q(i) BIT_ULL((i))
/* register operations */
#define wr32(a, reg, value) writel((value), ((a)->hw_addr + (reg)))
diff --git a/drivers/net/ethernet/wangxun/txgbe/txgbe_type.h b/drivers/net/ethernet/wangxun/txgbe/txgbe_type.h
index 27f8db89a5c7..c8981c27340c 100644
--- a/drivers/net/ethernet/wangxun/txgbe/txgbe_type.h
+++ b/drivers/net/ethernet/wangxun/txgbe/txgbe_type.h
@@ -264,7 +264,7 @@ struct txgbe_fdir_filter {
#define TXGBE_DEFAULT_RX_WORK 128
#endif
-#define TXGBE_INTR_MISC(A) BIT((A)->num_q_vectors)
+#define TXGBE_INTR_MISC(A) BIT_ULL((A)->num_q_vectors)
#define TXGBE_INTR_QALL(A) (TXGBE_INTR_MISC(A) - 1)
#define TXGBE_MAX_EITR GENMASK(11, 3)
diff --git a/drivers/net/gtp.c b/drivers/net/gtp.c
index 023a791f39d9..2437b77e9227 100644
--- a/drivers/net/gtp.c
+++ b/drivers/net/gtp.c
@@ -1544,6 +1544,8 @@ static int gtp_newlink(struct net *src_net, struct net_device *dev,
out_encap:
gtp_encap_disable(gtp);
out_hashtable:
+ /* Wait for RCU readers that may still reference this gtp_dev. */
+ synchronize_net();
kfree(gtp->addr_hash);
kfree(gtp->tid_hash);
return err;
diff --git a/drivers/net/ipa/ipa_main.c b/drivers/net/ipa/ipa_main.c
index f374a590d1c5..aa48ee2c9157 100644
--- a/drivers/net/ipa/ipa_main.c
+++ b/drivers/net/ipa/ipa_main.c
@@ -974,12 +974,12 @@ static void ipa_remove(struct platform_device *pdev)
}
if (ret) {
/*
- * Not cleaning up here properly might also yield a
- * crash later on. As the device is still unregistered
- * in this case, this might even yield a crash later on.
+ * Continuing teardown after failing to stop the modem
+ * could crash, so leave the remaining resources allocated.
*/
dev_err(dev, "Failed to stop modem (%pe), leaking resources\n",
ERR_PTR(ret));
+ pm_runtime_put_noidle(dev);
return;
}
diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c
index ec5c6be87446..bf11812438aa 100644
--- a/drivers/net/ntb_netdev.c
+++ b/drivers/net/ntb_netdev.c
@@ -164,8 +164,10 @@ static int __ntb_netdev_maybe_stop_tx(struct net_device *netdev,
static int ntb_netdev_maybe_stop_tx(struct net_device *ndev,
struct ntb_transport_qp *qp, int size)
{
- if (netif_queue_stopped(ndev) ||
- (ntb_transport_tx_free_entry(qp) >= size))
+ if (netif_queue_stopped(ndev))
+ return -EBUSY;
+
+ if (ntb_transport_tx_free_entry(qp) >= size)
return 0;
return __ntb_netdev_maybe_stop_tx(ndev, qp, size);
@@ -208,21 +210,30 @@ static netdev_tx_t ntb_netdev_start_xmit(struct sk_buff *skb,
struct ntb_netdev *dev = netdev_priv(ndev);
int rc;
- ntb_netdev_maybe_stop_tx(ndev, dev->qp, tx_stop);
+ if (unlikely(ntb_netdev_maybe_stop_tx(ndev, dev->qp, tx_stop)))
+ return NETDEV_TX_BUSY;
rc = ntb_transport_tx_enqueue(dev->qp, skb, skb->data, skb->len);
- if (rc)
- goto err;
+ if (rc) {
+ if (rc == -EAGAIN || rc == -EBUSY) {
+ netif_stop_queue(ndev);
+ mod_timer(&dev->tx_timer,
+ jiffies + usecs_to_jiffies(tx_time));
+ return NETDEV_TX_BUSY;
+ }
+
+ goto drop;
+ }
/* check for next submit */
ntb_netdev_maybe_stop_tx(ndev, dev->qp, tx_stop);
return NETDEV_TX_OK;
-err:
+drop:
+ dev_kfree_skb_any(skb);
ndev->stats.tx_dropped++;
- ndev->stats.tx_errors++;
- return NETDEV_TX_BUSY;
+ return NETDEV_TX_OK;
}
static void ntb_netdev_tx_timer(struct timer_list *t)
diff --git a/drivers/net/ppp/pppox.c b/drivers/net/ppp/pppox.c
index 08364f10a43f..61b3453469dc 100644
--- a/drivers/net/ppp/pppox.c
+++ b/drivers/net/ppp/pppox.c
@@ -74,7 +74,9 @@ int pppox_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
switch (cmd) {
case PPPIOCGCHAN: {
+ struct sk_buff *skb;
int index;
+
rc = -ENOTCONN;
if (!(sk->sk_state & PPPOX_CONNECTED))
break;
@@ -85,7 +87,22 @@ int pppox_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg)
break;
rc = 0;
+ /* PPPIOCGCHAN historically marks the userspace handoff to
+ * generic PPP; pppd then attaches the returned channel to
+ * /dev/ppp.
+ */
sk->sk_state |= PPPOX_BOUND;
+ /* Let lockless receive paths finish queueing against the old
+ * state.
+ */
+ synchronize_net();
+ /* Drain packets queued before the handoff because a bound
+ * socket is no longer readable.
+ */
+ while ((skb = skb_dequeue(&sk->sk_receive_queue))) {
+ skb_orphan(skb);
+ ppp_input(&po->chan, skb);
+ }
break;
}
default:
diff --git a/drivers/net/slip/slip.c b/drivers/net/slip/slip.c
index 117de9eb5e09..bc155dddae8b 100644
--- a/drivers/net/slip/slip.c
+++ b/drivers/net/slip/slip.c
@@ -886,8 +886,6 @@ static int slip_open(struct tty_struct *tty)
* Close down a SLIP channel.
* This means flushing out any pending queues, and then returning. This
* call is serialized against other ldisc functions.
- *
- * We also use this method fo a hangup event
*/
static void slip_close(struct tty_struct *tty)
@@ -915,11 +913,6 @@ static void slip_close(struct tty_struct *tty)
unregister_netdev(sl->dev);
/* sl_uninit() has dropped the slip_devs[] entry by now */
}
-
-static void slip_hangup(struct tty_struct *tty)
-{
- slip_close(tty);
-}
/************************************************************************
* STANDARD SLIP ENCAPSULATION *
************************************************************************/
@@ -1280,7 +1273,6 @@ static struct tty_ldisc_ops sl_ldisc = {
.name = "slip",
.open = slip_open,
.close = slip_close,
- .hangup = slip_hangup,
.ioctl = slip_ioctl,
.receive_buf = slip_receive_buf,
.write_wakeup = slip_write_wakeup,
diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index 7bb9aa20fab9..096f3ff36a43 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -907,9 +907,11 @@ static int tbnet_poll(struct napi_struct *napi, int budget)
le32_to_cpu(net->rx_hdr.frame_count) - 1;
rx_packets++;
- net->stats.rx_bytes += frame_size;
if (last) {
+ /* Before eth_type_trans() pulls the Ethernet header. */
+ net->stats.rx_packets++;
+ net->stats.rx_bytes += skb->len;
skb->protocol = eth_type_trans(skb, net->dev);
trace_tbnet_rx_skb(skb);
napi_gro_receive(&net->napi, skb);
@@ -917,8 +919,6 @@ static int tbnet_poll(struct napi_struct *napi, int budget)
}
}
- net->stats.rx_packets += rx_packets;
-
if (cleaned_count)
tbnet_alloc_rx_buffers(net, cleaned_count);
diff --git a/drivers/net/vxlan/vxlan_mdb.c b/drivers/net/vxlan/vxlan_mdb.c
index e15a8536dae9..166e590da160 100644
--- a/drivers/net/vxlan/vxlan_mdb.c
+++ b/drivers/net/vxlan/vxlan_mdb.c
@@ -1428,14 +1428,17 @@ static void vxlan_mdb_flush(struct vxlan_dev *vxlan,
struct vxlan_mdb_entry *mdb_entry;
struct hlist_node *tmp;
- /* The removal of an entry cannot trigger the removal of another entry
- * since entries are always added to the head of the list.
- */
hlist_for_each_entry_safe(mdb_entry, tmp, &vxlan->mdb_list, mdb_node) {
if (desc->src_vni && desc->src_vni != mdb_entry->key.vni)
continue;
vxlan_mdb_remotes_flush(vxlan, mdb_entry, desc);
+ /* The flush can remove the (S, G) entries created for the
+ * source list of this entry, including the one saved by
+ * hlist_for_each_entry_safe(), so re-read it while this entry
+ * is still linked.
+ */
+ tmp = mdb_entry->mdb_node.next;
/* Entry will only be removed if its remotes list is empty. */
vxlan_mdb_entry_put(vxlan, mdb_entry);
}
diff --git a/drivers/net/vxlan/vxlan_vnifilter.c b/drivers/net/vxlan/vxlan_vnifilter.c
index 272fa31ef074..b5854aceddf9 100644
--- a/drivers/net/vxlan/vxlan_vnifilter.c
+++ b/drivers/net/vxlan/vxlan_vnifilter.c
@@ -467,10 +467,8 @@ static int vxlan_vnifilter_dump(struct sk_buff *skb, struct netlink_callback *cb
static const struct nla_policy vni_filter_entry_policy[VXLAN_VNIFILTER_ENTRY_MAX + 1] = {
[VXLAN_VNIFILTER_ENTRY_START] = { .type = NLA_U32 },
[VXLAN_VNIFILTER_ENTRY_END] = { .type = NLA_U32 },
- [VXLAN_VNIFILTER_ENTRY_GROUP] = { .type = NLA_BINARY,
- .len = sizeof_field(struct iphdr, daddr) },
- [VXLAN_VNIFILTER_ENTRY_GROUP6] = { .type = NLA_BINARY,
- .len = sizeof(struct in6_addr) },
+ [VXLAN_VNIFILTER_ENTRY_GROUP] = NLA_POLICY_EXACT_LEN(sizeof_field(struct iphdr, daddr)),
+ [VXLAN_VNIFILTER_ENTRY_GROUP6] = NLA_POLICY_EXACT_LEN(sizeof(struct in6_addr)),
};
static const struct nla_policy vni_filter_policy[VXLAN_VNIFILTER_MAX + 1] = {
diff --git a/drivers/net/wireless/ath/ath10k/snoc.c b/drivers/net/wireless/ath/ath10k/snoc.c
index 59f7ccb33fde..ad215de068e1 100644
--- a/drivers/net/wireless/ath/ath10k/snoc.c
+++ b/drivers/net/wireless/ath/ath10k/snoc.c
@@ -5,6 +5,7 @@
#include <linux/bits.h>
#include <linux/clk.h>
+#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
@@ -1455,11 +1456,15 @@ static void ath10k_msa_dump_memory(struct ath10k *ar,
hdr->length = cpu_to_le32(ar->msa.mem_size);
if (current_region->len < ar->msa.mem_size) {
- memcpy(buf, ar->msa.vaddr, current_region->len);
+ memcpy_fromio(buf,
+ (const void __iomem __force *)ar->msa.vaddr,
+ current_region->len);
ath10k_warn(ar, "msa dump length is less than msa size %x, %x\n",
current_region->len, ar->msa.mem_size);
} else {
- memcpy(buf, ar->msa.vaddr, ar->msa.mem_size);
+ memcpy_fromio(buf,
+ (const void __iomem __force *)ar->msa.vaddr,
+ ar->msa.mem_size);
}
}
diff --git a/drivers/net/wireless/ath/ath11k/dp_rx.c b/drivers/net/wireless/ath/ath11k/dp_rx.c
index 6cc252d00ef1..4935cfdc9adc 100644
--- a/drivers/net/wireless/ath/ath11k/dp_rx.c
+++ b/drivers/net/wireless/ath/ath11k/dp_rx.c
@@ -2332,10 +2332,10 @@ static void ath11k_dp_rx_h_rate(struct ath11k *ar, struct hal_rx_desc *rx_desc,
case RX_MSDU_START_PKT_TYPE_11N:
rx_status->encoding = RX_ENC_HT;
if (rate_mcs > ATH11K_HT_MCS_MAX) {
- ath11k_warn(ar->ab,
- "Received with invalid mcs in HT mode %d\n",
- rate_mcs);
- break;
+ ath11k_dbg(ar->ab, ATH11K_DBG_DP_RX,
+ "Received HT frame with out-of-range mcs %d, capping to %d\n",
+ rate_mcs, ATH11K_HT_MCS_MAX);
+ rate_mcs = ATH11K_HT_MCS_MAX;
}
rx_status->rate_idx = rate_mcs + (8 * (nss - 1));
if (sgi)
@@ -2344,13 +2344,13 @@ static void ath11k_dp_rx_h_rate(struct ath11k *ar, struct hal_rx_desc *rx_desc,
break;
case RX_MSDU_START_PKT_TYPE_11AC:
rx_status->encoding = RX_ENC_VHT;
- rx_status->rate_idx = rate_mcs;
if (rate_mcs > ATH11K_VHT_MCS_MAX) {
- ath11k_warn(ar->ab,
- "Received with invalid mcs in VHT mode %d\n",
- rate_mcs);
- break;
+ ath11k_dbg(ar->ab, ATH11K_DBG_DP_RX,
+ "Received VHT frame with out-of-range mcs %d, capping to %d\n",
+ rate_mcs, ATH11K_VHT_MCS_MAX);
+ rate_mcs = ATH11K_VHT_MCS_MAX;
}
+ rx_status->rate_idx = rate_mcs;
rx_status->nss = nss;
if (sgi)
rx_status->enc_flags |= RX_ENC_FLAG_SHORT_GI;
@@ -2360,14 +2360,14 @@ static void ath11k_dp_rx_h_rate(struct ath11k *ar, struct hal_rx_desc *rx_desc,
rx_status->enc_flags |= RX_ENC_FLAG_LDPC;
break;
case RX_MSDU_START_PKT_TYPE_11AX:
- rx_status->rate_idx = rate_mcs;
+ rx_status->encoding = RX_ENC_HE;
if (rate_mcs > ATH11K_HE_MCS_MAX) {
- ath11k_warn(ar->ab,
- "Received with invalid mcs in HE mode %d\n",
- rate_mcs);
- break;
+ ath11k_dbg(ar->ab, ATH11K_DBG_DP_RX,
+ "Received HE frame with out-of-range mcs %d, capping to %d\n",
+ rate_mcs, ATH11K_HE_MCS_MAX);
+ rate_mcs = ATH11K_HE_MCS_MAX;
}
- rx_status->encoding = RX_ENC_HE;
+ rx_status->rate_idx = rate_mcs;
rx_status->nss = nss;
rx_status->he_gi = ath11k_mac_he_gi_to_nl80211_he_gi(sgi);
rx_status->bw = ath11k_mac_bw_to_mac80211_bw(bw);
diff --git a/drivers/net/wireless/ath/ath11k/wmi.c b/drivers/net/wireless/ath/ath11k/wmi.c
index 5f15f7acd513..a184a7e3452c 100644
--- a/drivers/net/wireless/ath/ath11k/wmi.c
+++ b/drivers/net/wireless/ath/ath11k/wmi.c
@@ -159,6 +159,8 @@ static const struct wmi_tlv_policy wmi_tlv_policies[] = {
.min_len = sizeof(struct ath11k_wmi_p2p_noa_info) },
[WMI_TAG_P2P_NOA_EVENT] = {
.min_len = sizeof(struct wmi_p2p_noa_event) },
+ [WMI_TAG_PDEV_CSA_SWITCH_COUNT_STATUS_EVENT] = {
+ .min_len = sizeof(struct wmi_pdev_csa_switch_ev) },
};
#define PRIMAP(_hw_mode_) \
@@ -262,6 +264,13 @@ const void **ath11k_wmi_tlv_parse_alloc(struct ath11k_base *ab,
return tb;
}
+static u32 ath11k_wmi_tlv_data_len(const void *data)
+{
+ const struct wmi_tlv *tlv = (const struct wmi_tlv *)data - 1;
+
+ return FIELD_GET(WMI_TLV_LEN, tlv->header);
+}
+
static int ath11k_wmi_cmd_send_nowait(struct ath11k_pdev_wmi *wmi, struct sk_buff *skb,
u32 cmd_id)
{
@@ -2411,8 +2420,8 @@ int ath11k_wmi_send_scan_start_cmd(struct ath11k *ar,
for (i = 0; i < params->num_hint_bssid; ++i) {
hint_bssid->freq_flags =
params->hint_bssid[i].freq_flags;
- ether_addr_copy(¶ms->hint_bssid[i].bssid.addr[0],
- &hint_bssid->bssid.addr[0]);
+ ether_addr_copy(&hint_bssid->bssid.addr[0],
+ ¶ms->hint_bssid[i].bssid.addr[0]);
hint_bssid++;
}
}
@@ -4747,14 +4756,16 @@ static int ath11k_wmi_tlv_mac_phy_caps_parse(struct ath11k_base *soc,
if (svc_rdy_ext->n_mac_phy_caps >= svc_rdy_ext->tot_phy_id)
return -ENOBUFS;
- len = min_t(u16, len, sizeof(struct wmi_mac_phy_capabilities));
if (!svc_rdy_ext->n_mac_phy_caps) {
- svc_rdy_ext->mac_phy_caps = kcalloc(svc_rdy_ext->tot_phy_id,
- len, GFP_ATOMIC);
+ svc_rdy_ext->mac_phy_caps =
+ kzalloc_objs(*svc_rdy_ext->mac_phy_caps,
+ svc_rdy_ext->tot_phy_id,
+ GFP_ATOMIC);
if (!svc_rdy_ext->mac_phy_caps)
return -ENOMEM;
}
+ len = min_t(u16, len, sizeof(struct wmi_mac_phy_capabilities));
memcpy(svc_rdy_ext->mac_phy_caps + svc_rdy_ext->n_mac_phy_caps, ptr, len);
svc_rdy_ext->n_mac_phy_caps++;
return 0;
@@ -5066,6 +5077,7 @@ static int ath11k_service_ready_ext_event(struct ath11k_base *ab,
return 0;
err:
+ kfree(svc_rdy_ext.mac_phy_caps);
ath11k_wmi_free_dbring_caps(ab);
return ret;
}
@@ -8302,15 +8314,23 @@ ath11k_wmi_process_csa_switch_count_event(struct ath11k_base *ab,
const struct wmi_pdev_csa_switch_ev *ev,
const u32 *vdev_ids)
{
- int i;
+ u32 vdev_ids_len = ath11k_wmi_tlv_data_len(vdev_ids);
+ u32 num_vdevs = ev->num_vdevs;
struct ath11k_vif *arvif;
+ int i;
/* Finish CSA once the switch count becomes NULL */
if (ev->current_switch_count)
return;
+ if (num_vdevs > vdev_ids_len / sizeof(*vdev_ids)) {
+ ath11k_warn(ab, "csa switch count num_vdevs %u exceeds tlv array length %u\n",
+ num_vdevs, vdev_ids_len);
+ return;
+ }
+
rcu_read_lock();
- for (i = 0; i < ev->num_vdevs; i++) {
+ for (i = 0; i < num_vdevs; i++) {
arvif = ath11k_mac_get_arvif_by_vdev_id(ab, vdev_ids[i]);
if (!arvif) {
@@ -8757,13 +8777,15 @@ static void ath11k_wmi_tlv_op_rx(struct ath11k_base *ab, struct sk_buff *skb)
struct wmi_cmd_hdr *cmd_hdr;
enum wmi_tlv_event_id id;
+ if (skb->len < sizeof(*cmd_hdr))
+ goto out;
+
cmd_hdr = (struct wmi_cmd_hdr *)skb->data;
id = FIELD_GET(WMI_CMD_HDR_CMD_ID, (cmd_hdr->cmd_id));
trace_ath11k_wmi_event(ab, id, skb->data, skb->len);
- if (skb_pull(skb, sizeof(struct wmi_cmd_hdr)) == NULL)
- goto out;
+ skb_pull(skb, sizeof(*cmd_hdr));
switch (id) {
/* Process all the WMI events here */
diff --git a/drivers/net/wireless/ath/ath12k/wmi.c b/drivers/net/wireless/ath/ath12k/wmi.c
index 761c169c992f..598a69f9e6df 100644
--- a/drivers/net/wireless/ath/ath12k/wmi.c
+++ b/drivers/net/wireless/ath/ath12k/wmi.c
@@ -2479,8 +2479,8 @@ int ath12k_wmi_send_scan_start_cmd(struct ath12k *ar,
for (i = 0; i < arg->num_hint_bssid; ++i) {
hint_bssid->freq_flags =
arg->hint_bssid[i].freq_flags;
- ether_addr_copy(&arg->hint_bssid[i].bssid.addr[0],
- &hint_bssid->bssid.addr[0]);
+ ether_addr_copy(&hint_bssid->bssid.addr[0],
+ &arg->hint_bssid[i].bssid.addr[0]);
hint_bssid++;
}
}
@@ -4073,14 +4073,16 @@ static int ath12k_wmi_mac_phy_caps_parse(struct ath12k_base *soc,
if (svc_rdy_ext->n_mac_phy_caps >= svc_rdy_ext->tot_phy_id)
return -ENOBUFS;
- len = min_t(u16, len, sizeof(struct ath12k_wmi_mac_phy_caps_params));
if (!svc_rdy_ext->n_mac_phy_caps) {
- svc_rdy_ext->mac_phy_caps = kzalloc((svc_rdy_ext->tot_phy_id) * len,
- GFP_ATOMIC);
+ svc_rdy_ext->mac_phy_caps =
+ kzalloc_objs(*svc_rdy_ext->mac_phy_caps,
+ svc_rdy_ext->tot_phy_id,
+ GFP_ATOMIC);
if (!svc_rdy_ext->mac_phy_caps)
return -ENOMEM;
}
+ len = min_t(u16, len, sizeof(struct ath12k_wmi_mac_phy_caps_params));
memcpy(svc_rdy_ext->mac_phy_caps + svc_rdy_ext->n_mac_phy_caps, ptr, len);
svc_rdy_ext->n_mac_phy_caps++;
return 0;
@@ -7196,12 +7198,12 @@ static void ath12k_wmi_op_rx(struct ath12k_base *ab, struct sk_buff *skb)
struct wmi_cmd_hdr *cmd_hdr;
enum wmi_tlv_event_id id;
- cmd_hdr = (struct wmi_cmd_hdr *)skb->data;
- id = le32_get_bits(cmd_hdr->cmd_id, WMI_CMD_HDR_CMD_ID);
-
- if (!skb_pull(skb, sizeof(struct wmi_cmd_hdr)))
+ cmd_hdr = skb_pull_data(skb, sizeof(*cmd_hdr));
+ if (!cmd_hdr)
goto out;
+ id = le32_get_bits(cmd_hdr->cmd_id, WMI_CMD_HDR_CMD_ID);
+
switch (id) {
/* Process all the WMI events here */
case WMI_SERVICE_READY_EVENTID:
diff --git a/drivers/net/wireless/ath/ath6kl/wmi.c b/drivers/net/wireless/ath/ath6kl/wmi.c
index 447896b871c3..eacaae752d8a 100644
--- a/drivers/net/wireless/ath/ath6kl/wmi.c
+++ b/drivers/net/wireless/ath/ath6kl/wmi.c
@@ -1296,6 +1296,9 @@ static int ath6kl_wmi_scan_complete_rx(struct wmi *wmi, u8 *datap, int len,
{
struct wmi_scan_complete_event *ev;
+ if (len < sizeof(*ev))
+ return -EINVAL;
+
ev = (struct wmi_scan_complete_event *) datap;
ath6kl_scan_complete_evt(vif, a_sle32_to_cpu(ev->status));
@@ -3372,7 +3375,12 @@ static int ath6kl_wmi_get_pmkid_list_event_rx(struct wmi *wmi, u8 *datap,
static int ath6kl_wmi_addba_req_event_rx(struct wmi *wmi, u8 *datap, int len,
struct ath6kl_vif *vif)
{
- struct wmi_addba_req_event *cmd = (struct wmi_addba_req_event *) datap;
+ struct wmi_addba_req_event *cmd;
+
+ if (len < sizeof(*cmd))
+ return -EINVAL;
+
+ cmd = (struct wmi_addba_req_event *)datap;
aggr_recv_addba_req_evt(vif, cmd->tid,
le16_to_cpu(cmd->st_seq_no), cmd->win_sz);
@@ -3383,7 +3391,12 @@ static int ath6kl_wmi_addba_req_event_rx(struct wmi *wmi, u8 *datap, int len,
static int ath6kl_wmi_delba_req_event_rx(struct wmi *wmi, u8 *datap, int len,
struct ath6kl_vif *vif)
{
- struct wmi_delba_event *cmd = (struct wmi_delba_event *) datap;
+ struct wmi_delba_event *cmd;
+
+ if (len < sizeof(*cmd))
+ return -EINVAL;
+
+ cmd = (struct wmi_delba_event *)datap;
aggr_recv_delba_req_evt(vif, cmd->tid);
diff --git a/drivers/net/wireless/intel/iwlwifi/fw/dump.c b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
index 8f107ceec407..0aa53212a993 100644
--- a/drivers/net/wireless/intel/iwlwifi/fw/dump.c
+++ b/drivers/net/wireless/intel/iwlwifi/fw/dump.c
@@ -488,7 +488,7 @@ static void iwl_fwrt_dump_fseq_regs(struct iwl_fw_runtime *fwrt)
void iwl_fwrt_dump_error_logs(struct iwl_fw_runtime *fwrt)
{
struct iwl_pc_data *pc_data;
- u8 count;
+ u32 count;
if (!test_bit(STATUS_DEVICE_ENABLED, &fwrt->trans->status)) {
IWL_ERR(fwrt,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
index ce787326aa69..47d5488c9504 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
/*
- * Copyright (C) 2018-2025 Intel Corporation
+ * Copyright (C) 2018-2026 Intel Corporation
*/
#include <linux/firmware.h>
#include "iwl-drv.h"
@@ -602,6 +602,9 @@ static int iwl_dbg_tlv_alloc_fragments(struct iwl_fw_runtime *fwrt,
cpu_to_le32(IWL_FW_INI_LOCATION_DRAM_PATH))
return 0;
+ if (!fw_mon_cfg->req_size)
+ return -EIO;
+
num_frags = le32_to_cpu(fw_mon_cfg->max_frags_num);
if (fwrt->trans->trans_cfg->device_family < IWL_DEVICE_FAMILY_AX210) {
if (alloc_id != IWL_FW_INI_ALLOCATION_ID_DBGC1)
@@ -612,6 +615,9 @@ static int iwl_dbg_tlv_alloc_fragments(struct iwl_fw_runtime *fwrt,
return -EIO;
}
+ if (!num_frags)
+ return -EIO;
+
remain_pages = DIV_ROUND_UP(le32_to_cpu(fw_mon_cfg->req_size),
PAGE_SIZE);
num_frags = min_t(u32, num_frags, BUF_ALLOC_MAX_NUM_FRAGS);
diff --git a/drivers/net/wireless/intel/iwlwifi/mei/main.c b/drivers/net/wireless/intel/iwlwifi/mei/main.c
index dce0b7cf7b26..1486702d37e7 100644
--- a/drivers/net/wireless/intel/iwlwifi/mei/main.c
+++ b/drivers/net/wireless/intel/iwlwifi/mei/main.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2021-2024 Intel Corporation
+ * Copyright (C) 2026 Intel Corporation
*/
#include <linux/etherdevice.h>
@@ -457,7 +458,7 @@ static int iwl_mei_send_sap_msg_payload(struct mei_cl_device *cldev,
notif_q = &dir->q_ctrl_blk[SAP_QUEUE_IDX_NOTIF];
q_head = mei->shared_mem.q_head[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_NOTIF];
q_sz = mei->shared_mem.q_size[SAP_DIRECTION_HOST_TO_ME][SAP_QUEUE_IDX_NOTIF];
- ret = iwl_mei_write_cyclic_buf(q_head, notif_q, q_head, hdr, q_sz);
+ ret = iwl_mei_write_cyclic_buf(cldev, notif_q, q_head, hdr, q_sz);
if (ret < 0)
return ret;
@@ -1147,6 +1148,11 @@ static void iwl_mei_handle_sap_rx_cmd(struct mei_cl_device *cldev,
iwl_mei_read_from_q(q_head, q_sz, &rd, wr, hdr, sizeof(*hdr));
valid_rx_sz -= sizeof(*hdr);
len = le16_to_cpu(hdr->len);
+ if (len + sizeof(*hdr) > PAGE_SIZE) {
+ dev_err(&cldev->dev,
+ "SAP message is too big: %u\n", len);
+ break;
+ }
if (valid_rx_sz < len)
break;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
index a7dbc0a5ea84..96de7edf1792 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
@@ -1010,7 +1010,7 @@ static void iwl_mvm_frob_txf_key_iter(struct ieee80211_hw *hw,
}
match++;
if (match == keylen) {
- memset(txf->buf + i - keylen, 0xAA, keylen);
+ memset(txf->buf + i + 1 - keylen, 0xAA, keylen);
match = 0;
}
}
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index 8940f8bb7bb5..e02741c5892e 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -459,8 +459,13 @@ mt76_dma_get_buf(struct mt76_dev *dev, struct mt76_queue *q, int idx,
t->ptr = NULL;
mt76_put_rxwi(dev, t);
- if (drop)
+#ifdef CONFIG_NET_MEDIATEK_SOC_WED
+ /* the WO MCU owns the RX path only on WED v2, on newer
+ * versions this buf1 bit carries no drop information
+ */
+ if (drop && dev->mmio.wed.version == 2)
*drop |= !!(buf1 & MT_DMA_CTL_WO_DROP);
+#endif
} else {
dma_sync_single_for_cpu(dev->dma_dev, e->dma_addr[0],
SKB_WITH_OVERHEAD(q->buf_size),
@@ -490,6 +495,15 @@ mt76_dma_dequeue(struct mt76_dev *dev, struct mt76_queue *q, bool flush,
q->desc[idx].ctrl |= cpu_to_le32(MT_DMA_CTL_DMA_DONE);
else if (!(q->desc[idx].ctrl & cpu_to_le32(MT_DMA_CTL_DMA_DONE)))
return NULL;
+#ifdef CONFIG_NET_MEDIATEK_SOC_WED
+ /* on WED v3 the M_DONE bit signals that WED is done reading
+ * the txfree descriptor; WED v2 does not set it
+ */
+ else if (dev->mmio.wed.version > 2 &&
+ mt76_queue_is_wed_tx_free(q) &&
+ !(q->desc[idx].ctrl & cpu_to_le32(MT_DMA_CTL_M_DONE)))
+ return NULL;
+#endif
}
q->tail = (q->tail + 1) % q->ndesc;
diff --git a/drivers/net/wireless/mediatek/mt76/dma.h b/drivers/net/wireless/mediatek/mt76/dma.h
index e3ddc7a83757..2e20d855f3ec 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.h
+++ b/drivers/net/wireless/mediatek/mt76/dma.h
@@ -11,7 +11,7 @@
#define MT_DMA_CTL_SD_LEN1 GENMASK(13, 0)
#define MT_DMA_CTL_LAST_SEC1 BIT(14)
-#define MT_DMA_CTL_BURST BIT(15)
+#define MT_DMA_CTL_M_DONE BIT(15)
#define MT_DMA_CTL_SD_LEN0 GENMASK(29, 16)
#define MT_DMA_CTL_LAST_SEC0 BIT(30)
#define MT_DMA_CTL_DMA_DONE BIT(31)
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index e538bb36a730..e1856664ca5d 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -644,6 +644,7 @@ mt76_alloc_device(struct device *pdev, unsigned int size,
dev = hw->priv;
dev->hw = hw;
dev->dev = pdev;
+ dev->init_wiphy = NULL;
dev->drv = drv_ops;
dev->dma_dev = pdev;
@@ -740,6 +741,12 @@ int mt76_register_device(struct mt76_dev *dev, bool vht,
mt76_check_sband(&dev->phy, &phy->sband_5g, NL80211_BAND_5GHZ);
mt76_check_sband(&dev->phy, &phy->sband_6g, NL80211_BAND_6GHZ);
+ if (dev->init_wiphy) {
+ ret = dev->init_wiphy(dev);
+ if (ret)
+ return ret;
+ }
+
if (IS_ENABLED(CONFIG_MT76_LEDS)) {
ret = mt76_led_init(phy);
if (ret)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index e7ebae0584f5..daa78f52c15e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -845,6 +845,9 @@ struct mt76_dev {
const struct mt76_bus_ops *bus;
const struct mt76_driver_ops *drv;
const struct mt76_mcu_ops *mcu_ops;
+
+ /* Optional callback to finalize wiphy state before registration. */
+ int (*init_wiphy)(struct mt76_dev *dev);
struct device *dev;
struct device *dma_dev;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76_connac.h b/drivers/net/wireless/mediatek/mt76/mt76_connac.h
index 445d0f0ab779..deb3c27ddc31 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76_connac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76_connac.h
@@ -177,6 +177,11 @@ static inline bool is_mt7925(struct mt76_dev *dev)
return mt76_chip(dev) == 0x7925;
}
+static inline bool is_320mhz_supported(struct mt76_dev *dev)
+{
+ return mt76_chip(dev) == 0x7927;
+}
+
static inline bool is_mt7920(struct mt76_dev *dev)
{
return mt76_chip(dev) == 0x7920;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
index d5db6ffd6d36..338f5b46c651 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.c
@@ -850,7 +850,7 @@ int mt76x02_mac_process_rx(struct mt76x02_dev *dev, struct sk_buff *skb,
}
}
- if (WARN_ON_ONCE(len > skb->len))
+ if (len > skb->len)
return -EINVAL;
if (pskb_trim(skb, len))
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
index b038568e0be9..93956d402041 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c
@@ -257,7 +257,7 @@ void mt7915_eeprom_parse_hw_cap(struct mt7915_dev *dev,
nss = min_t(u8, min_t(u8, nss_max, nss), path);
mphy->chainmask = BIT(path) - 1;
- if (band)
+ if (band && dev->dbdc_support)
mphy->chainmask <<= dev->chainshift;
mphy->antenna_mask = BIT(nss) - 1;
dev->chainmask |= mphy->chainmask;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/init.c b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
index b2220ff7d675..96871012bed3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/init.c
@@ -1253,14 +1253,19 @@ int mt7915_register_device(struct mt7915_dev *dev)
ret = mt7915_init_debugfs(&dev->phy);
if (ret)
- goto unreg_thermal;
+ goto unreg_ext_phy;
ret = mt7915_coredump_register(dev);
if (ret)
- goto unreg_thermal;
+ goto unreg_ext_phy;
return 0;
+unreg_ext_phy:
+ if (phy2) {
+ mt7915_unregister_ext_phy(dev);
+ phy2 = NULL;
+ }
unreg_thermal:
mt7915_unregister_thermal(&dev->phy);
unreg_dev:
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
index b058a38a88cf..f165e61ed28f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mac.c
@@ -444,7 +444,7 @@ mt7915_mac_fill_rx(struct mt7915_dev *dev, struct sk_buff *skb,
if (v0 & MT_PRXV_HT_AD_CODE)
status->enc_flags |= RX_ENC_FLAG_LDPC;
- status->chains = mphy->antenna_mask;
+ status->chains = mt7915_band_chainmask(phy);
status->chain_signal[0] = to_rssi(MT_PRXV_RCPI0, v1);
status->chain_signal[1] = to_rssi(MT_PRXV_RCPI1, v1);
status->chain_signal[2] = to_rssi(MT_PRXV_RCPI2, v1);
@@ -2371,8 +2371,10 @@ void mt7915_mac_add_twt_setup(struct ieee80211_hw *hw,
}
flow->tsf = le64_to_cpu(twt_agrt->twt);
- if (mt7915_mcu_twt_agrt_update(dev, msta->vif, flow, MCU_TWT_AGRT_ADD))
+ if (mt7915_mcu_twt_agrt_update(dev, msta->vif, flow, MCU_TWT_AGRT_ADD)) {
+ list_del(&flow->list);
goto unlock;
+ }
setup_cmd = TWT_SETUP_CMD_ACCEPT;
dev->twt.table_mask |= BIT(table_id);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/main.c b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
index f49dcdde3f2c..0e3ca0eb9f08 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/main.c
@@ -248,7 +248,7 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
idx = mt76_wcid_alloc(dev->mt76.wcid_mask, mt7915_wtbl_size(dev));
if (idx < 0) {
ret = -ENOSPC;
- goto out;
+ goto err;
}
INIT_LIST_HEAD(&mvif->sta.rc_list);
@@ -279,7 +279,17 @@ static int mt7915_add_interface(struct ieee80211_hw *hw,
mt7915_mcu_add_sta(dev, vif, NULL, CONN_STATE_PORT_SECURE, true);
rcu_assign_pointer(dev->mt76.wcid[idx], &mvif->sta.wcid);
+ mutex_unlock(&dev->mt76.mutex);
+
+ return 0;
+
+err:
+ dev->mt76.vif_mask &= ~BIT_ULL(mvif->mt76.idx);
+ phy->omac_mask &= ~BIT_ULL(mvif->mt76.omac_idx);
+ mt7915_mcu_add_dev_info(phy, vif, false);
out:
+ if (phy->monitor_vif == vif)
+ phy->monitor_vif = NULL;
mutex_unlock(&dev->mt76.mutex);
return ret;
@@ -296,7 +306,6 @@ static void mt7915_remove_interface(struct ieee80211_hw *hw,
mt7915_mcu_add_bss_info(phy, vif, false);
mt7915_mcu_add_sta(dev, vif, NULL, CONN_STATE_DISCONNECT, false);
- mt76_wcid_mask_clear(dev->mt76.wcid_mask, mvif->sta.wcid.idx);
mutex_lock(&dev->mt76.mutex);
mt76_testmode_reset(phy->mt76, true);
@@ -312,6 +321,7 @@ static void mt7915_remove_interface(struct ieee80211_hw *hw,
mutex_lock(&dev->mt76.mutex);
dev->mt76.vif_mask &= ~BIT_ULL(mvif->mt76.idx);
phy->omac_mask &= ~BIT_ULL(mvif->mt76.omac_idx);
+ mt76_wcid_mask_clear(dev->mt76.wcid_mask, mvif->sta.wcid.idx);
mutex_unlock(&dev->mt76.mutex);
spin_lock_bh(&dev->mt76.sta_poll_lock);
@@ -490,7 +500,7 @@ static int mt7915_config(struct ieee80211_hw *hw, u32 changed)
mt76_rmw_field(dev, MT_DMA_DCR0(band), MT_DMA_DCR0_RXD_G5_EN,
enabled);
- mt76_rmw_field(dev, MT_DMA_DCR0(band), MT_MDP_DCR0_RX_HDR_TRANS_EN,
+ mt76_rmw_field(dev, MT_MDP_DCR0, MT_MDP_DCR0_RX_HDR_TRANS_EN,
!dev->monitor_mask);
mt76_testmode_reset(phy->mt76, true);
mt76_wr(dev, MT_WF_RFCR(band), rxfilter);
@@ -1062,7 +1072,7 @@ mt7915_set_antenna(struct ieee80211_hw *hw, u32 tx_ant, u32 rx_ant)
struct mt7915_dev *dev = mt7915_hw_dev(hw);
struct mt7915_phy *phy = mt7915_hw_phy(hw);
int max_nss = hweight8(hw->wiphy->available_antennas_tx);
- u8 chainshift = dev->chainshift;
+ u8 shift = mt7915_band_chainshift(phy);
u8 band = phy->mt76->band_idx;
if (!tx_ant || tx_ant != rx_ant || ffs(tx_ant) > max_nss)
@@ -1075,9 +1085,9 @@ mt7915_set_antenna(struct ieee80211_hw *hw, u32 tx_ant, u32 rx_ant)
/* handle a variant of mt7916/mt7981 which has 3T3R but nss2 on 5 GHz band */
if ((is_mt7916(&dev->mt76) || is_mt7981(&dev->mt76)) &&
band && hweight8(tx_ant) == max_nss)
- phy->mt76->chainmask = (dev->chainmask >> chainshift) << chainshift;
+ phy->mt76->chainmask = (dev->chainmask >> shift) << shift;
else
- phy->mt76->chainmask = tx_ant << (chainshift * band);
+ phy->mt76->chainmask = tx_ant << shift;
mt76_set_stream_caps(phy->mt76, true);
mt7915_set_stream_vht_txbf_caps(phy);
@@ -1676,7 +1686,7 @@ mt7915_net_fill_forward_path(struct ieee80211_hw *hw,
path->mtk_wdma.wdma_idx = wed->wdma_idx;
path->mtk_wdma.bss = mvif->mt76.idx;
path->mtk_wdma.wcid = is_mt7915(&dev->mt76) ? msta->wcid.idx : 0x3ff;
- path->mtk_wdma.queue = phy != &dev->phy;
+ path->mtk_wdma.queue = phy->mt76->band_idx;
ctx->dev = NULL;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
index 32e9c0960657..343866dfc112 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.c
@@ -51,7 +51,7 @@ mt7915_mcu_get_sta_nss(u16 mcs_map)
break;
}
- return nss - 1;
+ return nss ? nss - 1 : 0;
}
static void
@@ -532,9 +532,9 @@ mt7915_mcu_bss_ra_tlv(struct sk_buff *skb, struct ieee80211_vif *vif,
ra->rx_streams = max_nss;
ra->algo = 4;
ra->train_up_rule = 2;
- ra->train_up_high_thres = 110;
- ra->train_up_rule_rssi = -70;
- ra->low_traffic_thres = 2;
+ ra->train_up_high_thres = cpu_to_le16(110);
+ ra->train_up_rule_rssi = cpu_to_le16(-70);
+ ra->low_traffic_thres = cpu_to_le16(2);
ra->phy_cap = cpu_to_le32(0xfdf);
ra->interval = cpu_to_le32(500);
ra->fast_interval = cpu_to_le32(100);
@@ -2758,7 +2758,7 @@ int mt7915_mcu_set_chan_info(struct mt7915_phy *phy, int cmd)
.center_ch = ieee80211_frequency_to_channel(freq1),
.bw = mt76_connac_chan_bw(chandef),
.tx_path_num = hweight16(phy->mt76->chainmask),
- .rx_path = phy->mt76->chainmask >> (dev->chainshift * band),
+ .rx_path = mt7915_band_chainmask(phy),
.band_idx = band,
.channel_band = ch_band[chandef->chan->band],
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
index b7e26c240944..9e57a369a423 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mcu.h
@@ -318,9 +318,9 @@ struct bss_info_ra {
u8 antenna_idx;
u8 train_up_rule;
u8 rsv[3];
- unsigned short train_up_high_thres;
- short train_up_rule_rssi;
- unsigned short low_traffic_thres;
+ __le16 train_up_high_thres;
+ __le16 train_up_rule_rssi;
+ __le16 low_traffic_thres;
__le16 max_phyrate;
__le32 phy_cap;
__le32 interval;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
index f49d6b2f81e4..623b3077ed6f 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h
@@ -383,6 +383,24 @@ mt7915_ext_phy(struct mt7915_dev *dev)
return phy->priv;
}
+/* without dbdc, the chainmask is stored unshifted, even if the phy is
+ * bound to band 1
+ */
+static inline u8 mt7915_band_chainshift(struct mt7915_phy *phy)
+{
+ struct mt7915_dev *dev = phy->dev;
+
+ if (!dev->dbdc_support)
+ return 0;
+
+ return phy->mt76->band_idx * dev->chainshift;
+}
+
+static inline u16 mt7915_band_chainmask(struct mt7915_phy *phy)
+{
+ return phy->mt76->chainmask >> mt7915_band_chainshift(phy);
+}
+
static inline u32 mt7915_check_adie(struct mt7915_dev *dev, bool sku)
{
u32 mask = sku ? MT_CONNINFRA_SKU_MASK : MT_ADIE_TYPE_MASK;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
index 07b0a5766eab..c24f1b12f064 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/pci.c
@@ -135,7 +135,6 @@ static int mt7915_pci_probe(struct pci_dev *pdev,
mdev = &dev->mt76;
mt7915_wfsys_reset(dev);
- hif2 = mt7915_pci_init_hif2(pdev);
ret = mt7915_mmio_wed_init(dev, pdev, true, &irq);
if (ret < 0)
@@ -145,16 +144,20 @@ static int mt7915_pci_probe(struct pci_dev *pdev,
hif2 = mt7915_pci_init_hif2(pdev);
ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES);
- if (ret < 0)
+ if (ret < 0) {
+ mt7915_put_hif2(hif2);
goto free_device;
+ }
irq = pdev->irq;
}
ret = devm_request_irq(mdev->dev, irq, mt7915_irq_handler,
IRQF_SHARED, KBUILD_MODNAME, dev);
- if (ret)
+ if (ret) {
+ mt7915_put_hif2(hif2);
goto free_wed_or_irq_vector;
+ }
/* master switch of PCIe tnterrupt enable */
mt76_wr(dev, MT_PCIE_MAC_INT_ENABLE, 0xff);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/soc.c b/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
index 90a6f61d1089..c0372f693cd4 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/soc.c
@@ -913,7 +913,7 @@ static void mt7986_wmac_clock_enable(struct mt7915_dev *dev, u32 adie_type)
read_poll_timeout(mt76_rr, cur, !(cur & MT_SLP_CTRL_BSY_MASK),
USEC_PER_MSEC, 50 * USEC_PER_MSEC, false,
- dev, MT_ADIE_SLP_CTRL_CK0(0));
+ dev, MT_ADIE_SLP_CTRL_CK0(1));
}
mt76_wmac_spi_unlock(dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c b/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
index d534fff5c952..2a9e5d9ff101 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7915/testmode.c
@@ -694,9 +694,7 @@ mt7915_tm_set_params(struct mt76_phy *mphy, struct nlattr **tb,
{
struct mt76_testmode_data *td = &mphy->test;
struct mt7915_phy *phy = mphy->priv;
- struct mt7915_dev *dev = phy->dev;
- u32 chainmask = mphy->chainmask, changed = 0;
- bool ext_phy = phy != &dev->phy;
+ u32 chainmask = mt7915_band_chainmask(phy), changed = 0;
int i;
BUILD_BUG_ON(NUM_TM_CHANGED >= 32);
@@ -705,7 +703,6 @@ mt7915_tm_set_params(struct mt76_phy *mphy, struct nlattr **tb,
td->state == MT76_TM_STATE_OFF)
return 0;
- chainmask = ext_phy ? chainmask >> dev->chainshift : chainmask;
if (td->tx_antenna_mask > chainmask)
return -EINVAL;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/main.c b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
index 1dbc6d22b066..bcc5ce829169 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/main.c
@@ -178,20 +178,36 @@ mt7925_init_eht_caps(struct mt792x_phy *phy, enum nl80211_band band,
IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMER |
IEEE80211_EHT_PHY_CAP0_SU_BEAMFORMEE;
+ if (band == NL80211_BAND_6GHZ && is_320mhz_supported(&phy->dev->mt76))
+ eht_cap_elem->phy_cap_info[0] |=
+ IEEE80211_EHT_PHY_CAP0_320MHZ_IN_6GHZ;
+
+ val = (sts > 3) ? sts - 1 : 3;
+
eht_cap_elem->phy_cap_info[0] |=
- u8_encode_bits(u8_get_bits(sts - 1, BIT(0)),
+ u8_encode_bits(u8_get_bits(val, BIT(0)),
IEEE80211_EHT_PHY_CAP0_BEAMFORMEE_SS_80MHZ_MASK);
eht_cap_elem->phy_cap_info[1] =
- u8_encode_bits(u8_get_bits(sts - 1, GENMASK(2, 1)),
+ u8_encode_bits(u8_get_bits(val, GENMASK(2, 1)),
IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_80MHZ_MASK) |
- u8_encode_bits(sts - 1,
+ u8_encode_bits(val,
IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_160MHZ_MASK);
+ if (band == NL80211_BAND_6GHZ && is_320mhz_supported(&phy->dev->mt76))
+ eht_cap_elem->phy_cap_info[1] |=
+ u8_encode_bits(val,
+ IEEE80211_EHT_PHY_CAP1_BEAMFORMEE_SS_320MHZ_MASK);
+
eht_cap_elem->phy_cap_info[2] =
u8_encode_bits(sts - 1, IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_80MHZ_MASK) |
u8_encode_bits(sts - 1, IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_160MHZ_MASK);
+ if (band == NL80211_BAND_6GHZ && is_320mhz_supported(&phy->dev->mt76))
+ eht_cap_elem->phy_cap_info[2] |=
+ u8_encode_bits(sts - 1,
+ IEEE80211_EHT_PHY_CAP2_SOUNDING_DIM_320MHZ_MASK);
+
eht_cap_elem->phy_cap_info[3] =
IEEE80211_EHT_PHY_CAP3_NG_16_SU_FEEDBACK |
IEEE80211_EHT_PHY_CAP3_NG_16_MU_FEEDBACK |
@@ -212,7 +228,8 @@ mt7925_init_eht_caps(struct mt792x_phy *phy, enum nl80211_band band,
u8_encode_bits(u8_get_bits(0x11, GENMASK(1, 0)),
IEEE80211_EHT_PHY_CAP5_MAX_NUM_SUPP_EHT_LTF_MASK);
- val = width == NL80211_CHAN_WIDTH_160 ? 0x7 :
+ val = width == NL80211_CHAN_WIDTH_320 ? 0xf :
+ width == NL80211_CHAN_WIDTH_160 ? 0x7 :
width == NL80211_CHAN_WIDTH_80 ? 0x3 : 0x1;
eht_cap_elem->phy_cap_info[6] =
u8_encode_bits(u8_get_bits(0x11, GENMASK(4, 2)),
@@ -234,6 +251,11 @@ mt7925_init_eht_caps(struct mt792x_phy *phy, enum nl80211_band band,
eht_nss->bw._160.rx_tx_mcs9_max_nss = val;
eht_nss->bw._160.rx_tx_mcs11_max_nss = val;
eht_nss->bw._160.rx_tx_mcs13_max_nss = val;
+ if (band == NL80211_BAND_6GHZ && is_320mhz_supported(&phy->dev->mt76)) {
+ eht_nss->bw._320.rx_tx_mcs9_max_nss = val;
+ eht_nss->bw._320.rx_tx_mcs11_max_nss = val;
+ eht_nss->bw._320.rx_tx_mcs13_max_nss = val;
+ }
}
int mt7925_init_mlo_caps(struct mt792x_phy *phy)
@@ -2165,6 +2187,40 @@ static void mt7925_unassign_vif_chanctx(struct ieee80211_hw *hw,
mutex_unlock(&dev->mt76.mutex);
}
+static void mt7925_sta_pre_rcu_remove(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta)
+{
+ struct mt76_phy *phy = hw->priv;
+ struct mt76_dev *dev = phy->dev;
+ struct mt76_wcid *wcid = (struct mt76_wcid *)sta->drv_priv;
+
+ mutex_lock(&dev->mutex);
+ spin_lock_bh(&dev->status_lock);
+
+ if (ieee80211_vif_is_mld(vif)) {
+ struct mt792x_sta *msta = (struct mt792x_sta *)sta->drv_priv;
+ struct mt792x_vif *mvif = (struct mt792x_vif *)vif->drv_priv;
+ unsigned long valid = mvif->valid_links;
+ struct mt792x_link_sta *mlink;
+ unsigned int link_id;
+
+ for_each_set_bit(link_id, &valid, IEEE80211_MLD_MAX_NUM_LINKS) {
+ mlink = mt792x_sta_to_link(msta, link_id);
+ if (!mlink || !mlink->wcid.sta)
+ continue;
+ if (mlink->wcid.idx < ARRAY_SIZE(dev->wcid))
+ rcu_assign_pointer(dev->wcid[mlink->wcid.idx],
+ NULL);
+ }
+ } else {
+ rcu_assign_pointer(dev->wcid[wcid->idx], NULL);
+ }
+
+ spin_unlock_bh(&dev->status_lock);
+ mutex_unlock(&dev->mutex);
+}
+
const struct ieee80211_ops mt7925_ops = {
.tx = mt792x_tx,
.start = mt7925_start,
@@ -2177,7 +2233,7 @@ const struct ieee80211_ops mt7925_ops = {
.start_ap = mt7925_start_ap,
.stop_ap = mt7925_stop_ap,
.sta_state = mt76_sta_state,
- .sta_pre_rcu_remove = mt76_sta_pre_rcu_remove,
+ .sta_pre_rcu_remove = mt7925_sta_pre_rcu_remove,
.set_key = mt7925_set_key,
.sta_set_decap_offload = mt7925_sta_set_decap_offload,
#if IS_ENABLED(CONFIG_IPV6)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
index 910775a2c39f..ac241c2dfb98 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7925/mcu.c
@@ -3602,7 +3602,7 @@ mt7925_mcu_rate_txpower_band(struct mt76_phy *phy,
memcpy(tx_power_tlv->alpha2, dev->alpha2, sizeof(dev->alpha2));
tx_power_tlv->n_chan = num_ch;
tx_power_tlv->tag = cpu_to_le16(0x1);
- tx_power_tlv->len = cpu_to_le16(msg_len);
+ tx_power_tlv->len = cpu_to_le16(msg_len - 4);
switch (band) {
case NL80211_BAND_2GHZ:
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
index 4c4ab0d4065d..a14d1d2849b6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/eeprom.c
@@ -51,6 +51,12 @@ mt7996_eeprom_load_default(struct mt7996_dev *dev)
goto out;
}
+ if (fw->size < MT7996_EEPROM_SIZE) {
+ dev_err(dev->mt76.dev, "Invalid default bin size\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
memcpy(eeprom, fw->data, MT7996_EEPROM_SIZE);
dev->flash_mode = true;
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/init.c b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
index fc2d46b10b72..d3d63f07d7b0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/init.c
@@ -1181,7 +1181,6 @@ mt7996_init_eht_caps(struct mt7996_phy *phy, enum nl80211_band band,
struct ieee80211_sta_eht_cap *eht_cap = &data->eht_cap;
struct ieee80211_eht_cap_elem_fixed *eht_cap_elem = &eht_cap->eht_cap_elem;
struct ieee80211_eht_mcs_nss_supp *eht_nss = &eht_cap->eht_mcs_nss_supp;
- enum nl80211_chan_width width = phy->mt76->chandef.width;
int nss = hweight8(phy->mt76->antenna_mask);
int sts = hweight16(phy->mt76->chainmask);
u8 val;
@@ -1250,11 +1249,16 @@ mt7996_init_eht_caps(struct mt7996_phy *phy, enum nl80211_band band,
u8_encode_bits(u8_get_bits(1, GENMASK(1, 0)),
IEEE80211_EHT_PHY_CAP5_MAX_NUM_SUPP_EHT_LTF_MASK);
- val = width == NL80211_CHAN_WIDTH_320 ? 0xf :
- width == NL80211_CHAN_WIDTH_160 ? 0x7 :
- width == NL80211_CHAN_WIDTH_80 ? 0x3 : 0x1;
- eht_cap_elem->phy_cap_info[6] =
- u8_encode_bits(val, IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_MASK);
+ eht_cap_elem->phy_cap_info[6] = IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_MASK;
+ if (band != NL80211_BAND_6GHZ) {
+ eht_cap_elem->phy_cap_info[6] &=
+ ~IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_320MHZ;
+
+ if (band != NL80211_BAND_5GHZ)
+ eht_cap_elem->phy_cap_info[6] &=
+ ~(IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_160MHZ |
+ IEEE80211_EHT_PHY_CAP6_MCS15_SUPP_80MHZ);
+ }
val = u8_encode_bits(nss, IEEE80211_EHT_MCS_NSS_RX) |
u8_encode_bits(nss, IEEE80211_EHT_MCS_NSS_TX);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
index 54bbc871c0bd..24e0346ea3d8 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mac.c
@@ -454,7 +454,13 @@ mt7996_mac_fill_rx(struct mt7996_dev *dev, enum mt76_rxq_id q,
memset(status, 0, sizeof(*status));
band_idx = FIELD_GET(MT_RXD1_NORMAL_BAND_IDX, rxd1);
+ if (!mt7996_band_valid(dev, band_idx))
+ return -EINVAL;
+
mphy = dev->mt76.phys[band_idx];
+ if (!mphy)
+ return -EINVAL;
+
phy = mphy->priv;
status->phy_idx = mphy->band_idx;
@@ -1792,6 +1798,7 @@ mt7996_mac_full_reset(struct mt7996_dev *dev)
phy3 = mt7996_phy3(dev);
dev->recovery.hw_full_reset = true;
+ set_bit(MT76_MCU_RESET, &dev->mphy.state);
wake_up(&dev->mt76.mcu.wait);
ieee80211_stop_queues(mt76_hw(dev));
if (phy2)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/main.c b/drivers/net/wireless/mediatek/mt76/mt7996/main.c
index 4d11083b86c0..bdb5e82d9f4e 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/main.c
@@ -1014,8 +1014,6 @@ static void mt7996_sta_statistics(struct ieee80211_hw *hw,
sinfo->txrate.flags = txrate->flags;
sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE);
}
- sinfo->txrate.flags = txrate->flags;
- sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_BITRATE);
sinfo->tx_failed = msta->wcid.stats.tx_failed;
sinfo->filled |= BIT_ULL(NL80211_STA_INFO_TX_FAILED);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
index a5693efee74b..e3f5b2b2e367 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.c
@@ -3651,21 +3651,31 @@ int mt7996_mcu_get_chip_config(struct mt7996_dev *dev, u32 *cap)
return ret;
/* fixed field */
+ if (skb->len < 4) {
+ dev_kfree_skb(skb);
+ return -EINVAL;
+ }
skb_pull(skb, 4);
buf = skb->data;
- while (buf - skb->data < skb->len) {
+ while (buf - skb->data + sizeof(struct tlv) <= skb->len) {
struct tlv *tlv = (struct tlv *)buf;
+ u16 tlv_len = le16_to_cpu(tlv->len);
+
+ if (tlv_len < sizeof(*tlv) ||
+ tlv_len > skb->len - (buf - skb->data))
+ break;
switch (le16_to_cpu(tlv->tag)) {
case UNI_EVENT_CHIP_CONFIG_EFUSE_VERSION:
- *cap = le32_to_cpu(*(__le32 *)(buf + sizeof(*tlv)));
+ if (tlv_len >= sizeof(*tlv) + sizeof(__le32))
+ *cap = le32_to_cpu(*(__le32 *)(buf + sizeof(*tlv)));
break;
default:
break;
}
- buf += le16_to_cpu(tlv->len);
+ buf += tlv_len;
}
dev_kfree_skb(skb);
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
index ded2fe721068..ce01f540972b 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mcu.h
@@ -804,7 +804,7 @@ enum {
#define MT7996_BEACON_UPDATE_SIZE (sizeof(struct bss_req_hdr) + \
sizeof(struct bss_bcn_content_tlv) + \
4 + MT_TXD_SIZE + \
- sizeof(struct bss_bcn_cntdwn_tlv) + \
+ sizeof(struct bss_bcn_cntdwn_tlv) * 2 + \
sizeof(struct bss_bcn_mbss_tlv))
#define MT7996_MAX_BSS_OFFLOAD_SIZE (MT7996_MAX_BEACON_SIZE + \
MT7996_BEACON_UPDATE_SIZE)
diff --git a/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c b/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
index bffee73b780c..01d4b54a1acc 100644
--- a/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt7996/mmio.c
@@ -17,6 +17,8 @@
static bool wed_enable;
module_param(wed_enable, bool, 0644);
+#define INVALID_REG_ADDR 0xffffffff
+
static const struct __base mt7996_reg_base[] = {
[WF_AGG_BASE] = { { 0x820e2000, 0x820f2000, 0x830e2000 } },
[WF_ARB_BASE] = { { 0x820e3000, 0x820f3000, 0x830e3000 } },
@@ -183,7 +185,7 @@ static u32 __mt7996_reg_addr(struct mt7996_dev *dev, u32 addr)
return dev->reg.map[i].mapped + ofs;
}
- return 0;
+ return INVALID_REG_ADDR;
}
static u32 __mt7996_reg_remap_addr(struct mt7996_dev *dev, u32 addr)
@@ -212,7 +214,7 @@ void mt7996_memcpy_fromio(struct mt7996_dev *dev, void *buf, u32 offset,
{
u32 addr = __mt7996_reg_addr(dev, offset);
- if (addr) {
+ if (addr != INVALID_REG_ADDR) {
memcpy_fromio(buf, dev->mt76.mmio.regs + addr, len);
return;
}
@@ -228,7 +230,7 @@ static u32 mt7996_rr(struct mt76_dev *mdev, u32 offset)
struct mt7996_dev *dev = container_of(mdev, struct mt7996_dev, mt76);
u32 addr = __mt7996_reg_addr(dev, offset), val;
- if (addr)
+ if (addr != INVALID_REG_ADDR)
return dev->bus_ops->rr(mdev, addr);
spin_lock_bh(&dev->reg_lock);
@@ -243,7 +245,7 @@ static void mt7996_wr(struct mt76_dev *mdev, u32 offset, u32 val)
struct mt7996_dev *dev = container_of(mdev, struct mt7996_dev, mt76);
u32 addr = __mt7996_reg_addr(dev, offset);
- if (addr) {
+ if (addr != INVALID_REG_ADDR) {
dev->bus_ops->wr(mdev, addr, val);
return;
}
@@ -258,7 +260,7 @@ static u32 mt7996_rmw(struct mt76_dev *mdev, u32 offset, u32 mask, u32 val)
struct mt7996_dev *dev = container_of(mdev, struct mt7996_dev, mt76);
u32 addr = __mt7996_reg_addr(dev, offset);
- if (addr)
+ if (addr != INVALID_REG_ADDR)
return dev->bus_ops->rmw(mdev, addr, mask, val);
spin_lock_bh(&dev->reg_lock);
diff --git a/drivers/net/wireless/mediatek/mt76/sdio.c b/drivers/net/wireless/mediatek/mt76/sdio.c
index 8e9576747052..e4c3dcc34f68 100644
--- a/drivers/net/wireless/mediatek/mt76/sdio.c
+++ b/drivers/net/wireless/mediatek/mt76/sdio.c
@@ -519,6 +519,10 @@ mt76s_tx_queue_skb(struct mt76_phy *phy, struct mt76_queue *q,
enum mt76_txq_id qid, struct sk_buff *skb,
struct mt76_wcid *wcid, struct ieee80211_sta *sta)
{
+ struct ieee80211_tx_status status = {
+ .sta = sta,
+ };
+
struct mt76_tx_info tx_info = {
.skb = skb,
};
@@ -531,8 +535,13 @@ mt76s_tx_queue_skb(struct mt76_phy *phy, struct mt76_queue *q,
skb->prev = skb->next = NULL;
err = dev->drv->tx_prepare_skb(dev, NULL, qid, wcid, sta, &tx_info);
- if (err < 0)
+ if (err < 0) {
+ status.skb = tx_info.skb;
+ spin_lock_bh(&dev->rx_lock);
+ ieee80211_tx_status_ext(dev->hw, &status);
+ spin_unlock_bh(&dev->rx_lock);
return err;
+ }
q->entry[q->head].skb = tx_info.skb;
q->entry[q->head].buf_sz = len;
diff --git a/drivers/net/wireless/mediatek/mt76/tx.c b/drivers/net/wireless/mediatek/mt76/tx.c
index 634b6dacd1e0..1a5acb2bcfb3 100644
--- a/drivers/net/wireless/mediatek/mt76/tx.c
+++ b/drivers/net/wireless/mediatek/mt76/tx.c
@@ -313,10 +313,6 @@ __mt76_tx_queue_skb(struct mt76_phy *phy, int qid, struct sk_buff *skb,
if (idx < 0 || !sta)
return idx;
- wcid = (struct mt76_wcid *)sta->drv_priv;
- if (!wcid->sta)
- return idx;
-
q->entry[idx].wcid = wcid->idx;
if (!non_aql)
@@ -668,8 +664,8 @@ static void mt76_txq_schedule_pending(struct mt76_phy *phy)
ret = mt76_txq_schedule_pending_wcid(phy, wcid, &wcid->tx_pending);
spin_lock(&phy->tx_lock);
- if (!skb_queue_empty(&wcid->tx_pending) &&
- !skb_queue_empty(&wcid->tx_offchannel) &&
+ if ((!skb_queue_empty(&wcid->tx_pending) ||
+ !skb_queue_empty(&wcid->tx_offchannel)) &&
list_empty(&wcid->tx_list))
list_add_tail(&wcid->tx_list, &phy->tx_list);
}
diff --git a/drivers/net/wireless/realtek/rtlwifi/pci.c b/drivers/net/wireless/realtek/rtlwifi/pci.c
index f0010336e78c..8fb9604c3c53 100644
--- a/drivers/net/wireless/realtek/rtlwifi/pci.c
+++ b/drivers/net/wireless/realtek/rtlwifi/pci.c
@@ -2226,13 +2226,17 @@ int rtl_pci_probe(struct pci_dev *pdev,
rtl_dbg(rtlpriv, COMP_INIT, DBG_DMESG,
"%s: failed to register IRQ handler\n",
wiphy_name(hw->wiphy));
- goto fail3;
+ goto fail6;
}
rtlpci->irq_alloc = 1;
set_bit(RTL_STATUS_INTERFACE_START, &rtlpriv->status);
return 0;
+fail6:
+ rtl_deinit_rfkill(hw);
+ rtl_debug_remove_one(hw);
+ ieee80211_unregister_hw(hw);
fail5:
rtl_pci_deinit(hw);
fail4:
diff --git a/drivers/net/wireless/realtek/rtw89/mac.c b/drivers/net/wireless/realtek/rtw89/mac.c
index 31465893f866..f323417a88d2 100644
--- a/drivers/net/wireless/realtek/rtw89/mac.c
+++ b/drivers/net/wireless/realtek/rtw89/mac.c
@@ -4730,7 +4730,7 @@ static void rtw89_mac_check_he_obss_narrow_bw_ru_iter(struct wiphy *wiphy,
elem = cfg80211_find_elem(WLAN_EID_EXT_CAPABILITY, ies->data,
ies->len);
- if (!elem || elem->datalen < 10 ||
+ if (!elem || elem->datalen < 11 ||
!(elem->data[10] & WLAN_EXT_CAPA10_OBSS_NARROW_BW_RU_TOLERANCE_SUPPORT))
*tolerated = false;
rcu_read_unlock();
diff --git a/drivers/net/wireless/virtual/mac80211_hwsim.c b/drivers/net/wireless/virtual/mac80211_hwsim.c
index 63fae67c8162..4622cbf4f5f9 100644
--- a/drivers/net/wireless/virtual/mac80211_hwsim.c
+++ b/drivers/net/wireless/virtual/mac80211_hwsim.c
@@ -2117,6 +2117,7 @@ static int mac80211_hwsim_start(struct ieee80211_hw *hw)
static void mac80211_hwsim_stop(struct ieee80211_hw *hw, bool suspend)
{
struct mac80211_hwsim_data *data = hw->priv;
+ struct sk_buff *skb;
int i;
data->started = false;
@@ -2124,8 +2125,8 @@ static void mac80211_hwsim_stop(struct ieee80211_hw *hw, bool suspend)
for (i = 0; i < ARRAY_SIZE(data->link_data); i++)
hrtimer_cancel(&data->link_data[i].beacon_timer);
- while (!skb_queue_empty(&data->pending))
- ieee80211_free_txskb(hw, skb_dequeue(&data->pending));
+ while ((skb = skb_dequeue(&data->pending)))
+ ieee80211_free_txskb(hw, skb);
wiphy_dbg(hw->wiphy, "%s\n", __func__);
}
diff --git a/drivers/net/wireless/zydas/zd1211rw/zd_usb.c b/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
index 9ae10f65f2af..8b8abcfbbfc8 100644
--- a/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
+++ b/drivers/net/wireless/zydas/zd1211rw/zd_usb.c
@@ -1352,6 +1352,14 @@ static int probe(struct usb_interface *intf, const struct usb_device_id *id)
struct zd_usb *usb;
struct ieee80211_hw *hw = NULL;
+ /*
+ * ZD1211 devices are single-function. Reject secondary interfaces
+ * to prevent multiple instances from conflicting on hardcoded endpoints
+ * and triggering recursive locking warnings.
+ */
+ if (intf->cur_altsetting->desc.bInterfaceNumber != 0)
+ return -ENODEV;
+
print_id(udev);
if (id->driver_info & DEVICE_INSTALLER)
diff --git a/drivers/nfc/pn533/pn533.c b/drivers/nfc/pn533/pn533.c
index 6abe7b38acfb..c804c16f4b3b 100644
--- a/drivers/nfc/pn533/pn533.c
+++ b/drivers/nfc/pn533/pn533.c
@@ -434,6 +434,18 @@ static int pn533_send_async_complete(struct pn533 *dev)
return rc;
}
+static int pn533_send_cmd_frame(struct pn533 *dev, struct pn533_cmd *cmd)
+{
+ struct sk_buff *req = cmd->req;
+ int rc;
+
+ skb_get(req);
+ dev->cmd = cmd;
+ rc = dev->phy_ops->send_frame(dev, req);
+ dev_kfree_skb(req);
+ return rc;
+}
+
static int __pn533_send_async(struct pn533 *dev, u8 cmd_code,
struct sk_buff *req,
pn533_send_async_complete_t complete_cb,
@@ -458,8 +470,7 @@ static int __pn533_send_async(struct pn533 *dev, u8 cmd_code,
mutex_lock(&dev->cmd_lock);
if (!dev->cmd_pending) {
- dev->cmd = cmd;
- rc = dev->phy_ops->send_frame(dev, req);
+ rc = pn533_send_cmd_frame(dev, cmd);
if (rc) {
dev->cmd = NULL;
goto error;
@@ -529,8 +540,7 @@ static int pn533_send_cmd_direct_async(struct pn533 *dev, u8 cmd_code,
pn533_build_cmd_frame(dev, cmd_code, req);
- dev->cmd = cmd;
- rc = dev->phy_ops->send_frame(dev, req);
+ rc = pn533_send_cmd_frame(dev, cmd);
if (rc < 0) {
dev->cmd = NULL;
kfree(cmd);
@@ -569,8 +579,7 @@ static void pn533_wq_cmd(struct work_struct *work)
mutex_unlock(&dev->cmd_lock);
- dev->cmd = cmd;
- rc = dev->phy_ops->send_frame(dev, cmd->req);
+ rc = pn533_send_cmd_frame(dev, cmd);
if (rc < 0) {
dev->cmd = NULL;
dev_kfree_skb(cmd->req);
diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 80ecbfb3c6e9..109444ae32de 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -885,6 +885,14 @@ static int discover_arenas(struct btt *btt)
arena->external_lba_start = cur_nlba;
parse_arena_meta(arena, super, cur_off);
+ if (arena->nfree < btt->nd_region->num_lanes) {
+ dev_err(to_dev(arena),
+ "nfree %u smaller than lane count %d\n",
+ arena->nfree, btt->nd_region->num_lanes);
+ ret = -ENODEV;
+ goto out;
+ }
+
ret = log_set_indices(arena);
if (ret) {
dev_err(to_dev(arena),
diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 584a9c27ce4b..93ff219d91fc 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -145,10 +145,21 @@ static int __nd_label_validate(struct nvdimm_drvdata *ndd)
/* label sizes larger than 128 arrived with v1.2 */
version = __le16_to_cpu(nsindex[i]->major) * 100
+ __le16_to_cpu(nsindex[i]->minor);
- if (version >= 102)
+ if (version >= 102) {
+ /*
+ * labelsize feeds the shift below; only 0 (128-byte)
+ * and 1 (256-byte) are valid -- a larger value would
+ * overflow or exceed the width of int.
+ */
+ if (nsindex[i]->labelsize > 1) {
+ dev_dbg(dev, "nsindex%d labelsize: %d invalid\n",
+ i, nsindex[i]->labelsize);
+ continue;
+ }
labelsize = 1 << (7 + nsindex[i]->labelsize);
- else
+ } else {
labelsize = 128;
+ }
if (labelsize != sizeof_namespace_label(ndd)) {
dev_dbg(dev, "nsindex%d labelsize %d invalid\n",
diff --git a/drivers/nvme/host/apple.c b/drivers/nvme/host/apple.c
index 839c6737dbb6..db8c20a60064 100644
--- a/drivers/nvme/host/apple.c
+++ b/drivers/nvme/host/apple.c
@@ -35,7 +35,6 @@
#include "nvme.h"
#define APPLE_ANS_BOOT_TIMEOUT USEC_PER_SEC
-#define APPLE_ANS_MAX_QUEUE_DEPTH 64
#define APPLE_ANS_COPROC_CPU_CONTROL 0x44
#define APPLE_ANS_COPROC_CPU_CONTROL_RUN BIT(4)
@@ -48,9 +47,6 @@
#define APPLE_ANS_BOOT_STATUS 0x1300
#define APPLE_ANS_BOOT_STATUS_OK 0xde71ce55
-#define APPLE_ANS_UNKNOWN_CTRL 0x24008
-#define APPLE_ANS_PRP_NULL_CHECK BIT(11)
-
#define APPLE_ANS_LINEAR_SQ_CTRL 0x24908
#define APPLE_ANS_LINEAR_SQ_EN BIT(0)
@@ -75,6 +71,8 @@
#define APPLE_NVME_AQ_DEPTH 2
#define APPLE_NVME_AQ_MQ_TAG_DEPTH (APPLE_NVME_AQ_DEPTH - 1)
+#define APPLE_NVME_IOSQES 7
+
/*
* These can be higher, but we need to ensure that any command doesn't
* require an sg allocation that needs more than a page of data.
@@ -142,6 +140,7 @@ struct apple_nvme_queue {
u32 __iomem *sq_db;
u32 __iomem *cq_db;
+ u16 sq_tail;
u16 cq_head;
u8 cq_phase;
@@ -149,6 +148,23 @@ struct apple_nvme_queue {
bool enabled;
};
+static inline bool apple_nvme_queue_enabled(struct apple_nvme_queue *q)
+{
+ /* Pair with apple_nvme_enable_queue(). */
+ return smp_load_acquire(&q->enabled);
+}
+
+static inline void apple_nvme_enable_queue(struct apple_nvme_queue *q)
+{
+ /* Publish queue initialization before setting q->enabled. */
+ smp_store_release(&q->enabled, true);
+}
+
+static inline void apple_nvme_disable_queue(struct apple_nvme_queue *q)
+{
+ WRITE_ONCE(q->enabled, false);
+}
+
/*
* The apple_nvme_iod describes the data in an I/O.
*
@@ -166,11 +182,17 @@ struct apple_nvme_iod {
struct scatterlist *sg;
};
+struct apple_nvme_hw {
+ bool has_lsq_nvmmu;
+ u32 max_queue_depth;
+};
+
struct apple_nvme {
struct device *dev;
void __iomem *mmio_coproc;
void __iomem *mmio_nvme;
+ const struct apple_nvme_hw *hw;
struct device **pd_dev;
struct device_link **pd_link;
@@ -215,10 +237,12 @@ static inline struct apple_nvme *queue_to_apple_nvme(struct apple_nvme_queue *q)
static unsigned int apple_nvme_queue_depth(struct apple_nvme_queue *q)
{
+ struct apple_nvme *anv = queue_to_apple_nvme(q);
+
if (q->is_adminq)
return APPLE_NVME_AQ_DEPTH;
- return APPLE_ANS_MAX_QUEUE_DEPTH;
+ return anv->hw->max_queue_depth;
}
static void apple_nvme_rtkit_crashed(void *cookie)
@@ -280,20 +304,43 @@ static void apple_nvmmu_inval(struct apple_nvme_queue *q, unsigned int tag)
"NVMMU TCB invalidation failed\n");
}
-static void apple_nvme_submit_cmd(struct apple_nvme_queue *q,
+static void apple_nvme_submit_cmd_t8015(struct apple_nvme_queue *q,
+ struct nvme_command *cmd)
+{
+ struct apple_nvme *anv = queue_to_apple_nvme(q);
+
+ spin_lock_irq(&anv->lock);
+
+ if (q->is_adminq)
+ memcpy(&q->sqes[q->sq_tail], cmd, sizeof(*cmd));
+ else
+ memcpy((void *)q->sqes + (q->sq_tail << APPLE_NVME_IOSQES),
+ cmd, sizeof(*cmd));
+
+ if (++q->sq_tail == apple_nvme_queue_depth(q))
+ q->sq_tail = 0;
+
+ writel(q->sq_tail, q->sq_db);
+ spin_unlock_irq(&anv->lock);
+}
+
+
+static void apple_nvme_submit_cmd_t8103(struct apple_nvme_queue *q,
struct nvme_command *cmd)
{
struct apple_nvme *anv = queue_to_apple_nvme(q);
u32 tag = nvme_tag_from_cid(cmd->common.command_id);
struct apple_nvmmu_tcb *tcb = &q->tcbs[tag];
- tcb->opcode = cmd->common.opcode;
+ tcb->opcode = 0;
tcb->prp1 = cmd->common.dptr.prp1;
tcb->prp2 = cmd->common.dptr.prp2;
tcb->length = cmd->rw.length;
tcb->command_id = tag;
- if (nvme_is_write(cmd))
+ if (!cmd->common.dptr.prp1)
+ tcb->dma_flags = 0;
+ else if (nvme_is_write(cmd))
tcb->dma_flags = APPLE_ANS_TCB_DMA_TO_DEVICE;
else
tcb->dma_flags = APPLE_ANS_TCB_DMA_FROM_DEVICE;
@@ -590,7 +637,8 @@ static inline void apple_nvme_handle_cqe(struct apple_nvme_queue *q,
__u16 command_id = READ_ONCE(cqe->command_id);
struct request *req;
- apple_nvmmu_inval(q, command_id);
+ if (anv->hw->has_lsq_nvmmu)
+ apple_nvmmu_inval(q, command_id);
req = nvme_find_rq(apple_nvme_queue_tagset(anv, q), command_id);
if (unlikely(!req)) {
@@ -645,7 +693,7 @@ static bool apple_nvme_handle_cq(struct apple_nvme_queue *q, bool force)
bool found;
DEFINE_IO_COMP_BATCH(iob);
- if (!READ_ONCE(q->enabled) && !force)
+ if (!apple_nvme_queue_enabled(q) && !force)
return false;
found = apple_nvme_poll_cq(q, &iob);
@@ -685,7 +733,7 @@ static int apple_nvme_create_cq(struct apple_nvme *anv)
c.create_cq.opcode = nvme_admin_create_cq;
c.create_cq.prp1 = cpu_to_le64(anv->ioq.cq_dma_addr);
c.create_cq.cqid = cpu_to_le16(1);
- c.create_cq.qsize = cpu_to_le16(APPLE_ANS_MAX_QUEUE_DEPTH - 1);
+ c.create_cq.qsize = cpu_to_le16(anv->hw->max_queue_depth - 1);
c.create_cq.cq_flags = cpu_to_le16(NVME_QUEUE_PHYS_CONTIG | NVME_CQ_IRQ_ENABLED);
c.create_cq.irq_vector = cpu_to_le16(0);
@@ -713,7 +761,7 @@ static int apple_nvme_create_sq(struct apple_nvme *anv)
c.create_sq.opcode = nvme_admin_create_sq;
c.create_sq.prp1 = cpu_to_le64(anv->ioq.sq_dma_addr);
c.create_sq.sqid = cpu_to_le16(1);
- c.create_sq.qsize = cpu_to_le16(APPLE_ANS_MAX_QUEUE_DEPTH - 1);
+ c.create_sq.qsize = cpu_to_le16(anv->hw->max_queue_depth - 1);
c.create_sq.sq_flags = cpu_to_le16(NVME_QUEUE_PHYS_CONTIG);
c.create_sq.cqid = cpu_to_le16(1);
@@ -748,7 +796,7 @@ static blk_status_t apple_nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
* We should not need to do this, but we're still using this to
* ensure we can drain requests on a dying queue.
*/
- if (unlikely(!READ_ONCE(q->enabled)))
+ if (unlikely(!apple_nvme_queue_enabled(q)))
return BLK_STS_IOERR;
if (!nvme_check_ready(&anv->ctrl, req, true))
@@ -765,7 +813,12 @@ static blk_status_t apple_nvme_queue_rq(struct blk_mq_hw_ctx *hctx,
}
nvme_start_request(req);
- apple_nvme_submit_cmd(q, cmnd);
+
+ if (anv->hw->has_lsq_nvmmu)
+ apple_nvme_submit_cmd_t8103(q, cmnd);
+ else
+ apple_nvme_submit_cmd_t8015(q, cmnd);
+
return BLK_STS_OK;
out_free_cmd:
@@ -826,7 +879,7 @@ static void apple_nvme_disable(struct apple_nvme *anv, bool shutdown)
nvme_quiesce_io_queues(&anv->ctrl);
if (!dead) {
- if (READ_ONCE(anv->ioq.enabled)) {
+ if (apple_nvme_queue_enabled(&anv->ioq)) {
apple_nvme_remove_sq(anv);
apple_nvme_remove_cq(anv);
}
@@ -850,8 +903,8 @@ static void apple_nvme_disable(struct apple_nvme *anv, bool shutdown)
nvme_disable_ctrl(&anv->ctrl, false);
}
- WRITE_ONCE(anv->ioq.enabled, false);
- WRITE_ONCE(anv->adminq.enabled, false);
+ apple_nvme_disable_queue(&anv->ioq);
+ apple_nvme_disable_queue(&anv->adminq);
mb(); /* ensure that nvme_queue_rq() sees that enabled is cleared */
nvme_quiesce_admin_queue(&anv->ctrl);
@@ -970,14 +1023,16 @@ static const struct blk_mq_ops apple_nvme_mq_ops = {
static void apple_nvme_init_queue(struct apple_nvme_queue *q)
{
unsigned int depth = apple_nvme_queue_depth(q);
+ struct apple_nvme *anv = queue_to_apple_nvme(q);
+ q->sq_tail = 0;
q->cq_head = 0;
q->cq_phase = 1;
- memset(q->tcbs, 0,
- APPLE_ANS_MAX_QUEUE_DEPTH * sizeof(struct apple_nvmmu_tcb));
+ if (anv->hw->has_lsq_nvmmu)
+ memset(q->tcbs, 0, anv->hw->max_queue_depth
+ * sizeof(struct apple_nvmmu_tcb));
memset(q->cqes, 0, depth * sizeof(struct nvme_completion));
- WRITE_ONCE(q->enabled, true);
- wmb(); /* ensure the first interrupt sees the initialization */
+ apple_nvme_enable_queue(q);
}
static void apple_nvme_reset_work(struct work_struct *work)
@@ -1057,33 +1112,23 @@ static void apple_nvme_reset_work(struct work_struct *work)
dma_set_max_seg_size(anv->dev, 0xffffffff);
- /*
- * Enable NVMMU and linear submission queues.
- * While we could keep those disabled and pretend this is slightly
- * more common NVMe controller we'd still need some quirks (e.g.
- * sq entries will be 128 bytes) and Apple might drop support for
- * that mode in the future.
- */
- writel(APPLE_ANS_LINEAR_SQ_EN,
- anv->mmio_nvme + APPLE_ANS_LINEAR_SQ_CTRL);
-
- /* Allow as many pending command as possible for both queues */
- writel(APPLE_ANS_MAX_QUEUE_DEPTH | (APPLE_ANS_MAX_QUEUE_DEPTH << 16),
- anv->mmio_nvme + APPLE_ANS_MAX_PEND_CMDS_CTRL);
+ if (anv->hw->has_lsq_nvmmu) {
+ /*
+ * Enable NVMMU and linear submission queues which is required
+ * since T6000.
+ */
+ writel(APPLE_ANS_LINEAR_SQ_EN,
+ anv->mmio_nvme + APPLE_ANS_LINEAR_SQ_CTRL);
- /* Setup the NVMMU for the maximum admin and IO queue depth */
- writel(APPLE_ANS_MAX_QUEUE_DEPTH - 1,
- anv->mmio_nvme + APPLE_NVMMU_NUM_TCBS);
+ /* Allow as many pending command as possible for both queues */
+ writel(anv->hw->max_queue_depth
+ | (anv->hw->max_queue_depth << 16), anv->mmio_nvme
+ + APPLE_ANS_MAX_PEND_CMDS_CTRL);
- /*
- * This is probably a chicken bit: without it all commands where any PRP
- * is set to zero (including those that don't use that field) fail and
- * the co-processor complains about "completed with err BAD_CMD-" or
- * a "NULL_PRP_PTR_ERR" in the syslog
- */
- writel(readl(anv->mmio_nvme + APPLE_ANS_UNKNOWN_CTRL) &
- ~APPLE_ANS_PRP_NULL_CHECK,
- anv->mmio_nvme + APPLE_ANS_UNKNOWN_CTRL);
+ /* Setup the NVMMU for the maximum admin and IO queue depth */
+ writel(anv->hw->max_queue_depth - 1,
+ anv->mmio_nvme + APPLE_NVMMU_NUM_TCBS);
+ }
/* Setup the admin queue */
aqa = APPLE_NVME_AQ_DEPTH - 1;
@@ -1092,14 +1137,16 @@ static void apple_nvme_reset_work(struct work_struct *work)
writeq(anv->adminq.sq_dma_addr, anv->mmio_nvme + NVME_REG_ASQ);
writeq(anv->adminq.cq_dma_addr, anv->mmio_nvme + NVME_REG_ACQ);
- /* Setup NVMMU for both queues */
- writeq(anv->adminq.tcb_dma_addr,
- anv->mmio_nvme + APPLE_NVMMU_ASQ_TCB_BASE);
- writeq(anv->ioq.tcb_dma_addr,
- anv->mmio_nvme + APPLE_NVMMU_IOSQ_TCB_BASE);
+ if (anv->hw->has_lsq_nvmmu) {
+ /* Setup NVMMU for both queues */
+ writeq(anv->adminq.tcb_dma_addr,
+ anv->mmio_nvme + APPLE_NVMMU_ASQ_TCB_BASE);
+ writeq(anv->ioq.tcb_dma_addr,
+ anv->mmio_nvme + APPLE_NVMMU_IOSQ_TCB_BASE);
+ }
anv->ctrl.sqsize =
- APPLE_ANS_MAX_QUEUE_DEPTH - 1; /* 0's based queue depth */
+ anv->hw->max_queue_depth - 1; /* 0's based queue depth */
anv->ctrl.cap = readq(anv->mmio_nvme + NVME_REG_CAP);
dev_dbg(anv->dev, "Enabling controller now");
@@ -1268,7 +1315,7 @@ static int apple_nvme_alloc_tagsets(struct apple_nvme *anv)
* must be marked as reserved in the IO queue.
*/
anv->tagset.reserved_tags = APPLE_NVME_AQ_DEPTH;
- anv->tagset.queue_depth = APPLE_ANS_MAX_QUEUE_DEPTH - 1;
+ anv->tagset.queue_depth = anv->hw->max_queue_depth - 1;
anv->tagset.timeout = NVME_IO_TIMEOUT;
anv->tagset.numa_node = NUMA_NO_NODE;
anv->tagset.cmd_size = sizeof(struct apple_nvme_iod);
@@ -1293,6 +1340,7 @@ static int apple_nvme_queue_alloc(struct apple_nvme *anv,
struct apple_nvme_queue *q)
{
unsigned int depth = apple_nvme_queue_depth(q);
+ size_t iosq_size;
q->cqes = dmam_alloc_coherent(anv->dev,
depth * sizeof(struct nvme_completion),
@@ -1300,22 +1348,28 @@ static int apple_nvme_queue_alloc(struct apple_nvme *anv,
if (!q->cqes)
return -ENOMEM;
- q->sqes = dmam_alloc_coherent(anv->dev,
- depth * sizeof(struct nvme_command),
+ if (anv->hw->has_lsq_nvmmu)
+ iosq_size = depth * sizeof(struct nvme_command);
+ else
+ iosq_size = depth << APPLE_NVME_IOSQES;
+
+ q->sqes = dmam_alloc_coherent(anv->dev, iosq_size,
&q->sq_dma_addr, GFP_KERNEL);
if (!q->sqes)
return -ENOMEM;
- /*
- * We need the maximum queue depth here because the NVMMU only has a
- * single depth configuration shared between both queues.
- */
- q->tcbs = dmam_alloc_coherent(anv->dev,
- APPLE_ANS_MAX_QUEUE_DEPTH *
- sizeof(struct apple_nvmmu_tcb),
- &q->tcb_dma_addr, GFP_KERNEL);
- if (!q->tcbs)
- return -ENOMEM;
+ if (anv->hw->has_lsq_nvmmu) {
+ /*
+ * We need the maximum queue depth here because the NVMMU only
+ * has a single depth configuration shared between both queues.
+ */
+ q->tcbs = dmam_alloc_coherent(anv->dev,
+ anv->hw->max_queue_depth *
+ sizeof(struct apple_nvmmu_tcb),
+ &q->tcb_dma_addr, GFP_KERNEL);
+ if (!q->tcbs)
+ return -ENOMEM;
+ }
/*
* initialize phase to make sure the allocated and empty memory
@@ -1399,6 +1453,12 @@ static struct apple_nvme *apple_nvme_alloc(struct platform_device *pdev)
anv->adminq.is_adminq = true;
platform_set_drvdata(pdev, anv);
+ anv->hw = of_device_get_match_data(&pdev->dev);
+ if (!anv->hw) {
+ ret = -ENODEV;
+ goto put_dev;
+ }
+
ret = apple_nvme_attach_genpd(anv);
if (ret < 0) {
dev_err_probe(dev, ret, "Failed to attach power domains");
@@ -1430,10 +1490,17 @@ static struct apple_nvme *apple_nvme_alloc(struct platform_device *pdev)
goto put_dev;
}
- anv->adminq.sq_db = anv->mmio_nvme + APPLE_ANS_LINEAR_ASQ_DB;
- anv->adminq.cq_db = anv->mmio_nvme + APPLE_ANS_ACQ_DB;
- anv->ioq.sq_db = anv->mmio_nvme + APPLE_ANS_LINEAR_IOSQ_DB;
- anv->ioq.cq_db = anv->mmio_nvme + APPLE_ANS_IOCQ_DB;
+ if (anv->hw->has_lsq_nvmmu) {
+ anv->adminq.sq_db = anv->mmio_nvme + APPLE_ANS_LINEAR_ASQ_DB;
+ anv->adminq.cq_db = anv->mmio_nvme + APPLE_ANS_ACQ_DB;
+ anv->ioq.sq_db = anv->mmio_nvme + APPLE_ANS_LINEAR_IOSQ_DB;
+ anv->ioq.cq_db = anv->mmio_nvme + APPLE_ANS_IOCQ_DB;
+ } else {
+ anv->adminq.sq_db = anv->mmio_nvme + NVME_REG_DBS;
+ anv->adminq.cq_db = anv->mmio_nvme + APPLE_ANS_ACQ_DB;
+ anv->ioq.sq_db = anv->mmio_nvme + NVME_REG_DBS + 8;
+ anv->ioq.cq_db = anv->mmio_nvme + APPLE_ANS_IOCQ_DB;
+ }
anv->sart = devm_apple_sart_get(dev);
if (IS_ERR(anv->sart)) {
@@ -1562,6 +1629,15 @@ static void apple_nvme_remove(struct platform_device *pdev)
nvme_stop_ctrl(&anv->ctrl);
nvme_remove_namespaces(&anv->ctrl);
apple_nvme_disable(anv, true);
+ if (anv->ctrl.admin_q && !blk_queue_dying(anv->ctrl.admin_q)) {
+ /*
+ * If the controller was reset during removal, it's possible
+ * user requests may be waiting on a stopped queue. Start the
+ * queue to flush these to completion.
+ */
+ nvme_unquiesce_admin_queue(&anv->ctrl);
+ blk_mq_destroy_queue(anv->ctrl.admin_q);
+ }
nvme_uninit_ctrl(&anv->ctrl);
if (apple_rtkit_is_running(anv->rtk))
@@ -1604,8 +1680,19 @@ static int apple_nvme_suspend(struct device *dev)
static DEFINE_SIMPLE_DEV_PM_OPS(apple_nvme_pm_ops, apple_nvme_suspend,
apple_nvme_resume);
+static const struct apple_nvme_hw apple_nvme_t8015_hw = {
+ .has_lsq_nvmmu = false,
+ .max_queue_depth = 16,
+};
+
+static const struct apple_nvme_hw apple_nvme_t8103_hw = {
+ .has_lsq_nvmmu = true,
+ .max_queue_depth = 64,
+};
+
static const struct of_device_id apple_nvme_of_match[] = {
- { .compatible = "apple,nvme-ans2" },
+ { .compatible = "apple,t8015-nvme-ans2", .data = &apple_nvme_t8015_hw },
+ { .compatible = "apple,nvme-ans2", .data = &apple_nvme_t8103_hw },
{},
};
MODULE_DEVICE_TABLE(of, apple_nvme_of_match);
diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index fdcd655ee743..78bba4b3cc93 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -1025,6 +1025,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
}
if (strlen(p) < 11 || strncmp(p, "DHHC-1:", 7)) {
pr_err("Invalid DH-CHAP secret %s\n", p);
+ kfree_sensitive(p);
ret = -EINVAL;
goto out;
}
@@ -1039,6 +1040,7 @@ static int nvmf_parse_options(struct nvmf_ctrl_options *opts,
}
if (strlen(p) < 11 || strncmp(p, "DHHC-1:", 7)) {
pr_err("Invalid DH-CHAP secret %s\n", p);
+ kfree_sensitive(p);
ret = -EINVAL;
goto out;
}
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 547763d859f9..fd0d99c0f97d 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2092,9 +2092,15 @@ __nvme_fc_init_request(struct nvme_fc_ctrl *ctrl,
dev_err(ctrl->dev,
"FCP Op failed - rspiu dma mapping failed.\n");
ret = -EFAULT;
+ goto out_unmap;
}
atomic_set(&op->state, FCPOP_STATE_IDLE);
+ return 0;
+
+out_unmap:
+ fc_dma_unmap_single(ctrl->lport->dev, op->fcp_req.cmddma,
+ sizeof(op->cmd_iu), DMA_TO_DEVICE);
out_on_error:
return ret;
}
@@ -2350,9 +2356,15 @@ nvme_fc_ctrl_free(struct kref *ref)
struct nvme_fc_ctrl *ctrl =
container_of(ref, struct nvme_fc_ctrl, ref);
unsigned long flags;
+ bool owns_opts;
- /* remove from rport list */
+ /*
+ * Presence on the rport list means nvme_fc_init_ctrl() completed,
+ * and with it ownership of the fabrics options passed to it. If it
+ * failed instead, the options still belong to nvmf_create_ctrl().
+ */
spin_lock_irqsave(&ctrl->rport->lock, flags);
+ owns_opts = !list_empty(&ctrl->ctrl_list);
list_del(&ctrl->ctrl_list);
spin_unlock_irqrestore(&ctrl->rport->lock, flags);
@@ -2362,7 +2374,7 @@ nvme_fc_ctrl_free(struct kref *ref)
nvme_fc_rport_put(ctrl->rport);
ida_free(&nvme_fc_ctrl_cnt, ctrl->cnum);
- if (ctrl->ctrl.opts)
+ if (owns_opts)
nvmf_free_options(ctrl->ctrl.opts);
kfree(ctrl);
}
@@ -3552,14 +3564,14 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING)) {
dev_err(ctrl->ctrl.device,
"NVME-FC{%d}: failed to init ctrl state\n", ctrl->cnum);
- goto fail_ctrl;
+ goto fail_unlist;
}
if (!queue_delayed_work(nvme_wq, &ctrl->connect_work, 0)) {
dev_err(ctrl->ctrl.device,
"NVME-FC{%d}: failed to schedule initial connect\n",
ctrl->cnum);
- goto fail_ctrl;
+ goto fail_unlist;
}
flush_delayed_work(&ctrl->connect_work);
@@ -3570,14 +3582,22 @@ nvme_fc_init_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
return &ctrl->ctrl;
+fail_unlist:
+ /*
+ * Leaving the list hands the options back to nvmf_create_ctrl();
+ * see nvme_fc_ctrl_free(). Re-init so that list_empty() there
+ * reports the controller as unlisted.
+ */
+ spin_lock_irqsave(&rport->lock, flags);
+ list_del_init(&ctrl->ctrl_list);
+ spin_unlock_irqrestore(&rport->lock, flags);
+
fail_ctrl:
nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING);
cancel_work_sync(&ctrl->ioerr_work);
cancel_work_sync(&ctrl->ctrl.reset_work);
cancel_delayed_work_sync(&ctrl->connect_work);
- ctrl->ctrl.opts = NULL;
-
if (ctrl->ctrl.admin_tagset)
nvme_remove_admin_tag_set(&ctrl->ctrl);
/* initiate nvme ctrl ref counting teardown */
diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c
index 930521c633d2..d3564b2eebc9 100644
--- a/drivers/nvme/host/ioctl.c
+++ b/drivers/nvme/host/ioctl.c
@@ -14,45 +14,54 @@ enum {
NVME_IOCTL_PARTITION = (1 << 1),
};
-static bool nvme_cmd_allowed(struct nvme_ns *ns, struct nvme_command *c,
- unsigned int flags, bool open_for_write)
+static bool nvme_admin_cmd_allowed(struct nvme_ctrl *ctrl,
+ struct nvme_command *c)
{
- u32 effects;
-
- /*
- * Do not allow unprivileged passthrough on partitions, as that allows an
- * escape from the containment of the partition.
- */
- if (flags & NVME_IOCTL_PARTITION)
- goto admin;
-
- /*
- * Do not allow unprivileged processes to send vendor specific or fabrics
- * commands as we can't be sure about their effects.
- */
- if (c->common.opcode >= nvme_cmd_vendor_start ||
- c->common.opcode == nvme_fabrics_command)
- goto admin;
-
/*
* Do not allow unprivileged passthrough of admin commands except
* for a subset of identify commands that contain information required
* to form proper I/O commands in userspace and do not expose any
* potentially sensitive information.
*/
- if (!ns) {
- if (c->common.opcode == nvme_admin_identify) {
- switch (c->identify.cns) {
- case NVME_ID_CNS_NS:
- case NVME_ID_CNS_CS_NS:
- case NVME_ID_CNS_NS_CS_INDEP:
- case NVME_ID_CNS_CS_CTRL:
- case NVME_ID_CNS_CTRL:
- return true;
- }
+ switch (c->common.opcode) {
+ case nvme_admin_identify:
+ switch (c->identify.cns) {
+ case NVME_ID_CNS_NS:
+ case NVME_ID_CNS_CS_NS:
+ case NVME_ID_CNS_NS_CS_INDEP:
+ case NVME_ID_CNS_CS_CTRL:
+ case NVME_ID_CNS_CTRL:
+ return true;
+ }
+ break;
+ case nvme_admin_set_features:
+ /*
+ * Reject Set Features that change controller state the driver
+ * manages itself; setting them behind the driver's back from
+ * userspace leaves it unable to react correctly. Keep Alive is
+ * only armed for fabrics - on other transports it has no
+ * reserved tag and harms idle power states.
+ */
+ switch (le32_to_cpu(c->features.fid) & 0xff) {
+ case NVME_FEAT_KATO:
+ if (ctrl->ops->flags & NVME_F_FABRICS)
+ break;
+ fallthrough;
+ case NVME_FEAT_HOST_BEHAVIOR:
+ case NVME_FEAT_HOST_MEM_BUF:
+ case NVME_FEAT_NUM_QUEUES:
+ case NVME_FEAT_AUTO_PST:
+ return false;
}
- goto admin;
+ break;
}
+ return capable(CAP_SYS_ADMIN);
+}
+
+static bool nvme_ns_cmd_allowed(struct nvme_ns *ns, struct nvme_command *c,
+ bool open_for_write)
+{
+ u32 effects;
/*
* Check if the controller provides a Commands Supported and Effects log
@@ -61,7 +70,7 @@ static bool nvme_cmd_allowed(struct nvme_ns *ns, struct nvme_command *c,
*/
effects = nvme_command_effects(ns->ctrl, ns, c->common.opcode);
if (!(effects & NVME_CMD_EFFECTS_CSUPP))
- goto admin;
+ return capable(CAP_SYS_ADMIN);
/*
* Don't allow passthrough for command that have intrusive (or unknown)
@@ -70,7 +79,7 @@ static bool nvme_cmd_allowed(struct nvme_ns *ns, struct nvme_command *c,
if (effects & ~(NVME_CMD_EFFECTS_CSUPP | NVME_CMD_EFFECTS_LBCC |
NVME_CMD_EFFECTS_UUID_SEL |
NVME_CMD_EFFECTS_SCOPE_MASK))
- goto admin;
+ return capable(CAP_SYS_ADMIN);
/*
* Only allow I/O commands that transfer data to the controller or that
@@ -79,11 +88,34 @@ static bool nvme_cmd_allowed(struct nvme_ns *ns, struct nvme_command *c,
*/
if ((nvme_is_write(c) || (effects & NVME_CMD_EFFECTS_LBCC)) &&
!open_for_write)
- goto admin;
+ return capable(CAP_SYS_ADMIN);
return true;
-admin:
- return capable(CAP_SYS_ADMIN);
+}
+
+static bool nvme_cmd_allowed(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
+ struct nvme_command *c, unsigned int flags,
+ bool open_for_write)
+{
+ /*
+ * Do not allow unprivileged passthrough on partitions, as that
+ * allows an escape from the containment of the partition.
+ */
+ if (flags & NVME_IOCTL_PARTITION)
+ return capable(CAP_SYS_ADMIN);
+
+ /*
+ * Do not allow unprivileged processes to send vendor specific or
+ * fabrics commands as we can't be sure about their effects.
+ */
+ if (c->common.opcode >= nvme_cmd_vendor_start ||
+ c->common.opcode == nvme_fabrics_command)
+ return capable(CAP_SYS_ADMIN);
+
+ if (!ns)
+ return nvme_admin_cmd_allowed(ctrl, c);
+
+ return nvme_ns_cmd_allowed(ns, c, open_for_write);
}
/*
@@ -220,7 +252,8 @@ static int nvme_submit_user_cmd(struct request_queue *q,
return ret;
}
-static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
+static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio,
+ unsigned int flags, bool open_for_write)
{
struct nvme_user_io io;
struct nvme_command c;
@@ -278,6 +311,9 @@ static int nvme_submit_io(struct nvme_ns *ns, struct nvme_user_io __user *uio)
c.rw.lbat = cpu_to_le16(io.apptag);
c.rw.lbatm = cpu_to_le16(io.appmask);
+ if (!nvme_cmd_allowed(ns->ctrl, ns, &c, flags, open_for_write))
+ return -EACCES;
+
return nvme_submit_user_cmd(ns->queue, &c, io.addr, length, metadata,
meta_len, NULL, 0, 0);
}
@@ -325,7 +361,7 @@ static int nvme_user_cmd(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
c.common.cdw14 = cpu_to_le32(cmd.cdw14);
c.common.cdw15 = cpu_to_le32(cmd.cdw15);
- if (!nvme_cmd_allowed(ns, &c, 0, open_for_write))
+ if (!nvme_cmd_allowed(ctrl, ns, &c, 0, open_for_write))
return -EACCES;
if (cmd.timeout_ms)
@@ -372,7 +408,7 @@ static int nvme_user_cmd64(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
c.common.cdw14 = cpu_to_le32(cmd.cdw14);
c.common.cdw15 = cpu_to_le32(cmd.cdw15);
- if (!nvme_cmd_allowed(ns, &c, flags, open_for_write))
+ if (!nvme_cmd_allowed(ctrl, ns, &c, flags, open_for_write))
return -EACCES;
if (cmd.timeout_ms)
@@ -458,6 +494,7 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd);
const struct nvme_uring_cmd *cmd = io_uring_sqe_cmd(ioucmd->sqe);
struct request_queue *q = ns ? ns->queue : ctrl->admin_q;
+ bool open_for_write = ioucmd->file->f_mode & FMODE_WRITE;
struct nvme_uring_data d;
struct nvme_command c;
struct request *req;
@@ -486,7 +523,7 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns,
c.common.cdw14 = cpu_to_le32(READ_ONCE(cmd->cdw14));
c.common.cdw15 = cpu_to_le32(READ_ONCE(cmd->cdw15));
- if (!nvme_cmd_allowed(ns, &c, 0, ioucmd->file->f_mode & FMODE_WRITE))
+ if (!nvme_cmd_allowed(ctrl, ns, &c, 0, open_for_write))
return -EACCES;
d.metadata = READ_ONCE(cmd->metadata);
@@ -582,7 +619,7 @@ static int nvme_ns_ioctl(struct nvme_ns *ns, unsigned int cmd,
case NVME_IOCTL_SUBMIT_IO32:
#endif
case NVME_IOCTL_SUBMIT_IO:
- return nvme_submit_io(ns, argp);
+ return nvme_submit_io(ns, argp, flags, open_for_write);
case NVME_IOCTL_IO64_CMD_VEC:
flags |= NVME_IOCTL_VEC;
fallthrough;
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 8b5cc155d8f2..545704f1efa6 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -684,6 +684,13 @@ static int nvme_tcp_handle_c2h_data(struct nvme_tcp_queue *queue,
return -ENOENT;
}
+ if (rq_data_dir(rq) != READ) {
+ dev_err(queue->ctrl->ctrl.device,
+ "queue %d tag %#x unexpected data for a write\n",
+ nvme_tcp_queue_id(queue), rq->tag);
+ return -EIO;
+ }
+
req = blk_mq_rq_to_pdu(rq);
if (!blk_rq_payload_bytes(rq) || !req->curr_bio || !req->data_len) {
dev_err(queue->ctrl->ctrl.device,
diff --git a/drivers/nvme/target/auth.c b/drivers/nvme/target/auth.c
index b47d675232d2..38d1ced21f53 100644
--- a/drivers/nvme/target/auth.c
+++ b/drivers/nvme/target/auth.c
@@ -241,6 +241,12 @@ void nvmet_auth_sq_free(struct nvmet_sq *sq)
sq->dhchap_skey = NULL;
}
+void nvmet_auth_sq_destroy(struct nvmet_sq *sq)
+{
+ cancel_delayed_work_sync(&sq->auth_expired_work);
+ nvmet_auth_sq_free(sq);
+}
+
void nvmet_destroy_auth(struct nvmet_ctrl *ctrl)
{
ctrl->shash_id = 0;
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index ce85b5783260..6267c6d576db 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -812,7 +812,7 @@ void nvmet_sq_destroy(struct nvmet_sq *sq)
wait_for_completion(&sq->confirm_done);
wait_for_completion(&sq->free_done);
percpu_ref_exit(&sq->ref);
- nvmet_auth_sq_free(sq);
+ nvmet_auth_sq_destroy(sq);
/*
* we must reference the ctrl again after waiting for inflight IO
diff --git a/drivers/nvme/target/fabrics-cmd-auth.c b/drivers/nvme/target/fabrics-cmd-auth.c
index 34924c4c73b9..4d7b5ef643eb 100644
--- a/drivers/nvme/target/fabrics-cmd-auth.c
+++ b/drivers/nvme/target/fabrics-cmd-auth.c
@@ -31,12 +31,16 @@ void nvmet_auth_sq_init(struct nvmet_sq *sq)
sq->dhchap_step = NVME_AUTH_DHCHAP_MESSAGE_NEGOTIATE;
}
-static u8 nvmet_auth_negotiate(struct nvmet_req *req, void *d)
+static u8 nvmet_auth_negotiate(struct nvmet_req *req, void *d, u32 tl)
{
struct nvmet_ctrl *ctrl = req->sq->ctrl;
struct nvmf_auth_dhchap_negotiate_data *data = d;
int i, hash_id = 0, fallback_hash_id = 0, dhgid, fallback_dhgid;
+ if (tl < sizeof(*data) +
+ sizeof(struct nvmf_auth_dhchap_protocol_descriptor))
+ return NVME_AUTH_DHCHAP_FAILURE_INCORRECT_PAYLOAD;
+
pr_debug("%s: ctrl %d qid %d: data sc_d %d napd %d authid %d halen %d dhlen %d\n",
__func__, ctrl->cntlid, req->sq->qid,
data->sc_c, data->napd, data->auth_protocol[0].dhchap.authid,
@@ -53,6 +57,10 @@ static u8 nvmet_auth_negotiate(struct nvmet_req *req, void *d)
NVME_AUTH_DHCHAP_AUTH_ID)
return NVME_AUTH_DHCHAP_FAILURE_INCORRECT_PAYLOAD;
+ if (data->auth_protocol[0].dhchap.dhlen > NVME_AUTH_DHCHAP_MAX_DH_IDS ||
+ data->auth_protocol[0].dhchap.halen > NVME_AUTH_DHCHAP_MAX_HASH_IDS)
+ return NVME_AUTH_DHCHAP_FAILURE_INCORRECT_PAYLOAD;
+
for (i = 0; i < data->auth_protocol[0].dhchap.halen; i++) {
u8 host_hmac_id = data->auth_protocol[0].dhchap.idlist[i];
@@ -270,7 +278,7 @@ void nvmet_execute_auth_send(struct nvmet_req *req)
} else if (data->auth_id != req->sq->dhchap_step)
goto done_failure1;
/* Validate negotiation parameters */
- dhchap_status = nvmet_auth_negotiate(req, d);
+ dhchap_status = nvmet_auth_negotiate(req, d, tl);
if (dhchap_status == 0)
req->sq->dhchap_step =
NVME_AUTH_DHCHAP_MESSAGE_CHALLENGE;
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 17a63bc7d80c..25bba2d09762 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -737,6 +737,7 @@ u8 nvmet_setup_auth(struct nvmet_ctrl *ctrl);
void nvmet_auth_sq_init(struct nvmet_sq *sq);
void nvmet_destroy_auth(struct nvmet_ctrl *ctrl);
void nvmet_auth_sq_free(struct nvmet_sq *sq);
+void nvmet_auth_sq_destroy(struct nvmet_sq *sq);
int nvmet_setup_dhgroup(struct nvmet_ctrl *ctrl, u8 dhgroup_id);
bool nvmet_check_auth_status(struct nvmet_req *req);
int nvmet_auth_host_hash(struct nvmet_req *req, u8 *response,
@@ -761,6 +762,7 @@ static inline void nvmet_auth_sq_init(struct nvmet_sq *sq)
}
static inline void nvmet_destroy_auth(struct nvmet_ctrl *ctrl) {};
static inline void nvmet_auth_sq_free(struct nvmet_sq *sq) {};
+static inline void nvmet_auth_sq_destroy(struct nvmet_sq *sq) {};
static inline bool nvmet_check_auth_status(struct nvmet_req *req)
{
return true;
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index eb971feece0f..36eb26c5a05d 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -658,18 +658,25 @@ static void nvmet_rdma_rw_ctx_destroy(struct nvmet_rdma_rsp *rsp)
req->sg, req->sg_cnt, nvmet_data_dir(req));
}
-static void nvmet_rdma_release_rsp(struct nvmet_rdma_rsp *rsp)
+static void nvmet_rdma_free_rsp_resources(struct nvmet_rdma_rsp *rsp)
{
struct nvmet_rdma_queue *queue = rsp->queue;
- atomic_add(1 + rsp->n_rdma, &queue->sq_wr_avail);
-
if (rsp->n_rdma)
nvmet_rdma_rw_ctx_destroy(rsp);
if (rsp->req.sg < rsp->cmd->inline_sg ||
rsp->req.sg >= rsp->cmd->inline_sg + queue->dev->inline_page_count)
nvmet_req_free_sgls(&rsp->req);
+}
+
+static void nvmet_rdma_release_rsp(struct nvmet_rdma_rsp *rsp)
+{
+ struct nvmet_rdma_queue *queue = rsp->queue;
+
+ atomic_add(1 + rsp->n_rdma, &queue->sq_wr_avail);
+
+ nvmet_rdma_free_rsp_resources(rsp);
if (unlikely(!list_empty_careful(&queue->rsp_wr_wait_list)))
nvmet_rdma_process_wr_wait_list(queue);
@@ -1340,9 +1347,27 @@ static int nvmet_rdma_create_queue_ib(struct nvmet_rdma_queue *queue)
goto out;
}
+static bool nvmet_rdma_reclaim_rsp(struct sbitmap *sb, unsigned int bitnr,
+ void *data)
+{
+ struct nvmet_rdma_queue *queue = data;
+
+ nvmet_rdma_free_rsp_resources(&queue->rsps[bitnr]);
+
+ return true;
+}
+
static void nvmet_rdma_destroy_queue_ib(struct nvmet_rdma_queue *queue)
{
ib_drain_qp(queue->qp);
+
+ /*
+ * Reclaim resources of a response that is still in-flight when the
+ * queue is being torn down. This happens when the connection was
+ * forcefully disconnected while an I/O is in flight.
+ */
+ sbitmap_for_each_set(&queue->rsp_tags, nvmet_rdma_reclaim_rsp, queue);
+
if (queue->cm_id)
rdma_destroy_id(queue->cm_id);
ib_destroy_qp(queue->qp);
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 01f4ef36383b..5509737bb4cb 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -104,6 +104,7 @@ enum nvmet_tcp_recv_state {
enum {
NVMET_TCP_F_INIT_FAILED = (1 << 0),
+ NVMET_TCP_F_R2T_SENT = (1 << 1),
};
struct nvmet_tcp_cmd {
@@ -785,6 +786,7 @@ static int nvmet_try_send_r2t(struct nvmet_tcp_cmd *cmd, bool last_in_batch)
return -EAGAIN;
cmd->queue->snd_cmd = NULL;
+ cmd->flags |= NVMET_TCP_F_R2T_SENT;
return 1;
}
@@ -1061,6 +1063,12 @@ static int nvmet_tcp_handle_h2c_data_pdu(struct nvmet_tcp_queue *queue)
cmd = &queue->connect;
}
+ if (unlikely(!(cmd->flags & NVMET_TCP_F_R2T_SENT))) {
+ pr_err("queue %d: unsolicited H2CData (ttag %u)\n",
+ queue->idx, data->ttag);
+ goto err_proto;
+ }
+
if (le32_to_cpu(data->data_offset) != cmd->rbytes_done) {
pr_err("ttag %u unexpected data offset %u (expected %u)\n",
data->ttag, le32_to_cpu(data->data_offset),
@@ -1301,6 +1309,8 @@ static int nvmet_tcp_try_recv_pdu(struct nvmet_tcp_queue *queue)
}
queue->left = hdr->hlen - queue->offset + hdgst;
+ if (queue->left > sizeof(queue->pdu) - queue->offset)
+ return -EPROTO;
goto recv;
}
diff --git a/drivers/nvme/target/zns.c b/drivers/nvme/target/zns.c
index af9e13be7678..40f261a4927c 100644
--- a/drivers/nvme/target/zns.c
+++ b/drivers/nvme/target/zns.c
@@ -120,7 +120,7 @@ void nvmet_execute_identify_ns_zns(struct nvmet_req *req)
mutex_unlock(&req->ns->subsys->lock);
}
- if (!bdev_is_zoned(req->ns->bdev)) {
+ if (!req->ns->bdev || !bdev_is_zoned(req->ns->bdev)) {
status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
req->error_loc = offsetof(struct nvme_identify, nsid);
goto out;
diff --git a/drivers/of/property.c b/drivers/of/property.c
index 906a33ae717f..0b1d28b249bd 100644
--- a/drivers/of/property.c
+++ b/drivers/of/property.c
@@ -630,6 +630,70 @@ struct device_node *of_graph_get_port_by_id(struct device_node *parent, u32 id)
}
EXPORT_SYMBOL(of_graph_get_port_by_id);
+/**
+ * of_graph_get_next_port() - get next port node.
+ * @parent: pointer to the parent device node, or parent ports node
+ * @prev: previous port node, or NULL to get first
+ *
+ * Parent device node can be used as @parent whether device node has ports node
+ * or not. It will work same as ports@0 node.
+ *
+ * Return: A 'port' node pointer with refcount incremented. Refcount
+ * of the passed @prev node is decremented.
+ */
+struct device_node *of_graph_get_next_port(const struct device_node *parent,
+ struct device_node *prev)
+{
+ if (!parent)
+ return NULL;
+
+ if (!prev) {
+ struct device_node *node __free(device_node) =
+ of_get_child_by_name(parent, "ports");
+
+ if (node)
+ parent = node;
+
+ return of_get_child_by_name(parent, "port");
+ }
+
+ do {
+ prev = of_get_next_child(parent, prev);
+ if (!prev)
+ break;
+ } while (!of_node_name_eq(prev, "port"));
+
+ return prev;
+}
+EXPORT_SYMBOL(of_graph_get_next_port);
+
+/**
+ * of_graph_get_next_port_endpoint() - get next endpoint node in port.
+ * If it reached to end of the port, it will return NULL.
+ * @port: pointer to the target port node
+ * @prev: previous endpoint node, or NULL to get first
+ *
+ * Return: An 'endpoint' node pointer with refcount incremented. Refcount
+ * of the passed @prev node is decremented.
+ */
+struct device_node *of_graph_get_next_port_endpoint(const struct device_node *port,
+ struct device_node *prev)
+{
+ while (1) {
+ prev = of_get_next_child(port, prev);
+ if (!prev)
+ break;
+ if (WARN(!of_node_name_eq(prev, "endpoint"),
+ "non endpoint node is used (%pOF)", prev))
+ continue;
+
+ break;
+ }
+
+ return prev;
+}
+EXPORT_SYMBOL(of_graph_get_next_port_endpoint);
+
/**
* of_graph_get_next_endpoint() - get next endpoint node
* @parent: pointer to the parent device node
@@ -823,6 +887,23 @@ unsigned int of_graph_get_endpoint_count(const struct device_node *np)
}
EXPORT_SYMBOL(of_graph_get_endpoint_count);
+/**
+ * of_graph_get_port_count() - get the number of port in a device or ports node
+ * @np: pointer to the device or ports node
+ *
+ * Return: count of port of this device or ports node
+ */
+unsigned int of_graph_get_port_count(struct device_node *np)
+{
+ unsigned int num = 0;
+
+ for_each_of_graph_port(np, port)
+ num++;
+
+ return num;
+}
+EXPORT_SYMBOL(of_graph_get_port_count);
+
/**
* of_graph_get_remote_node() - get remote parent device_node for given port/endpoint
* @node: pointer to parent device_node containing graph port/endpoint
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 96d2e7b63db3..bed3f7538aa9 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -1871,7 +1871,6 @@ static void attach_node_and_children(struct device_node *np)
*/
static int __init unittest_data_add(void)
{
- void *unittest_data;
void *unittest_data_align;
struct device_node *unittest_data_node = NULL, *np;
/*
@@ -1890,7 +1889,7 @@ static int __init unittest_data_add(void)
}
/* creating copy */
- unittest_data = kmalloc(size + FDT_ALIGN_SIZE, GFP_KERNEL);
+ void *unittest_data __free(kfree) = kmalloc(size + FDT_ALIGN_SIZE, GFP_KERNEL);
if (!unittest_data)
return -ENOMEM;
@@ -1900,12 +1899,10 @@ static int __init unittest_data_add(void)
ret = of_fdt_unflatten_tree(unittest_data_align, NULL, &unittest_data_node);
if (!ret) {
pr_warn("%s: unflatten testcases tree failed\n", __func__);
- kfree(unittest_data);
return -ENODATA;
}
if (!unittest_data_node) {
pr_warn("%s: testcases tree is empty\n", __func__);
- kfree(unittest_data);
return -ENODATA;
}
@@ -1924,7 +1921,6 @@ static int __init unittest_data_add(void)
/* attach the sub-tree to live tree */
if (!of_root) {
pr_warn("%s: no live tree to attach sub-tree\n", __func__);
- kfree(unittest_data);
rc = -ENODEV;
goto unlock;
}
@@ -1945,6 +1941,8 @@ static int __init unittest_data_add(void)
EXPECT_END(KERN_INFO,
"Duplicate name in testcase-data, renamed to \"duplicate-name#1\"");
+ retain_and_null_ptr(unittest_data);
+
unlock:
of_overlay_mutex_unlock();
diff --git a/drivers/parisc/eisa.c b/drivers/parisc/eisa.c
index 9eab974e6baf..b5b5413bfd1e 100644
--- a/drivers/parisc/eisa.c
+++ b/drivers/parisc/eisa.c
@@ -442,6 +442,11 @@ static int __init eisa_irq_setup(char *str)
val = (int) simple_strtoul(cur, &pe, 0);
if (val > 15 || val < 0) {
printk(KERN_ERR "eisa: EISA irq value are 0-15\n");
+ cur = strchr(cur, ',');
+ if (cur)
+ cur++;
+ else
+ break;
continue;
}
if (val == 2) {
diff --git a/drivers/pci/controller/cadence/pci-j721e.c b/drivers/pci/controller/cadence/pci-j721e.c
index c014220d2b24..e2a522d0f6c1 100644
--- a/drivers/pci/controller/cadence/pci-j721e.c
+++ b/drivers/pci/controller/cadence/pci-j721e.c
@@ -374,7 +374,7 @@ static const struct j721e_pcie_data j7200_pcie_rc_data = {
.quirk_detect_quiet_flag = true,
.linkdown_irq_regfield = J7200_LINK_DOWN,
.byte_access_allowed = true,
- .max_lanes = 2,
+ .max_lanes = 4,
};
static const struct j721e_pcie_data j7200_pcie_ep_data = {
@@ -382,7 +382,7 @@ static const struct j721e_pcie_data j7200_pcie_ep_data = {
.quirk_detect_quiet_flag = true,
.linkdown_irq_regfield = J7200_LINK_DOWN,
.quirk_disable_flr = true,
- .max_lanes = 2,
+ .max_lanes = 4,
};
static const struct j721e_pcie_data am64_pcie_rc_data = {
diff --git a/drivers/pci/controller/pci-xgene.c b/drivers/pci/controller/pci-xgene.c
index 1e2ebbfa36d1..7028d4b6abe6 100644
--- a/drivers/pci/controller/pci-xgene.c
+++ b/drivers/pci/controller/pci-xgene.c
@@ -53,13 +53,11 @@
#define XGENE_V1_PCI_EXP_CAP 0x40
/* PCIe IP version */
-#define XGENE_PCIE_IP_VER_UNKN 0
#define XGENE_PCIE_IP_VER_1 1
#define XGENE_PCIE_IP_VER_2 2
#if defined(CONFIG_PCI_XGENE) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
struct xgene_pcie {
- struct device_node *node;
struct device *dev;
struct clk *clk;
void __iomem *csr_base;
@@ -529,7 +527,7 @@ static void xgene_pcie_setup_ib_reg(struct xgene_pcie *port,
static int xgene_pcie_parse_map_dma_ranges(struct xgene_pcie *port)
{
- struct device_node *np = port->node;
+ struct device_node *np = port->dev->of_node;
struct of_pci_range range;
struct of_pci_range_parser parser;
struct device *dev = port->dev;
@@ -597,7 +595,6 @@ static struct pci_ops xgene_pcie_ops = {
static int xgene_pcie_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- struct device_node *dn = dev->of_node;
struct xgene_pcie *port;
struct pci_host_bridge *bridge;
int ret;
@@ -608,12 +605,8 @@ static int xgene_pcie_probe(struct platform_device *pdev)
port = pci_host_bridge_priv(bridge);
- port->node = of_node_get(dn);
port->dev = dev;
-
- port->version = XGENE_PCIE_IP_VER_UNKN;
- if (of_device_is_compatible(port->node, "apm,xgene-pcie"))
- port->version = XGENE_PCIE_IP_VER_1;
+ port->version = XGENE_PCIE_IP_VER_1;
ret = xgene_pcie_map_reg(port, pdev);
if (ret)
diff --git a/drivers/pci/controller/plda/pcie-starfive.c b/drivers/pci/controller/plda/pcie-starfive.c
index 0a0b5a7d84d7..aeaebe9cd360 100644
--- a/drivers/pci/controller/plda/pcie-starfive.c
+++ b/drivers/pci/controller/plda/pcie-starfive.c
@@ -405,7 +405,11 @@ static int starfive_pcie_probe(struct platform_device *pdev)
return ret;
pm_runtime_enable(&pdev->dev);
- pm_runtime_get_sync(&pdev->dev);
+ ret = pm_runtime_resume_and_get(&pdev->dev);
+ if (ret < 0) {
+ pm_runtime_disable(&pdev->dev);
+ return dev_err_probe(dev, ret, "failed to resume device\n");
+ }
plda->host_ops = &sf_host_ops;
plda->num_events = PLDA_MAX_EVENT_NUM;
@@ -431,9 +435,9 @@ static void starfive_pcie_remove(struct platform_device *pdev)
{
struct starfive_jh7110_pcie *pcie = platform_get_drvdata(pdev);
- pm_runtime_put(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
plda_pcie_host_deinit(&pcie->plda);
+ pm_runtime_put_sync(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
platform_set_drvdata(pdev, NULL);
}
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index b746038b59e7..fc3f526f76a2 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -798,7 +798,7 @@ static void aspm_l1ss_init(struct pcie_link_state *link)
static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
{
struct pci_dev *child = link->downstream, *parent = link->pdev;
- u32 parent_lnkcap, child_lnkcap;
+ struct pci_dev *fn;
u16 parent_lnkctl, child_lnkctl;
struct pci_bus *linkbus = parent->subordinate;
@@ -813,9 +813,8 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
* If ASPM not supported, don't mess with the clocks and link,
* bail out now.
*/
- pcie_capability_read_dword(parent, PCI_EXP_LNKCAP, &parent_lnkcap);
- pcie_capability_read_dword(child, PCI_EXP_LNKCAP, &child_lnkcap);
- if (!(parent_lnkcap & child_lnkcap & PCI_EXP_LNKCAP_ASPMS))
+ if (!(parent->aspm_l0s_support && child->aspm_l0s_support) &&
+ !(parent->aspm_l1_support && child->aspm_l1_support))
return;
/* Configure common clock before checking latencies */
@@ -827,18 +826,17 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
* read-only Link Capabilities may change depending on common clock
* configuration (PCIe r5.0, sec 7.5.3.6).
*/
- pcie_capability_read_dword(parent, PCI_EXP_LNKCAP, &parent_lnkcap);
- pcie_capability_read_dword(child, PCI_EXP_LNKCAP, &child_lnkcap);
pcie_capability_read_word(parent, PCI_EXP_LNKCTL, &parent_lnkctl);
pcie_capability_read_word(child, PCI_EXP_LNKCTL, &child_lnkctl);
/* Disable L0s/L1 before updating L1SS config */
if (FIELD_GET(PCI_EXP_LNKCTL_ASPMC, child_lnkctl) ||
FIELD_GET(PCI_EXP_LNKCTL_ASPMC, parent_lnkctl)) {
- pcie_capability_write_word(child, PCI_EXP_LNKCTL,
- child_lnkctl & ~PCI_EXP_LNKCTL_ASPMC);
- pcie_capability_write_word(parent, PCI_EXP_LNKCTL,
- parent_lnkctl & ~PCI_EXP_LNKCTL_ASPMC);
+ list_for_each_entry(fn, &linkbus->devices, bus_list)
+ pcie_capability_clear_and_set_word(fn, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_ASPMC, 0);
+ pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_ASPMC, 0);
}
/*
@@ -848,7 +846,7 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
* given link unless components on both sides of the link each
* support L0s.
*/
- if (parent_lnkcap & child_lnkcap & PCI_EXP_LNKCAP_ASPM_L0S)
+ if (parent->aspm_l0s_support && child->aspm_l0s_support)
link->aspm_support |= PCIE_LINK_STATE_L0S;
if (child_lnkctl & PCI_EXP_LNKCTL_ASPM_L0S)
@@ -857,7 +855,7 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
link->aspm_enabled |= PCIE_LINK_STATE_L0S_DW;
/* Setup L1 state */
- if (parent_lnkcap & child_lnkcap & PCI_EXP_LNKCAP_ASPM_L1)
+ if (parent->aspm_l1_support && child->aspm_l1_support)
link->aspm_support |= PCIE_LINK_STATE_L1;
if (parent_lnkctl & child_lnkctl & PCI_EXP_LNKCTL_ASPM_L1)
@@ -868,8 +866,13 @@ static void pcie_aspm_cap_init(struct pcie_link_state *link, int blacklist)
/* Restore L0s/L1 if they were enabled */
if (FIELD_GET(PCI_EXP_LNKCTL_ASPMC, child_lnkctl) ||
FIELD_GET(PCI_EXP_LNKCTL_ASPMC, parent_lnkctl)) {
- pcie_capability_write_word(parent, PCI_EXP_LNKCTL, parent_lnkctl);
- pcie_capability_write_word(child, PCI_EXP_LNKCTL, child_lnkctl);
+ pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_ASPMC,
+ parent_lnkctl & PCI_EXP_LNKCTL_ASPMC);
+ list_for_each_entry(fn, &linkbus->devices, bus_list)
+ pcie_capability_clear_and_set_word(fn, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_ASPMC,
+ child_lnkctl & PCI_EXP_LNKCTL_ASPMC);
}
/* Save default state */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index d8c5a957b70e..718293b4bdb9 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1589,6 +1589,13 @@ void set_pcie_port_type(struct pci_dev *pdev)
if (reg32 & PCI_EXP_LNKCAP_DLLLARC)
pdev->link_active_reporting = 1;
+#ifdef CONFIG_PCIEASPM
+ if (reg32 & PCI_EXP_LNKCAP_ASPM_L0S)
+ pdev->aspm_l0s_support = 1;
+ if (reg32 & PCI_EXP_LNKCAP_ASPM_L1)
+ pdev->aspm_l1_support = 1;
+#endif
+
parent = pci_upstream_bridge(pdev);
if (!parent)
return;
diff --git a/drivers/perf/arm_spe_pmu.c b/drivers/perf/arm_spe_pmu.c
index 2fce871a1882..5902ca3f057a 100644
--- a/drivers/perf/arm_spe_pmu.c
+++ b/drivers/perf/arm_spe_pmu.c
@@ -467,7 +467,7 @@ static u64 __arm_spe_pmu_next_off(struct perf_output_handle *handle)
* the page boundary following it. Keep the tail boundary if
* that's lower.
*/
- if (handle->wakeup < (handle->head + handle->size) && head <= wakeup)
+ if ((handle->wakeup - handle->head) < handle->size && head <= wakeup)
limit = min(limit, round_up(wakeup, PAGE_SIZE));
if (limit > head)
diff --git a/drivers/perf/cxl_pmu.c b/drivers/perf/cxl_pmu.c
index 24244096f514..ce97062a6619 100644
--- a/drivers/perf/cxl_pmu.c
+++ b/drivers/perf/cxl_pmu.c
@@ -632,7 +632,7 @@ static void cxl_pmu_event_start(struct perf_event *event, int flags)
cfg = cxl_pmu_config2_get_hdm_decoder(event);
else
cfg = GENMASK(31, 0); /* No filtering if 0xFFFF_FFFF */
- writeq(cfg, base + CXL_PMU_FILTER_CFG_REG(hwc->idx, 0));
+ writel(cfg, base + CXL_PMU_FILTER_CFG_REG(hwc->idx, 0));
}
cfg = readq(base + CXL_PMU_COUNTER_CFG_REG(hwc->idx));
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-combo.c b/drivers/phy/qualcomm/phy-qcom-qmp-combo.c
index 643045c9024e..535dffd3d7b0 100644
--- a/drivers/phy/qualcomm/phy-qcom-qmp-combo.c
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-combo.c
@@ -28,10 +28,16 @@
#include "phy-qcom-qmp-common.h"
#include "phy-qcom-qmp.h"
+#include "phy-qcom-qmp-pcs-aon-v6.h"
+#include "phy-qcom-qmp-pcs-aon-v8.h"
#include "phy-qcom-qmp-pcs-misc-v3.h"
+#include "phy-qcom-qmp-pcs-misc-v4.h"
+#include "phy-qcom-qmp-pcs-misc-v5.h"
+#include "phy-qcom-qmp-pcs-misc-v8.h"
#include "phy-qcom-qmp-pcs-usb-v4.h"
#include "phy-qcom-qmp-pcs-usb-v5.h"
#include "phy-qcom-qmp-pcs-usb-v6.h"
+#include "phy-qcom-qmp-pcs-usb-v8.h"
#include "phy-qcom-qmp-dp-com-v3.h"
@@ -40,6 +46,9 @@
#include "phy-qcom-qmp-dp-phy-v4.h"
#include "phy-qcom-qmp-dp-phy-v5.h"
#include "phy-qcom-qmp-dp-phy-v6.h"
+#include "phy-qcom-qmp-dp-phy-v8.h"
+
+#include "phy-qcom-qmp-usb43-pcs-v8.h"
/* QPHY_V3_DP_COM_RESET_OVRD_CTRL register bits */
/* DP PHY soft reset */
@@ -70,6 +79,7 @@ enum qphy_reg_layout {
QPHY_PCS_AUTONOMOUS_MODE_CTRL,
QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR,
QPHY_PCS_POWER_DOWN_CONTROL,
+ QPHY_PCS_CLAMP_ENABLE,
QPHY_COM_RESETSM_CNTRL,
QPHY_COM_C_READY_STATUS,
@@ -85,6 +95,8 @@ enum qphy_reg_layout {
QPHY_TX_HIGHZ_DRVR_EN,
QPHY_TX_TRANSCEIVER_BIAS_EN,
+ QPHY_AON_TOGGLE_ENABLE,
+ QPHY_DP_AON_TOGGLE_ENABLE,
/* Keep last to ensure regs_layout arrays are properly initialized */
QPHY_LAYOUT_SIZE
};
@@ -97,6 +109,8 @@ static const unsigned int qmp_v3_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V3_PCS_AUTONOMOUS_MODE_CTRL,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V3_PCS_LFPS_RXTERM_IRQ_CLEAR,
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V3_PCS_MISC_CLAMP_ENABLE,
+
[QPHY_COM_RESETSM_CNTRL] = QSERDES_V3_COM_RESETSM_CNTRL,
[QPHY_COM_C_READY_STATUS] = QSERDES_V3_COM_C_READY_STATUS,
[QPHY_COM_CMN_STATUS] = QSERDES_V3_COM_CMN_STATUS,
@@ -122,6 +136,8 @@ static const unsigned int qmp_v45_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V4_PCS_USB3_AUTONOMOUS_MODE_CTRL,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V4_PCS_USB3_LFPS_RXTERM_IRQ_CLEAR,
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V4_PCS_MISC_CLAMP_ENABLE,
+
[QPHY_COM_RESETSM_CNTRL] = QSERDES_V4_COM_RESETSM_CNTRL,
[QPHY_COM_C_READY_STATUS] = QSERDES_V4_COM_C_READY_STATUS,
[QPHY_COM_CMN_STATUS] = QSERDES_V4_COM_CMN_STATUS,
@@ -147,6 +163,8 @@ static const unsigned int qmp_v5_5nm_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V5_PCS_USB3_AUTONOMOUS_MODE_CTRL,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V5_PCS_USB3_LFPS_RXTERM_IRQ_CLEAR,
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V5_PCS_MISC_CLAMP_ENABLE,
+
[QPHY_COM_RESETSM_CNTRL] = QSERDES_V5_COM_RESETSM_CNTRL,
[QPHY_COM_C_READY_STATUS] = QSERDES_V5_COM_C_READY_STATUS,
[QPHY_COM_CMN_STATUS] = QSERDES_V5_COM_CMN_STATUS,
@@ -172,6 +190,8 @@ static const unsigned int qmp_v6_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V6_PCS_USB3_AUTONOMOUS_MODE_CTRL,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V6_PCS_USB3_LFPS_RXTERM_IRQ_CLEAR,
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V6_PCS_AON_CLAMP_ENABLE,
+
[QPHY_COM_RESETSM_CNTRL] = QSERDES_V6_COM_RESETSM_CNTRL,
[QPHY_COM_C_READY_STATUS] = QSERDES_V6_COM_C_READY_STATUS,
[QPHY_COM_CMN_STATUS] = QSERDES_V6_COM_CMN_STATUS,
@@ -197,6 +217,8 @@ static const unsigned int qmp_v6_n4_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V6_PCS_USB3_AUTONOMOUS_MODE_CTRL,
[QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V6_PCS_USB3_LFPS_RXTERM_IRQ_CLEAR,
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V6_PCS_AON_CLAMP_ENABLE,
+
[QPHY_COM_RESETSM_CNTRL] = QSERDES_V6_COM_RESETSM_CNTRL,
[QPHY_COM_C_READY_STATUS] = QSERDES_V6_COM_C_READY_STATUS,
[QPHY_COM_CMN_STATUS] = QSERDES_V6_COM_CMN_STATUS,
@@ -212,6 +234,262 @@ static const unsigned int qmp_v6_n4_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
[QPHY_TX_TRANSCEIVER_BIAS_EN] = QSERDES_V6_N4_TX_TRANSCEIVER_BIAS_EN,
};
+static const unsigned int qmp_v8_usb3phy_regs_layout[QPHY_LAYOUT_SIZE] = {
+ [QPHY_SW_RESET] = QPHY_V8_PCS_SW_RESET,
+ [QPHY_START_CTRL] = QPHY_V8_PCS_START_CONTROL,
+ [QPHY_PCS_STATUS] = QPHY_V8_PCS_PCS_STATUS1,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = QPHY_V8_PCS_POWER_DOWN_CONTROL,
+
+ /* In PCS_USB */
+ [QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V8_PCS_USB_AUTONOMOUS_MODE_CTRL,
+ [QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V8_PCS_USB_LFPS_RXTERM_IRQ_CLEAR,
+
+ [QPHY_COM_RESETSM_CNTRL] = QSERDES_V8_COM_RESETSM_CNTRL,
+ [QPHY_COM_C_READY_STATUS] = QSERDES_V8_COM_C_READY_STATUS,
+ [QPHY_COM_CMN_STATUS] = QSERDES_V8_COM_CMN_STATUS,
+ [QPHY_COM_BIAS_EN_CLKBUFLR_EN] = QSERDES_V8_COM_BIAS_EN_CLKBUFLR_EN,
+
+ [QPHY_DP_PHY_STATUS] = QSERDES_V6_DP_PHY_STATUS,
+ [QPHY_DP_PHY_VCO_DIV] = QSERDES_V6_DP_PHY_VCO_DIV,
+
+ [QPHY_TX_TX_POL_INV] = QSERDES_V8_TX_TX_POL_INV,
+ [QPHY_TX_TX_DRV_LVL] = QSERDES_V8_TX_TX_DRV_LVL,
+ [QPHY_TX_TX_EMP_POST1_LVL] = QSERDES_V8_TX_TX_EMP_POST1_LVL,
+ [QPHY_TX_HIGHZ_DRVR_EN] = QSERDES_V8_TX_HIGHZ_DRVR_EN,
+ [QPHY_TX_TRANSCEIVER_BIAS_EN] = QSERDES_V8_TX_TRANSCEIVER_BIAS_EN,
+};
+
+static const unsigned int qmp_v8_n3_usb43dpphy_regs_layout[QPHY_LAYOUT_SIZE] = {
+ [QPHY_SW_RESET] = QPHY_V8_USB43_PCS_SW_RESET,
+ [QPHY_START_CTRL] = QPHY_V8_USB43_PCS_START_CONTROL,
+ [QPHY_PCS_STATUS] = QPHY_V8_USB43_PCS_PCS_STATUS1,
+ [QPHY_PCS_POWER_DOWN_CONTROL] = QPHY_V8_USB43_PCS_POWER_DOWN_CONTROL,
+
+ /* In PCS_USB */
+ [QPHY_PCS_AUTONOMOUS_MODE_CTRL] = QPHY_V8_PCS_USB_AUTONOMOUS_MODE_CTRL,
+ [QPHY_PCS_LFPS_RXTERM_IRQ_CLEAR] = QPHY_V8_PCS_USB_LFPS_RXTERM_IRQ_CLEAR,
+
+ [QPHY_PCS_CLAMP_ENABLE] = QPHY_V8_PCS_AON_USB3_AON_CLAMP_ENABLE,
+ [QPHY_AON_TOGGLE_ENABLE] = QPHY_V8_PCS_AON_USB3_AON_TOGGLE_ENABLE,
+ [QPHY_DP_AON_TOGGLE_ENABLE] = QPHY_V8_PCS_AON_DP_AON_TOGGLE_ENABLE,
+
+ [QPHY_COM_RESETSM_CNTRL] = QSERDES_V8_COM_RESETSM_CNTRL,
+ [QPHY_COM_C_READY_STATUS] = QSERDES_V8_COM_C_READY_STATUS,
+ [QPHY_COM_CMN_STATUS] = QSERDES_V8_COM_CMN_STATUS,
+ [QPHY_COM_BIAS_EN_CLKBUFLR_EN] = QSERDES_V8_COM_BIAS_EN_CLKBUFLR_EN,
+
+ [QPHY_DP_PHY_STATUS] = QSERDES_V8_DP_PHY_STATUS,
+ [QPHY_DP_PHY_VCO_DIV] = QSERDES_V8_DP_PHY_VCO_DIV,
+
+ [QPHY_TX_TX_DRV_LVL] = QSERDES_V8_LALB_TX0_DRV_LVL,
+ [QPHY_TX_TX_EMP_POST1_LVL] = QSERDES_V8_LALB_TX0_EMP_POST1_LVL,
+ [QPHY_TX_HIGHZ_DRVR_EN] = QSERDES_V8_LALB_HIGHZ_DRVR_EN,
+ [QPHY_TX_TRANSCEIVER_BIAS_EN] = QSERDES_V8_LALB_TRANSMITTER_EN_CTRL,
+};
+
+static const struct qmp_phy_init_tbl glymur_usb43dp_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE1, 0xe1),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CP_CTRL_MODE1, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_RCTRL_MODE1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_CCTRL_MODE1, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORECLK_DIV_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE1, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE1, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE1, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MSB_MODE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE1, 0xab),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE1, 0xaa),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_SEL_1, 0x13),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE1, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE1, 0x4d),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE1, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE1, 0x95),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE1, 0x1e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0x4b),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0, 0xe1),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CP_CTRL_MODE0, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORECLK_DIV_MODE0, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE0, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MSB_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE0, 0xab),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0, 0xaa),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_HS_SWITCH_SEL_1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE0, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0, 0x4d),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BG_TIMER, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_EN_CENTER, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER1, 0x62),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER2, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_BUF_ENABLE, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_IVCO, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_IVCO_MODE1, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_EN_SEL, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP_EN, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP_CFG, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_CTRL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_MAP, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORE_CLK_EN, 0xa0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_CONFIG_1, 0x76),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SVS_MODE_CLK_SEL, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_SPARE_FOR_ECO, 0x40),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DCC_CAL_1, 0x40),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DCC_CAL_2, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DCC_CAL_3, 0x60),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PSM_CAL_EN, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1, 0x33),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL, 0xaf),
+};
+
+static const struct qmp_phy_init_tbl glymur_usb43dp_pcs_misc_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_MISC_PCS_MISC_CONFIG1, 0x01),
+};
+
+static const struct qmp_phy_init_tbl glymur_usb43dp_pcs_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG1, 0xc4),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG2, 0x89),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG3, 0x20),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG6, 0x13),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_REFGEN_REQ_CONFIG1, 0x21),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_RX_SIGDET_LVL, 0x55),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_RCVR_DTCT_DLY_P1U2_L, 0xe7),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_RCVR_DTCT_DLY_P1U2_H, 0x03),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_TSYNC_RSYNC_TIME, 0xa4),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_RX_CONFIG, 0x0a),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_TSYNC_DLY_TIME, 0x04),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_ALIGN_DETECT_CONFIG1, 0xd4),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_ALIGN_DETECT_CONFIG2, 0x30),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_PCS_TX_RX_CONFIG, 0x0c),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_EQ_CONFIG1, 0x4b),
+ QMP_PHY_INIT_CFG(QPHY_V8_USB43_PCS_EQ_CONFIG5, 0x10),
+};
+
+static const struct qmp_phy_init_tbl glymur_usb43dp_pcs_usb_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_LFPS_DET_HIGH_COUNT_VAL, 0xf8),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_RXEQTRAINING_DFE_TIME_S2, 0x07),
+};
+
+static const struct qmp_phy_init_tbl glymur_usb43dp_lalb_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CLKBUF_ENABLE, 0x81),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX_LVL_UPDATE_CTRL, 0x0d),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL3, 0x80),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL4, 0x8D),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TRANSMITTER_EN_CTRL, 0x13),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_1, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_3, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_4, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_RESTRIM_CAL_CTRL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_RESTRIM_CAL_CTRL, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_RESTRIM_POST_CAL_OFFSET, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_RESTRIM_VREF_SEL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_RESTRIM_VREF_SEL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_ANA_INTERFACE_SELECT2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCS_INTERFACE_SELECT1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B0, 0xa4),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B1, 0xa2),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B2, 0x6e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B3, 0x51),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B4, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B5, 0x26),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B6, 0x12),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE_0_1_B7, 0x2a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B0, 0x4c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B1, 0xc4),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B2, 0x38),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B3, 0x64),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B4, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B5, 0x4b),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B6, 0x12),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_MODE_RATE2_B7, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX_DCC_ANA_CTRL2, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE1, 0x26),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE1, 0x26),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE2, 0x26),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE2, 0x26),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_INIT_RATE_0_1, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_INIT_RATE_2_3, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE1, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE2, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE1, 0x15),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE2, 0x22),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CODE_OVRD_RATE_2_3, 0x22),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE1, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE2, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE1, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE2, 0x09),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_SUMMER_CAL_SPD_MODE_RATE_0123, 0x2f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_CAL_CTRL2, 0x85),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_CAL_CTRL3, 0x45),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_ENABLES, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_CNTRL, 0xa3),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_LVL, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_DEGLITCH_CNTRL, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_CAL_CTRL1, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_CAL_CTRL2_AND_CDR_LOCK_EDGE, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_SIGDET_CAL_TRIM, 0x66),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE1, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE2, 0x32),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE1, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE2, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE2, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_CCODE_RATE_01, 0x76),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_CCODE_RATE_23, 0x67),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_FAST_RATE_0_1, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_FAST_RATE_2_3, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_FLL_RATE_0_1, 0x33),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_FLL_RATE_2_3, 0x43),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_PLL_RATE_0_1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_RCODE_PLL_RATE_2_3, 0x51),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_FLL_DIV_RATIO_RATE_0123, 0xe5),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CAP_CODE_RATE_0123, 0xf5),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_TYPE_CONFIG, 0x1f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_EN_LOWFREQ, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_LOOP_FUNC_CTRL, 0xd0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_GM_CAL_EN, 0x1f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_GM_CAL_RES_RATE0_1, 0x88),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_GM_CAL_RES_RATE2_3, 0x88),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_AUX_CLK_CTRL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_EOM_CTRL1, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL2, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL3, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL4, 0xaa),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CTLE_POST_CAL_OFFSET_RATE_0_1_2, 0x77),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_VGA_CAL_CNTRL1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_VGA_CAL_MAN_VAL_RATE0_1, 0xdd),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_VGA_CAL_MAN_VAL_RATE2_3, 0xd8),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_DFE_TAP1_DAC_ENABLE, 0x1c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_DFE_TAP2_DAC_ENABLE, 0x1c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_DFE_TAP345_DAC_ENABLE, 0x18),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_DFE_TAP67_DAC_ENABLE, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_IQTUNE_CTRL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_IQTUNE_MAN_INDEX, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_IQTUNE_DIV2_CTRL_RATE0123, 0x1C),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CDR_VCO_CAP_CODE_OVRD_MUXES, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_DIG_BKUP_CTRL16, 0x37),
+};
+
static const struct qmp_phy_init_tbl qmp_v3_usb3_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V3_COM_PLL_IVCO, 0x07),
QMP_PHY_INIT_CFG(QSERDES_V3_COM_SYSCLK_EN_SEL, 0x14),
@@ -400,6 +678,57 @@ static const struct qmp_phy_init_tbl qmp_v3_usb3_pcs_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V3_PCS_RXEQTRAINING_RUN_TIME, 0x13),
};
+static const struct qmp_phy_init_tbl sar2130p_usb3_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_STEP_SIZE1_MODE1, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_STEP_SIZE2_MODE1, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_CP_CTRL_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_PLL_RCTRL_MODE1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_PLL_CCTRL_MODE1, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_CORECLK_DIV_MODE1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_LOCK_CMP1_MODE1, 0x2e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_LOCK_CMP2_MODE1, 0x82),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DEC_START_MODE1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DEC_START_MSB_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START1_MODE1, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START2_MODE1, 0xd5),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START3_MODE1, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_HSCLK_SEL_1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_VCO_TUNE1_MODE1, 0x25),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_VCO_TUNE2_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_BIN_VCOCAL_CMP_CODE1_MODE1, 0xb7),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_BIN_VCOCAL_CMP_CODE2_MODE1, 0x1e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0xb7),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x1e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_STEP_SIZE1_MODE0, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_STEP_SIZE2_MODE0, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_CP_CTRL_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_LOCK_CMP1_MODE0, 0x12),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_LOCK_CMP2_MODE0, 0x34),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DEC_START_MODE0, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DEC_START_MSB_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START1_MODE0, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START2_MODE0, 0xd5),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_DIV_FRAC_START3_MODE0, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_VCO_TUNE1_MODE0, 0x25),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_VCO_TUNE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_BG_TIMER, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_EN_CENTER, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_PER1, 0x31),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_PER2, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SYSCLK_BUF_ENABLE, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_SYSCLK_EN_SEL, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_LOCK_CMP_CFG, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_VCO_TUNE_MAP, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_CORE_CLK_EN, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_CMN_CONFIG_1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_AUTO_GAIN_ADJ_CTRL_1, 0xb6),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_AUTO_GAIN_ADJ_CTRL_2, 0x4b),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_AUTO_GAIN_ADJ_CTRL_3, 0x37),
+ QMP_PHY_INIT_CFG(QSERDES_V6_COM_ADDITIONAL_MISC, 0x0c),
+};
+
static const struct qmp_phy_init_tbl sm6350_usb3_rx_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V3_RX_UCDR_FASTLOCK_FO_GAIN, 0x0b),
QMP_PHY_INIT_CFG(QSERDES_V3_RX_RX_EQU_ADAPTOR_CNTRL2, 0x0f),
@@ -1047,6 +1376,38 @@ static const struct qmp_phy_init_tbl qmp_v6_n4_dp_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V6_COM_CORE_CLK_EN, 0x0f),
};
+static const struct qmp_phy_init_tbl qmp_v8_dp_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CP_CTRL_MODE0, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_RCTRL_MODE1, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_CCTRL_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORECLK_DIV_MODE0, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE0, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN1_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BG_TIMER, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_EN_CENTER, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_ADJ_PER1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_ENABLE1, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYS_CLK_CTRL, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_BUF_ENABLE, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_IVCO, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_EN_SEL, 0x3b),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP_EN, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_CTRL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_MAP, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_SELECT, 0x30),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORE_CLK_EN, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_CONFIG_1, 0x56),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SVS_MODE_CLK_SEL, 0x15),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD1, 0x24),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DCC_CAL_1, 0x40),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DCC_CAL_3, 0x60),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PSM_CAL_EN, 0x01),
+};
+
static const struct qmp_phy_init_tbl qmp_v6_dp_tx_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V6_TX_VMODE_CTRL1, 0x40),
QMP_PHY_INIT_CFG(QSERDES_V6_TX_PRE_STALL_LDO_BOOST_EN, 0x30),
@@ -1074,6 +1435,33 @@ static const struct qmp_phy_init_tbl qmp_v6_n4_dp_tx_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V6_N4_TX_TX_BAND, 0x1),
};
+static const struct qmp_phy_init_tbl qmp_v8_n3p_dp_tx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TRANSMITTER_EN_CTRL, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_VMODE_CTRL1, 0x40),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_ANA_INTERFACE_SELECT1, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_ANA_INTERFACE_SELECT2, 0x18),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_PCS_INTERFACE_SELECT1, 0x50),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_1, 0x0d),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_CLKBUF_ENABLE, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_RESET_TSYNC_EN_CTRL, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX_LVL_UPDATE_CTRL, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TRAN_DRVR_EMP_EN, 0x5f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_EMP_POST1_LVL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_EMP_POST1_LVL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_PRE1_EMPH, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_PRE1_EMPH, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_DRV_LVL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_DRV_LVL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_HIGHZ_DRVR_EN, 0x30),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_2, 0x50),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_LANE_MODE_3, 0x51),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX_DCC_ANA_CTRL2, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_RESTRIM_CAL_CTRL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_RESTRIM_CAL_CTRL, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX0_RESTRIM_POST_CAL_OFFSET, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_LALB_TX1_RESTRIM_POST_CAL_OFFSET, 0x10),
+};
+
static const struct qmp_phy_init_tbl qmp_v6_dp_serdes_tbl_rbr[] = {
QMP_PHY_INIT_CFG(QSERDES_V6_COM_HSCLK_SEL_1, 0x05),
QMP_PHY_INIT_CFG(QSERDES_V6_COM_DEC_START_MODE0, 0x34),
@@ -1190,6 +1578,109 @@ static const struct qmp_phy_init_tbl qmp_v6_n4_dp_serdes_tbl_hbr3[] = {
QMP_PHY_INIT_CFG(QSERDES_V6_COM_SSC_STEP_SIZE2_MODE0, 0x01),
};
+static const struct qmp_phy_init_tbl qmp_v8_dp_serdes_tbl_rbr[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_SEL_1, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0x7a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0, 0x83),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0, 0x37),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE0, 0x54),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0, 0xfe),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD3, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1, 0x30),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL, 0xa4),
+};
+
+static const struct qmp_phy_init_tbl qmp_v8_dp_serdes_tbl_hbr[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_SEL_1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0x21),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0, 0x18),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE0, 0x46),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0, 0xae),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD3, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL, 0xa3),
+};
+
+static const struct qmp_phy_init_tbl qmp_v8_dp_serdes_tbl_hbr2[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_SEL_1, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0xf6),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0, 0x0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE0, 0x46),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0, 0xae),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL, 0xbf),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIAS_EN_CLKBUFLR_EN, 0x1c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_RESETSM_CNTRL, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD3, 0x03),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1, 0x3f),
+};
+
+static const struct qmp_phy_init_tbl qmp_v8_dp_serdes_tbl_hbr3[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_HSCLK_SEL_1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0x63),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0, 0x5b),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CP_CTRL_MODE0, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORECLK_DIV_MODE0, 0x0a),
+
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0, 0x17),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0, 0x15),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DEC_START_MODE0, 0x4f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0, 0xa0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE0, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_INTEGLOOP_GAIN1_MODE0, 0x00),
+
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0, 0xa0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_ADJ_PER1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER1, 0x6b),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SSC_PER2, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_ENABLE1, 0x0c),
+
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYS_CLK_CTRL, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_BUF_ENABLE, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_PLL_IVCO, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SYSCLK_EN_SEL, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_CTRL, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_VCO_TUNE_MAP, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_SELECT, 0x30),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CORE_CLK_EN, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_CONFIG_1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_SVS_MODE_CLK_SEL, 0x15),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1, 0x30),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIAS_EN_CLKBUFLR_EN, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD3, 0x05),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_CMN_MODE_CONTD1, 0x24),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL, 0x84),
+};
+
static const struct qmp_phy_init_tbl sc8280xp_usb43dp_serdes_tbl[] = {
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_EN_CENTER, 0x01),
QMP_PHY_INIT_CFG(QSERDES_V5_COM_SSC_PER1, 0x31),
@@ -1420,20 +1911,154 @@ static const struct qmp_phy_init_tbl x1e80100_usb43dp_pcs_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V6_N4_PCS_EQ_CONFIG5, 0x10),
};
+static const struct qmp_phy_init_tbl sm8750_usb3_serdes_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_STEP_SIZE1_MODE1, 0xc0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_STEP_SIZE2_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_CP_CTRL_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_PLL_RCTRL_MODE1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_PLL_CCTRL_MODE1, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_CORECLK_DIV_MODE1, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_LOCK_CMP1_MODE1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_LOCK_CMP2_MODE1, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DEC_START_MODE1, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DEC_START_MSB_MODE1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START1_MODE1, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START2_MODE1, 0x75),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START3_MODE1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_HSCLK_SEL_1, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_VCO_TUNE1_MODE1, 0x25),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_VCO_TUNE2_MODE1, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE1_MODE1, 0x5c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE2_MODE1, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE1_MODE0, 0x5c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE2_MODE0, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_STEP_SIZE1_MODE0, 0xc0),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_STEP_SIZE2_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_CP_CTRL_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_PLL_RCTRL_MODE0, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_PLL_CCTRL_MODE0, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_LOCK_CMP1_MODE0, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_LOCK_CMP2_MODE0, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DEC_START_MODE0, 0x41),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DEC_START_MSB_MODE0, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START1_MODE0, 0x55),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START2_MODE0, 0x75),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_DIV_FRAC_START3_MODE0, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_VCO_TUNE1_MODE0, 0x25),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_VCO_TUNE2_MODE0, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_BG_TIMER, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_EN_CENTER, 0x01),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_PER1, 0x62),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SSC_PER2, 0x02),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SYSCLK_BUF_ENABLE, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_SYSCLK_EN_SEL, 0x1a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_LOCK_CMP_CFG, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_VCO_TUNE_MAP, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_CORE_CLK_EN, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_CMN_CONFIG_1, 0x16),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_1, 0xb6),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_2, 0x4a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_3, 0x36),
+ QMP_PHY_INIT_CFG(QSERDES_V8_COM_ADDITIONAL_MISC, 0x0c),
+};
+
+static const struct qmp_phy_init_tbl sm8750_usb3_tx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_RES_CODE_LANE_TX, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_RES_CODE_LANE_RX, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_RES_CODE_LANE_OFFSET_TX, 0x1f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_RES_CODE_LANE_OFFSET_RX, 0x09),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_LANE_MODE_1, 0xf5),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_LANE_MODE_3, 0x11),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_LANE_MODE_4, 0x31),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_LANE_MODE_5, 0x5f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_TX_RCV_DETECT_LVL_2, 0x12),
+ QMP_PHY_INIT_CFG_LANE(QSERDES_V8_TX_PI_QEC_CTRL, 0x21, 1),
+ QMP_PHY_INIT_CFG_LANE(QSERDES_V8_TX_PI_QEC_CTRL, 0x05, 2),
+};
+
+static const struct qmp_phy_init_tbl sm8750_usb3_rx_tbl[] = {
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_FO_GAIN, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SO_GAIN, 0x06),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_FASTLOCK_FO_GAIN, 0x2f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SO_SATURATION_AND_ENABLE, 0x7f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_FASTLOCK_COUNT_LOW, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_FASTLOCK_COUNT_HIGH, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_PI_CONTROLS, 0x99),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SB2_THRESH1, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SB2_THRESH2, 0x08),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SB2_GAIN1, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_UCDR_SB2_GAIN2, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_AUX_DATA_TCOARSE_TFINE, 0x20),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_VGA_CAL_CNTRL1, 0x54),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_VGA_CAL_CNTRL2, 0x0f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_GM_CAL, 0x13),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL2, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL3, 0x4a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL4, 0x0a),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_IDAC_TSETTLE_LOW, 0x07),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_IDAC_TSETTLE_HIGH, 0x00),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_EQ_OFFSET_ADAPTOR_CNTRL1, 0x27),
+
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_SIGDET_ENABLES, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_SIGDET_CNTRL, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_SIGDET_DEGLITCH_CNTRL, 0x0e),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_00_LOW, 0x3f),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_00_HIGH, 0xbf),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_00_HIGH2, 0xff),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_00_HIGH3, 0xdf),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_00_HIGH4, 0xed),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_01_LOW, 0x19),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_01_HIGH, 0x09),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_01_HIGH2, 0x91),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_01_HIGH3, 0xb7),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_RX_MODE_01_HIGH4, 0xaa),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_DFE_EN_TIMER, 0x04),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_DFE_CTLE_POST_CAL_OFFSET, 0x38),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_DCC_CTRL1, 0x0c),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_VTH_CODE, 0x10),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_SIGDET_CAL_CTRL1, 0x14),
+ QMP_PHY_INIT_CFG(QSERDES_V8_RX_SIGDET_CAL_TRIM, 0x08),
+};
+
+static const struct qmp_phy_init_tbl sm8750_usb3_pcs_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_LOCK_DETECT_CONFIG1, 0xc4),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_LOCK_DETECT_CONFIG2, 0x89),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_LOCK_DETECT_CONFIG3, 0x20),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_LOCK_DETECT_CONFIG6, 0x13),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_REFGEN_REQ_CONFIG1, 0x21),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_RX_SIGDET_LVL, 0x55),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_RCVR_DTCT_DLY_P1U2_L, 0xe7),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_RCVR_DTCT_DLY_P1U2_H, 0x03),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_CDR_RESET_TIME, 0x0a),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_ALIGN_DETECT_CONFIG1, 0x88),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_ALIGN_DETECT_CONFIG2, 0x13),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_PCS_TX_RX_CONFIG, 0x0c),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_EQ_CONFIG1, 0x4b),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_EQ_CONFIG5, 0x10),
+};
+
+static const struct qmp_phy_init_tbl sm8750_usb3_pcs_usb_tbl[] = {
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_LFPS_DET_HIGH_COUNT_VAL, 0xf8),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_RXEQTRAINING_DFE_TIME_S2, 0x07),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_RCVR_DTCT_DLY_U3_L, 0x40),
+ QMP_PHY_INIT_CFG(QPHY_V8_PCS_USB_RCVR_DTCT_DLY_U3_H, 0x00),
+};
+
static const struct qmp_phy_init_tbl x1e80100_usb43dp_pcs_usb_tbl[] = {
QMP_PHY_INIT_CFG(QPHY_V6_PCS_USB3_LFPS_DET_HIGH_COUNT_VAL, 0xf8),
QMP_PHY_INIT_CFG(QPHY_V6_PCS_USB3_RXEQTRAINING_DFE_TIME_S2, 0x07),
};
/* list of regulators */
-struct qmp_regulator_data {
- const char *name;
- unsigned int enable_load;
+static struct regulator_bulk_data qmp_phy_vreg_l[] = {
+ { .supply = "vdda-phy", .init_load_uA = 21800, },
+ { .supply = "vdda-pll", .init_load_uA = 36000, },
};
-static struct qmp_regulator_data qmp_phy_vreg_l[] = {
- { .name = "vdda-phy", .enable_load = 21800 },
- { .name = "vdda-pll", .enable_load = 36000 },
+static struct regulator_bulk_data qmp_phy_vreg_refgen[] = {
+ { .supply = "vdda-phy", .init_load_uA = 21800 },
+ { .supply = "vdda-pll", .init_load_uA = 36000 },
+ { .supply = "refgen", .init_load_uA = 3270 },
};
static const u8 qmp_dp_v3_pre_emphasis_hbr3_hbr2[4][4] = {
@@ -1474,7 +2099,7 @@ static const u8 qmp_dp_v4_pre_emphasis_hbr3_hbr2[4][4] = {
static const u8 qmp_dp_v4_pre_emphasis_hbr_rbr[4][4] = {
{ 0x00, 0x0d, 0x14, 0x1a },
{ 0x00, 0x0e, 0x15, 0xff },
- { 0x00, 0x0d, 0xff, 0xff },
+ { 0x00, 0x0e, 0xff, 0xff },
{ 0x03, 0xff, 0xff, 0xff }
};
@@ -1538,6 +2163,7 @@ struct qmp_combo_offsets {
u16 usb3_serdes;
u16 usb3_pcs_misc;
u16 usb3_pcs;
+ u16 usb3_pcs_aon;
u16 usb3_pcs_usb;
u16 dp_serdes;
u16 dp_txa;
@@ -1559,6 +2185,8 @@ struct qmp_phy_cfg {
int pcs_tbl_num;
const struct qmp_phy_init_tbl *pcs_usb_tbl;
int pcs_usb_tbl_num;
+ const struct qmp_phy_init_tbl *pcs_misc_tbl;
+ int pcs_misc_tbl_num;
const struct qmp_phy_init_tbl *dp_serdes_tbl;
int dp_serdes_tbl_num;
@@ -1582,6 +2210,7 @@ struct qmp_phy_cfg {
const u8 (*pre_emphasis_hbr3_hbr2)[4][4];
/* DP PHY callbacks */
+ int (*configure_dp_clocks)(struct qmp_combo *qmp);
int (*configure_dp_phy)(struct qmp_combo *qmp);
void (*configure_dp_tx)(struct qmp_combo *qmp);
int (*calibrate_dp_phy)(struct qmp_combo *qmp);
@@ -1591,7 +2220,7 @@ struct qmp_phy_cfg {
const char * const *reset_list;
int num_resets;
/* regulators to be requested */
- const struct qmp_regulator_data *vreg_list;
+ const struct regulator_bulk_data *vreg_list;
int num_vregs;
/* array of registers with different offsets */
@@ -1619,6 +2248,7 @@ struct qmp_combo {
void __iomem *tx2;
void __iomem *rx2;
void __iomem *pcs_misc;
+ void __iomem *pcs_aon;
void __iomem *pcs_usb;
void __iomem *dp_serdes;
@@ -1654,13 +2284,17 @@ struct qmp_combo {
static void qmp_v3_dp_aux_init(struct qmp_combo *qmp);
static void qmp_v3_configure_dp_tx(struct qmp_combo *qmp);
+static int qmp_v3_configure_dp_clocks(struct qmp_combo *qmp);
static int qmp_v3_configure_dp_phy(struct qmp_combo *qmp);
static int qmp_v3_calibrate_dp_phy(struct qmp_combo *qmp);
static void qmp_v4_dp_aux_init(struct qmp_combo *qmp);
static void qmp_v4_configure_dp_tx(struct qmp_combo *qmp);
static int qmp_v4_configure_dp_phy(struct qmp_combo *qmp);
-static int qmp_v4_calibrate_dp_phy(struct qmp_combo *qmp);
+
+static void qmp_v8_dp_aux_init(struct qmp_combo *qmp);
+static int qmp_v8_configure_dp_clocks(struct qmp_combo *qmp);
+static int qmp_v8_configure_dp_phy(struct qmp_combo *qmp);
static inline void qphy_setbits(void __iomem *base, u32 offset, u32 val)
{
@@ -1730,6 +2364,81 @@ static const struct qmp_combo_offsets qmp_combo_offsets_v5 = {
.dp_dp_phy = 0x2200,
};
+static const struct qmp_combo_offsets qmp_combo_offsets_v8 = {
+ .com = 0x0000,
+ .txa = 0x1400,
+ .rxa = 0x1600,
+ .txb = 0x1800,
+ .rxb = 0x1a00,
+ .usb3_serdes = 0x1000,
+ .usb3_pcs_misc = 0x1c00,
+ .usb3_pcs = 0x1e00,
+ .usb3_pcs_aon = 0x2000,
+ .usb3_pcs_usb = 0x2100,
+ .dp_serdes = 0x3000,
+ .dp_txa = 0x3400,
+ .dp_txb = 0x3800,
+ .dp_dp_phy = 0x3c00,
+};
+
+static const struct qmp_combo_offsets qmp_combo_usb43dp_offsets_v8 = {
+ .com = 0x0000,
+ .usb3_pcs_aon = 0x0100,
+ .usb3_serdes = 0x1000,
+ .usb3_pcs_misc = 0x1400,
+ .usb3_pcs = 0x1600,
+ .usb3_pcs_usb = 0x1900,
+ .dp_serdes = 0x2000,
+ .dp_dp_phy = 0x2400,
+ .txa = 0x4000,
+ .txb = 0x5000,
+};
+
+static const struct qmp_phy_cfg sar2130p_usb3dpphy_cfg = {
+ .offsets = &qmp_combo_offsets_v3,
+
+ .serdes_tbl = sar2130p_usb3_serdes_tbl,
+ .serdes_tbl_num = ARRAY_SIZE(sar2130p_usb3_serdes_tbl),
+ .tx_tbl = sm8550_usb3_tx_tbl,
+ .tx_tbl_num = ARRAY_SIZE(sm8550_usb3_tx_tbl),
+ .rx_tbl = sm8550_usb3_rx_tbl,
+ .rx_tbl_num = ARRAY_SIZE(sm8550_usb3_rx_tbl),
+ .pcs_tbl = sm8550_usb3_pcs_tbl,
+ .pcs_tbl_num = ARRAY_SIZE(sm8550_usb3_pcs_tbl),
+ .pcs_usb_tbl = sm8550_usb3_pcs_usb_tbl,
+ .pcs_usb_tbl_num = ARRAY_SIZE(sm8550_usb3_pcs_usb_tbl),
+
+ .dp_serdes_tbl = qmp_v6_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v6_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v6_dp_tx_tbl),
+
+ .serdes_tbl_rbr = qmp_v6_dp_serdes_tbl_rbr,
+ .serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_rbr),
+ .serdes_tbl_hbr = qmp_v6_dp_serdes_tbl_hbr,
+ .serdes_tbl_hbr_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr),
+ .serdes_tbl_hbr2 = qmp_v6_dp_serdes_tbl_hbr2,
+ .serdes_tbl_hbr2_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr2),
+ .serdes_tbl_hbr3 = qmp_v6_dp_serdes_tbl_hbr3,
+ .serdes_tbl_hbr3_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr3),
+
+ .swing_hbr_rbr = &qmp_dp_v5_voltage_swing_hbr_rbr,
+ .pre_emphasis_hbr_rbr = &qmp_dp_v6_pre_emphasis_hbr_rbr,
+ .swing_hbr3_hbr2 = &qmp_dp_v5_voltage_swing_hbr3_hbr2,
+ .pre_emphasis_hbr3_hbr2 = &qmp_dp_v5_pre_emphasis_hbr3_hbr2,
+
+ .dp_aux_init = qmp_v4_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
+ .configure_dp_phy = qmp_v4_configure_dp_phy,
+
+ .regs = qmp_v6_usb3phy_regs_layout,
+ .reset_list = msm8996_usb3phy_reset_l,
+ .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+};
+
static const struct qmp_phy_cfg sc7180_usb3dpphy_cfg = {
.offsets = &qmp_combo_offsets_v3,
@@ -1855,8 +2564,8 @@ static const struct qmp_phy_cfg sc8180x_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
@@ -1901,8 +2610,8 @@ static const struct qmp_phy_cfg sc8280xp_usb43dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
@@ -1946,8 +2655,8 @@ static const struct qmp_phy_cfg x1e80100_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
@@ -2034,8 +2743,8 @@ static const struct qmp_phy_cfg sm8250_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
@@ -2082,8 +2791,8 @@ static const struct qmp_phy_cfg sm8350_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.reset_list = msm8996_usb3phy_reset_l,
.num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
@@ -2129,8 +2838,8 @@ static const struct qmp_phy_cfg sm8550_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.regs = qmp_v6_usb3phy_regs_layout,
.reset_list = msm8996_usb3phy_reset_l,
@@ -2174,8 +2883,8 @@ static const struct qmp_phy_cfg sm8650_usb3dpphy_cfg = {
.dp_aux_init = qmp_v4_dp_aux_init,
.configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
.configure_dp_phy = qmp_v4_configure_dp_phy,
- .calibrate_dp_phy = qmp_v4_calibrate_dp_phy,
.regs = qmp_v6_usb3phy_regs_layout,
.reset_list = msm8996_usb3phy_reset_l,
@@ -2184,6 +2893,96 @@ static const struct qmp_phy_cfg sm8650_usb3dpphy_cfg = {
.num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
};
+static const struct qmp_phy_cfg sm8750_usb3dpphy_cfg = {
+ .offsets = &qmp_combo_offsets_v8,
+
+ .serdes_tbl = sm8750_usb3_serdes_tbl,
+ .serdes_tbl_num = ARRAY_SIZE(sm8750_usb3_serdes_tbl),
+ .tx_tbl = sm8750_usb3_tx_tbl,
+ .tx_tbl_num = ARRAY_SIZE(sm8750_usb3_tx_tbl),
+ .rx_tbl = sm8750_usb3_rx_tbl,
+ .rx_tbl_num = ARRAY_SIZE(sm8750_usb3_rx_tbl),
+ .pcs_tbl = sm8750_usb3_pcs_tbl,
+ .pcs_tbl_num = ARRAY_SIZE(sm8750_usb3_pcs_tbl),
+ .pcs_usb_tbl = sm8750_usb3_pcs_usb_tbl,
+ .pcs_usb_tbl_num = ARRAY_SIZE(sm8750_usb3_pcs_usb_tbl),
+
+ .dp_serdes_tbl = qmp_v6_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v6_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v6_dp_tx_tbl),
+
+ .serdes_tbl_rbr = qmp_v6_dp_serdes_tbl_rbr,
+ .serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_rbr),
+ .serdes_tbl_hbr = qmp_v6_dp_serdes_tbl_hbr,
+ .serdes_tbl_hbr_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr),
+ .serdes_tbl_hbr2 = qmp_v6_dp_serdes_tbl_hbr2,
+ .serdes_tbl_hbr2_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr2),
+ .serdes_tbl_hbr3 = qmp_v6_dp_serdes_tbl_hbr3,
+ .serdes_tbl_hbr3_num = ARRAY_SIZE(qmp_v6_dp_serdes_tbl_hbr3),
+
+ .swing_hbr_rbr = &qmp_dp_v6_voltage_swing_hbr_rbr,
+ .pre_emphasis_hbr_rbr = &qmp_dp_v6_pre_emphasis_hbr_rbr,
+ .swing_hbr3_hbr2 = &qmp_dp_v5_voltage_swing_hbr3_hbr2,
+ .pre_emphasis_hbr3_hbr2 = &qmp_dp_v5_pre_emphasis_hbr3_hbr2,
+
+ .dp_aux_init = qmp_v4_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v3_configure_dp_clocks,
+ .configure_dp_phy = qmp_v4_configure_dp_phy,
+
+ .regs = qmp_v8_usb3phy_regs_layout,
+ .reset_list = msm8996_usb3phy_reset_l,
+ .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
+ .vreg_list = qmp_phy_vreg_l,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_l),
+};
+
+static const struct qmp_phy_cfg glymur_usb3dpphy_cfg = {
+ .offsets = &qmp_combo_usb43dp_offsets_v8,
+
+ .serdes_tbl = glymur_usb43dp_serdes_tbl,
+ .serdes_tbl_num = ARRAY_SIZE(glymur_usb43dp_serdes_tbl),
+ .tx_tbl = glymur_usb43dp_lalb_tbl,
+ .tx_tbl_num = ARRAY_SIZE(glymur_usb43dp_lalb_tbl),
+ .pcs_tbl = glymur_usb43dp_pcs_tbl,
+ .pcs_tbl_num = ARRAY_SIZE(glymur_usb43dp_pcs_tbl),
+ .pcs_usb_tbl = glymur_usb43dp_pcs_usb_tbl,
+ .pcs_usb_tbl_num = ARRAY_SIZE(glymur_usb43dp_pcs_usb_tbl),
+ .pcs_misc_tbl = glymur_usb43dp_pcs_misc_tbl,
+ .pcs_misc_tbl_num = ARRAY_SIZE(glymur_usb43dp_pcs_misc_tbl),
+
+ .dp_serdes_tbl = qmp_v8_dp_serdes_tbl,
+ .dp_serdes_tbl_num = ARRAY_SIZE(qmp_v8_dp_serdes_tbl),
+ .dp_tx_tbl = qmp_v8_n3p_dp_tx_tbl,
+ .dp_tx_tbl_num = ARRAY_SIZE(qmp_v8_n3p_dp_tx_tbl),
+
+ .serdes_tbl_rbr = qmp_v8_dp_serdes_tbl_rbr,
+ .serdes_tbl_rbr_num = ARRAY_SIZE(qmp_v8_dp_serdes_tbl_rbr),
+ .serdes_tbl_hbr = qmp_v8_dp_serdes_tbl_hbr,
+ .serdes_tbl_hbr_num = ARRAY_SIZE(qmp_v8_dp_serdes_tbl_hbr),
+ .serdes_tbl_hbr2 = qmp_v8_dp_serdes_tbl_hbr2,
+ .serdes_tbl_hbr2_num = ARRAY_SIZE(qmp_v8_dp_serdes_tbl_hbr2),
+ .serdes_tbl_hbr3 = qmp_v8_dp_serdes_tbl_hbr3,
+ .serdes_tbl_hbr3_num = ARRAY_SIZE(qmp_v8_dp_serdes_tbl_hbr3),
+
+ .swing_hbr_rbr = &qmp_dp_v6_voltage_swing_hbr_rbr,
+ .pre_emphasis_hbr_rbr = &qmp_dp_v6_pre_emphasis_hbr_rbr,
+ .swing_hbr3_hbr2 = &qmp_dp_v5_voltage_swing_hbr3_hbr2,
+ .pre_emphasis_hbr3_hbr2 = &qmp_dp_v5_pre_emphasis_hbr3_hbr2,
+
+ .dp_aux_init = qmp_v8_dp_aux_init,
+ .configure_dp_tx = qmp_v4_configure_dp_tx,
+ .configure_dp_clocks = qmp_v8_configure_dp_clocks,
+ .configure_dp_phy = qmp_v8_configure_dp_phy,
+
+ .regs = qmp_v8_n3_usb43dpphy_regs_layout,
+ .reset_list = msm8996_usb3phy_reset_l,
+ .num_resets = ARRAY_SIZE(msm8996_usb3phy_reset_l),
+ .vreg_list = qmp_phy_vreg_refgen,
+ .num_vregs = ARRAY_SIZE(qmp_phy_vreg_refgen),
+};
+
static int qmp_combo_dp_serdes_init(struct qmp_combo *qmp)
{
const struct qmp_phy_cfg *cfg = qmp->cfg;
@@ -2346,7 +3145,7 @@ static bool qmp_combo_configure_dp_mode(struct qmp_combo *qmp)
return reverse;
}
-static int qmp_combo_configure_dp_clocks(struct qmp_combo *qmp)
+static int qmp_v3_configure_dp_clocks(struct qmp_combo *qmp)
{
const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
u32 phy_vco_div;
@@ -2393,7 +3192,7 @@ static int qmp_v3_configure_dp_phy(struct qmp_combo *qmp)
writel(0x05, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_TX0_TX1_LANE_CTL);
writel(0x05, qmp->dp_dp_phy + QSERDES_V3_DP_PHY_TX2_TX3_LANE_CTL);
- ret = qmp_combo_configure_dp_clocks(qmp);
+ ret = qmp_v3_configure_dp_clocks(qmp);
if (ret)
return ret;
@@ -2471,7 +3270,34 @@ static void qmp_v4_dp_aux_init(struct qmp_combo *qmp)
writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG7);
writel(0xb7, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG8);
writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG9);
- qmp->dp_aux_cfg = 0;
+
+ writel(PHY_AUX_STOP_ERR_MASK | PHY_AUX_DEC_ERR_MASK |
+ PHY_AUX_SYNC_ERR_MASK | PHY_AUX_ALIGN_ERR_MASK |
+ PHY_AUX_REQ_ERR_MASK,
+ qmp->dp_dp_phy + QSERDES_V4_DP_PHY_AUX_INTERRUPT_MASK);
+}
+
+static void qmp_v8_dp_aux_init(struct qmp_combo *qmp)
+{
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+
+ writel(DP_PHY_PD_CTL_PWRDN | DP_PHY_PD_CTL_PSR_PWRDN | DP_PHY_PD_CTL_AUX_PWRDN |
+ DP_PHY_PD_CTL_PLL_PWRDN | DP_PHY_PD_CTL_DP_CLAMP_EN,
+ qmp->dp_dp_phy + QSERDES_DP_PHY_PD_CTL);
+
+ /* Turn on BIAS current for PHY/PLL */
+ writel(0x1c, qmp->dp_serdes + cfg->regs[QPHY_COM_BIAS_EN_CLKBUFLR_EN]);
+
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG0);
+ writel(0x13, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
+ writel(0x06, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG2);
+ writel(0x00, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG3);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG4);
+ writel(0x26, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG5);
+ writel(0x0a, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG6);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG7);
+ writel(0xb7, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG8);
+ writel(0x03, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG9);
writel(PHY_AUX_STOP_ERR_MASK | PHY_AUX_DEC_ERR_MASK |
PHY_AUX_SYNC_ERR_MASK | PHY_AUX_ALIGN_ERR_MASK |
@@ -2493,6 +3319,58 @@ static void qmp_v4_configure_dp_tx(struct qmp_combo *qmp)
qmp_combo_configure_dp_swing(qmp);
}
+static int qmp_v8_configure_dp_clocks(struct qmp_combo *qmp)
+{
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
+ u32 phy_vco_div;
+ unsigned long pixel_freq;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+
+ switch (dp_opts->link_rate) {
+ case 1620:
+ phy_vco_div = 0x4;
+ pixel_freq = 1620000000UL / 2;
+ break;
+ case 2700:
+ phy_vco_div = 0x2;
+ pixel_freq = 2700000000UL / 2;
+ break;
+ case 5400:
+ phy_vco_div = 0x4;
+ pixel_freq = 5400000000UL / 4;
+ break;
+ case 8100:
+ phy_vco_div = 0x3;
+ pixel_freq = 8100000000UL / 6;
+ break;
+ default:
+ /* Other link rates aren't supported */
+ return -EINVAL;
+ }
+ writel(phy_vco_div, qmp->dp_dp_phy + cfg->regs[QPHY_DP_PHY_VCO_DIV]);
+
+ /* disable core reset tsync */
+ writel(0x09, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+
+ writel(0x04, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_AUXLESS_SETUP_CYC);
+ writel(0x08, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_AUXLESS_SILENCE_CYC);
+ writel(0x08, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_LFPS_CYC);
+ writel(0x11, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_LFPS_PERIOD);
+
+ writel(0x3e, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_TSYNC_OVRD);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_TX2_TX3_LANE_CTL);
+ writel(0x05, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_TX0_TX1_LANE_CTL);
+ writel(0x01, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_AUXLESS_CFG1);
+ writel(0x11, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_LFPS_PERIOD);
+ writel(0x1f, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_LN0_DRV_LVL);
+ writel(0x1f, qmp->dp_dp_phy + QSERDES_V8_DP_PHY_LN1_DRV_LVL);
+
+ clk_set_rate(qmp->dp_link_hw.clk, dp_opts->link_rate * 100000);
+ clk_set_rate(qmp->dp_pixel_hw.clk, pixel_freq);
+
+ return 0;
+}
+
static int qmp_v456_configure_dp_phy(struct qmp_combo *qmp)
{
const struct qmp_phy_cfg *cfg = qmp->cfg;
@@ -2509,7 +3387,7 @@ static int qmp_v456_configure_dp_phy(struct qmp_combo *qmp)
writel(0x05, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_TX0_TX1_LANE_CTL);
writel(0x05, qmp->dp_dp_phy + QSERDES_V4_DP_PHY_TX2_TX3_LANE_CTL);
- ret = qmp_combo_configure_dp_clocks(qmp);
+ ret = qmp->cfg->configure_dp_clocks(qmp);
if (ret)
return ret;
@@ -2623,20 +3501,58 @@ static int qmp_v4_configure_dp_phy(struct qmp_combo *qmp)
return 0;
}
-/*
- * We need to calibrate the aux setting here as many times
- * as the caller tries
- */
-static int qmp_v4_calibrate_dp_phy(struct qmp_combo *qmp)
+static int qmp_v8_configure_dp_phy(struct qmp_combo *qmp)
{
- static const u8 cfg1_settings[] = { 0x20, 0x13, 0x23, 0x1d };
- u8 val;
+ const struct qmp_phy_cfg *cfg = qmp->cfg;
+ bool reverse = (qmp->orientation == TYPEC_ORIENTATION_REVERSE);
+ const struct phy_configure_opts_dp *dp_opts = &qmp->dp_opts;
+ u32 bias0_en, drvr0_en, bias1_en, drvr1_en;
+ u32 status;
+ int ret;
- qmp->dp_aux_cfg++;
- qmp->dp_aux_cfg %= ARRAY_SIZE(cfg1_settings);
- val = cfg1_settings[qmp->dp_aux_cfg];
+ ret = qmp_v456_configure_dp_phy(qmp);
+ if (ret < 0)
+ return ret;
- writel(val, qmp->dp_dp_phy + QSERDES_DP_PHY_AUX_CFG1);
+ if (dp_opts->lanes == 1) {
+ bias0_en = reverse ? 0x3e : 0x15;
+ bias1_en = reverse ? 0x15 : 0x3e;
+ drvr0_en = reverse ? 0x13 : 0x10;
+ drvr1_en = reverse ? 0x10 : 0x13;
+ } else if (dp_opts->lanes == 2) {
+ bias0_en = reverse ? 0x3f : 0x15;
+ bias1_en = reverse ? 0x15 : 0x3f;
+ drvr0_en = 0x10;
+ drvr1_en = 0x10;
+ } else {
+ bias0_en = 0x3f;
+ bias1_en = 0x3f;
+ drvr0_en = 0x34;
+ drvr1_en = 0x34;
+ }
+
+ writel(drvr0_en, qmp->dp_tx + cfg->regs[QPHY_TX_HIGHZ_DRVR_EN]);
+ writel(bias0_en, qmp->dp_tx + cfg->regs[QPHY_TX_TRANSCEIVER_BIAS_EN]);
+ writel(drvr1_en, qmp->dp_tx2 + cfg->regs[QPHY_TX_HIGHZ_DRVR_EN]);
+ writel(bias1_en, qmp->dp_tx2 + cfg->regs[QPHY_TX_TRANSCEIVER_BIAS_EN]);
+
+ writel(0x08, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ udelay(100);
+ writel(0x09, qmp->dp_dp_phy + QSERDES_DP_PHY_CFG);
+ udelay(500);
+
+ if (readl_poll_timeout(qmp->dp_dp_phy + cfg->regs[QPHY_DP_PHY_STATUS],
+ status,
+ ((status & BIT(1)) > 0),
+ 500,
+ 10000))
+ return -ETIMEDOUT;
+
+ writel(0x00, qmp->dp_tx + cfg->regs[QPHY_TX_TX_DRV_LVL]);
+ writel(0x00, qmp->dp_tx2 + cfg->regs[QPHY_TX_TX_DRV_LVL]);
+
+ writel(0x2b, qmp->dp_tx + cfg->regs[QPHY_TX_TX_EMP_POST1_LVL]);
+ writel(0x2b, qmp->dp_tx2 + cfg->regs[QPHY_TX_TX_EMP_POST1_LVL]);
return 0;
}
@@ -2680,6 +3596,7 @@ static int qmp_combo_com_init(struct qmp_combo *qmp, bool force)
{
const struct qmp_phy_cfg *cfg = qmp->cfg;
void __iomem *com = qmp->com;
+ void __iomem *pcs_aon = qmp->pcs_aon;
int ret;
u32 val;
@@ -2715,6 +3632,12 @@ static int qmp_combo_com_init(struct qmp_combo *qmp, bool force)
SW_DPPHY_RESET_MUX | SW_DPPHY_RESET |
SW_USB3PHY_RESET_MUX | SW_USB3PHY_RESET);
+ /* override hardware control for reset of qmp phy */
+ if (pcs_aon && cfg->regs[QPHY_AON_TOGGLE_ENABLE]) {
+ qphy_clrbits(pcs_aon, cfg->regs[QPHY_AON_TOGGLE_ENABLE], 0x1);
+ qphy_clrbits(pcs_aon, cfg->regs[QPHY_DP_AON_TOGGLE_ENABLE], 0x1);
+ }
+
/* Use software based port select and switch on typec orientation */
val = SW_PORTSELECT_MUX;
if (qmp->orientation == TYPEC_ORIENTATION_REVERSE)
@@ -2867,6 +3790,8 @@ static int qmp_combo_usb_power_on(struct phy *phy)
qmp_configure_lane(qmp->dev, rx2, cfg->rx_tbl, cfg->rx_tbl_num, 2);
qmp_configure(qmp->dev, pcs, cfg->pcs_tbl, cfg->pcs_tbl_num);
+ qmp_configure(qmp->dev, qmp->pcs_misc, cfg->pcs_misc_tbl, cfg->pcs_misc_tbl_num);
+
if (pcs_usb)
qmp_configure(qmp->dev, pcs_usb, cfg->pcs_usb_tbl,
@@ -2993,6 +3918,7 @@ static void qmp_combo_enable_autonomous_mode(struct qmp_combo *qmp)
const struct qmp_phy_cfg *cfg = qmp->cfg;
void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
void __iomem *pcs_misc = qmp->pcs_misc;
+ void __iomem *pcs_aon = qmp->pcs_aon;
u32 intr_mask;
if (qmp->mode == PHY_MODE_USB_HOST_SS ||
@@ -3012,9 +3938,14 @@ static void qmp_combo_enable_autonomous_mode(struct qmp_combo *qmp)
/* Enable required PHY autonomous mode interrupts */
qphy_setbits(pcs_usb, cfg->regs[QPHY_PCS_AUTONOMOUS_MODE_CTRL], intr_mask);
- /* Enable i/o clamp_n for autonomous mode */
- if (pcs_misc)
- qphy_clrbits(pcs_misc, QPHY_V3_PCS_MISC_CLAMP_ENABLE, CLAMP_EN);
+ /*
+ * Enable i/o clamp_n for autonomous mode
+ * V6 and later versions use pcs aon clamp register
+ */
+ if (pcs_aon)
+ qphy_clrbits(pcs_aon, cfg->regs[QPHY_PCS_CLAMP_ENABLE], CLAMP_EN);
+ else if (pcs_misc)
+ qphy_clrbits(pcs_misc, cfg->regs[QPHY_PCS_CLAMP_ENABLE], CLAMP_EN);
}
static void qmp_combo_disable_autonomous_mode(struct qmp_combo *qmp)
@@ -3022,10 +3953,13 @@ static void qmp_combo_disable_autonomous_mode(struct qmp_combo *qmp)
const struct qmp_phy_cfg *cfg = qmp->cfg;
void __iomem *pcs_usb = qmp->pcs_usb ?: qmp->pcs;
void __iomem *pcs_misc = qmp->pcs_misc;
+ void __iomem *pcs_aon = qmp->pcs_aon;
/* Disable i/o clamp_n on resume for normal mode */
- if (pcs_misc)
- qphy_setbits(pcs_misc, QPHY_V3_PCS_MISC_CLAMP_ENABLE, CLAMP_EN);
+ if (pcs_aon)
+ qphy_setbits(pcs_aon, cfg->regs[QPHY_PCS_CLAMP_ENABLE], CLAMP_EN);
+ else if (pcs_misc)
+ qphy_setbits(pcs_misc, cfg->regs[QPHY_PCS_CLAMP_ENABLE], CLAMP_EN);
qphy_clrbits(pcs_usb, cfg->regs[QPHY_PCS_AUTONOMOUS_MODE_CTRL],
ARCVR_DTCT_EN | ARCVR_DTCT_EVENT_SEL | ALFPS_DTCT_EN);
@@ -3087,39 +4021,6 @@ static const struct dev_pm_ops qmp_combo_pm_ops = {
qmp_combo_runtime_resume, NULL)
};
-static int qmp_combo_vreg_init(struct qmp_combo *qmp)
-{
- const struct qmp_phy_cfg *cfg = qmp->cfg;
- struct device *dev = qmp->dev;
- int num = cfg->num_vregs;
- int ret, i;
-
- qmp->vregs = devm_kcalloc(dev, num, sizeof(*qmp->vregs), GFP_KERNEL);
- if (!qmp->vregs)
- return -ENOMEM;
-
- for (i = 0; i < num; i++)
- qmp->vregs[i].supply = cfg->vreg_list[i].name;
-
- ret = devm_regulator_bulk_get(dev, num, qmp->vregs);
- if (ret) {
- dev_err(dev, "failed at devm_regulator_bulk_get\n");
- return ret;
- }
-
- for (i = 0; i < num; i++) {
- ret = regulator_set_load(qmp->vregs[i].consumer,
- cfg->vreg_list[i].enable_load);
- if (ret) {
- dev_err(dev, "failed to set load at %s\n",
- qmp->vregs[i].supply);
- return ret;
- }
- }
-
- return 0;
-}
-
static int qmp_combo_reset_init(struct qmp_combo *qmp)
{
const struct qmp_phy_cfg *cfg = qmp->cfg;
@@ -3618,6 +4519,8 @@ static int qmp_combo_parse_dt(struct qmp_combo *qmp)
qmp->serdes = base + offs->usb3_serdes;
qmp->pcs_misc = base + offs->usb3_pcs_misc;
qmp->pcs = base + offs->usb3_pcs;
+ if (offs->usb3_pcs_aon)
+ qmp->pcs_aon = base + offs->usb3_pcs_aon;
qmp->pcs_usb = base + offs->usb3_pcs_usb;
qmp->dp_serdes = base + offs->dp_serdes;
@@ -3687,7 +4590,8 @@ static int qmp_combo_probe(struct platform_device *pdev)
if (ret)
return ret;
- ret = qmp_combo_vreg_init(qmp);
+ ret = devm_regulator_bulk_get_const(dev, qmp->cfg->num_vregs,
+ qmp->cfg->vreg_list, &qmp->vregs);
if (ret)
return ret;
@@ -3767,6 +4671,14 @@ static int qmp_combo_probe(struct platform_device *pdev)
}
static const struct of_device_id qmp_combo_of_match_table[] = {
+ {
+ .compatible = "qcom,glymur-qmp-usb3-dp-phy",
+ .data = &glymur_usb3dpphy_cfg,
+ },
+ {
+ .compatible = "qcom,sar2130p-qmp-usb3-dp-phy",
+ .data = &sar2130p_usb3dpphy_cfg,
+ },
{
.compatible = "qcom,sc7180-qmp-usb3-dp-phy",
.data = &sc7180_usb3dpphy_cfg,
@@ -3815,6 +4727,10 @@ static const struct of_device_id qmp_combo_of_match_table[] = {
.compatible = "qcom,sm8650-qmp-usb3-dp-phy",
.data = &sm8650_usb3dpphy_cfg,
},
+ {
+ .compatible = "qcom,sm8750-qmp-usb3-dp-phy",
+ .data = &sm8750_usb3dpphy_cfg,
+ },
{
.compatible = "qcom,x1e80100-qmp-usb3-dp-phy",
.data = &x1e80100_usb3dpphy_cfg,
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-dp-phy-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-dp-phy-v8.h
new file mode 100644
index 000000000000..b6a8ab59c2ff
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-dp-phy-v8.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2017, The Linux Foundation. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_DP_PHY_V8_H_
+#define QCOM_PHY_QMP_DP_PHY_V8_H_
+
+/* Only for QMP V8 PHY - DP PHY registers */
+#define QSERDES_V8_DP_PHY_VCO_DIV 0x070
+#define QSERDES_V8_DP_PHY_AUX_INTERRUPT_STATUS 0x0e0
+#define QSERDES_V8_DP_PHY_TSYNC_OVRD 0x074
+#define QSERDES_V8_DP_PHY_TX0_TX1_LANE_CTL 0x078
+#define QSERDES_V8_DP_PHY_TX2_TX3_LANE_CTL 0x0bc
+#define QSERDES_V8_DP_PHY_AUXLESS_CFG1 0x0c8
+#define QSERDES_V8_DP_PHY_LFPS_PERIOD 0x0d0
+#define QSERDES_V8_DP_PHY_LFPS_CYC 0x0d4
+#define QSERDES_V8_DP_PHY_AUXLESS_SETUP_CYC 0x0d8
+#define QSERDES_V8_DP_PHY_AUXLESS_SILENCE_CYC 0x0d8
+#define QSERDES_V8_DP_PHY_LN0_DRV_LVL 0x0e0
+#define QSERDES_V8_DP_PHY_LN1_DRV_LVL 0x0e4
+#define QSERDES_V8_DP_PHY_STATUS 0x114
+
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-dp-qserdes-com-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-dp-qserdes-com-v8.h
new file mode 100644
index 000000000000..2bef1eecdc56
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-dp-qserdes-com-v8.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2025 Linaro Ltd.
+ */
+
+#ifndef QCOM_PHY_QMP_DP_QSERDES_COM_V8_H_
+#define QCOM_PHY_QMP_DP_QSERDES_COM_V8_H_
+
+/* Only for DP QMP V8 PHY - QSERDES COM registers */
+#define DP_QSERDES_V8_COM_HSCLK_SEL_1 0x03c
+#define DP_QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE1_MODE0 0x058
+#define DP_QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE2_MODE0 0x05c
+#define DP_QSERDES_V8_COM_SSC_STEP_SIZE1_MODE0 0x060
+#define DP_QSERDES_V8_COM_SSC_STEP_SIZE2_MODE0 0x064
+#define DP_QSERDES_V8_COM_CP_CTRL_MODE0 0x070
+#define DP_QSERDES_V8_COM_PLL_RCTRL_MODE0 0x074
+#define DP_QSERDES_V8_COM_PLL_CCTRL_MODE0 0x078
+#define DP_QSERDES_V8_COM_CORECLK_DIV_MODE0 0x07c
+#define DP_QSERDES_V8_COM_LOCK_CMP1_MODE0 0x080
+#define DP_QSERDES_V8_COM_LOCK_CMP2_MODE0 0x084
+#define DP_QSERDES_V8_COM_DEC_START_MODE0 0x088
+#define DP_QSERDES_V8_COM_DIV_FRAC_START1_MODE0 0x090
+#define DP_QSERDES_V8_COM_DIV_FRAC_START2_MODE0 0x094
+#define DP_QSERDES_V8_COM_DIV_FRAC_START3_MODE0 0x098
+#define DP_QSERDES_V8_COM_INTEGLOOP_GAIN0_MODE0 0x0a0
+#define DP_QSERDES_V8_COM_VCO_TUNE1_MODE0 0x0a8
+#define DP_QSERDES_V8_COM_INTEGLOOP_GAIN1_MODE0 0x0a4
+#define DP_QSERDES_V8_COM_VCO_TUNE2_MODE0 0x0ac
+#define DP_QSERDES_V8_COM_BG_TIMER 0x0bc
+#define DP_QSERDES_V8_COM_SSC_EN_CENTER 0x0c0
+#define DP_QSERDES_V8_COM_SSC_ADJ_PER1 0x0c4
+#define DP_QSERDES_V8_COM_SSC_PER1 0x0cc
+#define DP_QSERDES_V8_COM_SSC_PER2 0x0d0
+#define DP_QSERDES_V8_COM_BIAS_EN_CLKBUFLR_EN 0x0dc
+#define DP_QSERDES_V8_COM_CLK_ENABLE1 0x0e0
+#define DP_QSERDES_V8_COM_SYS_CLK_CTRL 0x0e4
+#define DP_QSERDES_V8_COM_SYSCLK_BUF_ENABLE 0x0e8
+#define DP_QSERDES_V8_COM_PLL_IVCO 0x0f4
+#define DP_QSERDES_V8_COM_SYSCLK_EN_SEL 0x110
+#define DP_QSERDES_V8_COM_RESETSM_CNTRL 0x118
+#define DP_QSERDES_V8_COM_LOCK_CMP_EN 0x120
+#define DP_QSERDES_V8_COM_VCO_TUNE_CTRL 0x13c
+#define DP_QSERDES_V8_COM_VCO_TUNE_MAP 0x140
+#define DP_QSERDES_V8_COM_CLK_SELECT 0x164
+#define DP_QSERDES_V8_COM_CORE_CLK_EN 0x170
+#define DP_QSERDES_V8_COM_CMN_CONFIG_1 0x174
+#define DP_QSERDES_V8_COM_SVS_MODE_CLK_SEL 0x180
+#define DP_QSERDES_V8_COM_CLK_FWD_CONFIG_1 0x2f4
+#define DP_QSERDES_V8_COM_CMN_STATUS 0x314
+#define DP_QSERDES_V8_COM_C_READY_STATUS 0x33c
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v6.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v6.h
new file mode 100644
index 000000000000..52db31a7cf22
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v6.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025, Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_AON_V6_H_
+#define QCOM_PHY_QMP_PCS_AON_V6_H_
+
+/* Only for QMP V6 PHY - PCS_AON registers */
+#define QPHY_V6_PCS_AON_CLAMP_ENABLE 0x00
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v8.h
new file mode 100644
index 000000000000..f6a275c0938f
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-aon-v8.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025, Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_AON_V8_H_
+#define QCOM_PHY_QMP_PCS_AON_V8_H_
+
+/* Only for QMP V8 PHY - PCS_AON registers */
+#define QPHY_V8_PCS_AON_USB3_AON_CLAMP_ENABLE 0x00
+#define QPHY_V8_PCS_AON_USB4_AON_CLAMP_ENABLE 0x04
+#define QPHY_V8_PCS_AON_USB3_AON_TOGGLE_ENABLE 0x08
+#define QPHY_V8_PCS_AON_USB4_AON_TOGGLE_ENABLE 0x0c
+#define QPHY_V8_PCS_AON_DP_AON_TOGGLE_ENABLE 0x10
+#define QPHY_V8_PCS_AON_DUMMY_STATUS 0x14
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v5.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v5.h
new file mode 100644
index 000000000000..77d04c6a1644
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v5.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_MISC_V5_H_
+#define QCOM_PHY_QMP_PCS_MISC_V5_H_
+
+/* Only for QMP V5 PHY - PCS_MISC registers */
+#define QPHY_V5_PCS_MISC_CLAMP_ENABLE 0x0c
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v8.h
new file mode 100644
index 000000000000..a93ef2faa894
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-misc-v8.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_MISC_V8_H_
+#define QCOM_PHY_QMP_PCS_MISC_V8_H_
+
+/* Only for QMP V8 PHY - PCS_MISC registers */
+#define QPHY_V8_PCS_MISC_PCS_MISC_CONFIG1 0x08
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v8.h
new file mode 100644
index 000000000000..89ace8024bc0
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-usb-v8.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_USB_V8_H_
+#define QCOM_PHY_QMP_PCS_USB_V8_H_
+
+#define QPHY_V8_PCS_USB_POWER_STATE_CONFIG1 0x00
+#define QPHY_V8_PCS_USB_AUTONOMOUS_MODE_STATUS 0x04
+#define QPHY_V8_PCS_USB_AUTONOMOUS_MODE_CTRL 0x08
+#define QPHY_V8_PCS_USB_AUTONOMOUS_MODE_CTRL2 0x0c
+#define QPHY_V8_PCS_USB_LFPS_RXTERM_IRQ_SOURCE_STATUS 0x10
+#define QPHY_V8_PCS_USB_LFPS_RXTERM_IRQ_CLEAR 0x14
+#define QPHY_V8_PCS_USB_LFPS_DET_HIGH_COUNT_VAL 0x18
+#define QPHY_V8_PCS_USB_LFPS_TX_ECSTART 0x1c
+#define QPHY_V8_PCS_USB_LFPS_PER_TIMER_VAL 0x20
+#define QPHY_V8_PCS_USB_LFPS_TX_END_CNT_U3_START 0x24
+#define QPHY_V8_PCS_USB_LFPS_CONFIG1 0x28
+#define QPHY_V8_PCS_USB_RXEQTRAINING_LOCK_TIME 0x2c
+#define QPHY_V8_PCS_USB_RXEQTRAINING_WAIT_TIME 0x30
+#define QPHY_V8_PCS_USB_RXEQTRAINING_CTLE_TIME 0x34
+#define QPHY_V8_PCS_USB_RXEQTRAINING_WAIT_TIME_S2 0x38
+#define QPHY_V8_PCS_USB_RXEQTRAINING_DFE_TIME_S2 0x3c
+#define QPHY_V8_PCS_USB_RCVR_DTCT_DLY_U3_L 0x40
+#define QPHY_V8_PCS_USB_RCVR_DTCT_DLY_U3_H 0x44
+#define QPHY_V8_PCS_USB_ARCVR_DTCT_EN_PERIOD 0x48
+#define QPHY_V8_PCS_USB_ARCVR_DTCT_CM_DLY 0x4c
+#define QPHY_V8_PCS_USB_TXONESZEROS_RUN_LENGTH 0x50
+#define QPHY_V8_PCS_USB_ALFPS_DEGLITCH_VAL 0x54
+#define QPHY_V8_PCS_USB_SIGDET_STARTUP_TIMER_VAL 0x58
+#define QPHY_V8_PCS_USB_TEST_CONTROL 0x5c
+#define QPHY_V8_PCS_USB_RXTERMINATION_DLY_SEL 0x60
+#define QPHY_V8_PCS_USB_POWER_STATE_CONFIG2 0x64
+#define QPHY_V8_PCS_USB_POWER_STATE_CONFIG3 0x68
+#define QPHY_V8_PCS_USB_POWER_STATE_CONFIG4 0x6c
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-pcs-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-v8.h
new file mode 100644
index 000000000000..169fd5de7474
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-pcs-v8.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_PCS_V8_H_
+#define QCOM_PHY_QMP_PCS_V8_H_
+
+/* Only for QMP V8 PHY - USB/PCIe PCS registers */
+#define QPHY_V8_PCS_SW_RESET 0x000
+#define QPHY_V8_PCS_PCS_STATUS1 0x014
+#define QPHY_V8_PCS_POWER_DOWN_CONTROL 0x040
+#define QPHY_V8_PCS_START_CONTROL 0x044
+#define QPHY_V8_PCS_POWER_STATE_CONFIG1 0x090
+#define QPHY_V8_PCS_LOCK_DETECT_CONFIG1 0x0c4
+#define QPHY_V8_PCS_LOCK_DETECT_CONFIG2 0x0c8
+#define QPHY_V8_PCS_LOCK_DETECT_CONFIG3 0x0cc
+#define QPHY_V8_PCS_LOCK_DETECT_CONFIG6 0x0d8
+#define QPHY_V8_PCS_REFGEN_REQ_CONFIG1 0x0dc
+#define QPHY_V8_PCS_RX_SIGDET_LVL 0x188
+#define QPHY_V8_PCS_RCVR_DTCT_DLY_P1U2_L 0x190
+#define QPHY_V8_PCS_RCVR_DTCT_DLY_P1U2_H 0x194
+#define QPHY_V8_PCS_RATE_SLEW_CNTRL1 0x198
+#define QPHY_V8_PCS_CDR_RESET_TIME 0x1b0
+#define QPHY_V8_PCS_ALIGN_DETECT_CONFIG1 0x1c0
+#define QPHY_V8_PCS_ALIGN_DETECT_CONFIG2 0x1c4
+#define QPHY_V8_PCS_PCS_TX_RX_CONFIG 0x1d0
+#define QPHY_V8_PCS_EQ_CONFIG1 0x1dc
+#define QPHY_V8_PCS_EQ_CONFIG2 0x1e0
+#define QPHY_V8_PCS_EQ_CONFIG5 0x1ec
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-com-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-com-v8.h
new file mode 100644
index 000000000000..d3b2292257bc
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-com-v8.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_QSERDES_COM_V8_H_
+#define QCOM_PHY_QMP_QSERDES_COM_V8_H_
+
+/* Only for QMP V8 PHY - QSERDES COM registers */
+#define QSERDES_V8_COM_SSC_STEP_SIZE1_MODE1 0x000
+#define QSERDES_V8_COM_SSC_STEP_SIZE2_MODE1 0x004
+#define QSERDES_V8_COM_SSC_STEP_SIZE3_MODE1 0x008
+#define QSERDES_V8_COM_CP_CTRL_MODE1 0x010
+#define QSERDES_V8_COM_PLL_RCTRL_MODE1 0x014
+#define QSERDES_V8_COM_PLL_CCTRL_MODE1 0x018
+#define QSERDES_V8_COM_CORECLK_DIV_MODE1 0x01c
+#define QSERDES_V8_COM_LOCK_CMP1_MODE1 0x020
+#define QSERDES_V8_COM_LOCK_CMP2_MODE1 0x024
+#define QSERDES_V8_COM_DEC_START_MODE1 0x028
+#define QSERDES_V8_COM_DEC_START_MSB_MODE1 0x02c
+#define QSERDES_V8_COM_DIV_FRAC_START1_MODE1 0x030
+#define QSERDES_V8_COM_DIV_FRAC_START2_MODE1 0x034
+#define QSERDES_V8_COM_DIV_FRAC_START3_MODE1 0x038
+#define QSERDES_V8_COM_HSCLK_SEL_1 0x03c
+#define QSERDES_V8_COM_VCO_TUNE1_MODE1 0x048
+#define QSERDES_V8_COM_VCO_TUNE2_MODE1 0x04c
+#define QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE1_MODE1 0x050
+#define QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE2_MODE1 0x054
+#define QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE1_MODE0 0x058
+#define QSERDES_V8_COM_BIN_VCOCAL_CMP_CODE2_MODE0 0x05c
+#define QSERDES_V8_COM_SSC_STEP_SIZE1_MODE0 0x060
+#define QSERDES_V8_COM_SSC_STEP_SIZE2_MODE0 0x064
+#define QSERDES_V8_COM_CP_CTRL_MODE0 0x070
+#define QSERDES_V8_COM_PLL_RCTRL_MODE0 0x074
+#define QSERDES_V8_COM_PLL_CCTRL_MODE0 0x078
+#define QSERDES_V8_COM_LOCK_CMP1_MODE0 0x080
+#define QSERDES_V8_COM_LOCK_CMP2_MODE0 0x084
+#define QSERDES_V8_COM_DEC_START_MODE0 0x088
+#define QSERDES_V8_COM_DEC_START_MSB_MODE0 0x08c
+#define QSERDES_V8_COM_DIV_FRAC_START1_MODE0 0x090
+#define QSERDES_V8_COM_DIV_FRAC_START2_MODE0 0x094
+#define QSERDES_V8_COM_DIV_FRAC_START3_MODE0 0x098
+#define QSERDES_V8_COM_VCO_TUNE1_MODE0 0x0a8
+#define QSERDES_V8_COM_VCO_TUNE2_MODE0 0x0ac
+#define QSERDES_V8_COM_BG_TIMER 0x0bc
+#define QSERDES_V8_COM_SSC_EN_CENTER 0x0c0
+#define QSERDES_V8_COM_SSC_PER1 0x0cc
+#define QSERDES_V8_COM_SSC_PER2 0x0d0
+#define QSERDES_V8_COM_BIAS_EN_CLKBUFLR_EN 0x0dc
+#define QSERDES_V8_COM_SYSCLK_BUF_ENABLE 0x0e8
+#define QSERDES_V8_COM_SYSCLK_EN_SEL 0x110
+#define QSERDES_V8_COM_RESETSM_CNTRL 0x118
+#define QSERDES_V8_COM_LOCK_CMP_CFG 0x124
+#define QSERDES_V8_COM_VCO_TUNE_MAP 0x140
+#define QSERDES_V8_COM_CORE_CLK_EN 0x170
+#define QSERDES_V8_COM_CMN_CONFIG_1 0x174
+#define QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_1 0x1a4
+#define QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_2 0x1a8
+#define QSERDES_V8_COM_AUTO_GAIN_ADJ_CTRL_3 0x1ac
+#define QSERDES_V8_COM_ADDITIONAL_MISC 0x1b4
+#define QSERDES_V8_COM_CMN_STATUS 0x2c8
+#define QSERDES_V8_COM_C_READY_STATUS 0x2f0
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-lalb-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-lalb-v8.h
new file mode 100644
index 000000000000..60ba730620f8
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-lalb-v8.h
@@ -0,0 +1,639 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_QSERDES_V8_LALBH_
+#define QCOM_PHY_QMP_QSERDES_V8_LALBH_
+
+#define QSERDES_V8_LALB_BIST_MODE_LANENO 0x0
+#define QSERDES_V8_LALB_BIST_INVERT 0x4
+#define QSERDES_V8_LALB_PERL_LENGTH1 0x8
+#define QSERDES_V8_LALB_PERL_LENGTH2 0xc
+#define QSERDES_V8_LALB_BIST_PATTERN1 0x10
+#define QSERDES_V8_LALB_BIST_PATTERN2 0x14
+#define QSERDES_V8_LALB_BIST_PATTERN3 0x18
+#define QSERDES_V8_LALB_BIST_PATTERN4 0x1c
+#define QSERDES_V8_LALB_BIST_PATTERN5 0x20
+#define QSERDES_V8_LALB_BIST_PATTERN6 0x24
+#define QSERDES_V8_LALB_BIST_PATTERN7 0x28
+#define QSERDES_V8_LALB_BIST_PATTERN8 0x2c
+#define QSERDES_V8_LALB_PRBS_SEED1 0x30
+#define QSERDES_V8_LALB_PRBS_SEED2 0x34
+#define QSERDES_V8_LALB_PRBS_SEED3 0x38
+#define QSERDES_V8_LALB_PRBS_SEED4 0x3c
+#define QSERDES_V8_LALB_PRBS_SEED5 0x40
+#define QSERDES_V8_LALB_PRBS_SEED6 0x44
+#define QSERDES_V8_LALB_PRBS_SEED7 0x48
+#define QSERDES_V8_LALB_SW_RESET_PWRDNB 0x4c
+#define QSERDES_V8_LALB_RESET_GEN 0x50
+#define QSERDES_V8_LALB_RESET_TSYNC_EN_CTRL 0x54
+#define QSERDES_V8_LALB_CDR_EN_RXEQ_RESET 0x58
+#define QSERDES_V8_LALB_CLKBUF_ENABLE 0x5c
+#define QSERDES_V8_LALB_TX0_EMP_POST1_LVL 0x60
+#define QSERDES_V8_LALB_TX1_EMP_POST1_LVL 0x64
+#define QSERDES_V8_LALB_TX0_IDLE_CTRL 0x68
+#define QSERDES_V8_LALB_TX1_IDLE_CTRL 0x6c
+#define QSERDES_V8_LALB_TX0_DRV_LVL 0x70
+#define QSERDES_V8_LALB_TX0_DRV_LVL_OFFSET 0x74
+#define QSERDES_V8_LALB_TX1_DRV_LVL 0x78
+#define QSERDES_V8_LALB_TX1_DRV_LVL_OFFSET 0x7c
+#define QSERDES_V8_LALB_TRAN_DRVR_EMP_EN 0x80
+#define QSERDES_V8_LALB_TX_LVL_UPDATE_CTRL 0x84
+#define QSERDES_V8_LALB_TX0_PRE1_EMPH 0x88
+#define QSERDES_V8_LALB_TX1_PRE1_EMPH 0x8c
+#define QSERDES_V8_LALB_TX0_PRE2_EMPH 0x90
+#define QSERDES_V8_LALB_TX1_PRE2_EMPH 0x94
+#define QSERDES_V8_LALB_STALL_LDO_BOOST_EN 0x98
+#define QSERDES_V8_LALB_PRE_EMPH_EN_CTRL 0x9c
+#define QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL1 0xa0
+#define QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL2 0xa4
+#define QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL3 0xa8
+#define QSERDES_V8_LALB_PCIE5_TOP_LDO_CODE_CTRL4 0xac
+#define QSERDES_V8_LALB_TRANSMITTER_EN_CTRL 0xb0
+#define QSERDES_V8_LALB_HIGHZ_DRVR_EN 0xb4
+#define QSERDES_V8_LALB_TX_MISC_CTRL1 0xb8
+#define QSERDES_V8_LALB_LPB_EN_CTRL1 0xbc
+#define QSERDES_V8_LALB_LBP_EN_CTRL2 0xc0
+#define QSERDES_V8_LALB_TX0_SERDES_BYP_CTRL 0xc4
+#define QSERDES_V8_LALB_TX1_SERDES_BYP_CTRL 0xc8
+#define QSERDES_V8_LALB_LANE_MODE_1 0xcc
+#define QSERDES_V8_LALB_LANE_MODE_2 0xd0
+#define QSERDES_V8_LALB_LANE_MODE_3 0xd4
+#define QSERDES_V8_LALB_LANE_MODE_4 0xd8
+#define QSERDES_V8_LALB_ATB_SEL1 0xdc
+#define QSERDES_V8_LALB_ATB_SEL2 0xe0
+#define QSERDES_V8_LALB_TX0_RES_CODE_LANE 0xe4
+#define QSERDES_V8_LALB_TX0_RESTRIM_ICAL_OVRD 0xe8
+#define QSERDES_V8_LALB_TX0_RESTRIM_CAL_CTRL 0xec
+#define QSERDES_V8_LALB_TX0_RESTRIM_INIT_CODE 0xf0
+#define QSERDES_V8_LALB_TX0_RESTRIM_POST_CAL_OFFSET 0xf4
+#define QSERDES_V8_LALB_TX1_RES_CODE_LANE 0xf8
+#define QSERDES_V8_LALB_TX1_RESTRIM_ICAL_OVRD 0xfc
+#define QSERDES_V8_LALB_TX1_RESTRIM_CAL_CTRL 0x100
+#define QSERDES_V8_LALB_TX1_RESTRIM_INIT_CODE 0x104
+#define QSERDES_V8_LALB_TX1_RESTRIM_POST_CAL_OFFSET 0x108
+#define QSERDES_V8_LALB_TX0_RESTRIM_VREF_SEL 0x10c
+#define QSERDES_V8_LALB_TX1_RESTRIM_VREF_SEL 0x110
+#define QSERDES_V8_LALB_VMODE_CTRL1 0x114
+#define QSERDES_V8_LALB_SLEW_CNTL_RATE01 0x118
+#define QSERDES_V8_LALB_SLEW_CNTL_RATE23 0x11c
+#define QSERDES_V8_LALB_SLEW_CNTL_RATE4 0x120
+#define QSERDES_V8_LALB_ANA_INTERFACE_SELECT1 0x124
+#define QSERDES_V8_LALB_ANA_INTERFACE_SELECT2 0x128
+#define QSERDES_V8_LALB_ANA_INTERFACE_SELECT3 0x12c
+#define QSERDES_V8_LALB_PCS_INTERFACE_SELECT1 0x130
+#define QSERDES_V8_LALB_PCS_INTERFACE_SELECT2 0x134
+#define QSERDES_V8_LALB_LDO_TIMER_CTRL 0x138
+#define QSERDES_V8_LALB_AC_JTAG_ENABLE 0x13c
+#define QSERDES_V8_LALB_AC_JTAG_INITP 0x140
+#define QSERDES_V8_LALB_AC_JTAG_INITN 0x144
+#define QSERDES_V8_LALB_AC_JTAG_LVL 0x148
+#define QSERDES_V8_LALB_AC_JTAG_MODE 0x14c
+#define QSERDES_V8_LALB_AC_JTAG_RESET 0x150
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B0 0x154
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B1 0x158
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B2 0x15c
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B3 0x160
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B4 0x164
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B5 0x168
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B6 0x16c
+#define QSERDES_V8_LALB_RX_MODE_RATE_0_1_B7 0x170
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B0 0x174
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B1 0x178
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B2 0x17c
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B3 0x180
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B4 0x184
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B5 0x188
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B6 0x18c
+#define QSERDES_V8_LALB_RX_MODE_RATE2_B7 0x190
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B0 0x194
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B1 0x198
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B2 0x19c
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B3 0x1a0
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B4 0x1a4
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B5 0x1a8
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B6 0x1ac
+#define QSERDES_V8_LALB_RX_MODE_RATE3_B7 0x1b0
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B0 0x1b4
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B1 0x1b8
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B2 0x1bc
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B3 0x1c0
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B4 0x1c4
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B5 0x1c8
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B6 0x1cc
+#define QSERDES_V8_LALB_RX_MODE_RATE4_B7 0x1d0
+#define QSERDES_V8_LALB_TX_DCC_ANA_CTRL1 0x1d4
+#define QSERDES_V8_LALB_TX_DCC_ANA_CTRL2 0x1d8
+#define QSERDES_V8_LALB_CMUX_DCC_CTRL1 0x1dc
+#define QSERDES_V8_LALB_CMUX_DCC_POSTCAL_OFFSET 0x1e0
+#define QSERDES_V8_LALB_CMUX_DCC_OVRD 0x1e4
+#define QSERDES_V8_LALB_TX_DCC_CTRL 0x1e8
+#define QSERDES_V8_LALB_TX0_CTUNE_DCC_CONFIG 0x1ec
+#define QSERDES_V8_LALB_TX0_CTUNE_DCC_POSTCAL_OFFSET 0x1f0
+#define QSERDES_V8_LALB_TX0_CTUNE_DCC_OVRD 0x1f4
+#define QSERDES_V8_LALB_TX0_FTUNE_MSB_DCC_CONFIG 0x1f8
+#define QSERDES_V8_LALB_TX0_FTUNE_MSB_DCC_OFFSET_AND_OVRD 0x1fc
+#define QSERDES_V8_LALB_TX0_FTUNE_LSB_DCC_CONFIG 0x200
+#define QSERDES_V8_LALB_TX0_FTUNE_LSB_DCC_OFFSET_AND_OVRD 0x204
+#define QSERDES_V8_LALB_TX1_CTUNE_DCC_CONFIG 0x208
+#define QSERDES_V8_LALB_TX1_CTUNE_DCC_POSTCAL_OFFSET 0x20c
+#define QSERDES_V8_LALB_TX1_CTUNE_DCC_OVRD 0x210
+#define QSERDES_V8_LALB_TX1_FTUNE_MSB_DCC_CONFIG 0x214
+#define QSERDES_V8_LALB_TX1_FTUNE_MSB_DCC_OFFSET_AND_OVRD 0x218
+#define QSERDES_V8_LALB_TX1_FTUNE_LSB_DCC_CONFIG 0x21c
+#define QSERDES_V8_LALB_TX1_FTUNE_LSB_DCC_OFFSET_AND_OVRD 0x220
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CTRL 0x224
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CODE_OVRD_RATE0 0x228
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CODE_OVRD_RATE1 0x22c
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CODE_OVRD_RATE2 0x230
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CODE_OVRD_RATE3 0x234
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_CODE_OVRD_RATE4 0x238
+#define QSERDES_V8_LALB_CDR_VCO_CAL_CTRL 0x23c
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE0 0x240
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE0 0x244
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE1 0x248
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE1 0x24c
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE2 0x250
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE2 0x254
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE3 0x258
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE3 0x25c
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT1_RATE4 0x260
+#define QSERDES_V8_LALB_CDR_VCO_CTUNE_MEAS_CNT2_RATE4 0x264
+#define QSERDES_V8_LALB_CDR_VCTRL_RATE_0_1 0x268
+#define QSERDES_V8_LALB_CDR_VCTRL_RATE_2_3 0x26c
+#define QSERDES_V8_LALB_CDR_VCTRL_RATE_4 0x270
+#define QSERDES_V8_LALB_KVCO_INIT_RATE_0_1 0x274
+#define QSERDES_V8_LALB_KVCO_INIT_RATE_2_3 0x278
+#define QSERDES_V8_LALB_KVCO_INIT_RATE_4 0x27c
+#define QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE0 0x280
+#define QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE1 0x284
+#define QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE2 0x288
+#define QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE3 0x28c
+#define QSERDES_V8_LALB_KVCO_CODE_OVRD_RATE4 0x290
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_HIGH_RATE_0_1 0x294
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_HIGH_RATE_2_3 0x298
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_HIGH_RATE_4 0x29c
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_LOW_RATE_0_1 0x2a0
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_LOW_RATE_2_3 0x2a4
+#define QSERDES_V8_LALB_KVCO_CAL_VCTRL_LOW_RATE_4 0x2a8
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE0 0x2ac
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE0 0x2b0
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE1 0x2b4
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE1 0x2b8
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE2 0x2bc
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE2 0x2c0
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE3 0x2c4
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE3 0x2c8
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF1_RATE4 0x2cc
+#define QSERDES_V8_LALB_KVCO_IDEAL_FREQ_DIFF2_RATE4 0x2d0
+#define QSERDES_V8_LALB_KP_CDR_UP_DN 0x2d4
+#define QSERDES_V8_LALB_KP_CODE_OVRD_RATE_0_1 0x2d8
+#define QSERDES_V8_LALB_KP_CODE_OVRD_RATE_2_3 0x2dc
+#define QSERDES_V8_LALB_KP_CODE_OVRD_RATE4 0x2e0
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE0 0x2e4
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE0 0x2e8
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE1 0x2ec
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE1 0x2f0
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE2 0x2f4
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE2 0x2f8
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE3 0x2fc
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE3 0x300
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND1_RATE4 0x304
+#define QSERDES_V8_LALB_KP_CAL_UPPER_FREQ_DIFF_BND2_RATE4 0x308
+#define QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE0 0x30c
+#define QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE1 0x310
+#define QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE2 0x314
+#define QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE3 0x318
+#define QSERDES_V8_LALB_KP_CAL_LOWER_FREQ_DIFF_BND_RATE4 0x31c
+#define QSERDES_V8_LALB_CDR_KVCO_KP_CAL_FREQ_MEAS_CTRL 0x320
+#define QSERDES_V8_LALB_PLLLOCK_CMP_DEBUG_CTRL 0x324
+#define QSERDES_V8_LALB_PLLLOCK_CMP_DEBUG_CNT1 0x328
+#define QSERDES_V8_LALB_PLLLOCK_CMP_DEBUG_CNT2 0x32c
+#define QSERDES_V8_LALB_PLLLOCK_CMP_DEBUG_CNT3 0x330
+#define QSERDES_V8_LALB_RX_SUMMER_CAL_SPD_MODE_RATE_0123 0x334
+#define QSERDES_V8_LALB_RX_SUMMER_CAL_SPD_MODE_RATE_4 0x338
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE0 0x33c
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE1 0x340
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE2 0x344
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE3 0x348
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CODE_OVERRIDE_RATE4 0x34c
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CTRL1 0x350
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CTRL2 0x354
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CTRL3 0x358
+#define QSERDES_V8_LALB_RX_IVCM_CAL_CTRL4 0x35c
+#define QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE0 0x360
+#define QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE1 0x364
+#define QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE2 0x368
+#define QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE3 0x36c
+#define QSERDES_V8_LALB_RX_IVCM_POSTCAL_OFFSET_RATE4 0x370
+#define QSERDES_V8_LALB_RX_IDAC_I0_DC_OFFSETS 0x374
+#define QSERDES_V8_LALB_RX_IDAC_I0BAR_DC_OFFSETS 0x378
+#define QSERDES_V8_LALB_RX_IDAC_I1_DC_OFFSETS 0x37c
+#define QSERDES_V8_LALB_RX_IDAC_I1BAR_DC_OFFSETS 0x380
+#define QSERDES_V8_LALB_RX_IDAC_Q_DC_OFFSETS 0x384
+#define QSERDES_V8_LALB_RX_IDAC_QBAR_DC_OFFSETS 0x388
+#define QSERDES_V8_LALB_RX_IDAC_A_DC_OFFSETS 0x38c
+#define QSERDES_V8_LALB_RX_IDAC_ABAR_DC_OFFSETS 0x390
+#define QSERDES_V8_LALB_RX_IDAC_EN 0x394
+#define QSERDES_V8_LALB_DATA_SLICER_INIT_TIMER_CTRL 0x398
+#define QSERDES_V8_LALB_RX_IDAC_ENABLES 0x39c
+#define QSERDES_V8_LALB_RX_IDAC_SIGN 0x3a0
+#define QSERDES_V8_LALB_RX_IDAC_TSETTLE 0x3a4
+#define QSERDES_V8_LALB_SIGDET_ENABLES 0x3a8
+#define QSERDES_V8_LALB_SIGDET_CNTRL 0x3ac
+#define QSERDES_V8_LALB_SIGDET_LVL 0x3b0
+#define QSERDES_V8_LALB_SIGDET_DEGLITCH_CNTRL 0x3b4
+#define QSERDES_V8_LALB_SIGDET_CAL_CTRL1 0x3b8
+#define QSERDES_V8_LALB_SIGDET_CAL_CTRL2_AND_CDR_LOCK_EDGE 0x3bc
+#define QSERDES_V8_LALB_SIGDET_CAL_TRIM 0x3c0
+#define QSERDES_V8_LALB_IA_OFFSET_CENTER_CAL_CTRL 0x3c4
+#define QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE0 0x3c8
+#define QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE1 0x3cc
+#define QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE2 0x3d0
+#define QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE3 0x3d4
+#define QSERDES_V8_LALB_FREQ_LOCK_DET_DLY_RATE4 0x3d8
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_RATE0 0x3dc
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_RATE1 0x3e0
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_RATE2 0x3e4
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_RATE3 0x3e8
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_RATE4 0x3ec
+#define QSERDES_V8_LALB_CDR_LOCK_CTRL 0x3f0
+#define QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE0 0x3f4
+#define QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE1 0x3f8
+#define QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE2 0x3fc
+#define QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE3 0x400
+#define QSERDES_V8_LALB_CDR_CP_CUR_FLL_RATE4 0x404
+#define QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE0 0x408
+#define QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE1 0x40c
+#define QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE2 0x410
+#define QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE3 0x414
+#define QSERDES_V8_LALB_CDR_CP_CUR_PLL_RATE4 0x418
+#define QSERDES_V8_LALB_CDR_FLL_DIV_RATIO_RATE_0123 0x41c
+#define QSERDES_V8_LALB_CDR_FLL_DIV_RATIO_RATE4 0x420
+#define QSERDES_V8_LALB_CDR_LOOP_CCODE_RATE_01 0x424
+#define QSERDES_V8_LALB_CDR_LOOP_CCODE_RATE_23 0x428
+#define QSERDES_V8_LALB_CDR_LOOP_CCODE_RATE4 0x42c
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FAST_RATE_0_1 0x430
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FAST_RATE_2_3 0x434
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FAST_RATE4 0x438
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FLL_RATE_0_1 0x43c
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FLL_RATE_2_3 0x440
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_FLL_RATE4 0x444
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_PLL_RATE_0_1 0x448
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_PLL_RATE_2_3 0x44c
+#define QSERDES_V8_LALB_CDR_LOOP_RCODE_PLL_RATE4 0x450
+#define QSERDES_V8_LALB_CDR_VCO_CAP_CODE_RATE_0123 0x454
+#define QSERDES_V8_LALB_CDR_VCO_CAP_CODE_RATE4 0x458
+#define QSERDES_V8_LALB_CDR_VCO_TYPE_CONFIG 0x45c
+#define QSERDES_V8_LALB_CDR_VCO_EN_LOWFREQ 0x460
+#define QSERDES_V8_LALB_CDR_FAST_SLOW_VCO_OVRD 0x464
+#define QSERDES_V8_LALB_CDR_LOOP_FUNC_CTRL 0x468
+#define QSERDES_V8_LALB_CDR_FAST_LOCK_EN_CTRL 0x46c
+#define QSERDES_V8_LALB_RX_RCVR_EN 0x470
+#define QSERDES_V8_LALB_LANE_RATE_CTRL 0x474
+#define QSERDES_V8_LALB_RX_TERM_RCVR_CTRL 0x478
+#define QSERDES_V8_LALB_REC_DETECT_CTRL 0x47c
+#define QSERDES_V8_LALB_RCV_DETECT_LVL 0x480
+#define QSERDES_V8_LALB_GM_CAL_EN 0x484
+#define QSERDES_V8_LALB_GM_CAL_RES_RATE0_1 0x488
+#define QSERDES_V8_LALB_GM_CAL_RES_RATE2_3 0x48c
+#define QSERDES_V8_LALB_GM_CAL_RES_RATE4 0x490
+#define QSERDES_V8_LALB_RX_TERM_BW_RATE_0123 0x494
+#define QSERDES_V8_LALB_RX_TERM_BW_RATE4 0x498
+#define QSERDES_V8_LALB_AUX_CLK_CTRL 0x49c
+#define QSERDES_V8_LALB_AUX_OFFSET_CONTROL 0x4a0
+#define QSERDES_V8_LALB_AUXDATA_TB 0x4a4
+#define QSERDES_V8_LALB_EOM_CTRL1 0x4a8
+#define QSERDES_V8_LALB_EOM_CTRL2 0x4ac
+#define QSERDES_V8_LALB_EOM_CTRL3 0x4b0
+#define QSERDES_V8_LALB_EOM_CTRL4 0x4b4
+#define QSERDES_V8_LALB_DFE_EN_TIMER 0x4b8
+#define QSERDES_V8_LALB_RX_EQ_OFFSET_LSB 0x4bc
+#define QSERDES_V8_LALB_RX_EQ_OFFSET_MSB 0x4c0
+#define QSERDES_V8_LALB_RX_EQ_OFFSET_ADAPTOR_CNTRL1 0x4c4
+#define QSERDES_V8_LALB_RX_OFFSET_ADAPTOR_CNTRL2 0x4c8
+#define QSERDES_V8_LALB_RX_OFFSET_ADAPTOR_CNTRL3 0x4cc
+#define QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL1 0x4d0
+#define QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL2 0x4d4
+#define QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL3 0x4d8
+#define QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL4 0x4dc
+#define QSERDES_V8_LALB_RX_EQU_ADAPTOR_CNTRL5 0x4e0
+#define QSERDES_V8_LALB_RX_EQU_KEQ_UP_LSB 0x4e4
+#define QSERDES_V8_LALB_RX_EQU_KEQ_UP_MSB 0x4e8
+#define QSERDES_V8_LALB_RX_EQU_KEQ_DN_LSB 0x4ec
+#define QSERDES_V8_LALB_RX_EQU_KEQ_DN_MSB 0x4f0
+#define QSERDES_V8_LALB_CTLE_ADP_RESET_INIT_CODE_RATE_0_1_2 0x4f4
+#define QSERDES_V8_LALB_CTLE_ADP_RESET_INIT_CODE_RATE_3_4 0x4f8
+#define QSERDES_V8_LALB_CTLE_POST_CAL_OFFSET_RATE_0_1_2 0x4fc
+#define QSERDES_V8_LALB_CTLE_POST_CAL_OFFSET_RATE_3_4 0x500
+#define QSERDES_V8_LALB_RX_VGA_GAIN2_BLK1 0x504
+#define QSERDES_V8_LALB_RX_VGA_GAIN2_BLK2 0x508
+#define QSERDES_V8_LALB_VGA_CAL_CNTRL1 0x50c
+#define QSERDES_V8_LALB_VGA_CAL_CNTRL2 0x510
+#define QSERDES_V8_LALB_VGA_CAL_MAN_VAL_RATE0_1 0x514
+#define QSERDES_V8_LALB_VGA_CAL_MAN_VAL_RATE2_3 0x518
+#define QSERDES_V8_LALB_VGA_CAL_MAN_VAL_RATE4 0x51c
+#define QSERDES_V8_LALB_KVGA_CTRL1 0x520
+#define QSERDES_V8_LALB_KVGA_CTRL2 0x524
+#define QSERDES_V8_LALB_VTHRESH_CAL_CNTRL1 0x528
+#define QSERDES_V8_LALB_VTHRESH_CAL_CNTRL2 0x52c
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_VAL_RATE0 0x530
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_VAL_RATE1 0x534
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_VAL_RATE2 0x538
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_VAL_RATE3 0x53c
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_VAL_RATE4 0x540
+#define QSERDES_V8_LALB_VTHRESH_CAL_MAN_CAL_PAM3 0x544
+#define QSERDES_V8_LALB_VTH_POST_CAL_OFFSET_RATE_0_1 0x548
+#define QSERDES_V8_LALB_VTH_POST_CAL_OFFSET_RATE_2_3 0x54c
+#define QSERDES_V8_LALB_VTH_POST_CAL_OFFSET_RATE4 0x550
+#define QSERDES_V8_LALB_DFE_TAP1_CTRL 0x554
+#define QSERDES_V8_LALB_DFE_TAP1_MANVAL_KTAP 0x558
+#define QSERDES_V8_LALB_DFE_TAP1_POST_CAL_OFFSET_RATE_0_1_2 0x55c
+#define QSERDES_V8_LALB_DFE_TAP1_POST_CAL_OFFSET_RATE_3_4 0x560
+#define QSERDES_V8_LALB_DFE_TAP2_CTRL 0x564
+#define QSERDES_V8_LALB_DFE_TAP2_MANVAL_KTAP 0x568
+#define QSERDES_V8_LALB_DFE_TAP3_CTRL 0x56c
+#define QSERDES_V8_LALB_DFE_TAP3_MANVAL_KTAP 0x570
+#define QSERDES_V8_LALB_DFE_TAP4_CTRL 0x574
+#define QSERDES_V8_LALB_DFE_TAP4_MANVAL_KTAP 0x578
+#define QSERDES_V8_LALB_DFE_TAP5_CTRL 0x57c
+#define QSERDES_V8_LALB_DFE_TAP5_MANVAL_KTAP 0x580
+#define QSERDES_V8_LALB_DFE_TAP6_CTRL 0x584
+#define QSERDES_V8_LALB_DFE_TAP6_MANVAL_KTAP 0x588
+#define QSERDES_V8_LALB_DFE_TAP7_CTRL 0x58c
+#define QSERDES_V8_LALB_DFE_TAP7_MANVAL_KTAP 0x590
+#define QSERDES_V8_LALB_DFE_TAP1_DAC_ENABLE 0x594
+#define QSERDES_V8_LALB_DFE_TAP2_DAC_ENABLE 0x598
+#define QSERDES_V8_LALB_DFE_TAP345_DAC_ENABLE 0x59c
+#define QSERDES_V8_LALB_DFE_TAP67_DAC_ENABLE 0x5a0
+#define QSERDES_V8_LALB_CDR_IQTUNE_CTRL 0x5a4
+#define QSERDES_V8_LALB_CDR_IQTUNE_GAIN 0x5a8
+#define QSERDES_V8_LALB_CDR_IQTUNE_MAN_INDEX 0x5ac
+#define QSERDES_V8_LALB_CDR_IQTUNE_FILTER_CAL_CTRL1 0x5b0
+#define QSERDES_V8_LALB_CDR_IQTUNE_FILTER_CAL_CTRL2 0x5b4
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK0_CAL_CODE_RATE0 0x5b8
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK0_CAL_CODE_RATE1 0x5bc
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK0_CAL_CODE_RATE2 0x5c0
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK0_CAL_CODE_RATE3 0x5c4
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK0_CAL_CODE_RATE4 0x5c8
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK90_CAL_CODE_RATE0 0x5cc
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK90_CAL_CODE_RATE1 0x5d0
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK90_CAL_CODE_RATE2 0x5d4
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK90_CAL_CODE_RATE3 0x5d8
+#define QSERDES_V8_LALB_CDR_IQTUNE_CLK90_CAL_CODE_RATE4 0x5dc
+#define QSERDES_V8_LALB_CDR_IQTUNE_ANA_CTRL 0x5e0
+#define QSERDES_V8_LALB_CDR_IQTUNE_VDCC_CTRL 0x5e4
+#define QSERDES_V8_LALB_CDR_IQTUNE_DIV2_CTRL_RATE0123 0x5e8
+#define QSERDES_V8_LALB_CDR_IQTUNE_DIV2_CTRL_RATE4 0x5ec
+#define QSERDES_V8_LALB_BLW_CTRL 0x5f0
+#define QSERDES_V8_LALB_BLW_ANA_VER_CTRL 0x5f4
+#define QSERDES_V8_LALB_BLW_GAIN_CAL_CTRL 0x5f8
+#define QSERDES_V8_LALB_BLW_GAIN_FORCE_CODE 0x5fc
+#define QSERDES_V8_LALB_BLW_MAN_VAL_RATE3 0x600
+#define QSERDES_V8_LALB_BLW_MAN_VAL_RATE4 0x604
+#define QSERDES_V8_LALB_IVTH_CAL_CTRL1 0x608
+#define QSERDES_V8_LALB_IVTH_CAL_CTRL2 0x60c
+#define QSERDES_V8_LALB_IVTH_CAL_CTRL3 0x610
+#define QSERDES_V8_LALB_VTH_I_UP_CNTRL_VAL 0x614
+#define QSERDES_V8_LALB_VTH_I_DN_CNTRL_VAL 0x618
+#define QSERDES_V8_LALB_NRZ_EYE_HEIGHT_SEL_VAL 0x61c
+#define QSERDES_V8_LALB_IVTH_CAL_VAL_OVRD_MUX 0x620
+#define QSERDES_V8_LALB_CDR_VCO_CAP_CODE_OVRD_MUXES 0x624
+#define QSERDES_V8_LALB_VCO_CTUNE_LOWER_BND_RATE0 0x628
+#define QSERDES_V8_LALB_VCO_CTUNE_LOWER_BND_RATE1 0x62c
+#define QSERDES_V8_LALB_VCO_CTUNE_LOWER_BND_RATE2 0x630
+#define QSERDES_V8_LALB_VCO_CTUNE_LOWER_BND_RATE3 0x634
+#define QSERDES_V8_LALB_VCO_CTUNE_LOWER_BND_RATE4 0x638
+#define QSERDES_V8_LALB_VCO_CTUNE_UPPER_BND_RATE0 0x63c
+#define QSERDES_V8_LALB_VCO_CTUNE_UPPER_BND_RATE1 0x640
+#define QSERDES_V8_LALB_VCO_CTUNE_UPPER_BND_RATE2 0x644
+#define QSERDES_V8_LALB_VCO_CTUNE_UPPER_BND_RATE3 0x648
+#define QSERDES_V8_LALB_VCO_CTUNE_UPPER_BND_RATE4 0x64c
+#define QSERDES_V8_LALB_CDR_LOCK_KVCO_OFFSET_RATE0 0x650
+#define QSERDES_V8_LALB_CDR_LOCK_KVCO_OFFSET_RATE1 0x654
+#define QSERDES_V8_LALB_CDR_LOCK_KVCO_OFFSET_RATE2 0x658
+#define QSERDES_V8_LALB_CDR_LOCK_KVCO_OFFSET_RATE3 0x65c
+#define QSERDES_V8_LALB_CDR_LOCK_KVCO_OFFSET_RATE4 0x660
+#define QSERDES_V8_LALB_CDR_LOCK_KP_OFFSET_RATE0 0x664
+#define QSERDES_V8_LALB_CDR_LOCK_KP_OFFSET_RATE1 0x668
+#define QSERDES_V8_LALB_CDR_LOCK_KP_OFFSET_RATE2 0x66c
+#define QSERDES_V8_LALB_CDR_LOCK_KP_OFFSET_RATE3 0x670
+#define QSERDES_V8_LALB_CDR_LOCK_KP_OFFSET_RATE4 0x674
+#define QSERDES_V8_LALB_CDR_FASTLOCK_CP_CUR_PLL_RATE0 0x678
+#define QSERDES_V8_LALB_CDR_FASTLOCK_CP_CUR_PLL_RATE1 0x67c
+#define QSERDES_V8_LALB_CDR_FASTLOCK_CP_CUR_PLL_RATE2 0x680
+#define QSERDES_V8_LALB_CDR_FASTLOCK_CP_CUR_PLL_RATE3 0x684
+#define QSERDES_V8_LALB_CDR_FASTLOCK_CP_CUR_PLL_RATE4 0x688
+#define QSERDES_V8_LALB_DEBUG_BUS_SEL 0x68c
+#define QSERDES_V8_LALB_BIST_STATUS 0x690
+#define QSERDES_V8_LALB_BIST_ERROR_COUNT1 0x694
+#define QSERDES_V8_LALB_BIST_ERROR_COUNT2 0x698
+#define QSERDES_V8_LALB_AC_JTAG_OUTP 0x69c
+#define QSERDES_V8_LALB_AC_JTAG_OUTN 0x6a0
+#define QSERDES_V8_LALB_DATA_SLICER_DEBUG_STATUS 0x6a4
+#define QSERDES_V8_LALB_DATA_SLICER_TIMER1_STATUS 0x6a8
+#define QSERDES_V8_LALB_DATA_SLICER_TIMER2_STATUS 0x6ac
+#define QSERDES_V8_LALB_TX0_RESTRIM_CODE_STATUS 0x6b0
+#define QSERDES_V8_LALB_TX0_RESTRIM_ICAL_CODE_STATUS 0x6b4
+#define QSERDES_V8_LALB_TX0_RESTRIM_CAL_STATUS 0x6b8
+#define QSERDES_V8_LALB_TX1_RESTRIM_CODE_STATUS 0x6bc
+#define QSERDES_V8_LALB_TX1_RESTRIM_ICAL_CODE_STATUS 0x6c0
+#define QSERDES_V8_LALB_TX1_RESTRIM_CAL_STATUS 0x6c4
+#define QSERDES_V8_LALB_CMUX_DCC_CAL_FSM_STATUS 0x6c8
+#define QSERDES_V8_LALB_CMUX_DCC_READCODE_STATUS 0x6cc
+#define QSERDES_V8_LALB_TX_DCC_CAL_ANA_STATUS 0x6d0
+#define QSERDES_V8_LALB_TX0_CTUNE_DCC_FSM_DEBUG_STATUS 0x6d4
+#define QSERDES_V8_LALB_TX0_COARSE_DCC_READCODE_STATUS 0x6d8
+#define QSERDES_V8_LALB_TX0_FTUNE_MSB_DCC_FSM_DEBUG_STATUS 0x6dc
+#define QSERDES_V8_LALB_TX0_FTUNE_LSB_DCC_FSM_DEBUG_STATUS 0x6e0
+#define QSERDES_V8_LALB_TX0_FINE_DCC_READCODE_STATUS 0x6e4
+#define QSERDES_V8_LALB_TX1_CTUNE_DCC_FSM_DEBUG_STATUS 0x6e8
+#define QSERDES_V8_LALB_TX1_COARSE_DCC_READCODE_STATUS 0x6ec
+#define QSERDES_V8_LALB_TX1_FTUNE_MSB_DCC_FSM_DEBUG_STATUS 0x6f0
+#define QSERDES_V8_LALB_TX1_FTUNE_LSB_DCC_FSM_DEBUG_STATUS 0x6f4
+#define QSERDES_V8_LALB_TX1_FINE_DCC_READCODE_STATUS 0x6f8
+#define QSERDES_V8_LALB_CDR_VCO_CAL_STATUS 0x6fc
+#define QSERDES_V8_LALB_CDR_VCTRL_STATUS 0x700
+#define QSERDES_V8_LALB_CDR_VCO_CAP_CODE_STATUS 0x704
+#define QSERDES_V8_LALB_KVCO_CAL_DEBUG1_STATUS 0x708
+#define QSERDES_V8_LALB_KVCO_CAL_DEBUG2_STATUS 0x70c
+#define QSERDES_V8_LALB_KP_CAL_DEBUG1_STATUS 0x710
+#define QSERDES_V8_LALB_KP_CAL_DEBUG2_STATUS 0x714
+#define QSERDES_V8_LALB_CDR_VCO_FREQ_DEBUG1_STATUS 0x718
+#define QSERDES_V8_LALB_CDR_VCO_FREQ_DEBUG2_STATUS 0x71c
+#define QSERDES_V8_LALB_CDR_VCO_FREQ_DEBUG3_STATUS 0x720
+#define QSERDES_V8_LALB_CDR_VCO_FREQ_DEBUG4_STATUS 0x724
+#define QSERDES_V8_LALB_IVCM_CAL_STATUS 0x728
+#define QSERDES_V8_LALB_IVCM_CAL_DEBUG_STATUS 0x72c
+#define QSERDES_V8_LALB_IDAC_STATUS_I0 0x730
+#define QSERDES_V8_LALB_IDAC_STATUS_I0BAR 0x734
+#define QSERDES_V8_LALB_IDAC_STATUS_I1 0x738
+#define QSERDES_V8_LALB_IDAC_STATUS_I1BAR 0x73c
+#define QSERDES_V8_LALB_IDAC_STATUS_Q 0x740
+#define QSERDES_V8_LALB_IDAC_STATUS_QBAR 0x744
+#define QSERDES_V8_LALB_IDAC_STATUS_A 0x748
+#define QSERDES_V8_LALB_IDAC_STATUS_ABAR 0x74c
+#define QSERDES_V8_LALB_IDAC_STATUS_SM_ON 0x750
+#define QSERDES_V8_LALB_IDAC_STATUS_SIGNERROR 0x754
+#define QSERDES_V8_LALB_RX_SIGDET_STATUS 0x758
+#define QSERDES_V8_LALB_SIGDET_CAL_CODE_STATUS 0x75c
+#define QSERDES_V8_LALB_SIGDET_CAL_FSM_DEBUG_STATUS 0x760
+#define QSERDES_V8_LALB_CDR_FREQ_LOCK_CNT_STATUS 0x764
+#define QSERDES_V8_LALB_CDR_PHASE_LOCK_CNT_STATUS 0x768
+#define QSERDES_V8_LALB_CDR_LOCK_DEBUG_STATUS 0x76c
+#define QSERDES_V8_LALB_IDATA_HIGH_STATUS1 0x770
+#define QSERDES_V8_LALB_IDATA_HIGH_STATUS2 0x774
+#define QSERDES_V8_LALB_IDATA_HIGH_STATUS3 0x778
+#define QSERDES_V8_LALB_IDATA_HIGH_STATUS4 0x77c
+#define QSERDES_V8_LALB_IDATA_LOW_STATUS1 0x780
+#define QSERDES_V8_LALB_IDATA_LOW_STATUS2 0x784
+#define QSERDES_V8_LALB_IDATA_LOW_STATUS3 0x788
+#define QSERDES_V8_LALB_IDATA_LOW_STATUS4 0x78c
+#define QSERDES_V8_LALB_QDATA_STATUS1 0x790
+#define QSERDES_V8_LALB_QDATA_STATUS2 0x794
+#define QSERDES_V8_LALB_QDATA_STATUS3 0x798
+#define QSERDES_V8_LALB_QDATA_STATUS4 0x79c
+#define QSERDES_V8_LALB_IA_ERROR_COUNTER_LOW 0x7a0
+#define QSERDES_V8_LALB_IA_ERROR_COUNTER_HIGH 0x7a4
+#define QSERDES_V8_LALB_EOM_ERR_CNT_LSB_STATUS 0x7a8
+#define QSERDES_V8_LALB_EOM_ERR_CNT_MSB_STATUS 0x7ac
+#define QSERDES_V8_LALB_EOM_OP_STATUS 0x7b0
+#define QSERDES_V8_LALB_AUX_MIXER_INDEX_STATUS 0x7b4
+#define QSERDES_V8_LALB_AUX_OFFSET_STATUS 0x7b8
+#define QSERDES_V8_LALB_AUXDATA_TB_STATUS 0x7bc
+#define QSERDES_V8_LALB_AUX_MIXER_CTRL_0_STATUS 0x7c0
+#define QSERDES_V8_LALB_AUX_MIXER_CTRL_90_STATUS 0x7c4
+#define QSERDES_V8_LALB_AUX_MIXER_CTRL_180_STATUS 0x7c8
+#define QSERDES_V8_LALB_IQ_MIXER_INDEX_STATUS 0x7cc
+#define QSERDES_V8_LALB_IQTUNE_FLTR_INDEX_STATUS 0x7d0
+#define QSERDES_V8_LALB_IQ_MIXER_CTRL_0_STATUS 0x7d4
+#define QSERDES_V8_LALB_IQ_MIXER_CTRL_90_STATUS 0x7d8
+#define QSERDES_V8_LALB_IQ_MIXER_CTRL_180_STATUS 0x7dc
+#define QSERDES_V8_LALB_READ_EQCODE 0x7e0
+#define QSERDES_V8_LALB_READ_OFFSETCODE 0x7e4
+#define QSERDES_V8_LALB_VGA_READ_CODE 0x7e8
+#define QSERDES_V8_LALB_VTHRESH_READ_CODE 0x7ec
+#define QSERDES_V8_LALB_DFE_TAP1_READ_CODE 0x7f0
+#define QSERDES_V8_LALB_DFE_TAP2_READ_CODE 0x7f4
+#define QSERDES_V8_LALB_DFE_TAP3_READ_CODE 0x7f8
+#define QSERDES_V8_LALB_DFE_TAP4_READ_CODE 0x7fc
+#define QSERDES_V8_LALB_DFE_TAP5_READ_CODE 0x800
+#define QSERDES_V8_LALB_DFE_TAP6_READ_CODE 0x804
+#define QSERDES_V8_LALB_DFE_TAP7_READ_CODE 0x808
+#define QSERDES_V8_LALB_CDR_IQTUNE_FILTER_BIN_CODE 0x80c
+#define QSERDES_V8_LALB_CDR_IQTUNE_FILTER_CLK0_CODE 0x810
+#define QSERDES_V8_LALB_CDR_IQTUNE_FILTER_CLK90_CODE 0x814
+#define QSERDES_V8_LALB_BLW_READ_CODE 0x818
+#define QSERDES_V8_LALB_IA_OFFSET_CAL_DEBUG_STATUS 0x81c
+#define QSERDES_V8_LALB_IA_OFFSET_CAL_STATUS 0x820
+#define QSERDES_V8_LALB_IVTH_CAL_STATUS 0x824
+#define QSERDES_V8_LALB_IVTH_NRZ_EYE_HEIGHT_STATUS 0x828
+#define QSERDES_V8_LALB_IVTH_UPPER_EYE_MAX_STATUS 0x82c
+#define QSERDES_V8_LALB_IVTH_UPPER_EYE_MIN_STATUS 0x830
+#define QSERDES_V8_LALB_IVTH_LOWER_EYE_MAX_STATUS 0x834
+#define QSERDES_V8_LALB_IVTH_LOWER_EYE_MIN_STATUS 0x838
+#define QSERDES_V8_LALB_IVTH_UP_INIT_CTR_STATUS 0x83c
+#define QSERDES_V8_LALB_VTH_I_UP_CNTRL_STATUS 0x840
+#define QSERDES_V8_LALB_VTH_I_DN_CNTRL_STATUS 0x844
+#define QSERDES_V8_LALB_NRZ_EYE_HEIGHT_SEL_STATUS 0x848
+#define QSERDES_V8_LALB_DEBUG_BUS0 0x84c
+#define QSERDES_V8_LALB_DEBUG_BUS1 0x850
+#define QSERDES_V8_LALB_DEBUG_BUS2 0x854
+#define QSERDES_V8_LALB_DEBUG_BUS3 0x858
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL1 0x85c
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL2 0x860
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL3 0x864
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL4 0x868
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL5 0x86c
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL6 0x870
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL7 0x874
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL8 0x878
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL9 0x87c
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL10 0x880
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL11 0x884
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL12 0x888
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL13 0x88c
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL14 0x890
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL15 0x894
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL16 0x898
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL17 0x89c
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL18 0x8a0
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL19 0x8a4
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL20 0x8a8
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL21 0x8ac
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL22 0x8b0
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL23 0x8b4
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL24 0x8b8
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL25 0x8bc
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL26 0x8c0
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL27 0x8c4
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL28 0x8c8
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL29 0x8cc
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL30 0x8d0
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL31 0x8d4
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL32 0x8d8
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL_V2_1 0x8dc
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL_V2_2 0x8e0
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL_V2_3 0x8e4
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL_V2_4 0x8e8
+#define QSERDES_V8_LALB_DIG_BKUP_CTRL_V2_5 0x8ec
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS1 0x8f0
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS2 0x8f4
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS3 0x8f8
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS4 0x8fc
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS5 0x900
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS6 0x904
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS7 0x908
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS8 0x90c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS9 0x910
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS10 0x914
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS11 0x918
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS12 0x91c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS13 0x920
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS14 0x924
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS15 0x928
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS16 0x92c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS17 0x930
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS18 0x934
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS19 0x938
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS20 0x93c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS21 0x940
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS22 0x944
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS23 0x948
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS24 0x94c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS25 0x950
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS26 0x954
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS27 0x958
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS28 0x95c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS29 0x960
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS30 0x964
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS31 0x968
+#define QSERDES_V8_LALB_DIG_BKUP_RO_BUS32 0x96c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS1 0x970
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS2 0x974
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS3 0x978
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS4 0x97c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS5 0x980
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS6 0x984
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS7 0x988
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS8 0x98c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS9 0x990
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS10 0x994
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS11 0x998
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS12 0x99c
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS13 0x9a0
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS14 0x9a4
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS15 0x9a8
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS16 0x9ac
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS17 0x9b0
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS18 0x9b4
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS19 0x9b8
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS20 0x9bc
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS21 0x9c0
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS22 0x9c4
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS23 0x9c8
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS24 0x9cc
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS25 0x9d0
+#define QSERDES_V8_LALB_DIG_BKUP_RO_V2_BUS26 0x9d4
+#endif /* QCOM_PHY_QMP_QSERDES_V8_LALBH_ */
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-txrx-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-txrx-v8.h
new file mode 100644
index 000000000000..4cb8b1708607
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-qserdes-txrx-v8.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_QSERDES_TXRX_V8_H_
+#define QCOM_PHY_QMP_QSERDES_TXRX_V8_H_
+
+#define QSERDES_V8_TX_TX_EMP_POST1_LVL 0x00c
+#define QSERDES_V8_TX_TX_DRV_LVL 0x014
+#define QSERDES_V8_TX_RES_CODE_LANE_TX 0x034
+#define QSERDES_V8_TX_RES_CODE_LANE_RX 0x038
+#define QSERDES_V8_TX_RES_CODE_LANE_OFFSET_TX 0x03c
+#define QSERDES_V8_TX_RES_CODE_LANE_OFFSET_RX 0x040
+#define QSERDES_V8_TX_TRANSCEIVER_BIAS_EN 0x054
+#define QSERDES_V8_TX_HIGHZ_DRVR_EN 0x058
+#define QSERDES_V8_TX_TX_POL_INV 0x05c
+#define QSERDES_V8_TX_LANE_MODE_1 0x084
+#define QSERDES_V8_TX_LANE_MODE_2 0x088
+#define QSERDES_V8_TX_LANE_MODE_3 0x08c
+#define QSERDES_V8_TX_LANE_MODE_4 0x090
+#define QSERDES_V8_TX_LANE_MODE_5 0x094
+#define QSERDES_V8_TX_RCV_DETECT_LVL_2 0x0a4
+#define QSERDES_V8_TX_PI_QEC_CTRL 0x0e4
+
+#define QSERDES_V8_RX_UCDR_FO_GAIN 0x008
+#define QSERDES_V8_RX_UCDR_SO_GAIN 0x014
+#define QSERDES_V8_RX_UCDR_SVS_FO_GAIN 0x020
+#define QSERDES_V8_RX_UCDR_FASTLOCK_FO_GAIN 0x030
+#define QSERDES_V8_RX_UCDR_SO_SATURATION_AND_ENABLE 0x034
+#define QSERDES_V8_RX_UCDR_FASTLOCK_COUNT_LOW 0x03c
+#define QSERDES_V8_RX_UCDR_FASTLOCK_COUNT_HIGH 0x040
+#define QSERDES_V8_RX_UCDR_PI_CONTROLS 0x044
+#define QSERDES_V8_RX_UCDR_SB2_THRESH1 0x04c
+#define QSERDES_V8_RX_UCDR_SB2_THRESH2 0x050
+#define QSERDES_V8_RX_UCDR_SB2_GAIN1 0x054
+#define QSERDES_V8_RX_UCDR_SB2_GAIN2 0x058
+#define QSERDES_V8_RX_AUX_DATA_TCOARSE_TFINE 0x060
+#define QSERDES_V8_RX_VGA_CAL_CNTRL1 0x0d4
+#define QSERDES_V8_RX_VGA_CAL_CNTRL2 0x0d8
+#define QSERDES_V8_RX_GM_CAL 0x0dc
+#define QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL2 0x0ec
+#define QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL3 0x0f0
+#define QSERDES_V8_RX_RX_EQU_ADAPTOR_CNTRL4 0x0f4
+#define QSERDES_V8_RX_RX_IDAC_TSETTLE_LOW 0x0f8
+#define QSERDES_V8_RX_RX_IDAC_TSETTLE_HIGH 0x0fc
+#define QSERDES_V8_RX_RX_EQ_OFFSET_ADAPTOR_CNTRL1 0x110
+#define QSERDES_V8_RX_SIGDET_ENABLES 0x118
+#define QSERDES_V8_RX_SIGDET_CNTRL 0x11c
+#define QSERDES_V8_RX_SIGDET_DEGLITCH_CNTRL 0x124
+#define QSERDES_V8_RX_RX_MODE_00_LOW 0x15c
+#define QSERDES_V8_RX_RX_MODE_00_HIGH 0x160
+#define QSERDES_V8_RX_RX_MODE_00_HIGH2 0x164
+#define QSERDES_V8_RX_RX_MODE_00_HIGH3 0x168
+#define QSERDES_V8_RX_RX_MODE_00_HIGH4 0x16c
+#define QSERDES_V8_RX_RX_MODE_01_LOW 0x170
+#define QSERDES_V8_RX_RX_MODE_01_HIGH 0x174
+#define QSERDES_V8_RX_RX_MODE_01_HIGH2 0x178
+#define QSERDES_V8_RX_RX_MODE_01_HIGH3 0x17c
+#define QSERDES_V8_RX_RX_MODE_01_HIGH4 0x180
+#define QSERDES_V8_RX_DFE_EN_TIMER 0x1a0
+#define QSERDES_V8_RX_DFE_CTLE_POST_CAL_OFFSET 0x1a4
+#define QSERDES_V8_RX_DCC_CTRL1 0x1a8
+#define QSERDES_V8_RX_VTH_CODE 0x1b0
+#define QSERDES_V8_RX_SIGDET_CAL_CTRL1 0x1e4
+#define QSERDES_V8_RX_SIGDET_CAL_TRIM 0x1f8
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-usb-legacy.c b/drivers/phy/qualcomm/phy-qcom-qmp-usb-legacy.c
index 8bf951b0490c..fc490589c8e4 100644
--- a/drivers/phy/qualcomm/phy-qcom-qmp-usb-legacy.c
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-usb-legacy.c
@@ -542,6 +542,8 @@ struct qmp_usb {
enum phy_mode mode;
+ bool phy_initialized;
+
struct phy *phy;
struct clk_fixed_rate pipe_clk_fixed;
@@ -895,6 +897,7 @@ static int qmp_usb_legacy_power_off(struct phy *phy)
static int qmp_usb_legacy_enable(struct phy *phy)
{
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
int ret;
ret = qmp_usb_legacy_init(phy);
@@ -904,14 +907,19 @@ static int qmp_usb_legacy_enable(struct phy *phy)
ret = qmp_usb_legacy_power_on(phy);
if (ret)
qmp_usb_legacy_exit(phy);
+ else
+ qmp->phy_initialized = true;
return ret;
}
static int qmp_usb_legacy_disable(struct phy *phy)
{
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
int ret;
+ qmp->phy_initialized = false;
+
ret = qmp_usb_legacy_power_off(phy);
if (ret)
return ret;
@@ -988,7 +996,7 @@ static int __maybe_unused qmp_usb_legacy_runtime_suspend(struct device *dev)
dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qmp->mode);
- if (!qmp->phy->init_count) {
+ if (!qmp->phy_initialized) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
@@ -1009,7 +1017,7 @@ static int __maybe_unused qmp_usb_legacy_runtime_resume(struct device *dev)
dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qmp->mode);
- if (!qmp->phy->init_count) {
+ if (!qmp->phy_initialized) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-usb.c b/drivers/phy/qualcomm/phy-qcom-qmp-usb.c
index c12efd127a61..2f1b967864b9 100644
--- a/drivers/phy/qualcomm/phy-qcom-qmp-usb.c
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-usb.c
@@ -1256,6 +1256,8 @@ struct qmp_usb {
enum phy_mode mode;
+ bool phy_initialized;
+
struct phy *phy;
struct clk_fixed_rate pipe_clk_fixed;
@@ -1786,6 +1788,7 @@ static int qmp_usb_power_off(struct phy *phy)
static int qmp_usb_enable(struct phy *phy)
{
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
int ret;
ret = qmp_usb_init(phy);
@@ -1795,14 +1798,19 @@ static int qmp_usb_enable(struct phy *phy)
ret = qmp_usb_power_on(phy);
if (ret)
qmp_usb_exit(phy);
+ else
+ qmp->phy_initialized = true;
return ret;
}
static int qmp_usb_disable(struct phy *phy)
{
+ struct qmp_usb *qmp = phy_get_drvdata(phy);
int ret;
+ qmp->phy_initialized = false;
+
ret = qmp_usb_power_off(phy);
if (ret)
return ret;
@@ -1878,7 +1886,7 @@ static int __maybe_unused qmp_usb_runtime_suspend(struct device *dev)
dev_vdbg(dev, "Suspending QMP phy, mode:%d\n", qmp->mode);
- if (!qmp->phy->init_count) {
+ if (!qmp->phy_initialized) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
@@ -1898,7 +1906,7 @@ static int __maybe_unused qmp_usb_runtime_resume(struct device *dev)
dev_vdbg(dev, "Resuming QMP phy, mode:%d\n", qmp->mode);
- if (!qmp->phy->init_count) {
+ if (!qmp->phy_initialized) {
dev_vdbg(dev, "PHY not initialized, bailing out\n");
return 0;
}
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-usb43-pcs-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-usb43-pcs-v8.h
new file mode 100644
index 000000000000..4f387c8ed9e5
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-usb43-pcs-v8.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_USB43_PCS_V8_H_
+#define QCOM_PHY_QMP_USB43_PCS_V8_H_
+
+#define QPHY_V8_USB43_PCS_SW_RESET 0x000
+#define QPHY_V8_USB43_PCS_PCS_STATUS1 0x014
+#define QPHY_V8_USB43_PCS_POWER_DOWN_CONTROL 0x040
+#define QPHY_V8_USB43_PCS_START_CONTROL 0x044
+#define QPHY_V8_USB43_PCS_POWER_STATE_CONFIG1 0x090
+#define QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG1 0x0c4
+#define QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG2 0x0c8
+#define QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG3 0x0cc
+#define QPHY_V8_USB43_PCS_LOCK_DETECT_CONFIG6 0x0d8
+#define QPHY_V8_USB43_PCS_REFGEN_REQ_CONFIG1 0x0dc
+#define QPHY_V8_USB43_PCS_RX_SIGDET_LVL 0x188
+#define QPHY_V8_USB43_PCS_RCVR_DTCT_DLY_P1U2_L 0x190
+#define QPHY_V8_USB43_PCS_RCVR_DTCT_DLY_P1U2_H 0x194
+#define QPHY_V8_USB43_PCS_RATE_SLEW_CNTRL1 0x198
+#define QPHY_V8_USB43_PCS_TSYNC_RSYNC_TIME 0x1ac
+#define QPHY_V8_USB43_PCS_RX_CONFIG 0x1b0
+#define QPHY_V8_USB43_PCS_TSYNC_DLY_TIME 0x1b4
+#define QPHY_V8_USB43_PCS_ALIGN_DETECT_CONFIG1 0x1c0
+#define QPHY_V8_USB43_PCS_ALIGN_DETECT_CONFIG2 0x1c4
+#define QPHY_V8_USB43_PCS_PCS_TX_RX_CONFIG 0x1d0
+#define QPHY_V8_USB43_PCS_EQ_CONFIG1 0x1dc
+#define QPHY_V8_USB43_PCS_EQ_CONFIG2 0x1e0
+#define QPHY_V8_USB43_PCS_EQ_CONFIG5 0x1ec
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp-usb43-qserdes-com-v8.h b/drivers/phy/qualcomm/phy-qcom-qmp-usb43-qserdes-com-v8.h
new file mode 100644
index 000000000000..e9c743fce9d1
--- /dev/null
+++ b/drivers/phy/qualcomm/phy-qcom-qmp-usb43-qserdes-com-v8.h
@@ -0,0 +1,224 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2025 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef QCOM_PHY_QMP_USB43_QSERDES_COM_V8_H_
+#define QCOM_PHY_QMP_USB43_QSERDES_COM_V8_H_
+
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE1 0x000
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE1 0x004
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE3_MODE1 0x008
+#define QSERDES_V8_USB43_COM_CLK_EP_DIV_MODE1 0x00c
+#define QSERDES_V8_USB43_COM_CP_CTRL_MODE1 0x010
+#define QSERDES_V8_USB43_COM_PLL_RCTRL_MODE1 0x014
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_MODE1 0x018
+#define QSERDES_V8_USB43_COM_CORECLK_DIV_MODE1 0x01c
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_MODE1 0x020
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_MODE1 0x024
+#define QSERDES_V8_USB43_COM_DEC_START_MODE1 0x028
+#define QSERDES_V8_USB43_COM_DEC_START_MSB_MODE1 0x02c
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE1 0x030
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE1 0x034
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE1 0x038
+#define QSERDES_V8_USB43_COM_HSCLK_SEL_1 0x03c
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE1 0x040
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN1_MODE1 0x044
+#define QSERDES_V8_USB43_COM_VCO_TUNE1_MODE1 0x048
+#define QSERDES_V8_USB43_COM_VCO_TUNE2_MODE1 0x04c
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE1 0x050
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE1 0x054
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE0 0x058
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE0 0x05c
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE0 0x060
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE0 0x064
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE3_MODE0 0x068
+#define QSERDES_V8_USB43_COM_CLK_EP_DIV_MODE0 0x06c
+#define QSERDES_V8_USB43_COM_CP_CTRL_MODE0 0x070
+#define QSERDES_V8_USB43_COM_PLL_RCTRL_MODE0 0x074
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_MODE0 0x078
+#define QSERDES_V8_USB43_COM_CORECLK_DIV_MODE0 0x07c
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_MODE0 0x080
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_MODE0 0x084
+#define QSERDES_V8_USB43_COM_DEC_START_MODE0 0x088
+#define QSERDES_V8_USB43_COM_DEC_START_MSB_MODE0 0x08c
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE0 0x090
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE0 0x094
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE0 0x098
+#define QSERDES_V8_USB43_COM_HSCLK_HS_SWITCH_SEL_1 0x09c
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE0 0x0a0
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN1_MODE0 0x0a4
+#define QSERDES_V8_USB43_COM_VCO_TUNE1_MODE0 0x0a8
+#define QSERDES_V8_USB43_COM_VCO_TUNE2_MODE0 0x0ac
+#define QSERDES_V8_USB43_COM_ATB_SEL1 0x0b0
+#define QSERDES_V8_USB43_COM_ATB_SEL2 0x0b4
+#define QSERDES_V8_USB43_COM_FREQ_UPDATE 0x0b8
+#define QSERDES_V8_USB43_COM_BG_TIMER 0x0bc
+#define QSERDES_V8_USB43_COM_SSC_EN_CENTER 0x0c0
+#define QSERDES_V8_USB43_COM_SSC_ADJ_PER1 0x0c4
+#define QSERDES_V8_USB43_COM_SSC_ADJ_PER2 0x0c8
+#define QSERDES_V8_USB43_COM_SSC_PER1 0x0cc
+#define QSERDES_V8_USB43_COM_SSC_PER2 0x0d0
+#define QSERDES_V8_USB43_COM_POST_DIV 0x0d4
+#define QSERDES_V8_USB43_COM_POST_DIV_MUX 0x0d8
+#define QSERDES_V8_USB43_COM_BIAS_EN_CLKBUFLR_EN 0x0dc
+#define QSERDES_V8_USB43_COM_CLK_ENABLE1 0x0e0
+#define QSERDES_V8_USB43_COM_SYS_CLK_CTRL 0x0e4
+#define QSERDES_V8_USB43_COM_SYSCLK_BUF_ENABLE 0x0e8
+#define QSERDES_V8_USB43_COM_PLL_EN 0x0ec
+#define QSERDES_V8_USB43_COM_DEBUG_BUS_OVRD 0x0f0
+#define QSERDES_V8_USB43_COM_PLL_IVCO 0x0f4
+#define QSERDES_V8_USB43_COM_PLL_IVCO_MODE1 0x0f8
+#define QSERDES_V8_USB43_COM_CMN_IETRIM 0x0fc
+#define QSERDES_V8_USB43_COM_CMN_IPTRIM 0x100
+#define QSERDES_V8_USB43_COM_EP_CLOCK_DETECT_CTRL 0x104
+#define QSERDES_V8_USB43_COM_PLL_CNTRL 0x108
+#define QSERDES_V8_USB43_COM_BIAS_EN_CTRL_BY_PSM 0x10c
+#define QSERDES_V8_USB43_COM_SYSCLK_EN_SEL 0x110
+#define QSERDES_V8_USB43_COM_CML_SYSCLK_SEL 0x114
+#define QSERDES_V8_USB43_COM_RESETSM_CNTRL 0x118
+#define QSERDES_V8_USB43_COM_RESETSM_CNTRL2 0x11c
+#define QSERDES_V8_USB43_COM_LOCK_CMP_EN 0x120
+#define QSERDES_V8_USB43_COM_LOCK_CMP_CFG 0x124
+#define QSERDES_V8_USB43_COM_INTEGLOOP_INITVAL 0x128
+#define QSERDES_V8_USB43_COM_INTEGLOOP_EN 0x12c
+#define QSERDES_V8_USB43_COM_INTEGLOOP_P_PATH_GAIN0 0x130
+#define QSERDES_V8_USB43_COM_INTEGLOOP_P_PATH_GAIN1 0x134
+#define QSERDES_V8_USB43_COM_VCOCAL_DEADMAN_CTRL 0x138
+#define QSERDES_V8_USB43_COM_VCO_TUNE_CTRL 0x13c
+#define QSERDES_V8_USB43_COM_VCO_TUNE_MAP 0x140
+#define QSERDES_V8_USB43_COM_VCO_TUNE_INITVAL1 0x144
+#define QSERDES_V8_USB43_COM_VCO_TUNE_INITVAL2 0x148
+#define QSERDES_V8_USB43_COM_VCO_TUNE_MINVAL1 0x14c
+#define QSERDES_V8_USB43_COM_VCO_TUNE_MINVAL2 0x150
+#define QSERDES_V8_USB43_COM_VCO_TUNE_MAXVAL1 0x154
+#define QSERDES_V8_USB43_COM_VCO_TUNE_MAXVAL2 0x158
+#define QSERDES_V8_USB43_COM_VCO_TUNE_TIMER1 0x15c
+#define QSERDES_V8_USB43_COM_VCO_TUNE_TIMER2 0x160
+#define QSERDES_V8_USB43_COM_CLK_SELECT 0x164
+#define QSERDES_V8_USB43_COM_PLL_ANALOG 0x168
+#define QSERDES_V8_USB43_COM_SW_RESET 0x16c
+#define QSERDES_V8_USB43_COM_CORE_CLK_EN 0x170
+#define QSERDES_V8_USB43_COM_CMN_CONFIG_1 0x174
+#define QSERDES_V8_USB43_COM_CMN_CONFIG_3 0x178
+#define QSERDES_V8_USB43_COM_CMN_RATE_OVERRIDE 0x17c
+#define QSERDES_V8_USB43_COM_SVS_MODE_CLK_SEL 0x180
+#define QSERDES_V8_USB43_COM_DEBUG_BUS_SEL 0x184
+#define QSERDES_V8_USB43_COM_CMN_MISC1 0x188
+#define QSERDES_V8_USB43_COM_CMN_MODE 0x18c
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD 0x190
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD1 0x194
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD2 0x198
+#define QSERDES_V8_USB43_COM_VCO_DC_LEVEL_CTRL 0x19c
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_1 0x1a0
+#define QSERDES_V8_USB43_COM_ADDITIONAL_CTRL_1 0x1a4
+#define QSERDES_V8_USB43_COM_AUTO_GAIN_ADJ_CTRL_1 0x1a8
+#define QSERDES_V8_USB43_COM_AUTO_GAIN_ADJ_CTRL_2 0x1ac
+#define QSERDES_V8_USB43_COM_AUTO_GAIN_ADJ_CTRL_3 0x1b0
+#define QSERDES_V8_USB43_COM_AUTO_GAIN_ADJ_CTRL_4 0x1b4
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC 0x1b8
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_2 0x1bc
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_3 0x1c0
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_4 0x1c4
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_5 0x1c8
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE1_MODE2 0x1cc
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE2_MODE2 0x1d0
+#define QSERDES_V8_USB43_COM_SSC_STEP_SIZE3_MODE2 0x1d4
+#define QSERDES_V8_USB43_COM_CLK_EP_DIV_MODE2 0x1d8
+#define QSERDES_V8_USB43_COM_CP_CTRL_MODE2 0x1dc
+#define QSERDES_V8_USB43_COM_PLL_RCTRL_MODE2 0x1e0
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_MODE2 0x1e4
+#define QSERDES_V8_USB43_COM_CORECLK_DIV_MODE2 0x1e8
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_MODE2 0x1ec
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_MODE2 0x1f0
+#define QSERDES_V8_USB43_COM_DEC_START_MODE2 0x1f4
+#define QSERDES_V8_USB43_COM_DEC_START_MSB_MODE2 0x1f8
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START1_MODE2 0x1fc
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START2_MODE2 0x200
+#define QSERDES_V8_USB43_COM_DIV_FRAC_START3_MODE2 0x204
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN0_MODE2 0x208
+#define QSERDES_V8_USB43_COM_INTEGLOOP_GAIN1_MODE2 0x20c
+#define QSERDES_V8_USB43_COM_VCO_TUNE1_MODE2 0x210
+#define QSERDES_V8_USB43_COM_VCO_TUNE2_MODE2 0x214
+#define QSERDES_V8_USB43_COM_PLL_IVCO_MODE2 0x218
+#define QSERDES_V8_USB43_COM_HSCLK_SEL_2 0x21c
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE1_MODE2 0x220
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_CMP_CODE2_MODE2 0x224
+#define QSERDES_V8_USB43_COM_HSCLK_HS_SWITCH_SEL_2 0x228
+#define QSERDES_V8_USB43_COM_CMN_CONFIG_2 0x22c
+#define QSERDES_V8_USB43_COM_BIN_VCOCAL_HSCLK_SEL_2 0x230
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_0 0x234
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_1 0x238
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_2 0x23c
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_3 0x240
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_4 0x244
+#define QSERDES_V8_USB43_COM_IVCOCAL_CONFIG_5 0x248
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_EARLY_MODE0 0x24c
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_EARLY_MODE0 0x250
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_EARLY_MODE1 0x254
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_EARLY_MODE1 0x258
+#define QSERDES_V8_USB43_COM_LOCK_CMP1_EARLY_MODE2 0x25c
+#define QSERDES_V8_USB43_COM_LOCK_CMP2_EARLY_MODE2 0x260
+#define QSERDES_V8_USB43_COM_EARLY_LOCK_CONFIG_0 0x264
+#define QSERDES_V8_USB43_COM_EARLY_LOCK_CONFIG_1 0x268
+#define QSERDES_V8_USB43_COM_ADAPTIVE_ANALOG_CONFIG 0x26c
+#define QSERDES_V8_USB43_COM_CP_CTRL_ADAPTIVE_MODE0 0x270
+#define QSERDES_V8_USB43_COM_PLL_RCCTRL_ADAPTIVE_MODE0 0x274
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_ADAPTIVE_MODE0 0x278
+#define QSERDES_V8_USB43_COM_CP_CTRL_ADAPTIVE_MODE1 0x27c
+#define QSERDES_V8_USB43_COM_PLL_RCCTRL_ADAPTIVE_MODE1 0x280
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_ADAPTIVE_MODE1 0x284
+#define QSERDES_V8_USB43_COM_CP_CTRL_ADAPTIVE_MODE2 0x288
+#define QSERDES_V8_USB43_COM_PLL_RCCTRL_ADAPTIVE_MODE2 0x28c
+#define QSERDES_V8_USB43_COM_PLL_CCTRL_ADAPTIVE_MODE2 0x290
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD3 0x294
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD4 0x298
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD5 0x29c
+#define QSERDES_V8_USB43_COM_CMN_MODE_CONTD6 0x2a0
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_6 0x2a4
+#define QSERDES_V8_USB43_COM_ADDITIONAL_MISC_7 0x2a8
+#define QSERDES_V8_USB43_COM_VCO_WAIT_CYCLES 0x2ac
+#define QSERDES_V8_USB43_COM_BIAS_WAIT_CYCLES 0x2b0
+#define QSERDES_V8_USB43_COM_AUX_CLK_PSM_ENABLE 0x2b4
+#define QSERDES_V8_USB43_COM_PLL_SPARE_FOR_ECO 0x2b8
+#define QSERDES_V8_USB43_COM_PLL_SPARE_FOR_ECO_1 0x2bc
+#define QSERDES_V8_USB43_COM_PLL_SPARE_FOR_ECO_2 0x2c0
+#define QSERDES_V8_USB43_COM_LDO_CAL_1 0x2c4
+#define QSERDES_V8_USB43_COM_LDO_CAL_2 0x2c8
+#define QSERDES_V8_USB43_COM_LDO_CAL_3 0x2cc
+#define QSERDES_V8_USB43_COM_LDO_CAL_4 0x2d0
+#define QSERDES_V8_USB43_COM_LDO_CAL_5 0x2d4
+#define QSERDES_V8_USB43_COM_DCC_CAL_1 0x2d8
+#define QSERDES_V8_USB43_COM_DCC_CAL_2 0x2dc
+#define QSERDES_V8_USB43_COM_DCC_CAL_3 0x2e0
+#define QSERDES_V8_USB43_COM_DCC_CAL_4 0x2e4
+#define QSERDES_V8_USB43_COM_DCC_CAL_5 0x2e8
+#define QSERDES_V8_USB43_COM_DCC_CAL_6 0x2ec
+#define QSERDES_V8_USB43_COM_PSM_CAL_EN 0x2f0
+#define QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_1 0x2f4
+#define QSERDES_V8_USB43_COM_CLK_FWD_CONFIG_2 0x2f8
+#define QSERDES_V8_USB43_COM_IP_CTRL_AND_DP_SEL 0x2fc
+#define QSERDES_V8_USB43_COM_DCC_CAL_7 0x300
+#define QSERDES_V8_USB43_COM_DCC_CAL_8 0x304
+#define QSERDES_V8_USB43_COM_DCC_CAL_9 0x308
+#define QSERDES_V8_USB43_COM_MODE_OPERATION_STATUS 0x30c
+#define QSERDES_V8_USB43_COM_SYSCLK_DET_COMP_STATUS 0x310
+#define QSERDES_V8_USB43_COM_CMN_STATUS 0x314
+#define QSERDES_V8_USB43_COM_RESET_SM_STATUS 0x318
+#define QSERDES_V8_USB43_COM_RESTRIM_CODE_STATUS 0x31c
+#define QSERDES_V8_USB43_COM_PLLCAL_CODE1_STATUS 0x320
+#define QSERDES_V8_USB43_COM_PLLCAL_CODE2_STATUS 0x324
+#define QSERDES_V8_USB43_COM_INTEGLOOP_BINCODE_STATUS 0x328
+#define QSERDES_V8_USB43_COM_DEBUG_BUS0 0x32c
+#define QSERDES_V8_USB43_COM_DEBUG_BUS1 0x330
+#define QSERDES_V8_USB43_COM_DEBUG_BUS2 0x334
+#define QSERDES_V8_USB43_COM_DEBUG_BUS3 0x338
+#define QSERDES_V8_USB43_COM_C_READY_STATUS 0x33c
+#define QSERDES_V8_USB43_COM_READ_DUMMY_1 0x340
+#define QSERDES_V8_USB43_COM_READ_DUMMY_2 0x344
+#define QSERDES_V8_USB43_COM_READ_DUMMY_3 0x348
+#define QSERDES_V8_USB43_COM_IVCO_CAL_CODE_STATUS 0x34c
+#define QSERDES_V8_USB43_COM_PLL_LDO_CAL_STATUS_2 0x350
+#define QSERDES_V8_USB43_COM_PLL_LDO_CAL_STATUS_3 0x354
+
+#endif
diff --git a/drivers/phy/qualcomm/phy-qcom-qmp.h b/drivers/phy/qualcomm/phy-qcom-qmp.h
index d0f41e4aaa85..5d174eb82d93 100644
--- a/drivers/phy/qualcomm/phy-qcom-qmp.h
+++ b/drivers/phy/qualcomm/phy-qcom-qmp.h
@@ -30,6 +30,11 @@
#include "phy-qcom-qmp-qserdes-com-v7.h"
#include "phy-qcom-qmp-qserdes-txrx-v7.h"
+#include "phy-qcom-qmp-qserdes-com-v8.h"
+#include "phy-qcom-qmp-usb43-qserdes-com-v8.h"
+#include "phy-qcom-qmp-qserdes-txrx-v8.h"
+#include "phy-qcom-qmp-qserdes-lalb-v8.h"
+
#include "phy-qcom-qmp-qserdes-pll.h"
#include "phy-qcom-qmp-pcs-v2.h"
@@ -52,6 +57,8 @@
#include "phy-qcom-qmp-pcs-v7.h"
+#include "phy-qcom-qmp-pcs-v8.h"
+
/* QPHY_SW_RESET bit */
#define SW_RESET BIT(0)
/* QPHY_POWER_DOWN_CONTROL */
diff --git a/drivers/phy/qualcomm/phy-qcom-sgmii-eth.c b/drivers/phy/qualcomm/phy-qcom-sgmii-eth.c
index 5b1c82459c12..8de0fba25ecf 100644
--- a/drivers/phy/qualcomm/phy-qcom-sgmii-eth.c
+++ b/drivers/phy/qualcomm/phy-qcom-sgmii-eth.c
@@ -10,6 +10,7 @@
#include <linux/phy/phy.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
+#include <linux/regulator/consumer.h>
#include "phy-qcom-qmp-pcs-sgmii.h"
#include "phy-qcom-qmp-qserdes-com-v5.h"
@@ -25,7 +26,15 @@
#define QSERDES_PCS_SGMIIPHY_READY BIT(7)
#define QSERDES_COM_C_PLL_LOCKED BIT(1)
+static const struct regulator_bulk_data qcom_dwmac_sgmii_phy_vregs[] = {
+ { .supply = "vdda-0p9", .init_load_uA = 46000 },
+ { .supply = "vdda-1p2", .init_load_uA = 15000 },
+};
+
+#define QCOM_SGMII_NUM_SUPPLIES ARRAY_SIZE(qcom_dwmac_sgmii_phy_vregs)
+
struct qcom_dwmac_sgmii_phy_data {
+ struct regulator_bulk_data *vregs;
struct regmap *regmap;
struct clk *refclk;
int speed;
@@ -267,8 +276,28 @@ static int qcom_dwmac_sgmii_phy_calibrate(struct phy *phy)
static int qcom_dwmac_sgmii_phy_power_on(struct phy *phy)
{
struct qcom_dwmac_sgmii_phy_data *data = phy_get_drvdata(phy);
+ int ret;
+
+ ret = regulator_bulk_enable(QCOM_SGMII_NUM_SUPPLIES, data->vregs);
+ if (ret)
+ return ret;
- return clk_prepare_enable(data->refclk);
+ ret = clk_prepare_enable(data->refclk);
+ if (ret)
+ goto err_disable_regulators;
+
+ ret = qcom_dwmac_sgmii_phy_calibrate(phy);
+ if (ret)
+ goto err_disable_clk;
+
+ return 0;
+
+err_disable_clk:
+ clk_disable_unprepare(data->refclk);
+err_disable_regulators:
+ regulator_bulk_disable(QCOM_SGMII_NUM_SUPPLIES, data->vregs);
+
+ return ret;
}
static int qcom_dwmac_sgmii_phy_power_off(struct phy *phy)
@@ -283,6 +312,8 @@ static int qcom_dwmac_sgmii_phy_power_off(struct phy *phy)
clk_disable_unprepare(data->refclk);
+ regulator_bulk_disable(QCOM_SGMII_NUM_SUPPLIES, data->vregs);
+
return 0;
}
@@ -293,6 +324,9 @@ static int qcom_dwmac_sgmii_phy_set_speed(struct phy *phy, int speed)
if (speed != data->speed)
data->speed = speed;
+ if (phy->power_count == 0)
+ return 0;
+
return qcom_dwmac_sgmii_phy_calibrate(phy);
}
@@ -319,6 +353,7 @@ static int qcom_dwmac_sgmii_phy_probe(struct platform_device *pdev)
struct phy_provider *provider;
void __iomem *base;
struct phy *phy;
+ int ret;
data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
if (!data)
@@ -343,6 +378,12 @@ static int qcom_dwmac_sgmii_phy_probe(struct platform_device *pdev)
if (IS_ERR(data->refclk))
return PTR_ERR(data->refclk);
+ ret = devm_regulator_bulk_get_const(dev, QCOM_SGMII_NUM_SUPPLIES,
+ qcom_dwmac_sgmii_phy_vregs,
+ &data->vregs);
+ if (ret)
+ return ret;
+
provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
if (IS_ERR(provider))
return PTR_ERR(provider);
diff --git a/drivers/phy/qualcomm/phy-qcom-snps-femto-v2.c b/drivers/phy/qualcomm/phy-qcom-snps-femto-v2.c
index eb0b0f61d98e..980ad1fb1e2e 100644
--- a/drivers/phy/qualcomm/phy-qcom-snps-femto-v2.c
+++ b/drivers/phy/qualcomm/phy-qcom-snps-femto-v2.c
@@ -599,8 +599,18 @@ static int qcom_snps_hsphy_probe(struct platform_device *pdev)
return dev_err_probe(dev, ret,
"failed to get regulator supplies\n");
+ /*
+ * Enable runtime PM before creating the PHY, phy_create() only enables
+ * it on the PHY device if already enabled on the parent. Hold a usage
+ * reference so callbacks cannot run before the PHY is ready.
+ */
+ pm_runtime_get_noresume(dev);
pm_runtime_set_active(dev);
- pm_runtime_enable(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret) {
+ pm_runtime_put_noidle(dev);
+ return ret;
+ }
/*
* Prevent runtime pm from being ON by default. Users can enable
* it using power/control in sysfs.
@@ -611,6 +621,7 @@ static int qcom_snps_hsphy_probe(struct platform_device *pdev)
if (IS_ERR(generic_phy)) {
ret = PTR_ERR(generic_phy);
dev_err(dev, "failed to create phy, %d\n", ret);
+ pm_runtime_put_noidle(dev);
return ret;
}
hsphy->phy = generic_phy;
@@ -620,12 +631,15 @@ static int qcom_snps_hsphy_probe(struct platform_device *pdev)
qcom_snps_hsphy_read_override_param_seq(dev);
phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
- if (!IS_ERR(phy_provider))
- dev_dbg(dev, "Registered Qcom-SNPS HS phy\n");
- else
- pm_runtime_disable(dev);
+ if (IS_ERR(phy_provider)) {
+ pm_runtime_put_noidle(dev);
+ return PTR_ERR(phy_provider);
+ }
- return PTR_ERR_OR_ZERO(phy_provider);
+ dev_dbg(dev, "Registered Qcom-SNPS HS phy\n");
+ pm_runtime_put(dev);
+
+ return 0;
}
static struct platform_driver qcom_snps_hsphy_driver = {
diff --git a/drivers/phy/rockchip/phy-rockchip-inno-csidphy.c b/drivers/phy/rockchip/phy-rockchip-inno-csidphy.c
index 279e19e7546b..2c1f8e3645df 100644
--- a/drivers/phy/rockchip/phy-rockchip-inno-csidphy.c
+++ b/drivers/phy/rockchip/phy-rockchip-inno-csidphy.c
@@ -157,7 +157,7 @@ static const struct hsfreq_range rk1808_mipidphy_hsfreq_ranges[] = {
{ 299, 0x06}, { 399, 0x08}, { 499, 0x0b}, { 599, 0x0e},
{ 699, 0x10}, { 799, 0x12}, { 999, 0x16}, {1199, 0x1e},
{1399, 0x23}, {1599, 0x2d}, {1799, 0x32}, {1999, 0x37},
- {2199, 0x3c}, {2399, 0x41}, {2499, 0x46}
+ {2199, 0x3c}, {2399, 0x41}, {2500, 0x46}
};
static const struct hsfreq_range rk3326_mipidphy_hsfreq_ranges[] = {
diff --git a/drivers/phy/starfive/phy-jh7110-dphy-rx.c b/drivers/phy/starfive/phy-jh7110-dphy-rx.c
index 0b039e1f71c5..d06f21ad6332 100644
--- a/drivers/phy/starfive/phy-jh7110-dphy-rx.c
+++ b/drivers/phy/starfive/phy-jh7110-dphy-rx.c
@@ -150,6 +150,7 @@ static int stf_dphy_probe(struct platform_device *pdev)
{
struct phy_provider *phy_provider;
struct stf_dphy *dphy;
+ int ret;
dphy = devm_kzalloc(&pdev->dev, sizeof(*dphy), GFP_KERNEL);
if (!dphy)
@@ -190,7 +191,9 @@ static int stf_dphy_probe(struct platform_device *pdev)
return PTR_ERR(dphy->phy);
}
- pm_runtime_enable(&pdev->dev);
+ ret = devm_pm_runtime_enable(&pdev->dev);
+ if (ret)
+ return ret;
phy_set_drvdata(dphy->phy, dphy);
phy_provider = devm_of_phy_provider_register(&pdev->dev,
diff --git a/drivers/phy/starfive/phy-jh7110-dphy-tx.c b/drivers/phy/starfive/phy-jh7110-dphy-tx.c
index c64d1c91b130..181491a93807 100644
--- a/drivers/phy/starfive/phy-jh7110-dphy-tx.c
+++ b/drivers/phy/starfive/phy-jh7110-dphy-tx.c
@@ -392,6 +392,7 @@ static int stf_dphy_probe(struct platform_device *pdev)
{
struct phy_provider *phy_provider;
struct stf_dphy *dphy;
+ int ret;
dphy = devm_kzalloc(&pdev->dev, sizeof(*dphy), GFP_KERNEL);
if (!dphy)
@@ -406,7 +407,9 @@ static int stf_dphy_probe(struct platform_device *pdev)
if (IS_ERR(dphy->topsys))
return PTR_ERR(dphy->topsys);
- pm_runtime_enable(&pdev->dev);
+ ret = devm_pm_runtime_enable(&pdev->dev);
+ if (ret)
+ return ret;
dphy->txesc_clk = devm_clk_get(&pdev->dev, "txesc");
if (IS_ERR(dphy->txesc_clk))
diff --git a/drivers/phy/sunplus/phy-sunplus-usb2.c b/drivers/phy/sunplus/phy-sunplus-usb2.c
index 637a5fbae6d9..0ad4c7160d17 100644
--- a/drivers/phy/sunplus/phy-sunplus-usb2.c
+++ b/drivers/phy/sunplus/phy-sunplus-usb2.c
@@ -116,11 +116,11 @@ static int sp_uphy_init(struct phy *phy)
ret = clk_prepare_enable(usbphy->phy_clk);
if (ret)
- goto err_clk;
+ return ret;
ret = reset_control_deassert(usbphy->rstc);
if (ret)
- goto err_reset;
+ goto err_clk;
/* Default value modification */
writel(HIGH_MASK_BITS | 0x4002, usbphy->moon4_regs + UPHY_CONTROL0);
@@ -129,7 +129,7 @@ static int sp_uphy_init(struct phy *phy)
/* disconnect voltage */
ret = update_disc_vol(usbphy);
if (ret < 0)
- return ret;
+ goto err_reset;
/* board uphy 0 internal register modification for tid certification */
val = readl(usbphy->phy_regs + CONFIG9);
diff --git a/drivers/pinctrl/bcm/pinctrl-bcm2835.c b/drivers/pinctrl/bcm/pinctrl-bcm2835.c
index cc1fe0555e19..5e31a5704fcc 100644
--- a/drivers/pinctrl/bcm/pinctrl-bcm2835.c
+++ b/drivers/pinctrl/bcm/pinctrl-bcm2835.c
@@ -1348,7 +1348,6 @@ static int bcm2835_pinctrl_probe(struct platform_device *pdev)
pc->pctl_desc = *pdata->pctl_desc;
pc->pctl_dev = devm_pinctrl_register(dev, &pc->pctl_desc, pc);
if (IS_ERR(pc->pctl_dev)) {
- gpiochip_remove(&pc->gpio_chip);
return PTR_ERR(pc->pctl_dev);
}
diff --git a/drivers/pinctrl/mediatek/mtk-eint.c b/drivers/pinctrl/mediatek/mtk-eint.c
index 27f0a54e12bf..4c4ee1d3a3a9 100644
--- a/drivers/pinctrl/mediatek/mtk-eint.c
+++ b/drivers/pinctrl/mediatek/mtk-eint.c
@@ -1,17 +1,21 @@
// SPDX-License-Identifier: GPL-2.0
-// Copyright (c) 2014-2018 MediaTek Inc.
+// Copyright (c) 2014-2025 MediaTek Inc.
/*
* Library for MediaTek External Interrupt Support
*
* Author: Maoguang Meng <maoguang.meng@mediatek.com>
* Sean Wang <sean.wang@mediatek.com>
+ * Hao Chang <ot_chhao.chang@mediatek.com>
+ * Qingliang Li <qingliang.li@mediatek.com>
*
*/
#include <linux/delay.h>
+#include <linux/device.h>
#include <linux/err.h>
#include <linux/gpio/driver.h>
+#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/irqdomain.h>
@@ -68,13 +72,11 @@ static void __iomem *mtk_eint_get_offset(struct mtk_eint *eint,
unsigned int eint_num,
unsigned int offset)
{
- unsigned int eint_base = 0;
+ unsigned int idx = eint->pins[eint_num].index;
+ unsigned int inst = eint->pins[eint_num].instance;
void __iomem *reg;
- if (eint_num >= eint->hw->ap_num)
- eint_base = eint->hw->ap_num;
-
- reg = eint->base + offset + ((eint_num - eint_base) / 32) * 4;
+ reg = eint->base[inst] + offset + (idx / 32 * 4);
return reg;
}
@@ -83,7 +85,7 @@ static unsigned int mtk_eint_can_en_debounce(struct mtk_eint *eint,
unsigned int eint_num)
{
unsigned int sens;
- unsigned int bit = BIT(eint_num % 32);
+ unsigned int bit = BIT(eint->pins[eint_num].index % 32);
void __iomem *reg = mtk_eint_get_offset(eint, eint_num,
eint->regs->sens);
@@ -92,7 +94,7 @@ static unsigned int mtk_eint_can_en_debounce(struct mtk_eint *eint,
else
sens = MTK_EINT_EDGE_SENSITIVE;
- if (eint_num < eint->hw->db_cnt && sens != MTK_EINT_EDGE_SENSITIVE)
+ if (eint->pins[eint_num].debounce && sens != MTK_EINT_EDGE_SENSITIVE)
return 1;
else
return 0;
@@ -102,9 +104,9 @@ static int mtk_eint_flip_edge(struct mtk_eint *eint, int hwirq)
{
int start_level, curr_level;
unsigned int reg_offset;
- u32 mask = BIT(hwirq & 0x1f);
- u32 port = (hwirq >> 5) & eint->hw->port_mask;
- void __iomem *reg = eint->base + (port << 2);
+ unsigned int mask = BIT(eint->pins[hwirq].index & 0x1f);
+ unsigned int port = (eint->pins[hwirq].index >> 5) & eint->hw->port_mask;
+ void __iomem *reg = eint->base[eint->pins[hwirq].instance] + (port << 2);
curr_level = eint->gpio_xlate->get_gpio_state(eint->pctl, hwirq);
@@ -126,11 +128,13 @@ static int mtk_eint_flip_edge(struct mtk_eint *eint, int hwirq)
static void mtk_eint_mask(struct irq_data *d)
{
struct mtk_eint *eint = irq_data_get_irq_chip_data(d);
- u32 mask = BIT(d->hwirq & 0x1f);
+ unsigned int idx = eint->pins[d->hwirq].index;
+ unsigned int inst = eint->pins[d->hwirq].instance;
+ unsigned int mask = BIT(idx & 0x1f);
void __iomem *reg = mtk_eint_get_offset(eint, d->hwirq,
eint->regs->mask_set);
- eint->cur_mask[d->hwirq >> 5] &= ~mask;
+ eint->cur_mask[inst][idx >> 5] &= ~mask;
writel(mask, reg);
}
@@ -138,22 +142,24 @@ static void mtk_eint_mask(struct irq_data *d)
static void mtk_eint_unmask(struct irq_data *d)
{
struct mtk_eint *eint = irq_data_get_irq_chip_data(d);
- u32 mask = BIT(d->hwirq & 0x1f);
+ unsigned int idx = eint->pins[d->hwirq].index;
+ unsigned int inst = eint->pins[d->hwirq].instance;
+ unsigned int mask = BIT(idx & 0x1f);
void __iomem *reg = mtk_eint_get_offset(eint, d->hwirq,
eint->regs->mask_clr);
- eint->cur_mask[d->hwirq >> 5] |= mask;
+ eint->cur_mask[inst][idx >> 5] |= mask;
writel(mask, reg);
- if (eint->dual_edge[d->hwirq])
+ if (eint->pins[d->hwirq].dual_edge)
mtk_eint_flip_edge(eint, d->hwirq);
}
static unsigned int mtk_eint_get_mask(struct mtk_eint *eint,
unsigned int eint_num)
{
- unsigned int bit = BIT(eint_num % 32);
+ unsigned int bit = BIT(eint->pins[eint_num].index % 32);
void __iomem *reg = mtk_eint_get_offset(eint, eint_num,
eint->regs->mask);
@@ -163,7 +169,7 @@ static unsigned int mtk_eint_get_mask(struct mtk_eint *eint,
static void mtk_eint_ack(struct irq_data *d)
{
struct mtk_eint *eint = irq_data_get_irq_chip_data(d);
- u32 mask = BIT(d->hwirq & 0x1f);
+ unsigned int mask = BIT(eint->pins[d->hwirq].index & 0x1f);
void __iomem *reg = mtk_eint_get_offset(eint, d->hwirq,
eint->regs->ack);
@@ -174,7 +180,7 @@ static int mtk_eint_set_type(struct irq_data *d, unsigned int type)
{
struct mtk_eint *eint = irq_data_get_irq_chip_data(d);
bool masked;
- u32 mask = BIT(d->hwirq & 0x1f);
+ unsigned int mask = BIT(eint->pins[d->hwirq].index & 0x1f);
void __iomem *reg;
if (((type & IRQ_TYPE_EDGE_BOTH) && (type & IRQ_TYPE_LEVEL_MASK)) ||
@@ -186,9 +192,9 @@ static int mtk_eint_set_type(struct irq_data *d, unsigned int type)
}
if ((type & IRQ_TYPE_EDGE_BOTH) == IRQ_TYPE_EDGE_BOTH)
- eint->dual_edge[d->hwirq] = 1;
+ eint->pins[d->hwirq].dual_edge = 1;
else
- eint->dual_edge[d->hwirq] = 0;
+ eint->pins[d->hwirq].dual_edge = 0;
if (!mtk_eint_get_mask(eint, d->hwirq)) {
mtk_eint_mask(d);
@@ -223,27 +229,32 @@ static int mtk_eint_set_type(struct irq_data *d, unsigned int type)
static int mtk_eint_irq_set_wake(struct irq_data *d, unsigned int on)
{
struct mtk_eint *eint = irq_data_get_irq_chip_data(d);
- int shift = d->hwirq & 0x1f;
- int reg = d->hwirq >> 5;
+ unsigned int idx = eint->pins[d->hwirq].index;
+ unsigned int inst = eint->pins[d->hwirq].instance;
+ unsigned int shift = idx & 0x1f;
+ unsigned int port = idx >> 5;
if (on)
- eint->wake_mask[reg] |= BIT(shift);
+ eint->wake_mask[inst][port] |= BIT(shift);
else
- eint->wake_mask[reg] &= ~BIT(shift);
+ eint->wake_mask[inst][port] &= ~BIT(shift);
return 0;
}
static void mtk_eint_chip_write_mask(const struct mtk_eint *eint,
- void __iomem *base, u32 *buf)
+ unsigned int **buf)
{
- int port;
+ int inst, port, port_num;
void __iomem *reg;
- for (port = 0; port < eint->hw->ports; port++) {
- reg = base + (port << 2);
- writel_relaxed(~buf[port], reg + eint->regs->mask_set);
- writel_relaxed(buf[port], reg + eint->regs->mask_clr);
+ for (inst = 0; inst < eint->nbase; inst++) {
+ port_num = DIV_ROUND_UP(eint->base_pin_num[inst], 32);
+ for (port = 0; port < port_num; port++) {
+ reg = eint->base[inst] + (port << 2);
+ writel_relaxed(~buf[inst][port], reg + eint->regs->mask_set);
+ writel_relaxed(buf[inst][port], reg + eint->regs->mask_clr);
+ }
}
}
@@ -303,15 +314,18 @@ static struct irq_chip mtk_eint_irq_chip = {
static unsigned int mtk_eint_hw_init(struct mtk_eint *eint)
{
- void __iomem *dom_en = eint->base + eint->regs->dom_en;
- void __iomem *mask_set = eint->base + eint->regs->mask_set;
- unsigned int i;
-
- for (i = 0; i < eint->hw->ap_num; i += 32) {
- writel(0xffffffff, dom_en);
- writel(0xffffffff, mask_set);
- dom_en += 4;
- mask_set += 4;
+ void __iomem *dom_reg, *mask_reg;
+ unsigned int i, j;
+
+ for (i = 0; i < eint->nbase; i++) {
+ dom_reg = eint->base[i] + eint->regs->dom_en;
+ mask_reg = eint->base[i] + eint->regs->mask_set;
+ for (j = 0; j < eint->base_pin_num[i]; j += 32) {
+ writel(0xffffffff, dom_reg);
+ writel(0xffffffff, mask_reg);
+ dom_reg += 4;
+ mask_reg += 4;
+ }
}
return 0;
@@ -322,14 +336,16 @@ mtk_eint_debounce_process(struct mtk_eint *eint, int index)
{
unsigned int rst, ctrl_offset;
unsigned int bit, dbnc;
+ unsigned int inst = eint->pins[index].instance;
+ unsigned int idx = eint->pins[index].index;
- ctrl_offset = (index / 4) * 4 + eint->regs->dbnc_ctrl;
- dbnc = readl(eint->base + ctrl_offset);
- bit = MTK_EINT_DBNC_SET_EN << ((index % 4) * 8);
+ ctrl_offset = (idx / 4) * 4 + eint->regs->dbnc_ctrl;
+ dbnc = readl(eint->base[inst] + ctrl_offset);
+ bit = MTK_EINT_DBNC_SET_EN << ((idx % 4) * 8);
if ((bit & dbnc) > 0) {
- ctrl_offset = (index / 4) * 4 + eint->regs->dbnc_set;
- rst = MTK_EINT_DBNC_RST_BIT << ((index % 4) * 8);
- writel(rst, eint->base + ctrl_offset);
+ ctrl_offset = (idx / 4) * 4 + eint->regs->dbnc_set;
+ rst = MTK_EINT_DBNC_RST_BIT << ((idx % 4) * 8);
+ writel(rst, eint->base[inst] + ctrl_offset);
}
}
@@ -337,65 +353,68 @@ static void mtk_eint_irq_handler(struct irq_desc *desc)
{
struct irq_chip *chip = irq_desc_get_chip(desc);
struct mtk_eint *eint = irq_desc_get_handler_data(desc);
- unsigned int status, eint_num;
- int offset, mask_offset, index;
- void __iomem *reg = mtk_eint_get_offset(eint, 0, eint->regs->stat);
+ unsigned int i, j, port, status, shift, mask, eint_num;
+ void __iomem *reg;
int dual_edge, start_level, curr_level;
chained_irq_enter(chip, desc);
- for (eint_num = 0; eint_num < eint->hw->ap_num; eint_num += 32,
- reg += 4) {
- status = readl(reg);
- while (status) {
- offset = __ffs(status);
- mask_offset = eint_num >> 5;
- index = eint_num + offset;
- status &= ~BIT(offset);
-
- /*
- * If we get an interrupt on pin that was only required
- * for wake (but no real interrupt requested), mask the
- * interrupt (as would mtk_eint_resume do anyway later
- * in the resume sequence).
- */
- if (eint->wake_mask[mask_offset] & BIT(offset) &&
- !(eint->cur_mask[mask_offset] & BIT(offset))) {
- writel_relaxed(BIT(offset), reg -
- eint->regs->stat +
- eint->regs->mask_set);
- }
-
- dual_edge = eint->dual_edge[index];
- if (dual_edge) {
- /*
- * Clear soft-irq in case we raised it last
- * time.
- */
- writel(BIT(offset), reg - eint->regs->stat +
- eint->regs->soft_clr);
-
- start_level =
- eint->gpio_xlate->get_gpio_state(eint->pctl,
- index);
- }
-
- generic_handle_domain_irq(eint->domain, index);
-
- if (dual_edge) {
- curr_level = mtk_eint_flip_edge(eint, index);
+ for (i = 0; i < eint->nbase; i++) {
+ for (j = 0; j < eint->base_pin_num[i]; j += 32) {
+ port = j >> 5;
+ status = readl(eint->base[i] + port * 4 + eint->regs->stat);
+ while (status) {
+ shift = __ffs(status);
+ status &= ~BIT(shift);
+ mask = BIT(shift);
+ eint_num = eint->pin_list[i][shift + j];
/*
- * If level changed, we might lost one edge
- * interrupt, raised it through soft-irq.
+ * If we get an interrupt on pin that was only required
+ * for wake (but no real interrupt requested), mask the
+ * interrupt (as would mtk_eint_resume do anyway later
+ * in the resume sequence).
*/
- if (start_level != curr_level)
- writel(BIT(offset), reg -
- eint->regs->stat +
- eint->regs->soft_set);
+ if (eint->wake_mask[i][port] & mask &&
+ !(eint->cur_mask[i][port] & mask)) {
+ reg = mtk_eint_get_offset(eint, eint_num,
+ eint->regs->mask_set);
+ writel_relaxed(mask, reg);
+ }
+
+ dual_edge = eint->pins[eint_num].dual_edge;
+ if (dual_edge) {
+ /*
+ * Clear soft-irq in case we raised it last
+ * time.
+ */
+ reg = mtk_eint_get_offset(eint, eint_num,
+ eint->regs->soft_clr);
+ writel(mask, reg);
+
+ start_level =
+ eint->gpio_xlate->get_gpio_state(eint->pctl,
+ eint_num);
+ }
+
+ generic_handle_domain_irq(eint->domain, eint_num);
+
+ if (dual_edge) {
+ curr_level = mtk_eint_flip_edge(eint, eint_num);
+
+ /*
+ * If level changed, we might lost one edge
+ * interrupt, raised it through soft-irq.
+ */
+ if (start_level != curr_level) {
+ reg = mtk_eint_get_offset(eint, eint_num,
+ eint->regs->soft_set);
+ writel(mask, reg);
+ }
+ }
+
+ if (eint->pins[eint_num].debounce)
+ mtk_eint_debounce_process(eint, eint_num);
}
-
- if (index < eint->hw->db_cnt)
- mtk_eint_debounce_process(eint, index);
}
}
chained_irq_exit(chip, desc);
@@ -403,7 +422,7 @@ static void mtk_eint_irq_handler(struct irq_desc *desc)
int mtk_eint_do_suspend(struct mtk_eint *eint)
{
- mtk_eint_chip_write_mask(eint, eint->base, eint->wake_mask);
+ mtk_eint_chip_write_mask(eint, eint->wake_mask);
return 0;
}
@@ -411,7 +430,7 @@ EXPORT_SYMBOL_GPL(mtk_eint_do_suspend);
int mtk_eint_do_resume(struct mtk_eint *eint)
{
- mtk_eint_chip_write_mask(eint, eint->base, eint->cur_mask);
+ mtk_eint_chip_write_mask(eint, eint->cur_mask);
return 0;
}
@@ -423,17 +442,19 @@ int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_num,
int virq, eint_offset;
unsigned int set_offset, bit, clr_bit, clr_offset, rst, i, unmask,
dbnc;
+ unsigned int inst = eint->pins[eint_num].instance;
+ unsigned int idx = eint->pins[eint_num].index;
struct irq_data *d;
if (!eint->hw->db_time)
return -EOPNOTSUPP;
virq = irq_find_mapping(eint->domain, eint_num);
- eint_offset = (eint_num % 4) * 8;
+ eint_offset = (idx % 4) * 8;
d = irq_get_irq_data(virq);
- set_offset = (eint_num / 4) * 4 + eint->regs->dbnc_set;
- clr_offset = (eint_num / 4) * 4 + eint->regs->dbnc_clr;
+ set_offset = (idx / 4) * 4 + eint->regs->dbnc_set;
+ clr_offset = (idx / 4) * 4 + eint->regs->dbnc_clr;
if (!mtk_eint_can_en_debounce(eint, eint_num))
return -EINVAL;
@@ -454,12 +475,12 @@ int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_num,
}
clr_bit = 0xff << eint_offset;
- writel(clr_bit, eint->base + clr_offset);
+ writel(clr_bit, eint->base[inst] + clr_offset);
bit = ((dbnc << MTK_EINT_DBNC_SET_DBNC_BITS) | MTK_EINT_DBNC_SET_EN) <<
eint_offset;
rst = MTK_EINT_DBNC_RST_BIT << eint_offset;
- writel(rst | bit, eint->base + set_offset);
+ writel(rst | bit, eint->base[inst] + set_offset);
/*
* Delay a while (more than 2T) to wait for hw debounce counter reset
@@ -485,34 +506,89 @@ int mtk_eint_find_irq(struct mtk_eint *eint, unsigned long eint_n)
}
EXPORT_SYMBOL_GPL(mtk_eint_find_irq);
-int mtk_eint_do_init(struct mtk_eint *eint)
+static void mtk_eint_teardown(void *data)
+{
+ struct mtk_eint *eint = data;
+ unsigned int i, virq;
+
+ /* Detach the demux handler so it can no longer reference freed data. */
+ irq_set_chained_handler_and_data(eint->irq, NULL, NULL);
+
+ /* Wait for any in-flight handler to finish before tearing down. */
+ synchronize_irq(eint->irq);
+
+ /* Dispose of all child mappings before the domain is removed. */
+ for (i = 0; i < eint->hw->ap_num; i++) {
+ virq = irq_find_mapping(eint->domain, i);
+ if (virq)
+ irq_dispose_mapping(virq);
+ }
+
+ irq_domain_remove(eint->domain);
+}
+
+int mtk_eint_do_init(struct mtk_eint *eint, struct mtk_eint_pin *eint_pin)
{
- int i;
+ unsigned int size, i, port, virq, inst = 0;
/* If clients don't assign a specific regs, let's use generic one */
if (!eint->regs)
eint->regs = &mtk_generic_eint_regs;
- eint->wake_mask = devm_kcalloc(eint->dev, eint->hw->ports,
- sizeof(*eint->wake_mask), GFP_KERNEL);
- if (!eint->wake_mask)
+ eint->base_pin_num = devm_kmalloc_array(eint->dev, eint->nbase, sizeof(u16),
+ GFP_KERNEL | __GFP_ZERO);
+ if (!eint->base_pin_num)
return -ENOMEM;
- eint->cur_mask = devm_kcalloc(eint->dev, eint->hw->ports,
- sizeof(*eint->cur_mask), GFP_KERNEL);
- if (!eint->cur_mask)
- return -ENOMEM;
+ if (eint_pin) {
+ eint->pins = eint_pin;
+ for (i = 0; i < eint->hw->ap_num; i++) {
+ inst = eint->pins[i].instance;
+ if (inst >= eint->nbase)
+ continue;
+ eint->base_pin_num[inst]++;
+ }
+ } else {
+ size = eint->hw->ap_num * sizeof(struct mtk_eint_pin);
+ eint->pins = devm_kmalloc(eint->dev, size, GFP_KERNEL);
+ if (!eint->pins)
+ goto err_pins;
+
+ eint->base_pin_num[inst] = eint->hw->ap_num;
+ for (i = 0; i < eint->hw->ap_num; i++) {
+ eint->pins[i].instance = inst;
+ eint->pins[i].index = i;
+ eint->pins[i].debounce = (i < eint->hw->db_cnt) ? 1 : 0;
+ }
+ }
- eint->dual_edge = devm_kcalloc(eint->dev, eint->hw->ap_num,
- sizeof(int), GFP_KERNEL);
- if (!eint->dual_edge)
- return -ENOMEM;
+ eint->pin_list = devm_kmalloc(eint->dev, eint->nbase * sizeof(u16 *), GFP_KERNEL);
+ if (!eint->pin_list)
+ goto err_pin_list;
+
+ eint->wake_mask = devm_kmalloc(eint->dev, eint->nbase * sizeof(u32 *), GFP_KERNEL);
+ if (!eint->wake_mask)
+ goto err_wake_mask;
+
+ eint->cur_mask = devm_kmalloc(eint->dev, eint->nbase * sizeof(u32 *), GFP_KERNEL);
+ if (!eint->cur_mask)
+ goto err_cur_mask;
+
+ for (i = 0; i < eint->nbase; i++) {
+ eint->pin_list[i] = devm_kzalloc(eint->dev, eint->base_pin_num[i] * sizeof(u16),
+ GFP_KERNEL);
+ port = DIV_ROUND_UP(eint->base_pin_num[i], 32);
+ eint->wake_mask[i] = devm_kzalloc(eint->dev, port * sizeof(u32), GFP_KERNEL);
+ eint->cur_mask[i] = devm_kzalloc(eint->dev, port * sizeof(u32), GFP_KERNEL);
+ if (!eint->pin_list[i] || !eint->wake_mask[i] || !eint->cur_mask[i])
+ goto err_eint;
+ }
eint->domain = irq_domain_add_linear(eint->dev->of_node,
eint->hw->ap_num,
&irq_domain_simple_ops, NULL);
if (!eint->domain)
- return -ENOMEM;
+ goto err_eint;
if (eint->hw->db_time) {
for (i = 0; i < MTK_EINT_DBNC_MAX; i++)
@@ -523,8 +599,11 @@ int mtk_eint_do_init(struct mtk_eint *eint)
mtk_eint_hw_init(eint);
for (i = 0; i < eint->hw->ap_num; i++) {
- int virq = irq_create_mapping(eint->domain, i);
-
+ inst = eint->pins[i].instance;
+ if (inst >= eint->nbase)
+ continue;
+ eint->pin_list[inst][eint->pins[i].index] = i;
+ virq = irq_create_mapping(eint->domain, i);
irq_set_chip_and_handler(virq, &mtk_eint_irq_chip,
handle_level_irq);
irq_set_chip_data(virq, eint);
@@ -533,7 +612,28 @@ int mtk_eint_do_init(struct mtk_eint *eint)
irq_set_chained_handler_and_data(eint->irq, mtk_eint_irq_handler,
eint);
- return 0;
+ return devm_add_action_or_reset(eint->dev, mtk_eint_teardown, eint);
+
+err_eint:
+ for (i = 0; i < eint->nbase; i++) {
+ if (eint->cur_mask[i])
+ devm_kfree(eint->dev, eint->cur_mask[i]);
+ if (eint->wake_mask[i])
+ devm_kfree(eint->dev, eint->wake_mask[i]);
+ if (eint->pin_list[i])
+ devm_kfree(eint->dev, eint->pin_list[i]);
+ }
+ devm_kfree(eint->dev, eint->cur_mask);
+err_cur_mask:
+ devm_kfree(eint->dev, eint->wake_mask);
+err_wake_mask:
+ devm_kfree(eint->dev, eint->pin_list);
+err_pin_list:
+ if (!eint_pin)
+ devm_kfree(eint->dev, eint->pins);
+err_pins:
+ devm_kfree(eint->dev, eint->base_pin_num);
+ return -ENOMEM;
}
EXPORT_SYMBOL_GPL(mtk_eint_do_init);
diff --git a/drivers/pinctrl/mediatek/mtk-eint.h b/drivers/pinctrl/mediatek/mtk-eint.h
index 6139b16cd225..fc31a4c0c77b 100644
--- a/drivers/pinctrl/mediatek/mtk-eint.h
+++ b/drivers/pinctrl/mediatek/mtk-eint.h
@@ -1,10 +1,11 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
- * Copyright (C) 2014-2018 MediaTek Inc.
+ * Copyright (C) 2014-2025 MediaTek Inc.
*
* Author: Maoguang Meng <maoguang.meng@mediatek.com>
* Sean Wang <sean.wang@mediatek.com>
- *
+ * Hao Chang <ot_chhao.chang@mediatek.com>
+ * Qingliang Li <qingliang.li@mediatek.com>
*/
#ifndef __MTK_EINT_H
#define __MTK_EINT_H
@@ -40,6 +41,14 @@ struct mtk_eint_hw {
const unsigned int *db_time;
};
+struct mtk_eint_pin {
+ u16 number;
+ u8 instance;
+ u8 index;
+ bool debounce;
+ bool dual_edge;
+};
+
extern const unsigned int debounce_time_mt2701[];
extern const unsigned int debounce_time_mt6765[];
extern const unsigned int debounce_time_mt6795[];
@@ -56,17 +65,21 @@ struct mtk_eint_xt {
struct mtk_eint {
struct device *dev;
- void __iomem *base;
+ void __iomem **base;
+ int nbase;
+ u16 *base_pin_num;
struct irq_domain *domain;
int irq;
int *dual_edge;
- u32 *wake_mask;
- u32 *cur_mask;
+ u16 **pin_list;
+ u32 **wake_mask;
+ u32 **cur_mask;
/* Used to fit into various EINT device */
const struct mtk_eint_hw *hw;
const struct mtk_eint_regs *regs;
+ struct mtk_eint_pin *pins;
u16 num_db_time;
/* Used to fit into various pinctrl device */
@@ -75,7 +88,7 @@ struct mtk_eint {
};
#if IS_ENABLED(CONFIG_EINT_MTK)
-int mtk_eint_do_init(struct mtk_eint *eint);
+int mtk_eint_do_init(struct mtk_eint *eint, struct mtk_eint_pin *eint_pin);
int mtk_eint_do_suspend(struct mtk_eint *eint);
int mtk_eint_do_resume(struct mtk_eint *eint);
int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_n,
@@ -83,7 +96,8 @@ int mtk_eint_set_debounce(struct mtk_eint *eint, unsigned long eint_n,
int mtk_eint_find_irq(struct mtk_eint *eint, unsigned long eint_n);
#else
-static inline int mtk_eint_do_init(struct mtk_eint *eint)
+static inline int mtk_eint_do_init(struct mtk_eint *eint,
+ struct mtk_eint_pin *eint_pin)
{
return -EOPNOTSUPP;
}
diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.c b/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.c
index 54301fbba524..882733516b45 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.c
+++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.c
@@ -13,6 +13,7 @@
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/module.h>
+#include <linux/of_address.h>
#include <linux/of_irq.h>
#include "mtk-eint.h"
@@ -367,7 +368,7 @@ static const struct mtk_eint_xt mtk_eint_xt = {
int mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
- int ret;
+ int ret, i, j, count_reg_names;
if (!IS_ENABLED(CONFIG_EINT_MTK))
return 0;
@@ -379,10 +380,27 @@ int mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev)
if (!hw->eint)
return -ENOMEM;
- hw->eint->base = devm_platform_ioremap_resource_byname(pdev, "eint");
- if (IS_ERR(hw->eint->base)) {
- ret = PTR_ERR(hw->eint->base);
- goto err_free_eint;
+ count_reg_names = of_property_count_strings(np, "reg-names");
+ if (count_reg_names < 0)
+ return -EINVAL;
+
+ hw->eint->nbase = count_reg_names - (int)hw->soc->nbase_names;
+ if (hw->eint->nbase <= 0)
+ return -EINVAL;
+
+ hw->eint->base = devm_kmalloc_array(&pdev->dev, hw->eint->nbase,
+ sizeof(*hw->eint->base), GFP_KERNEL | __GFP_ZERO);
+ if (!hw->eint->base) {
+ ret = -ENOMEM;
+ goto err_free_base;
+ }
+
+ for (i = hw->soc->nbase_names, j = 0; i < count_reg_names; i++, j++) {
+ hw->eint->base[j] = of_iomap(np, i);
+ if (IS_ERR(hw->eint->base[j])) {
+ ret = PTR_ERR(hw->eint->base[j]);
+ goto err_free_eint;
+ }
}
hw->eint->irq = irq_of_parse_and_map(np, 0);
@@ -401,9 +419,19 @@ int mtk_build_eint(struct mtk_pinctrl *hw, struct platform_device *pdev)
hw->eint->pctl = hw;
hw->eint->gpio_xlate = &mtk_eint_xt;
- return mtk_eint_do_init(hw->eint);
+ ret = mtk_eint_do_init(hw->eint, hw->soc->eint_pin);
+ if (ret)
+ goto err_free_eint;
+
+ return 0;
err_free_eint:
+ for (j = 0; j < hw->eint->nbase; j++) {
+ if (hw->eint->base[j])
+ iounmap(hw->eint->base[j]);
+ }
+ devm_kfree(hw->dev, hw->eint->base);
+err_free_base:
devm_kfree(hw->dev, hw->eint);
hw->eint = NULL;
return ret;
diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.h b/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.h
index 23688ca6d04e..1dc2ed8a7df2 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.h
+++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.h
@@ -241,6 +241,7 @@ struct mtk_pin_soc {
unsigned int nfuncs;
const struct mtk_eint_regs *eint_regs;
const struct mtk_eint_hw *eint_hw;
+ struct mtk_eint_pin *eint_pin;
/* Specific parameters per SoC */
u8 gpio_m;
diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
index e5d6dc6b0069..efc9b07e6265 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
+++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
@@ -1015,9 +1015,15 @@ static int mtk_eint_init(struct mtk_pinctrl *pctl, struct platform_device *pdev)
if (!pctl->eint)
return -ENOMEM;
- pctl->eint->base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(pctl->eint->base))
- return PTR_ERR(pctl->eint->base);
+ pctl->eint->nbase = 1;
+ /* mtk-eint expects an array */
+ pctl->eint->base = devm_kzalloc(pctl->dev, sizeof(pctl->eint->base), GFP_KERNEL);
+ if (!pctl->eint->base)
+ return -ENOMEM;
+
+ pctl->eint->base[0] = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(pctl->eint->base[0]))
+ return PTR_ERR(pctl->eint->base[0]);
pctl->eint->irq = irq_of_parse_and_map(np, 0);
if (!pctl->eint->irq)
@@ -1033,7 +1039,7 @@ static int mtk_eint_init(struct mtk_pinctrl *pctl, struct platform_device *pdev)
pctl->eint->pctl = pctl;
pctl->eint->gpio_xlate = &mtk_eint_xt;
- return mtk_eint_do_init(pctl->eint);
+ return mtk_eint_do_init(pctl->eint, NULL);
}
/* This is used as a common probe function */
diff --git a/drivers/pinctrl/mediatek/pinctrl-paris.h b/drivers/pinctrl/mediatek/pinctrl-paris.h
index 948ce126aa0c..d8c1822662fc 100644
--- a/drivers/pinctrl/mediatek/pinctrl-paris.h
+++ b/drivers/pinctrl/mediatek/pinctrl-paris.h
@@ -49,6 +49,13 @@
__VA_ARGS__, { } }, \
}
+#define MTK_EINT_PIN(_number, _instance, _index, _debounce) { \
+ .number = _number, \
+ .instance = _instance, \
+ .index = _index, \
+ .debounce = _debounce, \
+ }
+
#define PINCTRL_PIN_GROUP(_name_, id) \
{ \
.grp = PINCTRL_PINGROUP(_name_,id##_pins, ARRAY_SIZE(id##_pins)), \
diff --git a/drivers/pinctrl/pinctrl-rockchip.c b/drivers/pinctrl/pinctrl-rockchip.c
index 894dde944684..bd3cfeffaac1 100644
--- a/drivers/pinctrl/pinctrl-rockchip.c
+++ b/drivers/pinctrl/pinctrl-rockchip.c
@@ -3409,6 +3409,16 @@ static struct rockchip_pin_ctrl *rockchip_pinctrl_get_soc_data(
pmu_offs = ctrl->pmu_mux_offset;
drv_pmu_offs = ctrl->pmu_drv_offset;
drv_grf_offs = ctrl->grf_drv_offset;
+
+ /*
+ * This function mutates the static per-SoC data. Most of it is
+ * idempotent: recalculated iomux and drv offsets anchor at the
+ * values calculated by a previous run. The pin count is not, so
+ * reset it here; otherwise it accumulates when the probe runs
+ * again after a probe deferral, shifting every bank's pin_base.
+ */
+ ctrl->nr_pins = 0;
+
bank = ctrl->pin_banks;
for (i = 0; i < ctrl->nr_banks; ++i, ++bank) {
int bank_pins = 0;
diff --git a/drivers/platform/chrome/cros_ec_debugfs.c b/drivers/platform/chrome/cros_ec_debugfs.c
index 839154c46e46..21b328ea9a33 100644
--- a/drivers/platform/chrome/cros_ec_debugfs.c
+++ b/drivers/platform/chrome/cros_ec_debugfs.c
@@ -525,7 +525,7 @@ static int cros_ec_debugfs_probe(struct platform_device *pd)
ret = blocking_notifier_chain_register(&ec->ec_dev->panic_notifier,
&debug_info->notifier_panic);
if (ret)
- goto remove_debugfs;
+ goto cleanup_console_log;
ec->debug_info = debug_info;
@@ -533,6 +533,8 @@ static int cros_ec_debugfs_probe(struct platform_device *pd)
return 0;
+cleanup_console_log:
+ cros_ec_cleanup_console_log(debug_info);
remove_debugfs:
debugfs_remove_recursive(debug_info->dir);
return ret;
@@ -542,6 +544,8 @@ static void cros_ec_debugfs_remove(struct platform_device *pd)
{
struct cros_ec_dev *ec = dev_get_drvdata(pd->dev.parent);
+ blocking_notifier_chain_unregister(&ec->ec_dev->panic_notifier,
+ &ec->debug_info->notifier_panic);
debugfs_remove_recursive(ec->debug_info->dir);
cros_ec_cleanup_console_log(ec->debug_info);
}
diff --git a/drivers/platform/chrome/cros_ec_sensorhub_ring.c b/drivers/platform/chrome/cros_ec_sensorhub_ring.c
index 302d037b90a1..b5970e8620bf 100644
--- a/drivers/platform/chrome/cros_ec_sensorhub_ring.c
+++ b/drivers/platform/chrome/cros_ec_sensorhub_ring.c
@@ -825,8 +825,15 @@ static void cros_ec_sensorhub_ring_handler(struct cros_ec_sensorhub *sensorhub)
sensorhub->msg->outsize = 1;
sensorhub->msg->insize = fifo_info_length;
- if (cros_ec_cmd_xfer_status(ec->ec_dev, sensorhub->msg) < 0)
+ ret = cros_ec_cmd_xfer_status(ec->ec_dev, sensorhub->msg);
+ if (ret < 0)
+ goto error;
+ if (ret != fifo_info_length) {
+ dev_warn_ratelimited(sensorhub->dev,
+ "Mismatch read length: size %d - expected %d\n",
+ ret, fifo_info_length);
goto error;
+ }
memcpy(fifo_info, &sensorhub->resp->fifo_info,
fifo_info_length);
diff --git a/drivers/platform/chrome/cros_ec_typec.c b/drivers/platform/chrome/cros_ec_typec.c
index ca665b901010..8c0b814973c0 100644
--- a/drivers/platform/chrome/cros_ec_typec.c
+++ b/drivers/platform/chrome/cros_ec_typec.c
@@ -989,6 +989,12 @@ static void cros_typec_register_partner_pdos(struct cros_typec_data *typec,
if (!resp->source_cap_count && !resp->sink_cap_count)
return;
+ if (resp->source_cap_count > PDO_MAX_OBJECTS ||
+ resp->sink_cap_count > PDO_MAX_OBJECTS) {
+ dev_warn(typec->dev, "Invalid PDO count from EC, port: %d\n", port_num);
+ return;
+ }
+
port->partner_pd = typec_partner_usb_power_delivery_register(port->partner, &desc);
if (IS_ERR(port->partner_pd)) {
dev_warn(typec->dev, "Failed to register partner PD device, port: %d\n", port_num);
diff --git a/drivers/platform/mellanox/mlxbf-bootctl.c b/drivers/platform/mellanox/mlxbf-bootctl.c
index dd5f370c3168..73f74c61df2a 100644
--- a/drivers/platform/mellanox/mlxbf-bootctl.c
+++ b/drivers/platform/mellanox/mlxbf-bootctl.c
@@ -10,6 +10,7 @@
#include <linux/acpi.h>
#include <linux/arm-smccc.h>
+#include <linux/bitfield.h>
#include <linux/delay.h>
#include <linux/if_ether.h>
#include <linux/iopoll.h>
diff --git a/drivers/platform/mellanox/mlxbf-pmc.c b/drivers/platform/mellanox/mlxbf-pmc.c
index 67d9b19731ed..fcc8d9eaf37c 100644
--- a/drivers/platform/mellanox/mlxbf-pmc.c
+++ b/drivers/platform/mellanox/mlxbf-pmc.c
@@ -2021,13 +2021,19 @@ static int mlxbf_pmc_map_counters(struct device *dev)
static int mlxbf_pmc_probe(struct platform_device *pdev)
{
- struct acpi_device *acpi_dev = ACPI_COMPANION(&pdev->dev);
- const char *hid = acpi_device_hid(acpi_dev);
struct device *dev = &pdev->dev;
+ struct acpi_device *acpi_dev;
struct arm_smccc_res res;
+ const char *hid;
guid_t guid;
int ret;
+ acpi_dev = ACPI_COMPANION(&pdev->dev);
+ if (!acpi_dev)
+ return -ENODEV;
+
+ hid = acpi_device_hid(acpi_dev);
+
/* Ensure we have the UUID we expect for this service. */
arm_smccc_smc(MLXBF_PMC_SIP_SVC_UID, 0, 0, 0, 0, 0, 0, 0, &res);
guid_parse(mlxbf_pmc_svc_uuid_str, &guid);
diff --git a/drivers/platform/surface/surface_acpi_notify.c b/drivers/platform/surface/surface_acpi_notify.c
index 14a9d8a267cb..7651075572d6 100644
--- a/drivers/platform/surface/surface_acpi_notify.c
+++ b/drivers/platform/surface/surface_acpi_notify.c
@@ -777,12 +777,16 @@ static int san_consumer_links_setup(struct platform_device *pdev)
static int san_probe(struct platform_device *pdev)
{
- struct acpi_device *san = ACPI_COMPANION(&pdev->dev);
struct ssam_controller *ctrl;
+ struct acpi_device *san;
struct san_data *data;
acpi_status astatus;
int status;
+ san = ACPI_COMPANION(&pdev->dev);
+ if (!san)
+ return -ENODEV;
+
ctrl = ssam_client_bind(&pdev->dev);
if (IS_ERR(ctrl))
return PTR_ERR(ctrl) == -ENODEV ? -EPROBE_DEFER : PTR_ERR(ctrl);
diff --git a/drivers/platform/x86/dell/dell-wmi-base.c b/drivers/platform/x86/dell/dell-wmi-base.c
index 01f3ff21c888..713ec2c12a04 100644
--- a/drivers/platform/x86/dell/dell-wmi-base.c
+++ b/drivers/platform/x86/dell/dell-wmi-base.c
@@ -839,9 +839,22 @@ static int __init dell_wmi_init(void)
err = dell_privacy_register_driver();
if (err)
- return err;
+ goto out_smbios;
- return wmi_driver_register(&dell_wmi_driver);
+ err = wmi_driver_register(&dell_wmi_driver);
+ if (err)
+ goto out_privacy;
+
+ return 0;
+
+out_privacy:
+ dell_privacy_unregister_driver();
+
+out_smbios:
+ if (wmi_requires_smbios_request)
+ dell_wmi_events_set_enabled(false);
+
+ return err;
}
late_initcall(dell_wmi_init);
diff --git a/drivers/platform/x86/dell/dell-wmi-privacy.c b/drivers/platform/x86/dell/dell-wmi-privacy.c
index 4b65e1655d42..7c2b6fae96c5 100644
--- a/drivers/platform/x86/dell/dell-wmi-privacy.c
+++ b/drivers/platform/x86/dell/dell-wmi-privacy.c
@@ -69,11 +69,11 @@ bool dell_privacy_has_mic_mute(void)
{
struct privacy_wmi_data *priv;
- mutex_lock(&list_mutex);
+ guard(mutex)(&list_mutex);
+
priv = list_first_entry_or_null(&wmi_list,
struct privacy_wmi_data,
list);
- mutex_unlock(&list_mutex);
return priv && (priv->features_present & BIT(DELL_PRIVACY_TYPE_AUDIO));
}
diff --git a/drivers/platform/x86/dell/dell-wmi-sysman/dell-wmi-sysman.h b/drivers/platform/x86/dell/dell-wmi-sysman/dell-wmi-sysman.h
index 817ee7ba07ca..b0810a02e6da 100644
--- a/drivers/platform/x86/dell/dell-wmi-sysman/dell-wmi-sysman.h
+++ b/drivers/platform/x86/dell/dell-wmi-sysman/dell-wmi-sysman.h
@@ -107,7 +107,7 @@ enum {
static int get_##type##_instance_id(struct kobject *kobj) \
{ \
int i; \
- for (i = 0; i <= wmi_priv.type##_instances_count; i++) { \
+ for (i = 0; i < wmi_priv.type##_instances_count; i++) { \
if (!(strcmp(kobj->name, wmi_priv.type##_data[i].attribute_name)))\
return i; \
} \
diff --git a/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c b/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
index 1ebc5afc850f..34740f29f41a 100644
--- a/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
+++ b/drivers/platform/x86/hp/hp-bioscfg/passwdobj-attributes.c
@@ -353,6 +353,11 @@ static int hp_populate_password_elements_from_package(union acpi_object *passwor
case PSWD_ENCODINGS:
size = min_t(u32, password_data->encodings_size, MAX_ENCODINGS_SIZE);
for (pos_values = 0; pos_values < size; pos_values++) {
+ if (elem + pos_values >= password_obj_count) {
+ pr_err("Error elem-objects package is too small\n");
+ return -EINVAL;
+ }
+
ret = hp_convert_hexstr_to_str(password_obj[elem + pos_values].string.pointer,
password_obj[elem + pos_values].string.length,
&str_value, &value_len);
diff --git a/drivers/platform/x86/lg-laptop.c b/drivers/platform/x86/lg-laptop.c
index 6af6cf477c5b..f4e43e09cbe4 100644
--- a/drivers/platform/x86/lg-laptop.c
+++ b/drivers/platform/x86/lg-laptop.c
@@ -98,6 +98,7 @@ static u32 inited;
#define INIT_SPARSE_KEYMAP 0x80
static int battery_limit_use_wmbb;
+static bool kbd_backlight_available;
static struct led_classdev kbd_backlight;
static enum led_brightness get_kbd_backlight_level(struct device *dev);
@@ -212,6 +213,7 @@ static union acpi_object *lg_wmbb(struct device *dev, u32 method_id, u32 arg1, u
static void wmi_notify(union acpi_object *obj, void *context)
{
long data = (long)context;
+ unsigned int brightness;
pr_debug("event guid %li\n", data);
if (!obj)
@@ -222,8 +224,11 @@ static void wmi_notify(union acpi_object *obj, void *context)
struct key_entry *key;
if (eventcode == 0x10000000) {
- led_classdev_notify_brightness_hw_changed(
- &kbd_backlight, get_kbd_backlight_level(kbd_backlight.dev->parent));
+ if (kbd_backlight_available) {
+ brightness = get_kbd_backlight_level(kbd_backlight.dev->parent);
+ led_classdev_notify_brightness_hw_changed(&kbd_backlight,
+ brightness);
+ }
} else {
key = sparse_keymap_entry_from_scancode(
wmi_input_dev, eventcode);
@@ -269,11 +274,6 @@ static void wmi_input_setup(void)
}
}
-static void acpi_notify(struct acpi_device *device, u32 event)
-{
- acpi_handle_debug(device->handle, "notify: %d\n", event);
-}
-
static ssize_t fan_mode_store(struct device *dev,
struct device_attribute *attr,
const char *buffer, size_t count)
@@ -753,13 +753,13 @@ static void lg_laptop_remove_address_space_handler(void *data)
&lg_laptop_address_space_handler);
}
-static int acpi_add(struct acpi_device *device)
+static int acpi_probe(struct platform_device *pdev)
{
struct platform_device_info pdev_info = {
- .fwnode = acpi_fwnode_handle(device),
.name = PLATFORM_NAME,
.id = PLATFORM_DEVID_NONE,
};
+ struct acpi_device *device;
acpi_status status;
int ret;
const char *product;
@@ -768,13 +768,19 @@ static int acpi_add(struct acpi_device *device)
if (pf_device)
return 0;
+ device = ACPI_COMPANION(&pdev->dev);
+ if (!device)
+ return -ENODEV;
+
+ pdev_info.fwnode = acpi_fwnode_handle(device),
+
status = acpi_install_address_space_handler(device->handle, LG_ADDRESS_SPACE_ID,
&lg_laptop_address_space_handler,
- NULL, &device->dev);
+ NULL, &pdev->dev);
if (ACPI_FAILURE(status))
return -ENODEV;
- ret = devm_add_action_or_reset(&device->dev, lg_laptop_remove_address_space_handler,
+ ret = devm_add_action_or_reset(&pdev->dev, lg_laptop_remove_address_space_handler,
device);
if (ret < 0)
return ret;
@@ -844,8 +850,13 @@ static int acpi_add(struct acpi_device *device)
goto out_platform_device;
/* LEDs are optional */
- led_classdev_register(&pf_device->dev, &kbd_backlight);
- led_classdev_register(&pf_device->dev, &tpad_led);
+ ret = devm_led_classdev_register(&pdev->dev, &kbd_backlight);
+ if (ret < 0)
+ kbd_backlight_available = false;
+ else
+ kbd_backlight_available = true;
+
+ devm_led_classdev_register(&pdev->dev, &tpad_led);
wmi_input_setup();
battery_hook_register(&battery_hook);
@@ -859,13 +870,10 @@ static int acpi_add(struct acpi_device *device)
return ret;
}
-static void acpi_remove(struct acpi_device *device)
+static void acpi_remove(struct platform_device *pdev)
{
sysfs_remove_group(&pf_device->dev.kobj, &dev_attribute_group);
- led_classdev_unregister(&tpad_led);
- led_classdev_unregister(&kbd_backlight);
-
battery_hook_unregister(&battery_hook);
wmi_input_destroy();
platform_device_unregister(pf_device);
@@ -879,34 +887,13 @@ static const struct acpi_device_id device_ids[] = {
};
MODULE_DEVICE_TABLE(acpi, device_ids);
-static struct acpi_driver acpi_driver = {
- .name = "LG Gram Laptop Support",
- .class = "lg-laptop",
- .ids = device_ids,
- .ops = {
- .add = acpi_add,
- .remove = acpi_remove,
- .notify = acpi_notify,
- },
+static struct platform_driver acpi_driver = {
+ .probe = acpi_probe,
+ .remove = acpi_remove,
+ .driver = {
+ .name = "LG Gram Laptop Support",
+ .acpi_match_table = device_ids,
+ },
};
-static int __init acpi_init(void)
-{
- int result;
-
- result = acpi_bus_register_driver(&acpi_driver);
- if (result < 0) {
- pr_debug("Error registering driver\n");
- return -ENODEV;
- }
-
- return 0;
-}
-
-static void __exit acpi_exit(void)
-{
- acpi_bus_unregister_driver(&acpi_driver);
-}
-
-module_init(acpi_init);
-module_exit(acpi_exit);
+module_platform_driver(acpi_driver);
diff --git a/drivers/pmdomain/bcm/bcm2835-power.c b/drivers/pmdomain/bcm/bcm2835-power.c
index 20130549a7a5..85406b4a99b0 100644
--- a/drivers/pmdomain/bcm/bcm2835-power.c
+++ b/drivers/pmdomain/bcm/bcm2835-power.c
@@ -682,7 +682,12 @@ static int bcm2835_power_probe(struct platform_device *pdev)
if (ret)
goto fail;
- of_genpd_add_provider_onecell(dev->parent->of_node, &power->pd_xlate);
+ ret = of_genpd_add_provider_onecell(dev->parent->of_node,
+ &power->pd_xlate);
+ if (ret) {
+ dev_err_probe(dev, ret, "failed to add genpd provider\n");
+ goto fail;
+ }
dev_info(dev, "Broadcom BCM2835 power domains driver");
return 0;
diff --git a/drivers/power/supply/bd99954-charger.h b/drivers/power/supply/bd99954-charger.h
index f58897925383..d053ac901274 100644
--- a/drivers/power/supply/bd99954-charger.h
+++ b/drivers/power/supply/bd99954-charger.h
@@ -279,17 +279,7 @@ enum bd9995x_fields {
F_VBUS_EXTID,
F_VBUS_IDRDET,
F_VBUS_INDO,
- F_VBUS_UCDSWEN,
- F_VBUS_RREF_EN,
- F_VBUS_DPPU_EN,
- F_VBUS_DPREF_EN,
- F_VBUS_DMREF_EN,
- F_VBUS_DPDET_EN,
- F_VBUS_DMDET_EN,
- F_VBUS_DPSINK_EN,
- F_VBUS_DMSINK_EN,
- F_VBUS_DP_BUFF_EN,
- F_VBUS_DM_BUFF_EN,
+
F_VBUS_EXTCLKENBL,
F_VBUS_PLSTESTEN,
F_VBUS_UCDSWEN_TSTENB,
@@ -626,17 +616,6 @@ static const struct reg_field bd9995x_reg_fields[] = {
[F_VBUS_EXTID] = REG_FIELD(VBUS_IDD_STATUS, 5, 5),
[F_VBUS_IDRDET] = REG_FIELD(VBUS_IDD_STATUS, 4, 4),
[F_VBUS_INDO] = REG_FIELD(VBUS_IDD_STATUS, 0, 3),
- [F_VBUS_UCDSWEN] = REG_FIELD(VCC_UCD_FCTRL_SET, 10, 10),
- [F_VBUS_RREF_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 9, 9),
- [F_VBUS_DPPU_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 8, 8),
- [F_VBUS_DPREF_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 7, 7),
- [F_VBUS_DMREF_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 6, 6),
- [F_VBUS_DPDET_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 5, 5),
- [F_VBUS_DMDET_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 4, 4),
- [F_VBUS_DPSINK_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 3, 3),
- [F_VBUS_DMSINK_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 2, 2),
- [F_VBUS_DP_BUFF_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 1, 1),
- [F_VBUS_DM_BUFF_EN] = REG_FIELD(VCC_UCD_FCTRL_SET, 0, 0),
[F_VBUS_EXTCLKENBL] = REG_FIELD(VBUS_UCD_FCTRL_EN, 15, 15),
[F_VBUS_PLSTESTEN] = REG_FIELD(VBUS_UCD_FCTRL_EN, 14, 14),
diff --git a/drivers/power/supply/bq27xxx_battery.c b/drivers/power/supply/bq27xxx_battery.c
index a5a3ab4f8a63..fb16354fcb97 100644
--- a/drivers/power/supply/bq27xxx_battery.c
+++ b/drivers/power/supply/bq27xxx_battery.c
@@ -324,7 +324,7 @@ static u8
[BQ27XXX_REG_FLAGS] = 0x0a,
[BQ27XXX_REG_TTE] = 0x16,
[BQ27XXX_REG_TTF] = INVALID_REG_ADDR,
- [BQ27XXX_REG_TTES] = 0x1c,
+ [BQ27XXX_REG_TTES] = 0x1a,
[BQ27XXX_REG_TTECP] = INVALID_REG_ADDR,
[BQ27XXX_REG_NAC] = 0x0c,
[BQ27XXX_REG_RC] = 0x10,
@@ -489,7 +489,7 @@ static u8
[BQ27XXX_REG_RC] = 0x10,
[BQ27XXX_REG_FCC] = 0x12,
[BQ27XXX_REG_CYCT] = 0x2a,
- [BQ27XXX_REG_AE] = 0x22,
+ [BQ27XXX_REG_AE] = INVALID_REG_ADDR,
[BQ27XXX_REG_SOC] = 0x2c,
[BQ27XXX_REG_DCAP] = 0x3c,
[BQ27XXX_REG_AP] = 0x22,
@@ -510,7 +510,7 @@ static u8
[BQ27XXX_REG_RC] = 0x10,
[BQ27XXX_REG_FCC] = 0x12,
[BQ27XXX_REG_CYCT] = 0x2a,
- [BQ27XXX_REG_AE] = 0x22,
+ [BQ27XXX_REG_AE] = INVALID_REG_ADDR,
[BQ27XXX_REG_SOC] = 0x2c,
[BQ27XXX_REG_DCAP] = 0x3c,
[BQ27XXX_REG_AP] = 0x22,
diff --git a/drivers/power/supply/isp1704_charger.c b/drivers/power/supply/isp1704_charger.c
index 860d8614c98f..71008c5d9d7f 100644
--- a/drivers/power/supply/isp1704_charger.c
+++ b/drivers/power/supply/isp1704_charger.c
@@ -482,6 +482,7 @@ static void isp1704_charger_remove(struct platform_device *pdev)
struct isp1704_charger *isp = platform_get_drvdata(pdev);
usb_unregister_notifier(isp->phy, &isp->nb);
+ cancel_work_sync(&isp->work);
power_supply_unregister(isp->psy);
isp1704_charger_set_power(isp, 0);
}
diff --git a/drivers/power/supply/sbs-battery.c b/drivers/power/supply/sbs-battery.c
index f80edceafc3c..4e2b8e252d5e 100644
--- a/drivers/power/supply/sbs-battery.c
+++ b/drivers/power/supply/sbs-battery.c
@@ -216,6 +216,7 @@ struct sbs_info {
u32 flags;
int technology;
char strings[NR_STRING_BUFFERS][I2C_SMBUS_BLOCK_MAX + 1];
+ char serial[5];
};
static char *sbs_get_string_buf(struct sbs_info *chip,
@@ -820,18 +821,18 @@ static int sbs_get_battery_capacity(struct i2c_client *client,
return 0;
}
-static char sbs_serial[5];
static int sbs_get_battery_serial_number(struct i2c_client *client,
union power_supply_propval *val)
{
+ struct sbs_info *chip = i2c_get_clientdata(client);
int ret;
ret = sbs_read_word_data(client, sbs_data[REG_SERIAL_NUMBER].addr);
if (ret < 0)
return ret;
- sprintf(sbs_serial, "%04x", ret);
- val->strval = sbs_serial;
+ sprintf(chip->serial, "%04x", ret);
+ val->strval = chip->serial;
return 0;
}
diff --git a/drivers/power/supply/sc2731_charger.c b/drivers/power/supply/sc2731_charger.c
index b3d8b1ca97da..0b3153eed8c4 100644
--- a/drivers/power/supply/sc2731_charger.c
+++ b/drivers/power/supply/sc2731_charger.c
@@ -466,6 +466,7 @@ static int sc2731_charger_probe(struct platform_device *pdev)
mutex_init(&info->lock);
info->dev = &pdev->dev;
INIT_WORK(&info->work, sc2731_charger_work);
+ platform_set_drvdata(pdev, info);
info->regmap = dev_get_regmap(pdev->dev.parent, NULL);
if (!info->regmap) {
@@ -516,6 +517,7 @@ static void sc2731_charger_remove(struct platform_device *pdev)
struct sc2731_charger_info *info = platform_get_drvdata(pdev);
usb_unregister_notifier(info->usb_phy, &info->usb_notify);
+ cancel_work_sync(&info->work);
}
static const struct of_device_id sc2731_charger_of_match[] = {
diff --git a/drivers/powercap/intel_rapl_tpmi.c b/drivers/powercap/intel_rapl_tpmi.c
index 1618138c5cac..1fae66a60929 100644
--- a/drivers/powercap/intel_rapl_tpmi.c
+++ b/drivers/powercap/intel_rapl_tpmi.c
@@ -313,7 +313,10 @@ static int intel_rapl_tpmi_probe(struct auxiliary_device *auxdev,
goto err;
}
- rapl_package_add_pmu(trp->rp);
+ ret = rapl_package_add_pmu(trp->rp);
+ if (ret)
+ dev_info(&auxdev->dev, "Failed to add RAPL PMU for Package%d, %d\n",
+ info->package_id, ret);
auxiliary_set_drvdata(auxdev, trp);
diff --git a/drivers/pps/clients/pps-gpio.c b/drivers/pps/clients/pps-gpio.c
index 1412f8af15f2..96e169a35ced 100644
--- a/drivers/pps/clients/pps-gpio.c
+++ b/drivers/pps/clients/pps-gpio.c
@@ -32,34 +32,37 @@ struct pps_gpio_device_data {
struct gpio_desc *echo_pin;
struct timer_list echo_timer; /* timer to reset echo active state */
bool assert_falling_edge;
- bool capture_clear;
unsigned int echo_active_ms; /* PPS echo active duration */
unsigned long echo_timeout; /* timer timeout value in jiffies */
+ struct pps_event_time ts; /* timestamp captured in hardirq */
};
/*
* Report the PPS event
*/
-static irqreturn_t pps_gpio_irq_handler(int irq, void *data)
+/*
+ * Primary hardirq handler -- runs in hardirq context even on PREEMPT_RT.
+ * Only captures the timestamp; all other work is deferred to the thread.
+ */
+static irqreturn_t pps_gpio_irq_hardirq(int irq, void *data)
{
- const struct pps_gpio_device_data *info;
- struct pps_event_time ts;
- int rising_edge;
+ struct pps_gpio_device_data *info = data;
- /* Get the time stamp first */
- pps_get_ts(&ts);
+ pps_get_ts(&info->ts);
- info = data;
+ return IRQ_WAKE_THREAD;
+}
+
+/*
+ * Threaded handler -- processes the PPS event using the timestamp
+ * captured in hardirq context above.
+ */
+static irqreturn_t pps_gpio_irq_thread(int irq, void *data)
+{
+ struct pps_gpio_device_data *info = data;
- rising_edge = gpiod_get_value(info->gpio_pin);
- if ((rising_edge && !info->assert_falling_edge) ||
- (!rising_edge && info->assert_falling_edge))
- pps_event(info->pps, &ts, PPS_CAPTUREASSERT, data);
- else if (info->capture_clear &&
- ((rising_edge && info->assert_falling_edge) ||
- (!rising_edge && !info->assert_falling_edge)))
- pps_event(info->pps, &ts, PPS_CAPTURECLEAR, data);
+ pps_event(info->pps, &info->ts, PPS_CAPTUREASSERT, data);
return IRQ_HANDLED;
}
@@ -75,11 +78,6 @@ static void pps_gpio_echo(struct pps_device *pps, int event, void *data)
if (pps->params.mode & PPS_ECHOASSERT)
gpiod_set_value(info->echo_pin, 1);
break;
-
- case PPS_CAPTURECLEAR:
- if (pps->params.mode & PPS_ECHOCLEAR)
- gpiod_set_value(info->echo_pin, 1);
- break;
}
/* fire the timer */
@@ -141,15 +139,8 @@ static int pps_gpio_setup(struct device *dev)
static unsigned long
get_irqf_trigger_flags(const struct pps_gpio_device_data *data)
{
- unsigned long flags = data->assert_falling_edge ?
- IRQF_TRIGGER_FALLING : IRQF_TRIGGER_RISING;
-
- if (data->capture_clear) {
- flags |= ((flags & IRQF_TRIGGER_RISING) ?
- IRQF_TRIGGER_FALLING : IRQF_TRIGGER_RISING);
- }
-
- return flags;
+ return data->assert_falling_edge ? IRQF_TRIGGER_FALLING :
+ IRQF_TRIGGER_RISING;
}
static int pps_gpio_probe(struct platform_device *pdev)
@@ -182,9 +173,6 @@ static int pps_gpio_probe(struct platform_device *pdev)
/* initialize PPS specific parts of the bookkeeping data structure. */
data->info.mode = PPS_CAPTUREASSERT | PPS_OFFSETASSERT |
PPS_ECHOASSERT | PPS_CANWAIT | PPS_TSFMT_TSPEC;
- if (data->capture_clear)
- data->info.mode |= PPS_CAPTURECLEAR | PPS_OFFSETCLEAR |
- PPS_ECHOCLEAR;
data->info.owner = THIS_MODULE;
snprintf(data->info.name, PPS_MAX_NAME_LEN - 1, "%s.%d",
pdev->name, pdev->id);
@@ -196,8 +184,6 @@ static int pps_gpio_probe(struct platform_device *pdev)
/* register PPS source */
pps_default_params = PPS_CAPTUREASSERT | PPS_OFFSETASSERT;
- if (data->capture_clear)
- pps_default_params |= PPS_CAPTURECLEAR | PPS_OFFSETCLEAR;
data->pps = pps_register_source(&data->info, pps_default_params);
if (IS_ERR(data->pps)) {
dev_err(dev, "failed to register IRQ %d as PPS source\n",
@@ -206,8 +192,10 @@ static int pps_gpio_probe(struct platform_device *pdev)
}
/* register IRQ interrupt handler */
- ret = request_irq(data->irq, pps_gpio_irq_handler,
- get_irqf_trigger_flags(data), data->info.name, data);
+ ret = request_threaded_irq(data->irq,
+ pps_gpio_irq_hardirq, pps_gpio_irq_thread,
+ get_irqf_trigger_flags(data) | IRQF_ONESHOT,
+ data->info.name, data);
if (ret) {
pps_unregister_source(data->pps);
dev_err(dev, "failed to acquire IRQ %d\n", data->irq);
diff --git a/drivers/pps/pps.c b/drivers/pps/pps.c
index c6b8b6478276..2e04636f4380 100644
--- a/drivers/pps/pps.c
+++ b/drivers/pps/pps.c
@@ -63,13 +63,19 @@ static int pps_cdev_pps_fetch(struct pps_device *pps, struct pps_fdata *fdata)
err = wait_event_interruptible(pps->queue,
ev != pps->last_ev);
else {
+ struct timespec64 ts;
unsigned long ticks;
dev_dbg(&pps->dev, "timeout %lld.%09d\n",
(long long) fdata->timeout.sec,
fdata->timeout.nsec);
- ticks = fdata->timeout.sec * HZ;
- ticks += fdata->timeout.nsec / (NSEC_PER_SEC / HZ);
+
+ if (fdata->timeout.sec < 0)
+ return -ETIMEDOUT;
+
+ ts.tv_sec = fdata->timeout.sec;
+ ts.tv_nsec = fdata->timeout.nsec;
+ ticks = timespec64_to_jiffies(&ts);
if (ticks != 0) {
err = wait_event_interruptible_timeout(
diff --git a/drivers/rapidio/rio-scan.c b/drivers/rapidio/rio-scan.c
index dcd6619a4b02..3cc25d053451 100644
--- a/drivers/rapidio/rio-scan.c
+++ b/drivers/rapidio/rio-scan.c
@@ -874,6 +874,7 @@ static struct rio_net *rio_scan_alloc_net(struct rio_mport *mport,
net->dev.release = rio_scan_release_dev;
if (rio_add_net(net)) {
put_device(&net->dev);
+ mport->net = NULL;
net = NULL;
}
}
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 264c8ab434f0..6c571edfaab9 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -27,6 +27,7 @@
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
#include <linux/module.h>
+#include <linux/workqueue.h>
#define CREATE_TRACE_POINTS
#include <trace/events/regulator.h>
@@ -6467,8 +6468,9 @@ static int __init regulator_init_complete(void)
* we'd only do this on systems that need it, and a kernel
* command line option might be useful.
*/
- schedule_delayed_work(®ulator_init_complete_work,
- msecs_to_jiffies(30000));
+ queue_delayed_work(system_freezable_wq,
+ ®ulator_init_complete_work,
+ msecs_to_jiffies(30000));
return 0;
}
diff --git a/drivers/regulator/qcom-rpmh-regulator.c b/drivers/regulator/qcom-rpmh-regulator.c
index 7870722b6ee2..62fd0da48407 100644
--- a/drivers/regulator/qcom-rpmh-regulator.c
+++ b/drivers/regulator/qcom-rpmh-regulator.c
@@ -77,6 +77,7 @@ enum rpmh_regulator_type {
* @hpm_min_load_uA: Minimum load current in microamps that requires
* high power mode (HPM) operation. This is used
* for LDO hardware type regulators only.
+ * @pmic_bypass_mode: The PMIC bypass mode value.
* @pmic_mode_map: Array indexed by regulator framework mode
* containing PMIC hardware modes. Must be large
* enough to index all framework modes supported
@@ -91,6 +92,7 @@ struct rpmh_vreg_hw_data {
int n_linear_ranges;
int n_voltages;
int hpm_min_load_uA;
+ int pmic_bypass_mode;
const int *pmic_mode_map;
unsigned int (*of_map_mode)(unsigned int mode);
};
@@ -277,7 +279,7 @@ static int rpmh_regulator_vrm_set_mode_bypass(struct rpmh_vreg *vreg,
return pmic_mode;
if (bypassed)
- cmd.data = PMIC4_BOB_MODE_PASS;
+ cmd.data = vreg->hw_data->pmic_bypass_mode;
else
cmd.data = pmic_mode;
@@ -692,6 +694,7 @@ static const struct rpmh_vreg_hw_data pmic4_bob = {
},
.n_linear_ranges = 1,
.n_voltages = 84,
+ .pmic_bypass_mode = PMIC4_BOB_MODE_PASS,
.pmic_mode_map = pmic_mode_map_pmic4_bob,
.of_map_mode = rpmh_regulator_pmic4_bob_of_map_mode,
};
@@ -900,6 +903,7 @@ static const struct rpmh_vreg_hw_data pmic5_bob = {
},
.n_linear_ranges = 1,
.n_voltages = 32,
+ .pmic_bypass_mode = PMIC5_BOB_MODE_PASS,
.pmic_mode_map = pmic_mode_map_pmic5_bob,
.of_map_mode = rpmh_regulator_pmic4_bob_of_map_mode,
};
diff --git a/drivers/regulator/tps6594-regulator.c b/drivers/regulator/tps6594-regulator.c
index ac53792e3fed..9ede06398648 100644
--- a/drivers/regulator/tps6594-regulator.c
+++ b/drivers/regulator/tps6594-regulator.c
@@ -21,10 +21,6 @@
#define BUCK_NB 5
#define LDO_NB 4
#define MULTI_PHASE_NB 4
-/* TPS6593 and LP8764 supports OV, UV, SC, ILIM */
-#define REGS_INT_NB 4
-/* TPS65224 supports OV or UV */
-#define TPS65224_REGS_INT_NB 1
enum tps6594_regulator_id {
/* DCDC's */
@@ -56,7 +52,7 @@ struct tps6594_regulator_irq_type {
unsigned long event;
};
-static struct tps6594_regulator_irq_type tps6594_ext_regulator_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_ext_regulator_irq_types[] = {
{ TPS6594_IRQ_NAME_VCCA_OV, "VCCA", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_VCCA_UV, "VCCA", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_VMON1_OV, "VMON1", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
@@ -69,7 +65,7 @@ static struct tps6594_regulator_irq_type tps6594_ext_regulator_irq_types[] = {
REGULATOR_EVENT_OVER_VOLTAGE_WARN },
};
-static struct tps6594_regulator_irq_type tps65224_ext_regulator_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_ext_regulator_irq_types[] = {
{ TPS65224_IRQ_NAME_VCCA_UVOV, "VCCA", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
{ TPS65224_IRQ_NAME_VMON1_UVOV, "VMON1", "voltage out of range",
@@ -80,13 +76,13 @@ static struct tps6594_regulator_irq_type tps65224_ext_regulator_irq_types[] = {
struct tps6594_regulator_irq_data {
struct device *dev;
- struct tps6594_regulator_irq_type *type;
+ const struct tps6594_regulator_irq_type *type;
struct regulator_dev *rdev;
};
struct tps6594_ext_regulator_irq_data {
struct device *dev;
- struct tps6594_regulator_irq_type *type;
+ const struct tps6594_regulator_irq_type *type;
};
#define TPS6594_REGULATOR(_name, _of, _id, _type, _ops, _n, _vr, _vm, _er, \
@@ -192,7 +188,7 @@ static const struct regulator_ops tps6594_ldos_4_ops = {
.map_voltage = regulator_map_voltage_linear_range,
};
-static const struct regulator_desc buck_regs[] = {
+static const struct regulator_desc tps6594_buck_regs[] = {
TPS6594_REGULATOR("BUCK1", "buck1", TPS6594_BUCK_1,
REGULATOR_VOLTAGE, tps6594_bucks_ops, TPS6594_MASK_BUCKS_VSET,
TPS6594_REG_BUCKX_VOUT_1(0),
@@ -262,7 +258,7 @@ static const struct regulator_desc tps65224_buck_regs[] = {
4, 0, 0, NULL, 0, 0),
};
-static struct tps6594_regulator_irq_type tps6594_buck1_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_buck1_irq_types[] = {
{ TPS6594_IRQ_NAME_BUCK1_OV, "BUCK1", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_BUCK1_UV, "BUCK1", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_BUCK1_SC, "BUCK1", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -270,7 +266,7 @@ static struct tps6594_regulator_irq_type tps6594_buck1_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_buck2_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_buck2_irq_types[] = {
{ TPS6594_IRQ_NAME_BUCK2_OV, "BUCK2", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_BUCK2_UV, "BUCK2", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_BUCK2_SC, "BUCK2", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -278,7 +274,7 @@ static struct tps6594_regulator_irq_type tps6594_buck2_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_buck3_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_buck3_irq_types[] = {
{ TPS6594_IRQ_NAME_BUCK3_OV, "BUCK3", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_BUCK3_UV, "BUCK3", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_BUCK3_SC, "BUCK3", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -286,7 +282,7 @@ static struct tps6594_regulator_irq_type tps6594_buck3_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_buck4_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_buck4_irq_types[] = {
{ TPS6594_IRQ_NAME_BUCK4_OV, "BUCK4", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_BUCK4_UV, "BUCK4", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_BUCK4_SC, "BUCK4", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -294,7 +290,7 @@ static struct tps6594_regulator_irq_type tps6594_buck4_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_buck5_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_buck5_irq_types[] = {
{ TPS6594_IRQ_NAME_BUCK5_OV, "BUCK5", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_BUCK5_UV, "BUCK5", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_BUCK5_SC, "BUCK5", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -302,7 +298,7 @@ static struct tps6594_regulator_irq_type tps6594_buck5_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_ldo1_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_ldo1_irq_types[] = {
{ TPS6594_IRQ_NAME_LDO1_OV, "LDO1", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_LDO1_UV, "LDO1", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_LDO1_SC, "LDO1", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -310,7 +306,7 @@ static struct tps6594_regulator_irq_type tps6594_ldo1_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_ldo2_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_ldo2_irq_types[] = {
{ TPS6594_IRQ_NAME_LDO2_OV, "LDO2", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_LDO2_UV, "LDO2", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_LDO2_SC, "LDO2", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -318,7 +314,7 @@ static struct tps6594_regulator_irq_type tps6594_ldo2_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_ldo3_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_ldo3_irq_types[] = {
{ TPS6594_IRQ_NAME_LDO3_OV, "LDO3", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_LDO3_UV, "LDO3", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_LDO3_SC, "LDO3", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -326,7 +322,7 @@ static struct tps6594_regulator_irq_type tps6594_ldo3_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps6594_ldo4_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps6594_ldo4_irq_types[] = {
{ TPS6594_IRQ_NAME_LDO4_OV, "LDO4", "overvoltage", REGULATOR_EVENT_OVER_VOLTAGE_WARN },
{ TPS6594_IRQ_NAME_LDO4_UV, "LDO4", "undervoltage", REGULATOR_EVENT_UNDER_VOLTAGE },
{ TPS6594_IRQ_NAME_LDO4_SC, "LDO4", "short circuit", REGULATOR_EVENT_REGULATION_OUT },
@@ -334,42 +330,42 @@ static struct tps6594_regulator_irq_type tps6594_ldo4_irq_types[] = {
REGULATOR_EVENT_OVER_CURRENT },
};
-static struct tps6594_regulator_irq_type tps65224_buck1_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_buck1_irq_types[] = {
{ TPS65224_IRQ_NAME_BUCK1_UVOV, "BUCK1", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_buck2_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_buck2_irq_types[] = {
{ TPS65224_IRQ_NAME_BUCK2_UVOV, "BUCK2", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_buck3_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_buck3_irq_types[] = {
{ TPS65224_IRQ_NAME_BUCK3_UVOV, "BUCK3", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_buck4_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_buck4_irq_types[] = {
{ TPS65224_IRQ_NAME_BUCK4_UVOV, "BUCK4", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_ldo1_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_ldo1_irq_types[] = {
{ TPS65224_IRQ_NAME_LDO1_UVOV, "LDO1", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_ldo2_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_ldo2_irq_types[] = {
{ TPS65224_IRQ_NAME_LDO2_UVOV, "LDO2", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type tps65224_ldo3_irq_types[] = {
+static const struct tps6594_regulator_irq_type tps65224_ldo3_irq_types[] = {
{ TPS65224_IRQ_NAME_LDO3_UVOV, "LDO3", "voltage out of range",
REGULATOR_EVENT_REGULATION_OUT },
};
-static struct tps6594_regulator_irq_type *tps6594_bucks_irq_types[] = {
+static const struct tps6594_regulator_irq_type *tps6594_bucks_irq_types[] = {
tps6594_buck1_irq_types,
tps6594_buck2_irq_types,
tps6594_buck3_irq_types,
@@ -377,21 +373,21 @@ static struct tps6594_regulator_irq_type *tps6594_bucks_irq_types[] = {
tps6594_buck5_irq_types,
};
-static struct tps6594_regulator_irq_type *tps6594_ldos_irq_types[] = {
+static const struct tps6594_regulator_irq_type *tps6594_ldos_irq_types[] = {
tps6594_ldo1_irq_types,
tps6594_ldo2_irq_types,
tps6594_ldo3_irq_types,
tps6594_ldo4_irq_types,
};
-static struct tps6594_regulator_irq_type *tps65224_bucks_irq_types[] = {
+static const struct tps6594_regulator_irq_type *tps65224_bucks_irq_types[] = {
tps65224_buck1_irq_types,
tps65224_buck2_irq_types,
tps65224_buck3_irq_types,
tps65224_buck4_irq_types,
};
-static struct tps6594_regulator_irq_type *tps65224_ldos_irq_types[] = {
+static const struct tps6594_regulator_irq_type *tps65224_ldos_irq_types[] = {
tps65224_ldo1_irq_types,
tps65224_ldo2_irq_types,
tps65224_ldo3_irq_types,
@@ -516,11 +512,11 @@ static irqreturn_t tps6594_regulator_irq_handler(int irq, void *data)
static int tps6594_request_reg_irqs(struct platform_device *pdev,
struct regulator_dev *rdev,
struct tps6594_regulator_irq_data *irq_data,
- struct tps6594_regulator_irq_type *regs_irq_types,
+ const struct tps6594_regulator_irq_type *regs_irq_types,
size_t interrupt_cnt,
int *irq_idx)
{
- struct tps6594_regulator_irq_type *irq_type;
+ const struct tps6594_regulator_irq_type *irq_type;
struct tps6594 *tps = dev_get_drvdata(pdev->dev.parent);
size_t j;
int irq;
@@ -549,6 +545,63 @@ static int tps6594_request_reg_irqs(struct platform_device *pdev,
return 0;
}
+struct tps6594_regulator_desc {
+ const struct regulator_desc *multi_phase_regs;
+ unsigned int num_multi_phase_regs;
+
+ const struct regulator_desc *buck_regs;
+ int num_buck_regs;
+
+ const struct regulator_desc *ldo_regs;
+ int num_ldo_regs;
+
+ const struct tps6594_regulator_irq_type **bucks_irq_types;
+ const struct tps6594_regulator_irq_type **ldos_irq_types;
+ int num_irq_types;
+
+ const struct tps6594_regulator_irq_type *ext_irq_types;
+ int num_ext_irqs;
+};
+
+static const struct tps6594_regulator_desc tps65224_reg_desc = {
+ .multi_phase_regs = tps65224_multi_regs,
+ .num_multi_phase_regs = ARRAY_SIZE(tps65224_multi_regs),
+ .buck_regs = tps65224_buck_regs,
+ .num_buck_regs = ARRAY_SIZE(tps65224_buck_regs),
+ .ldo_regs = tps65224_ldo_regs,
+ .num_ldo_regs = ARRAY_SIZE(tps65224_ldo_regs),
+ .bucks_irq_types = tps65224_bucks_irq_types,
+ .ldos_irq_types = tps65224_ldos_irq_types,
+ .num_irq_types = 1, /* OV or UV */
+ .ext_irq_types = tps65224_ext_regulator_irq_types,
+ .num_ext_irqs = ARRAY_SIZE(tps65224_ext_regulator_irq_types),
+};
+
+static const struct tps6594_regulator_desc tps6594_reg_desc = {
+ .multi_phase_regs = tps6594_multi_regs,
+ .num_multi_phase_regs = ARRAY_SIZE(tps6594_multi_regs),
+ .buck_regs = tps6594_buck_regs,
+ .num_buck_regs = ARRAY_SIZE(tps6594_buck_regs),
+ .ldo_regs = tps6594_ldo_regs,
+ .num_ldo_regs = ARRAY_SIZE(tps6594_ldo_regs),
+ .bucks_irq_types = tps6594_bucks_irq_types,
+ .ldos_irq_types = tps6594_ldos_irq_types,
+ .num_irq_types = 4, /* OV, UV, SC and ILIM */
+ .ext_irq_types = tps6594_ext_regulator_irq_types,
+ .num_ext_irqs = 2, /* only VCCA OV and UV */
+};
+
+static const struct tps6594_regulator_desc lp8764_reg_desc = {
+ .multi_phase_regs = tps6594_multi_regs,
+ .num_multi_phase_regs = ARRAY_SIZE(tps6594_multi_regs),
+ .buck_regs = tps6594_buck_regs,
+ .num_buck_regs = ARRAY_SIZE(tps6594_buck_regs),
+ .bucks_irq_types = tps6594_bucks_irq_types,
+ .num_irq_types = 4, /* OV, UV, SC and ILIM */
+ .ext_irq_types = tps6594_ext_regulator_irq_types,
+ .num_ext_irqs = ARRAY_SIZE(tps6594_ext_regulator_irq_types),
+};
+
static int tps6594_regulator_probe(struct platform_device *pdev)
{
struct tps6594 *tps = dev_get_drvdata(pdev->dev.parent);
@@ -558,42 +611,33 @@ static int tps6594_regulator_probe(struct platform_device *pdev)
struct regulator_config config = {};
struct tps6594_regulator_irq_data *irq_data;
struct tps6594_ext_regulator_irq_data *irq_ext_reg_data;
- struct tps6594_regulator_irq_type *irq_type;
- struct tps6594_regulator_irq_type *irq_types;
+ const struct tps6594_regulator_irq_type *irq_type;
bool buck_configured[BUCK_NB] = { false };
bool buck_multi[MULTI_PHASE_NB] = { false };
+ const struct tps6594_regulator_desc *desc;
+ const struct regulator_desc *multi_regs;
static const char *npname;
int error, i, irq, multi;
int irq_idx = 0;
int buck_idx = 0;
- int nr_ldo;
- int nr_buck;
- int nr_types;
- unsigned int irq_count;
- unsigned int multi_phase_cnt;
size_t reg_irq_nb;
- struct tps6594_regulator_irq_type **bucks_irq_types;
- const struct regulator_desc *multi_regs;
- struct tps6594_regulator_irq_type **ldos_irq_types;
- const struct regulator_desc *ldo_regs;
- size_t interrupt_count;
-
- if (tps->chip_id == TPS65224) {
- bucks_irq_types = tps65224_bucks_irq_types;
- interrupt_count = ARRAY_SIZE(tps65224_buck1_irq_types);
- multi_regs = tps65224_multi_regs;
- ldos_irq_types = tps65224_ldos_irq_types;
- ldo_regs = tps65224_ldo_regs;
- multi_phase_cnt = ARRAY_SIZE(tps65224_multi_regs);
- } else {
- bucks_irq_types = tps6594_bucks_irq_types;
- interrupt_count = ARRAY_SIZE(tps6594_buck1_irq_types);
- multi_regs = tps6594_multi_regs;
- ldos_irq_types = tps6594_ldos_irq_types;
- ldo_regs = tps6594_ldo_regs;
- multi_phase_cnt = ARRAY_SIZE(tps6594_multi_regs);
- }
+
+ switch (tps->chip_id) {
+ case TPS65224:
+ desc = &tps65224_reg_desc;
+ break;
+ case TPS6594:
+ case TPS6593:
+ desc = &tps6594_reg_desc;
+ break;
+ case LP8764:
+ desc = &lp8764_reg_desc;
+ break;
+ default:
+ dev_err(tps->dev, "unknown chip_id %lu\n", tps->chip_id);
+ return -EINVAL;
+ };
enum {
MULTI_BUCK12,
@@ -614,13 +658,21 @@ static int tps6594_regulator_probe(struct platform_device *pdev)
* In case of Multiphase configuration, value should be defined for
* buck_configured to avoid creating bucks for every buck in multiphase
*/
- for (multi = 0; multi < multi_phase_cnt; multi++) {
- np = of_find_node_by_name(tps->dev->of_node, multi_regs[multi].supply_name);
- npname = of_node_full_name(np);
- np_pmic_parent = of_get_parent(of_get_parent(np));
- if (of_node_cmp(of_node_full_name(np_pmic_parent), tps->dev->of_node->full_name))
+ for (multi = 0; multi < desc->num_multi_phase_regs; multi++) {
+ struct device_node *np_parent;
+
+ multi_regs = &desc->multi_phase_regs[multi];
+ np = of_find_node_by_name(tps->dev->of_node, multi_regs->supply_name);
+ if (!np)
continue;
- if (strcmp(npname, multi_regs[multi].supply_name) == 0) {
+
+ npname = of_node_full_name(np);
+ np_parent = of_get_parent(np);
+ np_pmic_parent = of_get_parent(np_parent);
+
+ if (np_pmic_parent &&
+ !of_node_cmp(of_node_full_name(np_pmic_parent), tps->dev->of_node->full_name) &&
+ strcmp(npname, multi_regs->supply_name) == 0) {
switch (multi) {
case MULTI_BUCK12:
buck_multi[0] = true;
@@ -651,30 +703,25 @@ static int tps6594_regulator_probe(struct platform_device *pdev)
break;
}
}
- }
- if (tps->chip_id == TPS65224) {
- nr_buck = ARRAY_SIZE(tps65224_buck_regs);
- nr_ldo = ARRAY_SIZE(tps65224_ldo_regs);
- nr_types = TPS65224_REGS_INT_NB;
- } else {
- nr_buck = ARRAY_SIZE(buck_regs);
- nr_ldo = (tps->chip_id == LP8764) ? 0 : ARRAY_SIZE(tps6594_ldo_regs);
- nr_types = REGS_INT_NB;
+ of_node_put(np_pmic_parent);
+ of_node_put(np_parent);
+ of_node_put(np);
}
- reg_irq_nb = nr_types * (nr_buck + nr_ldo);
+ reg_irq_nb = desc->num_irq_types * (desc->num_buck_regs + desc->num_ldo_regs);
irq_data = devm_kmalloc_array(tps->dev, reg_irq_nb,
sizeof(struct tps6594_regulator_irq_data), GFP_KERNEL);
if (!irq_data)
return -ENOMEM;
- for (i = 0; i < multi_phase_cnt; i++) {
+ for (i = 0; i < desc->num_multi_phase_regs; i++) {
if (!buck_multi[i])
continue;
- rdev = devm_regulator_register(&pdev->dev, &multi_regs[i], &config);
+ rdev = devm_regulator_register(&pdev->dev, &desc->multi_phase_regs[i],
+ &config);
if (IS_ERR(rdev))
return dev_err_probe(tps->dev, PTR_ERR(rdev),
"failed to register %s regulator\n",
@@ -685,91 +732,74 @@ static int tps6594_regulator_probe(struct platform_device *pdev)
buck_idx = 2;
error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- bucks_irq_types[buck_idx],
- interrupt_count, &irq_idx);
+ desc->bucks_irq_types[buck_idx],
+ desc->num_irq_types, &irq_idx);
if (error)
return error;
error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- bucks_irq_types[buck_idx + 1],
- interrupt_count, &irq_idx);
+ desc->bucks_irq_types[buck_idx + 1],
+ desc->num_irq_types, &irq_idx);
if (error)
return error;
if (i == MULTI_BUCK123 || i == MULTI_BUCK1234) {
error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- tps6594_bucks_irq_types[buck_idx + 2],
- interrupt_count,
+ desc->bucks_irq_types[buck_idx + 2],
+ desc->num_irq_types,
&irq_idx);
if (error)
return error;
}
if (i == MULTI_BUCK1234) {
error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- tps6594_bucks_irq_types[buck_idx + 3],
- interrupt_count,
+ desc->bucks_irq_types[buck_idx + 3],
+ desc->num_irq_types,
&irq_idx);
if (error)
return error;
}
}
- for (i = 0; i < nr_buck; i++) {
+ for (i = 0; i < desc->num_buck_regs; i++) {
if (buck_configured[i])
continue;
- const struct regulator_desc *buck_cfg = (tps->chip_id == TPS65224) ?
- tps65224_buck_regs : buck_regs;
-
- rdev = devm_regulator_register(&pdev->dev, &buck_cfg[i], &config);
+ rdev = devm_regulator_register(&pdev->dev, &desc->buck_regs[i], &config);
if (IS_ERR(rdev))
return dev_err_probe(tps->dev, PTR_ERR(rdev),
"failed to register %s regulator\n", pdev->name);
error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- bucks_irq_types[i], interrupt_count, &irq_idx);
+ desc->bucks_irq_types[i],
+ desc->num_irq_types, &irq_idx);
if (error)
return error;
}
- /* LP8764 doesn't have LDO */
- if (tps->chip_id != LP8764) {
- for (i = 0; i < nr_ldo; i++) {
- rdev = devm_regulator_register(&pdev->dev, &ldo_regs[i], &config);
- if (IS_ERR(rdev))
- return dev_err_probe(tps->dev, PTR_ERR(rdev),
- "failed to register %s regulator\n",
- pdev->name);
-
- error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
- ldos_irq_types[i], interrupt_count,
- &irq_idx);
- if (error)
- return error;
- }
- }
+ for (i = 0; i < desc->num_ldo_regs; i++) {
+ rdev = devm_regulator_register(&pdev->dev, &desc->ldo_regs[i], &config);
+ if (IS_ERR(rdev))
+ return dev_err_probe(tps->dev, PTR_ERR(rdev),
+ "failed to register %s regulator\n",
+ pdev->name);
- if (tps->chip_id == TPS65224) {
- irq_types = tps65224_ext_regulator_irq_types;
- irq_count = ARRAY_SIZE(tps65224_ext_regulator_irq_types);
- } else {
- irq_types = tps6594_ext_regulator_irq_types;
- if (tps->chip_id == LP8764)
- irq_count = ARRAY_SIZE(tps6594_ext_regulator_irq_types);
- else
- /* TPS6593 supports only VCCA OV and UV */
- irq_count = 2;
+ error = tps6594_request_reg_irqs(pdev, rdev, irq_data,
+ desc->ldos_irq_types[i],
+ desc->num_irq_types, &irq_idx);
+ if (error)
+ return error;
}
irq_ext_reg_data = devm_kmalloc_array(tps->dev,
- irq_count,
+ desc->num_ext_irqs,
sizeof(struct tps6594_ext_regulator_irq_data),
GFP_KERNEL);
if (!irq_ext_reg_data)
return -ENOMEM;
- for (i = 0; i < irq_count; ++i) {
- irq_type = &irq_types[i];
+ for (i = 0; i < desc->num_ext_irqs; ++i) {
+ irq_type = &desc->ext_irq_types[i];
irq = platform_get_irq_byname(pdev, irq_type->irq_name);
if (irq < 0)
return -EINVAL;
@@ -787,6 +817,7 @@ static int tps6594_regulator_probe(struct platform_device *pdev)
"failed to request %s IRQ %d\n",
irq_type->irq_name, irq);
}
+
return 0;
}
diff --git a/drivers/remoteproc/qcom_common.c b/drivers/remoteproc/qcom_common.c
index 5bbbe36d0964..506678d1633e 100644
--- a/drivers/remoteproc/qcom_common.c
+++ b/drivers/remoteproc/qcom_common.c
@@ -242,8 +242,10 @@ void qcom_add_glink_subdev(struct rproc *rproc, struct qcom_rproc_glink *glink,
return;
glink->ssr_name = kstrdup_const(ssr_name, GFP_KERNEL);
- if (!glink->ssr_name)
+ if (!glink->ssr_name) {
+ of_node_put(glink->node);
return;
+ }
glink->dev = dev;
glink->subdev.start = glink_subdev_start;
diff --git a/drivers/remoteproc/qcom_q6v5_adsp.c b/drivers/remoteproc/qcom_q6v5_adsp.c
index 223f6ca0745d..80ac739621be 100644
--- a/drivers/remoteproc/qcom_q6v5_adsp.c
+++ b/drivers/remoteproc/qcom_q6v5_adsp.c
@@ -355,6 +355,7 @@ static int adsp_map_carveout(struct rproc *rproc)
return ret;
sid = args.args[0] & SID_MASK_DEFAULT;
+ of_node_put(args.np);
/* Add SID configuration for ADSP Firmware to SMMU */
iova = adsp->mem_phys | (sid << 32);
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index b7011eb384a5..0c11a0b52908 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1016,60 +1016,55 @@ static rproc_handle_resource_t rproc_loading_handlers[RSC_LAST] = {
[RSC_VDEV] = rproc_handle_vdev,
};
-/* handle firmware resource entries before booting the remote processor */
-static int rproc_handle_resources(struct rproc *rproc,
- rproc_handle_resource_t handlers[RSC_LAST])
+struct rproc_rsc_cb_data {
+ struct rproc *rproc;
+ rproc_handle_resource_t *handlers;
+};
+
+static int rproc_handle_rsc_entry(u32 type, void *rsc, int offset,
+ int avail, void *data)
{
+ struct rproc_rsc_cb_data *d = data;
+ struct rproc *rproc = d->rproc;
struct device *dev = &rproc->dev;
rproc_handle_resource_t handler;
- int ret = 0, i;
-
- if (!rproc->table_ptr)
- return 0;
-
- for (i = 0; i < rproc->table_ptr->num; i++) {
- int offset = rproc->table_ptr->offset[i];
- struct fw_rsc_hdr *hdr = (void *)rproc->table_ptr + offset;
- int avail = rproc->table_sz - offset - sizeof(*hdr);
- void *rsc = (void *)hdr + sizeof(*hdr);
+ int ret;
- /* make sure table isn't truncated */
- if (avail < 0) {
- dev_err(dev, "rsc table is truncated\n");
- return -EINVAL;
- }
+ dev_dbg(dev, "rsc: type %d\n", type);
- dev_dbg(dev, "rsc: type %d\n", hdr->type);
+ if (type >= RSC_VENDOR_START && type <= RSC_VENDOR_END) {
+ ret = rproc_handle_rsc(rproc, type, rsc, offset, avail);
+ if (ret == RSC_HANDLED)
+ return 0;
+ if (ret < 0)
+ return ret;
+ dev_warn(dev, "unsupported vendor resource %d\n", type);
+ return 0;
+ }
- if (hdr->type >= RSC_VENDOR_START &&
- hdr->type <= RSC_VENDOR_END) {
- ret = rproc_handle_rsc(rproc, hdr->type, rsc,
- offset + sizeof(*hdr), avail);
- if (ret == RSC_HANDLED)
- continue;
- else if (ret < 0)
- break;
+ if (type >= RSC_LAST) {
+ dev_warn(dev, "unsupported resource %d\n", type);
+ return 0;
+ }
- dev_warn(dev, "unsupported vendor resource %d\n",
- hdr->type);
- continue;
- }
+ handler = d->handlers[type];
+ if (!handler)
+ return 0;
- if (hdr->type >= RSC_LAST) {
- dev_warn(dev, "unsupported resource %d\n", hdr->type);
- continue;
- }
+ return handler(rproc, rsc, offset, avail);
+}
- handler = handlers[hdr->type];
- if (!handler)
- continue;
+/* handle firmware resource entries before booting the remote processor */
+static int rproc_handle_resources(struct rproc *rproc,
+ rproc_handle_resource_t handlers[RSC_LAST])
+{
+ struct rproc_rsc_cb_data d = { .rproc = rproc, .handlers = handlers };
- ret = handler(rproc, rsc, offset + sizeof(*hdr), avail);
- if (ret)
- break;
- }
+ if (!rproc->table_ptr)
+ return 0;
- return ret;
+ return rsc_table_for_each_entry(rproc->table_ptr, rproc->table_sz,
+ &rproc->dev, rproc_handle_rsc_entry, &d);
}
static int rproc_prepare_subdevices(struct rproc *rproc)
@@ -1678,18 +1673,21 @@ static void rproc_auto_boot_callback(const struct firmware *fw, void *context)
release_firmware(fw);
}
+static void rproc_attach_work(struct work_struct *work)
+{
+ struct rproc *rproc = container_of(work, struct rproc, attach_work);
+
+ rproc_boot(rproc);
+}
+
static int rproc_trigger_auto_boot(struct rproc *rproc)
{
int ret;
- /*
- * Since the remote processor is in a detached state, it has already
- * been booted by another entity. As such there is no point in waiting
- * for a firmware image to be loaded, we can simply initiate the process
- * of attaching to it immediately.
- */
- if (rproc->state == RPROC_DETACHED)
- return rproc_boot(rproc);
+ if (rproc->state == RPROC_DETACHED) {
+ schedule_work(&rproc->attach_work);
+ return 0;
+ }
/*
* We're initiating an asynchronous firmware loading, so we can
@@ -1838,6 +1836,11 @@ int rproc_trigger_recovery(struct rproc *rproc)
if (ret)
return ret;
+ if (READ_ONCE(rproc->deleting)) {
+ ret = -ENODEV;
+ goto unlock_mutex;
+ }
+
/* State could have changed before we got the mutex */
if (rproc->state != RPROC_CRASHED)
goto unlock_mutex;
@@ -1870,6 +1873,11 @@ static void rproc_crash_handler_work(struct work_struct *work)
mutex_lock(&rproc->lock);
+ if (READ_ONCE(rproc->deleting)) {
+ mutex_unlock(&rproc->lock);
+ goto out;
+ }
+
if (rproc->state == RPROC_CRASHED) {
/* handle only the first crash detected */
mutex_unlock(&rproc->lock);
@@ -1925,9 +1933,9 @@ int rproc_boot(struct rproc *rproc)
return ret;
}
- if (rproc->state == RPROC_DELETED) {
+ if (READ_ONCE(rproc->deleting)) {
ret = -ENODEV;
- dev_err(dev, "can't boot deleted rproc %s\n", rproc->name);
+ dev_err(dev, "can't boot deleting rproc %s\n", rproc->name);
goto unlock_mutex;
}
@@ -1989,7 +1997,8 @@ EXPORT_SYMBOL(rproc_boot);
int rproc_shutdown(struct rproc *rproc)
{
struct device *dev = &rproc->dev;
- int ret = 0;
+ bool crashed;
+ int ret;
ret = mutex_lock_interruptible(&rproc->lock);
if (ret) {
@@ -1998,16 +2007,18 @@ int rproc_shutdown(struct rproc *rproc)
}
if (rproc->state != RPROC_RUNNING &&
- rproc->state != RPROC_ATTACHED) {
+ rproc->state != RPROC_ATTACHED &&
+ rproc->state != RPROC_CRASHED) {
ret = -EINVAL;
goto out;
}
+ crashed = rproc->state == RPROC_CRASHED;
/* if the remote proc is still needed, bail out */
if (!atomic_dec_and_test(&rproc->power))
goto out;
- ret = rproc_stop(rproc, false);
+ ret = rproc_stop(rproc, crashed);
if (ret) {
atomic_inc(&rproc->power);
goto out;
@@ -2517,7 +2528,9 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
INIT_LIST_HEAD(&rproc->subdevs);
INIT_LIST_HEAD(&rproc->dump_segments);
+ INIT_WORK(&rproc->attach_work, rproc_attach_work);
INIT_WORK(&rproc->crash_handler, rproc_crash_handler_work);
+ spin_lock_init(&rproc->crash_handler_lock);
rproc->state = RPROC_OFFLINE;
@@ -2581,16 +2594,21 @@ EXPORT_SYMBOL(rproc_put);
*/
int rproc_del(struct rproc *rproc)
{
+ unsigned long flags;
+
if (!rproc)
return -EINVAL;
+ spin_lock_irqsave(&rproc->crash_handler_lock, flags);
+ WRITE_ONCE(rproc->deleting, true);
+ spin_unlock_irqrestore(&rproc->crash_handler_lock, flags);
+
+ if (cancel_work_sync(&rproc->crash_handler))
+ pm_relax(rproc->dev.parent);
+
/* TODO: make sure this works with rproc->power > 1 */
rproc_shutdown(rproc);
- mutex_lock(&rproc->lock);
- rproc->state = RPROC_DELETED;
- mutex_unlock(&rproc->lock);
-
rproc_delete_debug_dir(rproc);
/* the rproc is downref'ed as soon as it's removed from the klist */
@@ -2702,18 +2720,26 @@ EXPORT_SYMBOL(rproc_get_by_child);
*/
void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type)
{
+ unsigned long flags;
+
if (!rproc) {
pr_err("NULL rproc pointer\n");
return;
}
+ spin_lock_irqsave(&rproc->crash_handler_lock, flags);
+ if (READ_ONCE(rproc->deleting)) {
+ spin_unlock_irqrestore(&rproc->crash_handler_lock, flags);
+ return;
+ }
+
/* Prevent suspend while the remoteproc is being recovered */
pm_stay_awake(rproc->dev.parent);
+ queue_work(rproc_recovery_wq, &rproc->crash_handler);
+ spin_unlock_irqrestore(&rproc->crash_handler_lock, flags);
dev_err(&rproc->dev, "crash detected in %s: type %s\n",
rproc->name, rproc_crash_to_string(type));
-
- queue_work(rproc_recovery_wq, &rproc->crash_handler);
}
EXPORT_SYMBOL(rproc_report_crash);
diff --git a/drivers/remoteproc/remoteproc_sysfs.c b/drivers/remoteproc/remoteproc_sysfs.c
index 138e752c5e4e..925b0cdbe577 100644
--- a/drivers/remoteproc/remoteproc_sysfs.c
+++ b/drivers/remoteproc/remoteproc_sysfs.c
@@ -168,7 +168,6 @@ static const char * const rproc_state_string[] = {
[RPROC_SUSPENDED] = "suspended",
[RPROC_RUNNING] = "running",
[RPROC_CRASHED] = "crashed",
- [RPROC_DELETED] = "deleted",
[RPROC_ATTACHED] = "attached",
[RPROC_DETACHED] = "detached",
[RPROC_LAST] = "invalid",
diff --git a/drivers/rpmsg/qcom_glink_native.c b/drivers/rpmsg/qcom_glink_native.c
index c2d39a3bcea7..c21a5e4da0f6 100644
--- a/drivers/rpmsg/qcom_glink_native.c
+++ b/drivers/rpmsg/qcom_glink_native.c
@@ -1395,26 +1395,28 @@ static int qcom_glink_announce_create(struct rpmsg_device *rpdev)
return 0;
}
-static void qcom_glink_destroy_ept(struct rpmsg_endpoint *ept)
+static void qcom_glink_remove_rpmsg_device(struct qcom_glink *glink, struct glink_channel *channel)
{
- struct glink_channel *channel = to_glink_channel(ept);
- struct qcom_glink *glink = channel->glink;
struct rpmsg_channel_info chinfo;
- unsigned long flags;
-
- spin_lock_irqsave(&channel->recv_lock, flags);
- channel->ept.cb = NULL;
- spin_unlock_irqrestore(&channel->recv_lock, flags);
- /* Decouple the potential rpdev from the channel */
if (channel->rpdev) {
strscpy_pad(chinfo.name, channel->name, sizeof(chinfo.name));
chinfo.src = RPMSG_ADDR_ANY;
chinfo.dst = RPMSG_ADDR_ANY;
-
rpmsg_unregister_device(glink->dev, &chinfo);
}
channel->rpdev = NULL;
+}
+
+static void qcom_glink_destroy_ept(struct rpmsg_endpoint *ept)
+{
+ struct glink_channel *channel = to_glink_channel(ept);
+ struct qcom_glink *glink = channel->glink;
+ unsigned long flags;
+
+ spin_lock_irqsave(&channel->recv_lock, flags);
+ channel->ept.cb = NULL;
+ spin_unlock_irqrestore(&channel->recv_lock, flags);
qcom_glink_send_close_req(glink, channel);
}
@@ -1704,7 +1706,6 @@ static int qcom_glink_rx_open(struct qcom_glink *glink, unsigned int rcid,
static void qcom_glink_rx_close(struct qcom_glink *glink, unsigned int rcid)
{
- struct rpmsg_channel_info chinfo;
struct glink_channel *channel;
unsigned long flags;
@@ -1720,14 +1721,7 @@ static void qcom_glink_rx_close(struct qcom_glink *glink, unsigned int rcid)
/* cancel pending rx_done work */
cancel_work_sync(&channel->intent_work);
- if (channel->rpdev) {
- strscpy_pad(chinfo.name, channel->name, sizeof(chinfo.name));
- chinfo.src = RPMSG_ADDR_ANY;
- chinfo.dst = RPMSG_ADDR_ANY;
-
- rpmsg_unregister_device(glink->dev, &chinfo);
- }
- channel->rpdev = NULL;
+ qcom_glink_remove_rpmsg_device(glink, channel);
qcom_glink_send_close_ack(glink, channel);
@@ -1741,7 +1735,6 @@ static void qcom_glink_rx_close(struct qcom_glink *glink, unsigned int rcid)
static void qcom_glink_rx_close_ack(struct qcom_glink *glink, unsigned int lcid)
{
- struct rpmsg_channel_info chinfo;
struct glink_channel *channel;
unsigned long flags;
@@ -1763,14 +1756,7 @@ static void qcom_glink_rx_close_ack(struct qcom_glink *glink, unsigned int lcid)
spin_unlock_irqrestore(&glink->idr_lock, flags);
/* Decouple the potential rpdev from the channel */
- if (channel->rpdev) {
- strscpy(chinfo.name, channel->name, sizeof(chinfo.name));
- chinfo.src = RPMSG_ADDR_ANY;
- chinfo.dst = RPMSG_ADDR_ANY;
-
- rpmsg_unregister_device(glink->dev, &chinfo);
- }
- channel->rpdev = NULL;
+ qcom_glink_remove_rpmsg_device(glink, channel);
kref_put(&channel->refcount, qcom_glink_channel_release);
}
diff --git a/drivers/rtc/rtc-gamecube.c b/drivers/rtc/rtc-gamecube.c
index 045d5d45ab4b..889028cecf4a 100644
--- a/drivers/rtc/rtc-gamecube.c
+++ b/drivers/rtc/rtc-gamecube.c
@@ -355,9 +355,7 @@ static int gamecube_rtc_probe(struct platform_device *pdev)
rtc->range_max = U32_MAX;
rtc->ops = &gamecube_rtc_ops;
- devm_rtc_register_device(rtc);
-
- return 0;
+ return devm_rtc_register_device(rtc);
}
static const struct of_device_id gamecube_rtc_of_match[] = {
diff --git a/drivers/rtc/rtc-pcf85363.c b/drivers/rtc/rtc-pcf85363.c
index 540042b9eec8..ccc7834e5759 100644
--- a/drivers/rtc/rtc-pcf85363.c
+++ b/drivers/rtc/rtc-pcf85363.c
@@ -426,8 +426,8 @@ static int pcf85363_probe(struct i2c_client *client)
err = pcf85363_load_capacitance(pcf85363, client->dev.of_node);
if (err < 0)
- dev_warn(&client->dev, "failed to set xtal load capacitance: %d",
- err);
+ return dev_err_probe(&client->dev, err,
+ "failed to set xtal load capacitance\n");
pcf85363->rtc->ops = &rtc_ops;
pcf85363->rtc->range_min = RTC_TIMESTAMP_BEGIN_2000;
@@ -436,9 +436,16 @@ static int pcf85363_probe(struct i2c_client *client)
wakeup_source = device_property_read_bool(&client->dev,
"wakeup-source");
if (client->irq > 0 || wakeup_source) {
- regmap_write(pcf85363->regmap, CTRL_FLAGS, 0);
- regmap_update_bits(pcf85363->regmap, CTRL_PIN_IO,
- PIN_IO_INTAPM, PIN_IO_INTA_OUT);
+ err = regmap_write(pcf85363->regmap, CTRL_FLAGS, 0);
+ if (err)
+ return dev_err_probe(&client->dev, err,
+ "failed to clear flags\n");
+
+ err = regmap_update_bits(pcf85363->regmap, CTRL_PIN_IO,
+ PIN_IO_INTAPM, PIN_IO_INTA_OUT);
+ if (err)
+ return dev_err_probe(&client->dev, err,
+ "failed to set interrupt pin mode\n");
}
if (client->irq > 0) {
diff --git a/drivers/rtc/rtc-pcf8563.c b/drivers/rtc/rtc-pcf8563.c
index 23b21b908915..a3d81afff046 100644
--- a/drivers/rtc/rtc-pcf8563.c
+++ b/drivers/rtc/rtc-pcf8563.c
@@ -494,7 +494,9 @@ static struct clk *pcf8563_clkout_register_clk(struct pcf8563 *pcf8563)
clk = devm_clk_register(&client->dev, &pcf8563->clkout_hw);
if (!IS_ERR(clk))
- of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ devm_of_clk_add_hw_provider(pcf8563->rtc->dev.parent,
+ of_clk_hw_simple_get,
+ &pcf8563->clkout_hw);
return clk;
}
diff --git a/drivers/rtc/rtc-rzn1.c b/drivers/rtc/rtc-rzn1.c
index 8570c8e63d70..c64eaf50bb39 100644
--- a/drivers/rtc/rtc-rzn1.c
+++ b/drivers/rtc/rtc-rzn1.c
@@ -219,7 +219,6 @@ static int rzn1_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
struct rzn1_rtc *rtc = dev_get_drvdata(dev);
struct rtc_time *tm = &alrm->time, tm_now;
unsigned long alarm, farest;
- unsigned int days_ahead, wday;
int ret;
ret = rzn1_rtc_read_time(dev, &tm_now);
@@ -232,13 +231,14 @@ static int rzn1_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
if (time_after(alarm, farest))
return -ERANGE;
- /* Convert alarm day into week day */
- days_ahead = tm->tm_mday - tm_now.tm_mday;
- wday = (tm_now.tm_wday + days_ahead) % 7;
+ /* Disable alarm interrupts before reprogramming the alarm. */
+ ret = rzn1_rtc_alarm_irq_enable(dev, 0);
+ if (ret)
+ return ret;
writel(bin2bcd(tm->tm_min), rtc->base + RZN1_RTC_ALM);
writel(bin2bcd(tm->tm_hour), rtc->base + RZN1_RTC_ALH);
- writel(BIT(wday), rtc->base + RZN1_RTC_ALW);
+ writel(BIT(tm->tm_wday), rtc->base + RZN1_RTC_ALW);
rzn1_rtc_alarm_irq_enable(dev, alrm->enabled);
diff --git a/drivers/rtc/rtc-zynqmp.c b/drivers/rtc/rtc-zynqmp.c
index f49af7d963fb..e77fb1433401 100644
--- a/drivers/rtc/rtc-zynqmp.c
+++ b/drivers/rtc/rtc-zynqmp.c
@@ -320,10 +320,9 @@ static int xlnx_rtc_probe(struct platform_device *pdev)
/* Getting the rtc info */
xrtcdev->rtc_clk = devm_clk_get_optional(&pdev->dev, "rtc");
- if (IS_ERR(xrtcdev->rtc_clk)) {
- if (PTR_ERR(xrtcdev->rtc_clk) != -EPROBE_DEFER)
- dev_warn(&pdev->dev, "Device clock not found.\n");
- }
+ if (IS_ERR(xrtcdev->rtc_clk))
+ return dev_err_probe(&pdev->dev, PTR_ERR(xrtcdev->rtc_clk),
+ "Failed to get rtc clock\n");
xrtcdev->freq = clk_get_rate(xrtcdev->rtc_clk);
if (!xrtcdev->freq) {
ret = of_property_read_u32(pdev->dev.of_node, "calibration",
diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index 3bac163057d8..ab29febebbfd 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -48,15 +48,19 @@ static void vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q);
* 1. matrix_dev->guests_lock: required to use the KVM pointer to update a KVM
* guest's APCB.
* 2. kvm->lock: required to update a guest's APCB
- * 3. matrix_dev->mdevs_lock: required to access data stored in a matrix_mdev
+ * 3. kvm->arch.crypto.pqap_hook_rwsem: required to update pqap_hook and
+ * serialize against PQAP intercepts
+ * 4. matrix_dev->mdevs_lock: required to access data stored in a matrix_mdev
*
- * Note: If @kvm is NULL, the KVM lock will not be taken.
+ * Note: If @kvm is NULL, the KVM lock and pqap_hook_rwsem will not be taken.
*/
static inline void get_update_locks_for_kvm(struct kvm *kvm)
{
mutex_lock(&matrix_dev->guests_lock);
- if (kvm)
+ if (kvm) {
mutex_lock(&kvm->lock);
+ down_write(&kvm->arch.crypto.pqap_hook_rwsem);
+ }
mutex_lock(&matrix_dev->mdevs_lock);
}
@@ -68,16 +72,19 @@ static inline void get_update_locks_for_kvm(struct kvm *kvm)
*
* The proper unlocking order is:
* 1. matrix_dev->mdevs_lock
- * 2. kvm->lock
- * 3. matrix_dev->guests_lock
+ * 2. kvm->arch.crypto.pqap_hook_rwsem
+ * 3. kvm->lock
+ * 4. matrix_dev->guests_lock
*
- * Note: If @kvm is NULL, the KVM lock will not be released.
+ * Note: If @kvm is NULL, the KVM lock and pqap_hook_rwsem will not be released.
*/
static inline void release_update_locks_for_kvm(struct kvm *kvm)
{
mutex_unlock(&matrix_dev->mdevs_lock);
- if (kvm)
+ if (kvm) {
+ up_write(&kvm->arch.crypto.pqap_hook_rwsem);
mutex_unlock(&kvm->lock);
+ }
mutex_unlock(&matrix_dev->guests_lock);
}
@@ -1817,26 +1824,17 @@ static const struct attribute_group *vfio_ap_mdev_attr_groups[] = {
static int vfio_ap_mdev_set_kvm(struct ap_matrix_mdev *matrix_mdev,
struct kvm *kvm)
{
- struct ap_matrix_mdev *m;
-
if (kvm->arch.crypto.crycbd) {
- down_write(&kvm->arch.crypto.pqap_hook_rwsem);
- kvm->arch.crypto.pqap_hook = &matrix_mdev->pqap_hook;
- up_write(&kvm->arch.crypto.pqap_hook_rwsem);
-
get_update_locks_for_kvm(kvm);
-
- list_for_each_entry(m, &matrix_dev->mdev_list, node) {
- if (m != matrix_mdev && m->kvm == kvm) {
- release_update_locks_for_kvm(kvm);
- return -EPERM;
- }
+ if (kvm->arch.crypto.pqap_hook) {
+ release_update_locks_for_kvm(kvm);
+ return -EPERM;
}
+ kvm->arch.crypto.pqap_hook = &matrix_mdev->pqap_hook;
kvm_get_kvm(kvm);
matrix_mdev->kvm = kvm;
vfio_ap_mdev_update_guest_apcb(matrix_mdev);
-
release_update_locks_for_kvm(kvm);
}
@@ -1879,18 +1877,15 @@ static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)
struct kvm *kvm = matrix_mdev->kvm;
if (kvm && kvm->arch.crypto.crycbd) {
- down_write(&kvm->arch.crypto.pqap_hook_rwsem);
- kvm->arch.crypto.pqap_hook = NULL;
- up_write(&kvm->arch.crypto.pqap_hook_rwsem);
-
get_update_locks_for_kvm(kvm);
+ kvm->arch.crypto.pqap_hook = NULL;
kvm_arch_crypto_clear_masks(kvm);
vfio_ap_mdev_reset_queues(matrix_mdev);
- kvm_put_kvm(kvm);
matrix_mdev->kvm = NULL;
release_update_locks_for_kvm(kvm);
+ kvm_put_kvm(kvm);
}
}
@@ -2266,6 +2261,23 @@ static ssize_t status_show(struct device *dev,
mutex_lock(&matrix_dev->mdevs_lock);
q = dev_get_drvdata(&apdev->device);
+
+ /*
+ * Make sure the drvdata has been set before proceeding. There is a
+ * possibility that the drvdata was not set if the vfio_ap_queue object
+ * could not be allocated when the queue device was probed. In that case,
+ * the locks used in vfio_ap_mdev_probe_queue() are released prior to
+ * removing the sysfs status attribute to avoid a lockdep
+ * splat. That opens a very small window where the status attribute is
+ * still available without the vfio_ap_queue object having been
+ * stored in the device drvdata. In that case, indicate the queue is not
+ * assigned.
+ */
+ if (!q) {
+ nchars = sysfs_emit(buf, "%s\n", AP_QUEUE_UNASSIGNED);
+ goto done;
+ }
+
matrix_mdev = vfio_ap_mdev_for_queue(q);
/* If the queue is assigned to the matrix mediated device, then
@@ -2293,6 +2305,7 @@ static ssize_t status_show(struct device *dev,
AP_QUEUE_UNASSIGNED);
}
+done:
mutex_unlock(&matrix_dev->mdevs_lock);
return nchars;
@@ -2366,14 +2379,17 @@ void vfio_ap_mdev_unregister(void)
int vfio_ap_mdev_probe_queue(struct ap_device *apdev)
{
- int ret;
+ int ret, apqn;
struct vfio_ap_queue *q;
DECLARE_BITMAP(apm_filtered, AP_DEVICES);
struct ap_matrix_mdev *matrix_mdev;
+ apqn = to_ap_queue(&apdev->device)->qid;
+ matrix_mdev = get_update_locks_by_apqn(apqn);
+
ret = sysfs_create_group(&apdev->device.kobj, &vfio_queue_attr_group);
if (ret)
- return ret;
+ goto err_release_locks;
q = kzalloc(sizeof(*q), GFP_KERNEL);
if (!q) {
@@ -2381,11 +2397,10 @@ int vfio_ap_mdev_probe_queue(struct ap_device *apdev)
goto err_remove_group;
}
- q->apqn = to_ap_queue(&apdev->device)->qid;
+ q->apqn = apqn;
q->saved_isc = VFIO_AP_ISC_INVALID;
memset(&q->reset_status, 0, sizeof(q->reset_status));
INIT_WORK(&q->reset_work, apq_reset_check);
- matrix_mdev = get_update_locks_by_apqn(q->apqn);
if (matrix_mdev) {
vfio_ap_mdev_link_queue(matrix_mdev, q);
@@ -2414,8 +2429,13 @@ int vfio_ap_mdev_probe_queue(struct ap_device *apdev)
return ret;
err_remove_group:
+ release_update_locks_for_mdev(matrix_mdev);
sysfs_remove_group(&apdev->device.kobj, &vfio_queue_attr_group);
return ret;
+
+err_release_locks:
+ release_update_locks_for_mdev(matrix_mdev);
+ return ret;
}
void vfio_ap_mdev_remove_queue(struct ap_device *apdev)
@@ -2510,24 +2530,28 @@ static void vfio_ap_mdev_hot_unplug_cfg(struct ap_matrix_mdev *matrix_mdev,
unsigned long *aqrem,
unsigned long *cdrem)
{
- int do_hotplug = 0;
+ bool do_hotplug = false;
- if (!bitmap_empty(aprem, AP_DEVICES)) {
- do_hotplug |= bitmap_andnot(matrix_mdev->shadow_apcb.apm,
- matrix_mdev->shadow_apcb.apm,
- aprem, AP_DEVICES);
+ if (bitmap_intersects(matrix_mdev->shadow_apcb.apm, aprem, AP_DEVICES)) {
+ bitmap_andnot(matrix_mdev->shadow_apcb.apm,
+ matrix_mdev->shadow_apcb.apm,
+ aprem, AP_DEVICES);
+ do_hotplug = true;
}
- if (!bitmap_empty(aqrem, AP_DOMAINS)) {
- do_hotplug |= bitmap_andnot(matrix_mdev->shadow_apcb.aqm,
- matrix_mdev->shadow_apcb.aqm,
- aqrem, AP_DEVICES);
+ if (bitmap_intersects(matrix_mdev->shadow_apcb.aqm, aqrem, AP_DOMAINS)) {
+ bitmap_andnot(matrix_mdev->shadow_apcb.aqm,
+ matrix_mdev->shadow_apcb.aqm,
+ aqrem, AP_DOMAINS);
+ do_hotplug = true;
}
- if (!bitmap_empty(cdrem, AP_DOMAINS))
- do_hotplug |= bitmap_andnot(matrix_mdev->shadow_apcb.adm,
- matrix_mdev->shadow_apcb.adm,
- cdrem, AP_DOMAINS);
+ if (bitmap_intersects(matrix_mdev->shadow_apcb.adm, cdrem, AP_DOMAINS)) {
+ bitmap_andnot(matrix_mdev->shadow_apcb.adm,
+ matrix_mdev->shadow_apcb.adm,
+ cdrem, AP_DOMAINS);
+ do_hotplug = true;
+ }
if (do_hotplug)
vfio_ap_mdev_update_guest_apcb(matrix_mdev);
@@ -2554,28 +2578,41 @@ static void vfio_ap_mdev_cfg_remove(unsigned long *ap_remove,
DECLARE_BITMAP(aprem, AP_DEVICES);
DECLARE_BITMAP(aqrem, AP_DOMAINS);
DECLARE_BITMAP(cdrem, AP_DOMAINS);
- int do_remove = 0;
+ int do_remove;
+ /*
+ * It is safe to traverse this list here because the
+ * required guard - matrix_dev->guests_lock - is taken in the
+ * vfio_ap_on_cfg_changed function prior to this function getting
+ * called.
+ */
list_for_each_entry(matrix_mdev, &matrix_dev->mdev_list, node) {
- mutex_lock(&matrix_mdev->kvm->lock);
+ /*
+ * The mdevs_lock must be held to access fields within matrix_mdev,
+ * and kvm->lock must be taken before mdevs_lock to satisfy the lock
+ * ordering requirement and prevent a lockdep splat.
+ */
+ if (matrix_mdev->kvm)
+ mutex_lock(&matrix_mdev->kvm->lock);
mutex_lock(&matrix_dev->mdevs_lock);
- do_remove |= bitmap_and(aprem, ap_remove,
- matrix_mdev->matrix.apm,
- AP_DEVICES);
+ do_remove = bitmap_and(aprem, ap_remove,
+ matrix_mdev->matrix.apm,
+ AP_DEVICES);
do_remove |= bitmap_and(aqrem, aq_remove,
matrix_mdev->matrix.aqm,
AP_DOMAINS);
- do_remove |= bitmap_andnot(cdrem, cd_remove,
- matrix_mdev->matrix.adm,
- AP_DOMAINS);
+ do_remove |= bitmap_and(cdrem, cd_remove,
+ matrix_mdev->matrix.adm,
+ AP_DOMAINS);
if (do_remove)
vfio_ap_mdev_hot_unplug_cfg(matrix_mdev, aprem, aqrem,
cdrem);
mutex_unlock(&matrix_dev->mdevs_lock);
- mutex_unlock(&matrix_mdev->kvm->lock);
+ if (matrix_mdev->kvm)
+ mutex_unlock(&matrix_mdev->kvm->lock);
}
}
@@ -2700,12 +2737,20 @@ static void vfio_ap_mdev_cfg_add(unsigned long *apm_add, unsigned long *aqm_add,
vfio_ap_filter_apid_by_qtype(apm_add, aqm_add);
list_for_each_entry(matrix_mdev, &matrix_dev->mdev_list, node) {
+ /*
+ * The mdevs_lock must be held in order to access fields
+ * within matrix_mdev
+ */
+ mutex_lock(&matrix_dev->mdevs_lock);
+
bitmap_and(matrix_mdev->apm_add,
matrix_mdev->matrix.apm, apm_add, AP_DEVICES);
bitmap_and(matrix_mdev->aqm_add,
matrix_mdev->matrix.aqm, aqm_add, AP_DOMAINS);
bitmap_and(matrix_mdev->adm_add,
matrix_mdev->matrix.adm, adm_add, AP_DEVICES);
+
+ mutex_unlock(&matrix_dev->mdevs_lock);
}
}
@@ -2772,8 +2817,14 @@ static void vfio_ap_mdev_hot_plug_cfg(struct ap_matrix_mdev *matrix_mdev)
DECLARE_BITMAP(apm_filtered, AP_DEVICES);
bool filter_domains, filter_adapters, filter_cdoms, do_hotplug = false;
- mutex_lock(&matrix_mdev->kvm->lock);
- mutex_lock(&matrix_dev->mdevs_lock);
+ /*
+ * Zero out the apm_filtered bitmap in case there are no adapters or
+ * domains to be added, but only control domains. In that case,
+ * vfio_ap_mdev_filter_matrix() - which initializes apm_filtered - will
+ * not get called and the reset_queues_for_apids will crash because it
+ * will access an uninitialized bitmap.
+ */
+ bitmap_zero(apm_filtered, AP_DEVICES);
filter_adapters = bitmap_intersects(matrix_mdev->matrix.apm,
matrix_mdev->apm_add, AP_DEVICES);
@@ -2792,9 +2843,6 @@ static void vfio_ap_mdev_hot_plug_cfg(struct ap_matrix_mdev *matrix_mdev)
vfio_ap_mdev_update_guest_apcb(matrix_mdev);
reset_queues_for_apids(matrix_mdev, apm_filtered);
-
- mutex_unlock(&matrix_dev->mdevs_lock);
- mutex_unlock(&matrix_mdev->kvm->lock);
}
void vfio_ap_on_scan_complete(struct ap_config_info *new_config_info,
@@ -2805,15 +2853,29 @@ void vfio_ap_on_scan_complete(struct ap_config_info *new_config_info,
mutex_lock(&matrix_dev->guests_lock);
list_for_each_entry(matrix_mdev, &matrix_dev->mdev_list, node) {
+ /*
+ * The mdevs_lock must be held to access fields within matrix_mdev,
+ * and kvm->lock must be taken before mdevs_lock to satisfy the lock
+ * ordering requirement and prevent a lockdep splat.
+ */
+ if (matrix_mdev->kvm)
+ mutex_lock(&matrix_mdev->kvm->lock);
+ mutex_lock(&matrix_dev->mdevs_lock);
+
if (bitmap_empty(matrix_mdev->apm_add, AP_DEVICES) &&
bitmap_empty(matrix_mdev->aqm_add, AP_DOMAINS) &&
bitmap_empty(matrix_mdev->adm_add, AP_DOMAINS))
- continue;
+ goto do_unlock;
vfio_ap_mdev_hot_plug_cfg(matrix_mdev);
bitmap_clear(matrix_mdev->apm_add, 0, AP_DEVICES);
bitmap_clear(matrix_mdev->aqm_add, 0, AP_DOMAINS);
bitmap_clear(matrix_mdev->adm_add, 0, AP_DOMAINS);
+
+do_unlock:
+ mutex_unlock(&matrix_dev->mdevs_lock);
+ if (matrix_mdev->kvm)
+ mutex_unlock(&matrix_mdev->kvm->lock);
}
mutex_unlock(&matrix_dev->guests_lock);
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
index c1d8f2c91a5e..0d78e9dbf641 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
@@ -5854,6 +5854,8 @@ _base_release_memory_pools(struct MPT3SAS_ADAPTER *ioc)
if (ioc->pcie_sgl_dma_pool) {
for (i = 0; i < ioc->scsiio_depth; i++) {
+ if (!ioc->pcie_sg_lookup[i].pcie_sgl)
+ continue;
dma_pool_free(ioc->pcie_sgl_dma_pool,
ioc->pcie_sg_lookup[i].pcie_sgl,
ioc->pcie_sg_lookup[i].pcie_sgl_dma);
diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c
index 5317f82c51fd..01383196d806 100644
--- a/drivers/scsi/pm8001/pm8001_init.c
+++ b/drivers/scsi/pm8001/pm8001_init.c
@@ -1027,8 +1027,8 @@ static u32 pm8001_request_msix(struct pm8001_hba_info *pm8001_ha)
&(pm8001_ha->irq_vector[i]));
if (rc) {
for (j = 0; j < i; j++) {
- free_irq(pci_irq_vector(pm8001_ha->pdev, i),
- &(pm8001_ha->irq_vector[i]));
+ free_irq(pci_irq_vector(pm8001_ha->pdev, j),
+ &pm8001_ha->irq_vector[j]);
}
pci_free_irq_vectors(pm8001_ha->pdev);
break;
diff --git a/drivers/scsi/qla2xxx/qla_attr.c b/drivers/scsi/qla2xxx/qla_attr.c
index e6ece30c4348..a5c031f92b0a 100644
--- a/drivers/scsi/qla2xxx/qla_attr.c
+++ b/drivers/scsi/qla2xxx/qla_attr.c
@@ -580,7 +580,6 @@ qla2x00_sysfs_read_vpd(struct file *filp, struct kobject *kobj,
ha->isp_ops->read_optrom(vha, ha->vpd, faddr, ha->vpd_size);
mutex_unlock(&ha->optrom_mutex);
- ha->isp_ops->read_optrom(vha, ha->vpd, faddr, ha->vpd_size);
skip:
return memory_read_from_buffer(buf, count, &off, ha->vpd, ha->vpd_size);
}
@@ -816,7 +815,9 @@ qla2x00_sysfs_write_reset(struct file *filp, struct kobject *kobj,
"Unable to allocate memory for VPD information update.\n");
return -ENOMEM;
}
+ mutex_lock(&ha->optrom_mutex);
ha->isp_ops->get_flash_version(vha, tmp_data);
+ mutex_unlock(&ha->optrom_mutex);
vfree(tmp_data);
break;
}
@@ -1660,10 +1661,8 @@ qla2x00_fw_state_show(struct device *dev, struct device_attribute *attr,
rval = qla2x00_get_firmware_state(vha, state);
mutex_unlock(&vha->hw->optrom_mutex);
out:
- if (rval != QLA_SUCCESS) {
+ if (rval != QLA_SUCCESS)
memset(state, -1, sizeof(state));
- rval = qla2x00_get_firmware_state(vha, state);
- }
return scnprintf(buf, PAGE_SIZE, "0x%x 0x%x 0x%x 0x%x 0x%x 0x%x\n",
state[0], state[1], state[2], state[3], state[4], state[5]);
diff --git a/drivers/scsi/qla2xxx/qla_bsg.c b/drivers/scsi/qla2xxx/qla_bsg.c
index f43969ed87bf..3982f7d576a8 100644
--- a/drivers/scsi/qla2xxx/qla_bsg.c
+++ b/drivers/scsi/qla2xxx/qla_bsg.c
@@ -1560,12 +1560,12 @@ qla2x00_update_fru_versions(struct bsg_job *bsg_job)
scsi_qla_host_t *vha = shost_priv(host);
struct qla_hw_data *ha = vha->hw;
int rval = 0;
- uint8_t bsg[DMA_POOL_SIZE];
+ uint8_t bsg[DMA_POOL_SIZE] = {};
struct qla_image_version_list *list = (void *)bsg;
struct qla_image_version *image;
uint32_t count;
dma_addr_t sfp_dma;
- void *sfp = dma_pool_alloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
+ void *sfp = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
if (!sfp) {
bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
@@ -1578,6 +1578,13 @@ qla2x00_update_fru_versions(struct bsg_job *bsg_job)
image = list->version;
count = list->count;
+
+ if (struct_size(list, version, count) > sizeof(bsg)) {
+ bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
+ EXT_STATUS_INVALID_PARAM;
+ goto dealloc;
+ }
+
while (count--) {
memcpy(sfp, &image->field_info, sizeof(image->field_info));
rval = qla2x00_write_sfp(vha, sfp_dma, sfp,
@@ -1613,10 +1620,10 @@ qla2x00_read_fru_status(struct bsg_job *bsg_job)
scsi_qla_host_t *vha = shost_priv(host);
struct qla_hw_data *ha = vha->hw;
int rval = 0;
- uint8_t bsg[DMA_POOL_SIZE];
+ uint8_t bsg[DMA_POOL_SIZE] = {};
struct qla_status_reg *sr = (void *)bsg;
dma_addr_t sfp_dma;
- uint8_t *sfp = dma_pool_alloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
+ uint8_t *sfp = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
if (!sfp) {
bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
@@ -1664,10 +1671,10 @@ qla2x00_write_fru_status(struct bsg_job *bsg_job)
scsi_qla_host_t *vha = shost_priv(host);
struct qla_hw_data *ha = vha->hw;
int rval = 0;
- uint8_t bsg[DMA_POOL_SIZE];
+ uint8_t bsg[DMA_POOL_SIZE] = {};
struct qla_status_reg *sr = (void *)bsg;
dma_addr_t sfp_dma;
- uint8_t *sfp = dma_pool_alloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
+ uint8_t *sfp = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
if (!sfp) {
bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
@@ -1711,10 +1718,10 @@ qla2x00_write_i2c(struct bsg_job *bsg_job)
scsi_qla_host_t *vha = shost_priv(host);
struct qla_hw_data *ha = vha->hw;
int rval = 0;
- uint8_t bsg[DMA_POOL_SIZE];
+ uint8_t bsg[DMA_POOL_SIZE] = {};
struct qla_i2c_access *i2c = (void *)bsg;
dma_addr_t sfp_dma;
- uint8_t *sfp = dma_pool_alloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
+ uint8_t *sfp = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
if (!sfp) {
bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
@@ -1725,6 +1732,12 @@ qla2x00_write_i2c(struct bsg_job *bsg_job)
sg_copy_to_buffer(bsg_job->request_payload.sg_list,
bsg_job->request_payload.sg_cnt, i2c, sizeof(*i2c));
+ if (i2c->length > sizeof(i2c->buffer)) {
+ bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
+ EXT_STATUS_INVALID_PARAM;
+ goto dealloc;
+ }
+
memcpy(sfp, i2c->buffer, i2c->length);
rval = qla2x00_write_sfp(vha, sfp_dma, sfp,
i2c->device, i2c->offset, i2c->length, i2c->option);
@@ -1757,10 +1770,10 @@ qla2x00_read_i2c(struct bsg_job *bsg_job)
scsi_qla_host_t *vha = shost_priv(host);
struct qla_hw_data *ha = vha->hw;
int rval = 0;
- uint8_t bsg[DMA_POOL_SIZE];
+ uint8_t bsg[DMA_POOL_SIZE] = {};
struct qla_i2c_access *i2c = (void *)bsg;
dma_addr_t sfp_dma;
- uint8_t *sfp = dma_pool_alloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
+ uint8_t *sfp = dma_pool_zalloc(ha->s_dma_pool, GFP_KERNEL, &sfp_dma);
if (!sfp) {
bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
@@ -1771,6 +1784,12 @@ qla2x00_read_i2c(struct bsg_job *bsg_job)
sg_copy_to_buffer(bsg_job->request_payload.sg_list,
bsg_job->request_payload.sg_cnt, i2c, sizeof(*i2c));
+ if (i2c->length > sizeof(i2c->buffer)) {
+ bsg_reply->reply_data.vendor_reply.vendor_rsp[0] =
+ EXT_STATUS_INVALID_PARAM;
+ goto dealloc;
+ }
+
rval = qla2x00_read_sfp(vha, sfp_dma, sfp,
i2c->device, i2c->offset, i2c->length, i2c->option);
diff --git a/drivers/scsi/qla2xxx/qla_dbg.c b/drivers/scsi/qla2xxx/qla_dbg.c
index 691ef827a5ab..37a308c020f9 100644
--- a/drivers/scsi/qla2xxx/qla_dbg.c
+++ b/drivers/scsi/qla2xxx/qla_dbg.c
@@ -16,7 +16,7 @@
* | | | 0x2127-0x2128 |
* | Queue Command and IO tracing | 0x3074 | 0x300b |
* | | | 0x3027-0x3028 |
- * | | | 0x303d-0x3041 |
+ * | | | 0x303e-0x3041 |
* | | | 0x302e,0x3033 |
* | | | 0x3036,0x3038 |
* | | | 0x303a |
diff --git a/drivers/scsi/qla2xxx/qla_dfs.c b/drivers/scsi/qla2xxx/qla_dfs.c
index 08273520c777..8772db7a5ed8 100644
--- a/drivers/scsi/qla2xxx/qla_dfs.c
+++ b/drivers/scsi/qla2xxx/qla_dfs.c
@@ -518,7 +518,9 @@ qla2x00_dfs_fce_write(struct file *file, const char __user *buffer,
return PTR_ERR(buf);
}
- enable = kstrtoul(buf, 0, 0);
+ rc = kstrtoul(buf, 0, &enable);
+ if (rc)
+ goto out_free;
rc = count;
mutex_lock(&ha->fce_mutex);
diff --git a/drivers/scsi/qla2xxx/qla_edif.c b/drivers/scsi/qla2xxx/qla_edif.c
index be20e2c457b8..dd5b64e73fd8 100644
--- a/drivers/scsi/qla2xxx/qla_edif.c
+++ b/drivers/scsi/qla2xxx/qla_edif.c
@@ -3491,6 +3491,9 @@ void qla_chk_edif_rx_sa_delete_pending(scsi_qla_host_t *vha,
struct scsi_cmnd *cmd = GET_CMD_SP(sp);
uint32_t handle;
+ if (!cmd)
+ return;
+
handle = (uint32_t)LSW(sts24->handle);
/* find out if this status iosb is for a scsi read */
diff --git a/drivers/scsi/qla2xxx/qla_gs.c b/drivers/scsi/qla2xxx/qla_gs.c
index 4cef9a349343..cdf50f71f76e 100644
--- a/drivers/scsi/qla2xxx/qla_gs.c
+++ b/drivers/scsi/qla2xxx/qla_gs.c
@@ -157,8 +157,8 @@ qla2x00_chk_ms_status(scsi_qla_host_t *vha, ms_iocb_entry_t *ms_pkt,
break;
case CS_PORT_LOGGED_OUT:
if (IS_FWI2_CAPABLE(ha)) {
- if (le16_to_cpu(ms_pkt->loop_id.extended) ==
- NPH_SNS)
+ if (le16_to_cpu(((struct ct_entry_24xx *)
+ ms_pkt)->nport_handle) == NPH_SNS)
lid_is_sns = true;
} else {
if (le16_to_cpu(ms_pkt->loop_id.extended) ==
diff --git a/drivers/scsi/qla2xxx/qla_init.c b/drivers/scsi/qla2xxx/qla_init.c
index 2ab1cecda8c7..f2ed5c559089 100644
--- a/drivers/scsi/qla2xxx/qla_init.c
+++ b/drivers/scsi/qla2xxx/qla_init.c
@@ -228,7 +228,7 @@ qla2x00_async_iocb_timeout(void *data)
srb_t *sp = data;
fc_port_t *fcport = sp->fcport;
struct srb_iocb *lio = &sp->u.iocb_cmd;
- int rc, h;
+ int rc, h, found;
unsigned long flags;
if (fcport) {
@@ -251,6 +251,7 @@ qla2x00_async_iocb_timeout(void *data)
lio->u.logio.data[1] =
lio->u.logio.flags & SRB_LOGIN_RETRIED ?
QLA_LOGIO_LOGIN_RETRIED : 0;
+ found = 0;
spin_lock_irqsave(sp->qpair->qp_lock_ptr, flags);
for (h = 1; h < sp->qpair->req->num_outstanding_cmds;
h++) {
@@ -258,11 +259,19 @@ qla2x00_async_iocb_timeout(void *data)
sp) {
sp->qpair->req->outstanding_cmds[h] =
NULL;
+ found = 1;
break;
}
}
spin_unlock_irqrestore(sp->qpair->qp_lock_ptr, flags);
- sp->done(sp, QLA_FUNCTION_TIMEOUT);
+ /*
+ * Only complete the command if this path removed it
+ * from outstanding_cmds. Otherwise the ISR already
+ * completed it and a second sp->done() would race the
+ * submitter's freeing of the on-stack completion.
+ */
+ if (found)
+ sp->done(sp, QLA_FUNCTION_TIMEOUT);
}
break;
case SRB_LOGOUT_CMD:
@@ -275,6 +284,7 @@ qla2x00_async_iocb_timeout(void *data)
default:
rc = qla24xx_async_abort_cmd(sp, false);
if (rc) {
+ found = 0;
spin_lock_irqsave(sp->qpair->qp_lock_ptr, flags);
for (h = 1; h < sp->qpair->req->num_outstanding_cmds;
h++) {
@@ -282,11 +292,19 @@ qla2x00_async_iocb_timeout(void *data)
sp) {
sp->qpair->req->outstanding_cmds[h] =
NULL;
+ found = 1;
break;
}
}
spin_unlock_irqrestore(sp->qpair->qp_lock_ptr, flags);
- sp->done(sp, QLA_FUNCTION_TIMEOUT);
+ /*
+ * Only complete the command if this path removed it
+ * from outstanding_cmds. Otherwise the ISR already
+ * completed it and a second sp->done() would race the
+ * submitter's freeing of the on-stack completion.
+ */
+ if (found)
+ sp->done(sp, QLA_FUNCTION_TIMEOUT);
}
break;
}
@@ -3764,11 +3782,27 @@ int qla2x00_alloc_fce_trace(scsi_qla_host_t *vha)
void qla2x00_free_fce_trace(struct qla_hw_data *ha)
{
- if (!ha->fce)
+ void *fce;
+ dma_addr_t fce_dma;
+ unsigned long flags;
+
+ /*
+ * Unpublish ha->fce under hardware_lock so a firmware dump in
+ * progress (which reads ha->fce under the same lock) cannot race
+ * with the buffer being freed.
+ */
+ spin_lock_irqsave(&ha->hardware_lock, flags);
+ if (!ha->fce) {
+ spin_unlock_irqrestore(&ha->hardware_lock, flags);
return;
- dma_free_coherent(&ha->pdev->dev, FCE_SIZE, ha->fce, ha->fce_dma);
+ }
+ fce = ha->fce;
+ fce_dma = ha->fce_dma;
ha->fce = NULL;
ha->fce_dma = 0;
+ spin_unlock_irqrestore(&ha->hardware_lock, flags);
+
+ dma_free_coherent(&ha->pdev->dev, FCE_SIZE, fce, fce_dma);
}
static void
@@ -9837,11 +9871,28 @@ int qla2xxx_delete_qpair(struct scsi_qla_host *vha, struct qla_qpair *qpair)
{
int ret = QLA_FUNCTION_FAILED;
struct qla_hw_data *ha = qpair->hw;
+ struct rsp_que *rsp = qpair->rsp;
qpair->delete_in_progress = 1;
qla_free_buf_pool(qpair);
+ /*
+ * The response-queue interrupt schedules qla_do_work(), which
+ * dereferences qpair->rsp->req. Release the interrupt and flush
+ * any pending work before the request queue is freed below so a
+ * late completion cannot touch the freed request queue. The
+ * firmware queue-delete order (request then response) is kept.
+ */
+ if (rsp && rsp->msix && rsp->msix->have_irq) {
+ free_irq(rsp->msix->vector, rsp->msix->handle);
+ rsp->msix->have_irq = 0;
+ rsp->msix->in_use = 0;
+ rsp->msix->handle = NULL;
+ }
+ if (rsp && ha->wq)
+ cancel_work_sync(&qpair->q_work);
+
ret = qla25xx_delete_req_que(vha, qpair->req);
if (ret != QLA_SUCCESS)
goto fail;
diff --git a/drivers/scsi/qla2xxx/qla_inline.h b/drivers/scsi/qla2xxx/qla_inline.h
index ef4b3cc1cd77..09ada9dfb5b8 100644
--- a/drivers/scsi/qla2xxx/qla_inline.h
+++ b/drivers/scsi/qla2xxx/qla_inline.h
@@ -54,6 +54,19 @@ qla2x00_debounce_register(volatile __le16 __iomem *addr)
return (first);
}
+static inline u8
+qla_calc_queue_count(u16 msix_count)
+{
+ /*
+ * Request/response queues are bounded by the MSI-X vector count less
+ * the mailbox vector. These counters are u8, so a board advertising
+ * e.g. 257 vectors would truncate msix_count - 1 (256) to 0 and hand
+ * kzalloc_objs() a zero count (ZERO_SIZE_PTR), faulting on the first
+ * ha->req_q_map[0] store. Clamp into [1, QLA_MAX_QUEUES - 1].
+ */
+ return clamp_t(u16, msix_count - 1, 1, QLA_MAX_QUEUES - 1);
+}
+
static inline void
qla2x00_poll(struct rsp_que *rsp)
{
diff --git a/drivers/scsi/qla2xxx/qla_isr.c b/drivers/scsi/qla2xxx/qla_isr.c
index 07b08119af0b..9c26e4a0afc1 100644
--- a/drivers/scsi/qla2xxx/qla_isr.c
+++ b/drivers/scsi/qla2xxx/qla_isr.c
@@ -205,6 +205,17 @@ void __qla_consume_iocb(struct scsi_qla_host *vha,
struct purex_entry_24xx *purex = *pkt;
entry_count_remaining = purex->entry_count;
+
+ /*
+ * The caller already advanced ring_ptr past the head IOCB, so mark
+ * the head processed and account for it here, then consume only the
+ * continuation IOCBs that follow.
+ */
+ ((response_t *)purex)->signature = RESPONSE_PROCESSED;
+ /* flush signature */
+ wmb();
+ --entry_count_remaining;
+
while (entry_count_remaining > 0) {
new_pkt = rsp_q->ring_ptr;
*pkt = new_pkt;
@@ -3406,6 +3417,14 @@ qla2x00_status_entry(scsi_qla_host_t *vha, struct rsp_que *rsp, void *pkt)
return;
}
+ /* Everything below is the SCSI fast path; reject other SRB types. */
+ if (sp->type != SRB_SCSI_CMD) {
+ ql_dbg(ql_dbg_io, vha, 0x303d,
+ "Unexpected SRB type %x for status IOCB, sp %p.\n",
+ sp->type, sp);
+ return;
+ }
+
/* Fast path completion. */
qla_chk_edif_rx_sa_delete_pending(vha, sp, sts24);
sp->qpair->cmd_completion_cnt++;
@@ -3463,6 +3482,18 @@ qla2x00_status_entry(scsi_qla_host_t *vha, struct rsp_que *rsp, void *pkt)
if (scsi_status & SS_RESPONSE_INFO_LEN_VALID) {
/* Sense data lies beyond any FCP RESPONSE data. */
if (IS_FWI2_CAPABLE(ha)) {
+ /*
+ * A hostile or buggy target may report an
+ * rsp_info_len larger than the IOCB data area.
+ * Clamp it so the par_sense_len subtraction cannot
+ * underflow and walk sense_data out of bounds.
+ */
+ if (rsp_info_len > par_sense_len) {
+ ql_log(ql_log_warn, fcport->vha, 0x3107,
+ "Truncating bogus rsp_info_len 0x%x to 0x%x.\n",
+ rsp_info_len, par_sense_len);
+ rsp_info_len = par_sense_len;
+ }
sense_data += rsp_info_len;
par_sense_len -= rsp_info_len;
}
@@ -3778,10 +3809,12 @@ qla2x00_error_entry(scsi_qla_host_t *vha, struct rsp_que *rsp, sts_entry_t *pkt)
"iocb type %xh with error status %xh, handle %xh, rspq id %d\n",
pkt->entry_type, pkt->entry_status, pkt->handle, rsp->id);
- if (que >= ha->max_req_queues || !ha->req_q_map[que])
+ if (que >= ha->max_req_queues)
goto fatal;
req = ha->req_q_map[que];
+ if (!req)
+ goto fatal;
if (pkt->entry_status & RF_BUSY)
res = DID_BUS_BUSY << 16;
@@ -4548,10 +4581,10 @@ qla24xx_enable_msix(struct qla_hw_data *ha, struct rsp_que *rsp)
ha->msix_count = ret;
/* Recalculate queue values */
if (ha->mqiobase && (ql2xmqsupport || ql2xnvmeenable)) {
- ha->max_req_queues = ha->msix_count - 1;
+ ha->max_req_queues = qla_calc_queue_count(ha->msix_count);
/* ATIOQ needs 1 vector. That's 1 less QPair */
- if (QLA_TGT_MODE_ENABLED())
+ if (QLA_TGT_MODE_ENABLED() && ha->max_req_queues > 1)
ha->max_req_queues--;
ha->max_rsp_queues = ha->max_req_queues;
diff --git a/drivers/scsi/qla2xxx/qla_mbx.c b/drivers/scsi/qla2xxx/qla_mbx.c
index 41435e98092a..5bb9241ebebd 100644
--- a/drivers/scsi/qla2xxx/qla_mbx.c
+++ b/drivers/scsi/qla2xxx/qla_mbx.c
@@ -2214,6 +2214,8 @@ qla2x00_get_firmware_state(scsi_qla_host_t *vha, uint16_t *states)
if (!ha->flags.fw_started)
return QLA_FUNCTION_FAILED;
+ memset(&mc, 0, sizeof(mc));
+
mcp->mb[0] = MBC_GET_FIRMWARE_STATE;
mcp->out_mb = MBX_0;
if (IS_FWI2_CAPABLE(vha->hw))
@@ -4146,6 +4148,7 @@ qla24xx_report_id_acquisition(scsi_qla_host_t *vha,
list_for_each_entry(vp, &ha->vp_list, list) {
if (rptid_entry->vp_idx == vp->vp_idx) {
found = 1;
+ atomic_inc(&vp->vref_count);
break;
}
}
@@ -4154,7 +4157,9 @@ qla24xx_report_id_acquisition(scsi_qla_host_t *vha,
if (!found)
return;
+ spin_lock_irqsave(&ha->vport_slock, flags);
qla_update_host_map(vp, id);
+ spin_unlock_irqrestore(&ha->vport_slock, flags);
/*
* Cannot configure here as we are still sitting on the
@@ -4163,6 +4168,10 @@ qla24xx_report_id_acquisition(scsi_qla_host_t *vha,
set_bit(VP_IDX_ACQUIRED, &vp->vp_flags);
set_bit(REGISTER_FC4_NEEDED, &vp->dpc_flags);
set_bit(REGISTER_FDMI_NEEDED, &vp->dpc_flags);
+
+ spin_lock_irqsave(&ha->vport_slock, flags);
+ atomic_dec(&vp->vref_count);
+ spin_unlock_irqrestore(&ha->vport_slock, flags);
}
set_bit(VP_DPC_NEEDED, &vha->dpc_flags);
qla2xxx_wake_dpc(vha);
@@ -4273,10 +4282,10 @@ qla24xx_modify_vp_config(scsi_qla_host_t *vha)
if (rval != QLA_SUCCESS) {
ql_dbg(ql_dbg_mbx, vha, 0x10bd,
"Failed to issue VP config IOCB (%x).\n", rval);
- } else if (vpmod->comp_status != 0) {
+ } else if (vpmod->entry_status != 0) {
ql_dbg(ql_dbg_mbx, vha, 0x10be,
"Failed to complete IOCB -- error status (%x).\n",
- vpmod->comp_status);
+ vpmod->entry_status);
rval = QLA_FUNCTION_FAILED;
} else if (vpmod->comp_status != cpu_to_le16(CS_COMPLETE)) {
ql_dbg(ql_dbg_mbx, vha, 0x10bf,
@@ -6433,6 +6442,7 @@ qla26xx_dport_diagnostics(scsi_qla_host_t *vha,
mbx_cmd_t mc;
mbx_cmd_t *mcp = &mc;
dma_addr_t dd_dma;
+ void *dd;
if (!IS_QLA83XX(vha->hw) && !IS_QLA27XX(vha->hw) &&
!IS_QLA28XX(vha->hw))
@@ -6441,15 +6451,12 @@ qla26xx_dport_diagnostics(scsi_qla_host_t *vha,
ql_dbg(ql_dbg_mbx + ql_dbg_verbose, vha, 0x119f,
"Entered %s.\n", __func__);
- dd_dma = dma_map_single(&vha->hw->pdev->dev,
- dd_buf, size, DMA_FROM_DEVICE);
- if (dma_mapping_error(&vha->hw->pdev->dev, dd_dma)) {
- ql_log(ql_log_warn, vha, 0x1194, "Failed to map dma buffer.\n");
+ dd = dma_alloc_coherent(&vha->hw->pdev->dev, size, &dd_dma, GFP_KERNEL);
+ if (!dd) {
+ ql_log(ql_log_warn, vha, 0x1194, "Failed to allocate dma buffer.\n");
return QLA_MEMORY_ALLOC_FAILED;
}
- memset(dd_buf, 0, size);
-
mcp->mb[0] = MBC_DPORT_DIAGNOSTICS;
mcp->mb[1] = options;
mcp->mb[2] = MSW(LSD(dd_dma));
@@ -6471,8 +6478,9 @@ qla26xx_dport_diagnostics(scsi_qla_host_t *vha,
"Done %s.\n", __func__);
}
- dma_unmap_single(&vha->hw->pdev->dev, dd_dma,
- size, DMA_FROM_DEVICE);
+ memcpy(dd_buf, dd, size);
+
+ dma_free_coherent(&vha->hw->pdev->dev, size, dd, dd_dma);
return rval;
}
diff --git a/drivers/scsi/qla2xxx/qla_nvme.c b/drivers/scsi/qla2xxx/qla_nvme.c
index 42eb65a62f1f..3160e212fab1 100644
--- a/drivers/scsi/qla2xxx/qla_nvme.c
+++ b/drivers/scsi/qla2xxx/qla_nvme.c
@@ -374,6 +374,7 @@ static int qla_nvme_xmt_ls_rsp(struct nvme_fc_local_port *lport,
srb_t *sp;
int rval = QLA_FUNCTION_FAILED;
uint8_t cnt = 0;
+ unsigned long flags;
if (!fcport || fcport->deleted)
goto out;
@@ -440,7 +441,11 @@ static int qla_nvme_xmt_ls_rsp(struct nvme_fc_local_port *lport,
a.vp_idx = vha->vp_idx;
a.nport_handle = uctx->nport_handle;
a.xchg_address = uctx->exchange_address;
- qla_nvme_ls_reject_iocb(vha, ha->base_qpair, &a, true);
+ if (ha->flags.fw_started) {
+ spin_lock_irqsave(ha->base_qpair->qp_lock_ptr, flags);
+ qla_nvme_ls_reject_iocb(vha, ha->base_qpair, &a, true);
+ spin_unlock_irqrestore(ha->base_qpair->qp_lock_ptr, flags);
+ }
kfree(uctx);
return rval;
}
@@ -463,8 +468,8 @@ static void qla_nvme_ls_abort(struct nvme_fc_local_port *lport,
}
spin_unlock_irqrestore(&priv->cmd_lock, flags);
- INIT_WORK(&priv->abort_work, qla_nvme_abort_work);
- schedule_work(&priv->abort_work);
+ if (!schedule_work(&priv->abort_work))
+ kref_put(&priv->sp->cmd_kref, priv->sp->put_fn);
}
static int qla_nvme_ls_req(struct nvme_fc_local_port *lport,
@@ -501,6 +506,7 @@ static int qla_nvme_ls_req(struct nvme_fc_local_port *lport,
priv->sp = sp;
kref_init(&sp->cmd_kref);
spin_lock_init(&priv->cmd_lock);
+ INIT_WORK(&priv->abort_work, qla_nvme_abort_work);
nvme = &sp->u.iocb_cmd;
priv->fd = fd;
nvme->u.nvme.desc = fd;
@@ -545,8 +551,8 @@ static void qla_nvme_fcp_abort(struct nvme_fc_local_port *lport,
}
spin_unlock_irqrestore(&priv->cmd_lock, flags);
- INIT_WORK(&priv->abort_work, qla_nvme_abort_work);
- schedule_work(&priv->abort_work);
+ if (!schedule_work(&priv->abort_work))
+ kref_put(&priv->sp->cmd_kref, priv->sp->put_fn);
}
static inline int qla2x00_start_nvme_mq(srb_t *sp)
@@ -811,6 +817,7 @@ static int qla_nvme_post_cmd(struct nvme_fc_local_port *lport,
kref_init(&sp->cmd_kref);
spin_lock_init(&priv->cmd_lock);
+ INIT_WORK(&priv->abort_work, qla_nvme_abort_work);
sp->priv = priv;
priv->sp = sp;
sp->type = SRB_NVME_CMD;
@@ -1127,6 +1134,10 @@ static void qla_nvme_lsrjt_pt_iocb(struct scsi_qla_host *vha,
lsrjt_iocb->rx_byte_count = 0;
}
+/*
+ * Allocates from and advances the request ring, so the caller must hold
+ * qp->qp_lock_ptr (the response-queue caller already holds it).
+ */
static int
qla_nvme_ls_reject_iocb(struct scsi_qla_host *vha, struct qla_qpair *qp,
struct qla_nvme_lsrjt_pt_arg *a, bool is_xchg_terminate)
@@ -1183,6 +1194,7 @@ qla2xxx_process_purls_pkt(struct scsi_qla_host *vha, struct purex_item *item)
{
struct qla_nvme_unsol_ctx *uctx = item->purls_context;
struct qla_nvme_lsrjt_pt_arg a;
+ unsigned long flags;
int ret = 1;
#if (IS_ENABLED(CONFIG_NVME_FC))
@@ -1195,7 +1207,14 @@ qla2xxx_process_purls_pkt(struct scsi_qla_host *vha, struct purex_item *item)
a.vp_idx = vha->vp_idx;
a.nport_handle = uctx->nport_handle;
a.xchg_address = uctx->exchange_address;
- qla_nvme_ls_reject_iocb(vha, vha->hw->base_qpair, &a, true);
+ if (vha->hw->flags.fw_started) {
+ spin_lock_irqsave(vha->hw->base_qpair->qp_lock_ptr,
+ flags);
+ qla_nvme_ls_reject_iocb(vha, vha->hw->base_qpair, &a,
+ true);
+ spin_unlock_irqrestore(vha->hw->base_qpair->qp_lock_ptr,
+ flags);
+ }
list_del(&uctx->elem);
kfree(uctx);
}
diff --git a/drivers/scsi/qla2xxx/qla_nx2.c b/drivers/scsi/qla2xxx/qla_nx2.c
index 41ff6fbdb933..04d7ab6ad037 100644
--- a/drivers/scsi/qla2xxx/qla_nx2.c
+++ b/drivers/scsi/qla2xxx/qla_nx2.c
@@ -3507,7 +3507,7 @@ qla8044_poll_flash_status_reg(struct scsi_qla_host *vha)
msleep(QLA8044_FLASH_STATUS_REG_POLL_DELAY);
}
- if (!retries)
+ if (retries == -1)
ret_val = QLA_FUNCTION_FAILED;
return ret_val;
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index b8631698e262..7a78cff128cf 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -2123,7 +2123,7 @@ qla2x00_iospace_config(struct qla_hw_data *ha)
ha->msix_count = msix + 1;
/* Max queues are bounded by available msix vectors */
/* MB interrupt uses 1 vector */
- ha->max_req_queues = ha->msix_count - 1;
+ ha->max_req_queues = qla_calc_queue_count(ha->msix_count);
ha->max_rsp_queues = ha->max_req_queues;
/* Queue pairs is the max value minus the base queue pair */
ha->max_qpairs = ha->max_rsp_queues - 1;
@@ -2209,10 +2209,10 @@ qla83xx_iospace_config(struct qla_hw_data *ha)
*/
if (ql2xmqsupport || ql2xnvmeenable) {
/* MB interrupt uses 1 vector */
- ha->max_req_queues = ha->msix_count - 1;
+ ha->max_req_queues = qla_calc_queue_count(ha->msix_count);
/* ATIOQ needs 1 vector. That's 1 less QPair */
- if (QLA_TGT_MODE_ENABLED())
+ if (QLA_TGT_MODE_ENABLED() && ha->max_req_queues > 1)
ha->max_req_queues--;
ha->max_rsp_queues = ha->max_req_queues;
@@ -3929,8 +3929,6 @@ qla2x00_remove_one(struct pci_dev *pdev)
qla2x00_dfs_remove(base_vha);
- qla84xx_put_chip(base_vha);
-
/* Disable timer */
if (base_vha->timer_active)
qla2x00_stop_timer(base_vha);
@@ -3955,6 +3953,8 @@ qla2x00_remove_one(struct pci_dev *pdev)
scsi_remove_host(base_vha->host);
+ qla84xx_put_chip(base_vha);
+
qla2x00_free_device(base_vha);
qla2x00_clear_drv_active(ha);
@@ -6826,8 +6826,6 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
qla2x00_dfs_remove(base_vha);
- qla84xx_put_chip(base_vha);
-
if (base_vha->timer_active)
qla2x00_stop_timer(base_vha);
@@ -6845,6 +6843,8 @@ qla2x00_disable_board_on_pci_error(struct work_struct *work)
scsi_remove_host(base_vha->host);
+ qla84xx_put_chip(base_vha);
+
base_vha->flags.init_done = 0;
qla25xx_delete_queues(base_vha);
qla2x00_free_fcports(base_vha);
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index ab5cf8460aca..7def1f64f1ec 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -2322,8 +2322,8 @@ static int sd_done(struct scsi_cmnd *SCpnt)
}
sdkp->medium_access_timed_out = 0;
- if (!scsi_status_is_check_condition(result) &&
- (!sense_valid || sense_deferred))
+ if (!scsi_status_is_check_condition(result) ||
+ !sense_valid || sense_deferred)
goto out;
switch (sshdr.sense_key) {
diff --git a/drivers/scsi/smartpqi/smartpqi_init.c b/drivers/scsi/smartpqi/smartpqi_init.c
index 4d751a237672..32c92678d591 100644
--- a/drivers/scsi/smartpqi/smartpqi_init.c
+++ b/drivers/scsi/smartpqi/smartpqi_init.c
@@ -65,6 +65,12 @@ static struct pqi_cmd_priv *pqi_cmd_priv(struct scsi_cmnd *cmd)
return scsi_cmd_priv(cmd);
}
+static int pqi_init_cmd_priv(struct Scsi_Host *shost, struct scsi_cmnd *cmd)
+{
+ memset(pqi_cmd_priv(cmd), 0, sizeof(struct pqi_cmd_priv));
+ return 0;
+}
+
static void pqi_verify_structures(void);
static void pqi_take_ctrl_offline(struct pqi_ctrl_info *ctrl_info,
enum pqi_ctrl_shutdown_reason ctrl_shutdown_reason);
@@ -5928,6 +5934,17 @@ void pqi_prep_for_scsi_done(struct scsi_cmnd *scmd)
struct pqi_scsi_dev *device;
struct completion *wait;
+ /*
+ * Clear the AIO-retry marker on final completion so the tag
+ * starts clean on its next dispatch. On DID_IMM_RETRY leave
+ * it intact: pqi_aio_io_complete() sets DID_IMM_RETRY and
+ * bumps the marker to steer the requeue onto the RAID path,
+ * and pqi_process_raid_io_error() consumes the non-zero
+ * marker to offline a misbehaving drive.
+ */
+ if (host_byte(scmd->result) != DID_IMM_RETRY)
+ pqi_cmd_priv(scmd)->this_residual = 0;
+
if (!scmd->device) {
set_host_byte(scmd, DID_NO_CONNECT);
return;
@@ -7582,6 +7599,7 @@ static const struct scsi_host_template pqi_driver_template = {
.sdev_groups = pqi_sdev_groups,
.shost_groups = pqi_shost_groups,
.cmd_size = sizeof(struct pqi_cmd_priv),
+ .init_cmd_priv = pqi_init_cmd_priv,
};
static int pqi_register_scsi(struct pqi_ctrl_info *ctrl_info)
diff --git a/drivers/soc/fsl/qe/qe_ic.c b/drivers/soc/fsl/qe/qe_ic.c
index bbae3d39c7be..f17de6000ff2 100644
--- a/drivers/soc/fsl/qe/qe_ic.c
+++ b/drivers/soc/fsl/qe/qe_ic.c
@@ -481,7 +481,6 @@ static struct platform_driver qe_ic_driver =
static int __init qe_ic_of_init(void)
{
- platform_driver_register(&qe_ic_driver);
- return 0;
+ return platform_driver_register(&qe_ic_driver);
}
subsys_initcall(qe_ic_of_init);
diff --git a/drivers/soc/qcom/rpmh-rsc.c b/drivers/soc/qcom/rpmh-rsc.c
index cc72a31a450e..1b0b7081c951 100644
--- a/drivers/soc/qcom/rpmh-rsc.c
+++ b/drivers/soc/qcom/rpmh-rsc.c
@@ -944,17 +944,30 @@ static int rpmh_rsc_pd_callback(struct notifier_block *nfb,
return NOTIFY_OK;
}
+static void rpmh_rsc_pd_detach(void *data)
+{
+ dev_pm_genpd_remove_notifier(data);
+}
+
static int rpmh_rsc_pd_attach(struct rsc_drv *drv, struct device *dev)
{
int ret;
- pm_runtime_enable(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ return ret;
+
drv->genpd_nb.notifier_call = rpmh_rsc_pd_callback;
ret = dev_pm_genpd_add_notifier(dev, &drv->genpd_nb);
if (ret)
- pm_runtime_disable(dev);
+ return ret;
- return ret;
+ return devm_add_action_or_reset(dev, rpmh_rsc_pd_detach, dev);
+}
+
+static void rpmh_rsc_cpu_pm_unregister(void *data)
+{
+ cpu_pm_unregister_notifier(data);
}
static int rpmh_probe_tcs_config(struct platform_device *pdev, struct rsc_drv *drv)
@@ -1110,7 +1123,15 @@ static int rpmh_rsc_probe(struct platform_device *pdev)
return ret;
} else {
drv->rsc_pm.notifier_call = rpmh_rsc_cpu_pm_callback;
- cpu_pm_register_notifier(&drv->rsc_pm);
+ ret = cpu_pm_register_notifier(&drv->rsc_pm);
+ if (ret)
+ return ret;
+
+ ret = devm_add_action_or_reset(&pdev->dev,
+ rpmh_rsc_cpu_pm_unregister,
+ &drv->rsc_pm);
+ if (ret)
+ return ret;
}
}
@@ -1125,13 +1146,7 @@ static int rpmh_rsc_probe(struct platform_device *pdev)
dev_set_drvdata(&pdev->dev, drv);
drv->dev = &pdev->dev;
- ret = devm_of_platform_populate(&pdev->dev);
- if (ret && pdev->dev.pm_domain) {
- dev_pm_genpd_remove_notifier(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
- }
-
- return ret;
+ return devm_of_platform_populate(&pdev->dev);
}
static const struct of_device_id rpmh_drv_match[] = {
diff --git a/drivers/soc/samsung/exynos-pmu.c b/drivers/soc/samsung/exynos-pmu.c
index 7291c2fb14d7..a294018ed758 100644
--- a/drivers/soc/samsung/exynos-pmu.c
+++ b/drivers/soc/samsung/exynos-pmu.c
@@ -276,8 +276,8 @@ static const struct mfd_cell exynos_pmu_devs[] = {
*/
struct regmap *exynos_get_pmu_regmap(void)
{
- struct device_node *np = of_find_matching_node(NULL,
- exynos_pmu_of_device_ids);
+ struct device_node *np __free(device_node) =
+ of_find_matching_node(NULL, exynos_pmu_of_device_ids);
if (np)
return exynos_get_pmu_regmap_by_phandle(np, NULL);
return ERR_PTR(-ENODEV);
diff --git a/drivers/soundwire/qcom.c b/drivers/soundwire/qcom.c
index aed57002fd0e..db7c8fb04f11 100644
--- a/drivers/soundwire/qcom.c
+++ b/drivers/soundwire/qcom.c
@@ -1204,7 +1204,7 @@ static int qcom_swrm_stream_alloc_ports(struct qcom_swrm_ctrl *ctrl,
else
pn = find_first_zero_bit(port_mask, maxport);
- if (pn > maxport) {
+ if (pn >= maxport) {
dev_err(ctrl->dev, "All ports busy\n");
ret = -EBUSY;
goto out;
diff --git a/drivers/spi/spi-bcm63xx-hsspi.c b/drivers/spi/spi-bcm63xx-hsspi.c
index 8df12efeea21..f5a8edef7b7d 100644
--- a/drivers/spi/spi-bcm63xx-hsspi.c
+++ b/drivers/spi/spi-bcm63xx-hsspi.c
@@ -921,7 +921,13 @@ static int bcm63xx_hsspi_resume(struct device *dev)
}
}
- spi_controller_resume(host);
+ ret = spi_controller_resume(host);
+ if (ret) {
+ if (bs->pll_clk)
+ clk_disable_unprepare(bs->pll_clk);
+ clk_disable_unprepare(bs->clk);
+ return ret;
+ }
return 0;
}
diff --git a/drivers/spi/spi-bcm63xx.c b/drivers/spi/spi-bcm63xx.c
index 746a61095ad4..6f7c4ab0057d 100644
--- a/drivers/spi/spi-bcm63xx.c
+++ b/drivers/spi/spi-bcm63xx.c
@@ -661,7 +661,11 @@ static int bcm63xx_spi_resume(struct device *dev)
if (ret)
return ret;
- spi_controller_resume(host);
+ ret = spi_controller_resume(host);
+ if (ret) {
+ clk_disable_unprepare(bs->clk);
+ return ret;
+ }
return 0;
}
diff --git a/drivers/spi/spi-bcmbca-hsspi.c b/drivers/spi/spi-bcmbca-hsspi.c
index 95fb181408e0..c666172f2b05 100644
--- a/drivers/spi/spi-bcmbca-hsspi.c
+++ b/drivers/spi/spi-bcmbca-hsspi.c
@@ -612,7 +612,13 @@ static int bcmbca_hsspi_resume(struct device *dev)
}
}
- spi_controller_resume(host);
+ ret = spi_controller_resume(host);
+ if (ret) {
+ if (bs->pll_clk)
+ clk_disable_unprepare(bs->pll_clk);
+ clk_disable_unprepare(bs->clk);
+ return ret;
+ }
return 0;
}
diff --git a/drivers/spi/spi-davinci.c b/drivers/spi/spi-davinci.c
index ad26c8409733..cfe4f8edfcbd 100644
--- a/drivers/spi/spi-davinci.c
+++ b/drivers/spi/spi-davinci.c
@@ -868,7 +868,7 @@ static int davinci_spi_probe(struct platform_device *pdev)
int ret = 0;
u32 spipc0;
- host = spi_alloc_host(&pdev->dev, sizeof(struct davinci_spi));
+ host = devm_spi_alloc_host(&pdev->dev, sizeof(struct davinci_spi));
if (host == NULL) {
ret = -ENOMEM;
goto err;
@@ -998,7 +998,6 @@ static int davinci_spi_probe(struct platform_device *pdev)
dma_release_channel(dspi->dma_tx);
}
free_host:
- spi_controller_put(host);
err:
return ret;
}
@@ -1022,6 +1021,8 @@ static void davinci_spi_remove(struct platform_device *pdev)
spi_bitbang_stop(&dspi->bitbang);
+ devm_free_irq(&pdev->dev, dspi->irq, dspi);
+
/* This bit needs to be cleared to disable dpsi->clk */
clear_io_bits(dspi->base + SPIGCR1, SPIGCR1_POWERDOWN_MASK);
@@ -1029,8 +1030,6 @@ static void davinci_spi_remove(struct platform_device *pdev)
dma_release_channel(dspi->dma_rx);
dma_release_channel(dspi->dma_tx);
}
-
- spi_controller_put(host);
}
static struct platform_driver davinci_spi_driver = {
diff --git a/drivers/spi/spi-img-spfi.c b/drivers/spi/spi-img-spfi.c
index 1e2a8cf9290f..07fd38ae959e 100644
--- a/drivers/spi/spi-img-spfi.c
+++ b/drivers/spi/spi-img-spfi.c
@@ -616,7 +616,7 @@ static int img_spfi_probe(struct platform_device *pdev)
ret = PTR_ERR(spfi->tx_ch);
spfi->tx_ch = NULL;
if (ret == -EPROBE_DEFER)
- goto disable_pm;
+ goto free_dma;
}
spfi->rx_ch = dma_request_chan(spfi->dev, "rx");
@@ -624,7 +624,7 @@ static int img_spfi_probe(struct platform_device *pdev)
ret = PTR_ERR(spfi->rx_ch);
spfi->rx_ch = NULL;
if (ret == -EPROBE_DEFER)
- goto disable_pm;
+ goto free_dma;
}
if (!spfi->tx_ch || !spfi->rx_ch) {
@@ -652,6 +652,7 @@ static int img_spfi_probe(struct platform_device *pdev)
disable_pm:
pm_runtime_disable(spfi->dev);
+free_dma:
if (spfi->rx_ch)
dma_release_channel(spfi->rx_ch);
if (spfi->tx_ch)
diff --git a/drivers/spi/spi-oc-tiny.c b/drivers/spi/spi-oc-tiny.c
index 7d8c5cd680d1..2e5df685c277 100644
--- a/drivers/spi/spi-oc-tiny.c
+++ b/drivers/spi/spi-oc-tiny.c
@@ -211,11 +211,11 @@ static int tiny_spi_probe(struct platform_device *pdev)
struct tiny_spi_platform_data *platp = dev_get_platdata(&pdev->dev);
struct tiny_spi *hw;
struct spi_controller *host;
- int err = -ENODEV;
+ int err;
- host = spi_alloc_host(&pdev->dev, sizeof(struct tiny_spi));
+ host = devm_spi_alloc_host(&pdev->dev, sizeof(struct tiny_spi));
if (!host)
- return err;
+ return -ENOMEM;
/* setup the host state. */
host->bus_num = pdev->id;
@@ -233,10 +233,8 @@ static int tiny_spi_probe(struct platform_device *pdev)
/* find and map our resources */
hw->base = devm_platform_ioremap_resource(pdev, 0);
- if (IS_ERR(hw->base)) {
- err = PTR_ERR(hw->base);
- goto exit;
- }
+ if (IS_ERR(hw->base))
+ return PTR_ERR(hw->base);
/* irq is optional */
hw->irq = platform_get_irq(pdev, 0);
if (hw->irq >= 0) {
@@ -244,7 +242,7 @@ static int tiny_spi_probe(struct platform_device *pdev)
err = devm_request_irq(&pdev->dev, hw->irq, tiny_spi_irq, 0,
pdev->name, hw);
if (err)
- goto exit;
+ return err;
}
/* find platform data */
if (platp) {
@@ -253,29 +251,23 @@ static int tiny_spi_probe(struct platform_device *pdev)
} else {
err = tiny_spi_of_probe(pdev);
if (err)
- goto exit;
+ return err;
}
/* register our spi controller */
err = spi_bitbang_start(&hw->bitbang);
if (err)
- goto exit;
+ return err;
dev_info(&pdev->dev, "base %p, irq %d\n", hw->base, hw->irq);
return 0;
-
-exit:
- spi_controller_put(host);
- return err;
}
static void tiny_spi_remove(struct platform_device *pdev)
{
struct tiny_spi *hw = platform_get_drvdata(pdev);
- struct spi_controller *host = hw->bitbang.ctlr;
spi_bitbang_stop(&hw->bitbang);
- spi_controller_put(host);
}
#ifdef CONFIG_OF
diff --git a/drivers/spi/spi-sprd-adi.c b/drivers/spi/spi-sprd-adi.c
index f25b34a91756..72f2f30e657e 100644
--- a/drivers/spi/spi-sprd-adi.c
+++ b/drivers/spi/spi-sprd-adi.c
@@ -553,6 +553,12 @@ static int sprd_adi_probe(struct platform_device *pdev)
return -ENXIO;
} else {
switch (ret) {
+ case 0:
+ /*
+ * Only reachable with CONFIG_HWSPINLOCK=n, where the
+ * of_hwspin_lock_get_id() stub returns 0.
+ */
+ fallthrough;
case -ENOENT:
dev_info(&pdev->dev, "no hardware spinlock supplied\n");
break;
diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c
index 319d3834476e..caf445d4a435 100644
--- a/drivers/spi/spi.c
+++ b/drivers/spi/spi.c
@@ -1229,6 +1229,8 @@ void spi_unmap_buf(struct spi_controller *ctlr, struct device *dev,
spi_unmap_buf_attrs(ctlr, dev, sgt, dir, 0);
}
+static int __spi_unmap_msg(struct spi_controller *ctlr, struct spi_message *msg);
+
static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg)
{
struct device *tx_dev, *rx_dev;
@@ -1252,7 +1254,13 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg)
else
rx_dev = ctlr->dev.parent;
- ret = -ENOMSG;
+ /*
+ * Store the devices before mapping so partial failures can be unwound
+ * with the device that created each mapping.
+ */
+ ctlr->cur_tx_dma_dev = tx_dev;
+ ctlr->cur_rx_dma_dev = rx_dev;
+
list_for_each_entry(xfer, &msg->transfers, transfer_list) {
/* The sync is done before each transfer. */
unsigned long attrs = DMA_ATTR_SKIP_CPU_SYNC;
@@ -1265,8 +1273,8 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg)
(void *)xfer->tx_buf,
xfer->len, DMA_TO_DEVICE,
attrs);
- if (ret != 0)
- return ret;
+ if (ret)
+ goto unwind;
xfer->tx_sg_mapped = true;
}
@@ -1275,25 +1283,19 @@ static int __spi_map_msg(struct spi_controller *ctlr, struct spi_message *msg)
ret = spi_map_buf_attrs(ctlr, rx_dev, &xfer->rx_sg,
xfer->rx_buf, xfer->len,
DMA_FROM_DEVICE, attrs);
- if (ret != 0) {
- spi_unmap_buf_attrs(ctlr, tx_dev,
- &xfer->tx_sg, DMA_TO_DEVICE,
- attrs);
-
- return ret;
- }
+ if (ret)
+ goto unwind;
xfer->rx_sg_mapped = true;
}
}
- /* No transfer has been mapped, bail out with success */
- if (ret)
- return 0;
-
- ctlr->cur_rx_dma_dev = rx_dev;
- ctlr->cur_tx_dma_dev = tx_dev;
return 0;
+
+unwind:
+ __spi_unmap_msg(ctlr, msg);
+
+ return ret;
}
static int __spi_unmap_msg(struct spi_controller *ctlr, struct spi_message *msg)
diff --git a/drivers/staging/fbtft/fbtft-sysfs.c b/drivers/staging/fbtft/fbtft-sysfs.c
index e45c90a03a90..7bd6cbf2f1e5 100644
--- a/drivers/staging/fbtft/fbtft-sysfs.c
+++ b/drivers/staging/fbtft/fbtft-sysfs.c
@@ -98,7 +98,7 @@ sprintf_gamma(struct fbtft_par *par, u32 *curves, char *buf)
mutex_lock(&par->gamma.lock);
for (i = 0; i < par->gamma.num_curves; i++) {
for (j = 0; j < par->gamma.num_values; j++)
- len += scnprintf(&buf[len], PAGE_SIZE,
+ len += sysfs_emit_at(buf, len,
"%04x ", curves[i * par->gamma.num_values + j]);
buf[len - 1] = '\n';
}
diff --git a/drivers/staging/greybus/audio_gb.c b/drivers/staging/greybus/audio_gb.c
index 9d8994fdb41a..144591f1a512 100644
--- a/drivers/staging/greybus/audio_gb.c
+++ b/drivers/staging/greybus/audio_gb.c
@@ -37,6 +37,19 @@ int gb_audio_gb_get_topology(struct gb_connection *connection,
return ret;
}
+ /*
+ * The size_* fields are supplied by the module and are used by
+ * gbaudio_tplg_parse_data() to compute offsets into the blob; make
+ * sure the sections fit within the fetched topology, so walking it
+ * cannot read out of bounds.
+ */
+ if ((u64)le32_to_cpu(topo->size_dais) + le32_to_cpu(topo->size_controls) +
+ le32_to_cpu(topo->size_widgets) + le32_to_cpu(topo->size_routes) >
+ size - sizeof(*topo)) {
+ kfree(topo);
+ return -EINVAL;
+ }
+
*topology = topo;
return 0;
diff --git a/drivers/staging/greybus/audio_manager_sysfs.c b/drivers/staging/greybus/audio_manager_sysfs.c
index fcd518f9540c..ff323ca8154f 100644
--- a/drivers/staging/greybus/audio_manager_sysfs.c
+++ b/drivers/staging/greybus/audio_manager_sysfs.c
@@ -23,7 +23,7 @@ static ssize_t manager_sysfs_add_store(struct kobject *kobj,
desc.name, &desc.vid, &desc.pid, &desc.intf_id,
&desc.ip_devices, &desc.op_devices);
- if (num != 7)
+ if (num != 6)
return -EINVAL;
num = gb_audio_manager_add(&desc);
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index e92d70e91fb7..fffafc77c8b6 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -506,8 +506,8 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
fmt_out = find_format(fmts, size, pixmp->pixelformat);
if (!fmt_out) {
- pixmp->pixelformat = V4L2_PIX_FMT_MPEG2;
- fmt_out = find_format(fmts, size, pixmp->pixelformat);
+ pixmp->pixelformat = fmts[0].pixfmt;
+ fmt_out = &fmts[0];
}
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index 358481d542eb..3c988039e0d0 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -285,8 +285,10 @@ static int cedrus_init_ctrls(struct cedrus_dev *dev, struct cedrus_ctx *ctx)
ctrl_size = sizeof(ctrl) * CEDRUS_CONTROLS_COUNT + 1;
ctx->ctrls = kzalloc(ctrl_size, GFP_KERNEL);
- if (!ctx->ctrls)
+ if (!ctx->ctrls) {
+ v4l2_ctrl_handler_free(hdl);
return -ENOMEM;
+ }
j = 0;
for (i = 0; i < CEDRUS_CONTROLS_COUNT; i++) {
diff --git a/drivers/staging/octeon/ethernet-mem.c b/drivers/staging/octeon/ethernet-mem.c
index 532594957ebc..68c3ef984e56 100644
--- a/drivers/staging/octeon/ethernet-mem.c
+++ b/drivers/staging/octeon/ethernet-mem.c
@@ -5,6 +5,7 @@
* Copyright (c) 2003-2010 Cavium Networks
*/
+#include <linux/platform_device.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/slab.h>
@@ -40,11 +41,13 @@ static int cvm_oct_fill_hw_skbuff(int pool, int size, int elements)
/**
* cvm_oct_free_hw_skbuff- free hardware pool skbuffs
+ * @pdev: Platform device for logging
* @pool: Pool to allocate an skbuff for
* @size: Size of the buffer needed for the pool
* @elements: Number of buffers to allocate
*/
-static void cvm_oct_free_hw_skbuff(int pool, int size, int elements)
+static void cvm_oct_free_hw_skbuff(struct platform_device *pdev,
+ int pool, int size, int elements)
{
char *memory;
@@ -59,22 +62,24 @@ static void cvm_oct_free_hw_skbuff(int pool, int size, int elements)
} while (memory);
if (elements < 0)
- pr_warn("Freeing of pool %u had too many skbuffs (%d)\n",
- pool, elements);
+ dev_warn(&pdev->dev, "Freeing of pool %u had too many skbuffs (%d)\n",
+ pool, elements);
else if (elements > 0)
- pr_warn("Freeing of pool %u is missing %d skbuffs\n",
- pool, elements);
+ dev_warn(&pdev->dev, "Freeing of pool %u is missing %d skbuffs\n",
+ pool, elements);
}
/**
* cvm_oct_fill_hw_memory - fill a hardware pool with memory.
+ * @pdev: Platform device for logging
* @pool: Pool to populate
* @size: Size of each buffer in the pool
* @elements: Number of buffers to allocate
*
* Returns the actual number of buffers allocated.
*/
-static int cvm_oct_fill_hw_memory(int pool, int size, int elements)
+static int cvm_oct_fill_hw_memory(struct platform_device *pdev, int pool, int size,
+ int elements)
{
char *memory;
char *fpa;
@@ -93,8 +98,8 @@ static int cvm_oct_fill_hw_memory(int pool, int size, int elements)
*/
memory = kmalloc(size + 256, GFP_ATOMIC);
if (unlikely(!memory)) {
- pr_warn("Unable to allocate %u bytes for FPA pool %d\n",
- elements * size, pool);
+ dev_warn(&pdev->dev, "Unable to allocate %u bytes for FPA pool %d\n",
+ elements * size, pool);
break;
}
fpa = (char *)(((unsigned long)memory + 256) & ~0x7fUL);
@@ -107,11 +112,13 @@ static int cvm_oct_fill_hw_memory(int pool, int size, int elements)
/**
* cvm_oct_free_hw_memory - Free memory allocated by cvm_oct_fill_hw_memory
+ * @pdev: Platform device for logging
* @pool: FPA pool to free
* @size: Size of each buffer in the pool
* @elements: Number of buffers that should be in the pool
*/
-static void cvm_oct_free_hw_memory(int pool, int size, int elements)
+static void cvm_oct_free_hw_memory(struct platform_device *pdev,
+ int pool, int size, int elements)
{
char *memory;
char *fpa;
@@ -127,28 +134,28 @@ static void cvm_oct_free_hw_memory(int pool, int size, int elements)
} while (fpa);
if (elements < 0)
- pr_warn("Freeing of pool %u had too many buffers (%d)\n",
- pool, elements);
+ dev_warn(&pdev->dev, "Freeing of pool %u had too many buffers (%d)\n",
+ pool, elements);
else if (elements > 0)
- pr_warn("Warning: Freeing of pool %u is missing %d buffers\n",
- pool, elements);
+ dev_warn(&pdev->dev, "Freeing of pool %u is missing %d buffers\n",
+ pool, elements);
}
-int cvm_oct_mem_fill_fpa(int pool, int size, int elements)
+int cvm_oct_mem_fill_fpa(struct platform_device *pdev, int pool, int size, int elements)
{
int freed;
if (pool == CVMX_FPA_PACKET_POOL)
freed = cvm_oct_fill_hw_skbuff(pool, size, elements);
else
- freed = cvm_oct_fill_hw_memory(pool, size, elements);
+ freed = cvm_oct_fill_hw_memory(pdev, pool, size, elements);
return freed;
}
-void cvm_oct_mem_empty_fpa(int pool, int size, int elements)
+void cvm_oct_mem_empty_fpa(struct platform_device *pdev, int pool, int size, int elements)
{
if (pool == CVMX_FPA_PACKET_POOL)
- cvm_oct_free_hw_skbuff(pool, size, elements);
+ cvm_oct_free_hw_skbuff(pdev, pool, size, elements);
else
- cvm_oct_free_hw_memory(pool, size, elements);
+ cvm_oct_free_hw_memory(pdev, pool, size, elements);
}
diff --git a/drivers/staging/octeon/ethernet-mem.h b/drivers/staging/octeon/ethernet-mem.h
index 692dcdb7154d..9279bb0de2db 100644
--- a/drivers/staging/octeon/ethernet-mem.h
+++ b/drivers/staging/octeon/ethernet-mem.h
@@ -5,5 +5,9 @@
* Copyright (c) 2003-2007 Cavium Networks
*/
-int cvm_oct_mem_fill_fpa(int pool, int size, int elements);
-void cvm_oct_mem_empty_fpa(int pool, int size, int elements);
+struct platform_device;
+
+int cvm_oct_mem_fill_fpa(struct platform_device *pdev, int pool, int size,
+ int elements);
+void cvm_oct_mem_empty_fpa(struct platform_device *pdev, int pool, int size,
+ int elements);
diff --git a/drivers/staging/octeon/ethernet-rx.c b/drivers/staging/octeon/ethernet-rx.c
index 965330eec80a..3e9d58d32156 100644
--- a/drivers/staging/octeon/ethernet-rx.c
+++ b/drivers/staging/octeon/ethernet-rx.c
@@ -5,6 +5,7 @@
* Copyright (c) 2003-2010 Cavium Networks
*/
+#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/cache.h>
@@ -31,12 +32,6 @@
static atomic_t oct_rx_ready = ATOMIC_INIT(0);
-static struct oct_rx_group {
- int irq;
- int group;
- struct napi_struct napi;
-} oct_rx_group[16];
-
/**
* cvm_oct_do_interrupt - interrupt handler.
* @irq: Interrupt number.
@@ -397,7 +392,7 @@ static int cvm_oct_poll(struct oct_rx_group *rx_group, int budget)
/* Restore the scratch area */
cvmx_scratch_write64(CVMX_SCR_SCRATCH, old_scratch);
}
- cvm_oct_rx_refill_pool(0);
+ cvm_oct_rx_refill_pool(rx_group->pdev, 0);
return rx_count;
}
@@ -434,24 +429,28 @@ static int cvm_oct_napi_poll(struct napi_struct *napi, int budget)
*/
void cvm_oct_poll_controller(struct net_device *dev)
{
+ struct platform_device *pdev = to_platform_device(dev->dev.parent);
+ struct octeon_ethernet_platform *plat = platform_get_drvdata(pdev);
int i;
if (!atomic_read(&oct_rx_ready))
return;
- for (i = 0; i < ARRAY_SIZE(oct_rx_group); i++) {
+ for (i = 0; i < ARRAY_SIZE(plat->rx_group); i++) {
if (!(pow_receive_groups & BIT(i)))
continue;
- cvm_oct_poll(&oct_rx_group[i], 16);
+ cvm_oct_poll(&plat->rx_group[i], 16);
}
}
#endif
-void cvm_oct_rx_initialize(void)
+void cvm_oct_rx_initialize(struct platform_device *pdev)
{
int i;
struct net_device *dev_for_napi = NULL;
+ struct octeon_ethernet_platform *plat = platform_get_drvdata(pdev);
+ struct oct_rx_group *rx_group = plat->rx_group;
for (i = 0; i < TOTAL_NUMBER_OF_PORTS; i++) {
if (cvm_oct_device[i]) {
@@ -463,27 +462,28 @@ void cvm_oct_rx_initialize(void)
if (!dev_for_napi)
panic("No net_devices were allocated.");
- for (i = 0; i < ARRAY_SIZE(oct_rx_group); i++) {
+ for (i = 0; i < ARRAY_SIZE(plat->rx_group); i++) {
int ret;
if (!(pow_receive_groups & BIT(i)))
continue;
- netif_napi_add_weight(dev_for_napi, &oct_rx_group[i].napi,
+ netif_napi_add_weight(dev_for_napi, &rx_group[i].napi,
cvm_oct_napi_poll, rx_napi_weight);
- napi_enable(&oct_rx_group[i].napi);
+ napi_enable(&rx_group[i].napi);
- oct_rx_group[i].irq = OCTEON_IRQ_WORKQ0 + i;
- oct_rx_group[i].group = i;
+ rx_group[i].irq = OCTEON_IRQ_WORKQ0 + i;
+ rx_group[i].group = i;
+ rx_group[i].pdev = pdev;
/* Register an IRQ handler to receive POW interrupts */
- ret = request_irq(oct_rx_group[i].irq, cvm_oct_do_interrupt, 0,
- "Ethernet", &oct_rx_group[i].napi);
+ ret = request_irq(rx_group[i].irq, cvm_oct_do_interrupt, 0,
+ "Ethernet", &rx_group[i].napi);
if (ret)
panic("Could not acquire Ethernet IRQ %d\n",
- oct_rx_group[i].irq);
+ rx_group[i].irq);
- disable_irq_nosync(oct_rx_group[i].irq);
+ disable_irq_nosync(rx_group[i].irq);
/* Enable POW interrupt when our port has at least one packet */
if (OCTEON_IS_MODEL(OCTEON_CN68XX)) {
@@ -515,16 +515,17 @@ void cvm_oct_rx_initialize(void)
/* Schedule NAPI now. This will indirectly enable the
* interrupt.
*/
- napi_schedule(&oct_rx_group[i].napi);
+ napi_schedule(&rx_group[i].napi);
}
atomic_inc(&oct_rx_ready);
}
-void cvm_oct_rx_shutdown(void)
+void cvm_oct_rx_shutdown(struct platform_device *pdev)
{
+ struct octeon_ethernet_platform *plat = platform_get_drvdata(pdev);
int i;
- for (i = 0; i < ARRAY_SIZE(oct_rx_group); i++) {
+ for (i = 0; i < ARRAY_SIZE(plat->rx_group); i++) {
if (!(pow_receive_groups & BIT(i)))
continue;
@@ -534,9 +535,11 @@ void cvm_oct_rx_shutdown(void)
else
cvmx_write_csr(CVMX_POW_WQ_INT_THRX(i), 0);
+ napi_disable(&plat->rx_group[i].napi);
+
/* Free the interrupt handler */
- free_irq(oct_rx_group[i].irq, cvm_oct_device);
+ free_irq(plat->rx_group[i].irq, &plat->rx_group[i].napi);
- netif_napi_del(&oct_rx_group[i].napi);
+ netif_napi_del(&plat->rx_group[i].napi);
}
}
diff --git a/drivers/staging/octeon/ethernet-rx.h b/drivers/staging/octeon/ethernet-rx.h
index ff6482fa20d6..6093694326cb 100644
--- a/drivers/staging/octeon/ethernet-rx.h
+++ b/drivers/staging/octeon/ethernet-rx.h
@@ -5,11 +5,14 @@
* Copyright (c) 2003-2007 Cavium Networks
*/
+struct platform_device;
+
void cvm_oct_poll_controller(struct net_device *dev);
-void cvm_oct_rx_initialize(void);
-void cvm_oct_rx_shutdown(void);
+void cvm_oct_rx_initialize(struct platform_device *pdev);
+void cvm_oct_rx_shutdown(struct platform_device *pdev);
-static inline void cvm_oct_rx_refill_pool(int fill_threshold)
+static inline void cvm_oct_rx_refill_pool(struct platform_device *pdev,
+ int fill_threshold)
{
int number_to_free;
int num_freed;
@@ -20,7 +23,7 @@ static inline void cvm_oct_rx_refill_pool(int fill_threshold)
if (number_to_free > fill_threshold) {
cvmx_fau_atomic_add32(FAU_NUM_PACKET_BUFFERS_TO_FREE,
-number_to_free);
- num_freed = cvm_oct_mem_fill_fpa(CVMX_FPA_PACKET_POOL,
+ num_freed = cvm_oct_mem_fill_fpa(pdev, CVMX_FPA_PACKET_POOL,
CVMX_FPA_PACKET_POOL_SIZE,
number_to_free);
if (num_freed != number_to_free) {
diff --git a/drivers/staging/octeon/ethernet-tx.c b/drivers/staging/octeon/ethernet-tx.c
index bbf33b88bb7c..e54b4a413dc4 100644
--- a/drivers/staging/octeon/ethernet-tx.c
+++ b/drivers/staging/octeon/ethernet-tx.c
@@ -711,4 +711,6 @@ void cvm_oct_tx_shutdown(void)
{
/* Free the interrupt handler */
free_irq(OCTEON_IRQ_TIMER1, cvm_oct_device);
+
+ tasklet_kill(&cvm_oct_tx_cleanup_tasklet);
}
diff --git a/drivers/staging/octeon/ethernet.c b/drivers/staging/octeon/ethernet.c
index a5e99cc78a45..b8a528881fca 100644
--- a/drivers/staging/octeon/ethernet.c
+++ b/drivers/staging/octeon/ethernet.c
@@ -104,11 +104,10 @@ struct net_device *cvm_oct_device[TOTAL_NUMBER_OF_PORTS];
u64 cvm_oct_tx_poll_interval;
-static void cvm_oct_rx_refill_worker(struct work_struct *work);
-static DECLARE_DELAYED_WORK(cvm_oct_rx_refill_work, cvm_oct_rx_refill_worker);
-
static void cvm_oct_rx_refill_worker(struct work_struct *work)
{
+ struct octeon_ethernet_platform *plat = container_of(work,
+ struct octeon_ethernet_platform, rx_refill_work.work);
/*
* FPA 0 may have been drained, try to refill it if we need
* more than num_packet_buffers / 2, otherwise normal receive
@@ -116,10 +115,10 @@ static void cvm_oct_rx_refill_worker(struct work_struct *work)
* could be received so cvm_oct_napi_poll would never be
* invoked to do the refill.
*/
- cvm_oct_rx_refill_pool(num_packet_buffers / 2);
+ cvm_oct_rx_refill_pool(plat->pdev, num_packet_buffers / 2);
if (!atomic_read(&cvm_oct_poll_queue_stopping))
- schedule_delayed_work(&cvm_oct_rx_refill_work, HZ);
+ schedule_delayed_work(&plat->rx_refill_work, HZ);
}
static void cvm_oct_periodic_worker(struct work_struct *work)
@@ -138,16 +137,16 @@ static void cvm_oct_periodic_worker(struct work_struct *work)
schedule_delayed_work(&priv->port_periodic_work, HZ);
}
-static void cvm_oct_configure_common_hw(void)
+static void cvm_oct_configure_common_hw(struct platform_device *pdev)
{
/* Setup the FPA */
cvmx_fpa_enable();
- cvm_oct_mem_fill_fpa(CVMX_FPA_PACKET_POOL, CVMX_FPA_PACKET_POOL_SIZE,
+ cvm_oct_mem_fill_fpa(pdev, CVMX_FPA_PACKET_POOL, CVMX_FPA_PACKET_POOL_SIZE,
num_packet_buffers);
- cvm_oct_mem_fill_fpa(CVMX_FPA_WQE_POOL, CVMX_FPA_WQE_POOL_SIZE,
+ cvm_oct_mem_fill_fpa(pdev, CVMX_FPA_WQE_POOL, CVMX_FPA_WQE_POOL_SIZE,
num_packet_buffers);
if (CVMX_FPA_OUTPUT_BUFFER_POOL != CVMX_FPA_PACKET_POOL)
- cvm_oct_mem_fill_fpa(CVMX_FPA_OUTPUT_BUFFER_POOL,
+ cvm_oct_mem_fill_fpa(pdev, CVMX_FPA_OUTPUT_BUFFER_POOL,
CVMX_FPA_OUTPUT_BUFFER_POOL_SIZE, 1024);
#ifdef __LITTLE_ENDIAN
@@ -678,6 +677,15 @@ static int cvm_oct_probe(struct platform_device *pdev)
int qos;
struct device_node *pip;
int mtu_overhead = ETH_HLEN + ETH_FCS_LEN;
+ struct octeon_ethernet_platform *plat;
+
+ plat = devm_kzalloc(&pdev->dev, sizeof(*plat), GFP_KERNEL);
+ if (!plat)
+ return -ENOMEM;
+
+ plat->pdev = pdev;
+ INIT_DELAYED_WORK(&plat->rx_refill_work, cvm_oct_rx_refill_worker);
+ platform_set_drvdata(pdev, plat);
#if IS_ENABLED(CONFIG_VLAN_8021Q)
mtu_overhead += VLAN_HLEN;
@@ -689,7 +697,7 @@ static int cvm_oct_probe(struct platform_device *pdev)
return -EINVAL;
}
- cvm_oct_configure_common_hw();
+ cvm_oct_configure_common_hw(pdev);
cvmx_helper_initialize_packet_io_global();
@@ -912,28 +920,29 @@ static int cvm_oct_probe(struct platform_device *pdev)
}
cvm_oct_tx_initialize();
- cvm_oct_rx_initialize();
+ cvm_oct_rx_initialize(pdev);
/*
* 150 uS: about 10 1500-byte packets at 1GE.
*/
cvm_oct_tx_poll_interval = 150 * (octeon_get_clock_rate() / 1000000);
- schedule_delayed_work(&cvm_oct_rx_refill_work, HZ);
+ schedule_delayed_work(&plat->rx_refill_work, HZ);
return 0;
}
static void cvm_oct_remove(struct platform_device *pdev)
{
+ struct octeon_ethernet_platform *plat = platform_get_drvdata(pdev);
int port;
cvmx_ipd_disable();
atomic_inc_return(&cvm_oct_poll_queue_stopping);
- cancel_delayed_work_sync(&cvm_oct_rx_refill_work);
+ cancel_delayed_work_sync(&plat->rx_refill_work);
- cvm_oct_rx_shutdown();
+ cvm_oct_rx_shutdown(pdev);
cvm_oct_tx_shutdown();
cvmx_pko_disable();
@@ -958,12 +967,12 @@ static void cvm_oct_remove(struct platform_device *pdev)
cvmx_ipd_free_ptr();
/* Free the HW pools */
- cvm_oct_mem_empty_fpa(CVMX_FPA_PACKET_POOL, CVMX_FPA_PACKET_POOL_SIZE,
+ cvm_oct_mem_empty_fpa(pdev, CVMX_FPA_PACKET_POOL, CVMX_FPA_PACKET_POOL_SIZE,
num_packet_buffers);
- cvm_oct_mem_empty_fpa(CVMX_FPA_WQE_POOL, CVMX_FPA_WQE_POOL_SIZE,
+ cvm_oct_mem_empty_fpa(pdev, CVMX_FPA_WQE_POOL, CVMX_FPA_WQE_POOL_SIZE,
num_packet_buffers);
if (CVMX_FPA_OUTPUT_BUFFER_POOL != CVMX_FPA_PACKET_POOL)
- cvm_oct_mem_empty_fpa(CVMX_FPA_OUTPUT_BUFFER_POOL,
+ cvm_oct_mem_empty_fpa(pdev, CVMX_FPA_OUTPUT_BUFFER_POOL,
CVMX_FPA_OUTPUT_BUFFER_POOL_SIZE, 128);
}
diff --git a/drivers/staging/octeon/octeon-ethernet.h b/drivers/staging/octeon/octeon-ethernet.h
index a6140705706f..0ac430db1e6e 100644
--- a/drivers/staging/octeon/octeon-ethernet.h
+++ b/drivers/staging/octeon/octeon-ethernet.h
@@ -11,6 +11,7 @@
#ifndef OCTEON_ETHERNET_H
#define OCTEON_ETHERNET_H
+#include <linux/netdevice.h>
#include <linux/of.h>
#include <linux/phy.h>
@@ -74,6 +75,19 @@ struct octeon_ethernet {
struct device_node *of_node;
};
+struct oct_rx_group {
+ int irq;
+ int group;
+ struct napi_struct napi;
+ struct platform_device *pdev;
+};
+
+struct octeon_ethernet_platform {
+ struct platform_device *pdev;
+ struct delayed_work rx_refill_work;
+ struct oct_rx_group rx_group[16];
+};
+
int cvm_oct_free_work(void *work_queue_entry);
int cvm_oct_rgmii_open(struct net_device *dev);
diff --git a/drivers/staging/rtl8723bs/core/rtw_cmd.c b/drivers/staging/rtl8723bs/core/rtw_cmd.c
index 84ce7307d8f3..e9a95a2733ad 100644
--- a/drivers/staging/rtl8723bs/core/rtw_cmd.c
+++ b/drivers/staging/rtl8723bs/core/rtw_cmd.c
@@ -894,7 +894,7 @@ u8 rtw_setstakey_cmd(struct adapter *padapter, struct sta_info *sta, u8 unicast_
if (enqueue) {
ph2c = rtw_zmalloc(sizeof(struct cmd_obj));
if (!ph2c) {
- kfree(psetstakey_para);
+ kfree_sensitive(psetstakey_para);
res = _FAIL;
goto exit;
}
@@ -902,7 +902,7 @@ u8 rtw_setstakey_cmd(struct adapter *padapter, struct sta_info *sta, u8 unicast_
psetstakey_rsp = rtw_zmalloc(sizeof(struct set_stakey_rsp));
if (!psetstakey_rsp) {
kfree(ph2c);
- kfree(psetstakey_para);
+ kfree_sensitive(psetstakey_para);
res = _FAIL;
goto exit;
}
@@ -913,7 +913,7 @@ u8 rtw_setstakey_cmd(struct adapter *padapter, struct sta_info *sta, u8 unicast_
res = rtw_enqueue_cmd(pcmdpriv, ph2c);
} else {
set_stakey_hdl(padapter, (u8 *)psetstakey_para);
- kfree(psetstakey_para);
+ kfree_sensitive(psetstakey_para);
}
exit:
return res;
@@ -953,7 +953,7 @@ u8 rtw_clearstakey_cmd(struct adapter *padapter, struct sta_info *sta, u8 enqueu
psetstakey_rsp = rtw_zmalloc(sizeof(struct set_stakey_rsp));
if (!psetstakey_rsp) {
kfree(ph2c);
- kfree(psetstakey_para);
+ kfree_sensitive(psetstakey_para);
res = _FAIL;
goto exit;
}
diff --git a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
index 56938ff47235..96a149239e85 100644
--- a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
+++ b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c
@@ -746,6 +746,10 @@ u8 *rtw_get_wps_attr(u8 *wps_ie, uint wps_ielen, u16 target_attr_id, u8 *buf_att
u16 attr_data_len = get_unaligned_be16(attr_ptr + 2);
u16 attr_len = attr_data_len + 4;
+ /* Reject attributes whose claimed length runs past the IE */
+ if (attr_ptr + attr_len > wps_ie + wps_ielen)
+ break;
+
if (attr_id == target_attr_id) {
target_attr_ptr = attr_ptr;
@@ -1149,6 +1153,9 @@ int rtw_action_frame_parse(const u8 *frame, u32 frame_len, u8 *category, u8 *act
u8 c;
u8 a = ACT_PUBLIC_MAX;
+ if (frame_len < sizeof(struct ieee80211_hdr_3addr) + 2)
+ return false;
+
fc = le16_to_cpu(((struct ieee80211_hdr_3addr *)frame)->frame_control);
if ((fc & (IEEE80211_FCTL_FTYPE|IEEE80211_FCTL_STYPE))
diff --git a/drivers/staging/rtl8723bs/os_dep/sdio_intf.c b/drivers/staging/rtl8723bs/os_dep/sdio_intf.c
index d64998be6b2b..c9a90d5293b0 100644
--- a/drivers/staging/rtl8723bs/os_dep/sdio_intf.c
+++ b/drivers/staging/rtl8723bs/os_dep/sdio_intf.c
@@ -309,8 +309,8 @@ static struct adapter *rtw_sdio_if1_init(struct dvobj_priv *dvobj, const struct
status = _SUCCESS;
free_hal_data:
- if (status != _SUCCESS && padapter->HalData)
- kfree(padapter->HalData);
+ if (status != _SUCCESS)
+ rtw_hal_data_deinit(padapter);
if (status != _SUCCESS) {
rtw_wdev_unregister(padapter->rtw_wdev);
diff --git a/drivers/staging/sm750fb/Kconfig b/drivers/staging/sm750fb/Kconfig
index 08bcccdd0f1c..25fe422f55f2 100644
--- a/drivers/staging/sm750fb/Kconfig
+++ b/drivers/staging/sm750fb/Kconfig
@@ -6,6 +6,7 @@ config FB_SM750
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select FB_IOMEM_FOPS
help
Frame buffer driver for the Silicon Motion SM750 chip
with 2D acceleration and dual head support.
diff --git a/drivers/staging/sm750fb/sm750.c b/drivers/staging/sm750fb/sm750.c
index 94a83ec34282..1452edab96cc 100644
--- a/drivers/staging/sm750fb/sm750.c
+++ b/drivers/staging/sm750fb/sm750.c
@@ -599,7 +599,7 @@ static int sm750fb_set_drv(struct lynxfb_par *par)
crtc = &par->crtc;
crtc->vidmem_size = sm750_dev->vidmem_size;
- if (sm750_dev->fb_count > 1)
+ if (g_dualview)
crtc->vidmem_size >>= 1;
/* setup crtc and output member */
@@ -966,7 +966,7 @@ static void sm750fb_setup(struct sm750_dev *sm750_dev, char *src)
NO_PARAM:
if (sm750_dev->revid != SM750LE_REVISION_ID) {
- if (sm750_dev->fb_count > 1) {
+ if (g_dualview) {
if (swap)
sm750_dev->dataflow = sm750_dual_swap;
else
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index 90b870f234f0..f0d7405d1b21 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -48,7 +48,7 @@ static struct iscsi_login *iscsi_login_init_conn(struct iscsit_conn *conn)
login->conn = conn;
login->first_request = 1;
- login->req_buf = kzalloc(MAX_KEY_VALUE_PAIRS, GFP_KERNEL);
+ login->req_buf = kzalloc(MAX_KEY_VALUE_PAIRS + 1, GFP_KERNEL);
if (!login->req_buf) {
pr_err("Unable to allocate memory for response buffer.\n");
goto out_login;
diff --git a/drivers/thermal/imx_thermal.c b/drivers/thermal/imx_thermal.c
index b8e85a405351..922358acff4b 100644
--- a/drivers/thermal/imx_thermal.c
+++ b/drivers/thermal/imx_thermal.c
@@ -832,12 +832,12 @@ static int imx_thermal_runtime_resume(struct device *dev)
ret = regmap_write(map, socdata->sensor_ctrl + REG_CLR,
socdata->power_down_mask);
if (ret)
- return ret;
+ goto disable_clk;
ret = regmap_write(map, socdata->sensor_ctrl + REG_SET,
socdata->measure_temp_mask);
if (ret)
- return ret;
+ goto disable_clk;
/*
* According to the temp sensor designers, it may require up to ~17us
@@ -846,6 +846,11 @@ static int imx_thermal_runtime_resume(struct device *dev)
usleep_range(20, 50);
return 0;
+
+disable_clk:
+ clk_disable_unprepare(data->thermal_clk);
+
+ return ret;
}
static const struct dev_pm_ops imx_thermal_pm_ops = {
diff --git a/drivers/thermal/intel/int340x_thermal/int3400_thermal.c b/drivers/thermal/intel/int340x_thermal/int3400_thermal.c
index f547d386ae80..0e18f4f6ccb8 100644
--- a/drivers/thermal/intel/int340x_thermal/int3400_thermal.c
+++ b/drivers/thermal/intel/int340x_thermal/int3400_thermal.c
@@ -363,8 +363,10 @@ static void cleanup_odvp(struct int3400_thermal_priv *priv)
kfree(priv->odvp_attrs[i].attr.attr.name);
}
kfree(priv->odvp_attrs);
+ priv->odvp_attrs = NULL;
}
kfree(priv->odvp);
+ priv->odvp = NULL;
priv->odvp_count = 0;
}
@@ -646,7 +648,6 @@ static int int3400_thermal_probe(struct platform_device *pdev)
acpi_remove_notify_handler(priv->adev->handle, ACPI_DEVICE_NOTIFY,
int3400_notify);
free_sysfs:
- cleanup_odvp(priv);
if (!ZERO_OR_NULL_PTR(priv->data_vault)) {
sysfs_remove_group(&pdev->dev.kobj, &data_attribute_group);
kfree(priv->data_vault);
@@ -660,6 +661,7 @@ static int int3400_thermal_probe(struct platform_device *pdev)
acpi_thermal_rel_misc_device_remove(priv->adev->handle);
thermal_zone_device_unregister(priv->thermal);
free_art_trt:
+ cleanup_odvp(priv);
kfree(priv->trts);
kfree(priv->arts);
free_priv:
diff --git a/drivers/thermal/qoriq_thermal.c b/drivers/thermal/qoriq_thermal.c
index aed2729f63d0..69d8afed3785 100644
--- a/drivers/thermal/qoriq_thermal.c
+++ b/drivers/thermal/qoriq_thermal.c
@@ -380,11 +380,20 @@ static int qoriq_tmu_resume(struct device *dev)
if (data->ver > TMU_VER1) {
ret = regmap_clear_bits(data->regmap, REGS_TMR, TMR_CMD);
if (ret)
- return ret;
+ goto disable_clk;
}
/* Enable monitoring */
- return regmap_update_bits(data->regmap, REGS_TMR, TMR_ME, TMR_ME);
+ ret = regmap_update_bits(data->regmap, REGS_TMR, TMR_ME, TMR_ME);
+ if (ret)
+ goto disable_clk;
+
+ return 0;
+
+disable_clk:
+ clk_disable_unprepare(data->clk);
+
+ return ret;
}
static DEFINE_SIMPLE_DEV_PM_OPS(qoriq_tmu_pm_ops,
diff --git a/drivers/thermal/renesas/rcar_thermal.c b/drivers/thermal/renesas/rcar_thermal.c
index ddc8341e5c3f..00517681a8bb 100644
--- a/drivers/thermal/renesas/rcar_thermal.c
+++ b/drivers/thermal/renesas/rcar_thermal.c
@@ -492,12 +492,6 @@ static int rcar_thermal_probe(struct platform_device *pdev)
"rcar_thermal", trips, ARRAY_SIZE(trips), priv,
&rcar_thermal_zone_ops, NULL, 0,
idle);
-
- ret = thermal_zone_device_enable(priv->zone);
- if (ret) {
- thermal_zone_device_unregister(priv->zone);
- priv->zone = ERR_PTR(ret);
- }
}
if (IS_ERR(priv->zone)) {
dev_err(dev, "can't register thermal zone\n");
@@ -506,11 +500,12 @@ static int rcar_thermal_probe(struct platform_device *pdev)
goto error_unregister;
}
- if (chip->use_of_thermal) {
+ if (chip->use_of_thermal)
ret = thermal_add_hwmon_sysfs(priv->zone);
- if (ret)
- goto error_unregister;
- }
+ else
+ ret = thermal_zone_device_enable(priv->zone);
+ if (ret)
+ goto error_unregister;
rcar_thermal_irq_enable(priv);
diff --git a/drivers/tty/hvc/Kconfig b/drivers/tty/hvc/Kconfig
index c2a4e88b328f..5866195de26a 100644
--- a/drivers/tty/hvc/Kconfig
+++ b/drivers/tty/hvc/Kconfig
@@ -79,7 +79,7 @@ config HVC_UDBG
config HVC_DCC
bool "ARM JTAG DCC console"
- depends on ARM || ARM64
+ depends on (ARM && (CPU_V6 || CPU_V6K || CPU_V7)) || ARM64
select HVC_DRIVER
select SERIAL_CORE_CONSOLE
help
diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c
index 8934fbdb0cf9..3e1932a3bfbb 100644
--- a/drivers/tty/serial/amba-pl011.c
+++ b/drivers/tty/serial/amba-pl011.c
@@ -2434,6 +2434,15 @@ static int pl011_console_setup(struct console *co, char *options)
return uart_set_options(&uap->port, co, baud, parity, bits, flow);
}
+static int pl011_console_exit(struct console *co)
+{
+ struct uart_amba_port *uap = amba_ports[co->index];
+
+ clk_unprepare(uap->clk);
+
+ return 0;
+}
+
/**
* pl011_console_match - non-standard console matching
* @co: registering console
@@ -2500,6 +2509,7 @@ static struct console amba_console = {
.write = pl011_console_write,
.device = uart_console_device,
.setup = pl011_console_setup,
+ .exit = pl011_console_exit,
.match = pl011_console_match,
.flags = CON_PRINTBUFFER | CON_ANYTIME,
.index = -1,
diff --git a/drivers/tty/serial/ma35d1_serial.c b/drivers/tty/serial/ma35d1_serial.c
index 3b4206e815fe..2c1c8d748023 100644
--- a/drivers/tty/serial/ma35d1_serial.c
+++ b/drivers/tty/serial/ma35d1_serial.c
@@ -608,8 +608,14 @@ static int __init ma35d1serial_console_setup(struct console *co, char *options)
if (!np || !p)
return -ENODEV;
- if (of_property_read_u32_array(np, "reg", val32, ARRAY_SIZE(val32)) != 0)
+ if (of_property_read_u32_array(np, "reg", val32, ARRAY_SIZE(val32)) != 0) {
+ of_node_put(np);
+ ma35d1serial_uart_nodes[co->index] = NULL;
return -EINVAL;
+ }
+
+ of_node_put(np);
+ ma35d1serial_uart_nodes[co->index] = NULL;
p->port.iobase = val32[1];
p->port.membase = ioremap(p->port.iobase, MA35_UART_REG_SIZE);
@@ -648,8 +654,10 @@ static void ma35d1serial_console_init_port(void)
of_node_get(np);
ma35d1serial_uart_nodes[i] = np;
i++;
- if (i == MA35_UART_NR)
+ if (i == MA35_UART_NR) {
+ of_node_put(np);
break;
+ }
}
}
}
diff --git a/drivers/tty/serial/qcom_geni_serial.c b/drivers/tty/serial/qcom_geni_serial.c
index 6e5b31ebd5d6..0cccc26e932b 100644
--- a/drivers/tty/serial/qcom_geni_serial.c
+++ b/drivers/tty/serial/qcom_geni_serial.c
@@ -132,6 +132,7 @@ struct qcom_geni_serial_port {
unsigned int tx_remaining;
unsigned int tx_queued;
+ bool tx_dma_stale;
int wakeup_irq;
bool rx_tx_swap;
bool cts_rts_swap;
@@ -659,6 +660,7 @@ static void qcom_geni_serial_start_tx_dma(struct uart_port *uport)
}
port->tx_remaining = xmit_size;
+ port->tx_dma_stale = false;
}
static void qcom_geni_serial_start_tx_fifo(struct uart_port *uport)
@@ -991,6 +993,7 @@ static void qcom_geni_serial_handle_tx_dma(struct uart_port *uport)
struct qcom_geni_serial_port *port = to_dev_port(uport);
struct tty_port *tport = &uport->state->port;
unsigned int fifo_len = kfifo_len(&tport->xmit_fifo);
+ bool tx_dma_stale = port->tx_dma_stale;
/*
* Only advance the kfifo if it still contains the bytes that were
@@ -1001,12 +1004,13 @@ static void qcom_geni_serial_handle_tx_dma(struct uart_port *uport)
* kfifo->in, making kfifo_len() wrap to UART_XMIT_SIZE - tx_remaining
* and triggering a spurious large DMA transfer of stale data.
*/
- if (fifo_len >= port->tx_remaining)
+ if (!tx_dma_stale && fifo_len >= port->tx_remaining)
uart_xmit_advance(uport, port->tx_remaining);
geni_se_tx_dma_unprep(&port->se, port->tx_dma_addr, port->tx_remaining);
port->tx_dma_addr = 0;
port->tx_remaining = 0;
+ port->tx_dma_stale = false;
if (!kfifo_is_empty(&tport->xmit_fifo))
qcom_geni_serial_start_tx_dma(uport);
@@ -1142,6 +1146,10 @@ static void qcom_geni_serial_shutdown(struct uart_port *uport)
static void qcom_geni_serial_flush_buffer_fifo(struct uart_port *uport)
{
+ struct qcom_geni_serial_port *port = to_dev_port(uport);
+
+ if (port->tx_dma_addr)
+ port->tx_dma_stale = true;
qcom_geni_serial_cancel_tx_cmd(uport);
}
diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c
index dcb1769c3625..5d372fd7423e 100644
--- a/drivers/tty/tty_io.c
+++ b/drivers/tty/tty_io.c
@@ -3181,8 +3181,10 @@ static int tty_cdev_add(struct tty_driver *driver, dev_t dev,
driver->cdevs[index]->ops = &tty_fops;
driver->cdevs[index]->owner = driver->owner;
err = cdev_add(driver->cdevs[index], dev, count);
- if (err)
+ if (err) {
kobject_put(&driver->cdevs[index]->kobj);
+ driver->cdevs[index] = NULL;
+ }
return err;
}
diff --git a/drivers/ufs/core/ufs-debugfs.c b/drivers/ufs/core/ufs-debugfs.c
index e3baed6c70bd..6dece24baad1 100644
--- a/drivers/ufs/core/ufs-debugfs.c
+++ b/drivers/ufs/core/ufs-debugfs.c
@@ -165,7 +165,7 @@ static ssize_t ufs_saved_err_write(struct file *file, const char __user *buf,
char val_str[16] = { };
int val, ret;
- if (count > sizeof(val_str))
+ if (count >= sizeof(val_str))
return -EINVAL;
if (copy_from_user(val_str, buf, count))
return -EFAULT;
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 98bf5bc6f383..22f4f5f08e74 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -1264,11 +1264,13 @@ static bool ufshcd_is_devfreq_scaling_required(struct ufs_hba *hba,
static u32 ufshcd_pending_cmds(struct ufs_hba *hba)
{
const struct scsi_device *sdev;
+ unsigned long flags;
u32 pending = 0;
- lockdep_assert_held(hba->host->host_lock);
+ spin_lock_irqsave(hba->host->host_lock, flags);
__shost_for_each_device(sdev, hba->host)
pending += sbitmap_weight(&sdev->budget_map);
+ spin_unlock_irqrestore(hba->host->host_lock, flags);
return pending;
}
@@ -1282,7 +1284,6 @@ static u32 ufshcd_pending_cmds(struct ufs_hba *hba)
static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba,
u64 wait_timeout_us)
{
- unsigned long flags;
int ret = 0;
u32 tm_doorbell;
u32 tr_pending;
@@ -1290,7 +1291,6 @@ static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba,
ktime_t start;
ufshcd_hold(hba);
- spin_lock_irqsave(hba->host->host_lock, flags);
/*
* Wait for all the outstanding tasks/transfer requests.
* Verify by checking the doorbell registers are clear.
@@ -1311,7 +1311,7 @@ static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba,
break;
}
- spin_unlock_irqrestore(hba->host->host_lock, flags);
+ __set_current_state(TASK_UNINTERRUPTIBLE);
io_schedule_timeout(msecs_to_jiffies(20));
if (ktime_to_us(ktime_sub(ktime_get(), start)) >
wait_timeout_us) {
@@ -1323,7 +1323,6 @@ static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba,
*/
do_last_check = true;
}
- spin_lock_irqsave(hba->host->host_lock, flags);
} while (tm_doorbell || tr_pending);
if (timeout) {
@@ -1333,7 +1332,6 @@ static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba,
ret = -EBUSY;
}
out:
- spin_unlock_irqrestore(hba->host->host_lock, flags);
ufshcd_release(hba);
return ret;
}
@@ -7127,12 +7125,13 @@ static int __ufshcd_issue_tm_cmd(struct ufs_hba *hba,
memcpy(hba->utmrdl_base_addr + task_tag, treq, sizeof(*treq));
ufshcd_vops_setup_task_mgmt(hba, task_tag, tm_function);
- /* send command to the controller */
__set_bit(task_tag, &hba->outstanding_tasks);
- ufshcd_writel(hba, 1 << task_tag, REG_UTP_TASK_REQ_DOOR_BELL);
spin_unlock_irqrestore(host->host_lock, flags);
+ /* send command to the controller */
+ ufshcd_writel(hba, 1 << task_tag, REG_UTP_TASK_REQ_DOOR_BELL);
+
ufshcd_add_tm_upiu_trace(hba, task_tag, UFS_TM_SEND);
/* wait until the task management command is completed */
diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index 004a549c6c7d..bce9635c7ca8 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -1051,6 +1051,11 @@ int __uio_register_device(struct module *owner,
err_request_irq:
uio_dev_del_attributes(idev);
err_uio_dev_add_attributes:
+ mutex_lock(&idev->info_lock);
+ idev->info = NULL;
+ mutex_unlock(&idev->info_lock);
+ wake_up_interruptible(&idev->wait);
+ kill_fasync(&idev->async_queue, SIGIO, POLL_HUP);
device_del(&idev->dev);
err_device_create:
uio_free_minor(idev->minor);
diff --git a/drivers/usb/atm/usbatm.c b/drivers/usb/atm/usbatm.c
index 2da6615fbb6f..a1130c325ca1 100644
--- a/drivers/usb/atm/usbatm.c
+++ b/drivers/usb/atm/usbatm.c
@@ -917,8 +917,8 @@ static int usbatm_atm_init(struct usbatm_data *instance)
instance->atm_dev = atm_dev;
- atm_dev->ci_range.vpi_bits = ATM_CI_MAX;
- atm_dev->ci_range.vci_bits = ATM_CI_MAX;
+ atm_dev->ci_range.vpi_bits = 8;
+ atm_dev->ci_range.vci_bits = 16;
atm_dev->signal = ATM_PHY_SIG_UNKNOWN;
/* temp init ATM device, set to 128kbit */
diff --git a/drivers/usb/common/common.c b/drivers/usb/common/common.c
index b7bea1015d7c..871cf199b6bf 100644
--- a/drivers/usb/common/common.c
+++ b/drivers/usb/common/common.c
@@ -415,6 +415,9 @@ EXPORT_SYMBOL_GPL(usb_of_get_companion_dev);
struct dentry *usb_debug_root;
EXPORT_SYMBOL_GPL(usb_debug_root);
+DEFINE_MUTEX(usb_dynids_lock);
+EXPORT_SYMBOL_GPL(usb_dynids_lock);
+
static int __init usb_common_init(void)
{
usb_debug_root = debugfs_create_dir("usb", NULL);
diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index 0c3f12daac79..528274e3cd7a 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -95,9 +95,9 @@ ssize_t usb_store_new_id(struct usb_dynids *dynids,
}
}
- spin_lock(&dynids->lock);
+ mutex_lock(&usb_dynids_lock);
list_add_tail(&dynid->node, &dynids->list);
- spin_unlock(&dynids->lock);
+ mutex_unlock(&usb_dynids_lock);
retval = driver_attach(driver);
@@ -160,7 +160,7 @@ static ssize_t remove_id_store(struct device_driver *driver, const char *buf,
if (fields < 2)
return -EINVAL;
- spin_lock(&usb_driver->dynids.lock);
+ guard(mutex)(&usb_dynids_lock);
list_for_each_entry_safe(dynid, n, &usb_driver->dynids.list, node) {
struct usb_device_id *id = &dynid->id;
@@ -171,7 +171,6 @@ static ssize_t remove_id_store(struct device_driver *driver, const char *buf,
break;
}
}
- spin_unlock(&usb_driver->dynids.lock);
return count;
}
@@ -220,27 +219,26 @@ static void usb_free_dynids(struct usb_driver *usb_drv)
{
struct usb_dynid *dynid, *n;
- spin_lock(&usb_drv->dynids.lock);
+ guard(mutex)(&usb_dynids_lock);
list_for_each_entry_safe(dynid, n, &usb_drv->dynids.list, node) {
list_del(&dynid->node);
kfree(dynid);
}
- spin_unlock(&usb_drv->dynids.lock);
}
static const struct usb_device_id *usb_match_dynamic_id(struct usb_interface *intf,
- struct usb_driver *drv)
+ const struct usb_driver *drv,
+ struct usb_device_id *id_copy)
{
struct usb_dynid *dynid;
- spin_lock(&drv->dynids.lock);
+ guard(mutex)(&usb_dynids_lock);
list_for_each_entry(dynid, &drv->dynids.list, node) {
if (usb_match_one_id(intf, &dynid->id)) {
- spin_unlock(&drv->dynids.lock);
- return &dynid->id;
+ *id_copy = dynid->id;
+ return id_copy;
}
}
- spin_unlock(&drv->dynids.lock);
return NULL;
}
@@ -324,6 +322,7 @@ static int usb_probe_interface(struct device *dev)
struct usb_interface *intf = to_usb_interface(dev);
struct usb_device *udev = interface_to_usbdev(intf);
const struct usb_device_id *id;
+ struct usb_device_id id_copy;
int error = -ENODEV;
int lpm_disable_error = -ENODEV;
@@ -343,7 +342,7 @@ static int usb_probe_interface(struct device *dev)
return error;
}
- id = usb_match_dynamic_id(intf, driver);
+ id = usb_match_dynamic_id(intf, driver, &id_copy);
if (!id)
id = usb_match_id(intf, driver->id_table);
if (!id)
@@ -893,8 +892,9 @@ static int usb_device_match(struct device *dev, const struct device_driver *drv)
} else if (is_usb_interface(dev)) {
struct usb_interface *intf;
- struct usb_driver *usb_drv;
+ const struct usb_driver *usb_drv;
const struct usb_device_id *id;
+ struct usb_device_id id_copy;
/* device drivers never match interfaces */
if (is_usb_device_driver(drv))
@@ -907,7 +907,7 @@ static int usb_device_match(struct device *dev, const struct device_driver *drv)
if (id)
return 1;
- id = usb_match_dynamic_id(intf, usb_drv);
+ id = usb_match_dynamic_id(intf, usb_drv, &id_copy);
if (id)
return 1;
}
@@ -1076,7 +1076,6 @@ int usb_register_driver(struct usb_driver *new_driver, struct module *owner,
new_driver->driver.owner = owner;
new_driver->driver.mod_name = mod_name;
new_driver->driver.dev_groups = new_driver->dev_groups;
- spin_lock_init(&new_driver->dynids.lock);
INIT_LIST_HEAD(&new_driver->dynids.list);
retval = driver_register(&new_driver->driver);
diff --git a/drivers/usb/gadget/configfs.c b/drivers/usb/gadget/configfs.c
index 82282373c786..6b3713066ac6 100644
--- a/drivers/usb/gadget/configfs.c
+++ b/drivers/usb/gadget/configfs.c
@@ -1177,7 +1177,7 @@ static ssize_t os_desc_qw_sign_show(struct config_item *item, char *page)
struct gadget_info *gi = os_desc_item_to_gadget_info(item);
int res;
- res = utf16s_to_utf8s((wchar_t *) gi->qw_sign, OS_STRING_QW_SIGN_LEN,
+ res = utf16s_to_utf8s((wchar_t *) gi->qw_sign, OS_STRING_QW_SIGN_LEN / 2,
UTF16_LITTLE_ENDIAN, page, PAGE_SIZE - 1);
page[res++] = '\n';
@@ -1199,7 +1199,7 @@ static ssize_t os_desc_qw_sign_store(struct config_item *item, const char *page,
mutex_lock(&gi->lock);
res = utf8s_to_utf16s(page, l,
UTF16_LITTLE_ENDIAN, (wchar_t *) gi->qw_sign,
- OS_STRING_QW_SIGN_LEN);
+ OS_STRING_QW_SIGN_LEN / 2);
if (res > 0)
res = len;
mutex_unlock(&gi->lock);
diff --git a/drivers/usb/gadget/function/f_fs.c b/drivers/usb/gadget/function/f_fs.c
index cbed66a52826..24bfde82c385 100644
--- a/drivers/usb/gadget/function/f_fs.c
+++ b/drivers/usb/gadget/function/f_fs.c
@@ -1678,13 +1678,13 @@ static int ffs_dmabuf_transfer(struct file *file,
/* In the meantime, endpoint got disabled or changed. */
if (epfile->ep != ep) {
ret = -ESHUTDOWN;
- goto err_fence_put;
+ goto err_fence_free;
}
usb_req = usb_ep_alloc_request(ep->ep, GFP_ATOMIC);
if (!usb_req) {
ret = -ENOMEM;
- goto err_fence_put;
+ goto err_fence_free;
}
/*
@@ -1733,9 +1733,9 @@ static int ffs_dmabuf_transfer(struct file *file,
return ret;
-err_fence_put:
+err_fence_free:
spin_unlock_irq(&epfile->ffs->eps_lock);
- dma_fence_put(&fence->base);
+ kfree(fence);
err_resv_unlock:
dma_resv_unlock(dmabuf->resv);
err_attachment_put:
diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index 74cb7e57a197..24908779bd0b 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -2961,7 +2961,7 @@ EXPORT_SYMBOL_GPL(fsg_common_create_lun);
int fsg_common_create_luns(struct fsg_common *common, struct fsg_config *cfg)
{
- char buf[8]; /* enough for 100000000 different numbers, decimal */
+ char buf[14];
int i, rc;
fsg_common_remove_luns(common);
diff --git a/drivers/usb/gadget/function/f_midi.c b/drivers/usb/gadget/function/f_midi.c
index bbd90d0c2b6b..1e29da01f131 100644
--- a/drivers/usb/gadget/function/f_midi.c
+++ b/drivers/usb/gadget/function/f_midi.c
@@ -878,7 +878,6 @@ static int f_midi_bind(struct usb_configuration *c, struct usb_function *f)
int status, n, jack = 1, i = 0, endpoint_descriptor_index = 0;
midi->gadget = cdev->gadget;
- INIT_WORK(&midi->work, f_midi_in_work);
status = f_midi_register_card(midi);
if (status < 0)
goto fail_register;
@@ -1371,6 +1370,7 @@ static struct usb_function *f_midi_alloc(struct usb_function_instance *fi)
status = -ENOMEM;
goto midi_free;
}
+ INIT_WORK(&midi->work, f_midi_in_work);
midi->out_ports = opts->out_ports;
midi->index = opts->index;
midi->buflen = opts->buflen;
diff --git a/drivers/usb/gadget/function/f_midi2.c b/drivers/usb/gadget/function/f_midi2.c
index 4e5376eccd85..307f2b3a9cb7 100644
--- a/drivers/usb/gadget/function/f_midi2.c
+++ b/drivers/usb/gadget/function/f_midi2.c
@@ -1145,7 +1145,7 @@ static int f_midi2_alloc_ep_reqs(struct f_midi2_usb_ep *usb_ep)
if (!usb_ep->reqs)
return -EINVAL;
- for (i = 0; i < midi2->info.num_reqs; i++) {
+ for (i = 0; i < usb_ep->num_reqs; i++) {
if (usb_ep->reqs[i].req)
continue;
usb_ep->reqs[i].req = alloc_ep_req(usb_ep->usb_ep,
@@ -1160,10 +1160,9 @@ static int f_midi2_alloc_ep_reqs(struct f_midi2_usb_ep *usb_ep)
/* Free allocated requests */
static void f_midi2_free_ep_reqs(struct f_midi2_usb_ep *usb_ep)
{
- struct f_midi2 *midi2 = usb_ep->card;
int i;
- for (i = 0; i < midi2->info.num_reqs; i++) {
+ for (i = 0; i < usb_ep->num_reqs; i++) {
if (!usb_ep->reqs[i].req)
continue;
free_ep_req(usb_ep->usb_ep, usb_ep->reqs[i].req);
@@ -2181,13 +2180,13 @@ static ssize_t f_midi2_opts_bool_store(struct f_midi2_opts *opts,
/* generic show/store for string */
static ssize_t f_midi2_opts_str_show(struct f_midi2_opts *opts,
- const char *str, char *page)
+ const char **strp, char *page)
{
int result = 0;
mutex_lock(&opts->lock);
- if (str)
- result = scnprintf(page, PAGE_SIZE, "%s\n", str);
+ if (*strp)
+ result = scnprintf(page, PAGE_SIZE, "%s\n", *strp);
mutex_unlock(&opts->lock);
return result;
}
@@ -2281,7 +2280,7 @@ static ssize_t f_midi2_block_opts_name_show(struct config_item *item,
{
struct f_midi2_block_opts *opts = to_f_midi2_block_opts(item);
- return f_midi2_opts_str_show(opts->ep->opts, opts->info.name, page);
+ return f_midi2_opts_str_show(opts->ep->opts, &opts->info.name, page);
}
static ssize_t f_midi2_block_opts_name_store(struct config_item *item,
@@ -2438,7 +2437,7 @@ static ssize_t f_midi2_ep_opts_##name##_show(struct config_item *item, \
char *page) \
{ \
struct f_midi2_ep_opts *opts = to_f_midi2_ep_opts(item); \
- return f_midi2_opts_str_show(opts->opts, opts->info.name, page);\
+ return f_midi2_opts_str_show(opts->opts, &opts->info.name, page);\
} \
\
static ssize_t f_midi2_ep_opts_##name##_store(struct config_item *item, \
@@ -2593,7 +2592,7 @@ static ssize_t f_midi2_opts_iface_name_show(struct config_item *item,
{
struct f_midi2_opts *opts = to_f_midi2_opts(item);
- return f_midi2_opts_str_show(opts, opts->info.iface_name, page);
+ return f_midi2_opts_str_show(opts, &opts->info.iface_name, page);
}
static ssize_t f_midi2_opts_iface_name_store(struct config_item *item,
diff --git a/drivers/usb/gadget/function/f_uac1_legacy.c b/drivers/usb/gadget/function/f_uac1_legacy.c
index 4981af8337ab..c234f9fea27c 100644
--- a/drivers/usb/gadget/function/f_uac1_legacy.c
+++ b/drivers/usb/gadget/function/f_uac1_legacy.c
@@ -888,60 +888,10 @@ UAC1_INT_ATTRIBUTE(req_buf_size);
UAC1_INT_ATTRIBUTE(req_count);
UAC1_INT_ATTRIBUTE(audio_buf_size);
-#define UAC1_STR_ATTRIBUTE(name) \
-static ssize_t f_uac1_opts_##name##_show(struct config_item *item, \
- char *page) \
-{ \
- struct f_uac1_legacy_opts *opts = to_f_uac1_opts(item); \
- int result; \
- \
- mutex_lock(&opts->lock); \
- result = sprintf(page, "%s\n", opts->name); \
- mutex_unlock(&opts->lock); \
- \
- return result; \
-} \
- \
-static ssize_t f_uac1_opts_##name##_store(struct config_item *item, \
- const char *page, size_t len) \
-{ \
- struct f_uac1_legacy_opts *opts = to_f_uac1_opts(item); \
- int ret = -EBUSY; \
- char *tmp; \
- \
- mutex_lock(&opts->lock); \
- if (opts->refcnt) \
- goto end; \
- \
- tmp = kstrndup(page, len, GFP_KERNEL); \
- if (tmp) { \
- ret = -ENOMEM; \
- goto end; \
- } \
- if (opts->name##_alloc) \
- kfree(opts->name); \
- opts->name##_alloc = true; \
- opts->name = tmp; \
- ret = len; \
- \
-end: \
- mutex_unlock(&opts->lock); \
- return ret; \
-} \
- \
-CONFIGFS_ATTR(f_uac1_opts_, name)
-
-UAC1_STR_ATTRIBUTE(fn_play);
-UAC1_STR_ATTRIBUTE(fn_cap);
-UAC1_STR_ATTRIBUTE(fn_cntl);
-
static struct configfs_attribute *f_uac1_attrs[] = {
&f_uac1_opts_attr_req_buf_size,
&f_uac1_opts_attr_req_count,
&f_uac1_opts_attr_audio_buf_size,
- &f_uac1_opts_attr_fn_play,
- &f_uac1_opts_attr_fn_cap,
- &f_uac1_opts_attr_fn_cntl,
NULL,
};
@@ -956,12 +906,6 @@ static void f_audio_free_inst(struct usb_function_instance *f)
struct f_uac1_legacy_opts *opts;
opts = container_of(f, struct f_uac1_legacy_opts, func_inst);
- if (opts->fn_play_alloc)
- kfree(opts->fn_play);
- if (opts->fn_cap_alloc)
- kfree(opts->fn_cap);
- if (opts->fn_cntl_alloc)
- kfree(opts->fn_cntl);
kfree(opts);
}
diff --git a/drivers/usb/gadget/function/u_uac1_legacy.h b/drivers/usb/gadget/function/u_uac1_legacy.h
index b5df9bcbbeba..b9ddae550ff3 100644
--- a/drivers/usb/gadget/function/u_uac1_legacy.h
+++ b/drivers/usb/gadget/function/u_uac1_legacy.h
@@ -62,9 +62,6 @@ struct f_uac1_legacy_opts {
char *fn_cap;
char *fn_cntl;
unsigned bound:1;
- unsigned fn_play_alloc:1;
- unsigned fn_cap_alloc:1;
- unsigned fn_cntl_alloc:1;
struct mutex lock;
int refcnt;
};
diff --git a/drivers/usb/gadget/functions.c b/drivers/usb/gadget/functions.c
index 203361a64212..70e31c40e267 100644
--- a/drivers/usb/gadget/functions.c
+++ b/drivers/usb/gadget/functions.c
@@ -70,7 +70,7 @@ void usb_put_function_instance(struct usb_function_instance *fi)
{
struct module *mod;
- if (!fi)
+ if (!fi || !fi->fd)
return;
mod = fi->fd->mod;
diff --git a/drivers/usb/gadget/legacy/inode.c b/drivers/usb/gadget/legacy/inode.c
index b6a30d88a800..81766838e4b2 100644
--- a/drivers/usb/gadget/legacy/inode.c
+++ b/drivers/usb/gadget/legacy/inode.c
@@ -1253,10 +1253,11 @@ ep0_poll (struct file *fd, poll_table *wait)
static long gadget_dev_ioctl (struct file *fd, unsigned code, unsigned long value)
{
struct dev_data *dev = fd->private_data;
- struct usb_gadget *gadget = dev->gadget;
+ struct usb_gadget *gadget;
long ret = -ENOTTY;
spin_lock_irq(&dev->lock);
+ gadget = dev->gadget;
if (dev->state == STATE_DEV_OPENED ||
dev->state == STATE_DEV_UNBOUND) {
/* Not bound to a UDC */
diff --git a/drivers/usb/gadget/udc/aspeed_udc.c b/drivers/usb/gadget/udc/aspeed_udc.c
index f4781e611aaa..efcceb886bb4 100644
--- a/drivers/usb/gadget/udc/aspeed_udc.c
+++ b/drivers/usb/gadget/udc/aspeed_udc.c
@@ -1434,25 +1434,12 @@ static void ast_udc_init_hw(struct ast_udc_dev *udc)
ast_udc_write(udc, 0, AST_UDC_EP0_CTRL);
}
-static void ast_udc_remove(struct platform_device *pdev)
+static void ast_udc_cleanup(struct platform_device *pdev)
{
struct ast_udc_dev *udc = platform_get_drvdata(pdev);
unsigned long flags;
u32 ctrl;
- usb_del_gadget_udc(&udc->gadget);
- if (udc->driver) {
- /*
- * This is broken as only some cleanup is skipped, *udev is
- * freed and the register mapping goes away. Any further usage
- * probably crashes. Also the device is unbound, so the skipped
- * cleanup is never catched up later.
- */
- dev_alert(&pdev->dev,
- "Driver is busy and still going away. Fasten your seat belts!\n");
- return;
- }
-
spin_lock_irqsave(&udc->lock, flags);
/* Disable upstream port connection */
@@ -1472,6 +1459,26 @@ static void ast_udc_remove(struct platform_device *pdev)
udc->ep0_buf = NULL;
}
+static void ast_udc_remove(struct platform_device *pdev)
+{
+ struct ast_udc_dev *udc = platform_get_drvdata(pdev);
+
+ usb_del_gadget_udc(&udc->gadget);
+ if (udc->driver) {
+ /*
+ * This is broken as only some cleanup is skipped, *udev is
+ * freed and the register mapping goes away. Any further usage
+ * probably crashes. Also the device is unbound, so the skipped
+ * cleanup is never catched up later.
+ */
+ dev_alert(&pdev->dev,
+ "Driver is busy and still going away. Fasten your seat belts!\n");
+ return;
+ }
+
+ ast_udc_cleanup(pdev);
+}
+
static int ast_udc_probe(struct platform_device *pdev)
{
enum usb_device_speed max_speed;
@@ -1524,6 +1531,12 @@ static int ast_udc_probe(struct platform_device *pdev)
AST_UDC_NUM_ENDPOINTS,
&udc->ep0_buf_dma, GFP_KERNEL);
+ if (!udc->ep0_buf) {
+ clk_disable_unprepare(udc->clk);
+ rc = -ENOMEM;
+ goto err;
+ }
+
udc->gadget.speed = USB_SPEED_UNKNOWN;
udc->gadget.max_speed = USB_SPEED_HIGH;
udc->creq = udc->reg + AST_UDC_SETUP0;
@@ -1553,20 +1566,20 @@ static int ast_udc_probe(struct platform_device *pdev)
udc->irq = platform_get_irq(pdev, 0);
if (udc->irq < 0) {
rc = udc->irq;
- goto err;
+ goto err_cleanup;
}
rc = devm_request_irq(&pdev->dev, udc->irq, ast_udc_isr, 0,
KBUILD_MODNAME, udc);
if (rc) {
dev_err(&pdev->dev, "Failed to request interrupt\n");
- goto err;
+ goto err_cleanup;
}
rc = usb_add_gadget_udc(&pdev->dev, &udc->gadget);
if (rc) {
dev_err(&pdev->dev, "Failed to add gadget udc\n");
- goto err;
+ goto err_cleanup;
}
dev_info(&pdev->dev, "Initialized udc in USB%s mode\n",
@@ -1574,9 +1587,10 @@ static int ast_udc_probe(struct platform_device *pdev)
return 0;
+err_cleanup:
+ ast_udc_cleanup(pdev);
err:
dev_err(&pdev->dev, "Failed to udc probe, rc:0x%x\n", rc);
- ast_udc_remove(pdev);
return rc;
}
diff --git a/drivers/usb/gadget/udc/r8a66597-udc.c b/drivers/usb/gadget/udc/r8a66597-udc.c
index db4a10a979f9..813b1e1af435 100644
--- a/drivers/usb/gadget/udc/r8a66597-udc.c
+++ b/drivers/usb/gadget/udc/r8a66597-udc.c
@@ -1951,7 +1951,6 @@ static int r8a66597_probe(struct platform_device *pdev)
return 0;
err_add_udc:
- r8a66597_free_request(&r8a66597->ep[0].ep, r8a66597->ep0_req);
clean_up2:
if (r8a66597->pdata->on_chip)
clk_disable_unprepare(r8a66597->clk);
diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index 931f46ae89ac..df2617f6b60b 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -781,21 +781,18 @@ static void xhci_giveback_urb_in_irq(struct xhci_hcd *xhci,
usb_hcd_giveback_urb(hcd, urb, status);
}
-static void xhci_unmap_td_bounce_buffer(struct xhci_hcd *xhci,
- struct xhci_ring *ring, struct xhci_td *td)
+static void xhci_unmap_one_bounce_buffer(struct xhci_hcd *xhci,
+ struct xhci_ring *ring, struct xhci_td *td,
+ struct xhci_segment *seg)
{
struct device *dev = xhci_to_hcd(xhci)->self.sysdev;
- struct xhci_segment *seg = td->bounce_seg;
struct urb *urb = td->urb;
size_t len;
- if (!ring || !seg || !urb)
- return;
-
if (usb_urb_dir_out(urb)) {
dma_unmap_single(dev, seg->bounce_dma, ring->bounce_buf_len,
DMA_TO_DEVICE);
- return;
+ goto done;
}
dma_unmap_single(dev, seg->bounce_dma, ring->bounce_buf_len,
@@ -811,10 +808,29 @@ static void xhci_unmap_td_bounce_buffer(struct xhci_hcd *xhci,
memcpy(urb->transfer_buffer + seg->bounce_offs, seg->bounce_buf,
seg->bounce_len);
}
+done:
seg->bounce_len = 0;
seg->bounce_offs = 0;
}
+static void xhci_unmap_td_bounce_buffer(struct xhci_hcd *xhci,
+ struct xhci_ring *ring, struct xhci_td *td)
+{
+ struct xhci_segment *seg;
+ int i = 0;
+
+ if (!td->bounce_seg || !ring || !td->urb)
+ return;
+
+ /* td->bounce_seg is the last one bounced, unmap them all */
+ for (seg = td->start_seg; i++ < ring->num_segs; seg = seg->next) {
+ if (seg->bounce_len)
+ xhci_unmap_one_bounce_buffer(xhci, ring, td, seg);
+ if (seg == td->bounce_seg)
+ break;
+ }
+}
+
static int xhci_td_cleanup(struct xhci_hcd *xhci, struct xhci_td *td,
struct xhci_ring *ep_ring, int status)
{
@@ -3725,7 +3741,7 @@ int xhci_queue_bulk_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
&trb_buff_len,
ring->enq_seg)) {
send_addr = ring->enq_seg->bounce_dma;
- /* assuming TD won't span 2 segs */
+ /* TD bounced at least, and last on this seg */
td->bounce_seg = ring->enq_seg;
}
}
diff --git a/drivers/usb/image/mdc800.c b/drivers/usb/image/mdc800.c
index 10d72562e4d2..afc8da9cddf1 100644
--- a/drivers/usb/image/mdc800.c
+++ b/drivers/usb/image/mdc800.c
@@ -1000,13 +1000,13 @@ static int __init usb_mdc800_init (void)
mdc800->downloaded = 0;
mdc800->written = 0;
- mdc800->irq_urb_buffer=kmalloc (8, GFP_KERNEL);
+ mdc800->irq_urb_buffer=kzalloc (8, GFP_KERNEL);
if (!mdc800->irq_urb_buffer)
goto cleanup_on_fail;
mdc800->write_urb_buffer=kmalloc (8, GFP_KERNEL);
if (!mdc800->write_urb_buffer)
goto cleanup_on_fail;
- mdc800->download_urb_buffer=kmalloc (64, GFP_KERNEL);
+ mdc800->download_urb_buffer=kzalloc (64, GFP_KERNEL);
if (!mdc800->download_urb_buffer)
goto cleanup_on_fail;
diff --git a/drivers/usb/misc/usb-ljca.c b/drivers/usb/misc/usb-ljca.c
index d9c21f783055..e6e3888451fa 100644
--- a/drivers/usb/misc/usb-ljca.c
+++ b/drivers/usb/misc/usb-ljca.c
@@ -585,6 +585,9 @@ static int ljca_enumerate_gpio(struct ljca_adapter *adap)
if (gpio_num > LJCA_MAX_GPIO_NUM)
return -EINVAL;
+ if (desc->bank_num > ARRAY_SIZE(valid_pin))
+ return -EINVAL;
+
/* construct platform data */
gpio_info = kzalloc(sizeof *gpio_info, GFP_KERNEL);
if (!gpio_info)
diff --git a/drivers/usb/mtu3/mtu3_core.c b/drivers/usb/mtu3/mtu3_core.c
index a3a6282893d0..ca8cb6c3a01b 100644
--- a/drivers/usb/mtu3/mtu3_core.c
+++ b/drivers/usb/mtu3/mtu3_core.c
@@ -1037,9 +1037,14 @@ int ssusb_gadget_suspend(struct ssusb_mtk *ssusb, pm_message_t msg)
if (!mtu->gadget_driver)
return 0;
- if (mtu->connected)
+ /* Prevent runtime suspend when active connection exists */
+ if (mtu->connected && PMSG_IS_AUTO(msg))
return -EBUSY;
+ /* Perform soft disconnect for system suspend */
+ if (mtu->softconnect && !PMSG_IS_AUTO(msg))
+ mtu3_dev_on_off(mtu, 0);
+
mtu3_dev_suspend(mtu);
synchronize_irq(mtu->irq);
@@ -1055,5 +1060,9 @@ int ssusb_gadget_resume(struct ssusb_mtk *ssusb, pm_message_t msg)
mtu3_dev_resume(mtu);
+ /* Restore soft connect for system resume */
+ if (mtu->softconnect && !PMSG_IS_AUTO(msg))
+ mtu3_dev_on_off(mtu, 1);
+
return 0;
}
diff --git a/drivers/usb/renesas_usbhs/common.c b/drivers/usb/renesas_usbhs/common.c
index ef9c3bd950ca..75600dc6703d 100644
--- a/drivers/usb/renesas_usbhs/common.c
+++ b/drivers/usb/renesas_usbhs/common.c
@@ -809,9 +809,6 @@ static void usbhs_remove(struct platform_device *pdev)
flush_delayed_work(&priv->notify_hotplug_work);
- usbhs_platform_call(priv, hardware_exit, pdev);
- reset_control_assert(priv->rsts);
-
/*
* Explicitly free the IRQ to ensure the interrupt handler is
* disabled and synchronized before freeing resources.
@@ -828,6 +825,9 @@ static void usbhs_remove(struct platform_device *pdev)
if (!usbhs_get_dparam(priv, runtime_pwctrl))
usbhsc_power_ctrl(priv, 0);
+ usbhs_platform_call(priv, hardware_exit, pdev);
+ reset_control_assert(priv->rsts);
+
usbhsc_clk_put(priv);
pm_runtime_disable(&pdev->dev);
}
diff --git a/drivers/usb/serial/bus.c b/drivers/usb/serial/bus.c
index d200e2c29a8f..2fea1b1db4a2 100644
--- a/drivers/usb/serial/bus.c
+++ b/drivers/usb/serial/bus.c
@@ -136,12 +136,11 @@ static void free_dynids(struct usb_serial_driver *drv)
{
struct usb_dynid *dynid, *n;
- spin_lock(&drv->dynids.lock);
+ guard(mutex)(&usb_dynids_lock);
list_for_each_entry_safe(dynid, n, &drv->dynids.list, node) {
list_del(&dynid->node);
kfree(dynid);
}
- spin_unlock(&drv->dynids.lock);
}
const struct bus_type usb_serial_bus_type = {
@@ -157,7 +156,6 @@ int usb_serial_bus_register(struct usb_serial_driver *driver)
int retval;
driver->driver.bus = &usb_serial_bus_type;
- spin_lock_init(&driver->dynids.lock);
INIT_LIST_HEAD(&driver->dynids.list);
retval = driver_register(&driver->driver);
diff --git a/drivers/usb/serial/usb-serial.c b/drivers/usb/serial/usb-serial.c
index 2ee0b64b8be0..c78ff40b1e5f 100644
--- a/drivers/usb/serial/usb-serial.c
+++ b/drivers/usb/serial/usb-serial.c
@@ -706,14 +706,12 @@ static const struct usb_device_id *match_dynamic_id(struct usb_interface *intf,
{
struct usb_dynid *dynid;
- spin_lock(&drv->dynids.lock);
+ guard(mutex)(&usb_dynids_lock);
list_for_each_entry(dynid, &drv->dynids.list, node) {
if (usb_match_one_id(intf, &dynid->id)) {
- spin_unlock(&drv->dynids.lock);
return &dynid->id;
}
}
- spin_unlock(&drv->dynids.lock);
return NULL;
}
diff --git a/drivers/usb/storage/ene_ub6250.c b/drivers/usb/storage/ene_ub6250.c
index 35913ce81940..de0aa765b3fa 100644
--- a/drivers/usb/storage/ene_ub6250.c
+++ b/drivers/usb/storage/ene_ub6250.c
@@ -2358,7 +2358,9 @@ static int ene_ub6250_probe(struct usb_interface *intf,
return result;
/* probe card type */
+ mutex_lock(&us->dev_mutex);
result = ene_get_card_type(us, REG_CARD_STATUS, info->bbuf);
+ mutex_unlock(&us->dev_mutex);
if (result != USB_STOR_XFER_GOOD) {
usb_stor_disconnect(intf);
return USB_STOR_TRANSPORT_ERROR;
diff --git a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c
index 18303b34594b..2a50960283e1 100644
--- a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c
+++ b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_pdphy.c
@@ -544,6 +544,8 @@ static void qcom_pmic_typec_pdphy_stop(struct pmic_typec *tcpm)
for (i = 0; i < pmic_typec_pdphy->nr_irqs; i++)
disable_irq(pmic_typec_pdphy->irq_data[i].irq);
+ cancel_work_sync(&pmic_typec_pdphy->reset_work);
+
qcom_pmic_typec_pdphy_reset_on(pmic_typec_pdphy);
regulator_disable(pmic_typec_pdphy->vdd_pdphy);
diff --git a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_port.c b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_port.c
index 4fc83dcfae64..67a31e170a07 100644
--- a/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_port.c
+++ b/drivers/usb/typec/tcpm/qcom/qcom_pmic_typec_port.c
@@ -683,6 +683,9 @@ static int qcom_pmic_typec_port_start(struct pmic_typec *tcpm,
enable_irq(pmic_typec_port->irq_data[i].irq);
done:
+ if (ret)
+ disable_delayed_work_sync(&pmic_typec_port->cc_debounce_dwork);
+
return ret;
}
@@ -693,6 +696,8 @@ static void qcom_pmic_typec_port_stop(struct pmic_typec *tcpm)
for (i = 0; i < pmic_typec_port->nr_irqs; i++)
disable_irq(pmic_typec_port->irq_data[i].irq);
+
+ disable_delayed_work_sync(&pmic_typec_port->cc_debounce_dwork);
}
int qcom_pmic_typec_port_probe(struct platform_device *pdev,
diff --git a/drivers/usb/typec/ucsi/debugfs.c b/drivers/usb/typec/ucsi/debugfs.c
index f67733cecfdf..2f5a3ae1fd85 100644
--- a/drivers/usb/typec/ucsi/debugfs.c
+++ b/drivers/usb/typec/ucsi/debugfs.c
@@ -89,6 +89,7 @@ void ucsi_debugfs_unregister(struct ucsi *ucsi)
debugfs_remove_recursive(ucsi->debugfs->dentry);
kfree(ucsi->debugfs);
+ ucsi->debugfs = NULL;
}
void ucsi_debugfs_init(void)
diff --git a/drivers/usb/typec/ucsi/displayport.c b/drivers/usb/typec/ucsi/displayport.c
index 7067f2561b84..8d2032d0762c 100644
--- a/drivers/usb/typec/ucsi/displayport.c
+++ b/drivers/usb/typec/ucsi/displayport.c
@@ -74,7 +74,7 @@ static int ucsi_displayport_enter(struct typec_altmode *alt, u32 *vdo)
cur = 0xff;
}
- if (cur != 0xff) {
+ if (cur < UCSI_MAX_ALTMODES) {
ret = dp->con->port_altmode[cur] == alt ? 0 : -EBUSY;
goto err_unlock;
}
diff --git a/drivers/usb/typec/ucsi/ucsi.c b/drivers/usb/typec/ucsi/ucsi.c
index 5e7226aed700..b80f68e34d21 100644
--- a/drivers/usb/typec/ucsi/ucsi.c
+++ b/drivers/usb/typec/ucsi/ucsi.c
@@ -2102,6 +2102,8 @@ void ucsi_unregister(struct ucsi *ucsi)
cancel_delayed_work_sync(&ucsi->work);
cancel_work_sync(&ucsi->resume_work);
+ ucsi_debugfs_unregister(ucsi);
+
/* Disable notifications */
ucsi->ops->async_control(ucsi, cmd);
diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
index 27abbe69b55c..25e2c234bcef 100644
--- a/drivers/vdpa/mlx5/core/mr.c
+++ b/drivers/vdpa/mlx5/core/mr.c
@@ -482,7 +482,7 @@ static int add_direct_chain(struct mlx5_vdpa_dev *mvdev,
return 0;
err_alloc:
- list_for_each_entry_safe(dmr, n, &mr->head, list) {
+ list_for_each_entry_safe(dmr, n, &tmp, list) {
list_del_init(&dmr->list);
unmap_direct_mr(mvdev, dmr);
kfree(dmr);
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 481c52be5023..825690c6fed1 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -231,8 +231,11 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr,
kthread_init_work(&vdpasim->work, vdpasim_work_fn);
vdpasim->worker = kthread_create_worker(0, "vDPA sim worker: %s",
dev_attr->name);
- if (IS_ERR(vdpasim->worker))
+ if (IS_ERR(vdpasim->worker)) {
+ ret = PTR_ERR(vdpasim->worker);
+ vdpasim->worker = NULL;
goto err_iommu;
+ }
mutex_init(&vdpasim->mutex);
spin_lock_init(&vdpasim->iommu_lock);
@@ -744,18 +747,24 @@ static void vdpasim_free(struct vdpa_device *vdpa)
struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
int i;
- kthread_cancel_work_sync(&vdpasim->work);
- kthread_destroy_worker(vdpasim->worker);
+ if (vdpasim->worker) {
+ kthread_cancel_work_sync(&vdpasim->work);
+ kthread_destroy_worker(vdpasim->worker);
+ }
- for (i = 0; i < vdpasim->dev_attr.nvqs; i++) {
- vringh_kiov_cleanup(&vdpasim->vqs[i].out_iov);
- vringh_kiov_cleanup(&vdpasim->vqs[i].in_iov);
+ if (vdpasim->vqs) {
+ for (i = 0; i < vdpasim->dev_attr.nvqs; i++) {
+ vringh_kiov_cleanup(&vdpasim->vqs[i].out_iov);
+ vringh_kiov_cleanup(&vdpasim->vqs[i].in_iov);
+ }
}
vdpasim->dev_attr.free(vdpasim);
- for (i = 0; i < vdpasim->dev_attr.nas; i++)
- vhost_iotlb_reset(&vdpasim->iommu[i]);
+ if (vdpasim->iommu) {
+ for (i = 0; i < vdpasim->dev_attr.nas; i++)
+ vhost_iotlb_reset(&vdpasim->iommu[i]);
+ }
kfree(vdpasim->iommu);
kfree(vdpasim->iommu_pt);
kfree(vdpasim->vqs);
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 8ea38e7421df..e1828547603c 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1257,6 +1257,7 @@ static int vfio_msi_cap_len(struct vfio_pci_core_device *vdev, u8 pos)
ret = init_pci_cap_msi_perm(vdev->msi_perm, len, flags);
if (ret) {
kfree(vdev->msi_perm);
+ vdev->msi_perm = NULL;
return ret;
}
diff --git a/drivers/video/fbdev/kyro/fbdev.c b/drivers/video/fbdev/kyro/fbdev.c
index 08ee8baa79f8..cf6062210c8b 100644
--- a/drivers/video/fbdev/kyro/fbdev.c
+++ b/drivers/video/fbdev/kyro/fbdev.c
@@ -369,6 +369,9 @@ static int kyro_dev_overlay_create(u32 ulWidth,
static int kyro_dev_overlay_viewport_set(u32 x, u32 y, u32 ulWidth, u32 ulHeight)
{
+ u32 right;
+ u32 bottom;
+
if (deviceInfo.ulOverlayOffset == 0)
/* probably haven't called CreateOverlay yet */
return -EINVAL;
@@ -378,11 +381,30 @@ static int kyro_dev_overlay_viewport_set(u32 x, u32 y, u32 ulWidth, u32 ulHeight
(x < 2 && ulWidth + 2 == 0))
return -EINVAL;
+ /*
+ * SetOverlayViewPort() adjusts X coordinates by +2 (left) and +1
+ * (right) before packing them into 16-bit register fields.
+ */
+ if (x > U16_MAX - 2 || y > U16_MAX)
+ return -EINVAL;
+
+ right = x + ulWidth;
+ bottom = y + ulHeight;
+
+ if (right < x || bottom < y)
+ return -EINVAL;
+
+ right--;
+ bottom--;
+
+ if (right > U16_MAX - 1 || bottom > U16_MAX)
+ return -EINVAL;
+
/* Stop Ramdac Output */
DisableRamdacOutput(deviceInfo.pSTGReg);
SetOverlayViewPort(deviceInfo.pSTGReg,
- x, y, x + ulWidth - 1, y + ulHeight - 1);
+ x, y, right, bottom);
EnableOverlayPlane(deviceInfo.pSTGReg);
/* Start Ramdac Output */
diff --git a/drivers/video/fbdev/tdfxfb.c b/drivers/video/fbdev/tdfxfb.c
index c5095358209e..eb8f33ae94e7 100644
--- a/drivers/video/fbdev/tdfxfb.c
+++ b/drivers/video/fbdev/tdfxfb.c
@@ -1383,7 +1383,7 @@ static int tdfxfb_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (err)
return err;
- err = pci_enable_device(pdev);
+ err = pcim_enable_device(pdev);
if (err) {
printk(KERN_ERR "tdfxfb: Can't enable pdev: %d\n", err);
return err;
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index b7b9fb80d6a0..02236ee98e59 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -7,6 +7,7 @@
*/
#include <linux/virtio.h>
+#include <uapi/linux/virtio_ring.h>
#include <linux/virtio_balloon.h>
#include <linux/swap.h>
#include <linux/workqueue.h>
@@ -1187,6 +1188,11 @@ static int virtballoon_validate(struct virtio_device *vdev)
else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON))
__virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING);
+ /*
+ * Disable indirect descriptors to avoid memory allocation in
+ * virtqueue_add during page reporting.
+ */
+ __virtio_clear_bit(vdev, VIRTIO_RING_F_INDIRECT_DESC);
__virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM);
return 0;
}
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index 189261d230d2..af31711f3861 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -502,7 +502,7 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs,
if (!avq_num)
return 0;
sprintf(avq->name, "avq.%u", avq->vq_index);
- vq = vp_setup_vq(vdev, queue_idx++, vp_modern_avq_done, avq->name,
+ vq = vp_setup_vq(vdev, avq->vq_index, vp_modern_avq_done, avq->name,
false, VIRTIO_MSI_NO_VECTOR,
&vp_dev->admin_vq.info);
if (IS_ERR(vq)) {
diff --git a/drivers/w1/masters/ds2482.c b/drivers/w1/masters/ds2482.c
index a2ecbb863c57..a7fc6ec31938 100644
--- a/drivers/w1/masters/ds2482.c
+++ b/drivers/w1/masters/ds2482.c
@@ -309,6 +309,10 @@ static u8 ds2482_w1_triplet(void *data, u8 dbit)
mutex_unlock(&pdev->access_lock);
+ /* On bus error, decode to 3 (no device responded) to abort the search */
+ if (status < 0)
+ status = 3 << 5;
+
/* Decode the status */
return (status >> 5);
}
diff --git a/drivers/watchdog/pseries-wdt.c b/drivers/watchdog/pseries-wdt.c
index 7f53b5293409..bad4c63b8310 100644
--- a/drivers/watchdog/pseries-wdt.c
+++ b/drivers/watchdog/pseries-wdt.c
@@ -13,61 +13,10 @@
#include <linux/platform_device.h>
#include <linux/time64.h>
#include <linux/watchdog.h>
+#include <asm/papr-watchdog.h>
#define DRV_NAME "pseries-wdt"
-/*
- * H_WATCHDOG Input
- *
- * R4: "flags":
- *
- * Bits 48-55: "operation"
- */
-#define PSERIES_WDTF_OP_START 0x100UL /* start timer */
-#define PSERIES_WDTF_OP_STOP 0x200UL /* stop timer */
-#define PSERIES_WDTF_OP_QUERY 0x300UL /* query timer capabilities */
-
-/*
- * Bits 56-63: "timeoutAction" (for "Start Watchdog" only)
- */
-#define PSERIES_WDTF_ACTION_HARD_POWEROFF 0x1UL /* poweroff */
-#define PSERIES_WDTF_ACTION_HARD_RESTART 0x2UL /* restart */
-#define PSERIES_WDTF_ACTION_DUMP_RESTART 0x3UL /* dump + restart */
-
-/*
- * H_WATCHDOG Output
- *
- * R3: Return code
- *
- * H_SUCCESS The operation completed.
- *
- * H_BUSY The hypervisor is too busy; retry the operation.
- *
- * H_PARAMETER The given "flags" are somehow invalid. Either the
- * "operation" or "timeoutAction" is invalid, or a
- * reserved bit is set.
- *
- * H_P2 The given "watchdogNumber" is zero or exceeds the
- * supported maximum value.
- *
- * H_P3 The given "timeoutInMs" is below the supported
- * minimum value.
- *
- * H_NOOP The given "watchdogNumber" is already stopped.
- *
- * H_HARDWARE The operation failed for ineffable reasons.
- *
- * H_FUNCTION The H_WATCHDOG hypercall is not supported by this
- * hypervisor.
- *
- * R4:
- *
- * - For the "Query Watchdog Capabilities" operation, a 64-bit
- * structure:
- */
-#define PSERIES_WDTQ_MIN_TIMEOUT(cap) (((cap) >> 48) & 0xffff)
-#define PSERIES_WDTQ_MAX_NUMBER(cap) (((cap) >> 32) & 0xffff)
-
static const unsigned long pseries_wdt_action[] = {
[0] = PSERIES_WDTF_ACTION_HARD_POWEROFF,
[1] = PSERIES_WDTF_ACTION_HARD_RESTART,
diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index dcf9182c8451..75efacd0125b 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -952,19 +952,27 @@ int xs_init(void)
int err;
struct task_struct *task;
- register_reboot_notifier(&xs_reboot_nb);
+ err = register_reboot_notifier(&xs_reboot_nb);
+ if (err)
+ return err;
/* Initialize the shared memory rings to talk to xenstored */
err = xb_init_comms();
if (err)
- return err;
+ goto err_unregister_reboot_notifier;
task = kthread_run(xenwatch_thread, NULL, "xenwatch");
- if (IS_ERR(task))
- return PTR_ERR(task);
+ if (IS_ERR(task)) {
+ err = PTR_ERR(task);
+ goto err_unregister_reboot_notifier;
+ }
/* shutdown watches for kexec boot */
xs_reset_watches();
return 0;
+
+err_unregister_reboot_notifier:
+ unregister_reboot_notifier(&xs_reboot_nb);
+ return err;
}
diff --git a/fs/afs/callback.c b/fs/afs/callback.c
index 4ddc2d666dfb..3e3ae49f28c1 100644
--- a/fs/afs/callback.c
+++ b/fs/afs/callback.c
@@ -138,6 +138,7 @@ static struct afs_volume *afs_lookup_volume_rcu(struct afs_cell *cell,
if (volume && afs_try_get_volume(volume, afs_volume_trace_get_callback))
break;
+ volume = NULL;
if (!need_seqretry(&cell->volume_lock, seq))
break;
}
diff --git a/fs/btrfs/verity.c b/fs/btrfs/verity.c
index e97ad824ae16..2e301f06c573 100644
--- a/fs/btrfs/verity.c
+++ b/fs/btrfs/verity.c
@@ -731,14 +731,18 @@ static struct page *btrfs_read_merkle_tree_page(struct inode *inode,
goto out;
folio_lock(folio);
- /* If it's not uptodate after we have the lock, we got a read error. */
- if (!folio_test_uptodate(folio)) {
+ /* Folio was truncated from mapping. */
+ if (!folio->mapping) {
folio_unlock(folio);
folio_put(folio);
- return ERR_PTR(-EIO);
+ goto again;
}
- folio_unlock(folio);
- goto out;
+ /* Another reader may have filled the folio while we waited. */
+ if (folio_test_uptodate(folio)) {
+ folio_unlock(folio);
+ goto out;
+ }
+ goto read_folio;
}
folio = filemap_alloc_folio(mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS),
@@ -755,6 +759,7 @@ static struct page *btrfs_read_merkle_tree_page(struct inode *inode,
return ERR_PTR(ret);
}
+read_folio:
/*
* Merkle item keys are indexed from byte 0 in the merkle tree.
* They have the form:
@@ -764,6 +769,7 @@ static struct page *btrfs_read_merkle_tree_page(struct inode *inode,
ret = read_key_bytes(BTRFS_I(inode), BTRFS_VERITY_MERKLE_ITEM_KEY, off,
folio_address(folio), PAGE_SIZE, folio);
if (ret < 0) {
+ folio_unlock(folio);
folio_put(folio);
return ERR_PTR(ret);
}
diff --git a/fs/ceph/file.c b/fs/ceph/file.c
index d86c1285fb02..e70f3e433c84 100644
--- a/fs/ceph/file.c
+++ b/fs/ceph/file.c
@@ -2411,6 +2411,54 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)
if (err < 0)
goto out;
+ /*
+ * For O_APPEND writes we may have waited for Fwx exclusive caps
+ * while the previous Fwx holder (another client) extended the
+ * file. i_size has been updated via the cap grant message from
+ * the MDS, but ki_pos is still the old EOF. Re-read i_size here
+ * (no extra MDS round-trip needed) and adjust ki_pos to the true
+ * EOF. Since we hold Fwx, no other client can change the file.
+ */
+ if (iocb->ki_flags & IOCB_APPEND) {
+ loff_t cur_eof = i_size_read(inode);
+
+ if (cur_eof != pos) {
+ doutc(cl,
+ "%p %llx.%llx O_APPEND: pos adjusted %lld -> %lld\n",
+ inode, ceph_vinop(inode), pos, cur_eof);
+ iocb->ki_pos = cur_eof;
+ pos = cur_eof;
+ if (pos >= limit) {
+ err = -EFBIG;
+ goto out_caps;
+ }
+ iov_iter_truncate(from, limit - pos);
+ count = iov_iter_count(from);
+
+ /*
+ * ceph_get_caps() validated the old endoff
+ * against i_max_size; adjusting ki_pos forward
+ * may have shifted the write range beyond the
+ * granted max_size. Re-check and truncate if
+ * necessary.
+ */
+ spin_lock(&ci->i_ceph_lock);
+ if (pos + count > (loff_t)ci->i_max_size) {
+ loff_t max_size = ci->i_max_size;
+
+ spin_unlock(&ci->i_ceph_lock);
+ if (pos >= max_size) {
+ err = -EFBIG;
+ goto out_caps;
+ }
+ iov_iter_truncate(from, max_size - pos);
+ count = iov_iter_count(from);
+ } else {
+ spin_unlock(&ci->i_ceph_lock);
+ }
+ }
+ }
+
err = file_update_time(file);
if (err)
goto out_caps;
diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c
index 44b3fb267ae2..b1f2ef990ff4 100644
--- a/fs/ceph/mds_client.c
+++ b/fs/ceph/mds_client.c
@@ -5795,11 +5795,13 @@ int ceph_mds_check_access(struct ceph_mds_client *mdsc, char *tpath, int mask)
doutc(cl, "tpath '%s', mask %d, caller_uid %d, caller_gid %d\n",
tpath, mask, caller_uid, caller_gid);
+ mutex_lock(&mdsc->mutex);
for (i = 0; i < mdsc->s_cap_auths_num; i++) {
struct ceph_mds_cap_auth *s = &mdsc->s_cap_auths[i];
err = ceph_mds_auth_match(mdsc, s, cred, tpath);
if (err < 0) {
+ mutex_unlock(&mdsc->mutex);
put_cred(cred);
return err;
} else if (err > 0) {
@@ -5821,6 +5823,7 @@ int ceph_mds_check_access(struct ceph_mds_client *mdsc, char *tpath, int mask)
doutc(cl, "root_squash_perms %d, rw_perms_s %p\n", root_squash_perms,
rw_perms_s);
if (root_squash_perms && rw_perms_s == NULL) {
+ mutex_unlock(&mdsc->mutex);
doutc(cl, "access allowed\n");
return 0;
}
@@ -5835,6 +5838,7 @@ int ceph_mds_check_access(struct ceph_mds_client *mdsc, char *tpath, int mask)
!!(mask & MAY_READ), !!(mask & MAY_WRITE));
}
doutc(cl, "access denied\n");
+ mutex_unlock(&mdsc->mutex);
return -EACCES;
}
diff --git a/fs/ceph/mds_client.h b/fs/ceph/mds_client.h
index 8987d0ff9625..b23cd39b1d2e 100644
--- a/fs/ceph/mds_client.h
+++ b/fs/ceph/mds_client.h
@@ -540,6 +540,7 @@ struct ceph_mds_client {
struct rw_semaphore pool_perm_rwsem;
struct rb_root pool_perm_tree;
+ /* protected by mutex */
u32 s_cap_auths_num;
struct ceph_mds_cap_auth *s_cap_auths;
diff --git a/fs/crypto/inline_crypt.c b/fs/crypto/inline_crypt.c
index 0a216d20e695..60d15d5e02ca 100644
--- a/fs/crypto/inline_crypt.c
+++ b/fs/crypto/inline_crypt.c
@@ -275,8 +275,7 @@ static bool bh_get_inode_and_lblk_num(const struct buffer_head *bh,
inode = mapping->host;
*inode_ret = inode;
- *lblk_num_ret = ((u64)folio->index << (PAGE_SHIFT - inode->i_blkbits)) +
- (bh_offset(bh) >> inode->i_blkbits);
+ *lblk_num_ret = (folio_pos(folio) + bh_offset(bh)) >> inode->i_blkbits;
return true;
}
diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig
index 2d36d7b1b344..6acdb0e02256 100644
--- a/fs/erofs/Kconfig
+++ b/fs/erofs/Kconfig
@@ -115,11 +115,13 @@ config EROFS_FS_ZIP_LZMA
If unsure, say N.
config EROFS_FS_ZIP_LZMA_DEFAULT_MAX_STREAMS
- int "EROFS LZMA default maximum decompression streams"
+ int "EROFS LZMA default maximum decompression streams" if SMP
depends on EROFS_FS_ZIP_LZMA
range 1 NR_CPUS if SMP
range 1 1 if !SMP
- default 16
+ default 16 if SMP && NR_CPUS >= 16
+ default NR_CPUS if SMP
+ default 1
help
By default EROFS allocates one LZMA decompression stream per CPU.
Each stream can hold a dictionary of up to 8 MiB taken from the
@@ -127,7 +129,7 @@ config EROFS_FS_ZIP_LZMA_DEFAULT_MAX_STREAMS
of memory. This caps the default; the lzma_streams module parameter
still overrides it.
- If unsure, keep the default of 16.
+ If unsure, keep the suggested default (16 or lower).
config EROFS_FS_ZIP_DEFLATE
bool "EROFS DEFLATE compressed data support"
diff --git a/fs/erofs/compress.h b/fs/erofs/compress.h
index 106fc3315dec..2704d7a592a5 100644
--- a/fs/erofs/compress.h
+++ b/fs/erofs/compress.h
@@ -30,29 +30,8 @@ struct z_erofs_decompressor {
char *name;
};
-/* some special page->private (unsigned long, see below) */
#define Z_EROFS_SHORTLIVED_PAGE (-1UL << 2)
-#define Z_EROFS_PREALLOCATED_PAGE (-2UL << 2)
-
-/*
- * For all pages in a pcluster, page->private should be one of
- * Type Last 2bits page->private
- * short-lived page 00 Z_EROFS_SHORTLIVED_PAGE
- * preallocated page (tryalloc) 00 Z_EROFS_PREALLOCATED_PAGE
- * cached/managed page 00 pointer to z_erofs_pcluster
- * online page (file-backed, 01/10/11 sub-index << 2 | count
- * some pages can be used for inplace I/O)
- *
- * page->mapping should be one of
- * Type page->mapping
- * short-lived page NULL
- * preallocated page NULL
- * cached/managed page non-NULL or NULL (invalidated/truncated page)
- * online page non-NULL
- *
- * For all managed pages, PG_private should be set with 1 extra refcount,
- * which is used for page reclaim / migration.
- */
+#define Z_EROFS_PREALLOCATED_FOLIO ((void *)(-2UL << 2))
/*
* Currently, short-lived pages are pages directly from buddy system
diff --git a/fs/erofs/decompressor.c b/fs/erofs/decompressor.c
index 87009c4d2cf3..39695addce07 100644
--- a/fs/erofs/decompressor.c
+++ b/fs/erofs/decompressor.c
@@ -326,7 +326,7 @@ static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq,
rq->outputsize -= cur;
}
- for (; rq->outputsize; rq->pageofs_in = 0, cur += PAGE_SIZE, ni++) {
+ for (; rq->outputsize; rq->pageofs_in = 0, cur += insz, ni++) {
insz = min(PAGE_SIZE - rq->pageofs_in, rq->outputsize);
rq->outputsize -= insz;
if (!rq->in[ni])
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index d625e3be9ec6..d36562c4afff 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -45,8 +45,8 @@ struct z_erofs_pcluster {
/* A: point to next chained pcluster or TAILs */
struct z_erofs_pcluster *next;
- /* I: start block address of this pcluster */
- erofs_off_t index;
+ /* I: start physical position of this pcluster */
+ erofs_off_t pos;
/* L: the maximum decompression size of this round */
unsigned int length;
@@ -74,6 +74,9 @@ struct z_erofs_pcluster {
/* I: compression algorithm format */
unsigned char algorithmformat;
+ /* I: whether compressed data is in-lined or not */
+ bool from_meta;
+
/* L: whether partial decompression or not */
bool partial;
@@ -100,14 +103,9 @@ struct z_erofs_decompressqueue {
bool eio, sync;
};
-static inline bool z_erofs_is_inline_pcluster(struct z_erofs_pcluster *pcl)
-{
- return !pcl->index;
-}
-
static inline unsigned int z_erofs_pclusterpages(struct z_erofs_pcluster *pcl)
{
- return PAGE_ALIGN(pcl->pclustersize) >> PAGE_SHIFT;
+ return PAGE_ALIGN(pcl->pageofs_in + pcl->pclustersize) >> PAGE_SHIFT;
}
static bool erofs_folio_is_managed(struct erofs_sb_info *sbi, struct folio *fo)
@@ -130,8 +128,18 @@ struct z_erofs_pcluster_slab {
#define _PCLP(n) { .maxpages = n }
static struct z_erofs_pcluster_slab pcluster_pool[] __read_mostly = {
- _PCLP(1), _PCLP(4), _PCLP(16), _PCLP(64), _PCLP(128),
- _PCLP(Z_EROFS_PCLUSTER_MAX_PAGES)
+ _PCLP(1),
+ _PCLP(4),
+#if Z_EROFS_PCLUSTER_MAX_PAGES > 16
+ _PCLP(16),
+#endif
+#if Z_EROFS_PCLUSTER_MAX_PAGES > 64
+ _PCLP(64),
+#endif
+#if Z_EROFS_PCLUSTER_MAX_PAGES > 128
+ _PCLP(128),
+#endif
+ _PCLP(Z_EROFS_PCLUSTER_MAX_PAGES + 1)
};
struct z_erofs_bvec_iter {
@@ -265,7 +273,6 @@ static struct z_erofs_pcluster *z_erofs_alloc_pcluster(unsigned int size)
pcl = kmem_cache_zalloc(pcs->slab, GFP_KERNEL);
if (!pcl)
return ERR_PTR(-ENOMEM);
- pcl->pclustersize = size;
return pcl;
}
return ERR_PTR(-EINVAL);
@@ -514,13 +521,12 @@ static void z_erofs_bind_cache(struct z_erofs_frontend *fe)
struct z_erofs_pcluster *pcl = fe->pcl;
unsigned int pclusterpages = z_erofs_pclusterpages(pcl);
bool shouldalloc = z_erofs_should_alloc_cache(fe);
- bool standalone = true;
- /*
- * optimistic allocation without direct reclaim since inplace I/O
- * can be used if low memory otherwise.
- */
+ pgoff_t poff = pcl->pos >> PAGE_SHIFT;
+ bool may_bypass = true;
+ /* Optimistic allocation, as in-place I/O can be used as a fallback */
gfp_t gfp = (mapping_gfp_mask(mc) & ~__GFP_DIRECT_RECLAIM) |
__GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
+ struct folio *folio, *newfolio;
unsigned int i;
if (i_blocksize(fe->inode) != PAGE_SIZE ||
@@ -528,47 +534,42 @@ static void z_erofs_bind_cache(struct z_erofs_frontend *fe)
return;
for (i = 0; i < pclusterpages; ++i) {
- struct page *page, *newpage;
-
/* Inaccurate check w/o locking to avoid unneeded lookups */
if (READ_ONCE(pcl->compressed_bvecs[i].page))
continue;
- page = find_get_page(mc, pcl->index + i);
- if (!page) {
- /* I/O is needed, no possible to decompress directly */
- standalone = false;
+ folio = filemap_get_folio(mc, poff + i);
+ if (IS_ERR(folio)) {
+ may_bypass = false;
if (!shouldalloc)
continue;
/*
- * Try cached I/O if allocation succeeds or fallback to
- * in-place I/O instead to avoid any direct reclaim.
+ * Allocate a managed folio for cached I/O, or it may be
+ * then filled with a file-backed folio for in-place I/O
*/
- newpage = erofs_allocpage(&fe->pagepool, gfp);
- if (!newpage)
+ newfolio = filemap_alloc_folio(gfp, 0);
+ if (!newfolio)
continue;
- set_page_private(newpage, Z_EROFS_PREALLOCATED_PAGE);
+ newfolio->private = Z_EROFS_PREALLOCATED_FOLIO;
+ folio = NULL;
}
spin_lock(&pcl->lockref.lock);
if (!pcl->compressed_bvecs[i].page) {
- pcl->compressed_bvecs[i].page = page ? page : newpage;
+ pcl->compressed_bvecs[i].page =
+ folio_page(folio ?: newfolio, 0);
spin_unlock(&pcl->lockref.lock);
continue;
}
spin_unlock(&pcl->lockref.lock);
-
- if (page)
- put_page(page);
- else if (newpage)
- erofs_pagepool_add(&fe->pagepool, newpage);
+ folio_put(folio ?: newfolio);
}
/*
- * don't do inplace I/O if all compressed pages are available in
- * managed cache since it can be moved to the bypass queue instead.
+ * Don't perform in-place I/O if all compressed pages are available in
+ * the managed cache, as the pcluster can be moved to the bypass queue.
*/
- if (standalone)
+ if (may_bypass)
fe->mode = Z_EROFS_PCLUSTER_FOLLOWED_NOINPLACE;
}
@@ -580,7 +581,7 @@ static int erofs_try_to_free_all_cached_folios(struct erofs_sb_info *sbi,
struct folio *folio;
int i;
- DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
+ DBG_BUGON(pcl->from_meta);
/* Each cached folio contains one page unless bs > ps is supported */
for (i = 0; i < pclusterpages; ++i) {
if (pcl->compressed_bvecs[i].page) {
@@ -612,7 +613,7 @@ static bool z_erofs_cache_release_folio(struct folio *folio, gfp_t gfp)
ret = false;
spin_lock(&pcl->lockref.lock);
if (pcl->lockref.count <= 0) {
- DBG_BUGON(z_erofs_is_inline_pcluster(pcl));
+ DBG_BUGON(pcl->from_meta);
for (; bvec < end; ++bvec) {
if (bvec->page && page_folio(bvec->page) == folio) {
bvec->page = NULL;
@@ -672,16 +673,20 @@ static int z_erofs_attach_page(struct z_erofs_frontend *fe,
int ret;
if (exclusive) {
- /* give priority for inplaceio to use file pages first */
- spin_lock(&pcl->lockref.lock);
- while (fe->icur > 0) {
- if (pcl->compressed_bvecs[--fe->icur].page)
- continue;
- pcl->compressed_bvecs[fe->icur] = *bvec;
+ /* Inplace I/O is limited to one page for uncompressed data */
+ if (pcl->algorithmformat < Z_EROFS_COMPRESSION_MAX ||
+ fe->icur <= 1) {
+ /* Try to prioritize inplace I/O here */
+ spin_lock(&pcl->lockref.lock);
+ while (fe->icur > 0) {
+ if (pcl->compressed_bvecs[--fe->icur].page)
+ continue;
+ pcl->compressed_bvecs[fe->icur] = *bvec;
+ spin_unlock(&pcl->lockref.lock);
+ return 0;
+ }
spin_unlock(&pcl->lockref.lock);
- return 0;
}
- spin_unlock(&pcl->lockref.lock);
/* otherwise, check if it can be used as a bvpage */
if (fe->mode >= Z_EROFS_PCLUSTER_FOLLOWED &&
@@ -716,28 +721,27 @@ static int z_erofs_register_pcluster(struct z_erofs_frontend *fe)
struct erofs_map_blocks *map = &fe->map;
struct super_block *sb = fe->inode->i_sb;
struct erofs_sb_info *sbi = EROFS_SB(sb);
- bool ztailpacking = map->m_flags & EROFS_MAP_META;
struct z_erofs_pcluster *pcl, *pre;
+ unsigned int pageofs_in;
int err;
- if (!(map->m_flags & EROFS_MAP_ENCODED) ||
- (!ztailpacking && !erofs_blknr(sb, map->m_pa))) {
- DBG_BUGON(1);
- return -EFSCORRUPTED;
- }
-
- /* no available pcluster, let's allocate one */
- pcl = z_erofs_alloc_pcluster(map->m_plen);
+ pageofs_in = erofs_blkoff(sb, map->m_pa);
+ pcl = z_erofs_alloc_pcluster(pageofs_in + map->m_plen);
if (IS_ERR(pcl))
return PTR_ERR(pcl);
spin_lock_init(&pcl->lockref.lock);
pcl->lockref.count = 1; /* one ref for this request */
pcl->algorithmformat = map->m_algorithmformat;
+ pcl->pclustersize = map->m_plen;
+ pcl->pageofs_in = pageofs_in;
pcl->length = 0;
pcl->partial = true;
pcl->next = fe->head;
+ pcl->pos = map->m_pa;
+ pcl->pageofs_in = pageofs_in;
pcl->pageofs_out = map->m_la & ~PAGE_MASK;
+ pcl->from_meta = map->m_flags & EROFS_MAP_META;
fe->mode = Z_EROFS_PCLUSTER_FOLLOWED;
/*
@@ -747,13 +751,10 @@ static int z_erofs_register_pcluster(struct z_erofs_frontend *fe)
mutex_init(&pcl->lock);
DBG_BUGON(!mutex_trylock(&pcl->lock));
- if (ztailpacking) {
- pcl->index = 0; /* which indicates ztailpacking */
- } else {
- pcl->index = erofs_blknr(sb, map->m_pa);
+ if (!pcl->from_meta) {
while (1) {
xa_lock(&sbi->managed_pslots);
- pre = __xa_cmpxchg(&sbi->managed_pslots, pcl->index,
+ pre = __xa_cmpxchg(&sbi->managed_pslots, pcl->pos,
NULL, pcl, GFP_KERNEL);
if (!pre || xa_is_err(pre) || z_erofs_get_pcluster(pre)) {
xa_unlock(&sbi->managed_pslots);
@@ -785,7 +786,6 @@ static int z_erofs_pcluster_begin(struct z_erofs_frontend *fe)
{
struct erofs_map_blocks *map = &fe->map;
struct super_block *sb = fe->inode->i_sb;
- erofs_blk_t blknr = erofs_blknr(sb, map->m_pa);
struct z_erofs_pcluster *pcl = NULL;
void *ptr = NULL;
int ret;
@@ -797,9 +797,9 @@ static int z_erofs_pcluster_begin(struct z_erofs_frontend *fe)
if (!(map->m_flags & EROFS_MAP_META)) {
while (1) {
rcu_read_lock();
- pcl = xa_load(&EROFS_SB(sb)->managed_pslots, blknr);
+ pcl = xa_load(&EROFS_SB(sb)->managed_pslots, map->m_pa);
if (!pcl || z_erofs_get_pcluster(pcl)) {
- DBG_BUGON(pcl && blknr != pcl->index);
+ DBG_BUGON(pcl && map->m_pa != pcl->pos);
rcu_read_unlock();
break;
}
@@ -841,7 +841,7 @@ static int z_erofs_pcluster_begin(struct z_erofs_frontend *fe)
z_erofs_bvec_iter_begin(&fe->biter, &fe->pcl->bvset,
Z_EROFS_INLINE_BVECS, fe->pcl->vcnt);
- if (!z_erofs_is_inline_pcluster(fe->pcl)) {
+ if (!fe->pcl->from_meta) {
/* bind cache first when cached decompression is preferred */
z_erofs_bind_cache(fe);
} else {
@@ -878,7 +878,7 @@ static bool __erofs_try_to_release_pcluster(struct erofs_sb_info *sbi,
* It's impossible to fail after the pcluster is freezed, but in order
* to avoid some race conditions, add a DBG_BUGON to observe this.
*/
- DBG_BUGON(__xa_erase(&sbi->managed_pslots, pcl->index) != pcl);
+ DBG_BUGON(__xa_erase(&sbi->managed_pslots, pcl->pos) != pcl);
lockref_mark_dead(&pcl->lockref);
return true;
@@ -1214,7 +1214,7 @@ static int z_erofs_parse_in_bvecs(struct z_erofs_backend *be, bool *overlapped)
}
be->compressed_pages[i] = page;
- if (z_erofs_is_inline_pcluster(pcl) ||
+ if (pcl->from_meta ||
erofs_folio_is_managed(EROFS_SB(be->sb), page_folio(page))) {
if (!PageUptodate(page))
err = -EIO;
@@ -1292,7 +1292,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, bool eio)
}, be->pagepool);
/* must handle all compressed pages before actual file pages */
- if (z_erofs_is_inline_pcluster(pcl)) {
+ if (pcl->from_meta) {
page = pcl->compressed_bvecs[0].page;
WRITE_ONCE(pcl->compressed_bvecs[0].page, NULL);
put_page(page);
@@ -1351,7 +1351,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, bool eio)
WRITE_ONCE(pcl->next, NULL);
mutex_unlock(&pcl->lock);
- if (z_erofs_is_inline_pcluster(pcl))
+ if (pcl->from_meta)
z_erofs_free_pcluster(pcl);
else
z_erofs_put_pcluster(sbi, pcl, try_free);
@@ -1479,11 +1479,14 @@ static void z_erofs_fill_bio_vec(struct bio_vec *bvec,
folio = page_folio(zbv.page);
/*
- * Handle preallocated cached folios. We tried to allocate such folios
- * without triggering direct reclaim. If allocation failed, inplace
- * file-backed folios will be used instead.
+ * Preallocated folios are added to the managed cache here rather than
+ * in z_erofs_bind_cache() in order to keep these folios locked in
+ * increasing (physical) address order.
+ * Clear folio->private before these folios become visible to others in
+ * the managed cache to avoid duplicate additions for unaligned extents.
*/
- if (folio->private == (void *)Z_EROFS_PREALLOCATED_PAGE) {
+ if (folio->private == Z_EROFS_PREALLOCATED_FOLIO) {
+ folio->private = NULL;
tocache = true;
goto out_tocache;
}
@@ -1519,14 +1522,8 @@ static void z_erofs_fill_bio_vec(struct bio_vec *bvec,
}
return;
}
- /*
- * Already linked with another pcluster, which only appears in
- * crafted images by fuzzers for now. But handle this anyway.
- */
- tocache = false; /* use temporary short-lived pages */
} else {
DBG_BUGON(1); /* referenced managed folios can't be truncated */
- tocache = true;
}
folio_unlock(folio);
folio_put(folio);
@@ -1548,7 +1545,7 @@ static void z_erofs_fill_bio_vec(struct bio_vec *bvec,
folio = page_folio(page);
out_tocache:
if (!tocache || bs != PAGE_SIZE ||
- filemap_add_folio(mc, folio, pcl->index + nr, gfp)) {
+ filemap_add_folio(mc, folio, (pcl->pos >> PAGE_SHIFT) + nr, gfp)) {
/* turn into a temporary shortlived folio (1 ref) */
folio->private = (void *)Z_EROFS_SHORTLIVED_PAGE;
return;
@@ -1665,19 +1662,20 @@ static void z_erofs_submit_queue(struct z_erofs_frontend *f,
pcl = next;
next = READ_ONCE(pcl->next);
- if (z_erofs_is_inline_pcluster(pcl)) {
+ if (pcl->from_meta) {
z_erofs_move_to_bypass_queue(pcl, next, qtail);
continue;
}
/* no device id here, thus it will always succeed */
mdev = (struct erofs_map_dev) {
- .m_pa = erofs_pos(sb, pcl->index),
+ .m_pa = round_down(pcl->pos, sb->s_blocksize),
};
(void)erofs_map_dev(sb, &mdev);
cur = mdev.m_pa;
- end = cur + pcl->pclustersize;
+ end = round_up(cur + pcl->pageofs_in + pcl->pclustersize,
+ sb->s_blocksize);
do {
bvec.bv_page = NULL;
if (bio && (cur != last_pa ||
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 7d949ed0ab5f..fe990ce16ddb 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -292,6 +292,13 @@ static ssize_t ext4_buffered_write_iter(struct kiocb *iocb,
return -EOPNOTSUPP;
inode_lock(inode);
+
+ /*
+ * Prevent concurrent direct I/O and buffered I/O to the same file
+ * range. Wait for in-flight DIO to finish before dirtying pages.
+ */
+ inode_dio_wait(inode);
+
ret = ext4_write_checks(iocb, from);
if (ret <= 0)
goto out;
diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c
index b55bf19c82f2..54eb10c37637 100644
--- a/fs/ext4/inline.c
+++ b/fs/ext4/inline.c
@@ -1554,6 +1554,8 @@ int ext4_read_inline_dir(struct file *file,
/* for other entry, the real offset in
* the buf has to be tuned accordingly.
*/
+ if (i + ext4_dir_rec_len(1, NULL) > extra_size)
+ break;
de = (struct ext4_dir_entry_2 *)
(dir_buf + i - extra_offset);
/* It's too expensive to do a full
@@ -1588,10 +1590,17 @@ int ext4_read_inline_dir(struct file *file,
continue;
}
+ /*
+ * de lives at dir_buf + ctx->pos - extra_offset, within the
+ * kmalloc(inline_size) buffer. Make sure its header fits before
+ * ext4_check_dir_entry() dereferences de->rec_len.
+ */
+ if (ctx->pos + ext4_dir_rec_len(1, NULL) > extra_size)
+ goto out;
de = (struct ext4_dir_entry_2 *)
(dir_buf + ctx->pos - extra_offset);
if (ext4_check_dir_entry(inode, file, de, iloc.bh, dir_buf,
- extra_size, ctx->pos))
+ inline_size, ctx->pos))
goto out;
if (le32_to_cpu(de->inode)) {
if (!dir_emit(ctx, de->name, de->name_len,
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index de8ad1f90af1..fa303effb18a 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -2489,13 +2489,25 @@ static int mpage_prepare_extent_to_map(struct mpage_da_data *mpd)
* page is already under writeback and we are not doing
* a data integrity writeback, skip the page
*/
- if (!folio_test_dirty(folio) ||
- (folio_test_writeback(folio) &&
- (mpd->wbc->sync_mode == WB_SYNC_NONE)) ||
+ if ((folio_test_writeback(folio) &&
+ mpd->wbc->sync_mode == WB_SYNC_NONE) ||
unlikely(folio->mapping != mapping)) {
folio_unlock(folio);
continue;
}
+ /*
+ * If the folio is clean, skip writing it back.
+ * Cycle the folio through the writeback state
+ * though, to clear stale xarray tags.
+ */
+ if (!folio_test_dirty(folio)) {
+ if (!folio_test_writeback(folio)) {
+ __folio_start_writeback(folio, false);
+ folio_end_writeback(folio);
+ }
+ folio_unlock(folio);
+ continue;
+ }
folio_wait_writeback(folio);
BUG_ON(folio_test_writeback(folio));
@@ -5975,6 +5987,16 @@ static int ext4_try_to_expand_extra_isize(struct inode *inode,
if (ext4_test_inode_state(inode, EXT4_STATE_NO_EXPAND))
return -EOVERFLOW;
+ /*
+ * Skip expansion during mount (!SB_ACTIVE). Expanding extra isize
+ * may move xattrs to external blocks and release ea_inodes via iput.
+ * When !SB_ACTIVE, iput triggers write_inode_now() which acquires
+ * s_writepages_rwsem, causing a deadlock with the caller's active
+ * jbd2 handle (lock order: s_writepages_rwsem -> jbd2_handle).
+ */
+ if (unlikely(!(inode->i_sb->s_flags & SB_ACTIVE)))
+ return -EBUSY;
+
/*
* In nojournal mode, we can immediately attempt to expand
* the inode. When journaled, we first need to obtain extra
diff --git a/fs/ext4/migrate.c b/fs/ext4/migrate.c
index 1b0dfd963d3f..84e3b703ab46 100644
--- a/fs/ext4/migrate.c
+++ b/fs/ext4/migrate.c
@@ -458,6 +458,7 @@ int ext4_ext_migrate(struct inode *inode)
if (IS_ERR(tmp_inode)) {
retval = PTR_ERR(tmp_inode);
ext4_journal_stop(handle);
+ tmp_inode = NULL;
goto out_unlock;
}
/*
@@ -585,9 +586,9 @@ int ext4_ext_migrate(struct inode *inode)
ext4_journal_stop(handle);
out_tmp_inode:
unlock_new_inode(tmp_inode);
- iput(tmp_inode);
out_unlock:
ext4_writepages_up_write(inode->i_sb, alloc_ctx);
+ iput(tmp_inode);
return retval;
}
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 29c7c0b8295f..9b2f688b5aaa 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -1499,6 +1499,8 @@ int ext4_search_dir(struct buffer_head *bh, char *search_buf, int buf_size,
/* this code is executed quadratically often */
/* do minimal checking `by hand' */
if (de->name + de->name_len <= dlimit &&
+ (!ext4_hash_in_dirent(dir) ||
+ (char *)de + ext4_dir_rec_len(de->name_len, dir) <= dlimit) &&
ext4_match(dir, fname, de)) {
/* found a match - just to be sure, do
* a full check */
diff --git a/fs/ext4/orphan.c b/fs/ext4/orphan.c
index e5a6cb533242..270a50f67960 100644
--- a/fs/ext4/orphan.c
+++ b/fs/ext4/orphan.c
@@ -388,7 +388,7 @@ void ext4_orphan_cleanup(struct super_block *sb, struct ext4_super_block *es)
struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info;
int inodes_per_ob = ext4_inodes_per_orphan_block(sb);
- if (!es->s_last_orphan && !oi->of_blocks) {
+ if (!es->s_last_orphan && ext4_orphan_file_empty(sb)) {
ext4_debug("no orphan inodes to clean up\n");
return;
}
@@ -573,6 +573,7 @@ int ext4_init_orphan_info(struct super_block *sb)
int i, j;
int ret;
int free;
+ int loaded = 0;
__le32 *bdata;
int inodes_per_ob = ext4_inodes_per_orphan_block(sb);
struct ext4_orphan_block_tail *ot;
@@ -616,6 +617,7 @@ int ext4_init_orphan_info(struct super_block *sb)
ret = -EIO;
goto out_free;
}
+ loaded++;
ot = ext4_orphan_block_tail(sb, oi->of_binfo[i].ob_bh);
if (le32_to_cpu(ot->ob_magic) != EXT4_ORPHAN_BLOCK_MAGIC) {
ext4_error(sb, "orphan file block %d: bad magic", i);
@@ -638,8 +640,10 @@ int ext4_init_orphan_info(struct super_block *sb)
iput(inode);
return 0;
out_free:
- for (i--; i >= 0; i--)
- brelse(oi->of_binfo[i].ob_bh);
+ while (loaded > 0) {
+ loaded--;
+ brelse(oi->of_binfo[loaded].ob_bh);
+ }
kvfree(oi->of_binfo);
out_put:
iput(inode);
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index ec816df4887c..f8154a431fc9 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1454,8 +1454,11 @@ static int __allocate_data_block(struct dnode_of_data *dn, int seg_type)
old_blkaddr = dn->data_blkaddr;
err = f2fs_allocate_data_block(sbi, NULL, old_blkaddr,
&dn->data_blkaddr, &sum, seg_type, NULL);
- if (err)
+ if (err) {
+ if (old_blkaddr == NULL_ADDR)
+ dec_valid_block_count(sbi, dn->inode, count);
return err;
+ }
if (GET_SEGNO(sbi, old_blkaddr) != NULL_SEGNO)
f2fs_invalidate_internal_cache(sbi, old_blkaddr);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index e1cbd3917b9d..095f2c2953f1 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1035,7 +1035,7 @@ static inline void set_new_dnode(struct dnode_of_data *dn, struct inode *inode,
#define NR_CURSEG_PERSIST_TYPE (NR_CURSEG_DATA_TYPE + NR_CURSEG_NODE_TYPE)
#define NR_CURSEG_TYPE (NR_CURSEG_INMEM_TYPE + NR_CURSEG_PERSIST_TYPE)
-enum {
+enum log_type {
CURSEG_HOT_DATA = 0, /* directory entry blocks */
CURSEG_WARM_DATA, /* data blocks */
CURSEG_COLD_DATA, /* multimedia or GCed data blocks */
@@ -3823,7 +3823,8 @@ void f2fs_replace_block(struct f2fs_sb_info *sbi, struct dnode_of_data *dn,
block_t old_addr, block_t new_addr,
unsigned char version, bool recover_curseg,
bool recover_newaddr);
-int f2fs_get_segment_temp(int seg_type);
+enum temp_type f2fs_get_segment_temp(struct f2fs_sb_info *sbi,
+ enum log_type seg_type);
int f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page,
block_t old_blkaddr, block_t *new_blkaddr,
struct f2fs_summary *sum, int type,
@@ -3857,10 +3858,26 @@ unsigned int f2fs_usable_blks_in_seg(struct f2fs_sb_info *sbi,
#define MIN_FRAGMENT_SIZE 1
#define MAX_FRAGMENT_SIZE 512
-static inline bool f2fs_need_rand_seg(struct f2fs_sb_info *sbi)
+static inline bool f2fs_need_rand_blk(struct f2fs_sb_info *sbi,
+ enum log_type type)
+{
+ if (type == CURSEG_COLD_DATA_PINNED)
+ return false;
+ return F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_BLK;
+}
+
+static inline bool f2fs_need_rand_seg(struct f2fs_sb_info *sbi,
+ enum log_type type)
+{
+ if (type == CURSEG_COLD_DATA_PINNED)
+ return false;
+ return F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_SEG;
+}
+
+static inline bool f2fs_need_rand_seg_blk(struct f2fs_sb_info *sbi,
+ enum log_type type)
{
- return F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_SEG ||
- F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_BLK;
+ return f2fs_need_rand_blk(sbi, type) || f2fs_need_rand_seg(sbi, type);
}
/*
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 5f69bdca95c2..129cdee4149a 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -35,23 +35,69 @@
#include <trace/events/f2fs.h>
#include <uapi/linux/f2fs.h>
-static void f2fs_zero_post_eof_page(struct inode *inode,
- loff_t new_size, bool lock)
+static int fill_zero(struct inode *inode, pgoff_t index,
+ loff_t start, loff_t len);
+
+static int do_zero_post_eof_page(struct inode *inode, loff_t new_size)
{
loff_t old_size = i_size_read(inode);
+ unsigned int offset, len;
+ pgoff_t index;
+ int err;
+
+ offset = old_size & (PAGE_SIZE - 1);
+
+ if (!offset)
+ return 0;
+
+ len = min_t(loff_t, PAGE_SIZE - offset, new_size - old_size);
+ index = old_size >> PAGE_SHIFT;
+
+ if (f2fs_has_inline_data(inode)) {
+ /* data post eof should be always zero */
+ if (new_size <= MAX_INLINE_DATA(inode))
+ return 0;
+ err = f2fs_convert_inline_inode(inode);
+ if (err)
+ return err;
+ }
+
+ err = fill_zero(inode, index, offset, len);
+ if (err)
+ return err;
+ return filemap_write_and_wait_range(inode->i_mapping,
+ old_size, old_size + len - 1);
+}
+
+static int f2fs_zero_post_eof_page(struct inode *inode,
+ loff_t new_size, bool lock, bool writeback)
+{
+ loff_t old_size = i_size_read(inode);
+ bool strict =
+ F2FS_OPTION(F2FS_I_SB(inode)).fsync_mode == FSYNC_MODE_STRICT;
if (old_size >= new_size)
- return;
+ return 0;
- if (mapping_empty(inode->i_mapping))
- return;
+ if (!strict && mapping_empty(inode->i_mapping))
+ return 0;
if (lock)
filemap_invalidate_lock(inode->i_mapping);
/* zero or drop pages only in range of [old_size, new_size] */
- truncate_inode_pages_range(inode->i_mapping, old_size, new_size);
+ truncate_inode_pages_range(inode->i_mapping, old_size, new_size - 1);
if (lock)
filemap_invalidate_unlock(inode->i_mapping);
+
+ if (!writeback || !strict)
+ return 0;
+ /*
+ * In fsync_mode=strict, when we expand an unaligned EOF size, we
+ * should zero post EOF data and writeback the data immediately,
+ * so that it can avoid exposing stale data after metadata flush
+ * and POR.
+ */
+ return do_zero_post_eof_page(inode, new_size);
}
static vm_fault_t f2fs_filemap_fault(struct vm_fault *vmf)
@@ -122,7 +168,10 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
- f2fs_zero_post_eof_page(inode, (folio->index + 1) << PAGE_SHIFT, true);
+ err = f2fs_zero_post_eof_page(inode,
+ (folio->index + 1) << PAGE_SHIFT, true, false);
+ if (err)
+ goto out_pagefault;
file_update_time(vmf->vma->vm_file);
filemap_invalidate_lock_shared(inode->i_mapping);
@@ -179,7 +228,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
out_sem:
filemap_invalidate_unlock_shared(inode->i_mapping);
-
+out_pagefault:
sb_end_pagefault(inode->i_sb);
out:
ret = vmf_fs_error(err);
@@ -1097,8 +1146,12 @@ int f2fs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- if (attr->ia_size > old_size)
- f2fs_zero_post_eof_page(inode, attr->ia_size, false);
+ if (attr->ia_size > old_size) {
+ err = f2fs_zero_post_eof_page(inode,
+ attr->ia_size, false, true);
+ if (err)
+ goto err_out;
+ }
truncate_setsize(inode, attr->ia_size);
if (attr->ia_size <= old_size)
@@ -1107,6 +1160,7 @@ int f2fs_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
* do not trim all blocks after i_size if target size is
* larger than i_size.
*/
+err_out:
filemap_invalidate_unlock(inode->i_mapping);
f2fs_up_write(&fi->i_gc_rwsem[WRITE]);
if (err)
@@ -1217,7 +1271,9 @@ static int f2fs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- f2fs_zero_post_eof_page(inode, offset + len, true);
+ ret = f2fs_zero_post_eof_page(inode, offset + len, true, false);
+ if (ret)
+ return ret;
pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
@@ -1502,7 +1558,9 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- f2fs_zero_post_eof_page(inode, offset + len, false);
+ ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
+ if (ret)
+ goto out_unlock;
f2fs_lock_op(sbi);
f2fs_drop_extent_tree(inode);
@@ -1510,6 +1568,7 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
ret = __exchange_data_block(inode, inode, end, start, nrpages - end, true);
f2fs_unlock_op(sbi);
+out_unlock:
filemap_invalidate_unlock(inode->i_mapping);
f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
@@ -1542,11 +1601,14 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len)
/* write out all moved pages, if possible */
filemap_invalidate_lock(inode->i_mapping);
- filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
+ ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
+ if (ret)
+ goto out_unlock;
truncate_pagecache(inode, offset);
new_size = i_size_read(inode) - len;
ret = f2fs_truncate_blocks(inode, new_size, true);
+out_unlock:
filemap_invalidate_unlock(inode->i_mapping);
if (!ret)
f2fs_i_size_write(inode, new_size);
@@ -1628,7 +1690,9 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
if (ret)
return ret;
- f2fs_zero_post_eof_page(inode, offset + len, true);
+ ret = f2fs_zero_post_eof_page(inode, offset + len, true, false);
+ if (ret)
+ return ret;
pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
@@ -1762,7 +1826,9 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(mapping);
- f2fs_zero_post_eof_page(inode, offset + len, false);
+ ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
+ if (ret)
+ goto out_unlock;
truncate_pagecache(inode, offset);
while (!ret && idx > pg_start) {
@@ -1778,6 +1844,7 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
idx + delta, nr, false);
f2fs_unlock_op(sbi);
}
+out_unlock:
filemap_invalidate_unlock(mapping);
f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
if (ret)
@@ -1820,7 +1887,9 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
if (err)
return err;
- f2fs_zero_post_eof_page(inode, offset + len, true);
+ err = f2fs_zero_post_eof_page(inode, offset + len, true, true);
+ if (err)
+ return err;
f2fs_balance_fs(sbi, true);
@@ -1841,8 +1910,9 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
block_t sec_len;
if (map.m_lblk % sec_blks) {
- map.m_lblk = rounddown(map.m_lblk, sec_blks);
- map.m_len = pg_end - map.m_lblk;
+ pg_start = rounddown(map.m_lblk, sec_blks);
+ map.m_lblk = pg_start;
+ map.m_len = pg_end - pg_start;
if (off_end)
map.m_len++;
}
@@ -3053,8 +3123,6 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
if (src == dst) {
if (pos_in == pos_out)
return 0;
- if (pos_out > pos_in && pos_out < pos_in + len)
- return -EINVAL;
}
inode_lock(src);
@@ -3080,6 +3148,8 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
goto out_unlock;
if (len == 0)
olen = len = src->i_size - pos_in;
+ if (src == dst && pos_out > pos_in && pos_out < pos_in + len)
+ goto out_unlock;
if (pos_in + len == src->i_size)
len = ALIGN(src->i_size, F2FS_BLKSIZE) - pos_in;
if (len == 0) {
@@ -4835,8 +4905,10 @@ static ssize_t f2fs_write_checks(struct kiocb *iocb, struct iov_iter *from)
if (err)
return err;
- f2fs_zero_post_eof_page(inode,
- iocb->ki_pos + iov_iter_count(from), true);
+ err = f2fs_zero_post_eof_page(inode,
+ iocb->ki_pos + iov_iter_count(from), true, true);
+ if (err)
+ return err;
return count;
}
@@ -4950,8 +5022,8 @@ static void f2fs_dio_write_submit_io(const struct iomap_iter *iter,
{
struct inode *inode = iter->inode;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int seg_type = f2fs_rw_hint_to_seg_type(sbi, inode->i_write_hint);
- enum temp_type temp = f2fs_get_segment_temp(seg_type);
+ enum log_type type = f2fs_rw_hint_to_seg_type(sbi, inode->i_write_hint);
+ enum temp_type temp = f2fs_get_segment_temp(sbi, type);
bio->bi_write_hint = f2fs_io_type_to_rw_hint(sbi, DATA, temp);
submit_bio(bio);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index b8a9412ca1dc..49d1acd26a4c 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -317,10 +317,11 @@ static void select_policy(struct f2fs_sb_info *sbi, int gc_type,
p->max_search = sbi->max_victim_search;
/* let's select beginning hot/small space first. */
- if (f2fs_need_rand_seg(sbi))
+ if (f2fs_need_rand_seg_blk(sbi, type)) {
p->offset = get_random_u32_below(MAIN_SECS(sbi) *
SEGS_PER_SEC(sbi));
- else if (type == CURSEG_HOT_DATA || IS_NODESEG(type))
+ SIT_I(sbi)->last_victim[p->gc_mode] = p->offset;
+ } else if (type == CURSEG_HOT_DATA || IS_NODESEG(type))
p->offset = 0;
else
p->offset = SIT_I(sbi)->last_victim[p->gc_mode];
@@ -2171,7 +2172,7 @@ static int free_segment_range(struct f2fs_sb_info *sbi,
mutex_unlock(&DIRTY_I(sbi)->seglist_lock);
/* Move out cursegs from the target range */
- for (type = CURSEG_HOT_DATA; type < NR_CURSEG_PERSIST_TYPE; type++) {
+ for (type = CURSEG_HOT_DATA; type < NR_CURSEG_TYPE; type++) {
err = f2fs_allocate_segment_for_resize(sbi, type, start, end);
if (err)
goto out;
diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index da213cd39fc7..1d204294abf9 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -672,15 +672,16 @@ static int f2fs_symlink(struct mnt_idmap *idmap, struct inode *dir,
* performance regression.
*/
if (!err) {
- filemap_write_and_wait_range(inode->i_mapping, 0,
- disk_link.len - 1);
+ err = filemap_write_and_wait_range(inode->i_mapping, 0,
+ disk_link.len - 1);
- if (IS_DIRSYNC(dir))
+ if (!err && IS_DIRSYNC(dir))
f2fs_sync_fs(sbi->sb, 1);
- } else {
- f2fs_unlink(dir, dentry);
}
+ if (err)
+ f2fs_unlink(dir, dentry);
+
f2fs_balance_fs(sbi, true);
goto out_free_encrypted_link;
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 4e47826b570a..e3797e99271e 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -2843,7 +2843,7 @@ static unsigned int __get_next_segno(struct f2fs_sb_info *sbi, int type)
sanity_check_seg_type(sbi, seg_type);
if (__is_large_section(sbi)) {
- if (f2fs_need_rand_seg(sbi)) {
+ if (f2fs_need_rand_seg_blk(sbi, type)) {
unsigned int hint = GET_SEC_FROM_SEG(sbi, curseg->segno);
if (GET_SEC_FROM_SEG(sbi, curseg->segno + 1) != hint)
@@ -2852,7 +2852,7 @@ static unsigned int __get_next_segno(struct f2fs_sb_info *sbi, int type)
GET_SEG_FROM_SEC(sbi, hint + 1) - 1);
}
return curseg->segno;
- } else if (f2fs_need_rand_seg(sbi)) {
+ } else if (f2fs_need_rand_seg_blk(sbi, type)) {
return get_random_u32_below(MAIN_SECS(sbi) * SEGS_PER_SEC(sbi));
}
@@ -2908,7 +2908,7 @@ static int new_curseg(struct f2fs_sb_info *sbi, int type, bool new_sec)
curseg->next_segno = segno;
reset_curseg(sbi, type, 1);
curseg->alloc_type = LFS;
- if (F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_BLK)
+ if (f2fs_need_rand_blk(sbi, type))
curseg->fragment_remained_chunk =
get_random_u32_inclusive(1, sbi->max_fragment_chunk);
return 0;
@@ -3610,18 +3610,35 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
}
}
-int f2fs_get_segment_temp(int seg_type)
+enum temp_type f2fs_get_segment_temp(struct f2fs_sb_info *sbi,
+ enum log_type type)
{
- if (IS_HOT(seg_type))
- return HOT;
- else if (IS_WARM(seg_type))
- return WARM;
- return COLD;
+ struct curseg_info *curseg = CURSEG_I(sbi, type);
+ enum temp_type temp = COLD;
+
+ switch (curseg->seg_type) {
+ case CURSEG_HOT_NODE:
+ case CURSEG_HOT_DATA:
+ temp = HOT;
+ break;
+ case CURSEG_WARM_NODE:
+ case CURSEG_WARM_DATA:
+ temp = WARM;
+ break;
+ case CURSEG_COLD_NODE:
+ case CURSEG_COLD_DATA:
+ temp = COLD;
+ break;
+ default:
+ f2fs_bug_on(sbi, 1);
+ }
+
+ return temp;
}
static int __get_segment_type(struct f2fs_io_info *fio)
{
- int type = 0;
+ enum log_type type = CURSEG_HOT_DATA;
switch (F2FS_OPTION(fio->sbi).active_logs) {
case 2:
@@ -3637,7 +3654,7 @@ static int __get_segment_type(struct f2fs_io_info *fio)
f2fs_bug_on(fio->sbi, true);
}
- fio->temp = f2fs_get_segment_temp(type);
+ fio->temp = f2fs_get_segment_temp(fio->sbi, type);
return type;
}
@@ -3695,7 +3712,7 @@ int f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page,
curseg->next_blkoff = f2fs_find_next_ssr_block(sbi, curseg);
} else {
curseg->next_blkoff++;
- if (F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_BLK)
+ if (f2fs_need_rand_blk(sbi, type))
f2fs_randomize_chunk(sbi, curseg);
}
if (curseg->next_blkoff >= f2fs_usable_blks_in_seg(sbi, curseg->segno))
diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h
index f8f94301350c..5e913db63360 100644
--- a/fs/f2fs/segment.h
+++ b/fs/f2fs/segment.h
@@ -32,10 +32,6 @@ static inline void sanity_check_seg_type(struct f2fs_sb_info *sbi,
f2fs_bug_on(sbi, seg_type >= NR_PERSISTENT_LOG);
}
-#define IS_HOT(t) ((t) == CURSEG_HOT_NODE || (t) == CURSEG_HOT_DATA)
-#define IS_WARM(t) ((t) == CURSEG_WARM_NODE || (t) == CURSEG_WARM_DATA)
-#define IS_COLD(t) ((t) == CURSEG_COLD_NODE || (t) == CURSEG_COLD_DATA)
-
#define IS_CURSEG(sbi, seg) \
(((seg) == CURSEG_I(sbi, CURSEG_HOT_DATA)->segno) || \
((seg) == CURSEG_I(sbi, CURSEG_WARM_DATA)->segno) || \
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 8ceffecf28c8..6bf2e0c7854c 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -2059,9 +2059,9 @@ static int f2fs_show_options(struct seq_file *seq, struct dentry *root)
seq_puts(seq, "adaptive");
else if (F2FS_OPTION(sbi).fs_mode == FS_MODE_LFS)
seq_puts(seq, "lfs");
- else if (F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_SEG)
+ else if (f2fs_need_rand_seg(sbi, NO_CHECK_TYPE))
seq_puts(seq, "fragment:segment");
- else if (F2FS_OPTION(sbi).fs_mode == FS_MODE_FRAGMENT_BLK)
+ else if (f2fs_need_rand_blk(sbi, NO_CHECK_TYPE))
seq_puts(seq, "fragment:block");
seq_printf(seq, ",active_logs=%u", F2FS_OPTION(sbi).active_logs);
if (test_opt(sbi, RESERVE_ROOT))
diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c
index 4e1021eb372e..56c50faa3ca5 100644
--- a/fs/f2fs/sysfs.c
+++ b/fs/f2fs/sysfs.c
@@ -528,7 +528,7 @@ static ssize_t __sbi_store(struct f2fs_attr *a,
return -EINVAL;
cprc->ckpt_thread_ioprio = IOPRIO_PRIO_VALUE(class, level);
- if (test_opt(sbi, MERGE_CHECKPOINT)) {
+ if (cprc->f2fs_issue_ckpt) {
ret = set_task_ioprio(cprc->f2fs_issue_ckpt,
cprc->ckpt_thread_ioprio);
if (ret)
@@ -924,13 +924,14 @@ static ssize_t f2fs_sbi_store(struct f2fs_attr *a,
ssize_t ret;
bool gc_entry = (!strcmp(a->attr.name, "gc_urgent") ||
a->struct_type == GC_THREAD);
+ bool thread_entry = !strcmp(a->attr.name, "ckpt_thread_ioprio");
- if (gc_entry) {
+ if (gc_entry || thread_entry) {
if (!down_read_trylock(&sbi->sb->s_umount))
return -EAGAIN;
}
ret = __sbi_store(a, sbi, buf, count);
- if (gc_entry)
+ if (gc_entry || thread_entry)
up_read(&sbi->sb->s_umount);
return ret;
diff --git a/fs/f2fs/xattr.c b/fs/f2fs/xattr.c
index eac4c0d934f7..a267ab81ddca 100644
--- a/fs/f2fs/xattr.c
+++ b/fs/f2fs/xattr.c
@@ -118,7 +118,7 @@ static int f2fs_xattr_advise_set(const struct xattr_handler *handler,
unsigned char old_advise = F2FS_I(inode)->i_advise;
unsigned char new_advise;
- if (!inode_owner_or_capable(&nop_mnt_idmap, inode))
+ if (!inode_owner_or_capable(idmap, inode))
return -EPERM;
if (value == NULL)
return -EINVAL;
diff --git a/fs/fat/nfs.c b/fs/fat/nfs.c
index 509eea96a457..6e1b371711ed 100644
--- a/fs/fat/nfs.c
+++ b/fs/fat/nfs.c
@@ -250,8 +250,10 @@ struct inode *fat_rebuild_parent(struct super_block *sb, int parent_logstart)
MSDOS_I(dummy_grand_parent)->i_pos = -1;
}
- if (!fat_scan_logstart(dummy_grand_parent, clus_to_match, &sinfo))
+ if (!fat_scan_logstart(dummy_grand_parent, clus_to_match, &sinfo)) {
parent = fat_build_inode(sb, sinfo.de, sinfo.i_pos);
+ brelse(sinfo.bh);
+ }
brelse(parent_bh);
iput(dummy_grand_parent);
diff --git a/fs/fuse/cuse.c b/fs/fuse/cuse.c
index 22e82dae6c6b..47c8e17cc028 100644
--- a/fs/fuse/cuse.c
+++ b/fs/fuse/cuse.c
@@ -647,6 +647,11 @@ static void __exit cuse_exit(void)
{
misc_deregister(&cuse_miscdev);
class_destroy(cuse_class);
+ /*
+ * Wait for pending call_rcu() callbacks that call back into
+ * this module via fc->release (cuse_fc_release).
+ */
+ rcu_barrier();
}
module_init(cuse_init);
diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index c3e7267517a8..865c857c72ae 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -1023,17 +1023,41 @@ static int fuse_copy_pages(struct fuse_copy_state *cs, unsigned nbytes,
struct fuse_req *req = cs->req;
struct fuse_args_pages *ap = container_of(req->args, typeof(*ap), args);
+ if (ap->uses_folios) {
+ for (i = 0; i < ap->num_folios && (nbytes || zeroing); i++) {
+ int err;
+ unsigned int offset = ap->folio_descs[i].offset;
+ unsigned int count = min(nbytes, ap->folio_descs[i].length);
+ struct page *orig, *pagep;
- for (i = 0; i < ap->num_pages && (nbytes || zeroing); i++) {
- int err;
- unsigned int offset = ap->descs[i].offset;
- unsigned int count = min(nbytes, ap->descs[i].length);
+ orig = pagep = &ap->folios[i]->page;
- err = fuse_copy_page(cs, &ap->pages[i], offset, count, zeroing);
- if (err)
- return err;
+ err = fuse_copy_page(cs, &pagep, offset, count, zeroing);
+ if (err)
+ return err;
+
+ nbytes -= count;
+
+ /*
+ * fuse_copy_page may have moved a page from a pipe
+ * instead of copying into our given page, so update
+ * the folios if it was replaced.
+ */
+ if (pagep != orig)
+ ap->folios[i] = page_folio(pagep);
+ }
+ } else {
+ for (i = 0; i < ap->num_pages && (nbytes || zeroing); i++) {
+ int err;
+ unsigned int offset = ap->descs[i].offset;
+ unsigned int count = min(nbytes, ap->descs[i].length);
- nbytes -= count;
+ err = fuse_copy_page(cs, &ap->pages[i], offset, count, zeroing);
+ if (err)
+ return err;
+
+ nbytes -= count;
+ }
}
return 0;
}
diff --git a/fs/fuse/dir.c b/fs/fuse/dir.c
index abb3cb8ae688..45cbe0cb22a0 100644
--- a/fs/fuse/dir.c
+++ b/fs/fuse/dir.c
@@ -366,7 +366,7 @@ int fuse_lookup_name(struct super_block *sb, u64 nodeid, const struct qstr *name
struct fuse_mount *fm = get_fuse_mount_super(sb);
FUSE_ARGS(args);
struct fuse_forget_link *forget;
- u64 attr_version;
+ u64 attr_version, evict_ctr;
int err;
*inode = NULL;
@@ -381,6 +381,7 @@ int fuse_lookup_name(struct super_block *sb, u64 nodeid, const struct qstr *name
goto out;
attr_version = fuse_get_attr_version(fm->fc);
+ evict_ctr = fuse_get_evict_ctr(fm->fc);
fuse_lookup_init(fm->fc, &args, nodeid, name, outarg);
err = fuse_simple_request(fm, &args);
@@ -398,7 +399,7 @@ int fuse_lookup_name(struct super_block *sb, u64 nodeid, const struct qstr *name
*inode = fuse_iget(sb, outarg->nodeid, outarg->generation,
&outarg->attr, ATTR_TIMEOUT(outarg),
- attr_version);
+ attr_version, evict_ctr);
err = -ENOMEM;
if (!*inode) {
fuse_queue_forget(fm->fc, forget, outarg->nodeid, 1);
@@ -691,7 +692,7 @@ static int fuse_create_open(struct mnt_idmap *idmap, struct inode *dir,
ff->nodeid = outentry.nodeid;
ff->open_flags = outopenp->open_flags;
inode = fuse_iget(dir->i_sb, outentry.nodeid, outentry.generation,
- &outentry.attr, ATTR_TIMEOUT(&outentry), 0);
+ &outentry.attr, ATTR_TIMEOUT(&outentry), 0, 0);
if (!inode) {
flags &= ~(O_CREAT | O_EXCL | O_TRUNC);
fuse_sync_release(NULL, ff, flags);
@@ -822,7 +823,7 @@ static int create_new_entry(struct mnt_idmap *idmap, struct fuse_mount *fm,
goto out_put_forget_req;
inode = fuse_iget(dir->i_sb, outarg.nodeid, outarg.generation,
- &outarg.attr, ATTR_TIMEOUT(&outarg), 0);
+ &outarg.attr, ATTR_TIMEOUT(&outarg), 0, 0);
if (!inode) {
fuse_queue_forget(fm->fc, forget, outarg.nodeid, 1);
return -ENOMEM;
@@ -2041,7 +2042,7 @@ int fuse_do_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
fuse_change_attributes_common(inode, &outarg.attr, NULL,
ATTR_TIMEOUT(&outarg),
- fuse_get_cache_mask(inode));
+ fuse_get_cache_mask(inode), 0);
oldsize = inode->i_size;
/* see the comment in fuse_change_attributes() */
if (!is_wb || is_truncate)
diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
index 28cf319c1c25..4518d12f4f35 100644
--- a/fs/fuse/fuse_i.h
+++ b/fs/fuse/fuse_i.h
@@ -291,6 +291,12 @@ struct fuse_page_desc {
unsigned int offset;
};
+/** FUSE folio descriptor */
+struct fuse_folio_desc {
+ unsigned int length;
+ unsigned int offset;
+};
+
struct fuse_args {
uint64_t nodeid;
uint32_t opcode;
@@ -319,9 +325,19 @@ struct fuse_args {
struct fuse_args_pages {
struct fuse_args args;
- struct page **pages;
- struct fuse_page_desc *descs;
- unsigned int num_pages;
+ union {
+ struct {
+ struct page **pages;
+ struct fuse_page_desc *descs;
+ unsigned int num_pages;
+ };
+ struct {
+ struct folio **folios;
+ struct fuse_folio_desc *folio_descs;
+ unsigned int num_folios;
+ };
+ };
+ bool uses_folios;
};
struct fuse_release_args {
@@ -890,6 +906,9 @@ struct fuse_conn {
/** Version counter for attribute changes */
atomic64_t attr_version;
+ /** Version counter for evict inode */
+ atomic64_t evict_ctr;
+
/** Called on final put */
void (*release)(struct fuse_conn *);
@@ -984,6 +1003,11 @@ static inline u64 fuse_get_attr_version(struct fuse_conn *fc)
return atomic64_read(&fc->attr_version);
}
+static inline u64 fuse_get_evict_ctr(struct fuse_conn *fc)
+{
+ return atomic64_read(&fc->evict_ctr);
+}
+
static inline bool fuse_stale_inode(const struct inode *inode, int generation,
struct fuse_attr *attr)
{
@@ -1043,7 +1067,8 @@ extern const struct dentry_operations fuse_root_dentry_operations;
*/
struct inode *fuse_iget(struct super_block *sb, u64 nodeid,
int generation, struct fuse_attr *attr,
- u64 attr_valid, u64 attr_version);
+ u64 attr_valid, u64 attr_version,
+ u64 evict_ctr);
int fuse_lookup_name(struct super_block *sb, u64 nodeid, const struct qstr *name,
struct fuse_entry_out *outarg, struct inode **inode);
@@ -1133,7 +1158,8 @@ void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
void fuse_change_attributes_common(struct inode *inode, struct fuse_attr *attr,
struct fuse_statx *sx,
- u64 attr_valid, u32 cache_mask);
+ u64 attr_valid, u32 cache_mask,
+ u64 evict_ctr);
u32 fuse_get_cache_mask(struct inode *inode);
diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index 153f3102f167..08a47565cdad 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -170,6 +170,14 @@ static void fuse_evict_inode(struct inode *inode)
fuse_cleanup_submount_lookup(fc, fi->submount_lookup);
fi->submount_lookup = NULL;
}
+ /*
+ * Evict of non-deleted inode may race with outstanding
+ * LOOKUP/READDIRPLUS requests and result in inconsistency when
+ * the request finishes. Deal with that here by bumping a
+ * counter that can be compared to the starting value.
+ */
+ if (inode->i_nlink > 0)
+ atomic64_inc(&fc->evict_ctr);
}
if (S_ISREG(inode->i_mode) && !fuse_is_bad(inode)) {
WARN_ON(fi->iocachectr != 0);
@@ -203,17 +211,30 @@ static ino_t fuse_squash_ino(u64 ino64)
void fuse_change_attributes_common(struct inode *inode, struct fuse_attr *attr,
struct fuse_statx *sx,
- u64 attr_valid, u32 cache_mask)
+ u64 attr_valid, u32 cache_mask,
+ u64 evict_ctr)
{
struct fuse_conn *fc = get_fuse_conn(inode);
struct fuse_inode *fi = get_fuse_inode(inode);
lockdep_assert_held(&fi->lock);
+ /*
+ * Clear basic stats from invalid mask.
+ *
+ * Don't do this if this is coming from a fuse_iget() call and there
+ * might have been a racing evict which would've invalidated the result
+ * if the attr_version would've been preserved.
+ *
+ * !evict_ctr -> this is create
+ * fi->attr_version != 0 -> this is not a new inode
+ * evict_ctr == fuse_get_evict_ctr() -> no evicts while during request
+ */
+ if (!evict_ctr || fi->attr_version || evict_ctr == fuse_get_evict_ctr(fc))
+ set_mask_bits(&fi->inval_mask, STATX_BASIC_STATS, 0);
+
fi->attr_version = atomic64_inc_return(&fc->attr_version);
fi->i_time = attr_valid;
- /* Clear basic stats from invalid mask */
- set_mask_bits(&fi->inval_mask, STATX_BASIC_STATS, 0);
inode->i_ino = fuse_squash_ino(attr->ino);
inode->i_mode = (inode->i_mode & S_IFMT) | (attr->mode & 07777);
@@ -292,9 +313,9 @@ u32 fuse_get_cache_mask(struct inode *inode)
return STATX_MTIME | STATX_CTIME | STATX_SIZE;
}
-void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
- struct fuse_statx *sx,
- u64 attr_valid, u64 attr_version)
+static void fuse_change_attributes_i(struct inode *inode, struct fuse_attr *attr,
+ struct fuse_statx *sx, u64 attr_valid,
+ u64 attr_version, u64 evict_ctr)
{
struct fuse_conn *fc = get_fuse_conn(inode);
struct fuse_inode *fi = get_fuse_inode(inode);
@@ -328,7 +349,8 @@ void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
}
old_mtime = inode_get_mtime(inode);
- fuse_change_attributes_common(inode, attr, sx, attr_valid, cache_mask);
+ fuse_change_attributes_common(inode, attr, sx, attr_valid, cache_mask,
+ evict_ctr);
oldsize = inode->i_size;
/*
@@ -369,6 +391,13 @@ void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
fuse_dax_dontcache(inode, attr->flags);
}
+void fuse_change_attributes(struct inode *inode, struct fuse_attr *attr,
+ struct fuse_statx *sx, u64 attr_valid,
+ u64 attr_version)
+{
+ fuse_change_attributes_i(inode, attr, sx, attr_valid, attr_version, 0);
+}
+
static void fuse_init_submount_lookup(struct fuse_submount_lookup *sl,
u64 nodeid)
{
@@ -423,7 +452,8 @@ static int fuse_inode_set(struct inode *inode, void *_nodeidp)
struct inode *fuse_iget(struct super_block *sb, u64 nodeid,
int generation, struct fuse_attr *attr,
- u64 attr_valid, u64 attr_version)
+ u64 attr_valid, u64 attr_version,
+ u64 evict_ctr)
{
struct inode *inode;
struct fuse_inode *fi;
@@ -484,8 +514,8 @@ struct inode *fuse_iget(struct super_block *sb, u64 nodeid,
fi->nlookup++;
spin_unlock(&fi->lock);
done:
- fuse_change_attributes(inode, attr, NULL, attr_valid, attr_version);
-
+ fuse_change_attributes_i(inode, attr, NULL, attr_valid, attr_version,
+ evict_ctr);
return inode;
}
@@ -937,6 +967,7 @@ void fuse_conn_init(struct fuse_conn *fc, struct fuse_mount *fm,
fc->initialized = 0;
fc->connected = 1;
atomic64_set(&fc->attr_version, 1);
+ atomic64_set(&fc->evict_ctr, 1);
get_random_bytes(&fc->scramble_key, sizeof(fc->scramble_key));
fc->pid_ns = get_pid_ns(task_active_pid_ns(current));
fc->user_ns = get_user_ns(user_ns);
@@ -998,7 +1029,7 @@ static struct inode *fuse_get_root_inode(struct super_block *sb, unsigned mode)
attr.mode = mode;
attr.ino = FUSE_ROOT_ID;
attr.nlink = 1;
- return fuse_iget(sb, FUSE_ROOT_ID, 0, &attr, 0, 0);
+ return fuse_iget(sb, FUSE_ROOT_ID, 0, &attr, 0, 0, 0);
}
struct fuse_inode_handle {
@@ -1607,7 +1638,10 @@ static int fuse_fill_super_submount(struct super_block *sb,
return -ENOMEM;
fuse_fill_attr_from_inode(&root_attr, parent_fi);
- root = fuse_iget(sb, parent_fi->nodeid, 0, &root_attr, 0, 0);
+ root = fuse_iget(sb, parent_fi->nodeid, 0, &root_attr, 0, 0,
+ fuse_get_evict_ctr(fm->fc));
+ if (!root)
+ return -ENOMEM;
/*
* This inode is just a duplicate, so it is not looked up and
* its nlookup should not be incremented. fuse_iget() does
diff --git a/fs/fuse/readdir.c b/fs/fuse/readdir.c
index 78e6ad3f8617..b9e46ad95b3e 100644
--- a/fs/fuse/readdir.c
+++ b/fs/fuse/readdir.c
@@ -153,7 +153,7 @@ static int parse_dirfile(char *buf, size_t nbytes, struct file *file,
static int fuse_direntplus_link(struct file *file,
struct fuse_direntplus *direntplus,
- u64 attr_version)
+ u64 attr_version, u64 evict_ctr)
{
struct fuse_entry_out *o = &direntplus->entry_out;
struct fuse_dirent *dirent = &direntplus->dirent;
@@ -237,7 +237,7 @@ static int fuse_direntplus_link(struct file *file,
} else {
inode = fuse_iget(dir->i_sb, o->nodeid, o->generation,
&o->attr, ATTR_TIMEOUT(o),
- attr_version);
+ attr_version, evict_ctr);
if (!inode)
inode = ERR_PTR(-ENOMEM);
@@ -288,7 +288,8 @@ static void fuse_force_forget(struct file *file, u64 nodeid)
}
static int parse_dirplusfile(char *buf, size_t nbytes, struct file *file,
- struct dir_context *ctx, u64 attr_version)
+ struct dir_context *ctx, u64 attr_version,
+ u64 evict_ctr)
{
struct fuse_direntplus *direntplus;
struct fuse_dirent *dirent;
@@ -323,7 +324,7 @@ static int parse_dirplusfile(char *buf, size_t nbytes, struct file *file,
buf += reclen;
nbytes -= reclen;
- ret = fuse_direntplus_link(file, direntplus, attr_version);
+ ret = fuse_direntplus_link(file, direntplus, attr_version, evict_ctr);
if (ret)
fuse_force_forget(file, direntplus->entry_out.nodeid);
}
@@ -335,26 +336,28 @@ static int fuse_readdir_uncached(struct file *file, struct dir_context *ctx)
{
int plus;
ssize_t res;
- struct page *page;
+ struct folio *folio;
struct inode *inode = file_inode(file);
struct fuse_mount *fm = get_fuse_mount(inode);
struct fuse_io_args ia = {};
struct fuse_args_pages *ap = &ia.ap;
- struct fuse_page_desc desc = { .length = PAGE_SIZE };
- u64 attr_version = 0;
+ struct fuse_folio_desc desc = { .length = PAGE_SIZE };
+ u64 attr_version = 0, evict_ctr = 0;
bool locked;
- page = alloc_page(GFP_KERNEL);
- if (!page)
+ folio = folio_alloc(GFP_KERNEL, 0);
+ if (!folio)
return -ENOMEM;
plus = fuse_use_readdirplus(inode, ctx);
ap->args.out_pages = true;
- ap->num_pages = 1;
- ap->pages = &page;
- ap->descs = &desc;
+ ap->uses_folios = true;
+ ap->num_folios = 1;
+ ap->folios = &folio;
+ ap->folio_descs = &desc;
if (plus) {
attr_version = fuse_get_attr_version(fm->fc);
+ evict_ctr = fuse_get_evict_ctr(fm->fc);
fuse_read_args_fill(&ia, file, ctx->pos, PAGE_SIZE,
FUSE_READDIRPLUS);
} else {
@@ -371,15 +374,16 @@ static int fuse_readdir_uncached(struct file *file, struct dir_context *ctx)
if (ff->open_flags & FOPEN_CACHE_DIR)
fuse_readdir_cache_end(file, ctx->pos);
} else if (plus) {
- res = parse_dirplusfile(page_address(page), res,
- file, ctx, attr_version);
+ res = parse_dirplusfile(folio_address(folio), res,
+ file, ctx, attr_version,
+ evict_ctr);
} else {
- res = parse_dirfile(page_address(page), res, file,
+ res = parse_dirfile(folio_address(folio), res, file,
ctx);
}
}
- __free_page(page);
+ folio_put(folio);
fuse_invalidate_atime(inode);
return res;
}
diff --git a/fs/hfsplus/catalog.c b/fs/hfsplus/catalog.c
index dc3f98ec5e2d..7a9c72e4e4f2 100644
--- a/fs/hfsplus/catalog.c
+++ b/fs/hfsplus/catalog.c
@@ -204,7 +204,7 @@ int hfsplus_find_cat(struct super_block *sb, u32 cnid,
return err;
type = be16_to_cpu(tmp.type);
- if (type != HFSPLUS_FOLDER_THREAD && type != HFSPLUS_FILE_THREAD) {
+ if (!is_hfs_thread_record_type(type)) {
pr_err("found bad thread record in catalog\n");
return -EIO;
}
@@ -350,23 +350,22 @@ int hfsplus_delete_cat(u32 cnid, struct inode *dir, const struct qstr *str)
goto out;
if (!str) {
- int len;
+ hfsplus_cat_entry entry = {0};
hfsplus_cat_build_key_with_cnid(sb, fd.search_key, cnid);
- err = hfs_brec_find(&fd, hfs_find_rec_by_key);
+ err = hfsplus_brec_read_cat(&fd, &entry);
if (err)
goto out;
- off = fd.entryoffset +
- offsetof(struct hfsplus_cat_thread, nodeName);
- fd.search_key->cat.parent = cpu_to_be32(dir->i_ino);
- hfs_bnode_read(fd.bnode,
- &fd.search_key->cat.name.length, off, 2);
- len = be16_to_cpu(fd.search_key->cat.name.length) * 2;
- hfs_bnode_read(fd.bnode,
- &fd.search_key->cat.name.unicode,
- off + 2, len);
- fd.search_key->key_len = cpu_to_be16(6 + len);
+ type = be16_to_cpu(entry.type);
+ if (!is_hfs_thread_record_type(type)) {
+ pr_err("found bad thread record in catalog\n");
+ err = -EIO;
+ goto out;
+ }
+
+ hfsplus_cat_build_key_uni(fd.search_key, dir->i_ino,
+ &entry.thread.nodeName);
} else {
err = hfsplus_cat_build_key(sb, fd.search_key, dir->i_ino, str);
if (unlikely(err))
diff --git a/fs/hfsplus/hfsplus_fs.h b/fs/hfsplus/hfsplus_fs.h
index 11cef30803d5..f4b9c7e2491c 100644
--- a/fs/hfsplus/hfsplus_fs.h
+++ b/fs/hfsplus/hfsplus_fs.h
@@ -543,6 +543,12 @@ static inline u32 hfsplus_cat_thread_size(const struct hfsplus_cat_thread *threa
be16_to_cpu(thread->nodeName.length) * sizeof(hfsplus_unichr);
}
+static inline
+bool is_hfs_thread_record_type(u16 type)
+{
+ return type == HFSPLUS_FOLDER_THREAD || type == HFSPLUS_FILE_THREAD;
+}
+
int hfsplus_brec_read_cat(struct hfs_find_data *fd, hfsplus_cat_entry *entry);
/*
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index b0c3b4399a79..734eb08bafc9 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -1506,7 +1506,8 @@ hugetlbfs_fill_super(struct super_block *sb, struct fs_context *fc)
goto out_free;
return 0;
out_free:
- kfree(sbinfo->spool);
+ if (sbinfo->spool)
+ hugepage_put_subpool(sbinfo->spool);
kfree(sbinfo);
return -ENOMEM;
}
diff --git a/fs/isofs/compress.c b/fs/isofs/compress.c
index 34d5baa5d88a..7d701eb7af52 100644
--- a/fs/isofs/compress.c
+++ b/fs/isofs/compress.c
@@ -65,12 +65,14 @@ static loff_t zisofs_uncompress_block(struct inode *inode, loff_t block_start,
/* Empty block? */
if (block_size == 0) {
for ( i = 0 ; i < pcount ; i++ ) {
+ unsigned int off = i ? 0 : poffset;
+
if (!pages[i])
continue;
- memzero_page(pages[i], 0, PAGE_SIZE);
+ memzero_page(pages[i], off, PAGE_SIZE - off);
SetPageUptodate(pages[i]);
}
- return ((loff_t)pcount) << PAGE_SHIFT;
+ return (((loff_t)pcount) << PAGE_SHIFT) - poffset;
}
/* Because zlib is not thread-safe, do all the I/O at the top. */
@@ -291,6 +293,7 @@ static int zisofs_fill_pages(struct inode *inode, int full_page, int pcount,
memzero_page(*pages, poffset, PAGE_SIZE - poffset);
SetPageUptodate(*pages);
}
+ brelse(bh);
return 0;
}
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index d99284e35126..fd41e38aedb7 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -565,6 +565,8 @@ int netfs_read_folio(struct file *file, struct folio *folio)
struct netfs_inode *ctx = netfs_inode(mapping->host);
int ret;
+ folio_wait_writeback(folio);
+
if (folio_test_dirty(folio)) {
trace_netfs_folio(folio, netfs_folio_trace_read_gaps);
return netfs_read_gaps(file, folio);
diff --git a/fs/nfs/blocklayout/dev.c b/fs/nfs/blocklayout/dev.c
index 22d4529c6193..0d04952454b6 100644
--- a/fs/nfs/blocklayout/dev.c
+++ b/fs/nfs/blocklayout/dev.c
@@ -85,15 +85,17 @@ bl_free_device(struct pnfs_block_dev *dev)
{
bl_unregister_dev(dev);
- if (dev->nr_children) {
+ if (dev->children) {
int i;
for (i = 0; i < dev->nr_children; i++)
bl_free_device(&dev->children[i]);
kfree(dev->children);
- } else {
- if (dev->bdev_file)
- fput(dev->bdev_file);
+ dev->children = NULL;
+ dev->nr_children = 0;
+ } else if (dev->bdev_file) {
+ fput(dev->bdev_file);
+ dev->bdev_file = NULL;
}
}
@@ -434,6 +436,7 @@ bl_parse_scsi(struct nfs_server *server, struct pnfs_block_dev *d,
out_blkdev_put:
fput(d->bdev_file);
+ d->bdev_file = NULL;
return error;
}
@@ -469,8 +472,11 @@ bl_parse_concat(struct nfs_server *server, struct pnfs_block_dev *d,
for (i = 0; i < v->concat.volumes_count; i++) {
ret = bl_parse_deviceid(server, &d->children[i],
volumes, v->concat.volumes[i], gfp_mask);
- if (ret)
+ if (ret) {
+ bl_free_device(&d->children[i]);
+ bl_free_device(d);
return ret;
+ }
d->nr_children++;
d->children[i].start += len;
@@ -498,8 +504,11 @@ bl_parse_stripe(struct nfs_server *server, struct pnfs_block_dev *d,
for (i = 0; i < v->stripe.volumes_count; i++) {
ret = bl_parse_deviceid(server, &d->children[i],
volumes, v->stripe.volumes[i], gfp_mask);
- if (ret)
+ if (ret) {
+ bl_free_device(&d->children[i]);
+ bl_free_device(d);
return ret;
+ }
d->nr_children++;
len += d->children[i].len;
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 8bdde6cc873f..bb3f882d8c24 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -211,9 +211,21 @@ static void nfs_cb_idr_remove_locked(struct nfs_client *clp)
{
struct nfs_net *nn = net_generic(clp->cl_net, nfs_net_id);
- if (clp->cl_cb_ident)
+ if (clp->cl_cb_ident) {
idr_remove(&nn->cb_ident_idr, clp->cl_cb_ident);
+ clp->cl_cb_ident = 0;
+ }
+}
+
+void nfs_cb_idr_remove(struct nfs_client *clp)
+{
+ struct nfs_net *nn = net_generic(clp->cl_net, nfs_net_id);
+
+ spin_lock(&nn->nfs_client_lock);
+ nfs_cb_idr_remove_locked(clp);
+ spin_unlock(&nn->nfs_client_lock);
}
+EXPORT_SYMBOL_GPL(nfs_cb_idr_remove);
static void pnfs_init_server(struct nfs_server *server)
{
@@ -1248,6 +1260,10 @@ void nfs_clients_init(struct net *net)
INIT_LIST_HEAD(&nn->nfs_volume_list);
#if IS_ENABLED(CONFIG_NFS_V4)
idr_init(&nn->cb_ident_idr);
+#endif
+#if IS_ENABLED(CONFIG_NFS_V4_1)
+ INIT_LIST_HEAD(&nn->nfs4_data_server_cache);
+ spin_lock_init(&nn->nfs4_data_server_lock);
#endif
spin_lock_init(&nn->nfs_client_lock);
nn->boot_time = ktime_get_real();
@@ -1265,6 +1281,9 @@ void nfs_clients_exit(struct net *net)
nfs_cleanup_cb_ident_idr(net);
WARN_ON_ONCE(!list_empty(&nn->nfs_client_list));
WARN_ON_ONCE(!list_empty(&nn->nfs_volume_list));
+#if IS_ENABLED(CONFIG_NFS_V4_1)
+ WARN_ON_ONCE(!list_empty(&nn->nfs4_data_server_cache));
+#endif
}
#ifdef CONFIG_PROC_FS
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index d409239fb00f..5c9c15683867 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -2287,6 +2287,13 @@ int nfs_atomic_open_v23(struct inode *dir, struct dentry *dentry,
if (open_flags & O_CREAT) {
error = nfs_do_create(dir, dentry, mode, open_flags);
if (!error) {
+ /* With UNCHECKED mode, a server may return NFS3_OK for
+ * a pre-existing non-regular file (e.g. a symlink).
+ * Let the VFS handle it; calling finish_open() would
+ * hit no_open() and return -ENXIO.
+ */
+ if (!d_is_reg(dentry))
+ return finish_no_open(file, NULL);
file->f_mode |= FMODE_CREATED;
return finish_open(file, dentry, NULL);
} else if (error != -EEXIST || open_flags & O_EXCL)
diff --git a/fs/nfs/filelayout/filelayoutdev.c b/fs/nfs/filelayout/filelayoutdev.c
index 29d9234d5c08..b878ea466361 100644
--- a/fs/nfs/filelayout/filelayoutdev.c
+++ b/fs/nfs/filelayout/filelayoutdev.c
@@ -170,7 +170,8 @@ nfs4_fl_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev,
goto out_err_free_deviceid;
}
- dsaddr->ds_list[i] = nfs4_pnfs_ds_add(net, &dsaddrs, gfp_flags);
+ dsaddr->ds_list[i] = nfs4_pnfs_ds_add(net, &dsaddrs, 4,
+ gfp_flags);
if (!dsaddr->ds_list[i])
goto out_err_drain_dsaddrs;
trace_fl_getdevinfo(server, &pdev->dev_id, dsaddr->ds_list[i]->ds_remotestr);
diff --git a/fs/nfs/flexfilelayout/flexfilelayoutdev.c b/fs/nfs/flexfilelayout/flexfilelayoutdev.c
index 9c127565870e..53655581f9cb 100644
--- a/fs/nfs/flexfilelayout/flexfilelayoutdev.c
+++ b/fs/nfs/flexfilelayout/flexfilelayoutdev.c
@@ -149,7 +149,8 @@ nfs4_ff_alloc_deviceid_node(struct nfs_server *server, struct pnfs_device *pdev,
new_ds->ds_versions = ds_versions;
new_ds->ds_versions_cnt = version_count;
- new_ds->ds = nfs4_pnfs_ds_add(net, &dsaddrs, gfp_flags);
+ new_ds->ds = nfs4_pnfs_ds_add(net, &dsaddrs, ds_versions[0].version,
+ gfp_flags);
if (!new_ds->ds)
goto out_err_drain_dsaddrs;
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 1b43331eb6ec..a1e65cf3d8c9 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1504,9 +1504,7 @@ int nfs_clear_invalid_mapping(struct address_space *mapping)
ret = nfs_invalidate_mapping(inode, mapping);
trace_nfs_invalidate_mapping_exit(inode, ret);
- clear_bit_unlock(NFS_INO_INVALIDATING, bitlock);
- smp_mb__after_atomic();
- wake_up_bit(bitlock, NFS_INO_INVALIDATING);
+ clear_and_wake_up_bit(NFS_INO_INVALIDATING, bitlock);
out:
return ret;
}
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 4dccacf51750..7808b06f2332 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -226,6 +226,7 @@ void nfs_server_copy_userdata(struct nfs_server *, struct nfs_server *);
extern void nfs_put_client(struct nfs_client *);
extern void nfs_free_client(struct nfs_client *);
+void nfs_cb_idr_remove(struct nfs_client *clp);
extern struct nfs_client *nfs4_find_client_ident(struct net *, int);
extern struct nfs_client *
nfs4_find_client_sessionid(struct net *, const struct sockaddr *,
diff --git a/fs/nfs/netns.h b/fs/nfs/netns.h
index a68b21603ea9..6ba3ea39e928 100644
--- a/fs/nfs/netns.h
+++ b/fs/nfs/netns.h
@@ -31,7 +31,11 @@ struct nfs_net {
unsigned short nfs_callback_tcpport;
unsigned short nfs_callback_tcpport6;
int cb_users[NFS4_MAX_MINOR_VERSION + 1];
-#endif
+#endif /* CONFIG_NFS_V4 */
+#if IS_ENABLED(CONFIG_NFS_V4_1)
+ struct list_head nfs4_data_server_cache;
+ spinlock_t nfs4_data_server_lock;
+#endif /* CONFIG_NFS_V4_1 */
struct nfs_netns_client *nfs_client;
spinlock_t nfs_client_lock;
ktime_t boot_time;
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index 3f31d05e87ae..9311944a47f5 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -270,6 +270,7 @@ struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init)
return clp;
error:
+ nfs_cb_idr_remove(clp);
nfs_free_client(clp);
return ERR_PTR(err);
}
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index facd4e28770b..54e991d598f7 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -7812,6 +7812,7 @@ static int nfs4_add_lease(struct file *file, int arg, struct file_lease **lease,
{
struct inode *inode = file_inode(file);
fmode_t type = arg == F_RDLCK ? FMODE_READ : FMODE_WRITE;
+ fl_owner_t owner = (*lease)->c.flc_owner;
int ret;
/* No delegation, no lease */
@@ -7821,7 +7822,8 @@ static int nfs4_add_lease(struct file *file, int arg, struct file_lease **lease,
if (ret || nfs4_have_delegation(inode, type, 0))
return ret;
/* We raced with a delegation return */
- nfs4_delete_lease(file, priv);
+ dprintk("%s: raced with a delegation return\n", __func__);
+ nfs4_delete_lease(file, &owner);
return -EAGAIN;
}
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index a430b3da5c1b..41081f393d39 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -2088,15 +2088,6 @@ static bool pnfs_is_first_layoutget(struct pnfs_layout_hdr *lo)
return test_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags);
}
-static void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo)
-{
- unsigned long *bitlock = &lo->plh_flags;
-
- clear_bit_unlock(NFS_LAYOUT_FIRST_LAYOUTGET, bitlock);
- smp_mb__after_atomic();
- wake_up_bit(bitlock, NFS_LAYOUT_FIRST_LAYOUTGET);
-}
-
static void _add_to_server_list(struct pnfs_layout_hdr *lo,
struct nfs_server *server)
{
@@ -2272,7 +2263,8 @@ pnfs_update_layout(struct inode *ino,
iomode, lo, lseg,
PNFS_UPDATE_LAYOUT_INVALID_OPEN);
nfs4_schedule_stateid_recovery(server, ctx->state);
- pnfs_clear_first_layoutget(lo);
+ clear_and_wake_up_bit(NFS_LAYOUT_FIRST_LAYOUTGET,
+ &lo->plh_flags);
pnfs_put_layout_hdr(lo);
goto lookup_again;
}
@@ -2341,7 +2333,8 @@ pnfs_update_layout(struct inode *ino,
if (!exception.retry)
goto out_put_layout_hdr;
if (first)
- pnfs_clear_first_layoutget(lo);
+ clear_and_wake_up_bit(NFS_LAYOUT_FIRST_LAYOUTGET,
+ &lo->plh_flags);
trace_pnfs_update_layout(ino, pos, count,
iomode, lo, lseg, PNFS_UPDATE_LAYOUT_RETRY);
pnfs_put_layout_hdr(lo);
@@ -2353,7 +2346,7 @@ pnfs_update_layout(struct inode *ino,
out_put_layout_hdr:
if (first)
- pnfs_clear_first_layoutget(lo);
+ clear_and_wake_up_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags);
trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg,
PNFS_UPDATE_LAYOUT_EXIT);
pnfs_put_layout_hdr(lo);
@@ -2445,7 +2438,7 @@ static void _lgopen_prepare_attached(struct nfs4_opendata *data,
lgp = pnfs_alloc_init_layoutget_args(ino, ctx, ¤t_stateid, &rng,
nfs_io_gfp_mask());
if (!lgp) {
- pnfs_clear_first_layoutget(lo);
+ clear_and_wake_up_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags);
nfs_layoutget_end(lo);
pnfs_put_layout_hdr(lo);
return;
@@ -2549,7 +2542,8 @@ void nfs4_lgopen_release(struct nfs4_layoutget *lgp)
{
if (lgp != NULL) {
if (lgp->lo) {
- pnfs_clear_first_layoutget(lgp->lo);
+ clear_and_wake_up_bit(NFS_LAYOUT_FIRST_LAYOUTGET,
+ &lgp->lo->plh_flags);
nfs_layoutget_end(lgp->lo);
}
pnfs_layoutget_free(lgp);
@@ -3262,15 +3256,6 @@ pnfs_generic_pg_readpages(struct nfs_pageio_descriptor *desc)
}
EXPORT_SYMBOL_GPL(pnfs_generic_pg_readpages);
-static void pnfs_clear_layoutcommitting(struct inode *inode)
-{
- unsigned long *bitlock = &NFS_I(inode)->flags;
-
- clear_bit_unlock(NFS_INO_LAYOUTCOMMITTING, bitlock);
- smp_mb__after_atomic();
- wake_up_bit(bitlock, NFS_INO_LAYOUTCOMMITTING);
-}
-
/*
* There can be multiple RW segments.
*/
@@ -3295,7 +3280,7 @@ static void pnfs_list_write_lseg_done(struct inode *inode, struct list_head *lis
pnfs_put_lseg(lseg);
}
- pnfs_clear_layoutcommitting(inode);
+ clear_and_wake_up_bit(NFS_INO_LAYOUTCOMMITTING, &NFS_I(inode)->flags);
}
void pnfs_set_lo_fail(struct pnfs_layout_segment *lseg)
@@ -3435,7 +3420,7 @@ pnfs_layoutcommit_inode(struct inode *inode, bool sync)
spin_unlock(&inode->i_lock);
kfree(data);
clear_layoutcommitting:
- pnfs_clear_layoutcommitting(inode);
+ clear_and_wake_up_bit(NFS_INO_LAYOUTCOMMITTING, &NFS_I(inode)->flags);
goto out;
}
EXPORT_SYMBOL_GPL(pnfs_layoutcommit_inode);
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 3db8f13d8fe4..7b3ba2e64e6f 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -63,6 +63,7 @@ struct nfs4_pnfs_ds {
const struct net *ds_net;
struct nfs_client *ds_clp;
refcount_t ds_count;
+ u32 ds_version; /* cache key, with ds_addrs */
unsigned long ds_state;
#define NFS4DS_CONNECTING 0 /* ds is establishing connection */
};
@@ -417,7 +418,7 @@ void pnfs_generic_write_commit_done(struct rpc_task *task, void *data);
void nfs4_pnfs_ds_put(struct nfs4_pnfs_ds *ds);
struct nfs4_pnfs_ds *nfs4_pnfs_ds_add(const struct net *net,
struct list_head *dsaddrs,
- gfp_t gfp_flags);
+ u32 version, gfp_t gfp_flags);
void nfs4_pnfs_v3_ds_connect_unload(void);
int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds,
struct nfs4_deviceid_node *devid, unsigned int timeo,
diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c
index bf6abca0c29d..634ee58ced12 100644
--- a/fs/nfs/pnfs_nfs.c
+++ b/fs/nfs/pnfs_nfs.c
@@ -16,6 +16,7 @@
#include "nfs4session.h"
#include "internal.h"
#include "pnfs.h"
+#include "netns.h"
#define NFSDBG_FACILITY NFSDBG_PNFS
@@ -504,14 +505,14 @@ EXPORT_SYMBOL_GPL(pnfs_generic_commit_pagelist);
/*
* Data server cache
*
- * Data servers can be mapped to different device ids.
- * nfs4_pnfs_ds reference counting
+ * Data servers can be mapped to different device ids, but should
+ * never be shared between net namespaces.
+ *
+ * nfs4_pnfs_ds reference counting:
* - set to 1 on allocation
* - incremented when a device id maps a data server already in the cache.
* - decremented when deviceid is removed from the cache.
*/
-static DEFINE_SPINLOCK(nfs4_ds_cache_lock);
-static LIST_HEAD(nfs4_data_server_cache);
/* Debug routines */
static void
@@ -601,15 +602,17 @@ _same_data_server_addrs_locked(const struct list_head *dsaddrs1,
}
/*
- * Lookup DS by addresses. nfs4_ds_cache_lock is held
+ * Lookup DS by addresses and NFS version. nfs4_ds_cache_lock is held
*/
static struct nfs4_pnfs_ds *
-_data_server_lookup_locked(const struct net *net, const struct list_head *dsaddrs)
+_data_server_lookup_locked(const struct nfs_net *nn,
+ const struct list_head *dsaddrs, u32 version)
{
struct nfs4_pnfs_ds *ds;
- list_for_each_entry(ds, &nfs4_data_server_cache, ds_node)
- if (ds->ds_net == net && _same_data_server_addrs_locked(&ds->ds_addrs, dsaddrs))
+ list_for_each_entry(ds, &nn->nfs4_data_server_cache, ds_node)
+ if (ds->ds_version == version &&
+ _same_data_server_addrs_locked(&ds->ds_addrs, dsaddrs))
return ds;
return NULL;
}
@@ -653,10 +656,11 @@ static void destroy_ds(struct nfs4_pnfs_ds *ds)
void nfs4_pnfs_ds_put(struct nfs4_pnfs_ds *ds)
{
- if (refcount_dec_and_lock(&ds->ds_count,
- &nfs4_ds_cache_lock)) {
+ struct nfs_net *nn = net_generic(ds->ds_net, nfs_net_id);
+
+ if (refcount_dec_and_lock(&ds->ds_count, &nn->nfs4_data_server_lock)) {
list_del_init(&ds->ds_node);
- spin_unlock(&nfs4_ds_cache_lock);
+ spin_unlock(&nn->nfs4_data_server_lock);
destroy_ds(ds);
}
}
@@ -716,8 +720,10 @@ nfs4_pnfs_remotestr(struct list_head *dsaddrs, gfp_t gfp_flags)
* uncached and return cached struct nfs4_pnfs_ds.
*/
struct nfs4_pnfs_ds *
-nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_flags)
+nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, u32 version,
+ gfp_t gfp_flags)
{
+ struct nfs_net *nn = net_generic(net, nfs_net_id);
struct nfs4_pnfs_ds *tmp_ds, *ds = NULL;
char *remotestr;
@@ -733,8 +739,8 @@ nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_fla
/* this is only used for debugging, so it's ok if its NULL */
remotestr = nfs4_pnfs_remotestr(dsaddrs, gfp_flags);
- spin_lock(&nfs4_ds_cache_lock);
- tmp_ds = _data_server_lookup_locked(net, dsaddrs);
+ spin_lock(&nn->nfs4_data_server_lock);
+ tmp_ds = _data_server_lookup_locked(nn, dsaddrs, version);
if (tmp_ds == NULL) {
INIT_LIST_HEAD(&ds->ds_addrs);
list_splice_init(dsaddrs, &ds->ds_addrs);
@@ -743,7 +749,8 @@ nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_fla
INIT_LIST_HEAD(&ds->ds_node);
ds->ds_net = net;
ds->ds_clp = NULL;
- list_add(&ds->ds_node, &nfs4_data_server_cache);
+ ds->ds_version = version;
+ list_add(&ds->ds_node, &nn->nfs4_data_server_cache);
dprintk("%s add new data server %s\n", __func__,
ds->ds_remotestr);
} else {
@@ -755,7 +762,7 @@ nfs4_pnfs_ds_add(const struct net *net, struct list_head *dsaddrs, gfp_t gfp_fla
refcount_read(&tmp_ds->ds_count));
ds = tmp_ds;
}
- spin_unlock(&nfs4_ds_cache_lock);
+ spin_unlock(&nn->nfs4_data_server_lock);
out:
return ds;
}
diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c
index 06f18fe86407..bc698bb66f79 100644
--- a/fs/nilfs2/page.c
+++ b/fs/nilfs2/page.c
@@ -244,6 +244,7 @@ static void nilfs_copy_folio(struct folio *dst, struct folio *src,
int nilfs_copy_dirty_pages(struct address_space *dmap,
struct address_space *smap)
{
+ struct inode *smap_inode = smap->host;
struct folio_batch fbatch;
unsigned int i;
pgoff_t index = 0;
@@ -259,8 +260,19 @@ int nilfs_copy_dirty_pages(struct address_space *dmap,
struct folio *folio = fbatch.folios[i], *dfolio;
folio_lock(folio);
- if (unlikely(!folio_test_dirty(folio)))
- NILFS_FOLIO_BUG(folio, "inconsistent dirty state");
+ if (unlikely(!folio_test_dirty(folio))) {
+ if (WARN_ONCE(!sb_rdonly(smap_inode->i_sb),
+ "inconsistent dirty state\n"))
+ goto unlock_folio;
+
+ /*
+ * If the filesystem has been forced to read-only
+ * due to metadata corruption.
+ */
+ folio_unlock(folio);
+ err = -EROFS;
+ break;
+ }
dfolio = filemap_grab_folio(dmap, folio->index);
if (IS_ERR(dfolio)) {
@@ -278,6 +290,7 @@ int nilfs_copy_dirty_pages(struct address_space *dmap,
folio_unlock(dfolio);
folio_put(dfolio);
+unlock_folio:
folio_unlock(folio);
}
folio_batch_release(&fbatch);
diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
index 657a72f941b9..c598c2a4262a 100644
--- a/fs/nilfs2/segment.c
+++ b/fs/nilfs2/segment.c
@@ -2580,6 +2580,10 @@ int nilfs_clean_segments(struct super_block *sb, struct nilfs_argv *argv,
break;
nilfs_warn(sb, "error %d cleaning segments", err);
+
+ if (unlikely(err == -EROFS))
+ goto out_unlock;
+
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(sci->sc_interval);
}
diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c
index ecd71c190885..6069353bb945 100644
--- a/fs/nilfs2/the_nilfs.c
+++ b/fs/nilfs2/the_nilfs.c
@@ -448,6 +448,12 @@ static int nilfs_store_disk_layout(struct the_nilfs *nilfs,
nilfs->ns_inode_size);
return -EINVAL;
}
+ if (NILFS_SR_BYTES(nilfs->ns_inode_size) > nilfs->ns_blocksize) {
+ nilfs_err(nilfs->ns_sb,
+ "too large inode size for super root: %d bytes",
+ nilfs->ns_inode_size);
+ return -EINVAL;
+ }
nilfs->ns_first_ino = le32_to_cpu(sbp->s_first_ino);
if (nilfs->ns_first_ino < NILFS_USER_INO) {
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index b89ad128bf09..c62eb37cfb24 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -970,11 +970,13 @@ static long fanotify_ioctl(struct file *file, unsigned int cmd, unsigned long ar
{
struct fsnotify_group *group;
struct fsnotify_event *fsn_event;
+ unsigned int info_mode;
void __user *p;
int ret = -ENOTTY;
size_t send_len = 0;
group = file->private_data;
+ info_mode = FAN_GROUP_FLAG(group, FANOTIFY_INFO_MODES);
p = (void __user *) arg;
@@ -982,7 +984,8 @@ static long fanotify_ioctl(struct file *file, unsigned int cmd, unsigned long ar
case FIONREAD:
spin_lock(&group->notification_lock);
list_for_each_entry(fsn_event, &group->notification_list, list)
- send_len += FAN_EVENT_METADATA_LEN;
+ send_len += fanotify_event_len(info_mode,
+ FANOTIFY_E(fsn_event));
spin_unlock(&group->notification_lock);
ret = put_user(send_len, (int __user *) p);
break;
@@ -1141,16 +1144,18 @@ static bool fanotify_mark_update_flags(struct fsnotify_mark *fsn_mark,
static bool fanotify_mark_add_to_mask(struct fsnotify_mark *fsn_mark,
__u32 mask, unsigned int fan_flags)
{
+ __u32 old_mask;
bool recalc;
spin_lock(&fsn_mark->lock);
- if (!(fan_flags & FANOTIFY_MARK_IGNORE_BITS))
+ if (!(fan_flags & FANOTIFY_MARK_IGNORE_BITS)) {
+ old_mask = fsn_mark->mask;
fsn_mark->mask |= mask;
- else
+ recalc = old_mask != fsn_mark->mask;
+ } else {
fsn_mark->ignore_mask |= mask;
-
- recalc = fsnotify_calc_mask(fsn_mark) &
- ~fsnotify_conn_mask(fsn_mark->connector);
+ recalc = true;
+ }
recalc |= fanotify_mark_update_flags(fsn_mark, fan_flags);
spin_unlock(&fsn_mark->lock);
diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
index 0ea4e99dc449..024be6cee849 100644
--- a/fs/notify/inotify/inotify_user.c
+++ b/fs/notify/inotify/inotify_user.c
@@ -539,7 +539,6 @@ static int inotify_update_existing_watch(struct fsnotify_group *group,
{
struct fsnotify_mark *fsn_mark;
struct inotify_inode_mark *i_mark;
- __u32 old_mask, new_mask;
int replace = !(arg & IN_MASK_ADD);
int create = (arg & IN_MASK_CREATE);
int ret;
@@ -555,27 +554,15 @@ static int inotify_update_existing_watch(struct fsnotify_group *group,
i_mark = container_of(fsn_mark, struct inotify_inode_mark, fsn_mark);
spin_lock(&fsn_mark->lock);
- old_mask = fsn_mark->mask;
if (replace) {
fsn_mark->mask = 0;
fsn_mark->flags &= ~INOTIFY_MARK_FLAGS;
}
fsn_mark->mask |= inotify_arg_to_mask(inode, arg);
fsn_mark->flags |= inotify_arg_to_flags(arg);
- new_mask = fsn_mark->mask;
spin_unlock(&fsn_mark->lock);
- if (old_mask != new_mask) {
- /* more bits in old than in new? */
- int dropped = (old_mask & ~new_mask);
- /* more bits in this fsn_mark than the inode's mask? */
- int do_inode = (new_mask & ~READ_ONCE(inode->i_fsnotify_mask));
-
- /* update the inode with this new fsn_mark */
- if (dropped || do_inode)
- fsnotify_recalc_mask(inode->i_fsnotify_marks);
-
- }
+ fsnotify_recalc_mask(fsn_mark->connector);
/* return the wd */
ret = i_mark->wd;
diff --git a/fs/ntfs3/attrib.c b/fs/ntfs3/attrib.c
index 940a8293abe9..526f86a6fe51 100644
--- a/fs/ntfs3/attrib.c
+++ b/fs/ntfs3/attrib.c
@@ -789,7 +789,8 @@ int attr_set_size(struct ntfs_inode *ni, enum ATTR_TYPE type,
if (err)
goto out;
- attr = mi_find_attr(mi, NULL, type, name, name_len, &le->id);
+ attr = mi_find_attr(ni, mi, NULL, type, name, name_len,
+ &le->id);
if (!attr) {
err = -EINVAL;
goto bad_inode;
@@ -1197,7 +1198,7 @@ int attr_data_get_block(struct ntfs_inode *ni, CLST vcn, CLST clen, CLST *lcn,
goto out;
}
- attr = mi_find_attr(mi, NULL, ATTR_DATA, NULL, 0, &le->id);
+ attr = mi_find_attr(ni, mi, NULL, ATTR_DATA, NULL, 0, &le->id);
if (!attr) {
err = -EINVAL;
goto out;
@@ -1821,7 +1822,7 @@ int attr_allocate_frame(struct ntfs_inode *ni, CLST frame, size_t compr_size,
goto out;
}
- attr = mi_find_attr(mi, NULL, ATTR_DATA, NULL, 0,
+ attr = mi_find_attr(ni, mi, NULL, ATTR_DATA, NULL, 0,
&le->id);
if (!attr) {
err = -EINVAL;
@@ -2066,8 +2067,8 @@ int attr_collapse_range(struct ntfs_inode *ni, u64 vbo, u64 bytes)
}
/* Look for required attribute. */
- attr = mi_find_attr(mi, NULL, ATTR_DATA, NULL,
- 0, &le->id);
+ attr = mi_find_attr(ni, mi, NULL, ATTR_DATA,
+ NULL, 0, &le->id);
if (!attr) {
err = -EINVAL;
goto out;
diff --git a/fs/ntfs3/frecord.c b/fs/ntfs3/frecord.c
index 65daa4f31d37..45e8d73d38f8 100644
--- a/fs/ntfs3/frecord.c
+++ b/fs/ntfs3/frecord.c
@@ -75,7 +75,7 @@ struct ATTR_STD_INFO *ni_std(struct ntfs_inode *ni)
{
const struct ATTRIB *attr;
- attr = mi_find_attr(&ni->mi, NULL, ATTR_STD, NULL, 0, NULL);
+ attr = mi_find_attr(ni, &ni->mi, NULL, ATTR_STD, NULL, 0, NULL);
return attr ? resident_data_ex(attr, sizeof(struct ATTR_STD_INFO)) :
NULL;
}
@@ -89,7 +89,7 @@ struct ATTR_STD_INFO5 *ni_std5(struct ntfs_inode *ni)
{
const struct ATTRIB *attr;
- attr = mi_find_attr(&ni->mi, NULL, ATTR_STD, NULL, 0, NULL);
+ attr = mi_find_attr(ni, &ni->mi, NULL, ATTR_STD, NULL, 0, NULL);
return attr ? resident_data_ex(attr, sizeof(struct ATTR_STD_INFO5)) :
NULL;
@@ -203,7 +203,8 @@ struct ATTRIB *ni_find_attr(struct ntfs_inode *ni, struct ATTRIB *attr,
*mi = &ni->mi;
/* Look for required attribute in primary record. */
- return mi_find_attr(&ni->mi, attr, type, name, name_len, NULL);
+ return mi_find_attr(ni, &ni->mi, attr, type, name, name_len,
+ NULL);
}
/* First look for list entry of required type. */
@@ -219,7 +220,7 @@ struct ATTRIB *ni_find_attr(struct ntfs_inode *ni, struct ATTRIB *attr,
return NULL;
/* Look for required attribute. */
- attr = mi_find_attr(m, NULL, type, name, name_len, &le->id);
+ attr = mi_find_attr(ni, m, NULL, type, name, name_len, &le->id);
if (!attr)
goto out;
@@ -260,7 +261,7 @@ struct ATTRIB *ni_enum_attr_ex(struct ntfs_inode *ni, struct ATTRIB *attr,
if (mi)
*mi = &ni->mi;
/* Enum attributes in primary record. */
- return mi_enum_attr(&ni->mi, attr);
+ return mi_enum_attr(ni, &ni->mi, attr);
}
/* Get next list entry. */
@@ -276,7 +277,7 @@ struct ATTRIB *ni_enum_attr_ex(struct ntfs_inode *ni, struct ATTRIB *attr,
*mi = mi2;
/* Find attribute in loaded record. */
- return rec_find_attr_le(mi2, le2);
+ return rec_find_attr_le(ni, mi2, le2);
}
/*
@@ -294,7 +295,8 @@ struct ATTRIB *ni_load_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
if (!ni->attr_list.size) {
if (pmi)
*pmi = &ni->mi;
- return mi_find_attr(&ni->mi, NULL, type, name, name_len, NULL);
+ return mi_find_attr(ni, &ni->mi, NULL, type, name, name_len,
+ NULL);
}
le = al_find_ex(ni, NULL, type, name, name_len, NULL);
@@ -320,7 +322,7 @@ struct ATTRIB *ni_load_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
if (pmi)
*pmi = mi;
- attr = mi_find_attr(mi, NULL, type, name, name_len, &le->id);
+ attr = mi_find_attr(ni, mi, NULL, type, name, name_len, &le->id);
if (!attr)
return NULL;
@@ -402,7 +404,8 @@ int ni_remove_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
int diff;
if (base_only || type == ATTR_LIST || !ni->attr_list.size) {
- attr = mi_find_attr(&ni->mi, NULL, type, name, name_len, id);
+ attr = mi_find_attr(ni, &ni->mi, NULL, type, name, name_len,
+ id);
if (!attr)
return -ENOENT;
@@ -441,7 +444,7 @@ int ni_remove_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
al_remove_le(ni, le);
- attr = mi_find_attr(mi, NULL, type, name, name_len, id);
+ attr = mi_find_attr(ni, mi, NULL, type, name, name_len, id);
if (!attr)
return -ENOENT;
@@ -489,7 +492,7 @@ ni_ins_new_attr(struct ntfs_inode *ni, struct mft_inode *mi,
name = le->name;
}
- attr = mi_insert_attr(mi, type, name, name_len, asize, name_off);
+ attr = mi_insert_attr(ni, mi, type, name, name_len, asize, name_off);
if (!attr) {
if (le_added)
al_remove_le(ni, le);
@@ -677,7 +680,7 @@ static int ni_try_remove_attr_list(struct ntfs_inode *ni)
if (err)
return err;
- attr_list = mi_find_attr(&ni->mi, NULL, ATTR_LIST, NULL, 0, NULL);
+ attr_list = mi_find_attr(ni, &ni->mi, NULL, ATTR_LIST, NULL, 0, NULL);
if (!attr_list)
return 0;
@@ -699,7 +702,7 @@ static int ni_try_remove_attr_list(struct ntfs_inode *ni)
if (!mi)
return 0;
- attr = mi_find_attr(mi, NULL, le->type, le_name(le),
+ attr = mi_find_attr(ni, mi, NULL, le->type, le_name(le),
le->name_len, &le->id);
if (!attr)
return 0;
@@ -735,7 +738,7 @@ static int ni_try_remove_attr_list(struct ntfs_inode *ni)
goto out;
}
- attr = mi_find_attr(mi, NULL, le->type, le_name(le),
+ attr = mi_find_attr(ni, mi, NULL, le->type, le_name(le),
le->name_len, &le->id);
if (!attr) {
/* Should never happened, 'cause already checked. */
@@ -744,7 +747,7 @@ static int ni_try_remove_attr_list(struct ntfs_inode *ni)
asize = le32_to_cpu(attr->size);
/* Insert into primary record. */
- attr_ins = mi_insert_attr(&ni->mi, le->type, le_name(le),
+ attr_ins = mi_insert_attr(ni, &ni->mi, le->type, le_name(le),
le->name_len, asize,
le16_to_cpu(attr->name_off));
if (!attr_ins) {
@@ -772,7 +775,7 @@ static int ni_try_remove_attr_list(struct ntfs_inode *ni)
if (!mi)
continue;
- attr = mi_find_attr(mi, NULL, le->type, le_name(le),
+ attr = mi_find_attr(ni, mi, NULL, le->type, le_name(le),
le->name_len, &le->id);
if (!attr)
continue;
@@ -820,10 +823,23 @@ int ni_create_attr_list(struct ntfs_inode *ni)
rs = sbi->record_size;
/*
- * Skip estimating exact memory requirement.
- * Looks like one record_size is always enough.
+ * Compute the exact size of the attribute list. Each attribute in the
+ * record yields one ATTR_LIST_ENTRY of le_size(name_len) bytes. The
+ * minimum on-disk attribute is SIZEOF_RESIDENT (0x18) bytes, but an
+ * unnamed one expands to le_size(0) (0x20) here, so a record crafted
+ * with many such attributes needs more than a single record_size; the
+ * previous fixed kzalloc(record_size) could therefore be overflowed by
+ * an attacker-controlled record.
*/
- le = kmalloc(al_aligned(rs), GFP_NOFS);
+ lsize = 0;
+ attr = NULL;
+ while ((attr = mi_enum_attr(ni, &ni->mi, attr)))
+ lsize += le_size(attr->name_len);
+
+ if (!lsize)
+ return -EINVAL;
+
+ le = kzalloc(al_aligned(lsize), GFP_NOFS);
if (!le)
return -ENOMEM;
@@ -833,9 +849,8 @@ int ni_create_attr_list(struct ntfs_inode *ni)
attr = NULL;
nb = 0;
free_b = 0;
- attr = NULL;
- for (; (attr = mi_enum_attr(&ni->mi, attr)); le = Add2Ptr(le, sz)) {
+ for (; (attr = mi_enum_attr(ni, &ni->mi, attr)); le = Add2Ptr(le, sz)) {
sz = le_size(attr->name_len);
le->type = attr->type;
le->size = cpu_to_le16(sz);
@@ -890,7 +905,7 @@ int ni_create_attr_list(struct ntfs_inode *ni)
u32 asize = le32_to_cpu(b->size);
u16 name_off = le16_to_cpu(b->name_off);
- attr = mi_insert_attr(mi, b->type, Add2Ptr(b, name_off),
+ attr = mi_insert_attr(ni, mi, b->type, Add2Ptr(b, name_off),
b->name_len, asize, name_off);
if (!attr)
goto out;
@@ -913,7 +928,7 @@ int ni_create_attr_list(struct ntfs_inode *ni)
goto out;
}
- attr = mi_insert_attr(&ni->mi, ATTR_LIST, NULL, 0,
+ attr = mi_insert_attr(ni, &ni->mi, ATTR_LIST, NULL, 0,
lsize + SIZEOF_RESIDENT, SIZEOF_RESIDENT);
if (!attr)
goto out;
@@ -997,13 +1012,13 @@ static int ni_ins_attr_ext(struct ntfs_inode *ni, struct ATTR_LIST_ENTRY *le,
mi = rb_entry(node, struct mft_inode, node);
if (is_mft_data &&
- (mi_enum_attr(mi, NULL) ||
+ (mi_enum_attr(ni, mi, NULL) ||
vbo <= ((u64)mi->rno << sbi->record_bits))) {
/* We can't accept this record 'cause MFT's bootstrapping. */
continue;
}
if (is_mft &&
- mi_find_attr(mi, NULL, ATTR_DATA, NULL, 0, NULL)) {
+ mi_find_attr(ni, mi, NULL, ATTR_DATA, NULL, 0, NULL)) {
/*
* This child record already has a ATTR_DATA.
* So it can't accept any other records.
@@ -1012,7 +1027,7 @@ static int ni_ins_attr_ext(struct ntfs_inode *ni, struct ATTR_LIST_ENTRY *le,
}
if ((type != ATTR_NAME || name_len) &&
- mi_find_attr(mi, NULL, type, name, name_len, NULL)) {
+ mi_find_attr(ni, mi, NULL, type, name, name_len, NULL)) {
/* Only indexed attributes can share same record. */
continue;
}
@@ -1161,7 +1176,7 @@ static int ni_insert_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
/* Estimate the result of moving all possible attributes away. */
attr = NULL;
- while ((attr = mi_enum_attr(&ni->mi, attr))) {
+ while ((attr = mi_enum_attr(ni, &ni->mi, attr))) {
if (attr->type == ATTR_STD)
continue;
if (attr->type == ATTR_LIST)
@@ -1179,7 +1194,7 @@ static int ni_insert_attr(struct ntfs_inode *ni, enum ATTR_TYPE type,
attr = NULL;
for (;;) {
- attr = mi_enum_attr(&ni->mi, attr);
+ attr = mi_enum_attr(ni, &ni->mi, attr);
if (!attr) {
/* We should never be here 'cause we have already check this case. */
err = -EINVAL;
@@ -1263,7 +1278,7 @@ static int ni_expand_mft_list(struct ntfs_inode *ni)
for (node = rb_first(&ni->mi_tree); node; node = rb_next(node)) {
mi = rb_entry(node, struct mft_inode, node);
- attr = mi_enum_attr(mi, NULL);
+ attr = mi_enum_attr(ni, mi, NULL);
if (!attr) {
mft_min = mi->rno;
@@ -1284,7 +1299,7 @@ static int ni_expand_mft_list(struct ntfs_inode *ni)
ni_remove_mi(ni, mi_new);
}
- attr = mi_find_attr(&ni->mi, NULL, ATTR_DATA, NULL, 0, NULL);
+ attr = mi_find_attr(ni, &ni->mi, NULL, ATTR_DATA, NULL, 0, NULL);
if (!attr) {
err = -EINVAL;
goto out;
@@ -1401,7 +1416,7 @@ int ni_expand_list(struct ntfs_inode *ni)
continue;
/* Find attribute in primary record. */
- attr = rec_find_attr_le(&ni->mi, le);
+ attr = rec_find_attr_le(ni, &ni->mi, le);
if (!attr) {
err = -EINVAL;
goto out;
@@ -3430,7 +3445,7 @@ int ni_write_inode(struct inode *inode, int sync, const char *hint)
if (!mi->dirty)
continue;
- is_empty = !mi_enum_attr(mi, NULL);
+ is_empty = !mi_enum_attr(ni, mi, NULL);
if (is_empty)
clear_rec_inuse(mi->mrec);
diff --git a/fs/ntfs3/fslog.c b/fs/ntfs3/fslog.c
index 49f4ce2e676f..f7fa1471d6b9 100644
--- a/fs/ntfs3/fslog.c
+++ b/fs/ntfs3/fslog.c
@@ -875,6 +875,9 @@ static inline struct RESTART_TABLE *extend_rsttbl(struct RESTART_TABLE *tbl,
u32 used = le16_to_cpu(tbl->used);
struct RESTART_TABLE *rt;
+ if (used + add > U16_MAX)
+ return NULL;
+
rt = init_rsttbl(esize, used + add);
if (!rt)
return NULL;
@@ -2296,7 +2299,15 @@ static int read_log_rec_buf(struct ntfs_log *log,
*/
for (;;) {
bool usa_error;
- u32 tail = log->page_size - off;
+ u32 tail;
+
+ /* off comes from the on-disk restart area; bound it. */
+ if (off > log->page_size) {
+ err = -EINVAL;
+ goto out;
+ }
+
+ tail = log->page_size - off;
if (tail >= data_len)
tail = data_len;
diff --git a/fs/ntfs3/fsntfs.c b/fs/ntfs3/fsntfs.c
index 5972f160e566..170836749de6 100644
--- a/fs/ntfs3/fsntfs.c
+++ b/fs/ntfs3/fsntfs.c
@@ -2317,8 +2317,8 @@ int ntfs_reparse_init(struct ntfs_sb_info *sbi)
goto out;
}
- root_r = resident_data(attr);
- if (root_r->type != ATTR_ZERO ||
+ root_r = resident_data_ex(attr, sizeof(struct INDEX_ROOT));
+ if (!root_r || root_r->type != ATTR_ZERO ||
root_r->rule != NTFS_COLLATION_TYPE_UINTS) {
err = -EINVAL;
goto out;
@@ -2355,8 +2355,8 @@ int ntfs_objid_init(struct ntfs_sb_info *sbi)
goto out;
}
- root = resident_data(attr);
- if (root->type != ATTR_ZERO ||
+ root = resident_data_ex(attr, sizeof(struct INDEX_ROOT));
+ if (!root || root->type != ATTR_ZERO ||
root->rule != NTFS_COLLATION_TYPE_UINTS) {
err = -EINVAL;
goto out;
diff --git a/fs/ntfs3/ntfs_fs.h b/fs/ntfs3/ntfs_fs.h
index 919fb3f36af0..11be9a6f041c 100644
--- a/fs/ntfs3/ntfs_fs.h
+++ b/fs/ntfs3/ntfs_fs.h
@@ -746,23 +746,24 @@ int mi_get(struct ntfs_sb_info *sbi, CLST rno, struct mft_inode **mi);
void mi_put(struct mft_inode *mi);
int mi_init(struct mft_inode *mi, struct ntfs_sb_info *sbi, CLST rno);
int mi_read(struct mft_inode *mi, bool is_mft);
-struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr);
-// TODO: id?
-struct ATTRIB *mi_find_attr(struct mft_inode *mi, struct ATTRIB *attr,
- enum ATTR_TYPE type, const __le16 *name,
- u8 name_len, const __le16 *id);
-static inline struct ATTRIB *rec_find_attr_le(struct mft_inode *rec,
+struct ATTRIB *mi_enum_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ struct ATTRIB *attr);
+struct ATTRIB *mi_find_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ struct ATTRIB *attr, enum ATTR_TYPE type,
+ const __le16 *name, u8 name_len, const __le16 *id);
+static inline struct ATTRIB *rec_find_attr_le(struct ntfs_inode *ni,
+ struct mft_inode *rec,
struct ATTR_LIST_ENTRY *le)
{
- return mi_find_attr(rec, NULL, le->type, le_name(le), le->name_len,
+ return mi_find_attr(ni, rec, NULL, le->type, le_name(le), le->name_len,
&le->id);
}
int mi_write(struct mft_inode *mi, int wait);
int mi_format_new(struct mft_inode *mi, struct ntfs_sb_info *sbi, CLST rno,
__le16 flags, bool is_mft);
-struct ATTRIB *mi_insert_attr(struct mft_inode *mi, enum ATTR_TYPE type,
- const __le16 *name, u8 name_len, u32 asize,
- u16 name_off);
+struct ATTRIB *mi_insert_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ enum ATTR_TYPE type, const __le16 *name,
+ u8 name_len, u32 asize, u16 name_off);
bool mi_remove_attr(struct ntfs_inode *ni, struct mft_inode *mi,
struct ATTRIB *attr);
diff --git a/fs/ntfs3/record.c b/fs/ntfs3/record.c
index f810f0419d25..0f8ba76094fe 100644
--- a/fs/ntfs3/record.c
+++ b/fs/ntfs3/record.c
@@ -31,7 +31,7 @@ static inline int compare_attr(const struct ATTRIB *left, enum ATTR_TYPE type,
*
* Return: Unused attribute id that is less than mrec->next_attr_id.
*/
-static __le16 mi_new_attt_id(struct mft_inode *mi)
+static __le16 mi_new_attt_id(struct ntfs_inode *ni, struct mft_inode *mi)
{
u16 free_id, max_id, t16;
struct MFT_REC *rec = mi->mrec;
@@ -52,7 +52,7 @@ static __le16 mi_new_attt_id(struct mft_inode *mi)
attr = NULL;
for (;;) {
- attr = mi_enum_attr(mi, attr);
+ attr = mi_enum_attr(ni, mi, attr);
if (!attr) {
rec->next_attr_id = cpu_to_le16(max_id + 1);
mi->dirty = true;
@@ -195,13 +195,14 @@ int mi_read(struct mft_inode *mi, bool is_mft)
* NOTE: mi->mrec - memory of size sbi->record_size
* here we sure that mi->mrec->total == sbi->record_size (see mi_read)
*/
-struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr)
+struct ATTRIB *mi_enum_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ struct ATTRIB *attr)
{
const struct MFT_REC *rec = mi->mrec;
u32 used = le32_to_cpu(rec->used);
u32 t32, off, asize, prev_type;
u16 t16;
- u64 data_size, alloc_size, tot_size;
+ u64 svcn, evcn, data_size, alloc_size, tot_size;
if (!attr) {
u32 total = le32_to_cpu(rec->total);
@@ -209,11 +210,11 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr)
off = le16_to_cpu(rec->attr_off);
if (used > total)
- return NULL;
+ goto out;
if (off >= used || off < MFTRECORD_FIXUP_OFFSET_1 ||
- !IS_ALIGNED(off, 4)) {
- return NULL;
+ !IS_ALIGNED(off, 8)) {
+ goto out;
}
/* Skip non-resident records. */
@@ -236,11 +237,14 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr)
off += asize;
}
- /* Can we use the first field (attr->type). */
- /* NOTE: this code also checks attr->size availability. */
+ /*
+ * Can we use the first fields:
+ * attr->type,
+ * attr->size
+ */
if (off + 8 > used) {
static_assert(ALIGN(sizeof(enum ATTR_TYPE), 8) == 8);
- return NULL;
+ goto out;
}
if (attr->type == ATTR_END) {
@@ -251,105 +255,144 @@ struct ATTRIB *mi_enum_attr(struct mft_inode *mi, struct ATTRIB *attr)
/* 0x100 is last known attribute for now. */
t32 = le32_to_cpu(attr->type);
if (!t32 || (t32 & 0xf) || (t32 > 0x100))
- return NULL;
+ goto out;
/* attributes in record must be ordered by type */
if (t32 < prev_type)
- return NULL;
+ goto out;
asize = le32_to_cpu(attr->size);
+ if (!IS_ALIGNED(asize, 8))
+ goto out;
+
/* Check overflow and boundary. */
if (off + asize < off || off + asize > used)
- return NULL;
+ goto out;
+
+ /* Can we use the field attr->non_res. */
+ if (off + 9 > used)
+ goto out;
/* Check size of attribute. */
if (!attr->non_res) {
/* Check resident fields. */
if (asize < SIZEOF_RESIDENT)
- return NULL;
+ goto out;
t16 = le16_to_cpu(attr->res.data_off);
if (t16 > asize)
- return NULL;
+ goto out;
if (le32_to_cpu(attr->res.data_size) > asize - t16)
- return NULL;
+ goto out;
t32 = sizeof(short) * attr->name_len;
if (t32 && le16_to_cpu(attr->name_off) + t32 > t16)
- return NULL;
+ goto out;
return attr;
}
/* Check nonresident fields. */
if (attr->non_res != 1)
- return NULL;
+ goto out;
/* Can we use memory including attr->nres.valid_size? */
if (asize < SIZEOF_NONRESIDENT)
- return NULL;
+ goto out;
t16 = le16_to_cpu(attr->nres.run_off);
if (t16 > asize)
- return NULL;
+ goto out;
t32 = sizeof(short) * attr->name_len;
if (t32 && le16_to_cpu(attr->name_off) + t32 > t16)
- return NULL;
+ goto out;
- /* Check start/end vcn. */
- if (le64_to_cpu(attr->nres.svcn) > le64_to_cpu(attr->nres.evcn) + 1)
- return NULL;
+ /*
+ * Check start/end vcn. svcn == 0 with evcn == -1 (U64_MAX) is the
+ * sentinel for an empty non-resident attribute (no allocated
+ * clusters) and must be accepted: "svcn > evcn + 1" tolerates it,
+ * since "(u64)-1 + 1" is 0 and "0 > 0" is false.
+ *
+ * For a non-empty attribute evcn is a cluster index and must lie
+ * within the volume (sbi->used.bitmap.nbits, set up in
+ * ntfs_init_from_boot() before any caller of mi_enum_attr() runs).
+ * Bounding evcn also prevents a malformed value close to U64_MAX
+ * from slipping through the near-wrap "evcn + 1" upper bound.
+ */
+ svcn = le64_to_cpu(attr->nres.svcn);
+ evcn = le64_to_cpu(attr->nres.evcn);
+ if (svcn > evcn + 1)
+ goto out;
+
+ if (is_attr_ext(attr)) {
+ /* sparsed/compressed attribute. */
+#ifdef CONFIG_NTFS3_64BIT_CLUSTER
+ /* No limits. */
+#else
+ /* Check evcn fits into 32 bits. */
+ if (evcn != U64_MAX && evcn >= (1ull << 32))
+ goto out;
+#endif
+ } else {
+ /* Check out of volume for normal attribute. */
+ if (evcn != U64_MAX && evcn >= mi->sbi->used.bitmap.nbits)
+ goto out;
+ }
data_size = le64_to_cpu(attr->nres.data_size);
if (le64_to_cpu(attr->nres.valid_size) > data_size)
- return NULL;
+ goto out;
alloc_size = le64_to_cpu(attr->nres.alloc_size);
if (data_size > alloc_size)
- return NULL;
+ goto out;
t32 = mi->sbi->cluster_mask;
if (alloc_size & t32)
- return NULL;
+ goto out;
if (!attr->nres.svcn && is_attr_ext(attr)) {
/* First segment of sparse/compressed attribute */
/* Can we use memory including attr->nres.total_size? */
if (asize < SIZEOF_NONRESIDENT_EX)
- return NULL;
+ goto out;
tot_size = le64_to_cpu(attr->nres.total_size);
if (tot_size & t32)
- return NULL;
+ goto out;
if (tot_size > alloc_size)
- return NULL;
+ goto out;
} else {
if (attr->nres.c_unit)
- return NULL;
+ goto out;
if (alloc_size > mi->sbi->volume.size)
- return NULL;
+ goto out;
}
return attr;
+
+out:
+ _ntfs_bad_inode(&ni->vfs_inode);
+ return NULL;
}
/*
* mi_find_attr - Find the attribute by type and name and id.
*/
-struct ATTRIB *mi_find_attr(struct mft_inode *mi, struct ATTRIB *attr,
- enum ATTR_TYPE type, const __le16 *name,
- u8 name_len, const __le16 *id)
+struct ATTRIB *mi_find_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ struct ATTRIB *attr, enum ATTR_TYPE type,
+ const __le16 *name, u8 name_len, const __le16 *id)
{
u32 type_in = le32_to_cpu(type);
u32 atype;
next_attr:
- attr = mi_enum_attr(mi, attr);
+ attr = mi_enum_attr(ni, mi, attr);
if (!attr)
return NULL;
@@ -457,9 +500,9 @@ int mi_format_new(struct mft_inode *mi, struct ntfs_sb_info *sbi, CLST rno,
*
* Return: Not full constructed attribute or NULL if not possible to create.
*/
-struct ATTRIB *mi_insert_attr(struct mft_inode *mi, enum ATTR_TYPE type,
- const __le16 *name, u8 name_len, u32 asize,
- u16 name_off)
+struct ATTRIB *mi_insert_attr(struct ntfs_inode *ni, struct mft_inode *mi,
+ enum ATTR_TYPE type, const __le16 *name,
+ u8 name_len, u32 asize, u16 name_off)
{
size_t tail;
struct ATTRIB *attr;
@@ -478,7 +521,7 @@ struct ATTRIB *mi_insert_attr(struct mft_inode *mi, enum ATTR_TYPE type,
* at which we should insert it.
*/
attr = NULL;
- while ((attr = mi_enum_attr(mi, attr))) {
+ while ((attr = mi_enum_attr(ni, mi, attr))) {
int diff = compare_attr(attr, type, name, name_len, upcase);
if (diff < 0)
@@ -498,7 +541,7 @@ struct ATTRIB *mi_insert_attr(struct mft_inode *mi, enum ATTR_TYPE type,
tail = used - PtrOffset(rec, attr);
}
- id = mi_new_attt_id(mi);
+ id = mi_new_attt_id(ni, mi);
memmove(Add2Ptr(attr, asize), attr, tail);
memset(attr, 0, asize);
diff --git a/fs/ntfs3/super.c b/fs/ntfs3/super.c
index 1af1500ec24b..b0c46840f75f 100644
--- a/fs/ntfs3/super.c
+++ b/fs/ntfs3/super.c
@@ -57,6 +57,7 @@
#include <linux/minmax.h>
#include <linux/module.h>
#include <linux/nls.h>
+#include <linux/overflow.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/statfs.h>
@@ -877,7 +878,7 @@ static int ntfs_init_from_boot(struct super_block *sb, u32 sector_size,
struct ntfs_sb_info *sbi = sb->s_fs_info;
int err;
u32 mb, gb, boot_sector_size, sct_per_clst, record_size;
- u64 sectors, clusters, mlcn, mlcn2, dev_size0;
+ u64 sectors, clusters, mlcn, mlcn2, mft_pos, mft2_pos, dev_size0;
struct NTFS_BOOT *boot;
struct buffer_head *bh;
struct MFT_REC *rec;
@@ -946,7 +947,15 @@ static int ntfs_init_from_boot(struct super_block *sb, u32 sector_size,
mlcn2 = le64_to_cpu(boot->mft2_clst);
sectors = le64_to_cpu(boot->sectors_per_volume);
- if (mlcn * sct_per_clst >= sectors || mlcn2 * sct_per_clst >= sectors) {
+ /*
+ * Convert mlcn/mlcn2 to sector positions before comparing with
+ * 'sectors'. All three are u64 values that come from the boot
+ * sector, so use check_mul_overflow() to keep a wraparound from
+ * silently bypassing the comparison.
+ */
+ if (check_mul_overflow(mlcn, (u64)sct_per_clst, &mft_pos) ||
+ check_mul_overflow(mlcn2, (u64)sct_per_clst, &mft2_pos) ||
+ mft_pos >= sectors || mft2_pos >= sectors) {
ntfs_err(
sb,
"%s: start of MFT 0x%llx (0x%llx) is out of volume 0x%llx.",
diff --git a/fs/ntfs3/xattr.c b/fs/ntfs3/xattr.c
index 287534823346..7cf9039198df 100644
--- a/fs/ntfs3/xattr.c
+++ b/fs/ntfs3/xattr.c
@@ -146,26 +146,29 @@ static int ntfs_read_ea(struct ntfs_inode *ni, struct EA_FULL **ea,
for (off = 0; off < size; off += ea_size) {
const struct EA_FULL *ef = Add2Ptr(ea_p, off);
u32 bytes = size - off;
+ size_t need;
/* Check if we can use field ea->size. */
if (bytes < sizeof(ef->size))
goto out1;
+ /* Check if we can use fields ef->name_len and ef->elength. */
+ if (bytes < offsetof(struct EA_FULL, name))
+ goto out1;
+
+ /* Size needed to hold this record's name and value. */
+ need = struct_size(ef, name,
+ 1 + ef->name_len + le16_to_cpu(ef->elength));
+
if (ef->size) {
ea_size = le32_to_cpu(ef->size);
- if (ea_size > bytes)
+ /* ef->size must fit the list and cover the record. */
+ if (ea_size > bytes || ea_size < need)
goto out1;
continue;
}
- /* Check if we can use fields ef->name_len and ef->elength. */
- if (bytes < offsetof(struct EA_FULL, name))
- goto out1;
-
- ea_size = ALIGN(struct_size(ef, name,
- 1 + ef->name_len +
- le16_to_cpu(ef->elength)),
- 4);
+ ea_size = ALIGN(need, 4);
if (ea_size > bytes)
goto out1;
}
diff --git a/fs/ocfs2/acl.c b/fs/ocfs2/acl.c
index 62464d194da3..095ce96c2484 100644
--- a/fs/ocfs2/acl.c
+++ b/fs/ocfs2/acl.c
@@ -109,8 +109,7 @@ static void *ocfs2_acl_to_xattr(const struct posix_acl *acl, size_t *size)
return ocfs2_acl;
}
-static struct posix_acl *ocfs2_get_acl_nolock(struct inode *inode,
- int type,
+static struct posix_acl *ocfs2_get_acl_nolock(struct inode *inode, int type,
struct buffer_head *di_bh)
{
int name_index;
@@ -348,63 +347,105 @@ int ocfs2_acl_chmod(struct inode *inode, struct buffer_head *bh)
* Initialize the ACLs of a new inode. If parent directory has default ACL,
* then clone to new inode. Called from ocfs2_mknod.
*/
-int ocfs2_init_acl(handle_t *handle,
- struct inode *inode,
- struct inode *dir,
- struct buffer_head *di_bh,
- struct buffer_head *dir_bh,
- struct ocfs2_alloc_context *meta_ac,
- struct ocfs2_alloc_context *data_ac)
+void ocfs2_acl_init_release(struct ocfs2_acl_state *state)
+{
+ posix_acl_release(state->default_acl);
+ posix_acl_release(state->acl);
+ state->default_acl = NULL;
+ state->acl = NULL;
+}
+
+int ocfs2_acl_init_prepare(struct inode *inode, struct inode *dir,
+ struct buffer_head *dir_bh,
+ struct ocfs2_acl_state *state)
{
struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
- struct posix_acl *acl = NULL;
- int ret = 0, ret2;
- umode_t mode;
-
- if (!S_ISLNK(inode->i_mode)) {
- if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
- down_read(&OCFS2_I(dir)->ip_xattr_sem);
- acl = ocfs2_get_acl_nolock(dir, ACL_TYPE_DEFAULT,
- dir_bh);
- up_read(&OCFS2_I(dir)->ip_xattr_sem);
- if (IS_ERR(acl))
- return PTR_ERR(acl);
+ int ret = 0;
+
+ state->default_acl = NULL;
+ state->acl = NULL;
+ state->mode = inode->i_mode;
+
+ if (S_ISLNK(inode->i_mode))
+ return 0;
+
+ if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
+ down_read(&OCFS2_I(dir)->ip_xattr_sem);
+ state->default_acl =
+ ocfs2_get_acl_nolock(dir, ACL_TYPE_DEFAULT, dir_bh);
+ up_read(&OCFS2_I(dir)->ip_xattr_sem);
+ if (IS_ERR(state->default_acl)) {
+ ret = PTR_ERR(state->default_acl);
+ state->default_acl = NULL;
+ return ret;
}
- if (!acl) {
- mode = inode->i_mode & ~current_umask();
- ret = ocfs2_acl_set_mode(inode, di_bh, handle, mode);
- if (ret) {
- mlog_errno(ret);
+ if (state->default_acl) {
+ state->acl = posix_acl_dup(state->default_acl);
+ if (!state->acl) {
+ ret = -ENOMEM;
goto cleanup;
}
+ ret = __posix_acl_create(&state->acl, GFP_NOFS,
+ &state->mode);
+ if (ret < 0)
+ goto cleanup;
+ if (ret == 0) {
+ posix_acl_release(state->acl);
+ state->acl = NULL;
+ }
+ if (!S_ISDIR(inode->i_mode)) {
+ posix_acl_release(state->default_acl);
+ state->default_acl = NULL;
+ }
+ } else {
+ state->mode &= ~current_umask();
}
+ } else {
+ state->mode &= ~current_umask();
}
- if ((osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) && acl) {
- if (S_ISDIR(inode->i_mode)) {
+
+ return 0;
+cleanup:
+ ocfs2_acl_init_release(state);
+ return ret;
+}
+
+int ocfs2_init_acl(handle_t *handle, struct inode *inode,
+ struct buffer_head *di_bh,
+ struct ocfs2_alloc_context *meta_ac,
+ struct ocfs2_alloc_context *data_ac,
+ struct ocfs2_acl_state *state)
+{
+ struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
+ int ret = 0;
+
+ if (S_ISLNK(inode->i_mode))
+ return 0;
+
+ if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
+ if (S_ISDIR(inode->i_mode) && state->default_acl) {
ret = ocfs2_set_acl(handle, inode, di_bh,
- ACL_TYPE_DEFAULT, acl,
- meta_ac, data_ac);
+ ACL_TYPE_DEFAULT,
+ state->default_acl, meta_ac,
+ data_ac);
if (ret)
- goto cleanup;
+ return ret;
}
- mode = inode->i_mode;
- ret = __posix_acl_create(&acl, GFP_NOFS, &mode);
- if (ret < 0)
- return ret;
+ }
- ret2 = ocfs2_acl_set_mode(inode, di_bh, handle, mode);
- if (ret2) {
- mlog_errno(ret2);
- ret = ret2;
- goto cleanup;
- }
- if (ret > 0) {
- ret = ocfs2_set_acl(handle, inode,
- di_bh, ACL_TYPE_ACCESS,
- acl, meta_ac, data_ac);
+ ret = ocfs2_acl_set_mode(inode, di_bh, handle, state->mode);
+ if (ret) {
+ mlog_errno(ret);
+ return ret;
+ }
+
+ if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
+ if (state->acl) {
+ ret = ocfs2_set_acl(handle, inode, di_bh,
+ ACL_TYPE_ACCESS, state->acl,
+ meta_ac, data_ac);
}
}
-cleanup:
- posix_acl_release(acl);
+
return ret;
}
diff --git a/fs/ocfs2/acl.h b/fs/ocfs2/acl.h
index 667c6f03fa60..a91f9ce278d6 100644
--- a/fs/ocfs2/acl.h
+++ b/fs/ocfs2/acl.h
@@ -20,9 +20,20 @@ struct posix_acl *ocfs2_iop_get_acl(struct inode *inode, int type, bool rcu);
int ocfs2_iop_set_acl(struct mnt_idmap *idmap, struct dentry *dentry,
struct posix_acl *acl, int type);
extern int ocfs2_acl_chmod(struct inode *, struct buffer_head *);
-extern int ocfs2_init_acl(handle_t *, struct inode *, struct inode *,
- struct buffer_head *, struct buffer_head *,
- struct ocfs2_alloc_context *,
- struct ocfs2_alloc_context *);
+struct ocfs2_acl_state {
+ struct posix_acl *default_acl;
+ struct posix_acl *acl;
+ umode_t mode;
+};
+
+int ocfs2_acl_init_prepare(struct inode *inode, struct inode *dir,
+ struct buffer_head *dir_bh,
+ struct ocfs2_acl_state *state);
+void ocfs2_acl_init_release(struct ocfs2_acl_state *state);
+int ocfs2_init_acl(handle_t *handle, struct inode *inode,
+ struct buffer_head *di_bh,
+ struct ocfs2_alloc_context *meta_ac,
+ struct ocfs2_alloc_context *data_ac,
+ struct ocfs2_acl_state *state);
#endif /* OCFS2_ACL_H */
diff --git a/fs/ocfs2/cluster/heartbeat.c b/fs/ocfs2/cluster/heartbeat.c
index 6914d615ec2d..6170e202e56e 100644
--- a/fs/ocfs2/cluster/heartbeat.c
+++ b/fs/ocfs2/cluster/heartbeat.c
@@ -210,6 +210,7 @@ struct o2hb_region {
/* protected by the hr_callback_sem */
struct task_struct *hr_task;
+ u8 hr_node_num;
unsigned int hr_blocks;
unsigned long long hr_start_block;
@@ -357,12 +358,12 @@ static void o2hb_disarm_timeout(struct o2hb_region *reg)
cancel_delayed_work_sync(®->hr_nego_timeout_work);
}
-static int o2hb_send_nego_msg(int key, int type, u8 target)
+static int o2hb_send_nego_msg(int key, int type, u8 target, u8 node_num)
{
struct o2hb_nego_msg msg;
int status, ret;
- msg.node_num = o2nm_this_node();
+ msg.node_num = node_num;
again:
ret = o2net_send_message(type, key, &msg, sizeof(msg),
target, &status);
@@ -380,8 +381,10 @@ static void o2hb_nego_timeout(struct work_struct *work)
unsigned long live_node_bitmap[BITS_TO_LONGS(O2NM_MAX_NODES)];
int master_node, i, ret;
struct o2hb_region *reg;
+ u8 node_num;
reg = container_of(work, struct o2hb_region, hr_nego_timeout_work.work);
+ node_num = reg->hr_node_num;
/* don't negotiate timeout if last hb failed since it is very
* possible io failed. Should let write timeout fence self.
*/
@@ -392,10 +395,10 @@ static void o2hb_nego_timeout(struct work_struct *work)
/* lowest node as master node to make negotiate decision. */
master_node = find_first_bit(live_node_bitmap, O2NM_MAX_NODES);
- if (master_node == o2nm_this_node()) {
+ if (master_node == node_num) {
if (!test_bit(master_node, reg->hr_nego_node_bitmap)) {
printk(KERN_NOTICE "o2hb: node %d hb write hung for %ds on region %s (%pg).\n",
- o2nm_this_node(), O2HB_NEGO_TIMEOUT_MS/1000,
+ node_num, O2HB_NEGO_TIMEOUT_MS / 1000,
config_item_name(®->hr_item), reg_bdev(reg));
set_bit(master_node, reg->hr_nego_node_bitmap);
}
@@ -424,7 +427,7 @@ static void o2hb_nego_timeout(struct work_struct *work)
mlog(ML_HEARTBEAT, "send NEGO_APPROVE msg to node %d\n", i);
ret = o2hb_send_nego_msg(reg->hr_key,
- O2HB_NEGO_APPROVE_MSG, i);
+ O2HB_NEGO_APPROVE_MSG, i, node_num);
if (ret)
mlog(ML_ERROR, "send NEGO_APPROVE msg to node %d fail %d\n",
i, ret);
@@ -432,10 +435,10 @@ static void o2hb_nego_timeout(struct work_struct *work)
} else {
/* negotiate timeout with master node. */
printk(KERN_NOTICE "o2hb: node %d hb write hung for %ds on region %s (%pg), negotiate timeout with node %d.\n",
- o2nm_this_node(), O2HB_NEGO_TIMEOUT_MS/1000, config_item_name(®->hr_item),
+ node_num, O2HB_NEGO_TIMEOUT_MS / 1000, config_item_name(®->hr_item),
reg_bdev(reg), master_node);
ret = o2hb_send_nego_msg(reg->hr_key, O2HB_NEGO_TIMEOUT_MSG,
- master_node);
+ master_node, node_num);
if (ret)
mlog(ML_ERROR, "send NEGO_TIMEOUT msg to node %d fail %d\n",
master_node, ret);
@@ -608,7 +611,9 @@ static int o2hb_issue_node_write(struct o2hb_region *reg,
o2hb_bio_wait_init(write_wc);
- slot = o2nm_this_node();
+ slot = reg->hr_node_num;
+ if (slot >= O2NM_MAX_NODES)
+ return -EINVAL;
bio = o2hb_setup_one_bio(reg, write_wc, &slot, slot+1,
REQ_OP_WRITE | REQ_SYNC);
@@ -677,8 +682,12 @@ static int o2hb_check_own_slot(struct o2hb_region *reg)
struct o2hb_disk_slot *slot;
struct o2hb_disk_heartbeat_block *hb_block;
char *errstr;
+ u8 node_num = reg->hr_node_num;
+
+ if (node_num >= O2NM_MAX_NODES)
+ return 0;
- slot = ®->hr_slots[o2nm_this_node()];
+ slot = ®->hr_slots[node_num];
/* Don't check on our 1st timestamp */
if (!slot->ds_last_time)
return 0;
@@ -719,7 +728,10 @@ static inline void o2hb_prepare_block(struct o2hb_region *reg,
struct o2hb_disk_slot *slot;
struct o2hb_disk_heartbeat_block *hb_block;
- node_num = o2nm_this_node();
+ node_num = reg->hr_node_num;
+ if (node_num >= O2NM_MAX_NODES)
+ return;
+
slot = ®->hr_slots[node_num];
hb_block = (struct o2hb_disk_heartbeat_block *)slot->ds_raw_block;
@@ -1213,7 +1225,7 @@ static int o2hb_thread(void *data)
set_user_nice(current, MIN_NICE);
/* Pin node */
- ret = o2nm_depend_this_node();
+ ret = o2nm_depend_node(reg->hr_node_num);
if (ret) {
mlog(ML_ERROR, "Node has been deleted, ret = %d\n", ret);
reg->hr_node_deleted = 1;
@@ -1222,7 +1234,8 @@ static int o2hb_thread(void *data)
}
while (!kthread_should_stop() &&
- !reg->hr_unclean_stop && !reg->hr_aborted_start) {
+ !reg->hr_unclean_stop && !reg->hr_aborted_start &&
+ o2nm_this_node() == reg->hr_node_num) {
/* We track the time spent inside
* o2hb_do_disk_heartbeat so that we avoid more than
* hr_timeout_ms between disk writes. On busy systems
@@ -1271,7 +1284,7 @@ static int o2hb_thread(void *data)
}
/* Unpin node */
- o2nm_undepend_this_node();
+ o2nm_undepend_node(reg->hr_node_num);
mlog(ML_HEARTBEAT|ML_KTHREAD, "o2hb thread exiting\n");
@@ -1463,13 +1476,38 @@ void o2hb_init(void)
o2hb_debug_init();
}
-/* if we're already in a callback then we're already serialized by the sem */
-static void o2hb_fill_node_map_from_callback(unsigned long *map,
- unsigned int bits)
+static void __o2hb_fill_node_map(unsigned long *map, unsigned int bits)
{
bitmap_copy(map, o2hb_live_node_bitmap, bits);
}
+void o2hb_callback_read_lock(void)
+{
+ down_read(&o2hb_callback_sem);
+}
+
+void o2hb_callback_read_unlock(void)
+{
+ up_read(&o2hb_callback_sem);
+}
+
+void o2hb_synchronize_callbacks(void)
+{
+ down_write(&o2hb_callback_sem);
+ up_write(&o2hb_callback_sem);
+}
+
+/*
+ * Callers must already hold o2hb_callback_sem for read or write so the copy
+ * stays serialized with callback delivery.
+ */
+void o2hb_fill_node_map_locked(unsigned long *map, unsigned int bits)
+{
+ spin_lock(&o2hb_live_lock);
+ __o2hb_fill_node_map(map, bits);
+ spin_unlock(&o2hb_live_lock);
+}
+
/*
* get a map of all nodes that are heartbeating in any regions
*/
@@ -1477,11 +1515,9 @@ void o2hb_fill_node_map(unsigned long *map, unsigned int bits)
{
/* callers want to serialize this map and callbacks so that they
* can trust that they don't miss nodes coming to the party */
- down_read(&o2hb_callback_sem);
- spin_lock(&o2hb_live_lock);
- o2hb_fill_node_map_from_callback(map, bits);
- spin_unlock(&o2hb_live_lock);
- up_read(&o2hb_callback_sem);
+ o2hb_callback_read_lock();
+ o2hb_fill_node_map_locked(map, bits);
+ o2hb_callback_read_unlock();
}
EXPORT_SYMBOL_GPL(o2hb_fill_node_map);
@@ -1778,23 +1814,24 @@ static ssize_t o2hb_region_dev_store(struct config_item *item,
int live_threshold;
if (reg->hr_bdev_file)
- goto out;
+ return -EINVAL;
/* We can't heartbeat without having had our node number
* configured yet. */
- if (o2nm_this_node() == O2NM_MAX_NODES)
- goto out;
+ reg->hr_node_num = o2nm_this_node();
+ if (reg->hr_node_num == O2NM_MAX_NODES)
+ return -EINVAL;
fd = simple_strtol(p, &p, 0);
if (!p || (*p && (*p != '\n')))
- goto out;
+ return -EINVAL;
if (fd < 0 || fd >= INT_MAX)
- goto out;
+ return -EINVAL;
f = fdget(fd);
if (fd_file(f) == NULL)
- goto out;
+ return -EINVAL;
if (reg->hr_blocks == 0 || reg->hr_start_block == 0 ||
reg->hr_block_bytes == 0)
@@ -1916,7 +1953,6 @@ static ssize_t o2hb_region_dev_store(struct config_item *item,
}
out2:
fdput(f);
-out:
return ret;
}
@@ -2018,6 +2054,7 @@ static struct config_item *o2hb_heartbeat_group_make_item(struct config_group *g
ret = -ENAMETOOLONG;
goto free;
}
+ reg->hr_node_num = O2NM_MAX_NODES;
spin_lock(&o2hb_live_lock);
reg->hr_region_num = 0;
@@ -2552,7 +2589,7 @@ int o2hb_check_node_heartbeating_no_sem(u8 node_num)
unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];
spin_lock(&o2hb_live_lock);
- o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES);
+ __o2hb_fill_node_map(testing_map, O2NM_MAX_NODES);
spin_unlock(&o2hb_live_lock);
if (!test_bit(node_num, testing_map)) {
mlog(ML_HEARTBEAT,
@@ -2569,7 +2606,7 @@ int o2hb_check_node_heartbeating_from_callback(u8 node_num)
{
unsigned long testing_map[BITS_TO_LONGS(O2NM_MAX_NODES)];
- o2hb_fill_node_map_from_callback(testing_map, O2NM_MAX_NODES);
+ o2hb_fill_node_map_locked(testing_map, O2NM_MAX_NODES);
if (!test_bit(node_num, testing_map)) {
mlog(ML_HEARTBEAT,
"node (%u) does not have heartbeating enabled.\n",
diff --git a/fs/ocfs2/cluster/heartbeat.h b/fs/ocfs2/cluster/heartbeat.h
index 8ef8c1b9eeb7..2ca2b657583c 100644
--- a/fs/ocfs2/cluster/heartbeat.h
+++ b/fs/ocfs2/cluster/heartbeat.h
@@ -58,6 +58,11 @@ int o2hb_register_callback(const char *region_uuid,
struct o2hb_callback_func *hc);
void o2hb_unregister_callback(const char *region_uuid,
struct o2hb_callback_func *hc);
+void o2hb_callback_read_lock(void);
+void o2hb_callback_read_unlock(void);
+void o2hb_synchronize_callbacks(void);
+void o2hb_fill_node_map_locked(unsigned long *map,
+ unsigned int bits);
void o2hb_fill_node_map(unsigned long *map,
unsigned int bits);
void o2hb_exit(void);
diff --git a/fs/ocfs2/cluster/nodemanager.c b/fs/ocfs2/cluster/nodemanager.c
index 5fffbed779da..b65fc6eeae6e 100644
--- a/fs/ocfs2/cluster/nodemanager.c
+++ b/fs/ocfs2/cluster/nodemanager.c
@@ -325,6 +325,7 @@ static ssize_t o2nm_node_local_store(struct config_item *item, const char *page,
struct o2nm_node *node = to_o2nm_node(item);
struct o2nm_cluster *cluster;
unsigned long tmp;
+ bool starting = false;
char *p = (char *)page;
ssize_t ret;
@@ -361,11 +362,13 @@ static ssize_t o2nm_node_local_store(struct config_item *item, const char *page,
ret = o2net_start_listening(node);
if (ret)
goto out;
+ starting = true;
}
if (!tmp && cluster->cl_has_local &&
cluster->cl_local_node == node->nd_num) {
o2net_stop_listening(node);
+ cluster->cl_has_local = 0;
cluster->cl_local_node = O2NM_INVALID_NODE_NUM;
}
@@ -373,6 +376,8 @@ static ssize_t o2nm_node_local_store(struct config_item *item, const char *page,
if (node->nd_local) {
cluster->cl_has_local = tmp;
cluster->cl_local_node = node->nd_num;
+ if (starting)
+ o2net_complete_start_listening(node);
}
ret = count;
@@ -787,12 +792,12 @@ void o2nm_undepend_item(struct config_item *item)
configfs_undepend_item(item);
}
-int o2nm_depend_this_node(void)
+int o2nm_depend_node(u8 node_num)
{
int ret = 0;
struct o2nm_node *local_node;
- local_node = o2nm_get_node_by_num(o2nm_this_node());
+ local_node = o2nm_get_node_by_num(node_num);
if (!local_node) {
ret = -EINVAL;
goto out;
@@ -805,17 +810,27 @@ int o2nm_depend_this_node(void)
return ret;
}
-void o2nm_undepend_this_node(void)
+void o2nm_undepend_node(u8 node_num)
{
struct o2nm_node *local_node;
- local_node = o2nm_get_node_by_num(o2nm_this_node());
+ local_node = o2nm_get_node_by_num(node_num);
BUG_ON(!local_node);
o2nm_undepend_item(&local_node->nd_item);
o2nm_node_put(local_node);
}
+int o2nm_depend_this_node(void)
+{
+ return o2nm_depend_node(o2nm_this_node());
+}
+
+void o2nm_undepend_this_node(void)
+{
+ o2nm_undepend_node(o2nm_this_node());
+}
+
static void __exit exit_o2nm(void)
{
diff --git a/fs/ocfs2/cluster/nodemanager.h b/fs/ocfs2/cluster/nodemanager.h
index 2f72f56996bd..ca3483fb5450 100644
--- a/fs/ocfs2/cluster/nodemanager.h
+++ b/fs/ocfs2/cluster/nodemanager.h
@@ -66,6 +66,8 @@ void o2nm_node_put(struct o2nm_node *node);
int o2nm_depend_item(struct config_item *item);
int o2nm_depend_item_unlocked(struct config_item *item);
void o2nm_undepend_item(struct config_item *item);
+int o2nm_depend_node(u8 node_num);
+void o2nm_undepend_node(u8 node_num);
int o2nm_depend_this_node(void);
void o2nm_undepend_this_node(void);
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index 2b8fa3e782fb..15b726a3a1fc 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -105,6 +105,8 @@ static struct socket *o2net_listen_sock;
* destroying the work queue.
*/
static struct workqueue_struct *o2net_wq;
+/* Heartbeat callbacks stay registered across local-node off/on. */
+static bool o2net_listening;
static struct work_struct o2net_listen_work;
static struct o2hb_callback_func o2net_hb_up, o2net_hb_down;
@@ -1691,6 +1693,19 @@ static void o2net_still_up(struct work_struct *work)
/* ------------------------------------------------------------ */
+static void o2net_hb_node_up(struct o2net_node *nn)
+{
+ /* ensure an immediate connect attempt */
+ nn->nn_last_connect_attempt = jiffies -
+ (msecs_to_jiffies(o2net_reconnect_delay()) + 1);
+
+ spin_lock(&nn->nn_lock);
+ atomic_set(&nn->nn_timeout, 0);
+ if (nn->nn_persistent_error)
+ o2net_set_nn_state(nn, NULL, 0, 0);
+ spin_unlock(&nn->nn_lock);
+}
+
void o2net_disconnect_node(struct o2nm_node *node)
{
struct o2net_node *nn = o2net_nn_from_num(node->nd_num);
@@ -1712,41 +1727,39 @@ void o2net_disconnect_node(struct o2nm_node *node)
static void o2net_hb_node_down_cb(struct o2nm_node *node, int node_num,
void *data)
{
+ u8 this_node;
+
o2quo_hb_down(node_num);
if (!node)
- return;
+ goto out;
- if (node_num != o2nm_this_node())
+ this_node = o2nm_this_node();
+ if (!READ_ONCE(o2net_listening) || this_node == O2NM_MAX_NODES)
+ goto out;
+
+ if (node_num != this_node)
o2net_disconnect_node(node);
+out:
BUG_ON(atomic_read(&o2net_connected_peers) < 0);
}
static void o2net_hb_node_up_cb(struct o2nm_node *node, int node_num,
void *data)
{
- struct o2net_node *nn = o2net_nn_from_num(node_num);
+ u8 this_node;
o2quo_hb_up(node_num);
BUG_ON(!node);
- /* ensure an immediate connect attempt */
- nn->nn_last_connect_attempt = jiffies -
- (msecs_to_jiffies(o2net_reconnect_delay()) + 1);
+ this_node = o2nm_this_node();
+ if (!READ_ONCE(o2net_listening) || this_node == O2NM_MAX_NODES)
+ return;
- if (node_num != o2nm_this_node()) {
- /* believe it or not, accept and node heartbeating testing
- * can succeed for this node before we got here.. so
- * only use set_nn_state to clear the persistent error
- * if that hasn't already happened */
- spin_lock(&nn->nn_lock);
- atomic_set(&nn->nn_timeout, 0);
- if (nn->nn_persistent_error)
- o2net_set_nn_state(nn, NULL, 0, 0);
- spin_unlock(&nn->nn_lock);
- }
+ if (node_num != this_node)
+ o2net_hb_node_up(o2net_nn_from_num(node_num));
}
void o2net_unregister_hb_callbacks(void)
@@ -1755,6 +1768,37 @@ void o2net_unregister_hb_callbacks(void)
o2hb_unregister_callback(NULL, &o2net_hb_down);
}
+/*
+ * Delay heartbeat-driven network work until the local node is fully published
+ * through o2nm_this_node(), then replay the nodes that are already live while
+ * callback delivery stays blocked.
+ */
+void o2net_complete_start_listening(struct o2nm_node *node)
+{
+ unsigned long live_nodes[BITS_TO_LONGS(O2NM_MAX_NODES)];
+ unsigned long node_num;
+ u8 local_node;
+
+ local_node = o2nm_this_node();
+ if (WARN_ON_ONCE(local_node == O2NM_MAX_NODES))
+ return;
+ if (WARN_ON_ONCE(local_node != node->nd_num))
+ return;
+ if (WARN_ON_ONCE(!o2net_wq))
+ return;
+
+ o2hb_callback_read_lock();
+ WRITE_ONCE(o2net_listening, true);
+ o2hb_fill_node_map_locked(live_nodes, O2NM_MAX_NODES);
+ for_each_set_bit(node_num, live_nodes, O2NM_MAX_NODES) {
+ if (node_num == local_node)
+ continue;
+
+ o2net_hb_node_up(o2net_nn_from_num(node_num));
+ }
+ o2hb_callback_read_unlock();
+}
+
int o2net_register_hb_callbacks(void)
{
int ret;
@@ -2033,6 +2077,8 @@ int o2net_start_listening(struct o2nm_node *node)
{
int ret = 0;
+ if (WARN_ON_ONCE(READ_ONCE(o2net_listening)))
+ return -EBUSY;
BUG_ON(o2net_wq != NULL);
BUG_ON(o2net_listen_sock != NULL);
@@ -2064,6 +2110,9 @@ void o2net_stop_listening(struct o2nm_node *node)
BUG_ON(o2net_wq == NULL);
BUG_ON(o2net_listen_sock == NULL);
+ WRITE_ONCE(o2net_listening, false);
+ o2hb_synchronize_callbacks();
+
/* stop the listening socket from generating work */
write_lock_bh(&sock->sk->sk_callback_lock);
sock->sk->sk_data_ready = sock->sk->sk_user_data;
diff --git a/fs/ocfs2/cluster/tcp.h b/fs/ocfs2/cluster/tcp.h
index a75b551d31c7..2e86d42b5faf 100644
--- a/fs/ocfs2/cluster/tcp.h
+++ b/fs/ocfs2/cluster/tcp.h
@@ -96,6 +96,7 @@ struct o2nm_node;
int o2net_register_hb_callbacks(void);
void o2net_unregister_hb_callbacks(void);
int o2net_start_listening(struct o2nm_node *node);
+void o2net_complete_start_listening(struct o2nm_node *node);
void o2net_stop_listening(struct o2nm_node *node);
void o2net_disconnect_node(struct o2nm_node *node);
int o2net_num_connected_peers(void);
diff --git a/fs/ocfs2/namei.c b/fs/ocfs2/namei.c
index 5550f8afa438..1dcd420878dc 100644
--- a/fs/ocfs2/namei.c
+++ b/fs/ocfs2/namei.c
@@ -252,6 +252,7 @@ static int ocfs2_mknod(struct mnt_idmap *idmap,
sigset_t oldset;
int did_block_signals = 0;
struct ocfs2_dentry_lock *dl = NULL;
+ struct ocfs2_acl_state acl_state = { 0 };
trace_ocfs2_mknod(dir, dentry, dentry->d_name.len, dentry->d_name.name,
(unsigned long long)OCFS2_I(dir)->ip_blkno,
@@ -326,10 +327,14 @@ static int ocfs2_mknod(struct mnt_idmap *idmap,
}
}
+ status = ocfs2_acl_init_prepare(inode, dir, parent_fe_bh, &acl_state);
+ if (status < 0)
+ goto leave;
+
/* calculate meta data/clusters for setting security and acl xattr */
- status = ocfs2_calc_xattr_init(dir, parent_fe_bh, mode,
- &si, &want_clusters,
- &xattr_credits, &want_meta);
+ status = ocfs2_calc_xattr_init(dir, mode, &si, &want_clusters,
+ &xattr_credits, &want_meta,
+ &acl_state);
if (status < 0) {
mlog_errno(status);
goto leave;
@@ -407,8 +412,8 @@ static int ocfs2_mknod(struct mnt_idmap *idmap,
inc_nlink(dir);
}
- status = ocfs2_init_acl(handle, inode, dir, new_fe_bh, parent_fe_bh,
- meta_ac, data_ac);
+ status = ocfs2_init_acl(handle, inode, new_fe_bh, meta_ac, data_ac,
+ &acl_state);
if (status < 0) {
mlog_errno(status);
@@ -473,6 +478,8 @@ static int ocfs2_mknod(struct mnt_idmap *idmap,
brelse(parent_fe_bh);
kfree(si.value);
+ ocfs2_acl_init_release(&acl_state);
+
ocfs2_free_dir_lookup_result(&lookup);
if (inode_ac)
diff --git a/fs/ocfs2/xattr.c b/fs/ocfs2/xattr.c
index 80f2e803def8..52156e849543 100644
--- a/fs/ocfs2/xattr.c
+++ b/fs/ocfs2/xattr.c
@@ -607,13 +607,10 @@ int ocfs2_calc_security_init(struct inode *dir,
return ret;
}
-int ocfs2_calc_xattr_init(struct inode *dir,
- struct buffer_head *dir_bh,
- umode_t mode,
+int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
struct ocfs2_security_xattr_info *si,
- int *want_clusters,
- int *xattr_credits,
- int *want_meta)
+ int *want_clusters, int *xattr_credits,
+ int *want_meta, struct ocfs2_acl_state *acl_state)
{
int ret = 0;
struct ocfs2_super *osb = OCFS2_SB(dir->i_sb);
@@ -624,19 +621,15 @@ int ocfs2_calc_xattr_init(struct inode *dir,
si->value_len);
if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
- down_read(&OCFS2_I(dir)->ip_xattr_sem);
- acl_len = ocfs2_xattr_get_nolock(dir, dir_bh,
- OCFS2_XATTR_INDEX_POSIX_ACL_DEFAULT,
- "", NULL, 0);
- up_read(&OCFS2_I(dir)->ip_xattr_sem);
- if (acl_len > 0) {
- a_size = ocfs2_xattr_entry_real_size(0, acl_len);
- if (S_ISDIR(mode))
- a_size <<= 1;
- } else if (acl_len != 0 && acl_len != -ENODATA) {
- ret = acl_len;
- mlog_errno(ret);
- return ret;
+ if (acl_state->default_acl && S_ISDIR(mode)) {
+ acl_len = acl_state->default_acl->a_count *
+ sizeof(struct ocfs2_acl_entry);
+ a_size += ocfs2_xattr_entry_real_size(0, acl_len);
+ }
+ if (acl_state->acl) {
+ acl_len = acl_state->acl->a_count *
+ sizeof(struct ocfs2_acl_entry);
+ a_size += ocfs2_xattr_entry_real_size(0, acl_len);
}
}
@@ -679,14 +672,33 @@ int ocfs2_calc_xattr_init(struct inode *dir,
new_clusters);
*want_clusters += new_clusters;
}
- if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL &&
- acl_len > OCFS2_XATTR_INLINE_SIZE) {
- /* for directory, it has DEFAULT and ACCESS two types of acls */
- new_clusters = (S_ISDIR(mode) ? 2 : 1) *
- ocfs2_clusters_for_bytes(dir->i_sb, acl_len);
- *xattr_credits += ocfs2_clusters_to_blocks(dir->i_sb,
- new_clusters);
- *want_clusters += new_clusters;
+ if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
+ if (acl_state->default_acl && S_ISDIR(mode)) {
+ acl_len = acl_state->default_acl->a_count *
+ sizeof(struct ocfs2_acl_entry);
+ if (acl_len > OCFS2_XATTR_INLINE_SIZE) {
+ new_clusters =
+ ocfs2_clusters_for_bytes(dir->i_sb,
+ acl_len);
+ *xattr_credits +=
+ ocfs2_clusters_to_blocks(dir->i_sb,
+ new_clusters);
+ *want_clusters += new_clusters;
+ }
+ }
+ if (acl_state->acl) {
+ acl_len = acl_state->acl->a_count *
+ sizeof(struct ocfs2_acl_entry);
+ if (acl_len > OCFS2_XATTR_INLINE_SIZE) {
+ new_clusters =
+ ocfs2_clusters_for_bytes(dir->i_sb,
+ acl_len);
+ *xattr_credits +=
+ ocfs2_clusters_to_blocks(dir->i_sb,
+ new_clusters);
+ *want_clusters += new_clusters;
+ }
+ }
}
return ret;
@@ -3445,9 +3457,10 @@ static int __ocfs2_xattr_set_handle(struct inode *inode,
}
/*
- * This function only called duing creating inode
- * for init security/acl xattrs of the new inode.
- * All transanction credits have been reserved in mknod.
+ * This helper is only for setting initial ACL or security xattrs on an inode
+ * that is still unpublished, unhashed, and unattached to a dentry.
+ * Ordinary xattr updates must use ocfs2_xattr_set().
+ * All transaction credits have been reserved in mknod or symlink callers.
*/
int ocfs2_xattr_set_handle(handle_t *handle,
struct inode *inode,
@@ -3504,8 +3517,6 @@ int ocfs2_xattr_set_handle(handle_t *handle,
xis.inode_bh = xbs.inode_bh = di_bh;
di = (struct ocfs2_dinode *)di_bh->b_data;
- down_write(&OCFS2_I(inode)->ip_xattr_sem);
-
ret = ocfs2_xattr_ibody_find(inode, name_index, name, &xis);
if (ret)
goto cleanup;
@@ -3518,7 +3529,6 @@ int ocfs2_xattr_set_handle(handle_t *handle,
ret = __ocfs2_xattr_set_handle(inode, di, &xi, &xis, &xbs, &ctxt);
cleanup:
- up_write(&OCFS2_I(inode)->ip_xattr_sem);
brelse(xbs.xattr_bh);
ocfs2_xattr_bucket_free(xbs.bucket);
@@ -7208,6 +7218,7 @@ int ocfs2_init_security_and_acl(struct inode *dir,
{
int ret = 0;
struct buffer_head *dir_bh = NULL;
+ struct ocfs2_acl_state acl_state = { 0 };
ret = ocfs2_init_security_get(inode, dir, qstr, NULL);
if (ret) {
@@ -7220,10 +7231,17 @@ int ocfs2_init_security_and_acl(struct inode *dir,
mlog_errno(ret);
goto leave;
}
- ret = ocfs2_init_acl(NULL, inode, dir, NULL, dir_bh, NULL, NULL);
+
+ ret = ocfs2_acl_init_prepare(inode, dir, dir_bh, &acl_state);
+ if (ret)
+ goto unlock;
+
+ ret = ocfs2_init_acl(NULL, inode, NULL, NULL, NULL, &acl_state);
if (ret)
mlog_errno(ret);
+unlock:
+ ocfs2_acl_init_release(&acl_state);
ocfs2_inode_unlock(dir, 0);
brelse(dir_bh);
leave:
diff --git a/fs/ocfs2/xattr.h b/fs/ocfs2/xattr.h
index 65e9aa743919..5cdd6c6b4006 100644
--- a/fs/ocfs2/xattr.h
+++ b/fs/ocfs2/xattr.h
@@ -55,9 +55,12 @@ int ocfs2_init_security_set(handle_t *, struct inode *,
int ocfs2_calc_security_init(struct inode *,
struct ocfs2_security_xattr_info *,
int *, int *, struct ocfs2_alloc_context **);
-int ocfs2_calc_xattr_init(struct inode *, struct buffer_head *,
- umode_t, struct ocfs2_security_xattr_info *,
- int *, int *, int *);
+
+struct ocfs2_acl_state;
+int ocfs2_calc_xattr_init(struct inode *dir, umode_t mode,
+ struct ocfs2_security_xattr_info *si,
+ int *want_clusters, int *xattr_credits,
+ int *want_meta, struct ocfs2_acl_state *acl_state);
/*
* xattrs can live inside an inode, as part of an external xattr block,
diff --git a/fs/smb/client/cifs_debug.c b/fs/smb/client/cifs_debug.c
index 9092051776fc..185a618f5cf1 100644
--- a/fs/smb/client/cifs_debug.c
+++ b/fs/smb/client/cifs_debug.c
@@ -682,7 +682,7 @@ static ssize_t cifs_stats_proc_write(struct file *file,
atomic_set(&server->smb2slowcmd[i], 0);
server->time_per_cmd[i] = 0;
server->slowest_cmd[i] = 0;
- server->fastest_cmd[0] = 0;
+ server->fastest_cmd[i] = 0;
}
#endif /* CONFIG_CIFS_STATS2 */
list_for_each_entry(ses, &server->smb_ses_list, smb_ses_list) {
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 5a8a9e4ba36d..bfb0298f6b10 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -4458,8 +4458,10 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
if (rc)
return rc;
- if (server == NULL)
- return -ECONNABORTED;
+ if (!server) {
+ rc = -ECONNABORTED;
+ goto free_req;
+ }
shdr = &req->hdr;
shdr->Id.SyncId.ProcessId = cpu_to_le32(io_parms->pid);
@@ -4490,8 +4492,10 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
rdata->mr = smbd_register_mr(server->smbd_conn, &rdata->subreq.io_iter,
true, need_invalidate);
- if (!rdata->mr)
- return -EAGAIN;
+ if (!rdata->mr) {
+ rc = -EAGAIN;
+ goto free_req;
+ }
req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
if (need_invalidate)
@@ -4534,6 +4538,10 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
*buf = req;
return rc;
+
+free_req:
+ cifs_small_buf_release(req);
+ return rc;
}
static void
diff --git a/fs/smb/server/auth.c b/fs/smb/server/auth.c
index bd2e7a8cfbb3..d0f940367623 100644
--- a/fs/smb/server/auth.c
+++ b/fs/smb/server/auth.c
@@ -547,6 +547,7 @@ int ksmbd_krb5_authenticate(struct ksmbd_session *sess, char *in_blob,
resp_ext = ksmbd_ipc_login_request_ext(resp->login_response.account);
user = ksmbd_alloc_user(&resp->login_response, resp_ext);
+ kvfree(resp_ext);
if (!user) {
ksmbd_debug(AUTH, "login failure\n");
retval = -ENOMEM;
diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c
index 5a896b146f43..c5f9705825b8 100644
--- a/fs/smb/server/connection.c
+++ b/fs/smb/server/connection.c
@@ -11,6 +11,7 @@
#include "server.h"
#include "smb_common.h"
#include "mgmt/ksmbd_ida.h"
+#include "mgmt/user_session.h"
#include "connection.h"
#include "transport_tcp.h"
#include "transport_rdma.h"
@@ -123,6 +124,7 @@ void ksmbd_conn_free(struct ksmbd_conn *conn)
kvfree(conn->request_buf);
kfree(conn->preauth_info);
kfree(conn->mechToken);
+ ksmbd_preauth_session_destroy(conn);
ksmbd_conn_put(conn);
}
@@ -161,6 +163,7 @@ struct ksmbd_conn *ksmbd_conn_alloc(void)
init_waitqueue_head(&conn->r_count_q);
INIT_LIST_HEAD(&conn->requests);
INIT_LIST_HEAD(&conn->async_requests);
+ INIT_LIST_HEAD(&conn->preauth_sess_table);
spin_lock_init(&conn->request_lock);
spin_lock_init(&conn->credits_lock);
ida_init(&conn->async_ida);
@@ -252,6 +255,22 @@ void ksmbd_all_conn_set_status(u64 sess_id, u32 status)
up_read(&conn_list_lock);
}
+void ksmbd_conn_abort(struct ksmbd_conn *conn)
+{
+ bool shutdown = false;
+
+ spin_lock(&conn->request_lock);
+ if (!ksmbd_conn_exiting(conn) && !ksmbd_conn_releasing(conn)) {
+ ksmbd_conn_set_exiting(conn);
+ shutdown = true;
+ }
+ spin_unlock(&conn->request_lock);
+ wake_up_all(&conn->req_running_q);
+
+ if (shutdown && conn->transport->ops->shutdown)
+ conn->transport->ops->shutdown(conn->transport);
+}
+
void ksmbd_conn_wait_idle(struct ksmbd_conn *conn)
{
wait_event(conn->req_running_q, atomic_read(&conn->req_running) < 2);
diff --git a/fs/smb/server/connection.h b/fs/smb/server/connection.h
index 8728899d5e75..96ef44133051 100644
--- a/fs/smb/server/connection.h
+++ b/fs/smb/server/connection.h
@@ -166,6 +166,7 @@ struct ksmbd_conn *ksmbd_conn_alloc(void);
void ksmbd_conn_free(struct ksmbd_conn *conn);
struct ksmbd_conn *ksmbd_conn_get(struct ksmbd_conn *conn);
void ksmbd_conn_put(struct ksmbd_conn *conn);
+void ksmbd_conn_abort(struct ksmbd_conn *conn);
int ksmbd_conn_wq_init(void);
void ksmbd_conn_wq_destroy(void);
bool ksmbd_conn_lookup_dialect(struct ksmbd_conn *c);
diff --git a/fs/smb/server/mgmt/share_config.c b/fs/smb/server/mgmt/share_config.c
index d3d5f99bdd34..4a5bc3c14428 100644
--- a/fs/smb/server/mgmt/share_config.c
+++ b/fs/smb/server/mgmt/share_config.c
@@ -158,6 +158,11 @@ static struct ksmbd_share_config *share_config_request(struct ksmbd_work *work,
atomic_set(&share->refcount, 1);
INIT_LIST_HEAD(&share->veto_list);
share->name = kstrdup(name, KSMBD_DEFAULT_GFP);
+ if (!share->name) {
+ kill_share(share);
+ share = NULL;
+ goto out;
+ }
if (!test_share_config_flag(share, KSMBD_SHARE_FLAG_PIPE)) {
int path_len = PATH_MAX;
@@ -199,7 +204,7 @@ static struct ksmbd_share_config *share_config_request(struct ksmbd_work *work,
share->path = NULL;
}
}
- if (ret || !share->name) {
+ if (ret) {
kill_share(share);
share = NULL;
goto out;
diff --git a/fs/smb/server/mgmt/user_config.c b/fs/smb/server/mgmt/user_config.c
index d051ac3bc831..53db14e91bfd 100644
--- a/fs/smb/server/mgmt/user_config.c
+++ b/fs/smb/server/mgmt/user_config.c
@@ -26,6 +26,7 @@ struct ksmbd_user *ksmbd_login_user(const char *account)
resp_ext = ksmbd_ipc_login_request_ext(account);
user = ksmbd_alloc_user(resp, resp_ext);
+ kvfree(resp_ext);
out:
kvfree(resp);
return user;
diff --git a/fs/smb/server/mgmt/user_session.c b/fs/smb/server/mgmt/user_session.c
index ecd511351f19..05fdee54209f 100644
--- a/fs/smb/server/mgmt/user_session.c
+++ b/fs/smb/server/mgmt/user_session.c
@@ -212,10 +212,21 @@ static void ksmbd_expire_session(struct ksmbd_conn *conn)
int ksmbd_session_register(struct ksmbd_conn *conn,
struct ksmbd_session *sess)
{
+ int ret;
+
sess->dialect = conn->dialect;
memcpy(sess->ClientGUID, conn->ClientGUID, SMB2_CLIENT_GUID_SIZE);
ksmbd_expire_session(conn);
- return xa_err(xa_store(&conn->sessions, sess->id, sess, KSMBD_DEFAULT_GFP));
+ ret = xa_err(xa_store(&conn->sessions, sess->id, sess,
+ KSMBD_DEFAULT_GFP));
+ if (ret) {
+ down_write(&sessions_table_lock);
+ hash_del(&sess->hlist);
+ up_write(&sessions_table_lock);
+ ksmbd_user_session_put(sess);
+ }
+
+ return ret;
}
static int ksmbd_chann_del(struct ksmbd_conn *conn, struct ksmbd_session *sess)
@@ -374,6 +385,17 @@ struct preauth_session *ksmbd_preauth_session_alloc(struct ksmbd_conn *conn,
return sess;
}
+void ksmbd_preauth_session_destroy(struct ksmbd_conn *conn)
+{
+ struct preauth_session *sess, *tmp;
+
+ list_for_each_entry_safe(sess, tmp, &conn->preauth_sess_table,
+ preauth_entry) {
+ list_del(&sess->preauth_entry);
+ kfree(sess);
+ }
+}
+
void destroy_previous_session(struct ksmbd_conn *conn,
struct ksmbd_user *user, u64 id)
{
diff --git a/fs/smb/server/mgmt/user_session.h b/fs/smb/server/mgmt/user_session.h
index cba7f688f6b5..525650d6876c 100644
--- a/fs/smb/server/mgmt/user_session.h
+++ b/fs/smb/server/mgmt/user_session.h
@@ -101,6 +101,7 @@ void destroy_previous_session(struct ksmbd_conn *conn,
struct ksmbd_user *user, u64 id);
struct preauth_session *ksmbd_preauth_session_alloc(struct ksmbd_conn *conn,
u64 sess_id);
+void ksmbd_preauth_session_destroy(struct ksmbd_conn *conn);
struct preauth_session *ksmbd_preauth_session_lookup(struct ksmbd_conn *conn,
unsigned long long id);
diff --git a/fs/smb/server/server.c b/fs/smb/server/server.c
index 4c7924e25417..a61863887501 100644
--- a/fs/smb/server/server.c
+++ b/fs/smb/server/server.c
@@ -170,8 +170,10 @@ static void __handle_ksmbd_work(struct ksmbd_work *work,
if (conn->ops->is_transform_hdr &&
conn->ops->is_transform_hdr(work->request_buf)) {
rc = conn->ops->decrypt_req(work);
- if (rc < 0)
+ if (rc < 0) {
+ ksmbd_conn_abort(conn);
return;
+ }
work->encrypted = true;
}
diff --git a/fs/smb/server/smb2ops.c b/fs/smb/server/smb2ops.c
index bcf05caa2304..4c50b7f57f76 100644
--- a/fs/smb/server/smb2ops.c
+++ b/fs/smb/server/smb2ops.c
@@ -262,8 +262,10 @@ void init_smb3_02_server(struct ksmbd_conn *conn)
if (server_conf.flags & KSMBD_GLOBAL_FLAG_SMB3_MULTICHANNEL)
conn->vals->req_capabilities |= SMB2_GLOBAL_CAP_MULTI_CHANNEL;
- if (server_conf.flags & KSMBD_GLOBAL_FLAG_DURABLE_HANDLE)
- conn->vals->req_capabilities |= SMB2_GLOBAL_CAP_PERSISTENT_HANDLES;
+ /*
+ * Durable handles are in-memory only. Do not advertise persistent
+ * handles until CA recovery and fencing are implemented.
+ */
}
/**
@@ -286,10 +288,7 @@ int init_smb3_11_server(struct ksmbd_conn *conn)
if (server_conf.flags & KSMBD_GLOBAL_FLAG_SMB3_MULTICHANNEL)
conn->vals->req_capabilities |= SMB2_GLOBAL_CAP_MULTI_CHANNEL;
- if (server_conf.flags & KSMBD_GLOBAL_FLAG_DURABLE_HANDLE)
- conn->vals->req_capabilities |= SMB2_GLOBAL_CAP_PERSISTENT_HANDLES;
-
- INIT_LIST_HEAD(&conn->preauth_sess_table);
+ /* See init_smb3_02_server(): persistent handles require CA recovery. */
return 0;
}
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index b0a3796a2816..ebb35e95aef5 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -1169,7 +1169,7 @@ int smb2_handle_negotiate(struct ksmbd_work *work)
KSMBD_DEFAULT_GFP);
if (!conn->preauth_info) {
rc = -ENOMEM;
- rsp->hdr.Status = STATUS_INVALID_PARAMETER;
+ rsp->hdr.Status = STATUS_INSUFFICIENT_RESOURCES;
goto err_out;
}
@@ -1266,7 +1266,7 @@ int smb2_handle_negotiate(struct ksmbd_work *work)
ksmbd_conn_set_need_setup(conn);
err_out:
- if (rc)
+ if (rc && rsp->hdr.Status == STATUS_SUCCESS)
rsp->hdr.Status = STATUS_INSUFFICIENT_RESOURCES;
if (!rc)
@@ -2056,12 +2056,11 @@ int smb2_tree_connect(struct ksmbd_work *work)
write_unlock(&sess->tree_conns_lock);
rsp->StructureSize = cpu_to_le16(16);
out_err1:
- if (server_conf.flags & KSMBD_GLOBAL_FLAG_DURABLE_HANDLE && share &&
- test_share_config_flag(share,
- KSMBD_SHARE_FLAG_CONTINUOUS_AVAILABILITY))
- rsp->Capabilities = SMB2_SHARE_CAP_CONTINUOUS_AVAILABILITY;
- else
- rsp->Capabilities = 0;
+ /*
+ * A configured CA share is not continuously available until persistent
+ * open recovery, ownership fencing, and failover are implemented.
+ */
+ rsp->Capabilities = 0;
rsp->Reserved = 0;
/* default manual caching */
rsp->ShareFlags = SMB2_SHAREFLAG_MANUAL_CACHING;
@@ -3018,29 +3017,23 @@ int smb2_open(struct ksmbd_work *work)
if (dh_info.reconnected == true) {
rc = smb2_check_durable_oplock(conn, share, dh_info.fp,
lc, sess->user, name);
- if (rc) {
- ksmbd_put_durable_fd(dh_info.fp);
+ if (rc)
goto err_out2;
- }
rc = ksmbd_reopen_durable_fd(work, dh_info.fp);
- if (rc) {
- ksmbd_put_durable_fd(dh_info.fp);
+ if (rc)
goto err_out2;
- }
fp = dh_info.fp;
if (ksmbd_override_fsids(work)) {
rc = -ENOMEM;
- ksmbd_put_durable_fd(dh_info.fp);
goto err_out2;
}
file_info = FILE_OPENED;
rc = ksmbd_vfs_getattr(&fp->filp->f_path, &stat);
- ksmbd_put_durable_fd(fp);
if (rc)
goto err_out2;
@@ -3810,6 +3803,20 @@ int smb2_open(struct ksmbd_work *work)
ksmbd_debug(SMB, "Error response: %x\n", rsp->hdr.Status);
}
+ if (dh_info.reconnected) {
+ /*
+ * If reconnect succeeded, fp was republished in the
+ * session file table. On a later error, ksmbd_fd_put()
+ * above drops the session reference; drop the durable
+ * lookup reference through the same session-aware path so
+ * final close removes the volatile id before freeing fp.
+ */
+ if (rc && fp == dh_info.fp)
+ ksmbd_fd_put(work, dh_info.fp);
+ else
+ ksmbd_put_durable_fd(dh_info.fp);
+ }
+
kfree(name);
kfree(lc);
@@ -6739,13 +6746,18 @@ static noinline int smb2_read_pipe(struct ksmbd_work *work)
}
aux_payload_buf =
- kvmalloc(rpc_resp->payload_sz, KSMBD_DEFAULT_GFP);
+ kvmalloc(ALIGN(rpc_resp->payload_sz, 8),
+ KSMBD_DEFAULT_GFP);
if (!aux_payload_buf) {
err = -ENOMEM;
goto out;
}
memcpy(aux_payload_buf, rpc_resp->payload, rpc_resp->payload_sz);
+ if (rpc_resp->payload_sz & 7)
+ memset(aux_payload_buf + rpc_resp->payload_sz, 0,
+ ALIGN(rpc_resp->payload_sz, 8) -
+ rpc_resp->payload_sz);
nbytes = rpc_resp->payload_sz;
err = ksmbd_iov_pin_rsp_read(work, (void *)rsp,
@@ -8193,9 +8205,20 @@ static inline int fsctl_set_sparse(struct ksmbd_work *work, u64 id,
int ret = 0;
__le32 old_fattr;
+ if (!test_tree_conn_flag(work->tcon, KSMBD_TREE_CONN_FLAG_WRITABLE)) {
+ ksmbd_debug(SMB, "User does not have write permission\n");
+ return -EACCES;
+ }
+
fp = ksmbd_lookup_fd_fast(work, id);
if (!fp)
return -ENOENT;
+
+ if (!(fp->daccess & (FILE_WRITE_DATA_LE | FILE_WRITE_ATTRIBUTES_LE))) {
+ ret = -EACCES;
+ goto out;
+ }
+
idmap = file_mnt_idmap(fp->filp);
old_fattr = fp->f_ci->m_fattr;
diff --git a/fs/smb/server/smbacl.c b/fs/smb/server/smbacl.c
index d3d0a22620f9..0879012aa63c 100644
--- a/fs/smb/server/smbacl.c
+++ b/fs/smb/server/smbacl.c
@@ -1471,7 +1471,7 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
DELETE;
ace = (struct smb_ace *)((char *)pdacl + sizeof(struct smb_acl));
- aces_size = acl_size - sizeof(struct smb_acl);
+ aces_size = pdacl_size - sizeof(struct smb_acl);
for (i = 0; i < le16_to_cpu(pdacl->num_aces); i++) {
if (aces_size < offsetof(struct smb_ace, sid) +
CIFS_SID_BASE_SIZE)
@@ -1495,7 +1495,7 @@ int smb_check_perm_dacl(struct ksmbd_conn *conn, const struct path *path,
id_to_sid(uid, sid_type, &sid);
ace = (struct smb_ace *)((char *)pdacl + sizeof(struct smb_acl));
- aces_size = acl_size - sizeof(struct smb_acl);
+ aces_size = pdacl_size - sizeof(struct smb_acl);
for (i = 0; i < le16_to_cpu(pdacl->num_aces); i++) {
if (aces_size < offsetof(struct smb_ace, sid) +
CIFS_SID_BASE_SIZE)
diff --git a/fs/smb/server/transport_ipc.c b/fs/smb/server/transport_ipc.c
index 2f5b0bedf2a9..a0b9b3def334 100644
--- a/fs/smb/server/transport_ipc.c
+++ b/fs/smb/server/transport_ipc.c
@@ -498,6 +498,9 @@ static int ipc_validate_msg(struct ipc_msg_table_entry *entry)
{
struct ksmbd_rpc_command *resp = entry->response;
+ if (entry->msg_sz < sizeof(struct ksmbd_rpc_command))
+ return -EINVAL;
+
if (check_add_overflow(sizeof(struct ksmbd_rpc_command),
resp->payload_sz, &msg_sz))
return -EINVAL;
@@ -507,6 +510,9 @@ static int ipc_validate_msg(struct ipc_msg_table_entry *entry)
{
struct ksmbd_spnego_authen_response *resp = entry->response;
+ if (entry->msg_sz < sizeof(struct ksmbd_spnego_authen_response))
+ return -EINVAL;
+
msg_sz = sizeof(struct ksmbd_spnego_authen_response) +
resp->session_key_len + resp->spnego_blob_len;
break;
@@ -515,6 +521,9 @@ static int ipc_validate_msg(struct ipc_msg_table_entry *entry)
{
struct ksmbd_share_config_response *resp = entry->response;
+ if (entry->msg_sz < sizeof(struct ksmbd_share_config_response))
+ return -EINVAL;
+
if (resp->payload_sz) {
if (resp->payload_sz < resp->veto_list_sz)
return -EINVAL;
@@ -529,6 +538,9 @@ static int ipc_validate_msg(struct ipc_msg_table_entry *entry)
{
struct ksmbd_login_response_ext *resp = entry->response;
+ if (entry->msg_sz < sizeof(struct ksmbd_login_response_ext))
+ return -EINVAL;
+
if (resp->ngroups) {
if (resp->ngroups < 0 ||
resp->ngroups > NGROUPS_MAX) {
diff --git a/fs/smb/server/transport_tcp.c b/fs/smb/server/transport_tcp.c
index d731bab98568..902a6265e323 100644
--- a/fs/smb/server/transport_tcp.c
+++ b/fs/smb/server/transport_tcp.c
@@ -467,6 +467,11 @@ static void ksmbd_tcp_disconnect(struct ksmbd_transport *t)
atomic_dec(&active_num_conn);
}
+static void ksmbd_tcp_shutdown(struct ksmbd_transport *t)
+{
+ kernel_sock_shutdown(TCP_TRANS(t)->sock, SHUT_RDWR);
+}
+
static void tcp_destroy_socket(struct socket *ksmbd_socket)
{
int ret;
@@ -722,5 +727,6 @@ static const struct ksmbd_transport_ops ksmbd_tcp_transport_ops = {
.read = ksmbd_tcp_read,
.writev = ksmbd_tcp_writev,
.disconnect = ksmbd_tcp_disconnect,
+ .shutdown = ksmbd_tcp_shutdown,
.free_transport = ksmbd_tcp_free_transport,
};
diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c
index d77c3e95645d..6bc19b164dfb 100644
--- a/fs/smb/server/vfs.c
+++ b/fs/smb/server/vfs.c
@@ -329,9 +329,6 @@ static int check_lock_range(struct file *filp, loff_t start, loff_t end,
struct file_lock_context *ctx = locks_inode_context(file_inode(filp));
int error = 0;
- if (start == end)
- return 0;
-
if (!ctx || list_empty_careful(&ctx->flc_posix))
return 0;
diff --git a/fs/smb/server/vfs_cache.c b/fs/smb/server/vfs_cache.c
index 8b5f50ac61e2..0b654021bec0 100644
--- a/fs/smb/server/vfs_cache.c
+++ b/fs/smb/server/vfs_cache.c
@@ -1103,9 +1103,12 @@ void ksmbd_launch_ksmbd_durable_scavenger(void)
server_conf.dh_task = kthread_run(ksmbd_durable_scavenger,
(void *)NULL, "ksmbd-durable-scavenger");
- if (IS_ERR(server_conf.dh_task))
+ if (IS_ERR(server_conf.dh_task)) {
pr_err("cannot start conn thread, err : %ld\n",
PTR_ERR(server_conf.dh_task));
+ server_conf.dh_task = NULL;
+ durable_scavenger_running = false;
+ }
mutex_unlock(&durable_scavenger_lock);
}
diff --git a/fs/squashfs/cache.c b/fs/squashfs/cache.c
index 25bf038b880a..2091af741632 100644
--- a/fs/squashfs/cache.c
+++ b/fs/squashfs/cache.c
@@ -295,7 +295,7 @@ int squashfs_copy_data(void *buffer, struct squashfs_cache_entry *entry,
{
int remaining = length;
- if (length == 0)
+ if (length == 0 || offset < 0)
return 0;
else if (buffer == NULL)
return min(length, entry->length - offset);
diff --git a/fs/super.c b/fs/super.c
index bccfadb37ff2..2418e10cffa4 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -2065,7 +2065,7 @@ int freeze_super(struct super_block *sb, enum freeze_holder who)
int ret;
if (!super_lock_excl(sb)) {
- WARN_ON_ONCE("Dying superblock while freezing!");
+ WARN_ONCE(1, "Dying superblock while freezing!");
return -EINVAL;
}
atomic_inc(&sb->s_active);
@@ -2219,7 +2219,7 @@ static int thaw_super_locked(struct super_block *sb, enum freeze_holder who)
int thaw_super(struct super_block *sb, enum freeze_holder who)
{
if (!super_lock_excl(sb)) {
- WARN_ON_ONCE("Dying superblock while thawing!");
+ WARN_ONCE(1, "Dying superblock while thawing!");
return -EINVAL;
}
return thaw_super_locked(sb, who);
diff --git a/fs/udf/super.c b/fs/udf/super.c
index c7c5fd45166b..f91b231b7d7b 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -2050,6 +2050,17 @@ static int udf_load_vrs(struct super_block *sb, struct udf_options *uopt,
return 0;
}
+static void udf_mark_buffer_dirty(struct buffer_head *bh)
+{
+ /*
+ * We set buffer uptodate unconditionally here to avoid spurious
+ * warnings from mark_buffer_dirty() when previous EIO has marked
+ * the buffer as !uptodate
+ */
+ set_buffer_uptodate(bh);
+ mark_buffer_dirty(bh);
+}
+
static void udf_finalize_lvid(struct logicalVolIntegrityDesc *lvid)
{
struct timespec64 ts;
@@ -2085,7 +2096,7 @@ static void udf_open_lvid(struct super_block *sb)
UDF_SET_FLAG(sb, UDF_FLAG_INCONSISTENT);
udf_finalize_lvid(lvid);
- mark_buffer_dirty(bh);
+ udf_mark_buffer_dirty(bh);
sbi->s_lvid_dirty = 0;
mutex_unlock(&sbi->s_alloc_mutex);
/* Make opening of filesystem visible on the media immediately */
@@ -2118,14 +2129,8 @@ static void udf_close_lvid(struct super_block *sb)
if (!UDF_QUERY_FLAG(sb, UDF_FLAG_INCONSISTENT))
lvid->integrityType = cpu_to_le32(LVID_INTEGRITY_TYPE_CLOSE);
- /*
- * We set buffer uptodate unconditionally here to avoid spurious
- * warnings from mark_buffer_dirty() when previous EIO has marked
- * the buffer as !uptodate
- */
- set_buffer_uptodate(bh);
udf_finalize_lvid(lvid);
- mark_buffer_dirty(bh);
+ udf_mark_buffer_dirty(bh);
sbi->s_lvid_dirty = 0;
mutex_unlock(&sbi->s_alloc_mutex);
/* Make closing of filesystem visible on the media immediately */
@@ -2407,7 +2412,7 @@ static int udf_sync_fs(struct super_block *sb, int wait)
* Blockdevice will be synced later so we don't have to submit
* the buffer for IO
*/
- mark_buffer_dirty(bh);
+ udf_mark_buffer_dirty(bh);
sbi->s_lvid_dirty = 0;
}
mutex_unlock(&sbi->s_alloc_mutex);
diff --git a/fs/udf/symlink.c b/fs/udf/symlink.c
index fe03745d09b1..a05d1888a2ba 100644
--- a/fs/udf/symlink.c
+++ b/fs/udf/symlink.c
@@ -36,6 +36,8 @@ static int udf_pc_to_char(struct super_block *sb, unsigned char *from,
/* Reserve one byte for terminating \0 */
tolen--;
while (elen < fromlen) {
+ if (fromlen - elen < sizeof(struct pathComponent))
+ return -EIO;
pc = (struct pathComponent *)(from + elen);
elen += sizeof(struct pathComponent);
switch (pc->componentType) {
diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h
index 62d368bcd9ec..31767c65be20 100644
--- a/include/acpi/cppc_acpi.h
+++ b/include/acpi/cppc_acpi.h
@@ -159,7 +159,7 @@ extern int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val);
extern int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val);
extern int cppc_get_epp_perf(int cpunum, u64 *epp_perf);
extern int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable);
-extern int cppc_get_auto_sel_caps(int cpunum, struct cppc_perf_caps *perf_caps);
+extern int cppc_get_auto_sel(int cpu, bool *enable);
extern int cppc_set_auto_sel(int cpu, bool enable);
extern int amd_get_highest_perf(unsigned int cpu, u32 *highest_perf);
extern int amd_get_boost_ratio_numerator(unsigned int cpu, u64 *numerator);
@@ -229,11 +229,11 @@ static inline int cppc_get_epp_perf(int cpunum, u64 *epp_perf)
{
return -EOPNOTSUPP;
}
-static inline int cppc_set_auto_sel(int cpu, bool enable)
+static inline int cppc_get_auto_sel(int cpu, bool *enable)
{
return -EOPNOTSUPP;
}
-static inline int cppc_get_auto_sel_caps(int cpunum, struct cppc_perf_caps *perf_caps)
+static inline int cppc_set_auto_sel(int cpu, bool enable)
{
return -EOPNOTSUPP;
}
diff --git a/include/acpi/pcc.h b/include/acpi/pcc.h
index 699c1a37b8e7..0462bb9da651 100644
--- a/include/acpi/pcc.h
+++ b/include/acpi/pcc.h
@@ -38,7 +38,6 @@ struct pcc_mbox_chan {
extern struct pcc_mbox_chan *
pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id);
extern void pcc_mbox_free_channel(struct pcc_mbox_chan *chan);
-extern int pcc_mbox_ioremap(struct mbox_chan *chan);
#else
static inline struct pcc_mbox_chan *
pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id)
@@ -46,10 +45,6 @@ pcc_mbox_request_channel(struct mbox_client *cl, int subspace_id)
return ERR_PTR(-ENODEV);
}
static inline void pcc_mbox_free_channel(struct pcc_mbox_chan *chan) { }
-static inline int pcc_mbox_ioremap(struct mbox_chan *chan)
-{
- return 0;
-};
#endif
#endif /* _PCC_H */
diff --git a/include/acpi/processor.h b/include/acpi/processor.h
index a17e97e634a6..960ce42386cc 100644
--- a/include/acpi/processor.h
+++ b/include/acpi/processor.h
@@ -418,6 +418,8 @@ int acpi_processor_power_init(struct acpi_processor *pr);
int acpi_processor_power_exit(struct acpi_processor *pr);
int acpi_processor_power_state_has_changed(struct acpi_processor *pr);
int acpi_processor_hotplug(struct acpi_processor *pr);
+void acpi_processor_register_idle_driver(void);
+void acpi_processor_unregister_idle_driver(void);
#else
static inline int acpi_processor_power_init(struct acpi_processor *pr)
{
diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h
index bacd111e75f1..cc894f07a435 100644
--- a/include/cxl/mailbox.h
+++ b/include/cxl/mailbox.h
@@ -3,12 +3,49 @@
#ifndef __CXL_MBOX_H__
#define __CXL_MBOX_H__
#include <linux/rcuwait.h>
+#include <uapi/linux/cxl_mem.h>
-struct cxl_mbox_cmd;
+/**
+ * struct cxl_mbox_cmd - A command to be submitted to hardware.
+ * @opcode: (input) The command set and command submitted to hardware.
+ * @payload_in: (input) Pointer to the input payload.
+ * @payload_out: (output) Pointer to the output payload. Must be allocated by
+ * the caller.
+ * @size_in: (input) Number of bytes to load from @payload_in.
+ * @size_out: (input) Max number of bytes loaded into @payload_out.
+ * (output) Number of bytes generated by the device. For fixed size
+ * outputs commands this is always expected to be deterministic. For
+ * variable sized output commands, it tells the exact number of bytes
+ * written.
+ * @min_out: (input) internal command output payload size validation
+ * @poll_count: (input) Number of timeouts to attempt.
+ * @poll_interval_ms: (input) Time between mailbox background command polling
+ * interval timeouts.
+ * @return_code: (output) Error code returned from hardware.
+ *
+ * This is the primary mechanism used to send commands to the hardware.
+ * All the fields except @payload_* correspond exactly to the fields described in
+ * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and
+ * @payload_out are written to, and read from the Command Payload Registers
+ * defined in CXL 2.0 8.2.8.4.8.
+ */
+struct cxl_mbox_cmd {
+ u16 opcode;
+ void *payload_in;
+ void *payload_out;
+ size_t size_in;
+ size_t size_out;
+ size_t min_out;
+ int poll_count;
+ int poll_interval_ms;
+ u16 return_code;
+};
/**
* struct cxl_mailbox - context for CXL mailbox operations
* @host: device that hosts the mailbox
+ * @enabled_cmds: mailbox commands that are enabled by the driver
+ * @exclusive_cmds: mailbox commands that are exclusive to the kernel
* @payload_size: Size of space for payload
* (CXL 3.1 8.2.8.4.3 Mailbox Capabilities Register)
* @mbox_mutex: mutex protects device mailbox and firmware
@@ -17,6 +54,8 @@ struct cxl_mbox_cmd;
*/
struct cxl_mailbox {
struct device *host;
+ DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX);
+ DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX);
size_t payload_size;
struct mutex mbox_mutex; /* lock to protect mailbox context */
struct rcuwait mbox_wait;
diff --git a/include/linux/bpf-cgroup-defs.h b/include/linux/bpf-cgroup-defs.h
index 0985221d5478..c9e6b26abab6 100644
--- a/include/linux/bpf-cgroup-defs.h
+++ b/include/linux/bpf-cgroup-defs.h
@@ -63,6 +63,7 @@ struct cgroup_bpf {
*/
struct hlist_head progs[MAX_CGROUP_BPF_ATTACH_TYPE];
u8 flags[MAX_CGROUP_BPF_ATTACH_TYPE];
+ u64 revisions[MAX_CGROUP_BPF_ATTACH_TYPE];
/* list of cgroup shared storages */
struct list_head storages;
diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 26ee360c345f..a0b510455812 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -420,7 +420,7 @@ int cgroup_bpf_prog_detach(const union bpf_attr *attr,
enum bpf_prog_type ptype);
int cgroup_bpf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog);
int cgroup_bpf_prog_query(const union bpf_attr *attr,
- union bpf_attr __user *uattr);
+ union bpf_attr __user *uattr, u32 uattr_size);
const struct bpf_func_proto *
cgroup_common_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog);
@@ -451,7 +451,8 @@ static inline int cgroup_bpf_link_attach(const union bpf_attr *attr,
}
static inline int cgroup_bpf_prog_query(const union bpf_attr *attr,
- union bpf_attr __user *uattr)
+ union bpf_attr __user *uattr,
+ u32 uattr_size)
{
return -EINVAL;
}
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 9e1731db76d8..47fdbf13abe1 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -500,7 +500,7 @@ static inline void bpf_obj_memcpy(struct btf_record *rec,
if (IS_ERR_OR_NULL(rec)) {
if (long_memcpy)
- bpf_long_memcpy(dst, src, round_up(size, 8));
+ bpf_long_memcpy(dst, src, size);
else
memcpy(dst, src, size);
return;
@@ -523,7 +523,7 @@ static inline void copy_map_value(struct bpf_map *map, void *dst, void *src)
static inline void copy_map_value_long(struct bpf_map *map, void *dst, void *src)
{
- bpf_obj_memcpy(map->record, dst, src, map->value_size, true);
+ bpf_obj_memcpy(map->record, dst, src, round_up(map->value_size, 8), true);
}
static inline void bpf_obj_memzero(struct btf_record *rec, void *dst, u32 size)
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 2f1bfd7562eb..06e8d3c69f9e 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -1239,8 +1239,8 @@ efi_call_acpi_prm_handler(efi_status_t (__efiapi *handler_addr)(u64, void *),
/*
* efi_runtime_service() function identifiers.
- * "NONE" is used by efi_recover_from_page_fault() to check if the page
- * fault happened while executing an efi runtime service.
+ * "NONE" is used by efi_crash_gracefully_on_page_fault() to check if the
+ * page fault happened while executing an efi runtime service.
*/
enum efi_rts_ids {
EFI_NONE,
diff --git a/include/linux/ima.h b/include/linux/ima.h
index abf8923f8fc5..8e08baf16c2f 100644
--- a/include/linux/ima.h
+++ b/include/linux/ima.h
@@ -11,6 +11,7 @@
#include <linux/fs.h>
#include <linux/security.h>
#include <linux/kexec.h>
+#include <linux/secure_boot.h>
#include <crypto/hash_info.h>
struct linux_binprm;
@@ -73,14 +74,8 @@ int ima_validate_range(phys_addr_t phys, size_t size);
#endif
#ifdef CONFIG_IMA_SECURE_AND_OR_TRUSTED_BOOT
-extern bool arch_ima_get_secureboot(void);
extern const char * const *arch_get_ima_policy(void);
#else
-static inline bool arch_ima_get_secureboot(void)
-{
- return false;
-}
-
static inline const char * const *arch_get_ima_policy(void)
{
return NULL;
diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h
index 8c4f3bb24429..e6de7ae55bda 100644
--- a/include/linux/kprobes.h
+++ b/include/linux/kprobes.h
@@ -181,6 +181,7 @@ struct kprobe_blacklist_entry {
struct list_head list;
unsigned long start_addr;
unsigned long end_addr;
+ struct rcu_head rcu;
};
#ifdef CONFIG_KPROBES
diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h
index c2c11004085e..ecbcc2b3e59a 100644
--- a/include/linux/maple_tree.h
+++ b/include/linux/maple_tree.h
@@ -528,7 +528,7 @@ void mas_destroy(struct ma_state *mas);
int mas_expected_entries(struct ma_state *mas, unsigned long nr_entries);
void *mas_prev(struct ma_state *mas, unsigned long min);
-void *mas_prev_range(struct ma_state *mas, unsigned long max);
+void *mas_prev_range(struct ma_state *mas, unsigned long min);
void *mas_next(struct ma_state *mas, unsigned long max);
void *mas_next_range(struct ma_state *mas, unsigned long max);
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index e1b41554a5fb..e4c24c94b9ef 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -344,17 +344,23 @@ enum page_memcg_data_flags {
__NR_MEMCG_DATA_FLAGS = (1UL << 2),
};
+#define __OBJEXTS_ALLOC_FAIL MEMCG_DATA_OBJEXTS
#define __FIRST_OBJEXT_FLAG __NR_MEMCG_DATA_FLAGS
#else /* CONFIG_MEMCG */
+#define __OBJEXTS_ALLOC_FAIL (1UL << 0)
#define __FIRST_OBJEXT_FLAG (1UL << 0)
#endif /* CONFIG_MEMCG */
enum objext_flags {
- /* slabobj_ext vector failed to allocate */
- OBJEXTS_ALLOC_FAIL = __FIRST_OBJEXT_FLAG,
+ /*
+ * Use bit 0 with zero other bits to signal that slabobj_ext vector
+ * failed to allocate. The same bit 0 with valid upper bits means
+ * MEMCG_DATA_OBJEXTS.
+ */
+ OBJEXTS_ALLOC_FAIL = __OBJEXTS_ALLOC_FAIL,
/* the next bit after the last actual flag */
__NR_OBJEXTS_FLAGS = (__FIRST_OBJEXT_FLAG << 1),
};
diff --git a/include/linux/memory.h b/include/linux/memory.h
index c0afee5d126e..f6109556ed10 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -19,6 +19,7 @@
#include <linux/node.h>
#include <linux/compiler.h>
#include <linux/mutex.h>
+#include <linux/memory_hotplug.h>
#define MIN_MEMORY_BLOCK_SIZE (1UL << SECTION_SIZE_BITS)
@@ -64,10 +65,22 @@ struct memory_group {
};
};
+enum memory_block_state {
+ /* These states are exposed to userspace as text strings in sysfs */
+ MEM_ONLINE, /* exposed to userspace */
+ MEM_GOING_OFFLINE, /* exposed to userspace */
+ MEM_OFFLINE, /* exposed to userspace */
+ MEM_GOING_ONLINE,
+ MEM_CANCEL_ONLINE,
+ MEM_CANCEL_OFFLINE,
+ MEM_PREPARE_ONLINE,
+ MEM_FINISH_OFFLINE,
+};
+
struct memory_block {
unsigned long start_section_nr;
- unsigned long state; /* serialized by the dev->lock */
- int online_type; /* for passing data to online routine */
+ enum memory_block_state state; /* serialized by the dev->lock */
+ enum mmop online_type; /* for passing data to online routine */
int nid; /* NID for this memory block */
/*
* The single zone of this memory block if all PFNs of this memory block
@@ -89,16 +102,6 @@ int arch_get_memory_phys_device(unsigned long start_pfn);
unsigned long memory_block_size_bytes(void);
int set_memory_block_size_order(unsigned int order);
-/* These states are exposed to userspace as text strings in sysfs */
-#define MEM_ONLINE (1<<0) /* exposed to userspace */
-#define MEM_GOING_OFFLINE (1<<1) /* exposed to userspace */
-#define MEM_OFFLINE (1<<2) /* exposed to userspace */
-#define MEM_GOING_ONLINE (1<<3)
-#define MEM_CANCEL_ONLINE (1<<4)
-#define MEM_CANCEL_OFFLINE (1<<5)
-#define MEM_PREPARE_ONLINE (1<<6)
-#define MEM_FINISH_OFFLINE (1<<7)
-
struct memory_notify {
/*
* The altmap_start_pfn and altmap_nr_pages fields are designated for
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index 99d6ba41ed29..d93a70c112ee 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -16,11 +16,8 @@ struct resource;
struct vmem_altmap;
struct dev_pagemap;
-#ifdef CONFIG_MEMORY_HOTPLUG
-struct page *pfn_to_online_page(unsigned long pfn);
-
/* Types for control the zone type of onlined and offlined memory */
-enum {
+enum mmop {
/* Offline the memory. */
MMOP_OFFLINE = 0,
/* Online the memory. Zone depends, see default_zone_for_pfn(). */
@@ -31,6 +28,9 @@ enum {
MMOP_ONLINE_MOVABLE,
};
+#ifdef CONFIG_MEMORY_HOTPLUG
+struct page *pfn_to_online_page(unsigned long pfn);
+
/* Flags for add_memory() and friends to specify memory hotplug details. */
typedef int __bitwise mhp_t;
@@ -144,8 +144,6 @@ extern u64 max_mem_size;
extern int mhp_online_type_from_str(const char *str);
-/* Default online_type (MMOP_*) when new memory blocks are added. */
-extern int mhp_default_online_type;
/* If movable_node boot option specified */
extern bool movable_node_enabled;
static inline bool movable_node_is_enabled(void)
@@ -304,7 +302,10 @@ static inline void __remove_memory(u64 start, u64 size) {}
#endif /* CONFIG_MEMORY_HOTREMOVE */
#ifdef CONFIG_MEMORY_HOTPLUG
-extern void __ref free_area_init_core_hotplug(struct pglist_data *pgdat);
+/* Default online_type (MMOP_*) when new memory blocks are added. */
+extern enum mmop mhp_get_default_online_type(void);
+extern void mhp_set_default_online_type(enum mmop online_type);
+int __ref free_area_init_core_hotplug(struct pglist_data *pgdat);
extern int __add_memory(int nid, u64 start, u64 size, mhp_t mhp_flags);
extern int add_memory(int nid, u64 start, u64 size, mhp_t mhp_flags);
extern int add_memory_resource(int nid, struct resource *resource,
@@ -326,8 +327,8 @@ extern void sparse_remove_section(unsigned long pfn, unsigned long nr_pages,
struct dev_pagemap *pgmap);
extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
unsigned long pnum);
-extern struct zone *zone_for_pfn_range(int online_type, int nid,
- struct memory_group *group, unsigned long start_pfn,
+extern struct zone *zone_for_pfn_range(enum mmop online_type,
+ int nid, struct memory_group *group, unsigned long start_pfn,
unsigned long nr_pages);
extern int arch_create_linear_mapping(int nid, u64 start, u64 size,
struct mhp_params *params);
diff --git a/include/linux/mmc/sdio_ids.h b/include/linux/mmc/sdio_ids.h
index 8fb8e7e87f6e..77edf5008337 100644
--- a/include/linux/mmc/sdio_ids.h
+++ b/include/linux/mmc/sdio_ids.h
@@ -110,6 +110,7 @@
#define SDIO_VENDOR_ID_MEDIATEK 0x037a
#define SDIO_DEVICE_ID_MEDIATEK_MT7663 0x7663
#define SDIO_DEVICE_ID_MEDIATEK_MT7668 0x7668
+#define SDIO_DEVICE_ID_MEDIATEK_MT7902 0x790a
#define SDIO_DEVICE_ID_MEDIATEK_MT7961 0x7961
#define SDIO_VENDOR_ID_MICROCHIP_WILC 0x0296
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index b6f948408746..4d0f9aad7b96 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -348,7 +348,7 @@ struct nfs4_layoutreturn {
struct nfs4_xdr_opaque_data ld_private;
};
-#define PNFS_LAYOUTSTATS_MAXSIZE 256
+#define PNFS_LAYOUTSTATS_MAXSIZE 384
struct nfs42_layoutstat_args;
struct nfs42_layoutstat_devinfo;
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 84d4f0657b7a..081f96a415d3 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -2038,4 +2038,85 @@ struct nvme_completion {
#define NVME_MINOR(ver) (((ver) >> 8) & 0xff)
#define NVME_TERTIARY(ver) ((ver) & 0xff)
+enum {
+ NVME_AEN_RESV_LOG_PAGE_AVALIABLE = 0x00,
+};
+
+enum {
+ NVME_PR_LOG_EMPTY_LOG_PAGE = 0x00,
+ NVME_PR_LOG_REGISTRATION_PREEMPTED = 0x01,
+ NVME_PR_LOG_RESERVATION_RELEASED = 0x02,
+ NVME_PR_LOG_RESERVATOPM_PREEMPTED = 0x03,
+};
+
+enum {
+ NVME_PR_NOTIFY_BIT_REG_PREEMPTED = 1,
+ NVME_PR_NOTIFY_BIT_RESV_RELEASED = 2,
+ NVME_PR_NOTIFY_BIT_RESV_PREEMPTED = 3,
+};
+
+struct nvme_pr_log {
+ __le64 count;
+ __u8 type;
+ __u8 nr_pages;
+ __u8 rsvd1[2];
+ __le32 nsid;
+ __u8 rsvd2[48];
+};
+
+struct nvmet_pr_register_data {
+ __le64 crkey;
+ __le64 nrkey;
+};
+
+struct nvmet_pr_acquire_data {
+ __le64 crkey;
+ __le64 prkey;
+};
+
+struct nvmet_pr_release_data {
+ __le64 crkey;
+};
+
+enum nvme_pr_capabilities {
+ NVME_PR_SUPPORT_PTPL = 1,
+ NVME_PR_SUPPORT_WRITE_EXCLUSIVE = 1 << 1,
+ NVME_PR_SUPPORT_EXCLUSIVE_ACCESS = 1 << 2,
+ NVME_PR_SUPPORT_WRITE_EXCLUSIVE_REG_ONLY = 1 << 3,
+ NVME_PR_SUPPORT_EXCLUSIVE_ACCESS_REG_ONLY = 1 << 4,
+ NVME_PR_SUPPORT_WRITE_EXCLUSIVE_ALL_REGS = 1 << 5,
+ NVME_PR_SUPPORT_EXCLUSIVE_ACCESS_ALL_REGS = 1 << 6,
+ NVME_PR_SUPPORT_IEKEY_VER_1_3_DEF = 1 << 7,
+};
+
+enum nvme_pr_register_action {
+ NVME_PR_REGISTER_ACT_REG = 0,
+ NVME_PR_REGISTER_ACT_UNREG = 1,
+ NVME_PR_REGISTER_ACT_REPLACE = 1 << 1,
+};
+
+enum nvme_pr_acquire_action {
+ NVME_PR_ACQUIRE_ACT_ACQUIRE = 0,
+ NVME_PR_ACQUIRE_ACT_PREEMPT = 1,
+ NVME_PR_ACQUIRE_ACT_PREEMPT_AND_ABORT = 1 << 1,
+};
+
+enum nvme_pr_release_action {
+ NVME_PR_RELEASE_ACT_RELEASE = 0,
+ NVME_PR_RELEASE_ACT_CLEAR = 1,
+};
+
+enum nvme_pr_change_ptpl {
+ NVME_PR_CPTPL_NO_CHANGE = 0,
+ NVME_PR_CPTPL_RESV = 1 << 30,
+ NVME_PR_CPTPL_CLEARED = 2 << 30,
+ NVME_PR_CPTPL_PERSIST = 3 << 30,
+};
+
+#define NVME_PR_IGNORE_KEY (1 << 3)
+
+/* Section 8.3.4.5.2 of the NVMe 2.1 */
+#define NVME_AUTH_DHCHAP_MAX_HASH_IDS 30
+#define NVME_AUTH_DHCHAP_MAX_DH_IDS 30
+
#endif /* _LINUX_NVME_H */
diff --git a/include/linux/of_graph.h b/include/linux/of_graph.h
index a4bea62bfa29..a692d9d979a6 100644
--- a/include/linux/of_graph.h
+++ b/include/linux/of_graph.h
@@ -11,6 +11,7 @@
#ifndef __LINUX_OF_GRAPH_H
#define __LINUX_OF_GRAPH_H
+#include <linux/cleanup.h>
#include <linux/types.h>
#include <linux/errno.h>
@@ -37,14 +38,43 @@ struct of_endpoint {
for (child = of_graph_get_next_endpoint(parent, NULL); child != NULL; \
child = of_graph_get_next_endpoint(parent, child))
+/**
+ * for_each_of_graph_port - iterate over every port in a device or ports node
+ * @parent: parent device or ports node containing port
+ * @child: loop variable pointing to the current port node
+ *
+ * When breaking out of the loop, and continue to use the @child, you need to
+ * use return_ptr(@child) or no_free_ptr(@child) not to call __free() for it.
+ */
+#define for_each_of_graph_port(parent, child) \
+ for (struct device_node *child __free(device_node) = of_graph_get_next_port(parent, NULL);\
+ child != NULL; child = of_graph_get_next_port(parent, child))
+
+/**
+ * for_each_of_graph_port_endpoint - iterate over every endpoint in a port node
+ * @parent: parent port node
+ * @child: loop variable pointing to the current endpoint node
+ *
+ * When breaking out of the loop, and continue to use the @child, you need to
+ * use return_ptr(@child) or no_free_ptr(@child) not to call __free() for it.
+ */
+#define for_each_of_graph_port_endpoint(parent, child) \
+ for (struct device_node *child __free(device_node) = of_graph_get_next_port_endpoint(parent, NULL);\
+ child != NULL; child = of_graph_get_next_port_endpoint(parent, child))
+
#ifdef CONFIG_OF
bool of_graph_is_present(const struct device_node *node);
int of_graph_parse_endpoint(const struct device_node *node,
struct of_endpoint *endpoint);
unsigned int of_graph_get_endpoint_count(const struct device_node *np);
+unsigned int of_graph_get_port_count(struct device_node *np);
struct device_node *of_graph_get_port_by_id(struct device_node *node, u32 id);
struct device_node *of_graph_get_next_endpoint(const struct device_node *parent,
struct device_node *previous);
+struct device_node *of_graph_get_next_port(const struct device_node *parent,
+ struct device_node *port);
+struct device_node *of_graph_get_next_port_endpoint(const struct device_node *port,
+ struct device_node *prev);
struct device_node *of_graph_get_endpoint_by_regs(
const struct device_node *parent, int port_reg, int reg);
struct device_node *of_graph_get_remote_endpoint(
@@ -73,6 +103,11 @@ static inline unsigned int of_graph_get_endpoint_count(const struct device_node
return 0;
}
+static inline unsigned int of_graph_get_port_count(struct device_node *np)
+{
+ return 0;
+}
+
static inline struct device_node *of_graph_get_port_by_id(
struct device_node *node, u32 id)
{
@@ -86,6 +121,20 @@ static inline struct device_node *of_graph_get_next_endpoint(
return NULL;
}
+static inline struct device_node *of_graph_get_next_port(
+ const struct device_node *parent,
+ struct device_node *previous)
+{
+ return NULL;
+}
+
+static inline struct device_node *of_graph_get_next_port_endpoint(
+ const struct device_node *parent,
+ struct device_node *previous)
+{
+ return NULL;
+}
+
static inline struct device_node *of_graph_get_endpoint_by_regs(
const struct device_node *parent, int port_reg, int reg)
{
diff --git a/include/linux/panic.h b/include/linux/panic.h
index 54d90b6c5f47..f0b779d2915e 100644
--- a/include/linux/panic.h
+++ b/include/linux/panic.h
@@ -44,6 +44,12 @@ void abort(void);
extern atomic_t panic_cpu;
#define PANIC_CPU_INVALID -1
+bool panic_try_start(void);
+void panic_reset(void);
+bool panic_in_progress(void);
+bool panic_on_this_cpu(void);
+bool panic_on_other_cpu(void);
+
/*
* Only to be used by arch init code. If the user over-wrote the default
* CONFIG_PANIC_TIMEOUT, honor it.
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 24e586153f62..2e47edd0f515 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -409,6 +409,8 @@ struct pci_dev {
u16 l1ss; /* L1SS Capability pointer */
#ifdef CONFIG_PCIEASPM
struct pcie_link_state *link_state; /* ASPM link state */
+ unsigned int aspm_l0s_support:1; /* ASPM L0s support */
+ unsigned int aspm_l1_support:1; /* ASPM L1 support */
unsigned int ltr_path:1; /* Latency Tolerance Reporting
supported from root to here */
#endif
diff --git a/include/linux/printk.h b/include/linux/printk.h
index a6c6fd107805..c2ae39c85a20 100644
--- a/include/linux/printk.h
+++ b/include/linux/printk.h
@@ -312,8 +312,6 @@ static inline bool pr_flush(int timeout_ms, bool reset_on_progress)
#endif
-bool this_cpu_in_panic(void);
-
#ifdef CONFIG_SMP
extern int __printk_cpu_sync_try_get(void);
extern void __printk_cpu_sync_wait(void);
diff --git a/include/linux/ptdump.h b/include/linux/ptdump.h
index 8dbd51ea8626..1c1eb1fae199 100644
--- a/include/linux/ptdump.h
+++ b/include/linux/ptdump.h
@@ -11,9 +11,12 @@ struct ptdump_range {
};
struct ptdump_state {
- /* level is 0:PGD to 4:PTE, or -1 if unknown */
- void (*note_page)(struct ptdump_state *st, unsigned long addr,
- int level, u64 val);
+ void (*note_page_pte)(struct ptdump_state *st, unsigned long addr, pte_t pte);
+ void (*note_page_pmd)(struct ptdump_state *st, unsigned long addr, pmd_t pmd);
+ void (*note_page_pud)(struct ptdump_state *st, unsigned long addr, pud_t pud);
+ void (*note_page_p4d)(struct ptdump_state *st, unsigned long addr, p4d_t p4d);
+ void (*note_page_pgd)(struct ptdump_state *st, unsigned long addr, pgd_t pgd);
+ void (*note_page_flush)(struct ptdump_state *st);
void (*effective_prot)(struct ptdump_state *st, int level, u64 val);
const struct ptdump_range *range;
};
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index b4795698d8c2..de98462d5888 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -37,279 +37,13 @@
#include <linux/types.h>
#include <linux/mutex.h>
+#include <linux/spinlock.h>
#include <linux/virtio.h>
#include <linux/cdev.h>
#include <linux/completion.h>
#include <linux/idr.h>
#include <linux/of.h>
-
-/**
- * struct resource_table - firmware resource table header
- * @ver: version number
- * @num: number of resource entries
- * @reserved: reserved (must be zero)
- * @offset: array of offsets pointing at the various resource entries
- *
- * A resource table is essentially a list of system resources required
- * by the remote processor. It may also include configuration entries.
- * If needed, the remote processor firmware should contain this table
- * as a dedicated ".resource_table" ELF section.
- *
- * Some resources entries are mere announcements, where the host is informed
- * of specific remoteproc configuration. Other entries require the host to
- * do something (e.g. allocate a system resource). Sometimes a negotiation
- * is expected, where the firmware requests a resource, and once allocated,
- * the host should provide back its details (e.g. address of an allocated
- * memory region).
- *
- * The header of the resource table, as expressed by this structure,
- * contains a version number (should we need to change this format in the
- * future), the number of available resource entries, and their offsets
- * in the table.
- *
- * Immediately following this header are the resource entries themselves,
- * each of which begins with a resource entry header (as described below).
- */
-struct resource_table {
- u32 ver;
- u32 num;
- u32 reserved[2];
- u32 offset[];
-} __packed;
-
-/**
- * struct fw_rsc_hdr - firmware resource entry header
- * @type: resource type
- * @data: resource data
- *
- * Every resource entry begins with a 'struct fw_rsc_hdr' header providing
- * its @type. The content of the entry itself will immediately follow
- * this header, and it should be parsed according to the resource type.
- */
-struct fw_rsc_hdr {
- u32 type;
- u8 data[];
-} __packed;
-
-/**
- * enum fw_resource_type - types of resource entries
- *
- * @RSC_CARVEOUT: request for allocation of a physically contiguous
- * memory region.
- * @RSC_DEVMEM: request to iommu_map a memory-based peripheral.
- * @RSC_TRACE: announces the availability of a trace buffer into which
- * the remote processor will be writing logs.
- * @RSC_VDEV: declare support for a virtio device, and serve as its
- * virtio header.
- * @RSC_LAST: just keep this one at the end of standard resources
- * @RSC_VENDOR_START: start of the vendor specific resource types range
- * @RSC_VENDOR_END: end of the vendor specific resource types range
- *
- * For more details regarding a specific resource type, please see its
- * dedicated structure below.
- *
- * Please note that these values are used as indices to the rproc_handle_rsc
- * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to
- * check the validity of an index before the lookup table is accessed, so
- * please update it as needed.
- */
-enum fw_resource_type {
- RSC_CARVEOUT = 0,
- RSC_DEVMEM = 1,
- RSC_TRACE = 2,
- RSC_VDEV = 3,
- RSC_LAST = 4,
- RSC_VENDOR_START = 128,
- RSC_VENDOR_END = 512,
-};
-
-#define FW_RSC_ADDR_ANY (-1)
-
-/**
- * struct fw_rsc_carveout - physically contiguous memory request
- * @da: device address
- * @pa: physical address
- * @len: length (in bytes)
- * @flags: iommu protection flags
- * @reserved: reserved (must be zero)
- * @name: human-readable name of the requested memory region
- *
- * This resource entry requests the host to allocate a physically contiguous
- * memory region.
- *
- * These request entries should precede other firmware resource entries,
- * as other entries might request placing other data objects inside
- * these memory regions (e.g. data/code segments, trace resource entries, ...).
- *
- * Allocating memory this way helps utilizing the reserved physical memory
- * (e.g. CMA) more efficiently, and also minimizes the number of TLB entries
- * needed to map it (in case @rproc is using an IOMMU). Reducing the TLB
- * pressure is important; it may have a substantial impact on performance.
- *
- * If the firmware is compiled with static addresses, then @da should specify
- * the expected device address of this memory region. If @da is set to
- * FW_RSC_ADDR_ANY, then the host will dynamically allocate it, and then
- * overwrite @da with the dynamically allocated address.
- *
- * We will always use @da to negotiate the device addresses, even if it
- * isn't using an iommu. In that case, though, it will obviously contain
- * physical addresses.
- *
- * Some remote processors needs to know the allocated physical address
- * even if they do use an iommu. This is needed, e.g., if they control
- * hardware accelerators which access the physical memory directly (this
- * is the case with OMAP4 for instance). In that case, the host will
- * overwrite @pa with the dynamically allocated physical address.
- * Generally we don't want to expose physical addresses if we don't have to
- * (remote processors are generally _not_ trusted), so we might want to
- * change this to happen _only_ when explicitly required by the hardware.
- *
- * @flags is used to provide IOMMU protection flags, and @name should
- * (optionally) contain a human readable name of this carveout region
- * (mainly for debugging purposes).
- */
-struct fw_rsc_carveout {
- u32 da;
- u32 pa;
- u32 len;
- u32 flags;
- u32 reserved;
- u8 name[32];
-} __packed;
-
-/**
- * struct fw_rsc_devmem - iommu mapping request
- * @da: device address
- * @pa: physical address
- * @len: length (in bytes)
- * @flags: iommu protection flags
- * @reserved: reserved (must be zero)
- * @name: human-readable name of the requested region to be mapped
- *
- * This resource entry requests the host to iommu map a physically contiguous
- * memory region. This is needed in case the remote processor requires
- * access to certain memory-based peripherals; _never_ use it to access
- * regular memory.
- *
- * This is obviously only needed if the remote processor is accessing memory
- * via an iommu.
- *
- * @da should specify the required device address, @pa should specify
- * the physical address we want to map, @len should specify the size of
- * the mapping and @flags is the IOMMU protection flags. As always, @name may
- * (optionally) contain a human readable name of this mapping (mainly for
- * debugging purposes).
- *
- * Note: at this point we just "trust" those devmem entries to contain valid
- * physical addresses, but this isn't safe and will be changed: eventually we
- * want remoteproc implementations to provide us ranges of physical addresses
- * the firmware is allowed to request, and not allow firmwares to request
- * access to physical addresses that are outside those ranges.
- */
-struct fw_rsc_devmem {
- u32 da;
- u32 pa;
- u32 len;
- u32 flags;
- u32 reserved;
- u8 name[32];
-} __packed;
-
-/**
- * struct fw_rsc_trace - trace buffer declaration
- * @da: device address
- * @len: length (in bytes)
- * @reserved: reserved (must be zero)
- * @name: human-readable name of the trace buffer
- *
- * This resource entry provides the host information about a trace buffer
- * into which the remote processor will write log messages.
- *
- * @da specifies the device address of the buffer, @len specifies
- * its size, and @name may contain a human readable name of the trace buffer.
- *
- * After booting the remote processor, the trace buffers are exposed to the
- * user via debugfs entries (called trace0, trace1, etc..).
- */
-struct fw_rsc_trace {
- u32 da;
- u32 len;
- u32 reserved;
- u8 name[32];
-} __packed;
-
-/**
- * struct fw_rsc_vdev_vring - vring descriptor entry
- * @da: device address
- * @align: the alignment between the consumer and producer parts of the vring
- * @num: num of buffers supported by this vring (must be power of two)
- * @notifyid: a unique rproc-wide notify index for this vring. This notify
- * index is used when kicking a remote processor, to let it know that this
- * vring is triggered.
- * @pa: physical address
- *
- * This descriptor is not a resource entry by itself; it is part of the
- * vdev resource type (see below).
- *
- * Note that @da should either contain the device address where
- * the remote processor is expecting the vring, or indicate that
- * dynamically allocation of the vring's device address is supported.
- */
-struct fw_rsc_vdev_vring {
- u32 da;
- u32 align;
- u32 num;
- u32 notifyid;
- u32 pa;
-} __packed;
-
-/**
- * struct fw_rsc_vdev - virtio device header
- * @id: virtio device id (as in virtio_ids.h)
- * @notifyid: a unique rproc-wide notify index for this vdev. This notify
- * index is used when kicking a remote processor, to let it know that the
- * status/features of this vdev have changes.
- * @dfeatures: specifies the virtio device features supported by the firmware
- * @gfeatures: a place holder used by the host to write back the
- * negotiated features that are supported by both sides.
- * @config_len: the size of the virtio config space of this vdev. The config
- * space lies in the resource table immediate after this vdev header.
- * @status: a place holder where the host will indicate its virtio progress.
- * @num_of_vrings: indicates how many vrings are described in this vdev header
- * @reserved: reserved (must be zero)
- * @vring: an array of @num_of_vrings entries of 'struct fw_rsc_vdev_vring'.
- *
- * This resource is a virtio device header: it provides information about
- * the vdev, and is then used by the host and its peer remote processors
- * to negotiate and share certain virtio properties.
- *
- * By providing this resource entry, the firmware essentially asks remoteproc
- * to statically allocate a vdev upon registration of the rproc (dynamic vdev
- * allocation is not yet supported).
- *
- * Note:
- * 1. unlike virtualization systems, the term 'host' here means
- * the Linux side which is running remoteproc to control the remote
- * processors. We use the name 'gfeatures' to comply with virtio's terms,
- * though there isn't really any virtualized guest OS here: it's the host
- * which is responsible for negotiating the final features.
- * Yeah, it's a bit confusing.
- *
- * 2. immediately following this structure is the virtio config space for
- * this vdev (which is specific to the vdev; for more info, read the virtio
- * spec). The size of the config space is specified by @config_len.
- */
-struct fw_rsc_vdev {
- u32 id;
- u32 notifyid;
- u32 dfeatures;
- u32 gfeatures;
- u32 config_len;
- u8 status;
- u8 num_of_vrings;
- u8 reserved[2];
- struct fw_rsc_vdev_vring vring[];
-} __packed;
+#include <linux/rsc_table.h>
struct rproc;
@@ -412,7 +146,6 @@ struct rproc_ops {
* a message.
* @RPROC_RUNNING: device is up and running
* @RPROC_CRASHED: device has crashed; need to start recovery
- * @RPROC_DELETED: device is deleted
* @RPROC_ATTACHED: device has been booted by another entity and the core
* has attached to it
* @RPROC_DETACHED: device has been booted by another entity and waiting
@@ -430,10 +163,9 @@ enum rproc_state {
RPROC_SUSPENDED = 1,
RPROC_RUNNING = 2,
RPROC_CRASHED = 3,
- RPROC_DELETED = 4,
- RPROC_ATTACHED = 5,
- RPROC_DETACHED = 6,
- RPROC_LAST = 7,
+ RPROC_ATTACHED = 4,
+ RPROC_DETACHED = 5,
+ RPROC_LAST = 6,
};
/**
@@ -526,7 +258,10 @@ enum rproc_features {
* @subdevs: list of subdevices, to following the running state
* @notifyids: idr for dynamically assigning rproc-wide unique notify ids
* @index: index of this rproc device
+ * @attach_work: workqueue for attaching rproc
* @crash_handler: workqueue for handling a crash
+ * @crash_handler_lock: serializes crash handler queueing and deletion
+ * @deleting: remoteproc deletion has begun
* @crash_cnt: crash counter
* @recovery_disabled: flag that state if recovery was disabled
* @max_notifyid: largest allocated notify id.
@@ -568,7 +303,10 @@ struct rproc {
struct list_head subdevs;
struct idr notifyids;
int index;
+ struct work_struct attach_work;
struct work_struct crash_handler;
+ spinlock_t crash_handler_lock;
+ bool deleting;
unsigned int crash_cnt;
bool recovery_disabled;
int max_notifyid;
diff --git a/include/linux/rsc_table.h b/include/linux/rsc_table.h
new file mode 100644
index 000000000000..4cef11a2e3a2
--- /dev/null
+++ b/include/linux/rsc_table.h
@@ -0,0 +1,364 @@
+/*
+ * Resource table and its types data structure
+ *
+ * Copyright(c) 2011 Texas Instruments, Inc.
+ * Copyright(c) 2011 Google, Inc.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name Texas Instruments nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef RSC_TABLE_H
+#define RSC_TABLE_H
+
+/**
+ * struct resource_table - firmware resource table header
+ * @ver: version number
+ * @num: number of resource entries
+ * @reserved: reserved (must be zero)
+ * @offset: array of offsets pointing at the various resource entries
+ *
+ * A resource table is essentially a list of system resources required
+ * by the remote processor. It may also include configuration entries.
+ * If needed, the remote processor firmware should contain this table
+ * as a dedicated ".resource_table" ELF section.
+ *
+ * Some resources entries are mere announcements, where the host is informed
+ * of specific remoteproc configuration. Other entries require the host to
+ * do something (e.g. allocate a system resource). Sometimes a negotiation
+ * is expected, where the firmware requests a resource, and once allocated,
+ * the host should provide back its details (e.g. address of an allocated
+ * memory region).
+ *
+ * The header of the resource table, as expressed by this structure,
+ * contains a version number (should we need to change this format in the
+ * future), the number of available resource entries, and their offsets
+ * in the table.
+ *
+ * Immediately following this header are the resource entries themselves,
+ * each of which begins with a resource entry header (as described below).
+ */
+struct resource_table {
+ u32 ver;
+ u32 num;
+ u32 reserved[2];
+ u32 offset[];
+} __packed;
+
+/**
+ * struct fw_rsc_hdr - firmware resource entry header
+ * @type: resource type
+ * @data: resource data
+ *
+ * Every resource entry begins with a 'struct fw_rsc_hdr' header providing
+ * its @type. The content of the entry itself will immediately follow
+ * this header, and it should be parsed according to the resource type.
+ */
+struct fw_rsc_hdr {
+ u32 type;
+ u8 data[];
+} __packed;
+
+/**
+ * enum fw_resource_type - types of resource entries
+ *
+ * @RSC_CARVEOUT: request for allocation of a physically contiguous
+ * memory region.
+ * @RSC_DEVMEM: request to iommu_map a memory-based peripheral.
+ * @RSC_TRACE: announces the availability of a trace buffer into which
+ * the remote processor will be writing logs.
+ * @RSC_VDEV: declare support for a virtio device, and serve as its
+ * virtio header.
+ * @RSC_LAST: just keep this one at the end of standard resources
+ * @RSC_VENDOR_START: start of the vendor specific resource types range
+ * @RSC_VENDOR_END: end of the vendor specific resource types range
+ *
+ * For more details regarding a specific resource type, please see its
+ * dedicated structure below.
+ *
+ * Please note that these values are used as indices to the rproc_handle_rsc
+ * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to
+ * check the validity of an index before the lookup table is accessed, so
+ * please update it as needed.
+ */
+enum fw_resource_type {
+ RSC_CARVEOUT = 0,
+ RSC_DEVMEM = 1,
+ RSC_TRACE = 2,
+ RSC_VDEV = 3,
+ RSC_LAST = 4,
+ RSC_VENDOR_START = 128,
+ RSC_VENDOR_END = 512,
+};
+
+#define FW_RSC_ADDR_ANY (-1)
+
+/**
+ * struct fw_rsc_carveout - physically contiguous memory request
+ * @da: device address
+ * @pa: physical address
+ * @len: length (in bytes)
+ * @flags: iommu protection flags
+ * @reserved: reserved (must be zero)
+ * @name: human-readable name of the requested memory region
+ *
+ * This resource entry requests the host to allocate a physically contiguous
+ * memory region.
+ *
+ * These request entries should precede other firmware resource entries,
+ * as other entries might request placing other data objects inside
+ * these memory regions (e.g. data/code segments, trace resource entries, ...).
+ *
+ * Allocating memory this way helps utilizing the reserved physical memory
+ * (e.g. CMA) more efficiently, and also minimizes the number of TLB entries
+ * needed to map it (in case @rproc is using an IOMMU). Reducing the TLB
+ * pressure is important; it may have a substantial impact on performance.
+ *
+ * If the firmware is compiled with static addresses, then @da should specify
+ * the expected device address of this memory region. If @da is set to
+ * FW_RSC_ADDR_ANY, then the host will dynamically allocate it, and then
+ * overwrite @da with the dynamically allocated address.
+ *
+ * We will always use @da to negotiate the device addresses, even if it
+ * isn't using an iommu. In that case, though, it will obviously contain
+ * physical addresses.
+ *
+ * Some remote processors needs to know the allocated physical address
+ * even if they do use an iommu. This is needed, e.g., if they control
+ * hardware accelerators which access the physical memory directly (this
+ * is the case with OMAP4 for instance). In that case, the host will
+ * overwrite @pa with the dynamically allocated physical address.
+ * Generally we don't want to expose physical addresses if we don't have to
+ * (remote processors are generally _not_ trusted), so we might want to
+ * change this to happen _only_ when explicitly required by the hardware.
+ *
+ * @flags is used to provide IOMMU protection flags, and @name should
+ * (optionally) contain a human readable name of this carveout region
+ * (mainly for debugging purposes).
+ */
+struct fw_rsc_carveout {
+ u32 da;
+ u32 pa;
+ u32 len;
+ u32 flags;
+ u32 reserved;
+ u8 name[32];
+} __packed;
+
+/**
+ * struct fw_rsc_devmem - iommu mapping request
+ * @da: device address
+ * @pa: physical address
+ * @len: length (in bytes)
+ * @flags: iommu protection flags
+ * @reserved: reserved (must be zero)
+ * @name: human-readable name of the requested region to be mapped
+ *
+ * This resource entry requests the host to iommu map a physically contiguous
+ * memory region. This is needed in case the remote processor requires
+ * access to certain memory-based peripherals; _never_ use it to access
+ * regular memory.
+ *
+ * This is obviously only needed if the remote processor is accessing memory
+ * via an iommu.
+ *
+ * @da should specify the required device address, @pa should specify
+ * the physical address we want to map, @len should specify the size of
+ * the mapping and @flags is the IOMMU protection flags. As always, @name may
+ * (optionally) contain a human readable name of this mapping (mainly for
+ * debugging purposes).
+ *
+ * Note: at this point we just "trust" those devmem entries to contain valid
+ * physical addresses, but this isn't safe and will be changed: eventually we
+ * want remoteproc implementations to provide us ranges of physical addresses
+ * the firmware is allowed to request, and not allow firmwares to request
+ * access to physical addresses that are outside those ranges.
+ */
+struct fw_rsc_devmem {
+ u32 da;
+ u32 pa;
+ u32 len;
+ u32 flags;
+ u32 reserved;
+ u8 name[32];
+} __packed;
+
+/**
+ * struct fw_rsc_trace - trace buffer declaration
+ * @da: device address
+ * @len: length (in bytes)
+ * @reserved: reserved (must be zero)
+ * @name: human-readable name of the trace buffer
+ *
+ * This resource entry provides the host information about a trace buffer
+ * into which the remote processor will write log messages.
+ *
+ * @da specifies the device address of the buffer, @len specifies
+ * its size, and @name may contain a human readable name of the trace buffer.
+ *
+ * After booting the remote processor, the trace buffers are exposed to the
+ * user via debugfs entries (called trace0, trace1, etc..).
+ */
+struct fw_rsc_trace {
+ u32 da;
+ u32 len;
+ u32 reserved;
+ u8 name[32];
+} __packed;
+
+/**
+ * struct fw_rsc_vdev_vring - vring descriptor entry
+ * @da: device address
+ * @align: the alignment between the consumer and producer parts of the vring
+ * @num: num of buffers supported by this vring (must be power of two)
+ * @notifyid: a unique rproc-wide notify index for this vring. This notify
+ * index is used when kicking a remote processor, to let it know that this
+ * vring is triggered.
+ * @pa: physical address
+ *
+ * This descriptor is not a resource entry by itself; it is part of the
+ * vdev resource type (see below).
+ *
+ * Note that @da should either contain the device address where
+ * the remote processor is expecting the vring, or indicate that
+ * dynamically allocation of the vring's device address is supported.
+ */
+struct fw_rsc_vdev_vring {
+ u32 da;
+ u32 align;
+ u32 num;
+ u32 notifyid;
+ u32 pa;
+} __packed;
+
+/**
+ * struct fw_rsc_vdev - virtio device header
+ * @id: virtio device id (as in virtio_ids.h)
+ * @notifyid: a unique rproc-wide notify index for this vdev. This notify
+ * index is used when kicking a remote processor, to let it know that the
+ * status/features of this vdev have changes.
+ * @dfeatures: specifies the virtio device features supported by the firmware
+ * @gfeatures: a place holder used by the host to write back the
+ * negotiated features that are supported by both sides.
+ * @config_len: the size of the virtio config space of this vdev. The config
+ * space lies in the resource table immediate after this vdev header.
+ * @status: a place holder where the host will indicate its virtio progress.
+ * @num_of_vrings: indicates how many vrings are described in this vdev header
+ * @reserved: reserved (must be zero)
+ * @vring: an array of @num_of_vrings entries of 'struct fw_rsc_vdev_vring'.
+ *
+ * This resource is a virtio device header: it provides information about
+ * the vdev, and is then used by the host and its peer remote processors
+ * to negotiate and share certain virtio properties.
+ *
+ * By providing this resource entry, the firmware essentially asks remoteproc
+ * to statically allocate a vdev upon registration of the rproc (dynamic vdev
+ * allocation is not yet supported).
+ *
+ * Note:
+ * 1. unlike virtualization systems, the term 'host' here means
+ * the Linux side which is running remoteproc to control the remote
+ * processors. We use the name 'gfeatures' to comply with virtio's terms,
+ * though there isn't really any virtualized guest OS here: it's the host
+ * which is responsible for negotiating the final features.
+ * Yeah, it's a bit confusing.
+ *
+ * 2. immediately following this structure is the virtio config space for
+ * this vdev (which is specific to the vdev; for more info, read the virtio
+ * spec). The size of the config space is specified by @config_len.
+ */
+struct fw_rsc_vdev {
+ u32 id;
+ u32 notifyid;
+ u32 dfeatures;
+ u32 gfeatures;
+ u32 config_len;
+ u8 status;
+ u8 num_of_vrings;
+ u8 reserved[2];
+ struct fw_rsc_vdev_vring vring[];
+} __packed;
+
+/**
+ * rsc_table_for_each_entry() - iterate over all entries in a resource table
+ * @table: pointer to the resource table
+ * @table_sz: total size of the table buffer in bytes
+ * @dev: device used for error logging
+ * @cb: callback invoked for each entry:
+ * @type - value from enum fw_resource_type
+ * @rsc - pointer to the entry payload (past struct fw_rsc_hdr)
+ * @offset - byte offset of the payload within the table; callers
+ * that write back into the table (e.g. to record a
+ * dynamically allocated address) use this to locate the
+ * entry for later update
+ * @avail - bytes available in the payload
+ * @data - caller-supplied private pointer
+ * Return 0 to continue iteration, non-zero to stop.
+ * @data: private pointer forwarded to @cb on every call
+ *
+ * Iterates over every resource entry in @table, performing the standard
+ * truncation check, and invokes @cb for each one. Iteration stops on the
+ * first non-zero return from @cb or on a malformed table.
+ *
+ * Returns 0 after a complete iteration, -EINVAL if the table is truncated,
+ * or the first non-zero value returned by @cb.
+ */
+static inline int rsc_table_for_each_entry(struct resource_table *table,
+ size_t table_sz,
+ struct device *dev,
+ int (*cb)(u32 type, void *rsc,
+ int offset, int avail,
+ void *data),
+ void *data) {
+ int i, ret;
+
+ for (i = 0; i < table->num; i++) {
+ u32 offset = table->offset[i];
+ struct fw_rsc_hdr *hdr;
+ int avail, rsc_offset;
+ void *rsc;
+
+ if (offset < sizeof(*table) || offset >= table_sz ||
+ table_sz - offset < sizeof(*hdr)) {
+ dev_err(dev, "rsc table is truncated\n");
+ return -EINVAL;
+ }
+
+ hdr = (void *)table + offset;
+ avail = table_sz - offset - sizeof(*hdr);
+ rsc_offset = offset + sizeof(*hdr);
+ rsc = (void *)hdr + sizeof(*hdr);
+
+ ret = cb(hdr->type, rsc, rsc_offset, avail, data);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+#endif /* RSC_TABLE_H */
diff --git a/include/linux/rtsx_usb.h b/include/linux/rtsx_usb.h
index 3247ed8e9ff0..5933751cf721 100644
--- a/include/linux/rtsx_usb.h
+++ b/include/linux/rtsx_usb.h
@@ -57,6 +57,9 @@ struct rtsx_ucr {
struct timer_list sg_timer;
struct mutex dev_mutex;
+
+ u16 card_status_cache;
+ bool card_status_valid;
};
/* buffer size */
diff --git a/include/linux/secure_boot.h b/include/linux/secure_boot.h
new file mode 100644
index 000000000000..d17e92351567
--- /dev/null
+++ b/include/linux/secure_boot.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ *
+ * Author: Coiby Xu <coxu@redhat.com>
+ */
+
+#ifndef _LINUX_SECURE_BOOT_H
+#define _LINUX_SECURE_BOOT_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_HAVE_ARCH_GET_SECUREBOOT
+/*
+ * Returns true if the platform secure boot is enabled.
+ * Returns false if disabled or not supported.
+ */
+bool arch_get_secureboot(void);
+#else
+static inline bool arch_get_secureboot(void) { return false; }
+#endif
+
+#endif /* _LINUX_SECURE_BOOT_H */
diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h
index 0a3cbbe4a4cd..6bc486214c53 100644
--- a/include/linux/sunrpc/xdr.h
+++ b/include/linux/sunrpc/xdr.h
@@ -695,6 +695,27 @@ xdr_stream_decode_u32(struct xdr_stream *xdr, __u32 *ptr)
return 0;
}
+/**
+ * xdr_stream_decode_be32 - Decode a big-endian 32-bit integer
+ * @xdr: pointer to xdr_stream
+ * @ptr: location to store integer
+ *
+ * Return values:
+ * %0 on success
+ * %-EBADMSG on XDR buffer overflow
+ */
+static inline ssize_t
+xdr_stream_decode_be32(struct xdr_stream *xdr, __be32 *ptr)
+{
+ const size_t count = sizeof(*ptr);
+ __be32 *p = xdr_inline_decode(xdr, count);
+
+ if (unlikely(!p))
+ return -EBADMSG;
+ *ptr = *p;
+ return 0;
+}
+
/**
* xdr_stream_decode_u64 - Decode a 64-bit integer
* @xdr: pointer to xdr_stream
diff --git a/include/linux/usb.h b/include/linux/usb.h
index 049b012638be..0012259ac910 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -1132,8 +1132,8 @@ static inline int usb_make_path(struct usb_device *dev, char *buf, size_t size)
/* ----------------------------------------------------------------------- */
/* Stuff for dynamic usb ids */
+extern struct mutex usb_dynids_lock;
struct usb_dynids {
- spinlock_t lock;
struct list_head list;
};
@@ -1159,7 +1159,8 @@ extern ssize_t usb_show_dynids(struct usb_dynids *dynids, char *buf);
* interface. It may also use usb_set_interface() to specify the
* appropriate altsetting. If unwilling to manage the interface,
* return -ENODEV, if genuine IO errors occurred, an appropriate
- * negative errno value.
+ * negative errno value. The usb_device_id parameter is only valid during
+ * probe.
* @disconnect: Called when the interface is no longer accessible, usually
* because its device has been (or is being) disconnected or the
* driver module is being unloaded.
@@ -1246,7 +1247,7 @@ struct usb_driver {
unsigned int disable_hub_initiated_lpm:1;
unsigned int soft_unbind:1;
};
-#define to_usb_driver(d) container_of(d, struct usb_driver, driver)
+#define to_usb_driver(d) container_of_const(d, struct usb_driver, driver)
/**
* struct usb_device_driver - identifies USB device driver to usbcore
diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h
index 02a9f4dc594d..d2cab1056964 100644
--- a/include/linux/virtio_net.h
+++ b/include/linux/virtio_net.h
@@ -6,6 +6,7 @@
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/udp.h>
+#include <net/tcp.h>
#include <uapi/linux/tcp.h>
#include <uapi/linux/virtio_net.h>
@@ -178,6 +179,9 @@ static inline int virtio_net_hdr_to_skb(struct sk_buff *skb,
if (skb->ip_summed == CHECKSUM_PARTIAL &&
skb->csum_offset != offsetof(struct tcphdr, check))
return -EINVAL;
+
+ BUILD_BUG_ON(TCP_MIN_GSO_SIZE * GSO_MAX_SEGS < GSO_MAX_SIZE);
+ gso_size = max(gso_size, TCP_MIN_GSO_SIZE);
break;
}
diff --git a/include/media/rc-map.h b/include/media/rc-map.h
index 4867eb2f931e..ab8c1b82dff6 100644
--- a/include/media/rc-map.h
+++ b/include/media/rc-map.h
@@ -262,7 +262,6 @@ struct rc_map *rc_map_get(const char *name);
#define RC_MAP_GENIUS_TVGO_A11MCE "rc-genius-tvgo-a11mce"
#define RC_MAP_GOTVIEW7135 "rc-gotview7135"
#define RC_MAP_HAUPPAUGE "rc-hauppauge"
-#define RC_MAP_HAUPPAUGE_NEW "rc-hauppauge"
#define RC_MAP_HISI_POPLAR "rc-hisi-poplar"
#define RC_MAP_HISI_TV_DEMO "rc-hisi-tv-demo"
#define RC_MAP_IMON_MCE "rc-imon-mce"
@@ -309,7 +308,6 @@ struct rc_map *rc_map_get(const char *name);
#define RC_MAP_PROTEUS_2309 "rc-proteus-2309"
#define RC_MAP_PURPLETV "rc-purpletv"
#define RC_MAP_PV951 "rc-pv951"
-#define RC_MAP_RC5_TV "rc-rc5-tv"
#define RC_MAP_RC6_MCE "rc-rc6-mce"
#define RC_MAP_REAL_AUDIO_220_32_KEYS "rc-real-audio-220-32-keys"
#define RC_MAP_REDDO "rc-reddo"
diff --git a/include/net/bluetooth/hci_core.h b/include/net/bluetooth/hci_core.h
index 34c4db7bee8c..4ed681b2dafe 100644
--- a/include/net/bluetooth/hci_core.h
+++ b/include/net/bluetooth/hci_core.h
@@ -892,9 +892,9 @@ static inline void hci_discovery_filter_clear(struct hci_dev *hdev)
hdev->discovery.result_filtering = false;
hdev->discovery.report_invalid_rssi = true;
hdev->discovery.rssi = HCI_RSSI_INVALID;
- hdev->discovery.uuid_count = 0;
spin_lock(&hdev->discovery.lock);
+ hdev->discovery.uuid_count = 0;
kfree(hdev->discovery.uuids);
hdev->discovery.uuids = NULL;
spin_unlock(&hdev->discovery.lock);
diff --git a/include/net/bluetooth/l2cap.h b/include/net/bluetooth/l2cap.h
index 8f0df86ee684..f5f91373022f 100644
--- a/include/net/bluetooth/l2cap.h
+++ b/include/net/bluetooth/l2cap.h
@@ -702,7 +702,12 @@ struct l2cap_rx_busy {
struct l2cap_pinfo {
struct bt_sock bt;
+
+ /* With owning sk_socket chan may be read without lock, other access
+ * should hold lock_sock.
+ */
struct l2cap_chan *chan;
+
struct list_head rx_busy;
};
diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
index bcc138ff087b..41354036b0c8 100644
--- a/include/net/inet_connection_sock.h
+++ b/include/net/inet_connection_sock.h
@@ -42,7 +42,9 @@ struct inet_connection_sock_af_ops {
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req);
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk));
u16 net_header_len;
u16 sockaddr_len;
int (*setsockopt)(struct sock *sk, int level, int optname,
@@ -296,14 +298,8 @@ reqsk_timeout(struct request_sock *req, unsigned long max_timeout)
return (unsigned long)min_t(u64, timeout, max_timeout);
}
-static inline void inet_csk_prepare_for_destroy_sock(struct sock *sk)
-{
- /* The below has to be done to allow calling inet_csk_destroy_sock */
- sock_set_flag(sk, SOCK_DEAD);
- this_cpu_inc(*sk->sk_prot->orphan_count);
-}
-
void inet_csk_destroy_sock(struct sock *sk);
+void inet_csk_prepare_for_destroy_sock(struct sock *sk);
void inet_csk_prepare_forced_close(struct sock *sk);
/*
diff --git a/include/net/inetpeer.h b/include/net/inetpeer.h
index f475757daafb..414e9adf4c51 100644
--- a/include/net/inetpeer.h
+++ b/include/net/inetpeer.h
@@ -35,6 +35,7 @@ struct inetpeer_addr {
struct inet_peer {
struct rb_node rb_node;
+ u64 hash;
struct inetpeer_addr daddr;
u32 metrics[RTAX_MAX];
@@ -125,6 +126,9 @@ static inline int inetpeer_addr_cmp(const struct inetpeer_addr *a,
{
int i, n;
+ if (a->family != b->family)
+ return a->family < b->family ? -1 : 1;
+
if (a->family == AF_INET)
n = sizeof(a->a4) / sizeof(u32);
else
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index 8a75c73fc555..f3649ae8cab6 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -12,6 +12,7 @@
#define DEFAULT_TX_QUEUE_LEN 1000
#define STAB_SIZE_LOG_MAX 30
+#define QDISC_PKT_LEN_MAX (1 << 20) /* 1 MiB */
struct qdisc_walker {
int stop;
diff --git a/include/net/tc_act/tc_ct.h b/include/net/tc_act/tc_ct.h
index 77f87c622a2e..c18e7cf22d03 100644
--- a/include/net/tc_act/tc_ct.h
+++ b/include/net/tc_act/tc_ct.h
@@ -13,7 +13,7 @@ struct tcf_ct_params {
struct nf_conntrack_helper *helper;
struct nf_conn *tmpl;
u16 zone;
-
+ int action;
u32 mark;
u32 mark_mask;
diff --git a/include/net/tc_act/tc_ctinfo.h b/include/net/tc_act/tc_ctinfo.h
index a04bcac7adf4..7fe01ab236da 100644
--- a/include/net/tc_act/tc_ctinfo.h
+++ b/include/net/tc_act/tc_ctinfo.h
@@ -7,6 +7,7 @@
struct tcf_ctinfo_params {
struct rcu_head rcu;
struct net *net;
+ int action;
u32 dscpmask;
u32 dscpstatemask;
u32 cpmarkmask;
diff --git a/include/net/tc_act/tc_skbedit.h b/include/net/tc_act/tc_skbedit.h
index 9649600fb3dc..31b2cd0bebb5 100644
--- a/include/net/tc_act/tc_skbedit.h
+++ b/include/net/tc_act/tc_skbedit.h
@@ -12,6 +12,7 @@
#include <linux/tc_act/tc_skbedit.h>
struct tcf_skbedit_params {
+ int action;
u32 flags;
u32 priority;
u32 mark;
diff --git a/include/net/tc_act/tc_skbmod.h b/include/net/tc_act/tc_skbmod.h
index 7c240d2fed4e..626704cd6241 100644
--- a/include/net/tc_act/tc_skbmod.h
+++ b/include/net/tc_act/tc_skbmod.h
@@ -12,6 +12,7 @@
struct tcf_skbmod_params {
struct rcu_head rcu;
u64 flags; /*up to 64 types of operations; extend if needed */
+ int action;
u8 eth_dst[ETH_ALEN];
u16 eth_type;
u8 eth_src[ETH_ALEN];
diff --git a/include/net/tc_act/tc_tunnel_key.h b/include/net/tc_act/tc_tunnel_key.h
index 879fe8cff581..0f1925f97520 100644
--- a/include/net/tc_act/tc_tunnel_key.h
+++ b/include/net/tc_act/tc_tunnel_key.h
@@ -14,6 +14,7 @@
struct tcf_tunnel_key_params {
struct rcu_head rcu;
int tcft_action;
+ int action;
struct metadata_dst *tcft_enc_metadata;
};
diff --git a/include/net/tc_act/tc_vlan.h b/include/net/tc_act/tc_vlan.h
index 904eddfc1826..2e39df2e717b 100644
--- a/include/net/tc_act/tc_vlan.h
+++ b/include/net/tc_act/tc_vlan.h
@@ -10,6 +10,7 @@
#include <linux/tc_act/tc_vlan.h>
struct tcf_vlan_params {
+ int action;
int tcfv_action;
unsigned char tcfv_push_dst[ETH_ALEN];
unsigned char tcfv_push_src[ETH_ALEN];
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 1cc25e0b3bde..8ebaeed6217b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -464,7 +464,9 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req);
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk));
int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb);
int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len);
int tcp_connect(struct sock *sk);
@@ -2702,6 +2704,11 @@ static inline int tcp_call_bpf_3arg(struct sock *sk, int op, u32 arg1, u32 arg2,
return tcp_call_bpf(sk, op, 3, args);
}
+static inline void tcp_clear_sock_ops_cb_flags(struct sock *sk)
+{
+ tcp_sk(sk)->bpf_sock_ops_cb_flags = 0;
+}
+
#else
static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args)
{
@@ -2719,6 +2726,10 @@ static inline int tcp_call_bpf_3arg(struct sock *sk, int op, u32 arg1, u32 arg2,
return -EPERM;
}
+static inline void tcp_clear_sock_ops_cb_flags(struct sock *sk)
+{
+}
+
#endif
static inline u32 tcp_timeout_init(struct sock *sk)
diff --git a/include/net/xsk_buff_pool.h b/include/net/xsk_buff_pool.h
index e21062cf6229..53da7697d009 100644
--- a/include/net/xsk_buff_pool.h
+++ b/include/net/xsk_buff_pool.h
@@ -28,7 +28,6 @@ struct xdp_buff_xsk {
dma_addr_t dma;
dma_addr_t frame_dma;
struct xsk_buff_pool *pool;
- u64 orig_addr;
struct list_head list_node;
};
@@ -121,7 +120,6 @@ void xp_free(struct xdp_buff_xsk *xskb);
static inline void xp_init_xskb_addr(struct xdp_buff_xsk *xskb, struct xsk_buff_pool *pool,
u64 addr)
{
- xskb->orig_addr = addr;
xskb->xdp.data_hard_start = pool->addrs + addr + pool->headroom;
}
@@ -223,14 +221,19 @@ static inline void xp_release(struct xdp_buff_xsk *xskb)
xskb->pool->free_heads[xskb->pool->free_heads_cnt++] = xskb;
}
-static inline u64 xp_get_handle(struct xdp_buff_xsk *xskb)
+static inline u64 xp_get_handle(struct xdp_buff_xsk *xskb,
+ struct xsk_buff_pool *pool)
{
- u64 offset = xskb->xdp.data - xskb->xdp.data_hard_start;
+ u64 orig_addr = xskb->xdp.data - pool->addrs;
+ u64 offset;
- offset += xskb->pool->headroom;
- if (!xskb->pool->unaligned)
- return xskb->orig_addr + offset;
- return xskb->orig_addr + (offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT);
+ if (!pool->unaligned)
+ return orig_addr;
+
+ offset = xskb->xdp.data - xskb->xdp.data_hard_start;
+ offset += pool->headroom;
+ orig_addr -= offset;
+ return orig_addr + (offset << XSK_UNALIGNED_BUF_OFFSET_SHIFT);
}
static inline bool xp_tx_metadata_enabled(const struct xsk_buff_pool *pool)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index a3d203ba238d..6c8f560e8637 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1831,6 +1831,11 @@ struct ib_dm {
struct ib_mr {
struct ib_device *device;
+ /*
+ * Due to IB_MR_REREG_PD pd is not a fixed pointer and can change. For a
+ * user MR, this value should only be read from a system call that holds
+ * the uobject lock, or the driver should disable in-place REREG_PD.
+ */
struct ib_pd *pd;
u32 lkey;
u32 rkey;
@@ -2469,6 +2474,15 @@ struct ib_device_ops {
int (*modify_cq)(struct ib_cq *cq, u16 cq_count, u16 cq_period);
int (*destroy_cq)(struct ib_cq *cq, struct ib_udata *udata);
int (*resize_cq)(struct ib_cq *cq, int cqe, struct ib_udata *udata);
+ /**
+ * pre_destroy_cq - Prevent a cq from generating any new work
+ * completions, but not free any kernel resources
+ */
+ int (*pre_destroy_cq)(struct ib_cq *cq);
+ /**
+ * post_destroy_cq - Free all kernel resources
+ */
+ void (*post_destroy_cq)(struct ib_cq *cq);
struct ib_mr *(*get_dma_mr)(struct ib_pd *pd, int mr_access_flags);
struct ib_mr *(*reg_user_mr)(struct ib_pd *pd, u64 start, u64 length,
u64 virt_addr, int mr_access_flags,
diff --git a/include/sound/core.h b/include/sound/core.h
index 1f3f5dccd736..0fcaa1cda166 100644
--- a/include/sound/core.h
+++ b/include/sound/core.h
@@ -314,6 +314,8 @@ static inline void snd_card_unref(struct snd_card *card)
put_device(&card->card_dev);
}
+DEFINE_FREE(snd_card_unref, struct snd_card *, if (_T) snd_card_unref(_T))
+
#define snd_card_set_dev(card, devptr) ((card)->dev = (devptr))
/* device.c */
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index c8fb6abfeca9..1d6dc90b2345 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1779,6 +1779,13 @@ union bpf_attr {
};
__u64 expected_revision;
} netkit;
+ struct {
+ union {
+ __u32 relative_fd;
+ __u32 relative_id;
+ };
+ __u64 expected_revision;
+ } cgroup;
};
} link_create;
diff --git a/include/uapi/linux/landlock.h b/include/uapi/linux/landlock.h
index c223572f8229..94cfad88082f 100644
--- a/include/uapi/linux/landlock.h
+++ b/include/uapi/linux/landlock.h
@@ -177,6 +177,7 @@ struct landlock_net_port_attr {
* device.
* - %LANDLOCK_ACCESS_FS_MAKE_DIR: Create (or rename) a directory.
* - %LANDLOCK_ACCESS_FS_MAKE_REG: Create (or rename or link) a regular file.
+ * This also guards the creation of whiteout objects as used in OverlayFS.
* - %LANDLOCK_ACCESS_FS_MAKE_SOCK: Create (or rename or link) a UNIX domain
* socket.
* - %LANDLOCK_ACCESS_FS_MAKE_FIFO: Create (or rename or link) a named pipe.
diff --git a/include/uapi/linux/raid/md_p.h b/include/uapi/linux/raid/md_p.h
index ff47b6f0ba0f..ac74133a4768 100644
--- a/include/uapi/linux/raid/md_p.h
+++ b/include/uapi/linux/raid/md_p.h
@@ -173,7 +173,7 @@ typedef struct mdp_superblock_s {
#else
#error unspecified endianness
#endif
- __u32 recovery_cp; /* 11 recovery checkpoint sector count */
+ __u32 recovery_cp; /* 11 resync checkpoint sector count */
/* There are only valid for minor_version > 90 */
__u64 reshape_position; /* 12,13 next address in array-space for reshape */
__u32 new_level; /* 14 new level we are reshaping to */
diff --git a/include/uapi/linux/time_types.h b/include/uapi/linux/time_types.h
index bcc0002115d3..03a0d8aaadca 100644
--- a/include/uapi/linux/time_types.h
+++ b/include/uapi/linux/time_types.h
@@ -30,7 +30,7 @@ struct __kernel_old_timeval {
struct __kernel_old_timespec {
__kernel_old_time_t tv_sec; /* seconds */
- long tv_nsec; /* nanoseconds */
+ __kernel_long_t tv_nsec; /* nanoseconds */
};
struct __kernel_old_itimerval {
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 187e4871b74b..c62c6f261ca0 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -523,6 +523,8 @@ static void arena_free_pages(struct bpf_arena *arena, long uaddr, long page_cnt)
uaddr = (u32)uaddr;
uaddr &= PAGE_MASK;
full_uaddr = clear_lo32(arena->user_vm_start) + uaddr;
+ if (full_uaddr < arena->user_vm_start)
+ return;
uaddr_end = min(arena->user_vm_end, full_uaddr + (page_cnt << PAGE_SHIFT));
if (full_uaddr >= uaddr_end)
return;
diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c
index 969a6ca362ec..140c63a816d0 100644
--- a/kernel/bpf/bpf_lsm.c
+++ b/kernel/bpf/bpf_lsm.c
@@ -293,7 +293,6 @@ BTF_ID(func, bpf_lsm_bpf_map_create)
BTF_ID(func, bpf_lsm_bpf_map_free)
BTF_ID(func, bpf_lsm_bpf_prog)
BTF_ID(func, bpf_lsm_bpf_prog_load)
-BTF_ID(func, bpf_lsm_bpf_prog_free)
BTF_ID(func, bpf_lsm_bpf_token_create)
BTF_ID(func, bpf_lsm_bpf_token_free)
BTF_ID(func, bpf_lsm_bpf_token_cmd)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 65006b4b84ca..fbf3e1718bfc 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -3565,7 +3565,7 @@ static int btf_repeat_fields(struct btf_field_info *info, int info_cnt,
static int btf_find_struct_field(const struct btf *btf,
const struct btf_type *t, u32 field_mask,
struct btf_field_info *info, int info_cnt,
- u32 level);
+ u32 level, u32 *seen_mask);
/* Find special fields in the struct type of a field.
*
@@ -3576,7 +3576,7 @@ static int btf_find_struct_field(const struct btf *btf,
static int btf_find_nested_struct(const struct btf *btf, const struct btf_type *t,
u32 off, u32 nelems,
u32 field_mask, struct btf_field_info *info,
- int info_cnt, u32 level)
+ int info_cnt, u32 level, u32 *seen_mask)
{
int ret, err, i;
@@ -3584,7 +3584,7 @@ static int btf_find_nested_struct(const struct btf *btf, const struct btf_type *
if (level >= MAX_RESOLVE_DEPTH)
return -E2BIG;
- ret = btf_find_struct_field(btf, t, field_mask, info, info_cnt, level);
+ ret = btf_find_struct_field(btf, t, field_mask, info, info_cnt, level, seen_mask);
if (ret <= 0)
return ret;
@@ -3641,7 +3641,7 @@ static int btf_find_field_one(const struct btf *btf,
if (expected_size && expected_size != sz * nelems)
return 0;
ret = btf_find_nested_struct(btf, var_type, off, nelems, field_mask,
- &info[0], info_cnt, level);
+ &info[0], info_cnt, level, seen_mask);
return ret;
}
@@ -3703,11 +3703,11 @@ static int btf_find_field_one(const struct btf *btf,
static int btf_find_struct_field(const struct btf *btf,
const struct btf_type *t, u32 field_mask,
struct btf_field_info *info, int info_cnt,
- u32 level)
+ u32 level, u32 *seen_mask)
{
int ret, idx = 0;
const struct btf_member *member;
- u32 i, off, seen_mask = 0;
+ u32 i, off;
for_each_member(i, t, member) {
const struct btf_type *member_type = btf_type_by_id(btf,
@@ -3721,7 +3721,7 @@ static int btf_find_struct_field(const struct btf *btf,
ret = btf_find_field_one(btf, t, member_type, i,
off, 0,
- field_mask, &seen_mask,
+ field_mask, seen_mask,
&info[idx], info_cnt - idx, level);
if (ret < 0)
return ret;
@@ -3732,11 +3732,11 @@ static int btf_find_struct_field(const struct btf *btf,
static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t,
u32 field_mask, struct btf_field_info *info,
- int info_cnt, u32 level)
+ int info_cnt, u32 level, u32 *seen_mask)
{
int ret, idx = 0;
const struct btf_var_secinfo *vsi;
- u32 i, off, seen_mask = 0;
+ u32 i, off;
for_each_vsi(i, t, vsi) {
const struct btf_type *var = btf_type_by_id(btf, vsi->type);
@@ -3744,7 +3744,7 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t,
off = vsi->offset;
ret = btf_find_field_one(btf, var, var_type, -1, off, vsi->size,
- field_mask, &seen_mask,
+ field_mask, seen_mask,
&info[idx], info_cnt - idx,
level);
if (ret < 0)
@@ -3758,10 +3758,12 @@ static int btf_find_field(const struct btf *btf, const struct btf_type *t,
u32 field_mask, struct btf_field_info *info,
int info_cnt)
{
+ u32 seen_mask = 0;
+
if (__btf_type_is_struct(t))
- return btf_find_struct_field(btf, t, field_mask, info, info_cnt, 0);
+ return btf_find_struct_field(btf, t, field_mask, info, info_cnt, 0, &seen_mask);
else if (btf_type_is_datasec(t))
- return btf_find_datasec_var(btf, t, field_mask, info, info_cnt, 0);
+ return btf_find_datasec_var(btf, t, field_mask, info, info_cnt, 0, &seen_mask);
return -EINVAL;
}
diff --git a/kernel/bpf/cgroup.c b/kernel/bpf/cgroup.c
index ad054a4c782e..b30ea11d93db 100644
--- a/kernel/bpf/cgroup.c
+++ b/kernel/bpf/cgroup.c
@@ -624,6 +624,116 @@ static struct bpf_prog_list *find_attach_entry(struct hlist_head *progs,
return NULL;
}
+static struct bpf_link *bpf_get_anchor_link(u32 flags, u32 id_or_fd)
+{
+ struct bpf_link *link = ERR_PTR(-EINVAL);
+
+ if (flags & BPF_F_ID)
+ link = bpf_link_by_id(id_or_fd);
+ else if (id_or_fd)
+ link = bpf_link_get_from_fd(id_or_fd);
+ return link;
+}
+
+static struct bpf_prog *bpf_get_anchor_prog(u32 flags, u32 id_or_fd)
+{
+ struct bpf_prog *prog = ERR_PTR(-EINVAL);
+
+ if (flags & BPF_F_ID)
+ prog = bpf_prog_by_id(id_or_fd);
+ else if (id_or_fd)
+ prog = bpf_prog_get(id_or_fd);
+ return prog;
+}
+
+static struct bpf_prog_list *get_prog_list(struct hlist_head *progs, struct bpf_prog *prog,
+ struct bpf_cgroup_link *link, u32 flags, u32 id_or_fd)
+{
+ bool is_link = flags & BPF_F_LINK, is_id = flags & BPF_F_ID;
+ struct bpf_prog_list *pltmp, *pl = ERR_PTR(-EINVAL);
+ bool preorder = flags & BPF_F_PREORDER;
+ struct bpf_link *anchor_link = NULL;
+ struct bpf_prog *anchor_prog = NULL;
+ bool is_before, is_after;
+
+ is_before = flags & BPF_F_BEFORE;
+ is_after = flags & BPF_F_AFTER;
+ if (is_link || is_id || id_or_fd) {
+ /* flags must have either BPF_F_BEFORE or BPF_F_AFTER */
+ if (is_before == is_after)
+ return ERR_PTR(-EINVAL);
+ if ((is_link && !link) || (!is_link && !prog))
+ return ERR_PTR(-EINVAL);
+ } else if (!hlist_empty(progs)) {
+ /* flags cannot have both BPF_F_BEFORE and BPF_F_AFTER */
+ if (is_before && is_after)
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (is_link) {
+ anchor_link = bpf_get_anchor_link(flags, id_or_fd);
+ if (IS_ERR(anchor_link))
+ return ERR_PTR(PTR_ERR(anchor_link));
+ } else if (is_id || id_or_fd) {
+ anchor_prog = bpf_get_anchor_prog(flags, id_or_fd);
+ if (IS_ERR(anchor_prog))
+ return ERR_PTR(PTR_ERR(anchor_prog));
+ }
+
+ if (!anchor_prog && !anchor_link) {
+ /* if there is no anchor_prog/anchor_link, then BPF_F_PREORDER
+ * doesn't matter since either prepend or append to a combined
+ * list of progs will end up with correct result.
+ */
+ hlist_for_each_entry(pltmp, progs, node) {
+ if (is_before)
+ return pltmp;
+ if (pltmp->node.next)
+ continue;
+ return pltmp;
+ }
+ return NULL;
+ }
+
+ hlist_for_each_entry(pltmp, progs, node) {
+ if ((anchor_prog && anchor_prog == pltmp->prog) ||
+ (anchor_link && anchor_link == &pltmp->link->link)) {
+ if (!!(pltmp->flags & BPF_F_PREORDER) != preorder)
+ goto out;
+ pl = pltmp;
+ goto out;
+ }
+ }
+
+ pl = ERR_PTR(-ENOENT);
+out:
+ if (anchor_link)
+ bpf_link_put(anchor_link);
+ else
+ bpf_prog_put(anchor_prog);
+ return pl;
+}
+
+static int insert_pl_to_hlist(struct bpf_prog_list *pl, struct hlist_head *progs,
+ struct bpf_prog *prog, struct bpf_cgroup_link *link,
+ u32 flags, u32 id_or_fd)
+{
+ struct bpf_prog_list *pltmp;
+
+ pltmp = get_prog_list(progs, prog, link, flags, id_or_fd);
+ if (IS_ERR(pltmp))
+ return PTR_ERR(pltmp);
+
+ if (!pltmp)
+ hlist_add_head(&pl->node, progs);
+ else if (flags & BPF_F_BEFORE)
+ hlist_add_before(&pl->node, &pltmp->node);
+ else
+ hlist_add_behind(&pl->node, &pltmp->node);
+
+ return 0;
+}
+
/**
* __cgroup_bpf_attach() - Attach the program or the link to a cgroup, and
* propagate the change to descendants
@@ -633,6 +743,8 @@ static struct bpf_prog_list *find_attach_entry(struct hlist_head *progs,
* @replace_prog: Previously attached program to replace if BPF_F_REPLACE is set
* @type: Type of attach operation
* @flags: Option flags
+ * @id_or_fd: Relative prog id or fd
+ * @revision: bpf_prog_list revision
*
* Exactly one of @prog or @link can be non-null.
* Must be called with cgroup_mutex held.
@@ -640,14 +752,17 @@ static struct bpf_prog_list *find_attach_entry(struct hlist_head *progs,
static int __cgroup_bpf_attach(struct cgroup *cgrp,
struct bpf_prog *prog, struct bpf_prog *replace_prog,
struct bpf_cgroup_link *link,
- enum bpf_attach_type type, u32 flags)
+ enum bpf_attach_type type, u32 flags, u32 id_or_fd,
+ u64 revision)
{
u32 saved_flags = (flags & (BPF_F_ALLOW_OVERRIDE | BPF_F_ALLOW_MULTI));
struct bpf_prog *old_prog = NULL;
struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE] = {};
struct bpf_cgroup_storage *new_storage[MAX_BPF_CGROUP_STORAGE_TYPE] = {};
+ struct bpf_cgroup_storage *old_storage[MAX_BPF_CGROUP_STORAGE_TYPE] = {};
struct bpf_prog *new_prog = prog ? : link->link.prog;
enum cgroup_bpf_attach_type atype;
+ u32 old_flags, old_pl_flags;
struct bpf_prog_list *pl;
struct hlist_head *progs;
int err;
@@ -656,6 +771,9 @@ static int __cgroup_bpf_attach(struct cgroup *cgrp,
((flags & BPF_F_REPLACE) && !(flags & BPF_F_ALLOW_MULTI)))
/* invalid combination */
return -EINVAL;
+ if ((flags & BPF_F_REPLACE) && (flags & (BPF_F_BEFORE | BPF_F_AFTER)))
+ /* only either replace or insertion with before/after */
+ return -EINVAL;
if (link && (prog || replace_prog))
/* only either link or prog/replace_prog can be specified */
return -EINVAL;
@@ -666,6 +784,8 @@ static int __cgroup_bpf_attach(struct cgroup *cgrp,
atype = bpf_cgroup_atype_find(type, new_prog->aux->attach_btf_id);
if (atype < 0)
return -EINVAL;
+ if (revision && revision != cgrp->bpf.revisions[atype])
+ return -ESTALE;
progs = &cgrp->bpf.progs[atype];
@@ -693,29 +813,28 @@ static int __cgroup_bpf_attach(struct cgroup *cgrp,
if (pl) {
old_prog = pl->prog;
+ old_pl_flags = pl->flags;
+ bpf_cgroup_storages_assign(old_storage, pl->storage);
} else {
- struct hlist_node *last = NULL;
-
pl = kmalloc(sizeof(*pl), GFP_KERNEL);
if (!pl) {
bpf_cgroup_storages_free(new_storage);
return -ENOMEM;
}
- if (hlist_empty(progs))
- hlist_add_head(&pl->node, progs);
- else
- hlist_for_each(last, progs) {
- if (last->next)
- continue;
- hlist_add_behind(&pl->node, last);
- break;
- }
+
+ err = insert_pl_to_hlist(pl, progs, prog, link, flags, id_or_fd);
+ if (err) {
+ kfree(pl);
+ bpf_cgroup_storages_free(new_storage);
+ return err;
+ }
}
pl->prog = prog;
pl->link = link;
pl->flags = flags;
bpf_cgroup_storages_assign(pl->storage, storage);
+ old_flags = cgrp->bpf.flags[atype];
cgrp->bpf.flags[atype] = saved_flags;
if (type == BPF_LSM_CGROUP) {
@@ -728,6 +847,7 @@ static int __cgroup_bpf_attach(struct cgroup *cgrp,
if (err)
goto cleanup_trampoline;
+ cgrp->bpf.revisions[atype] += 1;
if (old_prog) {
if (type == BPF_LSM_CGROUP)
bpf_trampoline_unlink_cgroup_shim(old_prog);
@@ -746,12 +866,15 @@ static int __cgroup_bpf_attach(struct cgroup *cgrp,
if (old_prog) {
pl->prog = old_prog;
pl->link = NULL;
+ pl->flags = old_pl_flags;
+ bpf_cgroup_storages_assign(pl->storage, old_storage);
}
bpf_cgroup_storages_free(new_storage);
if (!old_prog) {
hlist_del(&pl->node);
kfree(pl);
}
+ cgrp->bpf.flags[atype] = old_flags;
return err;
}
@@ -759,12 +882,13 @@ static int cgroup_bpf_attach(struct cgroup *cgrp,
struct bpf_prog *prog, struct bpf_prog *replace_prog,
struct bpf_cgroup_link *link,
enum bpf_attach_type type,
- u32 flags)
+ u32 flags, u32 id_or_fd, u64 revision)
{
int ret;
cgroup_lock();
- ret = __cgroup_bpf_attach(cgrp, prog, replace_prog, link, type, flags);
+ ret = __cgroup_bpf_attach(cgrp, prog, replace_prog, link, type, flags,
+ id_or_fd, revision);
cgroup_unlock();
return ret;
}
@@ -848,6 +972,20 @@ static void replace_effective_prog(struct cgroup *cgrp,
}
}
+static bool cgroup_bpf_storages_compatible(struct bpf_prog *old_prog,
+ struct bpf_prog *new_prog)
+{
+ enum bpf_cgroup_storage_type stype;
+
+ for_each_cgroup_storage_type(stype) {
+ if (old_prog->aux->cgroup_storage[stype] !=
+ new_prog->aux->cgroup_storage[stype])
+ return false;
+ }
+
+ return true;
+}
+
/**
* __cgroup_bpf_replace() - Replace link's program and propagate the change
* to descendants
@@ -886,6 +1024,10 @@ static int __cgroup_bpf_replace(struct cgroup *cgrp,
if (!found)
return -ENOENT;
+ if (!cgroup_bpf_storages_compatible(link->link.prog, new_prog))
+ return -EINVAL;
+
+ cgrp->bpf.revisions[atype] += 1;
old_prog = xchg(&link->link.prog, new_prog);
replace_effective_prog(cgrp, atype, pl);
bpf_prog_put(old_prog);
@@ -993,12 +1135,14 @@ static void purge_effective_progs(struct cgroup *cgrp, struct bpf_prog_list *pl,
* @prog: A program to detach or NULL
* @link: A link to detach or NULL
* @type: Type of detach operation
+ * @revision: bpf_prog_list revision
*
* At most one of @prog or @link can be non-NULL.
* Must be called with cgroup_mutex held.
*/
static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
- struct bpf_cgroup_link *link, enum bpf_attach_type type)
+ struct bpf_cgroup_link *link, enum bpf_attach_type type,
+ u64 revision)
{
enum cgroup_bpf_attach_type atype;
struct bpf_prog *old_prog;
@@ -1016,6 +1160,9 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
if (atype < 0)
return -EINVAL;
+ if (revision && revision != cgrp->bpf.revisions[atype])
+ return -ESTALE;
+
progs = &cgrp->bpf.progs[atype];
flags = cgrp->bpf.flags[atype];
@@ -1041,6 +1188,7 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
/* now can actually delete it from this cgroup list */
hlist_del(&pl->node);
+ cgrp->bpf.revisions[atype] += 1;
kfree(pl);
if (hlist_empty(progs))
@@ -1056,19 +1204,19 @@ static int __cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
}
static int cgroup_bpf_detach(struct cgroup *cgrp, struct bpf_prog *prog,
- enum bpf_attach_type type)
+ enum bpf_attach_type type, u64 revision)
{
int ret;
cgroup_lock();
- ret = __cgroup_bpf_detach(cgrp, prog, NULL, type);
+ ret = __cgroup_bpf_detach(cgrp, prog, NULL, type, revision);
cgroup_unlock();
return ret;
}
/* Must be called with cgroup_mutex held to avoid races. */
static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
- union bpf_attr __user *uattr)
+ union bpf_attr __user *uattr, u32 uattr_size)
{
__u32 __user *prog_attach_flags = u64_to_user_ptr(attr->query.prog_attach_flags);
bool effective_query = attr->query.query_flags & BPF_F_QUERY_EFFECTIVE;
@@ -1079,6 +1227,7 @@ static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
struct bpf_prog_array *effective;
int cnt, ret = 0, i;
int total_cnt = 0;
+ u64 revision = 0;
u32 flags;
if (effective_query && prog_attach_flags)
@@ -1116,6 +1265,11 @@ static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
return -EFAULT;
if (copy_to_user(&uattr->query.prog_cnt, &total_cnt, sizeof(total_cnt)))
return -EFAULT;
+ if (!effective_query && from_atype == to_atype)
+ revision = cgrp->bpf.revisions[from_atype];
+ if (uattr_size >= offsetofend(union bpf_attr, query.revision) &&
+ copy_to_user(&uattr->query.revision, &revision, sizeof(revision)))
+ return -EFAULT;
if (attr->query.prog_cnt == 0 || !prog_ids || !total_cnt)
/* return early if user requested only program count + flags */
return 0;
@@ -1167,12 +1321,12 @@ static int __cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
}
static int cgroup_bpf_query(struct cgroup *cgrp, const union bpf_attr *attr,
- union bpf_attr __user *uattr)
+ union bpf_attr __user *uattr, u32 uattr_size)
{
int ret;
cgroup_lock();
- ret = __cgroup_bpf_query(cgrp, attr, uattr);
+ ret = __cgroup_bpf_query(cgrp, attr, uattr, uattr_size);
cgroup_unlock();
return ret;
}
@@ -1198,7 +1352,8 @@ int cgroup_bpf_prog_attach(const union bpf_attr *attr,
}
ret = cgroup_bpf_attach(cgrp, prog, replace_prog, NULL,
- attr->attach_type, attr->attach_flags);
+ attr->attach_type, attr->attach_flags,
+ attr->relative_fd, attr->expected_revision);
if (replace_prog)
bpf_prog_put(replace_prog);
@@ -1220,7 +1375,7 @@ int cgroup_bpf_prog_detach(const union bpf_attr *attr, enum bpf_prog_type ptype)
if (IS_ERR(prog))
prog = NULL;
- ret = cgroup_bpf_detach(cgrp, prog, attr->attach_type);
+ ret = cgroup_bpf_detach(cgrp, prog, attr->attach_type, attr->expected_revision);
if (prog)
bpf_prog_put(prog);
@@ -1249,7 +1404,7 @@ static void bpf_cgroup_link_release(struct bpf_link *link)
}
WARN_ON(__cgroup_bpf_detach(cg_link->cgroup, NULL, cg_link,
- cg_link->type));
+ cg_link->type, 0));
if (cg_link->type == BPF_LSM_CGROUP)
bpf_trampoline_unlink_cgroup_shim(cg_link->link.prog);
@@ -1321,6 +1476,13 @@ static const struct bpf_link_ops bpf_cgroup_link_lops = {
.fill_link_info = bpf_cgroup_link_fill_link_info,
};
+#define BPF_F_LINK_ATTACH_MASK \
+ (BPF_F_ID | \
+ BPF_F_BEFORE | \
+ BPF_F_AFTER | \
+ BPF_F_PREORDER | \
+ BPF_F_LINK)
+
int cgroup_bpf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
{
struct bpf_link_primer link_primer;
@@ -1328,7 +1490,7 @@ int cgroup_bpf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
struct cgroup *cgrp;
int err;
- if (attr->link_create.flags)
+ if (attr->link_create.flags & (~BPF_F_LINK_ATTACH_MASK))
return -EINVAL;
cgrp = cgroup_get_from_fd(attr->link_create.target_fd);
@@ -1352,7 +1514,9 @@ int cgroup_bpf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
}
err = cgroup_bpf_attach(cgrp, NULL, NULL, link,
- link->type, BPF_F_ALLOW_MULTI);
+ link->type, BPF_F_ALLOW_MULTI | attr->link_create.flags,
+ attr->link_create.cgroup.relative_fd,
+ attr->link_create.cgroup.expected_revision);
if (err) {
bpf_link_cleanup(&link_primer);
goto out_put_cgroup;
@@ -1366,7 +1530,7 @@ int cgroup_bpf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog)
}
int cgroup_bpf_prog_query(const union bpf_attr *attr,
- union bpf_attr __user *uattr)
+ union bpf_attr __user *uattr, u32 uattr_size)
{
struct cgroup *cgrp;
int ret;
@@ -1375,7 +1539,7 @@ int cgroup_bpf_prog_query(const union bpf_attr *attr,
if (IS_ERR(cgrp))
return PTR_ERR(cgrp);
- ret = cgroup_bpf_query(cgrp, attr, uattr);
+ ret = cgroup_bpf_query(cgrp, attr, uattr, uattr_size);
cgroup_put(cgrp);
return ret;
@@ -2051,7 +2215,7 @@ int __cgroup_bpf_run_filter_getsockopt_kern(struct sock *sk, int level,
if (ret < 0)
return ret;
- if (ctx.optlen > *optlen)
+ if (ctx.optlen > *optlen || ctx.optlen < 0)
return -EFAULT;
/* BPF programs can shrink the buffer, export the modifications.
diff --git a/kernel/bpf/net_namespace.c b/kernel/bpf/net_namespace.c
index 868cc2c43899..a1d14bcb67c5 100644
--- a/kernel/bpf/net_namespace.c
+++ b/kernel/bpf/net_namespace.c
@@ -172,33 +172,28 @@ static int bpf_netns_link_update_prog(struct bpf_link *link,
struct net *net;
int idx, ret;
+ guard(mutex)(&netns_bpf_mutex);
+
if (old_prog && old_prog != link->prog)
return -EPERM;
if (new_prog->type != link->prog->type)
return -EINVAL;
- mutex_lock(&netns_bpf_mutex);
-
net = net_link->net;
- if (!net || !check_net(net)) {
+ if (!net || !check_net(net))
/* Link auto-detached or netns dying */
- ret = -ENOLINK;
- goto out_unlock;
- }
+ return -ENOLINK;
run_array = rcu_dereference_protected(net->bpf.run_array[type],
lockdep_is_held(&netns_bpf_mutex));
idx = link_index(net, type, net_link);
ret = bpf_prog_array_update_at(run_array, idx, new_prog);
if (ret)
- goto out_unlock;
+ return ret;
old_prog = xchg(&link->prog, new_prog);
bpf_prog_put(old_prog);
-
-out_unlock:
- mutex_unlock(&netns_bpf_mutex);
- return ret;
+ return 0;
}
static int bpf_netns_link_fill_info(const struct bpf_link *link,
diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
index 1f2c50480902..9e6037d88bda 100644
--- a/kernel/bpf/ringbuf.c
+++ b/kernel/bpf/ringbuf.c
@@ -429,7 +429,7 @@ static void *__bpf_ringbuf_reserve(struct bpf_ringbuf *rb, u64 size)
prod_pos = rb->producer_pos;
new_prod_pos = prod_pos + len;
- while (pend_pos < prod_pos) {
+ while (prod_pos - pend_pos > 0) {
hdr = (void *)rb->data + (pend_pos & rb->mask);
hdr_len = READ_ONCE(hdr->len);
if (hdr_len & BPF_RINGBUF_BUSY_BIT)
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 3bdfac9b4418..954b299f241b 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3091,9 +3091,10 @@ static const char *bpf_link_type_strs[] = {
static void bpf_link_show_fdinfo(struct seq_file *m, struct file *filp)
{
const struct bpf_link *link = filp->private_data;
- const struct bpf_prog *prog = link->prog;
+ const struct bpf_prog *prog;
enum bpf_link_type type = link->type;
char prog_tag[sizeof(prog->tag) * 2 + 1] = { };
+ u32 prog_id = 0;
if (type < ARRAY_SIZE(bpf_link_type_strs) && bpf_link_type_strs[type]) {
seq_printf(m, "link_type:\t%s\n", bpf_link_type_strs[type]);
@@ -3103,13 +3104,20 @@ static void bpf_link_show_fdinfo(struct seq_file *m, struct file *filp)
}
seq_printf(m, "link_id:\t%u\n", link->id);
+ rcu_read_lock();
+ prog = READ_ONCE(link->prog);
if (prog) {
bin2hex(prog_tag, prog->tag, sizeof(prog->tag));
+ prog_id = prog->aux->id;
+ }
+ rcu_read_unlock();
+
+ if (prog) {
seq_printf(m,
"prog_tag:\t%s\n"
"prog_id:\t%u\n",
prog_tag,
- prog->aux->id);
+ prog_id);
}
if (link->ops->show_fdinfo)
link->ops->show_fdinfo(link, m);
@@ -4047,6 +4055,25 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,
}
}
+static bool is_cgroup_prog_type(enum bpf_prog_type ptype, enum bpf_attach_type atype,
+ bool check_atype)
+{
+ switch (ptype) {
+ case BPF_PROG_TYPE_CGROUP_DEVICE:
+ case BPF_PROG_TYPE_CGROUP_SKB:
+ case BPF_PROG_TYPE_CGROUP_SOCK:
+ case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
+ case BPF_PROG_TYPE_CGROUP_SOCKOPT:
+ case BPF_PROG_TYPE_CGROUP_SYSCTL:
+ case BPF_PROG_TYPE_SOCK_OPS:
+ return true;
+ case BPF_PROG_TYPE_LSM:
+ return check_atype ? atype == BPF_LSM_CGROUP : true;
+ default:
+ return false;
+ }
+}
+
#define BPF_PROG_ATTACH_LAST_FIELD expected_revision
#define BPF_F_ATTACH_MASK_BASE \
@@ -4077,6 +4104,9 @@ static int bpf_prog_attach(const union bpf_attr *attr)
if (bpf_mprog_supported(ptype)) {
if (attr->attach_flags & ~BPF_F_ATTACH_MASK_MPROG)
return -EINVAL;
+ } else if (is_cgroup_prog_type(ptype, 0, false)) {
+ if (attr->attach_flags & ~(BPF_F_ATTACH_MASK_BASE | BPF_F_ATTACH_MASK_MPROG))
+ return -EINVAL;
} else {
if (attr->attach_flags & ~BPF_F_ATTACH_MASK_BASE)
return -EINVAL;
@@ -4094,6 +4124,11 @@ static int bpf_prog_attach(const union bpf_attr *attr)
return -EINVAL;
}
+ if (is_cgroup_prog_type(ptype, prog->expected_attach_type, true)) {
+ ret = cgroup_bpf_prog_attach(attr, ptype, prog);
+ goto out;
+ }
+
switch (ptype) {
case BPF_PROG_TYPE_SK_SKB:
case BPF_PROG_TYPE_SK_MSG:
@@ -4105,20 +4140,6 @@ static int bpf_prog_attach(const union bpf_attr *attr)
case BPF_PROG_TYPE_FLOW_DISSECTOR:
ret = netns_bpf_prog_attach(attr, prog);
break;
- case BPF_PROG_TYPE_CGROUP_DEVICE:
- case BPF_PROG_TYPE_CGROUP_SKB:
- case BPF_PROG_TYPE_CGROUP_SOCK:
- case BPF_PROG_TYPE_CGROUP_SOCK_ADDR:
- case BPF_PROG_TYPE_CGROUP_SOCKOPT:
- case BPF_PROG_TYPE_CGROUP_SYSCTL:
- case BPF_PROG_TYPE_SOCK_OPS:
- case BPF_PROG_TYPE_LSM:
- if (ptype == BPF_PROG_TYPE_LSM &&
- prog->expected_attach_type != BPF_LSM_CGROUP)
- ret = -EINVAL;
- else
- ret = cgroup_bpf_prog_attach(attr, ptype, prog);
- break;
case BPF_PROG_TYPE_SCHED_CLS:
if (attr->attach_type == BPF_TCX_INGRESS ||
attr->attach_type == BPF_TCX_EGRESS)
@@ -4129,7 +4150,7 @@ static int bpf_prog_attach(const union bpf_attr *attr)
default:
ret = -EINVAL;
}
-
+out:
if (ret)
bpf_prog_put(prog);
return ret;
@@ -4157,6 +4178,9 @@ static int bpf_prog_detach(const union bpf_attr *attr)
if (IS_ERR(prog))
return PTR_ERR(prog);
}
+ } else if (is_cgroup_prog_type(ptype, 0, false)) {
+ if (attr->attach_flags || attr->relative_fd)
+ return -EINVAL;
} else if (attr->attach_flags ||
attr->relative_fd ||
attr->expected_revision) {
@@ -4203,7 +4227,7 @@ static int bpf_prog_detach(const union bpf_attr *attr)
#define BPF_PROG_QUERY_LAST_FIELD query.revision
static int bpf_prog_query(const union bpf_attr *attr,
- union bpf_attr __user *uattr)
+ union bpf_attr __user *uattr, u32 uattr_size)
{
if (!bpf_net_capable())
return -EPERM;
@@ -4242,7 +4266,7 @@ static int bpf_prog_query(const union bpf_attr *attr,
case BPF_CGROUP_GETSOCKOPT:
case BPF_CGROUP_SETSOCKOPT:
case BPF_LSM_CGROUP:
- return cgroup_bpf_prog_query(attr, uattr);
+ return cgroup_bpf_prog_query(attr, uattr, uattr_size);
case BPF_LIRC_MODE2:
return lirc_prog_query(attr, uattr);
case BPF_FLOW_DISSECTOR:
@@ -4917,6 +4941,7 @@ static int bpf_link_get_info_by_fd(struct file *file,
{
struct bpf_link_info __user *uinfo = u64_to_user_ptr(attr->info.info);
struct bpf_link_info info;
+ const struct bpf_prog *prog;
u32 info_len = attr->info.info_len;
int err;
@@ -4931,8 +4956,12 @@ static int bpf_link_get_info_by_fd(struct file *file,
info.type = link->type;
info.id = link->id;
- if (link->prog)
- info.prog_id = link->prog->aux->id;
+
+ rcu_read_lock();
+ prog = READ_ONCE(link->prog);
+ if (prog)
+ info.prog_id = prog->aux->id;
+ rcu_read_unlock();
if (link->ops->fill_link_info) {
err = link->ops->fill_link_info(link, &info);
@@ -5691,7 +5720,7 @@ static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)
err = bpf_prog_detach(&attr);
break;
case BPF_PROG_QUERY:
- err = bpf_prog_query(&attr, uattr.user);
+ err = bpf_prog_query(&attr, uattr.user, size);
break;
case BPF_PROG_TEST_RUN:
err = bpf_prog_test_run(&attr, uattr.user);
diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c
index fc5f463ca529..f3a7a96a77d7 100644
--- a/kernel/bpf/task_iter.c
+++ b/kernel/bpf/task_iter.c
@@ -760,6 +760,7 @@ BPF_CALL_5(bpf_find_vma, struct task_struct *, task, u64, start,
struct mmap_unlock_irq_work *work = NULL;
struct vm_area_struct *vma;
bool irq_work_busy = false;
+ bool __maybe_unused mmput_needed = false;
struct mm_struct *mm;
int ret = -ENOENT;
@@ -769,14 +770,38 @@ BPF_CALL_5(bpf_find_vma, struct task_struct *, task, u64, start,
if (!task)
return -ENOENT;
- mm = task->mm;
+ if (task == current) {
+ mm = task->mm;
+ } else {
+ /*
+ * Foreign task: pin task->mm against a concurrent exit_mm().
+ * Use trylock on alloc_lock instead of get_task_mm()'s
+ * blocking task_lock() to avoid deadlocking the target task.
+ */
+ if (!IS_ENABLED(CONFIG_MMU))
+ return -EOPNOTSUPP;
+ if (irqs_disabled())
+ return -EBUSY;
+ if (!spin_trylock(&task->alloc_lock))
+ return -EBUSY;
+ mm = task->mm;
+ if (mm && !(task->flags & PF_KTHREAD)) {
+ mmget(mm);
+ mmput_needed = true;
+ } else {
+ mm = NULL;
+ }
+ spin_unlock(&task->alloc_lock);
+ }
if (!mm)
return -ENOENT;
irq_work_busy = bpf_mmap_unlock_get_irq_work(&work);
- if (irq_work_busy || !mmap_read_trylock(mm))
- return -EBUSY;
+ if (irq_work_busy || !mmap_read_trylock(mm)) {
+ ret = -EBUSY;
+ goto out;
+ }
vma = find_vma(mm, start);
@@ -786,6 +811,11 @@ BPF_CALL_5(bpf_find_vma, struct task_struct *, task, u64, start,
ret = 0;
}
bpf_mmap_unlock_mm(work, mm);
+out:
+#ifdef CONFIG_MMU
+ if (mmput_needed)
+ mmput_async(mm);
+#endif
return ret;
}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 99b3f539e431..f9f2095eab4a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -22781,13 +22781,25 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)
struct btf *bpf_get_btf_vmlinux(void)
{
- if (!btf_vmlinux && IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) {
+ /* Pairs with the smp_store_release() on the parse path below. */
+ struct btf *btf = smp_load_acquire(&btf_vmlinux);
+
+ if (!btf && IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) {
mutex_lock(&bpf_verifier_lock);
- if (!btf_vmlinux)
- btf_vmlinux = btf_parse_vmlinux();
+ btf = btf_vmlinux;
+ if (!btf) {
+ btf = btf_parse_vmlinux();
+ /*
+ * Order the parsed BTF contents and the globals the
+ * parse populated (e.g. bpf_ctx_convert.t) before
+ * the pointer publication. Pairs with the acquire
+ * on the lockless fast path above.
+ */
+ smp_store_release(&btf_vmlinux, btf);
+ }
mutex_unlock(&bpf_verifier_lock);
}
- return btf_vmlinux;
+ return btf;
}
int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u32 uattr_size)
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 3c7d466f13df..ce012aca272a 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -2081,6 +2081,11 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp)
for_each_subsys(ss, ssid)
INIT_LIST_HEAD(&cgrp->e_csets[ssid]);
+#ifdef CONFIG_CGROUP_BPF
+ for (int i = 0; i < ARRAY_SIZE(cgrp->bpf.revisions); i++)
+ cgrp->bpf.revisions[i] = 1;
+#endif
+
init_waitqueue_head(&cgrp->offline_waitq);
INIT_WORK(&cgrp->release_agent_work, cgroup1_release_agent);
}
diff --git a/kernel/cgroup/cpuset-internal.h b/kernel/cgroup/cpuset-internal.h
index 383963e28ac6..6c9916e85efd 100644
--- a/kernel/cgroup/cpuset-internal.h
+++ b/kernel/cgroup/cpuset-internal.h
@@ -166,7 +166,7 @@ struct cpuset {
* number of SCHED_DEADLINE tasks attached to this cpuset, so that we
* know when to rebuild associated root domain bandwidth information.
*/
- int nr_deadline_tasks;
+ atomic_t nr_deadline_tasks;
int nr_migrate_dl_tasks;
u64 sum_migrate_dl_bw;
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 4bc1ea983fe7..ba023c19f502 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -88,7 +88,6 @@ static struct list_head remote_children;
* A flag to force sched domain rebuild at the end of an operation.
* It can be set in
* - update_partition_sd_lb()
- * - remote_partition_check()
* - update_cpumasks_hier()
* - cpuset_update_flag()
* - cpuset_hotplug_update_tasks()
@@ -145,14 +144,14 @@ void inc_dl_tasks_cs(struct task_struct *p)
{
struct cpuset *cs = task_cs(p);
- cs->nr_deadline_tasks++;
+ atomic_inc(&cs->nr_deadline_tasks);
}
void dec_dl_tasks_cs(struct task_struct *p)
{
struct cpuset *cs = task_cs(p);
- cs->nr_deadline_tasks--;
+ atomic_dec(&cs->nr_deadline_tasks);
}
static inline int is_partition_valid(const struct cpuset *cs)
@@ -956,7 +955,7 @@ static void dl_update_tasks_root_domain(struct cpuset *cs)
struct css_task_iter it;
struct task_struct *task;
- if (cs->nr_deadline_tasks == 0)
+ if (atomic_read(&cs->nr_deadline_tasks) == 0)
return;
css_task_iter_start(&cs->css, 0, &it);
@@ -1356,20 +1355,57 @@ EXPORT_SYMBOL_GPL(cpuset_cpu_is_isolated);
* compute_effective_exclusive_cpumask - compute effective exclusive CPUs
* @cs: cpuset
* @xcpus: effective exclusive CPUs value to be set
- * Return: true if xcpus is not empty, false otherwise.
+ * @real_cs: the real cpuset (can be NULL)
+ * Return: 0 if there is no sibling conflict, > 0 otherwise
*
- * Starting with exclusive_cpus (cpus_allowed if exclusive_cpus is not set),
- * it must be a subset of parent's effective_xcpus.
+ * If exclusive_cpus isn't explicitly set or a real_cs is provided, we have to
+ * scan the sibling cpusets and exclude their exclusive_cpus or effective_xcpus
+ * as well. The provision of real_cs means that a cpumask is being changed and
+ * the given cs is a trial one.
*/
-static bool compute_effective_exclusive_cpumask(struct cpuset *cs,
- struct cpumask *xcpus)
+static int compute_effective_exclusive_cpumask(struct cpuset *cs,
+ struct cpumask *xcpus,
+ struct cpuset *real_cs)
{
+ struct cgroup_subsys_state *css;
struct cpuset *parent = parent_cs(cs);
+ struct cpuset *sibling;
+ int retval = 0;
if (!xcpus)
xcpus = cs->effective_xcpus;
- return cpumask_and(xcpus, user_xcpus(cs), parent->effective_xcpus);
+ cpumask_and(xcpus, user_xcpus(cs), parent->effective_xcpus);
+
+ if (!real_cs) {
+ if (!cpumask_empty(cs->exclusive_cpus))
+ return 0;
+ } else {
+ cs = real_cs;
+ }
+
+ /*
+ * Exclude exclusive CPUs from siblings
+ */
+ rcu_read_lock();
+ cpuset_for_each_child(sibling, css, parent) {
+ if (sibling == cs)
+ continue;
+
+ if (!cpumask_empty(sibling->exclusive_cpus) &&
+ cpumask_intersects(xcpus, sibling->exclusive_cpus)) {
+ cpumask_andnot(xcpus, xcpus, sibling->exclusive_cpus);
+ retval++;
+ continue;
+ }
+ if (!cpumask_empty(sibling->effective_xcpus) &&
+ cpumask_intersects(xcpus, sibling->effective_xcpus)) {
+ cpumask_andnot(xcpus, xcpus, sibling->effective_xcpus);
+ retval++;
+ }
+ }
+ rcu_read_unlock();
+ return retval;
}
static inline bool is_remote_partition(struct cpuset *cs)
@@ -1386,7 +1422,7 @@ static inline bool is_local_partition(struct cpuset *cs)
* remote_partition_enable - Enable current cpuset as a remote partition root
* @cs: the cpuset to update
* @new_prs: new partition_root_state
- * @tmp: temparary masks
+ * @tmp: temporary masks
* Return: 0 if successful, errcode if error
*
* Enable the current cpuset to become a remote partition root taking CPUs
@@ -1411,7 +1447,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
* remote partition root underneath it, its exclusive_cpus must
* have overlapped with subpartitions_cpus.
*/
- compute_effective_exclusive_cpumask(cs, tmp->new_cpus);
+ compute_effective_exclusive_cpumask(cs, tmp->new_cpus, NULL);
if (cpumask_empty(tmp->new_cpus) ||
cpumask_intersects(tmp->new_cpus, subpartitions_cpus) ||
cpumask_subset(top_cpuset.effective_cpus, tmp->new_cpus))
@@ -1420,12 +1456,14 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
spin_lock_irq(&callback_lock);
isolcpus_updated = partition_xcpus_add(new_prs, NULL, tmp->new_cpus);
list_add(&cs->remote_sibling, &remote_children);
+ cpumask_copy(cs->effective_xcpus, tmp->new_cpus);
spin_unlock_irq(&callback_lock);
update_unbound_workqueue_cpumask(isolcpus_updated);
+ cpuset_force_rebuild();
cs->prs_err = 0;
/*
- * Proprogate changes in top_cpuset's effective_cpus down the hierarchy.
+ * Propagate changes in top_cpuset's effective_cpus down the hierarchy.
*/
cpuset_update_tasks_cpumask(&top_cpuset, tmp->new_cpus);
update_sibling_cpumasks(&top_cpuset, NULL, tmp);
@@ -1435,7 +1473,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
/*
* remote_partition_disable - Remove current cpuset from remote partition list
* @cs: the cpuset to update
- * @tmp: temparary masks
+ * @tmp: temporary masks
*
* The effective_cpus is also updated.
*
@@ -1445,25 +1483,34 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp)
{
bool isolcpus_updated;
- compute_effective_exclusive_cpumask(cs, tmp->new_cpus);
WARN_ON_ONCE(!is_remote_partition(cs));
- WARN_ON_ONCE(!cpumask_subset(tmp->new_cpus, subpartitions_cpus));
+ /*
+ * When a CPU is offlined, top_cpuset may end up with no available CPUs,
+ * which should clear subpartitions_cpus. We should not emit a warning for this
+ * scenario: the hierarchy is updated from top to bottom, so subpartitions_cpus
+ * may already be cleared when disabling the partition.
+ */
+ WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus) &&
+ !cpumask_empty(subpartitions_cpus));
spin_lock_irq(&callback_lock);
list_del_init(&cs->remote_sibling);
isolcpus_updated = partition_xcpus_del(cs->partition_root_state,
- NULL, tmp->new_cpus);
+ NULL, cs->effective_xcpus);
if (cs->prs_err)
cs->partition_root_state = -cs->partition_root_state;
else
cs->partition_root_state = PRS_MEMBER;
+ /* effective_xcpus may need to be changed */
+ compute_effective_exclusive_cpumask(cs, NULL, NULL);
reset_partition_data(cs);
spin_unlock_irq(&callback_lock);
update_unbound_workqueue_cpumask(isolcpus_updated);
+ cpuset_force_rebuild();
/*
- * Proprogate changes in top_cpuset's effective_cpus down the hierarchy.
+ * Propagate changes in top_cpuset's effective_cpus down the hierarchy.
*/
cpuset_update_tasks_cpumask(&top_cpuset, tmp->new_cpus);
update_sibling_cpumasks(&top_cpuset, NULL, tmp);
@@ -1472,14 +1519,15 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp)
/*
* remote_cpus_update - cpus_exclusive change of remote partition
* @cs: the cpuset to be updated
- * @newmask: the new effective_xcpus mask
- * @tmp: temparary masks
+ * @xcpus: the new exclusive_cpus mask, if non-NULL
+ * @excpus: the new effective_xcpus mask
+ * @tmp: temporary masks
*
* top_cpuset and subpartitions_cpus will be updated or partition can be
* invalidated.
*/
-static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask,
- struct tmpmasks *tmp)
+static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,
+ struct cpumask *excpus, struct tmpmasks *tmp)
{
bool adding, deleting;
int prs = cs->partition_root_state;
@@ -1490,13 +1538,13 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask,
WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus));
- if (cpumask_empty(newmask)) {
+ if (cpumask_empty(excpus)) {
cs->prs_err = PERR_CPUSEMPTY;
goto invalidate;
}
- adding = cpumask_andnot(tmp->addmask, newmask, cs->effective_xcpus);
- deleting = cpumask_andnot(tmp->delmask, cs->effective_xcpus, newmask);
+ adding = cpumask_andnot(tmp->addmask, excpus, cs->effective_xcpus);
+ deleting = cpumask_andnot(tmp->delmask, cs->effective_xcpus, excpus);
/*
* Additions of remote CPUs is only allowed if those CPUs are
@@ -1518,11 +1566,20 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask,
isolcpus_updated += partition_xcpus_add(prs, NULL, tmp->addmask);
if (deleting)
isolcpus_updated += partition_xcpus_del(prs, NULL, tmp->delmask);
+ /*
+ * Need to update effective_xcpus and exclusive_cpus now as
+ * update_sibling_cpumasks() below may iterate back to the same cs.
+ */
+ cpumask_copy(cs->effective_xcpus, excpus);
+ if (xcpus)
+ cpumask_copy(cs->exclusive_cpus, xcpus);
spin_unlock_irq(&callback_lock);
update_unbound_workqueue_cpumask(isolcpus_updated);
+ if (adding || deleting)
+ cpuset_force_rebuild();
/*
- * Proprogate changes in top_cpuset's effective_cpus down the hierarchy.
+ * Propagate changes in top_cpuset's effective_cpus down the hierarchy.
*/
cpuset_update_tasks_cpumask(&top_cpuset, tmp->new_cpus);
update_sibling_cpumasks(&top_cpuset, NULL, tmp);
@@ -1532,47 +1589,6 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask,
remote_partition_disable(cs, tmp);
}
-/*
- * remote_partition_check - check if a child remote partition needs update
- * @cs: the cpuset to be updated
- * @newmask: the new effective_xcpus mask
- * @delmask: temporary mask for deletion (not in tmp)
- * @tmp: temparary masks
- *
- * This should be called before the given cs has updated its cpus_allowed
- * and/or effective_xcpus.
- */
-static void remote_partition_check(struct cpuset *cs, struct cpumask *newmask,
- struct cpumask *delmask, struct tmpmasks *tmp)
-{
- struct cpuset *child, *next;
- int disable_cnt = 0;
-
- /*
- * Compute the effective exclusive CPUs that will be deleted.
- */
- if (!cpumask_andnot(delmask, cs->effective_xcpus, newmask) ||
- !cpumask_intersects(delmask, subpartitions_cpus))
- return; /* No deletion of exclusive CPUs in partitions */
-
- /*
- * Searching the remote children list to look for those that will
- * be impacted by the deletion of exclusive CPUs.
- *
- * Since a cpuset must be removed from the remote children list
- * before it can go offline and holding cpuset_mutex will prevent
- * any change in cpuset status. RCU read lock isn't needed.
- */
- lockdep_assert_held(&cpuset_mutex);
- list_for_each_entry_safe(child, next, &remote_children, remote_sibling)
- if (cpumask_intersects(child->effective_cpus, delmask)) {
- remote_partition_disable(child, tmp);
- disable_cnt++;
- }
- if (disable_cnt)
- cpuset_force_rebuild();
-}
-
/*
* prstate_housekeeping_conflict - check for partition & housekeeping conflicts
* @prstate: partition root state to be checked
@@ -1645,6 +1661,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
bool nocpu;
lockdep_assert_held(&cpuset_mutex);
+ WARN_ON_ONCE(is_remote_partition(cs));
/*
* new_prs will only be changed for the partcmd_update and
@@ -1686,13 +1703,20 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
nocpu = tasks_nocpu_error(parent, cs, xcpus);
if ((cmd == partcmd_enable) || (cmd == partcmd_enablei)) {
+ /*
+ * Need to call compute_effective_exclusive_cpumask() in case
+ * exclusive_cpus not set. Sibling conflict should only happen
+ * if exclusive_cpus isn't set.
+ */
+ xcpus = tmp->new_cpus;
+ if (compute_effective_exclusive_cpumask(cs, xcpus, NULL))
+ WARN_ON_ONCE(!cpumask_empty(cs->exclusive_cpus));
+
/*
* Enabling partition root is not allowed if its
- * effective_xcpus is empty or doesn't overlap with
- * parent's effective_xcpus.
+ * effective_xcpus is empty.
*/
- if (cpumask_empty(xcpus) ||
- !cpumask_intersects(xcpus, parent->effective_xcpus))
+ if (cpumask_empty(xcpus))
return PERR_INVCPUS;
if (prstate_housekeeping_conflict(new_prs, xcpus))
@@ -1707,13 +1731,16 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
new_prs = (cmd == partcmd_enable) ? PRS_ROOT : PRS_ISOLATED;
} else if (cmd == partcmd_disable) {
/*
- * May need to add cpus to parent's effective_cpus for
- * valid partition root.
+ * May need to add cpus back to parent's effective_cpus
+ * (and maybe removed from subpartitions_cpus/isolated_cpus)
+ * for valid partition root. xcpus may contain CPUs that
+ * shouldn't be removed from the two global cpumasks.
*/
- adding = !is_prs_invalid(old_prs) &&
- cpumask_and(tmp->addmask, xcpus, parent->effective_xcpus);
- if (adding)
+ if (is_partition_valid(cs)) {
+ cpumask_copy(tmp->addmask, cs->effective_xcpus);
+ adding = true;
subparts_delta--;
+ }
new_prs = PRS_MEMBER;
} else if (newmask) {
/*
@@ -1723,6 +1750,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,
part_error = PERR_CPUSEMPTY;
goto write_error;
}
+
/* Check newmask again, whether cpus are available for parent/cs */
nocpu |= tasks_nocpu_error(parent, cs, newmask);
@@ -1939,7 +1967,7 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
* 2) All the effective_cpus will be used up and cp
* has tasks
*/
- compute_effective_exclusive_cpumask(cs, new_ecpus);
+ compute_effective_exclusive_cpumask(cs, new_ecpus, NULL);
cpumask_and(new_ecpus, new_ecpus, cpu_active_mask);
rcu_read_lock();
@@ -1947,6 +1975,11 @@ static void compute_partition_effective_cpumask(struct cpuset *cs,
if (!is_partition_valid(child))
continue;
+ /*
+ * There shouldn't be a remote partition underneath another
+ * partition root.
+ */
+ WARN_ON_ONCE(is_remote_partition(child));
child->prs_err = 0;
if (!cpumask_subset(child->effective_xcpus,
cs->effective_xcpus))
@@ -2002,32 +2035,39 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
bool remote = is_remote_partition(cp);
bool update_parent = false;
+ old_prs = new_prs = cp->partition_root_state;
+
/*
- * Skip descendent remote partition that acquires CPUs
- * directly from top cpuset unless it is cs.
+ * For child remote partition root (!= cs), we need to call
+ * remote_cpus_update() if effective_xcpus will be changed.
+ * Otherwise, we can skip the whole subtree.
+ *
+ * remote_cpus_update() will reuse tmp->new_cpus only after
+ * its value is being processed.
*/
if (remote && (cp != cs)) {
- pos_css = css_rightmost_descendant(pos_css);
- continue;
- }
+ compute_effective_exclusive_cpumask(cp, tmp->new_cpus, NULL);
+ if (cpumask_equal(cp->effective_xcpus, tmp->new_cpus)) {
+ pos_css = css_rightmost_descendant(pos_css);
+ continue;
+ }
+ rcu_read_unlock();
+ remote_cpus_update(cp, NULL, tmp->new_cpus, tmp);
+ rcu_read_lock();
- /*
- * Update effective_xcpus if exclusive_cpus set.
- * The case when exclusive_cpus isn't set is handled later.
- */
- if (!cpumask_empty(cp->exclusive_cpus) && (cp != cs)) {
- spin_lock_irq(&callback_lock);
- compute_effective_exclusive_cpumask(cp, NULL);
- spin_unlock_irq(&callback_lock);
+ /* Remote partition may be invalidated */
+ new_prs = cp->partition_root_state;
+ remote = (new_prs == old_prs);
}
- old_prs = new_prs = cp->partition_root_state;
- if (remote || (is_partition_valid(parent) &&
- is_partition_valid(cp)))
+ if (remote || (is_partition_valid(parent) && is_partition_valid(cp)))
compute_partition_effective_cpumask(cp, tmp->new_cpus);
else
compute_effective_cpumask(tmp->new_cpus, cp, parent);
+ if (remote)
+ goto get_css; /* Ready to update cpuset data */
+
/*
* A partition with no effective_cpus is allowed as long as
* there is no task associated with it. Call
@@ -2047,9 +2087,6 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
if (is_in_v2_mode() && !remote && cpumask_empty(tmp->new_cpus))
cpumask_copy(tmp->new_cpus, parent->effective_cpus);
- if (remote)
- goto get_css;
-
/*
* Skip the whole subtree if
* 1) the cpumask remains the same,
@@ -2110,6 +2147,9 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp,
spin_lock_irq(&callback_lock);
cpumask_copy(cp->effective_cpus, tmp->new_cpus);
cp->partition_root_state = new_prs;
+ if (!cpumask_empty(cp->exclusive_cpus) && (cp != cs))
+ compute_effective_exclusive_cpumask(cp, NULL, NULL);
+
/*
* Make sure effective_xcpus is properly set for a valid
* partition root.
@@ -2196,7 +2236,14 @@ static void update_sibling_cpumasks(struct cpuset *parent, struct cpuset *cs,
parent);
if (cpumask_equal(tmp->new_cpus, sibling->effective_cpus))
continue;
+ } else if (is_remote_partition(sibling)) {
+ /*
+ * Change in a sibling cpuset won't affect a remote
+ * partition root.
+ */
+ continue;
}
+
if (!css_tryget_online(&sibling->css))
continue;
@@ -2248,13 +2295,14 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
return -EINVAL;
/*
- * When exclusive_cpus isn't explicitly set, it is constrainted
+ * When exclusive_cpus isn't explicitly set, it is constrained
* by cpus_allowed and parent's effective_xcpus. Otherwise,
* trialcs->effective_xcpus is used as a temporary cpumask
* for checking validity of the partition root.
*/
+ trialcs->partition_root_state = PRS_MEMBER;
if (!cpumask_empty(trialcs->exclusive_cpus) || is_partition_valid(cs))
- compute_effective_exclusive_cpumask(trialcs, NULL);
+ compute_effective_exclusive_cpumask(trialcs, NULL, cs);
}
/* Nothing to do if the cpus didn't change */
@@ -2327,19 +2375,13 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs,
* Call remote_cpus_update() to handle valid remote partition
*/
if (is_remote_partition(cs))
- remote_cpus_update(cs, xcpus, &tmp);
+ remote_cpus_update(cs, NULL, xcpus, &tmp);
else if (invalidate)
update_parent_effective_cpumask(cs, partcmd_invalidate,
NULL, &tmp);
else
update_parent_effective_cpumask(cs, partcmd_update,
xcpus, &tmp);
- } else if (!cpumask_empty(cs->exclusive_cpus)) {
- /*
- * Use trialcs->effective_cpus as a temp cpumask
- */
- remote_partition_check(cs, trialcs->effective_xcpus,
- trialcs->effective_cpus, &tmp);
}
spin_lock_irq(&callback_lock);
@@ -2391,8 +2433,15 @@ static int update_exclusive_cpumask(struct cpuset *cs, struct cpuset *trialcs,
if (cpumask_equal(cs->exclusive_cpus, trialcs->exclusive_cpus))
return 0;
- if (*buf)
- compute_effective_exclusive_cpumask(trialcs, NULL);
+ if (*buf) {
+ trialcs->partition_root_state = PRS_MEMBER;
+ /*
+ * Reject the change if there is exclusive CPUs conflict with
+ * the siblings.
+ */
+ if (compute_effective_exclusive_cpumask(trialcs, NULL, cs))
+ return -EINVAL;
+ }
/*
* Check all the descendants in update_cpumasks_hier() if
@@ -2423,8 +2472,8 @@ static int update_exclusive_cpumask(struct cpuset *cs, struct cpuset *trialcs,
if (invalidate)
remote_partition_disable(cs, &tmp);
else
- remote_cpus_update(cs, trialcs->effective_xcpus,
- &tmp);
+ remote_cpus_update(cs, trialcs->exclusive_cpus,
+ trialcs->effective_xcpus, &tmp);
} else if (invalidate) {
update_parent_effective_cpumask(cs, partcmd_invalidate,
NULL, &tmp);
@@ -2432,12 +2481,6 @@ static int update_exclusive_cpumask(struct cpuset *cs, struct cpuset *trialcs,
update_parent_effective_cpumask(cs, partcmd_update,
trialcs->effective_xcpus, &tmp);
}
- } else if (!cpumask_empty(trialcs->exclusive_cpus)) {
- /*
- * Use trialcs->effective_cpus as a temp cpumask
- */
- remote_partition_check(cs, trialcs->effective_xcpus,
- trialcs->effective_cpus, &tmp);
}
spin_lock_irq(&callback_lock);
cpumask_copy(cs->exclusive_cpus, trialcs->exclusive_cpus);
@@ -2823,17 +2866,6 @@ static int update_prstate(struct cpuset *cs, int new_prs)
if (alloc_cpumasks(NULL, &tmpmask))
return -ENOMEM;
- /*
- * Setup effective_xcpus if not properly set yet, it will be cleared
- * later if partition becomes invalid.
- */
- if ((new_prs > 0) && cpumask_empty(cs->exclusive_cpus)) {
- spin_lock_irq(&callback_lock);
- cpumask_and(cs->effective_xcpus,
- cs->cpus_allowed, parent->effective_xcpus);
- spin_unlock_irq(&callback_lock);
- }
-
err = update_partition_exclusive(cs, new_prs);
if (err)
goto out;
@@ -3142,8 +3174,8 @@ static void cpuset_attach(struct cgroup_taskset *tset)
cs->old_mems_allowed = cpuset_attach_nodemask_to;
if (cs->nr_migrate_dl_tasks) {
- cs->nr_deadline_tasks += cs->nr_migrate_dl_tasks;
- oldcs->nr_deadline_tasks -= cs->nr_migrate_dl_tasks;
+ atomic_add(cs->nr_migrate_dl_tasks, &cs->nr_deadline_tasks);
+ atomic_sub(cs->nr_migrate_dl_tasks, &oldcs->nr_deadline_tasks);
reset_migrate_dl_data(cs);
}
@@ -3791,13 +3823,13 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
if (remote || (is_partition_valid(cs) && is_partition_valid(parent)))
compute_partition_effective_cpumask(cs, &new_cpus);
- if (remote && cpumask_empty(&new_cpus) &&
- partition_is_populated(cs, NULL)) {
+ if (remote && (cpumask_empty(subpartitions_cpus) ||
+ (cpumask_empty(&new_cpus) &&
+ partition_is_populated(cs, NULL)))) {
cs->prs_err = PERR_HOTPLUG;
remote_partition_disable(cs, tmp);
compute_effective_cpumask(&new_cpus, cs, parent);
remote = false;
- cpuset_force_rebuild();
}
/*
@@ -3806,9 +3838,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
* 1) empty effective cpus but not valid empty partition.
* 2) parent is invalid or doesn't grant any cpus to child
* partitions.
+ * 3) subpartitions_cpus is empty.
*/
- if (is_local_partition(cs) && (!is_partition_valid(parent) ||
- tasks_nocpu_error(parent, cs, &new_cpus)))
+ if (is_local_partition(cs) &&
+ (!is_partition_valid(parent) ||
+ tasks_nocpu_error(parent, cs, &new_cpus) ||
+ cpumask_empty(subpartitions_cpus)))
partcmd = partcmd_invalidate;
/*
* On the other hand, an invalid partition root may be transitioned
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index b8fe0b3d0ffb..c8bba1b6f59c 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -168,22 +168,21 @@ static bool dma_direct_use_pool(struct device *dev, gfp_t gfp)
return !gfpflags_allow_blocking(gfp) && !is_swiotlb_for_alloc(dev);
}
-static void *dma_direct_alloc_from_pool(struct device *dev, size_t size,
- dma_addr_t *dma_handle, gfp_t gfp)
+static struct page *dma_direct_alloc_from_pool(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, void **cpu_addr, gfp_t gfp)
{
struct page *page;
u64 phys_limit;
- void *ret;
if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_DMA_COHERENT_POOL)))
return NULL;
gfp |= dma_direct_optimal_gfp_mask(dev, &phys_limit);
- page = dma_alloc_from_pool(dev, size, &ret, gfp, dma_coherent_ok);
+ page = dma_alloc_from_pool(dev, size, cpu_addr, gfp, dma_coherent_ok);
if (!page)
return NULL;
*dma_handle = phys_to_dma_direct(dev, page_to_phys(page));
- return ret;
+ return page;
}
static void *dma_direct_alloc_no_mapping(struct device *dev, size_t size,
@@ -251,8 +250,11 @@ void *dma_direct_alloc(struct device *dev, size_t size,
* the atomic pools instead if we aren't allowed block.
*/
if ((remap || force_dma_unencrypted(dev)) &&
- dma_direct_use_pool(dev, gfp))
- return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp);
+ dma_direct_use_pool(dev, gfp)) {
+ page = dma_direct_alloc_from_pool(dev, size, dma_handle,
+ &ret, gfp);
+ return page ? ret : NULL;
+ }
/* we always manually zero the memory once we are done */
page = __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true);
@@ -361,7 +363,7 @@ struct page *dma_direct_alloc_pages(struct device *dev, size_t size,
void *ret;
if (force_dma_unencrypted(dev) && dma_direct_use_pool(dev, gfp))
- return dma_direct_alloc_from_pool(dev, size, dma_handle, gfp);
+ return dma_direct_alloc_from_pool(dev, size, dma_handle, &ret, gfp);
page = __dma_direct_alloc_pages(dev, size, gfp, false);
if (!page)
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index e2445b3af746..ffb7b021e2fb 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -268,9 +268,9 @@ void __init swiotlb_update_mem_attributes(void)
}
static void swiotlb_init_io_tlb_pool(struct io_tlb_pool *mem, phys_addr_t start,
- unsigned long nslabs, bool late_alloc, unsigned int nareas)
+ void *vaddr, unsigned long nslabs, bool late_alloc,
+ unsigned int nareas)
{
- void *vaddr = phys_to_virt(start);
unsigned long bytes = nslabs << IO_TLB_SHIFT, i;
mem->nslabs = nslabs;
@@ -411,7 +411,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags,
return;
}
- swiotlb_init_io_tlb_pool(mem, __pa(tlb), nslabs, false, nareas);
+ swiotlb_init_io_tlb_pool(mem, __pa(tlb), tlb, nslabs, false, nareas);
add_mem_pool(&io_tlb_default_mem, mem);
if (flags & SWIOTLB_VERBOSE)
@@ -509,7 +509,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
set_memory_decrypted((unsigned long)vstart,
(nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT);
- swiotlb_init_io_tlb_pool(mem, virt_to_phys(vstart), nslabs, true,
+ swiotlb_init_io_tlb_pool(mem, virt_to_phys(vstart), vstart, nslabs, true,
nareas);
add_mem_pool(&io_tlb_default_mem, mem);
@@ -607,25 +607,26 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, u64 phys_limit)
* @bytes: Size of the buffer.
* @phys_limit: Maximum allowed physical address of the buffer.
* @gfp: GFP flags for the allocation.
+ * @vaddr: Receives the virtual address for the allocated buffer.
*
* Return: Allocated pages, or %NULL on allocation failure.
*/
static struct page *swiotlb_alloc_tlb(struct device *dev, size_t bytes,
- u64 phys_limit, gfp_t gfp)
+ u64 phys_limit, gfp_t gfp, void **vaddr)
{
struct page *page;
+ *vaddr = NULL;
+
/*
* Allocate from the atomic pools if memory is encrypted and
* the allocation is atomic, because decrypting may block.
*/
if (!gfpflags_allow_blocking(gfp) && dev && force_dma_unencrypted(dev)) {
- void *vaddr;
-
if (!IS_ENABLED(CONFIG_DMA_COHERENT_POOL))
return NULL;
- return dma_alloc_from_pool(dev, bytes, &vaddr, gfp,
+ return dma_alloc_from_pool(dev, bytes, vaddr, gfp,
dma_coherent_ok);
}
@@ -647,6 +648,8 @@ static struct page *swiotlb_alloc_tlb(struct device *dev, size_t bytes,
return NULL;
}
+ if (page)
+ *vaddr = phys_to_virt(page_to_phys(page));
return page;
}
@@ -687,6 +690,7 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
{
struct io_tlb_pool *pool;
unsigned int slot_order;
+ void *tlb_vaddr;
struct page *tlb;
size_t pool_size;
size_t tlb_size;
@@ -703,7 +707,8 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
pool->areas = (void *)pool + sizeof(*pool);
tlb_size = nslabs << IO_TLB_SHIFT;
- while (!(tlb = swiotlb_alloc_tlb(dev, tlb_size, phys_limit, gfp))) {
+ while (!(tlb = swiotlb_alloc_tlb(dev, tlb_size, phys_limit, gfp,
+ &tlb_vaddr))) {
if (nslabs <= minslabs)
goto error_tlb;
nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE);
@@ -717,11 +722,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev,
if (!pool->slots)
goto error_slots;
- swiotlb_init_io_tlb_pool(pool, page_to_phys(tlb), nslabs, true, nareas);
+ swiotlb_init_io_tlb_pool(pool, page_to_phys(tlb), tlb_vaddr, nslabs,
+ true, nareas);
return pool;
error_slots:
- swiotlb_free_tlb(page_address(tlb), tlb_size);
+ swiotlb_free_tlb(tlb_vaddr, tlb_size);
error_tlb:
kfree(pool);
error:
@@ -1849,7 +1855,8 @@ static int rmem_swiotlb_device_init(struct reserved_mem *rmem,
set_memory_decrypted((unsigned long)phys_to_virt(rmem->base),
rmem->size >> PAGE_SHIFT);
- swiotlb_init_io_tlb_pool(pool, rmem->base, nslabs,
+ swiotlb_init_io_tlb_pool(pool, rmem->base, phys_to_virt(rmem->base),
+ nslabs,
false, nareas);
mem->force_bounce = true;
mem->for_alloc = true;
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index 5b6934e23c21..fc6552d065cc 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -45,6 +45,9 @@ long syscall_trace_enter(struct pt_regs *regs, long syscall,
ret = ptrace_report_syscall_entry(regs);
if (ret || (work & SYSCALL_WORK_SYSCALL_EMU))
return -1L;
+
+ /* ptrace might have changed work flags */
+ work = READ_ONCE(current_thread_info()->syscall_work);
}
/* Do seccomp after ptrace, to catch any tracer changes. */
diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c
index f4a69942780f..99176c18f686 100644
--- a/kernel/futex/requeue.c
+++ b/kernel/futex/requeue.c
@@ -149,8 +149,16 @@ static inline void futex_requeue_pi_complete(struct futex_q *q, int locked)
} while (!atomic_try_cmpxchg(&q->requeue_state, &old, new));
#ifdef CONFIG_PREEMPT_RT
- /* If the waiter interleaved with the requeue let it know */
- if (unlikely(old == Q_REQUEUE_PI_WAIT))
+ /*
+ * The waiter in futex_requeue_pi_wakeup_sync() can interleave with the
+ * wake below: It will assign Q_REQUEUE_PI_IN_PROGRESS and here it will
+ * be updated to Q_REQUEUE_PI_LOCKED (locked = 1). The rcuwait_wait_event()
+ * will already read Q_REQUEUE_PI_LOCKED and skip the schedule() invocation,
+ * leading to an access of futex_q::requeue_wait after the waiter returned.
+ * In this case only we skip the wake here and rely on following wake in
+ * requeue_pi_wake_futex() to perform the wake if needed.
+ */
+ if (unlikely(old == Q_REQUEUE_PI_WAIT) && new != Q_REQUEUE_PI_LOCKED)
rcuwait_wake_up(&q->requeue_wait);
#endif
}
diff --git a/kernel/kcsan/core.c b/kernel/kcsan/core.c
index 8a7baf4e332e..2db82661cd60 100644
--- a/kernel/kcsan/core.c
+++ b/kernel/kcsan/core.c
@@ -585,8 +585,14 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type, unsigned
* information is lost if dirtied by KCSAN.
*/
kcsan_save_irqtrace(current);
- if (!interrupt_watcher)
+ if (!interrupt_watcher) {
local_irq_save(irq_flags);
+ /*
+ * NMIs can still fire, disable checking for all interrupt
+ * contexts.
+ */
+ raw_cpu_ptr(&kcsan_cpu_ctx)->disable_count++;
+ }
watchpoint = insert_watchpoint((unsigned long)ptr, size, is_write);
if (watchpoint == NULL) {
@@ -699,8 +705,10 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type, unsigned
atomic_long_dec(&kcsan_counters[KCSAN_COUNTER_USED_WATCHPOINTS]);
out_unlock:
- if (!interrupt_watcher)
+ if (!interrupt_watcher) {
+ raw_cpu_ptr(&kcsan_cpu_ctx)->disable_count--;
local_irq_restore(irq_flags);
+ }
kcsan_restore_irqtrace(current);
ctx->disable_scoped--;
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 00ba56a27b66..dd48cb15c884 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -1393,8 +1393,14 @@ static bool __within_kprobe_blacklist(unsigned long addr)
/*
* If 'kprobe_blacklist' is defined, check the address and
* reject any probe registration in the prohibited area.
+ * Note: this can return true during transition period where
+ * (start_addr, end_addr) in the black list is shrinking
+ * but old entry has not been removed yet. This is acceptable
+ * because the worst case is that we reject more probes than
+ * we should.
*/
- list_for_each_entry(ent, &kprobe_blacklist, list) {
+ guard(rcu)();
+ list_for_each_entry_rcu(ent, &kprobe_blacklist, list) {
if (addr >= ent->start_addr && addr < ent->end_addr)
return true;
}
@@ -2474,7 +2480,7 @@ int kprobe_add_ksym_blacklist(unsigned long entry)
ent->start_addr = entry;
ent->end_addr = entry + size;
INIT_LIST_HEAD(&ent->list);
- list_add_tail(&ent->list, &kprobe_blacklist);
+ list_add_tail_rcu(&ent->list, &kprobe_blacklist);
return (int)size;
}
@@ -2568,8 +2574,8 @@ static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end)
list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) {
if (ent->start_addr < start || ent->start_addr >= end)
continue;
- list_del(&ent->list);
- kfree(ent);
+ list_del_rcu(&ent->list);
+ kfree_rcu(ent, rcu);
}
}
diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index 53c76dc71f3f..1422a04f950d 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -5402,6 +5402,8 @@ __lock_set_class(struct lockdep_map *lock, const char *name,
lock->wait_type_outer,
lock->lock_type);
class = register_lock_class(lock, subclass, 0);
+ if (!class)
+ return 0;
hlock->class_idx = class - lock_classes;
curr->lockdep_depth = i;
diff --git a/kernel/module/dups.c b/kernel/module/dups.c
index bd2149fbe117..c21b675c239b 100644
--- a/kernel/module/dups.c
+++ b/kernel/module/dups.c
@@ -30,6 +30,7 @@
#include <linux/ptrace.h>
#include <linux/async.h>
#include <linux/uaccess.h>
+#include <linux/refcount.h>
#include "internal.h"
@@ -38,27 +39,37 @@
static bool enable_dups_trace = IS_ENABLED(CONFIG_MODULE_DEBUG_AUTOLOAD_DUPS_TRACE);
module_param(enable_dups_trace, bool_enable_only, 0644);
-/*
- * Protects dup_kmod_reqs list, adds / removals with RCU.
- */
+/* A mutex-protected list of active kmod requests. */
static DEFINE_MUTEX(kmod_dup_mutex);
static LIST_HEAD(dup_kmod_reqs);
struct kmod_dup_req {
+ refcount_t refcount;
struct list_head list;
char name[MODULE_NAME_LEN];
struct completion first_req_done;
- struct work_struct complete_work;
struct delayed_work delete_work;
int dup_ret;
};
+static void get_kmod_req(struct kmod_dup_req *kmod_req)
+{
+ refcount_inc(&kmod_req->refcount);
+}
+
+static void put_kmod_req(struct kmod_dup_req *kmod_req)
+{
+ if (refcount_dec_and_test(&kmod_req->refcount))
+ kfree(kmod_req);
+}
+
static struct kmod_dup_req *kmod_dup_request_lookup(char *module_name)
{
struct kmod_dup_req *kmod_req;
- list_for_each_entry_rcu(kmod_req, &dup_kmod_reqs, list,
- lockdep_is_held(&kmod_dup_mutex)) {
+ lockdep_assert_held(&kmod_dup_mutex);
+
+ list_for_each_entry(kmod_req, &dup_kmod_reqs, list) {
if (strlen(kmod_req->name) == strlen(module_name) &&
!memcmp(kmod_req->name, module_name, strlen(module_name))) {
return kmod_req;
@@ -87,33 +98,10 @@ static void kmod_dup_request_delete(struct work_struct *work)
* just returning 0.
*/
mutex_lock(&kmod_dup_mutex);
- list_del_rcu(&kmod_req->list);
- synchronize_rcu();
+ list_del(&kmod_req->list);
mutex_unlock(&kmod_dup_mutex);
- kfree(kmod_req);
-}
-static void kmod_dup_request_complete(struct work_struct *work)
-{
- struct kmod_dup_req *kmod_req;
-
- kmod_req = container_of(work, struct kmod_dup_req, complete_work);
-
- /*
- * This will ensure that the kernel will let all the waiters get
- * informed its time to check the return value. It's time to
- * go home.
- */
- complete_all(&kmod_req->first_req_done);
-
- /*
- * Now that we have allowed prior request_module() calls to go on
- * with life, let's schedule deleting this entry. We don't have
- * to do it right away, but we *eventually* want to do it so to not
- * let this linger forever as this is just a boot optimization for
- * possible abuses of vmalloc() incurred by finit_module() thrashing.
- */
- queue_delayed_work(system_wq, &kmod_req->delete_work, 60 * HZ);
+ put_kmod_req(kmod_req);
}
bool kmod_dup_request_exists_wait(char *module_name, bool wait, int *dup_ret)
@@ -129,8 +117,8 @@ bool kmod_dup_request_exists_wait(char *module_name, bool wait, int *dup_ret)
if (!new_kmod_req)
return false;
- memcpy(new_kmod_req->name, module_name, strlen(module_name));
- INIT_WORK(&new_kmod_req->complete_work, kmod_dup_request_complete);
+ refcount_set(&new_kmod_req->refcount, 1);
+ strscpy(new_kmod_req->name, module_name);
INIT_DELAYED_WORK(&new_kmod_req->delete_work, kmod_dup_request_delete);
init_completion(&new_kmod_req->first_req_done);
@@ -161,10 +149,12 @@ bool kmod_dup_request_exists_wait(char *module_name, bool wait, int *dup_ret)
* keep tab on duplicates later.
*/
pr_debug("New request_module() for %s\n", module_name);
- list_add_rcu(&new_kmod_req->list, &dup_kmod_reqs);
+ list_add(&new_kmod_req->list, &dup_kmod_reqs);
mutex_unlock(&kmod_dup_mutex);
return false;
}
+
+ get_kmod_req(kmod_req);
mutex_unlock(&kmod_dup_mutex);
/* We are dealing with a duplicate request now */
@@ -194,7 +184,7 @@ bool kmod_dup_request_exists_wait(char *module_name, bool wait, int *dup_ret)
* calls bail out right away.
*/
*dup_ret = 0;
- return true;
+ goto out;
}
/*
@@ -209,12 +199,14 @@ bool kmod_dup_request_exists_wait(char *module_name, bool wait, int *dup_ret)
TASK_KILLABLE);
if (ret) {
*dup_ret = ret;
- return true;
+ goto out;
}
/* Now the duplicate request has the same exact return value as the first request */
*dup_ret = kmod_req->dup_ret;
+out:
+ put_kmod_req(kmod_req);
return true;
}
@@ -224,24 +216,31 @@ void kmod_dup_request_announce(char *module_name, int ret)
mutex_lock(&kmod_dup_mutex);
+ /*
+ * Look for a kmod_dup_req previously added in
+ * kmod_dup_request_exists_wait(). Note that a request_module_nowait()
+ * without its own kmod_dup_req entry can announce a result of
+ * a concurrent request_module() call.
+ */
kmod_req = kmod_dup_request_lookup(module_name);
- if (!kmod_req)
- goto out;
+ if (!kmod_req || completion_done(&kmod_req->first_req_done)) {
+ mutex_unlock(&kmod_dup_mutex);
+ return;
+ }
kmod_req->dup_ret = ret;
- /*
- * If we complete() here we may allow duplicate threads
- * to continue before the first one that submitted the
- * request. We're in no rush also, given that each and
- * every bounce back to userspace is slow we avoid that
- * with a slight delay here. So queueue up the completion
- * and let duplicates suffer, just wait a tad bit longer.
- * There is no rush. But we also don't want to hold the
- * caller up forever or introduce any boot delays.
- */
- queue_work(system_wq, &kmod_req->complete_work);
+ /* Inform all duplicate waiters to check the return value. */
+ complete_all(&kmod_req->first_req_done);
-out:
mutex_unlock(&kmod_dup_mutex);
+
+ /*
+ * Now that we have allowed prior request_module() calls to go on
+ * with life, let's schedule deleting this entry. We don't have
+ * to do it right away, but we *eventually* want to do it so to not
+ * let this linger forever as this is just a boot optimization for
+ * possible abuses of vmalloc() incurred by finit_module() thrashing.
+ */
+ queue_delayed_work(system_dfl_wq, &kmod_req->delete_work, 60 * HZ);
}
diff --git a/kernel/module/stats.c b/kernel/module/stats.c
index 3ba0e98b3c91..2a4e2f670896 100644
--- a/kernel/module/stats.c
+++ b/kernel/module/stats.c
@@ -253,7 +253,7 @@ int try_add_failed_module(const char *name, enum fail_dup_mod_reason reason)
mod_fail = kzalloc(sizeof(*mod_fail), GFP_KERNEL);
if (!mod_fail)
return -ENOMEM;
- memcpy(mod_fail->name, name, strlen(name));
+ strscpy(mod_fail->name, name);
__set_bit(reason, &mod_fail->dup_fail_mask);
atomic_long_inc(&mod_fail->count);
list_add_rcu(&mod_fail->list, &dup_failed_modules);
diff --git a/kernel/panic.c b/kernel/panic.c
index ddad0578355b..97c472164393 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -187,6 +187,59 @@ void __weak crash_smp_send_stop(void)
atomic_t panic_cpu = ATOMIC_INIT(PANIC_CPU_INVALID);
+bool panic_try_start(void)
+{
+ int old_cpu, this_cpu;
+
+ /*
+ * Only one CPU is allowed to execute the crash_kexec() code as with
+ * panic(). Otherwise parallel calls of panic() and crash_kexec()
+ * may stop each other. To exclude them, we use panic_cpu here too.
+ */
+ old_cpu = PANIC_CPU_INVALID;
+ this_cpu = raw_smp_processor_id();
+
+ return atomic_try_cmpxchg(&panic_cpu, &old_cpu, this_cpu);
+}
+EXPORT_SYMBOL(panic_try_start);
+
+void panic_reset(void)
+{
+ atomic_set(&panic_cpu, PANIC_CPU_INVALID);
+}
+EXPORT_SYMBOL(panic_reset);
+
+bool panic_in_progress(void)
+{
+ return unlikely(atomic_read(&panic_cpu) != PANIC_CPU_INVALID);
+}
+EXPORT_SYMBOL(panic_in_progress);
+
+/* Return true if a panic is in progress on the current CPU. */
+bool panic_on_this_cpu(void)
+{
+ /*
+ * We can use raw_smp_processor_id() here because it is impossible for
+ * the task to be migrated to the panic_cpu, or away from it. If
+ * panic_cpu has already been set, and we're not currently executing on
+ * that CPU, then we never will be.
+ */
+ return unlikely(atomic_read(&panic_cpu) == raw_smp_processor_id());
+}
+EXPORT_SYMBOL(panic_on_this_cpu);
+
+/*
+ * Return true if a panic is in progress on a remote CPU.
+ *
+ * On true, the local CPU should immediately release any printing resources
+ * that may be needed by the panic CPU.
+ */
+bool panic_on_other_cpu(void)
+{
+ return (panic_in_progress() && !panic_on_this_cpu());
+}
+EXPORT_SYMBOL(panic_on_other_cpu);
+
/*
* A variant of panic() called from NMI context. We return if we've already
* panicked on this CPU. If another CPU already panicked, loop in
diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index c56b08121dbc..38bab282d1b1 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -2803,9 +2803,10 @@ int snapshot_write_next(struct snapshot_handle *handle)
return error;
error = memory_bm_create(&zero_bm, GFP_ATOMIC, PG_ANY);
- if (error)
+ if (error) {
+ memory_bm_free(©_bm, PG_UNSAFE_CLEAR);
return error;
-
+ }
nr_zero_pages = 0;
hibernate_restore_protection_begin();
diff --git a/kernel/power/wakelock.c b/kernel/power/wakelock.c
index 4e941999a53b..5c2a248cf249 100644
--- a/kernel/power/wakelock.c
+++ b/kernel/power/wakelock.c
@@ -63,7 +63,7 @@ static unsigned int number_of_wakelocks;
static inline bool wakelocks_limit_exceeded(void)
{
- return number_of_wakelocks > CONFIG_PM_WAKELOCKS_LIMIT;
+ return number_of_wakelocks >= CONFIG_PM_WAKELOCKS_LIMIT;
}
static inline void increment_wakelocks_number(void)
diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h
index 5eef70000b43..53c4b3438a1c 100644
--- a/kernel/printk/internal.h
+++ b/kernel/printk/internal.h
@@ -62,6 +62,7 @@ struct dev_printk_info;
extern struct printk_ringbuffer *prb;
extern bool printk_kthreads_running;
+extern bool debug_non_panic_cpus;
__printf(4, 0)
int vprintk_store(int facility, int level,
@@ -327,7 +328,6 @@ struct printk_message {
unsigned long dropped;
};
-bool other_cpu_in_panic(void);
bool printk_get_next_message(struct printk_message *pmsg, u64 seq,
bool is_extended, bool may_supress);
diff --git a/kernel/printk/nbcon.c b/kernel/printk/nbcon.c
index e7a3af81b173..7b4dcb09ce99 100644
--- a/kernel/printk/nbcon.c
+++ b/kernel/printk/nbcon.c
@@ -254,7 +254,7 @@ static int nbcon_context_try_acquire_direct(struct nbcon_context *ctxt,
* opportunity to perform any necessary cleanup if they were
* interrupted by the panic CPU while printing.
*/
- if (other_cpu_in_panic() &&
+ if (panic_on_other_cpu() &&
(!is_reacquire || cur->unsafe_takeover)) {
return -EPERM;
}
@@ -309,7 +309,7 @@ static bool nbcon_waiter_matches(struct nbcon_state *cur, int expected_prio)
* Event #2 implies the new context is PANIC.
* Event #3 occurs when panic() has flushed the console.
* Event #4 occurs when a non-panic CPU reacquires.
- * Event #5 is not possible due to the other_cpu_in_panic() check
+ * Event #5 is not possible due to the panic_on_other_cpu() check
* in nbcon_context_try_acquire_handover().
*/
@@ -348,7 +348,7 @@ static int nbcon_context_try_acquire_requested(struct nbcon_context *ctxt,
struct nbcon_state new;
/* Note that the caller must still remove the request! */
- if (other_cpu_in_panic())
+ if (panic_on_other_cpu())
return -EPERM;
/*
@@ -446,7 +446,7 @@ static int nbcon_context_try_acquire_handover(struct nbcon_context *ctxt,
* nbcon_waiter_matches(). In particular, the assumption that
* lower priorities are ignored during panic.
*/
- if (other_cpu_in_panic())
+ if (panic_on_other_cpu())
return -EPERM;
/* Handover is not possible on the same CPU. */
@@ -1394,7 +1394,7 @@ enum nbcon_prio nbcon_get_default_prio(void)
{
unsigned int *cpu_emergency_nesting;
- if (this_cpu_in_panic())
+ if (panic_on_this_cpu())
return NBCON_PRIO_PANIC;
cpu_emergency_nesting = nbcon_get_cpu_emergency_nesting();
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 3a91b739e8f3..37f1d4637d90 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -17,6 +17,7 @@
* 01Mar01 Andrew Morton
*/
+#include "linux/panic.h"
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/kernel.h>
@@ -345,34 +346,6 @@ static void __up_console_sem(unsigned long ip)
}
#define up_console_sem() __up_console_sem(_RET_IP_)
-static bool panic_in_progress(void)
-{
- return unlikely(atomic_read(&panic_cpu) != PANIC_CPU_INVALID);
-}
-
-/* Return true if a panic is in progress on the current CPU. */
-bool this_cpu_in_panic(void)
-{
- /*
- * We can use raw_smp_processor_id() here because it is impossible for
- * the task to be migrated to the panic_cpu, or away from it. If
- * panic_cpu has already been set, and we're not currently executing on
- * that CPU, then we never will be.
- */
- return unlikely(atomic_read(&panic_cpu) == raw_smp_processor_id());
-}
-
-/*
- * Return true if a panic is in progress on a remote CPU.
- *
- * On true, the local CPU should immediately release any printing resources
- * that may be needed by the panic CPU.
- */
-bool other_cpu_in_panic(void)
-{
- return (panic_in_progress() && !this_cpu_in_panic());
-}
-
/*
* This is used for debugging the mess that is the VT code by
* keeping track if we have the console semaphore held. It's
@@ -2350,6 +2323,22 @@ void printk_legacy_allow_panic_sync(void)
}
}
+bool __read_mostly debug_non_panic_cpus;
+
+#ifdef CONFIG_PRINTK_CALLER
+static int __init debug_non_panic_cpus_setup(char *str)
+{
+ debug_non_panic_cpus = true;
+ pr_info("allow messages from non-panic CPUs in panic()\n");
+
+ return 0;
+}
+early_param("debug_non_panic_cpus", debug_non_panic_cpus_setup);
+module_param(debug_non_panic_cpus, bool, 0644);
+MODULE_PARM_DESC(debug_non_panic_cpus,
+ "allow messages from non-panic CPUs in panic()");
+#endif
+
asmlinkage int vprintk_emit(int facility, int level,
const struct dev_printk_info *dev_info,
const char *fmt, va_list args)
@@ -2366,7 +2355,9 @@ asmlinkage int vprintk_emit(int facility, int level,
* non-panic CPUs are generating any messages, they will be
* silently dropped.
*/
- if (other_cpu_in_panic() && !panic_triggering_all_cpu_backtrace)
+ if (panic_on_other_cpu() &&
+ !debug_non_panic_cpus &&
+ !panic_triggering_all_cpu_backtrace)
return 0;
printk_get_console_flush_type(&ft);
@@ -2800,7 +2791,7 @@ void console_lock(void)
might_sleep();
/* On panic, the console_lock must be left to the panic cpu. */
- while (other_cpu_in_panic())
+ while (panic_on_other_cpu())
msleep(1000);
down_console_sem();
@@ -2820,7 +2811,7 @@ EXPORT_SYMBOL(console_lock);
int console_trylock(void)
{
/* On panic, the console_lock must be left to the panic cpu. */
- if (other_cpu_in_panic())
+ if (panic_on_other_cpu())
return 0;
if (down_trylock_console_sem())
return 0;
@@ -3113,6 +3104,106 @@ static inline void printk_kthreads_check_locked(void) { }
#endif /* CONFIG_PRINTK */
+
+/*
+ * Print out one record for each console.
+ *
+ * @do_cond_resched is set by the caller. It can be true only in schedulable
+ * context.
+ *
+ * @next_seq is set to the sequence number after the last available record.
+ * The value is valid only when there is at least one usable console and all
+ * usable consoles were flushed.
+ *
+ * @handover will be set to true if a printk waiter has taken over the
+ * console_lock, in which case the caller is no longer holding the
+ * console_lock. Otherwise it is set to false.
+ *
+ * @any_usable will be set to true if there are any usable consoles.
+ *
+ * Returns true when there was at least one usable console and a record was
+ * flushed. A returned false indicates there were no records to flush for any
+ * of the consoles. It may also indicate that there were no usable consoles,
+ * the context has been lost or there is a panic suitation. Regardless the
+ * reason, the caller should assume it is not useful to immediately try again.
+ *
+ * Requires the console_lock.
+ */
+static bool console_flush_one_record(bool do_cond_resched, u64 *next_seq, bool *handover,
+ bool *any_usable)
+{
+ struct console_flush_type ft;
+ bool any_progress = false;
+ struct console *con;
+ int cookie;
+
+ printk_get_console_flush_type(&ft);
+
+ cookie = console_srcu_read_lock();
+ for_each_console_srcu(con) {
+ short flags = console_srcu_read_flags(con);
+ u64 printk_seq;
+ bool progress;
+
+ /*
+ * console_flush_one_record() is only responsible for
+ * nbcon consoles when the nbcon consoles cannot print via
+ * their atomic or threaded flushing.
+ */
+ if ((flags & CON_NBCON) && (ft.nbcon_atomic || ft.nbcon_offload))
+ continue;
+
+ if (!console_is_usable(con, flags, !do_cond_resched))
+ continue;
+ *any_usable = true;
+
+ if (flags & CON_NBCON) {
+ progress = nbcon_legacy_emit_next_record(con, handover, cookie,
+ !do_cond_resched);
+ } else {
+ progress = console_emit_next_record(con, handover, cookie);
+ }
+
+ /*
+ * If a handover has occurred, the SRCU read lock
+ * is already released.
+ */
+ if (*handover)
+ return false;
+
+ /*
+ * @con can be used here now that it is certain that this
+ * context is still holding the SRCU read lock.
+ */
+ if (flags & CON_NBCON)
+ printk_seq = nbcon_seq_read(con);
+ else
+ printk_seq = con->seq;
+
+ /* Track the next of the highest seq flushed. */
+ if (printk_seq > *next_seq)
+ *next_seq = printk_seq;
+
+ if (!progress)
+ continue;
+ any_progress = true;
+
+ /* Allow panic_cpu to take over the consoles safely. */
+ if (panic_on_other_cpu())
+ goto abandon;
+
+ if (do_cond_resched)
+ cond_resched();
+ }
+ console_srcu_read_unlock(cookie);
+
+ return any_progress;
+
+abandon:
+ console_srcu_read_unlock(cookie);
+ return false;
+}
+
/*
* Print out all remaining records to all consoles.
*
@@ -3138,77 +3229,24 @@ static inline void printk_kthreads_check_locked(void) { }
*/
static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handover)
{
- struct console_flush_type ft;
bool any_usable = false;
- struct console *con;
bool any_progress;
- int cookie;
*next_seq = 0;
*handover = false;
do {
- any_progress = false;
-
- printk_get_console_flush_type(&ft);
-
- cookie = console_srcu_read_lock();
- for_each_console_srcu(con) {
- short flags = console_srcu_read_flags(con);
- u64 printk_seq;
- bool progress;
-
- /*
- * console_flush_all() is only responsible for nbcon
- * consoles when the nbcon consoles cannot print via
- * their atomic or threaded flushing.
- */
- if ((flags & CON_NBCON) && (ft.nbcon_atomic || ft.nbcon_offload))
- continue;
-
- if (!console_is_usable(con, flags, !do_cond_resched))
- continue;
- any_usable = true;
-
- if (flags & CON_NBCON) {
- progress = nbcon_legacy_emit_next_record(con, handover, cookie,
- !do_cond_resched);
- printk_seq = nbcon_seq_read(con);
- } else {
- progress = console_emit_next_record(con, handover, cookie);
- printk_seq = con->seq;
- }
-
- /*
- * If a handover has occurred, the SRCU read lock
- * is already released.
- */
- if (*handover)
- return false;
-
- /* Track the next of the highest seq flushed. */
- if (printk_seq > *next_seq)
- *next_seq = printk_seq;
-
- if (!progress)
- continue;
- any_progress = true;
+ any_progress = console_flush_one_record(do_cond_resched, next_seq, handover,
+ &any_usable);
- /* Allow panic_cpu to take over the consoles safely. */
- if (other_cpu_in_panic())
- goto abandon;
+ if (*handover)
+ return false;
- if (do_cond_resched)
- cond_resched();
- }
- console_srcu_read_unlock(cookie);
+ if (panic_on_other_cpu())
+ return false;
} while (any_progress);
return any_usable;
-
-abandon:
- console_srcu_read_unlock(cookie);
- return false;
}
static void __console_flush_and_unlock(void)
diff --git a/kernel/printk/printk_ringbuffer.c b/kernel/printk/printk_ringbuffer.c
index 88e8f3a61922..e2a1b2d34d2b 100644
--- a/kernel/printk/printk_ringbuffer.c
+++ b/kernel/printk/printk_ringbuffer.c
@@ -2133,9 +2133,9 @@ static bool _prb_read_valid(struct printk_ringbuffer *rb, u64 *seq,
* there may be other finalized records beyond that
* need to be printed for a panic situation. If this
* is the panic CPU, skip this
- * non-existent/non-finalized record unless it is
- * at or beyond the head, in which case it is not
- * possible to continue.
+ * non-existent/non-finalized record unless non-panic
+ * CPUs are still running and their debugging is
+ * explicitly enabled.
*
* Note that new messages printed on panic CPU are
* finalized when we are here. The only exception
@@ -2143,10 +2143,13 @@ static bool _prb_read_valid(struct printk_ringbuffer *rb, u64 *seq,
* But it would have the sequence number returned
* by "prb_next_reserve_seq() - 1".
*/
- if (this_cpu_in_panic() && ((*seq + 1) < prb_next_reserve_seq(rb)))
+ if (panic_on_this_cpu() &&
+ (!debug_non_panic_cpus || legacy_allow_panic_sync) &&
+ ((*seq + 1) < prb_next_reserve_seq(rb))) {
(*seq)++;
- else
+ } else {
return false;
+ }
}
}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 47a44f6dede0..8708ca260929 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -973,7 +973,7 @@ void rcu_all_qs(void)
{
unsigned long flags;
- if (!raw_cpu_read(rcu_data.rcu_urgent_qs))
+ if (!READ_ONCE(*raw_cpu_ptr(&rcu_data.rcu_urgent_qs)))
return;
preempt_disable(); // For CONFIG_PREEMPT_COUNT=y kernels
/* Load rcu_urgent_qs before other flags. */
@@ -981,8 +981,8 @@ void rcu_all_qs(void)
preempt_enable();
return;
}
- this_cpu_write(rcu_data.rcu_urgent_qs, false);
- if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs))) {
+ WRITE_ONCE(*this_cpu_ptr(&rcu_data.rcu_urgent_qs), false);
+ if (unlikely(READ_ONCE(*this_cpu_ptr(&rcu_data.rcu_need_heavy_qs)))) {
local_irq_save(flags);
rcu_momentary_eqs();
local_irq_restore(flags);
@@ -1002,8 +1002,8 @@ void rcu_note_context_switch(bool preempt)
/* Load rcu_urgent_qs before other flags. */
if (!smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs)))
goto out;
- this_cpu_write(rcu_data.rcu_urgent_qs, false);
- if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs)))
+ WRITE_ONCE(*this_cpu_ptr(&rcu_data.rcu_urgent_qs), false);
+ if (unlikely(READ_ONCE(*this_cpu_ptr(&rcu_data.rcu_need_heavy_qs))))
rcu_momentary_eqs();
out:
rcu_tasks_qs(current, preempt);
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index 3d9606c2f32c..7c41ed3ec917 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -320,7 +320,7 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, u64 time,
* A CPU running a task which woken up after an IO operation can have its
* utilization boosted to speed up the completion of those IO operations.
* The IO boost value is increased each time a task wakes up from IO, in
- * sugov_iowait_apply(), and it's instead decreased by this function,
+ * sugov_iowait_boost(), and it's instead decreased by this function,
* each time an increase has not been requested (!iowait_boost_pending).
*
* A CPU which also appears to have been idle for at least one tick has also
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 5a09448e22c3..616ad9f86b22 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4416,7 +4416,8 @@ static inline void
update_tg_cfs_runnable(struct cfs_rq *cfs_rq, struct sched_entity *se, struct cfs_rq *gcfs_rq)
{
long delta_sum, delta_avg = gcfs_rq->avg.runnable_avg - se->avg.runnable_avg;
- u32 new_sum, divider;
+ u64 new_sum;
+ u32 divider;
/* Nothing to update */
if (!delta_avg)
@@ -4430,7 +4431,7 @@ update_tg_cfs_runnable(struct cfs_rq *cfs_rq, struct sched_entity *se, struct cf
/* Set new sched_entity's runnable */
se->avg.runnable_avg = gcfs_rq->avg.runnable_avg;
- new_sum = se->avg.runnable_avg * divider;
+ new_sum = (u64)se->avg.runnable_avg * divider;
delta_sum = (long)new_sum - (long)se->avg.runnable_sum;
se->avg.runnable_sum = new_sum;
@@ -10536,6 +10537,17 @@ static bool update_sd_pick_busiest(struct lb_env *env,
sds->local_stat.group_type != group_has_spare))
return false;
+ /*
+ * Candidate sg has no more than one task per CPU and has higher
+ * per-CPU capacity. Migrating tasks to less capable CPUs may harm
+ * throughput. Maximize throughput, power/energy consequences are not
+ * considered.
+ */
+ if ((env->sd->flags & SD_ASYM_CPUCAPACITY) &&
+ (sgs->group_type <= group_fully_busy) &&
+ (capacity_greater(sg->sgc->min_capacity, capacity_of(env->dst_cpu))))
+ return false;
+
if (sgs->group_type > busiest->group_type)
return true;
@@ -10637,17 +10649,6 @@ static bool update_sd_pick_busiest(struct lb_env *env,
break;
}
- /*
- * Candidate sg has no more than one task per CPU and has higher
- * per-CPU capacity. Migrating tasks to less capable CPUs may harm
- * throughput. Maximize throughput, power/energy consequences are not
- * considered.
- */
- if ((env->sd->flags & SD_ASYM_CPUCAPACITY) &&
- (sgs->group_type <= group_fully_busy) &&
- (capacity_greater(sg->sgc->min_capacity, capacity_of(env->dst_cpu))))
- return false;
-
return true;
}
diff --git a/kernel/sys.c b/kernel/sys.c
index 8283e35c9eeb..6c870636ea73 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -2153,7 +2153,7 @@ static int prctl_set_auxv(struct mm_struct *mm, unsigned long addr,
BUILD_BUG_ON(sizeof(user_auxv) != sizeof(mm->saved_auxv));
task_lock(current);
- memcpy(mm->saved_auxv, user_auxv, len);
+ memcpy(mm->saved_auxv, user_auxv, sizeof(user_auxv));
task_unlock(current);
return 0;
diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index df386912f961..e69c349a2fef 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -1510,8 +1510,12 @@ static int __init init_clocksource_sysfs(void)
{
int error = subsys_system_register(&clocksource_subsys, NULL);
- if (!error)
- error = device_register(&device_clocksource);
+ if (error)
+ return error;
+
+ error = device_register(&device_clocksource);
+ if (error)
+ bus_unregister(&clocksource_subsys);
return error;
}
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index b99171e71e0a..4d8927aeca97 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -2015,6 +2015,11 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,
* xtime_nsec_2 = xtime_nsec_1 - offset
* Which simplifies to:
* xtime_nsec -= offset
+ *
+ * When subtracting offset from xtime_nsec, the same amount
+ * (in appropriate units) has to be added to ntp_error, in
+ * order to correctly track the delta between the time
+ * reported in xtime_nsec, and the intended time.
*/
if ((mult_adj > 0) && (tk->tkr_mono.mult + mult_adj < mult_adj)) {
/* NTP adjustment caused clocksource mult overflow */
@@ -2025,6 +2030,7 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,
tk->tkr_mono.mult += mult_adj;
tk->xtime_interval += interval;
tk->tkr_mono.xtime_nsec -= offset;
+ tk->ntp_error += offset << tk->ntp_error_shift;
}
/*
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 14c1bcbce106..7347a4c27fab 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -76,6 +76,8 @@
.func_hash = &opsname.local_hash, \
.local_hash.regex_lock = __MUTEX_INITIALIZER(opsname.local_hash.regex_lock), \
.subop_list = LIST_HEAD_INIT(opsname.subop_list),
+/* Used only to synchronize the initialization of ftrace_ops */
+static DEFINE_MUTEX(ops_mutex);
#else
#define INIT_OPS_HASH(opsname)
#endif
@@ -160,11 +162,18 @@ const struct ftrace_ops ftrace_nop_ops = {
static inline void ftrace_ops_init(struct ftrace_ops *ops)
{
#ifdef CONFIG_DYNAMIC_FTRACE
- if (!(ops->flags & FTRACE_OPS_FL_INITIALIZED)) {
+ unsigned long flags = smp_load_acquire(&ops->flags);
+
+ if (!(flags & FTRACE_OPS_FL_INITIALIZED)) {
+ guard(mutex)(&ops_mutex);
+ /* Could have been initialized before lock taken */
+ if (unlikely(ops->flags & FTRACE_OPS_FL_INITIALIZED))
+ return;
mutex_init(&ops->local_hash.regex_lock);
INIT_LIST_HEAD(&ops->subop_list);
ops->func_hash = &ops->local_hash;
- ops->flags |= FTRACE_OPS_FL_INITIALIZED;
+ flags = ops->flags | FTRACE_OPS_FL_INITIALIZED;
+ smp_store_release(&ops->flags, flags);
}
#endif
}
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 47e6e1088e42..cfa08e9e6498 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -544,7 +544,6 @@ struct ring_buffer_per_cpu {
struct trace_buffer {
unsigned flags;
- int cpus;
atomic_t record_disabled;
atomic_t resizing;
cpumask_var_t cpumask;
@@ -2407,7 +2406,6 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
init_irq_work(&buffer->irq_work.work, rb_wake_up_waiters);
init_waitqueue_head(&buffer->irq_work.waiters);
- buffer->cpus = nr_cpu_ids;
bsize = sizeof(void *) * nr_cpu_ids;
buffer->buffers = kzalloc(ALIGN(bsize, cache_line_size()),
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 9d25992f0cf3..522b9e11d0ba 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -5823,17 +5823,27 @@ static inline void trace_insert_eval_map_file(struct module *mod,
struct trace_eval_map **start, int len) { }
#endif /* !CONFIG_TRACE_EVAL_MAP_FILE */
-static void trace_insert_eval_map(struct module *mod,
- struct trace_eval_map **start, int len)
+static void
+trace_event_update_with_eval_map(struct module *mod,
+ struct trace_eval_map **start,
+ int len)
{
struct trace_eval_map **map;
- if (len <= 0)
- return;
+ /* Always run sanitizer only if btf_type_tag attr exists. */
+ if (len <= 0) {
+ if (!(IS_ENABLED(CONFIG_DEBUG_INFO_BTF) &&
+ IS_ENABLED(CONFIG_PAHOLE_HAS_BTF_TAG) &&
+ __has_attribute(btf_type_tag)))
+ return;
+ }
map = start;
- trace_event_eval_update(map, len);
+ trace_event_update_all(map, len, mod);
+
+ if (len <= 0)
+ return;
trace_insert_eval_map_file(mod, start, len);
}
@@ -9805,7 +9815,7 @@ static void __init eval_map_work_func(struct work_struct *work)
int len;
len = __stop_ftrace_eval_maps - __start_ftrace_eval_maps;
- trace_insert_eval_map(NULL, __start_ftrace_eval_maps, len);
+ trace_event_update_with_eval_map(NULL, __start_ftrace_eval_maps, len);
}
static int __init trace_eval_init(void)
@@ -9840,9 +9850,6 @@ late_initcall_sync(trace_eval_sync);
#ifdef CONFIG_MODULES
static void trace_module_add_evals(struct module *mod)
{
- if (!mod->num_trace_evals)
- return;
-
/*
* Modules with bad taint do not have events created, do
* not bother with enums either.
@@ -9850,7 +9857,8 @@ static void trace_module_add_evals(struct module *mod)
if (trace_module_has_bad_taint(mod))
return;
- trace_insert_eval_map(mod, mod->trace_evals, mod->num_trace_evals);
+ /* Even if no trace_evals, this need to sanitize field types. */
+ trace_event_update_with_eval_map(mod, mod->trace_evals, mod->num_trace_evals);
}
#ifdef CONFIG_TRACE_EVAL_MAP_FILE
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index baa39009a25d..176e493f79db 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -2071,13 +2071,13 @@ static inline const char *get_syscall_name(int syscall)
#ifdef CONFIG_EVENT_TRACING
void trace_event_init(void);
-void trace_event_eval_update(struct trace_eval_map **map, int len);
+void trace_event_update_all(struct trace_eval_map **map, int len, struct module *mod);
/* Used from boot time tracer */
extern int ftrace_set_clr_event(struct trace_array *tr, char *buf, int set);
extern int trigger_process_regex(struct trace_event_file *file, char *buff);
#else
static inline void __init trace_event_init(void) { }
-static inline void trace_event_eval_update(struct trace_eval_map **map, int len) { }
+static inline void trace_event_update_all(struct trace_eval_map **map, int len, struct module *mod) { }
#endif
#ifdef CONFIG_TRACER_SNAPSHOT
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index a4cb8f760ee4..0da762d9696d 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -3015,43 +3015,120 @@ static void add_str_to_module(struct module *module, char *str)
list_add(&modstr->next, &module_strings);
}
+#define ATTRIBUTE_STR "__attribute__("
+#define ATTRIBUTE_STR_LEN (sizeof(ATTRIBUTE_STR) - 1)
+
+/* Remove all __attribute__() from @type. Return allocated string or @type. */
+static char *sanitize_field_type(const char *type)
+{
+ char *attr, *tmp, *next, *ret = (char *)type;
+ int depth;
+
+ next = (char *)type;
+ while ((attr = strstr(next, ATTRIBUTE_STR))) {
+ /* Retry if "__attribute__(" is a part of another word. */
+ if (attr != next && !isspace(attr[-1])) {
+ next = attr + ATTRIBUTE_STR_LEN;
+ continue;
+ }
+
+ if (ret == type) {
+ ret = kstrdup(type, GFP_KERNEL);
+ if (WARN_ON_ONCE(!ret))
+ return NULL;
+ attr = ret + (attr - type);
+ }
+
+ /* the ATTRIBUTE_STR already has the first '(' */
+ depth = 1;
+ next = attr + ATTRIBUTE_STR_LEN;
+ do {
+ tmp = strpbrk(next, "()");
+ /* There is unbalanced parentheses */
+ if (WARN_ON_ONCE(!tmp)) {
+ kfree(ret);
+ return (char *)type;
+ }
+
+ if (*tmp == '(')
+ depth++;
+ else
+ depth--;
+ next = tmp + 1;
+ } while (depth > 0);
+ next = skip_spaces(next);
+ strcpy(attr, next);
+ next = attr;
+ }
+ return ret;
+}
+
+static char *find_replacable_eval(const char *type, const char *eval_string,
+ int len)
+{
+ char *ptr;
+
+ if (!eval_string)
+ return NULL;
+
+ ptr = strchr(type, '[');
+ if (!ptr)
+ return NULL;
+ ptr++;
+
+ if (!isalpha(*ptr) && *ptr != '_')
+ return NULL;
+
+ if (strncmp(eval_string, ptr, len) != 0)
+ return NULL;
+
+ return ptr;
+}
+
static void update_event_fields(struct trace_event_call *call,
struct trace_eval_map *map)
{
struct ftrace_event_field *field;
+ const char *eval_string = NULL;
struct list_head *head;
+ int len = 0;
char *ptr;
char *str;
- int len = strlen(map->eval_string);
/* Dynamic events should never have field maps */
- if (WARN_ON_ONCE(call->flags & TRACE_EVENT_FL_DYNAMIC))
+ if (call->flags & TRACE_EVENT_FL_DYNAMIC)
return;
+ if (map) {
+ eval_string = map->eval_string;
+ len = strlen(map->eval_string);
+ }
+
head = trace_get_fields(call);
list_for_each_entry(field, head, link) {
- ptr = strchr(field->type, '[');
- if (!ptr)
- continue;
- ptr++;
-
- if (!isalpha(*ptr) && *ptr != '_')
- continue;
+ str = sanitize_field_type(field->type);
+ if (!str)
+ return;
- if (strncmp(map->eval_string, ptr, len) != 0)
- continue;
+ ptr = find_replacable_eval(str, eval_string, len);
+ if (ptr) {
+ if (str == field->type) {
+ str = kstrdup(field->type, GFP_KERNEL);
+ if (WARN_ON_ONCE(!str))
+ return;
+ ptr = str + (ptr - field->type);
+ }
- str = kstrdup(field->type, GFP_KERNEL);
- if (WARN_ON_ONCE(!str))
- return;
- ptr = str + (ptr - field->type);
- ptr = eval_replace(ptr, map, len);
- /* enum/sizeof string smaller than value */
- if (WARN_ON_ONCE(!ptr)) {
- kfree(str);
- continue;
+ ptr = eval_replace(ptr, map, len);
+ /* enum/sizeof string smaller than value */
+ if (WARN_ON_ONCE(!ptr)) {
+ kfree(str);
+ continue;
+ }
}
+ if (str == field->type)
+ continue;
/*
* If the event is part of a module, then we need to free the string
* when the module is removed. Otherwise, it will stay allocated
@@ -3061,20 +3138,28 @@ static void update_event_fields(struct trace_event_call *call,
add_str_to_module(call->module, str);
field->type = str;
+ if (field->filter_type == FILTER_OTHER)
+ field->filter_type = filter_assign_type(field->type);
}
}
-void trace_event_eval_update(struct trace_eval_map **map, int len)
+/* Update all events for replacing eval and sanitizing */
+void trace_event_update_all(struct trace_eval_map **map, int len, struct module *mod)
{
struct trace_event_call *call, *p;
const char *last_system = NULL;
bool first = false;
+ bool updated;
int last_i;
int i;
mutex_lock(&event_mutex);
down_write(&trace_event_sem);
list_for_each_entry_safe(call, p, &ftrace_events, list) {
+
+ if (mod && call->module != mod)
+ continue;
+
/* events are usually grouped together with systems */
if (!last_system || call->class->system != last_system) {
first = true;
@@ -3082,6 +3167,7 @@ void trace_event_eval_update(struct trace_eval_map **map, int len)
last_system = call->class->system;
}
+ updated = false;
/*
* Since calls are grouped by systems, the likelihood that the
* next call in the iteration belongs to the same system as the
@@ -3101,8 +3187,12 @@ void trace_event_eval_update(struct trace_eval_map **map, int len)
}
update_event_printk(call, map[i]);
update_event_fields(call, map[i]);
+ updated = true;
}
}
+ /* If not updated yet, update field for sanitizing. */
+ if (!updated)
+ update_event_fields(call, NULL);
cond_resched();
}
up_write(&trace_event_sem);
diff --git a/lib/dump_stack.c b/lib/dump_stack.c
index 388da1aea14a..0b41b0a34323 100644
--- a/lib/dump_stack.c
+++ b/lib/dump_stack.c
@@ -102,7 +102,7 @@ static void __dump_stack(const char *log_lvl)
*/
asmlinkage __visible void dump_stack_lvl(const char *log_lvl)
{
- bool in_panic = this_cpu_in_panic();
+ bool in_panic = panic_on_this_cpu();
unsigned long flags;
/*
diff --git a/lib/maple_tree.c b/lib/maple_tree.c
index e4b68b19ae9a..c7e3f91fd91c 100644
--- a/lib/maple_tree.c
+++ b/lib/maple_tree.c
@@ -4348,35 +4348,40 @@ int mas_alloc_cyclic(struct ma_state *mas, unsigned long *startp,
void *entry, unsigned long range_lo, unsigned long range_hi,
unsigned long *next, gfp_t gfp)
{
- unsigned long min = range_lo;
- int ret = 0;
-
- range_lo = max(min, *next);
- ret = mas_empty_area(mas, range_lo, range_hi, 1);
- if ((mas->tree->ma_flags & MT_FLAGS_ALLOC_WRAPPED) && ret == 0) {
- mas->tree->ma_flags &= ~MT_FLAGS_ALLOC_WRAPPED;
- ret = 1;
- }
- if (ret < 0 && range_lo > min) {
- mas_reset(mas);
- ret = mas_empty_area(mas, min, range_hi, 1);
- if (ret == 0)
- ret = 1;
- }
- if (ret < 0)
- return ret;
+ int ret;
+ unsigned long min;
+ min = range_lo;
do {
+ range_lo = max(min, *next);
+ ret = mas_empty_area(mas, range_lo, range_hi, 1);
+ if (ret < 0 && range_lo > min) {
+ mas_reset(mas);
+ ret = mas_empty_area(mas, min, range_hi, 1);
+ if (ret == 0)
+ ret = 1;
+ }
+ if (ret < 0)
+ goto out;
+
mas_insert(mas, entry);
} while (mas_nomem(mas, gfp));
- if (mas_is_err(mas))
- return xa_err(mas->node);
+ if (mas_is_err(mas)) {
+ ret = xa_err(mas->node);
+ goto out;
+ }
+
+ if ((mas->tree->ma_flags & MT_FLAGS_ALLOC_WRAPPED) && ret == 0) {
+ mas->tree->ma_flags &= ~MT_FLAGS_ALLOC_WRAPPED;
+ ret = 1;
+ }
*startp = mas->index;
*next = *startp + 1;
if (*next == 0)
mas->tree->ma_flags |= MT_FLAGS_ALLOC_WRAPPED;
+out:
mas_destroy(mas);
return ret;
}
diff --git a/lib/string.c b/lib/string.c
index e657809fa718..8d117fed3126 100644
--- a/lib/string.c
+++ b/lib/string.c
@@ -828,7 +828,8 @@ void *memchr_inv(const void *start, int c, size_t bytes)
{
u8 value = c;
u64 value64;
- unsigned int words, prefix;
+ size_t words;
+ unsigned int prefix;
if (bytes <= 16)
return check_bytes8(start, value, bytes);
diff --git a/lib/test_hmm.c b/lib/test_hmm.c
index 850d23469ef7..62d487223295 100644
--- a/lib/test_hmm.c
+++ b/lib/test_hmm.c
@@ -391,7 +391,7 @@ static int dmirror_fault(struct dmirror *dmirror, unsigned long start,
/* Since the mm is for the mirrored process, get a reference first. */
if (!mmget_not_zero(mm))
- return 0;
+ return -EFAULT;
for (addr = start; addr < end; addr = range.end) {
range.start = addr;
diff --git a/mm/Kconfig b/mm/Kconfig
index 763a9105ee18..620d33330f79 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -550,20 +550,63 @@ menuconfig MEMORY_HOTPLUG
if MEMORY_HOTPLUG
-config MEMORY_HOTPLUG_DEFAULT_ONLINE
- bool "Online the newly added memory blocks by default"
- depends on MEMORY_HOTPLUG
+choice
+ prompt "Memory Hotplug Default Online Type"
+ default MHP_DEFAULT_ONLINE_TYPE_OFFLINE
help
+ Default memory type for hotplugged memory.
+
This option sets the default policy setting for memory hotplug
onlining policy (/sys/devices/system/memory/auto_online_blocks) which
determines what happens to newly added memory regions. Policy setting
can always be changed at runtime.
+
+ The default is 'offline'.
+
+ Select offline to defer onlining to drivers and user policy.
+ Select auto to let the kernel choose what zones to utilize.
+ Select online_kernel to generally allow kernel usage of this memory.
+ Select online_movable to generally disallow kernel usage of this memory.
+
+ Example kernel usage would be page structs and page tables.
+
See Documentation/admin-guide/mm/memory-hotplug.rst for more information.
- Say Y here if you want all hot-plugged memory blocks to appear in
- 'online' state by default.
- Say N here if you want the default policy to keep all hot-plugged
- memory blocks in 'offline' state.
+config MHP_DEFAULT_ONLINE_TYPE_OFFLINE
+ bool "offline"
+ help
+ Hotplugged memory will not be onlined by default.
+ Choose this for systems with drivers and user policy that
+ handle onlining of hotplug memory policy.
+
+config MHP_DEFAULT_ONLINE_TYPE_ONLINE_AUTO
+ bool "auto"
+ help
+ Select this if you want the kernel to automatically online
+ hotplugged memory into the zone it thinks is reasonable.
+ This memory may be utilized for kernel data.
+
+config MHP_DEFAULT_ONLINE_TYPE_ONLINE_KERNEL
+ bool "kernel"
+ help
+ Select this if you want the kernel to automatically online
+ hotplugged memory into a zone capable of being used for kernel
+ data. This typically means ZONE_NORMAL.
+
+config MHP_DEFAULT_ONLINE_TYPE_ONLINE_MOVABLE
+ bool "movable"
+ help
+ Select this if you want the kernel to automatically online
+ hotplug memory into ZONE_MOVABLE. This memory will generally
+ not be utilized for kernel data.
+
+ This should only be used when the admin knows sufficient
+ ZONE_NORMAL memory is available to describe hotplug memory,
+ otherwise hotplug memory may fail to online. For example,
+ sufficient kernel-capable memory (ZONE_NORMAL) must be
+ available to allocate page structs to describe ZONE_MOVABLE.
+
+endchoice
config MEMORY_HOTREMOVE
bool "Allow for memory hot remove"
diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug
index 41a58536531d..32b65073d0cc 100644
--- a/mm/Kconfig.debug
+++ b/mm/Kconfig.debug
@@ -186,8 +186,9 @@ config ARCH_HAS_DEBUG_WX
config DEBUG_WX
bool "Warn on W+X mappings at boot"
depends on ARCH_HAS_DEBUG_WX
+ depends on ARCH_HAS_PTDUMP
depends on MMU
- select PTDUMP_CORE
+ select PTDUMP
help
Generate a warning if any W+X mappings are found at boot.
@@ -212,18 +213,18 @@ config DEBUG_WX
If in doubt, say "Y".
-config GENERIC_PTDUMP
+config ARCH_HAS_PTDUMP
bool
-config PTDUMP_CORE
+config PTDUMP
bool
config PTDUMP_DEBUGFS
bool "Export kernel pagetable layout to userspace via debugfs"
depends on DEBUG_KERNEL
depends on DEBUG_FS
- depends on GENERIC_PTDUMP
- select PTDUMP_CORE
+ depends on ARCH_HAS_PTDUMP
+ select PTDUMP
help
Say Y here if you want to show the kernel pagetable layout in a
debugfs file. This information is only useful for kernel developers
diff --git a/mm/Makefile b/mm/Makefile
index d5639b036166..3455c3f487d4 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -137,7 +137,7 @@ obj-$(CONFIG_ZONE_DEVICE) += memremap.o
obj-$(CONFIG_HMM_MIRROR) += hmm.o
obj-$(CONFIG_MEMFD_CREATE) += memfd.o
obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o
-obj-$(CONFIG_PTDUMP_CORE) += ptdump.o
+obj-$(CONFIG_PTDUMP) += ptdump.o
obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o
obj-$(CONFIG_IO_MAPPING) += io-mapping.o
obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 71264018cf2d..8901a80c8a21 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -1786,15 +1786,17 @@ static void damon_merge_two_regions(struct damon_target *t,
* sz_limit size upper limit of each region
*/
static void damon_merge_regions_of(struct damon_target *t, unsigned int thres,
- unsigned long sz_limit)
+ unsigned long sz_limit, bool count_age)
{
struct damon_region *r, *prev = NULL, *next;
damon_for_each_region_safe(r, next, t) {
- if (abs(r->nr_accesses - r->last_nr_accesses) > thres)
- r->age = 0;
- else
- r->age++;
+ if (count_age) {
+ if (abs(r->nr_accesses - r->last_nr_accesses) > thres)
+ r->age = 0;
+ else
+ r->age++;
+ }
if (prev && prev->ar.end == r->ar.start &&
abs(prev->nr_accesses - r->nr_accesses) <= thres &&
@@ -1828,15 +1830,18 @@ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold,
struct damon_target *t;
unsigned int nr_regions;
unsigned int max_thres;
+ bool count_age = true;
max_thres = c->attrs.aggr_interval /
(c->attrs.sample_interval ? c->attrs.sample_interval : 1);
do {
nr_regions = 0;
damon_for_each_target(t, c) {
- damon_merge_regions_of(t, threshold, sz_limit);
+ damon_merge_regions_of(t, threshold, sz_limit,
+ count_age);
nr_regions += damon_nr_regions(t);
}
+ count_age = false;
threshold = max(1, threshold * 2);
} while (nr_regions > c->attrs.max_nr_regions &&
threshold / 2 < max_thres);
diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 4b5638bbe061..dea662a6747c 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -80,8 +80,9 @@ int damon_hot_score(struct damon_ctx *c, struct damon_region *r,
unsigned int age_weight = s->quota.weight_age;
int hotness;
- freq_subscore = r->nr_accesses * DAMON_MAX_SUBSCORE /
- damon_max_nr_accesses(&c->attrs);
+ freq_subscore = mult_frac(r->nr_accesses_bp / 10000,
+ DAMON_MAX_SUBSCORE,
+ damon_max_nr_accesses(&c->attrs));
age_in_sec = (unsigned long)r->age * c->attrs.aggr_interval / 1000000;
for (age_in_log = 0; age_in_log < DAMON_MAX_AGE_IN_LOG && age_in_sec;
diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c
index d4a4d51750f5..a6b1de97b1e8 100644
--- a/mm/damon/paddr.c
+++ b/mm/damon/paddr.c
@@ -148,7 +148,7 @@ static bool damon_folio_young(struct folio *folio)
return accessed;
}
-static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz)
+static bool damon_pa_young(unsigned long paddr)
{
struct folio *folio = damon_get_folio(PHYS_PFN(paddr));
bool accessed;
@@ -157,7 +157,6 @@ static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz)
return false;
accessed = damon_folio_young(folio);
- *folio_sz = folio_size(folio);
folio_put(folio);
return accessed;
}
@@ -165,21 +164,10 @@ static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz)
static void __damon_pa_check_access(struct damon_region *r,
struct damon_attrs *attrs)
{
- static unsigned long last_addr;
- static unsigned long last_folio_sz = PAGE_SIZE;
- static bool last_accessed;
-
- /* If the region is in the last checked page, reuse the result */
- if (ALIGN_DOWN(last_addr, last_folio_sz) ==
- ALIGN_DOWN(r->sampling_addr, last_folio_sz)) {
- damon_update_region_access_rate(r, last_accessed, attrs);
- return;
- }
-
- last_accessed = damon_pa_young(r->sampling_addr, &last_folio_sz);
- damon_update_region_access_rate(r, last_accessed, attrs);
+ bool accessed;
- last_addr = r->sampling_addr;
+ accessed = damon_pa_young(r->sampling_addr);
+ damon_update_region_access_rate(r, accessed, attrs);
}
static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx)
diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c
index 998c7b5b4325..7e5fc797359a 100644
--- a/mm/damon/sysfs-schemes.c
+++ b/mm/damon/sysfs-schemes.c
@@ -197,6 +197,7 @@ static void damon_sysfs_scheme_regions_rm_dirs(
list_for_each_entry_safe(r, next, ®ions->regions_list, list) {
list_del(&r->list);
+ kobject_del(&r->kobj);
kobject_put(&r->kobj);
regions->nr_regions--;
}
@@ -560,8 +561,10 @@ static void damon_sysfs_scheme_filters_rm_dirs(
struct damon_sysfs_scheme_filter **filters_arr = filters->filters_arr;
int i;
- for (i = 0; i < filters->nr; i++)
+ for (i = 0; i < filters->nr; i++) {
+ kobject_del(&filters_arr[i]->kobj);
kobject_put(&filters_arr[i]->kobj);
+ }
filters->nr = 0;
kfree(filters_arr);
filters->filters_arr = NULL;
@@ -969,8 +972,10 @@ static void damos_sysfs_quota_goals_rm_dirs(
struct damos_sysfs_quota_goal **goals_arr = goals->goals_arr;
int i;
- for (i = 0; i < goals->nr; i++)
+ for (i = 0; i < goals->nr; i++) {
+ kobject_del(&goals_arr[i]->kobj);
kobject_put(&goals_arr[i]->kobj);
+ }
goals->nr = 0;
kfree(goals_arr);
goals->goals_arr = NULL;
@@ -1772,6 +1777,7 @@ void damon_sysfs_schemes_rm_dirs(struct damon_sysfs_schemes *schemes)
for (i = 0; i < schemes->nr; i++) {
damon_sysfs_scheme_rm_dirs(schemes_arr[i]);
+ kobject_del(&schemes_arr[i]->kobj);
kobject_put(&schemes_arr[i]->kobj);
}
schemes->nr = 0;
@@ -1813,13 +1819,15 @@ static int damon_sysfs_schemes_add_dirs(struct damon_sysfs_schemes *schemes,
goto out;
err = damon_sysfs_scheme_add_dirs(scheme);
if (err)
- goto out;
+ goto del_out;
schemes_arr[i] = scheme;
schemes->nr++;
}
return 0;
+del_out:
+ kobject_del(&scheme->kobj);
out:
damon_sysfs_schemes_rm_dirs(schemes);
kobject_put(&scheme->kobj);
diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c
index 6545ab35be36..5aa23c6545cb 100644
--- a/mm/damon/sysfs.c
+++ b/mm/damon/sysfs.c
@@ -107,8 +107,10 @@ static void damon_sysfs_regions_rm_dirs(struct damon_sysfs_regions *regions)
struct damon_sysfs_region **regions_arr = regions->regions_arr;
int i;
- for (i = 0; i < regions->nr; i++)
+ for (i = 0; i < regions->nr; i++) {
+ kobject_del(®ions_arr[i]->kobj);
kobject_put(®ions_arr[i]->kobj);
+ }
regions->nr = 0;
kfree(regions_arr);
regions->regions_arr = NULL;
@@ -305,6 +307,7 @@ static void damon_sysfs_targets_rm_dirs(struct damon_sysfs_targets *targets)
for (i = 0; i < targets->nr; i++) {
damon_sysfs_target_rm_dirs(targets_arr[i]);
+ kobject_del(&targets_arr[i]->kobj);
kobject_put(&targets_arr[i]->kobj);
}
targets->nr = 0;
@@ -343,13 +346,15 @@ static int damon_sysfs_targets_add_dirs(struct damon_sysfs_targets *targets,
err = damon_sysfs_target_add_dirs(target);
if (err)
- goto out;
+ goto del_out;
targets_arr[i] = target;
targets->nr++;
}
return 0;
+del_out:
+ kobject_del(&target->kobj);
out:
damon_sysfs_targets_rm_dirs(targets);
kobject_put(&target->kobj);
@@ -829,6 +834,7 @@ static void damon_sysfs_contexts_rm_dirs(struct damon_sysfs_contexts *contexts)
for (i = 0; i < contexts->nr; i++) {
damon_sysfs_context_rm_dirs(contexts_arr[i]);
+ kobject_del(&contexts_arr[i]->kobj);
kobject_put(&contexts_arr[i]->kobj);
}
contexts->nr = 0;
@@ -867,13 +873,15 @@ static int damon_sysfs_contexts_add_dirs(struct damon_sysfs_contexts *contexts,
err = damon_sysfs_context_add_dirs(context);
if (err)
- goto out;
+ goto del_out;
contexts_arr[i] = context;
contexts->nr++;
}
return 0;
+del_out:
+ kobject_del(&context->kobj);
out:
damon_sysfs_contexts_rm_dirs(contexts);
kobject_put(&context->kobj);
@@ -1688,6 +1696,7 @@ static void damon_sysfs_kdamonds_rm_dirs(struct damon_sysfs_kdamonds *kdamonds)
for (i = 0; i < kdamonds->nr; i++) {
damon_sysfs_kdamond_rm_dirs(kdamonds_arr[i]);
+ kobject_del(&kdamonds_arr[i]->kobj);
kobject_put(&kdamonds_arr[i]->kobj);
}
kdamonds->nr = 0;
@@ -1743,13 +1752,15 @@ static int damon_sysfs_kdamonds_add_dirs(struct damon_sysfs_kdamonds *kdamonds,
err = damon_sysfs_kdamond_add_dirs(kdamond);
if (err)
- goto out;
+ goto del_out;
kdamonds_arr[i] = kdamond;
kdamonds->nr++;
}
return 0;
+del_out:
+ kobject_del(&kdamond->kobj);
out:
damon_sysfs_kdamonds_rm_dirs(kdamonds);
kobject_put(&kdamond->kobj);
diff --git a/mm/damon/tests/core-kunit.h b/mm/damon/tests/core-kunit.h
index d7737f5df668..ff9f73579f7b 100644
--- a/mm/damon/tests/core-kunit.h
+++ b/mm/damon/tests/core-kunit.h
@@ -166,6 +166,10 @@ static void damon_test_split_at(struct kunit *test)
r->last_nr_accesses = 15;
damon_add_region(r, t);
damon_split_region_at(t, r, 25);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 2);
+ if (damon_nr_regions(t) != 2)
+ goto out;
+
KUNIT_EXPECT_EQ(test, r->ar.start, 0ul);
KUNIT_EXPECT_EQ(test, r->ar.end, 25ul);
@@ -177,6 +181,7 @@ static void damon_test_split_at(struct kunit *test)
KUNIT_EXPECT_EQ(test, r->nr_accesses, r_new->nr_accesses);
KUNIT_EXPECT_EQ(test, r->last_nr_accesses, r_new->last_nr_accesses);
+out:
damon_free_target(t);
damon_destroy_ctx(c);
}
@@ -261,14 +266,17 @@ static void damon_test_merge_regions_of(struct kunit *test)
damon_add_region(r, t);
}
- damon_merge_regions_of(t, 9, 9999);
+ damon_merge_regions_of(t, 9, 9999, true);
/* 0-112, 114-130, 130-156, 156-170 */
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 5u);
+ if (damon_nr_regions(t) != 5)
+ goto out;
for (i = 0; i < 5; i++) {
r = __nth_region_of(t, i);
KUNIT_EXPECT_EQ(test, r->ar.start, saddrs[i]);
KUNIT_EXPECT_EQ(test, r->ar.end, eaddrs[i]);
}
+out:
damon_free_target(t);
}
@@ -552,6 +560,8 @@ static void damos_test_filter_out(struct kunit *test)
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 1);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 2);
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 2);
+ if (damon_nr_regions(t) != 2)
+ goto out;
r2 = damon_next_region(r);
KUNIT_EXPECT_EQ(test, r2->ar.start, DAMON_MIN_REGION * 2);
KUNIT_EXPECT_EQ(test, r2->ar.end, DAMON_MIN_REGION * 4);
@@ -565,11 +575,14 @@ static void damos_test_filter_out(struct kunit *test)
KUNIT_EXPECT_EQ(test, r->ar.start, DAMON_MIN_REGION * 2);
KUNIT_EXPECT_EQ(test, r->ar.end, DAMON_MIN_REGION * 6);
KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 2);
+ if (damon_nr_regions(t) != 2)
+ goto out;
r2 = damon_next_region(r);
KUNIT_EXPECT_EQ(test, r2->ar.start, DAMON_MIN_REGION * 6);
KUNIT_EXPECT_EQ(test, r2->ar.end, DAMON_MIN_REGION * 8);
damon_destroy_region(r2, t);
+out:
damon_free_target(t);
damos_free_filter(f);
}
diff --git a/mm/damon/tests/vaddr-kunit.h b/mm/damon/tests/vaddr-kunit.h
index ed2845d9f9a9..9eead9882dcc 100644
--- a/mm/damon/tests/vaddr-kunit.h
+++ b/mm/damon/tests/vaddr-kunit.h
@@ -149,12 +149,17 @@ static void damon_do_test_apply_three_regions(struct kunit *test,
damon_set_regions(t, three_regions, 3);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), nr_expected / 2);
+ if (damon_nr_regions(t) != nr_expected / 2)
+ goto out;
+
for (i = 0; i < nr_expected / 2; i++) {
r = __nth_region_of(t, i);
KUNIT_EXPECT_EQ(test, r->ar.start, expected[i * 2]);
KUNIT_EXPECT_EQ(test, r->ar.end, expected[i * 2 + 1]);
}
+out:
damon_destroy_target(t);
}
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 52e5c244bd02..089ee23e07de 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -429,8 +429,6 @@ static void damon_va_prepare_access_checks(struct damon_ctx *ctx)
}
struct damon_young_walk_private {
- /* size of the folio for the access checked virtual memory address */
- unsigned long *folio_sz;
bool young;
};
@@ -466,7 +464,6 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
mmu_notifier_test_young(walk->mm,
addr))
priv->young = true;
- *priv->folio_sz = HPAGE_PMD_SIZE;
folio_put(folio);
huge_out:
spin_unlock(ptl);
@@ -488,7 +485,6 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
if (pte_young(ptent) || !folio_test_idle(folio) ||
mmu_notifier_test_young(walk->mm, addr))
priv->young = true;
- *priv->folio_sz = folio_size(folio);
folio_put(folio);
out:
pte_unmap_unlock(pte, ptl);
@@ -517,7 +513,6 @@ static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask,
if (pte_young(entry) || !folio_test_idle(folio) ||
mmu_notifier_test_young(walk->mm, addr))
priv->young = true;
- *priv->folio_sz = huge_page_size(h);
folio_put(folio);
@@ -535,11 +530,9 @@ static const struct mm_walk_ops damon_young_ops = {
.walk_lock = PGWALK_RDLOCK,
};
-static bool damon_va_young(struct mm_struct *mm, unsigned long addr,
- unsigned long *folio_sz)
+static bool damon_va_young(struct mm_struct *mm, unsigned long addr)
{
struct damon_young_walk_private arg = {
- .folio_sz = folio_sz,
.young = false,
};
@@ -556,29 +549,18 @@ static bool damon_va_young(struct mm_struct *mm, unsigned long addr,
* r the region to be checked
*/
static void __damon_va_check_access(struct mm_struct *mm,
- struct damon_region *r, bool same_target,
+ struct damon_region *r,
struct damon_attrs *attrs)
{
- static unsigned long last_addr;
- static unsigned long last_folio_sz = PAGE_SIZE;
- static bool last_accessed;
+ bool accessed;
if (!mm) {
damon_update_region_access_rate(r, false, attrs);
return;
}
- /* If the region is in the last checked page, reuse the result */
- if (same_target && (ALIGN_DOWN(last_addr, last_folio_sz) ==
- ALIGN_DOWN(r->sampling_addr, last_folio_sz))) {
- damon_update_region_access_rate(r, last_accessed, attrs);
- return;
- }
-
- last_accessed = damon_va_young(mm, r->sampling_addr, &last_folio_sz);
- damon_update_region_access_rate(r, last_accessed, attrs);
-
- last_addr = r->sampling_addr;
+ accessed = damon_va_young(mm, r->sampling_addr);
+ damon_update_region_access_rate(r, accessed, attrs);
}
static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)
@@ -587,16 +569,12 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)
struct mm_struct *mm;
struct damon_region *r;
unsigned int max_nr_accesses = 0;
- bool same_target;
damon_for_each_target(t, ctx) {
mm = damon_get_mm(t);
- same_target = false;
damon_for_each_region(r, t) {
- __damon_va_check_access(mm, r, same_target,
- &ctx->attrs);
+ __damon_va_check_access(mm, r, &ctx->attrs);
max_nr_accesses = max(r->nr_accesses, max_nr_accesses);
- same_target = true;
}
if (mm)
mmput(mm);
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index e910ed0c43d0..ea042e28c825 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -5523,6 +5523,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
bool adjust_reservation;
unsigned long last_addr_mask;
+ i_mmap_assert_write_locked(vma->vm_file->f_mapping);
WARN_ON(!is_vm_hugetlb_page(vma));
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
@@ -5613,7 +5614,10 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
/*
* Restore the reservation for anonymous page, otherwise the
- * backing page could be stolen by someone.
+ * backing page could be stolen by someone. Restore only on the
+ * last unmap, otherwise the owner could empty its resv map
+ * while the folio is still mapped by a child. Note that holding
+ * i_mmap_lock_write is needed to check the number of mappings.
* If there we are freeing a surplus, do not set the restore
* reservation bit.
*/
@@ -5621,7 +5625,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
spin_lock_irq(&hugetlb_lock);
if (!h->surplus_huge_pages && __vma_private_lock(vma) &&
- folio_test_anon(page_folio(page))) {
+ !folio_mapped(page_folio(page)) && folio_test_anon(page_folio(page))) {
folio_set_hugetlb_restore_reserve(page_folio(page));
/* Reservation to be adjusted after the spin lock */
adjust_reservation = true;
@@ -7519,14 +7523,14 @@ void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio, int re
* There is no need to transfer the per-node surplus state
* when we do not cross the node.
*/
- if (new_nid == old_nid)
- return;
- spin_lock_irq(&hugetlb_lock);
- if (h->surplus_huge_pages_node[old_nid]) {
- h->surplus_huge_pages_node[old_nid]--;
- h->surplus_huge_pages_node[new_nid]++;
+ if (new_nid != old_nid) {
+ spin_lock_irq(&hugetlb_lock);
+ if (h->surplus_huge_pages_node[old_nid]) {
+ h->surplus_huge_pages_node[old_nid]--;
+ h->surplus_huge_pages_node[new_nid]++;
+ }
+ spin_unlock_irq(&hugetlb_lock);
}
- spin_unlock_irq(&hugetlb_lock);
}
}
diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
index f8744f5630bb..1bad668aa337 100644
--- a/mm/memcontrol-v1.c
+++ b/mm/memcontrol-v1.c
@@ -120,7 +120,6 @@ enum {
RES_LIMIT,
RES_MAX_USAGE,
RES_FAILCNT,
- RES_SOFT_LIMIT,
};
#ifdef CONFIG_LOCKDEP
@@ -2440,6 +2439,30 @@ static int mem_cgroup_hierarchy_write(struct cgroup_subsys_state *css,
return -EINVAL;
}
+static u64 mem_cgroup_soft_limit_read(struct cgroup_subsys_state *css,
+ struct cftype *cft)
+{
+ return (u64)PAGE_COUNTER_MAX * PAGE_SIZE;
+}
+
+static ssize_t mem_cgroup_soft_limit_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ unsigned long nr_pages;
+ int ret;
+
+ ret = page_counter_memparse(strstrip(buf), "-1", &nr_pages);
+ if (ret)
+ return ret;
+
+ pr_warn_once("soft_limit_in_bytes is deprecated and will be removed. "
+ "Writing any value to this file has no effect. "
+ "Please report your usecase to linux-mm@kvack.org if you "
+ "depend on this functionality.\n");
+
+ return nbytes;
+}
+
static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,
struct cftype *cft)
{
@@ -2476,8 +2499,6 @@ static u64 mem_cgroup_read_u64(struct cgroup_subsys_state *css,
return (u64)counter->watermark * PAGE_SIZE;
case RES_FAILCNT:
return counter->failcnt;
- case RES_SOFT_LIMIT:
- return (u64)READ_ONCE(memcg->soft_limit) * PAGE_SIZE;
default:
BUG();
}
@@ -2572,17 +2593,6 @@ static ssize_t mem_cgroup_write(struct kernfs_open_file *of,
break;
}
break;
- case RES_SOFT_LIMIT:
- if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
- ret = -EOPNOTSUPP;
- } else {
- pr_warn_once("soft_limit_in_bytes is deprecated and will be removed. "
- "Please report your usecase to linux-mm@kvack.org if you "
- "depend on this functionality.\n");
- WRITE_ONCE(memcg->soft_limit, nr_pages);
- ret = 0;
- }
- break;
}
return ret ?: nbytes;
}
@@ -2918,9 +2928,8 @@ struct cftype mem_cgroup_legacy_files[] = {
},
{
.name = "soft_limit_in_bytes",
- .private = MEMFILE_PRIVATE(_MEM, RES_SOFT_LIMIT),
- .write = mem_cgroup_write,
- .read_u64 = mem_cgroup_read_u64,
+ .write = mem_cgroup_soft_limit_write,
+ .read_u64 = mem_cgroup_soft_limit_read,
},
{
.name = "failcnt",
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 9479cb19cecf..e21f59cf29d9 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -219,11 +219,30 @@ void put_online_mems(void)
bool movable_node_enabled = false;
-#ifndef CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
-int mhp_default_online_type = MMOP_OFFLINE;
-#else
-int mhp_default_online_type = MMOP_ONLINE;
-#endif
+static int mhp_default_online_type = -1;
+enum mmop mhp_get_default_online_type(void)
+{
+ if (mhp_default_online_type >= 0)
+ return mhp_default_online_type;
+
+ if (IS_ENABLED(CONFIG_MHP_DEFAULT_ONLINE_TYPE_OFFLINE))
+ mhp_default_online_type = MMOP_OFFLINE;
+ else if (IS_ENABLED(CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_AUTO))
+ mhp_default_online_type = MMOP_ONLINE;
+ else if (IS_ENABLED(CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_KERNEL))
+ mhp_default_online_type = MMOP_ONLINE_KERNEL;
+ else if (IS_ENABLED(CONFIG_MHP_DEFAULT_ONLINE_TYPE_ONLINE_MOVABLE))
+ mhp_default_online_type = MMOP_ONLINE_MOVABLE;
+ else
+ mhp_default_online_type = MMOP_OFFLINE;
+
+ return mhp_default_online_type;
+}
+
+void mhp_set_default_online_type(enum mmop online_type)
+{
+ mhp_default_online_type = online_type;
+}
static int __init setup_memhp_default_state(char *str)
{
@@ -1041,7 +1060,7 @@ static inline struct zone *default_zone_for_pfn(int nid, unsigned long start_pfn
return movable_node_enabled ? movable_zone : kernel_zone;
}
-struct zone *zone_for_pfn_range(int online_type, int nid,
+struct zone *zone_for_pfn_range(enum mmop online_type, int nid,
struct memory_group *group, unsigned long start_pfn,
unsigned long nr_pages)
{
@@ -1247,7 +1266,8 @@ static pg_data_t *hotadd_init_pgdat(int nid)
pgdat = NODE_DATA(nid);
/* init node's zones as empty zones, we don't have any present pages.*/
- free_area_init_core_hotplug(pgdat);
+ if (free_area_init_core_hotplug(pgdat))
+ return NULL;
/*
* The node we allocated has no zone fallback lists. For avoiding
@@ -1321,7 +1341,7 @@ static int check_hotplug_memory_range(u64 start, u64 size)
static int online_memory_block(struct memory_block *mem, void *arg)
{
- mem->online_type = mhp_default_online_type;
+ mem->online_type = mhp_get_default_online_type();
return device_online(&mem->dev);
}
@@ -1570,7 +1590,7 @@ int add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)
merge_system_ram_resource(res);
/* online pages if requested */
- if (mhp_default_online_type != MMOP_OFFLINE)
+ if (mhp_get_default_online_type() != MMOP_OFFLINE)
walk_memory_blocks(start, size, NULL, online_memory_block);
return ret;
@@ -2330,7 +2350,7 @@ EXPORT_SYMBOL_GPL(remove_memory);
static int try_offline_memory_block(struct memory_block *mem, void *arg)
{
- uint8_t online_type = MMOP_ONLINE_KERNEL;
+ enum mmop online_type = MMOP_ONLINE_KERNEL;
uint8_t **online_types = arg;
struct page *page;
int rc;
@@ -2363,7 +2383,7 @@ static int try_reonline_memory_block(struct memory_block *mem, void *arg)
int rc;
if (**online_types != MMOP_OFFLINE) {
- mem->online_type = **online_types;
+ mem->online_type = (enum mmop)**online_types;
rc = device_online(&mem->dev);
if (rc < 0)
pr_warn("%s: Failed to re-online memory: %d",
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 6adce609f4f1..37694789a966 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2438,7 +2438,7 @@ static unsigned long alloc_pages_bulk_array_weighted_interleave(gfp_t gfp,
prev_node = node;
/* create a local copy of node weights to operate on outside rcu */
- weights = kzalloc(nr_node_ids, GFP_KERNEL);
+ weights = kmalloc(nr_node_ids, gfp & GFP_RECLAIM_MASK);
if (!weights)
return total_allocated;
diff --git a/mm/mm_init.c b/mm/mm_init.c
index 21e01537c9f5..65f997ddd740 100644
--- a/mm/mm_init.c
+++ b/mm/mm_init.c
@@ -1310,7 +1310,7 @@ static void __init calculate_node_totalpages(struct pglist_data *pgdat,
unsigned long node_start_pfn,
unsigned long node_end_pfn)
{
- unsigned long realtotalpages = 0, totalpages = 0;
+ unsigned long realtotalpages = 0;
enum zone_type i;
for (i = 0; i < MAX_NR_ZONES; i++) {
@@ -1340,11 +1340,10 @@ static void __init calculate_node_totalpages(struct pglist_data *pgdat,
zone->present_early_pages = real_size;
#endif
- totalpages += spanned;
realtotalpages += real_size;
}
- pgdat->node_spanned_pages = totalpages;
+ pgdat->node_spanned_pages = node_end_pfn - node_start_pfn;
pgdat->node_present_pages = realtotalpages;
pr_debug("On node %d totalpages: %lu\n", pgdat->node_id, realtotalpages);
}
@@ -1520,7 +1519,7 @@ void __init set_pageblock_order(void)
* NOTE: this function is only called during memory hotplug
*/
#ifdef CONFIG_MEMORY_HOTPLUG
-void __ref free_area_init_core_hotplug(struct pglist_data *pgdat)
+int __ref free_area_init_core_hotplug(struct pglist_data *pgdat)
{
int nid = pgdat->node_id;
enum zone_type z;
@@ -1528,8 +1527,14 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat)
pgdat_init_internals(pgdat);
- if (pgdat->per_cpu_nodestats == &boot_nodestats)
- pgdat->per_cpu_nodestats = alloc_percpu(struct per_cpu_nodestat);
+ if (pgdat->per_cpu_nodestats == &boot_nodestats) {
+ struct per_cpu_nodestat __percpu *p;
+
+ p = alloc_percpu(struct per_cpu_nodestat);
+ if (!p)
+ return -ENOMEM;
+ pgdat->per_cpu_nodestats = p;
+ }
/*
* Reset the nr_zones, order and highest_zoneidx before reuse.
@@ -1567,6 +1572,8 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat)
zone->present_pages = 0;
zone_init_internals(zone, z, nid, 0);
}
+
+ return 0;
}
#endif
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 585a53f7b06f..1cc8fd382cd1 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -97,7 +97,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp)
static bool check_pte(struct page_vma_mapped_walk *pvmw, unsigned long pte_nr)
{
unsigned long pfn;
- pte_t ptent = ptep_get(pvmw->pte);
+ pte_t ptent;
+
+ if (is_vm_hugetlb_page(pvmw->vma))
+ ptent = huge_ptep_get(pvmw->vma->vm_mm, pvmw->address,
+ pvmw->pte);
+ else
+ ptent = ptep_get(pvmw->pte);
if (pvmw->flags & PVMW_MIGRATION) {
swp_entry_t entry;
diff --git a/mm/ptdump.c b/mm/ptdump.c
index 8c06fbf32685..9d1ed33c5537 100644
--- a/mm/ptdump.c
+++ b/mm/ptdump.c
@@ -19,7 +19,7 @@ static inline int note_kasan_page_table(struct mm_walk *walk,
{
struct ptdump_state *st = walk->private;
- st->note_page(st, addr, 4, pte_val(kasan_early_shadow_pte[0]));
+ st->note_page_pte(st, addr, kasan_early_shadow_pte[0]);
walk->action = ACTION_CONTINUE;
@@ -43,7 +43,7 @@ static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr,
st->effective_prot(st, 0, pgd_val(val));
if (pgd_leaf(val)) {
- st->note_page(st, addr, 0, pgd_val(val));
+ st->note_page_pgd(st, addr, val);
walk->action = ACTION_CONTINUE;
}
@@ -66,7 +66,7 @@ static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr,
st->effective_prot(st, 1, p4d_val(val));
if (p4d_leaf(val)) {
- st->note_page(st, addr, 1, p4d_val(val));
+ st->note_page_p4d(st, addr, val);
walk->action = ACTION_CONTINUE;
}
@@ -89,7 +89,7 @@ static int ptdump_pud_entry(pud_t *pud, unsigned long addr,
st->effective_prot(st, 2, pud_val(val));
if (pud_leaf(val)) {
- st->note_page(st, addr, 2, pud_val(val));
+ st->note_page_pud(st, addr, val);
walk->action = ACTION_CONTINUE;
}
@@ -110,7 +110,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,
if (st->effective_prot)
st->effective_prot(st, 3, pmd_val(val));
if (pmd_leaf(val)) {
- st->note_page(st, addr, 3, pmd_val(val));
+ st->note_page_pmd(st, addr, val);
walk->action = ACTION_CONTINUE;
}
@@ -126,7 +126,7 @@ static int ptdump_pte_entry(pte_t *pte, unsigned long addr,
if (st->effective_prot)
st->effective_prot(st, 4, pte_val(val));
- st->note_page(st, addr, 4, pte_val(val));
+ st->note_page_pte(st, addr, val);
return 0;
}
@@ -135,9 +135,31 @@ static int ptdump_hole(unsigned long addr, unsigned long next,
int depth, struct mm_walk *walk)
{
struct ptdump_state *st = walk->private;
-
- st->note_page(st, addr, depth, 0);
-
+ pte_t pte_zero = {0};
+ pmd_t pmd_zero = {0};
+ pud_t pud_zero = {0};
+ p4d_t p4d_zero = {0};
+ pgd_t pgd_zero = {0};
+
+ switch (depth) {
+ case 4:
+ st->note_page_pte(st, addr, pte_zero);
+ break;
+ case 3:
+ st->note_page_pmd(st, addr, pmd_zero);
+ break;
+ case 2:
+ st->note_page_pud(st, addr, pud_zero);
+ break;
+ case 1:
+ st->note_page_p4d(st, addr, p4d_zero);
+ break;
+ case 0:
+ st->note_page_pgd(st, addr, pgd_zero);
+ break;
+ default:
+ break;
+ }
return 0;
}
@@ -172,7 +194,7 @@ void ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm, pgd_t *pgd)
put_online_mems();
/* Flush out the last page */
- st->note_page(st, 0, -1, 0);
+ st->note_page_flush(st);
}
static int check_wx_show(struct seq_file *m, void *v)
diff --git a/mm/slab.h b/mm/slab.h
index 8e9fc3efc3fa..1b58c2a77140 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -258,6 +258,7 @@ struct kmem_cache_order_objects {
struct kmem_cache {
#ifndef CONFIG_SLUB_TINY
struct kmem_cache_cpu __percpu *cpu_slab;
+ struct lock_class_key lock_key;
#endif
/* Used for retrieving partial slabs, etc. */
slab_flags_t flags;
diff --git a/mm/slub.c b/mm/slub.c
index b61225c3503b..9087fd9a83b7 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1958,7 +1958,7 @@ static inline void handle_failed_objexts_alloc(unsigned long obj_exts,
* objects with no tag reference. Mark all references in this
* vector as empty to avoid warnings later on.
*/
- if (obj_exts & OBJEXTS_ALLOC_FAIL) {
+ if (obj_exts == OBJEXTS_ALLOC_FAIL) {
unsigned int i;
for (i = 0; i < objects; i++)
@@ -2161,8 +2161,15 @@ alloc_tagging_slab_free_hook(struct kmem_cache *s, struct slab *slab, void **p,
return;
obj_exts = slab_obj_exts(slab);
- if (!obj_exts)
+ if (!obj_exts) {
+ /*
+ * If obj_exts allocation failed, slab->obj_exts is set to
+ * OBJEXTS_ALLOC_FAIL. In this case, we end up here and should
+ * clear the flag.
+ */
+ slab->obj_exts = 0;
return;
+ }
for (i = 0; i < objects; i++) {
unsigned int off = obj_to_index(s, slab, p[i]);
@@ -3078,12 +3085,29 @@ static inline void note_cmpxchg_failure(const char *n,
static void init_kmem_cache_cpus(struct kmem_cache *s)
{
+#ifdef CONFIG_PREEMPT_RT
+ /*
+ * Register lockdep key for non-boot kmem caches to avoid
+ * WARN_ON_ONCE(static_obj(key))) in lockdep_register_key()
+ */
+ bool finegrain_lockdep = !init_section_contains(s, 1);
+#else
+ /*
+ * Don't bother with different lockdep classes for each
+ * kmem_cache, since we only use local_trylock_irqsave().
+ */
+ bool finegrain_lockdep = false;
+#endif
int cpu;
struct kmem_cache_cpu *c;
+ if (finegrain_lockdep)
+ lockdep_register_key(&s->lock_key);
for_each_possible_cpu(cpu) {
c = per_cpu_ptr(s->cpu_slab, cpu);
local_lock_init(&c->lock);
+ if (finegrain_lockdep)
+ lockdep_set_class(&c->lock, &s->lock_key);
c->tid = init_tid(cpu);
}
}
@@ -5304,6 +5328,10 @@ void __kmem_cache_release(struct kmem_cache *s)
{
cache_random_seq_destroy(s);
#ifndef CONFIG_SLUB_TINY
+#ifdef CONFIG_PREEMPT_RT
+ if (s->cpu_slab)
+ lockdep_unregister_key(&s->lock_key);
+#endif
free_percpu(s->cpu_slab);
#endif
free_kmem_cache_nodes(s);
@@ -5323,10 +5351,8 @@ static int init_kmem_cache_nodes(struct kmem_cache *s)
n = kmem_cache_alloc_node(kmem_cache_node,
GFP_KERNEL, node);
- if (!n) {
- free_kmem_cache_nodes(s);
+ if (!n)
return 0;
- }
init_kmem_cache_node(n);
s->node[node] = n;
diff --git a/net/batman-adv/bridge_loop_avoidance.c b/net/batman-adv/bridge_loop_avoidance.c
index cc6c14cac062..8942fbdf98b3 100644
--- a/net/batman-adv/bridge_loop_avoidance.c
+++ b/net/batman-adv/bridge_loop_avoidance.c
@@ -323,11 +323,6 @@ batadv_bla_del_backbone_claims(struct batadv_bla_backbone_gw *backbone_gw)
}
spin_unlock_bh(list_lock);
}
-
- /* all claims gone, initialize CRC */
- spin_lock_bh(&backbone_gw->crc_lock);
- backbone_gw->crc = BATADV_BLA_CRC_INIT;
- spin_unlock_bh(&backbone_gw->crc_lock);
}
/**
@@ -698,12 +693,14 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv,
struct batadv_bla_backbone_gw *old_backbone_gw;
struct batadv_bla_claim *claim;
struct batadv_bla_claim search_claim;
- bool remove_crc = false;
int hash_added;
+ u16 claim_crc;
+ bool changed;
ether_addr_copy(search_claim.addr, mac);
search_claim.vid = vid;
claim = batadv_claim_hash_find(bat_priv, &search_claim);
+ claim_crc = crc16(0, mac, ETH_ALEN);
/* create a new claim entry if it does not exist yet. */
if (!claim) {
@@ -735,43 +732,56 @@ static void batadv_bla_add_claim(struct batadv_priv *bat_priv,
kfree(claim);
return;
}
+
+ spin_lock_bh(&backbone_gw->crc_lock);
+ backbone_gw->crc ^= claim_crc;
+ spin_unlock_bh(&backbone_gw->crc_lock);
+
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
+
+ batadv_claim_put(claim);
+ return;
+ }
+
+ WRITE_ONCE(claim->lasttime, jiffies);
+
+ /* replace backbone_gw atomically and adjust reference counters */
+ spin_lock_bh(&claim->backbone_lock);
+ if (claim->backbone_gw != backbone_gw) {
+ changed = true;
+
+ old_backbone_gw = claim->backbone_gw;
+ kref_get(&backbone_gw->refcount);
+ claim->backbone_gw = backbone_gw;
} else {
- WRITE_ONCE(claim->lasttime, jiffies);
- if (claim->backbone_gw == backbone_gw)
- /* no need to register a new backbone */
- goto claim_free_ref;
+ old_backbone_gw = NULL;
+ changed = false;
+ }
+ spin_unlock_bh(&claim->backbone_lock);
+ if (changed) {
batadv_dbg(BATADV_DBG_BLA, bat_priv,
"%s(): changing ownership for %pM, vid %d to gw %pM\n",
__func__, mac, batadv_print_vid(vid),
backbone_gw->orig);
- remove_crc = true;
- }
+ /* add claim address to new backbone_gw */
+ spin_lock_bh(&backbone_gw->crc_lock);
+ backbone_gw->crc ^= claim_crc;
+ spin_unlock_bh(&backbone_gw->crc_lock);
- /* replace backbone_gw atomically and adjust reference counters */
- spin_lock_bh(&claim->backbone_lock);
- old_backbone_gw = claim->backbone_gw;
- kref_get(&backbone_gw->refcount);
- claim->backbone_gw = backbone_gw;
- spin_unlock_bh(&claim->backbone_lock);
+ WRITE_ONCE(backbone_gw->lasttime, jiffies);
+ }
- if (remove_crc) {
+ if (old_backbone_gw) {
/* remove claim address from old backbone_gw */
spin_lock_bh(&old_backbone_gw->crc_lock);
- old_backbone_gw->crc ^= crc16(0, claim->addr, ETH_ALEN);
+ old_backbone_gw->crc ^= claim_crc;
spin_unlock_bh(&old_backbone_gw->crc_lock);
- }
- batadv_backbone_gw_put(old_backbone_gw);
-
- /* add claim address to new backbone_gw */
- spin_lock_bh(&backbone_gw->crc_lock);
- backbone_gw->crc ^= crc16(0, claim->addr, ETH_ALEN);
- spin_unlock_bh(&backbone_gw->crc_lock);
- WRITE_ONCE(backbone_gw->lasttime, jiffies);
+ batadv_backbone_gw_put(old_backbone_gw);
+ }
-claim_free_ref:
batadv_claim_put(claim);
}
@@ -941,26 +951,18 @@ static bool batadv_handle_unclaim(struct batadv_priv *bat_priv,
const u8 *backbone_addr, const u8 *claim_addr,
unsigned short vid)
{
- struct batadv_bla_backbone_gw *backbone_gw;
-
/* unclaim in any case if it is our own */
if (primary_if && batadv_compare_eth(backbone_addr,
primary_if->net_dev->dev_addr))
batadv_bla_send_claim(bat_priv, claim_addr, vid,
BATADV_CLAIM_TYPE_UNCLAIM);
- backbone_gw = batadv_backbone_hash_find(bat_priv, backbone_addr, vid);
-
- if (!backbone_gw)
- return true;
-
/* this must be an UNCLAIM frame */
batadv_dbg(BATADV_DBG_BLA, bat_priv,
"%s(): UNCLAIM %pM on vid %d (sent by %pM)...\n", __func__,
- claim_addr, batadv_print_vid(vid), backbone_gw->orig);
+ claim_addr, batadv_print_vid(vid), backbone_addr);
batadv_bla_del_claim(bat_priv, claim_addr, vid);
- batadv_backbone_gw_put(backbone_gw);
return true;
}
diff --git a/net/batman-adv/distributed-arp-table.c b/net/batman-adv/distributed-arp-table.c
index ceebfa4f0107..45e58b348bd7 100644
--- a/net/batman-adv/distributed-arp-table.c
+++ b/net/batman-adv/distributed-arp-table.c
@@ -250,7 +250,10 @@ static u8 *batadv_arp_hw_src(struct sk_buff *skb, int hdr_size)
*/
static __be32 batadv_arp_ip_src(struct sk_buff *skb, int hdr_size)
{
- return *(__force __be32 *)(batadv_arp_hw_src(skb, hdr_size) + ETH_ALEN);
+ u8 *src = batadv_arp_hw_src(skb, hdr_size) + ETH_ALEN;
+ __be32 *ip = (__force __be32 *)src;
+
+ return get_unaligned(ip);
}
/**
@@ -275,8 +278,9 @@ static u8 *batadv_arp_hw_dst(struct sk_buff *skb, int hdr_size)
static __be32 batadv_arp_ip_dst(struct sk_buff *skb, int hdr_size)
{
u8 *dst = batadv_arp_hw_src(skb, hdr_size) + ETH_ALEN * 2 + 4;
+ __be32 *ip = (__force __be32 *)dst;
- return *(__force __be32 *)dst;
+ return get_unaligned(ip);
}
/**
@@ -372,18 +376,19 @@ batadv_dat_entry_hash_find(struct batadv_priv *bat_priv, __be32 ip,
static void batadv_dat_entry_add(struct batadv_priv *bat_priv, __be32 ip,
u8 *mac_addr, unsigned short vid)
{
+ u64 u64_mac = ether_addr_to_u64(mac_addr);
struct batadv_dat_entry *dat_entry;
int hash_added;
dat_entry = batadv_dat_entry_hash_find(bat_priv, ip, vid);
/* if this entry is already known, just update it */
if (dat_entry) {
- if (!batadv_compare_eth(dat_entry->mac_addr, mac_addr))
- ether_addr_copy(dat_entry->mac_addr, mac_addr);
+ atomic64_set(&dat_entry->mac_addr, u64_mac);
+
dat_entry->last_update = jiffies;
batadv_dbg(BATADV_DBG_DAT, bat_priv,
"Entry updated: %pI4 %pM (vid: %d)\n",
- &dat_entry->ip, dat_entry->mac_addr,
+ &dat_entry->ip, mac_addr,
batadv_print_vid(vid));
goto out;
}
@@ -394,7 +399,7 @@ static void batadv_dat_entry_add(struct batadv_priv *bat_priv, __be32 ip,
dat_entry->ip = ip;
dat_entry->vid = vid;
- ether_addr_copy(dat_entry->mac_addr, mac_addr);
+ atomic64_set(&dat_entry->mac_addr, u64_mac);
dat_entry->last_update = jiffies;
kref_init(&dat_entry->refcount);
@@ -410,7 +415,7 @@ static void batadv_dat_entry_add(struct batadv_priv *bat_priv, __be32 ip,
}
batadv_dbg(BATADV_DBG_DAT, bat_priv, "New entry added: %pI4 %pM (vid: %d)\n",
- &dat_entry->ip, dat_entry->mac_addr, batadv_print_vid(vid));
+ &dat_entry->ip, mac_addr, batadv_print_vid(vid));
out:
batadv_dat_entry_put(dat_entry);
@@ -865,6 +870,8 @@ batadv_dat_cache_dump_entry(struct sk_buff *msg, u32 portid,
struct netlink_callback *cb,
struct batadv_dat_entry *dat_entry)
{
+ u8 mac[ETH_ALEN];
+ u64 u64_mac;
int msecs;
void *hdr;
@@ -877,11 +884,12 @@ batadv_dat_cache_dump_entry(struct sk_buff *msg, u32 portid,
genl_dump_check_consistent(cb, hdr);
msecs = jiffies_to_msecs(jiffies - dat_entry->last_update);
+ u64_mac = atomic64_read(&dat_entry->mac_addr);
+ u64_to_ether_addr(u64_mac, mac);
if (nla_put_in_addr(msg, BATADV_ATTR_DAT_CACHE_IP4ADDRESS,
dat_entry->ip) ||
- nla_put(msg, BATADV_ATTR_DAT_CACHE_HWADDRESS, ETH_ALEN,
- dat_entry->mac_addr) ||
+ nla_put(msg, BATADV_ATTR_DAT_CACHE_HWADDRESS, ETH_ALEN, mac) ||
nla_put_u16(msg, BATADV_ATTR_DAT_CACHE_VID, dat_entry->vid) ||
nla_put_u32(msg, BATADV_ATTR_LAST_SEEN_MSECS, msecs)) {
genlmsg_cancel(msg, hdr);
@@ -1157,6 +1165,8 @@ bool batadv_dat_snoop_outgoing_arp_request(struct batadv_priv *bat_priv,
struct net_device *soft_iface = bat_priv->soft_iface;
int hdr_size = 0;
unsigned short vid;
+ u8 mac[ETH_ALEN];
+ u64 u64_mac;
if (!atomic_read(&bat_priv->distributed_arp_table))
goto out;
@@ -1184,6 +1194,9 @@ bool batadv_dat_snoop_outgoing_arp_request(struct batadv_priv *bat_priv,
dat_entry = batadv_dat_entry_hash_find(bat_priv, ip_dst, vid);
if (dat_entry) {
+ u64_mac = atomic64_read(&dat_entry->mac_addr);
+ u64_to_ether_addr(u64_mac, mac);
+
/* If the ARP request is destined for a local client the local
* client will answer itself. DAT would only generate a
* duplicate packet.
@@ -1192,7 +1205,7 @@ bool batadv_dat_snoop_outgoing_arp_request(struct batadv_priv *bat_priv,
* additional DAT answer may trigger kernel warnings about
* a packet coming from the wrong port.
*/
- if (batadv_is_my_client(bat_priv, dat_entry->mac_addr, vid)) {
+ if (batadv_is_my_client(bat_priv, mac, vid)) {
ret = true;
goto out;
}
@@ -1202,18 +1215,16 @@ bool batadv_dat_snoop_outgoing_arp_request(struct batadv_priv *bat_priv,
* the backbone gws belonging to our backbone has claimed the
* destination.
*/
- if (!batadv_bla_check_claim(bat_priv,
- dat_entry->mac_addr, vid)) {
+ if (!batadv_bla_check_claim(bat_priv, mac, vid)) {
batadv_dbg(BATADV_DBG_DAT, bat_priv,
"Device %pM claimed by another backbone gw. Don't send ARP reply!",
- dat_entry->mac_addr);
+ mac);
ret = true;
goto out;
}
skb_new = batadv_dat_arp_create_reply(bat_priv, ip_dst, ip_src,
- dat_entry->mac_addr,
- hw_src, vid);
+ mac, hw_src, vid);
if (!skb_new)
goto out;
@@ -1255,6 +1266,8 @@ bool batadv_dat_snoop_incoming_arp_request(struct batadv_priv *bat_priv,
struct batadv_dat_entry *dat_entry = NULL;
bool ret = false;
unsigned short vid;
+ u8 mac[ETH_ALEN];
+ u64 u64_mac;
int err;
if (!atomic_read(&bat_priv->distributed_arp_table))
@@ -1282,8 +1295,11 @@ bool batadv_dat_snoop_incoming_arp_request(struct batadv_priv *bat_priv,
if (!dat_entry)
goto out;
+ u64_mac = atomic64_read(&dat_entry->mac_addr);
+ u64_to_ether_addr(u64_mac, mac);
+
skb_new = batadv_dat_arp_create_reply(bat_priv, ip_dst, ip_src,
- dat_entry->mac_addr, hw_src, vid);
+ mac, hw_src, vid);
if (!skb_new)
goto out;
@@ -1374,6 +1390,8 @@ bool batadv_dat_snoop_incoming_arp_reply(struct batadv_priv *bat_priv,
u8 *hw_src, *hw_dst;
bool dropped = false;
unsigned short vid;
+ u8 mac[ETH_ALEN];
+ u64 u64_mac;
if (!atomic_read(&bat_priv->distributed_arp_table))
goto out;
@@ -1402,11 +1420,17 @@ bool batadv_dat_snoop_incoming_arp_reply(struct batadv_priv *bat_priv,
* this frame would lead to doubled receive of an ARP reply.
*/
dat_entry = batadv_dat_entry_hash_find(bat_priv, ip_src, vid);
- if (dat_entry && batadv_compare_eth(hw_src, dat_entry->mac_addr)) {
- batadv_dbg(BATADV_DBG_DAT, bat_priv, "Doubled ARP reply removed: ARP MSG = [src: %pM-%pI4 dst: %pM-%pI4]; dat_entry: %pM-%pI4\n",
- hw_src, &ip_src, hw_dst, &ip_dst,
- dat_entry->mac_addr, &dat_entry->ip);
- dropped = true;
+ if (dat_entry) {
+ u64_mac = atomic64_read(&dat_entry->mac_addr);
+ u64_to_ether_addr(u64_mac, mac);
+
+ if (batadv_compare_eth(hw_src, mac)) {
+ batadv_dbg(BATADV_DBG_DAT, bat_priv,
+ "Doubled ARP reply removed: ARP MSG = [src: %pM-%pI4 dst: %pM-%pI4]; dat_entry: %pM-%pI4\n",
+ hw_src, &ip_src, hw_dst, &ip_dst,
+ mac, &dat_entry->ip);
+ dropped = true;
+ }
}
/* Update our internal cache with both the IP addresses the node got
diff --git a/net/batman-adv/main.c b/net/batman-adv/main.c
index 87f798701dab..8091619981c0 100644
--- a/net/batman-adv/main.c
+++ b/net/batman-adv/main.c
@@ -458,6 +458,10 @@ int batadv_batman_skb_recv(struct sk_buff *skb, struct net_device *dev,
if (!skb)
goto err_put;
+ /* Merged fragments re-enter here with reused skb metadata. */
+ skb->dev = dev;
+ skb->skb_iif = dev->ifindex;
+
/* packet should hold at least type and version */
if (unlikely(!pskb_may_pull(skb, 2)))
goto err_free;
diff --git a/net/batman-adv/multicast_forw.c b/net/batman-adv/multicast_forw.c
index 6575e7cb2bcf..893cb9ce9971 100644
--- a/net/batman-adv/multicast_forw.c
+++ b/net/batman-adv/multicast_forw.c
@@ -1100,8 +1100,11 @@ static int batadv_mcast_forw_expand_head(struct batadv_priv *bat_priv,
return -EINVAL;
}
- if (skb_headroom(skb) < hdr_size &&
- pskb_expand_head(skb, hdr_size, 0, GFP_ATOMIC) < 0)
+ if (skb_cow(skb, hdr_size) < 0)
+ return -ENOMEM;
+
+ /* batadv_mcast_forw_scrape() + batadv_mcast_forw_packet() require linearized skb */
+ if (skb_linearize(skb) < 0)
return -ENOMEM;
return 0;
diff --git a/net/batman-adv/soft-interface.c b/net/batman-adv/soft-interface.c
index aaebe74c1bc0..47715a612320 100644
--- a/net/batman-adv/soft-interface.c
+++ b/net/batman-adv/soft-interface.c
@@ -271,6 +271,8 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
if (batadv_compare_eth(ethhdr->h_dest, ectp_addr))
goto dropped;
+ batadv_skb_set_priority(skb, 0);
+
gw_mode = atomic_read(&bat_priv->gw.mode);
if (is_multicast_ether_addr(ethhdr->h_dest)) {
/* if gw mode is off, broadcast every packet */
@@ -304,6 +306,9 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
send:
if (do_bcast && !is_broadcast_ether_addr(ethhdr->h_dest)) {
+ /* WARNING batadv_mcast_forw_mode might add more headers
+ * in front of the skb. and might even reallocate the skb
+ */
forw_mode = batadv_mcast_forw_mode(bat_priv, skb, vid,
&mcast_is_routable);
switch (forw_mode) {
@@ -321,8 +326,6 @@ static netdev_tx_t batadv_interface_tx(struct sk_buff *skb,
}
}
- batadv_skb_set_priority(skb, 0);
-
/* ethernet packet should be broadcasted */
if (do_bcast) {
primary_if = batadv_primary_if_get_selected(bat_priv);
diff --git a/net/batman-adv/types.h b/net/batman-adv/types.h
index 0022bee14574..1b8ab18dcf2c 100644
--- a/net/batman-adv/types.h
+++ b/net/batman-adv/types.h
@@ -2365,7 +2365,7 @@ struct batadv_dat_entry {
__be32 ip;
/** @mac_addr: the MAC address associated to the stored IPv4 */
- u8 mac_addr[ETH_ALEN];
+ atomic64_t mac_addr;
/** @vid: the vlan ID associated to this entry */
unsigned short vid;
diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c
index 18aafaa896b2..d2ad9a5dd1ac 100644
--- a/net/bluetooth/hci_conn.c
+++ b/net/bluetooth/hci_conn.c
@@ -287,8 +287,6 @@ static int hci_enhanced_setup_sync(struct hci_dev *hdev, void *data)
struct hci_cp_enhanced_setup_sync_conn cp;
const struct sco_param *param;
- kfree(conn_handle);
-
if (!hci_conn_valid(hdev, conn))
return -ECANCELED;
@@ -457,6 +455,15 @@ static bool hci_setup_sync_conn(struct hci_conn *conn, __u16 handle)
return true;
}
+static void hci_enhanced_setup_sync_destroy(struct hci_dev *hdev, void *data,
+ int err)
+{
+ struct conn_handle_t *conn_handle = data;
+
+ hci_conn_put(conn_handle->conn);
+ kfree(conn_handle);
+}
+
bool hci_setup_sync(struct hci_conn *conn, __u16 handle)
{
int result;
@@ -468,12 +475,15 @@ bool hci_setup_sync(struct hci_conn *conn, __u16 handle)
if (!conn_handle)
return false;
- conn_handle->conn = conn;
+ conn_handle->conn = hci_conn_get(conn);
conn_handle->handle = handle;
result = hci_cmd_sync_queue(conn->hdev, hci_enhanced_setup_sync,
- conn_handle, NULL);
- if (result < 0)
+ conn_handle,
+ hci_enhanced_setup_sync_destroy);
+ if (result < 0) {
+ hci_conn_put(conn);
kfree(conn_handle);
+ }
return result == 0;
}
diff --git a/net/bluetooth/hci_sync.c b/net/bluetooth/hci_sync.c
index cb6889ed9e0d..cc2b0b85426f 100644
--- a/net/bluetooth/hci_sync.c
+++ b/net/bluetooth/hci_sync.c
@@ -540,8 +540,6 @@ static int adv_timeout_expire_sync(struct hci_dev *hdev, void *data)
{
u8 instance = *(u8 *)data;
- kfree(data);
-
hci_clear_adv_instance_sync(hdev, NULL, instance, false);
if (list_empty(&hdev->adv_instances))
@@ -550,6 +548,12 @@ static int adv_timeout_expire_sync(struct hci_dev *hdev, void *data)
return 0;
}
+static void adv_timeout_expire_destroy(struct hci_dev *hdev, void *data,
+ int err)
+{
+ kfree(data);
+}
+
static void adv_timeout_expire(struct work_struct *work)
{
u8 *inst_ptr;
@@ -570,7 +574,9 @@ static void adv_timeout_expire(struct work_struct *work)
goto unlock;
*inst_ptr = hdev->cur_adv_instance;
- hci_cmd_sync_queue(hdev, adv_timeout_expire_sync, inst_ptr, NULL);
+ if (hci_cmd_sync_queue(hdev, adv_timeout_expire_sync, inst_ptr,
+ adv_timeout_expire_destroy) < 0)
+ kfree(inst_ptr);
unlock:
hci_dev_unlock(hdev);
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index 47ef01e2f7e2..d5b1b5a231a1 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1271,7 +1271,12 @@ static void l2cap_sock_kill(struct sock *sk)
BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));
+ /* Take lock to synchronize against access without owning sk->sk_socket,
+ * eg. in l2cap_sock_cleanup_listen(). proto_ops etc. don't need lock.
+ */
+ lock_sock(sk);
l2cap_sock_put_chan(sk);
+ release_sock(sk);
/* Kill poor orphan */
sock_set_flag(sk, SOCK_DEAD);
@@ -1474,14 +1479,10 @@ static void l2cap_sock_cleanup_listen(struct sock *parent)
* establish sk_lock -> conn->lock and invert the established
* conn->lock -> chan->lock -> sk_lock order (lockdep deadlock).
*
- * Instead, briefly take the child sk lock to fetch and pin its chan.
- * l2cap_conn_del() reaches the chan free only via
- * l2cap_chan_del() -> l2cap_sock_teardown_cb(), which itself takes
- * the child sk lock; holding it across l2cap_chan_hold_unless_zero()
- * therefore guarantees the chan cannot be freed while we read and
- * pin it (hold_unless_zero() additionally skips a chan already past
- * its last reference). We then drop the sk lock before taking
- * chan->lock, so sk and chan locks are never held together.
+ * Instead, briefly take the child sk lock to synchronize vs.
+ * l2cap_sock_kill that puts l2cap_pi(sk)->chan. We then drop the sk
+ * lock before taking chan->lock, so sk and chan locks are never held
+ * together.
*
* Since we cannot call l2cap_chan_close() without conn->lock,
* schedule l2cap_chan_timeout to close the channel; it already
@@ -1491,10 +1492,12 @@ static void l2cap_sock_cleanup_listen(struct sock *parent)
struct l2cap_chan *chan;
lock_sock_nested(sk, L2CAP_NESTING_NORMAL);
- chan = l2cap_chan_hold_unless_zero(l2cap_pi(sk)->chan);
+ chan = l2cap_pi(sk)->chan;
+ if (chan)
+ l2cap_chan_hold(chan);
release_sock(sk);
if (!chan) {
- /* l2cap_conn_del() already tearing this child down */
+ /* Already torn down */
sock_put(sk);
continue;
}
@@ -1526,6 +1529,11 @@ static int l2cap_sock_new_connection_cb(struct l2cap_chan *chan,
lock_sock(parent);
+ if (parent->sk_state != BT_LISTEN) {
+ release_sock(parent);
+ return -EINVAL;
+ }
+
/* Check for backlog size */
if (sk_acceptq_is_full(parent)) {
BT_DBG("backlog full %d", parent->sk_ack_backlog);
@@ -1690,10 +1698,14 @@ static void l2cap_sock_state_change_cb(struct l2cap_chan *chan, int state,
if (!sk)
return;
+ lock_sock(sk);
+
sk->sk_state = state;
if (err)
sk->sk_err = err;
+
+ release_sock(sk);
}
static struct sk_buff *l2cap_sock_alloc_skb_cb(struct l2cap_chan *chan,
@@ -1769,6 +1781,8 @@ static void l2cap_sock_resume_cb(struct l2cap_chan *chan)
if (!sk)
return;
+ lock_sock(sk);
+
if (test_and_clear_bit(FLAG_PENDING_SECURITY, &chan->flags)) {
sk->sk_state = BT_CONNECTED;
chan->state = BT_CONNECTED;
@@ -1776,6 +1790,8 @@ static void l2cap_sock_resume_cb(struct l2cap_chan *chan)
clear_bit(BT_SK_SUSPEND, &bt_sk(sk)->flags);
sk->sk_state_change(sk);
+
+ release_sock(sk);
}
static void l2cap_sock_set_shutdown_cb(struct l2cap_chan *chan)
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index 13b1ecb83e0e..cdb2425a16a2 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -2411,11 +2411,15 @@ static int send_cancel(struct hci_dev *hdev, void *data)
mgmt_cmd_complete(cmd->sk, hdev->id, MGMT_OP_MESH_SEND_CANCEL,
0, NULL, 0);
- mgmt_pending_free(cmd);
return 0;
}
+static void send_cancel_destroy(struct hci_dev *hdev, void *data, int err)
+{
+ mgmt_pending_free(data);
+}
+
static int mesh_send_cancel(struct sock *sk, struct hci_dev *hdev,
void *data, u16 len)
{
@@ -2436,7 +2440,8 @@ static int mesh_send_cancel(struct sock *sk, struct hci_dev *hdev,
if (!cmd)
err = -ENOMEM;
else
- err = hci_cmd_sync_queue(hdev, send_cancel, cmd, NULL);
+ err = hci_cmd_sync_queue(hdev, send_cancel, cmd,
+ send_cancel_destroy);
if (err < 0) {
err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_MESH_SEND_CANCEL,
@@ -6152,6 +6157,7 @@ static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
struct mgmt_pending_cmd *cmd;
const u16 max_uuid_count = ((U16_MAX - sizeof(*cp)) / 16);
u16 uuid_count, expected_len;
+ u8 (*uuids)[16] = NULL;
u8 status;
int err;
@@ -6228,12 +6234,10 @@ static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
hdev->discovery.result_filtering = true;
hdev->discovery.type = cp->type;
hdev->discovery.rssi = cp->rssi;
- hdev->discovery.uuid_count = uuid_count;
if (uuid_count > 0) {
- hdev->discovery.uuids = kmemdup(cp->uuids, uuid_count * 16,
- GFP_KERNEL);
- if (!hdev->discovery.uuids) {
+ uuids = kmemdup(cp->uuids, uuid_count * sizeof(*uuids), GFP_KERNEL);
+ if (!uuids) {
err = mgmt_cmd_complete(sk, hdev->id,
MGMT_OP_START_SERVICE_DISCOVERY,
MGMT_STATUS_FAILED,
@@ -6243,6 +6247,11 @@ static int start_service_discovery(struct sock *sk, struct hci_dev *hdev,
}
}
+ spin_lock(&hdev->discovery.lock);
+ hdev->discovery.uuids = uuids;
+ hdev->discovery.uuid_count = uuid_count;
+ spin_unlock(&hdev->discovery.lock);
+
err = hci_cmd_sync_queue(hdev, start_discovery_sync, cmd,
start_discovery_complete);
if (err < 0) {
@@ -10435,6 +10444,7 @@ static bool is_filter_match(struct hci_dev *hdev, s8 rssi, u8 *eir,
!test_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks))))
return false;
+ spin_lock(&hdev->discovery.lock);
if (hdev->discovery.uuid_count != 0) {
/* If a list of UUIDs is provided in filter, results with no
* matching UUID should be dropped.
@@ -10443,9 +10453,12 @@ static bool is_filter_match(struct hci_dev *hdev, s8 rssi, u8 *eir,
hdev->discovery.uuids) &&
!eir_has_uuids(scan_rsp, scan_rsp_len,
hdev->discovery.uuid_count,
- hdev->discovery.uuids))
+ hdev->discovery.uuids)) {
+ spin_unlock(&hdev->discovery.lock);
return false;
+ }
}
+ spin_unlock(&hdev->discovery.lock);
/* If duplicate filtering does not report RSSI changes, then restart
* scanning to ensure updated result with updated RSSI values.
diff --git a/net/bluetooth/msft.c b/net/bluetooth/msft.c
index 0da28910d00b..bb2cd4ea059b 100644
--- a/net/bluetooth/msft.c
+++ b/net/bluetooth/msft.c
@@ -165,6 +165,11 @@ static bool read_supported_features(struct hci_dev *hdev,
if (rp->sub_opcode != MSFT_OP_READ_SUPPORTED_FEATURES)
goto failed;
+ if (skb->len < sizeof(*rp) + rp->evt_prefix_len) {
+ bt_dev_err(hdev, "MSFT event prefix length mismatch");
+ goto failed;
+ }
+
if (rp->evt_prefix_len > 0) {
msft->evt_prefix = kmemdup(rp->evt_prefix, rp->evt_prefix_len,
GFP_KERNEL);
diff --git a/net/bluetooth/rfcomm/core.c b/net/bluetooth/rfcomm/core.c
index 130fe6434084..676989ff7573 100644
--- a/net/bluetooth/rfcomm/core.c
+++ b/net/bluetooth/rfcomm/core.c
@@ -1458,6 +1458,10 @@ static int rfcomm_apply_pn(struct rfcomm_dlc *d, int cr, struct rfcomm_pn *pn)
d->mtu = __le16_to_cpu(pn->mtu);
+ /* MTU 0 causes an infinite loop when fragmenting in sendmsg */
+ if (!d->mtu)
+ d->mtu = RFCOMM_DEFAULT_MTU;
+
if (cr && d->mtu > s->mtu)
d->mtu = s->mtu;
diff --git a/net/bridge/br_netlink_tunnel.c b/net/bridge/br_netlink_tunnel.c
index a713668ea34f..e7eceab5b515 100644
--- a/net/bridge/br_netlink_tunnel.c
+++ b/net/bridge/br_netlink_tunnel.c
@@ -302,7 +302,8 @@ int br_process_vlan_tunnel_info(const struct net_bridge *br,
if (!(tinfo_last->flags & BRIDGE_VLAN_INFO_RANGE_BEGIN))
return -EINVAL;
- if ((tinfo_curr->vid - tinfo_last->vid) !=
+ if (tinfo_curr->vid < tinfo_last->vid ||
+ (tinfo_curr->vid - tinfo_last->vid) !=
(tinfo_curr->tunid - tinfo_last->tunid))
return -EINVAL;
t = tinfo_last->tunid;
diff --git a/net/bridge/br_vlan.c b/net/bridge/br_vlan.c
index 1aa383035294..a61211a09e1e 100644
--- a/net/bridge/br_vlan.c
+++ b/net/bridge/br_vlan.c
@@ -1129,7 +1129,7 @@ int __br_vlan_set_default_pvid(struct net_bridge *br, u16 pvid,
if (err)
goto out;
- if (br_vlan_delete(br, old_pvid))
+ if (!br_vlan_delete(br, old_pvid))
br_vlan_notify(br, NULL, old_pvid, 0, RTM_DELVLAN);
br_vlan_notify(br, NULL, pvid, 0, RTM_NEWVLAN);
__set_bit(0, changed);
@@ -1151,7 +1151,7 @@ int __br_vlan_set_default_pvid(struct net_bridge *br, u16 pvid,
&vlchange, extack);
if (err)
goto err_port;
- if (nbp_vlan_delete(p, old_pvid))
+ if (!nbp_vlan_delete(p, old_pvid))
br_vlan_notify(br, p, old_pvid, 0, RTM_DELVLAN);
br_vlan_notify(p->br, p, pvid, 0, RTM_NEWVLAN);
__set_bit(p->port_no, changed);
diff --git a/net/ceph/messenger_v2.c b/net/ceph/messenger_v2.c
index 476d62bdeab8..8a6cecc19488 100644
--- a/net/ceph/messenger_v2.c
+++ b/net/ceph/messenger_v2.c
@@ -2172,6 +2172,11 @@ static int process_banner_prefix(struct ceph_connection *con)
payload_len = ceph_decode_16(&p);
dout("%s con %p payload_len %d\n", __func__, con, payload_len);
+ if (payload_len < sizeof(u64) + sizeof(u64)) {
+ con->error_msg = "protocol error, bad banner payload len";
+ return -EINVAL;
+ }
+
return prepare_read_banner_payload(con, payload_len);
}
diff --git a/net/core/dev.c b/net/core/dev.c
index e08150baf0aa..de1922f93ef2 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11335,6 +11335,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
free_percpu(dev->pcpu_refcnt);
free_dev:
#endif
+ ref_tracker_dir_exit(&dev->refcnt_tracker);
kvfree(dev);
return NULL;
}
diff --git a/net/core/filter.c b/net/core/filter.c
index 90e396423eb7..bf7c57b06ed9 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -8128,10 +8128,8 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
case BPF_CGROUP_UNIX_SENDMSG:
case BPF_CGROUP_INET4_GETPEERNAME:
case BPF_CGROUP_INET6_GETPEERNAME:
- case BPF_CGROUP_UNIX_GETPEERNAME:
case BPF_CGROUP_INET4_GETSOCKNAME:
case BPF_CGROUP_INET6_GETSOCKNAME:
- case BPF_CGROUP_UNIX_GETSOCKNAME:
return &bpf_sock_addr_setsockopt_proto;
default:
return NULL;
@@ -8151,10 +8149,8 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
case BPF_CGROUP_UNIX_SENDMSG:
case BPF_CGROUP_INET4_GETPEERNAME:
case BPF_CGROUP_INET6_GETPEERNAME:
- case BPF_CGROUP_UNIX_GETPEERNAME:
case BPF_CGROUP_INET4_GETSOCKNAME:
case BPF_CGROUP_INET6_GETSOCKNAME:
- case BPF_CGROUP_UNIX_GETSOCKNAME:
return &bpf_sock_addr_getsockopt_proto;
default:
return NULL;
diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c
index f1641ec8284b..d80899233441 100644
--- a/net/core/lwt_bpf.c
+++ b/net/core/lwt_bpf.c
@@ -160,10 +160,10 @@ static int bpf_output(struct net *net, struct sock *sk, struct sk_buff *skb)
return dst->lwtstate->orig_output(net, sk, skb);
}
-static int xmit_check_hhlen(struct sk_buff *skb, int hh_len)
+static int xmit_check_headroom(struct sk_buff *skb, int hroom)
{
- if (skb_headroom(skb) < hh_len) {
- int nhead = HH_DATA_ALIGN(hh_len - skb_headroom(skb));
+ if (skb_headroom(skb) < hroom) {
+ int nhead = hroom - skb_headroom(skb);
if (pskb_expand_head(skb, nhead, 0, GFP_ATOMIC))
return -ENOMEM;
@@ -275,7 +275,7 @@ static int bpf_xmit(struct sk_buff *skb)
bpf = bpf_lwt_lwtunnel(dst->lwtstate);
if (bpf->xmit.prog) {
- int hh_len = dst->dev->hard_header_len;
+ int hroom = LL_RESERVED_SPACE(dst->dev);
__be16 proto = skb->protocol;
int ret;
@@ -291,9 +291,12 @@ static int bpf_xmit(struct sk_buff *skb)
return -EINVAL;
}
/* If the header was expanded, headroom might be too
- * small for L2 header to come, expand as needed.
+ * small for the L2 header to come, expand as needed.
+ * neigh_hh_output() copies the cached header in
+ * HH_DATA_MOD aligned chunks, so match the reservation
+ * made before LWT xmit.
*/
- ret = xmit_check_hhlen(skb, hh_len);
+ ret = xmit_check_headroom(skb, hroom);
if (unlikely(ret))
return ret;
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index cc03e3fd85fc..613b2b07bad9 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -469,8 +469,12 @@ void net_passive_dec(struct net *net)
if (refcount_dec_and_test(&net->passive)) {
kfree(rcu_access_pointer(net->gen));
+#ifdef CONFIG_REF_TRACKER
/* There should not be any trackers left there. */
ref_tracker_dir_exit(&net->notrefcnt_tracker);
+ if (!net->refcnt_tracker.dead)
+ ref_tracker_dir_exit(&net->refcnt_tracker);
+#endif
/* Wait for an extra rcu_barrier() before final free. */
llist_add(&net->defer_free_list, &defer_free_list);
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 97ad4cc87be8..af4c7e62fe8d 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -484,6 +484,13 @@ static int page_pool_register_dma_index(struct page_pool *pool,
if (unlikely(!PP_DMA_INDEX_BITS))
goto out;
+ /*
+ * Drivers request GFP flags according to both the current context and
+ * the device constraints, but the XArray entry itself is by no mean
+ * used by the device, so remove zone/policy flags.
+ */
+ gfp &= ~(__GFP_DMA | __GFP_DMA32 | __GFP_HIGHMEM | __GFP_COMP);
+
if (in_softirq())
err = xa_alloc(&pool->dma_mapped, &id, netmem_to_page(netmem),
PP_DMA_INDEX_LIMIT, gfp);
@@ -500,29 +507,40 @@ static int page_pool_register_dma_index(struct page_pool *pool,
return err;
}
-static int page_pool_release_dma_index(struct page_pool *pool,
- netmem_ref netmem)
+static void __page_pool_unmap_netmem_dma(struct page_pool *pool,
+ netmem_ref netmem)
{
struct page *old, *page = netmem_to_page(netmem);
unsigned long id;
+ dma_addr_t dma;
- if (unlikely(!PP_DMA_INDEX_BITS))
- return 0;
-
- id = netmem_get_dma_index(netmem);
- if (!id)
- return -1;
+ if (!pool->dma_map)
+ return;
- if (in_softirq())
- old = xa_cmpxchg(&pool->dma_mapped, id, page, NULL, 0);
- else
- old = xa_cmpxchg_bh(&pool->dma_mapped, id, page, NULL, 0);
- if (old != page)
- return -1;
+ /* Cache dma_addr before xa_cmpxchg. The scrub path holds no page ref;
+ * the unref path calls put_page() regardless of cmpxchg outcome, so
+ * after the cmpxchg we cannot safely touch netmem fields.
+ */
+ dma = page_pool_get_dma_addr_netmem(netmem);
- netmem_set_dma_index(netmem, 0);
+ if (likely(PP_DMA_INDEX_BITS)) {
+ id = netmem_get_dma_index(netmem);
+ if (!id)
+ return;
+
+ if (in_softirq())
+ old = xa_cmpxchg(&pool->dma_mapped,
+ id, page, NULL, 0);
+ else
+ old = xa_cmpxchg_bh(&pool->dma_mapped,
+ id, page, NULL, 0);
+ if (old != page)
+ return;
+ }
- return 0;
+ dma_unmap_page_attrs(pool->p.dev, dma,
+ PAGE_SIZE << pool->p.order, pool->p.dma_dir,
+ DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING);
}
static bool page_pool_dma_map(struct page_pool *pool, netmem_ref netmem, gfp_t gfp)
@@ -726,27 +744,19 @@ void page_pool_clear_pp_info(netmem_ref netmem)
netmem_set_pp(netmem, NULL);
}
-static __always_inline void __page_pool_release_page_dma(struct page_pool *pool,
- netmem_ref netmem)
+static __always_inline void __page_pool_release_netmem_dma(struct page_pool *pool,
+ netmem_ref netmem)
{
- dma_addr_t dma;
-
+ /* Caller must hold a page ref: __page_pool_unmap_netmem_dma() is
+ * safe without a ref, but the field clears below require it.
+ */
if (!pool->dma_map)
- /* Always account for inflight pages, even if we didn't
- * map them
- */
return;
- if (page_pool_release_dma_index(pool, netmem))
- return;
-
- dma = page_pool_get_dma_addr_netmem(netmem);
-
- /* When page is unmapped, it cannot be returned to our pool */
- dma_unmap_page_attrs(pool->p.dev, dma,
- PAGE_SIZE << pool->p.order, pool->p.dma_dir,
- DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING);
+ __page_pool_unmap_netmem_dma(pool, netmem);
page_pool_set_dma_addr_netmem(netmem, 0);
+ if (likely(PP_DMA_INDEX_BITS))
+ netmem_set_dma_index(netmem, 0);
}
/* Disconnects a page (from a page_pool). API users can have a need
@@ -763,7 +773,7 @@ void page_pool_return_page(struct page_pool *pool, netmem_ref netmem)
if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops)
put = pool->mp_ops->release_netmem(pool, netmem);
else
- __page_pool_release_page_dma(pool, netmem);
+ __page_pool_release_netmem_dma(pool, netmem);
/* This may be the last page returned, releasing the pool, so
* it is not safe to reference pool afterwards.
@@ -1137,8 +1147,9 @@ static void page_pool_scrub(struct page_pool *pool)
synchronize_net();
}
+ /* No page ref, dma-unmap only. */
xa_for_each(&pool->dma_mapped, id, ptr)
- __page_pool_release_page_dma(pool, page_to_netmem(ptr));
+ __page_pool_unmap_netmem_dma(pool, page_to_netmem((struct page *)ptr));
}
/* No more consumers should exist, but producers could still
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index a3d329b2ccfd..b754125dcaad 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -1967,12 +1967,12 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
int i, order, psize, new_frags;
u32 d_off;
- if (skb_shared(skb) || skb_unclone(skb, gfp_mask))
- return -EINVAL;
-
if (!skb_frags_readable(skb))
return -EFAULT;
+ if (skb_shared(skb) || skb_unclone(skb, gfp_mask))
+ return -EINVAL;
+
if (!num_frags)
goto release;
@@ -3847,10 +3847,9 @@ skb_zerocopy(struct sk_buff *to, struct sk_buff *from, int len, int hlen)
skb_len_add(to, len + plen);
- if (unlikely(skb_orphan_frags(from, GFP_ATOMIC))) {
- skb_tx_error(from);
+ if (unlikely(skb_orphan_frags(from, GFP_ATOMIC)))
return -ENOMEM;
- }
+
skb_zerocopy_clone(to, from, GFP_ATOMIC);
for (i = 0; i < skb_shinfo(from)->nr_frags; i++) {
diff --git a/net/dccp/dccp.h b/net/dccp/dccp.h
index 1f748ed1279d..e4e055493203 100644
--- a/net/dccp/dccp.h
+++ b/net/dccp/dccp.h
@@ -267,7 +267,9 @@ struct sock *dccp_v4_request_recv_sock(const struct sock *sk, struct sk_buff *sk
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req);
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk));
struct sock *dccp_check_req(struct sock *sk, struct sk_buff *skb,
struct request_sock *req);
diff --git a/net/dccp/ipv4.c b/net/dccp/ipv4.c
index 5926159a6f20..b8f5e20368ec 100644
--- a/net/dccp/ipv4.c
+++ b/net/dccp/ipv4.c
@@ -410,7 +410,9 @@ struct sock *dccp_v4_request_recv_sock(const struct sock *sk,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct inet_request_sock *ireq;
struct inet_sock *newinet;
diff --git a/net/dccp/ipv6.c b/net/dccp/ipv6.c
index d6649246188d..c1db85366747 100644
--- a/net/dccp/ipv6.c
+++ b/net/dccp/ipv6.c
@@ -419,7 +419,9 @@ static struct sock *dccp_v6_request_recv_sock(const struct sock *sk,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct inet_request_sock *ireq = inet_rsk(req);
struct ipv6_pinfo *newnp;
@@ -434,7 +436,8 @@ static struct sock *dccp_v6_request_recv_sock(const struct sock *sk,
* v6 mapped
*/
newsk = dccp_v4_request_recv_sock(sk, skb, req, dst,
- req_unhash, own_req);
+ req_unhash, own_req,
+ opt_child_init);
if (newsk == NULL)
return NULL;
diff --git a/net/dccp/minisocks.c b/net/dccp/minisocks.c
index fecc8190064f..49939eba8703 100644
--- a/net/dccp/minisocks.c
+++ b/net/dccp/minisocks.c
@@ -187,7 +187,7 @@ struct sock *dccp_check_req(struct sock *sk, struct sk_buff *skb,
goto drop;
child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
- req, &own_req);
+ req, &own_req, NULL);
if (child) {
child = inet_csk_complete_hashdance(sk, child, req, own_req);
goto out;
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index fddc80c4bd24..988d35866d04 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -819,6 +819,8 @@ int hsr_dev_finalize(struct net_device *hsr_dev, struct net_device *slave[2],
hsr_del_ports(hsr);
err_add_master:
hsr_del_self_node(hsr);
+ hsr_del_nodes(&hsr->node_db);
+ hsr_del_nodes(&hsr->proxy_node_db);
if (unregister)
unregister_netdevice(hsr_dev);
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 91dcdb7567ac..117a424d1fee 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -1315,6 +1315,15 @@ void inet_csk_destroy_sock(struct sock *sk)
}
EXPORT_SYMBOL(inet_csk_destroy_sock);
+void inet_csk_prepare_for_destroy_sock(struct sock *sk)
+{
+ /* The below has to be done to allow calling inet_csk_destroy_sock */
+ if (sk->sk_protocol == IPPROTO_TCP)
+ tcp_clear_sock_ops_cb_flags(sk);
+ sock_set_flag(sk, SOCK_DEAD);
+ this_cpu_inc(*sk->sk_prot->orphan_count);
+}
+
/* This function allows to force a closure of a socket after the call to
* tcp/dccp_create_openreq_child().
*/
diff --git a/net/ipv4/inetpeer.c b/net/ipv4/inetpeer.c
index 9c5ffe3b5f77..c43e632b41c0 100644
--- a/net/ipv4/inetpeer.c
+++ b/net/ipv4/inetpeer.c
@@ -22,6 +22,7 @@
#include <net/ip.h>
#include <net/inetpeer.h>
#include <net/secure_seq.h>
+#include <linux/siphash.h>
/*
* Theory of operations.
@@ -53,6 +54,34 @@
*/
static struct kmem_cache *peer_cachep __ro_after_init;
+static siphash_aligned_key_t inetpeer_hash_key __read_mostly;
+
+static u64 inetpeer_addr_hash(const struct inetpeer_addr *a)
+{
+ net_get_random_once(&inetpeer_hash_key, sizeof(inetpeer_hash_key));
+
+ if (a->family == AF_INET)
+ return siphash_2u32((__force u32)a->a4.addr, a->a4.vif,
+ &inetpeer_hash_key);
+
+ return siphash_4u32((__force u32)a->a6.s6_addr32[0],
+ (__force u32)a->a6.s6_addr32[1],
+ (__force u32)a->a6.s6_addr32[2],
+ (__force u32)a->a6.s6_addr32[3],
+ &inetpeer_hash_key);
+}
+
+static int inetpeer_entry_cmp(u64 dhash,
+ const struct inetpeer_addr *daddr,
+ const struct inet_peer *p)
+{
+ if (dhash < p->hash)
+ return -1;
+ if (dhash > p->hash)
+ return 1;
+
+ return inetpeer_addr_cmp(daddr, &p->daddr);
+}
void inet_peer_base_init(struct inet_peer_base *bp)
{
@@ -86,6 +115,7 @@ void __init inet_initpeers(void)
/* Called with rcu_read_lock() or base->lock held */
static struct inet_peer *lookup(const struct inetpeer_addr *daddr,
+ u64 dhash,
struct inet_peer_base *base,
unsigned int seq,
struct inet_peer *gc_stack[],
@@ -107,7 +137,7 @@ static struct inet_peer *lookup(const struct inetpeer_addr *daddr,
break;
parent = next;
p = rb_entry(parent, struct inet_peer, rb_node);
- cmp = inetpeer_addr_cmp(daddr, &p->daddr);
+ cmp = inetpeer_entry_cmp(dhash, daddr, p);
if (cmp == 0) {
now = jiffies;
if (READ_ONCE(p->dtime) != now)
@@ -177,6 +207,7 @@ struct inet_peer *inet_getpeer(struct inet_peer_base *base,
const struct inetpeer_addr *daddr)
{
struct inet_peer *p, *gc_stack[PEER_MAX_GC];
+ u64 dhash = inetpeer_addr_hash(daddr);
struct rb_node **pp, *parent;
unsigned int gc_cnt, seq;
@@ -184,7 +215,7 @@ struct inet_peer *inet_getpeer(struct inet_peer_base *base,
* Because of a concurrent writer, we might not find an existing entry.
*/
seq = read_seqbegin(&base->lock);
- p = lookup(daddr, base, seq, NULL, &gc_cnt, &parent, &pp);
+ p = lookup(daddr, dhash, base, seq, NULL, &gc_cnt, &parent, &pp);
if (p)
return p;
@@ -196,11 +227,12 @@ struct inet_peer *inet_getpeer(struct inet_peer_base *base,
write_seqlock_bh(&base->lock);
gc_cnt = 0;
- p = lookup(daddr, base, seq, gc_stack, &gc_cnt, &parent, &pp);
+ p = lookup(daddr, dhash, base, seq, gc_stack, &gc_cnt, &parent, &pp);
if (!p) {
p = kmem_cache_alloc(peer_cachep, GFP_ATOMIC);
if (p) {
p->daddr = *daddr;
+ p->hash = dhash;
p->dtime = (__u32)jiffies;
refcount_set(&p->refcnt, 1);
atomic_set(&p->rid, 0);
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index facf0fa7d659..66d5272c5abb 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -202,7 +202,7 @@ struct sock *tcp_get_cookie_sock(struct sock *sk, struct sk_buff *skb,
bool own_req;
child = icsk->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
- NULL, &own_req);
+ NULL, &own_req, NULL);
if (child) {
refcount_set(&req->rsk_refcnt, 1);
sock_rps_save_rxhash(child, skb);
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 947109f01db6..5378ef3b6ab5 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -247,7 +247,7 @@ static struct sock *tcp_fastopen_create_child(struct sock *sk,
bool own_req;
child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
- NULL, &own_req);
+ NULL, &own_req, NULL);
if (!child)
return NULL;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 40b73e97d5fb..2b7c580e3728 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1742,7 +1742,9 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct inet_request_sock *ireq;
bool found_dup_sk = false;
@@ -1798,6 +1800,10 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
}
sk_setup_caps(newsk, dst);
+#if IS_ENABLED(CONFIG_IPV6)
+ if (opt_child_init)
+ opt_child_init(newsk, sk);
+#endif
tcp_ca_openreq_child(newsk, dst);
tcp_sync_mss(newsk, dst_mtu(dst));
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 7149ddb09840..f6657c571fb7 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -859,7 +859,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
* socket is created, wait for troubles.
*/
child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
- req, &own_req);
+ req, &own_req, NULL);
if (!child)
goto listen_overflow;
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 296102ed4a96..d08b50b85875 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3381,7 +3381,7 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)
avail_wnd = cur_mss;
}
- len = cur_mss * segs;
+ len = cur_mss * (tcp_urg_mode(tp) ? 1 : segs);
if (len > avail_wnd) {
len = rounddown(avail_wnd, cur_mss);
if (!len)
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 085fc97c5a5b..84efaf846f07 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1341,11 +1341,48 @@ static void tcp_v6_restore_cb(struct sk_buff *skb)
sizeof(struct inet6_skb_parm));
}
+/* Called from tcp_v4_syn_recv_sock() for v6_mapped children. */
+static void tcp_v6_mapped_child_init(struct sock *newsk, const struct sock *sk)
+{
+ struct inet_sock *newinet = inet_sk(newsk);
+ struct ipv6_pinfo *newnp;
+
+ newinet->pinet6 = newnp = tcp_inet6_sk(newsk);
+
+ memcpy(newnp, tcp_inet6_sk(sk), sizeof(struct ipv6_pinfo));
+
+ newnp->saddr = newsk->sk_v6_rcv_saddr;
+
+ inet_csk(newsk)->icsk_af_ops = &ipv6_mapped;
+ if (sk_is_mptcp(newsk))
+ mptcpv6_handle_mapped(newsk, true);
+ newsk->sk_backlog_rcv = tcp_v4_do_rcv;
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
+ tcp_sk(newsk)->af_specific = &tcp_sock_ipv6_mapped_specific;
+#endif
+
+ newnp->ipv6_mc_list = NULL;
+ newnp->ipv6_ac_list = NULL;
+ newnp->ipv6_fl_list = NULL;
+ newnp->pktoptions = NULL;
+ newnp->opt = NULL;
+
+ /* tcp_v4_syn_recv_sock() has initialized newinet->mc_{index,ttl} */
+ newnp->mcast_oif = newinet->mc_index;
+ newnp->mcast_hops = newinet->mc_ttl;
+
+ newnp->rcv_flowinfo = 0;
+ if (inet6_test_bit(REPFLOW, sk))
+ newnp->flow_label = 0;
+}
+
static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct inet_request_sock *ireq;
struct ipv6_pinfo *newnp;
@@ -1361,60 +1398,10 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
#endif
struct flowi6 fl6;
- if (skb->protocol == htons(ETH_P_IP)) {
- /*
- * v6 mapped
- */
-
- newsk = tcp_v4_syn_recv_sock(sk, skb, req, dst,
- req_unhash, own_req);
-
- if (!newsk)
- return NULL;
-
- inet_sk(newsk)->pinet6 = tcp_inet6_sk(newsk);
-
- newnp = tcp_inet6_sk(newsk);
- newtp = tcp_sk(newsk);
-
- memcpy(newnp, np, sizeof(struct ipv6_pinfo));
-
- newnp->saddr = newsk->sk_v6_rcv_saddr;
-
- inet_csk(newsk)->icsk_af_ops = &ipv6_mapped;
- if (sk_is_mptcp(newsk))
- mptcpv6_handle_mapped(newsk, true);
- newsk->sk_backlog_rcv = tcp_v4_do_rcv;
-#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
- newtp->af_specific = &tcp_sock_ipv6_mapped_specific;
-#endif
-
- newnp->ipv6_mc_list = NULL;
- newnp->ipv6_ac_list = NULL;
- newnp->ipv6_fl_list = NULL;
- newnp->pktoptions = NULL;
- newnp->opt = NULL;
- newnp->mcast_oif = inet_iif(skb);
- newnp->mcast_hops = ip_hdr(skb)->ttl;
- newnp->rcv_flowinfo = 0;
- if (inet6_test_bit(REPFLOW, sk))
- newnp->flow_label = 0;
-
- /*
- * No need to charge this sock to the relevant IPv6 refcnt debug socks count
- * here, tcp_create_openreq_child now does this for us, see the comment in
- * that function for the gory details. -acme
- */
-
- /* It is tricky place. Until this moment IPv4 tcp
- worked with IPv6 icsk.icsk_af_ops.
- Sync it now.
- */
- tcp_sync_mss(newsk, inet_csk(newsk)->icsk_pmtu_cookie);
-
- return newsk;
- }
-
+ if (skb->protocol == htons(ETH_P_IP))
+ return tcp_v4_syn_recv_sock(sk, skb, req, dst,
+ req_unhash, own_req,
+ tcp_v6_mapped_child_init);
ireq = inet_rsk(req);
if (sk_acceptq_is_full(sk))
diff --git a/net/ipv6/xfrm6_input.c b/net/ipv6/xfrm6_input.c
index 699a001ac166..c9e208c57a6b 100644
--- a/net/ipv6/xfrm6_input.c
+++ b/net/ipv6/xfrm6_input.c
@@ -249,7 +249,7 @@ int xfrm6_input_addr(struct sk_buff *skb, xfrm_address_t *daddr,
goto drop;
}
- if (1 + sp->len == XFRM_MAX_DEPTH) {
+ if (sp->len >= XFRM_MAX_DEPTH) {
XFRM_INC_STATS(net, LINUX_MIB_XFRMINBUFFERERROR);
goto drop;
}
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index 5736f06d8143..0340db5174a9 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -5,6 +5,7 @@
* Copyright (c) 2016 Tom Herbert <tom@herbertland.com>
*/
+#include <linux/rcupdate.h>
#include <linux/bpf.h>
#include <linux/errno.h>
#include <linux/errqueue.h>
@@ -389,7 +390,9 @@ static int kcm_parse_func_strparser(struct strparser *strp, struct sk_buff *skb)
struct bpf_prog *prog = psock->bpf_prog;
int res;
+ rcu_read_lock();
res = bpf_prog_run_pin_on_cpu(prog, skb);
+ rcu_read_unlock();
return res;
}
diff --git a/net/mac80211/link.c b/net/mac80211/link.c
index 29813394bbff..7e249e2f93bf 100644
--- a/net/mac80211/link.c
+++ b/net/mac80211/link.c
@@ -340,7 +340,8 @@ static int ieee80211_vif_update_links(struct ieee80211_sub_if_data *sdata,
link = links[link_id];
ieee80211_link_init(sdata, link_id, &link->data, &link->conf);
ieee80211_link_setup(&link->data);
- ieee80211_set_wmm_default(&link->data, true, non_sta);
+ if (sdata->vif.type != NL80211_IFTYPE_AP_VLAN)
+ ieee80211_set_wmm_default(&link->data, true, non_sta);
}
if (new_links == 0)
diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c
index 38549b5236b8..453801b93c32 100644
--- a/net/mac80211/mlme.c
+++ b/net/mac80211/mlme.c
@@ -6660,7 +6660,7 @@ ieee80211_mgd_check_cross_link_csa(struct ieee80211_sub_if_data *sdata,
prof = (void *)sta_profiles[link_id];
prof_elems = ieee802_11_parse_elems(prof->variable +
(prof->sta_info_len - 1),
- len -
+ len - sizeof(*prof) -
(prof->sta_info_len - 1),
false, NULL);
diff --git a/net/mac80211/s1g.c b/net/mac80211/s1g.c
index 2a23e431cbb5..657f281adf10 100644
--- a/net/mac80211/s1g.c
+++ b/net/mac80211/s1g.c
@@ -147,7 +147,7 @@ ieee80211_s1g_tx_twt_setup_fail(struct ieee80211_sub_if_data *sdata,
drv_twt_teardown_request(sdata->local, sdata, &sta->sta, flowid);
- ieee80211_s1g_send_twt_teardown(sdata, mgmt->sa, sdata->vif.addr,
+ ieee80211_s1g_send_twt_teardown(sdata, mgmt->da, sdata->vif.addr,
flowid);
}
diff --git a/net/mac80211/spectmgmt.c b/net/mac80211/spectmgmt.c
index 073ff9e0f397..1bdb241e1fd5 100644
--- a/net/mac80211/spectmgmt.c
+++ b/net/mac80211/spectmgmt.c
@@ -228,7 +228,7 @@ int ieee80211_parse_ch_switch_ie(struct ieee80211_sub_if_data *sdata,
{
enum nl80211_band new_band = current_band;
int new_freq;
- u8 new_chan_no = 0, new_op_class = 0;
+ int new_chan_no = -1, new_op_class = -1;
struct ieee80211_channel *new_chan;
struct cfg80211_chan_def new_chandef = {};
const struct ieee80211_sec_chan_offs_ie *sec_chan_offs;
@@ -257,7 +257,7 @@ int ieee80211_parse_ch_switch_ie(struct ieee80211_sub_if_data *sdata,
new_op_class = ext_chansw_elem->new_operating_class;
if (!ieee80211_operating_class_to_band(new_op_class, &new_band)) {
- new_op_class = 0;
+ new_op_class = -1;
if (!unprot_action)
sdata_info(sdata,
"cannot understand ECSA IE operating class, %d, ignoring\n",
@@ -269,14 +269,14 @@ int ieee80211_parse_ch_switch_ie(struct ieee80211_sub_if_data *sdata,
}
}
- if (!new_op_class && elems->ch_switch_ie) {
+ if (new_op_class < 0 && elems->ch_switch_ie) {
new_chan_no = elems->ch_switch_ie->new_ch_num;
csa_ie->count = elems->ch_switch_ie->count;
csa_ie->mode = elems->ch_switch_ie->mode;
}
/* nothing here we understand */
- if (!new_chan_no)
+ if (new_chan_no < 0)
return 1;
/* Mesh Channel Switch Parameters Element */
@@ -360,7 +360,8 @@ int ieee80211_parse_ch_switch_ie(struct ieee80211_sub_if_data *sdata,
get_unaligned_le16(bwi->info.optional);
} else if (!wide_bw_chansw_ie || !wbcs_elem_to_chandef(wide_bw_chansw_ie,
&new_chandef)) {
- if (!ieee80211_operating_class_to_chandef(new_op_class, new_chan,
+ if (new_op_class < 0 ||
+ !ieee80211_operating_class_to_chandef(new_op_class, new_chan,
&new_chandef))
new_chandef = csa_ie->chanreq.oper;
}
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index 7e23916c9189..8fc30a773c58 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -5051,7 +5051,8 @@ static void ieee80211_set_beacon_cntdwn(struct ieee80211_sub_if_data *sdata,
if (sdata->vif.type == NL80211_IFTYPE_AP && resp) {
u16 *resp_offsets = resp->cntdwn_counter_offsets;
- resp->data[resp_offsets[i]] = count;
+ if (resp_offsets[i])
+ resp->data[resp_offsets[i]] = count;
}
}
}
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index 26ea58691f79..4fc9a5a78876 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -812,7 +812,9 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct mptcp_subflow_context *listener = mptcp_subflow_ctx(sk);
struct mptcp_subflow_request_sock *subflow_req;
@@ -859,7 +861,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,
create_child:
child = listener->icsk_af_ops->syn_recv_sock(sk, skb, req, dst,
- req_unhash, own_req);
+ req_unhash, own_req, opt_child_init);
if (child && *own_req) {
struct mptcp_subflow_context *ctx = mptcp_subflow_ctx(child);
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index df2dc21304ef..346ac2152fa1 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -1180,7 +1180,7 @@ config NETFILTER_XT_MATCH_CGROUP
tristate '"control group" match support'
depends on NETFILTER_ADVANCED
depends on CGROUPS
- select CGROUP_NET_CLASSID
+ select SOCK_CGROUP_DATA
help
Socket/process control group matching allows you to match locally
generated packets based on which net_cls control group processes
diff --git a/net/netfilter/ipvs/ip_vs_ftp.c b/net/netfilter/ipvs/ip_vs_ftp.c
index 206c6700e200..d3be352a5ea8 100644
--- a/net/netfilter/ipvs/ip_vs_ftp.c
+++ b/net/netfilter/ipvs/ip_vs_ftp.c
@@ -103,7 +103,7 @@ static int ip_vs_ftp_get_addrport(char *data, char *data_limit,
char *s, c;
unsigned char p[6];
char edelim;
- __u16 hport;
+ __u32 hport;
int i = 0;
if (data_limit - data < plen) {
@@ -145,7 +145,11 @@ static int ip_vs_ftp_get_addrport(char *data, char *data_limit,
return -1;
c = *data;
if (isdigit(c)) {
- p[i] = p[i]*10 + c - '0';
+ unsigned int val = p[i] * 10 + c - '0';
+
+ if (val > 255)
+ return -1;
+ p[i] = val;
} else if (c == ',' && i < 5) {
i++;
p[i] = 0;
@@ -223,6 +227,8 @@ static int ip_vs_ftp_get_addrport(char *data, char *data_limit,
if (!isdigit(*s))
break;
hport = hport * 10 + *s - '0';
+ if (hport > 65535)
+ return -1;
}
if (s == data_limit || !hport || *s != edelim)
return -1;
diff --git a/net/netfilter/nf_nat_sip.c b/net/netfilter/nf_nat_sip.c
index a1c41defaf22..6b00c81084fe 100644
--- a/net/netfilter/nf_nat_sip.c
+++ b/net/netfilter/nf_nat_sip.c
@@ -267,12 +267,17 @@ static unsigned int nf_nat_sip(struct sk_buff *skb, unsigned int protoff,
SIP_HDR_CONTACT, &in_header,
&matchoff, &matchlen,
&addr, &port) > 0) {
+ int old_len = skb->len, delta;
+
if (!map_addr(skb, protoff, dataoff, dptr, datalen,
matchoff, matchlen,
&addr, port)) {
nf_ct_helper_log(skb, ct, "cannot mangle contact");
return NF_DROP;
}
+
+ delta = (int)skb->len - old_len;
+ coff += delta;
}
if (!map_sip_addr(skb, protoff, dataoff, dptr, datalen, SIP_HDR_FROM) ||
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index 12b9e1970883..2613ebfdc854 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -10271,10 +10271,6 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
return -EAGAIN;
}
- err = nft_flow_rule_offload_commit(net);
- if (err < 0)
- return err;
-
/* 1. Allocate space for next generation rules_gen_X[] */
list_for_each_entry_safe(trans, next, &nft_net->commit_list, list) {
struct nft_table *table = trans->table;
@@ -10299,6 +10295,16 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
}
}
+ /* must be last, so audit and chain blob set up does not leave hardware
+ * in consistent state.
+ */
+ err = nft_flow_rule_offload_commit(net);
+ if (err < 0) {
+ nf_tables_commit_chain_prepare_cancel(net);
+ nf_tables_commit_audit_free(&adl);
+ return err;
+ }
+
/* step 2. Make rules_gen_X visible to packet path */
list_for_each_entry(table, &nft_net->tables, list) {
list_for_each_entry(chain, &table->chains, list)
diff --git a/net/netfilter/xt_cgroup.c b/net/netfilter/xt_cgroup.c
index bfc98719684e..d9943c4f5ee4 100644
--- a/net/netfilter/xt_cgroup.c
+++ b/net/netfilter/xt_cgroup.c
@@ -23,6 +23,8 @@ MODULE_DESCRIPTION("Xtables: process control group matching");
MODULE_ALIAS("ipt_cgroup");
MODULE_ALIAS("ip6t_cgroup");
+#define NET_CLS_CLASSID_INVALID_MSG "classid invalid without net_cls cgroups\n"
+
static int cgroup_mt_check_v0(const struct xt_mtchk_param *par)
{
struct xt_cgroup_info_v0 *info = par->matchinfo;
@@ -30,6 +32,11 @@ static int cgroup_mt_check_v0(const struct xt_mtchk_param *par)
if (info->invert & ~1)
return -EINVAL;
+ if (!IS_ENABLED(CONFIG_CGROUP_NET_CLASSID)) {
+ pr_info_ratelimited(NET_CLS_CLASSID_INVALID_MSG);
+ return -EINVAL;
+ }
+
return 0;
}
@@ -42,7 +49,7 @@ static int cgroup_mt_check_v1(const struct xt_mtchk_param *par)
return -EINVAL;
if (!info->has_path && !info->has_classid) {
- pr_info("xt_cgroup: no path or classid specified\n");
+ pr_info_ratelimited("no path or classid specified\n");
return -EINVAL;
}
@@ -51,6 +58,11 @@ static int cgroup_mt_check_v1(const struct xt_mtchk_param *par)
return -EINVAL;
}
+ if (info->has_classid && !IS_ENABLED(CONFIG_CGROUP_NET_CLASSID)) {
+ pr_info_ratelimited(NET_CLS_CLASSID_INVALID_MSG);
+ return -EINVAL;
+ }
+
info->priv = NULL;
if (info->has_path) {
if (strnlen(info->path, sizeof(info->path)) >= sizeof(info->path))
@@ -77,7 +89,7 @@ static int cgroup_mt_check_v2(const struct xt_mtchk_param *par)
return -EINVAL;
if (!info->has_path && !info->has_classid) {
- pr_info("xt_cgroup: no path or classid specified\n");
+ pr_info_ratelimited("no path or classid specified\n");
return -EINVAL;
}
@@ -86,6 +98,11 @@ static int cgroup_mt_check_v2(const struct xt_mtchk_param *par)
return -EINVAL;
}
+ if (info->has_classid && !IS_ENABLED(CONFIG_CGROUP_NET_CLASSID)) {
+ pr_info_ratelimited(NET_CLS_CLASSID_INVALID_MSG);
+ return -EINVAL;
+ }
+
info->priv = NULL;
if (info->has_path) {
if (strnlen(info->path, sizeof(info->path)) >= sizeof(info->path))
diff --git a/net/netfilter/xt_hl.c b/net/netfilter/xt_hl.c
index c1a70f8f0441..59e93d97b507 100644
--- a/net/netfilter/xt_hl.c
+++ b/net/netfilter/xt_hl.c
@@ -6,6 +6,7 @@
* Hop Limit matching module
* (C) 2001-2002 Maciej Soltysiak <solt@dns.toxicfilms.tv>
*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/ip.h>
#include <linux/ipv6.h>
@@ -22,6 +23,18 @@ MODULE_LICENSE("GPL");
MODULE_ALIAS("ipt_ttl");
MODULE_ALIAS("ip6t_hl");
+static int ttl_mt_check(const struct xt_mtchk_param *par)
+{
+ const struct ipt_ttl_info *info = par->matchinfo;
+
+ if (info->mode > IPT_TTL_GT) {
+ pr_info_ratelimited("Unknown TTL match mode: %d\n", info->mode);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static bool ttl_mt(const struct sk_buff *skb, struct xt_action_param *par)
{
const struct ipt_ttl_info *info = par->matchinfo;
@@ -41,6 +54,18 @@ static bool ttl_mt(const struct sk_buff *skb, struct xt_action_param *par)
return false;
}
+static int hl_mt6_check(const struct xt_mtchk_param *par)
+{
+ const struct ip6t_hl_info *info = par->matchinfo;
+
+ if (info->mode > IP6T_HL_GT) {
+ pr_info_ratelimited("Unknown Hop Limit match mode: %d\n", info->mode);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static bool hl_mt6(const struct sk_buff *skb, struct xt_action_param *par)
{
const struct ip6t_hl_info *info = par->matchinfo;
@@ -65,6 +90,7 @@ static struct xt_match hl_mt_reg[] __read_mostly = {
.name = "ttl",
.revision = 0,
.family = NFPROTO_IPV4,
+ .checkentry = ttl_mt_check,
.match = ttl_mt,
.matchsize = sizeof(struct ipt_ttl_info),
.me = THIS_MODULE,
@@ -73,6 +99,7 @@ static struct xt_match hl_mt_reg[] __read_mostly = {
.name = "hl",
.revision = 0,
.family = NFPROTO_IPV6,
+ .checkentry = hl_mt6_check,
.match = hl_mt6,
.matchsize = sizeof(struct ip6t_hl_info),
.me = THIS_MODULE,
diff --git a/net/nfc/digital_core.c b/net/nfc/digital_core.c
index dae378f1d52b..b18021253224 100644
--- a/net/nfc/digital_core.c
+++ b/net/nfc/digital_core.c
@@ -127,7 +127,7 @@ static void digital_wq_cmd_complete(struct work_struct *work)
mutex_unlock(&ddev->cmd_lock);
- if (!IS_ERR(cmd->resp))
+ if (!IS_ERR_OR_NULL(cmd->resp))
print_hex_dump_debug("DIGITAL RX: ", DUMP_PREFIX_NONE, 16, 1,
cmd->resp->data, cmd->resp->len, false);
diff --git a/net/nfc/llcp_core.c b/net/nfc/llcp_core.c
index db050f4faa5b..7b86faf3f4ce 100644
--- a/net/nfc/llcp_core.c
+++ b/net/nfc/llcp_core.c
@@ -1286,10 +1286,9 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local,
{
struct nfc_llcp_sock *llcp_sock;
u8 dsap, ssap, type, length, tid, sap;
- const u8 *tlv;
- u16 tlv_len, offset;
+ const u8 *tlv, *tlv_end;
const char *service_name;
- size_t service_name_len;
+ int service_name_len;
struct nfc_llcp_sdp_tlv *sdp;
HLIST_HEAD(llc_sdres_list);
size_t sdres_tlvs_len;
@@ -1305,22 +1304,34 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local,
return;
}
+ /*
+ * Walk the SNL TLV list in the linear part of the skb only,
+ * bounded by skb_tail_pointer(). Each TLV needs a two-byte
+ * header (type, length) and its declared length must fit before
+ * the end; this also keeps the walk safe for very short frames.
+ */
tlv = &skb->data[LLCP_HEADER_SIZE];
- tlv_len = skb->len - LLCP_HEADER_SIZE;
- offset = 0;
+ tlv_end = skb_tail_pointer(skb);
sdres_tlvs_len = 0;
- while (offset < tlv_len) {
+ while (tlv + 2 < tlv_end) {
type = tlv[0];
length = tlv[1];
+ if (tlv + 2 + length > tlv_end)
+ break;
+
switch (type) {
case LLCP_TLV_SDREQ:
+ if (length < 1)
+ break;
+
tid = tlv[2];
service_name = (char *) &tlv[3];
service_name_len = length - 1;
- pr_debug("Looking for %.16s\n", service_name);
+ pr_debug("Looking for %.*s\n", service_name_len,
+ service_name);
if (service_name_len == strlen("urn:nfc:sn:sdp") &&
!strncmp(service_name, "urn:nfc:sn:sdp",
@@ -1380,6 +1391,9 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local,
break;
case LLCP_TLV_SDRES:
+ if (length != 2)
+ break;
+
mutex_lock(&local->sdreq_lock);
pr_debug("LLCP_TLV_SDRES: searching tid %d\n", tlv[2]);
@@ -1408,7 +1422,6 @@ static void nfc_llcp_recv_snl(struct nfc_llcp_local *local,
break;
}
- offset += length + 2;
tlv += length + 2;
}
diff --git a/net/nfc/llcp_sock.c b/net/nfc/llcp_sock.c
index 915929cd724f..507447bbceea 100644
--- a/net/nfc/llcp_sock.c
+++ b/net/nfc/llcp_sock.c
@@ -319,14 +319,22 @@ static int nfc_llcp_getsockopt(struct socket *sock, int level, int optname,
if (get_user(len, optlen))
return -EFAULT;
- local = llcp_sock->local;
- if (!local)
- return -ENODEV;
+ if (len < 0)
+ return -EINVAL;
+
+ if (len < sizeof(u32))
+ return -EINVAL;
len = min_t(u32, len, sizeof(u32));
lock_sock(sk);
+ local = llcp_sock->local;
+ if (!local) {
+ release_sock(sk);
+ return -ENODEV;
+ }
+
switch (optname) {
case NFC_LLCP_RW:
rw = llcp_sock->rw > LLCP_MAX_RW ? local->rw : llcp_sock->rw;
diff --git a/net/nfc/nci/data.c b/net/nfc/nci/data.c
index 7a3fb2a397a1..4db36cddf670 100644
--- a/net/nfc/nci/data.c
+++ b/net/nfc/nci/data.c
@@ -46,11 +46,11 @@ void nci_data_exchange_complete(struct nci_dev *ndev, struct sk_buff *skb,
del_timer_sync(&ndev->data_timer);
clear_bit(NCI_DATA_EXCHANGE_TO, &ndev->flags);
- /* Mark the exchange as done before calling the callback.
- * The callback (e.g. rawsock_data_exchange_complete) may
- * want to immediately queue another data exchange.
- */
- clear_bit(NCI_DATA_EXCHANGE, &ndev->flags);
+ /* Claim completion atomically -- both close and rx_work may race here */
+ if (!test_and_clear_bit(NCI_DATA_EXCHANGE, &ndev->flags)) {
+ kfree_skb(skb);
+ return;
+ }
if (cb) {
/* forward skb to nfc core */
diff --git a/net/nfc/nci/rsp.c b/net/nfc/nci/rsp.c
index fccd7ce69bb7..e5d6ce71ca33 100644
--- a/net/nfc/nci/rsp.c
+++ b/net/nfc/nci/rsp.c
@@ -50,11 +50,27 @@ static u8 nci_core_init_rsp_packet_v1(struct nci_dev *ndev,
const struct nci_core_init_rsp_1 *rsp_1 = (void *)skb->data;
const struct nci_core_init_rsp_2 *rsp_2;
+ /* Ensure that the status field can be accessed. */
+ if (skb_headlen(skb) < 1)
+ return NCI_STATUS_SYNTAX_ERROR;
+
pr_debug("status 0x%x\n", rsp_1->status);
if (rsp_1->status != NCI_STATUS_OK)
return rsp_1->status;
+ /* Success response must contain the full fixed-size header */
+ if (skb_headlen(skb) < sizeof(*rsp_1))
+ return NCI_STATUS_SYNTAX_ERROR;
+
+ /* Ensure the variable-length rf_interfaces array and trailing
+ * rsp_2 structure are fully contained within the skb.
+ */
+ if (skb_headlen(skb) < sizeof(*rsp_1) +
+ rsp_1->num_supported_rf_interfaces +
+ sizeof(*rsp_2))
+ return NCI_STATUS_SYNTAX_ERROR;
+
ndev->nfcc_features = __le32_to_cpu(rsp_1->nfcc_features);
ndev->num_supported_rf_interfaces = rsp_1->num_supported_rf_interfaces;
@@ -87,15 +103,25 @@ static u8 nci_core_init_rsp_packet_v2(struct nci_dev *ndev,
const struct sk_buff *skb)
{
const struct nci_core_init_rsp_nci_ver2 *rsp = (void *)skb->data;
- const u8 *supported_rf_interface = rsp->supported_rf_interfaces;
+ const u8 *supported_rf_interface;
u8 rf_interface_idx = 0;
u8 rf_extension_cnt = 0;
+ /* Ensure that the status field can be accessed. */
+ if (skb_headlen(skb) < 1)
+ return NCI_STATUS_SYNTAX_ERROR;
+
pr_debug("status %x\n", rsp->status);
if (rsp->status != NCI_STATUS_OK)
return rsp->status;
+ /* Success response must contain the full fixed-size header */
+ if (skb_headlen(skb) < sizeof(*rsp))
+ return NCI_STATUS_SYNTAX_ERROR;
+
+ supported_rf_interface = rsp->supported_rf_interfaces;
+
ndev->nfcc_features = __le32_to_cpu(rsp->nfcc_features);
ndev->num_supported_rf_interfaces = rsp->num_supported_rf_interfaces;
@@ -104,13 +130,22 @@ static u8 nci_core_init_rsp_packet_v2(struct nci_dev *ndev,
NCI_MAX_SUPPORTED_RF_INTERFACES);
while (rf_interface_idx < ndev->num_supported_rf_interfaces) {
- ndev->supported_rf_interfaces[rf_interface_idx++] = *supported_rf_interface++;
+ /* Each entry: [rf_interface_type (1B)] [ext_count (1B)] [ext...] */
+ if (supported_rf_interface + 2 > skb_tail_pointer(skb))
+ break;
+ ndev->supported_rf_interfaces[rf_interface_idx] = *supported_rf_interface++;
- /* skip rf extension parameters */
rf_extension_cnt = *supported_rf_interface++;
+ if (supported_rf_interface + rf_extension_cnt > skb_tail_pointer(skb))
+ break;
+
+ /* Only count the entry after full validation */
+ rf_interface_idx++;
supported_rf_interface += rf_extension_cnt;
}
+ ndev->num_supported_rf_interfaces = rf_interface_idx;
+
ndev->max_logical_connections = rsp->max_logical_connections;
ndev->max_routing_table_size =
__le16_to_cpu(rsp->max_routing_table_size);
diff --git a/net/openvswitch/conntrack.c b/net/openvswitch/conntrack.c
index fb68aa17e479..3f51f1968654 100644
--- a/net/openvswitch/conntrack.c
+++ b/net/openvswitch/conntrack.c
@@ -932,10 +932,14 @@ static int ovs_ct_check_limit(struct net *net,
const struct ovs_conntrack_info *info)
{
struct ovs_net *ovs_net = net_generic(net, ovs_net_id);
- const struct ovs_ct_limit_info *ct_limit_info = ovs_net->ct_limit_info;
+ const struct ovs_ct_limit_info *ct_limit_info;
u32 per_zone_limit, connections;
u32 conncount_key;
+ ct_limit_info = rcu_dereference(ovs_net->ct_limit_info);
+ if (!ct_limit_info)
+ return 0;
+
conncount_key = info->zone.id;
per_zone_limit = ct_limit_get(ct_limit_info, info->zone.id);
@@ -1584,42 +1588,56 @@ static void __ovs_ct_free_action(struct ovs_conntrack_info *ct_info)
#if IS_ENABLED(CONFIG_NETFILTER_CONNCOUNT)
static int ovs_ct_limit_init(struct net *net, struct ovs_net *ovs_net)
{
+ struct ovs_ct_limit_info *info;
int i, err;
- ovs_net->ct_limit_info = kmalloc(sizeof(*ovs_net->ct_limit_info),
- GFP_KERNEL);
- if (!ovs_net->ct_limit_info)
+ info = kmalloc(sizeof(*info), GFP_KERNEL);
+ if (!info)
return -ENOMEM;
- ovs_net->ct_limit_info->default_limit = OVS_CT_LIMIT_DEFAULT;
- ovs_net->ct_limit_info->limits =
+ info->default_limit = OVS_CT_LIMIT_DEFAULT;
+ info->limits =
kmalloc_array(CT_LIMIT_HASH_BUCKETS, sizeof(struct hlist_head),
GFP_KERNEL);
- if (!ovs_net->ct_limit_info->limits) {
- kfree(ovs_net->ct_limit_info);
+ if (!info->limits) {
+ kfree(info);
return -ENOMEM;
}
for (i = 0; i < CT_LIMIT_HASH_BUCKETS; i++)
- INIT_HLIST_HEAD(&ovs_net->ct_limit_info->limits[i]);
+ INIT_HLIST_HEAD(&info->limits[i]);
- ovs_net->ct_limit_info->data = nf_conncount_init(net, sizeof(u32));
+ info->data = nf_conncount_init(net, sizeof(u32));
- if (IS_ERR(ovs_net->ct_limit_info->data)) {
- err = PTR_ERR(ovs_net->ct_limit_info->data);
- kfree(ovs_net->ct_limit_info->limits);
- kfree(ovs_net->ct_limit_info);
+ if (IS_ERR(info->data)) {
+ err = PTR_ERR(info->data);
+ kfree(info->limits);
+ kfree(info);
pr_err("openvswitch: failed to init nf_conncount %d\n", err);
return err;
}
+ rcu_assign_pointer(ovs_net->ct_limit_info, info);
return 0;
}
-static void ovs_ct_limit_exit(struct net *net, struct ovs_net *ovs_net)
+static void *ovs_ct_limit_exit_start(struct ovs_net *ovs_net)
+{
+ return rcu_replace_pointer(ovs_net->ct_limit_info, NULL,
+ lockdep_ovsl_is_held());
+}
+
+/* The CT limit state must be detached by ovs_ct_limit_exit_start() and an
+ * RCU grace period must elapse before this function runs. The pernet core
+ * guarantees the grace period between the .pre_exit and .exit callbacks.
+ */
+static void ovs_ct_limit_exit_finish(struct net *net, void *data)
{
- const struct ovs_ct_limit_info *info = ovs_net->ct_limit_info;
+ const struct ovs_ct_limit_info *info = data;
int i;
+ if (!info)
+ return;
+
nf_conncount_destroy(net, info->data);
for (i = 0; i < CT_LIMIT_HASH_BUCKETS; ++i) {
struct hlist_head *head = &info->limits[i];
@@ -1627,7 +1645,7 @@ static void ovs_ct_limit_exit(struct net *net, struct ovs_net *ovs_net)
struct hlist_node *next;
hlist_for_each_entry_safe(ct_limit, next, head, hlist_node)
- kfree_rcu(ct_limit, rcu);
+ kfree(ct_limit);
}
kfree(info->limits);
kfree(info);
@@ -1666,12 +1684,13 @@ static bool check_zone_id(int zone_id, u16 *pzone)
return false;
}
-static int ovs_ct_limit_set_zone_limit(struct nlattr *nla_zone_limit,
- struct ovs_ct_limit_info *info)
+static int ovs_ct_limit_set_zone_limit(struct ovs_net *ovs_net,
+ struct nlattr *nla_zone_limit)
{
struct ovs_zone_limit *zone_limit;
- int rem;
+ struct ovs_ct_limit_info *info;
u16 zone;
+ int rem;
rem = NLA_ALIGN(nla_len(nla_zone_limit));
zone_limit = (struct ovs_zone_limit *)nla_data(nla_zone_limit);
@@ -1680,6 +1699,7 @@ static int ovs_ct_limit_set_zone_limit(struct nlattr *nla_zone_limit,
if (unlikely(zone_limit->zone_id ==
OVS_ZONE_LIMIT_DEFAULT_ZONE)) {
ovs_lock();
+ info = ovsl_dereference(ovs_net->ct_limit_info);
info->default_limit = zone_limit->limit;
ovs_unlock();
} else if (unlikely(!check_zone_id(
@@ -1697,6 +1717,7 @@ static int ovs_ct_limit_set_zone_limit(struct nlattr *nla_zone_limit,
ct_limit->limit = zone_limit->limit;
ovs_lock();
+ info = ovsl_dereference(ovs_net->ct_limit_info);
ct_limit_set(info, ct_limit);
ovs_unlock();
}
@@ -1711,12 +1732,13 @@ static int ovs_ct_limit_set_zone_limit(struct nlattr *nla_zone_limit,
return 0;
}
-static int ovs_ct_limit_del_zone_limit(struct nlattr *nla_zone_limit,
- struct ovs_ct_limit_info *info)
+static int ovs_ct_limit_del_zone_limit(struct ovs_net *ovs_net,
+ struct nlattr *nla_zone_limit)
{
struct ovs_zone_limit *zone_limit;
- int rem;
+ struct ovs_ct_limit_info *info;
u16 zone;
+ int rem;
rem = NLA_ALIGN(nla_len(nla_zone_limit));
zone_limit = (struct ovs_zone_limit *)nla_data(nla_zone_limit);
@@ -1725,6 +1747,7 @@ static int ovs_ct_limit_del_zone_limit(struct nlattr *nla_zone_limit,
if (unlikely(zone_limit->zone_id ==
OVS_ZONE_LIMIT_DEFAULT_ZONE)) {
ovs_lock();
+ info = ovsl_dereference(ovs_net->ct_limit_info);
info->default_limit = OVS_CT_LIMIT_DEFAULT;
ovs_unlock();
} else if (unlikely(!check_zone_id(
@@ -1732,6 +1755,7 @@ static int ovs_ct_limit_del_zone_limit(struct nlattr *nla_zone_limit,
OVS_NLERR(true, "zone id is out of range");
} else {
ovs_lock();
+ info = ovsl_dereference(ovs_net->ct_limit_info);
ct_limit_del(info, zone);
ovs_unlock();
}
@@ -1775,6 +1799,7 @@ static int __ovs_ct_limit_get_zone_limit(struct net *net,
return nla_put_nohdr(reply, sizeof(zone_limit), &zone_limit);
}
+/* Called with RCU read lock held. */
static int ovs_ct_limit_get_zone_limit(struct net *net,
struct nlattr *nla_zone_limit,
struct ovs_ct_limit_info *info,
@@ -1798,12 +1823,10 @@ static int ovs_ct_limit_get_zone_limit(struct net *net,
&zone))) {
OVS_NLERR(true, "zone id is out of range");
} else {
- rcu_read_lock();
limit = ct_limit_get(info, zone);
err = __ovs_ct_limit_get_zone_limit(
net, info->data, zone, limit, reply);
- rcu_read_unlock();
if (err)
return err;
}
@@ -1818,6 +1841,7 @@ static int ovs_ct_limit_get_zone_limit(struct net *net,
return 0;
}
+/* Called with RCU read lock held. */
static int ovs_ct_limit_get_all_zone_limit(struct net *net,
struct ovs_ct_limit_info *info,
struct sk_buff *reply)
@@ -1830,19 +1854,16 @@ static int ovs_ct_limit_get_all_zone_limit(struct net *net,
if (err)
return err;
- rcu_read_lock();
for (i = 0; i < CT_LIMIT_HASH_BUCKETS; ++i) {
head = &info->limits[i];
hlist_for_each_entry_rcu(ct_limit, head, hlist_node) {
err = __ovs_ct_limit_get_zone_limit(net, info->data,
ct_limit->zone, ct_limit->limit, reply);
if (err)
- goto exit_err;
+ return err;
}
}
-exit_err:
- rcu_read_unlock();
return err;
}
@@ -1852,7 +1873,6 @@ static int ovs_ct_limit_cmd_set(struct sk_buff *skb, struct genl_info *info)
struct sk_buff *reply;
struct ovs_header *ovs_reply_header;
struct ovs_net *ovs_net = net_generic(sock_net(skb->sk), ovs_net_id);
- struct ovs_ct_limit_info *ct_limit_info = ovs_net->ct_limit_info;
int err;
reply = ovs_ct_limit_cmd_reply_start(info, OVS_CT_LIMIT_CMD_SET,
@@ -1865,8 +1885,8 @@ static int ovs_ct_limit_cmd_set(struct sk_buff *skb, struct genl_info *info)
goto exit_err;
}
- err = ovs_ct_limit_set_zone_limit(a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT],
- ct_limit_info);
+ err = ovs_ct_limit_set_zone_limit(ovs_net,
+ a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT]);
if (err)
goto exit_err;
@@ -1886,7 +1906,6 @@ static int ovs_ct_limit_cmd_del(struct sk_buff *skb, struct genl_info *info)
struct sk_buff *reply;
struct ovs_header *ovs_reply_header;
struct ovs_net *ovs_net = net_generic(sock_net(skb->sk), ovs_net_id);
- struct ovs_ct_limit_info *ct_limit_info = ovs_net->ct_limit_info;
int err;
reply = ovs_ct_limit_cmd_reply_start(info, OVS_CT_LIMIT_CMD_DEL,
@@ -1899,8 +1918,8 @@ static int ovs_ct_limit_cmd_del(struct sk_buff *skb, struct genl_info *info)
goto exit_err;
}
- err = ovs_ct_limit_del_zone_limit(a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT],
- ct_limit_info);
+ err = ovs_ct_limit_del_zone_limit(ovs_net,
+ a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT]);
if (err)
goto exit_err;
@@ -1920,7 +1939,7 @@ static int ovs_ct_limit_cmd_get(struct sk_buff *skb, struct genl_info *info)
struct ovs_header *ovs_reply_header;
struct net *net = sock_net(skb->sk);
struct ovs_net *ovs_net = net_generic(net, ovs_net_id);
- struct ovs_ct_limit_info *ct_limit_info = ovs_net->ct_limit_info;
+ struct ovs_ct_limit_info *ct_limit_info;
int err;
reply = ovs_ct_limit_cmd_reply_start(info, OVS_CT_LIMIT_CMD_GET,
@@ -1934,18 +1953,19 @@ static int ovs_ct_limit_cmd_get(struct sk_buff *skb, struct genl_info *info)
goto exit_err;
}
+ rcu_read_lock();
+ ct_limit_info = rcu_dereference(ovs_net->ct_limit_info);
if (a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT]) {
err = ovs_ct_limit_get_zone_limit(
net, a[OVS_CT_LIMIT_ATTR_ZONE_LIMIT], ct_limit_info,
reply);
- if (err)
- goto exit_err;
} else {
err = ovs_ct_limit_get_all_zone_limit(net, ct_limit_info,
reply);
- if (err)
- goto exit_err;
}
+ rcu_read_unlock();
+ if (err)
+ goto exit_err;
nla_nest_end(reply, nla_reply);
genlmsg_end(reply, ovs_reply_header);
@@ -2020,12 +2040,29 @@ int ovs_ct_init(struct net *net)
return err;
}
-void ovs_ct_exit(struct net *net)
+/* Must be called with ovs_mutex held. Detaches the RCU-protected
+ * ct_limit_info and stores it in ovs_net->ct_limit_exit_data for
+ * ovs_ct_exit_finish() to complete the teardown after an RCU grace period.
+ */
+void ovs_ct_exit_start(struct net *net __maybe_unused)
+{
+#if IS_ENABLED(CONFIG_NETFILTER_CONNCOUNT)
+ struct ovs_net *ovs_net = net_generic(net, ovs_net_id);
+
+ ovs_net->ct_limit_exit_data = ovs_ct_limit_exit_start(ovs_net);
+#endif
+}
+
+/* Completes the CT limit teardown. The pernet core guarantees an RCU
+ * grace period between detaching the state in ovs_ct_exit_start() and
+ * this call, so no RCU readers remain.
+ */
+void ovs_ct_exit_finish(struct net *net)
{
struct ovs_net *ovs_net = net_generic(net, ovs_net_id);
#if IS_ENABLED(CONFIG_NETFILTER_CONNCOUNT)
- ovs_ct_limit_exit(net, ovs_net);
+ ovs_ct_limit_exit_finish(net, ovs_net->ct_limit_exit_data);
#endif
if (ovs_net->xt_label)
diff --git a/net/openvswitch/conntrack.h b/net/openvswitch/conntrack.h
index 317e525c8a11..ab21d032fde3 100644
--- a/net/openvswitch/conntrack.h
+++ b/net/openvswitch/conntrack.h
@@ -14,7 +14,8 @@ enum ovs_key_attr;
#if IS_ENABLED(CONFIG_NF_CONNTRACK)
int ovs_ct_init(struct net *);
-void ovs_ct_exit(struct net *);
+void ovs_ct_exit_start(struct net *net);
+void ovs_ct_exit_finish(struct net *net);
bool ovs_ct_verify(struct net *, enum ovs_key_attr attr);
int ovs_ct_copy_action(struct net *, const struct nlattr *,
const struct sw_flow_key *, struct sw_flow_actions **,
@@ -40,7 +41,8 @@ void ovs_ct_free_action(const struct nlattr *a);
static inline int ovs_ct_init(struct net *net) { return 0; }
-static inline void ovs_ct_exit(struct net *net) { }
+static inline void ovs_ct_exit_start(struct net *net) { }
+static inline void ovs_ct_exit_finish(struct net *net) { }
static inline bool ovs_ct_verify(struct net *net, int attr)
{
diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c
index 7d15be778474..1d11e72cf08d 100644
--- a/net/openvswitch/datapath.c
+++ b/net/openvswitch/datapath.c
@@ -2730,6 +2730,13 @@ static void __net_exit list_vports_from_net(struct net *net, struct net *dnet,
}
}
+static void __net_exit ovs_pre_exit_net(struct net *dnet)
+{
+ ovs_lock();
+ ovs_ct_exit_start(dnet);
+ ovs_unlock();
+}
+
static void __net_exit ovs_exit_net(struct net *dnet)
{
struct datapath *dp, *dp_next;
@@ -2740,7 +2747,7 @@ static void __net_exit ovs_exit_net(struct net *dnet)
ovs_lock();
- ovs_ct_exit(dnet);
+ ovs_ct_exit_finish(dnet);
list_for_each_entry_safe(dp, dp_next, &ovs_net->dps, list_node)
__dp_destroy(dp);
@@ -2764,6 +2771,7 @@ static void __net_exit ovs_exit_net(struct net *dnet)
static struct pernet_operations ovs_net_ops = {
.init = ovs_init_net,
+ .pre_exit = ovs_pre_exit_net,
.exit = ovs_exit_net,
.id = &ovs_net_id,
.size = sizeof(struct ovs_net),
diff --git a/net/openvswitch/datapath.h b/net/openvswitch/datapath.h
index 8453da5507e1..285c578dc30a 100644
--- a/net/openvswitch/datapath.h
+++ b/net/openvswitch/datapath.h
@@ -29,8 +29,8 @@
* datapath.
* @n_hit: Number of received packets for which a matching flow was found in
* the flow table.
- * @n_miss: Number of received packets that had no matching flow in the flow
- * table. The sum of @n_hit and @n_miss is the number of packets that have
+ * @n_missed: Number of received packets that had no matching flow in the flow
+ * table. The sum of @n_hit and @n_missed is the number of packets that have
* been received by the datapath.
* @n_lost: Number of received packets that had no matching flow in the flow
* table that could not be sent to userspace (normally due to an overflow in
@@ -40,6 +40,7 @@
* up per packet.
* @n_cache_hit: The number of received packets that had their mask found using
* the mask cache.
+ * @syncp: Synchronization point for 64bit counters.
*/
struct dp_stats_percpu {
u64 n_hit;
@@ -74,8 +75,10 @@ struct dp_nlsk_pids {
* ovs_mutex and RCU.
* @stats_percpu: Per-CPU datapath statistics.
* @net: Reference to net namespace.
- * @max_headroom: the maximum headroom of all vports in this datapath; it will
+ * @user_features: Bitmap of enabled %OVS_DP_F_* features.
+ * @max_headroom: The maximum headroom of all vports in this datapath; it will
* be used by all the internal vports in this dp.
+ * @meter_tbl: Meter table.
* @upcall_portids: RCU protected 'struct dp_nlsk_pids'.
*
* Context: See the comment on locking at the top of datapath.c for additional
@@ -128,10 +131,13 @@ struct ovs_skb_cb {
#define OVS_CB(skb) ((struct ovs_skb_cb *)(skb)->cb)
/**
- * struct dp_upcall - metadata to include with a packet to send to userspace
+ * struct dp_upcall_info - metadata to include with a packet sent to userspace
* @cmd: One of %OVS_PACKET_CMD_*.
* @userdata: If nonnull, its variable-length value is passed to userspace as
* %OVS_PACKET_ATTR_USERDATA.
+ * @actions: If nonnull, its variable-length value is passed to userspace as
+ * %OVS_PACKET_ATTR_ACTIONS.
+ * @actions_len: The length of the @actions.
* @portid: Netlink portid to which packet should be sent. If @portid is 0
* then no packet is sent and the packet is accounted in the datapath's @n_lost
* counter.
@@ -152,16 +158,22 @@ struct dp_upcall_info {
* struct ovs_net - Per net-namespace data for ovs.
* @dps: List of datapaths to enable dumping them all out.
* Protected by genl_mutex.
+ * @dp_notify_work: A work notifier to handle port unregistering.
+ * @masks_rebalance: A work to periodically optimize flow table caches.
+ * @ct_limit_info: Hash table of conntrack zone connection limits. Protected
+ * by RCU; updates and teardown are serialized by ovs_mutex. May be NULL during
+ * netns teardown.
+ * @ct_limit_exit_data: CT limit state detached at .pre_exit, freed at .exit.
+ * @xt_label: Whether connlables are configured for the network or not.
*/
struct ovs_net {
struct list_head dps;
struct work_struct dp_notify_work;
struct delayed_work masks_rebalance;
#if IS_ENABLED(CONFIG_NETFILTER_CONNCOUNT)
- struct ovs_ct_limit_info *ct_limit_info;
+ struct ovs_ct_limit_info __rcu *ct_limit_info;
+ struct ovs_ct_limit_info *ct_limit_exit_data;
#endif
-
- /* Module reference for configuring conntrack. */
bool xt_label;
};
diff --git a/net/openvswitch/vport.h b/net/openvswitch/vport.h
index 3e71ca8ad8a7..9f67b9dd49f9 100644
--- a/net/openvswitch/vport.h
+++ b/net/openvswitch/vport.h
@@ -97,6 +97,8 @@ struct vport {
* @desired_ifindex: New vport's ifindex.
* @dp: New vport's datapath.
* @port_no: New vport's port number.
+ * @upcall_portids: %OVS_VPORT_ATTR_UPCALL_PID attribute from Netlink message,
+ * %NULL if none was supplied.
*/
struct vport_parms {
const char *name;
@@ -125,6 +127,8 @@ struct vport_parms {
* have any configuration.
* @send: Send a packet on the device.
* zero for dropped packets or negative for error.
+ * @owner: Module that implements this vport type.
+ * @list: List entry in the global list of vport types.
*/
struct vport_ops {
enum ovs_vport_type type;
@@ -144,6 +148,7 @@ struct vport_ops {
/**
* struct vport_upcall_stats_percpu - per-cpu packet upcall statistics for
* a given vport.
+ * @syncp: Synchronization point for 64bit counters.
* @n_success: Number of packets that upcall to userspace succeed.
* @n_fail: Number of packets that upcall to userspace failed.
*/
@@ -164,6 +169,8 @@ void ovs_vport_free(struct vport *);
*
* @vport: vport to access
*
+ * Returns: A void pointer to a private data allocated in the @vport.
+ *
* If a nonzero size was passed in priv_size of vport_alloc() a private data
* area was allocated on creation. This allows that area to be accessed and
* used for any purpose needed by the vport implementer.
@@ -178,6 +185,8 @@ static inline void *vport_priv(const struct vport *vport)
*
* @priv: Start of private data area.
*
+ * Returns: A reference to a vport structure that contains @priv.
+ *
* It is sometimes useful to translate from a pointer to the private data
* area to the vport, such as in the case where the private data pointer is
* the result of a hash table lookup. @priv must point to the start of the
diff --git a/net/phonet/pep.c b/net/phonet/pep.c
index 2a8dee4a327e..4a350c437900 100644
--- a/net/phonet/pep.c
+++ b/net/phonet/pep.c
@@ -1116,7 +1116,7 @@ static int pep_getsockopt(struct sock *sk, int level, int optname,
len = min_t(unsigned int, sizeof(int), len);
if (put_user(len, optlen))
return -EFAULT;
- if (put_user(val, (int __user *) optval))
+ if (copy_to_user(optval, &val, len))
return -EFAULT;
return 0;
}
diff --git a/net/rds/cong.c b/net/rds/cong.c
index 8b689ebbd5b5..c85af56e42e7 100644
--- a/net/rds/cong.c
+++ b/net/rds/cong.c
@@ -256,9 +256,9 @@ void rds_cong_map_updated(struct rds_cong_map *map, uint64_t portmask)
map, &map->m_addr);
rds_stats_inc(s_cong_update_received);
atomic_inc(&rds_cong_generation);
- if (waitqueue_active(&map->m_waitq))
+ if (wq_has_sleeper(&map->m_waitq))
wake_up(&map->m_waitq);
- if (waitqueue_active(&rds_poll_waitq))
+ if (wq_has_sleeper(&rds_poll_waitq))
wake_up_all(&rds_poll_waitq);
if (portmask && !list_empty(&rds_cong_monitor)) {
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 6774edbdd528..a5330dfef412 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -452,7 +452,10 @@ static size_t tcf_action_shared_attrs_size(const struct tc_action *act)
/* TCA_STATS_QUEUE */
+ nla_total_size_64bit(sizeof(struct gnet_stats_queue))
+ nla_total_size(0) /* TCA_ACT_OPTIONS nested */
- + nla_total_size(sizeof(struct tcf_t)); /* TCA_GACT_TM */
+ /* TCA_GACT_TM; actions dump their tcf_t with nla_put_64bit(),
+ * which may emit an extra NLA_PAD attribute.
+ */
+ + nla_total_size_64bit(sizeof(struct tcf_t));
}
static size_t tcf_action_full_attrs_size(size_t sz)
diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c
index 284800b7d3c5..e99b9269ea1a 100644
--- a/net/sched/act_bpf.c
+++ b/net/sched/act_bpf.c
@@ -389,6 +389,31 @@ static void tcf_bpf_cleanup(struct tc_action *act)
tcf_bpf_cfg_cleanup(&tmp);
}
+static size_t tcf_bpf_get_fill_size(const struct tc_action *act)
+{
+ struct tcf_bpf *prog = to_bpf(act);
+ size_t size = nla_total_size(sizeof(struct tc_act_bpf));
+
+ /* bpf_ops and bpf_num_ops are published as separate stores under
+ * tcf_lock, so take it here as tcf_bpf_dump() does.
+ */
+ spin_lock_bh(&prog->tcf_lock);
+ if (tcf_bpf_is_ebpf(prog)) {
+ /* TCA_ACT_BPF_NAME */
+ size += nla_total_size(ACT_BPF_NAME_LEN + 1);
+ size += nla_total_size(sizeof(u32)); /* TCA_ACT_BPF_ID */
+ size += nla_total_size(BPF_TAG_SIZE); /* TCA_ACT_BPF_TAG */
+ } else {
+ size += nla_total_size(sizeof(u16)); /* TCA_ACT_BPF_OPS_LEN */
+ /* TCA_ACT_BPF_OPS */
+ size += nla_total_size(prog->bpf_num_ops *
+ sizeof(struct sock_filter));
+ }
+ spin_unlock_bh(&prog->tcf_lock);
+
+ return size;
+}
+
static struct tc_action_ops act_bpf_ops __read_mostly = {
.kind = "bpf",
.id = TCA_ID_BPF,
@@ -397,6 +422,7 @@ static struct tc_action_ops act_bpf_ops __read_mostly = {
.dump = tcf_bpf_dump,
.cleanup = tcf_bpf_cleanup,
.init = tcf_bpf_init,
+ .get_fill_size = tcf_bpf_get_fill_size,
.size = sizeof(struct tcf_bpf),
};
MODULE_ALIAS_NET_ACT("bpf");
diff --git a/net/sched/act_ct.c b/net/sched/act_ct.c
index ba6d257abdc0..8aed25cad9ee 100644
--- a/net/sched/act_ct.c
+++ b/net/sched/act_ct.c
@@ -994,7 +994,7 @@ TC_INDIRECT_SCOPE int tcf_ct_act(struct sk_buff *skb, const struct tc_action *a,
p = rcu_dereference_bh(c->params);
- retval = READ_ONCE(c->tcf_action);
+ retval = p->action;
commit = p->ct_action & TCA_CT_ACT_COMMIT;
clear = p->ct_action & TCA_CT_ACT_CLEAR;
tmpl = p->tmpl;
@@ -1443,6 +1443,7 @@ static int tcf_ct_init(struct net *net, struct nlattr *nla,
if (err)
goto cleanup;
+ params->action = parm->action;
spin_lock_bh(&c->tcf_lock);
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
params = rcu_replace_pointer(c->params, params,
@@ -1476,8 +1477,8 @@ static void tcf_ct_cleanup(struct tc_action *a)
}
static int tcf_ct_dump_key_val(struct sk_buff *skb,
- void *val, int val_type,
- void *mask, int mask_type,
+ const void *val, int val_type,
+ const void *mask, int mask_type,
int len)
{
int err;
@@ -1498,9 +1499,9 @@ static int tcf_ct_dump_key_val(struct sk_buff *skb,
return 0;
}
-static int tcf_ct_dump_nat(struct sk_buff *skb, struct tcf_ct_params *p)
+static int tcf_ct_dump_nat(struct sk_buff *skb, const struct tcf_ct_params *p)
{
- struct nf_nat_range2 *range = &p->range;
+ const struct nf_nat_range2 *range = &p->range;
if (!(p->ct_action & TCA_CT_ACT_NAT))
return 0;
@@ -1538,7 +1539,8 @@ static int tcf_ct_dump_nat(struct sk_buff *skb, struct tcf_ct_params *p)
return 0;
}
-static int tcf_ct_dump_helper(struct sk_buff *skb, struct nf_conntrack_helper *helper)
+static int tcf_ct_dump_helper(struct sk_buff *skb,
+ const struct nf_conntrack_helper *helper)
{
if (!helper)
return 0;
@@ -1555,9 +1557,8 @@ static inline int tcf_ct_dump(struct sk_buff *skb, struct tc_action *a,
int bind, int ref)
{
unsigned char *b = skb_tail_pointer(skb);
- struct tcf_ct *c = to_ct(a);
- struct tcf_ct_params *p;
-
+ const struct tcf_ct *c = to_ct(a);
+ const struct tcf_ct_params *p;
struct tc_ct opt = {
.index = c->tcf_index,
.refcnt = refcount_read(&c->tcf_refcnt) - ref,
@@ -1565,10 +1566,9 @@ static inline int tcf_ct_dump(struct sk_buff *skb, struct tc_action *a,
};
struct tcf_t t;
- spin_lock_bh(&c->tcf_lock);
- p = rcu_dereference_protected(c->params,
- lockdep_is_held(&c->tcf_lock));
- opt.action = c->tcf_action;
+ rcu_read_lock();
+ p = rcu_dereference(c->params);
+ opt.action = p->action;
if (tcf_ct_dump_key_val(skb,
&p->ct_action, TCA_CT_ACTION,
@@ -1613,11 +1613,11 @@ static inline int tcf_ct_dump(struct sk_buff *skb, struct tc_action *a,
tcf_tm_dump(&t, &c->tcf_tm);
if (nla_put_64bit(skb, TCA_CT_TM, sizeof(t), &t, TCA_CT_PAD))
goto nla_put_failure;
- spin_unlock_bh(&c->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&c->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
@@ -1655,6 +1655,51 @@ static int tcf_ct_offload_act_setup(struct tc_action *act, void *entry_data,
return 0;
}
+static size_t tcf_ct_get_fill_size(const struct tc_action *act)
+{
+ const struct tcf_ct_params *p;
+ size_t size;
+
+ size = nla_total_size(sizeof(struct tc_ct)) /* TCA_CT_PARMS */
+ + nla_total_size(sizeof(u16)); /* TCA_CT_ACTION */
+
+ rcu_read_lock();
+ p = rcu_dereference(to_ct(act)->params);
+
+ if (p->ct_action & TCA_CT_ACT_CLEAR)
+ goto out;
+
+ /* TCA_CT_MARK, TCA_CT_MARK_MASK */
+ if (IS_ENABLED(CONFIG_NF_CONNTRACK_MARK))
+ size += nla_total_size(sizeof(p->mark))
+ + nla_total_size(sizeof(p->mark_mask));
+
+ /* TCA_CT_LABELS, TCA_CT_LABELS_MASK */
+ if (IS_ENABLED(CONFIG_NF_CONNTRACK_LABELS))
+ size += nla_total_size(sizeof(p->labels))
+ + nla_total_size(sizeof(p->labels_mask));
+
+ if (IS_ENABLED(CONFIG_NF_CONNTRACK_ZONES))
+ size += nla_total_size(sizeof(p->zone)); /* TCA_CT_ZONE */
+
+ if (p->ct_action & TCA_CT_ACT_NAT)
+ /* TCA_CT_NAT_IPV6_{MIN,MAX}, the larger of the two address
+ * variants, plus TCA_CT_NAT_PORT_{MIN,MAX}.
+ */
+ size += 2 * nla_total_size(sizeof(struct in6_addr))
+ + 2 * nla_total_size(sizeof(__be16));
+
+ /* TCA_CT_HELPER_{NAME,FAMILY,PROTO} */
+ if (p->helper)
+ size += nla_total_size(NF_CT_HELPER_NAME_LEN)
+ + nla_total_size(sizeof(u8))
+ + nla_total_size(sizeof(u8));
+out:
+ rcu_read_unlock();
+
+ return size;
+}
+
static struct tc_action_ops act_ct_ops = {
.kind = "ct",
.id = TCA_ID_CT,
@@ -1664,6 +1709,7 @@ static struct tc_action_ops act_ct_ops = {
.init = tcf_ct_init,
.cleanup = tcf_ct_cleanup,
.stats_update = tcf_stats_update,
+ .get_fill_size = tcf_ct_get_fill_size,
.offload_act_setup = tcf_ct_offload_act_setup,
.size = sizeof(struct tcf_ct),
};
diff --git a/net/sched/act_ctinfo.c b/net/sched/act_ctinfo.c
index ae571bfe84c7..17abb1d064de 100644
--- a/net/sched/act_ctinfo.c
+++ b/net/sched/act_ctinfo.c
@@ -88,13 +88,11 @@ TC_INDIRECT_SCOPE int tcf_ctinfo_act(struct sk_buff *skb,
struct tcf_ctinfo_params *cp;
struct nf_conn *ct;
int proto, wlen;
- int action;
cp = rcu_dereference_bh(ca->params);
tcf_lastuse_update(&ca->tcf_tm);
tcf_action_update_bstats(&ca->common, skb);
- action = READ_ONCE(ca->tcf_action);
wlen = skb_network_offset(skb);
switch (skb_protocol(skb, true)) {
@@ -141,7 +139,7 @@ TC_INDIRECT_SCOPE int tcf_ctinfo_act(struct sk_buff *skb,
if (thash)
nf_ct_put(ct);
out:
- return action;
+ return cp->action;
}
static const struct nla_policy ctinfo_policy[TCA_CTINFO_MAX + 1] = {
@@ -258,6 +256,8 @@ static int tcf_ctinfo_init(struct net *net, struct nlattr *nla,
cp_new->mode |= CTINFO_MODE_CPMARK;
}
+ cp_new->action = actparm->action;
+
spin_lock_bh(&ci->tcf_lock);
goto_ch = tcf_action_set_ctrlact(*a, actparm->action, goto_ch);
cp_new = rcu_replace_pointer(ci->params, cp_new,
@@ -282,25 +282,24 @@ static int tcf_ctinfo_init(struct net *net, struct nlattr *nla,
static int tcf_ctinfo_dump(struct sk_buff *skb, struct tc_action *a,
int bind, int ref)
{
- struct tcf_ctinfo *ci = to_ctinfo(a);
+ const struct tcf_ctinfo *ci = to_ctinfo(a);
+ unsigned char *b = skb_tail_pointer(skb);
+ const struct tcf_ctinfo_params *cp;
struct tc_ctinfo opt = {
.index = ci->tcf_index,
.refcnt = refcount_read(&ci->tcf_refcnt) - ref,
.bindcnt = atomic_read(&ci->tcf_bindcnt) - bind,
};
- unsigned char *b = skb_tail_pointer(skb);
- struct tcf_ctinfo_params *cp;
struct tcf_t t;
- spin_lock_bh(&ci->tcf_lock);
- cp = rcu_dereference_protected(ci->params,
- lockdep_is_held(&ci->tcf_lock));
+ rcu_read_lock();
+ cp = rcu_dereference(ci->params);
tcf_tm_dump(&t, &ci->tcf_tm);
if (nla_put_64bit(skb, TCA_CTINFO_TM, sizeof(t), &t, TCA_CTINFO_PAD))
goto nla_put_failure;
- opt.action = ci->tcf_action;
+ opt.action = cp->action;
if (nla_put(skb, TCA_CTINFO_ACT, sizeof(opt), &opt))
goto nla_put_failure;
@@ -337,11 +336,11 @@ static int tcf_ctinfo_dump(struct sk_buff *skb, struct tc_action *a,
TCA_CTINFO_PAD))
goto nla_put_failure;
- spin_unlock_bh(&ci->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&ci->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
@@ -356,6 +355,16 @@ static void tcf_ctinfo_cleanup(struct tc_action *a)
kfree_rcu(cp, rcu);
}
+static size_t tcf_ctinfo_get_fill_size(const struct tc_action *act)
+{
+ return nla_total_size(sizeof(struct tc_ctinfo)) /* TCA_CTINFO_ACT */
+ + nla_total_size(sizeof(u16)) /* TCA_CTINFO_ZONE */
+ /* TCA_CTINFO_PARMS_{DSCP_MASK,DSCP_STATEMASK,CPMARK_MASK} */
+ + 3 * nla_total_size(sizeof(u32))
+ /* TCA_CTINFO_STATS_{DSCP_SET,DSCP_ERROR,CPMARK_SET} */
+ + 3 * nla_total_size_64bit(sizeof(u64));
+}
+
static struct tc_action_ops act_ctinfo_ops = {
.kind = "ctinfo",
.id = TCA_ID_CTINFO,
@@ -364,6 +373,7 @@ static struct tc_action_ops act_ctinfo_ops = {
.dump = tcf_ctinfo_dump,
.init = tcf_ctinfo_init,
.cleanup= tcf_ctinfo_cleanup,
+ .get_fill_size = tcf_ctinfo_get_fill_size,
.size = sizeof(struct tcf_ctinfo),
};
MODULE_ALIAS_NET_ACT("ctinfo");
diff --git a/net/sched/act_ife.c b/net/sched/act_ife.c
index 4ad01d4e820d..a90d7959fc43 100644
--- a/net/sched/act_ife.c
+++ b/net/sched/act_ife.c
@@ -878,6 +878,28 @@ TC_INDIRECT_SCOPE int tcf_ife_act(struct sk_buff *skb,
return tcf_ife_decode(skb, a, res);
}
+static size_t tcf_ife_get_fill_size(const struct tc_action *act)
+{
+ struct tcf_ife_info *ife = to_ife(act);
+ const struct tcf_ife_params *p;
+ struct tcf_meta_info *e;
+ size_t size = nla_total_size(sizeof(struct tc_ife)) /* TCA_IFE_PARMS */
+ + nla_total_size(ETH_ALEN) /* TCA_IFE_DMAC */
+ + nla_total_size(ETH_ALEN) /* TCA_IFE_SMAC */
+ + nla_total_size(2) /* TCA_IFE_TYPE */
+ + nla_total_size(0); /* TCA_IFE_METALST */
+
+ rcu_read_lock();
+ p = rcu_dereference(ife->params);
+ if (p) {
+ list_for_each_entry_rcu(e, &p->metalist, metalist)
+ size += nla_total_size(sizeof(u32));
+ }
+ rcu_read_unlock();
+
+ return size;
+}
+
static struct tc_action_ops act_ife_ops = {
.kind = "ife",
.id = TCA_ID_IFE,
@@ -886,6 +908,7 @@ static struct tc_action_ops act_ife_ops = {
.dump = tcf_ife_dump,
.cleanup = tcf_ife_cleanup,
.init = tcf_ife_init,
+ .get_fill_size = tcf_ife_get_fill_size,
.size = sizeof(struct tcf_ife_info),
};
MODULE_ALIAS_NET_ACT("ife");
diff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c
index 5aa91fb145a2..7ce50e303575 100644
--- a/net/sched/act_pedit.c
+++ b/net/sched/act_pedit.c
@@ -622,6 +622,29 @@ static int tcf_pedit_offload_act_setup(struct tc_action *act, void *entry_data,
return 0;
}
+static size_t tcf_pedit_get_fill_size(const struct tc_action *act)
+{
+ const struct tcf_pedit_parms *parms;
+ size_t size;
+
+ rcu_read_lock();
+ parms = rcu_dereference(to_pedit(act)->parms);
+ size = nla_total_size(struct_size_t(struct tc_pedit, keys,
+ parms->tcfp_nkeys));
+ if (parms->tcfp_keys_ex) {
+ /* TCA_PEDIT_KEYS_EX, holding one TCA_PEDIT_KEY_EX nest with a
+ * HTYPE and a CMD attribute per key.
+ */
+ size += nla_total_size(0)
+ + parms->tcfp_nkeys * (nla_total_size(0)
+ + nla_total_size(sizeof(u16))
+ + nla_total_size(sizeof(u16)));
+ }
+ rcu_read_unlock();
+
+ return size;
+}
+
static struct tc_action_ops act_pedit_ops = {
.kind = "pedit",
.id = TCA_ID_PEDIT,
@@ -631,6 +654,7 @@ static struct tc_action_ops act_pedit_ops = {
.dump = tcf_pedit_dump,
.cleanup = tcf_pedit_cleanup,
.init = tcf_pedit_init,
+ .get_fill_size = tcf_pedit_get_fill_size,
.offload_act_setup = tcf_pedit_offload_act_setup,
.size = sizeof(struct tcf_pedit),
};
diff --git a/net/sched/act_police.c b/net/sched/act_police.c
index 81d5f61be17e..1ad8a7948690 100644
--- a/net/sched/act_police.c
+++ b/net/sched/act_police.c
@@ -492,6 +492,17 @@ static int tcf_police_offload_act_setup(struct tc_action *act, void *entry_data,
return 0;
}
+static size_t tcf_police_get_fill_size(const struct tc_action *act)
+{
+ return nla_total_size(sizeof(struct tc_police)) /* TCA_POLICE_TBF */
+ + nla_total_size_64bit(sizeof(u64)) /* TCA_POLICE_RATE64 */
+ + nla_total_size_64bit(sizeof(u64)) /* TCA_POLICE_PEAKRATE64 */
+ + nla_total_size_64bit(sizeof(u64)) /* TCA_POLICE_PKTRATE64 */
+ + nla_total_size_64bit(sizeof(u64)) /* TCA_POLICE_PKTBURST64 */
+ + nla_total_size(sizeof(u32)) /* TCA_POLICE_RESULT */
+ + nla_total_size(sizeof(u32)); /* TCA_POLICE_AVRATE */
+}
+
MODULE_AUTHOR("Alexey Kuznetsov");
MODULE_DESCRIPTION("Policing actions");
MODULE_LICENSE("GPL");
@@ -505,6 +516,7 @@ static struct tc_action_ops act_police_ops = {
.dump = tcf_police_dump,
.init = tcf_police_init,
.cleanup = tcf_police_cleanup,
+ .get_fill_size = tcf_police_get_fill_size,
.offload_act_setup = tcf_police_offload_act_setup,
.size = sizeof(struct tcf_police),
};
diff --git a/net/sched/act_sample.c b/net/sched/act_sample.c
index 2ceb4d141b71..44319a159b55 100644
--- a/net/sched/act_sample.c
+++ b/net/sched/act_sample.c
@@ -315,6 +315,14 @@ static int tcf_sample_offload_act_setup(struct tc_action *act, void *entry_data,
return 0;
}
+static size_t tcf_sample_get_fill_size(const struct tc_action *act)
+{
+ return nla_total_size(sizeof(struct tc_sample)) /* TCA_SAMPLE_PARMS */
+ + nla_total_size(sizeof(u32)) /* TCA_SAMPLE_RATE */
+ + nla_total_size(sizeof(u32)) /* TCA_SAMPLE_TRUNC_SIZE */
+ + nla_total_size(sizeof(u32)); /* TCA_SAMPLE_PSAMPLE_GROUP */
+}
+
static struct tc_action_ops act_sample_ops = {
.kind = "sample",
.id = TCA_ID_SAMPLE,
@@ -324,6 +332,7 @@ static struct tc_action_ops act_sample_ops = {
.dump = tcf_sample_dump,
.init = tcf_sample_init,
.cleanup = tcf_sample_cleanup,
+ .get_fill_size = tcf_sample_get_fill_size,
.get_psample_group = tcf_sample_get_group,
.offload_act_setup = tcf_sample_offload_act_setup,
.size = sizeof(struct tcf_sample),
diff --git a/net/sched/act_skbedit.c b/net/sched/act_skbedit.c
index 2680196b09d0..5450c1293eb5 100644
--- a/net/sched/act_skbedit.c
+++ b/net/sched/act_skbedit.c
@@ -43,13 +43,11 @@ TC_INDIRECT_SCOPE int tcf_skbedit_act(struct sk_buff *skb,
{
struct tcf_skbedit *d = to_skbedit(a);
struct tcf_skbedit_params *params;
- int action;
tcf_lastuse_update(&d->tcf_tm);
bstats_update(this_cpu_ptr(d->common.cpu_bstats), skb);
params = rcu_dereference_bh(d->params);
- action = READ_ONCE(d->tcf_action);
if (params->flags & SKBEDIT_F_PRIORITY)
skb->priority = params->priority;
@@ -85,7 +83,7 @@ TC_INDIRECT_SCOPE int tcf_skbedit_act(struct sk_buff *skb,
}
if (params->flags & SKBEDIT_F_PTYPE)
skb->pkt_type = params->ptype;
- return action;
+ return params->action;
err:
qstats_drop_inc(this_cpu_ptr(d->common.cpu_qstats));
@@ -266,6 +264,7 @@ static int tcf_skbedit_init(struct net *net, struct nlattr *nla,
if (flags & SKBEDIT_F_MASK)
params_new->mask = *mask;
+ params_new->action = parm->action;
spin_lock_bh(&d->tcf_lock);
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
params_new = rcu_replace_pointer(d->params, params_new,
@@ -288,9 +287,9 @@ static int tcf_skbedit_init(struct net *net, struct nlattr *nla,
static int tcf_skbedit_dump(struct sk_buff *skb, struct tc_action *a,
int bind, int ref)
{
+ const struct tcf_skbedit *d = to_skbedit(a);
unsigned char *b = skb_tail_pointer(skb);
- struct tcf_skbedit *d = to_skbedit(a);
- struct tcf_skbedit_params *params;
+ const struct tcf_skbedit_params *params;
struct tc_skbedit opt = {
.index = d->tcf_index,
.refcnt = refcount_read(&d->tcf_refcnt) - ref,
@@ -299,10 +298,9 @@ static int tcf_skbedit_dump(struct sk_buff *skb, struct tc_action *a,
u64 pure_flags = 0;
struct tcf_t t;
- spin_lock_bh(&d->tcf_lock);
- params = rcu_dereference_protected(d->params,
- lockdep_is_held(&d->tcf_lock));
- opt.action = d->tcf_action;
+ rcu_read_lock();
+ params = rcu_dereference(d->params);
+ opt.action = params->action;
if (nla_put(skb, TCA_SKBEDIT_PARMS, sizeof(opt), &opt))
goto nla_put_failure;
@@ -337,12 +335,12 @@ static int tcf_skbedit_dump(struct sk_buff *skb, struct tc_action *a,
tcf_tm_dump(&t, &d->tcf_tm);
if (nla_put_64bit(skb, TCA_SKBEDIT_TM, sizeof(t), &t, TCA_SKBEDIT_PAD))
goto nla_put_failure;
- spin_unlock_bh(&d->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&d->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
diff --git a/net/sched/act_skbmod.c b/net/sched/act_skbmod.c
index dc0229693461..caa3cdfd2d3d 100644
--- a/net/sched/act_skbmod.c
+++ b/net/sched/act_skbmod.c
@@ -27,19 +27,17 @@ TC_INDIRECT_SCOPE int tcf_skbmod_act(struct sk_buff *skb,
struct tcf_result *res)
{
struct tcf_skbmod *d = to_skbmod(a);
- int action, max_edit_len, err;
struct tcf_skbmod_params *p;
+ int max_edit_len, err;
u64 flags;
tcf_lastuse_update(&d->tcf_tm);
bstats_update(this_cpu_ptr(d->common.cpu_bstats), skb);
- action = READ_ONCE(d->tcf_action);
- if (unlikely(action == TC_ACT_SHOT))
+ p = rcu_dereference_bh(d->skbmod_p);
+ if (unlikely(p->action == TC_ACT_SHOT))
goto drop;
- max_edit_len = skb_mac_header_len(skb);
- p = rcu_dereference_bh(d->skbmod_p);
flags = p->flags;
/* tcf_skbmod_init() guarantees "flags" to be one of the following:
@@ -52,14 +50,19 @@ TC_INDIRECT_SCOPE int tcf_skbmod_act(struct sk_buff *skb,
if (flags == SKBMOD_F_ECN) {
switch (skb_protocol(skb, true)) {
case cpu_to_be16(ETH_P_IP):
+ max_edit_len = sizeof(struct iphdr);
+ break;
case cpu_to_be16(ETH_P_IPV6):
- max_edit_len += skb_network_header_len(skb);
+ max_edit_len = sizeof(struct ipv6hdr);
break;
default:
goto out;
}
- } else if (!skb->dev || skb->dev->type != ARPHRD_ETHER) {
- goto out;
+ max_edit_len += skb_network_offset(skb);
+ } else {
+ if (!skb->dev || skb->dev->type != ARPHRD_ETHER)
+ goto out;
+ max_edit_len = ETH_HLEN;
}
err = skb_ensure_writable(skb, max_edit_len);
@@ -85,7 +88,7 @@ TC_INDIRECT_SCOPE int tcf_skbmod_act(struct sk_buff *skb,
INET_ECN_set_ce(skb);
out:
- return action;
+ return p->action;
drop:
qstats_overlimit_inc(this_cpu_ptr(d->common.cpu_qstats));
@@ -193,7 +196,7 @@ static int tcf_skbmod_init(struct net *net, struct nlattr *nla,
}
p->flags = lflags;
-
+ p->action = parm->action;
if (ovr)
spin_lock_bh(&d->tcf_lock);
/* Protected by tcf_lock if overwriting existing action. */
@@ -248,10 +251,9 @@ static int tcf_skbmod_dump(struct sk_buff *skb, struct tc_action *a,
opt.index = d->tcf_index;
opt.refcnt = refcount_read(&d->tcf_refcnt) - ref;
opt.bindcnt = atomic_read(&d->tcf_bindcnt) - bind;
- spin_lock_bh(&d->tcf_lock);
- opt.action = d->tcf_action;
- p = rcu_dereference_protected(d->skbmod_p,
- lockdep_is_held(&d->tcf_lock));
+ rcu_read_lock();
+ p = rcu_dereference(d->skbmod_p);
+ opt.action = p->action;
opt.flags = p->flags;
if (nla_put(skb, TCA_SKBMOD_PARMS, sizeof(opt), &opt))
goto nla_put_failure;
@@ -269,10 +271,10 @@ static int tcf_skbmod_dump(struct sk_buff *skb, struct tc_action *a,
if (nla_put_64bit(skb, TCA_SKBMOD_TM, sizeof(t), &t, TCA_SKBMOD_PAD))
goto nla_put_failure;
- spin_unlock_bh(&d->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&d->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c
index 74e9b5ea76a5..0134def61ddc 100644
--- a/net/sched/act_tunnel_key.c
+++ b/net/sched/act_tunnel_key.c
@@ -29,13 +29,11 @@ TC_INDIRECT_SCOPE int tunnel_key_act(struct sk_buff *skb,
{
struct tcf_tunnel_key *t = to_tunnel_key(a);
struct tcf_tunnel_key_params *params;
- int action;
params = rcu_dereference_bh(t->params);
tcf_lastuse_update(&t->tcf_tm);
tcf_action_update_bstats(&t->common, skb);
- action = READ_ONCE(t->tcf_action);
switch (params->tcft_action) {
case TCA_TUNNEL_KEY_ACT_RELEASE:
@@ -51,7 +49,7 @@ TC_INDIRECT_SCOPE int tunnel_key_act(struct sk_buff *skb,
break;
}
- return action;
+ return params->action;
}
static const struct nla_policy
@@ -538,6 +536,7 @@ static int tunnel_key_init(struct net *net, struct nlattr *nla,
params_new->tcft_action = parm->t_action;
params_new->tcft_enc_metadata = metadata;
+ params_new->action = parm->action;
spin_lock_bh(&t->tcf_lock);
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
params_new = rcu_replace_pointer(t->params, params_new,
@@ -732,10 +731,9 @@ static int tunnel_key_dump(struct sk_buff *skb, struct tc_action *a,
};
struct tcf_t tm;
- spin_lock_bh(&t->tcf_lock);
- params = rcu_dereference_protected(t->params,
- lockdep_is_held(&t->tcf_lock));
- opt.action = t->tcf_action;
+ rcu_read_lock();
+ params = rcu_dereference(t->params);
+ opt.action = params->action;
opt.t_action = params->tcft_action;
if (nla_put(skb, TCA_TUNNEL_KEY_PARMS, sizeof(opt), &opt))
@@ -772,12 +770,12 @@ static int tunnel_key_dump(struct sk_buff *skb, struct tc_action *a,
if (nla_put_64bit(skb, TCA_TUNNEL_KEY_TM, sizeof(tm),
&tm, TCA_TUNNEL_KEY_PAD))
goto nla_put_failure;
- spin_unlock_bh(&t->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&t->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
@@ -837,6 +835,85 @@ static int tcf_tunnel_key_offload_act_setup(struct tc_action *act,
return 0;
}
+static size_t
+tunnel_key_geneve_opts_fill_size(const struct ip_tunnel_info *info)
+{
+ const u8 *src = ip_tunnel_info_opts(info);
+ int len = info->options_len;
+ size_t size = 0;
+
+ while (len > 0) {
+ const struct geneve_opt *opt = (const struct geneve_opt *)src;
+
+ /* TCA_TUNNEL_KEY_ENC_OPT_GENEVE_{CLASS,TYPE,DATA} */
+ size += nla_total_size(2)
+ + nla_total_size(1)
+ + nla_total_size(opt->length * 4);
+
+ len -= sizeof(struct geneve_opt) + opt->length * 4;
+ src += sizeof(struct geneve_opt) + opt->length * 4;
+ }
+
+ return size;
+}
+
+static size_t tunnel_key_opts_fill_size(const struct ip_tunnel_info *info)
+{
+ size_t size;
+
+ if (!info->options_len)
+ return 0;
+
+ /* TCA_TUNNEL_KEY_ENC_OPTS and the per-protocol nest inside it */
+ size = nla_total_size(0) + nla_total_size(0);
+
+ if (test_bit(IP_TUNNEL_GENEVE_OPT_BIT, info->key.tun_flags)) {
+ size += tunnel_key_geneve_opts_fill_size(info);
+ } else if (test_bit(IP_TUNNEL_VXLAN_OPT_BIT, info->key.tun_flags)) {
+ /* TCA_TUNNEL_KEY_ENC_OPT_VXLAN_GBP */
+ size += nla_total_size(sizeof(u32));
+ } else if (test_bit(IP_TUNNEL_ERSPAN_OPT_BIT, info->key.tun_flags)) {
+ /* TCA_TUNNEL_KEY_ENC_OPT_ERSPAN_{VER,INDEX,DIR,HWID} */
+ size += nla_total_size(sizeof(u8))
+ + nla_total_size(sizeof(__be32))
+ + nla_total_size(sizeof(u8))
+ + nla_total_size(sizeof(u8));
+ }
+
+ return size;
+}
+
+static size_t tunnel_key_get_fill_size(const struct tc_action *act)
+{
+ struct tcf_tunnel_key *t = to_tunnel_key(act);
+ const struct tcf_tunnel_key_params *params;
+ /* TCA_TUNNEL_KEY_PARMS */
+ size_t size = nla_total_size(sizeof(struct tc_tunnel_key));
+
+ rcu_read_lock();
+ params = rcu_dereference(t->params);
+ if (params->tcft_action == TCA_TUNNEL_KEY_ACT_SET) {
+ const struct ip_tunnel_info *info =
+ ¶ms->tcft_enc_metadata->u.tun_info;
+
+ /* In dump order: TCA_TUNNEL_KEY_ENC_KEY_ID, the IPv6 address
+ * pair (larger than the IPv4 one), ..._ENC_DST_PORT,
+ * ..._NO_CSUM, ..._NO_FRAG, the options and ..._ENC_{TOS,TTL}.
+ */
+ size += nla_total_size(sizeof(__be32))
+ + 2 * nla_total_size(sizeof(struct in6_addr))
+ + nla_total_size(sizeof(__be16))
+ + nla_total_size(sizeof(u8))
+ + nla_total_size(0)
+ + tunnel_key_opts_fill_size(info)
+ + nla_total_size(sizeof(u8))
+ + nla_total_size(sizeof(u8));
+ }
+ rcu_read_unlock();
+
+ return size;
+}
+
static struct tc_action_ops act_tunnel_key_ops = {
.kind = "tunnel_key",
.id = TCA_ID_TUNNEL_KEY,
@@ -845,6 +922,7 @@ static struct tc_action_ops act_tunnel_key_ops = {
.dump = tunnel_key_dump,
.init = tunnel_key_init,
.cleanup = tunnel_key_release,
+ .get_fill_size = tunnel_key_get_fill_size,
.offload_act_setup = tcf_tunnel_key_offload_act_setup,
.size = sizeof(struct tcf_tunnel_key),
};
diff --git a/net/sched/act_vlan.c b/net/sched/act_vlan.c
index 383bf18b6862..a74621797d69 100644
--- a/net/sched/act_vlan.c
+++ b/net/sched/act_vlan.c
@@ -25,7 +25,6 @@ TC_INDIRECT_SCOPE int tcf_vlan_act(struct sk_buff *skb,
{
struct tcf_vlan *v = to_vlan(a);
struct tcf_vlan_params *p;
- int action;
int err;
u16 tci;
@@ -38,8 +37,6 @@ TC_INDIRECT_SCOPE int tcf_vlan_act(struct sk_buff *skb,
if (skb_at_tc_ingress(skb))
skb_push_rcsum(skb, skb->mac_len);
- action = READ_ONCE(v->tcf_action);
-
p = rcu_dereference_bh(v->vlan_p);
switch (p->tcfv_action) {
@@ -97,7 +94,7 @@ TC_INDIRECT_SCOPE int tcf_vlan_act(struct sk_buff *skb,
skb_pull_rcsum(skb, skb->mac_len);
skb_reset_mac_len(skb);
- return action;
+ return p->action;
drop:
tcf_action_inc_drop_qstats(&v->common);
@@ -255,6 +252,7 @@ static int tcf_vlan_init(struct net *net, struct nlattr *nla,
ETH_ALEN);
}
+ p->action = parm->action;
spin_lock_bh(&v->tcf_lock);
goto_ch = tcf_action_set_ctrlact(*a, parm->action, goto_ch);
p = rcu_replace_pointer(v->vlan_p, p, lockdep_is_held(&v->tcf_lock));
@@ -297,9 +295,9 @@ static int tcf_vlan_dump(struct sk_buff *skb, struct tc_action *a,
};
struct tcf_t t;
- spin_lock_bh(&v->tcf_lock);
- opt.action = v->tcf_action;
- p = rcu_dereference_protected(v->vlan_p, lockdep_is_held(&v->tcf_lock));
+ rcu_read_lock();
+ p = rcu_dereference(v->vlan_p);
+ opt.action = p->action;
opt.v_action = p->tcfv_action;
if (nla_put(skb, TCA_VLAN_PARMS, sizeof(opt), &opt))
goto nla_put_failure;
@@ -325,12 +323,12 @@ static int tcf_vlan_dump(struct sk_buff *skb, struct tc_action *a,
tcf_tm_dump(&t, &v->tcf_tm);
if (nla_put_64bit(skb, TCA_VLAN_TM, sizeof(t), &t, TCA_VLAN_PAD))
goto nla_put_failure;
- spin_unlock_bh(&v->tcf_lock);
+ rcu_read_unlock();
return skb->len;
nla_put_failure:
- spin_unlock_bh(&v->tcf_lock);
+ rcu_read_unlock();
nlmsg_trim(skb, b);
return -1;
}
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index 51a09d555310..53f5cab47e01 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -2242,6 +2242,12 @@ static bool is_ingress_or_clsact(struct tcf_block *block, struct Qdisc *q)
return tcf_block_shared(block) || (q && !!(q->flags & TCQ_F_INGRESS));
}
+enum tcf_tp_insert_state {
+ TP_NOT_CREATED = 0, /* did not create and insert a new tp */
+ TP_CREATED, /* created and inserted a new tp */
+ TP_NOT_OWNED, /* created a proto but failed to insert */
+};
+
static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n,
struct netlink_ext_ack *extack)
{
@@ -2262,12 +2268,12 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n,
unsigned long cl;
void *fh;
int err;
- int tp_created;
+ enum tcf_tp_insert_state tp_state;
bool rtnl_held = false;
u32 flags;
replay:
- tp_created = 0;
+ tp_state = TP_NOT_CREATED;
err = nlmsg_parse_deprecated(n, sizeof(*t), tca, TCA_MAX,
rtm_tca_policy, extack);
@@ -2390,13 +2396,15 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n,
goto errout_tp;
}
- tp_created = 1;
+ tp_state = TP_CREATED;
tp = tcf_chain_tp_insert_unique(chain, tp_new, protocol, prio,
rtnl_held);
if (IS_ERR(tp)) {
err = PTR_ERR(tp);
goto errout_tp;
}
+ if (tp != tp_new)
+ tp_state = TP_NOT_OWNED;
} else {
mutex_unlock(&chain->filter_chain_lock);
}
@@ -2450,13 +2458,13 @@ static int tc_new_tfilter(struct sk_buff *skb, struct nlmsghdr *n,
}
errout:
- if (err && tp_created)
+ if (err && tp_state == TP_CREATED)
tcf_chain_tp_delete_empty(chain, tp, rtnl_held, NULL);
errout_tp:
if (chain) {
if (tp && !IS_ERR(tp))
tcf_proto_put(tp, rtnl_held, NULL);
- if (!tp_created)
+ if (tp_state == TP_NOT_CREATED)
tcf_chain_put(chain);
}
tcf_block_release(q, block, rtnl_held);
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index e7d87e81bb92..1d76f7853404 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -608,8 +608,11 @@ void __qdisc_calculate_pkt_len(struct sk_buff *skb,
pkt_len <<= stab->szopts.size_log;
out:
- if (unlikely(pkt_len < 1))
- pkt_len = 1;
+ /* A size table can inflate qdisc_pkt_len() beyond any real packet
+ * (via overhead, the data table, or size_log); cap it so deficit
+ * schedulers such as DRR/ETS terminate their refill loops.
+ */
+ pkt_len = clamp_t(int, pkt_len, 1, QDISC_PKT_LEN_MAX);
qdisc_skb_cb(skb)->pkt_len = pkt_len;
}
diff --git a/net/sched/sch_codel.c b/net/sched/sch_codel.c
index 29f5a98a3bab..d6819c4e4198 100644
--- a/net/sched/sch_codel.c
+++ b/net/sched/sch_codel.c
@@ -203,7 +203,7 @@ static int codel_init(struct Qdisc *sch, struct nlattr *opt,
codel_params_init(&q->params);
codel_vars_init(&q->vars);
codel_stats_init(&q->stats);
- q->params.mtu = psched_mtu(qdisc_dev(sch));
+ q->params.mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 256, 1 << 20);
if (opt) {
int err = codel_change(sch, opt, extack);
diff --git a/net/sched/sch_fifo.c b/net/sched/sch_fifo.c
index e6bfd39ff339..1b6388d50967 100644
--- a/net/sched/sch_fifo.c
+++ b/net/sched/sch_fifo.c
@@ -19,7 +19,7 @@
static int bfifo_enqueue(struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff **to_free)
{
- if (likely(sch->qstats.backlog + qdisc_pkt_len(skb) <=
+ if (likely((u64)sch->qstats.backlog + qdisc_pkt_len(skb) <=
READ_ONCE(sch->limit)))
return qdisc_enqueue_tail(skb, sch);
diff --git a/net/sched/sch_fq.c b/net/sched/sch_fq.c
index 682daf79af37..6b85bb2a4f8a 100644
--- a/net/sched/sch_fq.c
+++ b/net/sched/sch_fq.c
@@ -909,7 +909,7 @@ static int fq_resize(struct Qdisc *sch, u32 log)
}
static const struct netlink_range_validation iq_range = {
- .max = INT_MAX,
+ .max = 1 << 20,
};
static const struct nla_policy fq_policy[TCA_FQ_MAX + 1] = {
@@ -1034,14 +1034,10 @@ static int fq_change(struct Qdisc *sch, struct nlattr *opt,
nla_get_u32(tb[TCA_FQ_FLOW_PLIMIT]));
if (tb[TCA_FQ_QUANTUM]) {
- u32 quantum = nla_get_u32(tb[TCA_FQ_QUANTUM]);
+ u32 quantum = clamp_t(u32, nla_get_u32(tb[TCA_FQ_QUANTUM]),
+ 256, 1 << 20);
- if (quantum > 0 && quantum <= (1 << 20)) {
- WRITE_ONCE(q->quantum, quantum);
- } else {
- NL_SET_ERR_MSG_MOD(extack, "invalid quantum");
- err = -EINVAL;
- }
+ WRITE_ONCE(q->quantum, quantum);
}
if (tb[TCA_FQ_INITIAL_QUANTUM])
@@ -1141,12 +1137,14 @@ static int fq_init(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
struct fq_sched_data *q = qdisc_priv(sch);
+ u32 mtu;
int i, err;
sch->limit = 10000;
q->flow_plimit = 100;
- q->quantum = 2 * psched_mtu(qdisc_dev(sch));
- q->initial_quantum = 10 * psched_mtu(qdisc_dev(sch));
+ mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 1, 1 << 20);
+ q->quantum = clamp_t(u32, 2 * mtu, 256, 1 << 20);
+ q->initial_quantum = min_t(u32, 10 * mtu, 1 << 20);
q->flow_refill_delay = msecs_to_jiffies(40);
q->flow_max_rate = ~0UL;
q->time_next_delayed_flow = ~0ULL;
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index f445507eff7a..dc07e77df5a8 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -504,6 +504,7 @@ static int fq_codel_init(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
struct fq_codel_sched_data *q = qdisc_priv(sch);
+ u32 mtu;
int i;
int err;
@@ -511,13 +512,14 @@ static int fq_codel_init(struct Qdisc *sch, struct nlattr *opt,
q->flows_cnt = 1024;
q->memory_limit = 32 << 20; /* 32 MBytes */
q->drop_batch_size = 64;
- q->quantum = psched_mtu(qdisc_dev(sch));
+ mtu = clamp_t(u32, psched_mtu(qdisc_dev(sch)), 256, FQ_CODEL_QUANTUM_MAX);
+ q->quantum = mtu;
INIT_LIST_HEAD(&q->new_flows);
INIT_LIST_HEAD(&q->old_flows);
codel_params_init(&q->cparams);
codel_stats_init(&q->cstats);
q->cparams.ecn = true;
- q->cparams.mtu = psched_mtu(qdisc_dev(sch));
+ q->cparams.mtu = mtu;
if (opt) {
err = fq_codel_change(sch, opt, extack);
diff --git a/net/sched/sch_fq_pie.c b/net/sched/sch_fq_pie.c
index ceba15465662..bae503d0a39f 100644
--- a/net/sched/sch_fq_pie.c
+++ b/net/sched/sch_fq_pie.c
@@ -423,7 +423,8 @@ static int fq_pie_init(struct Qdisc *sch, struct nlattr *opt,
pie_params_init(&q->p_params);
sch->limit = 10 * 1024;
q->p_params.limit = sch->limit;
- q->quantum = psched_mtu(qdisc_dev(sch));
+ q->quantum = clamp_t(u32, psched_mtu(qdisc_dev(sch)),
+ 256, 1 << 20);
q->sch = sch;
q->ecn_prob = 10;
q->flows_cnt = 1024;
diff --git a/net/sched/sch_gred.c b/net/sched/sch_gred.c
index 43b0343a7cd0..53b348b83c91 100644
--- a/net/sched/sch_gred.c
+++ b/net/sched/sch_gred.c
@@ -179,7 +179,7 @@ static int gred_enqueue(struct sk_buff *skb, struct Qdisc *sch,
* if no default DP has been configured. This
* allows for DP flows to be left untouched.
*/
- if (likely(sch->qstats.backlog + qdisc_pkt_len(skb) <=
+ if (likely((u64)sch->qstats.backlog + qdisc_pkt_len(skb) <=
sch->limit))
return qdisc_enqueue_tail(skb, sch);
else
@@ -244,7 +244,7 @@ static int gred_enqueue(struct sk_buff *skb, struct Qdisc *sch,
break;
}
- if (gred_backlog(t, q, sch) + qdisc_pkt_len(skb) <= q->limit) {
+ if ((u64)gred_backlog(t, q, sch) + qdisc_pkt_len(skb) <= q->limit) {
q->backlog += qdisc_pkt_len(skb);
return qdisc_enqueue_tail(skb, sch);
}
diff --git a/net/sched/sch_hhf.c b/net/sched/sch_hhf.c
index f9d2cf7533fa..0c406d11b22d 100644
--- a/net/sched/sch_hhf.c
+++ b/net/sched/sch_hhf.c
@@ -621,6 +621,10 @@ static int hhf_init(struct Qdisc *sch, struct nlattr *opt,
q->hhf_evict_timeout = HZ; /* 1 sec */
q->hhf_non_hh_weight = 2;
+ if ((int)q->quantum <= 0 ||
+ (u64)q->quantum * q->hhf_non_hh_weight > INT_MAX)
+ q->quantum = 256;
+
if (opt) {
int err = hhf_change(sch, opt, extack);
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 49f4599be14b..36cc9a343b54 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -223,6 +223,7 @@ static struct htb_class *htb_classify(struct sk_buff *skb, struct Qdisc *sch,
struct htb_class *cl;
struct tcf_result res;
struct tcf_proto *tcf;
+ unsigned int hops = 0;
int result;
/* allow to select class by setting skb->priority to valid classid;
@@ -265,6 +266,10 @@ static struct htb_class *htb_classify(struct sk_buff *skb, struct Qdisc *sch,
if (!cl->level)
return cl; /* we hit leaf; return it */
+ if (++hops > TC_HTB_MAXDEPTH) {
+ pr_warn_ratelimited("htb: classify loop detected, dropping packet\n");
+ return NULL;
+ }
/* we have got inner class; apply inner filter chain */
tcf = rcu_dereference_bh(cl->filter_list);
}
@@ -632,13 +637,11 @@ static int htb_enqueue(struct sk_buff *skb, struct Qdisc *sch,
} else {
return qdisc_drop(skb, sch, to_free);
}
-#ifdef CONFIG_NET_CLS_ACT
} else if (!cl) {
if (ret & __NET_XMIT_BYPASS)
qdisc_qstats_drop(sch);
__qdisc_drop(skb, to_free);
return ret;
-#endif
} else if ((ret = qdisc_enqueue(skb, cl->leaf.q,
to_free)) != NET_XMIT_SUCCESS) {
if (net_xmit_drop_count(ret)) {
diff --git a/net/sched/sch_plug.c b/net/sched/sch_plug.c
index cefb65201e17..b60ddfee6a68 100644
--- a/net/sched/sch_plug.c
+++ b/net/sched/sch_plug.c
@@ -89,7 +89,7 @@ static int plug_enqueue(struct sk_buff *skb, struct Qdisc *sch,
{
struct plug_sched_data *q = qdisc_priv(sch);
- if (likely(sch->qstats.backlog + skb->len <= q->limit)) {
+ if (likely((u64)sch->qstats.backlog + skb->len <= q->limit)) {
if (!q->unplug_indefinite)
q->pkts_current_epoch++;
return qdisc_enqueue_tail(skb, sch);
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index 11a7d5a25d6b..d3ac501e04c3 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -796,7 +796,8 @@ static int sfq_init(struct Qdisc *sch, struct nlattr *opt,
q->tail = NULL;
q->divisor = SFQ_DEFAULT_HASH_DIVISOR;
q->maxflows = SFQ_DEFAULT_FLOWS;
- q->quantum = psched_mtu(qdisc_dev(sch));
+ q->quantum = clamp_t(u32, psched_mtu(qdisc_dev(sch)),
+ 256, 1 << 20);
q->perturb_period = 0;
get_random_bytes(&q->perturbation, sizeof(q->perturbation));
diff --git a/net/sched/sch_teql.c b/net/sched/sch_teql.c
index 4d463ba5ce72..eb424b20027e 100644
--- a/net/sched/sch_teql.c
+++ b/net/sched/sch_teql.c
@@ -377,6 +377,7 @@ static netdev_tx_t teql_master_xmit(struct sk_buff *skb, struct net_device *dev)
nores = 1;
break;
}
+ skb->dev = dev;
__skb_pull(skb, skb_network_offset(skb));
} while ((q = rcu_dereference(NEXT_SLAVE(q))) != start);
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 7da93457e19a..4005cb1c0497 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -125,7 +125,9 @@ static struct sock *smc_tcp_syn_recv_sock(const struct sock *sk,
struct request_sock *req,
struct dst_entry *dst,
struct request_sock *req_unhash,
- bool *own_req)
+ bool *own_req,
+ void (*opt_child_init)(struct sock *newsk,
+ const struct sock *sk))
{
struct smc_sock *smc;
struct sock *child;
@@ -150,7 +152,7 @@ static struct sock *smc_tcp_syn_recv_sock(const struct sock *sk,
/* passthrough to original syn recv sock fct */
child = smc->ori_af_ops->syn_recv_sock(sk, skb, req, dst, req_unhash,
- own_req);
+ own_req, opt_child_init);
/* child must not inherit smc or its ops */
if (child) {
rcu_assign_sk_user_data(child, NULL);
@@ -408,13 +410,13 @@ void smc_sk_init(struct net *net, struct sock *sk, int protocol)
"sk_lock-AF_SMC", &smc_key);
spin_lock_init(&smc->accept_q_lock);
spin_lock_init(&smc->conn.send_lock);
- sk->sk_prot->hash(sk);
mutex_init(&smc->clcsock_release_lock);
smc_init_saved_callbacks(smc);
smc->limit_smc_hs = net->smc.limit_smc_hs;
smc->use_fallback = false; /* assume rdma capability first */
smc->fallback_rsn = 0;
smc_close_init(smc);
+ sk->sk_prot->hash(sk);
}
static struct sock *smc_sock_alloc(struct net *net, struct socket *sock,
diff --git a/net/smc/smc_inet.c b/net/smc/smc_inet.c
index a94084b4a498..520b666fdd8f 100644
--- a/net/smc/smc_inet.c
+++ b/net/smc/smc_inet.c
@@ -15,13 +15,16 @@
#include "smc_inet.h"
#include "smc.h"
+#include "smc_close.h"
static int smc_inet_init_sock(struct sock *sk);
+static void smc_inet_destroy_sock(struct sock *sk);
static struct proto smc_inet_prot = {
.name = "INET_SMC",
.owner = THIS_MODULE,
.init = smc_inet_init_sock,
+ .destroy = smc_inet_destroy_sock,
.hash = smc_hash_sk,
.unhash = smc_unhash_sk,
.release_cb = smc_release_cb,
@@ -68,6 +71,7 @@ static struct proto smc_inet6_prot = {
.name = "INET6_SMC",
.owner = THIS_MODULE,
.init = smc_inet_init_sock,
+ .destroy = smc_inet_destroy_sock,
.hash = smc_hash_sk,
.unhash = smc_unhash_sk,
.release_cb = smc_release_cb,
@@ -116,6 +120,18 @@ static int smc_inet_init_sock(struct sock *sk)
return smc_create_clcsk(net, sk, sk->sk_family);
}
+static void smc_inet_destroy_sock(struct sock *sk)
+{
+ /* The sock is hashed and smc_diag dumps dereference smc->clcsock
+ * without clcsock_release_lock, while sk_common_release() calls
+ * .destroy before .unhash. Unhash first, as __smc_release() does,
+ * so no dump can observe the clcsock being released; the second
+ * unhash is a no-op.
+ */
+ sk->sk_prot->unhash(sk);
+ smc_clcsock_release(smc_sk(sk));
+}
+
int __init smc_inet_init(void)
{
int rc;
diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c
index 149d63cff667..1932983ef763 100644
--- a/net/smc/smc_llc.c
+++ b/net/smc/smc_llc.c
@@ -320,6 +320,7 @@ int smc_llc_flow_initiate(struct smc_link_group *lgr,
void smc_llc_flow_stop(struct smc_link_group *lgr, struct smc_llc_flow *flow)
{
spin_lock_bh(&lgr->llc_flow_lock);
+ smc_llc_flow_qentry_del(flow);
memset(flow, 0, sizeof(*flow));
flow->type = SMC_LLC_FLOW_NONE;
spin_unlock_bh(&lgr->llc_flow_lock);
@@ -1902,6 +1903,8 @@ static void smc_llc_event_handler(struct smc_llc_qentry *qentry)
if (lgr->llc_flow_lcl.type ==
SMC_LLC_FLOW_REQ_ADD_LINK) {
/* server started add_link processing */
+ /* free any qentry stashed in REQ_ADD_LINK state */
+ smc_llc_flow_qentry_del(&lgr->llc_flow_lcl);
lgr->llc_flow_lcl.type = SMC_LLC_FLOW_ADD_LINK;
smc_llc_flow_qentry_set(&lgr->llc_flow_lcl,
qentry);
diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index 53bcca365fb1..2d74c660de66 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -490,6 +490,8 @@ static int rpcb_register_inet4(struct sunrpc_net *sn,
int result;
map->r_addr = rpc_sockaddr2uaddr(sap, GFP_KERNEL);
+ if (!map->r_addr)
+ return -ENOMEM;
msg->rpc_proc = &rpcb_procedures4[RPCBPROC_UNSET];
if (port != 0) {
@@ -516,6 +518,8 @@ static int rpcb_register_inet6(struct sunrpc_net *sn,
int result;
map->r_addr = rpc_sockaddr2uaddr(sap, GFP_KERNEL);
+ if (!map->r_addr)
+ return -ENOMEM;
msg->rpc_proc = &rpcb_procedures4[RPCBPROC_UNSET];
if (port != 0) {
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 1901f397b490..b021c9b8477c 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -1202,9 +1202,9 @@ static void xs_save_old_callbacks(struct sock_xprt *transport, struct sock *sk)
static void xs_restore_old_callbacks(struct sock_xprt *transport, struct sock *sk)
{
- sk->sk_data_ready = transport->old_data_ready;
+ WRITE_ONCE(sk->sk_data_ready, transport->old_data_ready);
sk->sk_state_change = transport->old_state_change;
- sk->sk_write_space = transport->old_write_space;
+ WRITE_ONCE(sk->sk_write_space, transport->old_write_space);
sk->sk_error_report = transport->old_error_report;
}
@@ -1663,6 +1663,7 @@ static void xs_udp_do_set_buffer_size(struct rpc_xprt *xprt)
{
struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt);
struct sock *sk = transport->inet;
+ void (*write_space)(struct sock *sock);
if (transport->rcvsize) {
sk->sk_userlocks |= SOCK_RCVBUF_LOCK;
@@ -1671,7 +1672,8 @@ static void xs_udp_do_set_buffer_size(struct rpc_xprt *xprt)
if (transport->sndsize) {
sk->sk_userlocks |= SOCK_SNDBUF_LOCK;
sk->sk_sndbuf = transport->sndsize * xprt->max_reqs * 2;
- sk->sk_write_space(sk);
+ write_space = READ_ONCE(sk->sk_write_space);
+ write_space(sk);
}
}
@@ -1987,8 +1989,8 @@ static int xs_local_finish_connecting(struct rpc_xprt *xprt,
xs_save_old_callbacks(transport, sk);
sk->sk_user_data = xprt;
- sk->sk_data_ready = xs_data_ready;
- sk->sk_write_space = xs_udp_write_space;
+ WRITE_ONCE(sk->sk_data_ready, xs_data_ready);
+ WRITE_ONCE(sk->sk_write_space, xs_udp_write_space);
sk->sk_state_change = xs_local_state_change;
sk->sk_error_report = xs_error_report;
sk->sk_use_task_frag = false;
@@ -2190,8 +2192,8 @@ static void xs_udp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock)
xs_save_old_callbacks(transport, sk);
sk->sk_user_data = xprt;
- sk->sk_data_ready = xs_data_ready;
- sk->sk_write_space = xs_udp_write_space;
+ WRITE_ONCE(sk->sk_data_ready, xs_data_ready);
+ WRITE_ONCE(sk->sk_write_space, xs_udp_write_space);
sk->sk_use_task_frag = false;
xprt_set_connected(xprt);
@@ -2377,9 +2379,9 @@ static int xs_tcp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock)
xs_save_old_callbacks(transport, sk);
sk->sk_user_data = xprt;
- sk->sk_data_ready = xs_data_ready;
+ WRITE_ONCE(sk->sk_data_ready, xs_data_ready);
sk->sk_state_change = xs_tcp_state_change;
- sk->sk_write_space = xs_tcp_write_space;
+ WRITE_ONCE(sk->sk_write_space, xs_tcp_write_space);
sk->sk_error_report = xs_error_report;
sk->sk_use_task_frag = false;
diff --git a/net/tls/tls_strp.c b/net/tls/tls_strp.c
index c72e88317627..c0611c432776 100644
--- a/net/tls/tls_strp.c
+++ b/net/tls/tls_strp.c
@@ -431,9 +431,10 @@ static int tls_strp_read_copy(struct tls_strparser *strp, bool qshort)
return 0;
}
-static bool tls_strp_check_queue_ok(struct tls_strparser *strp)
+static bool tls_strp_check_queue_ok(struct tls_strparser *strp,
+ unsigned int len)
{
- unsigned int len = strp->stm.offset + strp->stm.full_len;
+ unsigned int remaining = strp->stm.offset + len;
struct sk_buff *first, *skb;
u32 seq;
@@ -444,9 +445,9 @@ static bool tls_strp_check_queue_ok(struct tls_strparser *strp)
/* Make sure there's no duplicate data in the queue,
* and the decrypted status matches.
*/
- while (skb->len < len) {
+ while (skb->len < remaining) {
seq += skb->len;
- len -= skb->len;
+ remaining -= skb->len;
skb = skb->next;
if (TCP_SKB_CB(skb)->seq != seq)
@@ -525,6 +526,11 @@ static int tls_strp_read_sock(struct tls_strparser *strp)
tls_strp_load_anchor_with_queue(strp, inq);
if (!strp->stm.full_len) {
+ if (inq < TLS_HEADER_SIZE)
+ return tls_strp_read_copy(strp, true);
+ if (!tls_strp_check_queue_ok(strp, TLS_HEADER_SIZE))
+ return tls_strp_read_copy(strp, false);
+
sz = tls_rx_msg_size(strp, strp->anchor);
if (sz < 0)
return sz;
@@ -535,7 +541,7 @@ static int tls_strp_read_sock(struct tls_strparser *strp)
return tls_strp_read_copy(strp, true);
}
- if (!tls_strp_check_queue_ok(strp))
+ if (!tls_strp_check_queue_ok(strp, strp->stm.full_len))
return tls_strp_read_copy(strp, false);
WRITE_ONCE(strp->msg_ready, 1);
diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
index f03e00cae028..0cbe5a1a40d3 100644
--- a/net/vmw_vsock/af_vsock.c
+++ b/net/vmw_vsock/af_vsock.c
@@ -1671,12 +1671,10 @@ static int vsock_connect(struct socket *sock, struct sockaddr *addr,
prepare_to_wait(sk_sleep(sk), &wait, TASK_INTERRUPTIBLE);
}
- if (sk->sk_err) {
- err = -sk->sk_err;
+ err = sock_error(sk);
+ if (err) {
sk->sk_state = TCP_CLOSE;
sock->state = SS_UNCONNECTED;
- } else {
- err = 0;
}
out_wait:
@@ -1715,10 +1713,10 @@ static int vsock_accept(struct socket *sock, struct socket *newsock,
* created upon connection establishment.
*/
timeout = sock_rcvtimeo(listener, arg->flags & O_NONBLOCK);
- prepare_to_wait(sk_sleep(listener), &wait, TASK_INTERRUPTIBLE);
while ((connected = vsock_dequeue_accept(listener)) == NULL &&
- listener->sk_err == 0) {
+ timeout != 0) {
+ prepare_to_wait(sk_sleep(listener), &wait, TASK_INTERRUPTIBLE);
release_sock(listener);
timeout = schedule_timeout(timeout);
finish_wait(sk_sleep(listener), &wait);
@@ -1727,19 +1725,12 @@ static int vsock_accept(struct socket *sock, struct socket *newsock,
if (signal_pending(current)) {
err = sock_intr_errno(timeout);
goto out;
- } else if (timeout == 0) {
- err = -EAGAIN;
- goto out;
}
-
- prepare_to_wait(sk_sleep(listener), &wait, TASK_INTERRUPTIBLE);
}
- finish_wait(sk_sleep(listener), &wait);
-
- if (listener->sk_err)
- err = -listener->sk_err;
- if (connected) {
+ if (!connected) {
+ err = -EAGAIN;
+ } else {
sk_acceptq_removed(listener);
lock_sock_nested(connected, SINGLE_DEPTH_NESTING);
diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c
index da7e11e3bfad..36170e10deda 100644
--- a/net/xdp/xsk.c
+++ b/net/xdp/xsk.c
@@ -141,7 +141,7 @@ static int __xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff_xsk *xskb, u32 len,
u64 addr;
int err;
- addr = xp_get_handle(xskb);
+ addr = xp_get_handle(xskb, xskb->pool);
err = xskq_prod_reserve_desc(xs->rx, addr, len, flags);
if (err) {
xs->rx_queue_full++;
@@ -152,6 +152,17 @@ static int __xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff_xsk *xskb, u32 len,
return 0;
}
+static void __xsk_rcv_zc_safe(struct xdp_sock *xs, struct xdp_buff_xsk *xskb,
+ u32 len, u32 flags)
+{
+ u64 addr;
+
+ addr = xp_get_handle(xskb, xskb->pool);
+ __xskq_prod_reserve_desc(xs->rx, addr, len, flags);
+
+ xp_release(xskb);
+}
+
static int xsk_rcv_zc(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
{
struct xdp_buff_xsk *xskb = container_of(xdp, struct xdp_buff_xsk, xdp);
@@ -234,9 +245,11 @@ static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
u32 frame_size = __xsk_pool_get_rx_frame_size(xs->pool);
void *copy_from = xsk_copy_xdp_start(xdp), *copy_to;
u32 from_len, meta_len, rem, num_desc;
- struct xdp_buff_xsk *xskb;
+ struct xdp_buff_xsk *xskb, *tmp;
struct xdp_buff *xsk_xdp;
+ LIST_HEAD(xsk_buffs);
skb_frag_t *frag;
+ u32 i;
from_len = xdp->data_end - copy_from;
meta_len = xdp->data - copy_from;
@@ -279,22 +292,45 @@ static int __xsk_rcv(struct xdp_sock *xs, struct xdp_buff *xdp, u32 len)
frag = &sinfo->frags[0];
}
+ for (i = 0; i < num_desc; i++) {
+ xsk_xdp = xsk_buff_alloc(xs->pool);
+ if (!xsk_xdp)
+ goto err_alloc;
+
+ xskb = container_of(xsk_xdp, struct xdp_buff_xsk, xdp);
+ if (unlikely(!list_empty(&xskb->list_node)))
+ goto err_alloc;
+
+ list_add_tail(&xskb->list_node, &xsk_buffs);
+ }
+
do {
u32 to_len = frame_size + meta_len;
u32 copied;
- xsk_xdp = xsk_buff_alloc(xs->pool);
+ xskb = list_first_entry(&xsk_buffs, struct xdp_buff_xsk,
+ list_node);
+ list_del_init(&xskb->list_node);
+ xsk_xdp = &xskb->xdp;
copy_to = xsk_xdp->data - meta_len;
copied = xsk_copy_xdp(copy_to, ©_from, to_len, &from_len, &frag, rem);
rem -= copied;
- xskb = container_of(xsk_xdp, struct xdp_buff_xsk, xdp);
- __xsk_rcv_zc(xs, xskb, copied - meta_len, rem ? XDP_PKT_CONTD : 0);
+ __xsk_rcv_zc_safe(xs, xskb, copied - meta_len,
+ rem ? XDP_PKT_CONTD : 0);
meta_len = 0;
} while (rem);
return 0;
+
+err_alloc:
+ list_for_each_entry_safe(xskb, tmp, &xsk_buffs, list_node) {
+ list_del_init(&xskb->list_node);
+ xsk_buff_free(&xskb->xdp);
+ }
+ xs->rx_dropped++;
+ return -ENOMEM;
}
static bool xsk_tx_writeable(struct xdp_sock *xs)
@@ -665,6 +701,7 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs,
struct xsk_tx_metadata *meta = NULL;
struct net_device *dev = xs->dev;
struct sk_buff *skb = xs->skb;
+ u16 csum_start, csum_offset;
bool first_frag = false;
int err;
@@ -733,15 +770,17 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs,
}
if (meta->flags & XDP_TXMD_FLAGS_CHECKSUM) {
- if (unlikely(meta->request.csum_start +
- meta->request.csum_offset +
+ csum_start = READ_ONCE(meta->request.csum_start);
+ csum_offset = READ_ONCE(meta->request.csum_offset);
+
+ if (unlikely(csum_start + csum_offset +
sizeof(__sum16) > len)) {
err = -EINVAL;
goto free_err;
}
- skb->csum_start = hr + meta->request.csum_start;
- skb->csum_offset = meta->request.csum_offset;
+ skb->csum_start = hr + csum_start;
+ skb->csum_offset = csum_offset;
skb->ip_summed = CHECKSUM_PARTIAL;
if (unlikely(xs->pool->tx_sw_csum)) {
diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c
index 04fe499f7678..a22cedd280cb 100644
--- a/net/xdp/xsk_buff_pool.c
+++ b/net/xdp/xsk_buff_pool.c
@@ -442,8 +442,10 @@ static int xp_init_dma_info(struct xsk_buff_pool *pool, struct xsk_dma_map *dma_
for (i = 0; i < pool->heads_cnt; i++) {
struct xdp_buff_xsk *xskb = &pool->heads[i];
+ u64 orig_addr;
- xp_init_xskb_dma(xskb, pool, dma_map->dma_pages, xskb->orig_addr);
+ orig_addr = xskb->xdp.data_hard_start - pool->addrs - pool->headroom;
+ xp_init_xskb_dma(xskb, pool, dma_map->dma_pages, orig_addr);
}
}
diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h
index 9351e1298ef4..2ec64c185f8f 100644
--- a/net/xdp/xsk_queue.h
+++ b/net/xdp/xsk_queue.h
@@ -428,20 +428,26 @@ static inline void xskq_prod_write_addr_batch(struct xsk_queue *q, struct xdp_de
q->cached_prod = cached_prod;
}
-static inline int xskq_prod_reserve_desc(struct xsk_queue *q,
- u64 addr, u32 len, u32 flags)
+static inline void __xskq_prod_reserve_desc(struct xsk_queue *q,
+ u64 addr, u32 len, u32 flags)
{
struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring;
u32 idx;
- if (xskq_prod_is_full(q))
- return -ENOBUFS;
-
/* A, matches D */
idx = q->cached_prod++ & q->ring_mask;
ring->desc[idx].addr = addr;
ring->desc[idx].len = len;
ring->desc[idx].options = flags;
+}
+
+static inline int xskq_prod_reserve_desc(struct xsk_queue *q,
+ u64 addr, u32 len, u32 flags)
+{
+ if (xskq_prod_is_full(q))
+ return -ENOBUFS;
+
+ __xskq_prod_reserve_desc(q, addr, len, flags);
return 0;
}
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index a5b5d82bd755..53654c5fa9e9 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -627,10 +627,8 @@ static int xfrm_dev_direct_output(struct sock *sk, struct xfrm_state *x,
nf_reset_ct(skb);
err = skb_dst(skb)->ops->local_out(net, sk, skb);
- if (unlikely(err != 1)) {
- kfree_skb(skb);
+ if (unlikely(err != 1))
return err;
- }
/* In transport mode, network destination is
* directly reachable, while in tunnel mode,
diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c
index 3bb791e0b05e..a44ca313d916 100644
--- a/samples/ftrace/ftrace-direct-modify.c
+++ b/samples/ftrace/ftrace-direct-modify.c
@@ -237,9 +237,15 @@ static int __init ftrace_direct_init(void)
ftrace_set_filter_ip(&direct, (unsigned long) my_ip, 0, 0);
ret = register_ftrace_direct(&direct, my_tramp);
- if (!ret)
- simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn");
- return ret;
+ if (ret)
+ return ret;
+ simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn");
+ if (IS_ERR(simple_tsk)) {
+ unregister_ftrace_direct(&direct, my_tramp, true);
+ return PTR_ERR(simple_tsk);
+ }
+
+ return 0;
}
static void __exit ftrace_direct_exit(void)
diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c
index cc7f42999c00..34b396c9cbb7 100644
--- a/samples/ftrace/ftrace-direct-multi-modify.c
+++ b/samples/ftrace/ftrace-direct-multi-modify.c
@@ -265,9 +265,15 @@ static int __init ftrace_direct_multi_init(void)
ret = register_ftrace_direct(&direct, my_tramp);
- if (!ret)
- simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn");
- return ret;
+ if (ret)
+ return ret;
+ simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn");
+ if (IS_ERR(simple_tsk)) {
+ unregister_ftrace_direct(&direct, my_tramp, true);
+ return PTR_ERR(simple_tsk);
+ }
+
+ return 0;
}
static void __exit ftrace_direct_multi_exit(void)
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 07e8a4eb71d0..f17350bf3ddb 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -1567,14 +1567,14 @@ static void read_symbols(const char *modname)
struct elf_info info = { };
Elf_Sym *sym;
- if (!parse_elf(&info, modname))
- return;
-
if (!strends(modname, ".o")) {
error("%s: filename must be suffixed with .o\n", modname);
return;
}
+ if (!parse_elf(&info, modname))
+ return;
+
/* strip trailing .o */
mod = new_module(modname, strlen(modname) - strlen(".o"));
diff --git a/scripts/tags.sh b/scripts/tags.sh
index 191e0461d6d5..4c194ebf7a70 100755
--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -100,7 +100,7 @@ all_compiled_sources()
{
echo include/generated/autoconf.h
find $ignore -name "*.cmd" -exec \
- grep -Poh '(?<=^ )\S+|(?<== )\S+[^\\](?=$)' {} \+ |
+ grep -Poh '(?<=^ )\S+\.([chS]|rs)(?=\s)|(?<== )\S+\.(?1)(?=$)' {} \+ |
awk '!a[$0]++'
} | xargs realpath -esq $([ -z "$KBUILD_ABS_SRCTREE" ] && echo --relative-to=.) |
sort -u
diff --git a/security/apparmor/lib.c b/security/apparmor/lib.c
index cd569fbbfe36..1165d51072ce 100644
--- a/security/apparmor/lib.c
+++ b/security/apparmor/lib.c
@@ -448,24 +448,22 @@ bool aa_policy_init(struct aa_policy *policy, const char *prefix,
const char *name, gfp_t gfp)
{
char *hname;
+ size_t hname_sz;
+ INIT_LIST_HEAD(&policy->list);
+ INIT_LIST_HEAD(&policy->profiles);
+ hname_sz = (prefix ? strlen(prefix) + 2 : 0) + strlen(name) + 1;
/* freed by policy_free */
- if (prefix) {
- hname = aa_str_alloc(strlen(prefix) + strlen(name) + 3, gfp);
- if (hname)
- sprintf(hname, "%s//%s", prefix, name);
- } else {
- hname = aa_str_alloc(strlen(name) + 1, gfp);
- if (hname)
- strcpy(hname, name);
- }
+ hname = aa_str_alloc(hname_sz, gfp);
if (!hname)
return false;
+ if (prefix)
+ scnprintf(hname, hname_sz, "%s//%s", prefix, name);
+ else
+ strscpy(hname, name, hname_sz);
policy->hname = hname;
/* base.name is a substring of fqname */
policy->name = basename(policy->hname);
- INIT_LIST_HEAD(&policy->list);
- INIT_LIST_HEAD(&policy->profiles);
return true;
}
diff --git a/security/integrity/Makefile b/security/integrity/Makefile
index 92b63039c654..45dfdedbdad4 100644
--- a/security/integrity/Makefile
+++ b/security/integrity/Makefile
@@ -18,6 +18,7 @@ integrity-$(CONFIG_LOAD_IPL_KEYS) += platform_certs/load_ipl_s390.o
integrity-$(CONFIG_LOAD_PPC_KEYS) += platform_certs/efi_parser.o \
platform_certs/load_powerpc.o \
platform_certs/keyring_handler.o
+integrity-$(CONFIG_EFI) += efi_secureboot.o
# The relative order of the 'ima' and 'evm' LSMs depends on the order below.
obj-$(CONFIG_IMA) += ima/
obj-$(CONFIG_EVM) += evm/
diff --git a/security/integrity/efi_secureboot.c b/security/integrity/efi_secureboot.c
new file mode 100644
index 000000000000..bfd4260a83a3
--- /dev/null
+++ b/security/integrity/efi_secureboot.c
@@ -0,0 +1,56 @@
+// SPDX-License-Identifier: GPL-1.0+
+/*
+ * Copyright (C) 2018 IBM Corporation
+ */
+#include <linux/efi.h>
+#include <linux/secure_boot.h>
+#include <asm/efi.h>
+
+#ifndef arch_efi_boot_mode
+#define arch_efi_boot_mode efi_secureboot_mode_unset
+#endif
+
+static enum efi_secureboot_mode get_sb_mode(void)
+{
+ enum efi_secureboot_mode mode;
+
+ if (!efi_rt_services_supported(EFI_RT_SUPPORTED_GET_VARIABLE)) {
+ pr_info("integrity: secureboot mode unknown, no efi\n");
+ return efi_secureboot_mode_unknown;
+ }
+
+ mode = efi_get_secureboot_mode(efi.get_variable);
+ if (mode == efi_secureboot_mode_disabled)
+ pr_info("integrity: secureboot mode disabled\n");
+ else if (mode == efi_secureboot_mode_unknown)
+ pr_info("integrity: secureboot mode unknown\n");
+ else
+ pr_info("integrity: secureboot mode enabled\n");
+ return mode;
+}
+
+/*
+ * Query secure boot status
+ *
+ * Note don't call this function too early e.g. in __setup hook otherwise the
+ * kernel may hang when calling efi_get_secureboot_mode.
+ *
+ */
+bool arch_get_secureboot(void)
+{
+ static enum efi_secureboot_mode sb_mode;
+ static bool initialized;
+
+ if (!initialized && efi_enabled(EFI_BOOT)) {
+ sb_mode = arch_efi_boot_mode;
+
+ if (sb_mode == efi_secureboot_mode_unset)
+ sb_mode = get_sb_mode();
+ initialized = true;
+ }
+
+ if (sb_mode == efi_secureboot_mode_enabled)
+ return true;
+ else
+ return false;
+}
diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c
index b5c573332fa8..7eb2c2507611 100644
--- a/security/integrity/ima/ima_appraise.c
+++ b/security/integrity/ima/ima_appraise.c
@@ -26,7 +26,7 @@ core_param(ima_appraise, ima_appraise_cmdline_default, charp, 0);
void __init ima_appraise_parse_cmdline(void)
{
const char *str = ima_appraise_cmdline_default;
- bool sb_state = arch_ima_get_secureboot();
+ bool sb_state = arch_get_secureboot();
int appraisal_state = ima_appraise;
if (!str)
@@ -754,6 +754,8 @@ static int validate_hash_algo(struct dentry *dentry,
return -EACCES;
path = dentry_path(dentry, pathbuf, PATH_MAX);
+ if (IS_ERR(path))
+ path = NULL;
integrity_audit_msg(AUDIT_INTEGRITY_DATA, d_inode(dentry), path,
"set_data", errmsg, -EACCES, 0);
diff --git a/security/integrity/ima/ima_efi.c b/security/integrity/ima/ima_efi.c
index 138029bfcce1..78191879dd98 100644
--- a/security/integrity/ima/ima_efi.c
+++ b/security/integrity/ima/ima_efi.c
@@ -2,52 +2,9 @@
/*
* Copyright (C) 2018 IBM Corporation
*/
-#include <linux/efi.h>
#include <linux/module.h>
#include <linux/ima.h>
-#include <asm/efi.h>
-
-#ifndef arch_ima_efi_boot_mode
-#define arch_ima_efi_boot_mode efi_secureboot_mode_unset
-#endif
-
-static enum efi_secureboot_mode get_sb_mode(void)
-{
- enum efi_secureboot_mode mode;
-
- if (!efi_rt_services_supported(EFI_RT_SUPPORTED_GET_VARIABLE)) {
- pr_info("ima: secureboot mode unknown, no efi\n");
- return efi_secureboot_mode_unknown;
- }
-
- mode = efi_get_secureboot_mode(efi.get_variable);
- if (mode == efi_secureboot_mode_disabled)
- pr_info("ima: secureboot mode disabled\n");
- else if (mode == efi_secureboot_mode_unknown)
- pr_info("ima: secureboot mode unknown\n");
- else
- pr_info("ima: secureboot mode enabled\n");
- return mode;
-}
-
-bool arch_ima_get_secureboot(void)
-{
- static enum efi_secureboot_mode sb_mode;
- static bool initialized;
-
- if (!initialized && efi_enabled(EFI_BOOT)) {
- sb_mode = arch_ima_efi_boot_mode;
-
- if (sb_mode == efi_secureboot_mode_unset)
- sb_mode = get_sb_mode();
- initialized = true;
- }
-
- if (sb_mode == efi_secureboot_mode_enabled)
- return true;
- else
- return false;
-}
+#include <linux/secure_boot.h>
/* secureboot arch rules */
static const char * const sb_arch_rules[] = {
@@ -67,7 +24,7 @@ static const char * const sb_arch_rules[] = {
const char * const *arch_get_ima_policy(void)
{
- if (IS_ENABLED(CONFIG_IMA_ARCH_POLICY) && arch_ima_get_secureboot()) {
+ if (IS_ENABLED(CONFIG_IMA_ARCH_POLICY) && arch_get_secureboot()) {
if (IS_ENABLED(CONFIG_MODULE_SIG))
set_module_sig_enforced();
if (IS_ENABLED(CONFIG_KEXEC_SIG))
diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
index a0948057209e..62dc15ade559 100644
--- a/security/integrity/ima/ima_main.c
+++ b/security/integrity/ima/ima_main.c
@@ -907,8 +907,7 @@ static int ima_load_data(enum kernel_load_data_id id, bool contents)
switch (id) {
case LOADING_KEXEC_IMAGE:
- if (IS_ENABLED(CONFIG_KEXEC_SIG)
- && arch_ima_get_secureboot()) {
+ if (IS_ENABLED(CONFIG_KEXEC_SIG) && arch_get_secureboot()) {
pr_err("impossible to appraise a kernel image without a file descriptor; try using kexec_file_load syscall.\n");
return -EACCES;
}
diff --git a/security/integrity/integrity.h b/security/integrity/integrity.h
index c2c2da691123..17b037b3fd09 100644
--- a/security/integrity/integrity.h
+++ b/security/integrity/integrity.h
@@ -14,6 +14,7 @@
#include <linux/types.h>
#include <linux/integrity.h>
+#include <linux/secure_boot.h>
#include <crypto/sha1.h>
#include <crypto/hash.h>
#include <linux/key.h>
diff --git a/security/integrity/platform_certs/load_uefi.c b/security/integrity/platform_certs/load_uefi.c
index d1fdd113450a..c0d6948446c3 100644
--- a/security/integrity/platform_certs/load_uefi.c
+++ b/security/integrity/platform_certs/load_uefi.c
@@ -212,7 +212,7 @@ static int __init load_uefi_certs(void)
}
/* the MOK/MOKx can not be trusted when secure boot is disabled */
- if (!arch_ima_get_secureboot())
+ if (!arch_get_secureboot())
return 0;
mokx = get_cert_list(L"MokListXRT", &mok_var, &mokxsize, &status);
diff --git a/security/landlock/errata/abi-1.h b/security/landlock/errata/abi-1.h
index e8a2bff2e5b6..80930957e84d 100644
--- a/security/landlock/errata/abi-1.h
+++ b/security/landlock/errata/abi-1.h
@@ -14,3 +14,26 @@
* actions.
*/
LANDLOCK_ERRATUM(3)
+
+/**
+ * DOC: erratum_4
+ *
+ * Erratum 4: Creation of whiteout objects
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ * This fix changes the access rights required for the creation of whiteout
+ * objects through :manpage:`mknod(2)`, :manpage:`renameat2(2)`, or
+ * :manpage:`link(2)`. Creating whiteout objects is now guarded by
+ * ``LANDLOCK_ACCESS_FS_MAKE_REG`` instead of ``LANDLOCK_ACCESS_FS_MAKE_CHAR``.
+ *
+ * Whiteout objects are used in OverlayFS to mark the absence of a file in an
+ * upper file system. Despite being created with ``S_IFCHR``, whiteout objects
+ * do not count as character devices.
+ *
+ * Impact:
+ *
+ * Sandboxed programs that create OverlayFS whiteouts (such as fuse-overlayfs)
+ * now require ``LANDLOCK_ACCESS_FS_MAKE_REG`` instead of
+ * ``LANDLOCK_ACCESS_FS_MAKE_CHAR``.
+ */
+LANDLOCK_ERRATUM(4)
diff --git a/security/landlock/fs.c b/security/landlock/fs.c
index bb4a9e82156a..b96a4daa5b2a 100644
--- a/security/landlock/fs.c
+++ b/security/landlock/fs.c
@@ -20,6 +20,7 @@
#include <linux/falloc.h>
#include <linux/fs.h>
#include <linux/init.h>
+#include <linux/kdev_t.h>
#include <linux/kernel.h>
#include <linux/limits.h>
#include <linux/list.h>
@@ -954,7 +955,7 @@ static int current_check_access_path(const struct path *const path,
return check_access_path(dom, path, access_request);
}
-static access_mask_t get_mode_access(const umode_t mode)
+static access_mask_t get_mode_access(const umode_t mode, const dev_t dev)
{
switch (mode & S_IFMT) {
case S_IFLNK:
@@ -962,6 +963,9 @@ static access_mask_t get_mode_access(const umode_t mode)
case S_IFDIR:
return LANDLOCK_ACCESS_FS_MAKE_DIR;
case S_IFCHR:
+ /* Whiteout objects are guarded with MAKE_REG. */
+ if (dev == WHITEOUT_DEV)
+ return LANDLOCK_ACCESS_FS_MAKE_REG;
return LANDLOCK_ACCESS_FS_MAKE_CHAR;
case S_IFBLK:
return LANDLOCK_ACCESS_FS_MAKE_BLOCK;
@@ -978,6 +982,13 @@ static access_mask_t get_mode_access(const umode_t mode)
}
}
+static access_mask_t get_dentry_access(const struct dentry *const dentry)
+{
+ const struct inode *const inode = d_backing_inode(dentry);
+
+ return get_mode_access(inode->i_mode, inode->i_rdev);
+}
+
static access_mask_t maybe_remove(const struct dentry *const dentry)
{
if (d_is_negative(dentry))
@@ -1067,6 +1078,7 @@ static bool collect_domain_accesses(
* @new_dentry: Destination file or directory.
* @removable: Sets to true if it is a rename operation.
* @exchange: Sets to true if it is a rename operation with RENAME_EXCHANGE.
+ * @whiteout: Sets to true if it is a rename operation with RENAME_WHITEOUT.
*
* Because of its unprivileged constraints, Landlock relies on file hierarchies
* (and not only inodes) to tie access rights to files. Being able to link or
@@ -1115,7 +1127,8 @@ static bool collect_domain_accesses(
static int current_check_refer_path(struct dentry *const old_dentry,
const struct path *const new_dir,
struct dentry *const new_dentry,
- const bool removable, const bool exchange)
+ const bool removable, const bool exchange,
+ const bool whiteout)
{
const struct landlock_ruleset *const dom = get_current_fs_domain();
bool allow_parent1, allow_parent2;
@@ -1134,18 +1147,25 @@ static int current_check_refer_path(struct dentry *const old_dentry,
if (exchange) {
if (unlikely(d_is_negative(new_dentry)))
return -ENOENT;
- access_request_parent1 =
- get_mode_access(d_backing_inode(new_dentry)->i_mode);
+ access_request_parent1 = get_dentry_access(new_dentry);
} else {
access_request_parent1 = 0;
}
- access_request_parent2 =
- get_mode_access(d_backing_inode(old_dentry)->i_mode);
+ access_request_parent2 = get_dentry_access(old_dentry);
if (removable) {
access_request_parent1 |= maybe_remove(old_dentry);
access_request_parent2 |= maybe_remove(new_dentry);
}
+ /*
+ * In case of renameat2(2) with RENAME_WHITEOUT, a whiteout object is
+ * created in the source location, so we require an additional access
+ * right there.
+ */
+ if (whiteout)
+ access_request_parent1 |=
+ get_mode_access(S_IFCHR | WHITEOUT_MODE, WHITEOUT_DEV);
+
/* The mount points are the same for old and new paths, cf. EXDEV. */
if (old_dentry->d_parent == new_dir->dentry) {
/*
@@ -1417,7 +1437,7 @@ static int hook_path_link(struct dentry *const old_dentry,
struct dentry *const new_dentry)
{
return current_check_refer_path(old_dentry, new_dir, new_dentry, false,
- false);
+ false, false);
}
static int hook_path_rename(const struct path *const old_dir,
@@ -1428,7 +1448,8 @@ static int hook_path_rename(const struct path *const old_dir,
{
/* old_dir refers to old_dentry->d_parent and new_dir->mnt */
return current_check_refer_path(old_dentry, new_dir, new_dentry, true,
- !!(flags & RENAME_EXCHANGE));
+ !!(flags & RENAME_EXCHANGE),
+ !!(flags & RENAME_WHITEOUT));
}
static int hook_path_mkdir(const struct path *const dir,
@@ -1445,7 +1466,8 @@ static int hook_path_mknod(const struct path *const dir,
if (!dom)
return 0;
- return check_access_path(dom, dir, get_mode_access(mode));
+ return check_access_path(dom, dir,
+ get_mode_access(mode, new_decode_dev(dev)));
}
static int hook_path_symlink(const struct path *const dir,
diff --git a/security/smack/smack.h b/security/smack/smack.h
index deb2ef31b63a..f1eda7f269f5 100644
--- a/security/smack/smack.h
+++ b/security/smack/smack.h
@@ -302,8 +302,9 @@ int smack_populate_secattr(struct smack_known *skp);
* Shared data.
*/
extern int smack_enabled __initdata;
-extern int smack_cipso_direct;
-extern int smack_cipso_mapped;
+extern u8 smack_cipso_auto_level[2];
+#define smack_cipso_direct (+smack_cipso_auto_level[0])
+#define smack_cipso_mapped (+smack_cipso_auto_level[1])
extern struct smack_known *smack_net_ambient;
extern struct smack_known *smack_syslog_label;
#ifdef CONFIG_SECURITY_SMACK_BRINGUP
diff --git a/security/smack/smack_lsm.c b/security/smack/smack_lsm.c
index 7a9b20a576c6..c1c7b1409bf4 100644
--- a/security/smack/smack_lsm.c
+++ b/security/smack/smack_lsm.c
@@ -147,12 +147,13 @@ static int smk_bu_note(char *note, struct smack_known *sskp,
#define smk_bu_note(note, sskp, oskp, mode, RC) (RC)
#endif
-#ifdef CONFIG_SECURITY_SMACK_BRINGUP
-static int smk_bu_current(char *note, struct smack_known *oskp,
- int mode, int rc)
+static int
+smk_bu_tsk_to_obj(struct task_struct *tsk, const struct task_smack *tsp,
+ char *note, struct smack_known *oskp, int mode, int rc)
{
- struct task_smack *tsp = smack_cred(current_cred());
+#ifdef CONFIG_SECURITY_SMACK_BRINGUP
char acc[SMK_NUM_ACCESS_TYPE + 1];
+ char comm[TASK_COMM_LEN];
if (rc <= 0)
return rc;
@@ -160,14 +161,22 @@ static int smk_bu_current(char *note, struct smack_known *oskp,
rc = 0;
smk_bu_mode(mode, acc);
+
pr_info("Smack %s: (%s %s %s) %s %s\n", smk_bu_mess[rc],
- tsp->smk_task->smk_known, oskp->smk_known,
- acc, current->comm, note);
+ smk_of_task(tsp)->smk_known, oskp->smk_known,
+ acc, get_task_comm(comm, tsk), note);
return 0;
-}
#else
-#define smk_bu_current(note, oskp, mode, RC) (RC)
+ return rc;
#endif
+}
+
+static int smk_bu_current(char *note, struct smack_known *oskp,
+ int mode, int rc)
+{
+ return smk_bu_tsk_to_obj(current, smack_cred(current_cred()),
+ note, oskp, mode, rc);
+}
#ifdef CONFIG_SECURITY_SMACK_BRINGUP
static int smk_bu_task(struct task_struct *otp, int mode, int rc)
@@ -1329,7 +1338,7 @@ static int smack_inode_getattr(const struct path *path)
*/
static int smack_inode_xattr_skipcap(const char *name)
{
- if (strncmp(name, XATTR_SMACK_SUFFIX, strlen(XATTR_SMACK_SUFFIX)))
+ if (strncmp(name, XATTR_SMACK_SUFFIX, strlen(XATTR_SMACK_SUFFIX)) == 0)
return 0;
if (strcmp(name, XATTR_NAME_SMACK) == 0 ||
@@ -3360,14 +3369,20 @@ static int smack_sem_semop(struct kern_ipc_perm *isp, struct sembuf *sops,
}
/**
- * smk_curacc_msq : helper to check if current has access on msq
- * @isp : the msq
+ * smk_tskacc_msq : helper to check if tsk has access on msq
+ * @tsk: the task that requests access
+ * @isp : the sysv msg queue permissions
* @access : access requested
*
- * return 0 if current has access, error otherwise
+ * return 0 if tsk has access, error otherwise
*/
-static int smk_curacc_msq(struct kern_ipc_perm *isp, int access)
+static int
+smk_tskacc_msq(struct task_struct *tsk, struct kern_ipc_perm *isp, int access)
{
+ const bool tsk_is_current = (tsk == current);
+ const struct cred * const tsk_cred =
+ (tsk_is_current ? current_cred() : get_task_cred(tsk));
+ struct task_smack * const tsp = smack_cred(tsk_cred);
struct smack_known *msp = smack_of_ipc(isp);
struct smk_audit_info ad;
int rc;
@@ -3376,11 +3391,25 @@ static int smk_curacc_msq(struct kern_ipc_perm *isp, int access)
smk_ad_init(&ad, __func__, LSM_AUDIT_DATA_IPC);
ad.a.u.ipc_id = isp->id;
#endif
- rc = smk_curacc(msp, access, &ad);
- rc = smk_bu_current("msq", msp, access, rc);
+ rc = smk_tskacc(tsp, msp, access, &ad);
+ rc = smk_bu_tsk_to_obj(tsk, tsp, "msq", msp, access, rc);
+ if (!tsk_is_current)
+ put_cred(tsk_cred);
return rc;
}
+/**
+ * smk_curacc_msq : helper to check if current has access on msq
+ * @isp : the sysv msg queue permissions
+ * @access : access requested
+ *
+ * return 0 if current has access, error otherwise
+ */
+static int smk_curacc_msq(struct kern_ipc_perm *isp, int access)
+{
+ return smk_tskacc_msq(current, isp, access);
+}
+
/**
* smack_msg_queue_associate - Smack access check for msg_queue
* @isp: the object
@@ -3448,21 +3477,21 @@ static int smack_msg_queue_msgsnd(struct kern_ipc_perm *isp, struct msg_msg *msg
}
/**
- * smack_msg_queue_msgrcv - Smack access check for msg_queue
+ * smack_msg_queue_msgrcv - check it target has r/w access to msg_queue
* @isp: the object
* @msg: unused
- * @target: unused
+ * @target: the task that msgrcv() from the queue
* @type: unused
* @mode: unused
*
- * Returns 0 if current has read and write access, error code otherwise
+ * Returns 0 if target has read and write access, error code otherwise
*/
static int smack_msg_queue_msgrcv(struct kern_ipc_perm *isp,
struct msg_msg *msg,
struct task_struct *target, long type,
int mode)
{
- return smk_curacc_msq(isp, MAY_READWRITE);
+ return smk_tskacc_msq(target, isp, MAY_READWRITE);
}
/**
diff --git a/security/smack/smackfs.c b/security/smack/smackfs.c
index 109ad155ffc2..0f304667d857 100644
--- a/security/smack/smackfs.c
+++ b/security/smack/smackfs.c
@@ -81,18 +81,27 @@ static DEFINE_MUTEX(smk_net6addr_lock);
struct smack_known *smack_net_ambient;
/*
- * This is the level in a CIPSO header that indicates a
+ * Sensitivity levels for automatically created CIPSO labels.
+ * See smack_access.c`smack_populate_secattr()
+ *
+ * [0] "direct" labeling, label length < SMK_CIPSOLEN(24):
* smack label is contained directly in the category set.
* It can be reset via smackfs/direct
- */
-int smack_cipso_direct = SMACK_CIPSO_DIRECT_DEFAULT;
-
-/*
- * This is the level in a CIPSO header that indicates a
+ *
+ * [1] "mapped" labeling, label length >= SMK_CIPSOLEN(24):
* secid is contained directly in the category set.
* It can be reset via smackfs/mapped
*/
-int smack_cipso_mapped = SMACK_CIPSO_MAPPED_DEFAULT;
+u8 smack_cipso_auto_level[2] = {
+ SMACK_CIPSO_DIRECT_DEFAULT,
+ SMACK_CIPSO_MAPPED_DEFAULT,
+};
+
+static int
+smk_cipso_auto_level_idx(const struct file *file)
+{
+ return (file_inode(file)->i_ino != SMK_DIRECT);
+}
#ifdef CONFIG_SECURITY_SMACK_BRINGUP
/*
@@ -1616,24 +1625,17 @@ static ssize_t smk_read_doi(struct file *filp, char __user *buf,
static ssize_t smk_write_doi(struct file *file, const char __user *buf,
size_t count, loff_t *ppos)
{
- char temp[80];
- unsigned long u;
+ int ret;
+ u32 u;
if (!smack_privileged(CAP_MAC_ADMIN))
return -EPERM;
- if (count >= sizeof(temp) || count == 0)
- return -EINVAL;
-
- if (copy_from_user(temp, buf, count) != 0)
- return -EFAULT;
+ ret = kstrtou32_from_user(buf, count, 10, &u);
+ if (unlikely(ret))
+ return ret;
- temp[count] = '\0';
-
- if (kstrtoul(temp, 10, &u))
- return -EINVAL;
-
- if (u == CIPSO_V4_DOI_UNKNOWN || u > U32_MAX)
+ if (u == CIPSO_V4_DOI_UNKNOWN)
return -EINVAL;
return smk_cipso_doi(u, GFP_KERNEL) ? : count;
@@ -1646,158 +1648,80 @@ static const struct file_operations smk_doi_ops = {
};
/**
- * smk_read_direct - read() for /smack/direct
- * @filp: file pointer, not actually used
+ * smk_read_cipso_auto_level - read() for smackfs/direct and smackfs/mapped
+ * @filp: file pointer
* @buf: where to put the result
* @count: maximum to send along
* @ppos: where to start
*
* Returns number of bytes read or error code, as appropriate
*/
-static ssize_t smk_read_direct(struct file *filp, char __user *buf,
+static ssize_t smk_read_cipso_auto_level(struct file *filp, char __user *buf,
size_t count, loff_t *ppos)
{
- char temp[80];
- ssize_t rc;
+ char temp[sizeof "255"];
+ int n;
if (*ppos != 0)
return 0;
- sprintf(temp, "%d", smack_cipso_direct);
- rc = simple_read_from_buffer(buf, count, ppos, temp, strlen(temp));
-
- return rc;
+ n = sprintf(temp, "%u", (unsigned int)smack_cipso_auto_level[
+ smk_cipso_auto_level_idx(filp)]);
+ return simple_read_from_buffer(buf, count, ppos, temp, n);
}
/**
- * smk_write_direct - write() for /smack/direct
- * @file: file pointer, not actually used
+ * smk_write_cipso_auto_level - write() for smackfs/direct and smackfs/mapped
+ * @filp: file pointer
* @buf: where to get the data from
* @count: bytes sent
* @ppos: where to start
*
* Returns number of bytes written or error code, as appropriate
*/
-static ssize_t smk_write_direct(struct file *file, const char __user *buf,
- size_t count, loff_t *ppos)
+static ssize_t
+smk_write_cipso_auto_level(struct file *filp, const char __user *buf,
+ size_t count, loff_t *ppos)
{
- struct smack_known *skp;
- char temp[80];
- int i;
+ int ret, idx;
+ u8 i, old_lvl;
if (!smack_privileged(CAP_MAC_ADMIN))
return -EPERM;
-
- if (count >= sizeof(temp) || count == 0)
- return -EINVAL;
-
- if (copy_from_user(temp, buf, count) != 0)
- return -EFAULT;
-
- temp[count] = '\0';
-
- if (sscanf(temp, "%d", &i) != 1)
- return -EINVAL;
-
/*
- * Don't do anything if the value hasn't actually changed.
- * If it is changing reset the level on entries that were
- * set up to be direct when they were created.
+ * draft-ietf-cipso-ipsecurity-01 (CIPSO 2.2), 3.4.2.4:
+ * "Sensitivity Level is 1 octet in length. Its value is from 0 to 255"
*/
- if (smack_cipso_direct != i) {
- mutex_lock(&smack_known_lock);
- list_for_each_entry_rcu(skp, &smack_known_list, list)
- if (skp->smk_netlabel.attr.mls.lvl ==
- smack_cipso_direct)
- skp->smk_netlabel.attr.mls.lvl = i;
- smack_cipso_direct = i;
- mutex_unlock(&smack_known_lock);
- }
-
- return count;
-}
-
-static const struct file_operations smk_direct_ops = {
- .read = smk_read_direct,
- .write = smk_write_direct,
- .llseek = default_llseek,
-};
-
-/**
- * smk_read_mapped - read() for /smack/mapped
- * @filp: file pointer, not actually used
- * @buf: where to put the result
- * @count: maximum to send along
- * @ppos: where to start
- *
- * Returns number of bytes read or error code, as appropriate
- */
-static ssize_t smk_read_mapped(struct file *filp, char __user *buf,
- size_t count, loff_t *ppos)
-{
- char temp[80];
- ssize_t rc;
-
- if (*ppos != 0)
- return 0;
-
- sprintf(temp, "%d", smack_cipso_mapped);
- rc = simple_read_from_buffer(buf, count, ppos, temp, strlen(temp));
-
- return rc;
-}
-
-/**
- * smk_write_mapped - write() for /smack/mapped
- * @file: file pointer, not actually used
- * @buf: where to get the data from
- * @count: bytes sent
- * @ppos: where to start
- *
- * Returns number of bytes written or error code, as appropriate
- */
-static ssize_t smk_write_mapped(struct file *file, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- struct smack_known *skp;
- char temp[80];
- int i;
-
- if (!smack_privileged(CAP_MAC_ADMIN))
- return -EPERM;
-
- if (count >= sizeof(temp) || count == 0)
- return -EINVAL;
-
- if (copy_from_user(temp, buf, count) != 0)
- return -EFAULT;
-
- temp[count] = '\0';
-
- if (sscanf(temp, "%d", &i) != 1)
- return -EINVAL;
+ ret = kstrtou8_from_user(buf, count, 10, &i);
+ if (unlikely(ret))
+ return ret;
/*
* Don't do anything if the value hasn't actually changed.
* If it is changing reset the level on entries that were
- * set up to be mapped when they were created.
+ * set up to be "auto" level when they were created.
*/
- if (smack_cipso_mapped != i) {
+ idx = smk_cipso_auto_level_idx(filp);
+ old_lvl = smack_cipso_auto_level[idx];
+
+ if (old_lvl != i) {
+ struct smack_known *skp;
mutex_lock(&smack_known_lock);
list_for_each_entry_rcu(skp, &smack_known_list, list)
if (skp->smk_netlabel.attr.mls.lvl ==
- smack_cipso_mapped)
+ old_lvl)
skp->smk_netlabel.attr.mls.lvl = i;
- smack_cipso_mapped = i;
+ smack_cipso_auto_level[idx] = i;
mutex_unlock(&smack_known_lock);
}
return count;
}
-static const struct file_operations smk_mapped_ops = {
- .read = smk_read_mapped,
- .write = smk_write_mapped,
+static const struct file_operations
+smk_cipso_auto_level_ops = {
+ .read = smk_read_cipso_auto_level,
+ .write = smk_write_cipso_auto_level,
.llseek = default_llseek,
};
@@ -2196,22 +2120,15 @@ static ssize_t smk_read_logging(struct file *filp, char __user *buf,
static ssize_t smk_write_logging(struct file *file, const char __user *buf,
size_t count, loff_t *ppos)
{
- char temp[32];
- int i;
+ int i, ret;
if (!smack_privileged(CAP_MAC_ADMIN))
return -EPERM;
- if (count >= sizeof(temp) || count == 0)
- return -EINVAL;
+ ret = kstrtos32_from_user(buf, count, 10, &i);
+ if (unlikely(ret))
+ return ret;
- if (copy_from_user(temp, buf, count) != 0)
- return -EFAULT;
-
- temp[count] = '\0';
-
- if (sscanf(temp, "%d", &i) != 1)
- return -EINVAL;
if (i < 0 || i > 3)
return -EINVAL;
log_policy = i;
@@ -2854,22 +2771,15 @@ static ssize_t smk_read_ptrace(struct file *filp, char __user *buf,
static ssize_t smk_write_ptrace(struct file *file, const char __user *buf,
size_t count, loff_t *ppos)
{
- char temp[32];
- int i;
+ int i, ret;
if (!smack_privileged(CAP_MAC_ADMIN))
return -EPERM;
- if (*ppos != 0 || count >= sizeof(temp) || count == 0)
- return -EINVAL;
-
- if (copy_from_user(temp, buf, count) != 0)
- return -EFAULT;
-
- temp[count] = '\0';
+ ret = kstrtos32_from_user(buf, count, 10, &i);
+ if (unlikely(ret))
+ return ret;
- if (sscanf(temp, "%d", &i) != 1)
- return -EINVAL;
if (i < SMACK_PTRACE_DEFAULT || i > SMACK_PTRACE_MAX)
return -EINVAL;
smack_ptrace_rule = i;
@@ -2904,7 +2814,7 @@ static int smk_fill_super(struct super_block *sb, struct fs_context *fc)
[SMK_DOI] = {
"doi", &smk_doi_ops, S_IRUGO|S_IWUSR},
[SMK_DIRECT] = {
- "direct", &smk_direct_ops, S_IRUGO|S_IWUSR},
+ "direct", &smk_cipso_auto_level_ops, 0644},
[SMK_AMBIENT] = {
"ambient", &smk_ambient_ops, S_IRUGO|S_IWUSR},
[SMK_NET4ADDR] = {
@@ -2918,7 +2828,7 @@ static int smk_fill_super(struct super_block *sb, struct fs_context *fc)
[SMK_ACCESSES] = {
"access", &smk_access_ops, S_IRUGO|S_IWUGO},
[SMK_MAPPED] = {
- "mapped", &smk_mapped_ops, S_IRUGO|S_IWUSR},
+ "mapped", &smk_cipso_auto_level_ops, 0644},
[SMK_LOAD2] = {
"load2", &smk_load2_ops, S_IRUGO|S_IWUSR},
[SMK_LOAD_SELF2] = {
diff --git a/sound/core/control.c b/sound/core/control.c
index 6ceb5f977fcd..1e861f83e812 100644
--- a/sound/core/control.c
+++ b/sound/core/control.c
@@ -2219,7 +2219,6 @@ EXPORT_SYMBOL_GPL(snd_ctl_request_layer);
*/
void snd_ctl_register_layer(struct snd_ctl_layer_ops *lops)
{
- struct snd_card *card;
int card_number;
scoped_guard(rwsem_write, &snd_ctl_layer_rwsem) {
@@ -2227,11 +2226,12 @@ void snd_ctl_register_layer(struct snd_ctl_layer_ops *lops)
snd_ctl_layer = lops;
}
for (card_number = 0; card_number < SNDRV_CARDS; card_number++) {
- card = snd_card_ref(card_number);
+ struct snd_card *card __free(snd_card_unref) =
+ snd_card_ref(card_number);
+
if (card) {
scoped_guard(rwsem_read, &card->controls_rwsem)
lops->lregister(card);
- snd_card_unref(card);
}
}
}
diff --git a/sound/core/control_led.c b/sound/core/control_led.c
index e33dfcf863cf..ae903af93dd3 100644
--- a/sound/core/control_led.c
+++ b/sound/core/control_led.c
@@ -240,8 +240,6 @@ static void snd_ctl_led_notify(struct snd_card *card, unsigned int mask,
}
}
-DEFINE_FREE(snd_card_unref, struct snd_card *, if (_T) snd_card_unref(_T))
-
static int snd_ctl_led_set_id(int card_number, struct snd_ctl_elem_id *id,
unsigned int group, bool set)
{
@@ -257,6 +255,8 @@ static int snd_ctl_led_set_id(int card_number, struct snd_ctl_elem_id *id,
kctl = snd_ctl_find_id(card, id);
if (!kctl)
return -ENOENT;
+ if (!kctl->info || !kctl->get)
+ return -EINVAL;
ioff = snd_ctl_get_ioff(kctl, id);
vd = &kctl->vd[ioff];
access = vd->access & SNDRV_CTL_ELEM_ACCESS_LED_MASK;
@@ -758,18 +758,17 @@ static int __init snd_ctl_led_init(void)
static void __exit snd_ctl_led_exit(void)
{
struct snd_ctl_led *led;
- struct snd_card *card;
unsigned int group, card_number;
snd_ctl_disconnect_layer(&snd_ctl_led_lops);
for (card_number = 0; card_number < SNDRV_CARDS; card_number++) {
if (!snd_ctl_led_card_valid[card_number])
continue;
- card = snd_card_ref(card_number);
- if (card) {
+ struct snd_card *card __free(snd_card_unref) =
+ snd_card_ref(card_number);
+
+ if (card)
snd_ctl_led_sysfs_remove(card);
- snd_card_unref(card);
- }
}
for (group = 0; group < MAX_LED; group++) {
led = &snd_ctl_leds[group];
diff --git a/sound/core/init.c b/sound/core/init.c
index e0be60981285..c35e105ef132 100644
--- a/sound/core/init.c
+++ b/sound/core/init.c
@@ -579,6 +579,8 @@ EXPORT_SYMBOL_GPL(snd_card_disconnect_sync);
static int snd_card_do_free(struct snd_card *card)
{
+ bool managed = card->managed;
+
card->releasing = true;
#if IS_ENABLED(CONFIG_SND_MIXER_OSS)
if (snd_mixer_oss_notify_callback)
@@ -593,7 +595,7 @@ static int snd_card_do_free(struct snd_card *card)
}
if (card->release_completion)
complete(card->release_completion);
- if (!card->managed)
+ if (!managed)
kfree(card);
return 0;
}
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 706fab8ee038..7bfc9aafad3c 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -1441,6 +1441,8 @@ static int snd_pcm_pre_start(struct snd_pcm_substream *substream,
struct snd_pcm_runtime *runtime = substream->runtime;
if (runtime->state != SNDRV_PCM_STATE_PREPARED)
return -EBADFD;
+ if (atomic_read(&runtime->buffer_accessing) < 0)
+ return -EBADFD; /* during hw_params, hw_free or prepare */
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK &&
!snd_pcm_playback_data(substream))
return -EPIPE;
diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c
index 348ce1b7725e..81ede597d317 100644
--- a/sound/core/rawmidi.c
+++ b/sound/core/rawmidi.c
@@ -779,7 +779,7 @@ int snd_rawmidi_input_params(struct snd_rawmidi_substream *substream,
substream->framing = framing;
substream->clock_type = clock_type;
}
- return 0;
+ return err;
}
EXPORT_SYMBOL(snd_rawmidi_input_params);
diff --git a/sound/core/seq/seq_clientmgr.c b/sound/core/seq/seq_clientmgr.c
index 530a7d9be677..35f099e02ca7 100644
--- a/sound/core/seq/seq_clientmgr.c
+++ b/sound/core/seq/seq_clientmgr.c
@@ -579,7 +579,7 @@ static int bounce_error_event(struct snd_seq_client *client,
struct snd_seq_event *event,
int err, int atomic, int hop)
{
- struct snd_seq_event bounce_ev;
+ struct snd_seq_event bounce_ev, quoted;
int result;
if (client == NULL ||
@@ -595,15 +595,19 @@ static int bounce_error_event(struct snd_seq_client *client,
* For user clients, send SNDRV_SEQ_EVENT_BOUNCE with the
* original event embedded as variable-length data. This
* avoids exposing data.quote.event (a kernel pointer) to
- * userspace. The variable-length path in snd_seq_event_dup()
- * copies the event data from data.ext.ptr into chained cells,
- * and snd_seq_expand_var_event() copies only the data content
- * -- never the pointer -- to userspace.
+ * userspace. Sanitise the embedded copy too - a queued
+ * variable-length event carries the address of its own
+ * extension cell, and the payload goes out verbatim.
*/
+ quoted = *event;
+ if (snd_seq_ev_is_variable("ed)) {
+ quoted.data.ext.len &= ~SNDRV_SEQ_EXT_MASK;
+ quoted.data.ext.ptr = NULL;
+ }
bounce_ev.type = SNDRV_SEQ_EVENT_BOUNCE;
bounce_ev.flags = SNDRV_SEQ_EVENT_LENGTH_VARIABLE;
bounce_ev.data.ext.len = sizeof(struct snd_seq_event);
- bounce_ev.data.ext.ptr = (char *)event;
+ bounce_ev.data.ext.ptr = (char *)"ed;
} else {
/*
* For kernel clients, quote the event pointer directly.
diff --git a/sound/core/seq/seq_midi.c b/sound/core/seq/seq_midi.c
index 38cd631223c6..b18af999d350 100644
--- a/sound/core/seq/seq_midi.c
+++ b/sound/core/seq/seq_midi.c
@@ -42,9 +42,11 @@ struct seq_midisynth {
struct snd_rawmidi *rmidi;
int device;
int subdevice;
+ struct snd_rawmidi_substream __rcu *input_substream;
+ snd_use_lock_t input_use_lock; /* in-flight event_input users */
struct snd_rawmidi_file input_rfile;
- spinlock_t output_lock; /* protects output_rfile publication */
snd_use_lock_t output_use_lock; /* in-flight event_input users */
+ struct snd_rawmidi_substream __rcu *output_substream;
struct snd_rawmidi_file output_rfile;
int seq_client;
int seq_port;
@@ -76,6 +78,14 @@ static void snd_midi_input_event(struct snd_rawmidi_substream *substream)
msynth = runtime->private_data;
if (msynth == NULL)
return;
+
+ scoped_guard(rcu) {
+ if (rcu_dereference(msynth->input_substream) != substream)
+ return;
+
+ snd_use_lock_use(&msynth->input_use_lock);
+ }
+
memset(&ev, 0, sizeof(ev));
while (runtime->avail > 0) {
res = snd_rawmidi_kernel_read(substream, buf, sizeof(buf));
@@ -95,6 +105,8 @@ static void snd_midi_input_event(struct snd_rawmidi_substream *substream)
memset(&ev, 0, sizeof(ev));
}
}
+
+ snd_use_lock_free(&msynth->input_use_lock);
}
static int dump_midi(struct snd_rawmidi_substream *substream, const char *buf, int count)
@@ -134,8 +146,8 @@ static int event_process_midi(struct snd_seq_event *ev, int direct,
if (snd_BUG_ON(!msynth))
return -EINVAL;
- scoped_guard(spinlock_irqsave, &msynth->output_lock) {
- substream = msynth->output_rfile.output;
+ scoped_guard(rcu) {
+ substream = rcu_dereference(msynth->output_substream);
if (!substream)
return -ENODEV;
snd_use_lock_use(&msynth->output_use_lock);
@@ -177,7 +189,7 @@ static int snd_seq_midisynth_new(struct seq_midisynth *msynth,
msynth->card = card;
msynth->device = device;
msynth->subdevice = subdevice;
- spin_lock_init(&msynth->output_lock);
+ snd_use_lock_init(&msynth->input_use_lock);
snd_use_lock_init(&msynth->output_use_lock);
return 0;
}
@@ -188,28 +200,31 @@ static int midisynth_subscribe(void *private_data, struct snd_seq_port_subscribe
int err;
struct seq_midisynth *msynth = private_data;
struct snd_rawmidi_runtime *runtime;
+ struct snd_rawmidi_file rfile = {};
struct snd_rawmidi_params params;
/* open midi port */
err = snd_rawmidi_kernel_open(msynth->rmidi, msynth->subdevice,
SNDRV_RAWMIDI_LFLG_INPUT,
- &msynth->input_rfile);
+ &rfile);
if (err < 0) {
pr_debug("ALSA: seq_midi: midi input open failed!!!\n");
return err;
}
- runtime = msynth->input_rfile.input->runtime;
+ runtime = rfile.input->runtime;
memset(¶ms, 0, sizeof(params));
params.avail_min = 1;
params.buffer_size = input_buffer_size;
- err = snd_rawmidi_input_params(msynth->input_rfile.input, ¶ms);
+ err = snd_rawmidi_input_params(rfile.input, ¶ms);
if (err < 0) {
- snd_rawmidi_kernel_release(&msynth->input_rfile);
+ snd_rawmidi_kernel_release(&rfile);
return err;
}
snd_midi_event_reset_encode(msynth->parser);
runtime->event = snd_midi_input_event;
runtime->private_data = msynth;
+ msynth->input_rfile = rfile;
+ rcu_assign_pointer(msynth->input_substream, rfile.input);
snd_rawmidi_kernel_read(msynth->input_rfile.input, NULL, 0);
return 0;
}
@@ -219,10 +234,19 @@ static int midisynth_unsubscribe(void *private_data, struct snd_seq_port_subscri
{
int err;
struct seq_midisynth *msynth = private_data;
+ struct snd_rawmidi_file rfile;
- if (snd_BUG_ON(!msynth->input_rfile.input))
+ rcu_assign_pointer(msynth->input_substream, NULL);
+ synchronize_rcu();
+ snd_use_lock_sync(&msynth->input_use_lock);
+
+ rfile = msynth->input_rfile;
+ msynth->input_rfile = (struct snd_rawmidi_file){};
+
+ if (snd_BUG_ON(!rfile.input))
return -EINVAL;
- err = snd_rawmidi_kernel_release(&msynth->input_rfile);
+
+ err = snd_rawmidi_kernel_release(&rfile);
return err;
}
@@ -252,8 +276,8 @@ static int midisynth_use(void *private_data, struct snd_seq_port_subscribe *info
return err;
}
snd_midi_event_reset_decode(msynth->parser);
- scoped_guard(spinlock_irqsave, &msynth->output_lock)
- msynth->output_rfile = rfile;
+ msynth->output_rfile = rfile;
+ rcu_assign_pointer(msynth->output_substream, rfile.output);
return 0;
}
@@ -261,17 +285,16 @@ static int midisynth_use(void *private_data, struct snd_seq_port_subscribe *info
static int midisynth_unuse(void *private_data, struct snd_seq_port_subscribe *info)
{
struct seq_midisynth *msynth = private_data;
- struct snd_rawmidi_file rfile = {};
+ struct snd_rawmidi_file rfile;
- scoped_guard(spinlock_irqsave, &msynth->output_lock) {
- rfile = msynth->output_rfile;
- msynth->output_rfile = (struct snd_rawmidi_file){};
- }
+ rcu_assign_pointer(msynth->output_substream, NULL);
+ synchronize_rcu();
+ snd_use_lock_sync(&msynth->output_use_lock);
+ rfile = msynth->output_rfile;
+ msynth->output_rfile = (struct snd_rawmidi_file){};
if (snd_BUG_ON(!rfile.output))
return -EINVAL;
-
- snd_use_lock_sync(&msynth->output_use_lock);
snd_rawmidi_drain_output(rfile.output);
return snd_rawmidi_kernel_release(&rfile);
}
diff --git a/sound/parisc/harmony.c b/sound/parisc/harmony.c
index db9c296dd688..b9cc9e5e7056 100644
--- a/sound/parisc/harmony.c
+++ b/sound/parisc/harmony.c
@@ -889,6 +889,9 @@ snd_harmony_create(struct snd_card *card,
goto free_and_ret;
}
+ spin_lock_init(&h->mixer_lock);
+ spin_lock_init(&h->lock);
+
err = request_irq(padev->irq, snd_harmony_interrupt, 0,
"harmony", h);
if (err) {
@@ -898,9 +901,6 @@ snd_harmony_create(struct snd_card *card,
}
h->irq = padev->irq;
- spin_lock_init(&h->mixer_lock);
- spin_lock_init(&h->lock);
-
err = snd_device_new(card, SNDRV_DEV_LOWLEVEL, h, &ops);
if (err < 0)
goto free_and_ret;
diff --git a/sound/pci/asihpi/hpi6000.c b/sound/pci/asihpi/hpi6000.c
index 72aa135d69f8..8aa5832f1f2a 100644
--- a/sound/pci/asihpi/hpi6000.c
+++ b/sound/pci/asihpi/hpi6000.c
@@ -537,6 +537,11 @@ static short create_adapter_obj(struct hpi_adapter_obj *pao,
hr1.size = sizeof(hr1);
error = hpi6000_message_response_sequence(pao, 0, &hm, &hr0);
+ if (error) {
+ HPI_DEBUG_LOG(ERROR, "message transport error %d\n",
+ error);
+ return error;
+ }
if (hr0.error) {
HPI_DEBUG_LOG(DEBUG, "message error %d\n", hr0.error);
return hr0.error;
diff --git a/sound/pci/hda/hda_proc.c b/sound/pci/hda/hda_proc.c
index 00c2eeb2c472..e8a4cf3c84d8 100644
--- a/sound/pci/hda/hda_proc.c
+++ b/sound/pci/hda/hda_proc.c
@@ -624,7 +624,7 @@ static void print_conn_list(struct snd_info_buffer *buffer,
/* Get Cache connections info */
cache_len = snd_hda_get_conn_list(codec, nid, &list);
if (cache_len >= 0 && (cache_len != conn_len ||
- memcmp(list, conn, conn_len) != 0)) {
+ memcmp(list, conn, conn_len * sizeof(*conn)) != 0)) {
snd_iprintf(buffer, " In-driver Connection: %d\n", cache_len);
if (cache_len > 0) {
snd_iprintf(buffer, " ");
diff --git a/sound/pci/ice1712/ice1712.c b/sound/pci/ice1712/ice1712.c
index ad2180659187..45cb3e7b8d21 100644
--- a/sound/pci/ice1712/ice1712.c
+++ b/sound/pci/ice1712/ice1712.c
@@ -2549,7 +2549,7 @@ static int snd_ice1712_probe(struct pci_dev *pci,
const struct pci_device_id *pci_id)
{
static int dev;
- struct snd_card *card;
+ struct snd_card *card __free(snd_card_unref) = NULL;
struct snd_ice1712 *ice;
int pcm_dev = 0, err;
const struct snd_ice1712_card_info * const *tbl, *c;
@@ -2561,8 +2561,8 @@ static int snd_ice1712_probe(struct pci_dev *pci,
return -ENOENT;
}
- err = snd_card_new(&pci->dev, index[dev], id[dev], THIS_MODULE,
- sizeof(*ice), &card);
+ err = snd_devm_card_new(&pci->dev, index[dev], id[dev], THIS_MODULE,
+ sizeof(*ice), &card);
if (err < 0)
return err;
ice = card->private_data;
@@ -2666,6 +2666,7 @@ static int snd_ice1712_probe(struct pci_dev *pci,
if (err < 0)
return err;
pci_set_drvdata(pci, card);
+ card = NULL; /* probe succeeded, don't release as error */
dev++;
return 0;
}
diff --git a/sound/pci/via82xx_modem.c b/sound/pci/via82xx_modem.c
index a0a49b8d1511..60235e953a62 100644
--- a/sound/pci/via82xx_modem.c
+++ b/sound/pci/via82xx_modem.c
@@ -576,24 +576,18 @@ static inline unsigned int calc_linear_pos(struct via82xx_modem *chip,
viadev->bufsize2, viadev->idx_table[idx].offset,
viadev->idx_table[idx].size, count);
#endif
- if (count && size < count) {
+ if (! count)
+ /* bogus count 0 on the DMA boundary? */
+ res = viadev->idx_table[idx].offset;
+ else
+ /* count register returns full size
+ * when end of buffer is reached
+ */
+ res = viadev->idx_table[idx].offset + size;
+ if (check_invalid_pos(viadev, res)) {
dev_dbg(chip->card->dev,
- "invalid via82xx_cur_ptr, using last valid pointer\n");
+ "invalid via82xx_cur_ptr (2), using last valid pointer\n");
res = viadev->lastpos;
- } else {
- if (! count)
- /* bogus count 0 on the DMA boundary? */
- res = viadev->idx_table[idx].offset;
- else
- /* count register returns full size
- * when end of buffer is reached
- */
- res = viadev->idx_table[idx].offset + size;
- if (check_invalid_pos(viadev, res)) {
- dev_dbg(chip->card->dev,
- "invalid via82xx_cur_ptr (2), using last valid pointer\n");
- res = viadev->lastpos;
- }
}
}
viadev->lastpos = res; /* remember the last position */
diff --git a/sound/soc/apple/mca.c b/sound/soc/apple/mca.c
index 4a4ec1c09e13..d89e57414a5d 100644
--- a/sound/soc/apple/mca.c
+++ b/sound/soc/apple/mca.c
@@ -210,10 +210,10 @@ static void mca_fe_early_trigger(struct snd_pcm_substream *substream, int cmd,
SERDES_STATUS_EN | SERDES_STATUS_RST,
SERDES_STATUS_RST);
/*
- * Experiments suggest that it takes at most ~1 us
- * for the bit to clear, so wait 2 us for good measure.
+ * The SERDES cluster needs a bit of time to reset itself
+ * and settle before we start poking it. This is... slow...
*/
- udelay(2);
+ udelay(25);
WARN_ON(readl_relaxed(cl->base + serdes_unit + REG_SERDES_STATUS) &
SERDES_STATUS_RST);
mca_modify(cl, serdes_conf, SERDES_CONF_SYNC_SEL,
diff --git a/sound/soc/codecs/cs35l33.c b/sound/soc/codecs/cs35l33.c
index b03aab147530..b35fac87b19a 100644
--- a/sound/soc/codecs/cs35l33.c
+++ b/sound/soc/codecs/cs35l33.c
@@ -40,6 +40,7 @@ struct cs35l33_private {
struct regmap *regmap;
struct gpio_desc *reset_gpio;
bool amp_cal;
+ bool irq_requested;
int mclk_int;
struct regulator_bulk_data core_supplies[2];
int num_core_supplies;
@@ -881,6 +882,9 @@ static int __maybe_unused cs35l33_runtime_resume(struct device *dev)
goto err;
}
+ if (cs35l33->irq_requested)
+ enable_irq(to_i2c_client(dev)->irq);
+
return 0;
err:
@@ -900,6 +904,10 @@ static int __maybe_unused cs35l33_runtime_suspend(struct device *dev)
/* redo the calibration in next power up */
cs35l33->amp_cal = false;
+ /* Drain and block the threaded IRQ before cache_only/power-off. */
+ if (cs35l33->irq_requested)
+ disable_irq(to_i2c_client(dev)->irq);
+
regcache_cache_only(cs35l33->regmap, true);
regcache_mark_dirty(cs35l33->regmap);
regulator_bulk_disable(cs35l33->num_core_supplies,
@@ -1156,10 +1164,12 @@ static int cs35l33_i2c_probe(struct i2c_client *i2c_client)
}
ret = devm_request_threaded_irq(&i2c_client->dev, i2c_client->irq, NULL,
- cs35l33_irq_thread, IRQF_ONESHOT | IRQF_TRIGGER_LOW,
- "cs35l33", cs35l33);
+ cs35l33_irq_thread, IRQF_ONESHOT | IRQF_TRIGGER_LOW,
+ "cs35l33", cs35l33);
if (ret != 0)
dev_warn(&i2c_client->dev, "Failed to request IRQ: %d\n", ret);
+ else
+ cs35l33->irq_requested = true;
/* We could issue !RST or skip it based on AMP topology */
cs35l33->reset_gpio = devm_gpiod_get_optional(&i2c_client->dev,
diff --git a/sound/soc/codecs/cs35l34.c b/sound/soc/codecs/cs35l34.c
index 287b27476a10..da9ccf349bbb 100644
--- a/sound/soc/codecs/cs35l34.c
+++ b/sound/soc/codecs/cs35l34.c
@@ -45,6 +45,7 @@ struct cs35l34_private {
int num_core_supplies;
int mclk_int;
bool tdm_mode;
+ bool irq_requested;
struct gpio_desc *reset_gpio; /* Active-low reset GPIO */
};
@@ -1032,10 +1033,12 @@ static int cs35l34_i2c_probe(struct i2c_client *i2c_client)
}
ret = devm_request_threaded_irq(&i2c_client->dev, i2c_client->irq, NULL,
- cs35l34_irq_thread, IRQF_ONESHOT | IRQF_TRIGGER_LOW,
- "cs35l34", cs35l34);
+ cs35l34_irq_thread, IRQF_ONESHOT | IRQF_TRIGGER_LOW,
+ "cs35l34", cs35l34);
if (ret != 0)
dev_err(&i2c_client->dev, "Failed to request IRQ: %d\n", ret);
+ else
+ cs35l34->irq_requested = true;
cs35l34->reset_gpio = devm_gpiod_get_optional(&i2c_client->dev,
"reset", GPIOD_OUT_LOW);
@@ -1140,6 +1143,9 @@ static int __maybe_unused cs35l34_runtime_resume(struct device *dev)
dev_err(dev, "Failed to restore register cache\n");
goto err;
}
+
+ if (cs35l34->irq_requested)
+ enable_irq(to_i2c_client(dev)->irq);
return 0;
err:
regcache_cache_only(cs35l34->regmap, true);
@@ -1153,6 +1159,10 @@ static int __maybe_unused cs35l34_runtime_suspend(struct device *dev)
{
struct cs35l34_private *cs35l34 = dev_get_drvdata(dev);
+ /* Drain and block the threaded IRQ before cache_only/power-off. */
+ if (cs35l34->irq_requested)
+ disable_irq(to_i2c_client(dev)->irq);
+
regcache_cache_only(cs35l34->regmap, true);
regcache_mark_dirty(cs35l34->regmap);
diff --git a/sound/soc/codecs/hdac_hda.c b/sound/soc/codecs/hdac_hda.c
index 29c88de5508b..b3e5f4629ac1 100644
--- a/sound/soc/codecs/hdac_hda.c
+++ b/sound/soc/codecs/hdac_hda.c
@@ -643,10 +643,8 @@ static int hdac_hda_dev_probe(struct hdac_device *hdev)
&hdac_hda_codec, hdac_hda_dais,
ARRAY_SIZE(hdac_hda_dais));
- if (ret < 0) {
+ if (ret < 0)
dev_err(&hdev->dev, "%s: failed to register HDA codec %d\n", __func__, ret);
- return ret;
- }
snd_hdac_ext_bus_link_put(hdev->bus, hlink);
diff --git a/sound/soc/codecs/rt700-sdw.c b/sound/soc/codecs/rt700-sdw.c
index 24cb895b759f..366a09aa32ee 100644
--- a/sound/soc/codecs/rt700-sdw.c
+++ b/sound/soc/codecs/rt700-sdw.c
@@ -459,10 +459,8 @@ static int rt700_sdw_remove(struct sdw_slave *slave)
{
struct rt700_priv *rt700 = dev_get_drvdata(&slave->dev);
- if (rt700->hw_init) {
- cancel_delayed_work_sync(&rt700->jack_detect_work);
- cancel_delayed_work_sync(&rt700->jack_btn_check_work);
- }
+ cancel_delayed_work_sync(&rt700->jack_detect_work);
+ cancel_delayed_work_sync(&rt700->jack_btn_check_work);
pm_runtime_disable(&slave->dev);
diff --git a/sound/soc/fsl/fsl-asoc-card.c b/sound/soc/fsl/fsl-asoc-card.c
index a0c2ce84c32b..5a69f9d255ac 100644
--- a/sound/soc/fsl/fsl-asoc-card.c
+++ b/sound/soc/fsl/fsl-asoc-card.c
@@ -648,8 +648,8 @@ static int fsl_asoc_card_probe(struct platform_device *pdev)
cpu_pdev = of_find_device_by_node(cpu_np);
if (!cpu_pdev) {
- dev_err(&pdev->dev, "failed to find CPU DAI device\n");
- ret = -EINVAL;
+ ret = dev_err_probe(&pdev->dev, -EPROBE_DEFER,
+ "failed to find CPU DAI device\n");
goto fail;
}
diff --git a/sound/soc/fsl/fsl_audmix.c b/sound/soc/fsl/fsl_audmix.c
index d9b0bd61755d..c9a7a41ef388 100644
--- a/sound/soc/fsl/fsl_audmix.c
+++ b/sound/soc/fsl/fsl_audmix.c
@@ -454,6 +454,9 @@ static const struct of_device_id fsl_audmix_ids[] = {
};
MODULE_DEVICE_TABLE(of, fsl_audmix_ids);
+static int fsl_audmix_runtime_resume(struct device *dev);
+static int fsl_audmix_runtime_suspend(struct device *dev);
+
static int fsl_audmix_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -485,13 +488,25 @@ static int fsl_audmix_probe(struct platform_device *pdev)
spin_lock_init(&priv->lock);
platform_set_drvdata(pdev, priv);
pm_runtime_enable(dev);
+ if (!pm_runtime_enabled(dev)) {
+ ret = fsl_audmix_runtime_resume(dev);
+ if (ret)
+ goto err_disable_pm;
+ }
+
+ ret = pm_runtime_resume_and_get(dev);
+ if (ret < 0)
+ goto err_pm_get_sync;
+
+ /* To enable regmap cache only when runtime PM enabled */
+ pm_runtime_put(dev);
ret = devm_snd_soc_register_component(dev, &fsl_audmix_component,
fsl_audmix_dai,
ARRAY_SIZE(fsl_audmix_dai));
if (ret) {
dev_err(dev, "failed to register ASoC DAI\n");
- goto err_disable_pm;
+ goto err_pm_get_sync;
}
/*
@@ -503,12 +518,15 @@ static int fsl_audmix_probe(struct platform_device *pdev)
if (IS_ERR(priv->pdev)) {
ret = PTR_ERR(priv->pdev);
dev_err(dev, "failed to register platform: %d\n", ret);
- goto err_disable_pm;
+ goto err_pm_get_sync;
}
}
return 0;
+err_pm_get_sync:
+ if (!pm_runtime_status_suspended(dev))
+ fsl_audmix_runtime_suspend(dev);
err_disable_pm:
pm_runtime_disable(dev);
return ret;
@@ -519,6 +537,8 @@ static void fsl_audmix_remove(struct platform_device *pdev)
struct fsl_audmix *priv = dev_get_drvdata(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ if (!pm_runtime_status_suspended(&pdev->dev))
+ fsl_audmix_runtime_suspend(&pdev->dev);
if (priv->pdev)
platform_device_unregister(priv->pdev);
diff --git a/sound/soc/fsl/mpc5200_psc_i2s.c b/sound/soc/fsl/mpc5200_psc_i2s.c
index 9ad44eeed6ad..7831136f4f12 100644
--- a/sound/soc/fsl/mpc5200_psc_i2s.c
+++ b/sound/soc/fsl/mpc5200_psc_i2s.c
@@ -170,6 +170,7 @@ static int psc_i2s_of_probe(struct platform_device *op)
psc_i2s_dai, ARRAY_SIZE(psc_i2s_dai));
if (rc != 0) {
pr_err("Failed to register DAI\n");
+ mpc5200_audio_dma_destroy(op);
return rc;
}
diff --git a/sound/soc/intel/atom/sst/sst_pci.c b/sound/soc/intel/atom/sst/sst_pci.c
index d1e64c3500be..296bea33e16f 100644
--- a/sound/soc/intel/atom/sst/sst_pci.c
+++ b/sound/soc/intel/atom/sst/sst_pci.c
@@ -140,13 +140,15 @@ static int intel_sst_probe(struct pci_dev *pci,
sst_drv_ctx->pci = pci_dev_get(pci);
ret = sst_platform_get_resources(sst_drv_ctx);
if (ret < 0)
- goto do_free_drv_ctx;
+ goto do_put_pci;
pci_set_drvdata(pci, sst_drv_ctx);
sst_configure_runtime_pm(sst_drv_ctx);
return ret;
+do_put_pci:
+ pci_dev_put(sst_drv_ctx->pci);
do_free_drv_ctx:
sst_context_cleanup(sst_drv_ctx);
dev_err(sst_drv_ctx->dev, "Probe failed with %d\n", ret);
diff --git a/sound/soc/loongson/loongson_card.c b/sound/soc/loongson/loongson_card.c
index 7910d5d9ac4f..ea895fe6b5e9 100644
--- a/sound/soc/loongson/loongson_card.c
+++ b/sound/soc/loongson/loongson_card.c
@@ -91,7 +91,7 @@ static int loongson_card_parse_acpi(struct loongson_card_data *data)
const char *codec_dai_name;
struct acpi_device *adev;
struct device *phy_dev;
- int i;
+ int i, ret;
/* fixup platform name based on reference node */
adev = loongson_card_acpi_find_device(card, "cpu");
@@ -108,7 +108,9 @@ static int loongson_card_parse_acpi(struct loongson_card_data *data)
return -ENOENT;
snprintf(codec_name, sizeof(codec_name), "i2c-%s", acpi_dev_name(adev));
- device_property_read_string(card->dev, "codec-dai-name", &codec_dai_name);
+ ret = device_property_read_string(card->dev, "codec-dai-name", &codec_dai_name);
+ if (ret)
+ return ret;
for (i = 0; i < card->num_links; i++) {
loongson_dai_links[i].platforms->name = dev_name(phy_dev);
diff --git a/sound/soc/meson/meson-card-utils.c b/sound/soc/meson/meson-card-utils.c
index ad38c74166a4..c2a2531b30f9 100644
--- a/sound/soc/meson/meson-card-utils.c
+++ b/sound/soc/meson/meson-card-utils.c
@@ -50,25 +50,20 @@ int meson_card_reallocate_links(struct snd_soc_card *card,
num_links * sizeof(*priv->card.dai_link),
GFP_KERNEL | __GFP_ZERO);
if (!links)
- goto err_links;
+ return -ENOMEM;
+
+ priv->card.dai_link = links;
+ priv->card.num_links = num_links;
ldata = krealloc(priv->link_data,
num_links * sizeof(*priv->link_data),
GFP_KERNEL | __GFP_ZERO);
+ /* meson_card_clean_references() will free the links on this error path */
if (!ldata)
- goto err_ldata;
+ return -ENOMEM;
- priv->card.dai_link = links;
priv->link_data = ldata;
- priv->card.num_links = num_links;
return 0;
-
-err_ldata:
- kfree(links);
-err_links:
- dev_err(priv->card.dev, "failed to allocate links\n");
- return -ENOMEM;
-
}
EXPORT_SYMBOL_GPL(meson_card_reallocate_links);
diff --git a/sound/soc/pxa/pxa-ssp.c b/sound/soc/pxa/pxa-ssp.c
index b8a3cb8b7597..ba8cae8e85ef 100644
--- a/sound/soc/pxa/pxa-ssp.c
+++ b/sound/soc/pxa/pxa-ssp.c
@@ -769,13 +769,10 @@ static int pxa_ssp_probe(struct snd_soc_dai *dai)
goto err_priv;
}
- priv->extclk = devm_clk_get(dev, "extclk");
+ priv->extclk = devm_clk_get_optional(dev, "extclk");
if (IS_ERR(priv->extclk)) {
ret = PTR_ERR(priv->extclk);
- if (ret == -EPROBE_DEFER)
- goto err_priv;
-
- priv->extclk = NULL;
+ goto err_priv;
}
} else {
priv->ssp = pxa_ssp_request(dai->id + 1, "SoC audio");
diff --git a/sound/soc/qcom/qdsp6/q6apm.c b/sound/soc/qcom/qdsp6/q6apm.c
index e2ad6dc2dc85..b6d957f7c7ab 100644
--- a/sound/soc/qcom/qdsp6/q6apm.c
+++ b/sound/soc/qcom/qdsp6/q6apm.c
@@ -697,14 +697,17 @@ EXPORT_SYMBOL_GPL(q6apm_graph_prepare);
int q6apm_graph_start(struct q6apm_graph *graph)
{
struct audioreach_graph *ar_graph = graph->ar_graph;
- int ret = 0;
+ int ret;
- if (ar_graph->start_count == 0)
+ if (ar_graph->start_count == 0) {
ret = audioreach_graph_mgmt_cmd(ar_graph, APM_CMD_GRAPH_START);
+ if (ret)
+ return ret;
+ }
ar_graph->start_count++;
- return ret;
+ return 0;
}
EXPORT_SYMBOL_GPL(q6apm_graph_start);
@@ -712,6 +715,9 @@ int q6apm_graph_stop(struct q6apm_graph *graph)
{
struct audioreach_graph *ar_graph = graph->ar_graph;
+ if (ar_graph->start_count == 0)
+ return 0;
+
if (--ar_graph->start_count > 0)
return 0;
diff --git a/sound/soc/samsung/aries_wm8994.c b/sound/soc/samsung/aries_wm8994.c
index 01716df0c842..9c45c64fccd5 100644
--- a/sound/soc/samsung/aries_wm8994.c
+++ b/sound/soc/samsung/aries_wm8994.c
@@ -658,6 +658,7 @@ static int aries_audio_probe(struct platform_device *pdev)
goto out;
}
+ of_node_get(aries_dai[0].cpus->of_node);
aries_dai[0].platforms->of_node = aries_dai[0].cpus->of_node;
/* Set CPU of_node for BT DAI */
diff --git a/sound/soc/soc-dapm.c b/sound/soc/soc-dapm.c
index 196799b2fe24..82f745628838 100644
--- a/sound/soc/soc-dapm.c
+++ b/sound/soc/soc-dapm.c
@@ -3582,7 +3582,7 @@ int snd_soc_dapm_put_enum_double(struct snd_kcontrol *kcontrol,
val = snd_soc_enum_item_to_val(e, item[0]) << e->shift_l;
mask = e->mask << e->shift_l;
if (e->shift_l != e->shift_r) {
- if (item[1] > e->items)
+ if (item[1] >= e->items)
return -EINVAL;
val |= snd_soc_enum_item_to_val(e, item[1]) << e->shift_r;
mask |= e->mask << e->shift_r;
diff --git a/sound/soc/sof/topology.c b/sound/soc/sof/topology.c
index e6831d5aa15e..b011490d3991 100644
--- a/sound/soc/sof/topology.c
+++ b/sound/soc/sof/topology.c
@@ -827,6 +827,7 @@ static int sof_control_load_volume(struct snd_soc_component *scomp,
struct snd_soc_tplg_mixer_control *mc =
container_of(hdr, struct snd_soc_tplg_mixer_control, hdr);
int tlv[SOF_TLV_ITEMS];
+ u32 min, max;
unsigned int mask;
int ret;
@@ -834,6 +835,11 @@ static int sof_control_load_volume(struct snd_soc_component *scomp,
if (le32_to_cpu(mc->num_channels) > SND_SOC_TPLG_MAX_CHAN)
return -EINVAL;
+ min = le32_to_cpu(mc->min);
+ max = le32_to_cpu(mc->max);
+ if (min > max || max >= INT_MAX)
+ return -EINVAL;
+
/*
* If control has more than 2 channels we need to override the info. This is because even if
* ASoC layer has defined topology's max channel count to SND_SOC_TPLG_MAX_CHAN = 8, the
@@ -844,12 +850,12 @@ static int sof_control_load_volume(struct snd_soc_component *scomp,
kc->info = snd_sof_volume_info;
scontrol->comp_id = sdev->next_comp_id;
- scontrol->min_volume_step = le32_to_cpu(mc->min);
- scontrol->max_volume_step = le32_to_cpu(mc->max);
+ scontrol->min_volume_step = min;
+ scontrol->max_volume_step = max;
scontrol->num_channels = le32_to_cpu(mc->num_channels);
- scontrol->max = le32_to_cpu(mc->max);
- if (le32_to_cpu(mc->max) == 1)
+ scontrol->max = max;
+ if (max == 1)
goto skip;
/* extract tlv data */
@@ -859,7 +865,7 @@ static int sof_control_load_volume(struct snd_soc_component *scomp,
}
/* set up volume table */
- ret = set_up_volume_table(scontrol, tlv, le32_to_cpu(mc->max) + 1);
+ ret = set_up_volume_table(scontrol, tlv, max + 1);
if (ret < 0) {
dev_err(scomp->dev, "error: setting up volume table\n");
return ret;
@@ -892,7 +898,7 @@ static int sof_control_load_volume(struct snd_soc_component *scomp,
return 0;
err:
- if (le32_to_cpu(mc->max) > 1)
+ if (max > 1)
kfree(scontrol->volume_table);
return ret;
diff --git a/sound/soc/xilinx/xlnx_formatter_pcm.c b/sound/soc/xilinx/xlnx_formatter_pcm.c
index 45b178148b98..4d90e96a677c 100644
--- a/sound/soc/xilinx/xlnx_formatter_pcm.c
+++ b/sound/soc/xilinx/xlnx_formatter_pcm.c
@@ -384,7 +384,7 @@ static int xlnx_formatter_pcm_open(struct snd_soc_component *component,
if (err) {
dev_err(component->dev,
"Unable to set constraint on period bytes\n");
- return err;
+ goto error;
}
/* Resize the buffer bytes as divisible by 64 */
@@ -394,7 +394,7 @@ static int xlnx_formatter_pcm_open(struct snd_soc_component *component,
if (err) {
dev_err(component->dev,
"Unable to set constraint on buffer bytes\n");
- return err;
+ goto error;
}
/* Set periods as integer multiple */
@@ -403,7 +403,7 @@ static int xlnx_formatter_pcm_open(struct snd_soc_component *component,
if (err < 0) {
dev_err(component->dev,
"Unable to set constraint on periods to be integer\n");
- return err;
+ goto error;
}
/* enable DMA IOC irq */
@@ -412,6 +412,14 @@ static int xlnx_formatter_pcm_open(struct snd_soc_component *component,
writel(val, stream_data->mmio + XLNX_AUD_CTRL);
return 0;
+
+error:
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
+ adata->play_stream = NULL;
+ else
+ adata->capture_stream = NULL;
+ kfree(stream_data);
+ return err;
}
static int xlnx_formatter_pcm_close(struct snd_soc_component *component,
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 1b6109041c11..cb9bcfa22afc 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -974,6 +974,8 @@ static void snd_usbmidi_us122l_output(struct snd_usb_midi_out_endpoint *ep,
default:
count = 2;
}
+ if (ep->max_transfer < count)
+ return;
count = snd_rawmidi_transmit(ep->ports[0].substream,
urb->transfer_buffer,
count);
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index b89bd792c1d5..a9c8af7db19a 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -659,8 +659,6 @@ static int do_show_subset(int argc, char **argv)
show_map_close_json(fds[i], &info);
else
show_map_close_plain(fds[i], &info);
-
- close(fds[i]);
}
if (json_output && nb_fds > 1)
jsonw_end_array(json_wtr); /* root array */
@@ -787,6 +785,12 @@ static int maps_have_btf(int *fds, int nb_fds)
static struct btf *btf_vmlinux;
+static void free_btf_vmlinux(void)
+{
+ btf__free(btf_vmlinux);
+ btf_vmlinux = NULL;
+}
+
static int get_map_kv_btf(const struct bpf_map_info *info, struct btf **btf)
{
int err = 0;
@@ -886,7 +890,6 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
exit_free:
free(key);
free(value);
- close(fd);
free_map_kv_btf(btf);
return err;
@@ -935,6 +938,7 @@ static int do_dump(int argc, char **argv)
for (i = 0; i < nb_fds; i++) {
if (bpf_map_get_info_by_fd(fds[i], &info, &len)) {
p_err("can't get map info: %s", strerror(errno));
+ err = -1;
break;
}
err = map_dump(fds[i], &info, wtr, nb_fds > 1);
@@ -955,7 +959,7 @@ static int do_dump(int argc, char **argv)
close(fds[i]);
exit_free:
free(fds);
- btf__free(btf_vmlinux);
+ free_btf_vmlinux();
return err;
}
@@ -1046,7 +1050,7 @@ static void print_key_value(struct bpf_map_info *info, void *key,
btf_wtr = get_btf_writer();
if (!btf_wtr) {
p_info("failed to create json writer for btf. falling back to plain output");
- btf__free(btf);
+ free_map_kv_btf(btf);
btf = NULL;
print_entry_plain(info, key, value);
} else {
@@ -1062,7 +1066,7 @@ static void print_key_value(struct bpf_map_info *info, void *key,
} else {
print_entry_plain(info, key, value);
}
- btf__free(btf);
+ free_map_kv_btf(btf);
}
static int do_lookup(int argc, char **argv)
diff --git a/tools/bpf/bpftool/struct_ops.c b/tools/bpf/bpftool/struct_ops.c
index aa43dead249c..835e5e561f7f 100644
--- a/tools/bpf/bpftool/struct_ops.c
+++ b/tools/bpf/bpftool/struct_ops.c
@@ -643,6 +643,10 @@ int do_struct_ops(int argc, char **argv)
err = cmd_select(cmds, argc, argv, do_help);
btf__free(btf_vmlinux);
+ btf_vmlinux = NULL;
+ map_info_type = NULL;
+ map_info_alloc_len = 0;
+ map_info_type_id = 0;
return err;
}
diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index ffd117135094..24e9acf84708 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -110,7 +110,8 @@ FEATURE_TESTS_EXTRA := \
libbpf-bpf_create_map \
libpfm4 \
libdebuginfod \
- clang-bpf-co-re
+ clang-bpf-co-re \
+ bpftool-skeletons
FEATURE_TESTS ?= $(FEATURE_TESTS_BASIC)
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index 592ca17b4b74..13eec60f1cec 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -426,6 +426,9 @@ $(OUTPUT)test-file-handle.bin:
$(OUTPUT)test-libpfm4.bin:
$(BUILD) -lpfm
+$(OUTPUT)test-bpftool-skeletons.bin:
+ $(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons' \
+ > $(@:.bin=.make.output) 2>&1
###############################
clean:
diff --git a/tools/include/nolibc/arch-powerpc.h b/tools/include/nolibc/arch-powerpc.h
index ee2fdb8d601d..f468b6750543 100644
--- a/tools/include/nolibc/arch-powerpc.h
+++ b/tools/include/nolibc/arch-powerpc.h
@@ -23,7 +23,7 @@
*/
#define _NOLIBC_SYSCALL_CLOBBERLIST \
- "memory", "cr0", "r12", "r11", "r10", "r9"
+ "memory", "cr0", "ctr", "xer", "r12", "r11", "r10", "r9"
#define my_syscall0(num) \
({ \
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index c8fb6abfeca9..1d6dc90b2345 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1779,6 +1779,13 @@ union bpf_attr {
};
__u64 expected_revision;
} netkit;
+ struct {
+ union {
+ __u32 relative_fd;
+ __u32 relative_id;
+ };
+ __u64 expected_revision;
+ } cgroup;
};
} link_create;
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 9bbf7a3b87de..ffb0d1f7e9a0 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -11946,13 +11946,14 @@ static const char *arch_specific_lib_paths(void)
/* Get full path to program/shared library. */
static int resolve_full_path(const char *file, char *result, size_t result_sz)
{
- const char *search_paths[3] = {};
+ const char *search_paths[4] = {};
int i, perm;
if (str_has_sfx(file, ".so") || strstr(file, ".so.")) {
search_paths[0] = getenv("LD_LIBRARY_PATH");
search_paths[1] = "/usr/lib64:/usr/lib";
search_paths[2] = arch_specific_lib_paths();
+ search_paths[3] = "/lib64:/lib";
perm = R_OK;
} else {
search_paths[0] = getenv("PATH");
diff --git a/tools/net/sunrpc/xdrgen/README b/tools/net/sunrpc/xdrgen/README
index 92f7738ad50c..27218a78ab40 100644
--- a/tools/net/sunrpc/xdrgen/README
+++ b/tools/net/sunrpc/xdrgen/README
@@ -150,6 +150,23 @@ Pragma directives specify exceptions to the normal generation of
encoding and decoding functions. Currently one directive is
implemented: "public".
+Pragma big_endian
+------ ----------
+
+ pragma big_endian <enum> ;
+
+For variables that might contain only a small number values, it
+is more efficient to avoid the byte-swap when encoding or decoding
+on little-endian machines. Such is often the case with error status
+codes. For example:
+
+ pragma big_endian nfsstat3;
+
+In this case, when generating an XDR struct or union containing a
+field of type "nfsstat3", xdrgen will make the type of that field
+"__be32" instead of "enum nfsstat3". XDR unions then switch on the
+non-byte-swapped value of that field.
+
Pragma exclude
------ -------
diff --git a/tools/net/sunrpc/xdrgen/generators/enum.py b/tools/net/sunrpc/xdrgen/generators/enum.py
index 855e43f4ae38..b05553981f27 100644
--- a/tools/net/sunrpc/xdrgen/generators/enum.py
+++ b/tools/net/sunrpc/xdrgen/generators/enum.py
@@ -4,7 +4,7 @@
"""Generate code to handle XDR enum types"""
from generators import SourceGenerator, create_jinja2_environment
-from xdr_ast import _XdrEnum, public_apis
+from xdr_ast import _XdrEnum, public_apis, big_endian
class XdrEnumGenerator(SourceGenerator):
@@ -30,15 +30,24 @@ class XdrEnumGenerator(SourceGenerator):
for enumerator in node.enumerators:
print(template.render(name=enumerator.name, value=enumerator.value))
- template = self.environment.get_template("definition/close.j2")
+ if node.name in big_endian:
+ template = self.environment.get_template("definition/close_be.j2")
+ else:
+ template = self.environment.get_template("definition/close.j2")
print(template.render(name=node.name))
def emit_decoder(self, node: _XdrEnum) -> None:
"""Emit one decoder function for an XDR enum type"""
- template = self.environment.get_template("decoder/enum.j2")
+ if node.name in big_endian:
+ template = self.environment.get_template("decoder/enum_be.j2")
+ else:
+ template = self.environment.get_template("decoder/enum.j2")
print(template.render(name=node.name))
def emit_encoder(self, node: _XdrEnum) -> None:
"""Emit one encoder function for an XDR enum type"""
- template = self.environment.get_template("encoder/enum.j2")
+ if node.name in big_endian:
+ template = self.environment.get_template("encoder/enum_be.j2")
+ else:
+ template = self.environment.get_template("encoder/enum.j2")
print(template.render(name=node.name))
diff --git a/tools/net/sunrpc/xdrgen/generators/union.py b/tools/net/sunrpc/xdrgen/generators/union.py
index 7974967bbb9f..4522a5b7a943 100644
--- a/tools/net/sunrpc/xdrgen/generators/union.py
+++ b/tools/net/sunrpc/xdrgen/generators/union.py
@@ -8,7 +8,7 @@ from jinja2 import Environment
from generators import SourceGenerator
from generators import create_jinja2_environment, get_jinja2_template
-from xdr_ast import _XdrBasic, _XdrUnion, _XdrVoid
+from xdr_ast import _XdrBasic, _XdrUnion, _XdrVoid, big_endian
from xdr_ast import _XdrDeclaration, _XdrCaseSpec, public_apis
@@ -77,13 +77,18 @@ def emit_union_switch_spec_decoder(
print(template.render(name=node.name, type=node.spec.type_name))
-def emit_union_case_spec_decoder(environment: Environment, node: _XdrCaseSpec) -> None:
+def emit_union_case_spec_decoder(
+ environment: Environment, node: _XdrCaseSpec, big_endian_discriminant: bool
+) -> None:
"""Emit decoder functions for an XDR union's case arm"""
if isinstance(node.arm, _XdrVoid):
return
- template = get_jinja2_template(environment, "decoder", "case_spec")
+ if big_endian_discriminant:
+ template = get_jinja2_template(environment, "decoder", "case_spec_be")
+ else:
+ template = get_jinja2_template(environment, "decoder", "case_spec")
for case in node.values:
print(template.render(case=case))
@@ -136,7 +141,11 @@ def emit_union_decoder(environment: Environment, node: _XdrUnion) -> None:
emit_union_switch_spec_decoder(environment, node.discriminant)
for case in node.cases:
- emit_union_case_spec_decoder(environment, case)
+ emit_union_case_spec_decoder(
+ environment,
+ case,
+ node.discriminant.spec.type_name in big_endian,
+ )
emit_union_default_spec_decoder(environment, node)
@@ -153,17 +162,21 @@ def emit_union_switch_spec_encoder(
print(template.render(name=node.name, type=node.spec.type_name))
-def emit_union_case_spec_encoder(environment: Environment, node: _XdrCaseSpec) -> None:
+def emit_union_case_spec_encoder(
+ environment: Environment, node: _XdrCaseSpec, big_endian_discriminant: bool
+) -> None:
"""Emit encoder functions for an XDR union's case arm"""
if isinstance(node.arm, _XdrVoid):
return
- template = get_jinja2_template(environment, "encoder", "case_spec")
+ if big_endian_discriminant:
+ template = get_jinja2_template(environment, "encoder", "case_spec_be")
+ else:
+ template = get_jinja2_template(environment, "encoder", "case_spec")
for case in node.values:
print(template.render(case=case))
- assert isinstance(node.arm, _XdrBasic)
template = get_jinja2_template(environment, "encoder", node.arm.template)
print(
template.render(
@@ -192,7 +205,6 @@ def emit_union_default_spec_encoder(environment: Environment, node: _XdrUnion) -
print(template.render())
return
- assert isinstance(default_case.arm, _XdrBasic)
template = get_jinja2_template(environment, "encoder", default_case.arm.template)
print(
template.render(
@@ -210,7 +222,11 @@ def emit_union_encoder(environment, node: _XdrUnion) -> None:
emit_union_switch_spec_encoder(environment, node.discriminant)
for case in node.cases:
- emit_union_case_spec_encoder(environment, case)
+ emit_union_case_spec_encoder(
+ environment,
+ case,
+ node.discriminant.spec.type_name in big_endian,
+ )
emit_union_default_spec_encoder(environment, node)
diff --git a/tools/net/sunrpc/xdrgen/grammars/xdr.lark b/tools/net/sunrpc/xdrgen/grammars/xdr.lark
index f3c4552e548d..2eb30dddcaea 100644
--- a/tools/net/sunrpc/xdrgen/grammars/xdr.lark
+++ b/tools/net/sunrpc/xdrgen/grammars/xdr.lark
@@ -87,12 +87,14 @@ procedure_def : type_specifier identifier "(" type_specifier ")" "=" c
pragma_def : "pragma" directive identifier [ identifier ] ";"
-directive : exclude_directive
+directive : big_endian_directive
+ | exclude_directive
| header_directive
| pages_directive
| public_directive
| skip_directive
+big_endian_directive : "big_endian"
exclude_directive : "exclude"
header_directive : "header"
pages_directive : "pages"
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/close.j2
index ab1e576c9531..c7ae506076bb 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/close.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/declaration/close.j2
@@ -1,4 +1,3 @@
{# SPDX-License-Identifier: GPL-2.0 #}
-
-bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, enum {{ name }} *ptr);
-bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, enum {{ name }} value);
+bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr);
+bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value);
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2
index 341d829afeda..6482984f1cb7 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum.j2
@@ -8,7 +8,7 @@ bool
{% else %}
static bool __maybe_unused
{% endif %}
-xdrgen_decode_{{ name }}(struct xdr_stream *xdr, enum {{ name }} *ptr)
+xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr)
{
u32 val;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2
new file mode 100644
index 000000000000..44c391c10b42
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/decoder/enum_be.j2
@@ -0,0 +1,14 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+
+{% if annotate %}
+/* enum {{ name }} (big-endian) */
+{% endif %}
+{% if name in public_apis %}
+bool
+{% else %}
+static bool __maybe_unused
+{% endif %}
+xdrgen_decode_{{ name }}(struct xdr_stream *xdr, {{ name }} *ptr)
+{
+ return xdr_stream_decode_be32(xdr, ptr) == 0;
+}
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2
index 9e62344a976a..446266ad6d17 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close.j2
@@ -1,2 +1,4 @@
{# SPDX-License-Identifier: GPL-2.0 #}
};
+
+typedef enum {{ name }} {{ name }};
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2
new file mode 100644
index 000000000000..cfeee2287e68
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/definition/close_be.j2
@@ -0,0 +1,4 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+};
+
+typedef __be32 {{ name }};
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2
index bd0a770e50f2..67245b9a914d 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum.j2
@@ -8,7 +8,7 @@ bool
{% else %}
static bool __maybe_unused
{% endif %}
-xdrgen_encode_{{ name }}(struct xdr_stream *xdr, enum {{ name }} value)
+xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value)
{
return xdr_stream_encode_u32(xdr, value) == XDR_UNIT;
}
diff --git a/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2
new file mode 100644
index 000000000000..fbbcc45948d6
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/enum/encoder/enum_be.j2
@@ -0,0 +1,14 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+
+{% if annotate %}
+/* enum {{ name }} (big-endian) */
+{% endif %}
+{% if name in public_apis %}
+bool
+{% else %}
+static bool __maybe_unused
+{% endif %}
+xdrgen_encode_{{ name }}(struct xdr_stream *xdr, {{ name }} value)
+{
+ return xdr_stream_encode_be32(xdr, value) == XDR_UNIT;
+}
diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2
index 1d477c2d197a..5bf00070ae95 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_opaque.j2
@@ -2,7 +2,9 @@
{% if annotate %}
/* member {{ name }} (variable-length opaque) */
{% endif %}
+{% if maxsize != "0" %}
if (value->{{ name }}.len > {{ maxsize }})
return false;
+{% endif %}
if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0)
return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_string.j2 b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_string.j2
index cf65b71eaef3..7ddc2bf3edac 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_string.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/pointer/encoder/variable_length_string.j2
@@ -2,7 +2,9 @@
{% if annotate %}
/* member {{ name }} (variable-length string) */
{% endif %}
+{% if maxsize != "0" %}
if (value->{{ name }}.len > {{ maxsize }})
return false;
+{% endif %}
if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0)
return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2
index 1d477c2d197a..5bf00070ae95 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_opaque.j2
@@ -2,7 +2,9 @@
{% if annotate %}
/* member {{ name }} (variable-length opaque) */
{% endif %}
+{% if maxsize != "0" %}
if (value->{{ name }}.len > {{ maxsize }})
return false;
+{% endif %}
if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0)
return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_string.j2 b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_string.j2
index cf65b71eaef3..7ddc2bf3edac 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_string.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/struct/encoder/variable_length_string.j2
@@ -2,7 +2,9 @@
{% if annotate %}
/* member {{ name }} (variable-length string) */
{% endif %}
+{% if maxsize != "0" %}
if (value->{{ name }}.len > {{ maxsize }})
return false;
+{% endif %}
if (xdr_stream_encode_opaque(xdr, value->{{ name }}.data, value->{{ name }}.len) < 0)
return false;
diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2
new file mode 100644
index 000000000000..917f3a1c4588
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/union/decoder/case_spec_be.j2
@@ -0,0 +1,2 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+ case __constant_cpu_to_be32({{ case }}):
diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2
index 01d716d0099e..19ee759d70c6 100644
--- a/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2
+++ b/tools/net/sunrpc/xdrgen/templates/C/union/definition/close.j2
@@ -1,8 +1,3 @@
{# SPDX-License-Identifier: GPL-2.0 #}
} u;
};
-{%- if name in public_apis %}
-
-bool xdrgen_decode_{{ name }}(struct xdr_stream *xdr, struct {{ name }} *ptr);
-bool xdrgen_encode_{{ name }}(struct xdr_stream *xdr, const struct {{ name }} *ptr);
-{%- endif -%}
diff --git a/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2 b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2
new file mode 100644
index 000000000000..917f3a1c4588
--- /dev/null
+++ b/tools/net/sunrpc/xdrgen/templates/C/union/encoder/case_spec_be.j2
@@ -0,0 +1,2 @@
+{# SPDX-License-Identifier: GPL-2.0 #}
+ case __constant_cpu_to_be32({{ case }}):
diff --git a/tools/net/sunrpc/xdrgen/xdr_ast.py b/tools/net/sunrpc/xdrgen/xdr_ast.py
index dbd3fcf9c957..d90dd6ce6100 100644
--- a/tools/net/sunrpc/xdrgen/xdr_ast.py
+++ b/tools/net/sunrpc/xdrgen/xdr_ast.py
@@ -12,10 +12,10 @@ from lark.tree import Meta
this_module = sys.modules[__name__]
+big_endian = []
excluded_apis = []
header_name = "none"
public_apis = []
-enums = set()
structs = set()
pass_by_reference = set()
@@ -294,8 +294,6 @@ class ParseToAst(Transformer):
c_classifier = ""
if isinstance(children[0], _XdrIdentifier):
name = children[0].symbol
- if name in enums:
- c_classifier = "enum "
if name in structs:
c_classifier = "struct "
return _XdrDefinedType(
@@ -320,7 +318,6 @@ class ParseToAst(Transformer):
def enum(self, children):
"""Instantiate one _XdrEnum object"""
enum_name = children[0].symbol
- enums.add(enum_name)
i = 0
enumerators = []
@@ -484,6 +481,8 @@ class ParseToAst(Transformer):
"""Instantiate one _Pragma object"""
directive = children[0].children[0].data
match directive:
+ case "big_endian_directive":
+ big_endian.append(children[1].symbol)
case "exclude_directive":
excluded_apis.append(children[1].symbol)
case "header_directive":
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index f6fe896d0d65..e19bf7621daa 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -208,7 +208,8 @@ static bool is_rust_noreturn(const struct symbol *func)
* as well as changes to the source code itself between versions (since
* these come from the Rust standard library).
*/
- return str_ends_with(func->name, "_4core5sliceSp15copy_from_slice17len_mismatch_fail") ||
+ return str_ends_with(func->name, "_4core3num22from_ascii_radix_panic") ||
+ str_ends_with(func->name, "_4core5sliceSp15copy_from_slice17len_mismatch_fail") ||
str_ends_with(func->name, "_4core6option13expect_failed") ||
str_ends_with(func->name, "_4core6option13unwrap_failed") ||
str_ends_with(func->name, "_4core6result13unwrap_failed") ||
diff --git a/tools/perf/builtin-sched.c b/tools/perf/builtin-sched.c
index 64bf3ac237f2..57ef161e9b52 100644
--- a/tools/perf/builtin-sched.c
+++ b/tools/perf/builtin-sched.c
@@ -51,6 +51,7 @@
#define COMM_LEN 20
#define SYM_LEN 129
#define MAX_PID 1024000
+#define PID_MAX_LIMIT 4194304 /* kernel limit on 64-bit */
#define MAX_PRIO 140
static const char *cpu_list;
@@ -440,17 +441,28 @@ static void add_sched_event_sleep(struct perf_sched *sched, struct task_desc *ta
static struct task_desc *register_pid(struct perf_sched *sched,
unsigned long pid, const char *comm)
{
- struct task_desc *task;
+ struct task_desc *task, **tasks_p;
static int pid_max;
+ /* perf.data is untrusted — cap pid to prevent overflow in size calculations */
+ if (pid >= PID_MAX_LIMIT) {
+ pr_err("pid %lu exceeds limit %d, skipping\n", pid, PID_MAX_LIMIT);
+ return NULL;
+ }
+
if (sched->pid_to_task == NULL) {
if (sysctl__read_int("kernel/pid_max", &pid_max) < 0)
pid_max = MAX_PID;
- BUG_ON((sched->pid_to_task = calloc(pid_max, sizeof(struct task_desc *))) == NULL);
+ sched->pid_to_task = calloc(pid_max, sizeof(struct task_desc *));
+ if (sched->pid_to_task == NULL)
+ return NULL;
}
if (pid >= (unsigned long)pid_max) {
- BUG_ON((sched->pid_to_task = realloc(sched->pid_to_task, (pid + 1) *
- sizeof(struct task_desc *))) == NULL);
+ void *p = realloc(sched->pid_to_task, (pid + 1) * sizeof(struct task_desc *));
+
+ if (p == NULL)
+ return NULL;
+ sched->pid_to_task = p;
while (pid >= (unsigned long)pid_max)
sched->pid_to_task[pid_max++] = NULL;
}
@@ -461,9 +473,11 @@ static struct task_desc *register_pid(struct perf_sched *sched,
return task;
task = zalloc(sizeof(*task));
+ if (task == NULL)
+ return NULL;
task->pid = pid;
- task->nr = sched->nr_tasks;
- strcpy(task->comm, comm);
+ if (comm)
+ strlcpy(task->comm, comm, sizeof(task->comm));
/*
* every task starts in sleeping state - this gets ignored
* if there's no wakeup pointing to this sleep state:
@@ -471,10 +485,12 @@ static struct task_desc *register_pid(struct perf_sched *sched,
add_sched_event_sleep(sched, task, 0, 0);
sched->pid_to_task[pid] = task;
- sched->nr_tasks++;
- sched->tasks = realloc(sched->tasks, sched->nr_tasks * sizeof(struct task_desc *));
- BUG_ON(!sched->tasks);
- sched->tasks[task->nr] = task;
+ tasks_p = realloc(sched->tasks, (sched->nr_tasks + 1) * sizeof(struct task_desc *));
+ if (!tasks_p)
+ return NULL;
+ sched->tasks = tasks_p;
+ sched->tasks[sched->nr_tasks] = task;
+ task->nr = sched->nr_tasks++;
if (verbose > 0)
printf("registered task #%ld, PID %ld (%s)\n", sched->nr_tasks, pid, comm);
@@ -838,6 +854,8 @@ replay_wakeup_event(struct perf_sched *sched,
waker = register_pid(sched, sample->tid, "<unknown>");
wakee = register_pid(sched, pid, comm);
+ if (waker == NULL || wakee == NULL)
+ return -1;
add_sched_event_wakeup(sched, waker, sample->time, wakee);
return 0;
@@ -880,6 +898,8 @@ static int replay_switch_event(struct perf_sched *sched,
prev = register_pid(sched, prev_pid, prev_comm);
next = register_pid(sched, next_pid, next_comm);
+ if (prev == NULL || next == NULL)
+ return -1;
sched->cpu_last_switched[cpu] = timestamp;
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index e476598de808..34c7cfdfec2d 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -2648,7 +2648,8 @@ int cmd_stat(int argc, const char **argv)
output = fopen(output_name, mode);
if (!output) {
perror("failed to create output file");
- return -1;
+ status = -1;
+ goto out;
}
if (!stat_config.json_output) {
clock_gettime(CLOCK_REALTIME, &tm);
@@ -2659,7 +2660,8 @@ int cmd_stat(int argc, const char **argv)
output = fdopen(output_fd, mode);
if (!output) {
perror("Failed opening logfd");
- return -errno;
+ status = -errno;
+ goto out;
}
}
@@ -2668,7 +2670,8 @@ int cmd_stat(int argc, const char **argv)
parse_options_usage(stat_usage, stat_options, "o", 1);
parse_options_usage(NULL, stat_options, "log-fd", 0);
parse_options_usage(NULL, stat_options, "interval-clear", 0);
- return -1;
+ status = -1;
+ goto out;
}
stat_config.output = output;
diff --git a/tools/perf/pmu-events/arch/x86/amdzen5/floating-point.json b/tools/perf/pmu-events/arch/x86/amdzen5/floating-point.json
index 9204bfb1d69e..50d38434f8d3 100644
--- a/tools/perf/pmu-events/arch/x86/amdzen5/floating-point.json
+++ b/tools/perf/pmu-events/arch/x86/amdzen5/floating-point.json
@@ -179,6 +179,30 @@
"BriefDescription": "Retired scalar floating-point blend ops.",
"UMask": "0x09"
},
+ {
+ "EventName": "fp_ops_retired_by_type.scalar_mov",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired scalar floating-point MOV ops.",
+ "UMask": "0x0a"
+ },
+ {
+ "EventName": "fp_ops_retired_by_type.scalar_shuffle",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired scalar floating-point shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
+ "UMask": "0x0b"
+ },
+ {
+ "EventName": "fp_ops_retired_by_type.scalar_bfloat",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired scalar floating-point bfloat ops.",
+ "UMask": "0x0c"
+ },
+ {
+ "EventName": "fp_ops_retired_by_type.scalar_logical",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired scalar floating-point logical ops.",
+ "UMask": "0x0d"
+ },
{
"EventName": "fp_ops_retired_by_type.scalar_other",
"EventCode": "0x0a",
@@ -245,12 +269,24 @@
"BriefDescription": "Retired vector floating-point blend ops.",
"UMask": "0x90"
},
+ {
+ "EventName": "fp_ops_retired_by_type.vector_mov",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired vector floating-point MOV ops.",
+ "UMask": "0xa0"
+ },
{
"EventName": "fp_ops_retired_by_type.vector_shuffle",
"EventCode": "0x0a",
"BriefDescription": "Retired vector floating-point shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0xb0"
},
+ {
+ "EventName": "fp_ops_retired_by_type.vector_bfloat",
+ "EventCode": "0x0a",
+ "BriefDescription": "Retired vector floating-point bfloat ops.",
+ "UMask": "0xc0"
+ },
{
"EventName": "fp_ops_retired_by_type.vector_logical",
"EventCode": "0x0a",
@@ -278,7 +314,7 @@
{
"EventName": "sse_avx_ops_retired.mmx_add",
"EventCode": "0x0b",
- "BriefDescription": "Retired MMX integer add.",
+ "BriefDescription": "Retired MMX integer add ops.",
"UMask": "0x01"
},
{
@@ -299,16 +335,34 @@
"BriefDescription": "Retired MMX integer multiply-accumulate ops.",
"UMask": "0x04"
},
+ {
+ "EventName": "sse_avx_ops_retired.mmx_aes",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired MMX integer AES ops.",
+ "UMask": "0x05"
+ },
+ {
+ "EventName": "sse_avx_ops_retired.mmx_sha",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired MMX integer SHA ops.",
+ "UMask": "0x06"
+ },
{
"EventName": "sse_avx_ops_retired.mmx_cmp",
"EventCode": "0x0b",
"BriefDescription": "Retired MMX integer compare ops.",
"UMask": "0x07"
},
+ {
+ "EventName": "sse_avx_ops_retired.mmx_cvt",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired MMX integer convert or pack ops.",
+ "UMask": "0x08"
+ },
{
"EventName": "sse_avx_ops_retired.mmx_shift",
"EventCode": "0x0b",
- "BriefDescription": "Retired MMX integer shift ops.",
+ "BriefDescription": "Retired MMX integer shift or rotate ops.",
"UMask": "0x09"
},
{
@@ -323,10 +377,17 @@
"BriefDescription": "Retired MMX integer shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0x0b"
},
+ {
+ "EventName": "sse_avx_ops_retired.mmx_vnni",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired MMX integer VNNI ops.",
+ "UMask": "0x0c"
+ },
{
"EventName": "sse_avx_ops_retired.mmx_pack",
"EventCode": "0x0b",
- "BriefDescription": "Retired MMX integer pack ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event sse_avx_ops_retired.mmx_vnni",
+ "Deprecated": "1",
"UMask": "0x0c"
},
{
@@ -389,16 +450,23 @@
"BriefDescription": "Retired SSE and AVX integer compare ops.",
"UMask": "0x70"
},
+ {
+ "EventName": "sse_avx_ops_retired.sse_avx_cvt",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired SSE and AVX integer convert or pack ops.",
+ "UMask": "0x80"
+ },
{
"EventName": "sse_avx_ops_retired.sse_avx_clm",
"EventCode": "0x0b",
- "BriefDescription": "Retired SSE and AVX integer CLM ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event sse_avx_ops_retired.sse_avx_cvt",
+ "Deprecated": "1",
"UMask": "0x80"
},
{
"EventName": "sse_avx_ops_retired.sse_avx_shift",
"EventCode": "0x0b",
- "BriefDescription": "Retired SSE and AVX integer shift ops.",
+ "BriefDescription": "Retired SSE and AVX integer shift or rotate ops.",
"UMask": "0x90"
},
{
@@ -413,10 +481,17 @@
"BriefDescription": "Retired SSE and AVX integer shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0xb0"
},
+ {
+ "EventName": "sse_avx_ops_retired.sse_avx_vnni",
+ "EventCode": "0x0b",
+ "BriefDescription": "Retired SSE and AVX integer VNNI ops.",
+ "UMask": "0xc0"
+ },
{
"EventName": "sse_avx_ops_retired.sse_avx_pack",
"EventCode": "0x0b",
- "BriefDescription": "Retired SSE and AVX integer pack ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event sse_avx_ops_retired.sse_avx_vnni",
+ "Deprecated": "1",
"UMask": "0xc0"
},
{
@@ -497,12 +572,24 @@
"BriefDescription": "Retired 128-bit packed floating-point blend ops.",
"UMask": "0x09"
},
+ {
+ "EventName": "fp_pack_ops_retired.fp128_mov",
+ "EventCode": "0x0c",
+ "BriefDescription": "Retired 128-bit packed floating-point MOV ops.",
+ "UMask": "0x0a"
+ },
{
"EventName": "fp_pack_ops_retired.fp128_shuffle",
"EventCode": "0x0c",
"BriefDescription": "Retired 128-bit packed floating-point shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0x0b"
},
+ {
+ "EventName": "fp_pack_ops_retired.fp128_bfloat",
+ "EventCode": "0x0c",
+ "BriefDescription": "Retired 128-bit packed floating-point bfloat ops.",
+ "UMask": "0x0c"
+ },
{
"EventName": "fp_pack_ops_retired.fp128_logical",
"EventCode": "0x0c",
@@ -575,12 +662,24 @@
"BriefDescription": "Retired 256-bit packed floating-point blend ops.",
"UMask": "0x90"
},
+ {
+ "EventName": "fp_pack_ops_retired.fp256_mov",
+ "EventCode": "0x0c",
+ "BriefDescription": "Retired 256-bit packed floating-point MOV ops.",
+ "UMask": "0xa0"
+ },
{
"EventName": "fp_pack_ops_retired.fp256_shuffle",
"EventCode": "0x0c",
"BriefDescription": "Retired 256-bit packed floating-point shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0xb0"
},
+ {
+ "EventName": "fp_pack_ops_retired.fp256_bfloat",
+ "EventCode": "0x0c",
+ "BriefDescription": "Retired 256-bit packed floating-point bfloat ops.",
+ "UMask": "0xc0"
+ },
{
"EventName": "fp_pack_ops_retired.fp256_logical",
"EventCode": "0x0c",
@@ -647,16 +746,23 @@
"BriefDescription": "Retired 128-bit packed integer compare ops.",
"UMask": "0x07"
},
+ {
+ "EventName": "packed_int_op_type.int128_cvt",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 128-bit packed integer convert or pack ops.",
+ "UMask": "0x08"
+ },
{
"EventName": "packed_int_op_type.int128_clm",
"EventCode": "0x0d",
- "BriefDescription": "Retired 128-bit packed integer CLM ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event packed_int_op_type.int128_cvt",
+ "Deprecated": "1",
"UMask": "0x08"
},
{
"EventName": "packed_int_op_type.int128_shift",
"EventCode": "0x0d",
- "BriefDescription": "Retired 128-bit packed integer shift ops.",
+ "BriefDescription": "Retired 128-bit packed integer shift or rotate ops.",
"UMask": "0x09"
},
{
@@ -671,10 +777,17 @@
"BriefDescription": "Retired 128-bit packed integer shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0x0b"
},
+ {
+ "EventName": "packed_int_op_type.int128_vnni",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 128-bit packed integer VNNI ops.",
+ "UMask": "0x0c"
+ },
{
"EventName": "packed_int_op_type.int128_pack",
"EventCode": "0x0d",
- "BriefDescription": "Retired 128-bit packed integer pack ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event packed_int_op_type.int128_vnni",
+ "Deprecated": "1",
"UMask": "0x0c"
},
{
@@ -719,16 +832,34 @@
"BriefDescription": "Retired 256-bit packed integer multiply-accumulate ops.",
"UMask": "0x40"
},
+ {
+ "EventName": "packed_int_op_type.int256_aes",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 256-bit packed integer AES ops.",
+ "UMask": "0x50"
+ },
+ {
+ "EventName": "packed_int_op_type.int256_sha",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 256-bit packed integer SHA ops.",
+ "UMask": "0x60"
+ },
{
"EventName": "packed_int_op_type.int256_cmp",
"EventCode": "0x0d",
"BriefDescription": "Retired 256-bit packed integer compare ops.",
"UMask": "0x70"
},
+ {
+ "EventName": "packed_int_op_type.int256_cvt",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 256-bit packed integer convert or pack ops.",
+ "UMask": "0x80"
+ },
{
"EventName": "packed_int_op_type.int256_shift",
"EventCode": "0x0d",
- "BriefDescription": "Retired 256-bit packed integer shift ops.",
+ "BriefDescription": "Retired 256-bit packed integer shift or rotate ops.",
"UMask": "0x90"
},
{
@@ -743,10 +874,17 @@
"BriefDescription": "Retired 256-bit packed integer shuffle ops (may include instructions not necessarily thought of as including shuffles e.g. horizontal add, dot product, and certain MOV instructions).",
"UMask": "0xb0"
},
+ {
+ "EventName": "packed_int_op_type.int256_vnni",
+ "EventCode": "0x0d",
+ "BriefDescription": "Retired 256-bit packed integer VNNI ops.",
+ "UMask": "0xc0"
+ },
{
"EventName": "packed_int_op_type.int256_pack",
"EventCode": "0x0d",
- "BriefDescription": "Retired 256-bit packed integer pack ops.",
+ "BriefDescription": "This event is deprecated. Refer to new event packed_int_op_type.int256_vnni",
+ "Deprecated": "1",
"UMask": "0xc0"
},
{
diff --git a/tools/perf/pmu-events/arch/x86/amdzen5/load-store.json b/tools/perf/pmu-events/arch/x86/amdzen5/load-store.json
index 917f9d68f85a..d1ac7ff52a4d 100644
--- a/tools/perf/pmu-events/arch/x86/amdzen5/load-store.json
+++ b/tools/perf/pmu-events/arch/x86/amdzen5/load-store.json
@@ -8,9 +8,15 @@
{
"EventName": "ls_locks.bus_lock",
"EventCode": "0x25",
- "BriefDescription": "Retired Lock instructions which caused a bus lock.",
+ "BriefDescription": "Retired lock instructions which caused a bus lock.",
"UMask": "0x01"
},
+ {
+ "EventName": "ls_locks.all",
+ "EventCode": "0x25",
+ "BriefDescription": "Retired lock instructions of all types.",
+ "UMask": "0x1f"
+ },
{
"EventName": "ls_ret_cl_flush",
"EventCode": "0x26",
diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
index 4145e0277753..87cbbafb41e0 100755
--- a/tools/perf/pmu-events/jevents.py
+++ b/tools/perf/pmu-events/jevents.py
@@ -540,13 +540,14 @@ const struct pmu_table_entry {_pending_events_tblname}[] = {{
def print_pending_metrics() -> None:
"""Optionally close metrics table."""
- def metric_cmp_key(j: JsonEvent) -> Tuple[bool, str, str]:
+ def metric_cmp_key(j: JsonEvent) -> Tuple[str, str, str, str]:
def fix_none(s: Optional[str]) -> str:
if s is None:
return ''
return s
- return (j.desc is not None, fix_none(j.pmu), fix_none(j.metric_name))
+ return (fix_none(j.pmu), fix_none(j.metric_name), j.metric_expr.ToPerfJson(),
+ fix_none(j.desc))
global _pending_metrics
if not _pending_metrics:
diff --git a/tools/perf/pmu-events/metric.py b/tools/perf/pmu-events/metric.py
index 92acd89ed97a..ac322891c4f1 100644
--- a/tools/perf/pmu-events/metric.py
+++ b/tools/perf/pmu-events/metric.py
@@ -445,7 +445,11 @@ class Metric:
def __lt__(self, other):
"""Sort order."""
- return self.name < other.name
+ if self.name != other.name:
+ return self.name < other.name
+ if not self.expr.Equals(other.expr):
+ return self.expr.ToPerfJson() < other.expr.ToPerfJson()
+ return self.description < other.description
def AddToMetricGroup(self, group):
"""Callback used when being added to a MetricGroup."""
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
index 3954bd1587ce..6d1c6be1d918 100644
--- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
+++ b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
@@ -23,16 +23,6 @@
#include "../../../util/srcline.h"
#include "../../../util/srccode.h"
-#if PY_MAJOR_VERSION < 3
-#define _PyCapsule_GetPointer(arg1, arg2) \
- PyCObject_AsVoidPtr(arg1)
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
- PyString_FromStringAndSize((arg1), (arg2))
-#define _PyUnicode_AsUTF8(arg) \
- PyString_AsString(arg)
-
-PyMODINIT_FUNC initperf_trace_context(void);
-#else
#define _PyCapsule_GetPointer(arg1, arg2) \
PyCapsule_GetPointer((arg1), (arg2))
#define _PyBytes_FromStringAndSize(arg1, arg2) \
@@ -41,7 +31,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
PyUnicode_AsUTF8(arg)
PyMODINIT_FUNC PyInit_perf_trace_context(void);
-#endif
static struct scripting_context *get_args(PyObject *args, const char *name, PyObject **arg2)
{
@@ -202,12 +191,6 @@ static PyMethodDef ContextMethods[] = {
{ NULL, NULL, 0, NULL}
};
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf_trace_context(void)
-{
- (void) Py_InitModule("perf_trace_context", ContextMethods);
-}
-#else
PyMODINIT_FUNC PyInit_perf_trace_context(void)
{
static struct PyModuleDef moduledef = {
@@ -229,4 +212,3 @@ PyMODINIT_FUNC PyInit_perf_trace_context(void)
return mod;
}
-#endif
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 470a9709427d..e35b138f4bd0 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -8,6 +8,7 @@
#include <errno.h>
#include <poll.h>
#include <unistd.h>
+#include <setjmp.h>
#include <string.h>
#include <stdlib.h>
#include <sys/types.h>
@@ -139,12 +140,6 @@ static struct test_suite *generic_tests[] = {
NULL,
};
-static struct test_suite **tests[] = {
- generic_tests,
- arch_tests,
- NULL, /* shell tests created at runtime. */
-};
-
static struct test_workload *workloads[] = {
&workload__noploop,
&workload__thloop,
@@ -153,52 +148,57 @@ static struct test_workload *workloads[] = {
&workload__brstack,
&workload__datasym,
&workload__landlock,
+ &workload__traploop,
};
-static int num_subtests(const struct test_suite *t)
+#define workloads__for_each(workload) \
+ for (unsigned i = 0; i < ARRAY_SIZE(workloads) && ({ workload = workloads[i]; 1; }); i++)
+
+#define test_suite__for_each_test_case(suite, idx) \
+ for (idx = 0; (suite)->test_cases && (suite)->test_cases[idx].name != NULL; idx++)
+
+static int test_suite__num_test_cases(const struct test_suite *t)
{
int num;
- if (!t->test_cases)
- return 0;
-
- num = 0;
- while (t->test_cases[num].name)
- num++;
+ test_suite__for_each_test_case(t, num);
return num;
}
-static bool has_subtests(const struct test_suite *t)
-{
- return num_subtests(t) > 1;
-}
-
-static const char *skip_reason(const struct test_suite *t, int subtest)
+static const char *skip_reason(const struct test_suite *t, int test_case)
{
if (!t->test_cases)
return NULL;
- return t->test_cases[subtest >= 0 ? subtest : 0].skip_reason;
+ return t->test_cases[test_case >= 0 ? test_case : 0].skip_reason;
}
-static const char *test_description(const struct test_suite *t, int subtest)
+static const char *test_description(const struct test_suite *t, int test_case)
{
- if (t->test_cases && subtest >= 0)
- return t->test_cases[subtest].desc;
+ if (t->test_cases && test_case >= 0)
+ return t->test_cases[test_case].desc;
return t->desc;
}
-static test_fnptr test_function(const struct test_suite *t, int subtest)
+static test_fnptr test_function(const struct test_suite *t, int test_case)
{
- if (subtest <= 0)
+ if (test_case <= 0)
return t->test_cases[0].run_case;
- return t->test_cases[subtest].run_case;
+ return t->test_cases[test_case].run_case;
}
-static bool perf_test__matches(const char *desc, int curr, int argc, const char *argv[])
+static bool test_exclusive(const struct test_suite *t, int test_case)
+{
+ if (test_case <= 0)
+ return t->test_cases[0].exclusive;
+
+ return t->test_cases[test_case].exclusive;
+}
+
+static bool perf_test__matches(const char *desc, int suite_num, int argc, const char *argv[])
{
int i;
@@ -210,7 +210,7 @@ static bool perf_test__matches(const char *desc, int curr, int argc, const char
long nr = strtoul(argv[i], &end, 10);
if (*end == '\0') {
- if (nr == curr + 1)
+ if (nr == suite_num + 1)
return true;
continue;
}
@@ -225,38 +225,69 @@ static bool perf_test__matches(const char *desc, int curr, int argc, const char
struct child_test {
struct child_process process;
struct test_suite *test;
- int test_num;
- int subtest;
+ int suite_num;
+ int test_case_num;
};
+static jmp_buf run_test_jmp_buf;
+
+static void child_test_sig_handler(int sig)
+{
+ siglongjmp(run_test_jmp_buf, sig);
+}
+
static int run_test_child(struct child_process *process)
{
+ const int signals[] = {
+ SIGABRT, SIGBUS, SIGFPE, SIGILL, SIGINT, SIGPIPE, SIGQUIT, SIGSEGV, SIGTERM,
+ };
struct child_test *child = container_of(process, struct child_test, process);
int err;
+ err = sigsetjmp(run_test_jmp_buf, 1);
+ if (err) {
+ fprintf(stderr, "\n---- unexpected signal (%d) ----\n", err);
+ err = err > 0 ? -err : -1;
+ goto err_out;
+ }
+
+ for (size_t i = 0; i < ARRAY_SIZE(signals); i++)
+ signal(signals[i], child_test_sig_handler);
+
pr_debug("--- start ---\n");
pr_debug("test child forked, pid %d\n", getpid());
- err = test_function(child->test, child->subtest)(child->test, child->subtest);
+ err = test_function(child->test, child->test_case_num)(child->test, child->test_case_num);
pr_debug("---- end(%d) ----\n", err);
+
+err_out:
fflush(NULL);
+ for (size_t i = 0; i < ARRAY_SIZE(signals); i++)
+ signal(signals[i], SIG_DFL);
return -err;
}
-static int print_test_result(struct test_suite *t, int i, int subtest, int result, int width)
+#define TEST_RUNNING -3
+
+static int print_test_result(struct test_suite *t, int curr_suite, int curr_test_case,
+ int result, int width, int running)
{
- if (has_subtests(t)) {
+ if (test_suite__num_test_cases(t) > 1) {
int subw = width > 2 ? width - 2 : width;
- pr_info("%3d.%1d: %-*s:", i + 1, subtest + 1, subw, test_description(t, subtest));
+ pr_info("%3d.%1d: %-*s:", curr_suite + 1, curr_test_case + 1, subw,
+ test_description(t, curr_test_case));
} else
- pr_info("%3d: %-*s:", i + 1, width, test_description(t, subtest));
+ pr_info("%3d: %-*s:", curr_suite + 1, width, test_description(t, curr_test_case));
switch (result) {
+ case TEST_RUNNING:
+ color_fprintf(stderr, PERF_COLOR_YELLOW, " Running (%d active)\n", running);
+ break;
case TEST_OK:
pr_info(" Ok\n");
break;
case TEST_SKIP: {
- const char *reason = skip_reason(t, subtest);
+ const char *reason = skip_reason(t, curr_test_case);
if (reason)
color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip (%s)\n", reason);
@@ -273,34 +304,44 @@ static int print_test_result(struct test_suite *t, int i, int subtest, int resul
return 0;
}
-static int finish_test(struct child_test *child_test, int width)
+static void finish_test(struct child_test **child_tests, int running_test, int child_test_num,
+ int width)
{
- struct test_suite *t = child_test->test;
- int i = child_test->test_num;
- int subi = child_test->subtest;
- int err = child_test->process.err;
- bool err_done = err <= 0;
+ struct child_test *child_test = child_tests[running_test];
+ struct test_suite *t;
+ int curr_suite, curr_test_case, err;
+ bool err_done = false;
struct strbuf err_output = STRBUF_INIT;
+ int last_running = -1;
int ret;
+ if (child_test == NULL) {
+ /* Test wasn't started. */
+ return;
+ }
+ t = child_test->test;
+ curr_suite = child_test->suite_num;
+ curr_test_case = child_test->test_case_num;
+ err = child_test->process.err;
/*
* For test suites with subtests, display the suite name ahead of the
* sub test names.
*/
- if (has_subtests(t) && subi == 0)
- pr_info("%3d: %-*s:\n", i + 1, width, test_description(t, -1));
+ if (test_suite__num_test_cases(t) > 1 && curr_test_case == 0)
+ pr_info("%3d: %-*s:\n", curr_suite + 1, width, test_description(t, -1));
/*
* Busy loop reading from the child's stdout/stderr that are set to be
* non-blocking until EOF.
*/
- if (!err_done)
+ if (err > 0)
fcntl(err, F_SETFL, O_NONBLOCK);
if (verbose > 1) {
- if (has_subtests(t))
- pr_info("%3d.%1d: %s:\n", i + 1, subi + 1, test_description(t, subi));
+ if (test_suite__num_test_cases(t) > 1)
+ pr_info("%3d.%1d: %s:\n", curr_suite + 1, curr_test_case + 1,
+ test_description(t, curr_test_case));
else
- pr_info("%3d: %s:\n", i + 1, test_description(t, -1));
+ pr_info("%3d: %s:\n", curr_suite + 1, test_description(t, -1));
}
while (!err_done) {
struct pollfd pfds[1] = {
@@ -308,64 +349,99 @@ static int finish_test(struct child_test *child_test, int width)
.events = POLLIN | POLLERR | POLLHUP | POLLNVAL,
},
};
- char buf[512];
- ssize_t len;
-
- /* Poll to avoid excessive spinning, timeout set for 100ms. */
- poll(pfds, ARRAY_SIZE(pfds), /*timeout=*/100);
- if (!err_done && pfds[0].revents) {
- errno = 0;
- len = read(err, buf, sizeof(buf) - 1);
-
- if (len <= 0) {
- err_done = errno != EAGAIN;
- } else {
- buf[len] = '\0';
- if (verbose > 1)
- fprintf(stdout, "%s", buf);
- else
+ if (perf_use_color_default) {
+ int running = 0;
+
+ for (int y = running_test; y < child_test_num; y++) {
+ if (child_tests[y] == NULL)
+ continue;
+ if (check_if_command_finished(&child_tests[y]->process) == 0)
+ running++;
+ }
+ if (running != last_running) {
+ if (last_running != -1) {
+ /*
+ * Erase "Running (.. active)" line
+ * printed before poll/sleep.
+ */
+ fprintf(debug_file(), PERF_COLOR_DELETE_LINE);
+ }
+ print_test_result(t, curr_suite, curr_test_case, TEST_RUNNING,
+ width, running);
+ last_running = running;
+ }
+ }
+
+ err_done = true;
+ if (err <= 0) {
+ /* No child stderr to poll, sleep for 10ms for child to complete. */
+ usleep(10 * 1000);
+ } else {
+ /* Poll to avoid excessive spinning, timeout set for 100ms. */
+ poll(pfds, ARRAY_SIZE(pfds), /*timeout=*/100);
+ if (pfds[0].revents) {
+ char buf[512];
+ ssize_t len;
+
+ len = read(err, buf, sizeof(buf) - 1);
+
+ if (len > 0) {
+ err_done = false;
+ buf[len] = '\0';
strbuf_addstr(&err_output, buf);
+ }
}
}
+ if (err_done)
+ err_done = check_if_command_finished(&child_test->process);
+ }
+ if (perf_use_color_default && last_running != -1) {
+ /* Erase "Running (.. active)" line printed before poll/sleep. */
+ fprintf(debug_file(), PERF_COLOR_DELETE_LINE);
}
/* Clean up child process. */
ret = finish_command(&child_test->process);
- if (verbose == 1 && ret == TEST_FAIL) {
- /* Add header for test that was skipped above. */
- if (has_subtests(t))
- pr_info("%3d.%1d: %s:\n", i + 1, subi + 1, test_description(t, subi));
- else
- pr_info("%3d: %s:\n", i + 1, test_description(t, -1));
+ if (verbose > 1 || (verbose == 1 && ret == TEST_FAIL))
fprintf(stderr, "%s", err_output.buf);
- }
+
strbuf_release(&err_output);
- print_test_result(t, i, subi, ret, width);
+ print_test_result(t, curr_suite, curr_test_case, ret, width, /*running=*/0);
if (err > 0)
close(err);
- return 0;
+ zfree(&child_tests[running_test]);
}
-static int start_test(struct test_suite *test, int i, int subi, struct child_test **child,
- int width)
+static int start_test(struct test_suite *test, int curr_suite, int curr_test_case,
+ struct child_test **child, int width, int pass)
{
int err;
*child = NULL;
if (dont_fork) {
- pr_debug("--- start ---\n");
- err = test_function(test, subi)(test, subi);
- pr_debug("---- end ----\n");
- print_test_result(test, i, subi, err, width);
+ if (pass == 1) {
+ pr_debug("--- start ---\n");
+ err = test_function(test, curr_test_case)(test, curr_test_case);
+ pr_debug("---- end ----\n");
+ print_test_result(test, curr_suite, curr_test_case, err, width,
+ /*running=*/0);
+ }
+ return 0;
+ }
+ if (pass == 1 && !sequential && test_exclusive(test, curr_test_case)) {
+ /* When parallel, skip exclusive tests on the first pass. */
+ return 0;
+ }
+ if (pass != 1 && (sequential || !test_exclusive(test, curr_test_case))) {
+ /* Sequential and non-exclusive tests were run on the first pass. */
return 0;
}
-
*child = zalloc(sizeof(**child));
if (!*child)
return -ENOMEM;
(*child)->test = test;
- (*child)->test_num = i;
- (*child)->subtest = subi;
+ (*child)->suite_num = curr_suite;
+ (*child)->test_case_num = curr_test_case;
(*child)->process.pid = -1;
(*child)->process.no_stdin = 1;
if (verbose <= 0) {
@@ -377,40 +453,43 @@ static int start_test(struct test_suite *test, int i, int subi, struct child_tes
(*child)->process.err = -1;
}
(*child)->process.no_exec_cmd = run_test_child;
- err = start_command(&(*child)->process);
- if (err || !sequential)
- return err;
- return finish_test(*child, width);
+ if (sequential || pass == 2) {
+ err = start_command(&(*child)->process);
+ if (err)
+ return err;
+ finish_test(child, /*running_test=*/0, /*child_test_num=*/1, width);
+ return 0;
+ }
+ return start_command(&(*child)->process);
}
-#define for_each_test(j, k, t) \
- for (j = 0, k = 0; j < ARRAY_SIZE(tests); j++, k = 0) \
- while ((t = tests[j][k++]) != NULL)
+/* State outside of __cmd_test for the sake of the signal handler. */
-static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist)
+static size_t num_tests;
+static struct child_test **child_tests;
+static jmp_buf cmd_test_jmp_buf;
+
+static void cmd_test_sig_handler(int sig)
{
- struct test_suite *t;
- unsigned int j, k;
- int i = 0;
- int width = 0;
- size_t num_tests = 0;
- struct child_test **child_tests;
- int child_test_num = 0;
+ siglongjmp(cmd_test_jmp_buf, sig);
+}
- for_each_test(j, k, t) {
- int len = strlen(test_description(t, -1));
+static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
+ struct intlist *skiplist)
+{
+ static int width = 0;
+ int err = 0;
+
+ for (struct test_suite **t = suites; *t; t++) {
+ int i, len = strlen(test_description(*t, -1));
if (width < len)
width = len;
- if (has_subtests(t)) {
- for (int subi = 0, subn = num_subtests(t); subi < subn; subi++) {
- len = strlen(test_description(t, subi));
- if (width < len)
- width = len;
- num_tests++;
- }
- } else {
+ test_suite__for_each_test_case(*t, i) {
+ len = strlen(test_description(*t, i));
+ if (width < len)
+ width = len;
num_tests++;
}
}
@@ -418,85 +497,111 @@ static int __cmd_test(int argc, const char *argv[], struct intlist *skiplist)
if (!child_tests)
return -ENOMEM;
- for_each_test(j, k, t) {
- int curr = i++;
+ err = sigsetjmp(cmd_test_jmp_buf, 1);
+ if (err) {
+ pr_err("\nSignal (%d) while running tests.\nTerminating tests with the same signal\n",
+ err);
+ for (size_t x = 0; x < num_tests; x++) {
+ struct child_test *child_test = child_tests[x];
- if (!perf_test__matches(test_description(t, -1), curr, argc, argv)) {
- bool skip = true;
-
- for (int subi = 0, subn = num_subtests(t); subi < subn; subi++) {
- if (perf_test__matches(test_description(t, subi),
- curr, argc, argv))
- skip = false;
- }
-
- if (skip)
+ if (!child_test || child_test->process.pid <= 0)
continue;
- }
- if (intlist__find(skiplist, i)) {
- pr_info("%3d: %-*s:", curr + 1, width, test_description(t, -1));
- color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip (user override)\n");
- continue;
+ pr_debug3("Killing %d pid %d\n",
+ child_test->suite_num + 1,
+ child_test->process.pid);
+ kill(child_test->process.pid, err);
}
+ goto err_out;
+ }
+ signal(SIGINT, cmd_test_sig_handler);
+ signal(SIGTERM, cmd_test_sig_handler);
- if (!has_subtests(t)) {
- int err = start_test(t, curr, -1, &child_tests[child_test_num++], width);
+ /*
+ * In parallel mode pass 1 runs non-exclusive tests in parallel, pass 2
+ * runs the exclusive tests sequentially. In other modes all tests are
+ * run in pass 1.
+ */
+ for (int pass = 1; pass <= 2; pass++) {
+ int child_test_num = 0;
+ int curr_suite = 0;
+
+ for (struct test_suite **t = suites; *t; t++, curr_suite++) {
+ int curr_test_case;
+
+ if (!perf_test__matches(test_description(*t, -1), curr_suite, argc, argv)) {
+ /*
+ * Test suite shouldn't be run based on
+ * description. See if any test case should.
+ */
+ bool skip = true;
+
+ test_suite__for_each_test_case(*t, curr_test_case) {
+ if (perf_test__matches(test_description(*t, curr_test_case),
+ curr_suite, argc, argv)) {
+ skip = false;
+ break;
+ }
+ }
+ if (skip)
+ continue;
+ }
- if (err) {
- /* TODO: if !sequential waitpid the already forked children. */
- free(child_tests);
- return err;
+ if (intlist__find(skiplist, curr_suite + 1)) {
+ pr_info("%3d: %-*s:", curr_suite + 1, width,
+ test_description(*t, -1));
+ color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip (user override)\n");
+ continue;
}
- } else {
- for (int subi = 0, subn = num_subtests(t); subi < subn; subi++) {
- int err;
- if (!perf_test__matches(test_description(t, subi),
- curr, argc, argv))
+ test_suite__for_each_test_case(*t, curr_test_case) {
+ if (!perf_test__matches(test_description(*t, curr_test_case),
+ curr_suite, argc, argv))
continue;
- err = start_test(t, curr, subi, &child_tests[child_test_num++],
- width);
+ err = start_test(*t, curr_suite, curr_test_case,
+ &child_tests[child_test_num++],
+ width, pass);
if (err)
- return err;
+ goto err_out;
}
}
- }
- for (i = 0; i < child_test_num; i++) {
if (!sequential) {
- int ret = finish_test(child_tests[i], width);
-
- if (ret)
- return ret;
+ /* Parallel mode starts tests but doesn't finish them. Do that now. */
+ for (size_t x = 0; x < num_tests; x++)
+ finish_test(child_tests, x, num_tests, width);
}
- free(child_tests[i]);
+ }
+err_out:
+ signal(SIGINT, SIG_DFL);
+ signal(SIGTERM, SIG_DFL);
+ if (err) {
+ pr_err("Internal test harness failure. Completing any started tests:\n:");
+ for (size_t x = 0; x < num_tests; x++)
+ finish_test(child_tests, x, num_tests, width);
}
free(child_tests);
- return 0;
+ return err;
}
-static int perf_test__list(int argc, const char **argv)
+static int perf_test__list(FILE *fp, struct test_suite **suites, int argc, const char **argv)
{
- unsigned int j, k;
- struct test_suite *t;
- int i = 0;
+ int curr_suite = 0;
- for_each_test(j, k, t) {
- int curr = i++;
+ for (struct test_suite **t = suites; *t; t++, curr_suite++) {
+ int curr_test_case;
- if (!perf_test__matches(test_description(t, -1), curr, argc, argv))
+ if (!perf_test__matches(test_description(*t, -1), curr_suite, argc, argv))
continue;
- pr_info("%3d: %s\n", i, test_description(t, -1));
+ fprintf(fp, "%3d: %s\n", curr_suite + 1, test_description(*t, -1));
- if (has_subtests(t)) {
- int subn = num_subtests(t);
- int subi;
+ if (test_suite__num_test_cases(*t) <= 1)
+ continue;
- for (subi = 0; subi < subn; subi++)
- pr_info("%3d:%1d: %s\n", i, subi + 1,
- test_description(t, subi));
+ test_suite__for_each_test_case(*t, curr_test_case) {
+ fprintf(fp, "%3d.%1d: %s\n", curr_suite + 1, curr_test_case + 1,
+ test_description(*t, curr_test_case));
}
}
return 0;
@@ -504,11 +609,9 @@ static int perf_test__list(int argc, const char **argv)
static int run_workload(const char *work, int argc, const char **argv)
{
- unsigned int i = 0;
struct test_workload *twl;
- for (i = 0; i < ARRAY_SIZE(workloads); i++) {
- twl = workloads[i];
+ workloads__for_each(twl) {
if (!strcmp(twl->name, work))
return twl->func(argc, argv);
}
@@ -526,6 +629,63 @@ static int perf_test__config(const char *var, const char *value,
return 0;
}
+static struct test_suite **build_suites(void)
+{
+ /*
+ * TODO: suites is static to avoid needing to clean up the scripts tests
+ * for leak sanitizer.
+ */
+ static struct test_suite **suites[] = {
+ generic_tests,
+ arch_tests,
+ NULL,
+ };
+ struct test_suite **result;
+ struct test_suite *t;
+ size_t n = 0, num_suites = 0;
+
+ if (suites[2] == NULL)
+ suites[2] = create_script_test_suites();
+
+#define for_each_suite(suite) \
+ for (size_t i = 0, j = 0; i < ARRAY_SIZE(suites); i++, j = 0) \
+ while ((suite = suites[i][j++]) != NULL)
+
+ for_each_suite(t) {
+ if (t->setup) {
+ int ret = t->setup(t);
+
+ if (ret < 0) {
+ errno = -ret;
+ return NULL;
+ }
+ }
+ num_suites++;
+ }
+
+ result = calloc(num_suites + 1, sizeof(struct test_suite *));
+ if (!result)
+ return NULL;
+
+ for (int pass = 1; pass <= 2; pass++) {
+ for_each_suite(t) {
+ bool exclusive = false;
+ int curr_test_case;
+
+ test_suite__for_each_test_case(t, curr_test_case) {
+ if (test_exclusive(t, curr_test_case)) {
+ exclusive = true;
+ break;
+ }
+ }
+ if ((!exclusive && pass == 1) || (exclusive && pass == 2))
+ result[n++] = t;
+ }
+ }
+ return result;
+#undef for_each_suite
+}
+
int cmd_test(int argc, const char **argv)
{
const char *test_usage[] = {
@@ -552,6 +712,7 @@ int cmd_test(int argc, const char **argv)
const char * const test_subcommands[] = { "list", NULL };
struct intlist *skiplist = NULL;
int ret = hists__init();
+ struct test_suite **suites;
if (ret < 0)
return ret;
@@ -561,10 +722,15 @@ int cmd_test(int argc, const char **argv)
/* Unbuffered output */
setvbuf(stdout, NULL, _IONBF, 0);
- tests[2] = create_script_test_suites();
argc = parse_options_subcommand(argc, argv, test_options, test_subcommands, test_usage, 0);
- if (argc >= 1 && !strcmp(argv[0], "list"))
- return perf_test__list(argc - 1, argv + 1);
+ if (argc >= 1 && !strcmp(argv[0], "list")) {
+ suites = build_suites();
+ if (!suites)
+ return errno ? -errno : -ENOMEM;
+ ret = perf_test__list(stdout, suites, argc - 1, argv + 1);
+ free(suites);
+ return ret;
+ }
if (workload)
return run_workload(workload, argc, argv);
@@ -577,6 +743,7 @@ int cmd_test(int argc, const char **argv)
symbol_conf.priv_size = sizeof(int);
symbol_conf.try_vmlinux_path = true;
+
if (symbol__init(NULL) < 0)
return -1;
@@ -588,5 +755,15 @@ int cmd_test(int argc, const char **argv)
*/
rlimit__bump_memlock();
- return __cmd_test(argc, argv, skiplist);
+ suites = build_suites();
+ if (!suites) {
+ int err = errno;
+
+ intlist__delete(skiplist);
+ return err ? -err : -ENOMEM;
+ }
+ ret = __cmd_test(suites, argc, argv, skiplist);
+ free(suites);
+ intlist__delete(skiplist);
+ return ret;
}
diff --git a/tools/perf/tests/shell/lib/perf_metric_validation.py b/tools/perf/tests/shell/lib/perf_metric_validation.py
index 0b94216c9c46..32b81e085515 100644
--- a/tools/perf/tests/shell/lib/perf_metric_validation.py
+++ b/tools/perf/tests/shell/lib/perf_metric_validation.py
@@ -381,10 +381,13 @@ class Validator:
wl = workload.split()
command.extend(wl)
print(" ".join(command))
- cmd = subprocess.run(command, stderr=subprocess.PIPE, encoding='utf-8')
- data = [x+'}' for x in cmd.stderr.split('}\n') if x]
- if data[0][0] != '{':
- data[0] = data[0][data[0].find('{'):]
+ cmd = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8')
+ lines = cmd.stderr.splitlines() + cmd.stdout.splitlines()
+ data = []
+ for line in lines:
+ line = line.strip()
+ if line.startswith('{') and line.endswith('}'):
+ data.append(line)
return data
def collect_perf(self, workload: str):
diff --git a/tools/perf/tests/shell/stat_all_metrics.sh b/tools/perf/tests/shell/stat_all_metrics.sh
index 54774525e18a..feeb34c6fa6d 100755
--- a/tools/perf/tests/shell/stat_all_metrics.sh
+++ b/tools/perf/tests/shell/stat_all_metrics.sh
@@ -2,42 +2,147 @@
# perf all metrics test
# SPDX-License-Identifier: GPL-2.0
-err=0
+ParanoidAndNotRoot()
+{
+ [ "$(id -u)" != 0 ] && [ "$(cat /proc/sys/kernel/perf_event_paranoid)" -gt $1 ]
+}
+
+test_prog="sleep 0.01"
+system_wide_flag="-a"
+if ParanoidAndNotRoot 0
+then
+ system_wide_flag=""
+ test_prog="perf test -w noploop 0.01"
+fi
+
+check_metric() {
+ local output="$1"
+ local status="$2"
+ local metric="$3"
+
+ if [[ $status -ne 0 || ! "$output" =~ ${metric:0:50} ]]; then
+ return 1
+ fi
+
+ if [[ "$output" =~ "<not counted>" || "$output" =~ "<not supported>" ]]; then
+ return 1
+ fi
+
+ return 0
+}
+
+skip=0
+err=3
for m in $(perf list --raw-dump metrics); do
echo "Testing $m"
- result=$(perf stat -M "$m" true 2>&1)
- if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "<not supported>" ]]
- then
+ result=$(perf stat -M "$m" $system_wide_flag -- $test_prog 2>&1)
+ result_err=$?
+
+ if check_metric "$result" $result_err "$m"; then
+ if [[ "$err" -ne 1 ]]
+ then
+ err=0
+ fi
continue
fi
- # Failed so try system wide.
- result=$(perf stat -M "$m" -a sleep 0.01 2>&1)
- if [[ "$result" =~ ${m:0:50} ]]
+
+ if [[ "$result" =~ "Access to performance monitoring and observability operations is limited" || \
+ "$result" =~ "in per-thread mode, enable system wide" || \
+ "$result" =~ "<not supported>" || \
+ "$result" =~ "Cannot resolve IDs for" || \
+ "$result" =~ "No supported events found" || \
+ "$result" =~ "FP_ARITH" || \
+ "$result" =~ "AMX" || \
+ "$result" =~ "PMM" ]]
then
- continue
+ true
+ else
+ result=$(perf stat -M "$m" $system_wide_flag -- perf test -w noploop 0.1 2>&1)
+ result_err=$?
+
+ if check_metric "$result" $result_err "$m"; then
+ if [[ "$err" -ne 1 ]]
+ then
+ err=0
+ fi
+ continue
+ fi
fi
- # Failed again, possibly the workload was too small so retry with something
- # longer.
- result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
- if [[ "$result" =~ ${m:0:50} ]]
+
+ # If retry also failed, determine if we skip, ignore, or fail
+ if [[ "$result" =~ "Access to performance monitoring and observability operations is limited" ]]
then
+ echo "[Skipped $m] Permission failure"
+ echo $result
+ if [[ $err -eq 0 ]]
+ then
+ skip=1
+ fi
continue
- fi
- echo "Metric '$m' not printed in:"
- echo "$result"
- if [[ "$err" != "1" ]]
+ elif [[ "$result" =~ "in per-thread mode, enable system wide" ]]
then
- err=2
- if [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]]
+ echo "[Skipped $m] Permissions - need system wide mode"
+ echo $result
+ if [[ $err -eq 0 ]]
then
- echo "Skip, not fail, for FP issues"
- elif [[ "$result" =~ "PMM" ]]
+ skip=1
+ fi
+ continue
+ elif [[ "$result" =~ "<not supported>" || \
+ "$result" =~ "Cannot resolve IDs for" || \
+ "$result" =~ "No supported events found" ]]
+ then
+ if [[ $(perf list --raw-dump $m) == "Default"* ]]
then
- echo "Skip, not fail, for Optane memory issues"
- else
- err=1
+ echo "[Ignored $m] failed but as a Default metric this can be expected"
+ echo $result
+ continue
fi
+ echo "[Skipped $m] Not supported events"
+ echo $result
+ if [[ $err -eq 0 ]]
+ then
+ skip=1
+ fi
+ continue
+ elif [[ "$result" =~ "<not counted>" ]]
+ then
+ echo "[Skipped $m] Not counted events"
+ echo $result
+ if [[ $err -eq 0 ]]
+ then
+ skip=1
+ fi
+ continue
+ elif [[ "$result" =~ "FP_ARITH" || "$result" =~ "AMX" ]]
+ then
+ echo "[Skipped $m] FP issues"
+ echo $result
+ if [[ $err -eq 0 ]]
+ then
+ skip=1
+ fi
+ continue
+ elif [[ "$result" =~ "PMM" ]]
+ then
+ echo "[Skipped $m] Optane memory issues"
+ echo $result
+ if [[ $err -eq 0 ]]
+ then
+ skip=1
+ fi
+ continue
fi
+
+ echo "[Failed $m] has non-zero error '$result_err' or not printed/counted in:"
+ echo "$result"
+ err=1
done
+# return SKIP only if no success returned
+if [[ "$err" -eq 3 && "$skip" -eq 1 ]]
+then
+ err=2
+fi
+
exit "$err"
diff --git a/tools/perf/tests/shell/stat_bpf_counters.sh b/tools/perf/tests/shell/stat_bpf_counters.sh
index f250b7d6f773..76d4a7f15a43 100755
--- a/tools/perf/tests/shell/stat_bpf_counters.sh
+++ b/tools/perf/tests/shell/stat_bpf_counters.sh
@@ -4,21 +4,26 @@
set -e
-workload="perf test -w brstack"
+# Get the first allowed CPU
+CPU=$(taskset -c -p $$ | awk -F': ' '{print $2}' | awk -F'[,-]' '{print $1}')
+if [ -z "$CPU" ]; then
+ CPU=0
+fi
+workload=(taskset -c "$CPU" awk 'BEGIN { for (i=0; i<10000000; i++) sum+=i }')
-# check whether $2 is within +/- 20% of $1
+# check whether $2 is within +/- 15% of $1
compare_number()
{
first_num=$1
second_num=$2
- # upper bound is first_num * 120%
- upper=$(expr $first_num + $first_num / 5 )
- # lower bound is first_num * 80%
- lower=$(expr $first_num - $first_num / 5 )
+ # upper bound is first_num * 115%
+ upper=$(expr $first_num + $first_num / 20 \* 3 )
+ # lower bound is first_num * 85%
+ lower=$(expr $first_num - $first_num / 20 \* 3 )
if [ $second_num -gt $upper ] || [ $second_num -lt $lower ]; then
- echo "The difference between $first_num and $second_num are greater than 20%."
+ echo "The difference between $first_num and $second_num are greater than 15%."
exit 1
fi
}
@@ -41,8 +46,15 @@ check_counts()
test_bpf_counters()
{
printf "Testing --bpf-counters "
- base_instructions=$(perf stat --no-big-num -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
- bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
+ base_instructions=$(perf stat --no-big-num -e instructions:u -- "${workload[@]}" 2>&1 | \
+ awk -v i=0 -v c=0 '/instructions/ { \
+ if ($1 != "<not") { i++; c += $1 } \
+ } END { if (i > 0) printf "%.0f", c; else print "<not" }')
+ bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions:u \
+ -- "${workload[@]}" 2>&1 | \
+ awk -v i=0 -v c=0 '/instructions/ { \
+ if ($1 != "<not") { i++; c += $1 } \
+ } END { if (i > 0) printf "%.0f", c; else print "<not" }')
check_counts $base_instructions $bpf_instructions
compare_number $base_instructions $bpf_instructions
echo "[Success]"
@@ -51,9 +63,17 @@ test_bpf_counters()
test_bpf_modifier()
{
printf "Testing bpf event modifier "
- stat_output=$(perf stat --no-big-num -e instructions/name=base_instructions/,instructions/name=bpf_instructions/b -- $workload 2>&1)
- base_instructions=$(echo "$stat_output"| awk '/base_instructions/ {print $1}')
- bpf_instructions=$(echo "$stat_output"| awk '/bpf_instructions/ {print $1}')
+ stat_output=$(perf stat --no-big-num \
+ -e instructions/name=base_instructions/u,instructions/name=bpf_instructions/bu \
+ -- "${workload[@]}" 2>&1)
+ base_instructions=$(echo "$stat_output"| \
+ awk -v i=0 -v c=0 '/base_instructions/ { \
+ if ($1 != "<not") { i++; c += $1 } \
+ } END { if (i > 0) printf "%.0f", c; else print "<not" }')
+ bpf_instructions=$(echo "$stat_output"| \
+ awk -v i=0 -v c=0 '/bpf_instructions/ { \
+ if ($1 != "<not") { i++; c += $1 } \
+ } END { if (i > 0) printf "%.0f", c; else print "<not" }')
check_counts $base_instructions $bpf_instructions
compare_number $base_instructions $bpf_instructions
echo "[Success]"
diff --git a/tools/perf/tests/shell/stat_metrics_values.sh b/tools/perf/tests/shell/stat_metrics_values.sh
index 279f19c5919a..94969023cf8e 100755
--- a/tools/perf/tests/shell/stat_metrics_values.sh
+++ b/tools/perf/tests/shell/stat_metrics_values.sh
@@ -8,6 +8,13 @@ shelldir=$(dirname "$0")
grep -q GenuineIntel /proc/cpuinfo || { echo Skipping non-Intel; exit 2; }
+# Skip if no permission to record system-wide events
+if ! perf stat -a -e instructions sleep 0.01 >/dev/null 2>&1; then
+ echo "Skipping: no permission to record system-wide events (-a)"
+ exit 2
+fi
+
+
pythonvalidator=$(dirname $0)/lib/perf_metric_validation.py
rulefile=$(dirname $0)/lib/perf_metric_validation_rules.json
tmpdir=$(mktemp -d /tmp/__perf_test.program.XXXXX)
diff --git a/tools/perf/tests/shell/test_brstack.sh b/tools/perf/tests/shell/test_brstack.sh
index 5f14d0cb013f..71550e0b37ba 100755
--- a/tools/perf/tests/shell/test_brstack.sh
+++ b/tools/perf/tests/shell/test_brstack.sh
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
# Check branch stack sampling
# SPDX-License-Identifier: GPL-2.0
@@ -17,37 +17,168 @@ fi
skip_test_missing_symbol brstack_bench
+err=0
TMPDIR=$(mktemp -d /tmp/__perf_test.program.XXXXX)
TESTPROG="perf test -w brstack"
cleanup() {
rm -rf $TMPDIR
+ trap - EXIT TERM INT
}
-trap cleanup EXIT TERM INT
+trap_cleanup() {
+ set +e
+ echo "Unexpected signal in ${FUNCNAME[1]}"
+ cleanup
+ exit 1
+}
+trap trap_cleanup EXIT TERM INT
+
+is_arm64() {
+ [ "$(uname -m)" = "aarch64" ];
+}
+
+has_kaslr_bug() {
+ [ "$(uname -m)" != "aarch64" ];
+}
+
+check_branches() {
+ if ! tr -s ' ' '\n' < "$TMPDIR/perf.script" | grep -E -m1 -q "$1"; then
+ echo "ERROR: Branches missing $1"
+ err=1
+ fi
+}
test_user_branches() {
echo "Testing user branch stack sampling"
- perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u -- ${TESTPROG} > /dev/null 2>&1
- perf script -i $TMPDIR/perf.data --fields brstacksym | xargs -n1 > $TMPDIR/perf.script
+ start_err=$err
+ err=0
+ perf record -o "$TMPDIR/perf.data" --branch-filter any,save_type,u -- ${TESTPROG} > "$TMPDIR/record.txt" 2>&1
+ perf script -i "$TMPDIR/perf.data" --fields brstacksym > "$TMPDIR/perf.script"
# example of branch entries:
# brstack_foo+0x14/brstack_bar+0x40/P/-/-/0/CALL
- set -x
- grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$" $TMPDIR/perf.script
- grep -E -m1 "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$" $TMPDIR/perf.script
- set +x
+ expected=(
+ "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/IND_CALL/.*$"
+ "^brstack_foo\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
+ "^brstack_bench\+[^ ]*/brstack_foo\+[^ ]*/CALL/.*$"
+ "^brstack_bench\+[^ ]*/brstack_bar\+[^ ]*/CALL/.*$"
+ "^brstack_bar\+[^ ]*/brstack_foo\+[^ ]*/RET/.*$"
+ "^brstack_foo\+[^ ]*/brstack_bench\+[^ ]*/RET/.*$"
+ "^brstack_bench\+[^ ]*/brstack_bench\+[^ ]*/COND/.*$"
+ "^brstack\+[^ ]*/brstack\+[^ ]*/UNCOND/.*$"
+ )
+ for x in "${expected[@]}"
+ do
+ check_branches "$x"
+ done
+ # Dump addresses only this time
+ perf script -i "$TMPDIR/perf.data" --fields brstack | \
+ tr ' ' '\n' > "$TMPDIR/perf.script"
+
+ # There should be no kernel addresses in the target with the u option.
+ local regex="0x[89a-f][0-9a-f]{15}"
+ if has_kaslr_bug; then
+ # If the system has a kaslr bug that may leak kernel addresses
+ # in the source of something like an ERET/SYSRET. Make the regex
+ # more specific and just check the target address is in user
+ # code.
+ regex="^0x[0-9a-f]{0,16}/0x[89a-f][0-9a-f]{15}/"
+ fi
+ if grep -q -E -m1 "$regex" $TMPDIR/perf.script; then
+ echo "Testing user branch stack sampling [Failed kernel address found in user mode]"
+ err=1
+ fi
# some branch types are still not being tested:
- # IND COND_CALL COND_RET SYSCALL SYSRET IRQ SERROR NO_TX
+ # IND COND_CALL COND_RET SYSRET SERROR NO_TX
+ if [ $err -eq 0 ]; then
+ echo "Testing user branch stack sampling [Passed]"
+ err=$start_err
+ else
+ echo "Testing user branch stack sampling [Failed]"
+ fi
+}
+
+test_trap_eret_branches() {
+ echo "Testing trap & eret branches"
+
+ if ! is_arm64; then
+ echo "Testing trap & eret branches [Skipped not arm64]"
+ return
+ fi
+ start_err=$err
+ local ret=1
+ for loops in 1000 10000 100000; do
+ err=0
+ perf record -o $TMPDIR/perf.data --branch-filter any,save_type,u,k -- \
+ perf test -w traploop $loops > "$TMPDIR/record.txt" 2>&1
+ perf script -i $TMPDIR/perf.data --fields brstacksym | \
+ tr ' ' '\n' > $TMPDIR/perf.script
+
+ # BRBINF<n>.TYPE == TRAP are mapped to PERF_BR_IRQ by the BRBE driver
+ check_branches "^trap_bench\+[^ ]+/[^ ]/IRQ/"
+ check_branches "^[^ ]+/trap_bench\+[^ ]+/ERET/"
+ if [ $err -eq 0 ]; then
+ ret=0
+ break
+ fi
+ done
+
+ if [ $ret -eq 0 ]; then
+ echo "Testing trap & eret branches [Passed]"
+ err=$start_err
+ else
+ echo "Testing trap & eret branches [Failed]"
+ err=1
+ fi
+}
+
+test_kernel_branches() {
+ echo "Testing kernel branch sampling"
+
+ if ! perf record --branch-filter any,k -o- -- true > "$TMPDIR/record.txt" 2>&1; then
+ echo "Testing that k option [Skipped not enough privileges]"
+ return
+ fi
+ start_err=$err
+ local ret=1
+ for loops in 1000 10000 100000; do
+ err=0
+ perf record -o $TMPDIR/perf.data --branch-filter any,k -- \
+ perf bench syscall basic --loop $loops > "$TMPDIR/record.txt" 2>&1
+ perf script -i $TMPDIR/perf.data --fields brstack | \
+ tr ' ' '\n' > $TMPDIR/perf.script
+
+ # Example of branch entries:
+ # "0xffffffff93bda241/0xffffffff93bda20f/M/-/-/..."
+ # Source addresses come first in user or kernel code. Next is the target
+ # address that must be in the kernel.
+
+ # Look for source addresses with top bit set
+ if ! grep -q -E -m1 "^0x[89a-f][0-9a-f]{15}" $TMPDIR/perf.script; then
+ err=1
+ fi
+ # Look for no target addresses without top bit set
+ if grep -q -E -m1 "^0x[0-9a-f]{0,16}/0x[0-7][0-9a-f]{1,15}/" \
+ $TMPDIR/perf.script; then
+ err=1
+ fi
+ if [ $err -eq 0 ]; then
+ ret=0
+ break
+ fi
+ done
+
+ if [ $ret -eq 0 ]; then
+ echo "Testing kernel branch sampling [Passed]"
+ err=$start_err
+ else
+ echo "Testing kernel branch sampling [Failed]"
+ err=1
+ fi
}
# first argument <arg0> is the argument passed to "--branch-stack <arg0>,save_type,u"
@@ -57,26 +188,86 @@ test_filter() {
test_filter_expect=$2
echo "Testing branch stack filtering permutation ($test_filter_filter,$test_filter_expect)"
-
- perf record -o $TMPDIR/perf.data --branch-filter $test_filter_filter,save_type,u -- ${TESTPROG} > /dev/null 2>&1
- perf script -i $TMPDIR/perf.data --fields brstack | xargs -n1 > $TMPDIR/perf.script
+ perf record -o "$TMPDIR/perf.data" --branch-filter "$test_filter_filter,save_type,u" -- \
+ ${TESTPROG} > "$TMPDIR/record.txt" 2>&1
+ perf script -i "$TMPDIR/perf.data" --fields brstack > "$TMPDIR/perf.script"
# fail if we find any branch type that doesn't match any of the expected ones
# also consider UNKNOWN branch types (-)
- if grep -E -vm1 "^[^ ]*/($test_filter_expect|-|( *))/.*$" $TMPDIR/perf.script; then
- return 1
+ if [ ! -s "$TMPDIR/perf.script" ]
+ then
+ echo "Testing branch stack filtering [Failed empty script output]"
+ err=1
+ return
+ fi
+ # Look for lines not matching test_filter_expect ignoring issues caused
+ # by empty output
+ tr -s ' ' '\n' < "$TMPDIR/perf.script" | grep '.' | \
+ grep -E -vm1 "^[^ ]*/($test_filter_expect|-|( *))/.*$" \
+ > "$TMPDIR/perf.script-filtered" || true
+ if [ -s "$TMPDIR/perf.script-filtered" ]
+ then
+ echo "Testing branch stack filtering [Failed unexpected branch filter]"
+ cat "$TMPDIR/perf.script"
+ err=1
+ return
fi
+ echo "Testing branch stack filtering [Passed]"
}
+test_syscall() {
+ echo "Testing syscalls"
+ # skip if perf doesn't have enough privileges
+ if ! perf record --branch-filter any,k -o- -- true > "$TMPDIR/record.txt" 2>&1; then
+ echo "Testing syscalls [Skipped: not enough privileges]"
+ return
+ fi
+ start_err=$err
+ local ret=1
+ for loops in 8000 30000 100000; do
+ err=0
+ perf record -o $TMPDIR/perf.data --branch-filter \
+ any_call,save_type,u,k -c 10007 -- \
+ perf bench syscall basic --loop $loops > "$TMPDIR/record.txt" 2>&1
+ perf script -i $TMPDIR/perf.data --fields brstacksym | \
+ tr ' ' '\n' > $TMPDIR/perf.script
+
+ check_branches "getppid[^ ]*/SYSCALL/"
+ if [ $err -eq 0 ]; then
+ ret=0
+ break
+ fi
+ done
+
+ if [ $ret -eq 0 ]; then
+ echo "Testing syscalls [Passed]"
+ err=$start_err
+ else
+ echo "Testing syscalls [Failed]"
+ err=1
+ fi
+}
set -e
test_user_branches
+test_syscall
+test_kernel_branches
+test_trap_eret_branches
-test_filter "any_call" "CALL|IND_CALL|COND_CALL|SYSCALL|IRQ"
+any_call="CALL|IND_CALL|COND_CALL|SYSCALL|IRQ"
+
+if is_arm64; then
+ any_call="$any_call|FAULT_DATA|FAULT_INST"
+fi
+
+test_filter "any_call" "$any_call"
test_filter "call" "CALL|SYSCALL"
test_filter "cond" "COND"
test_filter "any_ret" "RET|COND_RET|SYSRET|ERET"
test_filter "call,cond" "CALL|SYSCALL|COND"
-test_filter "any_call,cond" "CALL|IND_CALL|COND_CALL|IRQ|SYSCALL|COND"
-test_filter "cond,any_call,any_ret" "COND|CALL|IND_CALL|COND_CALL|SYSCALL|IRQ|RET|COND_RET|SYSRET|ERET"
+test_filter "any_call,cond" "$any_call|COND"
+test_filter "any_call,cond,any_ret" "$any_call|COND|RET|COND_RET"
+
+cleanup
+exit $err
diff --git a/tools/perf/tests/task-exit.c b/tools/perf/tests/task-exit.c
index d33d0952025c..8e328bbd509d 100644
--- a/tools/perf/tests/task-exit.c
+++ b/tools/perf/tests/task-exit.c
@@ -152,4 +152,11 @@ static int test__task_exit(struct test_suite *test __maybe_unused, int subtest _
return err;
}
-DEFINE_SUITE("Number of exit events of a simple workload", task_exit);
+struct test_case tests__task_exit[] = {
+ TEST_CASE_EXCLUSIVE("Number of exit events of a simple workload", task_exit),
+ { .name = NULL, }
+};
+struct test_suite suite__task_exit = {
+ .desc = "Number of exit events of a simple workload",
+ .test_cases = tests__task_exit,
+};
diff --git a/tools/perf/tests/tests-scripts.c b/tools/perf/tests/tests-scripts.c
index b6986d50dde6..b8651eb7109d 100644
--- a/tools/perf/tests/tests-scripts.c
+++ b/tools/perf/tests/tests-scripts.c
@@ -175,6 +175,7 @@ static void append_script(int dir_fd, const char *name, char *desc,
struct test_suite *test_suite, **result_tmp;
struct test_case *tests;
size_t len;
+ char *exclusive;
snprintf(link, sizeof(link), "/proc/%d/fd/%d", getpid(), dir_fd);
len = readlink(link, filename, sizeof(filename));
@@ -191,9 +192,13 @@ static void append_script(int dir_fd, const char *name, char *desc,
return;
}
tests[0].name = strdup_check(name);
+ exclusive = strstr(desc, " (exclusive)");
+ if (exclusive != NULL) {
+ tests[0].exclusive = true;
+ exclusive[0] = '\0';
+ }
tests[0].desc = strdup_check(desc);
tests[0].run_case = shell_test__run;
-
test_suite = zalloc(sizeof(*test_suite));
if (!test_suite) {
pr_err("Out of memory while building script test suite list\n");
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 6ea2be86b7bf..b98b5a6e90ca 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -36,12 +36,15 @@ struct test_case {
const char *desc;
const char *skip_reason;
test_fnptr run_case;
+ bool exclusive;
+ void *priv;
};
struct test_suite {
const char *desc;
struct test_case *test_cases;
void *priv;
+ int (*setup)(struct test_suite *suite);
};
#define DECLARE_SUITE(name) \
@@ -62,6 +65,14 @@ struct test_suite {
.skip_reason = _reason, \
}
+#define TEST_CASE_EXCLUSIVE(description, _name) \
+ { \
+ .name = #_name, \
+ .desc = description, \
+ .run_case = test__##_name, \
+ .exclusive = true, \
+ }
+
#define DEFINE_SUITE(description, _name) \
struct test_case tests__##_name[] = { \
TEST_CASE(description, _name), \
@@ -206,6 +217,7 @@ DECLARE_WORKLOAD(sqrtloop);
DECLARE_WORKLOAD(brstack);
DECLARE_WORKLOAD(datasym);
DECLARE_WORKLOAD(landlock);
+DECLARE_WORKLOAD(traploop);
extern const char *dso_to_test;
extern const char *test_objdump_path;
diff --git a/tools/perf/tests/workloads/Build b/tools/perf/tests/workloads/Build
index 5af17206f04d..fb1012cc4fc3 100644
--- a/tools/perf/tests/workloads/Build
+++ b/tools/perf/tests/workloads/Build
@@ -7,8 +7,10 @@ perf-test-y += sqrtloop.o
perf-test-y += brstack.o
perf-test-y += datasym.o
perf-test-y += landlock.o
+perf-test-y += traploop.o
CFLAGS_sqrtloop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
CFLAGS_leafloop.o = -g -O0 -fno-inline -fno-omit-frame-pointer -U_FORTIFY_SOURCE
CFLAGS_brstack.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
CFLAGS_datasym.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
+CFLAGS_traploop.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
diff --git a/tools/perf/tests/workloads/traploop.c b/tools/perf/tests/workloads/traploop.c
new file mode 100644
index 000000000000..68dec399a735
--- /dev/null
+++ b/tools/perf/tests/workloads/traploop.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#include "../tests.h"
+
+#define BENCH_RUNS 999999
+
+#ifdef __aarch64__
+static void trap_bench(void)
+{
+ unsigned long val;
+
+ asm("mrs %0, ID_AA64ISAR0_EL1" : "=r" (val)); /* TRAP + ERET */
+}
+#else
+static void trap_bench(void) { }
+#endif
+
+static int traploop(int argc, const char **argv)
+{
+ int num_loops = BENCH_RUNS;
+
+ if (argc > 0)
+ num_loops = atoi(argv[0]);
+
+ for (int i = 0; i < num_loops; i++)
+ trap_bench();
+
+ return 0;
+}
+
+DEFINE_WORKLOAD(traploop);
diff --git a/tools/perf/ui/browsers/hists.c b/tools/perf/ui/browsers/hists.c
index 3283b6313bab..00d1ca3295b4 100644
--- a/tools/perf/ui/browsers/hists.c
+++ b/tools/perf/ui/browsers/hists.c
@@ -3033,15 +3033,15 @@ static int evsel__hists_browse(struct evsel *evsel, int nr_events, const char *h
browser->min_pcnt = min_pcnt;
hist_browser__update_nr_entries(browser);
+ memset(options, 0, sizeof(options));
+ memset(actions, 0, sizeof(actions));
+
browser->pstack = pstack__new(3);
if (browser->pstack == NULL)
goto out;
ui_helpline__push(helpline);
- memset(options, 0, sizeof(options));
- memset(actions, 0, sizeof(actions));
-
if (symbol_conf.col_width_list_str)
perf_hpp__set_user_width(symbol_conf.col_width_list_str);
diff --git a/tools/perf/util/auxtrace.c b/tools/perf/util/auxtrace.c
index ca8682966fae..0927cfb42b7c 100644
--- a/tools/perf/util/auxtrace.c
+++ b/tools/perf/util/auxtrace.c
@@ -237,8 +237,12 @@ static int auxtrace_queues__grow(struct auxtrace_queues *queues,
{
unsigned int nr_queues = queues->nr_queues;
struct auxtrace_queue *queue_array;
+ struct auxtrace_queue *old_array = queues->queue_array;
unsigned int i;
+ if (!new_nr_queues)
+ return -EINVAL;
+
if (!nr_queues)
nr_queues = AUXTRACE_INIT_NR_QUEUES;
@@ -253,16 +257,17 @@ static int auxtrace_queues__grow(struct auxtrace_queues *queues,
return -ENOMEM;
for (i = 0; i < queues->nr_queues; i++) {
- list_splice_tail(&queues->queue_array[i].head,
+ list_splice_tail(&old_array[i].head,
&queue_array[i].head);
- queue_array[i].tid = queues->queue_array[i].tid;
- queue_array[i].cpu = queues->queue_array[i].cpu;
- queue_array[i].set = queues->queue_array[i].set;
- queue_array[i].priv = queues->queue_array[i].priv;
+ queue_array[i].tid = old_array[i].tid;
+ queue_array[i].cpu = old_array[i].cpu;
+ queue_array[i].set = old_array[i].set;
+ queue_array[i].priv = old_array[i].priv;
}
queues->nr_queues = nr_queues;
queues->queue_array = queue_array;
+ free(old_array);
return 0;
}
diff --git a/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c b/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c
index e4352881e3fa..95ad8d7c1a19 100644
--- a/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c
+++ b/tools/perf/util/bpf_skel/augmented_raw_syscalls.bpf.c
@@ -428,15 +428,88 @@ static bool pid_filter__has(struct pids_filtered *pids, pid_t pid)
return bpf_map_lookup_elem(pids, &pid) != NULL;
}
+u64 ZERO = 0;
+
+/*
+ * Determine what type of argument and how many bytes to read from user space, using the
+ * value in the beauty_map. This is the relation of parameter type and its corresponding
+ * value in the beauty map, and how many bytes we read eventually:
+ *
+ * string: 1 -> size of string
+ * struct: size of struct -> size of struct
+ * buffer: -1 * (index of paired len) -> value of paired len (maximum: TRACE_AUG_MAX_BUF)
+ */
+static inline int augment_arg(struct syscall_enter_args *args, int i,
+ unsigned int *beauty_map,
+ struct beauty_payload_enter *payload, u64 offset)
+{
+ int index, value_size = sizeof(struct augmented_arg) - offsetof(struct augmented_arg, value);
+ struct augmented_arg *payload_offset;
+ s64 aug_size, size;
+ bool augmented;
+ void *arg;
+
+ arg = (void *)args->args[i];
+ augmented = false;
+ size = beauty_map[i];
+ aug_size = size; /* size of the augmented data read from user space */
+
+ if (size == 0 || arg == NULL)
+ return 0;
+
+ /* bounds check for the verifier */
+ if (offset > sizeof(payload->aug_args) - sizeof(payload->aug_args[0]))
+ return -1;
+ barrier_var(offset);
+ payload_offset = (struct augmented_arg *)((void *)&payload->aug_args + offset);
+
+ if (size == 1) { /* string */
+ aug_size = bpf_probe_read_user_str(payload_offset->value, value_size, arg);
+ /* minimum of 0 to pass the verifier */
+ if (aug_size < 0)
+ aug_size = 0;
+
+ augmented = true;
+ } else if (size > 0 && size <= value_size) { /* struct */
+ if (!bpf_probe_read_user(payload_offset->value, size, arg))
+ augmented = true;
+ } else if ((int)size < 0 && size >= -6) { /* buffer */
+ index = -(size + 1);
+ barrier_var(index); // Prevent clang (noticed with v18) from removing the &= 7 trick.
+ index &= 7; // Satisfy the bounds checking with the verifier in some kernels.
+ aug_size = args->args[index] > TRACE_AUG_MAX_BUF ? TRACE_AUG_MAX_BUF : args->args[index];
+
+ if (aug_size > 0) {
+ if (!bpf_probe_read_user(payload_offset->value, aug_size, arg))
+ augmented = true;
+ }
+ }
+
+ /* Augmented data size is limited to sizeof(augmented_arg->unnamed union with value field) */
+ if (aug_size > value_size)
+ aug_size = value_size;
+
+ /* write data to payload */
+ if (augmented) {
+ int written = offsetof(struct augmented_arg, value) + aug_size;
+
+ if (written < 0 || written > sizeof(struct augmented_arg))
+ return -1;
+
+ payload_offset->size = aug_size;
+ return written;
+ }
+
+ return 0;
+}
+
static int augment_sys_enter(void *ctx, struct syscall_enter_args *args)
{
- bool augmented, do_output = false;
- int zero = 0, index, value_size = sizeof(struct augmented_arg) - offsetof(struct augmented_arg, value);
+ bool do_output = false;
+ int i, zero = 0, written;
u64 output = 0; /* has to be u64, otherwise it won't pass the verifier */
- s64 aug_size, size;
unsigned int nr, *beauty_map;
struct beauty_payload_enter *payload;
- void *arg, *payload_offset;
/* fall back to do predefined tail call */
if (args == NULL)
@@ -448,7 +521,6 @@ static int augment_sys_enter(void *ctx, struct syscall_enter_args *args)
/* set up payload for output */
payload = bpf_map_lookup_elem(&beauty_payload_enter_map, &zero);
- payload_offset = (void *)&payload->aug_args;
if (beauty_map == NULL || payload == NULL)
return 1;
@@ -456,61 +528,30 @@ static int augment_sys_enter(void *ctx, struct syscall_enter_args *args)
/* copy the sys_enter header, which has the syscall_nr */
__builtin_memcpy(&payload->args, args, sizeof(struct syscall_enter_args));
- /*
- * Determine what type of argument and how many bytes to read from user space, using the
- * value in the beauty_map. This is the relation of parameter type and its corresponding
- * value in the beauty map, and how many bytes we read eventually:
- *
- * string: 1 -> size of string
- * struct: size of struct -> size of struct
- * buffer: -1 * (index of paired len) -> value of paired len (maximum: TRACE_AUG_MAX_BUF)
- */
- for (int i = 0; i < 6; i++) {
- arg = (void *)args->args[i];
- augmented = false;
- size = beauty_map[i];
- aug_size = size; /* size of the augmented data read from user space */
-
- if (size == 0 || arg == NULL)
- continue;
-
- if (size == 1) { /* string */
- aug_size = bpf_probe_read_user_str(((struct augmented_arg *)payload_offset)->value, value_size, arg);
- /* minimum of 0 to pass the verifier */
- if (aug_size < 0)
- aug_size = 0;
-
- augmented = true;
- } else if (size > 0 && size <= value_size) { /* struct */
- if (!bpf_probe_read_user(((struct augmented_arg *)payload_offset)->value, size, arg))
- augmented = true;
- } else if ((int)size < 0 && size >= -6) { /* buffer */
- index = -(size + 1);
- barrier_var(index); // Prevent clang (noticed with v18) from removing the &= 7 trick.
- index &= 7; // Satisfy the bounds checking with the verifier in some kernels.
- aug_size = args->args[index] > TRACE_AUG_MAX_BUF ? TRACE_AUG_MAX_BUF : args->args[index];
-
- if (aug_size > 0) {
- if (!bpf_probe_read_user(((struct augmented_arg *)payload_offset)->value, aug_size, arg))
- augmented = true;
+ if (bpf_ksym_exists(bpf_iter_num_new)) {
+ bpf_for(i, 0, 6) {
+ written = augment_arg(args, i, beauty_map, payload, output);
+ if (written < 0)
+ return 1;
+ if (written > 0) {
+ output += written;
+ /*
+ * guide the verifier to forget range of `output`, which
+ * helps to prove convergence of the loop
+ */
+ output += ZERO;
+ do_output = true;
}
}
-
- /* Augmented data size is limited to sizeof(augmented_arg->unnamed union with value field) */
- if (aug_size > value_size)
- aug_size = value_size;
-
- /* write data to payload */
- if (augmented) {
- int written = offsetof(struct augmented_arg, value) + aug_size;
-
- if (written < 0 || written > sizeof(struct augmented_arg))
+ } else {
+ for (i = 0; i < 6; i++) {
+ written = augment_arg(args, i, beauty_map, payload, output);
+ if (written < 0)
return 1;
-
- ((struct augmented_arg *)payload_offset)->size = aug_size;
- output += written;
- payload_offset += written;
- do_output = true;
+ if (written > 0) {
+ output += written;
+ do_output = true;
+ }
}
}
diff --git a/tools/perf/util/color.h b/tools/perf/util/color.h
index 01f7bed21c9b..4b9f8d5d4439 100644
--- a/tools/perf/util/color.h
+++ b/tools/perf/util/color.h
@@ -22,6 +22,7 @@
#define MIN_GREEN 0.5
#define MIN_RED 5.0
+#define PERF_COLOR_DELETE_LINE "\033[A\33[2K\r"
/*
* This variable stores the value of color.ui
*/
diff --git a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
index 9bf09a856b44..c1257c5ef425 100644
--- a/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
+++ b/tools/perf/util/cs-etm-decoder/cs-etm-decoder.c
@@ -399,6 +399,8 @@ cs_etm_decoder__buffer_packet(struct cs_etm_queue *etmq,
packet_queue->packet_buffer[et].flags = 0;
packet_queue->packet_buffer[et].exception_number = UINT32_MAX;
packet_queue->packet_buffer[et].trace_chan_id = trace_chan_id;
+ packet_queue->packet_buffer[et].el = ocsd_EL_unknown;
+ packet_queue->packet_buffer[et].tid = -1;
if (packet_queue->packet_count == CS_ETM_PACKET_MAX_BUFFER - 1)
return OCSD_RESP_WAIT;
@@ -446,6 +448,7 @@ cs_etm_decoder__buffer_range(struct cs_etm_queue *etmq,
packet->last_instr_type = elem->last_i_type;
packet->last_instr_subtype = elem->last_i_subtype;
packet->last_instr_cond = elem->last_instr_cond;
+ packet->el = elem->context.exception_level;
if (elem->last_i_type == OCSD_INSTR_BR || elem->last_i_type == OCSD_INSTR_BR_INDIRECT)
packet->last_instr_taken_branch = elem->last_instr_exec;
@@ -522,7 +525,9 @@ cs_etm_decoder__set_tid(struct cs_etm_queue *etmq,
const ocsd_generic_trace_elem *elem,
const uint8_t trace_chan_id)
{
+ struct cs_etm_packet *packet;
pid_t tid = -1;
+ int ret;
/*
* Process the PE_CONTEXT packets if we have a valid contextID or VMID.
@@ -543,12 +548,18 @@ cs_etm_decoder__set_tid(struct cs_etm_queue *etmq,
break;
}
- if (cs_etm__etmq_set_tid_el(etmq, tid, trace_chan_id,
- elem->context.exception_level))
+ if (cs_etm__etmq_update_decode_context(etmq, trace_chan_id,
+ elem->context.exception_level, tid))
return OCSD_RESP_FATAL_SYS_ERR;
- if (tid == -1)
- return OCSD_RESP_CONT;
+ ret = cs_etm_decoder__buffer_packet(etmq, packet_queue, trace_chan_id,
+ CS_ETM_CONTEXT);
+ if (ret != OCSD_RESP_CONT && ret != OCSD_RESP_WAIT)
+ return ret;
+
+ packet = &packet_queue->packet_buffer[packet_queue->tail];
+ packet->tid = tid;
+ packet->el = elem->context.exception_level;
/*
* A timestamp is generated after a PE_CONTEXT element so make sure
@@ -556,7 +567,7 @@ cs_etm_decoder__set_tid(struct cs_etm_queue *etmq,
*/
cs_etm_decoder__reset_timestamp(packet_queue);
- return OCSD_RESP_CONT;
+ return ret;
}
static ocsd_datapath_resp_t cs_etm_decoder__gen_trace_elem_printer(
diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c
index 0bf9e5c27b59..a6344ff19f74 100644
--- a/tools/perf/util/cs-etm.c
+++ b/tools/perf/util/cs-etm.c
@@ -86,15 +86,22 @@ struct cs_etm_traceid_queue {
u64 period_instructions;
size_t last_branch_pos;
union perf_event *event_buf;
- struct thread *thread;
- struct thread *prev_packet_thread;
- ocsd_ex_level prev_packet_el;
- ocsd_ex_level el;
struct branch_stack *last_branch;
struct branch_stack *last_branch_rb;
struct cs_etm_packet *prev_packet;
struct cs_etm_packet *packet;
struct cs_etm_packet_queue packet_queue;
+
+ struct thread *decode_thread;
+ ocsd_ex_level decode_el;
+
+ /*
+ * The frontend accesses the EL from '[prev_]packet' because it needs
+ * previous EL for branch and current EL for instruction samples. It's
+ * not possible to change thread in a single branch sample so no need to
+ * store or access the thread through the packet.
+ */
+ struct thread *frontend_thread;
};
enum cs_etm_format {
@@ -608,10 +615,13 @@ static int cs_etm__init_traceid_queue(struct cs_etm_queue *etmq,
queue = &etmq->etm->queues.queue_array[etmq->queue_nr];
tidq->trace_chan_id = trace_chan_id;
- tidq->el = tidq->prev_packet_el = ocsd_EL_unknown;
- tidq->thread = machine__findnew_thread(&etm->session->machines.host, -1,
+ tidq->decode_el = ocsd_EL_unknown;
+ tidq->frontend_thread = machine__findnew_thread(&etm->session->machines.host, -1,
+ queue->tid);
+ tidq->decode_thread = machine__findnew_thread(&etm->session->machines.host, -1,
queue->tid);
- tidq->prev_packet_thread = machine__idle_thread(&etm->session->machines.host);
+ if (!tidq->frontend_thread || !tidq->decode_thread)
+ goto out;
tidq->packet = zalloc(sizeof(struct cs_etm_packet));
if (!tidq->packet)
@@ -646,6 +656,8 @@ static int cs_etm__init_traceid_queue(struct cs_etm_queue *etmq,
zfree(&tidq->prev_packet);
zfree(&tidq->packet);
out:
+ thread__zput(tidq->frontend_thread);
+ thread__zput(tidq->decode_thread);
return rc;
}
@@ -744,21 +756,10 @@ static void cs_etm__packet_swap(struct cs_etm_auxtrace *etm,
/*
* Swap PACKET with PREV_PACKET: PACKET becomes PREV_PACKET for
* the next incoming packet.
- *
- * Threads and exception levels are also tracked for both the
- * previous and current packets. This is because the previous
- * packet is used for the 'from' IP for branch samples, so the
- * thread at that time must also be assigned to that sample.
- * Across discontinuity packets the thread can change, so by
- * tracking the thread for the previous packet the branch sample
- * will have the correct info.
*/
tmp = tidq->packet;
tidq->packet = tidq->prev_packet;
tidq->prev_packet = tmp;
- tidq->prev_packet_el = tidq->el;
- thread__put(tidq->prev_packet_thread);
- tidq->prev_packet_thread = thread__get(tidq->thread);
}
}
@@ -931,8 +932,8 @@ static void cs_etm__free_traceid_queues(struct cs_etm_queue *etmq)
/* Free this traceid_queue from the array */
tidq = etmq->traceid_queues[idx];
- thread__zput(tidq->thread);
- thread__zput(tidq->prev_packet_thread);
+ thread__zput(tidq->frontend_thread);
+ thread__zput(tidq->decode_thread);
zfree(&tidq->event_buf);
zfree(&tidq->last_branch);
zfree(&tidq->last_branch_rb);
@@ -1077,47 +1078,43 @@ static u8 cs_etm__cpu_mode(struct cs_etm_queue *etmq, u64 address,
}
}
-static u32 cs_etm__mem_access(struct cs_etm_queue *etmq, u8 trace_chan_id,
- u64 address, size_t size, u8 *buffer,
- const ocsd_mem_space_acc_t mem_space)
+static u32 __cs_etm__mem_access(struct cs_etm_queue *etmq,
+ u64 address, size_t size, u8 *buffer,
+ const ocsd_mem_space_acc_t mem_space,
+ ocsd_ex_level el, struct thread *thread)
{
u8 cpumode;
u64 offset;
int len;
struct addr_location al;
struct dso *dso;
- struct cs_etm_traceid_queue *tidq;
int ret = 0;
if (!etmq)
return 0;
addr_location__init(&al);
- tidq = cs_etm__etmq_get_traceid_queue(etmq, trace_chan_id);
- if (!tidq)
- goto out;
/*
- * We've already tracked EL along side the PID in cs_etm__set_thread()
- * so double check that it matches what OpenCSD thinks as well. It
- * doesn't distinguish between EL0 and EL1 for this mem access callback
- * so we had to do the extra tracking. Skip validation if it's any of
- * the 'any' values.
+ * We track EL for the frontend and the backend when receiving context
+ * and range packets. OpenCSD doesn't distinguish between EL0 and EL1
+ * for this mem access callback so we had to do the extra tracking. Skip
+ * validation if it's any of the 'any' values.
*/
if (!(mem_space == OCSD_MEM_SPACE_ANY ||
mem_space == OCSD_MEM_SPACE_N || mem_space == OCSD_MEM_SPACE_S)) {
if (mem_space & OCSD_MEM_SPACE_EL1N) {
/* Includes both non secure EL1 and EL0 */
- assert(tidq->el == ocsd_EL1 || tidq->el == ocsd_EL0);
+ assert(el == ocsd_EL1 || el == ocsd_EL0);
} else if (mem_space & OCSD_MEM_SPACE_EL2)
- assert(tidq->el == ocsd_EL2);
+ assert(el == ocsd_EL2);
else if (mem_space & OCSD_MEM_SPACE_EL3)
- assert(tidq->el == ocsd_EL3);
+ assert(el == ocsd_EL3);
}
- cpumode = cs_etm__cpu_mode(etmq, address, tidq->el);
+ cpumode = cs_etm__cpu_mode(etmq, address, el);
- if (!thread__find_map(tidq->thread, cpumode, address, &al))
+ if (!thread__find_map(thread, cpumode, address, &al))
goto out;
dso = map__dso(al.map);
@@ -1132,7 +1129,7 @@ static u32 cs_etm__mem_access(struct cs_etm_queue *etmq, u8 trace_chan_id,
map__load(al.map);
- len = dso__data_read_offset(dso, maps__machine(thread__maps(tidq->thread)),
+ len = dso__data_read_offset(dso, maps__machine(thread__maps(thread)),
offset, buffer, size);
if (len <= 0) {
@@ -1152,6 +1149,30 @@ static u32 cs_etm__mem_access(struct cs_etm_queue *etmq, u8 trace_chan_id,
return ret;
}
+static u32 cs_etm__frontend_mem_access(struct cs_etm_queue *etmq,
+ struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet,
+ u64 address, size_t size, u8 *buffer)
+{
+ return __cs_etm__mem_access(etmq, address, size, buffer, 0, packet->el,
+ tidq->frontend_thread);
+}
+
+static u32 cs_etm__decoder_mem_access(struct cs_etm_queue *etmq, u8 trace_chan_id,
+ u64 address, size_t size, u8 *buffer,
+ const ocsd_mem_space_acc_t mem_space)
+{
+ struct cs_etm_traceid_queue *tidq;
+
+ tidq = cs_etm__etmq_get_traceid_queue(etmq, trace_chan_id);
+ if (!tidq)
+ return 0;
+
+ return __cs_etm__mem_access(etmq, address, size, buffer,
+ mem_space, tidq->decode_el,
+ tidq->decode_thread);
+}
+
static struct cs_etm_queue *cs_etm__alloc_queue(void)
{
struct cs_etm_queue *etmq = zalloc(sizeof(*etmq));
@@ -1327,12 +1348,13 @@ void cs_etm__reset_last_branch_rb(struct cs_etm_traceid_queue *tidq)
}
static inline int cs_etm__t32_instr_size(struct cs_etm_queue *etmq,
- u8 trace_chan_id, u64 addr)
+ struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet, u64 addr)
{
u8 instrBytes[2];
- cs_etm__mem_access(etmq, trace_chan_id, addr, ARRAY_SIZE(instrBytes),
- instrBytes, 0);
+ cs_etm__frontend_mem_access(etmq, tidq, packet, addr,
+ ARRAY_SIZE(instrBytes), instrBytes);
/*
* T32 instruction size is indicated by bits[15:11] of the first
* 16-bit word of the instruction: 0b11101, 0b11110 and 0b11111
@@ -1365,16 +1387,16 @@ u64 cs_etm__last_executed_instr(const struct cs_etm_packet *packet)
}
static inline u64 cs_etm__instr_addr(struct cs_etm_queue *etmq,
- u64 trace_chan_id,
- const struct cs_etm_packet *packet,
+ struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet,
u64 offset)
{
if (packet->isa == CS_ETM_ISA_T32) {
u64 addr = packet->start_addr;
while (offset) {
- addr += cs_etm__t32_instr_size(etmq,
- trace_chan_id, addr);
+ addr += cs_etm__t32_instr_size(etmq, tidq, packet,
+ addr);
offset--;
}
return addr;
@@ -1462,38 +1484,54 @@ cs_etm__get_trace(struct cs_etm_queue *etmq)
etmq->buf_used = 0;
etmq->buf_len = aux_buffer->size;
etmq->buf = aux_buffer->data;
-
- return etmq->buf_len;
+ return 0;
}
-static void cs_etm__set_thread(struct cs_etm_queue *etmq,
- struct cs_etm_traceid_queue *tidq, pid_t tid,
- ocsd_ex_level el)
+/*
+ * Convert a raw thread number to a thread struct and assign it to **thread.
+ */
+static int cs_etm__etmq_update_thread(struct cs_etm_queue *etmq,
+ ocsd_ex_level el, pid_t tid,
+ struct thread **thread)
{
struct machine *machine = cs_etm__get_machine(etmq, el);
+ if (!machine || !*thread)
+ return -EINVAL;
+
if (tid != -1) {
- thread__zput(tidq->thread);
- tidq->thread = machine__find_thread(machine, -1, tid);
+ thread__zput(*thread);
+ *thread = machine__find_thread(machine, -1, tid);
}
/* Couldn't find a known thread */
- if (!tidq->thread)
- tidq->thread = machine__idle_thread(machine);
+ if (!*thread)
+ *thread = machine__idle_thread(machine);
- tidq->el = el;
+ return 0;
}
-int cs_etm__etmq_set_tid_el(struct cs_etm_queue *etmq, pid_t tid,
- u8 trace_chan_id, ocsd_ex_level el)
+/*
+ * Set the thread and EL of the decode context which is ahead in time of the
+ * frontend context.
+ */
+int cs_etm__etmq_update_decode_context(struct cs_etm_queue *etmq,
+ u8 trace_chan_id,
+ ocsd_ex_level el, pid_t tid)
{
struct cs_etm_traceid_queue *tidq;
+ int ret;
tidq = cs_etm__etmq_get_traceid_queue(etmq, trace_chan_id);
if (!tidq)
return -EINVAL;
- cs_etm__set_thread(etmq, tidq, tid, el);
+ ret = cs_etm__etmq_update_thread(etmq, el, tid,
+ &tidq->decode_thread);
+ if (ret)
+ return ret;
+
+ tidq->decode_el = el;
return 0;
}
@@ -1503,8 +1541,8 @@ bool cs_etm__etmq_is_timeless(struct cs_etm_queue *etmq)
}
static void cs_etm__copy_insn(struct cs_etm_queue *etmq,
- u64 trace_chan_id,
- const struct cs_etm_packet *packet,
+ struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet,
struct perf_sample *sample)
{
/*
@@ -1521,14 +1559,14 @@ static void cs_etm__copy_insn(struct cs_etm_queue *etmq,
* cs_etm__t32_instr_size().
*/
if (packet->isa == CS_ETM_ISA_T32)
- sample->insn_len = cs_etm__t32_instr_size(etmq, trace_chan_id,
+ sample->insn_len = cs_etm__t32_instr_size(etmq, tidq, packet,
sample->ip);
/* Otherwise, A64 and A32 instruction size are always 32-bit. */
else
sample->insn_len = 4;
- cs_etm__mem_access(etmq, trace_chan_id, sample->ip, sample->insn_len,
- (void *)sample->insn, 0);
+ cs_etm__frontend_mem_access(etmq, tidq, packet, sample->ip,
+ sample->insn_len, (void *)sample->insn);
}
u64 cs_etm__convert_sample_time(struct cs_etm_queue *etmq, u64 cs_timestamp)
@@ -1555,6 +1593,7 @@ static inline u64 cs_etm__resolve_sample_time(struct cs_etm_queue *etmq,
static int cs_etm__synth_instruction_sample(struct cs_etm_queue *etmq,
struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet,
u64 addr, u64 period)
{
int ret = 0;
@@ -1563,23 +1602,23 @@ static int cs_etm__synth_instruction_sample(struct cs_etm_queue *etmq,
struct perf_sample sample = {.ip = 0,};
event->sample.header.type = PERF_RECORD_SAMPLE;
- event->sample.header.misc = cs_etm__cpu_mode(etmq, addr, tidq->el);
+ event->sample.header.misc = cs_etm__cpu_mode(etmq, addr, packet->el);
event->sample.header.size = sizeof(struct perf_event_header);
/* Set time field based on etm auxtrace config. */
sample.time = cs_etm__resolve_sample_time(etmq, tidq);
sample.ip = addr;
- sample.pid = thread__pid(tidq->thread);
- sample.tid = thread__tid(tidq->thread);
+ sample.pid = thread__pid(tidq->frontend_thread);
+ sample.tid = thread__tid(tidq->frontend_thread);
sample.id = etmq->etm->instructions_id;
sample.stream_id = etmq->etm->instructions_id;
sample.period = period;
- sample.cpu = tidq->packet->cpu;
+ sample.cpu = packet->cpu;
sample.flags = tidq->prev_packet->flags;
sample.cpumode = event->sample.header.misc;
- cs_etm__copy_insn(etmq, tidq->trace_chan_id, tidq->packet, &sample);
+ cs_etm__copy_insn(etmq, tidq, packet, &sample);
if (etm->synth_opts.last_branch)
sample.branch_stack = tidq->last_branch;
@@ -1623,15 +1662,15 @@ static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq,
event->sample.header.type = PERF_RECORD_SAMPLE;
event->sample.header.misc = cs_etm__cpu_mode(etmq, ip,
- tidq->prev_packet_el);
+ tidq->prev_packet->el);
event->sample.header.size = sizeof(struct perf_event_header);
/* Set time field based on etm auxtrace config. */
sample.time = cs_etm__resolve_sample_time(etmq, tidq);
sample.ip = ip;
- sample.pid = thread__pid(tidq->prev_packet_thread);
- sample.tid = thread__tid(tidq->prev_packet_thread);
+ sample.pid = thread__pid(tidq->frontend_thread);
+ sample.tid = thread__tid(tidq->frontend_thread);
sample.addr = cs_etm__first_executed_instr(tidq->packet);
sample.id = etmq->etm->branches_id;
sample.stream_id = etmq->etm->branches_id;
@@ -1640,8 +1679,7 @@ static int cs_etm__synth_branch_sample(struct cs_etm_queue *etmq,
sample.flags = tidq->prev_packet->flags;
sample.cpumode = event->sample.header.misc;
- cs_etm__copy_insn(etmq, tidq->trace_chan_id, tidq->prev_packet,
- &sample);
+ cs_etm__copy_insn(etmq, tidq, tidq->prev_packet, &sample);
/*
* perf report cannot handle events without a branch stack
@@ -1765,7 +1803,6 @@ static int cs_etm__sample(struct cs_etm_queue *etmq,
{
struct cs_etm_auxtrace *etm = etmq->etm;
int ret;
- u8 trace_chan_id = tidq->trace_chan_id;
u64 instrs_prev;
/* Get instructions remainder from previous packet */
@@ -1851,10 +1888,10 @@ static int cs_etm__sample(struct cs_etm_queue *etmq,
* been executed, but PC has not advanced to next
* instruction)
*/
- addr = cs_etm__instr_addr(etmq, trace_chan_id,
- tidq->packet, offset - 1);
+ addr = cs_etm__instr_addr(etmq, tidq, tidq->packet,
+ offset - 1);
ret = cs_etm__synth_instruction_sample(
- etmq, tidq, addr,
+ etmq, tidq, tidq->packet, addr,
etm->instructions_sample_period);
if (ret)
return ret;
@@ -1936,7 +1973,7 @@ static int cs_etm__flush(struct cs_etm_queue *etmq,
addr = cs_etm__last_executed_instr(tidq->prev_packet);
err = cs_etm__synth_instruction_sample(
- etmq, tidq, addr,
+ etmq, tidq, tidq->prev_packet, addr,
tidq->period_instructions);
if (err)
return err;
@@ -1991,7 +2028,7 @@ static int cs_etm__end_block(struct cs_etm_queue *etmq,
addr = cs_etm__last_executed_instr(tidq->prev_packet);
err = cs_etm__synth_instruction_sample(
- etmq, tidq, addr,
+ etmq, tidq, tidq->prev_packet, addr,
tidq->period_instructions);
if (err)
return err;
@@ -2001,6 +2038,45 @@ static int cs_etm__end_block(struct cs_etm_queue *etmq,
return 0;
}
+
+static int cs_etm__flush_stack_cb(struct thread *thread,
+ void *data __maybe_unused)
+{
+ thread_stack__flush(thread);
+ return 0;
+}
+
+static void cs_etm__flush_machine_stack(struct cs_etm_queue *etmq, pid_t pid)
+{
+ struct machine *machine;
+
+ machine = machines__find(&etmq->etm->session->machines, pid);
+ if (machine)
+ machine__for_each_thread(machine, cs_etm__flush_stack_cb, NULL);
+}
+
+static void cs_etm__flush_all_stack(struct cs_etm_queue *etmq)
+{
+ enum cs_etm_pid_fmt pid_fmt = cs_etm__get_pid_fmt(etmq);
+
+ if (!etmq->etm->synth_opts.last_branch)
+ return;
+
+ switch (pid_fmt) {
+ case CS_ETM_PIDFMT_CTXTID2:
+ /* Clear the guest stack if virtualization is supported */
+ cs_etm__flush_machine_stack(etmq, DEFAULT_GUEST_KERNEL_ID);
+ fallthrough;
+ case CS_ETM_PIDFMT_CTXTID:
+ cs_etm__flush_machine_stack(etmq, HOST_KERNEL_ID);
+ break;
+ case CS_ETM_PIDFMT_NONE:
+ default:
+ break;
+
+ }
+}
+
/*
* cs_etm__get_data_block: Fetch a block from the auxtrace_buffer queue
* if need be.
@@ -2012,25 +2088,38 @@ static int cs_etm__get_data_block(struct cs_etm_queue *etmq)
{
int ret;
- if (!etmq->buf_len) {
- ret = cs_etm__get_trace(etmq);
- if (ret <= 0)
- return ret;
- /*
- * We cannot assume consecutive blocks in the data file
- * are contiguous, reset the decoder to force re-sync.
- */
- ret = cs_etm_decoder__reset(etmq->decoder);
- if (ret)
- return ret;
- }
+ /* The current block is not finished */
+ if (etmq->buf_len)
+ return 1;
+
+ ret = cs_etm__get_trace(etmq);
+ if (ret < 0)
+ return ret;
+
+ /* No more buffer to read */
+ if (!etmq->buf_len)
+ return 0;
+
+ /*
+ * We cannot assume consecutive blocks in the data file
+ * are contiguous, reset the decoder to force re-sync.
+ */
+ ret = cs_etm_decoder__reset(etmq->decoder);
+ if (ret)
+ return ret;
- return etmq->buf_len;
+ /*
+ * Since the decoder is reset, this causes a global trace
+ * discontinuity. Flush all thread stacks.
+ */
+ cs_etm__flush_all_stack(etmq);
+
+ return 1;
}
-static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq, u8 trace_chan_id,
- struct cs_etm_packet *packet,
- u64 end_addr)
+static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq,
+ struct cs_etm_traceid_queue *tidq,
+ struct cs_etm_packet *packet, u64 end_addr)
{
/* Initialise to keep compiler happy */
u16 instr16 = 0;
@@ -2052,8 +2141,8 @@ static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq, u8 trace_chan_id,
* so below only read 2 bytes as instruction size for T32.
*/
addr = end_addr - 2;
- cs_etm__mem_access(etmq, trace_chan_id, addr, sizeof(instr16),
- (u8 *)&instr16, 0);
+ cs_etm__frontend_mem_access(etmq, tidq, packet, addr,
+ sizeof(instr16), (u8 *)&instr16);
if ((instr16 & 0xFF00) == 0xDF00)
return true;
@@ -2068,8 +2157,8 @@ static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq, u8 trace_chan_id,
* +---------+---------+-------------------------+
*/
addr = end_addr - 4;
- cs_etm__mem_access(etmq, trace_chan_id, addr, sizeof(instr32),
- (u8 *)&instr32, 0);
+ cs_etm__frontend_mem_access(etmq, tidq, packet, addr,
+ sizeof(instr32), (u8 *)&instr32);
if ((instr32 & 0x0F000000) == 0x0F000000 &&
(instr32 & 0xF0000000) != 0xF0000000)
return true;
@@ -2085,8 +2174,8 @@ static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq, u8 trace_chan_id,
* +-----------------------+---------+-----------+
*/
addr = end_addr - 4;
- cs_etm__mem_access(etmq, trace_chan_id, addr, sizeof(instr32),
- (u8 *)&instr32, 0);
+ cs_etm__frontend_mem_access(etmq, tidq, packet, addr,
+ sizeof(instr32), (u8 *)&instr32);
if ((instr32 & 0xFFE0001F) == 0xd4000001)
return true;
@@ -2102,7 +2191,6 @@ static bool cs_etm__is_svc_instr(struct cs_etm_queue *etmq, u8 trace_chan_id,
static bool cs_etm__is_syscall(struct cs_etm_queue *etmq,
struct cs_etm_traceid_queue *tidq, u64 magic)
{
- u8 trace_chan_id = tidq->trace_chan_id;
struct cs_etm_packet *packet = tidq->packet;
struct cs_etm_packet *prev_packet = tidq->prev_packet;
@@ -2117,7 +2205,7 @@ static bool cs_etm__is_syscall(struct cs_etm_queue *etmq,
*/
if (magic == __perf_cs_etmv4_magic) {
if (packet->exception_number == CS_ETMV4_EXC_CALL &&
- cs_etm__is_svc_instr(etmq, trace_chan_id, prev_packet,
+ cs_etm__is_svc_instr(etmq, tidq, prev_packet,
prev_packet->end_addr))
return true;
}
@@ -2155,7 +2243,6 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq,
struct cs_etm_traceid_queue *tidq,
u64 magic)
{
- u8 trace_chan_id = tidq->trace_chan_id;
struct cs_etm_packet *packet = tidq->packet;
struct cs_etm_packet *prev_packet = tidq->prev_packet;
@@ -2181,7 +2268,7 @@ static bool cs_etm__is_sync_exception(struct cs_etm_queue *etmq,
* (SMC, HVC) are taken as sync exceptions.
*/
if (packet->exception_number == CS_ETMV4_EXC_CALL &&
- !cs_etm__is_svc_instr(etmq, trace_chan_id, prev_packet,
+ !cs_etm__is_svc_instr(etmq, tidq, prev_packet,
prev_packet->end_addr))
return true;
@@ -2205,7 +2292,6 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq,
{
struct cs_etm_packet *packet = tidq->packet;
struct cs_etm_packet *prev_packet = tidq->prev_packet;
- u8 trace_chan_id = tidq->trace_chan_id;
u64 magic;
int ret;
@@ -2286,11 +2372,11 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq,
if (prev_packet->flags == (PERF_IP_FLAG_BRANCH |
PERF_IP_FLAG_RETURN |
PERF_IP_FLAG_INTERRUPT) &&
- cs_etm__is_svc_instr(etmq, trace_chan_id,
- packet, packet->start_addr))
+ cs_etm__is_svc_instr(etmq, tidq, packet, packet->start_addr)) {
prev_packet->flags = PERF_IP_FLAG_BRANCH |
PERF_IP_FLAG_RETURN |
PERF_IP_FLAG_SYSCALLRET;
+ }
break;
case CS_ETM_DISCONTINUITY:
/*
@@ -2371,6 +2457,7 @@ static int cs_etm__set_sample_flags(struct cs_etm_queue *etmq,
PERF_IP_FLAG_RETURN |
PERF_IP_FLAG_INTERRUPT;
break;
+ case CS_ETM_CONTEXT:
case CS_ETM_EMPTY:
default:
break;
@@ -2446,6 +2533,19 @@ static int cs_etm__process_traceid_queue(struct cs_etm_queue *etmq,
*/
cs_etm__sample(etmq, tidq);
break;
+ case CS_ETM_CONTEXT:
+ /*
+ * Update context but don't swap packet. Keep the
+ * previous one for branch source address info, if
+ * tracing the kernel the context packet will be emitted
+ * between two ranges.
+ */
+ ret = cs_etm__etmq_update_thread(etmq, tidq->packet->el,
+ tidq->packet->tid,
+ &tidq->frontend_thread);
+ if (ret)
+ goto out;
+ break;
case CS_ETM_EXCEPTION:
case CS_ETM_EXCEPTION_RET:
/*
@@ -2474,6 +2574,7 @@ static int cs_etm__process_traceid_queue(struct cs_etm_queue *etmq,
}
}
+out:
return ret;
}
@@ -2597,7 +2698,7 @@ static int cs_etm__process_timeless_queues(struct cs_etm_auxtrace *etm,
if (!tidq)
continue;
- if (tid == -1 || thread__tid(tidq->thread) == tid)
+ if (tid == -1 || thread__tid(tidq->frontend_thread) == tid)
cs_etm__run_per_thread_timeless_decoder(etmq);
} else
cs_etm__run_per_cpu_timeless_decoder(etmq);
@@ -3308,7 +3409,7 @@ static int cs_etm__create_queue_decoders(struct cs_etm_queue *etmq)
*/
if (cs_etm_decoder__add_mem_access_cb(etmq->decoder,
0x0L, ((u64) -1L),
- cs_etm__mem_access))
+ cs_etm__decoder_mem_access))
goto out_free_decoder;
zfree(&t_params);
diff --git a/tools/perf/util/cs-etm.h b/tools/perf/util/cs-etm.h
index a8caeea720aa..dfc9aeacfd0b 100644
--- a/tools/perf/util/cs-etm.h
+++ b/tools/perf/util/cs-etm.h
@@ -158,6 +158,7 @@ enum cs_etm_sample_type {
CS_ETM_DISCONTINUITY,
CS_ETM_EXCEPTION,
CS_ETM_EXCEPTION_RET,
+ CS_ETM_CONTEXT,
};
enum cs_etm_isa {
@@ -184,6 +185,8 @@ struct cs_etm_packet {
u8 last_instr_size;
u8 trace_chan_id;
int cpu;
+ int el;
+ pid_t tid;
};
#define CS_ETM_PACKET_MAX_BUFFER 1024
@@ -244,8 +247,9 @@ enum cs_etm_pid_fmt {
#include <opencsd/ocsd_if_types.h>
int cs_etm__get_cpu(struct cs_etm_queue *etmq, u8 trace_chan_id, int *cpu);
enum cs_etm_pid_fmt cs_etm__get_pid_fmt(struct cs_etm_queue *etmq);
-int cs_etm__etmq_set_tid_el(struct cs_etm_queue *etmq, pid_t tid,
- u8 trace_chan_id, ocsd_ex_level el);
+int cs_etm__etmq_update_decode_context(struct cs_etm_queue *etmq,
+ u8 trace_chan_id, ocsd_ex_level el,
+ pid_t tid);
bool cs_etm__etmq_is_timeless(struct cs_etm_queue *etmq);
void cs_etm__etmq_set_traceid_queue_timestamp(struct cs_etm_queue *etmq,
u8 trace_chan_id);
diff --git a/tools/perf/util/data-convert-json.c b/tools/perf/util/data-convert-json.c
index 20bfb0884e9e..395b231cf16c 100644
--- a/tools/perf/util/data-convert-json.c
+++ b/tools/perf/util/data-convert-json.c
@@ -242,6 +242,7 @@ static int process_sample_event(const struct perf_tool *tool,
trace_seq_init(&s);
tep_print_field(&s, sample->raw_data, fields[i]);
output_json_key_string(out, true, 3, fields[i]->name, s.buffer);
+ trace_seq_destroy(&s);
i++;
}
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 5c6e85fdae0d..9de78061a3f2 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -385,7 +385,9 @@ int dso__decompress_kmodule_path(struct dso *dso, const char *name,
{
int fd = decompress_kmodule(dso, name, pathname, len);
- close(fd);
+ /* decompress_kmodule() returns -1 on failure, don't close(-1) */
+ if (fd >= 0)
+ close(fd);
return fd >= 0 ? 0 : -1;
}
@@ -937,7 +939,20 @@ static ssize_t dso_cache__memcpy(struct dso_cache *cache, u64 offset, u8 *data,
u64 size, bool out)
{
u64 cache_offset = offset - cache->offset;
- u64 cache_size = min(cache->size - cache_offset, size);
+ u64 cache_size;
+
+ /*
+ * The RB tree matches using DSO__DATA_CACHE_SIZE, but a short
+ * pread may leave cache->size smaller. For a regular file a
+ * short pread only happens at end-of-file, so an offset past
+ * the valid data is EOF: return 0, matching what a direct
+ * pread() at that offset would return, and cached_io() then
+ * stops its read loop.
+ */
+ if (cache_offset >= cache->size)
+ return 0;
+
+ cache_size = min(cache->size - cache_offset, size);
if (out)
memcpy(data, cache->data + cache_offset, cache_size);
@@ -961,7 +976,7 @@ static ssize_t file_read(struct dso *dso, struct machine *machine,
if (dso__data(dso)->fd < 0) {
dso__data(dso)->status = DSO_DATA_STATUS_ERROR;
- ret = -errno;
+ ret = dso__data(dso)->fd;
goto out;
}
@@ -1084,8 +1099,8 @@ static int file_size(struct dso *dso, struct machine *machine)
try_to_open_dso(dso, machine);
if (dso__data(dso)->fd < 0) {
- ret = -errno;
dso__data(dso)->status = DSO_DATA_STATUS_ERROR;
+ ret = dso__data(dso)->fd;
goto out;
}
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index c0472a41147c..bb0238acd583 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -9,6 +9,7 @@
#include <stdbool.h>
#include <stdio.h>
#include <linux/bitops.h>
+#include <string.h>
#include "build-id.h"
#include "mutex.h"
#include <internal/rc_check.h>
@@ -19,6 +20,40 @@ struct perf_env;
#define DSO__NAME_KALLSYMS "[kernel.kallsyms]"
#define DSO__NAME_KCORE "[kernel.kcore]"
+#define DSO__NAME_GUEST_KALLSYMS "[guest.kernel.kallsyms]"
+#define DSO__NAME_GUEST_KALLSYMS_PID_PREFIX "[guest.kernel.kallsyms."
+
+/*
+ * Validate names of the form "[guest.kernel.kallsyms.<pid>]", where
+ * <pid> is the PID of the guest VM and varies per guest, so it
+ * cannot be matched with strcmp() against a fixed string.
+ *
+ * Every character after the fixed prefix must be a decimal digit,
+ * with ']' immediately terminating the digit run and nothing
+ * following it. This rules out '/', "..", or any other character
+ * being smuggled into the name.
+ */
+static inline bool is_guest_kallsyms_pid_name(const char *name)
+{
+ const size_t prefix_len = sizeof(DSO__NAME_GUEST_KALLSYMS_PID_PREFIX) - 1;
+ size_t digits;
+
+ if (strncmp(name, DSO__NAME_GUEST_KALLSYMS_PID_PREFIX, prefix_len) != 0)
+ return false;
+
+ digits = strspn(name + prefix_len, "0123456789");
+ if (digits == 0)
+ return false;
+
+ /* ']' must terminate the digit run, with nothing trailing it */
+ if (name[prefix_len + digits] != ']')
+ return false;
+
+ if (name[prefix_len + digits + 1] != '\0')
+ return false;
+
+ return true;
+}
enum dso_binary_type {
DSO_BINARY_TYPE__KALLSYMS = 0,
@@ -809,8 +844,28 @@ static inline bool dso__is_kcore(const struct dso *dso)
static inline bool dso__is_kallsyms(const struct dso *dso)
{
enum dso_binary_type bt = dso__binary_type(dso);
+ const char *name;
+
+ if (bt == DSO_BINARY_TYPE__KALLSYMS || bt == DSO_BINARY_TYPE__GUEST_KALLSYMS)
+ return true;
+
+ if (bt != DSO_BINARY_TYPE__NOT_FOUND)
+ return false;
+
+ if (!dso__kernel(dso))
+ return false;
+
+ name = dso__long_name(dso);
+ if (!name)
+ return false;
+
+ if (!strcmp(name, DSO__NAME_KALLSYMS))
+ return true;
+
+ if (!strcmp(name, DSO__NAME_GUEST_KALLSYMS))
+ return true;
- return bt == DSO_BINARY_TYPE__KALLSYMS || bt == DSO_BINARY_TYPE__GUEST_KALLSYMS;
+ return is_guest_kallsyms_pid_name(name);
}
bool dso__is_object_file(const struct dso *dso);
diff --git a/tools/perf/util/intel-bts.c b/tools/perf/util/intel-bts.c
index 27d9b5c9fec8..056dde772eab 100644
--- a/tools/perf/util/intel-bts.c
+++ b/tools/perf/util/intel-bts.c
@@ -829,7 +829,7 @@ int intel_bts_process_auxtrace_info(union perf_event *event,
struct perf_session *session)
{
struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info;
- size_t min_sz = sizeof(u64) * INTEL_BTS_SNAPSHOT_MODE;
+ size_t min_sz = sizeof(u64) * (INTEL_BTS_SNAPSHOT_MODE + 1);
struct intel_bts *bts;
int err;
diff --git a/tools/perf/util/intel-pt.c b/tools/perf/util/intel-pt.c
index 61f10578e121..fa4eb009a963 100644
--- a/tools/perf/util/intel-pt.c
+++ b/tools/perf/util/intel-pt.c
@@ -4359,7 +4359,7 @@ int intel_pt_process_auxtrace_info(union perf_event *event,
struct perf_session *session)
{
struct perf_record_auxtrace_info *auxtrace_info = &event->auxtrace_info;
- size_t min_sz = sizeof(u64) * INTEL_PT_PER_CPU_MMAPS;
+ size_t min_sz = sizeof(u64) * (INTEL_PT_PER_CPU_MMAPS + 1);
struct intel_pt *pt;
void *info_end;
__u64 *info;
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 20fd742984e3..77050eff8fcb 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -289,16 +289,22 @@ struct machine *machines__findnew(struct machines *machines, pid_t pid)
if ((pid != HOST_KERNEL_ID) &&
(pid != DEFAULT_GUEST_KERNEL_ID) &&
(symbol_conf.guestmount)) {
- sprintf(path, "%s/%d", symbol_conf.guestmount, pid);
+ if (snprintf(path, sizeof(path), "%s/%d",
+ symbol_conf.guestmount, pid) >= (int)sizeof(path)) {
+ pr_err("Guest path too long for pid %d\n", pid);
+ machine = NULL;
+ goto out;
+ }
if (access(path, R_OK)) {
static struct strlist *seen;
if (!seen)
seen = strlist__new(NULL, NULL);
- if (!strlist__has_entry(seen, path)) {
+ if (!seen || !strlist__has_entry(seen, path)) {
pr_err("Can't access file %s\n", path);
- strlist__add(seen, path);
+ if (seen)
+ strlist__add(seen, path);
}
machine = NULL;
goto out;
@@ -1204,27 +1210,35 @@ int machines__create_guest_kernel_maps(struct machines *machines)
for (i = 0; i < items; i++) {
if (!isdigit(namelist[i]->d_name[0])) {
/* Filter out . and .. */
+ free(namelist[i]);
continue;
}
+ errno = 0;
pid = (pid_t)strtol(namelist[i]->d_name, &endp, 10);
if ((*endp != '\0') ||
(endp == namelist[i]->d_name) ||
(errno == ERANGE)) {
pr_debug("invalid directory (%s). Skipping.\n",
namelist[i]->d_name);
+ free(namelist[i]);
continue;
}
- sprintf(path, "%s/%s/proc/kallsyms",
- symbol_conf.guestmount,
- namelist[i]->d_name);
- ret = access(path, R_OK);
- if (ret) {
+ if (snprintf(path, sizeof(path), "%s/%s/proc/kallsyms",
+ symbol_conf.guestmount,
+ namelist[i]->d_name) >= (int)sizeof(path)) {
+ pr_debug("Guest kallsyms path too long for %s. Skipping.\n",
+ namelist[i]->d_name);
+ free(namelist[i]);
+ continue;
+ }
+ if (access(path, R_OK)) {
pr_debug("Can't access file %s\n", path);
- goto failure;
+ free(namelist[i]);
+ continue;
}
machines__create_kernel_maps(machines, pid);
+ free(namelist[i]);
}
-failure:
free(namelist);
}
@@ -1843,7 +1857,8 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event
* (fork) event that would have removed the thread was lost. Assume the
* latter case and continue on as best we can.
*/
- if (thread__pid(parent) != (pid_t)event->fork.ppid) {
+ if (parent != NULL &&
+ thread__pid(parent) != (pid_t)event->fork.ppid) {
dump_printf("removing erroneous parent thread %d/%d\n",
thread__pid(parent), thread__tid(parent));
machine__remove_thread(machine, parent);
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 4dff3e925a47..6d6bfcebb97a 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -1814,8 +1814,10 @@ int metricgroup__copy_metric_events(struct evlist *evlist, struct cgroup *cgrp,
new_expr->metric_expr = old_expr->metric_expr;
new_expr->metric_threshold = old_expr->metric_threshold;
new_expr->metric_name = strdup(old_expr->metric_name);
- if (!new_expr->metric_name)
+ if (!new_expr->metric_name) {
+ free(new_expr);
return -ENOMEM;
+ }
new_expr->metric_unit = old_expr->metric_unit;
new_expr->runtime = old_expr->runtime;
@@ -1827,6 +1829,7 @@ int metricgroup__copy_metric_events(struct evlist *evlist, struct cgroup *cgrp,
alloc_size = sizeof(*new_expr->metric_refs);
new_expr->metric_refs = calloc(nr + 1, alloc_size);
if (!new_expr->metric_refs) {
+ zfree(&new_expr->metric_name);
free(new_expr);
return -ENOMEM;
}
@@ -1843,6 +1846,7 @@ int metricgroup__copy_metric_events(struct evlist *evlist, struct cgroup *cgrp,
alloc_size = sizeof(*new_expr->metric_events);
new_expr->metric_events = calloc(nr + 1, alloc_size);
if (!new_expr->metric_events) {
+ zfree(&new_expr->metric_name);
zfree(&new_expr->metric_refs);
free(new_expr);
return -ENOMEM;
@@ -1853,6 +1857,7 @@ int metricgroup__copy_metric_events(struct evlist *evlist, struct cgroup *cgrp,
evsel = old_expr->metric_events[idx];
evsel = evlist__find_evsel(evlist, evsel->core.idx);
if (evsel == NULL) {
+ zfree(&new_expr->metric_name);
zfree(&new_expr->metric_events);
zfree(&new_expr->metric_refs);
free(new_expr);
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index e7f36ea9e2fa..6614c2303bb9 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -26,40 +26,14 @@
#include <internal/lib.h>
#include "../builtin.h"
-#if PY_MAJOR_VERSION < 3
-#define _PyUnicode_FromString(arg) \
- PyString_FromString(arg)
-#define _PyUnicode_AsString(arg) \
- PyString_AsString(arg)
-#define _PyUnicode_FromFormat(...) \
- PyString_FromFormat(__VA_ARGS__)
-#define _PyLong_FromLong(arg) \
- PyInt_FromLong(arg)
-
-#else
-
#define _PyUnicode_FromString(arg) \
PyUnicode_FromString(arg)
#define _PyUnicode_FromFormat(...) \
PyUnicode_FromFormat(__VA_ARGS__)
#define _PyLong_FromLong(arg) \
PyLong_FromLong(arg)
-#endif
-
-#ifndef Py_TYPE
-#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
-#endif
-
-/* Define PyVarObject_HEAD_INIT for python 2.5 */
-#ifndef PyVarObject_HEAD_INIT
-# define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size,
-#endif
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf(void);
-#else
PyMODINIT_FUNC PyInit_perf(void);
-#endif
#define member_def(type, member, ptype, help) \
{ #member, ptype, \
@@ -117,7 +91,7 @@ static PyObject *pyrf_mmap_event__repr(struct pyrf_event *pevent)
pevent->event.mmap.pgoff, pevent->event.mmap.filename) < 0) {
ret = PyErr_NoMemory();
} else {
- ret = _PyUnicode_FromString(s);
+ ret = PyUnicode_FromString(s);
free(s);
}
return ret;
@@ -148,7 +122,7 @@ static PyMemberDef pyrf_task_event__members[] = {
static PyObject *pyrf_task_event__repr(struct pyrf_event *pevent)
{
- return _PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
+ return PyUnicode_FromFormat("{ type: %s, pid: %u, ppid: %u, tid: %u, "
"ptid: %u, time: %" PRI_lu64 "}",
pevent->event.header.type == PERF_RECORD_FORK ? "fork" : "exit",
pevent->event.fork.pid,
@@ -181,7 +155,7 @@ static PyMemberDef pyrf_comm_event__members[] = {
static PyObject *pyrf_comm_event__repr(struct pyrf_event *pevent)
{
- return _PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
+ return PyUnicode_FromFormat("{ type: comm, pid: %u, tid: %u, comm: %s }",
pevent->event.comm.pid,
pevent->event.comm.tid,
pevent->event.comm.comm);
@@ -212,7 +186,7 @@ static PyObject *pyrf_throttle_event__repr(struct pyrf_event *pevent)
{
struct perf_record_throttle *te = (struct perf_record_throttle *)(&pevent->event.header + 1);
- return _PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
+ return PyUnicode_FromFormat("{ type: %sthrottle, time: %" PRI_lu64 ", id: %" PRI_lu64
", stream_id: %" PRI_lu64 " }",
pevent->event.header.type == PERF_RECORD_THROTTLE ? "" : "un",
te->time, te->id, te->stream_id);
@@ -247,7 +221,7 @@ static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
pevent->event.lost.id, pevent->event.lost.lost) < 0) {
ret = PyErr_NoMemory();
} else {
- ret = _PyUnicode_FromString(s);
+ ret = PyUnicode_FromString(s);
free(s);
}
return ret;
@@ -274,7 +248,7 @@ static PyMemberDef pyrf_read_event__members[] = {
static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
{
- return _PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
+ return PyUnicode_FromFormat("{ type: read, pid: %u, tid: %u }",
pevent->event.read.pid,
pevent->event.read.tid);
/*
@@ -309,7 +283,7 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
if (asprintf(&s, "{ type: sample }") < 0) {
ret = PyErr_NoMemory();
} else {
- ret = _PyUnicode_FromString(s);
+ ret = PyUnicode_FromString(s);
free(s);
}
return ret;
@@ -343,7 +317,7 @@ tracepoint_field(struct pyrf_event *pe, struct tep_format_field *field)
}
if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
- ret = _PyUnicode_FromString((char *)data + offset);
+ ret = PyUnicode_FromString((char *)data + offset);
} else {
ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
field->flags &= ~TEP_FIELD_IS_STRING;
@@ -432,7 +406,7 @@ static PyObject *pyrf_context_switch_event__repr(struct pyrf_event *pevent)
!!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT)) < 0) {
ret = PyErr_NoMemory();
} else {
- ret = _PyUnicode_FromString(s);
+ ret = PyUnicode_FromString(s);
free(s);
}
return ret;
@@ -662,6 +636,114 @@ static int pyrf_thread_map__setup_types(void)
return PyType_Ready(&pyrf_thread_map__type);
}
+struct pyrf_counts_values {
+ PyObject_HEAD
+
+ struct perf_counts_values values;
+};
+
+static const char pyrf_counts_values__doc[] = PyDoc_STR("perf counts values object.");
+
+static void pyrf_counts_values__delete(struct pyrf_counts_values *pcounts_values)
+{
+ Py_TYPE(pcounts_values)->tp_free((PyObject *)pcounts_values);
+}
+
+#define counts_values_member_def(member, ptype, help) \
+ { #member, ptype, \
+ offsetof(struct pyrf_counts_values, values.member), \
+ 0, help }
+
+static PyMemberDef pyrf_counts_values_members[] = {
+ counts_values_member_def(val, T_ULONGLONG, "Value of event"),
+ counts_values_member_def(ena, T_ULONGLONG, "Time for which enabled"),
+ counts_values_member_def(run, T_ULONGLONG, "Time for which running"),
+ counts_values_member_def(id, T_ULONGLONG, "Unique ID for an event"),
+ counts_values_member_def(lost, T_ULONGLONG, "Num of lost samples"),
+ { .name = NULL, },
+};
+
+static PyObject *pyrf_counts_values_get_values(struct pyrf_counts_values *self, void *closure)
+{
+ PyObject *vals = PyList_New(5);
+
+ if (!vals)
+ return NULL;
+ for (int i = 0; i < 5; i++) {
+ PyObject *val = PyLong_FromUnsignedLongLong(self->values.values[i]);
+
+ if (!val) {
+ Py_DECREF(vals);
+ return NULL;
+ }
+ PyList_SetItem(vals, i, val);
+ }
+
+ return vals;
+}
+
+static int pyrf_counts_values_set_values(struct pyrf_counts_values *self, PyObject *list,
+ void *closure)
+{
+ Py_ssize_t size;
+ PyObject *item = NULL;
+
+ if (list == NULL) {
+ PyErr_SetString(PyExc_TypeError, "cannot delete attribute");
+ return -1;
+ }
+
+ if (!PyList_Check(list)) {
+ PyErr_SetString(PyExc_TypeError, "Value assigned must be a list");
+ return -1;
+ }
+
+ size = PyList_Size(list);
+ if (size != 5) {
+ PyErr_SetString(PyExc_ValueError, "List must have exactly 5 entries");
+ return -1;
+ }
+
+ for (Py_ssize_t i = 0; i < size; i++) {
+ unsigned long long val;
+
+ item = PyList_GetItem(list, i);
+ if (!PyLong_Check(item)) {
+ PyErr_SetString(PyExc_TypeError, "List members should be numbers");
+ return -1;
+ }
+ val = PyLong_AsUnsignedLongLong(item);
+ if (val == (unsigned long long)-1 && PyErr_Occurred())
+ return -1;
+ self->values.values[i] = val;
+ }
+
+ return 0;
+}
+
+static PyGetSetDef pyrf_counts_values_getset[] = {
+ {"values", (getter)pyrf_counts_values_get_values, (setter)pyrf_counts_values_set_values,
+ "Name field", NULL},
+ { .name = NULL, },
+};
+
+static PyTypeObject pyrf_counts_values__type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ .tp_name = "perf.counts_values",
+ .tp_basicsize = sizeof(struct pyrf_counts_values),
+ .tp_dealloc = (destructor)pyrf_counts_values__delete,
+ .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+ .tp_doc = pyrf_counts_values__doc,
+ .tp_members = pyrf_counts_values_members,
+ .tp_getset = pyrf_counts_values_getset,
+};
+
+static int pyrf_counts_values__setup_types(void)
+{
+ pyrf_counts_values__type.tp_new = PyType_GenericNew;
+ return PyType_Ready(&pyrf_counts_values__type);
+}
+
struct pyrf_evsel {
PyObject_HEAD
@@ -923,17 +1005,8 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,
for (i = 0; i < evlist->core.pollfd.nr; ++i) {
PyObject *file;
-#if PY_MAJOR_VERSION < 3
- FILE *fp = fdopen(evlist->core.pollfd.entries[i].fd, "r");
-
- if (fp == NULL)
- goto free_list;
-
- file = PyFile_FromFile(fp, "perf", "r", NULL);
-#else
file = PyFile_FromFd(evlist->core.pollfd.entries[i].fd, "perf", "r", -1,
NULL, NULL, NULL, 0);
-#endif
if (file == NULL)
goto free_list;
@@ -1241,9 +1314,9 @@ static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
tp_format = trace_event__tp_format(sys, name);
if (IS_ERR(tp_format))
- return _PyLong_FromLong(-1);
+ return PyLong_FromLong(-1);
- return _PyLong_FromLong(tp_format->id);
+ return PyLong_FromLong(tp_format->id);
#endif // HAVE_LIBTRACEEVENT
}
@@ -1257,18 +1330,11 @@ static PyMethodDef perf__methods[] = {
{ .ml_name = NULL, }
};
-#if PY_MAJOR_VERSION < 3
-PyMODINIT_FUNC initperf(void)
-#else
PyMODINIT_FUNC PyInit_perf(void)
-#endif
{
PyObject *obj;
int i;
PyObject *dict;
-#if PY_MAJOR_VERSION < 3
- PyObject *module = Py_InitModule("perf", perf__methods);
-#else
static struct PyModuleDef moduledef = {
PyModuleDef_HEAD_INIT,
"perf", /* m_name */
@@ -1281,19 +1347,15 @@ PyMODINIT_FUNC PyInit_perf(void)
NULL, /* m_free */
};
PyObject *module = PyModule_Create(&moduledef);
-#endif
if (module == NULL ||
pyrf_event__setup_types() < 0 ||
pyrf_evlist__setup_types() < 0 ||
pyrf_evsel__setup_types() < 0 ||
pyrf_thread_map__setup_types() < 0 ||
- pyrf_cpu_map__setup_types() < 0)
-#if PY_MAJOR_VERSION < 3
- return;
-#else
+ pyrf_cpu_map__setup_types() < 0 ||
+ pyrf_counts_values__setup_types() < 0)
return module;
-#endif
/* The page_size is placed in util object. */
page_size = sysconf(_SC_PAGE_SIZE);
@@ -1337,12 +1399,15 @@ PyMODINIT_FUNC PyInit_perf(void)
Py_INCREF(&pyrf_cpu_map__type);
PyModule_AddObject(module, "cpu_map", (PyObject*)&pyrf_cpu_map__type);
+ Py_INCREF(&pyrf_counts_values__type);
+ PyModule_AddObject(module, "counts_values", (PyObject *)&pyrf_counts_values__type);
+
dict = PyModule_GetDict(module);
if (dict == NULL)
goto error;
for (i = 0; perf__constants[i].name != NULL; i++) {
- obj = _PyLong_FromLong(perf__constants[i].value);
+ obj = PyLong_FromLong(perf__constants[i].value);
if (obj == NULL)
goto error;
PyDict_SetItemString(dict, perf__constants[i].name, obj);
@@ -1352,9 +1417,7 @@ PyMODINIT_FUNC PyInit_perf(void)
error:
if (PyErr_Occurred())
PyErr_SetString(PyExc_ImportError, "perf: Init failed!");
-#if PY_MAJOR_VERSION >= 3
return module;
-#endif
}
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index d7183134b669..f1d461d47d73 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -58,22 +58,6 @@
#include "mem-events.h"
#include "util/perf_regs.h"
-#if PY_MAJOR_VERSION < 3
-#define _PyUnicode_FromString(arg) \
- PyString_FromString(arg)
-#define _PyUnicode_FromStringAndSize(arg1, arg2) \
- PyString_FromStringAndSize((arg1), (arg2))
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
- PyString_FromStringAndSize((arg1), (arg2))
-#define _PyLong_FromLong(arg) \
- PyInt_FromLong(arg)
-#define _PyLong_AsLong(arg) \
- PyInt_AsLong(arg)
-#define _PyCapsule_New(arg1, arg2, arg3) \
- PyCObject_FromVoidPtr((arg1), (arg2))
-
-PyMODINIT_FUNC initperf_trace_context(void);
-#else
#define _PyUnicode_FromString(arg) \
PyUnicode_FromString(arg)
#define _PyUnicode_FromStringAndSize(arg1, arg2) \
@@ -88,7 +72,6 @@ PyMODINIT_FUNC initperf_trace_context(void);
PyCapsule_New((arg1), (arg2), (arg3))
PyMODINIT_FUNC PyInit_perf_trace_context(void);
-#endif
#ifdef HAVE_LIBTRACEEVENT
#define TRACE_EVENT_TYPE_MAX \
@@ -181,17 +164,7 @@ static int get_argument_count(PyObject *handler)
{
int arg_count = 0;
- /*
- * The attribute for the code object is func_code in Python 2,
- * whereas it is __code__ in Python 3.0+.
- */
- PyObject *code_obj = PyObject_GetAttrString(handler,
- "func_code");
- if (PyErr_Occurred()) {
- PyErr_Clear();
- code_obj = PyObject_GetAttrString(handler,
- "__code__");
- }
+ PyObject *code_obj = code_obj = PyObject_GetAttrString(handler, "__code__");
PyErr_Clear();
if (code_obj) {
PyObject *arg_count_obj = PyObject_GetAttrString(code_obj,
@@ -1899,12 +1872,6 @@ static void set_table_handlers(struct tables *tables)
tables->synth_handler = get_handler("synth_data");
}
-#if PY_MAJOR_VERSION < 3
-static void _free_command_line(const char **command_line, int num)
-{
- free(command_line);
-}
-#else
static void _free_command_line(wchar_t **command_line, int num)
{
int i;
@@ -1912,7 +1879,6 @@ static void _free_command_line(wchar_t **command_line, int num)
PyMem_RawFree(command_line[i]);
free(command_line);
}
-#endif
/*
@@ -1922,30 +1888,12 @@ static int python_start_script(const char *script, int argc, const char **argv,
struct perf_session *session)
{
struct tables *tables = &tables_global;
-#if PY_MAJOR_VERSION < 3
- const char **command_line;
-#else
wchar_t **command_line;
-#endif
- /*
- * Use a non-const name variable to cope with python 2.6's
- * PyImport_AppendInittab prototype
- */
- char buf[PATH_MAX], name[19] = "perf_trace_context";
+ char buf[PATH_MAX];
int i, err = 0;
FILE *fp;
scripting_context->session = session;
-#if PY_MAJOR_VERSION < 3
- command_line = malloc((argc + 1) * sizeof(const char *));
- if (!command_line)
- return -1;
-
- command_line[0] = script;
- for (i = 1; i < argc + 1; i++)
- command_line[i] = argv[i - 1];
- PyImport_AppendInittab(name, initperf_trace_context);
-#else
command_line = malloc((argc + 1) * sizeof(wchar_t *));
if (!command_line)
return -1;
@@ -1953,15 +1901,10 @@ static int python_start_script(const char *script, int argc, const char **argv,
command_line[0] = Py_DecodeLocale(script, NULL);
for (i = 1; i < argc + 1; i++)
command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
- PyImport_AppendInittab(name, PyInit_perf_trace_context);
-#endif
+ PyImport_AppendInittab("perf_trace_context", PyInit_perf_trace_context);
Py_Initialize();
-#if PY_MAJOR_VERSION < 3
- PySys_SetArgv(argc + 1, (char **)command_line);
-#else
PySys_SetArgv(argc + 1, command_line);
-#endif
fp = fopen(script, "r");
if (!fp) {
diff --git a/tools/perf/util/synthetic-events.c b/tools/perf/util/synthetic-events.c
index a58444c4aed1..660689968b79 100644
--- a/tools/perf/util/synthetic-events.c
+++ b/tools/perf/util/synthetic-events.c
@@ -1045,7 +1045,7 @@ int perf_event__synthesize_threads(const struct perf_tool *tool,
else
thread_nr = nr_threads_synthesize;
- if (thread_nr <= 1) {
+ if (thread_nr <= 1 || n <= 1) {
err = __perf_event__synthesize_threads(tool, process,
machine,
needs_mmap, mmap_data,
@@ -1091,8 +1091,8 @@ int perf_event__synthesize_threads(const struct perf_tool *tool,
}
err = 0;
out_join:
- for (i = 0; i < thread_nr; i++)
- pthread_join(synthesize_threads[i], NULL);
+ for (j = 0; j < i; j++)
+ pthread_join(synthesize_threads[j], NULL);
free(args);
free_threads:
free(synthesize_threads);
diff --git a/tools/perf/util/thread-stack.c b/tools/perf/util/thread-stack.c
index c6a0a27b12c2..47d5922efdee 100644
--- a/tools/perf/util/thread-stack.c
+++ b/tools/perf/util/thread-stack.c
@@ -642,7 +642,7 @@ void thread_stack__br_sample(struct thread *thread, int cpu,
sz -= nr;
be = &dst->entries[nr];
nr = min(ts->br_stack_pos, sz);
- memcpy(be, &src->entries[0], bsz * ts->br_stack_pos);
+ memcpy(be, &src->entries[0], bsz * nr);
}
}
diff --git a/tools/perf/util/trace-event-read.c b/tools/perf/util/trace-event-read.c
index 1162c49b8082..b0d3957c48e6 100644
--- a/tools/perf/util/trace-event-read.c
+++ b/tools/perf/util/trace-event-read.c
@@ -25,18 +25,18 @@ static int input_fd;
static ssize_t trace_data_size;
static bool repipe;
-static int __do_read(int fd, void *buf, int size)
+static ssize_t __do_read(int fd, void *buf, size_t size)
{
- int rsize = size;
+ size_t rsize = size;
while (size) {
- int ret = read(fd, buf, size);
+ ssize_t ret = read(fd, buf, size);
if (ret <= 0)
return -1;
if (repipe) {
- int retw = write(STDOUT_FILENO, buf, ret);
+ ssize_t retw = write(STDOUT_FILENO, buf, ret);
if (retw <= 0 || retw != ret) {
pr_debug("repiping input file");
@@ -51,13 +51,13 @@ static int __do_read(int fd, void *buf, int size)
return rsize;
}
-static int do_read(void *data, int size)
+static ssize_t do_read(void *data, size_t size)
{
- int r;
+ ssize_t r;
r = __do_read(input_fd, data, size);
if (r <= 0) {
- pr_debug("reading input file (size expected=%d received=%d)",
+ pr_debug("reading input file (size expected=%zu received=%zd)",
size, r);
return -1;
}
@@ -68,10 +68,10 @@ static int do_read(void *data, int size)
}
/* If it fails, the next read will report it */
-static void skip(int size)
+static void skip(size_t size)
{
char buf[BUFSIZ];
- int r;
+ size_t r;
while (size) {
r = size > BUFSIZ ? BUFSIZ : size;
@@ -127,6 +127,11 @@ static char *read_string(void)
}
}
+ if (size >= (int)sizeof(buf) - 1) {
+ pr_debug("string too long (max %zu bytes)", sizeof(buf) - 1);
+ goto out;
+ }
+
buf[size++] = c;
if (!c)
@@ -197,7 +202,7 @@ static int read_header_files(struct tep_handle *pevent)
unsigned long long size;
char *header_page;
char buf[BUFSIZ];
- int ret = 0;
+ ssize_t ret = 0;
if (do_read(buf, 12) < 0)
return -1;
@@ -245,7 +250,7 @@ static int read_header_files(struct tep_handle *pevent)
static int read_ftrace_file(struct tep_handle *pevent, unsigned long long size)
{
- int ret;
+ ssize_t ret;
char *buf;
buf = malloc(size);
@@ -271,7 +276,7 @@ static int read_ftrace_file(struct tep_handle *pevent, unsigned long long size)
static int read_event_file(struct tep_handle *pevent, char *sys,
unsigned long long size)
{
- int ret;
+ ssize_t ret;
char *buf;
buf = malloc(size);
@@ -317,7 +322,7 @@ static int read_event_files(struct tep_handle *pevent)
int systems;
int count;
int i,x;
- int ret;
+ ssize_t ret;
systems = read4(pevent);
@@ -345,7 +350,7 @@ static int read_saved_cmdline(struct tep_handle *pevent)
{
unsigned long long size;
char *buf;
- int ret;
+ ssize_t ret;
/* it can have 0 size */
size = read8(pevent);
diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.bpf.c
index b722baf6da4b..05e3b4495ac5 100644
--- a/tools/sched_ext/scx_flatcg.bpf.c
+++ b/tools/sched_ext/scx_flatcg.bpf.c
@@ -261,7 +261,7 @@ static void cgrp_cap_budget(struct cgv_node *cgv_node, struct fcg_cgrp_ctx *cgc)
* and thus can't be updated and repositioned. Instead, we collect the
* vtime deltas separately and apply it asynchronously here.
*/
- delta = __sync_fetch_and_sub(&cgc->cvtime_delta, cgc->cvtime_delta);
+ delta = __sync_fetch_and_and(&cgc->cvtime_delta, 0);
cvtime = cgv_node->cvtime + delta;
/*
@@ -571,7 +571,8 @@ void BPF_STRUCT_OPS(fcg_stopping, struct task_struct *p, bool runnable)
cgc = find_cgrp_ctx(cgrp);
if (cgc) {
__sync_fetch_and_add(&cgc->cvtime_delta,
- p->se.sum_exec_runtime - taskc->bypassed_at);
+ (p->se.sum_exec_runtime - taskc->bypassed_at) *
+ FCG_HWEIGHT_ONE / (cgc->hweight ?: 1));
taskc->bypassed_at = 0;
}
bpf_cgroup_release(cgrp);
@@ -769,10 +770,18 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_struct *prev)
* cgroup to execute but the latter needs to be done in a loop
* and we can't keep the lock held. Oh well...
*/
+ s64 delta = now - cpuc->cur_at - cgrp_slice_ns;
+
bpf_spin_lock(&cgv_tree_lock);
- __sync_fetch_and_add(&cgc->cvtime_delta,
- (cpuc->cur_at + cgrp_slice_ns - now) *
- FCG_HWEIGHT_ONE / (cgc->hweight ?: 1));
+ /* keep the dividends positive, BPF division is unsigned */
+ if (delta >= 0)
+ __sync_fetch_and_add(&cgc->cvtime_delta,
+ (u64)delta * FCG_HWEIGHT_ONE /
+ (cgc->hweight ?: 1));
+ else
+ __sync_fetch_and_sub(&cgc->cvtime_delta,
+ (u64)-delta * FCG_HWEIGHT_ONE /
+ (cgc->hweight ?: 1));
bpf_spin_unlock(&cgv_tree_lock);
} else {
stat_inc(FCG_STAT_CNS_GONE);
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index 5f2afd95de43..076d62bbac13 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -59,10 +59,18 @@ LLVM_SUFFIX := $(LLVM)
endif
$(call allow-override,CC,$(LLVM_PREFIX)clang$(LLVM_SUFFIX))
+$(call allow-override,CLANG,$(LLVM_PREFIX)clang$(LLVM_SUFFIX))
+$(call allow-override,HOSTCC,$(LLVM_PREFIX)clang$(LLVM_SUFFIX))
$(call allow-override,AR,$(LLVM_PREFIX)llvm-ar$(LLVM_SUFFIX))
+$(call allow-override,HOSTAR,$(LLVM_PREFIX)llvm-ar$(LLVM_SUFFIX))
$(call allow-override,LD,$(LLVM_PREFIX)ld.lld$(LLVM_SUFFIX))
+$(call allow-override,HOSTLD,$(LLVM_PREFIX)ld.lld$(LLVM_SUFFIX))
$(call allow-override,CXX,$(LLVM_PREFIX)clang++$(LLVM_SUFFIX))
$(call allow-override,STRIP,$(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX))
+$(call allow-override,LLVM_STRIP,$(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX))
+$(call allow-override,LLC,$(LLVM_PREFIX)llc$(LLVM_SUFFIX))
+$(call allow-override,LLVM_CONFIG,$(LLVM_PREFIX)llvm-config$(LLVM_SUFFIX))
+$(call allow-override,LLVM_OBJCOPY,$(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX))
else
# Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix.
$(call allow-override,CC,$(CROSS_COMPILE)gcc)
@@ -70,26 +78,24 @@ $(call allow-override,AR,$(CROSS_COMPILE)ar)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
$(call allow-override,CXX,$(CROSS_COMPILE)g++)
$(call allow-override,STRIP,$(CROSS_COMPILE)strip)
+
+# Host versions aren't prefixed
+$(call allow-override,HOSTAR,ar)
+$(call allow-override,HOSTCC,gcc)
+$(call allow-override,HOSTLD,ld)
+
+# Some tools still require Clang, LLC and/or LLVM utils
+$(call allow-override,CLANG,clang)
+$(call allow-override,LLC,llc)
+$(call allow-override,LLVM_CONFIG,llvm-config)
+$(call allow-override,LLVM_OBJCOPY,llvm-objcopy)
+$(call allow-override,LLVM_STRIP,llvm-strip)
endif
CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?)
-ifneq ($(LLVM),)
-HOSTAR ?= $(LLVM_PREFIX)llvm-ar$(LLVM_SUFFIX)
-HOSTCC ?= $(LLVM_PREFIX)clang$(LLVM_SUFFIX)
-HOSTLD ?= $(LLVM_PREFIX)ld.lld$(LLVM_SUFFIX)
-else
-HOSTAR ?= ar
-HOSTCC ?= gcc
-HOSTLD ?= ld
-endif
-
-# Some tools require Clang, LLC and/or LLVM utils
-CLANG ?= clang
-LLC ?= llc
-LLVM_CONFIG ?= llvm-config
-LLVM_OBJCOPY ?= llvm-objcopy
-LLVM_STRIP ?= llvm-strip
+# Some tools require bpftool
+SYSTEM_BPFTOOL ?= bpftool
ifeq ($(CC_NO_CLANG), 1)
EXTRA_WARNINGS += -Wstrict-aliasing=3
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index bc74088c458a..ebe97ba520ce 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -121,7 +121,7 @@ def _list_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest)
lines.pop()
# Filter out any extraneous non-test output that might have gotten mixed in.
- return [l for l in output if re.match(r'^[^\s.]+\.[^\s.]+$', l)]
+ return [l for l in lines if re.match(r'^[^\s.]+\.[^\s.]+$', l)]
def _list_tests_attr(linux: kunit_kernel.LinuxSourceTree, request: KunitExecRequest) -> Iterable[str]:
args = ['kunit.action=list_attr']
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 6e6200d52540..79e05c333178 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -884,6 +884,18 @@ class KUnitMainTest(unittest.TestCase):
self.linux_source_mock.run_kernel.assert_called_once_with(
args=['kunit.action=list'], build_dir='.kunit', filter_glob='suite*', filter='', filter_action=None, timeout=300)
+ def test_list_tests_with_prefix(self):
+ want = ['suite.test1', 'suite.test2', 'suite2.test1']
+ self.linux_source_mock.run_kernel.return_value = [
+ '[ 0.100000] TAP version 14',
+ '[ 0.200000] suite.test1',
+ '[ 0.200000] suite.test2',
+ '[ 0.300000] suite2.test1']
+
+ got = kunit._list_tests(self.linux_source_mock,
+ kunit.KunitExecRequest(None, None, None, False, False, '.kunit', 300, 'suite*', '', None, None, 'suite', False, False, False))
+ self.assertEqual(got, want)
+
@mock.patch.object(kunit, '_list_tests')
def test_run_isolated_by_suite(self, mock_tests):
mock_tests.return_value = ['suite.test1', 'suite.test2', 'suite2.test1']
diff --git a/tools/testing/selftests/arm64/mte/check_buffer_fill.c b/tools/testing/selftests/arm64/mte/check_buffer_fill.c
index 1dbbbd47dd50..639ea469969a 100644
--- a/tools/testing/selftests/arm64/mte/check_buffer_fill.c
+++ b/tools/testing/selftests/arm64/mte/check_buffer_fill.c
@@ -406,6 +406,8 @@ int main(int argc, char *argv[])
size_t page_size = getpagesize();
int item = ARRAY_SIZE(sizes);
+ ksft_print_header();
+
sizes[item - 3] = page_size - 1;
sizes[item - 2] = page_size;
sizes[item - 1] = page_size + 1;
diff --git a/tools/testing/selftests/arm64/mte/check_child_memory.c b/tools/testing/selftests/arm64/mte/check_child_memory.c
index 7597fc632cad..7017a4851ff4 100644
--- a/tools/testing/selftests/arm64/mte/check_child_memory.c
+++ b/tools/testing/selftests/arm64/mte/check_child_memory.c
@@ -146,6 +146,8 @@ int main(int argc, char *argv[])
int err;
int item = ARRAY_SIZE(sizes);
+ ksft_print_header();
+
page_size = getpagesize();
if (!page_size) {
ksft_print_msg("ERR: Unable to get page size\n");
diff --git a/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c b/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
index 325bca0de0f6..d23f154d3288 100644
--- a/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
+++ b/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c
@@ -131,6 +131,7 @@ int main(int argc, char *argv[])
if (err)
return err;
+ ksft_print_header();
ksft_set_plan(1);
evaluate_test(mte_gcr_fork_test(),
diff --git a/tools/testing/selftests/arm64/mte/check_hugetlb_options.c b/tools/testing/selftests/arm64/mte/check_hugetlb_options.c
new file mode 100644
index 000000000000..cd7667187ed5
--- /dev/null
+++ b/tools/testing/selftests/arm64/mte/check_hugetlb_options.c
@@ -0,0 +1,298 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2024 Ampere Computing LLC
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <ucontext.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "kselftest.h"
+#include "mte_common_util.h"
+#include "mte_def.h"
+
+#define TAG_CHECK_ON 0
+#define TAG_CHECK_OFF 1
+
+static unsigned long default_huge_page_size(void)
+{
+ unsigned long hps = 0;
+ char *line = NULL;
+ size_t linelen = 0;
+ FILE *f = fopen("/proc/meminfo", "r");
+
+ if (!f)
+ return 0;
+ while (getline(&line, &linelen, f) > 0) {
+ if (sscanf(line, "Hugepagesize: %lu kB", &hps) == 1) {
+ hps <<= 10;
+ break;
+ }
+ }
+
+ free(line);
+ fclose(f);
+ return hps;
+}
+
+static bool is_hugetlb_allocated(void)
+{
+ unsigned long hps = 0;
+ char *line = NULL;
+ size_t linelen = 0;
+ FILE *f = fopen("/proc/meminfo", "r");
+
+ if (!f)
+ return false;
+ while (getline(&line, &linelen, f) > 0) {
+ if (sscanf(line, "Hugetlb: %lu kB", &hps) == 1) {
+ hps <<= 10;
+ break;
+ }
+ }
+
+ free(line);
+ fclose(f);
+
+ if (hps > 0)
+ return true;
+
+ return false;
+}
+
+static void write_sysfs(char *str, unsigned long val)
+{
+ FILE *f;
+
+ f = fopen(str, "w");
+ if (!f) {
+ ksft_print_msg("ERR: missing %s\n", str);
+ return;
+ }
+ fprintf(f, "%lu", val);
+ fclose(f);
+}
+
+static void allocate_hugetlb()
+{
+ write_sysfs("/proc/sys/vm/nr_hugepages", 2);
+}
+
+static void free_hugetlb()
+{
+ write_sysfs("/proc/sys/vm/nr_hugepages", 0);
+}
+
+static int check_child_tag_inheritance(char *ptr, int size, int mode)
+{
+ int i, parent_tag, child_tag, fault, child_status;
+ pid_t child;
+
+ parent_tag = MT_FETCH_TAG((uintptr_t)ptr);
+ fault = 0;
+
+ child = fork();
+ if (child == -1) {
+ ksft_print_msg("FAIL: child process creation\n");
+ return KSFT_FAIL;
+ } else if (child == 0) {
+ mte_initialize_current_context(mode, (uintptr_t)ptr, size);
+ /* Do copy on write */
+ memset(ptr, '1', size);
+ mte_wait_after_trig();
+ if (cur_mte_cxt.fault_valid == true) {
+ fault = 1;
+ goto check_child_tag_inheritance_err;
+ }
+ for (i = 0; i < size; i += MT_GRANULE_SIZE) {
+ child_tag = MT_FETCH_TAG((uintptr_t)(mte_get_tag_address(ptr + i)));
+ if (parent_tag != child_tag) {
+ ksft_print_msg("FAIL: child mte tag (%d) mismatch\n", i);
+ fault = 1;
+ goto check_child_tag_inheritance_err;
+ }
+ }
+check_child_tag_inheritance_err:
+ _exit(fault);
+ }
+ /* Wait for child process to terminate */
+ wait(&child_status);
+ if (WIFEXITED(child_status))
+ fault = WEXITSTATUS(child_status);
+ else
+ fault = 1;
+ return (fault) ? KSFT_FAIL : KSFT_PASS;
+}
+
+static int check_mte_memory(char *ptr, int size, int mode, int tag_check)
+{
+ mte_initialize_current_context(mode, (uintptr_t)ptr, size);
+ memset(ptr, '1', size);
+ mte_wait_after_trig();
+ if (cur_mte_cxt.fault_valid == true)
+ return KSFT_FAIL;
+
+ return KSFT_PASS;
+}
+
+static int check_hugetlb_memory_mapping(int mem_type, int mode, int mapping, int tag_check)
+{
+ char *ptr, *map_ptr;
+ int result;
+ unsigned long map_size;
+
+ map_size = default_huge_page_size();
+
+ mte_switch_mode(mode, MTE_ALLOW_NON_ZERO_TAG);
+ map_ptr = (char *)mte_allocate_memory(map_size, mem_type, mapping, false);
+ if (check_allocated_memory(map_ptr, map_size, mem_type, false) != KSFT_PASS)
+ return KSFT_FAIL;
+
+ mte_initialize_current_context(mode, (uintptr_t)map_ptr, map_size);
+ /* Only mte enabled memory will allow tag insertion */
+ ptr = mte_insert_tags((void *)map_ptr, map_size);
+ if (!ptr || cur_mte_cxt.fault_valid == true) {
+ ksft_print_msg("FAIL: Insert tags on anonymous mmap memory\n");
+ munmap((void *)map_ptr, map_size);
+ return KSFT_FAIL;
+ }
+ result = check_mte_memory(ptr, map_size, mode, tag_check);
+ mte_clear_tags((void *)ptr, map_size);
+ mte_free_memory((void *)map_ptr, map_size, mem_type, false);
+ if (result == KSFT_FAIL)
+ return KSFT_FAIL;
+
+ return KSFT_PASS;
+}
+
+static int check_clear_prot_mte_flag(int mem_type, int mode, int mapping)
+{
+ char *map_ptr;
+ int prot_flag, result;
+ unsigned long map_size;
+
+ prot_flag = PROT_READ | PROT_WRITE;
+ mte_switch_mode(mode, MTE_ALLOW_NON_ZERO_TAG);
+ map_size = default_huge_page_size();
+ map_ptr = (char *)mte_allocate_memory_tag_range(map_size, mem_type, mapping,
+ 0, 0);
+ if (check_allocated_memory_range(map_ptr, map_size, mem_type,
+ 0, 0) != KSFT_PASS)
+ return KSFT_FAIL;
+ /* Try to clear PROT_MTE property and verify it by tag checking */
+ if (mprotect(map_ptr, map_size, prot_flag)) {
+ mte_free_memory_tag_range((void *)map_ptr, map_size, mem_type,
+ 0, 0);
+ ksft_print_msg("FAIL: mprotect not ignoring clear PROT_MTE property\n");
+ return KSFT_FAIL;
+ }
+ result = check_mte_memory(map_ptr, map_size, mode, TAG_CHECK_ON);
+ mte_free_memory_tag_range((void *)map_ptr, map_size, mem_type, 0, 0);
+ if (result != KSFT_PASS)
+ return KSFT_FAIL;
+
+ return KSFT_PASS;
+}
+
+static int check_child_hugetlb_memory_mapping(int mem_type, int mode, int mapping)
+{
+ char *ptr;
+ int result;
+ unsigned long map_size;
+
+ map_size = default_huge_page_size();
+
+ mte_switch_mode(mode, MTE_ALLOW_NON_ZERO_TAG);
+ ptr = (char *)mte_allocate_memory_tag_range(map_size, mem_type, mapping,
+ 0, 0);
+ if (check_allocated_memory_range(ptr, map_size, mem_type,
+ 0, 0) != KSFT_PASS)
+ return KSFT_FAIL;
+ result = check_child_tag_inheritance(ptr, map_size, mode);
+ mte_free_memory_tag_range((void *)ptr, map_size, mem_type, 0, 0);
+ if (result == KSFT_FAIL)
+ return result;
+
+ return KSFT_PASS;
+}
+
+int main(int argc, char *argv[])
+{
+ int err;
+ void *map_ptr;
+ unsigned long map_size;
+
+ ksft_print_header();
+
+ err = mte_default_setup();
+ if (err)
+ return err;
+
+ /* Register signal handlers */
+ mte_register_signal(SIGBUS, mte_default_handler);
+ mte_register_signal(SIGSEGV, mte_default_handler);
+
+ allocate_hugetlb();
+
+ if (!is_hugetlb_allocated()) {
+ ksft_print_msg("ERR: Unable allocate hugetlb pages\n");
+ return KSFT_FAIL;
+ }
+
+ /* Check if MTE supports hugetlb mappings */
+ map_size = default_huge_page_size();
+ map_ptr = mmap(NULL, map_size, PROT_READ | PROT_MTE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
+ if (map_ptr == MAP_FAILED)
+ ksft_exit_skip("PROT_MTE not supported with MAP_HUGETLB mappings\n");
+ else
+ munmap(map_ptr, map_size);
+
+ /* Set test plan */
+ ksft_set_plan(12);
+
+ mte_enable_pstate_tco();
+
+ evaluate_test(check_hugetlb_memory_mapping(USE_MMAP, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_OFF),
+ "Check hugetlb memory with private mapping, sync error mode, mmap memory and tag check off\n");
+
+ mte_disable_pstate_tco();
+ evaluate_test(check_hugetlb_memory_mapping(USE_MMAP, MTE_NONE_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_OFF),
+ "Check hugetlb memory with private mapping, no error mode, mmap memory and tag check off\n");
+
+ evaluate_test(check_hugetlb_memory_mapping(USE_MMAP, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_ON),
+ "Check hugetlb memory with private mapping, sync error mode, mmap memory and tag check on\n");
+ evaluate_test(check_hugetlb_memory_mapping(USE_MPROTECT, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_ON),
+ "Check hugetlb memory with private mapping, sync error mode, mmap/mprotect memory and tag check on\n");
+ evaluate_test(check_hugetlb_memory_mapping(USE_MMAP, MTE_ASYNC_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_ON),
+ "Check hugetlb memory with private mapping, async error mode, mmap memory and tag check on\n");
+ evaluate_test(check_hugetlb_memory_mapping(USE_MPROTECT, MTE_ASYNC_ERR, MAP_PRIVATE | MAP_HUGETLB, TAG_CHECK_ON),
+ "Check hugetlb memory with private mapping, async error mode, mmap/mprotect memory and tag check on\n");
+
+ evaluate_test(check_clear_prot_mte_flag(USE_MMAP, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check clear PROT_MTE flags with private mapping, sync error mode and mmap memory\n");
+ evaluate_test(check_clear_prot_mte_flag(USE_MPROTECT, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check clear PROT_MTE flags with private mapping and sync error mode and mmap/mprotect memory\n");
+
+ evaluate_test(check_child_hugetlb_memory_mapping(USE_MMAP, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check child hugetlb memory with private mapping, sync error mode and mmap memory\n");
+ evaluate_test(check_child_hugetlb_memory_mapping(USE_MMAP, MTE_ASYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check child hugetlb memory with private mapping, async error mode and mmap memory\n");
+ evaluate_test(check_child_hugetlb_memory_mapping(USE_MPROTECT, MTE_SYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check child hugetlb memory with private mapping, sync error mode and mmap/mprotect memory\n");
+ evaluate_test(check_child_hugetlb_memory_mapping(USE_MPROTECT, MTE_ASYNC_ERR, MAP_PRIVATE | MAP_HUGETLB),
+ "Check child hugetlb memory with private mapping, async error mode and mmap/mprotect memory\n");
+
+ mte_restore_setup();
+ free_hugetlb();
+ ksft_print_cnts();
+ return ksft_get_fail_cnt() == 0 ? KSFT_PASS : KSFT_FAIL;
+}
diff --git a/tools/testing/selftests/arm64/mte/check_ksm_options.c b/tools/testing/selftests/arm64/mte/check_ksm_options.c
index 88c74bc46d4f..a947e6413e56 100644
--- a/tools/testing/selftests/arm64/mte/check_ksm_options.c
+++ b/tools/testing/selftests/arm64/mte/check_ksm_options.c
@@ -6,6 +6,7 @@
#include <errno.h>
#include <fcntl.h>
#include <signal.h>
+#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -22,6 +23,20 @@
static size_t page_sz;
static unsigned long ksm_sysfs[5];
+static bool has_merge_across_nodes;
+
+static bool merge_across_nodes_available(void)
+{
+ const char *path = PATH_KSM "merge_across_nodes";
+
+ if (!access(path, R_OK | W_OK))
+ return true;
+ if (errno == ENOENT)
+ return false;
+
+ ksft_exit_skip("Unable to read and write %s: %s\n", path,
+ strerror(errno));
+}
static unsigned long read_sysfs(char *str)
{
@@ -56,8 +71,10 @@ static void write_sysfs(char *str, unsigned long val)
static void mte_ksm_setup(void)
{
- ksm_sysfs[0] = read_sysfs(PATH_KSM "merge_across_nodes");
- write_sysfs(PATH_KSM "merge_across_nodes", 1);
+ if (has_merge_across_nodes) {
+ ksm_sysfs[0] = read_sysfs(PATH_KSM "merge_across_nodes");
+ write_sysfs(PATH_KSM "merge_across_nodes", 1);
+ }
ksm_sysfs[1] = read_sysfs(PATH_KSM "sleep_millisecs");
write_sysfs(PATH_KSM "sleep_millisecs", 0);
ksm_sysfs[2] = read_sysfs(PATH_KSM "run");
@@ -70,7 +87,8 @@ static void mte_ksm_setup(void)
static void mte_ksm_restore(void)
{
- write_sysfs(PATH_KSM "merge_across_nodes", ksm_sysfs[0]);
+ if (has_merge_across_nodes)
+ write_sysfs(PATH_KSM "merge_across_nodes", ksm_sysfs[0]);
write_sysfs(PATH_KSM "sleep_millisecs", ksm_sysfs[1]);
write_sysfs(PATH_KSM "run", ksm_sysfs[2]);
write_sysfs(PATH_KSM "max_page_sharing", ksm_sysfs[3]);
@@ -132,9 +150,16 @@ int main(int argc, char *argv[])
{
int err;
+ ksft_print_header();
+
err = mte_default_setup();
if (err)
return err;
+
+ if (geteuid() != 0)
+ ksft_exit_skip("Please run the test as root\n");
+
+ has_merge_across_nodes = merge_across_nodes_available();
page_sz = getpagesize();
if (!page_sz) {
ksft_print_msg("ERR: Unable to get page size\n");
diff --git a/tools/testing/selftests/arm64/mte/check_mmap_options.c b/tools/testing/selftests/arm64/mte/check_mmap_options.c
index 17694caaff53..703ccd4685a8 100644
--- a/tools/testing/selftests/arm64/mte/check_mmap_options.c
+++ b/tools/testing/selftests/arm64/mte/check_mmap_options.c
@@ -188,6 +188,8 @@ int main(int argc, char *argv[])
int err;
int item = ARRAY_SIZE(sizes);
+ ksft_print_header();
+
err = mte_default_setup();
if (err)
return err;
diff --git a/tools/testing/selftests/arm64/mte/check_tags_inclusion.c b/tools/testing/selftests/arm64/mte/check_tags_inclusion.c
index a3d1e23fe02a..c73b4641d01a 100644
--- a/tools/testing/selftests/arm64/mte/check_tags_inclusion.c
+++ b/tools/testing/selftests/arm64/mte/check_tags_inclusion.c
@@ -175,6 +175,8 @@ int main(int argc, char *argv[])
{
int err;
+ ksft_print_header();
+
err = mte_default_setup();
if (err)
return err;
diff --git a/tools/testing/selftests/arm64/mte/check_user_mem.c b/tools/testing/selftests/arm64/mte/check_user_mem.c
index f4ae5f87a3b7..8c790331d67c 100644
--- a/tools/testing/selftests/arm64/mte/check_user_mem.c
+++ b/tools/testing/selftests/arm64/mte/check_user_mem.c
@@ -201,6 +201,8 @@ int main(int argc, char *argv[])
int tag_offsets[] = {page_sz, MT_GRANULE_SIZE};
char test_name[TEST_NAME_MAX];
+ ksft_print_header();
+
page_sz = getpagesize();
if (!page_sz) {
ksft_print_msg("ERR: Unable to get page size\n");
diff --git a/tools/testing/selftests/bpf/README.rst b/tools/testing/selftests/bpf/README.rst
index 776fbe3cb8f9..a36afb460e26 100644
--- a/tools/testing/selftests/bpf/README.rst
+++ b/tools/testing/selftests/bpf/README.rst
@@ -107,12 +107,12 @@ Docker container and local rootfs image. The overall steps are as follows:
tools/testing/selftests/bpf/vmtest.sh \
-l <path of local rootfs image> -- \
./test_progs -d \
- \"$(cat tools/testing/selftests/bpf/DENYLIST.riscv64 \
+ "$(cat tools/testing/selftests/bpf/DENYLIST.riscv64 \
| cut -d'#' -f1 \
| sed -e 's/^[[:space:]]*//' \
-e 's/[[:space:]]*$//' \
| tr -s '\n' ',' \
- )\"
+ )"
Link: https://github.com/pulehui/riscv-bpf-vmtest.git [0]
Link: https://github.com/libbpf/ci/blob/main/rootfs/mkrootfs_debian.sh [1]
diff --git a/tools/testing/selftests/bpf/network_helpers.c b/tools/testing/selftests/bpf/network_helpers.c
index af0ee70a53f9..cd92745c4948 100644
--- a/tools/testing/selftests/bpf/network_helpers.c
+++ b/tools/testing/selftests/bpf/network_helpers.c
@@ -97,7 +97,7 @@ int settimeo(int fd, int timeout_ms)
int start_server_addr(int type, const struct sockaddr_storage *addr, socklen_t addrlen,
const struct network_helper_opts *opts)
{
- int fd;
+ int on = 1, fd;
if (!opts)
opts = &default_opts;
@@ -111,6 +111,12 @@ int start_server_addr(int type, const struct sockaddr_storage *addr, socklen_t a
if (settimeo(fd, opts->timeout_ms))
goto error_close;
+ if ((type & SOCK_TYPE_MASK) == SOCK_STREAM &&
+ setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on))) {
+ log_err("Failed to enable SO_REUSEADDR");
+ goto error_close;
+ }
+
if (opts->post_socket_cb &&
opts->post_socket_cb(fd, opts->cb_opts)) {
log_err("Failed to call post_socket_cb");
@@ -122,7 +128,7 @@ int start_server_addr(int type, const struct sockaddr_storage *addr, socklen_t a
goto error_close;
}
- if (type == SOCK_STREAM) {
+ if ((type & SOCK_TYPE_MASK) == SOCK_STREAM) {
if (listen(fd, opts->backlog ? MAX(opts->backlog, 0) : 1) < 0) {
log_err("Failed to listed on socket");
goto error_close;
diff --git a/tools/testing/selftests/bpf/network_helpers.h b/tools/testing/selftests/bpf/network_helpers.h
index 5764155b6d25..a80916a26779 100644
--- a/tools/testing/selftests/bpf/network_helpers.h
+++ b/tools/testing/selftests/bpf/network_helpers.h
@@ -22,6 +22,11 @@ typedef __u16 __sum16;
#define VIP_NUM 5
#define MAGIC_BYTES 123
+/* include/linux/net.h */
+#ifndef SOCK_TYPE_MASK
+#define SOCK_TYPE_MASK 0xf
+#endif
+
struct network_helper_opts {
int timeout_ms;
int proto;
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_attr_size.c b/tools/testing/selftests/bpf/prog_tests/bpf_attr_size.c
new file mode 100644
index 000000000000..32159dc64da8
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_attr_size.c
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Google LLC */
+#include <linux/bpf.h>
+#include <unistd.h>
+#include <sys/syscall.h>
+#include <test_progs.h>
+#include <cgroup_helpers.h>
+#include "cgroup_skb_direct_packet_access.skel.h"
+
+#define OLD_QUERY_SIZE offsetofend(union bpf_attr, query.prog_cnt)
+#define FULL_QUERY_SIZE offsetofend(union bpf_attr, query.revision)
+
+static void test_query_size_boundaries(void)
+{
+ struct cgroup_skb_direct_packet_access *skel;
+ struct bpf_link *link = NULL;
+ union bpf_attr attr;
+ int cg_fd = -1;
+ int err;
+
+ skel = cgroup_skb_direct_packet_access__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_load"))
+ return;
+
+ cg_fd = test__join_cgroup("/attr_size_cg");
+ if (!ASSERT_GE(cg_fd, 0, "join_cgroup"))
+ goto cleanup;
+
+ link = bpf_program__attach_cgroup(skel->progs.direct_packet_access,
+ cg_fd);
+ if (!ASSERT_OK_PTR(link, "cg_attach"))
+ goto cleanup;
+
+ memset(&attr, 0, sizeof(attr));
+ attr.query.target_fd = cg_fd;
+ attr.query.attach_type = BPF_CGROUP_INET_INGRESS;
+ attr.query.revision = 0xdeadbeefdeadbeefULL;
+
+ err = syscall(__NR_bpf, BPF_PROG_QUERY, &attr, OLD_QUERY_SIZE);
+ if (ASSERT_OK(err, "query_old_size")) {
+ ASSERT_EQ(attr.query.prog_cnt, 1, "prog_cnt_written_old");
+ ASSERT_EQ(attr.query.revision, 0xdeadbeefdeadbeefULL,
+ "revision_not_written_old");
+ }
+
+ memset(&attr, 0, sizeof(attr));
+ attr.query.target_fd = cg_fd;
+ attr.query.attach_type = BPF_CGROUP_INET_INGRESS;
+
+ err = syscall(__NR_bpf, BPF_PROG_QUERY, &attr, FULL_QUERY_SIZE);
+ if (!ASSERT_OK(err, "query_full_size"))
+ goto cleanup;
+
+ ASSERT_EQ(attr.query.prog_cnt, 1, "prog_cnt_written");
+ ASSERT_GT(attr.query.revision, 0, "revision_written");
+
+cleanup:
+ if (link)
+ bpf_link__destroy(link);
+ if (cg_fd >= 0)
+ close(cg_fd);
+ cgroup_skb_direct_packet_access__destroy(skel);
+}
+
+void test_bpf_attr_size(void)
+{
+ if (test__start_subtest("query_size_boundaries"))
+ test_query_size_boundaries();
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/global_map_resize.c b/tools/testing/selftests/bpf/prog_tests/global_map_resize.c
index 56b5baef35c8..602ce30f1720 100644
--- a/tools/testing/selftests/bpf/prog_tests/global_map_resize.c
+++ b/tools/testing/selftests/bpf/prog_tests/global_map_resize.c
@@ -23,6 +23,7 @@ static void global_map_resize_bss_subtest(void)
struct bpf_map *map;
const __u32 desired_sz = sizeof(skel->bss->sum) + sysconf(_SC_PAGE_SIZE) * 2;
size_t array_len, actual_sz, new_sz;
+ int *array;
skel = test_global_map_resize__open();
if (!ASSERT_OK_PTR(skel, "test_global_map_resize__open"))
@@ -58,10 +59,13 @@ static void global_map_resize_bss_subtest(void)
goto teardown;
/* fill the newly resized array with ones,
- * skipping the first element which was previously set
+ * skipping the first element which was previously set;
+ * access through a plain pointer to avoid -Warray-bounds
+ * since the array was resized beyond its declared length.
*/
+ array = skel->bss->array;
for (int i = 1; i < array_len; i++)
- skel->bss->array[i] = 1;
+ array[i] = 1;
/* set global const values before loading */
skel->rodata->pid = getpid();
diff --git a/tools/testing/selftests/bpf/prog_tests/mptcp.c b/tools/testing/selftests/bpf/prog_tests/mptcp.c
index d2ca32fa3b21..e7158a8e95e1 100644
--- a/tools/testing/selftests/bpf/prog_tests/mptcp.c
+++ b/tools/testing/selftests/bpf/prog_tests/mptcp.c
@@ -275,7 +275,7 @@ static int verify_mptcpify(int server_fd, int client_fd)
return err;
}
-static int run_mptcpify(int cgroup_fd)
+static int run_mptcpify(int cgroup_fd, int type)
{
int server_fd, client_fd, err = 0;
struct mptcpify *mptcpify_skel;
@@ -291,7 +291,7 @@ static int run_mptcpify(int cgroup_fd)
goto out;
/* without MPTCP */
- server_fd = start_server(AF_INET, SOCK_STREAM, NULL, 0, 0);
+ server_fd = start_server(AF_INET, type, NULL, 0, 0);
if (!ASSERT_GE(server_fd, 0, "start_server")) {
err = -EIO;
goto out;
@@ -328,7 +328,14 @@ static void test_mptcpify(void)
if (!ASSERT_OK_PTR(nstoken, "create_netns"))
goto fail;
- ASSERT_OK(run_mptcpify(cgroup_fd), "run_mptcpify");
+ ASSERT_OK(run_mptcpify(cgroup_fd, SOCK_STREAM), "run_mptcpify");
+ /* userspace sets flags such as SOCK_CLOEXEC together with the type;
+ * the BPF prog must still upgrade the socket to MPTCP. See
+ * update_socket_protocol() in net/socket.c, which runs before the
+ * type is masked with SOCK_TYPE_MASK.
+ */
+ ASSERT_OK(run_mptcpify(cgroup_fd, SOCK_STREAM | SOCK_CLOEXEC),
+ "run_mptcpify_cloexec");
fail:
cleanup_netns(nstoken);
diff --git a/tools/testing/selftests/bpf/progs/bpf_tracing_net.h b/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
index 59843b430f76..674120405e49 100644
--- a/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
+++ b/tools/testing/selftests/bpf/progs/bpf_tracing_net.h
@@ -8,6 +8,9 @@
#define AF_INET 2
#define AF_INET6 10
+/* include/linux/net.h */
+#define SOCK_TYPE_MASK 0xf
+
#define SOL_SOCKET 1
#define SO_REUSEADDR 2
#define SO_SNDBUF 7
diff --git a/tools/testing/selftests/bpf/progs/mptcpify.c b/tools/testing/selftests/bpf/progs/mptcpify.c
index cbdc730c3a47..e3f8cb54dbe9 100644
--- a/tools/testing/selftests/bpf/progs/mptcpify.c
+++ b/tools/testing/selftests/bpf/progs/mptcpify.c
@@ -15,7 +15,7 @@ int BPF_PROG(mptcpify, int family, int type, int protocol)
return protocol;
if ((family == AF_INET || family == AF_INET6) &&
- type == SOCK_STREAM &&
+ (type & SOCK_TYPE_MASK) == SOCK_STREAM &&
(!protocol || protocol == IPPROTO_TCP)) {
return IPPROTO_MPTCP;
}
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index fa829a7854f2..3a89cf01db90 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -468,18 +468,19 @@ bool test__start_subtest(const char *subtest_name)
struct test_state *state = env.test_state;
struct subtest_state *subtest_state;
size_t sub_state_size = sizeof(*subtest_state);
+ void *tmp;
if (env.subtest_state)
test__end_subtest();
state->subtest_num++;
- state->subtest_states =
- realloc(state->subtest_states,
- state->subtest_num * sub_state_size);
- if (!state->subtest_states) {
+ tmp = realloc(state->subtest_states, state->subtest_num * sub_state_size);
+ if (!tmp) {
+ state->subtest_num--;
fprintf(stderr, "Not enough memory to allocate subtest result\n");
return false;
}
+ state->subtest_states = tmp;
subtest_state = &state->subtest_states[state->subtest_num - 1];
@@ -1561,7 +1562,7 @@ static void server_main(void)
data[i].worker_id = i;
data[i].sock_fd = env.worker_socks[i];
rc = pthread_create(&dispatcher_threads[i], NULL, dispatch_thread, &data[i]);
- if (rc < 0) {
+ if (rc) {
perror("Failed to launch dispatcher thread");
exit(EXIT_ERR_SETUP_INFRA);
}
diff --git a/tools/testing/selftests/bpf/test_sockmap.c b/tools/testing/selftests/bpf/test_sockmap.c
index 61a747afcd05..7b237f737624 100644
--- a/tools/testing/selftests/bpf/test_sockmap.c
+++ b/tools/testing/selftests/bpf/test_sockmap.c
@@ -514,7 +514,8 @@ static int msg_alloc_iov(struct msghdr *msg,
return 0;
unwind_iov:
for (i--; i >= 0 ; i--)
- free(msg->msg_iov[i].iov_base);
+ free(iov[i].iov_base);
+ free(iov);
return -ENOMEM;
}
diff --git a/tools/testing/selftests/bpf/veristat.c b/tools/testing/selftests/bpf/veristat.c
index e9d7acff6bbb..6a457a50289f 100644
--- a/tools/testing/selftests/bpf/veristat.c
+++ b/tools/testing/selftests/bpf/veristat.c
@@ -405,6 +405,40 @@ static bool is_bpf_obj_file(const char *path) {
return err == 0;
}
+/* Exact filter match */
+static bool name_filter_matches(struct filter *f, const char *filename, const char *prog_name)
+{
+ if (f->any_glob)
+ return glob_matches(filename, f->any_glob) ||
+ (prog_name && glob_matches(prog_name, f->any_glob));
+ if (f->file_glob && f->prog_glob)
+ return prog_name &&
+ glob_matches(filename, f->file_glob) &&
+ glob_matches(prog_name, f->prog_glob);
+ if (f->file_glob)
+ return glob_matches(filename, f->file_glob);
+ if (f->prog_glob)
+ return prog_name && glob_matches(prog_name, f->prog_glob);
+ return false;
+}
+
+/* Check if the filter does not outright reject the file name */
+static bool name_filter_may_match(struct filter *f, const char *filename)
+{
+ if (f->file_glob)
+ return glob_matches(filename, f->file_glob);
+ /*
+ * If we don't know program name yet, any_glob filter
+ * has to assume that current BPF object file might be
+ * relevant; we'll check again later on after opening
+ * BPF object file, at which point program name will
+ * be known finally.
+ */
+ if (f->any_glob || f->prog_glob)
+ return true;
+ return false;
+}
+
static bool should_process_file_prog(const char *filename, const char *prog_name)
{
struct filter *f;
@@ -412,16 +446,7 @@ static bool should_process_file_prog(const char *filename, const char *prog_name
for (i = 0; i < env.deny_filter_cnt; i++) {
f = &env.deny_filters[i];
- if (f->kind != FILTER_NAME)
- continue;
-
- if (f->any_glob && glob_matches(filename, f->any_glob))
- return false;
- if (f->any_glob && prog_name && glob_matches(prog_name, f->any_glob))
- return false;
- if (f->file_glob && glob_matches(filename, f->file_glob))
- return false;
- if (f->prog_glob && prog_name && glob_matches(prog_name, f->prog_glob))
+ if (f->kind == FILTER_NAME && name_filter_matches(f, filename, prog_name))
return false;
}
@@ -431,24 +456,15 @@ static bool should_process_file_prog(const char *filename, const char *prog_name
continue;
allow_cnt++;
- if (f->any_glob) {
- if (glob_matches(filename, f->any_glob))
- return true;
- /* If we don't know program name yet, any_glob filter
- * has to assume that current BPF object file might be
- * relevant; we'll check again later on after opening
- * BPF object file, at which point program name will
- * be known finally.
- */
- if (!prog_name || glob_matches(prog_name, f->any_glob))
- return true;
- } else {
- if (f->file_glob && !glob_matches(filename, f->file_glob))
- continue;
- if (f->prog_glob && prog_name && !glob_matches(prog_name, f->prog_glob))
- continue;
+ if (prog_name && name_filter_matches(f, filename, prog_name))
+ return true;
+ /*
+ * If there is no prog_name and the file name is not blocked by
+ * the filter, allow to open the file. Afterwards there would be
+ * a second refining query with prog_name set.
+ */
+ if (!prog_name && name_filter_may_match(f, filename))
return true;
- }
}
/* if there are no file/prog name allow filters, allow all progs,
@@ -594,6 +610,12 @@ static int append_filter(struct filter **filters, int *cnt, const char *str)
}
}
+ if ((!f->any_glob && !f->file_glob && !f->prog_glob) ||
+ (f->any_glob && strcmp(f->any_glob, "") == 0)) {
+ fprintf(stderr, "Invalid filter: '%s'\n", str);
+ return -EINVAL;
+ }
+
*cnt += 1;
return 0;
}
diff --git a/tools/testing/selftests/bpf/vmtest.sh b/tools/testing/selftests/bpf/vmtest.sh
index b0aca33d7591..8282058d700f 100755
--- a/tools/testing/selftests/bpf/vmtest.sh
+++ b/tools/testing/selftests/bpf/vmtest.sh
@@ -419,8 +419,17 @@ main()
if [[ $# -eq 0 && "${debug_shell}" == "no" ]]; then
echo "No command specified, will run ${DEFAULT_COMMAND} in the vm"
- else
- command="$@"
+ elif [[ $# -gt 0 ]]; then
+ # Quote each argument so the command survives into the guest: the
+ # host expands ${command} into the generated init script, which
+ # the guest bash then parses as shell source. Without the %q
+ # escapes an argument with a space or a glob (e.g. -a 'verifier_*')
+ # is re-split and expanded against /root/bpf there.
+ #
+ # Skip this when there is no command: printf '%q ' would still
+ # apply the format once and emit '', which is not the empty
+ # command that -s (debug shell) expects.
+ command=$(printf '%q ' "$@")
fi
local kconfig_file="${OUTPUT_DIR}/latest.config"
diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
index 400a696a0d21..f6bc762705ab 100755
--- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh
+++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh
@@ -88,22 +88,30 @@ echo "" > test/cpuset.cpus
# If isolated CPUs have been reserved at boot time (as shown in
# cpuset.cpus.isolated), these isolated CPUs should be outside of CPUs 0-8
# that will be used by this script for testing purpose. If not, some of
-# the tests may fail incorrectly. These pre-isolated CPUs should stay in
-# an isolated state throughout the testing process for now.
+# the tests may fail incorrectly. Wait a bit and retry again just in case
+# these isolated CPUs are leftover from previous run and have just been
+# cleaned up earlier in this script.
+#
+# These pre-isolated CPUs should stay in an isolated state throughout the
+# testing process for now.
#
BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+[[ -n "$BOOT_ISOLCPUS" ]] && {
+ sleep 0.5
+ BOOT_ISOLCPUS=$(cat $CGROUP2/cpuset.cpus.isolated)
+}
if [[ -n "$BOOT_ISOLCPUS" ]]
then
[[ $(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//") -le 8 ]] &&
skip_test "Pre-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested"
echo "Pre-isolated CPUs: $BOOT_ISOLCPUS"
fi
+
cleanup()
{
online_cpus
cd $CGROUP2
- rmdir A1/A2/A3 A1/A2 A1 B1 > /dev/null 2>&1
- rmdir test > /dev/null 2>&1
+ rmdir A1/A2/A3 A1/A2 A1 B1 test/A1 test/B1 test > /dev/null 2>&1
[[ -n "$SCHED_DEBUG" ]] &&
echo "$SCHED_DEBUG" > /sys/kernel/debug/sched/verbose
}
@@ -173,14 +181,22 @@ test_add_proc()
#
# Cgroup test hierarchy
#
-# root -- A1 -- A2 -- A3
-# +- B1
+# root
+# |
+# +------+------+
+# | |
+# A1 B1
+# |
+# A2
+# |
+# A3
#
# P<v> = set cpus.partition (0:member, 1:root, 2:isolated)
# C<l> = add cpu-list to cpuset.cpus
# X<l> = add cpu-list to cpuset.cpus.exclusive
# S<p> = use prefix in subtree_control
# T = put a task into cgroup
+# CX<l> = add cpu-list to both cpuset.cpus and cpuset.cpus.exclusive
# O<c>=<v> = Write <v> to CPU online file of <c>
#
# ECPUs - effective CPUs of cpusets
@@ -207,130 +223,130 @@ TEST_MATRIX=(
" C0-1:P1 . . C2-3 S+:C4-5 . . . 0 A1:4-5"
" C0-1 . . C2-3:P1 . . . C2 0 "
" C0-1 . . C2-3:P1 . . . C4-5 0 B1:4-5"
- "C0-3:P1:S+ C2-3:P1 . . . . . . 0 A1:0-1,A2:2-3"
- "C0-3:P1:S+ C2-3:P1 . . C1-3 . . . 0 A1:1,A2:2-3"
- "C2-3:P1:S+ C3:P1 . . C3 . . . 0 A1:,A2:3 A1:P1,A2:P1"
- "C2-3:P1:S+ C3:P1 . . C3 P0 . . 0 A1:3,A2:3 A1:P1,A2:P0"
- "C2-3:P1:S+ C2:P1 . . C2-4 . . . 0 A1:3-4,A2:2"
- "C2-3:P1:S+ C3:P1 . . C3 . . C0-2 0 A1:,B1:0-2 A1:P1,A2:P1"
- "$SETUP_A123_PARTITIONS . C2-3 . . . 0 A1:,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
+ "C0-3:P1:S+ C2-3:P1 . . . . . . 0 A1:0-1|A2:2-3"
+ "C0-3:P1:S+ C2-3:P1 . . C1-3 . . . 0 A1:1|A2:2-3"
+ "C2-3:P1:S+ C3:P1 . . C3 . . . 0 A1:|A2:3 A1:P1|A2:P1"
+ "C2-3:P1:S+ C3:P1 . . C3 P0 . . 0 A1:3|A2:3 A1:P1|A2:P0"
+ "C2-3:P1:S+ C2:P1 . . C2-4 . . . 0 A1:3-4|A2:2"
+ "C2-3:P1:S+ C3:P1 . . C3 . . C0-2 0 A1:|B1:0-2 A1:P1|A2:P1"
+ "$SETUP_A123_PARTITIONS . C2-3 . . . 0 A1:|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
# CPU offlining cases:
- " C0-1 . . C2-3 S+ C4-5 . O2=0 0 A1:0-1,B1:3"
- "C0-3:P1:S+ C2-3:P1 . . O2=0 . . . 0 A1:0-1,A2:3"
- "C0-3:P1:S+ C2-3:P1 . . O2=0 O2=1 . . 0 A1:0-1,A2:2-3"
- "C0-3:P1:S+ C2-3:P1 . . O1=0 . . . 0 A1:0,A2:2-3"
- "C0-3:P1:S+ C2-3:P1 . . O1=0 O1=1 . . 0 A1:0-1,A2:2-3"
- "C2-3:P1:S+ C3:P1 . . O3=0 O3=1 . . 0 A1:2,A2:3 A1:P1,A2:P1"
- "C2-3:P1:S+ C3:P2 . . O3=0 O3=1 . . 0 A1:2,A2:3 A1:P1,A2:P2"
- "C2-3:P1:S+ C3:P1 . . O2=0 O2=1 . . 0 A1:2,A2:3 A1:P1,A2:P1"
- "C2-3:P1:S+ C3:P2 . . O2=0 O2=1 . . 0 A1:2,A2:3 A1:P1,A2:P2"
- "C2-3:P1:S+ C3:P1 . . O2=0 . . . 0 A1:,A2:3 A1:P1,A2:P1"
- "C2-3:P1:S+ C3:P1 . . O3=0 . . . 0 A1:2,A2: A1:P1,A2:P1"
- "C2-3:P1:S+ C3:P1 . . T:O2=0 . . . 0 A1:3,A2:3 A1:P1,A2:P-1"
- "C2-3:P1:S+ C3:P1 . . . T:O3=0 . . 0 A1:2,A2:2 A1:P1,A2:P-1"
- "$SETUP_A123_PARTITIONS . O1=0 . . . 0 A1:,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . O2=0 . . . 0 A1:1,A2:,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . O3=0 . . . 0 A1:1,A2:2,A3: A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . T:O1=0 . . . 0 A1:2-3,A2:2-3,A3:3 A1:P1,A2:P-1,A3:P-1"
- "$SETUP_A123_PARTITIONS . . T:O2=0 . . 0 A1:1,A2:3,A3:3 A1:P1,A2:P1,A3:P-1"
- "$SETUP_A123_PARTITIONS . . . T:O3=0 . 0 A1:1,A2:2,A3:2 A1:P1,A2:P1,A3:P-1"
- "$SETUP_A123_PARTITIONS . T:O1=0 O1=1 . . 0 A1:1,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . . T:O2=0 O2=1 . 0 A1:1,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . . . T:O3=0 O3=1 0 A1:1,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . T:O1=0 O2=0 O1=1 . 0 A1:1,A2:,A3:3 A1:P1,A2:P1,A3:P1"
- "$SETUP_A123_PARTITIONS . T:O1=0 O2=0 O2=1 . 0 A1:2-3,A2:2-3,A3:3 A1:P1,A2:P-1,A3:P-1"
+ " C0-1 . . C2-3 S+ C4-5 . O2=0 0 A1:0-1|B1:3"
+ "C0-3:P1:S+ C2-3:P1 . . O2=0 . . . 0 A1:0-1|A2:3"
+ "C0-3:P1:S+ C2-3:P1 . . O2=0 O2=1 . . 0 A1:0-1|A2:2-3"
+ "C0-3:P1:S+ C2-3:P1 . . O1=0 . . . 0 A1:0|A2:2-3"
+ "C0-3:P1:S+ C2-3:P1 . . O1=0 O1=1 . . 0 A1:0-1|A2:2-3"
+ "C2-3:P1:S+ C3:P1 . . O3=0 O3=1 . . 0 A1:2|A2:3 A1:P1|A2:P1"
+ "C2-3:P1:S+ C3:P2 . . O3=0 O3=1 . . 0 A1:2|A2:3 A1:P1|A2:P2"
+ "C2-3:P1:S+ C3:P1 . . O2=0 O2=1 . . 0 A1:2|A2:3 A1:P1|A2:P1"
+ "C2-3:P1:S+ C3:P2 . . O2=0 O2=1 . . 0 A1:2|A2:3 A1:P1|A2:P2"
+ "C2-3:P1:S+ C3:P1 . . O2=0 . . . 0 A1:|A2:3 A1:P1|A2:P1"
+ "C2-3:P1:S+ C3:P1 . . O3=0 . . . 0 A1:2|A2: A1:P1|A2:P1"
+ "C2-3:P1:S+ C3:P1 . . T:O2=0 . . . 0 A1:3|A2:3 A1:P1|A2:P-1"
+ "C2-3:P1:S+ C3:P1 . . . T:O3=0 . . 0 A1:2|A2:2 A1:P1|A2:P-1"
+ "$SETUP_A123_PARTITIONS . O1=0 . . . 0 A1:|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . O2=0 . . . 0 A1:1|A2:|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . O3=0 . . . 0 A1:1|A2:2|A3: A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . T:O1=0 . . . 0 A1:2-3|A2:2-3|A3:3 A1:P1|A2:P-1|A3:P-1"
+ "$SETUP_A123_PARTITIONS . . T:O2=0 . . 0 A1:1|A2:3|A3:3 A1:P1|A2:P1|A3:P-1"
+ "$SETUP_A123_PARTITIONS . . . T:O3=0 . 0 A1:1|A2:2|A3:2 A1:P1|A2:P1|A3:P-1"
+ "$SETUP_A123_PARTITIONS . T:O1=0 O1=1 . . 0 A1:1|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . . T:O2=0 O2=1 . 0 A1:1|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . . . T:O3=0 O3=1 0 A1:1|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . T:O1=0 O2=0 O1=1 . 0 A1:1|A2:|A3:3 A1:P1|A2:P1|A3:P1"
+ "$SETUP_A123_PARTITIONS . T:O1=0 O2=0 O2=1 . 0 A1:2-3|A2:2-3|A3:3 A1:P1|A2:P-1|A3:P-1"
# old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
# ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
#
# Remote partition and cpuset.cpus.exclusive tests
#
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 . . . 0 A1:0-3,A2:1-3,A3:2-3,XA1:2-3"
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3:P2 . . 0 A1:0-1,A2:2-3,A3:2-3 A1:P0,A2:P2 2-3"
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X3:P2 . . 0 A1:0-2,A2:3,A3:3 A1:P0,A2:P2 3"
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2 . 0 A1:0-1,A2:1,A3:2-3 A1:P0,A3:P2 2-3"
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:C3 . 0 A1:0-1,A2:1,A3:2-3 A1:P0,A3:P2 2-3"
- " C0-3:S+ C1-3:S+ C2-3 C2-3 . . . P2 0 A1:0-3,A2:1-3,A3:2-3,B1:2-3 A1:P0,A3:P0,B1:P-2"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 . . . 0 A1:0-3|A2:1-3|A3:2-3|XA1:2-3"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3:P2 . . 0 A1:0-1|A2:2-3|A3:2-3 A1:P0|A2:P2 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X3:P2 . . 0 A1:0-2|A2:3|A3:3 A1:P0|A2:P2 3"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2 . 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:C3 . 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 C2-3 . . . P2 0 A1:0-3|A2:1-3|A3:2-3|B1:2-3 A1:P0|A3:P0|B1:P-2"
" C0-3:S+ C1-3:S+ C2-3 C4-5 . . . P2 0 B1:4-5 B1:P2 4-5"
- " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2 P2 0 A3:2-3,B1:4 A3:P2,B1:P2 2-4"
- " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2:C1-3 P2 0 A3:2-3,B1:4 A3:P2,B1:P2 2-4"
- " C0-3:S+ C1-3:S+ C2-3 C4 X1-3 X1-3:P2 P2 . 0 A2:1,A3:2-3 A2:P2,A3:P2 1-3"
- " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2 P2:C4-5 0 A3:2-3,B1:4-5 A3:P2,B1:P2 2-5"
- " C4:X0-3:S+ X1-3:S+ X2-3 . . P2 . . 0 A1:4,A2:1-3,A3:1-3 A2:P2 1-3"
- " C4:X0-3:S+ X1-3:S+ X2-3 . . . P2 . 0 A1:4,A2:4,A3:2-3 A3:P2 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2 P2 0 A3:2-3|B1:4 A3:P2|B1:P2 2-4"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2:C1-3 P2 0 A3:2-3|B1:4 A3:P2|B1:P2 2-4"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X1-3 X1-3:P2 P2 . 0 A2:1|A3:2-3 A2:P2|A3:P2 1-3"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3 X2-3:P2 P2:C4-5 0 A3:2-3|B1:4-5 A3:P2|B1:P2 2-5"
+ " C4:X0-3:S+ X1-3:S+ X2-3 . . P2 . . 0 A1:4|A2:1-3|A3:1-3 A2:P2 1-3"
+ " C4:X0-3:S+ X1-3:S+ X2-3 . . . P2 . 0 A1:4|A2:4|A3:2-3 A3:P2 2-3"
# Nested remote/local partition tests
- " C0-3:S+ C1-3:S+ C2-3 C4-5 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:,A3:2-3,B1:4-5 \
- A1:P0,A2:P1,A3:P2,B1:P1 2-3"
- " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:,A3:2-3,B1:4 \
- A1:P0,A2:P1,A3:P2,B1:P1 2-4,2-3"
- " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3:P1 . P1 0 A1:0-1,A2:2-3,A3:2-3,B1:4 \
- A1:P0,A2:P1,A3:P0,B1:P1"
- " C0-3:S+ C1-3:S+ C3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1,A2:2,A3:3,B1:4 \
- A1:P0,A2:P1,A3:P2,B1:P1 2-4,3"
- " C0-4:S+ C1-4:S+ C2-4 . X2-4 X2-4:P2 X4:P1 . 0 A1:0-1,A2:2-3,A3:4 \
- A1:P0,A2:P2,A3:P1 2-4,2-3"
- " C0-4:S+ C1-4:S+ C2-4 . X2-4 X2-4:P2 X3-4:P1 . 0 A1:0-1,A2:2,A3:3-4 \
- A1:P0,A2:P2,A3:P1 2"
+ " C0-3:S+ C1-3:S+ C2-3 C4-5 X2-3 X2-3:P1 P2 P1 0 A1:0-1|A2:|A3:2-3|B1:4-5 \
+ A1:P0|A2:P1|A3:P2|B1:P1 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1|A2:|A3:2-3|B1:4 \
+ A1:P0|A2:P1|A3:P2|B1:P1 2-4|2-3"
+ " C0-3:S+ C1-3:S+ C2-3 C4 X2-3 X2-3:P1 . P1 0 A1:0-1|A2:2-3|A3:2-3|B1:4 \
+ A1:P0|A2:P1|A3:P0|B1:P1"
+ " C0-3:S+ C1-3:S+ C3 C4 X2-3 X2-3:P1 P2 P1 0 A1:0-1|A2:2|A3:3|B1:4 \
+ A1:P0|A2:P1|A3:P2|B1:P1 2-4|3"
+ " C0-4:S+ C1-4:S+ C2-4 . X2-4 X2-4:P2 X4:P1 . 0 A1:0-1|A2:2-3|A3:4 \
+ A1:P0|A2:P2|A3:P1 2-4|2-3"
+ " C0-4:S+ C1-4:S+ C2-4 . X2-4 X2-4:P2 X3-4:P1 . 0 A1:0-1|A2:2|A3:3-4 \
+ A1:P0|A2:P2|A3:P1 2"
" C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \
- . . X5 . . 0 A1:0-4,A2:1-4,A3:2-4 \
- A1:P0,A2:P-2,A3:P-1"
+ . . X5 . . 0 A1:0-4|A2:1-4|A3:2-4 \
+ A1:P0|A2:P-2|A3:P-1"
" C0-4:X2-4:S+ C1-4:X2-4:S+:P2 C2-4:X4:P1 \
- . . . X1 . 0 A1:0-1,A2:2-4,A3:2-4 \
- A1:P0,A2:P2,A3:P-1 2-4"
+ . . . X1 . 0 A1:0-1|A2:2-4|A3:2-4 \
+ A1:P0|A2:P2|A3:P-1 2-4"
# Remote partition offline tests
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1,A2:1,A3:3 A1:P0,A3:P2 2-3"
- " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1,A2:1,A3:2-3 A1:P0,A3:P2 2-3"
- " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2,A2:1-2,A3: A1:P0,A3:P2 3"
- " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 T:P2:O3=0 . 0 A1:0-2,A2:1-2,A3:1-2 A1:P0,A3:P-2 3,"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
+ " C0-3:S+ C1-3:S+ C2-3 . X2-3 X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3"
+ " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3"
+ " C0-3:S+ C1-3:S+ C3 . X2-3 X2-3 T:P2:O3=0 . 0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|"
# An invalidated remote partition cannot self-recover from hotplug
- " C0-3:S+ C1-3:S+ C2 . X2-3 X2-3 T:P2:O2=0 O2=1 0 A1:0-3,A2:1-3,A3:2 A1:P0,A3:P-2"
+ " C0-3:S+ C1-3:S+ C2 . X2-3 X2-3 T:P2:O2=0 O2=1 0 A1:0-3|A2:1-3|A3:2 A1:P0|A3:P-2"
# cpus.exclusive.effective clearing test
- " C0-3:S+ C1-3:S+ C2 . X2-3:X . . . 0 A1:0-3,A2:1-3,A3:2,XA1:"
+ " C0-3:S+ C1-3:S+ C2 . X2-3:X . . . 0 A1:0-3|A2:1-3|A3:2|XA1:"
# Invalid to valid remote partition transition test
- " C0-3:S+ C1-3 . . . X3:P2 . . 0 A1:0-3,A2:1-3,XA2: A2:P-2"
+ " C0-3:S+ C1-3 . . . X3:P2 . . 0 A1:0-3|A2:1-3|XA2: A2:P-2"
" C0-3:S+ C1-3:X3:P2
- . . X2-3 P2 . . 0 A1:0-2,A2:3,XA2:3 A2:P2 3"
+ . . X2-3 P2 . . 0 A1:0-2|A2:3|XA2:3 A2:P2 3"
# Invalid to valid local partition direct transition tests
- " C1-3:S+:P2 X4:P2 . . . . . . 0 A1:1-3,XA1:1-3,A2:1-3:XA2: A1:P2,A2:P-2 1-3"
- " C1-3:S+:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2,XA1:1-3,A2:3:XA2:3 A1:P2,A2:P2 1-3"
- " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4,B1:4-6 A1:P-2,B1:P0"
- " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3,B1:4-6 A1:P2,B1:P0 0-3"
- " C0-3:P2 . . C3-5:C4-5 . . . . 0 A1:0-3,B1:4-5 A1:P2,B1:P0 0-3"
+ " C1-3:S+:P2 X4:P2 . . . . . . 0 A1:1-3|XA1:1-3|A2:1-3:XA2: A1:P2|A2:P-2 1-3"
+ " C1-3:S+:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2|XA1:1-3|A2:3:XA2:3 A1:P2|A2:P2 1-3"
+ " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4|B1:4-6 A1:P-2|B1:P0"
+ " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3|B1:4-6 A1:P2|B1:P0 0-3"
+ " C0-3:P2 . . C3-5:C4-5 . . . . 0 A1:0-3|B1:4-5 A1:P2|B1:P0 0-3"
# Local partition invalidation tests
" C0-3:X1-3:S+:P2 C1-3:X2-3:S+:P2 C2-3:X3:P2 \
- . . . . . 0 A1:1,A2:2,A3:3 A1:P2,A2:P2,A3:P2 1-3"
+ . . . . . 0 A1:1|A2:2|A3:3 A1:P2|A2:P2|A3:P2 1-3"
" C0-3:X1-3:S+:P2 C1-3:X2-3:S+:P2 C2-3:X3:P2 \
- . . X4 . . 0 A1:1-3,A2:1-3,A3:2-3,XA2:,XA3: A1:P2,A2:P-2,A3:P-2 1-3"
+ . . X4 . . 0 A1:1-3|A2:1-3|A3:2-3|XA2:|XA3: A1:P2|A2:P-2|A3:P-2 1-3"
" C0-3:X1-3:S+:P2 C1-3:X2-3:S+:P2 C2-3:X3:P2 \
- . . C4:X . . 0 A1:1-3,A2:1-3,A3:2-3,XA2:,XA3: A1:P2,A2:P-2,A3:P-2 1-3"
+ . . C4:X . . 0 A1:1-3|A2:1-3|A3:2-3|XA2:|XA3: A1:P2|A2:P-2|A3:P-2 1-3"
# Local partition CPU change tests
- " C0-5:S+:P2 C4-5:S+:P1 . . . C3-5 . . 0 A1:0-2,A2:3-5 A1:P2,A2:P1 0-2"
- " C0-5:S+:P2 C4-5:S+:P1 . . C1-5 . . . 0 A1:1-3,A2:4-5 A1:P2,A2:P1 1-3"
+ " C0-5:S+:P2 C4-5:S+:P1 . . . C3-5 . . 0 A1:0-2|A2:3-5 A1:P2|A2:P1 0-2"
+ " C0-5:S+:P2 C4-5:S+:P1 . . C1-5 . . . 0 A1:1-3|A2:4-5 A1:P2|A2:P1 1-3"
# cpus_allowed/exclusive_cpus update tests
" C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \
- . X:C4 . P2 . 0 A1:4,A2:4,XA2:,XA3:,A3:4 \
- A1:P0,A3:P-2"
+ . X:C4 . P2 . 0 A1:4|A2:4|XA2:|XA3:|A3:4 \
+ A1:P0|A3:P-2"
" C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \
- . X1 . P2 . 0 A1:0-3,A2:1-3,XA1:1,XA2:,XA3:,A3:2-3 \
- A1:P0,A3:P-2"
+ . X1 . P2 . 0 A1:0-3|A2:1-3|XA1:1|XA2:|XA3:|A3:2-3 \
+ A1:P0|A3:P-2"
" C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3 \
- . . X3 P2 . 0 A1:0-2,A2:1-2,XA2:3,XA3:3,A3:3 \
- A1:P0,A3:P2 3"
+ . . X3 P2 . 0 A1:0-2|A2:1-2|XA2:3|XA3:3|A3:3 \
+ A1:P0|A3:P2 3"
" C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3:P2 \
- . . X3 . . 0 A1:0-3,A2:1-3,XA2:3,XA3:3,A3:2-3 \
- A1:P0,A3:P-2"
+ . . X3 . . 0 A1:0-2|A2:1-2|XA2:3|XA3:3|A3:3|XA3:3 \
+ A1:P0|A3:P2 3"
" C0-3:X2-3:S+ C1-3:X2-3:S+ C2-3:X2-3:P2 \
- . X4 . . . 0 A1:0-3,A2:1-3,A3:2-3,XA1:4,XA2:,XA3 \
- A1:P0,A3:P-2"
+ . X4 . . . 0 A1:0-3|A2:1-3|A3:2-3|XA1:4|XA2:|XA3 \
+ A1:P0|A3:P-2"
# old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
# ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
@@ -339,68 +355,68 @@ TEST_MATRIX=(
#
# Adding CPUs to partition root that are not in parent's
# cpuset.cpus is allowed, but those extra CPUs are ignored.
- "C2-3:P1:S+ C3:P1 . . . C2-4 . . 0 A1:,A2:2-3 A1:P1,A2:P1"
+ "C2-3:P1:S+ C3:P1 . . . C2-4 . . 0 A1:|A2:2-3 A1:P1|A2:P1"
# Taking away all CPUs from parent or itself if there are tasks
# will make the partition invalid.
- "C2-3:P1:S+ C3:P1 . . T C2-3 . . 0 A1:2-3,A2:2-3 A1:P1,A2:P-1"
- " C3:P1:S+ C3 . . T P1 . . 0 A1:3,A2:3 A1:P1,A2:P-1"
- "$SETUP_A123_PARTITIONS . T:C2-3 . . . 0 A1:2-3,A2:2-3,A3:3 A1:P1,A2:P-1,A3:P-1"
- "$SETUP_A123_PARTITIONS . T:C2-3:C1-3 . . . 0 A1:1,A2:2,A3:3 A1:P1,A2:P1,A3:P1"
+ "C2-3:P1:S+ C3:P1 . . T C2-3 . . 0 A1:2-3|A2:2-3 A1:P1|A2:P-1"
+ " C3:P1:S+ C3 . . T P1 . . 0 A1:3|A2:3 A1:P1|A2:P-1"
+ "$SETUP_A123_PARTITIONS . T:C2-3 . . . 0 A1:2-3|A2:2-3|A3:3 A1:P1|A2:P-1|A3:P-1"
+ "$SETUP_A123_PARTITIONS . T:C2-3:C1-3 . . . 0 A1:1|A2:2|A3:3 A1:P1|A2:P1|A3:P1"
# Changing a partition root to member makes child partitions invalid
- "C2-3:P1:S+ C3:P1 . . P0 . . . 0 A1:2-3,A2:3 A1:P0,A2:P-1"
- "$SETUP_A123_PARTITIONS . C2-3 P0 . . 0 A1:2-3,A2:2-3,A3:3 A1:P1,A2:P0,A3:P-1"
+ "C2-3:P1:S+ C3:P1 . . P0 . . . 0 A1:2-3|A2:3 A1:P0|A2:P-1"
+ "$SETUP_A123_PARTITIONS . C2-3 P0 . . 0 A1:2-3|A2:2-3|A3:3 A1:P1|A2:P0|A3:P-1"
# cpuset.cpus can contains cpus not in parent's cpuset.cpus as long
# as they overlap.
- "C2-3:P1:S+ . . . . C3-4:P1 . . 0 A1:2,A2:3 A1:P1,A2:P1"
+ "C2-3:P1:S+ . . . . C3-4:P1 . . 0 A1:2|A2:3 A1:P1|A2:P1"
# Deletion of CPUs distributed to child cgroup is allowed.
- "C0-1:P1:S+ C1 . C2-3 C4-5 . . . 0 A1:4-5,A2:4-5"
+ "C0-1:P1:S+ C1 . C2-3 C4-5 . . . 0 A1:4-5|A2:4-5"
# To become a valid partition root, cpuset.cpus must overlap parent's
# cpuset.cpus.
- " C0-1:P1 . . C2-3 S+ C4-5:P1 . . 0 A1:0-1,A2:0-1 A1:P1,A2:P-1"
+ " C0-1:P1 . . C2-3 S+ C4-5:P1 . . 0 A1:0-1|A2:0-1 A1:P1|A2:P-1"
# Enabling partition with child cpusets is allowed
- " C0-1:S+ C1 . C2-3 P1 . . . 0 A1:0-1,A2:1 A1:P1"
+ " C0-1:S+ C1 . C2-3 P1 . . . 0 A1:0-1|A2:1 A1:P1"
- # A partition root with non-partition root parent is invalid, but it
+ # A partition root with non-partition root parent is invalid| but it
# can be made valid if its parent becomes a partition root too.
- " C0-1:S+ C1 . C2-3 . P2 . . 0 A1:0-1,A2:1 A1:P0,A2:P-2"
- " C0-1:S+ C1:P2 . C2-3 P1 . . . 0 A1:0,A2:1 A1:P1,A2:P2"
+ " C0-1:S+ C1 . C2-3 . P2 . . 0 A1:0-1|A2:1 A1:P0|A2:P-2"
+ " C0-1:S+ C1:P2 . C2-3 P1 . . . 0 A1:0|A2:1 A1:P1|A2:P2"
# A non-exclusive cpuset.cpus change will invalidate partition and its siblings
- " C0-1:P1 . . C2-3 C0-2 . . . 0 A1:0-2,B1:2-3 A1:P-1,B1:P0"
- " C0-1:P1 . . P1:C2-3 C0-2 . . . 0 A1:0-2,B1:2-3 A1:P-1,B1:P-1"
- " C0-1 . . P1:C2-3 C0-2 . . . 0 A1:0-2,B1:2-3 A1:P0,B1:P-1"
+ " C0-1:P1 . . C2-3 C0-2 . . . 0 A1:0-2|B1:2-3 A1:P-1|B1:P0"
+ " C0-1:P1 . . P1:C2-3 C0-2 . . . 0 A1:0-2|B1:2-3 A1:P-1|B1:P-1"
+ " C0-1 . . P1:C2-3 C0-2 . . . 0 A1:0-2|B1:2-3 A1:P0|B1:P-1"
# cpuset.cpus can overlap with sibling cpuset.cpus.exclusive but not subsumed by it
- " C0-3 . . C4-5 X5 . . . 0 A1:0-3,B1:4-5"
+ " C0-3 . . C4-5 X5 . . . 0 A1:0-3|B1:4-5"
# Child partition root that try to take all CPUs from parent partition
# with tasks will remain invalid.
- " C1-4:P1:S+ P1 . . . . . . 0 A1:1-4,A2:1-4 A1:P1,A2:P-1"
- " C1-4:P1:S+ P1 . . . C1-4 . . 0 A1,A2:1-4 A1:P1,A2:P1"
- " C1-4:P1:S+ P1 . . T C1-4 . . 0 A1:1-4,A2:1-4 A1:P1,A2:P-1"
+ " C1-4:P1:S+ P1 . . . . . . 0 A1:1-4|A2:1-4 A1:P1|A2:P-1"
+ " C1-4:P1:S+ P1 . . . C1-4 . . 0 A1|A2:1-4 A1:P1|A2:P1"
+ " C1-4:P1:S+ P1 . . T C1-4 . . 0 A1:1-4|A2:1-4 A1:P1|A2:P-1"
# Clearing of cpuset.cpus with a preset cpuset.cpus.exclusive shouldn't
# affect cpuset.cpus.exclusive.effective.
- " C1-4:X3:S+ C1:X3 . . . C . . 0 A2:1-4,XA2:3"
+ " C1-4:X3:S+ C1:X3 . . . C . . 0 A2:1-4|XA2:3"
# old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS
# ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
# Failure cases:
# A task cannot be added to a partition with no cpu
- "C2-3:P1:S+ C3:P1 . . O2=0:T . . . 1 A1:,A2:3 A1:P1,A2:P1"
+ "C2-3:P1:S+ C3:P1 . . O2=0:T . . . 1 A1:|A2:3 A1:P1|A2:P1"
# Changes to cpuset.cpus.exclusive that violate exclusivity rule is rejected
- " C0-3 . . C4-5 X0-3 . . X3-5 1 A1:0-3,B1:4-5"
+ " C0-3 . . C4-5 X0-3 . . X3-5 1 A1:0-3|B1:4-5"
# cpuset.cpus cannot be a subset of sibling cpuset.cpus.exclusive
- " C0-3 . . C4-5 X3-5 . . . 1 A1:0-3,B1:4-5"
+ " C0-3 . . C4-5 X3-5 . . . 1 A1:0-3|B1:4-5"
)
#
@@ -412,6 +428,7 @@ write_cpu_online()
CPU=${1%=*}
VAL=${1#*=}
CPUFILE=//sys/devices/system/cpu/cpu${CPU}/online
+ echo $VAL > $CPUFILE || return 1
if [[ $VAL -eq 0 ]]
then
OFFLINE_CPUS="$OFFLINE_CPUS $CPU"
@@ -421,7 +438,6 @@ write_cpu_online()
sort | uniq -u)
}
fi
- echo $VAL > $CPUFILE
pause 0.05
}
@@ -453,25 +469,26 @@ set_ctrl_state()
PFILE=$CGRP/cpuset.cpus.partition
CFILE=$CGRP/cpuset.cpus
XFILE=$CGRP/cpuset.cpus.exclusive
- S=$(expr substr $CMD 1 1)
- if [[ $S = S ]]
- then
- PREFIX=${CMD#?}
+ case $CMD in
+ S*) PREFIX=${CMD#?}
COMM="echo ${PREFIX}${CTRL} > $SFILE"
eval $COMM $REDIRECT
- elif [[ $S = X ]]
- then
+ ;;
+ X*)
CPUS=${CMD#?}
COMM="echo $CPUS > $XFILE"
eval $COMM $REDIRECT
- elif [[ $S = C ]]
- then
- CPUS=${CMD#?}
+ ;;
+ CX*)
+ CPUS=${CMD#??}
+ COMM="echo $CPUS > $CFILE; echo $CPUS > $XFILE"
+ eval $COMM $REDIRECT
+ ;;
+ C*) CPUS=${CMD#?}
COMM="echo $CPUS > $CFILE"
eval $COMM $REDIRECT
- elif [[ $S = P ]]
- then
- VAL=${CMD#?}
+ ;;
+ P*) VAL=${CMD#?}
case $VAL in
0) VAL=member
;;
@@ -486,15 +503,18 @@ set_ctrl_state()
esac
COMM="echo $VAL > $PFILE"
eval $COMM $REDIRECT
- elif [[ $S = O ]]
- then
- VAL=${CMD#?}
- write_cpu_online $VAL
- elif [[ $S = T ]]
- then
- COMM="echo 0 > $TFILE"
+ ;;
+ O*) VAL=${CMD#?}
+ COMM="write_cpu_online $VAL"
eval $COMM $REDIRECT
- fi
+ ;;
+ T*) COMM="echo 0 > $TFILE"
+ eval $COMM $REDIRECT
+ ;;
+ *) echo "Unknown command: $CMD"
+ exit 1
+ ;;
+ esac
RET=$?
[[ $RET -ne 0 ]] && {
[[ -n "$SHOWERR" ]] && {
@@ -532,21 +552,18 @@ online_cpus()
}
#
-# Return 1 if the list of effective cpus isn't the same as the initial list.
+# Remove all the test cgroup directories
#
reset_cgroup_states()
{
echo 0 > $CGROUP2/cgroup.procs
online_cpus
- rmdir A1/A2/A3 A1/A2 A1 B1 > /dev/null 2>&1
- pause 0.02
- set_ctrl_state . R-
- pause 0.01
+ rmdir $RESET_LIST > /dev/null 2>&1
}
dump_states()
{
- for DIR in . A1 A1/A2 A1/A2/A3 B1
+ for DIR in $CGROUP_LIST
do
CPUS=$DIR/cpuset.cpus
ECPUS=$DIR/cpuset.cpus.effective
@@ -565,18 +582,34 @@ dump_states()
done
}
+#
+# Set the actual cgroup directory into $CGRP_DIR
+# $1 - cgroup name
+#
+set_cgroup_dir()
+{
+ CGRP_DIR=$1
+ [[ $CGRP_DIR = A2 ]] && CGRP_DIR=A1/A2
+ [[ $CGRP_DIR = A3 ]] && CGRP_DIR=A1/A2/A3
+ [[ $CGRP_DIR = c11 ]] && CGRP_DIR=p1/c11
+ [[ $CGRP_DIR = c12 ]] && CGRP_DIR=p1/c12
+ [[ $CGRP_DIR = c21 ]] && CGRP_DIR=p2/c21
+ [[ $CGRP_DIR = c22 ]] && CGRP_DIR=p2/c22
+}
+
#
# Check effective cpus
-# $1 - check string, format: <cgroup>:<cpu-list>[,<cgroup>:<cpu-list>]*
+# $1 - check string, format: <cgroup>:<cpu-list>[|<cgroup>:<cpu-list>]*
#
check_effective_cpus()
{
CHK_STR=$1
- for CHK in $(echo $CHK_STR | sed -e "s/,/ /g")
+ for CHK in $(echo $CHK_STR | sed -e "s/|/ /g")
do
set -- $(echo $CHK | sed -e "s/:/ /g")
CGRP=$1
- CPUS=$2
+ EXPECTED_CPUS=$2
+ ACTUAL_CPUS=
if [[ $CGRP = X* ]]
then
CGRP=${CGRP#X}
@@ -584,41 +617,39 @@ check_effective_cpus()
else
FILE=cpuset.cpus.effective
fi
- [[ $CGRP = A2 ]] && CGRP=A1/A2
- [[ $CGRP = A3 ]] && CGRP=A1/A2/A3
- [[ -e $CGRP/$FILE ]] || return 1
- [[ $CPUS = $(cat $CGRP/$FILE) ]] || return 1
+ set_cgroup_dir $CGRP
+ [[ -e $CGRP_DIR/$FILE ]] || return 1
+ ACTUAL_CPUS=$(cat $CGRP_DIR/$FILE)
+ [[ $EXPECTED_CPUS = $ACTUAL_CPUS ]] || return 1
done
}
#
# Check cgroup states
-# $1 - check string, format: <cgroup>:<state>[,<cgroup>:<state>]*
+# $1 - check string, format: <cgroup>:<state>[|<cgroup>:<state>]*
#
check_cgroup_states()
{
CHK_STR=$1
- for CHK in $(echo $CHK_STR | sed -e "s/,/ /g")
+ for CHK in $(echo $CHK_STR | sed -e "s/|/ /g")
do
set -- $(echo $CHK | sed -e "s/:/ /g")
CGRP=$1
- CGRP_DIR=$CGRP
- STATE=$2
+ EXPECTED_STATE=$2
FILE=
- EVAL=$(expr substr $STATE 2 2)
- [[ $CGRP = A2 ]] && CGRP_DIR=A1/A2
- [[ $CGRP = A3 ]] && CGRP_DIR=A1/A2/A3
+ EVAL=$(expr substr $EXPECTED_STATE 2 2)
- case $STATE in
+ set_cgroup_dir $CGRP
+ case $EXPECTED_STATE in
P*) FILE=$CGRP_DIR/cpuset.cpus.partition
;;
- *) echo "Unknown state: $STATE!"
+ *) echo "Unknown state: $EXPECTED_STATE!"
exit 1
;;
esac
- VAL=$(cat $FILE)
+ ACTUAL_STATE=$(cat $FILE)
- case "$VAL" in
+ case "$ACTUAL_STATE" in
member) VAL=0
;;
root) VAL=1
@@ -642,7 +673,7 @@ check_cgroup_states()
[[ $VAL -eq 1 && $VERBOSE -gt 0 ]] && {
DOMS=$(cat $CGRP_DIR/cpuset.cpus.effective)
[[ -n "$DOMS" ]] &&
- echo " [$CGRP] sched-domain: $DOMS" > $CONSOLE
+ echo " [$CGRP_DIR] sched-domain: $DOMS" > $CONSOLE
}
done
return 0
@@ -665,22 +696,22 @@ check_cgroup_states()
#
check_isolcpus()
{
- EXPECT_VAL=$1
- ISOLCPUS=
+ EXPECTED_ISOLCPUS=$1
+ ISCPUS=${CGROUP2}/cpuset.cpus.isolated
+ ISOLCPUS=$(cat $ISCPUS)
LASTISOLCPU=
SCHED_DOMAINS=/sys/kernel/debug/sched/domains
- ISCPUS=${CGROUP2}/cpuset.cpus.isolated
- if [[ $EXPECT_VAL = . ]]
+ if [[ $EXPECTED_ISOLCPUS = . ]]
then
- EXPECT_VAL=
- EXPECT_VAL2=
- elif [[ $(expr $EXPECT_VAL : ".*,.*") > 0 ]]
+ EXPECTED_ISOLCPUS=
+ EXPECTED_SDOMAIN=
+ elif [[ $(expr $EXPECTED_ISOLCPUS : ".*|.*") > 0 ]]
then
- set -- $(echo $EXPECT_VAL | sed -e "s/,/ /g")
- EXPECT_VAL=$1
- EXPECT_VAL2=$2
+ set -- $(echo $EXPECTED_ISOLCPUS | sed -e "s/|/ /g")
+ EXPECTED_ISOLCPUS=$2
+ EXPECTED_SDOMAIN=$1
else
- EXPECT_VAL2=$EXPECT_VAL
+ EXPECTED_SDOMAIN=$EXPECTED_ISOLCPUS
fi
#
@@ -689,20 +720,21 @@ check_isolcpus()
# to make appending those CPUs easier.
#
[[ -n "$BOOT_ISOLCPUS" ]] && {
- EXPECT_VAL=${EXPECT_VAL:+${EXPECT_VAL},}${BOOT_ISOLCPUS}
- EXPECT_VAL2=${EXPECT_VAL2:+${EXPECT_VAL2},}${BOOT_ISOLCPUS}
+ EXPECTED_ISOLCPUS=${EXPECTED_ISOLCPUS:+${EXPECTED_ISOLCPUS},}${BOOT_ISOLCPUS}
+ EXPECTED_SDOMAIN=${EXPECTED_SDOMAIN:+${EXPECTED_SDOMAIN},}${BOOT_ISOLCPUS}
}
#
# Check cpuset.cpus.isolated cpumask
#
- [[ "$EXPECT_VAL2" != "$ISOLCPUS" ]] && {
+ [[ "$EXPECTED_ISOLCPUS" != "$ISOLCPUS" ]] && {
# Take a 50ms pause and try again
pause 0.05
ISOLCPUS=$(cat $ISCPUS)
}
- [[ "$EXPECT_VAL2" != "$ISOLCPUS" ]] && return 1
+ [[ "$EXPECTED_ISOLCPUS" != "$ISOLCPUS" ]] && return 1
ISOLCPUS=
+ EXPECTED_ISOLCPUS=$EXPECTED_SDOMAIN
#
# Use the sched domain in debugfs to check isolated CPUs, if available
@@ -736,7 +768,7 @@ check_isolcpus()
done
[[ "$ISOLCPUS" = *- ]] && ISOLCPUS=${ISOLCPUS}$LASTISOLCPU
- [[ "$EXPECT_VAL" = "$ISOLCPUS" ]]
+ [[ "$EXPECTED_SDOMAIN" = "$ISOLCPUS" ]]
}
test_fail()
@@ -773,6 +805,63 @@ null_isolcpus_check()
exit 1
}
+#
+# Check state transition test result
+# $1 - Test number
+# $2 - Expected effective CPU values
+# $3 - Expected partition states
+# $4 - Expected isolated CPUs
+#
+check_test_results()
+{
+ _NR=$1
+ _ECPUS="$2"
+ _PSTATES="$3"
+ _ISOLCPUS="$4"
+
+ [[ -n "$_ECPUS" && "$_ECPUS" != . ]] && {
+ check_effective_cpus $_ECPUS
+ [[ $? -ne 0 ]] && test_fail $_NR "effective CPU" \
+ "Cgroup $CGRP: expected $EXPECTED_CPUS, got $ACTUAL_CPUS"
+ }
+
+ [[ -n "$_PSTATES" && "$_PSTATES" != . ]] && {
+ check_cgroup_states $_PSTATES
+ [[ $? -ne 0 ]] && test_fail $_NR states \
+ "Cgroup $CGRP: expected $EXPECTED_STATE, got $ACTUAL_STATE"
+ }
+
+ # Compare the expected isolated CPUs with the actual ones,
+ # if available
+ [[ -n "$_ISOLCPUS" ]] && {
+ check_isolcpus $_ISOLCPUS
+ [[ $? -ne 0 ]] && {
+ [[ -n "$BOOT_ISOLCPUS" ]] && _ISOLCPUS=${_ISOLCPUS},${BOOT_ISOLCPUS}
+ test_fail $_NR "isolated CPU" \
+ "Expect $_ISOLCPUS, get $ISOLCPUS instead"
+ }
+ }
+ reset_cgroup_states
+ #
+ # Check to see if effective cpu list changes
+ #
+ _NEWLIST=$(cat $CGROUP2/cpuset.cpus.effective)
+ RETRY=0
+ while [[ $_NEWLIST != $CPULIST && $RETRY -lt 8 ]]
+ do
+ # Wait a bit longer & recheck a few times
+ pause 0.02
+ ((RETRY++))
+ _NEWLIST=$(cat $CGROUP2/cpuset.cpus.effective)
+ done
+ [[ $_NEWLIST != $CPULIST ]] && {
+ echo "Effective cpus changed to $_NEWLIST after test $_NR!"
+ exit 1
+ }
+ null_isolcpus_check
+ [[ $VERBOSE -gt 0 ]] && echo "Test $I done."
+}
+
#
# Run cpuset state transition test
# $1 - test matrix name
@@ -785,6 +874,8 @@ run_state_test()
{
TEST=$1
CONTROLLER=cpuset
+ CGROUP_LIST=". A1 A1/A2 A1/A2/A3 B1"
+ RESET_LIST="A1/A2/A3 A1/A2 A1 B1"
I=0
eval CNT="\${#$TEST[@]}"
@@ -824,45 +915,7 @@ run_state_test()
[[ $RETVAL -ne $RESULT ]] && test_fail $I result
- [[ -n "$ECPUS" && "$ECPUS" != . ]] && {
- check_effective_cpus $ECPUS
- [[ $? -ne 0 ]] && test_fail $I "effective CPU"
- }
-
- [[ -n "$STATES" && "$STATES" != . ]] && {
- check_cgroup_states $STATES
- [[ $? -ne 0 ]] && test_fail $I states
- }
-
- # Compare the expected isolated CPUs with the actual ones,
- # if available
- [[ -n "$ICPUS" ]] && {
- check_isolcpus $ICPUS
- [[ $? -ne 0 ]] && {
- [[ -n "$BOOT_ISOLCPUS" ]] && ICPUS=${ICPUS},${BOOT_ISOLCPUS}
- test_fail $I "isolated CPU" \
- "Expect $ICPUS, get $ISOLCPUS instead"
- }
- }
- reset_cgroup_states
- #
- # Check to see if effective cpu list changes
- #
- NEWLIST=$(cat cpuset.cpus.effective)
- RETRY=0
- while [[ $NEWLIST != $CPULIST && $RETRY -lt 8 ]]
- do
- # Wait a bit longer & recheck a few times
- pause 0.02
- ((RETRY++))
- NEWLIST=$(cat cpuset.cpus.effective)
- done
- [[ $NEWLIST != $CPULIST ]] && {
- echo "Effective cpus changed to $NEWLIST after test $I!"
- exit 1
- }
- null_isolcpus_check
- [[ $VERBOSE -gt 0 ]] && echo "Test $I done."
+ check_test_results $I "$ECPUS" "$STATES" "$ICPUS"
((I++))
done
echo "All $I tests of $TEST PASSED."
@@ -932,6 +985,7 @@ test_isolated()
echo $$ > $CGROUP2/cgroup.procs
[[ -d A1 ]] && rmdir A1
null_isolcpus_check
+ pause 0.05
}
#
@@ -997,6 +1051,8 @@ test_inotify()
else
echo "Inotify test PASSED"
fi
+ echo member > cpuset.cpus.partition
+ echo "" > cpuset.cpus
}
trap cleanup 0 2 3 6
diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index a4e5b8613bab..4c652d2a8076 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -75,7 +75,7 @@ static inline void __kselftest_memset_safe(void *s, int c, size_t n)
memset(s, c, n);
}
-#define KSELFTEST_PRIO_TEST_F 20000
+#define KSELFTEST_PRIO_TEST 20000
#define KSELFTEST_PRIO_XFAIL 20001
#define TEST_TIMEOUT_DEFAULT 30
@@ -196,7 +196,7 @@ static inline void __kselftest_memset_safe(void *s, int c, size_t n)
.fixture = &_fixture_global, \
.termsig = _signal, \
.timeout = TEST_TIMEOUT_DEFAULT, }; \
- static void __attribute__((constructor)) _register_##test_name(void) \
+ static void __attribute__((constructor(KSELFTEST_PRIO_TEST))) _register_##test_name(void) \
{ \
__register_test(&_##test_name##_object); \
} \
@@ -240,7 +240,7 @@ static inline void __kselftest_memset_safe(void *s, int c, size_t n)
FIXTURE_VARIANT(fixture_name); \
static struct __fixture_metadata _##fixture_name##_fixture_object = \
{ .name = #fixture_name, }; \
- static void __attribute__((constructor)) \
+ static void __attribute__((constructor(KSELFTEST_PRIO_TEST))) \
_register_##fixture_name##_data(void) \
{ \
__register_fixture(&_##fixture_name##_fixture_object); \
@@ -366,7 +366,7 @@ static inline void __kselftest_memset_safe(void *s, int c, size_t n)
_##fixture_name##_##variant_name##_object = \
{ .name = #variant_name, \
.data = &_##fixture_name##_##variant_name##_variant}; \
- static void __attribute__((constructor)) \
+ static void __attribute__((constructor(KSELFTEST_PRIO_TEST))) \
_register_##fixture_name##_##variant_name(void) \
{ \
__register_fixture_variant(&_##fixture_name##_fixture_object, \
@@ -467,8 +467,8 @@ static inline void __kselftest_memset_safe(void *s, int c, size_t n)
} \
__test_check_assert(_metadata); \
} \
- static struct __test_metadata *_##fixture_name##_##test_name##_object; \
- static void __attribute__((constructor(KSELFTEST_PRIO_TEST_F))) \
+ static struct __test_metadata *_##fixture_name##_##test_name##_object __maybe_unused; \
+ static void __attribute__((constructor(KSELFTEST_PRIO_TEST))) \
_register_##fixture_name##_##test_name(void) \
{ \
struct __test_metadata *object = mmap(NULL, sizeof(*object), \
@@ -1335,7 +1335,7 @@ static int test_harness_run(int argc, char **argv)
return KSFT_FAIL;
}
-static void __attribute__((constructor)) __constructor_order_first(void)
+static void __attribute__((constructor(KSELFTEST_PRIO_TEST))) __constructor_order_first(void)
{
__constructor_order_forward = true;
}
diff --git a/tools/testing/selftests/lsm/common.c b/tools/testing/selftests/lsm/common.c
index 9ad258912646..927dce4f04cb 100644
--- a/tools/testing/selftests/lsm/common.c
+++ b/tools/testing/selftests/lsm/common.c
@@ -76,7 +76,7 @@ int attr_lsm_count(void)
return 0;
if (read_sysfs_lsms(names, sysconf(_SC_PAGESIZE)))
- return 0;
+ goto out;
if (strstr(names, "selinux"))
count++;
@@ -85,5 +85,7 @@ int attr_lsm_count(void)
if (strstr(names, "apparmor"))
count++;
+out:
+ free(names);
return count;
}
diff --git a/tools/testing/selftests/mm/cow.c b/tools/testing/selftests/mm/cow.c
index d87c5b1763ff..c0ca9483c448 100644
--- a/tools/testing/selftests/mm/cow.c
+++ b/tools/testing/selftests/mm/cow.c
@@ -112,9 +112,12 @@ struct comm_pipes {
static int setup_comm_pipes(struct comm_pipes *comm_pipes)
{
- if (pipe(comm_pipes->child_ready) < 0)
+ if (pipe(comm_pipes->child_ready) < 0) {
+ ksft_perror("pipe()");
return -errno;
+ }
if (pipe(comm_pipes->parent_ready) < 0) {
+ ksft_perror("pipe()");
close(comm_pipes->child_ready[0]);
close(comm_pipes->child_ready[1]);
return -errno;
@@ -207,13 +210,14 @@ static void do_test_cow_in_parent(char *mem, size_t size, bool do_mprotect,
ret = setup_comm_pipes(&comm_pipes);
if (ret) {
- ksft_test_result_fail("pipe() failed\n");
+ log_test_result(KSFT_FAIL);
return;
}
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
} else if (!ret) {
exit(fn(mem, size, &comm_pipes));
@@ -228,9 +232,18 @@ static void do_test_cow_in_parent(char *mem, size_t size, bool do_mprotect,
* write-faults by directly mapping pages writable.
*/
ret = mprotect(mem, size, PROT_READ);
- ret |= mprotect(mem, size, PROT_READ|PROT_WRITE);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
+ write(comm_pipes.parent_ready[1], "0", 1);
+ wait(&ret);
+ goto close_comm_pipes;
+ }
+
+ ret = mprotect(mem, size, PROT_READ|PROT_WRITE);
+ if (ret) {
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
write(comm_pipes.parent_ready[1], "0", 1);
wait(&ret);
goto close_comm_pipes;
@@ -248,16 +261,16 @@ static void do_test_cow_in_parent(char *mem, size_t size, bool do_mprotect,
ret = -EINVAL;
if (!ret) {
- ksft_test_result_pass("No leak from parent into child\n");
+ log_test_result(KSFT_PASS);
} else if (xfail) {
/*
* With hugetlb, some vmsplice() tests are currently expected to
* fail because (a) harder to fix and (b) nobody really cares.
* Flag them as expected failure for now.
*/
- ksft_test_result_xfail("Leak from parent into child\n");
+ log_test_result(KSFT_XFAIL);
} else {
- ksft_test_result_fail("Leak from parent into child\n");
+ log_test_result(KSFT_FAIL);
}
close_comm_pipes:
close_comm_pipes(&comm_pipes);
@@ -306,26 +319,29 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size,
ret = setup_comm_pipes(&comm_pipes);
if (ret) {
- ksft_test_result_fail("pipe() failed\n");
+ log_test_result(KSFT_FAIL);
goto free;
}
if (pipe(fds) < 0) {
- ksft_test_result_fail("pipe() failed\n");
+ ksft_perror("pipe() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
if (before_fork) {
transferred = vmsplice(fds[1], &iov, 1, 0);
if (transferred <= 0) {
- ksft_test_result_fail("vmsplice() failed\n");
+ ksft_print_msg("vmsplice() failed\n");
+ log_test_result(KSFT_FAIL);
goto close_pipe;
}
}
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed\n");
+ log_test_result(KSFT_FAIL);
goto close_pipe;
} else if (!ret) {
write(comm_pipes.child_ready[1], "0", 1);
@@ -339,7 +355,8 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size,
if (!before_fork) {
transferred = vmsplice(fds[1], &iov, 1, 0);
if (transferred <= 0) {
- ksft_test_result_fail("vmsplice() failed\n");
+ ksft_perror("vmsplice() failed");
+ log_test_result(KSFT_FAIL);
wait(&ret);
goto close_pipe;
}
@@ -348,7 +365,8 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size,
while (read(comm_pipes.child_ready[0], &buf, 1) != 1)
;
if (munmap(mem, size) < 0) {
- ksft_test_result_fail("munmap() failed\n");
+ ksft_perror("munmap() failed");
+ log_test_result(KSFT_FAIL);
goto close_pipe;
}
write(comm_pipes.parent_ready[1], "0", 1);
@@ -356,7 +374,8 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size,
/* Wait until the child is done writing. */
wait(&ret);
if (!WIFEXITED(ret)) {
- ksft_test_result_fail("wait() failed\n");
+ ksft_perror("wait() failed");
+ log_test_result(KSFT_FAIL);
goto close_pipe;
}
@@ -364,22 +383,23 @@ static void do_test_vmsplice_in_parent(char *mem, size_t size,
for (total = 0; total < transferred; total += cur) {
cur = read(fds[0], new + total, transferred - total);
if (cur < 0) {
- ksft_test_result_fail("read() failed\n");
+ ksft_perror("read() failed");
+ log_test_result(KSFT_FAIL);
goto close_pipe;
}
}
if (!memcmp(old, new, transferred)) {
- ksft_test_result_pass("No leak from child into parent\n");
+ log_test_result(KSFT_PASS);
} else if (xfail) {
/*
* With hugetlb, some vmsplice() tests are currently expected to
* fail because (a) harder to fix and (b) nobody really cares.
* Flag them as expected failure for now.
*/
- ksft_test_result_xfail("Leak from child into parent\n");
+ log_test_result(KSFT_XFAIL);
} else {
- ksft_test_result_fail("Leak from child into parent\n");
+ log_test_result(KSFT_FAIL);
}
close_pipe:
close(fds[0]);
@@ -416,13 +436,14 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
ret = setup_comm_pipes(&comm_pipes);
if (ret) {
- ksft_test_result_fail("pipe() failed\n");
+ log_test_result(KSFT_FAIL);
return;
}
file = tmpfile();
if (!file) {
- ksft_test_result_fail("tmpfile() failed\n");
+ ksft_perror("tmpfile() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
fd = fileno(file);
@@ -430,14 +451,16 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
tmp = malloc(size);
if (!tmp) {
- ksft_test_result_fail("malloc() failed\n");
+ ksft_print_msg("malloc() failed\n");
+ log_test_result(KSFT_FAIL);
goto close_file;
}
/* Skip on errors, as we might just lack kernel support. */
ret = io_uring_queue_init(1, &ring, 0);
if (ret < 0) {
- ksft_test_result_skip("io_uring_queue_init() failed\n");
+ ksft_print_msg("io_uring_queue_init() failed\n");
+ log_test_result(KSFT_SKIP);
goto free_tmp;
}
@@ -452,7 +475,8 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
iov.iov_len = size;
ret = io_uring_register_buffers(&ring, &iov, 1);
if (ret) {
- ksft_test_result_skip("io_uring_register_buffers() failed\n");
+ ksft_print_msg("io_uring_register_buffers() failed\n");
+ log_test_result(KSFT_SKIP);
goto queue_exit;
}
@@ -463,7 +487,8 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
*/
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed");
+ log_test_result(KSFT_FAIL);
goto unregister_buffers;
} else if (!ret) {
write(comm_pipes.child_ready[1], "0", 1);
@@ -483,10 +508,17 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
* if the page is mapped R/O vs. R/W).
*/
ret = mprotect(mem, size, PROT_READ);
+ if (ret) {
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
+ goto unregister_buffers;
+ }
+
clear_softdirty();
- ret |= mprotect(mem, size, PROT_READ | PROT_WRITE);
+ ret = mprotect(mem, size, PROT_READ | PROT_WRITE);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto unregister_buffers;
}
}
@@ -498,25 +530,29 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
memset(mem, 0xff, size);
sqe = io_uring_get_sqe(&ring);
if (!sqe) {
- ksft_test_result_fail("io_uring_get_sqe() failed\n");
+ ksft_print_msg("io_uring_get_sqe() failed\n");
+ log_test_result(KSFT_FAIL);
goto quit_child;
}
io_uring_prep_write_fixed(sqe, fd, mem, size, 0, 0);
ret = io_uring_submit(&ring);
if (ret < 0) {
- ksft_test_result_fail("io_uring_submit() failed\n");
+ ksft_print_msg("io_uring_submit() failed\n");
+ log_test_result(KSFT_FAIL);
goto quit_child;
}
ret = io_uring_wait_cqe(&ring, &cqe);
if (ret < 0) {
- ksft_test_result_fail("io_uring_wait_cqe() failed\n");
+ ksft_print_msg("io_uring_wait_cqe() failed\n");
+ log_test_result(KSFT_FAIL);
goto quit_child;
}
if (cqe->res != size) {
- ksft_test_result_fail("write_fixed failed\n");
+ ksft_print_msg("write_fixed failed\n");
+ log_test_result(KSFT_FAIL);
goto quit_child;
}
io_uring_cqe_seen(&ring, cqe);
@@ -526,15 +562,18 @@ static void do_test_iouring(char *mem, size_t size, bool use_fork)
while (total < size) {
cur = pread(fd, tmp + total, size - total, total);
if (cur < 0) {
- ksft_test_result_fail("pread() failed\n");
+ ksft_print_msg("pread() failed\n");
+ log_test_result(KSFT_FAIL);
goto quit_child;
}
total += cur;
}
/* Finally, check if we read what we expected. */
- ksft_test_result(!memcmp(mem, tmp, size),
- "Longterm R/W pin is reliable\n");
+ if (!memcmp(mem, tmp, size))
+ log_test_result(KSFT_PASS);
+ else
+ log_test_result(KSFT_FAIL);
quit_child:
if (use_fork) {
@@ -582,19 +621,21 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,
int ret;
if (gup_fd < 0) {
- ksft_test_result_skip("gup_test not available\n");
+ ksft_print_msg("gup_test not available\n");
+ log_test_result(KSFT_SKIP);
return;
}
tmp = malloc(size);
if (!tmp) {
- ksft_test_result_fail("malloc() failed\n");
+ ksft_print_msg("malloc() failed\n");
+ log_test_result(KSFT_FAIL);
return;
}
ret = setup_comm_pipes(&comm_pipes);
if (ret) {
- ksft_test_result_fail("pipe() failed\n");
+ log_test_result(KSFT_FAIL);
goto free_tmp;
}
@@ -609,7 +650,8 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,
*/
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
} else if (!ret) {
write(comm_pipes.child_ready[1], "0", 1);
@@ -646,7 +688,8 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,
clear_softdirty();
ret |= mprotect(mem, size, PROT_READ | PROT_WRITE);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
break;
@@ -661,9 +704,11 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,
ret = ioctl(gup_fd, PIN_LONGTERM_TEST_START, &args);
if (ret) {
if (errno == EINVAL)
- ksft_test_result_skip("PIN_LONGTERM_TEST_START failed\n");
+ ret = KSFT_SKIP;
else
- ksft_test_result_fail("PIN_LONGTERM_TEST_START failed\n");
+ ret = KSFT_FAIL;
+ ksft_perror("PIN_LONGTERM_TEST_START failed");
+ log_test_result(ret);
goto wait;
}
@@ -676,22 +721,26 @@ static void do_test_ro_pin(char *mem, size_t size, enum ro_pin_test test,
*/
tmp_val = (__u64)(uintptr_t)tmp;
ret = ioctl(gup_fd, PIN_LONGTERM_TEST_READ, &tmp_val);
- if (ret)
- ksft_test_result_fail("PIN_LONGTERM_TEST_READ failed\n");
- else
- ksft_test_result(!memcmp(mem, tmp, size),
- "Longterm R/O pin is reliable\n");
+ if (ret) {
+ ksft_perror("PIN_LONGTERM_TEST_READ failed");
+ log_test_result(KSFT_FAIL);
+ } else {
+ if (!memcmp(mem, tmp, size))
+ log_test_result(KSFT_PASS);
+ else
+ log_test_result(KSFT_FAIL);
+ }
ret = ioctl(gup_fd, PIN_LONGTERM_TEST_STOP);
if (ret)
- ksft_print_msg("[INFO] PIN_LONGTERM_TEST_STOP failed\n");
+ ksft_perror("PIN_LONGTERM_TEST_STOP failed");
wait:
switch (test) {
case RO_PIN_TEST_SHARED:
write(comm_pipes.parent_ready[1], "0", 1);
wait(&ret);
if (!WIFEXITED(ret))
- ksft_print_msg("[INFO] wait() failed\n");
+ ksft_perror("wait() failed");
break;
default:
break;
@@ -746,14 +795,16 @@ static void do_run_with_base_page(test_fn fn, bool swapout)
mem = mmap(NULL, pagesize, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
return;
}
ret = madvise(mem, pagesize, MADV_NOHUGEPAGE);
/* Ignore if not around on a kernel. */
if (ret && errno != EINVAL) {
- ksft_test_result_fail("MADV_NOHUGEPAGE failed\n");
+ ksft_perror("MADV_NOHUGEPAGE failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -763,7 +814,8 @@ static void do_run_with_base_page(test_fn fn, bool swapout)
if (swapout) {
madvise(mem, pagesize, MADV_PAGEOUT);
if (!pagemap_is_swapped(pagemap_fd, mem)) {
- ksft_test_result_skip("MADV_PAGEOUT did not work, is swap enabled?\n");
+ ksft_print_msg("MADV_PAGEOUT did not work, is swap enabled?\n");
+ log_test_result(KSFT_SKIP);
goto munmap;
}
}
@@ -775,13 +827,13 @@ static void do_run_with_base_page(test_fn fn, bool swapout)
static void run_with_base_page(test_fn fn, const char *desc)
{
- ksft_print_msg("[RUN] %s ... with base page\n", desc);
+ log_test_start("%s ... with base page", desc);
do_run_with_base_page(fn, false);
}
static void run_with_base_page_swap(test_fn fn, const char *desc)
{
- ksft_print_msg("[RUN] %s ... with swapped out base page\n", desc);
+ log_test_start("%s ... with swapped out base page", desc);
do_run_with_base_page(fn, true);
}
@@ -807,7 +859,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
mmap_mem = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (mmap_mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
return;
}
@@ -816,7 +869,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
ret = madvise(mem, thpsize, MADV_HUGEPAGE);
if (ret) {
- ksft_test_result_fail("MADV_HUGEPAGE failed\n");
+ ksft_perror("MADV_HUGEPAGE failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -826,7 +880,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
*/
mem[0] = 1;
if (!pagemap_is_populated(pagemap_fd, mem + thpsize - pagesize)) {
- ksft_test_result_skip("Did not get a THP populated\n");
+ ksft_print_msg("Did not get a THP populated\n");
+ log_test_result(KSFT_SKIP);
goto munmap;
}
memset(mem, 1, thpsize);
@@ -846,12 +901,14 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
*/
ret = mprotect(mem + pagesize, pagesize, PROT_READ);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
ret = mprotect(mem + pagesize, pagesize, PROT_READ | PROT_WRITE);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
break;
@@ -863,7 +920,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
*/
ret = madvise(mem + pagesize, thpsize - pagesize, MADV_DONTNEED);
if (ret) {
- ksft_test_result_fail("MADV_DONTNEED failed\n");
+ ksft_perror("MADV_DONTNEED failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
size = pagesize;
@@ -877,13 +935,15 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
mremap_mem = mmap(NULL, mremap_size, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (mremap_mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
tmp = mremap(mem + mremap_size, mremap_size, mremap_size,
MREMAP_MAYMOVE | MREMAP_FIXED, mremap_mem);
if (tmp != mremap_mem) {
- ksft_test_result_fail("mremap() failed\n");
+ ksft_perror("mremap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
size = mremap_size;
@@ -896,12 +956,14 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
*/
ret = madvise(mem + pagesize, thpsize - pagesize, MADV_DONTFORK);
if (ret) {
- ksft_test_result_fail("MADV_DONTFORK failed\n");
+ ksft_perror("MADV_DONTFORK failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
} else if (!ret) {
exit(0);
@@ -910,7 +972,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
/* Allow for sharing all pages again. */
ret = madvise(mem + pagesize, thpsize - pagesize, MADV_DOFORK);
if (ret) {
- ksft_test_result_fail("MADV_DOFORK failed\n");
+ ksft_perror("MADV_DOFORK failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
break;
@@ -924,7 +987,8 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
case THP_RUN_SINGLE_PTE_SWAPOUT:
madvise(mem, size, MADV_PAGEOUT);
if (!range_is_swapped(mem, size)) {
- ksft_test_result_skip("MADV_PAGEOUT did not work, is swap enabled?\n");
+ ksft_print_msg("MADV_PAGEOUT did not work, is swap enabled?\n");
+ log_test_result(KSFT_SKIP);
goto munmap;
}
break;
@@ -941,56 +1005,56 @@ static void do_run_with_thp(test_fn fn, enum thp_run thp_run, size_t thpsize)
static void run_with_thp(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with THP (%zu kB)\n",
+ log_test_start("%s ... with THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PMD, size);
}
static void run_with_thp_swap(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with swapped-out THP (%zu kB)\n",
+ log_test_start("%s ... with swapped-out THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PMD_SWAPOUT, size);
}
static void run_with_pte_mapped_thp(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with PTE-mapped THP (%zu kB)\n",
+ log_test_start("%s ... with PTE-mapped THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PTE, size);
}
static void run_with_pte_mapped_thp_swap(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with swapped-out, PTE-mapped THP (%zu kB)\n",
+ log_test_start("%s ... with swapped-out, PTE-mapped THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PTE_SWAPOUT, size);
}
static void run_with_single_pte_of_thp(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with single PTE of THP (%zu kB)\n",
+ log_test_start("%s ... with single PTE of THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_SINGLE_PTE, size);
}
static void run_with_single_pte_of_thp_swap(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with single PTE of swapped-out THP (%zu kB)\n",
+ log_test_start("%s ... with single PTE of swapped-out THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_SINGLE_PTE_SWAPOUT, size);
}
static void run_with_partial_mremap_thp(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with partially mremap()'ed THP (%zu kB)\n",
+ log_test_start("%s ... with partially mremap()'ed THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PARTIAL_MREMAP, size);
}
static void run_with_partial_shared_thp(test_fn fn, const char *desc, size_t size)
{
- ksft_print_msg("[RUN] %s ... with partially shared THP (%zu kB)\n",
+ log_test_start("%s ... with partially shared THP (%zu kB)",
desc, size / 1024);
do_run_with_thp(fn, THP_RUN_PARTIAL_SHARED, size);
}
@@ -1000,14 +1064,15 @@ static void run_with_hugetlb(test_fn fn, const char *desc, size_t hugetlbsize)
int flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB;
char *mem, *dummy;
- ksft_print_msg("[RUN] %s ... with hugetlb (%zu kB)\n", desc,
+ log_test_start("%s ... with hugetlb (%zu kB)", desc,
hugetlbsize / 1024);
flags |= __builtin_ctzll(hugetlbsize) << MAP_HUGE_SHIFT;
mem = mmap(NULL, hugetlbsize, PROT_READ | PROT_WRITE, flags, -1, 0);
if (mem == MAP_FAILED) {
- ksft_test_result_skip("need more free huge pages\n");
+ ksft_perror("need more free huge pages");
+ log_test_result(KSFT_SKIP);
return;
}
@@ -1020,7 +1085,8 @@ static void run_with_hugetlb(test_fn fn, const char *desc, size_t hugetlbsize)
*/
dummy = mmap(NULL, hugetlbsize, PROT_READ | PROT_WRITE, flags, -1, 0);
if (dummy == MAP_FAILED) {
- ksft_test_result_skip("need more free huge pages\n");
+ ksft_perror("need more free huge pages");
+ log_test_result(KSFT_SKIP);
goto munmap;
}
munmap(dummy, hugetlbsize);
@@ -1226,7 +1292,7 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
ret = setup_comm_pipes(&comm_pipes);
if (ret) {
- ksft_test_result_fail("pipe() failed\n");
+ log_test_result(KSFT_FAIL);
return;
}
@@ -1236,12 +1302,14 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
*/
ret = mprotect(mem + pagesize, pagesize, PROT_READ);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
ret = mprotect(mem + pagesize, pagesize, PROT_READ | PROT_WRITE);
if (ret) {
- ksft_test_result_fail("mprotect() failed\n");
+ ksft_perror("mprotect() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
@@ -1250,8 +1318,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
/* Collapse before actually COW-sharing the page. */
ret = madvise(mem, size, MADV_COLLAPSE);
if (ret) {
- ksft_test_result_skip("MADV_COLLAPSE failed: %s\n",
- strerror(errno));
+ ksft_perror("MADV_COLLAPSE failed");
+ log_test_result(KSFT_SKIP);
goto close_comm_pipes;
}
break;
@@ -1262,7 +1330,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
/* Don't COW-share the upper part of the THP. */
ret = madvise(mem + size / 2, size / 2, MADV_DONTFORK);
if (ret) {
- ksft_test_result_fail("MADV_DONTFORK failed\n");
+ ksft_perror("MADV_DONTFORK failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
break;
@@ -1270,7 +1339,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
/* Don't COW-share the lower part of the THP. */
ret = madvise(mem, size / 2, MADV_DONTFORK);
if (ret) {
- ksft_test_result_fail("MADV_DONTFORK failed\n");
+ ksft_perror("MADV_DONTFORK failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
}
break;
@@ -1280,7 +1350,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
ret = fork();
if (ret < 0) {
- ksft_test_result_fail("fork() failed\n");
+ ksft_perror("fork() failed");
+ log_test_result(KSFT_FAIL);
goto close_comm_pipes;
} else if (!ret) {
switch (test) {
@@ -1314,7 +1385,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
*/
ret = madvise(mem, size, MADV_DOFORK);
if (ret) {
- ksft_test_result_fail("MADV_DOFORK failed\n");
+ ksft_perror("MADV_DOFORK failed");
+ log_test_result(KSFT_FAIL);
write(comm_pipes.parent_ready[1], "0", 1);
wait(&ret);
goto close_comm_pipes;
@@ -1324,8 +1396,8 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
/* Collapse before anyone modified the COW-shared page. */
ret = madvise(mem, size, MADV_COLLAPSE);
if (ret) {
- ksft_test_result_skip("MADV_COLLAPSE failed: %s\n",
- strerror(errno));
+ ksft_perror("MADV_COLLAPSE failed");
+ log_test_result(KSFT_SKIP);
write(comm_pipes.parent_ready[1], "0", 1);
wait(&ret);
goto close_comm_pipes;
@@ -1345,7 +1417,10 @@ static void do_test_anon_thp_collapse(char *mem, size_t size,
else
ret = -EINVAL;
- ksft_test_result(!ret, "No leak from parent into child\n");
+ if (!ret)
+ log_test_result(KSFT_PASS);
+ else
+ log_test_result(KSFT_FAIL);
close_comm_pipes:
close_comm_pipes(&comm_pipes);
}
@@ -1430,7 +1505,7 @@ static void run_anon_thp_test_cases(void)
for (i = 0; i < ARRAY_SIZE(anon_thp_test_cases); i++) {
struct test_case const *test_case = &anon_thp_test_cases[i];
- ksft_print_msg("[RUN] %s\n", test_case->desc);
+ log_test_start("%s", test_case->desc);
do_run_with_thp(test_case->fn, THP_RUN_PMD, pmdsize);
}
}
@@ -1453,8 +1528,10 @@ static void test_cow(char *mem, const char *smem, size_t size)
memset(mem, 0xff, size);
/* See if we still read the old values via the other mapping. */
- ksft_test_result(!memcmp(smem, old, size),
- "Other mapping not modified\n");
+ if (!memcmp(smem, old, size))
+ log_test_result(KSFT_PASS);
+ else
+ log_test_result(KSFT_FAIL);
free(old);
}
@@ -1472,18 +1549,20 @@ static void run_with_zeropage(non_anon_test_fn fn, const char *desc)
{
char *mem, *smem, tmp;
- ksft_print_msg("[RUN] %s ... with shared zeropage\n", desc);
+ log_test_start("%s ... with shared zeropage", desc);
mem = mmap(NULL, pagesize, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANON, -1, 0);
if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
return;
}
smem = mmap(NULL, pagesize, PROT_READ, MAP_PRIVATE | MAP_ANON, -1, 0);
- if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ if (smem == MAP_FAILED) {
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1504,10 +1583,11 @@ static void run_with_huge_zeropage(non_anon_test_fn fn, const char *desc)
size_t mmap_size;
int ret;
- ksft_print_msg("[RUN] %s ... with huge zeropage\n", desc);
+ log_test_start("%s ... with huge zeropage", desc);
if (!has_huge_zeropage) {
- ksft_test_result_skip("Huge zeropage not enabled\n");
+ ksft_print_msg("Huge zeropage not enabled\n");
+ log_test_result(KSFT_SKIP);
return;
}
@@ -1516,13 +1596,15 @@ static void run_with_huge_zeropage(non_anon_test_fn fn, const char *desc)
mmap_mem = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (mmap_mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
return;
}
mmap_smem = mmap(NULL, mmap_size, PROT_READ,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (mmap_smem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1531,9 +1613,15 @@ static void run_with_huge_zeropage(non_anon_test_fn fn, const char *desc)
smem = (char *)(((uintptr_t)mmap_smem + pmdsize) & ~(pmdsize - 1));
ret = madvise(mem, pmdsize, MADV_HUGEPAGE);
+ if (ret != 0) {
+ ksft_perror("madvise()");
+ log_test_result(KSFT_FAIL);
+ goto munmap;
+ }
ret |= madvise(smem, pmdsize, MADV_HUGEPAGE);
- if (ret) {
- ksft_test_result_fail("MADV_HUGEPAGE failed\n");
+ if (ret != 0) {
+ ksft_perror("madvise()");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1562,29 +1650,33 @@ static void run_with_memfd(non_anon_test_fn fn, const char *desc)
char *mem, *smem, tmp;
int fd;
- ksft_print_msg("[RUN] %s ... with memfd\n", desc);
+ log_test_start("%s ... with memfd", desc);
fd = memfd_create("test", 0);
if (fd < 0) {
- ksft_test_result_fail("memfd_create() failed\n");
+ ksft_perror("memfd_create() failed");
+ log_test_result(KSFT_FAIL);
return;
}
/* File consists of a single page filled with zeroes. */
if (fallocate(fd, 0, 0, pagesize)) {
- ksft_test_result_fail("fallocate() failed\n");
+ ksft_perror("fallocate() failed");
+ log_test_result(KSFT_FAIL);
goto close;
}
/* Create a private mapping of the memfd. */
mem = mmap(NULL, pagesize, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto close;
}
smem = mmap(NULL, pagesize, PROT_READ, MAP_SHARED, fd, 0);
- if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ if (smem == MAP_FAILED) {
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1607,35 +1699,45 @@ static void run_with_tmpfile(non_anon_test_fn fn, const char *desc)
FILE *file;
int fd;
- ksft_print_msg("[RUN] %s ... with tmpfile\n", desc);
+ log_test_start("%s ... with tmpfile", desc);
file = tmpfile();
if (!file) {
- ksft_test_result_fail("tmpfile() failed\n");
+ ksft_perror("tmpfile() failed");
+ log_test_result(KSFT_FAIL);
return;
}
fd = fileno(file);
if (fd < 0) {
- ksft_test_result_skip("fileno() failed\n");
+ ksft_perror("fileno() failed");
+ log_test_result(KSFT_SKIP);
return;
}
/* File consists of a single page filled with zeroes. */
if (fallocate(fd, 0, 0, pagesize)) {
- ksft_test_result_fail("fallocate() failed\n");
+ if (errno == EOPNOTSUPP) {
+ ksft_print_msg("fallocate() not supported by filesystem\n");
+ log_test_result(KSFT_SKIP);
+ } else {
+ ksft_perror("fallocate() failed");
+ log_test_result(KSFT_FAIL);
+ }
goto close;
}
/* Create a private mapping of the memfd. */
mem = mmap(NULL, pagesize, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto close;
}
smem = mmap(NULL, pagesize, PROT_READ, MAP_SHARED, fd, 0);
- if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ if (smem == MAP_FAILED) {
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1659,20 +1761,22 @@ static void run_with_memfd_hugetlb(non_anon_test_fn fn, const char *desc,
char *mem, *smem, tmp;
int fd;
- ksft_print_msg("[RUN] %s ... with memfd hugetlb (%zu kB)\n", desc,
+ log_test_start("%s ... with memfd hugetlb (%zu kB)", desc,
hugetlbsize / 1024);
flags |= __builtin_ctzll(hugetlbsize) << MFD_HUGE_SHIFT;
fd = memfd_create("test", flags);
if (fd < 0) {
- ksft_test_result_skip("memfd_create() failed\n");
+ ksft_perror("memfd_create() failed");
+ log_test_result(KSFT_SKIP);
return;
}
/* File consists of a single page filled with zeroes. */
if (fallocate(fd, 0, 0, hugetlbsize)) {
- ksft_test_result_skip("need more free huge pages\n");
+ ksft_perror("need more free huge pages");
+ log_test_result(KSFT_SKIP);
goto close;
}
@@ -1680,12 +1784,14 @@ static void run_with_memfd_hugetlb(non_anon_test_fn fn, const char *desc,
mem = mmap(NULL, hugetlbsize, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd,
0);
if (mem == MAP_FAILED) {
- ksft_test_result_skip("need more free huge pages\n");
+ ksft_perror("need more free huge pages");
+ log_test_result(KSFT_SKIP);
goto close;
}
smem = mmap(NULL, hugetlbsize, PROT_READ, MAP_SHARED, fd, 0);
- if (mem == MAP_FAILED) {
- ksft_test_result_fail("mmap() failed\n");
+ if (smem == MAP_FAILED) {
+ ksft_perror("mmap() failed");
+ log_test_result(KSFT_FAIL);
goto munmap;
}
@@ -1696,7 +1802,7 @@ static void run_with_memfd_hugetlb(non_anon_test_fn fn, const char *desc,
fn(mem, smem, hugetlbsize);
munmap:
munmap(mem, hugetlbsize);
- if (mem != MAP_FAILED)
+ if (smem != MAP_FAILED)
munmap(smem, hugetlbsize);
close:
close(fd);
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 8a4d34cce36b..6822eb7ea58e 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -1190,6 +1190,11 @@ int main(int argc, char **argv)
.read_ahead_kb = 0,
};
+ if (!thp_is_enabled()) {
+ printf("Transparent Hugepages not available\n");
+ return KSFT_SKIP;
+ }
+
parse_test_type(argc, argv);
setbuf(stdout, NULL);
diff --git a/tools/testing/selftests/mm/ksm_tests.c b/tools/testing/selftests/mm/ksm_tests.c
index b748c48908d9..ecb4b1057c6b 100644
--- a/tools/testing/selftests/mm/ksm_tests.c
+++ b/tools/testing/selftests/mm/ksm_tests.c
@@ -15,6 +15,7 @@
#include "../kselftest.h"
#include <include/vdso/time64.h>
#include "vm_util.h"
+#include "thp_settings.h"
#define KSM_SYSFS_PATH "/sys/kernel/mm/ksm/"
#define KSM_FP(s) (KSM_SYSFS_PATH s)
@@ -202,12 +203,12 @@ static void *allocate_memory(void *ptr, int prot, int mapping, char data, size_
void *map_ptr = mmap(ptr, map_size, PROT_WRITE, mapping, -1, 0);
if (!map_ptr) {
- perror("mmap");
+ ksft_perror("mmap");
return NULL;
}
memset(map_ptr, data, map_size);
if (mprotect(map_ptr, map_size, prot)) {
- perror("mprotect");
+ ksft_perror("mprotect");
munmap(map_ptr, map_size);
return NULL;
}
@@ -228,11 +229,11 @@ static int ksm_do_scan(int scan_count, struct timespec start_time, int timeout)
if (ksm_read_sysfs(KSM_FP("full_scans"), &cur_scan))
return 1;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &cur_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return 1;
}
if ((cur_time.tv_sec - start_time.tv_sec) > timeout) {
- printf("Scan time limit exceeded\n");
+ ksft_print_msg("Scan time limit exceeded\n");
return 1;
}
}
@@ -245,12 +246,12 @@ static int ksm_merge_pages(int merge_type, void *addr, size_t size,
{
if (merge_type == KSM_MERGE_MADVISE) {
if (madvise(addr, size, MADV_MERGEABLE)) {
- perror("madvise");
+ ksft_perror("madvise");
return 1;
}
} else if (merge_type == KSM_MERGE_PRCTL) {
if (prctl(PR_SET_MEMORY_MERGE, 1, 0, 0, 0)) {
- perror("prctl");
+ ksft_perror("prctl");
return 1;
}
}
@@ -269,7 +270,7 @@ static int ksm_unmerge_pages(void *addr, size_t size,
struct timespec start_time, int timeout)
{
if (madvise(addr, size, MADV_UNMERGEABLE)) {
- perror("madvise");
+ ksft_perror("madvise");
return 1;
}
return 0;
@@ -315,8 +316,8 @@ static bool assert_ksm_pages_count(long dupl_page_count)
static int ksm_save_def(struct ksm_sysfs *ksm_sysfs)
{
if (ksm_read_sysfs(KSM_FP("max_page_sharing"), &ksm_sysfs->max_page_sharing) ||
- numa_available() ? 0 :
- ksm_read_sysfs(KSM_FP("merge_across_nodes"), &ksm_sysfs->merge_across_nodes) ||
+ (numa_available() ? 0 :
+ ksm_read_sysfs(KSM_FP("merge_across_nodes"), &ksm_sysfs->merge_across_nodes)) ||
ksm_read_sysfs(KSM_FP("sleep_millisecs"), &ksm_sysfs->sleep_millisecs) ||
ksm_read_sysfs(KSM_FP("pages_to_scan"), &ksm_sysfs->pages_to_scan) ||
ksm_read_sysfs(KSM_FP("run"), &ksm_sysfs->run) ||
@@ -331,8 +332,8 @@ static int ksm_save_def(struct ksm_sysfs *ksm_sysfs)
static int ksm_restore(struct ksm_sysfs *ksm_sysfs)
{
if (ksm_write_sysfs(KSM_FP("max_page_sharing"), ksm_sysfs->max_page_sharing) ||
- numa_available() ? 0 :
- ksm_write_sysfs(KSM_FP("merge_across_nodes"), ksm_sysfs->merge_across_nodes) ||
+ (numa_available() ? 0 :
+ ksm_write_sysfs(KSM_FP("merge_across_nodes"), ksm_sysfs->merge_across_nodes)) ||
ksm_write_sysfs(KSM_FP("pages_to_scan"), ksm_sysfs->pages_to_scan) ||
ksm_write_sysfs(KSM_FP("run"), ksm_sysfs->run) ||
ksm_write_sysfs(KSM_FP("sleep_millisecs"), ksm_sysfs->sleep_millisecs) ||
@@ -351,7 +352,7 @@ static int check_ksm_merge(int merge_type, int mapping, int prot,
struct timespec start_time;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
@@ -365,7 +366,6 @@ static int check_ksm_merge(int merge_type, int mapping, int prot,
/* verify that the right number of pages are merged */
if (assert_ksm_pages_count(page_count)) {
- printf("OK\n");
munmap(map_ptr, page_size * page_count);
if (merge_type == KSM_MERGE_PRCTL)
prctl(PR_SET_MEMORY_MERGE, 0, 0, 0, 0);
@@ -373,7 +373,6 @@ static int check_ksm_merge(int merge_type, int mapping, int prot,
}
err_out:
- printf("Not OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_FAIL;
}
@@ -385,7 +384,7 @@ static int check_ksm_unmerge(int merge_type, int mapping, int prot, int timeout,
int page_count = 2;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
@@ -407,13 +406,11 @@ static int check_ksm_unmerge(int merge_type, int mapping, int prot, int timeout,
/* check that unmerging was successful and 0 pages are currently merged */
if (assert_ksm_pages_count(0)) {
- printf("OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_PASS;
}
err_out:
- printf("Not OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_FAIL;
}
@@ -425,7 +422,7 @@ static int check_ksm_zero_page_merge(int merge_type, int mapping, int prot, long
struct timespec start_time;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
@@ -452,12 +449,10 @@ static int check_ksm_zero_page_merge(int merge_type, int mapping, int prot, long
else if (!use_zero_pages && !assert_ksm_pages_count(page_count))
goto err_out;
- printf("OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_PASS;
err_out:
- printf("Not OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_FAIL;
}
@@ -473,9 +468,9 @@ static int get_next_mem_node(int node)
mem_node = i % (max_node + 1);
node_size = numa_node_size(mem_node, NULL);
if (node_size > 0)
- break;
+ return mem_node;
}
- return mem_node;
+ return -ENODEV;
}
static int get_first_mem_node(void)
@@ -488,31 +483,33 @@ static int check_ksm_numa_merge(int merge_type, int mapping, int prot, int timeo
{
void *numa1_map_ptr, *numa2_map_ptr;
struct timespec start_time;
+ int first_node, second_node;
int page_count = 2;
- int first_node;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
if (numa_available() < 0) {
- perror("NUMA support not enabled");
+ ksft_print_msg("NUMA support not enabled\n");
return KSFT_SKIP;
}
- if (numa_num_configured_nodes() <= 1) {
- printf("At least 2 NUMA nodes must be available\n");
+ first_node = get_first_mem_node();
+ second_node = get_next_mem_node(first_node);
+
+ if (second_node < 0) {
+ ksft_print_msg("At least 2 NUMA nodes with memory must be available\n");
return KSFT_SKIP;
}
if (ksm_write_sysfs(KSM_FP("merge_across_nodes"), merge_across_nodes))
return KSFT_FAIL;
/* allocate 2 pages in 2 different NUMA nodes and fill them with the same data */
- first_node = get_first_mem_node();
numa1_map_ptr = numa_alloc_onnode(page_size, first_node);
- numa2_map_ptr = numa_alloc_onnode(page_size, get_next_mem_node(first_node));
+ numa2_map_ptr = numa_alloc_onnode(page_size, second_node);
if (!numa1_map_ptr || !numa2_map_ptr) {
- perror("numa_alloc_onnode");
+ ksft_perror("numa_alloc_onnode");
return KSFT_FAIL;
}
@@ -537,13 +534,11 @@ static int check_ksm_numa_merge(int merge_type, int mapping, int prot, int timeo
numa_free(numa1_map_ptr, page_size);
numa_free(numa2_map_ptr, page_size);
- printf("OK\n");
return KSFT_PASS;
err_out:
numa_free(numa1_map_ptr, page_size);
numa_free(numa2_map_ptr, page_size);
- printf("Not OK\n");
return KSFT_FAIL;
}
@@ -555,6 +550,11 @@ static int ksm_merge_hugepages_time(int merge_type, int mapping, int prot,
unsigned long scan_time_ns;
int pagemap_fd, n_normal_pages, n_huge_pages;
+ if (!thp_is_enabled()) {
+ ksft_print_msg("Transparent Hugepages not available\n");
+ return KSFT_SKIP;
+ }
+
map_size *= MB;
size_t len = map_size;
@@ -581,36 +581,35 @@ static int ksm_merge_hugepages_time(int merge_type, int mapping, int prot,
else
n_huge_pages++;
}
- printf("Number of normal pages: %d\n", n_normal_pages);
- printf("Number of huge pages: %d\n", n_huge_pages);
+ ksft_print_msg("Number of normal pages: %d\n", n_normal_pages);
+ ksft_print_msg("Number of huge pages: %d\n", n_huge_pages);
memset(map_ptr, '*', len);
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
if (ksm_merge_pages(merge_type, map_ptr, map_size, start_time, timeout))
goto err_out;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &end_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
scan_time_ns = (end_time.tv_sec - start_time.tv_sec) * NSEC_PER_SEC +
(end_time.tv_nsec - start_time.tv_nsec);
- printf("Total size: %lu MiB\n", map_size / MB);
- printf("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
+ ksft_print_msg("Total size: %lu MiB\n", map_size / MB);
+ ksft_print_msg("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
scan_time_ns % NSEC_PER_SEC);
- printf("Average speed: %.3f MiB/s\n", (map_size / MB) /
+ ksft_print_msg("Average speed: %.3f MiB/s\n", (map_size / MB) /
((double)scan_time_ns / NSEC_PER_SEC));
munmap(map_ptr_orig, len + HPAGE_SIZE);
return KSFT_PASS;
err_out:
- printf("Not OK\n");
munmap(map_ptr_orig, len + HPAGE_SIZE);
return KSFT_FAIL;
}
@@ -628,30 +627,29 @@ static int ksm_merge_time(int merge_type, int mapping, int prot, int timeout, si
return KSFT_FAIL;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
if (ksm_merge_pages(merge_type, map_ptr, map_size, start_time, timeout))
goto err_out;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &end_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
scan_time_ns = (end_time.tv_sec - start_time.tv_sec) * NSEC_PER_SEC +
(end_time.tv_nsec - start_time.tv_nsec);
- printf("Total size: %lu MiB\n", map_size / MB);
- printf("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
+ ksft_print_msg("Total size: %lu MiB\n", map_size / MB);
+ ksft_print_msg("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
scan_time_ns % NSEC_PER_SEC);
- printf("Average speed: %.3f MiB/s\n", (map_size / MB) /
+ ksft_print_msg("Average speed: %.3f MiB/s\n", (map_size / MB) /
((double)scan_time_ns / NSEC_PER_SEC));
munmap(map_ptr, map_size);
return KSFT_PASS;
err_out:
- printf("Not OK\n");
munmap(map_ptr, map_size);
return KSFT_FAIL;
}
@@ -668,37 +666,36 @@ static int ksm_unmerge_time(int merge_type, int mapping, int prot, int timeout,
if (!map_ptr)
return KSFT_FAIL;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
if (ksm_merge_pages(merge_type, map_ptr, map_size, start_time, timeout))
goto err_out;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
if (ksm_unmerge_pages(map_ptr, map_size, start_time, timeout))
goto err_out;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &end_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
scan_time_ns = (end_time.tv_sec - start_time.tv_sec) * NSEC_PER_SEC +
(end_time.tv_nsec - start_time.tv_nsec);
- printf("Total size: %lu MiB\n", map_size / MB);
- printf("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
+ ksft_print_msg("Total size: %lu MiB\n", map_size / MB);
+ ksft_print_msg("Total time: %ld.%09ld s\n", scan_time_ns / NSEC_PER_SEC,
scan_time_ns % NSEC_PER_SEC);
- printf("Average speed: %.3f MiB/s\n", (map_size / MB) /
+ ksft_print_msg("Average speed: %.3f MiB/s\n", (map_size / MB) /
((double)scan_time_ns / NSEC_PER_SEC));
munmap(map_ptr, map_size);
return KSFT_PASS;
err_out:
- printf("Not OK\n");
munmap(map_ptr, map_size);
return KSFT_FAIL;
}
@@ -717,24 +714,24 @@ static int ksm_cow_time(int merge_type, int mapping, int prot, int timeout, size
return KSFT_FAIL;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
for (size_t i = 0; i < page_count - 1; i = i + 2)
memset(map_ptr + page_size * i, '-', 1);
if (clock_gettime(CLOCK_MONOTONIC_RAW, &end_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
return KSFT_FAIL;
}
cow_time_ns = (end_time.tv_sec - start_time.tv_sec) * NSEC_PER_SEC +
(end_time.tv_nsec - start_time.tv_nsec);
- printf("Total size: %lu MiB\n\n", (page_size * page_count) / MB);
- printf("Not merged pages:\n");
- printf("Total time: %ld.%09ld s\n", cow_time_ns / NSEC_PER_SEC,
+ ksft_print_msg("Total size: %lu MiB\n\n", (page_size * page_count) / MB);
+ ksft_print_msg("Not merged pages:\n");
+ ksft_print_msg("Total time: %ld.%09ld s\n", cow_time_ns / NSEC_PER_SEC,
cow_time_ns % NSEC_PER_SEC);
- printf("Average speed: %.3f MiB/s\n\n", ((page_size * (page_count / 2)) / MB) /
+ ksft_print_msg("Average speed: %.3f MiB/s\n\n", ((page_size * (page_count / 2)) / MB) /
((double)cow_time_ns / NSEC_PER_SEC));
/* Create 2000 pairs of duplicate pages */
@@ -746,30 +743,29 @@ static int ksm_cow_time(int merge_type, int mapping, int prot, int timeout, size
goto err_out;
if (clock_gettime(CLOCK_MONOTONIC_RAW, &start_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
for (size_t i = 0; i < page_count - 1; i = i + 2)
memset(map_ptr + page_size * i, '-', 1);
if (clock_gettime(CLOCK_MONOTONIC_RAW, &end_time)) {
- perror("clock_gettime");
+ ksft_perror("clock_gettime");
goto err_out;
}
cow_time_ns = (end_time.tv_sec - start_time.tv_sec) * NSEC_PER_SEC +
(end_time.tv_nsec - start_time.tv_nsec);
- printf("Merged pages:\n");
- printf("Total time: %ld.%09ld s\n", cow_time_ns / NSEC_PER_SEC,
+ ksft_print_msg("Merged pages:\n");
+ ksft_print_msg("Total time: %ld.%09ld s\n", cow_time_ns / NSEC_PER_SEC,
cow_time_ns % NSEC_PER_SEC);
- printf("Average speed: %.3f MiB/s\n", ((page_size * (page_count / 2)) / MB) /
+ ksft_print_msg("Average speed: %.3f MiB/s\n", ((page_size * (page_count / 2)) / MB) /
((double)cow_time_ns / NSEC_PER_SEC));
munmap(map_ptr, page_size * page_count);
return KSFT_PASS;
err_out:
- printf("Not OK\n");
munmap(map_ptr, page_size * page_count);
return KSFT_FAIL;
}
@@ -787,6 +783,10 @@ int main(int argc, char *argv[])
bool use_zero_pages = KSM_USE_ZERO_PAGES_DEFAULT;
bool merge_across_nodes = KSM_MERGE_ACROSS_NODES_DEFAULT;
long size_MB = 0;
+ const char *test_descr = "KSM merging";
+
+ ksft_print_header();
+ ksft_set_plan(1);
while ((opt = getopt(argc, argv, "dha:p:l:z:m:s:t:MUZNPCHD")) != -1) {
switch (opt) {
@@ -795,17 +795,13 @@ int main(int argc, char *argv[])
break;
case 'p':
page_count = atol(optarg);
- if (page_count <= 0) {
- printf("The number of pages must be greater than 0\n");
- return KSFT_FAIL;
- }
+ if (page_count <= 0)
+ ksft_exit_fail_msg("The number of pages must be greater than 0\n");
break;
case 'l':
ksm_scan_limit_sec = atoi(optarg);
- if (ksm_scan_limit_sec <= 0) {
- printf("Timeout value must be greater than 0\n");
- return KSFT_FAIL;
- }
+ if (ksm_scan_limit_sec <= 0)
+ ksft_exit_fail_msg("Timeout value must be greater than 0\n");
break;
case 'h':
print_help();
@@ -827,19 +823,15 @@ int main(int argc, char *argv[])
break;
case 's':
size_MB = atoi(optarg);
- if (size_MB <= 0) {
- printf("Size must be greater than 0\n");
- return KSFT_FAIL;
- }
+ if (size_MB <= 0)
+ ksft_exit_fail_msg("Size must be greater than 0\n");
break;
case 't':
{
int tmp = atoi(optarg);
- if (tmp < 0 || tmp > KSM_MERGE_LAST) {
- printf("Invalid merge type\n");
- return KSFT_FAIL;
- }
+ if (tmp < 0 || tmp > KSM_MERGE_LAST)
+ ksft_exit_fail_msg("Invalid merge type\n");
merge_type = tmp;
}
break;
@@ -867,82 +859,80 @@ int main(int argc, char *argv[])
test_name = KSM_COW_TIME;
break;
default:
- return KSFT_FAIL;
+ ksft_exit_fail_msg("Unknown option\n");
}
}
if (prot == 0)
prot = str_to_prot(KSM_PROT_STR_DEFAULT);
- if (access(KSM_SYSFS_PATH, F_OK)) {
- printf("Config KSM not enabled\n");
- return KSFT_SKIP;
- }
+ if (access(KSM_SYSFS_PATH, F_OK))
+ ksft_exit_skip("Config KSM not enabled\n");
- if (ksm_save_def(&ksm_sysfs_old)) {
- printf("Cannot save default tunables\n");
- return KSFT_FAIL;
- }
+ if (ksm_save_def(&ksm_sysfs_old))
+ ksft_exit_fail_msg("Cannot save default tunables\n");
if (ksm_write_sysfs(KSM_FP("run"), 2) ||
ksm_write_sysfs(KSM_FP("sleep_millisecs"), 0) ||
- numa_available() ? 0 :
- ksm_write_sysfs(KSM_FP("merge_across_nodes"), 1) ||
+ (numa_available() ? 0 :
+ ksm_write_sysfs(KSM_FP("merge_across_nodes"), 1)) ||
ksm_write_sysfs(KSM_FP("pages_to_scan"), page_count))
- return KSFT_FAIL;
+ ksft_exit_fail_msg("Cannot set up KSM tunables\n");
switch (test_name) {
case CHECK_KSM_MERGE:
+ test_descr = "KSM merging";
ret = check_ksm_merge(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot, page_count,
ksm_scan_limit_sec, page_size);
break;
case CHECK_KSM_UNMERGE:
+ test_descr = "KSM unmerging";
ret = check_ksm_unmerge(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, page_size);
break;
case CHECK_KSM_ZERO_PAGE_MERGE:
+ test_descr = "KSM zero page merging";
ret = check_ksm_zero_page_merge(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
page_count, ksm_scan_limit_sec, use_zero_pages,
page_size);
break;
case CHECK_KSM_NUMA_MERGE:
+ test_descr = "KSM NUMA merging";
ret = check_ksm_numa_merge(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, merge_across_nodes, page_size);
break;
case KSM_MERGE_TIME:
- if (size_MB == 0) {
- printf("Option '-s' is required.\n");
- return KSFT_FAIL;
- }
+ if (size_MB == 0)
+ ksft_exit_fail_msg("Option '-s' is required\n");
+ test_descr = "KSM merge time";
ret = ksm_merge_time(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, size_MB);
break;
case KSM_MERGE_TIME_HUGE_PAGES:
- if (size_MB == 0) {
- printf("Option '-s' is required.\n");
- return KSFT_FAIL;
- }
+ if (size_MB == 0)
+ ksft_exit_fail_msg("Option '-s' is required\n");
+ test_descr = "KSM merge time with huge pages";
ret = ksm_merge_hugepages_time(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, size_MB);
break;
case KSM_UNMERGE_TIME:
- if (size_MB == 0) {
- printf("Option '-s' is required.\n");
- return KSFT_FAIL;
- }
+ if (size_MB == 0)
+ ksft_exit_fail_msg("Option '-s' is required\n");
+ test_descr = "KSM unmerge time";
ret = ksm_unmerge_time(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, size_MB);
break;
case KSM_COW_TIME:
+ test_descr = "KSM COW time";
ret = ksm_cow_time(merge_type, MAP_PRIVATE | MAP_ANONYMOUS, prot,
ksm_scan_limit_sec, page_size);
break;
}
- if (ksm_restore(&ksm_sysfs_old)) {
- printf("Cannot restore default tunables\n");
- return KSFT_FAIL;
- }
+ if (ksm_restore(&ksm_sysfs_old))
+ ksft_print_msg("Cannot restore default tunables\n");
+
+ ksft_test_result_report(ret, "%s\n", test_descr);
- return ret;
+ ksft_finished();
}
diff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c
index c883ef420d3b..7ad742c8c132 100644
--- a/tools/testing/selftests/mm/migration.c
+++ b/tools/testing/selftests/mm/migration.c
@@ -5,6 +5,8 @@
*/
#include "../kselftest_harness.h"
+#include "thp_settings.h"
+
#include <strings.h>
#include <pthread.h>
#include <numa.h>
@@ -186,6 +188,9 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME)
uint64_t *ptr;
int i;
+ if (!thp_is_enabled())
+ SKIP(return, "Transparent Hugepages not available");
+
if (self->nthreads < 2 || self->n1 < 0 || self->n2 < 0)
SKIP(return, "Not enough threads or NUMA nodes available");
@@ -205,4 +210,106 @@ TEST_F_TIMEOUT(migration, private_anon_thp, 2*RUNTIME)
ASSERT_EQ(pthread_cancel(self->threads[i]), 0);
}
+/*
+ * migration test with shared anon THP page
+ */
+
+TEST_F_TIMEOUT(migration, shared_anon_thp, 2*RUNTIME)
+{
+ pid_t pid;
+ uint64_t *ptr;
+ int i;
+
+ if (!thp_is_enabled())
+ SKIP(return, "Transparent Hugepages not available");
+
+ if (self->nthreads < 2 || self->n1 < 0 || self->n2 < 0)
+ SKIP(return, "Not enough threads or NUMA nodes available");
+
+ ptr = mmap(NULL, 2 * TWOMEG, PROT_READ | PROT_WRITE,
+ MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+ ASSERT_NE(ptr, MAP_FAILED);
+
+ ptr = (uint64_t *) ALIGN((uintptr_t) ptr, TWOMEG);
+ ASSERT_EQ(madvise(ptr, TWOMEG, MADV_HUGEPAGE), 0);
+
+ memset(ptr, 0xde, TWOMEG);
+ for (i = 0; i < self->nthreads - 1; i++) {
+ pid = fork();
+ if (!pid) {
+ prctl(PR_SET_PDEATHSIG, SIGHUP);
+ /* Parent may have died before prctl so check now. */
+ if (getppid() == 1)
+ kill(getpid(), SIGHUP);
+ access_mem(ptr);
+ } else {
+ self->pids[i] = pid;
+ }
+ }
+
+ ASSERT_EQ(migrate(ptr, self->n1, self->n2), 0);
+ for (i = 0; i < self->nthreads - 1; i++)
+ ASSERT_EQ(kill(self->pids[i], SIGTERM), 0);
+}
+
+/*
+ * migration test with private anon hugetlb page
+ */
+TEST_F_TIMEOUT(migration, private_anon_htlb, 2*RUNTIME)
+{
+ uint64_t *ptr;
+ int i;
+
+ if (self->nthreads < 2 || self->n1 < 0 || self->n2 < 0)
+ SKIP(return, "Not enough threads or NUMA nodes available");
+
+ ptr = mmap(NULL, TWOMEG, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
+ ASSERT_NE(ptr, MAP_FAILED);
+
+ memset(ptr, 0xde, TWOMEG);
+ for (i = 0; i < self->nthreads - 1; i++)
+ if (pthread_create(&self->threads[i], NULL, access_mem, ptr))
+ perror("Couldn't create thread");
+
+ ASSERT_EQ(migrate(ptr, self->n1, self->n2), 0);
+ for (i = 0; i < self->nthreads - 1; i++)
+ ASSERT_EQ(pthread_cancel(self->threads[i]), 0);
+}
+
+/*
+ * migration test with shared anon hugetlb page
+ */
+TEST_F_TIMEOUT(migration, shared_anon_htlb, 2*RUNTIME)
+{
+ pid_t pid;
+ uint64_t *ptr;
+ int i;
+
+ if (self->nthreads < 2 || self->n1 < 0 || self->n2 < 0)
+ SKIP(return, "Not enough threads or NUMA nodes available");
+
+ ptr = mmap(NULL, TWOMEG, PROT_READ | PROT_WRITE,
+ MAP_SHARED | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
+ ASSERT_NE(ptr, MAP_FAILED);
+
+ memset(ptr, 0xde, TWOMEG);
+ for (i = 0; i < self->nthreads - 1; i++) {
+ pid = fork();
+ if (!pid) {
+ prctl(PR_SET_PDEATHSIG, SIGHUP);
+ /* Parent may have died before prctl so check now. */
+ if (getppid() == 1)
+ kill(getpid(), SIGHUP);
+ access_mem(ptr);
+ } else {
+ self->pids[i] = pid;
+ }
+ }
+
+ ASSERT_EQ(migrate(ptr, self->n1, self->n2), 0);
+ for (i = 0; i < self->nthreads - 1; i++)
+ ASSERT_EQ(kill(self->pids[i], SIGTERM), 0);
+}
+
TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/mm/soft-dirty.c b/tools/testing/selftests/mm/soft-dirty.c
index 7b91df12ce5b..ca65ff3096b2 100644
--- a/tools/testing/selftests/mm/soft-dirty.c
+++ b/tools/testing/selftests/mm/soft-dirty.c
@@ -6,8 +6,10 @@
#include <stdint.h>
#include <malloc.h>
#include <sys/mman.h>
+
#include "../kselftest.h"
#include "vm_util.h"
+#include "thp_settings.h"
#define PAGEMAP_FILE_PATH "/proc/self/pagemap"
#define TEST_ITERATIONS 10000
@@ -78,8 +80,13 @@ static void test_hugepage(int pagemap_fd, int pagesize)
{
char *map;
int i, ret;
- size_t hpage_len = read_pmd_pagesize();
+ if (!thp_is_enabled()) {
+ ksft_test_result_skip("Transparent Hugepages not available\n");
+ return;
+ }
+
+ size_t hpage_len = read_pmd_pagesize();
if (!hpage_len)
ksft_exit_fail_msg("Reading PMD pagesize failed");
diff --git a/tools/testing/selftests/mm/thp_settings.c b/tools/testing/selftests/mm/thp_settings.c
index 577eaab6266f..38f70155bfbf 100644
--- a/tools/testing/selftests/mm/thp_settings.c
+++ b/tools/testing/selftests/mm/thp_settings.c
@@ -381,3 +381,14 @@ unsigned long thp_shmem_supported_orders(void)
{
return __thp_supported_orders(true);
}
+
+bool thp_is_enabled(void)
+{
+ if (access(THP_SYSFS, F_OK) != 0)
+ return false;
+
+ int mode = thp_read_string("enabled", thp_enabled_strings);
+
+ /* THP is considered enabled if it's either "always" or "madvise" */
+ return mode == 1 || mode == 3;
+}
diff --git a/tools/testing/selftests/mm/thp_settings.h b/tools/testing/selftests/mm/thp_settings.h
index 876235a23460..079a58a662b0 100644
--- a/tools/testing/selftests/mm/thp_settings.h
+++ b/tools/testing/selftests/mm/thp_settings.h
@@ -84,4 +84,6 @@ void thp_set_read_ahead_path(char *path);
unsigned long thp_supported_orders(void);
unsigned long thp_shmem_supported_orders(void);
+bool thp_is_enabled(void);
+
#endif /* __THP_SETTINGS_H__ */
diff --git a/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h b/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
index d3133587d996..5a29d6bec51f 100644
--- a/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
+++ b/tools/testing/selftests/rseq/rseq-x86-thread-pointer.h
@@ -8,13 +8,11 @@
#ifndef _RSEQ_X86_THREAD_POINTER
#define _RSEQ_X86_THREAD_POINTER
-#include <features.h>
-
#ifdef __cplusplus
extern "C" {
#endif
-#if __GNUC_PREREQ (11, 1)
+#if __GNUC__ > 11 || (__GNUC__ == 11 && __GNUC_MINOR__ >= 1)
static inline void *rseq_thread_pointer(void)
{
return __builtin_thread_pointer();
diff --git a/tools/testing/selftests/sched_ext/prog_run.c b/tools/testing/selftests/sched_ext/prog_run.c
index 3cd57ef8daaa..a75323fe8a2c 100644
--- a/tools/testing/selftests/sched_ext/prog_run.c
+++ b/tools/testing/selftests/sched_ext/prog_run.c
@@ -28,7 +28,8 @@ static enum scx_test_status setup(void **ctx)
static enum scx_test_status run(void *ctx)
{
struct prog_run *skel = ctx;
- struct bpf_link *link;
+ struct bpf_link *link = NULL;
+ enum scx_test_status status = SCX_TEST_PASS;
int prog_fd, err = 0;
prog_fd = bpf_program__fd(skel->progs.prog_run_syscall);
@@ -42,23 +43,40 @@ static enum scx_test_status run(void *ctx)
link = bpf_map__attach_struct_ops(skel->maps.prog_run_ops);
if (!link) {
SCX_ERR("Failed to attach scheduler");
- close(prog_fd);
- return SCX_TEST_FAIL;
+ status = SCX_TEST_FAIL;
+ goto out;
}
err = bpf_prog_test_run_opts(prog_fd, &topts);
- SCX_EQ(err, 0);
+ if (err) {
+ SCX_ERR("BPF_PROG_RUN failed (%d)", err);
+ status = SCX_TEST_FAIL;
+ goto out;
+ }
/* Assumes uei.kind is written last */
while (skel->data->uei.kind == EXIT_KIND(SCX_EXIT_NONE))
sched_yield();
- SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_UNREG_BPF));
- SCX_EQ(skel->data->uei.exit_code, 0xdeadbeef);
+ if (skel->data->uei.kind != EXIT_KIND(SCX_EXIT_UNREG_BPF)) {
+ SCX_ERR("Unexpected exit kind: %llu",
+ (unsigned long long)skel->data->uei.kind);
+ status = SCX_TEST_FAIL;
+ goto out;
+ }
+ if (skel->data->uei.exit_code != 0xdeadbeef) {
+ SCX_ERR("Unexpected exit code: %lld",
+ (long long)skel->data->uei.exit_code);
+ status = SCX_TEST_FAIL;
+ goto out;
+ }
+
+out:
close(prog_fd);
- bpf_link__destroy(link);
+ if (link)
+ bpf_link__destroy(link);
- return SCX_TEST_PASS;
+ return status;
}
static void cleanup(void *ctx)
diff --git a/tools/testing/selftests/timers/leap-a-day.c b/tools/testing/selftests/timers/leap-a-day.c
index 986abbdb1521..eb99683e5ced 100644
--- a/tools/testing/selftests/timers/leap-a-day.c
+++ b/tools/testing/selftests/timers/leap-a-day.c
@@ -9,16 +9,19 @@
* kernel's leap-second behavior, as well as how well applications
* handle the leap-second discontinuity.
*
- * Usage: leap-a-day [-s] [-i <num>]
+ * Usage: leap-a-day [-w] [-i <num>] [-t]
*
* Options:
- * -s: Each iteration, set the date to 10 seconds before midnight GMT.
- * This speeds up the number of leapsecond transitions tested,
- * but because it calls settimeofday frequently, advancing the
- * time by 24 hours every ~16 seconds, it may cause application
- * disruption.
+ * -w: Only set the leap-second flag and wait for the leap second
+ * each iteration, instead of advancing the time. By default the
+ * date is set to 10 seconds before midnight GMT, which speeds up
+ * the number of leapsecond transitions tested, but because it
+ * calls settimeofday frequently, advancing the time by 24 hours
+ * every ~16 seconds, it may cause application disruption.
*
- * -i: Number of iterations to run (default: infinite)
+ * -i: Number of iterations to run (-1 = infinite, default: 10)
+ *
+ * -t: Print TAI time.
*
* Other notes: Disabling NTP prior to running this is advised, as the two
* may conflict in their commands to the kernel.
@@ -186,7 +189,7 @@ int main(int argc, char **argv)
int opt;
/* Process arguments */
- while ((opt = getopt(argc, argv, "sti:")) != -1) {
+ while ((opt = getopt(argc, argv, "wti:")) != -1) {
switch (opt) {
case 'w':
printf("Only setting leap-flag, not changing time. It could take up to a day for leap to trigger.\n");
diff --git a/tools/testing/selftests/zram/zram_lib.sh b/tools/testing/selftests/zram/zram_lib.sh
index 21ec1966de76..0d44d83888f9 100755
--- a/tools/testing/selftests/zram/zram_lib.sh
+++ b/tools/testing/selftests/zram/zram_lib.sh
@@ -37,7 +37,7 @@ kernel_gte()
if [ $kernel_major -gt $major ]; then
return 0
- elif [[ $kernel_major -eq $major && $kernel_minor -ge $minor ]]; then
+ elif [ $kernel_major -eq $major ] && [ $kernel_minor -ge $minor ]; then
return 0
fi
prev parent reply other threads:[~2026-09-14 11:58 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-14 11:58 Greg Kroah-Hartman
2026-09-14 11:58 ` Greg Kroah-Hartman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2026091412-sniff-drank-bc23@gregkh \
--to=gregkh@linuxfoundation.org \
--cc=akpm@linux-foundation.org \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=lwn@lwn.net \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®